The recent advancements in large language models (LLMs) are largely attributed to the scale and diversity of their training datasets. These datasets encompass pretraining corpora and finetuning datasets, sourced from academia, synthetic model generation, and platforms like Hugging Face. While some documentation efforts exist, there's a growing lack of attribution and understanding of the raw data sources used in new models. This lack of transparency stems from the sheer scale of modern data collection, making proper attribution challenging, coupled with increased copyright scrutiny. Consequently, there's a decline in datasheets, non-disclosure of training sources, and ultimately, a diminished understanding of training data. This lack of understanding leads to several critical issues: data leakage between training and testing sets, exposure of personally identifiable information (PII), unintended biases, and lower-than-anticipated model quality. Beyond these practical challenges, inadequate documentation incurs substantial ethical and legal risks, as evidenced by instances of model releases contradicting data terms of use, post-release license revisions, and even copyright lawsuits. Given the expense and irreversibility of model training, these risks are not easily mitigated. This research defines 'data provenance' as encompassing a dataset's sourcing, creation, licensing heritage, and characteristics, and addresses the urgency for tools facilitating informed and responsible data usage.

The paper references numerous existing works highlighting the challenges of data transparency and documentation in the context of AI model training. These include studies on data leakage, bias in models, and the limitations of existing data documentation standards like datasheets. The authors cite several examples of copyright lawsuits and licensing discrepancies in the current landscape of LLMs and their training data. The literature review also discusses the evolving legal framework surrounding copyright and fair use in the context of machine learning, emphasizing the uncertainties around training models on copyrighted content and the creation of derivative works.

This research undertakes a large-scale audit of data provenance, focusing on 1,858 finetuning datasets from 44 widely used text data collections. A pipeline was developed, in collaboration with legal experts, to trace dataset lineage, encompassing original sources, licenses, creators, and subsequent use. The pipeline combined automated and manual methods. Automated methods included extracting license information from Hugging Face and GitHub, utilizing the Semantic Scholar API to retrieve release dates and citation counts, and computing text metrics. Manual annotation involved rigorously reviewing and categorizing licenses based on conditions impacting model development lifecycles (commercial use, attribution requirements, share-alike clauses). Additional manual annotation included detailed dataset characteristics (languages, tasks, topics, text metrics, format, source, creators, attribution). GPT-4 was leveraged to aid in text topic annotation and source identification. The study compared self-reported license information with that documented on GitHub, Hugging Face, and Papers with Code, identifying inconsistencies and gaps. The comprehensive audit resulted in two deliverables: (1) the DPCollection, a repository of annotated data provenance; and (2) the DPExplorer, an open-source, interactive interface enabling filtering and exploration of the dataset provenance and characteristics. The DPExplorer automatically generates data provenance cards, which serve as a standardized format for documenting the composition and risks of training data.

The audit revealed a critical situation regarding data licensing and attribution in the AI community. More than 70% of licenses for popular datasets were unspecified, creating substantial information gaps and legal uncertainty for developers. The analysis indicated a significant rate of license miscategorization on platforms like Hugging Face and GitHub, often with licenses appearing more permissive than originally intended by the authors. The study identified a stark divide between commercially open and closed datasets, with the latter dominating more diverse and creative data sources. Non-commercial and academic-only datasets demonstrated greater diversity in tasks, topics, sources, and target text lengths. A notable 45% of these datasets were synthetic, often generated using APIs with non-commercial terms of use. Language representation was heavily skewed toward English and Western European languages, reflecting a lack of coverage for the Global South. The majority of dataset curation was attributed to academic organizations, industry labs, and research institutions, predominantly located in the United States and China. Analysis of data sources revealed a reliance on online encyclopedias, social media, and the web, with limited representation of commerce, reviews, legal documents, academic papers, and search queries. The study highlighted open legal questions concerning copyright and model training, particularly the applicability of fair use for datasets created specifically for machine learning. The use of OpenAI-generated data was also discussed, noting potential legal ambiguities concerning its terms of use and the implications for training competing models. The DPCollection's data shows that non-commercial datasets tend to contain more diverse tasks (such as brainstorming, explanation, logic, math, and creative writing), topics, and sources than commercially available datasets, which usually focus on short text generation, translation, and classification tasks.

The findings underscore a pressing need for improved data transparency and responsible data practices in AI development. The high rates of license omissions and miscategorizations highlight the risks associated with the current ecosystem. The observed disparity in licensing between commercially open and closed datasets raises concerns about equitable access to diverse training data. The bias in language representation points to potential limitations and biases in models trained on these datasets. The analysis of data sources and licensing practices offers practical guidance for developers and policymakers. The study's tools, particularly the Data Provenance Explorer, offer a mechanism for improved data attribution and informed decision-making. The open-source nature of the tools ensures broader community involvement in promoting responsible AI practices.

This research provides the most extensive public audit of AI data provenance to date, tracing the lineage of over 1,800 text datasets. It reveals a critical situation regarding data licensing and attribution, underscoring a need for enhanced transparency and responsible practices. The study's contribution lies in its creation of the DPCollection and DPExplorer, which equip developers with the tools to make more informed decisions about dataset selection and usage. Future research should explore expanding the audit to other data modalities and incorporate more sophisticated legal analyses to address the complexities of copyright and fair use in the context of AI.

While the study represents a significant advancement in data provenance auditing, some limitations exist. The selection of datasets, though based on widespread adoption, might not fully represent the entire landscape of AI training data. The manual annotation process, while rigorous, remains subject to human error. The legal analysis provided offers interpretations of existing legal frameworks but cannot constitute legal advice. Finally, the study focuses primarily on the English language and may not fully capture nuances in other languages or regions.