3. Problem Definition¶

3.1 The Research Data Challenge¶

The research community faces significant barriers to making data outputs FAIR (Findable, Accessible, Interoperable and Reusable). The following challenges were identified through the global community consultation on agentic AI in research conducted by the RDA and Microsoft in November 2025 and further developed and prioritised during the Blueprint programme.

Each is framed as a gap, a defined deficit between current practice and what is needed to share data effectively, providing a structured basis for the solutions and recommendations that follow. These gaps are wide-ranging and deeply embedded in research culture, infrastructure, and incentive structures. The Blueprint does not claim to address them in full; rather, it responds to the most common and pressing challenges identified by the community, providing a foundation that implementers can build on. The gaps are presented here to set the scene and establish the context for the Data Director's design, not to imply that a single tool can resolve them entirely. The gaps are presented here in order of community priority.

3.1.1 Core Gaps: Directly addressed by the Data Director¶

Capacity Gap: Researchers lack the time and expertise to manage data responsibly across the research lifecycle

A central challenge is that researchers lack both the training and the incentive to produce high-quality metadata. Throughout this Blueprint, metadata is understood in its full scope, encompassing not only discovery fields but variable definitions, data structure, provenance, quality assessments, access conditions, and applicable governance constraints. Metadata creation is not typically part of academic training or curricula, meaning many researchers do not know what good metadata looks like, how to produce it, or why it matters. They are not rewarded for investing time in it, and where a dataset is produced by a cross-disciplinary or cross-institutional team, priorities around metadata quality are not always shared. The result is that researchers tend to fulfil only the minimum requirements, with the burden of support falling on libraries and institutions. This reflects a deeper cultural issue: research data management is not yet treated as a core research responsibility, but as an administrative obligation to be minimised.

This challenge spans the full research lifecycle, not only the point of data publication. Decisions made early in a project, such as file formats, storage, and documentation practice, have significant consequences for whether data can ultimately be shared and reused, yet researchers are often unaware of these dependencies until problems arise. This is becoming more acute as AI systems are increasingly used to process research data, making metadata precision a prerequisite not only for discoverability but for scientific replicability and accountability.

A related challenge is that many researchers are not familiar with community-recognised repositories, standards registries, or vocabulary services, and lack a reliable basis for evaluating the trustworthiness of the resources they do encounter. Data Management Plans (DMPs) illustrate the broader problem well: researchers struggle to navigate competing institutional, funder, and collaborator requirements, to balance open science with research security and data protection, and to keep DMPs current as projects evolve.

Standards Gap: Inconsistency in metadata standards creates significant barriers to data sharing and reuse

There is no universally agreed interpretation of the FAIR principles, though the principles themselves provide a broadly accepted direction of travel that this Blueprint adopts as its orienting framework. What constitutes a complete metadata record varies significantly between communities and data repository types. This variability is compounded by the growth of interdisciplinary research, which requires researchers to navigate multiple, sometimes conflicting, standards frameworks simultaneously. The difficulty of gaining a reliable overview of the standards relevant to any given discipline compounds this further.

Where standard vocabularies or ontologies do not exist, these challenges deepen further. The harmonisation of heterogeneous metadata schemas, the conversion between them, and the consistent definition of variables, frequently overlooked but fundamental to interoperability, remain largely unresolved and under-supported. Efforts to achieve standardisation have historically struggled, as community-specific needs often sit in tension with universal solutions.

These inconsistencies affect data discovery, making it difficult to locate relevant data outside one's own discipline, and carry equity implications: the costs of meeting evolving standards may disadvantage smaller repositories, risking the loss of important community resources.

Support Gap: Researchers lack access to skilled human support

Access to professional data support, such as data stewards or librarians, is unequally distributed across the research community. While smaller institutions are most visibly affected, the challenge is broader: large, devolved organisations face similar problems due to asymmetrical provision across faculties and disciplines. Even where support staff exist, they typically offer general guidance rather than hands-on assistance with tasks such as metadata creation. Access to timely support is a further constraint. Researchers frequently need help at short notice, and availability cannot be assumed.

Many researchers are unaware that support services exist at all, even at well-resourced institutions. Where support is available, it tends to concentrate around the point of data publication, with limited sustained curation effort afterwards, particularly as new standards emerge that may affect already-published datasets. A significant volume of research data has already been published without FAIR-aligned metadata, and the remediation of this existing material represents an important but largely unmet need.

Policy Gap: Researchers face uncertainty navigating divergent and sometimes conflicting data policies

The challenge here is broader than data repository selection alone. Researchers often lack clarity about what they are expected or required to do when publishing research data, and must navigate institutional, funder, and journal requirements that do not always align. Inconsistency in policy communication is as much an issue as policy conflict itself. Where institutions provide clear direction, the problem is significantly reduced.

Data repository selection is one practical manifestation of this challenge. The repository landscape is fragmented, leaving researchers without reliable criteria to guide their choices and forcing many to rely on general search engines as a starting point. Despite longstanding community efforts, consensus on what constitutes a good data repository has not been reached. The cost of data storage adds further complexity, as funding constraints can limit available options regardless of policy requirements. Journals also play a role here: many journals mandate that research data underpinning a journal article publication must be deposited in a specific repository or meet particular standards. These journal data policies must be considered alongside funder and institutional requirements when selecting a repository for data publication and can introduce additional or conflicting constraints.

The gap is best understood as one of comprehension: researchers struggle to understand the full range of requirements they are expected to meet, rather than facing simply a question of which data repository to use.

3.1.2 Boundary Conditions: Gaps that shape how the Data Director should behave¶

Quality and Provenance Gap: Researchers and tools must be able to demonstrate the trustworthiness and traceability of data and metadata

A core challenge is provenance: the need to know where a result comes from, how it was produced, and whether it can be trusted as the basis for new work. This concern is growing in urgency as increasing volumes of data and metadata are AI-generated, making traceability more critical and more difficult simultaneously. Any output from an agentic AI tool must include basic provenance information: the sources used, the model involved, the processing date, and a persistent identifier, as well as a record of whether human curation has taken place.

Closely related is the challenge of quality: both the quality of metadata, assessed against community standards and canonical semantics (a standard, universal representation of meaning and structure across datasets), and the quality of the data itself, which should be reflected as an explicit field within the metadata record. These two challenges are deeply interconnected: poor provenance undermines quality assessment, and poor quality metadata makes provenance harder to establish.

Provenance is already a core component of the FAIR principles, meaning tools that support FAIR-aligned metadata creation should produce appropriate provenance records as a matter of course. As AI plays a greater role in research data workflows, robust and standardised approaches to recording provenance and assessing quality become a fundamental requirement for research integrity.

Governance Gap: Researcher uncertainty about how to prepare data for sharing under ethical, legal, cultural or proprietary constraints

A significant challenge is researcher uncertainty about how to prepare data for sharing where ethical, legal, cultural, or commercial constraints apply. This uncertainty extends to data privacy, affecting not only researchers but data support professionals themselves. At its core, the gap reflects a lack of clarity about what is required and permissible when handling sensitive data, particularly around consent, confidentiality, and reuse. Cross-national research compounds this further, requiring researchers to navigate differing and sometimes conflicting requirements around data protection, rights holder obligations, and permitted reuse across jurisdictions.

The CARE principles for Indigenous data¹ represent a specific and important dimension of this gap. Questions have been raised about whether agentic AI is compatible with CARE principles at all, and how alignment between the two might be achieved. Meaningful engagement with the CARE principles requires direct engagement with relevant Indigenous and community stakeholders and may involve interaction with external systems such as Local Contexts.² This remains a complex but important area for consideration in the development of any tool operating in this space.

Research data governance considerations are best addressed at the data management planning stage. However, the degree to which an AI tool can support governance decisions in a meaningful and trustworthy way is limited, particularly for edge cases that are difficult even for experienced human practitioners, and human oversight remains essential.

3.1.3 Acknowledged Gaps: Not directly addressed by the Data Director¶

Sustainability Gap: Difficulty funding and sustaining research data infrastructure at institutional scale

While the Data Director supports PID assignment as part of its core functionality, the broader challenge of funding and sustaining PID infrastructure at institutional scale falls outside its scope. In practice, the cost burden varies considerably; some repositories integrate PID assignment directly at no additional cost, and the significance of this as a barrier differs across institutions and contexts.

More pressing are the wider infrastructure challenges faced particularly by smaller institutions, where limited technical knowledge and restricted access to IT specialists make it difficult to automate even basic research data management processes. The longer-term sustainability of repositories hosting research data, including appropriate recognition and credit for those services, is also a concern that falls beyond what the Data Director can address.

There is a risk that intensive delegation to the agentic AI tools, such as the Data Director, could create dependency and hinder the development of in-house data management capacity over time. This is an important consideration for institutions deploying the tool.

Recognition Gap: Academic reward systems do not adequately recognise data outputs or data management efforts

Academic reward systems are weighted towards journal article publications, leaving research data management and sharing largely unrecognised. The immaturity of data citation metrics makes it difficult to demonstrate the impact of data sharing, creating a structural disincentive to investing in good research data management practice. While this gap cannot be addressed through tooling alone, reducing the effort required to share data well and providing researchers with metrics on how their data is being reused can lower barriers and offer a degree of recognition within the current system.

The Sustainability and Recognition Gaps are documented here to provide a complete picture of the research data landscape. They fall outside the scope of the Data Director in its current form but may be considered in future iterations or addressed by other tools and systems in development.

The four core gaps: Capacity, Standards, Support, and Policy, define the primary research data challenges the Data Director is designed to address. The Quality and Provenance and Governance Gaps establish the boundary conditions that shape how the tool should behave. Together, these six gaps provide the evidence base for the functional specification that follows.

3.2 Vision and Goals¶

The Data Director will support any researcher, regardless of institution, discipline, or level of data expertise, and the data support professionals who assist them, to prepare and deposit research data for publication in alignment with the FAIR principles, narrowing the gap between those with access to skilled data support and those without.

The following goals represent the primary starting point for the Data Director in this iteration. They are not exhaustive; further goals and capabilities are identified throughout the Blueprint and in the roadmap for future iterations in Section 13.2.

Simplify data repository selection

The Data Director will recommend appropriate data repositories based on funder, journal, and institutional requirements, so that researchers can make well-informed deposit decisions with confidence, regardless of their familiarity with the policy landscape.

Reduce the burden of research data documentation and metadata creation

The Data Director will enable researchers to select the most appropriate community and domain-relevant standards and schemas for their data, and automatically suggest metadata and supporting documentation, including data dictionaries and README files, so that researchers can produce FAIR-aligned records without relying on specialist data support.

Support the assignment of persistent identifiers (PIDs) for datasets

The Data Director will support the assignment of persistent identifiers (PIDs) for research datasets, so that data outputs are citable, discoverable, and persistently accessible regardless of the researcher's institutional resources.

Connect datasets to the broader research landscape

The Data Director will link datasets to related research outputs, including journal article publications, grants, software, and people, so that research data is discoverable in context and its contribution to the wider research record is visible.

3.3 Out of Scope¶

While the Data Director addresses several key challenges researchers face when depositing research data for publication, this Blueprint iteration has defined boundaries.

The Data Director does not:

Support comprehensive data management across the full research data lifecycle; it is designed to support research data publication only.
Replace human expertise, substitute for specialist research data management professionals, or assess the scientific validity or methodological soundness of research data.
Take irreversible actions, including depositing research data for publication, without explicit human approval.
Advise on how to meet legal, ethical, or governance requirements; it may flag relevant requirements, but researchers and institutions remain responsible for decisions that depend on local context and specialist knowledge. Note that the governance requirements shaping how the Data Director itself is designed and deployed are a separate matter, addressed in Section 5.
Store, preserve, or provide repository infrastructure; it supports the preparation and submission of data for deposit but does not retain data itself.

The full boundary between what is included in this iteration and what may be reserved for future development is set out in the uses table in Section 4.1.