| License | CC BY-4.0 |
|---|
This repository contains the material and information associated with our poster presentation and talk at the Bioinformatics Open Source Conference (BOSC) 2026. We are presenting a study evaluating how well large language models (LLMs) can generate NIH-compliant Data Management Plans (DMPs) off-the-shelf.
Data Management Plans (DMPs) are now required by most funders for grant proposals to ensure researchers plan in advance for effective data management and sharing. Preparing a DMP is usually difficult for researchers because they often lack training and knowledge in data management practices. Moreover, funding organizations have different policies that require adherence to specific data management strategies and DMP formats. In this work, we evaluated the performance of Large Language Models (LLMs) in drafting DMPs compliant with the National Institutes of Health (NIH)’s 2023 guidelines. The goal was to investigate whether LLMs were ready, off-the-shelf, to assist biomedical researchers in preparing their DMPs. We evaluated the performance of an open-source LLM, Llama 3.3, and a commercial LLM, GPT 4, in drafting NIH-compliant DMPs. We used comparisons to NIH-provided human-written DMP samples to quantify the performance of Llama and GPT models. We also asked domain experts to evaluate LLM-generated DMPs. Overall, we found that LLMs can accelerate initial draft creation, but would still require expert review before submission to ensure full correctness and compliance. Moreover, open-source LLMs may not be as ready off-the-shelf as commercial ones for drafting funding-compliant DMPs.
| Type | Date & Time | Authors | Session / Location | Details |
|---|---|---|---|---|
| Poster | July 16, 2026, at 10:00-11:00 am and 4:00-4:40 pm | Nahid Zeinali (presenter), Xuebin Dong, Rebecca Hofstein Grady, Maria Praetzellis, Brian Riley, Bhavesh Patel | Poster Session D, Columbia Ballroom | https://www.open-bio.org/events/bosc-2026/bosc-2026-schedule/ |
| Oral Talk | July 15, 2026, 15:20-15:25 pm | Nahid Zeinali (presenter), Xuebin Dong, Rebecca Hofstein Grady, Maria Praetzellis, Brian Riley, Bhavesh Patel | Session 5b: AI | https://www.open-bio.org/events/bosc-2026/bosc-2026-schedule/ |
- Poster #927 — conference poster. (to be added closer to the event)
- Presentation slides — slides of our talk. (to be added closer to the event)
We list here major resources relevant to our poster and talk.
| Description | Link |
|---|---|
| LLM DMP Generation Code | https://github.com/fairdataihub/nih-dmp-llm-generation ; https://doi.org/10.5281/zenodo.20145725 |
| Evaluation & Analysis Code | https://github.com/fairdataihub/nih-dmp-llm-evaluation-paper-code ; https://doi.org/10.5281/zenodo.19462642 |
| Dataset | https://doi.org/10.5281/zenodo.19456208 |
The material in this repository is licensed under a Creative Commons Attribution 4.0 International License.
For submitting feedback or getting in touch:
- GitHub Issues — Submit a question or suggestion
Washington, DC
ISMB 2026 Website | BOSC 2026 Website
ISMB/ECCB 2026 runs July 12–16, 2026, in Washington, DC.
BOSC is a multi-day COSI within ISMB (July 14-15).
CoFest (collaborative sprinting event) follows on July 17–18.
Specific session times for our talks and posters are TBD. Check back as the conference schedule is released.