Skip to content

Commit ab51821

Browse files
committed
Clean up guide references and stabilize citation anchors
- Replace weak or stale active citations with stronger sources for BMI guidance, growthcleanr, code review, iterative refinement, structured output, documentation, privacy, and data dictionary claims - Remove uncited legacy bibliography entries and consolidate duplicate reference records - Convert numeric `refN` anchors to stable semantic source-key anchors across chapters, appendices, and templates - Trim repeated nearby citations while preserving source support at first use - Add a note that reference numbers are display-only and citation links use stable anchors Verification: - Reference integrity check: 45 defined, 45 cited, 0 missing, 0 unused, 0 duplicate - No remaining `#refN` links or numeric reference anchors - `git diff --check` - `bundle exec jekyll build` passed with existing Just the Docs Sass deprecation warnings
1 parent 88ed344 commit ab51821

18 files changed

Lines changed: 448 additions & 450 deletions

‎chapters/Chapter01_PlanAndContext.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ Once the goal and context are clear, the next logical step is to **plan the sub-
3030

3131
Finally, before you even begin to prompt an LLM, it is essential to **set success criteria**. This involves determining what the final deliverables should be and how you will validate whether your solution is successful. In our BMI example, success might be defined by the creation of a clean dataset containing one representative BMI per person, a summary table showing the distribution of individuals across different BMI categories, and a data dictionary that clearly explains the contents of the cleaned dataset. Establishing these criteria upfront allows for an objective evaluation of the LLM's contribution and the overall outcome of your workflow.
3232

33-
For modern coding agents, success criteria should also include **executable verification**. Instead of only writing "the script should clean BMI data," specify what evidence would convince you that the script works: a smoke test on synthetic EHR records, `testthat` unit tests for BMI boundary cases, an expected output schema, and the exact commands the agent should run before it claims completion. Current agent guidance from multiple vendors emphasizes giving agents concrete verification steps and asking for command output or other evidence, not just a verbal assertion that the work is done [102](../docs/References.md#ref102), [103](../docs/References.md#ref103).
33+
For modern coding agents, success criteria should also include **executable verification**. Instead of only writing "the script should clean BMI data," specify what evidence would convince you that the script works: a smoke test on synthetic EHR records, `testthat` unit tests for BMI boundary cases, an expected output schema, and the exact commands the agent should run before it claims completion. Current agent guidance from multiple vendors emphasizes giving agents concrete verification steps and asking for command output or other evidence, not just a verbal assertion that the work is done [GitHub Copilot agent best practices](../docs/References.md#github-copilot-agent-best-practices), [Claude Code best practices](../docs/References.md#claude-code-best-practices).
3434

3535
This planning stage is also where you should define **data and tool boundaries**. For clinical, EHR, or genomic work, decide what the LLM or coding agent is allowed to see before any prompt is written:
3636

@@ -39,7 +39,7 @@ This planning stage is also where you should define **data and tool boundaries**
3939
- Confirm whether your institution, data use agreement, IRB protocol, or sponsor policy allows AI-assisted processing.
4040
- If an agent can browse the web, install packages, call external APIs, or use Model Context Protocol (MCP) tools, document which actions are allowed and which require human approval.
4141

42-
These boundaries are not just administrative details. NIH guidance warns that sharing controlled-access human genomic data with public generative AI tools can violate non-transferability obligations, and HIPAA regulations specify requirements for de-identifying protected health information [109](../docs/References.md#ref109), [110](../docs/References.md#ref110). Tool access also creates a security surface: MCP and LLM security guidance emphasize least-privilege access, validation, and careful handling of sensitive information [107](../docs/References.md#ref107), [108](../docs/References.md#ref108).
42+
These boundaries are not just administrative details. NIH guidance warns that sharing controlled-access human genomic data with public generative AI tools can violate non-transferability obligations, and HIPAA regulations specify requirements for de-identifying protected health information [NIH genomic data guidance](../docs/References.md#nih-human-genomic-data-genai), [45 CFR 164.514](../docs/References.md#hipaa-deidentification-164-514). Tool access also creates a security surface: MCP and LLM security guidance emphasize least-privilege access, validation, and careful handling of sensitive information [MCP security best practices](../docs/References.md#mcp-security-best-practices), [OWASP LLM Top 10](../docs/References.md#owasp-llm-top-10-2025).
4343

4444
Even at this initial planning stage, LLMs themselves can be valuable tools. You can **leverage LLMs in the planning phase** by prompting them to help brainstorm potential sub-tasks or to consider challenges you might encounter. For example, you could ask: "Based on the goal of harmonizing BMI data from EHR files, what are the typical sub-tasks involved in data cleaning and summarization?" The responses from the LLM can offer valuable suggestions and help you structure your workflow more effectively.
4545

@@ -84,16 +84,16 @@ In essence, this foundational planning phase is crucial for setting the stage fo
8484

8585
To see how a high-level plan can evolve over time, refer to these example project plan files in the [docs/templates/](../docs/templates/) folder:
8686

87-
- **[ProjectPlan_InitialDraft.md](../docs/templates/ProjectPlan_InitialDraft.md)**
87+
- **[ProjectPlan_InitialDraft.md](../docs/templates/ProjectPlan_InitialDraft.md)**
8888
A simple, bare-bones plan outlining essential tasks like data ingestion, cleaning, and representative record selection.
8989

90-
- **[ProjectPlan_Improved.md](../docs/templates/ProjectPlan_Improved.md)**
90+
- **[ProjectPlan_Improved.md](../docs/templates/ProjectPlan_Improved.md)**
9191
A more detailed version that covers filtering, validation steps, and additional best practices.
9292

93-
- **[ProjectPlan_Advanced.md](../docs/templates/ProjectPlan_Advanced.md)**
93+
- **[ProjectPlan_Advanced.md](../docs/templates/ProjectPlan_Advanced.md)**
9494
A comprehensive plan that integrates demographic data, applies race-specific BMI categorizations, and includes extensive summary reporting.
9595

96-
- **[RedTeamingPromptExample.md](../docs/templates/RedTeamingPromptExample.md)**
96+
- **[RedTeamingPromptExample.md](../docs/templates/RedTeamingPromptExample.md)**
9797
A template for conducting adversarial review of your project plans to identify potential methodological weaknesses and edge cases.
9898

9999
These examples illustrate how an initial idea can be refined from a broad outline into a structured, domain-specific project plan that supports effective LLM utilization, and how red teaming can strengthen your methodology before implementation.

‎chapters/Chapter02_KnowledgeIsPower.md‎

Lines changed: 20 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,9 @@ nav_order: 2
77

88
# Chapter 2: Knowledge is Power — Do Your Research
99

10-
With a solid plan and a clear understanding of the context established, the next crucial step in our 8-step process is to arm ourselves with the necessary knowledge. Before we even think about prompting an LLM to generate code, we need to **gather foundational knowledge** relevant to each of the sub-tasks we identified in the planning phase [6](../docs/References.md#ref6). This involves researching the methods, libraries, or tools that are commonly used to accomplish these tasks within our respective domains. For instance, in our BMI harmonization example, this would mean investigating appropriate R packages for data cleaning and manipulation, such as `dplyr` or `data.table`, as well as looking into established medical guidelines and definitions for BMI categories [6](../docs/References.md#ref6). This domain-specific knowledge will be invaluable in guiding the LLM later on and in evaluating the quality and appropriateness of its suggestions.
10+
With a solid plan and a clear understanding of the context established, the next crucial step in our 8-step process is to arm ourselves with the necessary knowledge. Before we even think about prompting an LLM to generate code, we need to **gather foundational knowledge** relevant to each of the sub-tasks we identified in the planning phase [bioinformatics LLM review](../docs/References.md#bioinformatics-llm-review). This involves researching the methods, libraries, or tools that are commonly used to accomplish these tasks within our respective domains. For instance, in our BMI harmonization example, this would mean investigating appropriate R packages for data cleaning and manipulation, such as `dplyr` or `data.table`, as well as looking into established medical guidelines and definitions for BMI categories. This domain-specific knowledge will be invaluable in guiding the LLM later on and in evaluating the quality and appropriateness of its suggestions.
1111

12-
Furthermore, it is important to take this time to **clarify uncertainties** that might exist regarding the task at hand [8](../docs/References.md#ref8). Before we ask an LLM for code, we should strive to resolve any ambiguities or domain-specific questions we might have. In our BMI example, this could involve determining what range of BMI values would be considered implausible and should be filtered out, or deciding on the most appropriate method for selecting a single representative BMI measurement for each person if multiple measurements exist in the EHR data. Addressing these uncertainties upfront will lead to more focused and effective prompts when we eventually interact with the LLM.
12+
Furthermore, it is important to take this time to **clarify uncertainties** that might exist regarding the task at hand. Before we ask an LLM for code, we should strive to resolve any ambiguities or domain-specific questions we might have. In our BMI example, this could involve determining what range of BMI values would be considered implausible and should be filtered out, or deciding on the most appropriate method for selecting a single representative BMI measurement for each person if multiple measurements exist in the EHR data. Addressing these uncertainties upfront will lead to more focused and effective prompts when we eventually interact with the LLM.
1313

1414
```mermaid
1515
flowchart TB
@@ -33,14 +33,14 @@ It is also crucial to recognize that **human-friendly definitions** for categori
3333

3434
For example, a real-valued BMI of **39.95** is **less than 40**—which might suggest it belongs in the 35–39.9 bin from a human perspective—but it is also **greater than 39.9** if you compare it literally. Depending on how your code or LLM-generated logic interprets the boundary, you could misclassify or not classify this measurement at all:
3535

36-
- **Rounding vs. Exact Floats:** If you floor BMI to one decimal place, `39.95` becomes `40.0`, thus pushing it out of the 35–39.9 category (contrary to the guideline’s intended 39.9 upper limit).
37-
- **Choosing `<` vs. `<=`:** If your logic says `bmi <= 39.9` for Obesity class 2, then 39.95 is excluded. But if you code it as `bmi < 40`, 39.95 is included.
36+
- **Rounding vs. Exact Floats:** If you floor BMI to one decimal place, `39.95` becomes `40.0`, thus pushing it out of the 35–39.9 category (contrary to the guideline’s intended 39.9 upper limit).
37+
- **Choosing `<` vs. `<=`:** If your logic says `bmi <= 39.9` for Obesity class 2, then 39.95 is excluded. But if you code it as `bmi < 40`, 39.95 is included.
3838
- **Guideline vs. Code Reality:** Guidelines often say “39.9” to highlight that 40.0+ is a higher risk category (Obesity class 3). In code, you may prefer `< 40` for clarity and to avoid floating-point confusion.
3939

4040
### Suggestions for Handling These Subtleties
4141

42-
1. **Be Explicit About Boundaries:** Decide whether to interpret a textual cutoff like “39.9” as `<= 39.9` or `< 40.0`. Document this thoroughly so that future analysts understand the rationale.
43-
2. **Consider Rounding:** If your data is recorded at multiple decimals (e.g., 39.95) but guidelines show a single decimal, clarify how (or if) you round before classification.
42+
1. **Be Explicit About Boundaries:** Decide whether to interpret a textual cutoff like “39.9” as `<= 39.9` or `< 40.0`. Document this thoroughly so that future analysts understand the rationale.
43+
2. **Consider Rounding:** If your data is recorded at multiple decimals (e.g., 39.95) but guidelines show a single decimal, clarify how (or if) you round before classification.
4444
3. **Align Code With Human Logic:** For most clinical or epidemiological scenarios, using `< 40` (rather than `<= 39.9`) better reflects the guideline’s intention without risking floating-point mismatch. Still, you should confirm that this tiny difference does not conflict with domain expectations.
4545

4646
Such attention to detail prevents **unresolvable edge cases** in which the code’s binning disagrees with the intended clinical guideline, leading to misclassification and confusion. Always keep in mind that **guidelines are written for humans**, while your code needs precise comparisons for borderline values. This ensures both clinicians and data scientists understand how an exact BMI of 39.95 is ultimately categorized.
@@ -51,36 +51,36 @@ During the research phase, we may uncover important clinical practice guidelines
5151

5252
1. **European Association for the Study of Obesity (EASO)** – The EASO offers position statements and guidelines discussing the use of BMI and related anthropometric measures in both clinical and epidemiological settings. This information can help frame how BMI (as well as height and weight) should be interpreted in population studies. It also provides insight into **epidemiological nuances behind BMI thresholds**, including considerations tailored to different demographic groups.
5353

54-
2. **NICE Guideline on Overweight and Obesity Management** (Reference number: NG246, published 14 January 2025, last updated 8 January 2026) – This guideline provides a detailed classification of overweight and obesity in adults [9](../docs/References.md#ref9), including:
55-
- **Healthy weight:** BMI 18.5 kg/m<sup>2</sup> to 24.9 kg/m<sup>2</sup>
56-
- **Overweight:** BMI 25 kg/m<sup>2</sup> to 29.9 kg/m<sup>2</sup>
57-
- **Obesity class 1:** BMI 30 kg/m<sup>2</sup> to 34.9 kg/m<sup>2</sup>
58-
- **Obesity class 2:** BMI 35 kg/m<sup>2</sup> to 39.9 kg/m<sup>2</sup>
59-
- **Obesity class 3:** BMI 40 kg/m<sup>2</sup> or more
54+
2. **NICE Guideline on Overweight and Obesity Management** (Reference number: NG246, published 14 January 2025, last updated 8 January 2026) – This guideline provides a detailed classification of overweight and obesity in adults [NICE NG246](../docs/References.md#nice-ng246-bmi), including:
55+
- **Healthy weight:** BMI 18.5 kg/m<sup>2</sup> to 24.9 kg/m<sup>2</sup>
56+
- **Overweight:** BMI 25 kg/m<sup>2</sup> to 29.9 kg/m<sup>2</sup>
57+
- **Obesity class 1:** BMI 30 kg/m<sup>2</sup> to 34.9 kg/m<sup>2</sup>
58+
- **Obesity class 2:** BMI 35 kg/m<sup>2</sup> to 39.9 kg/m<sup>2</sup>
59+
- **Obesity class 3:** BMI 40 kg/m<sup>2</sup> or more
6060

6161
Crucially, it underscores that people of **South Asian, Chinese, Middle Eastern, Black African, or African–Caribbean** backgrounds are prone to central adiposity and may present higher cardiometabolic risk at **lower BMI thresholds**:
62-
- **Overweight:** BMI 23 kg/m<sup>2</sup> to 27.4 kg/m<sup>2</sup>
63-
- **Obesity:** BMI 27.5 kg/m<sup>2</sup> or above
62+
- **Overweight:** BMI 23 kg/m<sup>2</sup> to 27.4 kg/m<sup>2</sup>
63+
- **Obesity:** BMI 27.5 kg/m<sup>2</sup> or above
6464

65-
Classes 2 and 3 of obesity in these populations can be identified by subtracting 2.5 kg/m<sup>2</sup> from the mainstream thresholds. These racial- and ethnicity-specific cutoffs are highly relevant when harmonizing multi-ethnic EHR data.
65+
Classes 2 and 3 of obesity in these populations can be identified by subtracting 2.5 kg/m<sup>2</sup> from the mainstream thresholds. These racial- and ethnicity-specific cutoffs are highly relevant when harmonizing multi-ethnic EHR data.
6666

6767
Integrating such guidelines into our workflow ensures **proper categorization** of BMI across diverse populations, enhancing both the clinical relevance of our analysis and the robustness of any downstream results.
6868

6969
## Leveraging Tools for Data Cleaning and Validation
7070

7171
In addition to reviewing guidelines, we should seek out robust packages or software that can streamline data cleaning. One such tool is:
7272

73-
**`growthcleanr`** – An R package focused on cleaning anthropometric measurements (height, weight, and derived BMI) from EHR systems [10](../docs/References.md#ref10). It implements a series of algorithms to flag implausible values without deleting them outright. While its original emphasis was on pediatric growth curves, it has been expanded to handle adult measurements. Highlights include:
73+
**`growthcleanr`** – An R package focused on cleaning anthropometric measurements (height, weight, and derived BMI) from EHR systems [growthcleanr manual](../docs/References.md#growthcleanr-manual). It implements a series of algorithms to flag implausible values without deleting them outright. While its original emphasis was on pediatric growth curves, it has been expanded to handle adult measurements. Highlights include:
7474

75-
- **Biologically implausible value detection**: Uses patient-specific longitudinal analysis, outlier flagging, and thresholds derived from known growth curve references.
76-
- **Adult algorithm**: Accommodates stable height and weight fluctuations in adults and flags erroneous or extreme measurements.
77-
- **Easy integration**: Offers additional utilities to calculate Z-scores and percentiles, potentially saving a great deal of time in data preprocessing.
75+
- **Biologically implausible value detection**: Uses patient-specific longitudinal analysis, outlier flagging, and thresholds derived from known growth curve references.
76+
- **Adult algorithm**: Accommodates stable height and weight fluctuations in adults and flags erroneous or extreme measurements.
77+
- **Easy integration**: Offers additional utilities to calculate Z-scores and percentiles, potentially saving a great deal of time in data preprocessing.
7878

7979
Because `growthcleanr` specifically targets **clinical anthropometric data**, it aligns well with tasks such as identifying outlier BMI values in EHR-based cohorts. By using such a tool, you can systematically flag data issues and maintain consistent filtering methods in your workflow.
8080

8181
## Refining the Plan Based on Research
8282

83-
The information we gather during this research phase should then be used to **refine our approach based on research**. This means taking the insights gained from authoritative guidelines, specialized tools, and any domain expertise to update your initial plan before coding. For example, the NICE guideline might prompt you to apply **lower BMI thresholds** for certain ethnic populations [9](../docs/References.md#ref9), and `growthcleanr` could help automate outlier detection [10](../docs/References.md#ref10). Integrating these best practices ensures that your final solution—once you prompt the LLM—is both clinically sound and methodologically rigorous. The result might look like this: **[ProjectPlan_Advanced.md](../docs/templates/ProjectPlan_Advanced.md)**
83+
The information we gather during this research phase should then be used to **refine our approach based on research**. This means taking the insights gained from authoritative guidelines, specialized tools, and any domain expertise to update your initial plan before coding. For example, the NICE guideline might prompt you to apply **lower BMI thresholds** for certain ethnic populations [NICE NG246](../docs/References.md#nice-ng246-bmi), and `growthcleanr` could help automate outlier detection [growthcleanr manual](../docs/References.md#growthcleanr-manual). Integrating these best practices ensures that your final solution—once you prompt the LLM—is both clinically sound and methodologically rigorous. The result might look like this: **[ProjectPlan_Advanced.md](../docs/templates/ProjectPlan_Advanced.md)**
8484

8585
## Conclusion
8686

0 commit comments

Comments
 (0)