33
44<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
55
6- Why my ` nbs/sidebar.yml ` file is erased when running ` nbdev_prepare ` ?
7-
8- Here is my settings.ini:
9-
10- ![ pasted_image_139aef0e-db47-4fc7-92b5-0d761129545c.png] ( index_files/figure-commonmark/37de56d3-1-139aef0e-db47-4fc7-92b5-0d761129545c.png )
11-
12- ##### 🤖Reply🤖<!-- SOLVEIT_SEPARATOR_7f3a9b2c -->
13-
14- Looking at your ` settings.ini ` , I can see the issue. You have:
15-
16- custom_sidebar = True
17-
18- When ` custom_sidebar = True ` , nbdev expects you to manually maintain
19- your ` sidebar.yml ` and won’t overwrite it. However, if the file doesn’t
20- exist or gets erased, nbdev might be regenerating it.
6+ [ ![ PyPI] ( https://img.shields.io/pypi/v/evaluatr.png )] ( https://pypi.org/project/evaluatr/ )
7+ [ ![ License] ( https://img.shields.io/badge/license-MIT-blue.svg )] ( LICENSE )
8+ [ ![ Documentation] ( https://img.shields.io/badge/docs-GitHub%20Pages-blue.png )] ( https://franckalbinet.github.io/evaluatr/ )
219
22- The problem is likely that you also have:
10+ ## What is Evaluatr?
2311
24- doc_host = https://francklabinet.github.io
25- doc_baseurl = /evaluatr/
12+ ` Evaluatr ` is an AI-powered system that automates mapping evaluation
13+ reports against structured frameworks while maintaining interpretability
14+ and human oversight. Initially developed for [ IOM (International
15+ Organization for Migration)] ( https://www.iom.int ) evaluation reports and
16+ the [ Strategic Results Framework (SRF)] ( https://srf.iom.int ) , it
17+ transforms a traditionally manual, time-intensive process into an
18+ efficient, transparent workflow.
2619
27- But your repository structure might not match what nbdev expects. Try
28- this:
20+ The system maps evaluation reports against hierarchical frameworks like
21+ the SRF (objectives, enablers, cross-cutting priorities, outcomes,
22+ outputs, indicators) and connects to broader frameworks like the
23+ [ Sustainable Development Goals (SDGs)] ( https://sdgs.un.org ) for
24+ interoperability.
2925
30- 1 . Set ` custom_sidebar = False ` temporarily to let nbdev generate the
31- sidebar
32- 2 . Run ` nbdev_prepare `
33- 3 . Once generated, modify the sidebar as needed
34- 4 . Then set ` custom_sidebar = True ` again
26+ Beyond automation, ` Evaluatr ` prioritizes ** interpretability and
27+ human-AI collaboration ** —enabling evaluators to understand the mapping
28+ process, audit AI decisions, perform error analysis, and build training
29+ datasets over time, ensuring the system aligns with organizational needs
30+ through actionable, transparent, auditable methodology.
3531
36- Alternatively, if you want to keep ` custom_sidebar = True ` , make sure
37- your ` nbs/sidebar.yml ` file exists and has valid YAML content before
38- running ` nbdev_prepare ` . Even a minimal valid structure like:
32+ ## The Challenge We Solve
3933
40- ``` yaml
41- website :
42- sidebar :
43- contents :
44- - index.ipynb
45- ` ` `
34+ IOM evaluators possess deep expertise in mapping evaluation reports
35+ against frameworks like the Strategic Results Framework (SRF), but face
36+ significant operational challenges when processing reports that often
37+ exceed 150 pages of diverse content across multiple projects and
38+ contexts.
4639
47- Would you like me to help you create a proper ` sidebar.yml` structure?
40+ The core challenges are:
4841
49- [](https://pypi.org/project/evaluatr/)
50- [](LICENSE)
51- [](https://franckalbinet.github.io/evaluatr/)
42+ - ** Time-intensive process** : Hundreds of staff-hours required per
43+ comprehensive mapping exercise
44+ - ** Individual consistency** : Even expert evaluators may categorize the
45+ same content differently across sessions
46+ - ** Cross-evaluator consistency** : Different evaluators may interpret
47+ and map identical content to different framework outputs
48+ - ** Scale vs. thoroughness** : Growing volume of evaluation reports
49+ creates pressure to choose between speed and comprehensive analysis
5250
5351## Understanding Evaluation Mapping in the UN Context
5452
@@ -80,47 +78,6 @@ relevant reports) and recall (avoiding irrelevant ones).
8078> Throughout this documentation, we use “mapping” and “tagging”
8179> interchangeably.
8280
83- # # The Challenge We Solve
84-
85- IOM evaluators possess deep expertise in mapping evaluation reports
86- against frameworks like the Strategic Results Framework (SRF), but face
87- significant operational challenges when processing reports that often
88- exceed 150 pages of diverse content across multiple projects and
89- contexts.
90-
91- The core challenges are :
92-
93- - **Time-intensive process**: Hundreds of staff-hours required per
94- comprehensive mapping exercise
95- - **Individual consistency**: Even expert evaluators may categorize the
96- same content differently across sessions
97- - **Cross-evaluator consistency**: Different evaluators may interpret
98- and map identical content to different framework outputs
99- - **Scale vs. thoroughness**: Growing volume of evaluation reports
100- creates pressure to choose between speed and comprehensive analysis
101-
102- # # What is Evaluatr?
103-
104- ` Evaluatr` is an AI-powered system that automates mapping evaluation
105- reports against structured frameworks while maintaining interpretability
106- and human oversight. Initially developed for [IOM (International
107- Organization for Migration)](https://www.iom.int) evaluation reports and
108- the [Strategic Results Framework (SRF)](https://srf.iom.int), it
109- transforms a traditionally manual, time-intensive process into an
110- efficient, transparent workflow.
111-
112- The system maps evaluation reports against hierarchical frameworks like
113- the SRF (objectives, enablers, cross-cutting priorities, outcomes,
114- outputs, indicators) and connects to broader frameworks like the
115- [Sustainable Development Goals (SDGs)](https://sdgs.un.org) for
116- interoperability.
117-
118- Beyond automation, `Evaluatr` prioritizes **interpretability and
119- human-AI collaboration**—enabling evaluators to understand the mapping
120- process, audit AI decisions, perform error analysis, and build training
121- datasets over time, ensuring the system aligns with organizational needs
122- through actionable, transparent, auditable methodology.
123-
12481## Key Features
12582
12683### 1. Document Preparation Pipeline ✅ ** Available**
@@ -210,7 +167,7 @@ model provider beyond the examples above.
210167
211168## Quick Start
212169
213- # ## IOM Workflow (Programmatic)
170+ ### Reading & standardizing evaluations repository
214171
215172For IOM evaluators working with the official evaluation repository,
216173download the most recent evaluations from
@@ -231,9 +188,37 @@ evals[0]
231188** Year:** 2025 \| ** Organization:** IOM \| ** Countries:** Worldwide
232189
233190** Documents:** 2 available
234- **ID:** `99923109...`
191+ ** ID:** ` 9992310969aa2f428bc8aba29f865cf3 `
192+
193+ To find a particular evaluation by ` title ` or ` url ` :
194+
195+ ``` python
196+ from evaluatr.readers import find_eval
197+
198+ title = ' Evaluation of IOM Accountability to Affected Populations'
199+ find_eval(evals, title, by = ' title' )
200+ ```
201+
202+ ### Evaluation of IOM Accountability to Affected Populations
203+
204+ ** Year:** 2025 \| ** Organization:** IOM \| ** Countries:** Worldwide
205+
206+ ** Documents:** 4 available
207+ ** ID:** ` 6c3c2cf3fa479112967612b0baddab72 `
208+
209+ ``` python
210+ url = " https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/AAP%20E valuation%20Report_final_.pdf"
211+ find_eval(evals, url, by = ' url' )
212+ ```
213+
214+ ### Evaluation of IOM Accountability to Affected Populations
215+
216+ ** Year:** 2025 \| ** Organization:** IOM \| ** Countries:** Worldwide
217+
218+ ** Documents:** 4 available
219+ ** ID:** ` 6c3c2cf3fa479112967612b0baddab72 `
235220
236- ** Downloading Evaluation Documents**
221+ ### Downloading Evaluation Documents
237222
238223``` python
239224from evaluatr.downloaders import download_docs
@@ -244,7 +229,7 @@ base_dir = Path("files/test/pdf_library")
244229download_docs(fname, base_dir = base_dir, n_workers = 0 , overwrite = True )
245230```
246231
247- # ## Universal CLI Workflow
232+ ### Command Line Interface ( CLI) Workflow
248233
249234Process any evaluation report from PDF to tagged outputs using three
250235streamlined commands.
0 commit comments