Skip to content

Latest commit

 

History

History
126 lines (94 loc) · 5.55 KB

File metadata and controls

126 lines (94 loc) · 5.55 KB

02 — Dataset design

What an RLFT dataset is not

An RLFT example carries no target response. The schema is:

{
  "systemInstruction": {"role": "system", "parts": [{"text": "..."}]},
  "contents":          [{"role": "user",   "parts": [{"text": "..."}]}],
  "references":        {"key": "value"}
}

That absence is the whole point. Supervised fine-tuning on 843 poems would teach the model to reproduce those 843 poems. Here the corpus supplies only (a) realistic task specifications and (b) the per-example fingerprint that stops the policy from reciting the original. The poem itself is never shown to the model.

Source corpus

«مدوّنة الشعر الشعبي التونسي» — the 10-volume corpus of Tunisian folk poetry, OCR'd and classified by a separate pipeline that emits classified.json: 3,757 scanned pages → 2,224 units → 1,511 with one of the four usul.

The corpus is not vendored into this repository. It is supplied at runtime by path. Only two derived artifacts ship here:

  • src/tunifolk/resources/tunisianity.json.gz — contrastive n-gram weights, not text;
  • per-example source_fingerprint — irreversible 48-bit hashes of lines and n-grams.

Neither reconstructs the poems. The tuning dataset in GCS contains prompts and hashes, no corpus verse.

Prompt families

Two, both attested tasks in the living tradition:

compose — form + purpose only. Mirrors a poet at a maḥfal being asked for "a malzūma in al-akhḍar".

continue — form + purpose + a required opening hemistich. Mirrors the convention that later poets in a session keep the gharaḍ the first one set — «الغناء بالضدّ، اللي سمعت ردّ».

Each prompt restates the rhyme topology the reward will actually measure. That is deliberate: an RL task the policy cannot infer from the prompt is a guessing game, and the gradient goes into decoding the instruction rather than into the craft.

انظم الملزومة في غرض «الأخضر/الغزل».

ملزومة: تبدأ بطالع يضبط القافية الأمّ، ثمّ أعراف (مقاطع). لكلّ عُرف قافية
داخلية خاصّة به، ويُختم كلّ عُرف بغصن يرجع إلى قافية الطالع (الرجوع). …

ولا تقلّ القصيدة عن 14 شطرًا.

The output contract

The system instruction fixes an output format so the reward can parse it: one hemistich per line, blank line between strophes, no commentary, no tashkīl. parse_poem is additionally tolerant of markdown fences and the *** stanza rule, so a well-formed poem is never punished for cosmetic wrapping.

The system instruction also sets the persona and, importantly, the register: "دارجة تونسية أدبية كلغة المدوّنة، لا فصحى مدرسية ولا عامية عصرية" — literary Tunisian as the corpus writes it, neither schoolroom MSA nor modern street Derja.

references

Everything the reward needs, as a flat string→string map:

key purpose
form which validator to apply (qasim | malzuma | mawqif | musaddas)
form_ar, gharad surfaced to the autorater prompt via {{references.gharad}}
min_lines the length ramp in the structural scorer
mode compose | continue, for slicing metrics afterwards
source_uid provenance, for error analysis
source_fingerprint novelty guard
wazn_sub recorded, deliberately not scored (see 03)

Selection and stratification

Attestation filter. Only poems that pass their own form validator at ≥0.6 become prompts: 1,511 → 889. Only ~41% of poems labelled malzuma actually exhibit the ruju' that defines the form, because the automatic classifier defaults to the majority class and OCR strophe breaks are imperfect. Asking the policy for a structure that most genuine examples fail would be optimising a target the tradition does not attest. Implemented in tunifolk.rl.dataset.attested.

Stratification by form. malzuma and qasim dominate the corpus; mawqif and musaddas have tens of exemplars. Rather than truncating the head forms, every source poem is kept and the rare forms are oversampled: any form with fewer than 200 source poems emits both prompt modes per poem. The emitted training set is 51.1 % malzuma, 39.1 % qasim, 7.4 % mawqif, 2.4 % musaddas.

Split by source poem, not by example. A poem never contributes to both train and validation — otherwise its fingerprint would leak across the split and validation novelty would be measured against a poem the policy had already been rewarded for avoiding.

Result

train validation
examples 843 174
size 2.4 MB 0.5 MB
qasim 330 72
malzuma 431 94
mawqif 62 6
musaddas 20 2
compose / continue 402 / 441 87 / 87

Well inside the service limits (≤5,000 train, ≤500 validation, ≤1 GB, ≤32,768 tokens per example). scripts/build_dataset.py asserts every one of those before upload rather than discovering a violation at job-creation time.

A note on scale

843 prompts is small for RL. It is bounded by the corpus, not by choice: 889 attested poems is what the Mudawwana yields after honest filtering. The mitigation is samplesPerPrompt: 16 — 16 sampled completions per prompt per step means the gradient comes from 13,488 scored generations per epoch, not 843. The binding constraint on this project is prompt diversity, not sample count, and that is a corpus limit we report rather than paper over.