Skip to content
This repository was archived by the owner on Aug 6, 2026. It is now read-only.

0.10.md

Latest commit

 

History

History
81 lines (61 loc) · 3.41 KB

0.10.md

File metadata and controls

81 lines (61 loc) · 3.41 KB
Raw

0.10 — Turkish Morphological Pipeline: Results

Version: 0.10 Date: 2026-05-22 Status: Complete — Sprint 3.1 deliverable

Summary

Built an end-to-end pipeline that simulates Zemberek morphological analysis for Turkish, constructs Nested Semantic Trees from morphological parses, runs subtree search queries against a 15-document corpus, and evaluates with precision/recall/MRR metrics.

Architecture

Turkish sentence → ZemberekSimulator.analyze_sentence()
    → [morpheme analyses per word]
    → build_nst_from_morphology()
    → NestedSemanticTree (VERB_PHRASE, SUBJECT, OBJECT, ADJUNCT subtrees)
    → search_corpus()
    → ranked results
    → evaluate_searches()
    → P@k, R@k, F1@k, MRR

Key Results

Ultrametric Property Verification

All 15 Turkish documents satisfy the strong triangle inequality:

Status Count
PASS (0 violations) 15/15
FAIL 0/15

This confirms that the NST construction from morphological parses preserves ultrametric structure. Turkish agglutination, when modeled as hierarchical morpheme attachment, produces trees that are fully ultrametric.

Search Evaluation (10 queries)

Query P@5 R@5 F1@5 MRR Relevant
Q_buy_bread (ekmek aldi) 0.60 0.75 0.67 1.00 D1-D4
Q_woman_buy (kadin ekmek aldi) 0.40 1.00 0.57 1.00 D1, D3
Q_store (dukkan) 0.40 0.50 0.44 0.50 D2,D3,D7,D11
Q_man_buy (adam ekmek aldi) 0.20 1.00 0.33 0.50 D2
Q_give (verdi) 0.20 0.50 0.29 0.20 D5, D15
Q_come (geldi) 0.00 0.00 0.00 0.17 D6, D8
Q_go (gitti) 0.00 0.00 0.00 0.14 D7
Q_open (acti) 0.00 0.00 0.00 0.08 D13,D14
Q_see (gordu) 0.00 0.00 0.00 0.11 D9-D11
Q_woman_see_man 0.00 0.00 0.00 0.12 D9

Aggregate: Mean P@5 = 0.180, Mean R@5 = 0.375, Mean F1@5 = 0.230, Mean MRR = 0.382

Analysis

What works:

  • Queries matching multi-word patterns with direct tree correspondence (buy + bread, woman + buy + bread) get good results
  • The MRR for these queries (1.00 for Q_buy_bread, Q_woman_buy) shows the top result is consistently relevant
  • Store location queries (Q_store) successfully retrieve documents where store appears in any case role

What doesn't work (yet):

  • Single-verb queries (Q_come, Q_go, Q_open, Q_see) score zero because the exact subtree match requires the verb root node + its tense/person children, but queries constructed from isolated words produce trees that differ structurally from corpus trees
  • This is a known limitation of Type I (exact subtree) matching, not a pipeline failure

Path forward:

  • S3.2 (Evaluation Framework) should formalize these metrics across all 3 match types
  • Type II partial matching (already in 0.5.py) would boost recall on single-verb queries
  • Type III tree edit distance (0.8.py) would handle the morphological variation

Files

File Purpose
0.10.py Pipeline implementation (Zemberek simulator, NST builder, search, evaluation)
0.10.md This document — results and analysis

Grounding

  • 0.4.md: Cross-linguistic examples — Turkish, Mohawk, English map to same NST
  • 0.5.py: NST framework + Type I/II matching + ranking engine
  • 0.8.py: Type III tree edit distance (future integration)
  • Zemberek: https://github.com/ahmetaa/zemberek-nlp (simulated, not installed)