text-forensics-toolkit

A lightweight text-forensics and stylometry toolkit implemented in Clojure, featuring:

• Linguistic profiling
• Education-level estimation
• Regional (UK/US) English detection
• Psycholinguistic signal analysis
• Authorship similarity using cosine distance

Originally developed during an exploration of computational linguistics, this project is now organized as a standalone text-analysis toolkit.

---

🚀 Features

🔹 Linguistic Profiling

Extracts core linguistic metrics:

• Total words
• Unique vocabulary count
• Type–Token Ratio (TTR)
• Sentence count
• Average sentence length
• POS tag distribution (via OpenNLP)

---

🔹 Education-Level Estimation

Estimates the writer’s educational background using:

• Lexical diversity
• Syntactic complexity
• Adjective/adverb density

---

🔹 Regional English Detection

Identifies lexical tendencies toward:

• 🇬🇧 UK English — colour, organise, theatre, centre
• 🇺🇸 US English — color, organize, theater, center

---

🔹 Psycholinguistic State Analysis

Extracts emotional and stylistic cues, analyzing:

• Positive/negative sentiment words
• Negations
• First-person usage
• Passive sentence frequency
• Overall mental/emotional tendencies

---

🔹 Authorship Similarity

Compares two texts using:

• Token frequency vectors
• Cosine similarity

Useful for:

• Stylometry
• Forensic document comparison
• Threat-letter analysis
• Writer profiling

---

🗂 Project Structure

group7-text-forensics/
├── src/group7_text_forensics/   # Core NLP modules (Clojure)
├── resources/                   # OpenNLP model files (.bin)
├── test/group7_text_forensics/  # Tests / sample scripts
├── archive/                     # Editor caches (not part of runtime)
├── README.md
└── project.clj                  # Leiningen project definition

---

🧪 Example Usage

The -main function (in core.clj) demonstrates:

• Linguistic profiling
• Education-level estimation
• Regionality detection
• Psychological-state analysis
• Authorship similarity

Run with:

lein run

Example output:

================== Linguistic Profiling ==================
{:total-words 42
 :unique-words 31
 :type-token-ratio 0.7381
 :sentence-count 3
 :avg-sentence-length 14.0
 :pos-distribution {...}}

Education-level analysis:
High education background

Regional analysis:
Tend to be UK English

Psychological-state analysis:
Negative emotions are obvious, and depression or anxiety may be present

================== Authorship Similarity ==================
Similarity between text1 and text2 => 0.8123

---

⚙️ Requirements

• Clojure
• Leiningen
• OpenNLP models (included in resources/)

Install & run:

lein deps
lein run

---

🎯 Why This Project Matters

This toolkit demonstrates:

• NLP feature engineering
• Linguistic & psycholinguistic analysis
• Functional programming in Clojure
• Authorship attribution algorithms
• Clean modular design for text processing pipelines

Applicable for:

• Computational linguistics research
• Stylometry experiments
• Forensic text analysis
• NLP education and demos

---

📄 License

MIT License