Semantic Internal Linker

Suggest top-N internal links for each category using semantic embeddings (multilingual)

Given a flat CSV of category names and their search volumes, this tool computes semantic similarity between all category pairs using a multilingual sentence-transformers model and produces ranked internal-link suggestions for each category — weighted by volume so high-traffic pages are preferred as link targets.

---

Features

• Multilingual embeddings via paraphrase-multilingual-MiniLM-L12-v2 (50+ languages, no translation step needed)
• Volume-weighted ranking — blends cosine similarity with normalised search volume (tunable via ALPHA)
• Parent-category pinning — root categories (single-word, heuristic-detected) are always suggested first for their subcategories
• Single-CSV in / single-CSV out — drop one file in, get one file back
• CLI arguments — override input path, output path, and alpha without editing the script

---

Requirements

• Python 3.9+
• ~8 GB RAM recommended (model is ~120 MB; larger datasets need more headroom)

---

Installation

pip install -r requirements.txt

The model weights (~120 MB) are downloaded automatically on first run from the Hugging Face Hub. An internet connection is required for the first run only; subsequent runs use the cached model.

---

Input CSV format

The input CSV must contain exactly these two columns:

Column	Type	Description
Category Name	string	Human-readable category label
Volume	integer	Monthly search volume (or any proxy)

See sample_input.csv for a working example.

Note on column naming: The original Turkish-language version of this script used Kategori Adı and Hacim. Those column names have been changed to Category Name and Volume for this public release. If you have existing CSVs with the Turkish headers, rename the columns before running.

---

Usage

Default (reads input.csv, writes link_suggestions.csv):

python semantic_linker.py

Custom paths and alpha:

python semantic_linker.py --input my_categories.csv --output results.csv --alpha 0.5

CLI arguments

Argument	Default	Description
--input	input.csv	Path to input CSV
--output	link_suggestions.csv	Path for output CSV
--alpha	0.3	Volume-weight exponent (0 = ignore, 1 = heavy)

---

Configuration constants

Edit these at the top of semantic_linker.py if you want permanent defaults:

Constant	Default	Description
MODEL_NAME	paraphrase-multilingual-MiniLM-L12-v2	Sentence-transformers model identifier
INPUT_FILE	input.csv	Fallback input path (overridden by CLI)
OUTPUT_FILE	link_suggestions.csv	Fallback output path (overridden by CLI)
ALPHA	0.3	Volume-weight exponent
TOP_N	5	Number of suggestions per category

---

Output

The output CSV contains all original columns plus:

Column	Description
Suggestion 1	Best internal link target
Suggestion 2	Second-best internal link target
...	...
Suggestion 5	Fifth-best internal link target

---

How it works

1. Encode — every category name is encoded into a dense vector using the multilingual MiniLM model.
2. Cosine similarity — an N×N cosine similarity matrix is computed across all category pairs.
3. Volume weighting — each column of the matrix is multiplied by (volume / max_volume) ** alpha, promoting high-volume targets.
4. Parent pinning — single-word categories are treated as roots. Subcategories (multi-word) that contain a root name as a suffix or word are linked to that root first, regardless of similarity score.
5. Output — for each category, the top-5 unique targets are written as Suggestion 1 through Suggestion 5.

---

License

MIT — see LICENSE.