Automated Domain Question Mapping (DQM)

This project turns textbooks into interactive question maps. Instead of manually creating concept maps, it automatically generates questions from educational content and shows how they connect to each other.

Overview

Building concept maps by hand takes a lot of time, and doing it automatically is tricky because concepts can mean different things in different contexts. This project takes a different approach: it generates specific questions from textbook content and figures out how those questions relate to each other.

The system reads through textbook sections, creates a question for each chunk, determines which questions are more general or specific than others, and builds a visual map of these connections.

Features

Question Generation
Trains models (T5-Large-Squad-QG, BART-Large, Pegasus-Large, GPT-2-Medium) using SQuAD 2.0 and KhanQ datasets to create questions from text.

Relationship Classification
Uses a fine-tuned GPT-2 model to figure out how questions relate: general(parent-to-child order), specific(child-to-parent order) or unrelated to another?

Graph Construction

• Creates one question per text section
• Calculates relationships between question pairs
• Reduces nodes to a manageable number
• Builds a clean structure

Visualization
Generates different views of the knowledge graph:

• t-SNE embeddings showing topic clusters
• Threshold-based graphs
• Final graph wmade with Maximum Spanning Trees (MST)

Setup

We recommend using Anaconda for easier package management.

Create Conda Environment

conda create -n environment_name python=3.10
conda activate environment_name

Install Core Packages

Install main packages through conda for better stability:

conda install -c conda-forge pytorch torchvision torchaudio transformers tokenizers nltk rouge-score wandb scikit-learn pandas numpy matplotlib seaborn

Install Remaining Dependencies

pip3 install -r requirements.txt

Data Preparation

First, create a data folder in your main project directory:

mkdir data
cd data

Download the datasets from Kaggle extract them and move them to data folder

• QG Data: qa_data/ folder contains SQuAD 2.0 and KhanQ datsets
• Relationship Data: rel_datasets/ folder has IRBook turned as dataset with labels

After setup, your directory structure should look like:

project_dqm/
├── data/
│   ├── qa_data/
│   │   ├── khanq/all.json
│   │   └── squad2/dev.json, train.json
│   ├── rel_datasets/
│   │   └── irbook_all.jsonl
│   │   └── irbook_all_chunks.jsonl
│   │   └── irbook_rel_dataset.csv
├── scripts/
├── src/jandi
├── checkpoints/
├── generated/
└── results/

Usage

You have two options to use this project:

Option 1: Training Models from Scratch

Follow these five phases to train models and generate question maps.

Phase 1: Train Question Generation Model

python3 scripts/train_qa_generation.py --no_wandb

Trains a model LLM (T5-Large-Squad-QG) on SQuAD 2.0 and KhanQ datasets. Saves checkpoint to checkpoints/ folder.

NOTE: If you want model training statistics, you need to connect your account with wandb. Remove the --no_wandb flag to enable tracking.

NOTE: Set --wandb_entity to your wandb username for experiment tracking. At line 217 in train_qa_generation.py:

parser.add_argument("--wandb_entity", type=str, default="here_use_your_wandb_id")

Phase 2: Train Relationship Model

python3 scripts/train_rel.py --no_wandb

Trains gpt2-medium to classify question relationships (general, specific, or other). Saves checkpoint to checkpoints/ folder.

NOTE: Set --wandb_entity to your wandb username for experiment tracking. At line 109 in train_rel.py:

parser.add_argument("--wandb_entity", type=str, default="here_use_your_wandb_id")

Phase 3: Generate Questions

python3 scripts/qgen.py

Loads your textbook file and generates questions. Outputs to results/ folder.

Important: Move the output file from results/ to generated/ before continuing.

mv results/.json* .jsonl* ../generated/

Phase 4: Build the Graph

python3 scripts/pre_graph.py

Takes generated questions and builds a weighted graph. Outputs to results/ folder.

Important: Move the graph related files to generated/ folder.

mv results/.json* .jsonl* ../generated/

Phase 5: Visualize

python3 scripts/graph.py

Creates PNG visualizations and saves them to results/ folder.

---

Option 2: Using Fine-Tuned Model

If you want to skip training and use fine-trained models directly, follow these steps:

Step 1: Download Fine-Tuned Models

Download the fine-tuned model checkpoints from Kaggle

Step 2: Extract and Setup Checkpoints

# Extract the downloaded file
unzip DQM trained models.zip

# Create checkpoints folder in your project directory
mkdir checkpoints

# Move the .pt files into checkpoints folder
mv *.pt checkpoints/

Your checkpoints folder should now contain:

• finetuned_qgen_t5_large_squad_qg_combined.pt (Question Generation model)
• rel_gpt2_medium.pt (Relationship Classification model)

Step 3: Continue from Question Generation

Now you can skip Phases 1-2 and start directly from Phase 3:

# Phase 3: Generate Questions
python3 scripts/qgen.py

# Move output files
mv results/.json* .jsonl* ../generated/

# Phase 4: Build the Graph
python3 scripts/pre_graph.py

# Move graph files
mv results/.json* .jsonl* ../generated/

# Phase 5: Visualize
python3 scripts/graph.py

Your visualizations will be saved in the results/ folder.

Results

All the models work well on both datasets. Particularly T5-Large-Squad-QG gave the best results for question generation, and GPT-2 medium got 90% F1-score for relationship classification.

Visualizations

t-SNE Question Embeddings

Shows how questions group by topic.

Edge Weight Threshold Graph

Dense graph before pruning with 0.7 weight threshold applied.

Final Maximum Spanning Tree

Clean, connected graph ready to use.

Sample Learning Path

Example questions along a path in the final graph.

Technical Details

Models:

• BART-large (406M parameters)
• Pegasus-large (568M parameters)
• T5-Large-Squad-QG (770M parameters) - Recommended to use
• GPT-2 (355M parameters)

Datasets:

• SQuAD2.0
• KhanQ

Graph Processing:
Uses Sentence-BERT for semantic similarity combined with specificity scores. Applies Kruskal's algorithm to find the Maximum Spanning Tree, which keeps the graph connected while removing cycles.

Requirements

• Python 3.10+
• PyTorch
• Anaconda (recommended)

License

This project is licensed under the MIT License

Acknowledgments

Research and Contribution:

• Mukhesh Raghava Katragadda
• Jiho Noh

Affiliation: Cognition and Learning Design Lab, Department of Computer Science, Kennesaw State University, GA, 30060