AI/ML Internship Assessment | Data Analysis & NLP
An end-to-end data analysis project for categorizing multilingual purchase order items (Arabic & English) from a Saudi-based procurement system.
Given a real-world dataset of 3,150 purchase order items — with item names in Arabic, English, and mixed language — the goal was to:
- Clean and preprocess the raw data
- Handle multilingual text (Arabic + English)
- Develop a categorization strategy with documented reasoning
- Identify spending patterns and business insights
- Loaded and profiled a 3,150-row, 11-column Excel dataset (SAR-denominated procurement data)
- Identified and handled 240 missing Item Names (7.6% of data)
- Dropped uninformative columns:
Project ID(100% null),Tax ID(constant),Account ID(near-constant) - Standardized all monetary values to SAR (converted 65 USD rows using a fixed rate)
- Engineered a new feature:
Item price_in_SAR(unit price = total ÷ quantity) - Detected and documented a consistent 2.04% tax pattern across 15 rows where
Total ≠ SubTotal - Built a multilingual language detection pipeline without external libraries:
- Arabic-only: 1,044 records (33%)
- English-only: 1,670 records (53%)
- Mixed: 196 records (6%)
- No text (missing): 240 records (8%)
- Extracted top word frequencies separately for Arabic and English tokens
- Produced summary visualizations: language distribution, price distribution, missing data heatmap, word frequency
Explored 3 distinct categorization approaches with documented trade-offs:
Categorized items by their share of total spend:
| Tier | % of Total Spend | # of Items |
|---|---|---|
| High-value (≥ 1%) | ~50% of revenue | 19 items |
| Mid-value (0.1–1%) | ~28% of revenue | 80 items |
| Low-value (0.01–0.1%) | ~19% of revenue | 1,140 items |
| Micro (< 0.01%) | ~3% of revenue | 1,911 items |
Key finding: 19 items drive ~50% of total revenue — a clear Pareto pattern.
- Extracted and ranked all tokens across Arabic and English text separately
- Applied domain-aware stop word filtering (units, measurements, filler words)
- Identified top content-bearing keywords in both languages
- Conclusion: Effective but requires domain expert annotation (~1–2 hours) to map keywords to categories. Suggested 3 macro categories:
Construction,Mechanical,Electrical
- Reverse-engineered the Product ID encoding
- Discovered that digits at index positions 10–12 encode a product category code
- Identified 26 distinct category codes across 2,910 products with IDs
- Conclusion: Most systematic and scalable approach for large datasets, but requires internal documentation of the encoding scheme
Start with revenue-based segmentation immediately for financial decision-making.
Engage a domain expert to annotate the keyword list.
Standardize Product ID assignment going forward as the long-term categorization backbone.
| Tool | Use |
|---|---|
| Python 3.10 | Core language |
| Pandas | Data manipulation |
| NumPy | Numerical operations |
| Matplotlib / Seaborn | Visualization |
| re / collections | Text processing (no external NLP libraries) |
aajil-purchase-order-categorization/
│
├── notebooks/
│ ├── 01_data_processing.ipynb # EDA, cleaning, multilingual text analysis
│ └── 02_Item Categorization.ipynb # 3 categorization approaches + recommendation
│
├── data/
│ └── purchase-order-items.xlsx # Raw dataset
│
└── README.md
- 19 items account for ~50% of total procurement spend
- Item names are highly inconsistent — same product appears with different names in Arabic and English
- Product ID encoding contains implicit category information that could power automated classification
- Multilingual NLP is possible without heavy frameworks using simple Unicode range detection
Assessment completed as part of an AI/ML internship application process.