Skip to content

Commit 565583b

Browse files
Merge pull request #6 from madijonovsardorbek544-cmyk/refactor-app-architecture-and-improve-features
Move MVP toward evaluation-driven pilot platform
2 parents 9d85b3c + 3939e14 commit 565583b

31 files changed

Lines changed: 1019 additions & 559 deletions

README.md

Lines changed: 85 additions & 164 deletions
Large diffs are not rendered by default.

docs/COUNSELOR_INTERVIEW_GUIDE.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Counselor Interview Guide
2+
3+
## Interview principles
4+
- Ask about recent workflows before showing product screens.
5+
- Avoid asking “Would this be useful?” first; ask what they currently do and where it fails.
6+
- Do not lead with desired answers such as “Does the analyst view help?”
7+
- Collect concrete examples as redacted patterns, not raw student messages.
8+
9+
## Questions for counselors
10+
1. What scam or confusing-message patterns do international students bring to you most often?
11+
2. How do you currently decide whether to escalate a message?
12+
3. Which official channels do you want students to use before paying, clicking, or sending documents?
13+
4. Which result wording would reduce panic while still encouraging action?
14+
5. What would make a dashboard report credible enough for monthly awareness planning?
15+
16+
## Questions for education centers
17+
1. How do you verify agents, payment instructions, scholarship offers, and admissions portals?
18+
2. What countries, platforms, and languages create the most confusion?
19+
3. What kinds of false alarms would damage trust with students or partner institutions?
20+
4. What documentation would you need before recommending the tool?
21+
22+
## Questions for students and parents
23+
1. When you receive an urgent study-abroad message, what do you usually do first?
24+
2. Which parts of the result are clear or confusing?
25+
3. Would you know which official office or website to contact after seeing the script?
26+
4. What private data would you worry about entering?
27+
5. Did the score feel too low, accurate, or too high?
28+
29+
## Evidence to collect
30+
- Screen-level confusion points.
31+
- Misunderstood terms or icons.
32+
- Missing official verification channels.
33+
- False-positive and missed-risk categories.
34+
- Confidence in next action, not certainty of fraud.

docs/PILOT_PLAYBOOK.md

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# Institution Pilot Playbook
2+
3+
## 30-day process
4+
5+
### Week 1 — Setup
6+
- Confirm official payment, admissions, visa, housing, testing, and agent-verification resources.
7+
- Name counselor/staff owners for escalations.
8+
- Review privacy language and decide whether Firebase is configured or local/sample mode is used.
9+
- Define pilot success and stopping criteria.
10+
11+
### Week 2 — Awareness launch
12+
- Share the checker link with a limited student group.
13+
- Run a short awareness session using synthetic examples from the case library.
14+
- Explain that the product detects risk indicators, not certainty.
15+
- Remind students never to paste passports, card numbers, passwords, OTPs, or private document images.
16+
17+
### Week 3 — Review and adjust
18+
- Review anonymized dashboard categories, channels, claimed authorities, and feedback trends.
19+
- Identify top false-positive and missed-risk categories.
20+
- Update warnings and resource links if counselors find confusing wording.
21+
- Do not collect raw messages through informal channels.
22+
23+
### Week 4 — Export and decision
24+
- Export the Markdown report from the dashboard.
25+
- Compare success metrics to pilot goals.
26+
- Decide whether to continue, revise, expand, or stop.
27+
28+
## Data collected
29+
- Redacted report payloads only after consent.
30+
- Risk score, level, context, platform, country/region, and claimed authority if provided.
31+
- Anonymous feedback fields: helpfulness, official verification status, calibration, optional category.
32+
33+
## Data never collected by default
34+
- Raw suspicious messages.
35+
- Passport scans, student IDs, bank/card details, passwords, OTPs, screenshots with personal data, or legal case files.
36+
- Certainty labels that claim a person or organization committed fraud.
37+
38+
## Awareness session structure
39+
- Start with common cross-border pressure tactics.
40+
- Show synthetic examples and resource packs.
41+
- Demonstrate official verification scripts.
42+
- Close with escalation contacts and “pause before paying/clicking/sharing” guidance.
43+
44+
## Exporting reports
45+
- Open Dashboard.
46+
- Select sample/local/Firebase source mode and date range.
47+
- Confirm the source label is accurate.
48+
- Click “Download Markdown report.”
49+
50+
## Success/failure criteria
51+
- Continue if students and counselors find next steps clear, privacy concerns are manageable, and benchmark checks remain stable.
52+
- Revise if false positives cluster around legitimate deadlines or payment reminders.
53+
- Stop or narrow scope if users misunderstand scores as certainty or if privacy expectations cannot be met.

docs/PRIVACY_THREAT_MODEL.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Privacy Threat Model
2+
3+
## Assets
4+
- Student messages and contextual metadata.
5+
- Redacted report payloads.
6+
- Anonymous feedback records.
7+
- Institution dashboard trend exports.
8+
- Official resource links and verification scripts.
9+
10+
## Threat actors
11+
- Scammers attempting to learn detection patterns.
12+
- Curious insiders or misconfigured Firebase readers.
13+
- Users accidentally pasting sensitive documents.
14+
- Institutions over-interpreting scores as proof.
15+
- Attackers trying to inject malicious links into shared examples.
16+
17+
## Privacy risks
18+
- Raw messages may contain passports, card details, login codes, addresses, or immigration identifiers.
19+
- Trend reports could expose small-group patterns if a pilot group is tiny.
20+
- Feedback could become sensitive if free-text raw messages were allowed.
21+
22+
## Misuse risks
23+
- Treating scores as certainty or legal findings.
24+
- Blocking legitimate institutional communications solely because of a score.
25+
- Collecting raw scam reports outside the redacted consent flow.
26+
- Using the tool for populations outside its validated scope without retesting.
27+
28+
## Mitigations
29+
- Local analysis by default.
30+
- No raw message storage in feedback.
31+
- Redacted report preview plus consent.
32+
- Clear false-positive warning and “risk indicators detected” language.
33+
- Dashboard source labels for sample/local/Firebase data.
34+
- Documentation for pilot privacy boundaries and validation needs.
35+
36+
## Remaining limitations
37+
- Redaction is best effort and can miss unusual identifiers.
38+
- Rule-based scoring can miss new adversarial language.
39+
- Firebase deployment requires project-specific security rules, authentication, retention, and access review.
40+
- Small pilots need aggregation thresholds before sharing institution-wide conclusions.

docs/PRODUCT_REQUIREMENTS.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Product Requirements
2+
3+
## Target users
4+
- International students and families.
5+
- School counselors, international offices, admissions, housing, testing, and financial-aid staff.
6+
- Education centers and verified advisers.
7+
- Institutions running pre-arrival scam-awareness pilots.
8+
9+
## Problem
10+
International students receive cross-border messages involving visas, admissions, scholarships, housing, testing, payments, documents, and agents. Scammers exploit unfamiliar bureaucracy, urgency, language gaps, and family pressure. Students need safe verification steps before paying, clicking, or sharing documents.
11+
12+
## Non-goals
13+
- Do not claim certainty or make legal findings.
14+
- Do not replace immigration, legal, financial, emergency, or law-enforcement advice.
15+
- Do not collect raw sensitive messages by default.
16+
- Do not market as AI if the system is rules/evaluation driven.
17+
18+
## MVP features
19+
- Local-first scam checker.
20+
- Transparent analyzer rules and combination boosts.
21+
- Counselor / Analyst View.
22+
- Case library and resource packs.
23+
- Redacted reporting flow.
24+
- Anonymous feedback loop.
25+
- Institution dashboard with Markdown export.
26+
- Benchmark evaluation dataset and tests.
27+
28+
## Future features
29+
- Institution-specific verified resource configuration.
30+
- Aggregation thresholds and retention controls.
31+
- Multilingual UX validation.
32+
- Counselor review queue for consented, redacted cases.
33+
- Secure authenticated institution dashboards.
34+
35+
## Success metrics
36+
- High-risk benchmark examples do not score low.
37+
- Low-risk legitimate examples remain low/medium.
38+
- Students can identify official next steps.
39+
- Counselors trust explanations enough for advising.
40+
- Feedback produces actionable false-positive and missed-risk categories without raw message collection.

docs/VALIDATION_PLAN.md

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Validation Plan
2+
3+
## What needs to be validated
4+
- Analyzer calibration: high-risk examples should not be missed as low risk, and normal university reminders should not be pushed into high/critical without strong indicators.
5+
- Counselor usefulness: explanations, evidence, and verification scripts must support real advising conversations.
6+
- Student usability: international students and parents should understand “risk indicators detected” without panic or certainty claims.
7+
- Privacy comfort: users must understand that checker input is local by default and feedback does not store raw messages.
8+
- Pilot dashboard value: institutions need actionable categories, channels, authorities, feedback insights, and recommended interventions.
9+
10+
## Target users
11+
- International students before arrival and during first year.
12+
- Parents and guardians supporting cross-border decisions.
13+
- School counselors, admissions staff, international student offices, housing offices, and financial-aid teams.
14+
- Education centers and verified advisers who help students apply abroad.
15+
16+
## Success metrics
17+
- 90%+ of seeded high-risk benchmark examples are not scored low.
18+
- Legitimate low-risk reminders remain low/medium in benchmark checks.
19+
- Students can identify at least one official verification step after using a result.
20+
- Counselors rate analyst explanations as usable for advising sessions.
21+
- Feedback submission rate is sufficient to identify false-positive and missed-risk patterns without raw message collection.
22+
23+
## False-positive / false-negative tracking
24+
- False positives: collect anonymous category, context, platform, level, and “too high” calibration signals.
25+
- False negatives: collect anonymous category, context, platform, level, and “too low” or “missed risk” signals.
26+
- Review weekly during pilots; update rules only when evidence appears across multiple users or counselor review.
27+
- Keep raw messages out of feedback; if a real case needs review, use a separate consented and redacted workflow.
28+
29+
## Counselor interview goals
30+
- Learn whether the risk-area language matches how counselors explain scams.
31+
- Identify missing institution workflows, escalation paths, and country-specific resource needs.
32+
- Test whether false-positive warnings reduce overreaction.
33+
- Ask what would make the dashboard useful enough for a 30-day pilot.
34+
35+
## Student usability testing
36+
- Observe whether students know what to do after a high, medium, or low result.
37+
- Test comprehension of “risk indicators detected” vs. certainty claims.
38+
- Confirm students can copy and adapt verification scripts.
39+
- Confirm students know not to paste passports, cards, OTPs, or private documents.

src/analyzer.test.ts

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ import { describe, expect, it } from 'vitest';
22
import { analyzeMessage } from './analyzer';
33
import { redactSensitiveText } from './redaction';
44
import { createAnonymizedReportPayload, payloadContainsRawSensitiveData } from './lib/privacy/reportSchema';
5+
import { createAnonymousFeedbackRecord, feedbackContainsRawMessage } from './lib/feedback/feedbackSchema';
56
import type { CheckInput } from './types';
67

78
const base: CheckInput = { language: 'English', countryRegion: 'India', destinationCountry: 'Canada', platform: 'email', context: 'visa', message: '' };
@@ -129,6 +130,14 @@ describe('redactSensitiveText', () => {
129130
expect(result.highRiskMarkers).toContain('bank/account words near numbers');
130131
});
131132

133+
it('keeps raw messages out of anonymous feedback records', () => {
134+
const input = { ...base, message: 'We detected unauthorized login activity on your account. Your card is temporarily locked. Click here immediately to verify your identity.' };
135+
const result = analyzeMessage(input);
136+
const feedback = createAnonymousFeedbackRecord({ helpful: 'yes', verifiedOfficialChannel: 'not yet', calibration: 'accurate', category: 'useful' }, result, input.context, input.platform);
137+
expect(feedbackContainsRawMessage(feedback)).toBe(false);
138+
expect(JSON.stringify(feedback)).not.toContain(input.message);
139+
});
140+
132141
it('keeps raw sensitive data out of report payload creation', () => {
133142
const input = { ...base, message: 'Email student@example.com passport A1234567 card 4111111111111111 Telegram @visa_agent_now https://bad.test' };
134143
const result = analyzeMessage(input);

src/components/CaseCard.tsx

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
import { packsForText } from '../data/resourcePacks';
2+
import type { ScamCase } from '../types';
3+
4+
export function CaseCard({ item }: { item: ScamCase }) {
5+
const relatedPacks = packsForText(`${item.scamType} ${item.messageSample}`, item.tags);
6+
return (
7+
<article className="case-card">
8+
<p className="eyebrow">{item.sourceType} case · {item.platform}</p>
9+
<h2>{item.title}</h2>
10+
<p className="case-summary">{item.shortSummary}</p>
11+
<p><strong>Target:</strong> {item.targetGroup}</p>
12+
<p><strong>Claimed authority:</strong> {item.fakeAuthority}</p>
13+
<blockquote>{item.messageSample}</blockquote>
14+
<h3>Risk indicators detected</h3>
15+
<ul className="red-flags">{item.redFlags.map((flag) => <li key={flag}>{flag}</li>)}</ul>
16+
<h3>Safe response</h3><p>{item.safeResponse}</p>
17+
{relatedPacks.length > 0 && <><h3>Relevant resource packs</h3><div className="tags">{relatedPacks.map((pack) => <span key={pack.id}>{pack.title}</span>)}</div></>}
18+
<div className="tags">{item.tags.map((tag) => <span key={tag}>{tag}</span>)}</div>
19+
</article>
20+
);
21+
}

src/components/Chart.tsx

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
export function Chart({ title, data }: { title: string; data: Record<string, number> }) {
2+
const max = Math.max(1, ...Object.values(data));
3+
return <section className="panel"><h2>{title}</h2><div className="bars">{Object.entries(data).sort((a, b) => b[1] - a[1]).map(([label, value]) => <div className="bar" key={label}><span>{label}</span><div aria-hidden="true"><i style={{ width: `${(value / max) * 100}%` }} /></div><strong>{value}</strong></div>)}</div></section>;
4+
}
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
import { demoMessages } from '../data/demoMessages';
2+
3+
export function DemoMessageSelector({ demoId, onSelect }: { demoId: string; onSelect: (id: string) => void }) {
4+
return (
5+
<section className="demo-box" aria-labelledby="sample-heading">
6+
<h2 id="sample-heading">Try a sample message</h2>
7+
<p className="muted">Synthetic samples help counselors and students understand common risk indicators without entering private data.</p>
8+
<div className="chip-row">
9+
{demoMessages.map((demo) => <button key={demo.id} type="button" className={demoId === demo.id ? 'active-chip' : 'chip'} onClick={() => onSelect(demo.id)}>{demo.label}</button>)}
10+
</div>
11+
</section>
12+
);
13+
}

0 commit comments

Comments
 (0)