Skip to content

Commit cf5cd20

Browse files
Merge pull request #9 from madijonovsardorbek544-cmyk/prepare-project-for-controlled-beta-testing
Prepare controlled beta QA readiness
2 parents bcfea60 + 4aeb077 commit cf5cd20

17 files changed

Lines changed: 597 additions & 36 deletions

README.md

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,21 @@ This is a public MVP under validation hosted on GitHub Pages. It is being built
88

99
## Readiness status
1010

11+
### Controlled beta status
12+
1113
- Ready for self-testing and controlled friends/student feedback with clear disclaimers.
12-
- Partially ready for counselor review of wording, workflows, and resource gaps.
14+
- Ready for counselor feedback on wording, workflow, safe-next-step usefulness, and resource gaps.
1315
- Not ready for education-center or institution pilots until Firebase configuration, Firestore rules emulator tests, data retention, admin access, and human validation are completed.
1416
- Not ready for paid institution use. It is not certified fraud detection and must not be marketed as guaranteed scam detection.
1517
- Sample dashboard data is synthetic/sample data, not real institution data. Local dashboard data exists only in the user's browser.
18+
- Firebase dashboard mode is **not connected yet**; it is a labeled placeholder until authenticated Firestore queries, rules tests, retention, deletion, and admin access are implemented.
19+
- Local reports are browser-only localStorage records. They are not encrypted, not synced, and not institutional storage.
20+
21+
Controlled beta references:
22+
23+
- [Manual beta QA checklist](docs/BETA_QA_CHECKLIST.md)
24+
- [Controlled beta guide](docs/CONTROLLED_BETA_GUIDE.md)
25+
- [Firebase rules test plan](docs/FIREBASE_RULES_TEST_PLAN.md)
1626

1727
## Positioning
1828

@@ -55,6 +65,19 @@ React + Vite + TypeScript
5565

5666
Routing remains hash-based and the Vite base path remains `/cross-border-scam-analyzer/` for GitHub Pages.
5767

68+
## Controlled beta testing
69+
70+
Automated coverage includes analyzer/privacy regression tests and jsdom-based controlled-beta flow tests for hash routing, homepage CTAs, the bank/card phishing demo, feedback storage, local redacted reports, dashboard local mode, case search, eval dashboard loading, and invalid-route fallback. Playwright E2E was attempted for this pass, but package installation was blocked by registry policy (`403 Forbidden` for `@playwright/test`), so browser E2E remains a pre-pilot TODO.
71+
72+
Run:
73+
74+
```bash
75+
npm test
76+
npm run build
77+
```
78+
79+
Before any institution pilot, also implement the emulator suite in [Firebase rules test plan](docs/FIREBASE_RULES_TEST_PLAN.md).
80+
5881
## Evaluation-driven analyzer
5982

6083
The analyzer uses transparent rules and combination boosts rather than fake AI claims. Rules cover urgency, account lock threats, identity verification, click/action pressure, credential/OTP risk, financial account/card risk, authority impersonation, sensitive-data requests, payment pressure, suspicious links/domains, unofficial payment methods, vague institutions, unrealistic guarantees, visa threats, housing scarcity, test-score upgrade claims, personal-account payments, cross-border bureaucracy confusion, language mismatch, and platform risk.

docs/BETA_QA_CHECKLIST.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# Controlled Beta Manual QA Checklist
2+
3+
Use this checklist before sharing a beta link with students, friends, or counselors. Record browser, device, date, build/commit, and whether Firebase environment variables were configured.
4+
5+
| Area | Manual step | Expected result |
6+
| --- | --- | --- |
7+
| Live site | Open the live site URL. | Homepage loads without console-breaking errors and clearly says this is an educational MVP that detects risk indicators, not certainty. |
8+
| Homepage CTAs | Click “Check a suspicious message,” “View case library,” and “Explore institution pilot.” | Each CTA updates the hash route and shows the expected page without a full app failure. |
9+
| Bank/card phishing demo | Open `#checker`, select “Bank/card phishing,” and run analysis. | Result is high or critical; account/card/identity/click indicators appear in top reasons or analyst view. |
10+
| Scholarship scam demo | Select “Scholarship fee scam” and run analysis. | Result shows scholarship/payment/sensitive-data risk indicators and safe verification steps. |
11+
| Legitimate university reminder | Paste a normal university reminder that points to the official portal and does not request unusual payment or secrets. | Result remains low or medium and copy warns that legitimate messages can still be verified. |
12+
| Custom suspicious message | Paste a short custom suspicious message, then enter a real country/region and destination country. | Analysis button enables only after required fields are present; result avoids fraud-certainty language. |
13+
| Copy script | In the result page, use the copyable verification script control or confirm the script is visible if clipboard permissions are blocked. | Script is readable, official-channel oriented, and does not tell the student to click suspicious links. |
14+
| Save feedback | Save anonymous feedback after analysis. | Loading state appears, then success indicates storage mode; local fallback warning appears if Firebase failed; raw message is not included. |
15+
| Submit local report | Open `#report`, paste text, review redacted preview, enter country/destination, consent, and submit with Firebase unconfigured. | Report saves locally only, shows browser-only warning, and offers dashboard navigation. |
16+
| Dashboard local mode | Open `#dashboard`, select local report source mode. | Locally saved redacted report appears in metrics/trends; dashboard states local mode is browser-only and no raw messages are included. |
17+
| Dashboard sample mode | Select sample source mode. | Dashboard clearly labels sample/synthetic data and does not imply real institution metrics. |
18+
| Eval dashboard | Open `#eval`. | Evaluation dashboard loads and shows total examples plus guardrails that benchmark results are not real-world accuracy. |
19+
| Case library search | Open `#cases`, search for `visa`, `housing`, or `IELTS`, and use filters. | Results filter without crashing; empty states are understandable. |
20+
| Report redaction preview | Paste text containing an email, phone, passport-like ID, URL, and card-like number. | Preview replaces sensitive patterns where detected; user is warned redaction is best effort. |
21+
| Privacy page | Open `#privacy`. | Page explains local analysis, redacted reports, structured feedback, and limits of local/Firebase storage. |
22+
| Mobile width | Test around 375px width. | Navigation, forms, results, report preview, and dashboard controls remain usable without horizontal scrolling. |
23+
| Browser back/forward | Navigate home → checker → cases → dashboard, then use browser back/forward. | Visible page and URL hash stay consistent. |
24+
| Refresh direct routes | Refresh on `#checker` and `#eval`. | Correct page loads directly after refresh. |
25+
| Invalid route | Open an invalid hash such as `#not-real`. | App falls back safely to home and normal navigation still works. |
26+
| No raw message in feedback | After saving feedback, inspect localStorage key `crossBorderScamSafety.feedback.v1`. | Stored feedback contains structured score/context/tactic data only; it does not contain raw, redacted, full, or original message text. |
27+
| No raw message in local reports | After saving a report, inspect localStorage key `crossBorderScamSafety.reports.v1`. | Stored report contains redacted payload only; original raw message is not present. |
28+
| Clear local reports | Use “Clear saved local reports” on the report page and confirm. | Local reports are removed from browser localStorage; the control warns this is not institutional storage. |

docs/CONTROLLED_BETA_GUIDE.md

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# Controlled Beta Guide
2+
3+
## Current status
4+
5+
- **Ready for self-testing and small student/friend testing** with clear disclaimers and privacy instructions.
6+
- **Ready for counselor feedback** on wording, workflows, safe-next-step usefulness, and resource gaps.
7+
- **Not ready for an education-center pilot** until Firebase/rules/security and data governance are validated.
8+
- **Not ready for paid institution use** and must not be marketed as certified fraud detection.
9+
10+
## Who can test now
11+
12+
- The project owner.
13+
- Trusted friends or students who understand this is an educational MVP under validation.
14+
- Counselors or international-student support staff reviewing content and workflow, not relying on it for operational decisions.
15+
16+
## Who should not use it yet
17+
18+
- Students in urgent financial, immigration, housing, safety, or law-enforcement situations without human support.
19+
- Minors unless a trusted adult/counselor is involved.
20+
- Education centers or institutions needing production records, dashboards, retention, deletion, or admin access.
21+
- Paid customers or anyone expecting guaranteed fraud detection.
22+
23+
## What testers must be told
24+
25+
- The checker detects **risk indicators**, not certainty.
26+
- It can miss scams and can over-score legitimate messages.
27+
- It does not replace official university, embassy, bank, police, legal, immigration, or emergency channels.
28+
- The checker runs locally by default, but optional reports are still best-effort redacted and should be reviewed carefully.
29+
- Local reports and feedback in localStorage are browser-only and not encrypted institutional storage.
30+
31+
## What testers must not paste
32+
33+
Tell testers not to paste:
34+
35+
- passport scans or full passport numbers,
36+
- student IDs,
37+
- card or bank-account numbers,
38+
- passwords, OTPs, PINs, recovery codes, or login links,
39+
- exact home addresses,
40+
- private documents or screenshots with personal data,
41+
- names/contact details of real people unless removed first.
42+
43+
## How to collect feedback safely
44+
45+
- Prefer structured observations: “Which warning was confusing?” “Did the script help?” “Was the risk level too high/too low?”
46+
- Do not ask testers to send raw suspicious messages in chat, email, forms, or screenshots.
47+
- If examples are needed, ask testers to paraphrase and remove identifiers.
48+
- Summarize trends by category, channel, country/destination context, and risk indicator—not by raw message text.
49+
50+
## How to handle suspicious real messages
51+
52+
1. Ask the tester to stop before paying, clicking, replying, or sending documents.
53+
2. Have them verify through a published official channel typed manually or a saved official app/bookmark.
54+
3. If money, credentials, card data, documents, or OTPs were already shared, direct them to the relevant bank/payment provider, real institution, counselor, and appropriate local reporting channel.
55+
4. If the situation involves immediate danger, coercion, or a minor, stop product testing and involve a trusted adult, counselor, emergency service, or local authority as appropriate.
56+
57+
## When to stop testing
58+
59+
Stop the beta session if:
60+
61+
- a tester is distressed, pressured, or in immediate danger,
62+
- a tester starts sharing private documents or secrets,
63+
- the app gives confusing guidance for a high-stakes issue,
64+
- Firebase/dashboard behavior appears connected when it is not,
65+
- localStorage contains raw message text unexpectedly,
66+
- a counselor identifies unsafe or misleading wording.
67+
68+
## When counselor review is needed
69+
70+
Counselor/international-office review is needed before using examples or guidance for:
71+
72+
- visa or immigration deadlines,
73+
- tuition/payment diversion,
74+
- housing deposits,
75+
- scholarship/admission claims,
76+
- minors or vulnerable students,
77+
- country-specific official resources,
78+
- escalation paths after money or documents are shared.
79+
80+
## How to summarize results without raw messages
81+
82+
Use aggregate, non-identifying notes such as:
83+
84+
- “3 testers found the verification script useful.”
85+
- “2 testers were confused by local vs Firebase dashboard mode.”
86+
- “Scholarship-fee examples were understood; housing-deposit examples need clearer next steps.”
87+
- “One custom message triggered high risk due to urgency + payment + sensitive-data requests.”
88+
89+
Do not include raw messages, names, exact phone numbers, email addresses, URLs, passport/student IDs, card/bank details, or screenshots in beta summaries.

docs/FIREBASE_RULES_TEST_PLAN.md

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
# Firebase Rules Test Plan Before Any Institution Pilot
2+
3+
This project is not ready for an education-center or institution pilot until Firestore rules are tested with the Firebase emulator and the resulting checks run in CI.
4+
5+
## Scope
6+
7+
Collections covered by the first rules test suite:
8+
9+
- `reports`
10+
- `anonymousFeedback`
11+
- representative public-deny checks for update/delete/read paths
12+
13+
## Required tooling
14+
15+
1. Add dev dependencies:
16+
- `firebase-tools`
17+
- `@firebase/rules-unit-testing`
18+
2. Add a script such as:
19+
- `"test:rules": "firebase emulators:exec --only firestore \"vitest run firestore.rules.test.ts\""`
20+
3. Run with an isolated emulator project ID, for example `cross-border-scam-safety-rules-test`.
21+
22+
## Exact tests to implement
23+
24+
### Reports collection
25+
26+
1. **Valid redacted report create is allowed**
27+
- Create `/reports/{id}` with only allowed fields from `reportHasOnlyRedactedFields()`.
28+
- Include `reportId`, `createdAtIso`, `redactedMessage`, `redactionCounts`, `highRiskMarkers`, `language`, `countryRegion`, `destinationCountry`, `platform`, `context`, `score`, `level`, `scamTypeGuess`, `consentVersion`, and `deletionInstructions`.
29+
- Expected: `assertSucceeds`.
30+
31+
2. **Report with `rawMessage` is denied**
32+
- Add `rawMessage: "original text"` to an otherwise valid report.
33+
- Expected: `assertFails`.
34+
35+
3. **Report with `message` is denied**
36+
- Add `message: "original text"`.
37+
- Expected: `assertFails`.
38+
39+
4. **Report with `fullMessage` is denied**
40+
- Add `fullMessage: "original text"`.
41+
- Expected: `assertFails`.
42+
43+
5. **Report with `unredactedMessage` is denied**
44+
- Add `unredactedMessage: "original text"`.
45+
- Expected: `assertFails`.
46+
47+
6. **Report with `attachments` is denied**
48+
- Add `attachments: []` or any attachment-like metadata.
49+
- Expected: `assertFails`.
50+
51+
7. **Overlong redacted report is denied**
52+
- Set `redactedMessage` to more than 5,000 characters.
53+
- Expected: `assertFails`.
54+
55+
8. **Public read of reports is denied**
56+
- Try to read `/reports/{id}` as an unauthenticated client.
57+
- Expected: `assertFails`.
58+
59+
9. **Public update/delete of reports is denied**
60+
- Try update and delete as an unauthenticated client.
61+
- Expected: `assertFails`.
62+
63+
### Anonymous feedback collection
64+
65+
1. **Valid anonymous feedback create is allowed**
66+
- Create `/anonymousFeedback/{id}` with only allowed structured fields from `feedbackHasOnlyStructuredFields()`.
67+
- Include `id`, `createdAtIso`, `helpful`, `verifiedOfficialChannel`, `calibration`, `category`, `score`, `level`, `context`, `platform`, `tacticIds`, `riskAreaLevels`, `storageMode`, and `schemaVersion`.
68+
- Expected: `assertSucceeds`.
69+
70+
2. **Feedback with `rawMessage` is denied**
71+
- Add `rawMessage`.
72+
- Expected: `assertFails`.
73+
74+
3. **Feedback with `message` is denied**
75+
- Add `message`.
76+
- Expected: `assertFails`.
77+
78+
4. **Feedback with `redactedMessage` is denied**
79+
- Add `redactedMessage`.
80+
- Expected: `assertFails`.
81+
82+
5. **Feedback with `fullMessage` is denied**
83+
- Add `fullMessage`.
84+
- Expected: `assertFails`.
85+
86+
6. **Feedback with `unredactedMessage` is denied**
87+
- Add `unredactedMessage`.
88+
- Expected: `assertFails`.
89+
90+
7. **Feedback with invalid enums is denied**
91+
- Try `helpful: "maybe"`, `calibration: "unknown"`, or `level: "certain fraud"`.
92+
- Expected: `assertFails`.
93+
94+
8. **Public read of feedback is denied**
95+
- Try unauthenticated read of `/anonymousFeedback/{id}`.
96+
- Expected: `assertFails`.
97+
98+
9. **Public update/delete of feedback is denied**
99+
- Try update and delete as an unauthenticated client.
100+
- Expected: `assertFails`.
101+
102+
10. **Admin feedback read is allowed only with admin role claim**
103+
- Read with auth token `{ role: "admin" }`.
104+
- Expected: `assertSucceeds`.
105+
- Read with no token or a non-admin token.
106+
- Expected: `assertFails`.
107+
108+
## CI go/no-go rule
109+
110+
An education-center pilot must not start until:
111+
112+
- all tests above pass in CI,
113+
- Firebase dashboard querying is either implemented behind authenticated role-based access or hidden/disabled,
114+
- data retention/deletion ownership is documented,
115+
- admin access and incident response are reviewed by a human privacy/security owner.

0 commit comments

Comments
 (0)