Network Operations (NOA demo)

Fabric-first Network Operations Agent demo over telco/network data (Tier-1 European mobile operator profile): Fabric Graph + Ontology + Operations Agent branched on real-time telemetry. Autonomous multi-agent remediation (Microsoft Foundry — Niobe/Pal/SONiC) = phase 2.

Contents: At a glance · What it demonstrates · Architecture · Project layout · Testing gate · Quick start · Platform support · Decks · Public-repo hygiene · Key facts

Sister project of Financial_Platform — same idempotent deploy pattern (state.json). Full design: docs/ARCHITECTURE.md.

---

At a glance

Goal	Full NOC journey — detect → diagnose (RCA) → impact + handoff
Fabric stack	Eventhouse/KQL · Lakehouse · Fabric IQ (Ontology + Graph) · Direct Lake semantic model · Data Activator · Operations Agent · dual-source Data Agent
Phase 2	Microsoft Foundry — autonomous closed-loop remediation
Workspace	Network Operations (set in src/config.yaml) · any Fabric F-SKU · your Entra tenant
Storyline	PFC storm on SW-TOR-05 (Lille) degrades downstream cells

---

What it demonstrates (full NOC journey)

1. Detect — a PFC storm on SW-TOR-05 + KPI drop → Data Activator fires an incident (live injection).
2. Diagnose / RCA — the Operations Agent (NL→KQL + NL→GQL) localizes the fault by traversing Cell → Device → Interface → Link.
3. Impact + handoff — graph traversal Device → Cells → Sites → Services → Customers surfaces VIP/Gold impact, then hands off to Foundry (phase 2).

---

Architecture (Fabric ⟷ Foundry)

• Fabric (this project) — Eventhouse/KQL (telemetry), Lakehouse (topology), Fabric IQ (Ontology + Graph), Data Activator (alerts), Operations Agent (Fabric Data Agent).
• Foundry (phase 2) — Niobe (supervisor), Pal (RCA), SONiC (device exec), Miles (field ops), Ticketing, Security/Compliance — autonomous closed-loop remediation.

---

Project layout

.github/workflows/         — CI: leak guard + pytest
scripts/                   — check_no_client_leak.py (canonical, shared verbatim)
                             + check_repo_leaks.py (entry point; run before pushing)
docs/ARCHITECTURE.md         — full design + data model + demo storyboard
src/config.yaml              — workspace, item names, topology + telemetry config
src/generate_data.py         — synthetic topology + telemetry (with the scripted PFC storm)
src/inject_event.py          — LIVE telemetry injector (--loop animates the storm in real time)
src/deploy_kql_shortcuts.py  — OneLake availability (mirroring) + Lakehouse shortcuts
src/deploy_semantic_model.py — SM_Network_Analytics (Direct Lake, 32 DAX measures)
src/helpers.py               — Fabric API + Kusto (Eventhouse) helpers
src/platform_env.py          — cross-platform PATH bootstrap (Windows registry self-heal)
src/state.json               — deployment IDs (idempotent)
data/raw/*.csv               — generated data
presentation/build_deck.js   — deck generator (the .pptx are NOT committed)
tests/test_smoke.py          — offline gate (run before any deploy)

---

Mandatory testing gate

Before running ANY deploy_*.py / generator:

python -m pytest tests/ -v --tb=short

If anything fails → STOP, fix, re-run.

---

Quick start

pip install -r requirements.txt
cp src/config.example.yaml src/config.yaml   # then fill capacity_id + tenant_id
cp src/state.example.json  src/state.json
python src/generate_data.py        # writes data/raw/*.csv
python -m pytest tests/ -q         # gate

config.yaml / state.json are gitignored — they hold your tenant, capacity and item IDs. Every one of them can also come from the environment instead of the file (NOA_TENANT_ID, NOA_CAPACITY_ID, NOA_WORKSPACE_NAME, NOA_AZ_SUBSCRIPTION, NOA_WORKSPACE_ID, …); env wins over the file, and a missing value fails with an explicit message. No real identifier is ever committed.

Then deploy (scripts added in the deploy phase), in order:

workspace → lakehouse → eventhouse → preload telemetry → ontology →
graph (deploy_graph.py + RefreshGraph) → kql dashboard → data activator →
operations agent → kql shortcuts → semantic model → data agent

One command: python deploy_all.py (idempotent, resumable with --from <step>).

During the demo, animate the storm live:

python src/inject_event.py --loop      # ramps SW-TOR-05 to a full PFC storm, Ctrl+C to stop

---

Platform support

Everything under src/ runs on Windows, macOS and Linux. There is no .ps1, .cmd or .bat in this repository — every entry point is a python src/… command, identical on the three platforms. The only OS-specific behaviour is isolated in src/platform_env.py:

	Windows	macOS / Linux
IS_WINDOWS / AZ_NEEDS_SHELL	True	False
restore_path()	rebuilds PATH from the registry (HKLM + HKCU Environment), because activating a venv in some terminals wipes it and az then disappears	no-op — the inherited PATH is authoritative
configure_stdout()	forces UTF-8 (the console defaults to cp1252)	same, usually already UTF-8
bootstrap()	restore_path() then configure_stdout()	idem, in the same order
find_executable()	shutil.which, retried once after restore_path()	shutil.which
az invocation	shell=True (the CLI ships as an az.cmd shim)	shell=False — on POSIX shell=True with an argv list would drop every argument

Every script starts with exactly the same three-line prologue, right after its docstring (the first line keeps whatever stdlib modules that script needs):

import os, sys
from platform_env import bootstrap
bootstrap()

import winreg is guarded by if IS_WINDOWS: inside platform_env.py and must not appear anywhere else. Three regression tests in tests/test_smoke.py enforce the contract — no top-level winreg import, a byte-identical prologue in every script, no hard-coded shell=True — and CI runs the suite on ubuntu-latest.

Requirements common to every platform: Python 3.12, the Azure CLI on PATH with an active az login, and network access to the Fabric APIs.

---

Presentation decks

The .pptx files are not committed (binary Office files carry authoring metadata and sensitivity labels — unwanted in a public repo). Regenerate the architecture-vision deck:

cd presentation
npm install
node build_deck.js          # writes presentation/NOA_Architecture_Vision.pptx

Set your own name on the title/closing slides (defaults to a neutral placeholder):

DECK_AUTHOR="Jane Doe" DECK_AUTHOR_ROLE="Solution Engineer · Data & AI" node build_deck.js

NOA_Demo_Overview.pptx was authored by hand and has no generator — rebuild it from docs/ARCHITECTURE.md if you need it.

---

Public-repo hygiene

This is a synthetic demo: sites are generic French metro areas, and every customer, service and hardware vendor is fictional. Two guards enforce it, in CI and locally:

python scripts/check_repo_leaks.py   # real GUIDs, Fabric endpoints, personal paths
python -m pytest tests/ -q

The scan reads git ls-files only — never the working tree, because __pycache__/*.pyc and node_modules embed absolute build paths and produce false positives.

Detection is by shape, never by name. scripts/check_no_client_leak.py is a byte-identical copy of the canonical scanner shared across the sibling repos; do not edit it, configure it from scripts/check_repo_leaks.py instead.

An optional customer-name rule is driven by a list that is never committed:

• in CI, the CLIENT_DENYLIST repository secret (one entry per line);
• locally, a .clientdeny file at the repo root (git-ignored, same format, # comments).

Without it the rule is skipped with a warning — it never fails the build. Matches are reported as file:line plus the entry rank only, so the terms are not echoed back into public CI logs.

---

Key facts

• Workspace Network Operations (set workspace_name in src/config.yaml) · Capacity any Fabric F-SKU · Tenant your own Entra tenant.
• Culprit device (storyline): SW-TOR-05 (Lille site) — its downstream cells degrade during the PFC storm.
• Telemetry split: topology → Lakehouse (NonTimeSeries) · KPIs/PFC/alarms/logs → Eventhouse (TimeSeries).
• Graph deployment uses the build-and-push workaround (see Azure-Brain/Fabric-Brain/agents/graph-agent).
• Data Agent is dual-source: the ontology answers topology / RCA / impact in GQL, and the SM_Network_Analytics semantic model answers every telemetry NUMBER in DAX. This is deliberate — the Fabric IQ ontology TimeSeries path returns no values, so telemetry questions routed to the ontology alone come back empty.
• RTI dashboard has 3 pages: NOC Overview · RCA Drilldown · Carte Reseau (geospatial — the sites are real French metros, so a basemap is the right abstraction).

---

License

MIT.