Skip to content

Latest commit

 

History

History
462 lines (353 loc) · 15.8 KB

06_OUTPUT_INTERPRETATION.md

File metadata and controls

462 lines (353 loc) · 15.8 KB

Understanding Output

Complete guide to interpreting Aethelix's reports, visualizations, and confidence scores.

Report Output Example

ROOT CAUSE RANKING ANALYSIS
========================================================================

Most Likely Root Causes (by posterior probability):

1. solar_degradation         P= 46.3%  Confidence=93.3%
   Evidence: solar_input deviation, battery_charge deviation
   Mechanism: Reduced solar input is propagating through the power 
   subsystem. This suggests solar panel degradation or shadowing, which 
   reduces available power for charging the battery.

2. battery_aging             P= 18.8%  Confidence=71.7%
   Evidence: battery_charge deviation, battery_voltage deviation
   Mechanism: Aged battery cells have reduced capacity and efficiency,
   causing lower voltage and charge retention.

3. battery_thermal           P= 18.7%  Confidence=75.0%
   Evidence: battery_temp deviation, battery_voltage deviation
   Mechanism: Excessive battery temperature increases internal resistance,
   reducing charging efficiency and output voltage.

4. sensor_bias               P= 16.3%  Confidence=75.0%
   Evidence: battery_voltage deviation
   Mechanism: Sensor calibration drift could cause all voltage readings
   to be systematically offset, explaining the deviation.

ANOMALY DETECTION REPORT
========================================================================

Most Anomalous Variables (by deviation from nominal):

1. solar_input              Deviation: -59.47 W    (-9.91%)    Onset: 6.48h
2. battery_charge           Deviation: -23.90 %    (-25.04%)   Onset: 6.30h
3. battery_voltage          Deviation: -1.46 V     (-5.21%)    Onset: 7.46h
4. bus_voltage              Deviation: -0.59 V     (-2.11%)    Onset: 7.44h

Overall Severity Score: 20.68%

Mean Deviations:
  solar_input              :    59.47 W
  battery_charge           :    23.90 %
  battery_voltage          :     1.46 V
  bus_voltage              :     0.59 V

Report Components Explained

Root Cause Ranking

Format:

[Rank]. [Cause Name]     P= [Probability]%   Confidence=[Confidence]%
Evidence: [What deviations support this]
Mechanism: [English explanation]

Probability (P)

  • What it means: Posterior probability that this cause explains the observed anomalies
  • Range: 0-100%
  • Important: Probabilities sum to 100% across all hypotheses
  • Interpretation:
    • P > 70%: Very likely, act on this hypothesis
    • P = 30-70%: Possible, needs investigation
    • P < 10%: Unlikely, but don't completely rule out

Example:

  • P = 46.3% means there's a 46.3% chance solar_degradation explains what we observe
  • It's the most likely cause, but not certain (not 90%+)

Confidence

  • What it means: How certain we are about this probability (not about the cause itself)
  • Range: 0-100%
  • Calculation: Based on evidence quality and consistency with the causal graph
  • Interpretation:
    • Confidence > 80%: Strong evidence, high trust in ranking
    • Confidence = 50-80%: Moderate evidence, reasonable trust
    • Confidence < 50%: Weak evidence, low trust in ranking

Important distinction:

High probability + High confidence: "This is probably the cause, and we're sure"
High probability + Low confidence: "This looks likely, but the evidence is weak"
Low probability + High confidence: "This is unlikely, but if true, we're sure"

Evidence

  • What it means: Which measured variables support this hypothesis
  • How it works: The framework traces paths through the causal graph and identifies variables that would change if this cause were active
  • Example: If solar degradation is true, we expect:
    • Lower solar_input (direct cause)
    • Lower battery_charge (consequence of lower input)
    • Potentially higher battery_temp (consequence of longer discharge)

Mechanism

  • What it means: English-language explanation of how this cause produces the effects
  • Not a formula: These are textual descriptions that help operators understand the reasoning
  • Examples:
    • "Reduced solar input -> lower available power -> slower battery charging -> lower battery charge percentage"
    • "Aged battery cells -> reduced capacity -> lower voltage output -> bus voltage drop"

Anomaly Detection Report

Shows which sensors have unusual readings compared to nominal operation.

Format:

[Variable]    Deviation: [Absolute]  ([Percentage])  Onset: [Time]

Deviation: Absolute Change

  • Measured value minus nominal value
  • Same units as the variable
  • Example: -59.47 W means 59.47 W lower than normal

Deviation: Percentage Change

  • (Measured - Nominal) / Nominal x 100%
  • Easier to compare across variables with different scales
  • Example: -9.91% means 9.91% lower than nominal

Onset Time

  • When the anomaly first became significant (>threshold)
  • Helpful for correlating with events or fault injection times
  • Example: 6.48h means anomaly started 6.48 hours into the mission

Severity Score

  • Overall quantification of how wrong the system is
  • Aggregate across all anomalies
  • 0% = completely nominal
  • 100% = completely failed
  • 20.68% = roughly 1/5 of the way to complete failure

Visualization Output

Telemetry Comparison Plot (comparison.png)

Two panels, side-by-side comparison:

LEFT PANEL: NOMINAL                RIGHT PANEL: DEGRADED
+------------------------+         +------------------------+
| Solar Input (W)        |         | Solar Input (W)        |
|  600 --------------    |         |  600 ------------      |
|      |     (down)DROP       |         |      | RED ZONE (fault) |
|  400                   |         |  400 +----------        |
|      +----------------->|         |      +-----------------> |
+------------------------+         +------------------------+
| Battery Charge (%)     |         | Battery Charge (%)     |
|  100 ------------      |         |  100 --------          |
|      |                 |         |      | (down)SLOW RECOVERY  |
|   50 +-----------------|         |   50 +--------------    |
+------------------------+         +------------------------+
| ... (6 more variables) |         | ... (6 more variables) |
+------------------------+         +------------------------+
| Time (hours) ->         |         | Time (hours) ->         |
+------------------------+         +------------------------+

How to read it:

  1. Left panel: What healthy operation looks like (baseline)
  2. Right panel: What we actually observed
  3. Red shaded area: Period when faults were injected (if known)
  4. Deviations: Differences between left and right panels

What to look for:

  • Timing: When do variables change?
  • Magnitude: How much do they deviate?
  • Relationships: Do multiple variables change together (correlated)?
  • Recovery: Do variables recover after the fault period?

Example interpretation:

TIME: 6 hours
LEFT:  Solar input stays ~600W (steady)
RIGHT: Solar input drops to ~400W (and stays low)

CONCLUSION: Solar fault appears to start at t=6h and persists

Residual Analysis Plot (residuals.png)

Shows deviation magnitude over time:

RESIDUAL (DEVIATION)
    100 W |                          /\/\/\
     50 W |           (down)FAULT       /\
       0 W |----------------------
     -50 W |                   \/\/
    -100 W |
          +------------------------------> Time (hours)
            0h        6h         24h

What this shows:

  • How far each variable is from nominal
  • Positive deviation = higher than nominal
  • Negative deviation = lower than nominal
  • Magnitude = how abnormal the system is

What to look for:

  1. Start time: When does deviation become significant?
  2. Magnitude: How big is the deviation?
  3. Trend: Does it worsen, stabilize, or improve?
  4. Correlations: Do multiple variables deviate together?

Example:

Solar input residual: starts at 0, drops at t=6h to -400W, stays there
Battery charge residual: stays near 0 until t=6h, then slowly decreases

INTERPRETATION: Solar fault directly causes battery to discharge

Confidence Intervals

When reported:

solar_degradation: 46.3%  +-  5.2%

This means:

  • Point estimate: 46.3% probability
  • Uncertainty: +- 5.2% (confidence interval)
  • Range: 41.1% - 51.5% (95% confidence interval)

Wider interval = less confident in the exact probability Narrower interval = more confident in the exact probability

Decision Rules

For Operators

Rule 1: Single high-confidence hypothesis

IF P > 60% AND Confidence > 80%
THEN: Trust this diagnosis, take action based on mechanism

Rule 2: Multiple plausible hypotheses

IF multiple causes have P > 20%
THEN: Ambiguous diagnosis, collect more data or request diagnostics

Rule 3: Low confidence overall

IF max(Confidence) < 50%
THEN: Weak evidence, system may be partially masked

For Automated Systems

Automated Response

def get_recommended_action(hypotheses):
    best = hypotheses[0]
    
    if best.probability > 0.7 and best.confidence > 0.8:
        if best.name == "solar_degradation":
            return "rotate_solar_panels"
        elif best.name == "battery_thermal":
            return "reduce_power_load"
        elif best.name == "battery_aging":
            return "plan_battery_replacement"
    
    elif best.probability > 0.4:
        return "request_manual_investigation"
    
    else:
        return "no_action_continue_monitoring"

Common Patterns

Pattern 1: Single Root Cause

solar_degradation: P=70%, Confidence=85%
battery_aging:     P=15%, Confidence=60%
battery_thermal:   P=15%, Confidence=60%

Interpretation: One dominant hypothesis explains observations well

What to do: Act on solar_degradation diagnosis

Pattern 2: Multi-fault Ambiguity

solar_degradation: P=40%, Confidence=65%
battery_aging:     P=35%, Confidence=60%
battery_thermal:   P=25%, Confidence=55%

Interpretation: Multiple causes could explain observations

What to do:

  1. Request additional diagnostics
  2. Isolate each subsystem
  3. Inject test signals to disambiguate

Pattern 3: Weak Signal

solar_degradation: P=25%, Confidence=40%
battery_aging:     P=25%, Confidence=40%
battery_thermal:   P=25%, Confidence=40%
sensor_bias:       P=25%, Confidence=40%

Interpretation: Evidence is too weak, system behavior is ambiguous

What to do:

  1. Wait for more data accumulation
  2. Check for sensor faults
  3. Verify nominal baseline is correct

Pattern 4: High Confidence, Low Probability

solar_degradation: P=15%, Confidence=80%
battery_aging:     P=85%, Confidence=75%

Interpretation: We're confident solar is NOT the cause, battery aging is likely

What to do: Focus on battery aging diagnosis

Debugging Output

No significant anomalies detected

Cause: Deviation threshold too high or nominal scenario incorrect

Solution:

# Lower threshold
analyzer = ResidualAnalyzer(deviation_threshold=0.10)

# Or check nominal baseline
print(f"Nominal solar input: {nominal.solar_input}")
print(f"Degraded solar input: {degraded.solar_input}")

All hypotheses equally likely

Cause: Causal graph is too disconnected or evidence is insufficient

Solution:

# Check graph structure
for edge in graph.edges[:10]:
    print(f"{edge.source} -> {edge.target} (weight: {edge.weight})")

# Or inject stronger faults
power_deg = power_sim.run_degraded(solar_factor=0.3)  # 70% loss instead of 30%

Hypothesis with mechanism but low probability

Cause: Hypothesis is plausible but not well-supported by evidence

Solution:

# This is actually correct behavior - mechanism is good but evidence weak
# System is working as designed

# To increase probability, either:
# 1. Inject stronger faults
# 2. Lower deviation threshold
# 3. Adjust prior probabilities

Exporting Results

Save as JSON

import json

output = {
    "hypotheses": [
        {
            "name": h.name,
            "probability": float(h.probability),
            "confidence": float(h.confidence),
            "mechanisms": h.mechanisms,
            "evidence": h.evidence,
        }
        for h in hypotheses
    ],
    "severity": stats["overall_severity"],
    "timestamp": "2026-01-25T10:30:00Z",
}

with open("output/diagnosis.json", "w") as f:
    json.dump(output, f, indent=2)

Save as CSV

import csv

with open("output/diagnosis.csv", "w", newline="") as f:
    writer = csv.writer(f)
    writer.writerow(["Rank", "Cause", "Probability", "Confidence", "Mechanism"])
    
    for i, h in enumerate(hypotheses, 1):
        writer.writerow([i, h.name, f"{h.probability:.1%}", f"{h.confidence:.1%}", h.mechanisms[0]])

Next Steps


Continue to: Architecture Guide ->


Empirical Flight Validation & DAG Layout Engine (v2.0)

This documentation page includes updates from the v2.0 evaluation suite, incorporating empirical validation on European Space Agency (ESA) flight telemetry and improved hierarchical network rendering:

1. Evaluation on ESA Flight Telemetry (OPS-SAT & ESA-ADB)

  • ESA OPS-SAT (OPSSAT-AD): Evaluated on ADCS reaction wheel magnetic interference and magnetometer attitude anomalies. By applying subsystem-aware persistence filtering ($N=15$ consecutive samples for noisy magnetometer channels, $N=3$ for clean optical sensors), Aethelix achieves a 78.3% F1 score, comparable to deep LSTM autoencoder baselines (78.0% F1) without requiring historical model training.
  • ESA-ADB Multi-Mission: Evaluated on satellite telemetry subject to periodic orbital thermal variation. To prevent false alarms caused by 90-minute day/night baseline oscillations, the engine evaluates innovation residuals ($\Delta x_t = x_t - x_{t-1}$). Decoupling slow orbital dynamics from step-change fault signatures isolates transient subsystem anomalies.

2. Hierarchical Causal DAG Rendering

  • Bounded Functional Corridors: The visualization script (scripts/generate_dag_visuals.py) enforces strict vertical coordinate bounding (maximum node altitude $y=0.68$), ensuring clean separation from structural layer labels ($y=0.87$).
  • Normalized Ranking Score Visualization: In accordance with project semantics, node color intensity maps directly to normalized ranking scores (e.g., 0.94 normalized score weight for PCDU Regulator Failure on GSAT-6A; 0.89 normalized score weight for Reaction Wheel Magnetic Interference on OPS-SAT). Active causal propagation paths connecting dominant root causes to observed telemetry deviations are highlighted along directed edges.

3. CLI Reproduction Commands

Benchmark datasets and diagnostic charts can be generated locally using the standalone scripts:

# Render hierarchical causal DAGs with normalized score weighting
python3 scripts/generate_dag_visuals.py

# Generate empirical validation charts across ESA datasets
python3 scripts/generate_validation_plots.py

# Execute benchmark suite evaluation
python3 scripts/esa_benchmark.py --dataset all

Generated diagnostic charts in docs/:

  • causal_dag_intensity_gsat6a.png — Multi-subsystem causal DAG mapping normalized ranking scores.
  • causal_dag_intensity_opssat.png — ADCS magnetometer interference DAG structure.
  • validation_signal_overlay.png — Telemetry Z-score deviations and persistence window thresholds.
  • validation_confusion_matrix.png — Detection performance comparison against sequence-level baselines.
  • validation_subsystem_metrics.png — Subsystem-level precision, recall, and F1 evaluation.
  • validation_causal_attribution.png — Normalized root-cause score distribution across benchmark scenarios.