feat(wazuh_decoder_rule_tool): Add RAG engine for context-aware decoder generation (#92)

* Add RAG engine integration with ChromaDB vector store for decoder retrieval

- New rag_engine.py module: builds vector store from official Wazuh decoders
  and approved feedback/training pairs; provides retrieve() and format_rag_context()
- Updated main.py: RAG init in lifespan, API endpoint /api/rag/status,
  RAG context injection in AI prompt builder, RAG rebuild on ml_refresh
- Updated requirements.txt: added chromadb>=0.5.0
- Updated .gitignore: exclude .DS_Store, test files, response.json, architecture.html

* Update README with new RAG architecture details and setup instructions

* Add high-level Mermaid architecture diagram to README

* Improve AI prompt building: handle extra_context rule detection, program_name strategy hints, and logtest correction context

* Add async background startup, RAG engine integration, cached logtest health check, and AI prompt improvements

* feat(wazuh_decoder_rule_tool): update README and main.py with enhanced features

- README: complete rewrite with all current features documented
  - RAG engine (ChromaDB, 1700+ Wazuh decoder XMLs)
  - Enhanced ML ensemble (TF-IDF 30% + SBERT 70%)
  - Multi-provider AI (Ollama, DashScope, OpenRouter)
  - SSH remote mode and local sudo mode
  - Rule ML model section
  - All training scripts documented (build_dataset, train_similarity, train_osregex, generate_finetuning_data)
  - Full API reference (13 endpoints)
  - Complete env var table (21 variables) with accurate defaults

- main.py: latest version with all improvements
  - Async background startup with lifespan context manager
  - Cached wazuh-logtest accessibility check (refreshed every 30s)
  - RAG integration for LLM prompt building
  - SBERT 70% + TF-IDF 30% ensemble scoring
  - Rule ML model support
  - Improved OS_Regex generalization heuristics
  - CEF key=value log support
  - Multi-decoder split mode
  - Child rule conditions (field, match, static)

* docs(wazuh_decoder_rule_tool): update README with full UI guide and current features

- Add complete step-by-step UI walkthrough (5 panels)
  - Panel 1 (AI Generate): 7-step guide covering all fields, options,
    log samples, field extraction, AI context, generation buttons,
    and reviewing output XML with validation badges
  - Panel 2 (Test): install decoder, run wazuh-logtest, uninstall steps
  - Panel 3 (Feedback): approve/reject workflow to improve ML model
  - Panel 4 (ML Status): refresh and rebuild model/RAG store
  - Panel 5 (History): session history and reload
- Add UI layout ASCII diagram with status pill descriptions
- Add all new environment variables (21 total) with accurate defaults
  - WAZUH_USE_SUDO, WAZUH_SUDO_PASSWORD, WAZUH_REPO_BRANCH
  - WAZUH_RULESET_REPO_DIR, ML_MODEL_DIR
  - DASHSCOPE_BASE_URL, OPENROUTER_BASE_URL, AI_DEFAULT_MODEL
- Add /api/ai/generate-validated endpoint to API reference
- Add Rule ML Model section
- Update Ollama default URL to http://localhost:11434
- Document all 4 training scripts

* docs: fix UI layout diagram to match actual app screenshot

- Fix broken ASCII box — sidebar and main content area now
  properly separated by a clean vertical divider
- Correct sidebar icons to match actual UI (✳ ML Status, 🔘 History)
- Add 'Main Content Area' label inside the content pane
- Update status pill descriptions to match exact UI text:
  '● Wazuh Local', '● Wazuh Remote', '● Wazuh Local (unavailable)'

* Fix formatting in README.md for Wazuh Decoder Studio

* docs: sync latest codebase and update README (demo gif excluded due to size)

* docs: add optimized demo GIF

* Address PR #92 review: fix health() regression, sudo password exposure, unsupervised refresh thread

- health() now reports wazuh_logtest_exists via the real binary-existence
  check (find_wazuh_logtest()) instead of WAZUH_REMOTE_ENABLED, restoring
  correct connectivity status in local mode.
- run_local_sudo_command()/run_wazuh_logtest() no longer prepend
  WAZUH_SUDO_PASSWORD to stdin shared with user-controlled input; sudo now
  authenticates via a SUDO_ASKPASS helper. All subprocess stdout/stderr
  returned through API responses is redacted for configured secrets as
  defense in depth.
- The background Wazuh-status refresh loop now logs failures instead of
  swallowing them, and is supervised so it restarts (with logging) if it
  ever dies instead of silently freezing the health signal.
- rag_engine.py: escape user-submitted feedback fields before interpolating
  them into decoder XML, preventing structure/content injection.
- .opencode/ added to .gitignore and its tracked file removed.
- chromadb pinned with an upper bound.
- Removed a redundant shadowed `import time`.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

Author: Hasitha9796

.gitignore

@@ -0,0 +1,38 @@
+# Virtual environments
+venv/
+.venv/
+test_venv/
+
+# Python cache
+__pycache__/
+*.pyc
+
+# Local scratch and temp folders
+scratch/
+.claude/
+
+# Cache directories and ML models
+data/models/
+data/wazuh_repo/
+data/wazuh_ruleset_repo/
+.cache_decoders/
+
+
+# Generated output and TLS materials
+generated/
+certs/
+*.pem
+*.key
+*.crt
+
+# OS files
+.DS_Store
+
+# Test files
+test*
+test_*
+
+# Generated artifacts
+response.json
+architecture.html
+data/rag_store/

integrations/wazuh_decoder_rule_tool/.gitignore

@@ -10,6 +10,7 @@ __pycache__/
 # Local scratch and temp folders
 scratch/
 .claude/
+.opencode/
 
 # Cache directories and ML models
 data/models/
@@ -24,3 +25,15 @@ certs/
 *.pem
 *.key
 *.crt
+
+# OS files
+.DS_Store
+
+# Test files
+test*
+test_*
+
+# Generated artifacts
+response.json
+architecture.html
+data/rag_store/