feat(red-team): add curated attack datasets

[ccarvalho-eng]

Motivation

Aludel needs reusable security regression inputs that work with its existing datasets, suites, policies, reporters, dashboard, CLI, and ExUnit integrations. This adds a stable catalog and a safe materialization boundary without introducing a separate execution path.

Test Plan

• Added coverage for catalog stability, category and case selection, alternate variables, optional judges, invalid inputs, atomic rollback, configuration drift, payload tampering, idempotent reruns, and database uniqueness.
• Verified the additive migrations apply, roll back, reapply, and recover the concurrent index path safely.
• Verified the full test suite, static analysis, dependency security checks, production compilation, strict documentation generation, and Hex package construction.

Next Steps

A later focused change can build generated red-team workflows on top of this catalog.

Summary by CodeRabbit

• New Features

  • Added a curated, versioned red-team catalog covering key adversarial evaluation categories.
  • Added tools to browse, select, and materialize red-team cases into reusable datasets.
  • Materialized cases include provenance, severity, technique, checksums, and deterministic assertions.
  • Added optional model-based judging and support for alternate prompt variables.
  • Repeated materialization is safely deduplicated, with configuration conflicts reported clearly.

• Documentation

  • Added comprehensive red-team guidance and updated evaluation, feature, and API documentation.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: f96ed0bc-da26-4014-9107-d9f253382c7f

📥 Commits

Reviewing files that changed from the base of the PR and between 4b5bc24 and 43e3868.

📒 Files selected for processing (13)

• README.md
• guides/evaluations.md
• guides/features.md
• guides/red_team.md
• lib/aludel/datasets/dataset_entry.ex
• lib/aludel/red_team.ex
• lib/aludel/red_team/catalog.ex
• mix.exs
• priv/repo/migrations/20260904221403_add_red_team_deduplication_key.exs
• priv/repo/migrations/20260904221404_add_red_team_deduplication_index.exs
• test/aludel/red_team_test.exs
• test/support/migrations/20260904221418_add_red_team_deduplication_key.exs
• test/support/migrations/20260904221419_add_red_team_deduplication_index.exs

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

Adds a versioned Aludel.RedTeam catalog and API for materializing adversarial cases into datasets. The workflow supports filtering, judging, provenance metadata, deterministic deduplication, conflict detection, transactional persistence, tests, and documentation.

Changes

Red-team dataset catalog

Layer / File(s)	Summary
Catalog contract and case definitions lib/aludel/red_team/catalog.ex, lib/aludel/red_team.ex	Defines versioned adversarial templates, metadata, lookup functions, checksums, canary assertions, and public red-team types.
Dataset materialization and deduplication lib/aludel/red_team.ex, lib/aludel/datasets/dataset_entry.ex, priv/repo/migrations/*red_team_deduplication*	Validates materialization options, selects cases, locks datasets, creates entries, records provenance, and detects exact matches or conflicts through a unique deduplication key.
Materialization and database validation test/aludel/red_team_test.exs, test/support/migrations/*red_team_deduplication*	Tests catalog data, materialization, filtering, judging, validation errors, rollback behavior, configuration drift, payload changes, and database uniqueness.
Public usage documentation README.md, guides/evaluations.md, guides/features.md, guides/red_team.md, mix.exs	Documents catalog inspection, materialization options, judging, provenance, deduplication, downstream execution, and the published red-team guide.

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: ⚪ Minimal · up to 43e38

The red-team catalog, materialization workflow, migrations, tests, and documentation are ready to merge.

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant RedTeam as Aludel.RedTeam
  participant Catalog as Aludel.RedTeam.Catalog
  participant Repo as Aludel.Repo
  participant DatasetEntry
  Caller->>RedTeam: materialize(dataset, options)
  RedTeam->>Catalog: select catalog cases
  RedTeam->>Repo: lock dataset and load existing keys
  RedTeam->>DatasetEntry: insert or compare entries
  DatasetEntry-->>RedTeam: created, skipped, or conflict
  RedTeam-->>Caller: materialization result

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 54.55% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 22 functions across 9 files. (4 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: adding curated red-team attack datasets.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 54.55% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 22 functions across 9 files. (4 skipped: 4 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch ccarvalho/curated-red-team-datasets

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}