feat(usta): complete manager-supported catalog migration (#7656) - #7703
Open
Hugo Dupras (jabesq) wants to merge 6 commits into
Open
Hugo Dupras (jabesq) wants to merge 6 commits into
Hugo Dupras (jabesq) wants to merge 6 commits into
Conversation
|
The following packages appear to be unused:
|
|
🔴 Connector Linter errors detected
|
Contributor
There was a problem hiding this comment.
Pull request overview
Migrates USTA to the canonical connectors-sdk configuration and OpenCTI connector catalog format, including secure API-key handling and generated metadata.
Changes:
- Replaced local configuration models with SDK equivalents.
- Added catalog metadata, schema, documentation, and manager support.
- Added validation, secret-redaction, and helper compatibility tests.
- Updated deployment configuration and documentation.
Reviewed changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Summary |
|---|---|
external-import/usta/tests/test_main.py |
Added integration and helper compatibility coverage. |
external-import/usta/tests/test_connector/test_settings.py |
Added settings validation and secret-redaction tests. |
external-import/usta/src/requirements.txt |
Added the connectors-sdk dependency. |
external-import/usta/src/connector/settings.py |
Migrated settings to SDK models with secure secret handling. |
external-import/usta/src/__init__.py |
Re-exported ConnectorSettings. |
external-import/usta/README.md |
Updated configuration and deployment documentation. |
external-import/usta/docker-compose.yml |
Preserved deployment environment settings. |
external-import/usta/.env.sample |
Organized configuration variables; CONNECTOR_LOG_LEVEL remains active despite the optional block. |
external-import/usta/__metadata__/connector_manifest.json |
Enabled manager support. |
external-import/usta/__metadata__/connector_config_schema.json |
Added generated catalog schema. |
external-import/usta/__metadata__/CONNECTOR_CONFIG_DOC.md |
Added generated configuration documentation. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+8
to
+10
| # Connector parameters - OPTIONAL (default values are defined in src/connector/settings.py) | ||
| # CONNECTOR_NAME=USTA | ||
| # CONNECTOR_SCOPE=indicator,observable,malware,identity,incident,user-account,report,threat-actor |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
connectors-sdkbase classes.settings.pycarried its own private copies ofBaseConfigModel,_OpenCTIConfig,_BaseConnectorConfig,BaseExternalImportConnectorConfig,BaseConnectorSettingsandto_helper_config(). Duplicated base classes drift from the SDK over time and each connector then fixes the same bug independently, so these now import fromconnectors-sdk. Field names, types, defaults and environment variable names are preserved 1:1 — this is a no-behaviour-change refactor apart from the security fix below.USTA_API_KEYinto the helper config. The removed hand-rolled@field_serializer("api_key")called.get_secret_value()duringmodel_dump(), which defeated the whole point ofSecretStrand pushed the plaintext API key into the dict handed toOpenCTIConnectorHelper— where it could surface in helper state and debug output. The SDK'sSecretStrhandling redacts it instead, whileconnector.pycontinues to read the real token via.get_secret_value()at the call site where it is actually needed. This is the most reviewer-relevant change in the PR.connector.scopeto the SDKListFromStringtype. The scope is semantically a list, and modelling it as one lets the generated config schema describe it correctly for the connector catalog. The values are identical and pycti still receives the comma-joined string, so deployed behaviour is unchanged.connectors-sdkdependency tosrc/requirements.txt. Without it the config-schema generator refuses to run, so the connector could never be onboarded into the catalog.src/__init__.pyre-exportingConnectorSettings. The schema generator imports the settings class from the package root; this matches the layout already used bycybelangel,elastic-security-incidentsandalienvault.connector.ida repo-unique UUIDv4 default. Catalog deployment has no opportunity to inject aCONNECTOR_ID, so a required-with-no-default field made the connector undeployable from the catalog.CONNECTOR_ID=ChangeMeis deliberately kept indocker-compose.ymland.env.sample: operators running multiple USTA instances must still override it to avoid two connectors sharing an identity, and the README now says so explicitly.manager_supported: truein__metadata__/connector_manifest.jsonand commit the generatedconnector_config_schema.jsonandCONNECTOR_CONFIG_DOC.md, which together form the machine-readable contract the OpenCTI connector manager consumes..env.sampleis regrouped into REQUIRED / OPTIONAL blocks with the optional variables commented out, sosettings.pyremains the single source of truth for defaults rather than having the same value restated — and silently diverging — in three places. The README configuration table is updated to match, including the newly documentedconnector.scopeandconnector.log_levelrows.ConfigValidationError, an assertion on theconnector.iddefault, atest_api_key_is_a_secretregression test pinning the redaction fix, and ato_helper_config()pycti-compatibility test. 222 tests pass.Related issues
Checklist
Further comments
This is a deliberately structure-preserving pass: file structure, file names, class names, logging and the
schedule_process/duration_periodscheduling mechanism are all unchanged, which keeps the diff reviewable and lets the API-key redaction fix land without being buried in a restructuring. USTA was already partially migrated on master (a Pydanticsettings.pyexisted andmain.pyalready wiredConnectorSettings()→to_helper_config()), but it was not actually catalog-ready — this branch finishes that job.The connector is now manager-supported but not yet "verified": package restructuring, the logging refactor and the STIX-ID determinism review are intentionally left for a separate pass.
Validation run locally: isort + black clean,
flake8 --ignore=E,Wclean, 222/222 tests passing on Python 3.12, the custom STIX-ID pylint plugin rated 10.00/10, andmise gs external-import/ustareproduces the committed schema and config doc with no diff.