Skip to content

Commit 07b65dd

Browse files
committed
WIP(#27237): contract review r3 — fit the eval sentence under its token ceiling
Claude-Session: https://claude.ai/code/session_01Utp6CaqVRa7ow6CjE1JwaG Co-authored-by: Claude <noreply@anthropic.com>
1 parent aff9d4f commit 07b65dd

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

‎skills/objectstack-automation/evals/integrations-declared.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
{
1515
"id": 2,
1616
"prompt": "Every night, read our Stripe account balance and write the available amount and the sync time onto our own company's `crm_account` row (it is flagged `is_house_account`). The Stripe secret key must not appear in metadata.",
17-
"expected_output": "A `connectors:` entry `defineConnector({ provider: 'openapi', providerConfig: { spec: './…/stripe.trimmed.json' }, auth: { type: 'bearer', credentialRef: 'STRIPE_SECRET_KEY' } })` over a trimmed, packaged OpenAPI 3.x document holding `GetBalance`; the administrator sets `STRIPE_SECRET_KEY` (an environment variable of that name, or `PUT /api/v1/automation/connectors/credential-refs/STRIPE_SECRET_KEY`); until it is set the instance is degraded, and the app still boots. A `type: 'schedule'` flow with the cadence on the start node's `config.schedule`, `runAs: 'system'`, a `connector_action` with `actionId: 'GetBalance'` and a literal `input: {}`, then an `update_record` on `crm_account` with `filter: { is_house_account: true }` and `fields` read from the node's answer via CEL (`bal.body.available[0].amount / 100.0`, `isoDatetime(now())`). `requires` lists `automation`, `triggers`, `job`. No `http` node with the key in a header, no plugin.",
17+
"expected_output": "A `connectors:` entry `defineConnector({ provider: 'openapi', providerConfig: { spec: './…/stripe.trimmed.json' }, auth: { type: 'bearer', credentialRef: 'STRIPE_SECRET_KEY' } })` over a trimmed, packaged OpenAPI 3.x document holding `GetBalance`; the administrator sets `STRIPE_SECRET_KEY`; an unset ref degrades the instance, not boot. A `type: 'schedule'` flow with the cadence on the start node's `config.schedule`, `runAs: 'system'`, a `connector_action` with `actionId: 'GetBalance'` and a literal `input: {}`, then an `update_record` on `crm_account` with `filter: { is_house_account: true }` and `fields` read from the node's answer via CEL (`bal.body.available[0].amount / 100.0`, `isoDatetime(now())`). `requires` lists `automation`, `triggers`, `job`. No `http` node with the key in a header, no plugin.",
1818
"files": [],
1919
"assertions": {
2020
"must_contain": ["provider: 'openapi'", "credentialRef", "type: 'schedule'", "runAs: 'system'", "connector_action", "update_record", "dialect: 'cel'"],

0 commit comments

Comments
 (0)