objectstack/skills/objectstack-data/evals/objects-fields-relationships.json at 5039eb23986ee206b6513fa99042d7675e5d41c8 · objectstack-ai/objectstack · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
{
  "skill_name": "objectstack-data",
  "evals": [
    {
      "id": 1,
      "prompt": "Create `src/objects/vendor.object.ts` for our vendors: company name, contact email, a status (Active / Inactive), the API key we use to call their system, and the password for their supplier portal.",
      "expected_output": "Emits `ObjectSchema.create({...})` with a snake_case `name`, an authored `sharingModel` (a publish with none is refused), `label` / `pluralLabel` / `icon`, `nameField`, and fields with snake_case keys and camelCase config properties: `text` (required), `email`, a `select` whose `options` carry lowercase `value`s (`active`, `inactive`), and BOTH credentials as `type: 'secret'` (encrypted at rest) — never `type: 'password'`, which is plaintext at rest on a generic object. No `namespace` or `tableName` key.",
      "files": [],
      "assertions": {
        "must_contain": [
          "ObjectSchema.create",
          "sharingModel",
          "type: 'secret'",
          "options",
          "nameField"
        ],
        "must_not_contain": [
          "type: 'password'",
          "namespace:",
          "vendorName"
        ]
      }
    },
    {
      "id": 2,
      "prompt": "Model invoices with line items, and let invoices be tagged with many tags where each tagging records who added it and when. A rep who owns a line item but can only read the invoice may still correct that line.",
      "expected_output": "Line items: `invoice_line` with a `master_detail` field to `invoice`, `required: true`, `deleteBehavior: 'cascade'` (or `'restrict'` — `set_null` is refused on master_detail), `inlineEdit` on that relationship, and a `summary` roll-up field on the parent. Tagging carries attributes, so it is a junction object (`invoice_tag`) with two `lookup` fields plus `added_by` / `added_at` — not a `multiple: true` lookup (an array of ids) and not a `many_to_many` type, which does not exist. The line object is `sharingModel: 'controlled_by_parent'` with `sharing: { controlledByParent: { ownerMayEdit: true } }` (by-id update only; every other write still needs master edit). `added_by` is `type: 'user'` — never a `lookup` to a `user` object, which does not exist.",
      "files": [],
      "assertions": {
        "must_contain": [
          "master_detail",
          "deleteBehavior",
          "inlineEdit",
          "summary",
          "lookup",
          "ownerMayEdit",
          "type: 'user'"
        ],
        "must_not_contain": [
          "many_to_many",
          "deleteBehavior: 'set_null'",
          "reference: 'user'"
        ]
      }
    },
    {
      "id": 3,
      "prompt": "On `employee`: email must be unique within each organization, `status` may only move draft → active → terminated, and a `contract_end` earlier than `contract_start` must be rejected.",
      "expected_output": "Uniqueness is an index, not a validation type: `indexes: [{ fields: ['email'], unique: 'organization' }]` (the scope is stated; `type` / `partial` are retired). The lifecycle is one `state_machine` validation with `field: 'status'`, `transitions: { draft: ['active'], active: ['terminated'], terminated: [] }` and a `message`. The date rule is a `script` validation (inverted: fails when the expression is true) or `cross_field`, written as bare CEL over `record.` with explicit `!= null` guards.",
      "files": [],
      "assertions": {
        "must_contain": [
          "unique: 'organization'",
          "state_machine",
          "transitions",
          "record."
        ],
        "must_not_contain": [
          "type: 'unique'",
          "partial:"
        ]
      }
    },
    {
      "id": 4,
      "prompt": "On `invoice`, `paid_at` should only appear — and become mandatory — once the status is `paid`, and `total` must lock after payment.",
      "expected_output": "Declares the rules on the FIELD definitions (they belong to the data model, so every form and API write honours them): `visibleWhen: P`record.status == 'paid'``, `requiredWhen: P`record.status == 'paid'`` on `paid_at`, and `readonlyWhen: P`record.status == 'paid'`` on `total`. Bare CEL with `record.` — never brace-wrapped, never Salesforce syntax, never the removed `conditionalRequired` alias, and not in a form view.",
      "files": [],
      "assertions": {
        "must_contain": [
          "visibleWhen",
          "requiredWhen",
          "readonlyWhen",
          "record.status"
        ],
        "must_not_contain": [
          "conditionalRequired:",
          "{record."
        ]
      }
    },
    {
      "id": 5,
      "prompt": "We installed HotCRM (`hotcrm`, namespace `crm`). Add a `region` field to its `crm_account` object for our own use, without touching HotCRM's source.",
      "expected_output": "An object extension authored in HotCRM's paired customization package `hotcrm.custom` (never a replacement of the object, never an edit of HotCRM itself): `defineObjectExtension({ extend: 'crm_account', fields: { region__c: Field.text({ label: 'Region' }) } })` registered in that stack's `objectExtensions`. The field name ends in `__c` because the suffix is reserved to the pair; an unsuffixed field in a `.custom` extension is refused at save, publish, install, `os validate` and `os build`. A change to one of HotCRM's own fields goes through `fieldPatches` (display properties only), and the object takes no `overrides` declaration.",
      "files": [],
      "assertions": {
        "must_contain": [
          "hotcrm.custom",
          "defineObjectExtension",
          "extend: 'crm_account'",
          "region__c"
        ],
        "must_not_contain": [
          "overrides:",
          "name: 'crm_account'"
        ]
      }
    },
    {
      "id": 6,
      "prompt": "On `ticket`, `owner` is the agent who holds it and must always be set; deleting that agent's user account must be blocked while they still hold tickets. `watchers` is several users, and a deleted user should just drop out of the list.",
      "expected_output": "Both are `type: 'user'` fields, each with an authored `deleteBehavior` — a `user` field joins the referential delete walk only when it declares one. `owner`: `required: true` with `deleteBehavior: 'restrict'` (deleting a held user is refused with `409 DELETE_RESTRICTED`; `set_null` on a required field would refuse too). `watchers`: `multiple: true` with `deleteBehavior: 'set_null'` (only the deleted member is removed; an emptied set is `[]`). Neither is a `lookup` to a `user` object, which does not exist, and a `user` field with no `deleteBehavior` keeps the deleted user's id.",
      "files": [],
      "assertions": {
        "must_contain": [
          "type: 'user'",
          "deleteBehavior: 'restrict'",
          "deleteBehavior: 'set_null'",
          "multiple: true",
          "required: true"
        ],
        "must_not_contain": [
          "reference: 'user'",
          "deleteBehavior: 'cascade'"
        ]
      }
    }
  ]
}