Add yarn twenty pull to write an installed application back to source - #25410
Add yarn twenty pull to write an installed application back to source#25410Weiko wants to merge 7 commits into
yarn twenty pull to write an installed application back to source#25410Conversation
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
Greptile SummaryThis PR adds the experimental
Confidence Score: 3/5The PR is not safe to merge until pull prevents partial commits and preserves pre-existing files at unrecognized generated-path collisions. A commit-phase filesystem error can leave only part of the planned pull applied, and generated writes can silently remove local files that scanning did not associate with the exported entity. Files Needing Attention: packages/twenty-sdk/src/cli/utilities/pull/apply-pull-writes.ts, packages/twenty-sdk/src/cli/utilities/pull/plan-pull-writes.ts Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart LR
CLI[yarn twenty pull] --> Export[Export application]
Export --> Scan[Scan local define files]
Scan --> Base[Read pull base]
Base --> Plan[Plan writes and deletions]
Plan --> Stage[Stage generated files]
Stage --> Commit[Replace and delete destinations]
Commit --> Save[Record new pull base]
Save --> Report[Print writes and coverage]
Reviews (1): Last reviewed commit: "Mark pull as experimental and report def..." | Re-trigger Greptile |
🤖 PR Review
🛡️ Security Review✅ No high-severity vulnerabilities detected. Summary
🚦 Auto-approve🙋 Manual review recommended for the following reason(s):
Automated pre-review — human approval still required. |
🟡 Quality review · 1 finding
High-level — SDK-only pull command with sound architecture (no FE/BE shape divergence, calls an existing server query); the +3110-line size was flagged by the bot and acknowledged by the author, so not re-raised. 💬 1 inline comment on the diff. Reviewed against the |
|
App docs drift check
|
| | `dev` | Watch source files and live-sync changes | [Quick Start](/developers/extend/apps/getting-started/quick-start) | | ||
| | `plan` | Preview metadata changes without applying them | [Syncing & recovery](/developers/extend/apps/operations/sync-and-recovery#previewing-changes-plan) | | ||
| | `apply` | Apply metadata changes after showing the plan | [Syncing & recovery](/developers/extend/apps/operations/sync-and-recovery) | | ||
| | `pull` | **Experimental.** Write the installed application back to local source files | [Syncing & recovery](/developers/extend/apps/operations/sync-and-recovery#pulling-an-installed-app-back-to-source) | |
There was a problem hiding this comment.
would deserve a pull-plan mode (ok for doing it in another PR but i would start by this personnally)
There was a problem hiding this comment.
Agreed, and I would like it too. Two things make me want it as a follow-up rather than here.
It needs the writer to produce content without touching the tree, which is already how it works: the planner returns the full content of every write, and applying it is a separate step. So a plan mode is mostly a flag that stops before applyPullWrites and prints the same report, plus a diff of each file against what is on disk. That diff is the part worth designing properly rather than bolting on, since the useful output is "this file would change in these ways", not just "this file would be rewritten".
The other reason is size: this PR is already flagged as too large to review reliably, so I would rather not add a surface to it.
Happy to take it as the next one if you want it before the remaining entity families.
|
how does conflicts are handled? Like if you pull an object from the remove with a universalId you already use in your local app |
There was a problem hiding this comment.
All reported issues were addressed across 1 file (changes from recent commits).
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
There was a problem hiding this comment.
All reported issues were addressed
Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.
Re-trigger cubic
Short answer: the identifier is the identity, so pull treats that as the same entity and rewrites the file that defines it. The server wins, and the file appears under That is deliberate, but it is worth being precise about the three cases, because only one of them is a real conflict:
There is no case where two genuinely different entities collide on one identifier, because an identifier is unique within an application and the export only returns the identifiers that application owns. What can collide is the file name, and the review caught a bug there which is fixed in 24c38f8: a generated name that would land on an unrelated file is now qualified instead of overwriting it. If you want case 3 surfaced in the report before this ships, that is a small addition and I am happy to add it here. |
|
Review addressed in 24c38f8 and 9090708. Two data-loss bugs were real and are fixed, with tests that fail without the fix:
Also fixed: carriage returns and the two Unicode line separators are escaped (a carriage return really did produce an unterminated literal, verified by evaluating the output); a One suggested fix I did not take as written. Quoting On splitting the SDK foundation into an earlier PR: fair in principle, but the |
There was a problem hiding this comment.
All reported issues were addressed across 14 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
| : undefined; | ||
| }; | ||
|
|
||
| const buildConfigByUniversalIdentifier = ( |
There was a problem hiding this comment.
🟡 Nit · Low-level · naming — name for what it returns
buildConfigByUniversalIdentifier returns a map of serialized JSON strings, not configs
The returned Map<string, string> holds JSON.stringify(entity.config) values for equality comparison, so a caller reading .get(id) gets a string rather than a config. Rename to reflect the serialized value, e.g. buildSerializedConfigByUniversalIdentifier.
What
yarn twenty pullreads an installed application out of a workspace and writes it back as local define files. It is the inverse ofapply, and it consumes theexportApplicationquery added in #25311.The command is experimental and says so: a warning on every run, plus a callout in the docs and a marker in both command tables.
How it works
exportApplicationis added toApplicationApiandApiService, following the existing query on that class. Typed refusals (not exportable, standard application, not found) are relayed as the server worded them.universalIdentifierto the file that currently defines it. The application is matched by kind rather than identifier, so pulling into a scaffold rewrites the config file it already ships instead of leaving twodefineApplicationfiles behind.dev:addlayout..twenty/and moved into place at the end, so a failure halfway leaves the tree as it was. No folder is ever deleted.-vlists identifiers.The base lives at
.twenty/pull-base.json, gitignored and per checkout. A checkout without one writes everything and deletes nothing.Scope
The export fills the application header, objects with their fields, standalone fields on objects the app does not own, and authored indexes. Everything else comes back classified and is reported, not silently dropped. Package files and stored source are not restored here, because the export returns no files yet.
Consequence worth stating plainly: a pruning
applyof a pulled tree destroys the kinds that are not covered yet. That is why the docs and the report both point at--no-delete, and whyplanlists exactly what would go.Three things the implementation had to correct
plan's typecheck. The writer emits symbols with a generated import block. Runningtscover a pulled tree is what caught the number data type case.namefield loseswritability,isUIEditable,isUniqueandisLabelSyncedWithName, which the export carries and the injection does not restore.Objects whose label identifier is engine-derived
Junction objects created in the UI point their label identifier at the object's
idfield, which the engine derives and the export therefore drops. That pointer is portable: it is the deterministic identifier of that field, seeded on the application and object identifiers, so any workspace derives the same value.Two things in the SDK blocked it, and both are fixed here:
defineObjectrejected a label identifier naming no field in the object's ownfields. It is now a warning saying the pointer must name a field the engine derives.namefield into any object lacking one, which would have added a real column to a junction table. That injection is now skipped for exactly this configuration.This cannot affect any existing app, because that configuration was previously a hard error, so no app can be in it.
Verification
Against the seeded Custom application on a running server:
plan --no-deletereportsNo changes. Twenty metadata matches your manifest.;plandestroys exactly the kinds the coverage report named, and nothing else.Offline, building the written tree reproduces the exported objects, fields, indexes and application header exactly once key and collection order are normalised, and a second pull produces zero writes with every entity unchanged.
Tests
write-define-file— enum symbols and the import block, members whose name differs from their value, wrapped imports, quoting and escaping.build-pull-entities— application property stripping, file naming for fields on standard objects and for indexes, the engine-derived label identifier, skipping an index whose object is absent.plan-pull-writes— first pull, base-driven second pull, regeneration into an existing file, deletion, local-only entities, placement beside existing files, collision qualification.format-pull-report— file grouping, per-kind counts, verbose listing and its cap, unreadable files.get-default-fields-in-object-fields— the injection rule, including the new condition.pull-round-tripintegration — writes a fixture export into a temp project, builds it, and asserts the built manifest matches the export, junction object included.define-objectfor the warning that replaced the error.Docs
A
pullrow in the CLI command table, a row in the "which command, when" table, and a section on the sync and recovery page covering what is written, what is not, the report, the base file and the second pull.