Skip to content

Commit c4b49c3

Browse files
fix(rest)!: the CSV export writes a text cell in a form its consumer reads as text, and the CSV import reverses it (#25078) (#25130)
Fixes #25078 Clause-②: no (narrowing) Metadata-protocol: none The CSV export now writes a text cell its consumer would interpret in a form that consumer reads as text, and the CSV import reverses exactly that. This is this repository's half of the maintainer's choice B on #25078 (comment `6319255545`): the rule is defined once, server-side, and both directions read it. Both landing prerequisites are on `main`: objectstack-ai/objectui#13316, merged as `56cf4d8e89`, and this repository's `.objectui-sha` bump over it, PR #25160, merged as `06900c9c2`. ## Contract **Served answers that change** - `GET /api/v1/data/:object/export?format=csv`, on the metadata path and on the raw path (no field metadata). A CSV text cell is served with one apostrophe (U+0027) in front of its content, before the RFC-4180 quoting, when both of these hold: - its first character is one of a small fixed set that the file's consumer interprets; - it is not made only of digits, spaces and the characters a signed number or a phone number is written with. - A text cell is a string in a column whose field is not declared numeric, boolean, date or datetime. The declared type decides, never the JS type. Header labels and reference display titles are text cells too. On the raw path every string cell is one. - Every other byte of the file is unchanged. - `parseCsvToRows`, exported from `@objectstack/rest`, and through it the CSV payload of three doors: - `POST /api/v1/data/:object/import`; - `POST /api/v1/data/:object/import/jobs`; - `POST /api/v1/auth/admin/import-users`. - Every cell, header row included, loses exactly one leading apostrophe when the rest of the cell is one the export would mark. Cells without that shape are unchanged. - The reversal runs at the parse, before an inline or a named mapping, and it reads the cell text only. With a named mapping, rows keep their source headers until core's `applyMappingToRows`, so no target field type is known at that point. **Unchanged by construction:** phone numbers and signed amounts; every cell that does not begin with one of the set; numeric, boolean, date and datetime cells; the xlsx and json export formats; json and xlsx import payloads. **Stated consequences** - A user-authored CSV cell that is an apostrophe followed by a cell the export would mark now imports without that apostrophe. A cell that begins with an apostrophe followed by anything else imports unchanged. - A stored text value of that same shape is written unchanged by the export, so it comes back without its leading apostrophe. Measured with a one-shot script against this branch: it is the one stored shape the round trip does not return unchanged. The maintainer kept the rule as chosen (ruling record `6328219098` on the card). - A value beginning with TAB or CR is marked, and the import removes the mark. Core's default `trimWhitespace: true` then trims the leading TAB or CR, exactly as it did before this change. With `trimWhitespace: false` the value returns exactly. Both cases are pinned in `csv-text-cell-round-trip.test.ts`. The rule lives in `packages/rest/src/export-format.ts` (`csvTextCellNeedsMark`, `markCsvTextCell`, `unmarkCsvTextCell`, `isCsvTextColumn`). The writer is `formatCsvCell` / `rowsToCsv` in `rest-server.ts`. The reversal is in `parseCsvToRows` in `import-prepare.ts`. ## Files - `packages/rest/src/export-format.ts`: the rule, defined once. The set of non-text types is composed from the spec's `NUMERIC_VALUE_TYPES`, `BOOLEAN_VALUE_TYPES`, `CALENDAR_DATE_TYPES` and `INSTANT_TYPES`, not re-listed. - `packages/rest/src/rest-server.ts`: `formatCsvCell` takes a text-column flag, and `rowsToCsv` derives that flag per column from the declared type. - `packages/rest/src/import-prepare.ts`: `parseCsvToRows` reverses every cell, and reverses header cells before trimming and mapping. - `packages/rest/src/data-action-routes.ts`: comment only. The export route's doc said the raw path streams stored values unchanged, which this change makes false for csv. - Pins: - `packages/rest/src/csv-text-cell.test.ts` (new); - `packages/rest/src/csv-text-cell-round-trip.test.ts` (new); - one new `describe` in `packages/plugins/plugin-auth/src/admin-import-users.test.ts`, on that file's existing engine double, so the engine-double ledger does not move. - `.changeset/25078-csv-text-cell.md`: `@objectstack/rest` `patch`. It carries the BREAKING banner and one ADR-0087 marker, `not-required (no-migration-prescription)`. Its "What to do" line points at the rule in this repository's `packages/rest/src/export-format.ts`, not at a function name the package does not publish. ## Tests The tables and the ablations below were measured at `bf8244cb20`. The suites were run again on the merged head `2e4476001`. - **The predicate:** one table of 56 cases, with the boundary characters on both halves of the rule. - The code-point neighbours of each trigger as a first character. - Every character of the numeric-text set, after each trigger character the rule's second half applies to. - A letter or a trailing word outside that set. - **The writer table**, per value, on the metadata path and on the raw path: - plain words after a trigger character are marked; - two phone numbers, a signed amount, a company name and an unsigned amount are byte-identical; - `number`, `currency` and `percent` fields holding a negative amount stay numeric and unprefixed; - a string in a numeric, boolean, date or datetime column is never marked; - header labels and reference titles that start with a trigger character are marked. - **The round trip, through each CSV door:** - the sync import, with an inline mapping and with a named mapping; - the async import job; - the admin user import, whose file comes from the real export route of `RestServer`. - Each case asserts that the file carried the marks, and that every stored value came back exactly, including a value that begins with an apostrophe followed by a plain word. - **Suites, at `2e4476001`:** - This PR's pins: `csv-text-cell.test.ts` and `csv-text-cell-round-trip.test.ts`, 2 files / 133 tests passed; `admin-import-users.test.ts`, 41 of 41 passed. - `@objectstack/rest` local project: 314 files / 6318 tests passed, 326 skipped. - `@objectstack/plugin-auth`: 183 files / 3464 tests passed, 10 skipped. - Both packages' `typecheck`, the test layer included, exit 0. - `@objectstack/dogfood` (the full battery, owed by the `!` title): 287 files passed and 3 skipped; 2395 tests passed and 18 skipped. **Ablations,** measured at `bf8244cb20` and not run again: this round changes no tested behaviour. Each leg goes through `scripts/ablation-replace.mjs`. The anchor must hit, and the restore is proven by blob equal to HEAD and an empty `git diff HEAD`. The whole tree's porcelain is empty after every leg. - Writer ablated (`formatCsvCell` never marks): 20 red of 133. - Header marking ablated: 5 red. - Declared-type decision ablated (`isCsvTextColumn` always true): 12 red. - The predicate's second half removed: 36 red. - Cell reversal removed: 14 red. - Header reversal removed: 6 red. - Admin door, which resolves `@objectstack/rest` through `dist/`: - The reversal was removed and rest was rebuilt. `ablation-dist-preflight` then reported the marker absent from all 6 built files, and the admin round trip went red, 1 of 1. - After the restore and a rebuild, the preflight reported the marker present in 2 built files and a clean tree, and the pin was green again. ## Gates All readings are on the merged head `2e4476001`, with each exit code captured before any pipe. - `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derived 71 commands from the 8 changed paths against the merge base `06900c9c2`. All 71 exited 0 in one run, after a full workspace build. - The reconciliation printed: `✓ dispatch-gates --ran: 71 derived famil(ies) accounted for — 71 run, 0 NOT-MEASURED (a DERIVED zero — all 71 recorded an exit code and none of them is 3)`. - `node scripts/check-changeset-no-major.mjs --base origin/main --event` with this body: exit 0, `LEVEL AXIS ... a non-protocol change at patch`. - The families the derivation declares too wide to place, and `pnpm lint`, are CI's runs and were not run locally. - CI runs the full farm. `main` at `fccbd9d0d5` was merged once into this branch. The incoming commits touch no file this PR edits. One of them adds the NOT NULL tenant column, and that is why the round-trip harness calls `installDefaultOrganization`. `main` at `06900c9c2` was then merged for landing, with no conflict. Of the files this PR edits, it changes only `rest-server.ts`, and only comments there (PR #25124). The pre-landing round, which is that merge and the changeset's "What to do" line, is by session `session_01AFrKwndmrCLtF5V9hNtrw3`. ## Acceptance notes - **The changeset level is `patch`**, with `Metadata-protocol: none`. That follows AGENTS.md Post-Task Checklist step 3 as it reads on `main` since `f2b6cf230b`: a non-protocol change, widening or breaking, ships `patch` and keeps its BREAKING banner and its ADR-0087 disposition. - **For objectstack-ai/objectui#13315:** an inline mapping sent with CSV text must name a column by its header as this parse reads it, that is after the reversal. This only matters for a column whose label begins with one of the set. At the pin `56cf4d8e89`, the wizard sends no inline mapping. Its hand-mapped rows are keyed by field name in the browser, and a named mapping travels by name (`buildImportRequest` in `ImportWizard.tsx`, read with `git show`). - **The round-trip harness calls `installDefaultOrganization`.** The async job door's `sys_import_job` write needs the Default Organization on the NOT NULL tenant column that landed on `main` in `5e9b100743`. Without that call one test is red, measured. --- _Generated by [Claude Code](https://claude.ai/code/session_01LmjU2x1faLZ39DVTFyVca6)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 82cfaf7 commit c4b49c3

8 files changed

Lines changed: 740 additions & 10 deletions

File tree

‎.changeset/25078-csv-text-cell.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
'@objectstack/rest': patch
3+
---
4+
5+
fix(rest)!: the CSV export writes a text cell its consumer would interpret in a form that consumer reads as text, and the CSV import reverses exactly that
6+
7+
Clause-②: no (narrowing) — the CSV export serves different bytes for some text cells, and the CSV import parse reads a cell of the same shape differently; nothing is widened.
8+
Metadata-protocol: none (a REST export's bytes and a REST import's parse; nothing a metadata author writes changes)
9+
10+
<!-- adr-0087: not-required (no-migration-prescription) No metadata moves: no spec key, authorable spelling, object definition or stored shape is added, removed, renamed or re-shaped, and no exported declaration is removed or narrowed, so there is nothing for `objectstack migrate meta` to rewrite. What changes is the bytes one REST export format serves for some text cells and how the CSV import parse reads a cell of that same shape; stored rows are not touched. The other categories are closed on facts: the package publishes (not unpublished); no ADR-0087 id is named or touched (not registered or already-registered); and nothing exported is removed or narrowed (not runtime-interface-only, type-surface-only or runtime-export-retired). -->
11+
12+
**BREAKING** (the CSV export's bytes change for some text cells, and the CSV import reads a user-authored cell of the same shape differently), shipped as `patch` because the metadata protocol does not move.
13+
14+
The CSV export (`GET /api/v1/data/:object/export?format=csv`) wrote every text cell exactly as stored, so a cell whose leading character the program the file is opened in interprets was read there as something other than text. The rule below is defined once, in `csvTextCellNeedsMark` (`packages/rest/src/export-format.ts`), and both directions read it.
15+
16+
- **The export.** A CSV text cell whose first character is one of a small fixed set that its consumer interprets is written with one apostrophe (U+0027) in front of its content, before the usual RFC-4180 quoting. A text cell is a string in a column whose field is not declared numeric, boolean, date or datetime, a header label, or a reference field's display title; with no field metadata (the raw path) every string cell is one. A cell made only of digits, spaces and the characters a signed number or a phone number is written with is never marked, so phone numbers and signed amounts are byte-identical.
17+
- **The import.** `parseCsvToRows` removes exactly one leading apostrophe from a cell, header row included, when the rest of the cell is one the export would mark. It serves the sync import (`POST /api/v1/data/:object/import`), the async import job (`POST /api/v1/data/:object/import/jobs`) and plugin-auth's admin user import (`POST /api/v1/auth/admin/import-users`), with an inline mapping or a named one. So a CSV exported and imported back returns the stored value, with the one exception in the next point.
18+
- **A cell that begins with an apostrophe** followed by a cell the export would mark now imports without that apostrophe; the file cannot tell it from an exported one. That holds for a user-authored cell, and for a stored text value of that shape, which the export writes unchanged, so it comes back without its leading apostrophe. A cell beginning with an apostrophe followed by anything else imports unchanged.
19+
- **Measured.** A value beginning with TAB or CR is marked; the import removes the mark, and the default `trimWhitespace: true` then trims the leading TAB or CR exactly as it did before. With `trimWhitespace: false` it returns exactly.
20+
- **Unchanged.** Every other cell; numeric, boolean, date and datetime cells; the xlsx and json export formats; json and xlsx import payloads.
21+
- **What to do.** Nothing for a file that goes back through these import doors. A program that reads the CSV export some other way and needs the stored text applies the same reversal: remove one leading apostrophe from a cell whose rest the export would mark, by the rule in this repository's `packages/rest/src/export-format.ts`.

‎packages/plugins/plugin-auth/src/admin-import-users.test.ts‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ import { readFileSync } from 'node:fs';
55
import { dirname, resolve } from 'node:path';
66
import { fileURLToPath } from 'node:url';
77
import { assertEngineUpdateDispatch } from '@objectstack/objectql';
8+
import { RestServer } from '@objectstack/rest';
89
import { runAdminImportUsers, IMPORT_USERS_MAX_ROWS, type IdentityImportDeps } from './admin-import-users.js';
910
import { settleImportResetDelivery } from './account-invitation-marker.js';
1011
import type { AdminActor } from './admin-user-endpoints.js';
@@ -719,6 +720,75 @@ describe('runAdminImportUsers — CSV payloads', () => {
719720
});
720721
});
721722

723+
/**
724+
* [#25078] Export → admin user import round trip, through the CSV text-cell rule.
725+
*
726+
* `@objectstack/rest`'s CSV export writes a text cell its consumer would
727+
* interpret in a form that consumer reads as text, and its CSV import parse
728+
* (`parseCsvToRows`, reached through `prepareImportRequest`) reverses exactly
729+
* that. This door parses a `csv` body through that same `prepareImportRequest`,
730+
* so a user list exported as CSV and imported back here must hand better-auth's
731+
* `createUser` every stored name exactly.
732+
*
733+
* The file comes from the REAL export route of `@objectstack/rest`'s
734+
* `RestServer`, over a protocol double serving a two-field `sys_user` schema and
735+
* the rows below. The rule is pinned at its readers in `@objectstack/rest`
736+
* (`csv-text-cell.test.ts`), and its other CSV doors in that package's
737+
* `csv-text-cell-round-trip.test.ts`.
738+
*/
739+
describe('runAdminImportUsers — CSV export → import round trip (#25078)', () => {
740+
/** Plain words after a trigger character: marked by the export. */
741+
const MARKED = ['=Total', '+Name', '-Note', '@acme'];
742+
/** Ordinary business values, and a user-authored name that begins with the mark itself. */
743+
const UNMARKED = ['+86 138 0013 8000', '+1 555 010 0000', '-120.50', 'Acme Ltd', '120.50', "'Quoted"];
744+
const NAMES = [...MARKED, ...UNMARKED];
745+
746+
async function exportUsersCsv(rows: Array<Record<string, string>>): Promise<string> {
747+
const noop = () => {};
748+
const server = { get: noop, post: noop, put: noop, delete: noop, patch: noop, use: noop, listen: async () => {}, close: async () => {} };
749+
const schema = { name: 'sys_user', fields: { email: { name: 'email', type: 'email' }, name: { name: 'name', type: 'text' } } };
750+
const protocol = {
751+
getMetaItem: vi.fn(async () => ({ type: 'object', name: 'sys_user', item: schema })),
752+
findData: vi.fn(async () => ({ object: 'sys_user', records: rows, total: rows.length })),
753+
};
754+
const rest = new RestServer(server as any, protocol as any, { api: { requireAuth: false } } as any);
755+
(rest as any).resolveExecCtx = async () => ({ userId: ACTOR.id });
756+
rest.registerRoutes();
757+
const route: any = rest.getRoutes().find(
758+
(r: any) => r.method === 'GET' && r.path === '/api/v1/data/:object/export',
759+
);
760+
expect(route).toBeDefined();
761+
const chunks: string[] = [];
762+
const res: any = {
763+
write: (s: unknown) => { chunks.push(String(s)); return true; },
764+
end: () => {},
765+
header: () => res,
766+
status: () => res,
767+
json: () => res,
768+
};
769+
await route.handler({ params: { object: 'sys_user' }, query: { format: 'csv' } }, res);
770+
return chunks.join('');
771+
}
772+
773+
it('hands createUser every exported name exactly', async () => {
774+
const rows = NAMES.map((name, i) => ({ email: `user${i + 1}@example.com`, name }));
775+
const csv = await exportUsersCsv(rows);
776+
777+
// The file carried the marks, so the round trip below is not a pass-through.
778+
expect(csv.split('\r\n')[0]).toBe('email,name');
779+
for (const [i, v] of MARKED.entries()) expect(csv).toContain(`user${i + 1}@example.com,'${v}\r\n`);
780+
expect(csv).toContain(',+86 138 0013 8000\r\n');
781+
expect(csv).toContain(",'Quoted\r\n");
782+
783+
const m = makeDeps();
784+
const res = await runAdminImportUsers(m.deps, makeRequest({ passwordPolicy: 'temporary', csv }), ACTOR);
785+
expect(res.status).toBe(200);
786+
expect((res.body.data as any).summary.created).toBe(NAMES.length);
787+
const byEmail = new Map(m.createUser.mock.calls.map((c: any[]) => [c[0].body.email, c[0].body.name]));
788+
expect(rows.map((r) => byEmail.get(r.email))).toEqual(NAMES);
789+
});
790+
});
791+
722792
/**
723793
* [#17422] The IMPLEMENTOR half of #16952's contract.
724794
*
Lines changed: 228 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,228 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [#25078] Export → import round trip through each REST door that parses CSV.
5+
*
6+
* The CSV export writes a text cell its consumer would interpret in a form that
7+
* consumer reads as text; the CSV import parse reverses exactly that. The rule
8+
* is pinned at its readers in `csv-text-cell.test.ts`. This file drives the
9+
* REAL routes over a REAL `ObjectQL` engine with a better-sqlite3 `:memory:`
10+
* driver: rows are stored in `note`, exported as CSV through
11+
* `GET /data/note/export`, and that file is imported into `note_copy`'s text
12+
* fields through
13+
*
14+
* - the sync door, `POST /data/note_copy/import`, with an inline mapping;
15+
* - the sync door with a NAMED mapping, whose rows keep their source headers
16+
* until core's `applyMappingToRows` — the reversal must not depend on a
17+
* target field's type;
18+
* - the async door, `POST /data/note_copy/import/jobs`.
19+
*
20+
* Every stored value comes back exactly. Each case also asserts that the file
21+
* it imported carried the marks, so the round trip cannot pass by the export
22+
* never having marked anything. plugin-auth's admin user import is pinned in
23+
* its own package (`admin-import-users.test.ts`).
24+
*
25+
* Control characters are written as escapes in this file, never as raw bytes.
26+
*/
27+
28+
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
29+
import { ObjectQL } from '@objectstack/objectql';
30+
import { SqlDriver } from '@objectstack/driver-sql';
31+
import { ObjectStackProtocolImplementation } from '@objectstack/metadata-protocol';
32+
import { SysMetadataAuditObject, SysMetadataCommitObject, SysMetadataHistoryObject, SysMetadataObject } from '@objectstack/metadata-core';
33+
import { SysImportJob } from '@objectstack/platform-objects/audit';
34+
import { RestServer } from './rest-server';
35+
import { installDefaultOrganization } from './default-organization.testkit.js';
36+
37+
const liveEngines: ObjectQL[] = [];
38+
afterEach(async () => {
39+
while (liveEngines.length) {
40+
try { await liveEngines.pop()?.destroy(); } catch { /* noop */ }
41+
}
42+
});
43+
44+
/**
45+
* Text fields only, so every value lands in a text field. `memo`'s label starts
46+
* with a trigger character, so the header row carries a marked label too.
47+
*/
48+
function noteObject(name: string) {
49+
return {
50+
name,
51+
label: name,
52+
systemFields: false,
53+
fields: {
54+
id: { name: 'id', type: 'text' as const, primaryKey: true, label: 'ID' },
55+
body: { name: 'body', type: 'text' as const, label: 'Body' },
56+
memo: { name: 'memo', type: 'text' as const, label: '-Memo' },
57+
},
58+
};
59+
}
60+
61+
/** Plain words after a trigger character: marked by the export. */
62+
const MARKED = ['=Total', '+Name', '-Note', '@acme'];
63+
/** Ordinary business values, and a user-authored cell that begins with the mark itself. */
64+
const UNMARKED = ['+86 138 0013 8000', '+1 555 010 0000', '-120.50', 'Acme Ltd', '120.50', "'Quoted"];
65+
const VALUES = [...MARKED, ...UNMARKED];
66+
67+
const INLINE_MAPPING = { ID: 'id', Body: 'body', '-Memo': 'memo' };
68+
69+
function createMockServer() {
70+
const noop = () => {};
71+
return { get: noop, post: noop, put: noop, delete: noop, patch: noop, use: noop, listen: async () => {}, close: async () => {} };
72+
}
73+
74+
function makeRes() {
75+
const chunks: string[] = [];
76+
const res: any = {
77+
write: (s: string) => { chunks.push(typeof s === 'string' ? s : String(s)); return true; },
78+
end: () => {},
79+
header: () => res,
80+
status: (code: number) => { res._status = code; return res; },
81+
json: (body: any) => { res._json = body; return res; },
82+
};
83+
return { res, text: () => chunks.join('') };
84+
}
85+
86+
async function boot(values: string[]) {
87+
const engine = new ObjectQL();
88+
liveEngines.push(engine);
89+
engine.registerDriver(new SqlDriver({ client: 'better-sqlite3', connection: { filename: ':memory:' }, useNullAsDefault: true }), true);
90+
await engine.init();
91+
engine.registry.registerObject(noteObject('note') as any);
92+
engine.registry.registerObject(noteObject('note_copy') as any);
93+
engine.registry.registerObject(SysImportJob as any);
94+
await engine.syncSchemas();
95+
// A real `single` install's Default Organization: the system writes below
96+
// derive it (ADR-0131 D9), as a fresh NOT NULL tenant column requires (D1).
97+
await installDefaultOrganization(engine);
98+
for (let i = 0; i < values.length; i++) {
99+
await engine.insert('note', { id: `r${i + 1}`, body: values[i], memo: values[i] });
100+
}
101+
for (const o of [SysMetadataObject, SysMetadataHistoryObject, SysMetadataAuditObject, SysMetadataCommitObject]) {
102+
if (!engine.registry.getObject(o.name)) engine.registry.registerObject(o as any);
103+
}
104+
engine.registry.registerItem('mapping', {
105+
name: 'note_copy_feed',
106+
label: 'Note copy feed',
107+
sourceFormat: 'csv',
108+
targetObject: 'note_copy',
109+
fieldMapping: [
110+
{ source: 'ID', target: 'id', transform: 'none' },
111+
{ source: 'Body', target: 'body', transform: 'none' },
112+
{ source: '-Memo', target: 'memo', transform: 'none' },
113+
],
114+
} as any, 'name');
115+
const protocol = new ObjectStackProtocolImplementation(engine as any);
116+
const rest = new RestServer(createMockServer() as any, protocol as any, { api: { requireAuth: false } } as any);
117+
(rest as any).resolveExecCtx = async () => ({ userId: 'test-user' });
118+
rest.registerRoutes();
119+
const routes = rest.getRoutes();
120+
const find = (method: string, path: string) => routes.find((r: any) => r.method === method && r.path === path)!;
121+
return {
122+
engine,
123+
exportRoute: find('GET', '/api/v1/data/:object/export'),
124+
importRoute: find('POST', '/api/v1/data/:object/import'),
125+
jobCreate: find('POST', '/api/v1/data/:object/import/jobs'),
126+
jobProgress: find('GET', '/api/v1/data/import/jobs/:jobId'),
127+
};
128+
}
129+
130+
type Booted = Awaited<ReturnType<typeof boot>>;
131+
132+
async function exportCsv(ctx: Booted): Promise<string> {
133+
const { res, text } = makeRes();
134+
await ctx.exportRoute.handler({ params: { object: 'note' }, query: { format: 'csv' } } as any, res);
135+
return text();
136+
}
137+
138+
async function importSync(ctx: Booted, body: Record<string, unknown>): Promise<any> {
139+
const { res } = makeRes();
140+
await ctx.importRoute.handler({ params: { object: 'note_copy' }, body } as any, res);
141+
return res._json;
142+
}
143+
144+
async function importAsync(ctx: Booted, body: Record<string, unknown>): Promise<any> {
145+
const { res } = makeRes();
146+
await ctx.jobCreate.handler({ params: { object: 'note_copy' }, body } as any, res);
147+
expect(res._status).toBe(201);
148+
const jobId = res._json.jobId;
149+
for (let i = 0; i < 200; i++) {
150+
const p = makeRes();
151+
await ctx.jobProgress.handler({ params: { jobId }, query: {} } as any, p.res);
152+
const status = p.res._json?.status;
153+
if (status === 'succeeded' || status === 'failed' || status === 'cancelled') return p.res._json;
154+
await new Promise((r) => setTimeout(r, 5));
155+
}
156+
throw new Error(`import job ${jobId} did not finish`);
157+
}
158+
159+
async function storedCopies(ctx: Booted, count: number): Promise<Array<{ body: unknown; memo: unknown }>> {
160+
const out: Array<{ body: unknown; memo: unknown }> = [];
161+
for (let i = 1; i <= count; i++) {
162+
const row: any = await ctx.engine.findOne('note_copy', { where: { id: `r${i}` } });
163+
out.push({ body: row?.body, memo: row?.memo });
164+
}
165+
return out;
166+
}
167+
168+
/** The file carried the marks: the round trip below is not a pass-through. */
169+
function expectMarkedFile(csv: string): void {
170+
expect(csv.split('\r\n')[0]).toBe("ID,Body,'-Memo");
171+
for (const v of MARKED) expect(csv).toContain(`,'${v},'${v}\r\n`);
172+
for (const v of ['+86 138 0013 8000', '-120.50', 'Acme Ltd']) expect(csv).toContain(`,${v},${v}\r\n`);
173+
}
174+
175+
describe('CSV export → import round trip, every value returns exactly (#25078)', () => {
176+
let ctx: Booted;
177+
beforeEach(async () => {
178+
ctx = await boot(VALUES);
179+
});
180+
181+
it('sync door, inline mapping', async () => {
182+
const csv = await exportCsv(ctx);
183+
expectMarkedFile(csv);
184+
const res = await importSync(ctx, { format: 'csv', csv, mapping: INLINE_MAPPING });
185+
expect(res).toMatchObject({ total: VALUES.length, ok: VALUES.length, errors: 0, created: VALUES.length });
186+
expect(await storedCopies(ctx, VALUES.length)).toEqual(VALUES.map((v) => ({ body: v, memo: v })));
187+
});
188+
189+
it('sync door, named mapping (reversal runs before the mapping renames the columns)', async () => {
190+
const csv = await exportCsv(ctx);
191+
expectMarkedFile(csv);
192+
const res = await importSync(ctx, { format: 'csv', csv, mappingName: 'note_copy_feed' });
193+
expect(res).toMatchObject({ total: VALUES.length, ok: VALUES.length, errors: 0, created: VALUES.length });
194+
expect(await storedCopies(ctx, VALUES.length)).toEqual(VALUES.map((v) => ({ body: v, memo: v })));
195+
});
196+
197+
it('async import-job door', async () => {
198+
const csv = await exportCsv(ctx);
199+
expectMarkedFile(csv);
200+
const done = await importAsync(ctx, { format: 'csv', csv, mapping: INLINE_MAPPING });
201+
expect(done).toMatchObject({ status: 'succeeded', total: VALUES.length, created: VALUES.length, errors: 0 });
202+
expect(await storedCopies(ctx, VALUES.length)).toEqual(VALUES.map((v) => ({ body: v, memo: v })));
203+
});
204+
});
205+
206+
describe('CSV round trip of a value that begins with TAB or CR (#25078)', () => {
207+
const LEADING_WHITESPACE = ['\tTotal', '\rTotal'];
208+
let ctx: Booted;
209+
beforeEach(async () => {
210+
ctx = await boot(LEADING_WHITESPACE);
211+
});
212+
213+
it('the file carries the mark, and `trimWhitespace: false` returns the stored value exactly', async () => {
214+
const csv = await exportCsv(ctx);
215+
expect(csv).toContain("r1,'\tTotal,'\tTotal\r\n");
216+
expect(csv).toContain(`r2,"'\rTotal","'\rTotal"\r\n`);
217+
const res = await importSync(ctx, { format: 'csv', csv, mapping: INLINE_MAPPING, trimWhitespace: false });
218+
expect(res).toMatchObject({ ok: 2, errors: 0 });
219+
expect(await storedCopies(ctx, 2)).toEqual(LEADING_WHITESPACE.map((v) => ({ body: v, memo: v })));
220+
});
221+
222+
it("under the default `trimWhitespace`, core's text trim still removes the leading TAB / CR after the reversal", async () => {
223+
const csv = await exportCsv(ctx);
224+
const res = await importSync(ctx, { format: 'csv', csv, mapping: INLINE_MAPPING });
225+
expect(res).toMatchObject({ ok: 2, errors: 0 });
226+
expect(await storedCopies(ctx, 2)).toEqual([{ body: 'Total', memo: 'Total' }, { body: 'Total', memo: 'Total' }]);
227+
});
228+
});

0 commit comments

Comments
 (0)