Repository navigation
Conversation
Adds get-actor-version, the read side of the source tools that replace push-actor and pull-actor, in a new source category that is not enabled by default. It returns a version's metadata, a file manifest (path, decoded size, hash, format), a revision for the whole file set, and the content the caller asks for within MAX_INLINE_BYTES. Without paths it returns every text file only when all of them fit, so one large file cannot crowd out the rest; paths: [] gives the listing only; named paths fill the limit in order, and a single large text file can be read in line ranges. Environment variables come back as names and isSecret only. The hash is the first 16 hex characters of the SHA-256 of the file's bytes, and the revision hashes the sorted path and hash lines, so both are the same for a version stored inline and one stored as a zip. Zip-stored versions are read only from a key-value store record on the session's own API host, with the caller's token. The zip is parsed from its central directory with no new dependency, because apify push writes zeros for sizes in local headers. Archives over 50 MiB, over 10,000 entries, or over 64 MiB uncompressed are refused, as are encrypted, zip64, symlink, and unsafe or duplicate names, and every entry's length and CRC-32 are checked after inflating. Git, gist, and outside zip sources return only their URL, without its query string.
Zip reading: - Refuse an archive whose end record declares fewer entries than the central directory holds, and an entry whose local header names another file or method, so the files reported match what other zip readers see. - Check entry names for length, NUL, absolute paths, and '..' first, so no refusal message repeats an overlong name. - Skip a root folder entry such as './' instead of refusing the archive. - Detect text with isUtf8 and decode only the files that are returned, so a listing holds only the inflated bytes the 64 MiB cap bounds. Inline files: - Read a file stored without format as TEXT and one without content as empty, as the build worker does, instead of failing or breaking the output schema. - Normalize inline paths the way the zip reader does, so the same files give the same manifest and revision however they are stored. - Return a UTF-8 file stored as BASE64 as text, as a zip read does. The manifest keeps the stored format. Results: - A line over the 256 KiB limit no longer fails the call. The file goes to omittedPaths and the summary names the startLine that skips it, or says a single-line file cannot be read by lines. - The summary no longer tells the caller to name a base64 file too large to return; it points to the Apify CLI instead. - Remove the query string from Git and gist URLs too, and http user and password from every URL, before returning them or hashing the revision. - Word the outside zip summary without claiming the source is not on Apify. - The description says the API hides the source of most Actors the caller cannot modify, and gives the macOS hash command. Tests cover each rule and the boundaries that no test pinned before: hidden sources for URL types, a record URL on another host, local extra fields that differ from central ones, byte order of the manifest, base64 files next to text at the limit, and the entry count, size, CRC, and encryption flag edges.
Add two write tools to the opt-in source category. Both write only to the caller's own account and store files inline in the version. create-actor creates a private Actor with one version in a single POST, from files (which must include .actor/actor.json) or a Git repository. It refuses a name the account already uses, and warns about a missing Dockerfile and empty files. update-actor-version applies write, edit, delete, and move operations to a fresh read of the version and saves the result with one version PUT that carries only the source keys and buildTag, never envVars. Each operation has a precondition (an absent path, expectedHash, or an exact-once match), and expectedRevision guards replaceFiles and gitRepoUrl. A failure writes nothing and names the operation, the path, and a reason code. It refuses zip-stored versions and results over the platform's 3 MiB limit, and refuses file operations on Git and gist versions. A save that lands between the read and the PUT is not detected until the platform gets a conditional version PUT. Both take autoBuild, which starts a build after the write and returns without waiting. get-actor-version's description now says its hashes and revision are what update-actor-version takes, when that tool is loaded.
- Check for a cancel right before the version PUT and the Actor POST, so a request cancelled during the reads writes nothing. - create-actor builds its files, hashes, and revision from the files the platform stored, since it sets the name field of .actor/actor.json on create, and warns when that changed the file. A taken name (including the race case, name-not-unique) now gives the full name update-actor-version takes. The missing Dockerfile warning says the build uses the platform's default Node.js Dockerfile, and matches paths regardless of case. - update-actor-version refuses a missing .actor/actor.json only when the changes remove it, or when a Git version switches to stored files. - Sending back the Git URL that get-actor-version shows no longer removes the stored credentials. Another URL on the same host keeps them, with a warning, and a URL on another host is refused. - Stored entries the call leaves alone go back verbatim and in place, duplicates included; a changed file keeps its stored name. - Refuse a result where a file path is also a folder of other files or a folder entry. - Cap the NO_MATCH context at 4 KiB, show a window of a long line, show the lines around newText when only it is found, and suggest reading again when no context is shown. - Share splitLines, formatMib, the MiB and 255-character constants, and the 500-file cap across the source tools, and give the 2 MiB refusal text that fits each tool. - Output schema texts no longer name another tool, and describe sourceType and changes as the tool returns them. - Add tests for the new rules and for the cases the reviewers found untested.
Move the files input rules, the manifest builder, and the missing Dockerfile warning from create-actor to source_helpers.ts, so create-actor-version can use the same rules. create-actor behaves as before. (cherry picked from commit ea8fbb4)
The Apify API cannot read or change single files of an Actor's source, so the version GET returns every file and a zip-stored version is downloaded and unpacked whole, even for one file. TODO comments mark both places for when such an API exists.
…t/update-actor-version
The Apify API cannot change single files of an Actor's source, so a one-line edit reads the whole version and writes all its files back, which also leaves the gap between the read and the PUT. A TODO marks the place for when an API for atomic source changes exists.
…y it Remove the zip reader, its test helper, and its tests. A version stored as a zip in a key-value store of this API is now refused as a user error; a zip at an outside URL is still reported by its URL. A TODO marks where zip reading comes back and the library planned for it. Other simplifications, with the same output data: - One summary function replaces the format helpers. - A line over the limit is reported only when nothing of the range fits. - No Apify CLI hint for files too large to return. - Paths sort in plain JavaScript string order instead of UTF-8 byte order, for the manifest and the revision lines. - One hash helper, and the manifest builder and folder check live in source_files.ts.
# Conflicts: # src/tools/AGENTS.md # src/tools/source/get_actor_version.ts # src/tools/source/source_archive.ts # src/tools/source/source_files.ts
After the merge of get-actor-version's simplification: - create-actor and update-actor-version use buildFilesManifest and isFolderEntry from source_files.ts, and source_helpers.ts drops its copies. - The absolute path check (ABSOLUTE_NAME_REGEX) now lives in source_files.ts, since the zip reader it came from is gone. - update-actor-version refuses a zip-stored version with the same wording as get-actor-version and no longer points to apify push.
- Say that the tool does not download zips when it reports an outside zip URL, since a URL on the session's own API host is not outside the Apify API. - Describe the path normalization correctly: empty and `.` segments are dropped as the build worker does, and backslashes become `/`, which the build worker does not do for inline names. - Pass only the API base URL to the version reader, since it no longer calls the API, and fix comments that still mention the source store. - Stop exporting the hash helper and the format type, and read an entry's name, format, and content in one destructuring. - Test that a BASE64 file with a text extension and invalid UTF-8 comes back as base64, that a byte order mark is kept, and that the last of two entries with the same path wins.
# Conflicts: # src/tools/source/source_files.ts
Move the rule that turns a stored entry into bytes into source_files.ts as decodeSourceFileEntry, and use it for the hash that get-actor-version reports, for the unchanged-file check in update-actor-version, and for the config comparison in create-actor. The three can no longer disagree about how a missing format or content is read. A null content now reads as empty in all three, as it already did for the hash.
Narrow the tool to what the spec asks for: - Without paths, return the file list only. Drop pathPrefix and the read-all-text-files default. - Keep basic line ranges; a range over the limit goes to omittedPaths. Drop the automatic range and the long-line handling. - Refuse every version not stored as files with one message that names its URL without credentials. Drop Git, gist, and zip reporting, the URL revision, and the split of Git URLs. - Read the version from the Actor GET, which already hides the source the same way. Drop the second GET, the 4xx catch, and the no-versions branch. - Drop env var names, the stored format, the source type, and the build tag from the output, and keep the text result to one sentence. - Normalize stored names with posix.normalize, as the build worker does.
Hide the password of a URL in any scheme, not only http and https, so an ssh:// Git URL with a password no longer shows it in the refusal. The SSH user stays. Test that a BASE64 file with a binary extension comes back as base64 even when its bytes are valid UTF-8.
# Conflicts: # README.md # src/tools/AGENTS.md # src/tools/source/get_actor_version.ts # src/tools/source/source_files.ts
…rsion simplification The simplified get-actor-version dropped helpers that create-actor and update-actor-version still import. Define them in source_helpers.ts until the write tools are simplified too.
Narrow both tools to the updated spec and leave to the platform the checks it already makes. - update-actor-version takes write, edit, and delete operations, reads the version from the one Actor GET, and saves it with one PUT of sourceType and sourceFiles, skipped when the revision does not change. Move, replaceFiles, gitRepoUrl, buildTag, allOccurrences, edit excerpts, match context, caps, and the size and path checks are gone. The edit logic moves in from source_edits.ts, which is deleted. - create-actor takes files only and sends versionNumber and buildTag only when given. The name rules, token check, defaults, caps, path and base64 checks, actor.json rules, Git source, and extra warnings are gone. - The Actor lookup no longer compares the owner with users/me, and API errors reach the agent unchanged. - The Actor lookup, version choice, and refusal of versions not stored as files move to source_helpers.ts, shared with get-actor-version. The build start after a write and its next step are written once. - Results are shorter: update returns revision, changed, and changes with each file's new hash; both return build or buildError and the empty-files warning.
… simplification create-actor sends version number 0.0 again when the caller gives none. The platform refuses a version without a number, so the default call failed and created nothing. The build tag is still sent only when given, since the platform sets latest on the only version. The create-actor test mock no longer fills in a version number the platform does not add, and the update-actor-version test comment now describes the PUT file order the code produces.
# Conflicts: # tests/unit/tools.mode_contract.test.ts
# Conflicts: # tests/unit/tools.mode_contract.test.ts
Any other error from starting the build is a bug, so it goes to the tool-call engine as a server error instead of reading as a failed start.
# Conflicts: # README.md # src/tools/registry.ts
# Conflicts: # README.md # src/tools/registry.ts
startBuild returns ABORT only for a passed signal or a wait, and respondAfterWrite passes neither, so its ABORT branch could not run.
respondAfterWrite checks the signal before it starts a build, and the SDK drops the result of a cancelled request.
It bounds the worst-case search times the cap; the 21-edit test pins the cap itself. At 3 s it had too little room on CI, which runs this file about 4.5x slower.
apify push falls back to 0.0 for an empty version, so create-actor does too.
…not given AJV's coerceTypes turns null into an empty string before the tool runs. No file, version, or revision has an empty hash or number, so an empty string now counts as not given in update-actor-version, get-actor-version, and create-actor, which falls back to the actor.json version, then 0.0.
UTF-8 stores a lone surrogate as U+FFFD, so the stored name, the returned change path and the revision no longer matched what was sent. validatePathCharacters now refuses such a path in create-actor and update-actor-version, before any request.
fetchActor now goes through getResourceByIdOrName: a name that is not ID-shaped goes out once as ~name, and an ID-shaped value is tried as an ID first, then as ~name. The actor descriptions, the not-found text, and the create-actor retried-POST text now say a name alone works for your own Actors.
#1565 removed 'Verify the tool name and input parameters.' from tool errors with an HTTP status, so the create-actor case for a taken name failed on every transport.
|
from codex: Non-blocking: A no-op returns “runs use these files only after the version is built,” although the unchanged files may already be in the build used by runs. Could the With |
|
Why it's unable to merger |
|
from codex: #1452 lists evals for creating an Actor, fixing one line, and changing several files, each ending in a successful build and run. The eval jobs on both current PR heads are skipped. Is there evidence covering those flows elsewhere? |
jirispilka
left a comment
There was a problem hiding this comment.
I did a quick pass only. I'm sorry, I would need a bit more time.
I left one comment so far.
Codex left a couple of comments as well.
| schedules: [createSchedule, getSchedule, updateSchedule, deleteSchedule], | ||
| builds: [getActorBuild, getActorBuildLog, getActorBuildList, buildActor], | ||
| source: [getActorVersion], | ||
| source: [getActorVersion, createActor, updateActorVersion], |
There was a problem hiding this comment.
source is a poor category name. It's a term we made up: neither the Apify API nor the Apify CLI uses it. The API calls this "Actor versions", and the CLI has no matching group (Actor code goes through actors push and actors pull).
I'd rename it to versions. That's not a great name either, because createActor isn't strictly a versions endpoint. But it matches the API vocabulary, and it stays consistent with runs, builds and tasks.
There was a problem hiding this comment.
Agree. versions sounds better.
The Apify API and the Apify CLI call these Actor versions, which also matches the runs, builds, and tasks categories. tools=source is now a retired selector, like get-actor-log: it loads no tools and is never fetched as an Actor.
The tools folders match the category names, like builds, runs, and tasks. Only import paths and the folder name in src/tools/AGENTS.md change.
With autoBuild and nothing to write, the call now starts no build and says "Nothing changed, so no build was started." Without autoBuild, it says only "No build was started.", since the stored files may already be in the build runs use. create-actor always writes and keeps its text.
build-actor now looks the Actor up with getResourceByIdOrName, like the versions tools: a name that is not ID-shaped goes out once as ~name, and an ID-shaped value is tried as an ID first, then as ~name. The actor description and the not-found text say a name alone works for your own Actors. The live bare-name case also builds by the bare name.
The actorId filter of get-actor-build-list and get-actor-run-list now goes through getResourceByIdOrName. The list endpoints take ~name, so a name costs no extra request; only an ID-shaped value that matches no ID is listed a second time, as ~name. The actorId descriptions and the not-found text say a name alone works for your own Actors.
Agent evals for the version toolsFive cases cover the three flows from #1452. The agent gets only the MCP tools. A trial passes when the judge passes it and no tool call fails. Platform check: the final build started after the last write and succeeded, the final run used it and returned the expected item, and only the intended files changed. Server: a local merge of #1454 at
Previous round (same cases, without the bare-name changes): judge 15/15, no tool errors 7/15, 8 tool errors: 7 Actor names without a username in Remaining error: Opus 4.8, Not an error: in Sonnet 5.5's Cleanup: all Actors from the runs were deleted, and none are left. The cases are in the staging dataset 🤖 Generated with Claude Code |
What
Two write tools in the
versionscategory, which this PR renames fromsource:create-actorcreates a private Actor with its first version and files. It never touches an existing Actor.update-actor-versionwrites, edits, or deletes files of a version in one call. Files the call does not mention stay as they are.Why
They replace the drafted
push-actor(#1333), following the Actor source tools spec (#1452). Closes #1463. An edit sends only the changed text, not whole files.How
create-actor:name,title,description,files,versionNumber,buildTag, andautoBuild. One POST creates the Actor and its version. It returns the files as the platform stored them, since the platform sets the name in.actor/actor.json.versionNumber, the version is theversionin.actor/actor.jsonwhen that is a non-empty string, and0.0otherwise, the orderapify pushuses. Aversionthat is notMAJOR.MINOR, such as1.0.0, goes to the API as is, and the API refuses it.update-actor-version:actor,versionNumber,operations,expectedRevision, andautoBuild.writecreates a file, or replaces one given itsexpectedHash.editreplaces text; eacholdTextmust match exactly once, overlapping matches included, and anexpectedHash, when given, must match the file. In a file with CRLF line breaks, an LF edit is applied as CRLF, so the file keeps one line ending.deleteremoves a file given itsexpectedHash. A rename is adeleteand awrite.actorargument (update-actor-version, andget-actor-versionon master) takes an ID, a full name (username/nameorusername~name), or a bare name, which resolves to the Actor of that name in your account.fetchActorusesgetResourceByIdOrName, which tasks and schedules already use: a bare name that cannot be an ID goes out once as~name, and an ID-shaped value is tried as an ID first, then as~name. That helper and its two siblings move fromtask_helpers.tstosrc/utils/apify_resource_id.ts, since they resolve Actors, tasks, and schedules alike. The move is its own commit with no behavior change.build-actor,get-actor-build-list, andget-actor-run-listresolve a bare name the same way, so a name one call accepts works in the next (Resolve bare Actor names in build-actor, call-actor, and the other Actor tools #1614).FILE_EXISTS,FILE_NOT_FOUND,HASH_MISMATCH,NO_MATCH,MULTIPLE_MATCHES,NOT_TEXT, orREVISION_MISMATCH.sourceTypeandsourceFiles, never env vars or the build tag. A call that changes nothing sends no PUT and starts no build, even withautoBuild; it says "No build was started."update-actor-versiontakes at most 100 operations and at most 20 edits in total across them; a larger call is refused before the version is read. Each edit searches the whole file text, which can be close to 3 MiB, and blocks the event loop while it runs: up to about 20 ms for anoldTextover 250 characters, so 20 edits take up to about 0.4 s. Writes and deletes never search a stored file. AnoldTextover 250 characters is searched with Knuth-Morris-Pratt, since V8'sindexOfkeeps its Boyer-Moore tables for only the last 250 characters of a pattern and slows down to the file length times the pattern length past them.create-actorchecks its paths with a set and a map instead of comparing each file with every earlier one, which took 8 s for 20,000 files.[path, hash]pairs, sorted by path. JSON quotes and escapes each path, so two different file sets never give the same revision.get-actor-versionon master uses the same function, so its revision values change once with this PR.encodingdefaults tobase64for binary extensions such as.png.warningsnames empty files, which the build skips.autoBuild(default false) starts a build without waiting for it; a build that cannot start is reported inbuildError, and the write stands. Both tools haveopenWorldHint: true, sinceautoBuildstarts a build, asbuild-actordoes.nullinputs: the input check (AJV withcoerceTypes) turns anullinto an empty string before the tool runs. An emptyexpectedHash,expectedRevision, orversionNumbercounts as not given, since no stored hash, revision, or version number is empty. An emptycontentstays an empty file: the tool cannot tell it from anull, andwarningsnames empty files.redactArgs. The logged copy is an allowlist: it keeps the Actor, version, paths, operation types, encodings, and hashes, and replaces file content and theoldTextandnewTextof edits with[REDACTED]. File content can hold secrets, such as an API key written into the code./or..) or at a folder path, a new file whose path collides with a file or folder, the same path twice increate-actor, base64 that does not decode without loss, and text with a lone UTF-16 surrogate. The platform accepts all of these, and the build then fails or stores a corrupt file. UTF-8 stores a lone surrogate as U+FFFD, so the stored name would no longer match the revision.TOOL_GUIDELINES.md: descriptions, refusals, failed checks, andnextStepstate facts and never ask for another call, for example "No build was started; runs use these files only after the version is built."create-actorPOST, and the platform then refuses the name as taken. The tool says an earlier attempt of the same call may have created the Actor, and that if it holds the files sent, this call created it.apify-api-write(feat: Add apify-api-write to call any Apify API write method #1445, on master) covers env vars, build tags, Git sources, and replacing every file at once.get_actor_version.tstosource_helpers.ts, so all source tools share them.Testing
Unit tests cover each operation and reason code, all or nothing, the PUT body, CRLF edits, encodings, warnings,
expectedRevision, refusals, API errors, cancellation,autoBuild, and schema conformance. Round-trip tests check that therevisionthese tools return is the oneget-actor-versionthen reads.Regression tests for the reviews:
oldTextover 250 characters on a file of almost 3 MiB finish within 6 s. They take about 0.5 s locally, and CI runs this file about 4.5 times slower.oldTextagainst a file of almost 3 MiB returnsNO_MATCHwithin a second (it took over 4 s), andoldTextover 250 characters still finds overlapping matches and single matches after partial ones.create-actorchecks 20,000 files within a second (it took about 8 s), and file and folder collisions two levels deep keep their messages.oldText,newText, and unknown keys, and the arguments the tool receives are unchanged.nullexpectedHash,expectedRevision, orversionNumberpasses the input check and counts as not given, inupdate-actor-version,get-actor-version, andcreate-actor. A write over an existing file with anullexpectedHashstill getsFILE_EXISTS.~name; a 17-character value is tried as an ID, then as~name;username/name,username~name, and~nameare looked up once.create-actortakes the version from.actor/actor.json, sends0.0for no file, no version, an empty or non-string version, or a file that is not JSON, and sendsversionNumberwhen given.update-actor-versiondescription lists the reason codes and asks for no retry.type-check,lint,format,test:unit, andcheck:agentspass.Five live integration cases in
tests/test_kit/cases/versions.cases.tsrun in CI against the platform:create-actor, thenget-actor-versionreading back the same hashes and revision, andbuild-actorstarting a build, both given the bare nameupdate-actor-versioncall that creates, replaces, edits, and deletes files and leaves the other files, env vars, and build tag as they wereEach case uses its own
test-source-*Actor and deletes it at the end. Only thecreate-actorcase starts a build, and it aborts it right away.Notes
get-actor-version(feat: Add get-actor-version tool #1450) is on master.create-actor-versionanddelete-actor-version(feat: Add create-actor-version and delete-actor-version tools #1454) stack on this PR.get-actor-version, which is on master: itsactortext and not-found text change, a bare name now resolves, and anullversionNumbercounts as not given. The second review round also changed the served text andopenWorldHintof the two new tools.build-actor,get-actor-build-list, andget-actor-run-list, also on master: a bare Actor name now resolves to one of your own Actors, and theiractororactorIdtext and not-found text say so.sourcecategory is renamed toversions.sourceshipped in v0.17.2 with onlyget-actor-version.tools=sourceis now a retired selector, likeget-actor-log: it loads no tools, and the README says to usetools=versions. The tools moved fromsrc/tools/source/tosrc/tools/versions/as pure renames.apify-mcp-server-internalkeysCATEGORY_METAonsource(src/web/src/constants.ts), so the package bump that brings this rename needs that key renamed toversionsthere.call-actorandfetch-actor-detailsresolve a bare name too. In the evals, agents passed a bare name tocall-actor8 times once the versions tools andbuild-actoraccepted it; with this PR alone, those calls fail.build-actorcall sent in the same turn as anupdate-actor-versionwrite can start before the write lands and build the old files.autoBuildavoids this. In the evals it happened once, and the platform started that build after the write.delete-actorstill refuses a bare name, and theSpecify versionNumber; this Actor has versions: ...text, whichbuild-actorshares, still asks for a value. Both are better changed on master for all tools at once.update-actor-versionreads and writes a whole version, since the Apify API has no per-file operations.AI disclosure: implemented with Claude Code; awaiting human review.
🤖 Generated with Claude Code