Add an image-to-3d optional workflow for the Tripo API - #13
Open
uayten wants to merge 11 commits into
Open
Conversation
Workflow skill for dressing 3D character base renders with AI-generated clothing, aimed at a Blender -> image -> image-to-3D -> Blender pipeline. The core finding it encodes: describing anatomy in the prompt makes the model redraw the body from text instead of preserving the reference. On one character that collapsed the arm span from 88.9% to 74.3% of the frame width. Rebuilt with an IMAGE 1 / IMAGE 2 / TASK / PRESERVE EXACTLY architecture and no body description, the same render came back at 89.2%. Contents: - SKILL.md - workflow, parameters, verification thresholds, known residuals - references/prompt-template.md - the block architecture, with a working example and the counter-example that failed - references/resolutions-and-proportions.md - aspect ratios, per-edge margins, Blender export settings, mandatory pre-flight check - scripts/measure.py - bounding box occupancy and margins, with a pass/regenerate verdict on horizontal drift Also adds desktop.ini to .gitignore and replaces a personal-notes entry with generic fork-local patterns; the notes file it named moved into this skill. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Extends the pipeline the character-reference-sheets skill stops at. That skill ends with an isolated garment image and the instruction to send it to an image-to-3D generator; this one takes it from there and returns a GLB. scripts/tripo.py uses the standard library only, so there is nothing to install. It uploads the image, runs Tripo's two-step multiview pipeline, polls, and downloads immediately -- result URLs expire five minutes after a task succeeds, which is the main reason this is a script rather than something done by hand. Without --yes it prints the plan, the credit cost and the account balance and stops, so the cost rule is enforced by the tool instead of relying on whoever runs it to remember. Two findings from the docs worth recording, both counter-intuitive: - You supply one image, not four. Tripo generates the four views itself and reconstructs from those. Generating views separately in an image model produces views that disagree with each other, which reconstructs worse than a single clean view. - The best input is a front view on a clean background; Tripo states that side views reduce quality. That is the opposite of the three-quarter view one would reach for to convey depth. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
# Conflicts: # skills/character-reference-sheets/SKILL.md # skills/character-reference-sheets/references/resolutions-and-proportions.md # skills/character-reference-sheets/scripts/measure.py
Two corrections from running the pipeline for real on a pair of cargo
trousers.
The skill claimed you supply one image and Tripo makes the other three.
That is true of image-to-multiview, not of multiview-to-model, which
accepts images directly as [{front}, {left}, {back}, {right}] -- front
required, at least two in total. Letting Tripo invent the views wraps
front detail around onto the sides: the trousers came back with pockets
and buckles on the outer leg where none exist. Supplying real views fixes
that and costs 10 credits less, since the invention step is skipped.
Consistency between supplied views is the thing to watch. Generating each
one with the front image as a reference, asking for the same object at
another angle with the same framing, held the heights within 1.3 points
of each other across front, back and side. For a symmetric object the
left view mirrors into the right at no cost.
Texture is now opt-in via --texture rather than opt-out. Reconstruction
textures are baked from the input views and get replaced in the DCC
anyway, and the mesh is identical either way, so the default should not
be the one that costs more.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three lessons from taking a sneaker and a shirt through the pipeline, plus the script the first one needs. normalize_views.py rescales a set of views so the subject is the same height in each. Views drift in scale even when the prompt asks for identical framing, because each is framed by whichever axis constrains it: a side profile squeezed to fit a long object across the frame comes back with a shorter subject than the front view of the same object. Height is the dimension every view of a standing object shares, so it is the one to match on. On the sneaker this closed a 15-point spread and, as a side effect, corrected a front view that was too wide -- both errors came from the same scale drift, so both fell to the same fix. The other two lessons are prompt rules, now in SKILL.md: Anchoring a view on a reference that does not show what the view needs does not constrain it. The sneaker's inner side, generated from the front view, came back with different colour-blocking from its outer side, because the front shows no side panel and left the model free to invent one. Re-anchored on the outer profile it matched to within a point. Omitting a view beats fabricating one. Mirroring left into right is free and correct for an object symmetric about that axis, and wrong otherwise: a sneaker carries its emblem on one face, and a shirt's chest pocket proved visible in profile, so mirroring would have asserted a pocket on the side that lacks one. Dropping the view is allowed -- two is the minimum -- and missing information reconstructs better than contradictory information. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The script offered seven of the parameters the generation endpoints accept. It now offers all of them, grouped in --help by texture, mesh and other, with the combinations Tripo refuses rejected locally rather than after a submitted job is paid for: face_limit outside the range smart_low_poly allows, generate_parts alongside textures or quads, auto_size without a texture. Two things the API taught us along the way. Querying a finished task shows the views stored as a positional list of exactly four slots, so a request built by compacting away the missing views risks having a back view read as a left view; four slots are now always sent. And glTF has no quads, so a --quad job answers with FBX -- naming that file .glb produced something no importer opens, with nothing revealing the mistake until the import failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
character-reference-sheets was moved out of skills/ and into optional-workflows/ upstream. image-to-3d is the same kind of thing -- an opt-in workflow with its own scripts and dependencies rather than a skill that loads for everyone -- so it follows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The repo asks a workflow to say when a parameter's behaviour is an assumption
rather than something tested against the live API. Five parameters were
confirmed by reading GET /v3/tasks/{id} back after a paid job and matching
input and credits_consumed against the request; eleven were only accepted by a
dry run, which proves the flag parses and nothing more. The split is now
written down, along with the pricing that comes from the published table rather
than from an observed charge.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The fork carries scratch patterns of its own, one block of which says in its own comment that it must never reach an upstream PR. They belong to the fork, not to a workflow contribution, so this branch restores .gitignore to what upstream has and leaves the PR touching only optional-workflows/. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
view_slots padded the inputs array to four entries, using {} for a view
that was not supplied. The API rejects that:
1004 inputs[3] must include one of front|left|back|right with a URL,
file_token, or object value
Any run that did not supply all four views failed, and that is the normal
case -- the API documents two views as the minimum, and a garment usually
needs three.
The padding came from conflating the two input formats. The legacy
positional format takes exactly four bare strings with "" for a skipped
view, so position carries the meaning and padding is required. The
view-key format this script uses names each view explicitly, the server
canonicalises the order, and every entry must carry exactly one key. A
missing view is therefore an omitted entry, not an empty one, and the two
formats cannot be mixed.
Dropping the padding also removes the concern the old docstring raised:
with named keys, a run supplying only --back cannot have that view land
in the left position, because the key says which view it is.
Verified against the live API: front, back and left now reconstruct,
where the same call previously failed instantly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this workflow is for
image-to-3dturns a set of generated views of one object into a mesh throughthe Tripo API, and downloads the result. It is the other half of the pipeline
character-reference-sheetsalready covers: that workflow produces the isolatedgarment or prop image, this one reconstructs it.
It sits in
optional-workflows/for the reasons the README gives — it is narrow(you need a Tripo account, an API key and credits), it has script dependencies,
and it would be dead weight in an installation that never touches 3D.
What it does
scripts/tripo.pyhandles upload, submission, polling and download overmultiview-to-model,image-to-modelandimage-to-multiview, and exposes theparameters those endpoints accept — texture, mesh and seed options, including
smart_low_polyandquad, the Smart Mesh of the web UI.scripts/normalize_views.pybrings a set of views to a common subject heightbefore upload, which is the failure the workflow spends most of its words on:
generated views drift in scale even when the prompt asks for identical framing,
because each view is framed by whichever axis constrains it.
Cost discipline
Nothing is submitted without
--yes. A run without it prints the plan, theadd-ons, the total and the account balance, then stops. The combinations Tripo
refuses are rejected locally rather than after a job is paid for: a
face_limitoutside the range
smart_low_polyallows,generate_partsalongside texturesor quads,
auto_sizewithout a texture.What I ran it against
An armoured game character, from four generated views normalised to a common
height, on a Windows machine with Python 3 and Blender for inspection.
multiview-to-modelfrom four supplied views: 30 credits, GLBwith 1,419,756 triangles.
--smart-low-poly --quad --face-limit 10000: 45credits, and a mesh of 9859 quads plus 2386 triangles, one UV map, one
material.
Both were checked by reading
GET /v3/tasks/{id}back afterwards and comparingthe stored
inputandcredits_consumedagainst what was requested.Two things worth knowing, both documented in the SKILL.md
--quadreturns FBX, not glTF. glTF has no quads, so the model URL of aquad job carries a
.fbxfile —Kaydara FBX Binaryin the first bytes.Naming it
.glbproduces something no importer opens, and nothing reveals themistake until the import fails, so the script takes the extension from the URL.
The view list is positional. Reading a finished task back shows the views
stored as
files, exactly four entries in the order[front, left, back, right], with the names discarded. The documented way to omit a view is to leaveits slot without a file token rather than to shorten the list, so the script
always sends four slots — otherwise a request supplying only a front and a back
view risks having the back view read as a left view.
On verified versus assumed
Following the contributing note about saying what was actually verified, the
SKILL.md carries a section that splits the two: five parameters confirmed
against the live API by reading the task back, and eleven that were only
accepted by a dry run, which proves the flag parses and the arithmetic adds up
and nothing more. Their pricing comes from the published table rather than from
an observed charge, and the single-image paths were not run.
Notes
optional-workflows/, so per the README no version bump and nonpm testflow applies.scripts/tripo.pyis standard library only.scripts/normalize_views.pyneeds Pillow and NumPy, declared in
requirements.txt.🤖 Generated with Claude Code