From f08ae140064986e6825c0375f5ea88a004301a58 Mon Sep 17 00:00:00 2001 From: Danyal Sarfraz Date: Mon, 26 Oct 2026 04:26:00 +0900 Subject: [PATCH 1/3] Expand render pipeline, outputs, lighting, import and pre-flight (1.9.0) Render pipeline - Render All runs modally: status bar progress and time left, Esc stops after the image in flight, scene state always restored - Skip Existing resumes a crashed or cancelled batch - Per-product render ticks and Render Active; labels stay stable when products are unticked - Per-view checkboxes (Custom view set) and per-custom-view toggles - Reframe Before Rendering rebuilds cameras at render time Outputs - Naming templates for files and folders ({label} {prefix} {number} {product} {view} {mode}), with clash detection - Contact sheet per product with captions - Turntables: frames plus H.264 MP4 and looping GIF - Post-processed copies: trim, pad to square, flatten, PNG/JPEG/WebP, max size and extra sizes; originals untouched - CSV/JSON manifest, merged across partial runs Look and lighting - Lights follow the camera so every view is lit like Front - Ground: shadow catcher or floor, placed under each product - HDRI background, optionally hidden from camera - Line art, mask and AO passes - Material presets for the batch or per product, applied through a temporary Geometry Nodes modifier - Save and delete your own lighting presets Import and geometry - STEP parts import as separate named objects with their CAD colours, read from the STEP file since console FreeCAD drops them - IGES via FreeCAD; OBJ, PLY, glTF/GLB, FBX via Blender; 3MF via a small built-in reader - Add a whole folder, optionally recursive - Real size and units recorded at import; Keep Proportions scale mode; Rest on Floor - Auto Orient (lay flat or principal axes) and Reset Orientation Pre-flight - Checks the batch before rendering, with one-click fixes; errors stop Render All, warnings ask first; empty products are skipped Fixes - FreeCAD import meshed assembly containers and origin planes, duplicating every part of a multi-part STEP - Stray-object check crashed on stale view layer entries - Stop setting deprecated use_nodes on Blender 5 --- README.md | 410 ++- multiview_product_renderer/__init__.py | 4698 ++++++++++++++++++++++-- 2 files changed, 4686 insertions(+), 422 deletions(-) diff --git a/README.md b/README.md index aae6f45..2d3bb2f 100644 --- a/README.md +++ b/README.md @@ -2,13 +2,13 @@ A Blender add-on for turning CAD products into consistent, batch-rendered image sets. -Import STEP and STL files, standardize them at the world origin, build a rig of orthographic and -perspective cameras plus 3-point studio lighting, then render every product across every -view in one click, with predictable file names like `AA_01_Widget_Front_clay.png`. +Import STEP, IGES and mesh files, standardize them at the world origin, build a rig of +orthographic and perspective cameras plus studio lighting, then render every product across +every view in one click, with predictable file names like `AA_01_Widget_Front_clay.png`. - **Blender:** 4.0 and newer - **Panel location:** `3D Viewport → Sidebar (N key) → MultiView` -- **Version:** 1.4.0 +- **Version:** 1.9.0 --- @@ -43,38 +43,57 @@ Restart Blender and enable the add-on as in step 4 above. --- -## STEP import setup (optional) +## STEP / IGES import setup (optional) -You only need this if you want to import `.step` / `.stp` files directly. If you already -have meshes in your scene, skip it. The add-on works with any geometry. +You only need this if you want to import `.step` / `.stp` or `.iges` / `.igs` files +directly. If you already have meshes in your scene, skip it. The add-on works with any +geometry. -**`.stl` files need none of this.** An STL already holds a mesh, so it is imported straight -through Blender and never touches FreeCAD or STEPper, whatever the STEP Import Mode is set -to. From the moment it lands it is treated exactly like a converted STEP file: same product -collection, same Fit on Import, same standardizing, same batch render and file naming. +**Mesh files need none of this.** STL, OBJ, PLY, glTF/GLB, FBX and 3MF already hold a mesh, +so they are imported straight through Blender and never touch FreeCAD or STEPper, whatever +the STEP Import Mode is set to. From the moment one lands it is treated exactly like a +converted STEP file: same product collection, same Fit on Import, same standardizing, same +batch render and file naming. + +| Format | Imported by | Notes | +| --- | --- | --- | +| STEP (`.step`, `.stp`) | FreeCAD or STEPper | Parts keep their names and colours (FreeCAD route) | +| IGES (`.iges`, `.igs`) | FreeCAD | | +| STL, PLY | Blender | Read as Z-up, which is what CAD writes | +| OBJ, glTF / GLB, FBX | Blender | Up axis converted the way the format defines it. Cameras and lights in the file are dropped | +| 3MF | MultiView's own reader | Meshes, components, build transforms and base-material colours. Blender has no 3MF importer | Open `Edit → Preferences → Add-ons → MultiView Product Renderer` and expand the preferences: | Setting | What it does | | --- | --- | -| **STEP Import Mode** | `Auto` tries the STEPper add-on first, then falls back to FreeCAD. Or force one. STEP only; STL ignores it. | -| **FreeCAD Executable** | Path to `freecadcmd.exe` (Windows) or `freecadcmd` (macOS/Linux). STEP only. | +| **STEP Import Mode** | `Auto` tries the STEPper add-on first, then falls back to FreeCAD. Or force one. STEP only: IGES always uses FreeCAD, and mesh files ignore it. | +| **FreeCAD Executable** | Path to `freecadcmd.exe` (Windows) or `freecadcmd` (macOS/Linux). STEP and IGES. | | **Tessellation Deflection** | FreeCAD mode only. Lower = finer mesh, bigger files. Default `0.1`. | **FreeCAD route**: install [FreeCAD](https://www.freecad.org/) and point the preference at its console binary, typically: ``` -C:\Program Files\FreeCAD 0.21\bin\freecadcmd.exe +C:\Program Files\FreeCAD 1.1\bin\freecadcmd.exe ``` -The add-on shells out to FreeCAD, tessellates each solid into a mesh, writes a temporary -OBJ, and imports that. Nothing is left behind in your FreeCAD install, and the scratch +The add-on shells out to FreeCAD, tessellates each part into a mesh, writes temporary OBJs, +and imports those. Nothing is left behind in your FreeCAD install, and the scratch directory is removed even if you cancel. FreeCAD is run with its standard input closed, so a build that stops to ask a question fails immediately instead of stalling the import. Any single conversion is given 15 minutes before it is stopped. +**Parts and colours.** Each part of a STEP assembly comes in as its own object, named the +way the CAD file names it (`Housing`, `Lid`, `Screw`...), all inside the one product +collection. Colours set in the CAD file become materials (`MV_Color_RRGGBB`), including +faces coloured differently from the rest of their part, and show in Full renders unless a +material preset replaces them. FreeCAD's console mode cannot read STEP colours itself, so +MultiView reads them from the file and passes them along. It understands the usual ways CAD +programs write colour; a file that colours parts some other way imports in grey, never +fails. + **STEPper route**: if you own the STEPper add-on, install and enable it; MultiView detects and uses its import operator automatically. @@ -86,8 +105,9 @@ The sidebar is laid out in the order you should use it. ### 1. Scene Setup -Pick a lighting preset and a background, then **Apply Scene Setup**. This builds an -`MV_Lighting` collection and switches the scene to the add-on's own `MV_World`. +Pick a lighting preset, a background and a ground, then **Apply Scene Setup**. This builds +an `MV_Lighting` collection, an `MV_Stage` collection for the ground, and switches the scene +to the add-on's own `MV_World`. Your existing world is left intact. It is only deselected, not edited, so an HDRI setup you built yourself survives and can be picked again from World properties. @@ -98,17 +118,81 @@ you built yourself survives and can be picked again from World properties. | **Studio Hard** | Smaller lights, crisper shadows and highlights. | | **Dramatic** | Strong key, deep falloff, minimal fill. | | **High Key** | Bright, near-shadowless, white-on-white product look. | +| **No Lights (HDRI only)** | No lamps at all; the HDRI does the lighting. | + +**Intensity** multiplies the preset's light energies. **Lighting Only** and **Background +Only** re-run either half on its own. + +#### Lights Follow Camera + +On by default. The rig is laid out for the Front view, and with this off the lights stay +where they are while the cameras go round, so Back is lit mostly by the rim light and Bottom +barely at all. With it on, the whole rig turns with each camera during a render, so every +view is lit the way Front is: key from the same side, fill from the other, rim behind. The +lights go back to their Front position when the batch finishes. + +The rig hangs off an empty called `MV_LightRig` at the centre of the batch. A file whose +lighting was built before 1.7 has no rig yet: press **Lighting Only** once. + +#### Your own lighting presets + +Move the lights, change their colour or strength, or add your own lights to `MV_Lighting`, +then press **+** next to the preset list and give the setup a name. It is saved in +Blender's user presets folder (`scripts/presets/multiview_lighting/`), so it is available in +every file, and appears in the list after the built-in presets. **−** deletes the selected +saved preset; built-in ones cannot be deleted. + +Presets are saved relative to the product: positions and sizes as multiples of its size, +laid out against the Front view. Loaded onto a product of a different size, or with a +different **Front Axis**, the rig scales and turns to match. + +#### Background + +- **Color**: a flat background in **Background Color**. +- **HDRI**: pick an `.hdr` or `.exr` environment. It lights the product and shows in its + reflections. **Strength** and **Rotation** adjust it. With **Show HDRI** off (the default) + the camera still sees the flat Background Color, which is usually what a product shot + wants; tick it to see the environment itself. + +**Transparent Background** renders alpha instead of either. + +#### Ground + +Something for the product to stand on, in Full renders only. Clay and the extra passes +are about the product's shape, so the ground is left out of them. + +- **Shadow Catcher**: an invisible floor that only shows the product's shadow, on the + background colour or on transparency. Cycles only; EEVEE renders it as a plain floor, + and the panel warns. +- **Floor**: a visible floor in **Floor Color**. Lower **Roughness** makes it reflect the + product. + +The ground is moved under each product as it is rendered, so short and tall products all +stand on it. It is hidden for any view looking up from below it, so **Bottom** still shows +the product. + +#### Materials -**Intensity** multiplies the preset's light energies. **Transparent Background** renders -alpha instead of the **Background Color**. `Rebuild Lighting` and `Apply Background` re-run -either half on its own. +The **Materials** sub-panel replaces the products' materials in Full renders with one of: +Matte Plastic, Glossy Plastic, Rubber, Brushed Metal, Polished Metal or Glass, in a colour +you pick. **Keep Own** (the default) leaves materials alone. + +Set it for the whole batch, or select a product in **Products to render** and give it its +own preset and colour. **Batch Default** follows the batch setting; **Keep Own** keeps that +product's materials even when the batch has a preset. + +The products' own materials are never edited. The preset is applied by a temporary +Geometry Nodes modifier, `MV_Material`, added while the product renders and removed +afterwards, so it also works on STL imports, which arrive with no materials at all. ### 2. Import Products -Add one or more `.step` / `.stp` / `.stl` files to the list and click **Import CAD Files**. -Each file becomes its own collection under `MV_Products`. That grouping is what the batch -renderer iterates over. You can mix STEP and STL freely in one list; each file is routed by -its extension, and a file that fails does not stop the ones after it. +Add files to the list with **+**, or a whole folder with the folder button, then click +**Import Files**. **Add Folder** picks up every supported file in the folder (tick **Include +Subfolders** in the file browser's sidebar to go deeper, on by default) and skips files that +are already in the list. Each file becomes its own collection under `MV_Products`. That +grouping is what the batch renderer iterates over. You can mix formats freely in one list; +each file is routed by its extension, and a file that fails does not stop the ones after it. Import runs in the background: the status bar shows `MultiView: 3/12 housing.step (converting)`, Blender stays responsive, and **Esc** cancels after the file in flight. @@ -116,10 +200,16 @@ Files that fail are skipped rather than stopping the batch, and FreeCAD's own er message is reported instead of a generic failure -- run `Window > Toggle System Console` to see the full list. -STL files skip the conversion step entirely, so they land immediately rather than showing -`(converting)`. A `.stl` that Blender cannot read is reported and skipped rather than +Mesh files skip the conversion step entirely, so they land immediately rather than showing +`(converting)`. A file that Blender cannot read is reported and skipped rather than becoming an empty product collection that would render as blank frames. +**Mesh Units** says what unit the numbers in STL, OBJ, PLY, glTF and FBX files are in +(millimetres by default, which is what CAD exports almost always use). It does not change +the geometry; it is only used to report each product's real size and to scale products +against each other with **Keep Proportions**. STEP and IGES come out of FreeCAD in +millimetres, and 3MF files state their own unit. + You can also click **Add Empty Product Collection** and drag existing meshes into it, which is how you use MultiView with geometry that didn't come from a CAD file. @@ -135,10 +225,22 @@ origin. Turn it off if you want to place and scale products yourself. **Standardize Products** applies the same fit to everything already under `MV_Products`, which is what you want after dragging in your own meshes or importing with the option off: -- **Recenter**: moves each product's bounding box centre onto the world origin. -- **Rescale**: uniformly scales each product so its largest dimension equals **Target - Size**. This is what makes a tiny screw and a large housing frame read identically in - the final images. +- **Recenter**: moves each product's bounding box centre onto the world origin. With + **Rest on Floor**, the bottom of the box goes on the origin instead, so the product stands + on the ground rather than straddling it. +- **Rescale**: uniformly scales each product, one of two ways: + - **Fit Each** (default): every product's largest dimension becomes **Target Size**. This + is what makes a tiny screw and a large housing frame read identically in the final + images. + - **Keep Proportions**: the biggest product in the batch becomes Target Size and the rest + keep their real size relative to it, so a 10 mm screw next to a 400 mm housing renders + at a fortieth of its size. Uses the real sizes recorded at import, converted through + each file's units. + +Each product's real size, measured before anything rescaled it, is shown under the +Standardize button for the active product (`120 x 40 x 15.5 mm`) and written to the +manifest. Products assembled by hand are measured the first time Standardize runs on them, +in metres. - **Auto Smooth**: fixes the faceted look a tessellated CAD import arrives with. FreeCAD writes per-face normals that Blender keeps as custom split normals, and some Blender @@ -170,6 +272,12 @@ Choose a view set and click **Build Cameras**. Cameras land in `MV_Cameras`, nam | **Orthographic** | Front, Back, Top, Bottom, Right, Left (6) | | **Standard** | Front, Top, Right, Perspective (4) | | **Minimal** | Front, Top, Right (3) | +| **Custom** | Whichever of the nine you tick | + +Under the dropdown is a checkbox for each of the nine views. With a preset selected they are +greyed out and show what the preset includes. Pick **Custom** and they become editable, +starting from the preset you had selected, so going from Standard to "Standard plus Back" +takes one tick. **Front Axis** tells the add-on which world axis points out of the front of your product. CAD exports usually land on `+X` (the default); Blender's own convention is `-Y`. Getting @@ -183,7 +291,8 @@ All orthographic views share one framing, and all perspective views share anothe product reads at the same scale across the whole sheet rather than being re-fitted per view. Need an angle that isn't in the presets? Add a **Custom View** with an X/Y/Z direction -vector and an ortho/perspective toggle, and it gets built alongside the rest. +vector and an ortho/perspective toggle, and it gets built alongside the rest. The checkbox at +the start of its row switches it off without deleting it. #### Orient Products @@ -217,9 +326,19 @@ through. Rotation is about the product's own centre, so it never drifts out of f multi-part assembly turns as one rigid piece. **Rotate Step** sets how far one press turns it, 90 degrees by default, so four presses of the same button return to where you started. -Turning a product changes its bounding box, so **click Build Cameras again once you are -done orienting**. Framing is measured across the whole batch, and it is measured before you -start rotating. +**Auto Orient** turns the product for you, and **All** next to it does the whole batch: + +| Method | What it does | +| --- | --- | +| **Lay Flat** (default) | Rests the product on its largest flat face, then turns it about the vertical so its footprint lines up with the axes and its longest side runs across the Front view. Right for boxes, housings, panels and most machined parts. | +| **Principal Axes** | Lines the product's longest spread up across the Front view, the next longest towards the Front camera, and the thinnest upright. Better for rounded or organic shapes with no flat face. It may leave a product upside down; Up/Down twice fixes that. | + +**Reset** turns the product back to the way it faced when it was imported, undoing every +Auto Orient and arrow press since; **All** resets the whole batch. A product you built by +hand gets its starting orientation recorded the first time you turn it., and framing is measured across the whole batch. +With **Reframe Before Rendering** on (the default) you don't need to do anything about that: +Render All rebuilds the cameras before the first image. If you have switched it off, click +**Build Cameras** again once you are done orienting. ### 5. Batch Render @@ -232,7 +351,104 @@ Set an **Output Directory**, choose a **Render Mode**, and hit **Render All**. | **Both** | Writes a `_render` and a `_clay` image per view. | Every product is isolated in turn (all others excluded from the view layer), so nothing -bleeds between shots. Progress prints to the system console. +bleeds between shots. + +#### Extra passes + +Tick any of these under **Extra passes** to write another image per view alongside the +render mode's. Each has its own `{mode}` in the file name, and gets its own contact sheet. + +| Pass | `{mode}` | What it is | +| --- | --- | --- | +| **Line Art** | `lines` | A technical drawing: silhouettes and creases as lines. **Crease Angle** sets how sharp a fold has to be to get a line; smooth curves get only their outline. **Fill** is White (black on white), Clay (lines over the clay render) or None (lines on transparency). | +| **Mask** | `mask` | White product on black, antialiased, for use as a matte. | +| **AO** | `ao` | Ambient occlusion: a white product darkened in its creases and recesses, with lighting and materials taken out. **Distance** is how far occlusion reaches, as a fraction of the product's size. Needs the Cycles add-on (on by default). | + +Line Art and Mask are fast Workbench renders whatever your engine is. Line Art finds its +lines from the geometry's normals rather than with Freestyle, so it needs nothing set up +and does not slow your Full render down. + +#### Pre-flight check + +The **Pre-flight** section under Batch Render looks for anything that would waste a render, +before any render time is spent. **Check Batch** runs it on demand; with **On Render** ticked +(the default) it also runs when you press Render All. + +| Severity | Found | One-click fix | +| --- | --- | --- | +| Error | No products, or none ticked | | +| Error | Output format is a video rather than an image | | +| Error | Output path is a file, or the folder cannot be written | | +| Error | No views selected, or no cameras with Reframe off | Build cameras | +| Error | The file name template would give two images the same name | | +| Error | AO pass on, but the Cycles add-on is off | | +| Warning | Output folder is `//` relative but the `.blend` was never saved, so `//` points nowhere useful | Save the file | +| Warning | Output folder does not exist yet | Create it | +| Warning | Cameras are out of date (Reframe off): a product was turned, scaled or added, or the views, padding or resolution changed since Build Cameras | Rebuild cameras | +| Warning | An empty product. It is skipped when rendering rather than producing blank frames | | +| Warning | Faces with flipped normals, or a whole mesh inside out | Recalculate normals | +| Warning | Products are spread around the scene instead of centred, so each renders as a speck | Recenter | +| Warning | Objects outside `MV_Products` that would appear in every shot | Switch off their render visibility | +| Warning | Full render with no lights and no HDRI | Build lighting | +| Warning | Lights Follow Camera is on but the lighting predates the rig | Rebuild lighting | +| Warning | Shadow Catcher with an engine other than Cycles; HDRI file missing | | + +Each issue is listed in the panel with the product it concerns, and a wrench button where +there is a fix. Errors stop Render All with a message saying why. Warnings bring up a +dialog listing them, with **Render Anyway** and **Cancel**. Called from a script, Render All +stops on errors the same way but never asks; warnings are left for the panel. + +Normals are checked by looking at every edge: two faces sharing an edge should run along it +in opposite directions, and where they run the same way one of them is flipped. A closed +mesh whose faces all agree but enclose a negative volume is inside out as a whole. The fix +recalculates every face of the product to point outwards. + +#### Progress and cancelling + +The batch runs one image at a time with Blender still responsive. The status bar shows how +far it has got, which image is next, and roughly how long is left: + +``` +MultiView: 12/90 AA_03_Bracket Top ~4m 10s left - Esc to cancel +``` + +**Esc** stops the batch after the image currently rendering. The time left is an average of +the images so far, so it settles after the first few. Each file is also printed to the +system console as it is written. + +#### Choosing what to render + +The **Products to render** list shows every product under `MV_Products` with the label its +files will get. Untick a product to leave it out. The two buttons beside the list tick or +untick all of them, and the main button changes to **Render N Ticked** when some are off. + +**Render Active** renders only the product selected in the list, whether it is ticked or +not. Clicking a row also makes it the product **Orient Products** works on. + +Leaving a product out never renumbers the rest. Labels come from each product's place in the +whole list, so `AA_03_Knob` is still `AA_03` when `AA_02` is unticked, and re-rendering one +product later writes into the same folder it used before. + +#### Resuming with Skip Existing + +With **Skip Existing** ticked, images that are already in the output folder are left alone +and only the missing ones are rendered. Use it to pick up a batch that crashed or that you +cancelled. The counter and the time left then cover only the remaining images. A zero-byte +file, which is what an interrupted write leaves behind, counts as missing. + +Leave it off (the default) when you have changed a setting and want every image redone. +Skip Existing only checks whether a file is there, not what settings it was rendered with. + +#### Reframing + +**Reframe Before Rendering** (on by default) rebuilds the camera rig from the current view +selection, Front Axis, padding, output resolution and product positions before the first +image. A product you rotated, or a resolution you changed after building cameras, can't +leave a stale framing behind. + +Framing always covers the whole batch, ticked or not, so a product rendered on its own comes +out at the same scale as the run it belongs to. Turn this off if you have moved or adjusted +the `MV_Cam_*` cameras by hand and want those used as they are. **Only products are isolated.** Geometry that lives outside `MV_Products` is not a product, so it is never hidden and turns up in every shot of every product. The usual offender is the @@ -286,6 +502,107 @@ The `AA_01` label is controlled by three fields: letter pair. After `ZZ` the label grows a character rather than wrapping, so it runs `AAA`, `BBB`, ... and every product keeps a folder of its own. +#### Naming templates + +The layout above is the default. **Naming → Folder** and **Naming → File Name** let you +change it with tokens: + +| Token | Example | Meaning | +| --- | --- | --- | +| `{label}` | `AA_01` | The product's label | +| `{prefix}` | `AA` | The letter part of the label | +| `{number}` | `01` | The number part of the label | +| `{product}` | `Widget` | The product collection's name | +| `{view}` | `Front` | The camera view, or `Sheet` / `Turntable` | +| `{mode}` | `clay` | `clay`, `render` (Both mode), or empty for Full Render | + +The defaults are `{label}_{product}` for the folder and `{label}_{product}_{view}_{mode}` for +the file. A token that comes out empty takes one separator next to it along with it, so +`..._Front_{mode}` gives `..._Front` in Full Render mode rather than a trailing underscore. + +- Use `/` in the folder template to nest folders, e.g. `{mode}/{label}_{product}` puts clay + and full renders in separate trees. Leave it empty to put everything straight in the + output directory. +- An unknown token is left in the name as typed, so a typo shows up in the preview under the + fields (the first image of the batch, as it will be written) instead of silently vanishing. +- If the template would give two images the same name (no `{view}`, or no `{mode}` in Both + mode), the render stops before it starts and says so. + +#### Contact sheets + +Tick **Contact Sheet** and each product also gets one image with all of its views in a grid, +the product's name across the top and each view's name underneath: +`AA_01_Widget/AA_01_Widget_Sheet_clay.png`. Both mode makes one sheet per mode. + +- **Columns**: views per row. `0` picks a roughly square grid. +- **Cell Width**: how wide each view is drawn, in pixels. +- **Labels**, **Background**, **Text**: the captions and colours. + +With Post-Processing on, the sheet is built from the processed copies, so trimmed images +make a tighter sheet. Labels need a Blender recent enough to draw text into an image (5.0 +does); on an older one the sheet is drawn without captions. + +#### Turntables + +Tick **Turntable** and each product is also rendered spinning one full turn, then encoded: + +- **Style**: Clay (fast, matches the clay stills) or Full Render. +- **Frames** and **FPS**: 36 frames at 24 fps is a 1.5 second loop. +- **Elevation**: how far above the product the camera looks down from. +- **MP4** (H.264) and/or **GIF**. The GIF is scaled so its longest edge is **GIF Size**, + and uses one palette for the whole loop so the background doesn't flicker. + +The product turns about its own centre under the fixed studio lights, as on a real +turntable. Every product is framed by one camera wide enough for the whole batch at every +angle, so turntables share a scale the same way the stills do. Frames are kept in a +`turntable` folder next to the stills (`AA_01_Widget/turntable/..._001.png`); the video and +GIF go next to the stills. Transparent renders are put on the scene's **Background Color**, +since neither format does transparency well. The product is put back exactly where it +was afterwards, even if you cancel half way through. + +#### Post-processing + +Tick **Post-Processing** to write a processed copy of every image. **The original renders +are never touched**: the copies go in a separate folder (**Folder**, default `processed`) +that mirrors the original layout, so you can change the settings and re-process without +re-rendering. + +| Option | What it does | +| --- | --- | +| **Trim to Content** | Crops the empty background around the product, leaving **Margin** (a fraction of the product's size) around it. Each image then fills its own frame, so views no longer share one scale. | +| **Pad to Square** | Extends the shorter side with background so the image is square. | +| **Flatten** | Puts transparent renders on a solid colour. JPEG always does this. | +| **Format** | PNG, JPEG or WebP, with **Quality** for the lossy two. | +| **Max Size** | Shrinks the image so its longest edge is at most this many pixels. `0` keeps the size. | +| **Extra sizes** | Smaller copies too, e.g. `1024, 256` writes `..._Front_clay_1024.jpg` and `..._Front_clay_256.jpg`. Images are never scaled up. | + +Trimming works on transparency when the render has it, and otherwise on whatever colour the +four corners share, so it works on a solid studio background too. + +With **Skip Existing** on, an original that is already on disk is not re-rendered, but a +missing processed copy of it is still made. To re-process everything after changing the +settings, delete the `processed` folder and render again with Skip Existing on: only the +copies are rebuilt. + +#### Manifest + +**Manifest** (CSV + JSON by default) writes `manifest.csv` and/or `manifest.json` into the +output directory, listing every output with: + +`label`, `product`, `kind` (render, processed, sheet, turntable), `view`, `mode`, `path` +(relative to the output directory), `width`, `height`, `source_file`, `source_size` and +`source_units`. + +`source_file`, `source_size` and `source_units` are recorded when a file is imported: +`source_size` is the product's bounding box before Fit on Import rescaled it, in +`source_units` (`mm` for STEP and IGES, the file's own unit for 3MF, **Mesh Units** for +other meshes, empty after a STEPper import). Products assembled by hand have no source file +and are measured in metres the first time Standardize runs. + +Rendering part of a batch updates the rows for those products and keeps the rest, so the +manifest always describes everything in the output directory. Rows for files that have +since been deleted are dropped. + --- ## Collections the add-on creates @@ -294,9 +611,11 @@ The `AA_01` label is controlled by three fields: | --- | --- | | `MV_Products` | One child collection per product. This is what gets iterated and isolated. | | `MV_Cameras` | Generated cameras, named `MV_Cam_`. | -| `MV_Lighting` | The lights from the active lighting preset. | +| `MV_Lighting` | The lights from the active lighting preset, hung off the `MV_LightRig` empty. | +| `MV_Stage` | The ground plane, `MV_Ground`, when Ground is not None. | -It also creates one world datablock, `MV_World`, for the background. +It also creates one world datablock, `MV_World`, for the background, and a few materials +named `MV_...` for the ground, the AO pass and the material presets. Rebuilding cameras or lighting clears and regenerates only that collection, discarding the camera and light datablocks it made last time rather than leaving them behind. Your @@ -306,7 +625,17 @@ products, and any other world in the file, are never touched. ## Troubleshooting -**"No cameras. Build cameras first."**: run *Build Cameras* in the Cameras / Views panel. +**"No cameras. Build cameras first."**: run *Build Cameras* in the Cameras / Views panel, or +switch **Reframe Before Rendering** back on. + +**"No views selected."**: the view set is **Custom** with nothing ticked (and no enabled +custom views). Tick at least one view. + +**"No products are ticked for rendering."**: tick products in the **Products to render** +list, or use **Render Active** for the selected one. + +**A cancelled or crashed batch**: tick **Skip Existing** and render again. Only the images +that are missing get rendered. **Renders come back blank, or with something unexpected in them**: check the warning above **Render All**. Anything outside `MV_Products` is rendered along with every product, and a @@ -333,7 +662,8 @@ the default while orienting. Click **Show All** in Cameras / Views, or untick th Renders are unaffected either way. **Renders are cropped after rotating products**: rotating changes the bounding box the -framing was measured from. Click *Build Cameras* again. +framing was measured from. Switch on **Reframe Before Rendering**, or click *Build Cameras* +again. **Imported STEP geometry is blocky**: lower **Tessellation Deflection** in preferences (try `0.02`) and re-import. diff --git a/multiview_product_renderer/__init__.py b/multiview_product_renderer/__init__.py index e673500..ac16f67 100644 --- a/multiview_product_renderer/__init__.py +++ b/multiview_product_renderer/__init__.py @@ -1,24 +1,35 @@ bl_info = { "name": "MultiView Product Renderer", "author": "Danyal Sarfraz", - "version": (1, 4, 0), + "version": (1, 9, 0), "blender": (4, 0, 0), "location": "View3D > Sidebar > MultiView", - "description": "Import STEP and STL files, set up cameras and 3-point lighting, batch render products across views.", + "description": "Import CAD and mesh files, set up cameras and studio lighting, batch render products across views.", "category": "Render", } import bpy +import blf import os +import re +import csv +import json import math +import struct import subprocess import tempfile import shutil import time -from mathutils import Vector, Matrix +import traceback +import zlib +from contextlib import contextmanager +from datetime import datetime +import numpy as np +from mathutils import Vector, Matrix, Quaternion from bpy.props import ( StringProperty, EnumProperty, BoolProperty, FloatProperty, - IntProperty, FloatVectorProperty, CollectionProperty, PointerProperty, + IntProperty, FloatVectorProperty, BoolVectorProperty, CollectionProperty, + PointerProperty, ) from bpy.types import ( Operator, Panel, PropertyGroup, AddonPreferences, UIList, @@ -79,7 +90,35 @@ {"name": "Rim", "pos": ( 0.0, 1.8, 2.4), "size": 3.5, "energy": 900, "color": (1, 1, 1)}, ], }, + "NONE": { + "label": "No Lights (HDRI only)", + "lights": [], + }, +} + +# Saved lighting presets are tagged with this prefix in the enum, so they can +# never collide with a built-in key. +USER_PRESET_PREFIX = "USER:" +LIGHT_RIG = "MV_LightRig" + +MATERIAL_PRESETS = { + "MATTE_PLASTIC": {"label": "Matte Plastic", "inputs": {"Roughness": 0.55}}, + "GLOSSY_PLASTIC": {"label": "Glossy Plastic", "inputs": {"Roughness": 0.12, "Coat Weight": 0.4}}, + "RUBBER": {"label": "Rubber", "inputs": {"Roughness": 0.9}}, + "BRUSHED_METAL": {"label": "Brushed Metal", "inputs": {"Metallic": 1.0, "Roughness": 0.35, + "Anisotropic": 0.8}}, + "POLISHED_METAL": {"label": "Polished Metal", "inputs": {"Metallic": 1.0, "Roughness": 0.06}}, + "GLASS": {"label": "Glass", "inputs": {"Roughness": 0.02, "IOR": 1.45, + "Transmission Weight": 1.0}}, } +MATERIAL_ITEMS = [(k, v["label"], f"Render with a {v['label'].lower()} material") + for k, v in MATERIAL_PRESETS.items()] +MATERIAL_MOD = "MV_Material" +MATERIAL_GROUP = "MV_SetMaterial" + +UNIT_ITEMS = [("mm", "Millimetres", ""), ("cm", "Centimetres", ""), ("m", "Metres", ""), + ("in", "Inches", ""), ("ft", "Feet", "")] +UNIT_TO_MM = {"um": 0.001, "mm": 1.0, "cm": 10.0, "m": 1000.0, "in": 25.4, "ft": 304.8} # Object types that put something in front of a camera. Lights and cameras # are not here; neither are empties, which only carry a transform. @@ -88,22 +127,33 @@ CAMERA_COLL = "MV_Cameras" LIGHT_COLL = "MV_Lighting" PRODUCT_COLL = "MV_Products" +STAGE_COLL = "MV_Stage" WORLD_NAME = "MV_World" +GROUND_NAME = "MV_Ground" GUIDE_LINES = [ - "1. Scene Setup: pick a lighting preset and background, then Apply.", - "2. Import: add STEP or STL files, then click Import. STEP needs", - " FreeCAD or STEPper; STL is already a mesh and imports directly.", - " Each file becomes a collection under MV_Products.", + "1. Scene Setup: pick lighting (or save your own with +), a colour or", + " HDRI background and a ground, then Apply. Materials sets preset", + " materials for Full renders, for the batch or one product.", + "2. Import: add files or a whole folder, then click Import. STEP and", + " IGES need FreeCAD (STEP can use STEPper); STL, OBJ, PLY, glTF,", + " FBX and 3MF are meshes and import directly. Each file becomes a", + " collection under MV_Products; STEP parts keep names and colours.", " You can also create empty product collections and drop meshes in.", - "3. Standardize: centre and scale every product to one size, and fix", - " the faceted shading a CAD tessellation arrives with. Fit on Import", - " does the same automatically as each file lands.", - "4. Cameras: pick a view set and front axis, click Build.", - " Then use Orient Products to walk the batch one at a time and turn", - " anything that faces the wrong way, before rebuilding the cameras.", + "3. Standardize: centre and scale every product, to one size each or", + " keeping their real proportions, and fix the faceted shading a CAD", + " tessellation arrives with. Fit on Import does it as each file lands.", + "4. Cameras: pick a view set (or Custom and tick views), front axis,", + " click Build. Then use Orient Products to walk the batch one at a", + " time: Auto Orient lays products flat, Reset undoes any turning.", "5. Render: set output folder, mode (full/clay/both), starting index,", - " then Render All. Files are named PREFIX_NN_Product_View.", + " tick the products you want, then Render All or Render Active.", + " Pre-flight checks the batch first and offers one-click fixes.", + " Esc stops after the current image; Skip Existing resumes.", + " Cameras are reframed first. Files are named PREFIX_NN_Product_View,", + " or however the Naming templates say.", + " Optional extras under Batch Render: contact sheets, turntables", + " (MP4/GIF), post-processed copies, and a CSV/JSON manifest.", " Only products are hidden between shots, so anything outside", " MV_Products lands in every image. The panel warns you first.", "", @@ -151,6 +201,11 @@ class MV_StepFileItem(PropertyGroup): class MV_CustomView(PropertyGroup): + enabled: BoolProperty( + name="Enabled", + description="Build a camera for this view and render it", + default=True, + ) name: StringProperty(name="Name", default="CustomView") dir_x: FloatProperty(name="X", default=1.0) dir_y: FloatProperty(name="Y", default=-1.0) @@ -158,6 +213,70 @@ class MV_CustomView(PropertyGroup): is_ortho: BoolProperty(name="Ortho", default=False) +def user_preset_dir(create=False): + return bpy.utils.user_resource( + 'SCRIPTS', path=os.path.join("presets", "multiview_lighting"), create=create) + + +def user_lighting_presets(): + """{enum key: (label, path)} for every saved lighting preset. + + Kept in Blender's user scripts folder rather than in the .blend, so a rig + saved once is there in every file, the way Blender's own presets are. + """ + folder = user_preset_dir() + found = {} + if folder and os.path.isdir(folder): + for name in sorted(os.listdir(folder)): + if name.lower().endswith(".json"): + stem = name[:-5] + found[USER_PRESET_PREFIX + stem] = (stem, os.path.join(folder, name)) + return found + + +# Blender keeps only pointers to the strings a dynamic enum returns, so the +# list has to outlive the callback or the labels turn to garbage. +_lighting_items = [] + + +def lighting_preset_items(self, context): + # Built-ins keep the numbers they had as a fixed enum (0, 1, 2, ...), so + # files saved before presets could be added still open on their choice. + # Saved ones get a number from their name, which stays put as others are + # added and removed around them. + items = [(k, v["label"], v["label"], i) for i, (k, v) in enumerate(LIGHTING_PRESETS.items())] + for key, (label, path) in user_lighting_presets().items(): + items.append((key, label, f"Saved preset: {path}", + 1000 + zlib.crc32(key.encode("utf-8")) % 1000000)) + _lighting_items[:] = items + return _lighting_items + + +def lighting_specs(key): + """The light list for a preset key, or None if it cannot be read.""" + if key in LIGHTING_PRESETS: + return LIGHTING_PRESETS[key]["lights"] + entry = user_lighting_presets().get(key) + if entry is None: + return None + try: + with open(entry[1], encoding="utf-8") as f: + lights = json.load(f)["lights"] + for spec in lights: + Vector(spec["pos"]) + float(spec["energy"]) + except (OSError, ValueError, KeyError, TypeError): + return None + return lights + + +class MV_Issue(PropertyGroup): + severity: StringProperty() + message: StringProperty() + product: StringProperty() + fix: StringProperty() + + def _update_isolate(self, context): """Apply the isolate toggle the moment it is ticked, rather than waiting for the next rotate or step. @@ -170,6 +289,25 @@ def _update_isolate(self, context): isolate_product(context, coll.name if (self.isolate_active and coll) else None) +def _update_active_product(self, context): + """Picking a product in the render list makes it the one being oriented, + so isolation follows the click the same way it follows Previous/Next.""" + if self.isolate_active: + coll, _ = active_product(context) + if coll is not None: + isolate_product(context, coll.name) + + +def _update_view_set(self, context): + """Copy a preset into the per-view checkboxes, so switching to Custom + afterwards starts from the views you were just looking at rather than + from a blank or stale selection.""" + if self.view_set == 'CUSTOM': + return + names = {v[0] for v in VIEW_SETS[self.view_set]} + self.views_enabled = [v[0] in names for v in VIEWS_ALL] + + class MV_Settings(PropertyGroup): step_files: CollectionProperty(type=MV_StepFileItem) step_files_index: IntProperty(default=0) @@ -181,8 +319,16 @@ class MV_Settings(PropertyGroup): ("ORTHO", "Orthographic", "Front, Back, Top, Bottom, Right, Left"), ("STD", "Standard", "Front, Top, Side, Perspective"), ("MIN", "Minimal", "Front, Top, Side"), + ("CUSTOM", "Custom", "Tick the views you want individually"), ], default="ALL", + update=_update_view_set, + ) + views_enabled: BoolVectorProperty( + name="Views", + description="Which of the standard views get a camera when View Set is Custom", + size=len(VIEWS_ALL), + default=(True,) * len(VIEWS_ALL), ) front_axis: EnumProperty( @@ -199,20 +345,81 @@ class MV_Settings(PropertyGroup): lighting_preset: EnumProperty( name="Lighting", - items=[(k, v["label"], v["label"]) for k, v in LIGHTING_PRESETS.items()], - default="STUDIO_SOFT", + description="Light rig to build. Saved presets appear after the built-in ones", + items=lighting_preset_items, + default=0, ) lighting_intensity: FloatProperty( name="Intensity", description="Multiplier on preset light energies", default=1.0, min=0.0, soft_max=5.0, ) + lights_follow_camera: BoolProperty( + name="Lights Follow Camera", + description="Turn the light rig with each camera while rendering, so every view " + "is lit the way Front is. Off, the lights stay put and views from " + "behind or below are lit mostly by the rim light", + default=True, + ) + world_mode: EnumProperty( + name="Background", + items=[('COLOR', "Color", "A flat colour behind the product"), + ('HDRI', "HDRI", "An environment image that lights the product and, " + "optionally, shows behind it")], + default='COLOR', + ) transparent_bg: BoolProperty(name="Transparent Background", default=False) bg_color: FloatVectorProperty( name="Background Color", subtype='COLOR', size=4, min=0.0, max=1.0, default=(0.5, 0.5, 0.5, 1.0), ) + hdri_path: StringProperty( + name="HDRI", subtype='FILE_PATH', + description="Environment image, usually .hdr or .exr", + ) + hdri_strength: FloatProperty(name="Strength", default=1.0, min=0.0, soft_max=10.0) + hdri_rotation: FloatProperty( + name="Rotation", description="Turn the environment about the vertical axis", + default=0.0, subtype='ANGLE', + ) + hdri_visible: BoolProperty( + name="Show HDRI", + description="See the environment behind the product. Off, it still lights and " + "reflects in the product but the camera sees the background colour", + default=False, + ) + + ground_mode: EnumProperty( + name="Ground", + description="Something for the product to stand on in Full renders", + items=[('NONE', "None", "The product floats"), + ('SHADOW', "Shadow Catcher", "An invisible floor that only shows the " + "product's shadow. Cycles only"), + ('FLOOR', "Floor", "A visible floor in the colour below")], + default='NONE', + ) + ground_color: FloatVectorProperty( + name="Floor Color", subtype='COLOR', size=3, + min=0.0, max=1.0, default=(0.8, 0.8, 0.8), + ) + ground_roughness: FloatProperty( + name="Roughness", description="Lower makes the floor reflect the product", + default=0.5, min=0.0, max=1.0, subtype='FACTOR', + ) + + material_preset: EnumProperty( + name="Material", + description="Material every product renders with in Full renders, unless the " + "product picks its own", + items=[('KEEP', "Keep Own", "Use the materials the products already have")] + + MATERIAL_ITEMS, + default='KEEP', + ) + material_color: FloatVectorProperty( + name="Color", subtype='COLOR', size=3, + min=0.0, max=1.0, default=(0.8, 0.8, 0.8), + ) render_output_dir: StringProperty( name="Output Directory", subtype='DIR_PATH', default="//renders/", @@ -230,6 +437,228 @@ class MV_Settings(PropertyGroup): name="Clay Color", subtype='COLOR', size=3, min=0.0, max=1.0, default=(0.75, 0.75, 0.75), ) + pass_lines: BoolProperty( + name="Line Art", + description="Also write a technical-drawing image per view: outlines and creases " + "drawn as lines. Named with mode 'lines'", + default=False, + ) + pass_mask: BoolProperty( + name="Mask", + description="Also write a silhouette per view: white product on black. " + "Named with mode 'mask'", + default=False, + ) + pass_ao: BoolProperty( + name="AO", + description="Also write an ambient occlusion image per view: white product " + "darkened in its creases and contact areas. Needs Cycles. Named " + "with mode 'ao'", + default=False, + ) + lines_angle: FloatProperty( + name="Crease Angle", + description="Edges where faces meet more sharply than this are drawn", + default=math.radians(30.0), min=math.radians(1.0), max=math.radians(179.0), + subtype='ANGLE', + ) + lines_width: IntProperty(name="Line Width", description="In pixels", default=2, min=1, max=12) + lines_color: FloatVectorProperty( + name="Line Color", subtype='COLOR', size=3, + min=0.0, max=1.0, default=(0.0, 0.0, 0.0), + ) + lines_fill: EnumProperty( + name="Fill", + items=[('WHITE', "White", "Black lines on white: a technical drawing"), + ('CLAY', "Clay", "Lines drawn over the clay render"), + ('NONE', "None", "Lines only, on transparency")], + default='WHITE', + ) + ao_samples: IntProperty(name="AO Samples", default=32, min=1, max=4096) + ao_distance: FloatProperty( + name="AO Distance", + description="How far occlusion reaches, as a fraction of the product's size", + default=0.5, min=0.001, soft_max=2.0, + ) + preflight_on_render: BoolProperty( + name="Check Before Rendering", + description="Run the pre-flight check when Render All is pressed. Errors stop the " + "render; warnings ask before going ahead", + default=True, + ) + preflight_issues: CollectionProperty(type=MV_Issue) + preflight_time: StringProperty() + skip_existing: BoolProperty( + name="Skip Existing", + description="Leave images that are already in the output folder alone and " + "render only the missing ones. Use it to resume a batch that " + "crashed or was cancelled. Turn it off to overwrite after " + "changing a setting", + default=False, + ) + reframe_on_render: BoolProperty( + name="Reframe Before Rendering", + description="Rebuild the cameras from the current view settings and product " + "positions when a render starts, so rotating a product or changing " + "resolution never leaves a stale framing. Turn off to keep cameras " + "you have adjusted by hand", + default=True, + ) + + file_template: StringProperty( + name="File Name", + description="Name of each image, without extension. Tokens: {label} (AA_01), " + "{prefix} (AA), {number} (01), {product}, {view}, {mode} (render, " + "clay, or nothing for Full Render). A token that comes out empty " + "takes one separator next to it with it", + default="{label}_{product}_{view}_{mode}", + ) + folder_template: StringProperty( + name="Folder", + description="Folder each image goes in, under the output directory. Same tokens " + "as the file name; use / to nest folders, or leave empty to put " + "everything straight in the output directory", + default="{label}_{product}", + ) + + sheet_enabled: BoolProperty( + name="Contact Sheet", + description="After each product, lay all its views out on one labelled image", + default=False, + ) + sheet_columns: IntProperty( + name="Columns", + description="Views per row. 0 picks a roughly square grid", + default=0, min=0, max=16, + ) + sheet_cell_width: IntProperty( + name="Cell Width", + description="Width in pixels each view is scaled to on the sheet", + default=512, min=64, max=4096, + ) + sheet_labels: BoolProperty( + name="Labels", + description="Print the product name across the top and the view name under each image", + default=True, + ) + sheet_bg: FloatVectorProperty( + name="Background", subtype='COLOR', size=4, + min=0.0, max=1.0, default=(1.0, 1.0, 1.0, 1.0), + ) + sheet_text_color: FloatVectorProperty( + name="Text", subtype='COLOR', size=3, + min=0.0, max=1.0, default=(0.1, 0.1, 0.1), + ) + + turntable_enabled: BoolProperty( + name="Turntable", + description="Also render each product spinning a full turn, one frame per step", + default=False, + ) + turntable_frames: IntProperty( + name="Frames", + description="Frames in one full turn", + default=36, min=4, max=720, + ) + turntable_elevation: FloatProperty( + name="Elevation", + description="How far above the product the turntable camera looks down from", + default=math.radians(20.0), min=math.radians(-89.0), max=math.radians(89.0), + subtype='ANGLE', + ) + turntable_style: EnumProperty( + name="Style", + items=[('CLAY', "Clay", "Fast Workbench clay, like the clay stills"), + ('FULL', "Full Render", "Your current engine and materials")], + default='CLAY', + ) + turntable_mp4: BoolProperty( + name="MP4", + description="Encode the frames into an H.264 video", + default=True, + ) + turntable_gif: BoolProperty( + name="GIF", + description="Encode the frames into a looping animated GIF", + default=False, + ) + turntable_fps: IntProperty(name="FPS", default=24, min=1, max=60) + turntable_gif_size: IntProperty( + name="GIF Size", + description="Longest edge of the GIF in pixels. GIFs get large fast", + default=480, min=64, max=2048, + ) + + post_enabled: BoolProperty( + name="Post-Processing", + description="Write a processed copy of every image: trimmed, padded, resized " + "or converted. The original renders are kept as they are", + default=False, + ) + post_folder: StringProperty( + name="Folder", + description="Subfolder of the output directory the processed copies go in. " + "It mirrors the folder layout of the originals", + default="processed", + ) + post_trim: BoolProperty( + name="Trim to Content", + description="Crop away the empty background around the product. Each image " + "then fills its own frame, so views no longer share one scale", + default=False, + ) + post_margin: FloatProperty( + name="Margin", + description="Space left around the product after trimming, as a fraction of its size", + default=0.05, min=0.0, max=0.5, subtype='FACTOR', + ) + post_square: BoolProperty( + name="Pad to Square", + description="Extend the shorter side with background so the image is square", + default=False, + ) + post_flatten: BoolProperty( + name="Flatten Transparency", + description="Put transparent renders on a solid colour. JPEG always does this", + default=False, + ) + post_bg_color: FloatVectorProperty( + name="Flatten Color", subtype='COLOR', size=3, + min=0.0, max=1.0, default=(1.0, 1.0, 1.0), + ) + post_format: EnumProperty( + name="Format", + items=[('PNG', "PNG", "Lossless, keeps transparency"), + ('JPEG', "JPEG", "Small, no transparency"), + ('WEBP', "WebP", "Small, keeps transparency")], + default='PNG', + ) + post_quality: IntProperty( + name="Quality", description="JPEG and WebP quality", + default=90, min=1, max=100, subtype='PERCENTAGE', + ) + post_max_size: IntProperty( + name="Max Size", + description="Shrink so the longest edge is at most this many pixels. 0 keeps the size", + default=0, min=0, max=16384, + ) + post_extra_sizes: StringProperty( + name="Extra Sizes", + description="Also write smaller copies with these longest edges, e.g. '1024, 256'. " + "Each gets the size added to its name: AA_01_Widget_Front_256", + default="", + ) + + manifest_format: EnumProperty( + name="Manifest", + description="List every output with its product, view, size and source file, " + "in the output directory", + items=[('NONE', "None", "Do not write a manifest"), + ('CSV', "CSV", "manifest.csv"), + ('JSON', "JSON", "manifest.json"), + ('BOTH', "CSV + JSON", "manifest.csv and manifest.json")], + default='BOTH', + ) start_prefix: StringProperty( name="Start Prefix", @@ -284,6 +713,38 @@ class MV_Settings(PropertyGroup): default=math.radians(30.0), min=0.0, max=math.radians(180.0), subtype='ANGLE', ) + scale_mode: EnumProperty( + name="Scale", + items=[('FIT', "Fit Each", "Every product is scaled to Target Size, so all of them " + "fill the frame the same"), + ('RELATIVE', "Keep Proportions", "The biggest product is scaled to Target " + "Size and the rest keep their real size " + "relative to it")], + default='FIT', + ) + rest_on_floor: BoolProperty( + name="Rest on Floor", + description="Recenter puts the bottom of each product at the origin instead of " + "its middle, so it stands on the ground", + default=False, + ) + mesh_units: EnumProperty( + name="Mesh Units", + description="Unit the numbers in STL, OBJ, PLY, glTF and FBX files are in. Only " + "used to report real sizes and to scale products against each " + "other; STEP, IGES and 3MF know their own", + items=UNIT_ITEMS, + default='mm', + ) + orient_method: EnumProperty( + name="Method", + items=[('FLAT', "Lay Flat", "Rest the product on its largest flat face, longest " + "side across the Front view"), + ('PCA', "Principal Axes", "Line the product's longest spread up across the " + "Front view and its thinnest upright. Better for " + "rounded shapes with no flat face")], + default='FLAT', + ) fit_on_import: BoolProperty( name="Fit on Import", description="Straight after importing, scale each product to fit inside a cube of " @@ -295,8 +756,9 @@ class MV_Settings(PropertyGroup): active_product_index: IntProperty( name="Active Product", - description="Which product the orientation buttons act on", + description="Which product the orientation buttons and Render Active act on", default=0, min=0, + update=_update_active_product, ) isolate_active: BoolProperty( name="Isolate Active Product", @@ -363,10 +825,16 @@ def stray_render_objects(context): products = set() for pc in get_products(): products.update(pc.all_objects) + # The ground is the add-on's own, placed under each product on purpose. + stage = bpy.data.collections.get(STAGE_COLL) + if stage: + products.update(stage.all_objects) strays = [] for obj in context.view_layer.objects: - if obj.type not in GEOMETRY_TYPES or obj in products or obj.hide_render: + # Straight after objects are deleted the view layer can still list a + # slot for them, empty until it next resyncs. + if obj is None or obj.type not in GEOMETRY_TYPES or obj in products or obj.hide_render: continue # An object linked into several collections still renders as long as # one of them is on, so this takes all() rather than any(). @@ -435,6 +903,8 @@ def clear_collection_objects(coll): bpy.data.cameras.remove(data) elif isinstance(data, bpy.types.Light): bpy.data.lights.remove(data) + elif isinstance(data, bpy.types.Mesh): + bpy.data.meshes.remove(data) def collection_meshes(coll): @@ -448,13 +918,15 @@ def collection_roots(coll): return [o for o in members if o.parent is None or o.parent not in members] -def fit_collection(coll, target_size=None, recenter=True): +def fit_collection(coll, target_size=None, recenter=True, floor=False): """Uniformly scale a product so its bounding box fits inside a cube of target_size, and optionally move that box onto the world origin. - Pass target_size=None to only recenter. Returns False if the collection - holds no geometry. The whole product is treated as one rigid assembly, so - parts keep their relative positions. + Pass target_size=None to only recenter. With floor, recentering puts the + bottom of the box on the origin rather than its middle, so the product + stands on the ground plane instead of straddling it. Returns False if the + collection holds no geometry. The whole product is treated as one rigid + assembly, so parts keep their relative positions. """ objs = collection_meshes(coll) if not objs: @@ -467,7 +939,12 @@ def fit_collection(coll, target_size=None, recenter=True): if max_dim > 1e-9: factor = target_size / max_dim - dest = Vector((0, 0, 0)) if recenter else center + if not recenter: + dest = center + elif floor: + dest = Vector((0.0, 0.0, size.z * factor * 0.5)) + else: + dest = Vector((0.0, 0.0, 0.0)) if abs(factor - 1.0) < 1e-9 and (dest - center).length < 1e-9: return True @@ -479,6 +956,256 @@ def fit_collection(coll, target_size=None, recenter=True): return True +def record_source_size(coll, units): + """Remember how big a product really is, before anything rescales it. + + Called at import, and by Standardize for products assembled by hand, + which are measured the first time they are rescaled. Never overwrites a + size already recorded, since by then the product may have been scaled. + """ + if any(coll.mv_source_size): + return + meshes = collection_meshes(coll) + if meshes: + coll.mv_source_size = bbox_of(meshes)[1] + coll.mv_source_units = units + + +def real_extent_mm(coll): + """Largest dimension of the product as imported, in millimetres, or None.""" + size = max(coll.mv_source_size) + if size <= 0: + return None + return size * UNIT_TO_MM.get(coll.mv_source_units, 1.0) + + +def format_real_size(coll): + size = coll.mv_source_size + if not any(size): + return None + unit = coll.mv_source_units or "units" + return " x ".join(f"{v:.4g}" for v in size) + f" {unit}" + + +def scale_targets(settings, products): + """The size each product's largest dimension should be scaled to. + + Fit Each gives every product Target Size, which is what makes a screw and + a housing read the same in the renders. Keep Proportions gives the batch + one scale instead: the biggest product gets Target Size and the rest keep + their real size relative to it, so a screw stays a fraction of a housing. + A product with no recorded size cannot be placed on that scale and falls + back to Target Size. + """ + target = settings.target_size + if settings.scale_mode != 'RELATIVE': + return {pc.name: target for pc in products} + extents = {pc.name: real_extent_mm(pc) for pc in products} + biggest = max((e for e in extents.values() if e), default=None) + return {name: (target * e / biggest if e and biggest else target) + for name, e in extents.items()} + + +def standardize_products(context, products, scale=True, recenter=True): + """Scale and/or recenter products and bake the result. Returns the names + of objects whose transform could not be baked.""" + s = context.scene.mv_settings + for pc in products: + # Hand-built products are in Blender units, which are metres. + record_source_size(pc, "m") + targets = scale_targets(s, products) if scale else {} + unbaked = set() + context.view_layer.update() + for pc in products: + if fit_collection(pc, targets.get(pc.name), recenter=recenter, floor=s.rest_on_floor): + _, skipped = bake_product(pc, apply_scale=scale, origin_to_world=recenter) + unbaked.update(skipped) + context.view_layer.update() + return unbaked + + +# -- orientation ------------------------------------------------------------ + +HOME_KEY = "mv_home_rotation" + + +def remember_home(coll): + """Record each root's rotation as the product's home orientation, unless + one is recorded already. Reset Orientation returns the product to it.""" + roots = collection_roots(coll) + if any(HOME_KEY in o for o in roots): + return + for obj in roots: + obj[HOME_KEY] = list(obj.matrix_world.to_quaternion()) + + +def turn_product(coll, rotation): + """Rotate a product rigidly about its own bounding box centre.""" + objs = collection_meshes(coll) + if not objs: + return False + center, _ = bbox_of(objs) + xform = (Matrix.Translation(center) @ rotation.to_4x4() @ Matrix.Translation(-center)) + for obj in collection_roots(coll): + obj.matrix_world = xform @ obj.matrix_world + return True + + +def reset_orientation(coll): + """Turn a product back to the orientation it had when imported. False if + no home was ever recorded for it.""" + roots = [o for o in collection_roots(coll) if HOME_KEY in o] + if not roots: + return False + home = Quaternion(roots[0][HOME_KEY]) + now = roots[0].matrix_world.to_quaternion() + # Every root turned together, so undoing one root's turn undoes them all. + return turn_product(coll, (home @ now.inverted()).to_matrix()) + + +def world_triangles(coll): + """(n, 3, 3) array of the product's triangles in world space.""" + chunks = [] + for obj in collection_meshes(coll): + if obj.type != 'MESH' or not len(obj.data.polygons): + continue + me = obj.data + me.calc_loop_triangles() + co = np.empty(len(me.vertices) * 3, np.float64) + me.vertices.foreach_get("co", co) + idx = np.empty(len(me.loop_triangles) * 3, np.int64) + me.loop_triangles.foreach_get("vertices", idx) + m = np.array(obj.matrix_world) + world = co.reshape(-1, 3) @ m[:3, :3].T + m[:3, 3] + chunks.append(world[idx.reshape(-1, 3)]) + return np.concatenate(chunks) if chunks else None + + +def front_axes(settings): + """World directions of the Front view: across it, and towards its camera.""" + angle = FRONT_AXIS_ANGLES[settings.front_axis] + return (np.array(rotate_dir_z((1.0, 0.0, 0.0), angle)), + np.array(rotate_dir_z((0.0, -1.0, 0.0), angle))) + + +def sample_points(tris, limit=20000): + pts = tris.reshape(-1, 3) + if len(pts) > limit: + pts = pts[np.linspace(0, len(pts) - 1, limit).astype(np.int64)] + return pts + + +def flat_rotation(tris): + """Rotation that lays the product on its largest flat face. + + Triangles are grouped by the way they face, to within a few degrees, and + the group with the most area wins: for a box that is a large side, for a + phone its back. That face is turned to point straight down. + """ + cross = np.cross(tris[:, 1] - tris[:, 0], tris[:, 2] - tris[:, 0]) + area = np.linalg.norm(cross, axis=1) + keep = area > 1e-12 + normals = cross[keep] / area[keep, None] + area = area[keep] + if not len(area): + return Matrix.Identity(3) + keys, inverse = np.unique(np.round(normals * 24).astype(np.int64), axis=0, + return_inverse=True) + inverse = inverse.reshape(-1) + best = np.argmax(np.bincount(inverse, weights=area)) + n = (normals[inverse == best] * area[inverse == best, None]).sum(axis=0) + n /= max(np.linalg.norm(n), 1e-12) + return Vector(n).rotation_difference(Vector((0.0, 0.0, -1.0))).to_matrix() + + +def convex_hull_2d(points): + """Andrew's monotone chain, counter-clockwise, no repeated end point.""" + pts = [tuple(p) for p in np.unique(np.round(points, 9), axis=0)] + if len(pts) < 3: + return np.array(pts) + + def half(seq): + chain = [] + for p in seq: + while len(chain) >= 2 and ( + (chain[-1][0] - chain[-2][0]) * (p[1] - chain[-2][1]) + - (chain[-1][1] - chain[-2][1]) * (p[0] - chain[-2][0])) <= 0: + chain.pop() + chain.append(p) + return chain + + lower, upper = half(pts), half(reversed(pts)) + return np.array(lower[:-1] + upper[:-1]) + + +def footprint_rotation(points, across): + """Turn about the vertical so the product's footprint lines up with the + axes, longest side across the Front view. + + The smallest rectangle around a shape always has a side along one of its + convex hull's edges, so only those angles need trying, and the answer is + exact rather than the nearest step of a sweep. + """ + xy = points[:, :2] - points[:, :2].mean(axis=0) + hull = convex_hull_2d(xy) + if len(hull) < 3: + hull = xy + edges = np.roll(hull, -1, axis=0) - hull + candidates = np.unique(np.round(np.mod(-np.arctan2(edges[:, 1], edges[:, 0]), + math.pi / 2), 9)) + best_angle, best_area = 0.0, float("inf") + for angle in candidates: + c, s = math.cos(angle), math.sin(angle) + x = hull[:, 0] * c - hull[:, 1] * s + y = hull[:, 0] * s + hull[:, 1] * c + area = np.ptp(x) * np.ptp(y) + if area < best_area - 1e-12: + best_angle, best_area = float(angle), area + turn = Matrix.Rotation(best_angle, 3, 'Z') + turned = np.asarray(xy) @ np.array(turn)[:2, :2].T + # Extent along the direction across the Front view, against the other way. + along = np.ptp(turned @ across[:2]) + other = np.ptp(turned @ np.array((-across[1], across[0]))) + if along < other: + turn = Matrix.Rotation(best_angle + math.pi / 2, 3, 'Z') + return turn + + +def principal_rotation(points, across, towards): + """Rotation that puts the product's longest spread across the Front view, + its next longest pointing at the Front camera, and its thinnest upright.""" + centered = points - points.mean(axis=0) + _, vectors = np.linalg.eigh(np.cov(centered.T)) + small, middle, large = vectors[:, 0], vectors[:, 1], vectors[:, 2] + source = np.column_stack([large, middle, small]) + target = np.column_stack([across, towards, (0.0, 0.0, 1.0)]) + rot = target @ source.T + if np.linalg.det(rot) < 0: + source[:, 1] *= -1 + rot = target @ source.T + return Matrix(rot.tolist()) + + +def auto_orient(context, coll, method): + """Turn a product to a sensible resting orientation. False if it has no + mesh to measure.""" + tris = world_triangles(coll) + if tris is None or not len(tris): + return False + remember_home(coll) + across, towards = front_axes(context.scene.mv_settings) + points = sample_points(tris) + if method == 'PCA': + rot = principal_rotation(points, across, towards) + else: + lay = flat_rotation(tris) + laid = points @ np.array(lay).T + rot = footprint_rotation(laid, across) @ lay + turn_product(coll, rot) + context.view_layer.update() + return True + + def parent_depth(obj): depth, parent = 0, obj.parent while parent is not None: @@ -764,36 +1491,45 @@ def safe_name(name, fallback="unnamed"): return cleaned or fallback -STL_EXTENSIONS = {".stl"} - -# Blender 4.2 replaced the Python STL importer with a C++ one under a new name -# and retired the old operator soon after, so both are tried in turn. Forward -# Y / up Z is the identity mapping and is what CAD writes; it is passed -# explicitly rather than trusting an importer default to stay put. -STL_OPERATORS = ( - ("wm.stl_import", {"forward_axis": 'Y', "up_axis": 'Z'}), - ("import_mesh.stl", {"axis_forward": 'Y', "axis_up": 'Z'}), -) - - -def is_stl(path): - return os.path.splitext(path)[1].lower() in STL_EXTENSIONS +STEP_EXTENSIONS = {".step", ".stp"} +IGES_EXTENSIONS = {".iges", ".igs"} +CAD_EXTENSIONS = STEP_EXTENSIONS | IGES_EXTENSIONS + +# Each format lists its importers newest first. Blender has renamed several +# of them (4.2 replaced the Python STL importer with a C++ one under a new +# name), and calling an operator Blender does not have raises rather than +# returning, so a missing name just falls through to the next. +# +# Axes are spelled out only for STL and PLY, which have no up axis of their +# own and come out of CAD Z-up: forward Y / up Z is the identity mapping, and +# it is passed explicitly rather than trusting an importer default to stay +# put. OBJ, glTF and FBX carry or imply their own up axis, and the importers' +# defaults already convert it. +MESH_IMPORTERS = { + ".stl": (("wm.stl_import", {"forward_axis": 'Y', "up_axis": 'Z'}), + ("import_mesh.stl", {"axis_forward": 'Y', "axis_up": 'Z'})), + ".ply": (("wm.ply_import", {"forward_axis": 'Y', "up_axis": 'Z'}), + ("import_mesh.ply", {})), + ".obj": (("wm.obj_import", {}), ("import_scene.obj", {})), + ".gltf": (("import_scene.gltf", {}),), + ".glb": (("import_scene.gltf", {}),), + ".fbx": (("import_scene.fbx", {}),), +} +MESH_EXTENSIONS = set(MESH_IMPORTERS) | {".3mf"} +SUPPORTED_EXTENSIONS = CAD_EXTENSIONS | MESH_EXTENSIONS +FILE_FILTER = ";".join(f"*{e};*{e.upper()}" for e in sorted(SUPPORTED_EXTENSIONS)) +# Imported alongside the geometry by glTF and FBX, but a camera or lamp inside +# a product would light or frame every render of it. +NON_PRODUCT_TYPES = {'CAMERA', 'LIGHT', 'LIGHT_PROBE', 'SPEAKER'} -def import_stl_file(path): - """Import an STL straight into the scene. True only on a real success. +def file_ext(path): + return os.path.splitext(path)[1].lower() - An STL already holds a mesh, so it needs neither STEPper nor the FreeCAD - round trip a STEP file goes through. Everything after this point -- the - product collection, fitting, standardizing, rendering -- treats what lands - here exactly like a converted STEP file. - Calling an operator Blender does not have raises rather than returning, so - a missing name falls through to the next candidate; a genuine read failure - exhausts the list and reports False, which keeps one bad file from taking - the rest of the batch down with it. - """ - for op_path, axes in STL_OPERATORS: +def run_importer(candidates, path): + """True only on a real success from the first importer Blender has.""" + for op_path, axes in candidates: module, func = op_path.split(".", 1) mod = getattr(bpy.ops, module, None) op = getattr(mod, func, None) if mod else None @@ -809,6 +1545,170 @@ def import_stl_file(path): return False +def import_mesh_file(path): + """Import a mesh file straight into the scene: (success, units or None). + + A mesh needs neither STEPper nor the FreeCAD round trip a STEP file goes + through. Everything after this point -- the product collection, fitting, + standardizing, rendering -- treats what lands here exactly like a + converted STEP file. Only 3MF says what unit it is in. + """ + ext = file_ext(path) + if ext == ".3mf": + return import_3mf(path) + return run_importer(MESH_IMPORTERS.get(ext, ()), path), None + + +def srgb_to_linear(value): + return value / 12.92 if value <= 0.04045 else ((value + 0.055) / 1.055) ** 2.4 + + +def color_material(rgb): + """One shared material per colour a CAD or 3MF file asked for. + + The colours arrive as display values, so they are linearized for Base + Color, and also set as the viewport colour so Solid mode shows them. + """ + rgb = tuple(max(0.0, min(1.0, float(c))) for c in rgb[:3]) + name = "MV_Color_%02X%02X%02X" % tuple(int(round(c * 255)) for c in rgb) + mat = bpy.data.materials.get(name) + if mat is not None: + return mat + mat = bpy.data.materials.new(name) + linear = tuple(srgb_to_linear(c) for c in rgb) + (1.0,) + mat.diffuse_color = linear + tree = node_tree_of(mat) + bsdf = next((n for n in tree.nodes if n.bl_idname == "ShaderNodeBsdfPrincipled"), None) + if bsdf is None: + tree.nodes.clear() + bsdf = tree.nodes.new("ShaderNodeBsdfPrincipled") + out = tree.nodes.new("ShaderNodeOutputMaterial") + out.location = (300, 0) + tree.links.new(bsdf.outputs[0], out.inputs["Surface"]) + bsdf.inputs["Base Color"].default_value = linear + return mat + + +# -- 3MF ---------------------------------------------------------------------- + +NS_3MF = "{http://schemas.microsoft.com/3dmanufacturing/core/2015/02}" +UNITS_3MF = {"micron": "um", "millimeter": "mm", "centimeter": "cm", + "inch": "in", "foot": "ft", "meter": "m"} + + +def matrix_3mf(text): + """A 3MF transform as a Blender matrix. + + 3MF writes twelve numbers for a row-vector matrix (point times matrix, + translation in the last row); Blender's are column vectors, so it is the + transpose. + """ + if not text: + return Matrix.Identity(4) + m = [float(v) for v in text.split()] + return Matrix(((m[0], m[3], m[6], m[9]), + (m[1], m[4], m[7], m[10]), + (m[2], m[5], m[8], m[11]), + (0.0, 0.0, 0.0, 1.0))) + + +def color_3mf(text): + text = (text or "").lstrip("#") + if len(text) < 6: + return None + try: + return tuple(int(text[i:i + 2], 16) / 255.0 for i in (0, 2, 4)) + except ValueError: + return None + + +def import_3mf(path): + """A small 3MF reader: meshes, components, build transforms and base + material colours. Blender has no 3MF importer of its own. + + Each mesh object the build places becomes a Blender object, positioned by + the build item's and any component's transforms. Vertex values are kept in + the file's unit, the same as an STL, so its size is recorded as is. + """ + import zipfile + import xml.etree.ElementTree as ET + + try: + with zipfile.ZipFile(path) as z: + model = "3D/3dmodel.model" + try: + rels = ET.fromstring(z.read("_rels/.rels")) + for rel in rels: + if rel.get("Type", "").endswith("/3dmodel"): + model = rel.get("Target", model).lstrip("/") + except KeyError: + pass + root = ET.fromstring(z.read(model)) + except (OSError, KeyError, zipfile.BadZipFile, ET.ParseError): + return False, None + + units = UNITS_3MF.get(root.get("unit", "millimeter"), "mm") + resources = root.find(NS_3MF + "resources") + build = root.find(NS_3MF + "build") + if resources is None or build is None: + return False, None + + colours = {} + for group in resources.findall(NS_3MF + "basematerials"): + for i, base in enumerate(group.findall(NS_3MF + "base")): + colours[(group.get("id"), str(i))] = color_3mf(base.get("displaycolor")) + objects = {o.get("id"): o for o in resources.findall(NS_3MF + "object")} + stem = os.path.splitext(os.path.basename(path))[0] + made = [] + + def make_mesh(node, matrix, name): + mesh = node.find(NS_3MF + "mesh") + verts = [(float(v.get("x")), float(v.get("y")), float(v.get("z"))) + for v in mesh.find(NS_3MF + "vertices")] + tris, keys = [], [] + default = (node.get("pid"), node.get("pindex")) + for t in mesh.find(NS_3MF + "triangles"): + tris.append((int(t.get("v1")), int(t.get("v2")), int(t.get("v3")))) + pid = t.get("pid", default[0]) + keys.append((pid, t.get("p1", default[1] if pid == default[0] else "0"))) + me = bpy.data.meshes.new(name) + me.from_pydata(verts, [], tris) + slots = [] + for key in keys: + if key not in slots: + slots.append(key) + if any(colours.get(k) for k in slots): + for key in slots: + me.materials.append(color_material(colours.get(key) or (0.8, 0.8, 0.8))) + me.polygons.foreach_set("material_index", [slots.index(k) for k in keys]) + me.update() + obj = bpy.data.objects.new(name, me) + bpy.context.scene.collection.objects.link(obj) + obj.matrix_world = matrix + made.append(obj) + + def place(obj_id, matrix, depth=0): + node = objects.get(obj_id) + if node is None or depth > 32: + return + name = node.get("name") or stem + if node.find(NS_3MF + "mesh") is not None: + make_mesh(node, matrix, name) + comps = node.find(NS_3MF + "components") + if comps is not None: + for comp in comps.findall(NS_3MF + "component"): + place(comp.get("objectid"), matrix @ matrix_3mf(comp.get("transform")), depth + 1) + + try: + for item in build.findall(NS_3MF + "item"): + place(item.get("objectid"), matrix_3mf(item.get("transform"))) + except (TypeError, ValueError, AttributeError): + for obj in made: + bpy.data.objects.remove(obj, do_unlink=True) + return False, None + return bool(made), units + + def has_geometry(objs): """True when at least one of these objects carries geometry worth keeping. @@ -869,39 +1769,284 @@ def try_stepper_import(filepath): return False -def freecad_prepare(step_path, freecad_exe, deflection): - """Write the conversion script for one STEP file. None if FreeCAD is not - configured. Nothing is launched yet.""" +STEP_ENTITY = re.compile(r"#(\d+)\s*=\s*([A-Z0-9_]+)\s*\((.*?)\)\s*;", re.S) +STEP_REF = re.compile(r"#(\d+)") +STEP_SOLIDS = {"MANIFOLD_SOLID_BREP", "BREP_WITH_VOIDS", "SHELL_BASED_SURFACE_MODEL", + "FACETED_BREP"} +STEP_SHELLS = {"CLOSED_SHELL", "OPEN_SHELL"} +STEP_FACES = {"ADVANCED_FACE", "FACE_SURFACE"} +STEP_REPS = {"ADVANCED_BREP_SHAPE_REPRESENTATION", "MANIFOLD_SURFACE_SHAPE_REPRESENTATION", + "SHAPE_REPRESENTATION", "FACETED_BREP_SHAPE_REPRESENTATION"} +STEP_STYLE_CHAIN = {"PRESENTATION_STYLE_ASSIGNMENT", "SURFACE_STYLE_USAGE", "SURFACE_SIDE_STYLE", + "SURFACE_STYLE_FILL_AREA", "FILL_AREA_STYLE", "FILL_AREA_STYLE_COLOUR", + "COLOUR_RGB", "DRAUGHTING_PRE_DEFINED_COLOUR"} +STEP_NEEDED = (STEP_SOLIDS | STEP_SHELLS | STEP_FACES | STEP_REPS | STEP_STYLE_CHAIN + | {"STYLED_ITEM", "OVER_RIDING_STYLED_ITEM", "SHAPE_DEFINITION_REPRESENTATION", + "SHAPE_REPRESENTATION_RELATIONSHIP", "PRODUCT_DEFINITION_SHAPE", + "PRODUCT_DEFINITION", "PRODUCT_DEFINITION_FORMATION", + "PRODUCT_DEFINITION_FORMATION_WITH_SPECIFIED_SOURCE", "PRODUCT"}) +STEP_NAMED_COLOURS = {"red": (1, 0, 0), "green": (0, 1, 0), "blue": (0, 0, 1), + "yellow": (1, 1, 0), "magenta": (1, 0, 1), "cyan": (0, 1, 1), + "black": (0, 0, 0), "white": (1, 1, 1)} + + +def step_string(args): + """The first quoted string in a STEP argument list, unescaped.""" + m = re.match(r"\s*'((?:[^']|'')*)'", args) + if not m: + return "" + text = m.group(1).replace("''", "'") + text = re.sub(r"\\X2\\((?:[0-9A-F]{4})+)\\X0\\", + lambda g: "".join(chr(int(g.group(1)[i:i + 4], 16)) + for i in range(0, len(g.group(1)), 4)), text) + return re.sub(r"\\X\\([0-9A-F]{2})", lambda g: chr(int(g.group(1), 16)), text) + + +def step_colors(path): + """Colours a STEP file gives its parts and faces, keyed by product name. + + {name: {"color": (r, g, b) or None, "faces": {face index: (r, g, b)}}} + + FreeCAD reads STEP colours only into its GUI, so a console conversion + loses every one of them. They are read here instead, from the text: each + STYLED_ITEM points at a solid or a face, a solid belongs to a shape + representation, and the representation leads through the product + definition to the PRODUCT whose name FreeCAD gives the part. Face indices + count through the solid's shells in file order, which is the order the + faces come out of FreeCAD in. + + Best effort: anything unexpected yields fewer colours, never an error. + """ + try: + with open(path, "r", encoding="utf-8", errors="replace") as f: + text = f.read() + except OSError: + return {} + if "STYLED_ITEM" not in text: + return {} + + ents = {} + for m in STEP_ENTITY.finditer(text): + if m.group(2) in STEP_NEEDED: + ents[int(m.group(1))] = (m.group(2), m.group(3)) + refs = {i: [int(r) for r in STEP_REF.findall(args)] for i, (_, args) in ents.items()} + + def kind(i): + return ents.get(i, ("", ""))[0] + + def colour_of(i, depth=0): + t = kind(i) + if t == "COLOUR_RGB": + nums = re.findall(r"[-+]?\d*\.?\d+(?:[eE][-+]?\d+)?", ents[i][1].split(",", 1)[-1]) + return tuple(float(v) for v in nums[-3:]) if len(nums) >= 3 else None + if t == "DRAUGHTING_PRE_DEFINED_COLOUR": + return STEP_NAMED_COLOURS.get(step_string(ents[i][1]).lower()) + if t not in STEP_STYLE_CHAIN or depth > 8: + return None + for r in refs[i]: + found = colour_of(r, depth + 1) + if found: + return found + return None + + # Solid -> its faces in order, and face -> (solid, index). + solid_faces, face_home = {}, {} + for i in ents: + if kind(i) not in STEP_SOLIDS: + continue + faces = [] + for shell in refs[i]: + if kind(shell) in STEP_SHELLS: + faces.extend(f for f in refs[shell] if kind(f) in STEP_FACES) + solid_faces[i] = faces + for n, face in enumerate(faces): + face_home.setdefault(face, (i, n)) + + # Solid -> product name, through representation and product definition. + rep_of = {} + for i in ents: + if kind(i) in STEP_REPS: + for item in refs[i]: + if kind(item) in STEP_SOLIDS: + rep_of.setdefault(item, i) + rep_links = {} + for i in ents: + if kind(i) == "SHAPE_REPRESENTATION_RELATIONSHIP" and len(refs[i]) >= 2: + a, b = refs[i][-2], refs[i][-1] + rep_links.setdefault(a, []).append(b) + rep_links.setdefault(b, []).append(a) + pds_of = {} + for i in ents: + if kind(i) == "SHAPE_DEFINITION_REPRESENTATION" and len(refs[i]) >= 2: + pds_of[refs[i][1]] = refs[i][0] + + def product_name(rep): + for candidate in [rep] + rep_links.get(rep, []): + pds = pds_of.get(candidate) + if pds is None or kind(pds) != "PRODUCT_DEFINITION_SHAPE": + continue + for pd in refs[pds]: + if kind(pd) != "PRODUCT_DEFINITION": + continue + for pdf in refs[pd]: + if kind(pdf).startswith("PRODUCT_DEFINITION_FORMATION"): + for product in refs[pdf]: + if kind(product) == "PRODUCT": + return step_string(ents[product][1]) + return None + + per_solid, per_face = {}, {} + # Plain styles first, so an over-riding style wins where both apply. + for t in ("STYLED_ITEM", "OVER_RIDING_STYLED_ITEM"): + for i in ents: + if kind(i) != t or not refs[i]: + continue + colour = next((c for c in (colour_of(r) for r in refs[i][:-1]) if c), None) + target = refs[i][-1] + if t == "OVER_RIDING_STYLED_ITEM" and len(refs[i]) >= 2: + target = next((r for r in refs[i] if kind(r) in STEP_SOLIDS | STEP_FACES), target) + if colour is None: + continue + if kind(target) in STEP_SOLIDS: + per_solid[target] = colour + elif kind(target) in STEP_SHELLS: + for solid, faces in solid_faces.items(): + if target in refs[solid]: + per_solid[solid] = colour + elif target in face_home: + per_face[target] = colour + + result = {} + for solid, faces in solid_faces.items(): + name = product_name(rep_of.get(solid)) + if name is None: + continue + entry = result.setdefault(name, {"colors": []}) + base = per_solid.get(solid) + entry["colors"].extend(per_face.get(face, base) for face in faces) + out = {} + for name, entry in result.items(): + colours = entry["colors"] + if not any(colours): + continue + if len(set(colours)) == 1: + out[name] = {"color": colours[0], "faces": {}} + else: + out[name] = {"color": None, + "faces": {str(n): c for n, c in enumerate(colours) if c}} + return out + + +# The script FreeCAD runs. It reads its settings from job.json beside it, so +# no path or name is ever pasted into code: a path holding an apostrophe -- +# C:\Users\Dan's Files\part.step -- once closed a string early and surfaced as +# a bare "conversion failed". +FREECAD_SCRIPT = r''' +import json, os, re, sys, traceback +import FreeCAD, Part, MeshPart, Import + +here = os.path.dirname(os.path.abspath(__file__)) +job = json.load(open(os.path.join(here, "job.json"), encoding="utf-8")) +Import.open(job["source"]) +doc = FreeCAD.ActiveDocument +colors = job.get("colors", {}) + + +def shaped(o): + return (o.isDerivedFrom("Part::Feature") and o.Shape and not o.Shape.isNull() + and len(o.Shape.Faces) > 0) + + +# Leaves only. A STEP assembly arrives as an App::Part container whose shape +# is all of its children combined, plus origin planes with shapes of their +# own; meshing everything with a shape doubled every part. +leaves = [o for o in doc.Objects if shaped(o) and not any(shaped(c) for c in o.OutList)] if doc else [] +if not leaves: + # Console FreeCAD opens IGES into an empty document. Reading the shape + # directly works for it, and for anything else Import.open passes over. + doc = FreeCAD.newDocument("MV") + whole = doc.addObject("Part::Feature", os.path.splitext(os.path.basename(job["source"]))[0]) + whole.Shape = Part.read(job["source"]) + leaves = [whole] if shaped(whole) else [] + + +def world_shape(o): + shape = o.Shape.copy() + try: + shape.Placement = o.getGlobalPlacement() + except Exception: + pass + return shape + + +def mesh(shape): + return MeshPart.meshFromShape(Shape=shape, LinearDeflection=job["deflection"], + AngularDeflection=0.5, Relative=False) + + +def colour_info(label): + if label in colors: + return colors[label] + return colors.get(re.sub(r"\d{3}$", "", label), {}) + + +try: + parts = [] + for i, o in enumerate(leaves): + shape = world_shape(o) + info = colour_info(o.Label) + base = tuple(info["color"]) if info.get("color") else None + face_colors = info.get("faces") or {} + groups = {} + if face_colors: + for n, face in enumerate(shape.Faces): + key = tuple(face_colors.get(str(n)) or base or ()) or None + groups.setdefault(key, []).append(face) + else: + groups[base] = None + files = [] + for g, (colour, faces) in enumerate(groups.items()): + piece = shape if faces is None else Part.makeCompound(faces) + name = "part_%04d_%d.obj" % (i, g) + m = mesh(piece) + if m.CountFacets: + m.write(os.path.join(here, name)) + files.append({"file": name, "color": list(colour) if colour else None}) + if files: + parts.append({"name": o.Label, "files": files}) + json.dump({"parts": parts}, open(os.path.join(here, "parts.json"), "w", encoding="utf-8")) +except Exception: + # Whatever went wrong, still hand back the geometry in one piece. + traceback.print_exc() + merged = None + for o in leaves: + try: + m = mesh(world_shape(o)) + except Exception as e: + print("mesh failed", o.Name, e) + continue + if merged is None: + merged = m + else: + merged.addMesh(m) + if merged is not None: + merged.write(os.path.join(here, "model.obj")) +''' + + +def freecad_prepare(source, freecad_exe, deflection, colors=None): + """Write the conversion job for one STEP or IGES file. None if FreeCAD is + not configured. Nothing is launched yet.""" if not freecad_exe or not os.path.isfile(freecad_exe): return None tmp_dir = tempfile.mkdtemp(prefix="mv_step_") - obj_out = os.path.join(tmp_dir, os.path.splitext(os.path.basename(step_path))[0] + ".obj") - # Paths go in as %r, not hand-escaped into '...'. A path holding an - # apostrophe -- C:\Users\Dan's Files\part.step -- closed the string early - # and the script died of a SyntaxError that surfaced as a bare "conversion - # failed". repr() handles quotes, backslashes and newlines in one step. - script = ( - "import Import, MeshPart, FreeCAD\n" - "Import.open(%r)\n" % step_path + - "doc = FreeCAD.ActiveDocument\n" - "meshes = []\n" - "for obj in doc.Objects:\n" - " if hasattr(obj, 'Shape') and obj.Shape and not obj.Shape.isNull():\n" - " try:\n" - " meshes.append(MeshPart.meshFromShape(Shape=obj.Shape, " - "LinearDeflection=%r, AngularDeflection=0.5, Relative=False))\n" % float(deflection) + - " except Exception as e:\n" - " print('mesh failed', obj.Name, e)\n" - "if meshes:\n" - " m = meshes[0]\n" - " for extra in meshes[1:]:\n" - " m.addMesh(extra)\n" - " m.write(%r)\n" % obj_out - ) + with open(os.path.join(tmp_dir, "job.json"), "w", encoding="utf-8") as f: + json.dump({"source": source, "deflection": float(deflection), + "colors": colors or {}}, f) script_path = os.path.join(tmp_dir, "convert.py") with open(script_path, "w", encoding="utf-8") as f: - f.write(script) - return {"tmp_dir": tmp_dir, "obj": obj_out, "script": script_path, + f.write(FREECAD_SCRIPT) + return {"tmp_dir": tmp_dir, "script": script_path, "log": os.path.join(tmp_dir, "freecad.log"), "exe": freecad_exe, "proc": None, "handle": None} @@ -932,7 +2077,13 @@ def freecad_running(job): def freecad_collect(job): - """Reap a finished job: (obj path or None, whatever FreeCAD printed).""" + """Reap a finished job: (result or None, whatever FreeCAD printed). + + The result is {"parts": [{"name", "files": [{"path", "color"}]}]}: one + entry per part, each split by colour. A conversion that could not split + the parts falls back to a single merged OBJ, which comes back as one + nameless part. + """ if job.get("handle") is not None: job["handle"].close() job["handle"] = None @@ -942,9 +2093,20 @@ def freecad_collect(job): log = f.read() except OSError: pass - obj = job["obj"] - if os.path.isfile(obj) and os.path.getsize(obj) > 0: - return obj, log + here = job["tmp_dir"] + try: + with open(os.path.join(here, "parts.json"), encoding="utf-8") as f: + parts = json.load(f)["parts"] + for part in parts: + for entry in part["files"]: + entry["path"] = os.path.join(here, entry["file"]) + if parts: + return {"parts": parts}, log + except (OSError, ValueError, KeyError, TypeError): + pass + merged = os.path.join(here, "model.obj") + if file_ok(merged): + return {"parts": [{"name": None, "files": [{"path": merged, "color": None}]}]}, log return None, log @@ -1013,6 +2175,39 @@ def import_obj_file(path): return False +def import_freecad_result(result): + """Bring a converted STEP or IGES into the scene, one object per part. + + A part with faces in several colours arrives as one OBJ per colour; those + are joined back into a single object with a material slot per colour, and + named after the part, so the outliner reads like the CAD tree. + """ + for part in result["parts"]: + pieces = [] + for entry in part["files"]: + before = set(bpy.data.objects) + if not import_obj_file(entry["path"]): + continue + new = [o for o in bpy.data.objects if o not in before and o.type == 'MESH'] + if entry.get("color"): + mat = color_material(entry["color"]) + for obj in new: + obj.data.materials.clear() + obj.data.materials.append(mat) + pieces.extend(new) + if not pieces: + continue + if len(pieces) > 1: + with bpy.context.temp_override(active_object=pieces[0], object=pieces[0], + selected_objects=pieces, + selected_editable_objects=pieces): + bpy.ops.object.join() + obj = pieces[0] + if part.get("name"): + obj.name = part["name"] + obj.data.name = part["name"] + + PERSP_LENS = 50.0 SENSOR_WIDTH = 36.0 @@ -1080,6 +2275,23 @@ def persp_distance(local, tan_x, tan_y, pad): return distance +def planned_views(settings): + """(name, kind, direction) for every view a camera should be built for. + + A preset is read from VIEW_SETS rather than from the checkboxes, so a file + saved before the checkboxes existed still gets exactly the views it chose. + """ + if settings.view_set == 'CUSTOM': + views = [v for v, on in zip(VIEWS_ALL, settings.views_enabled) if on] + else: + views = list(VIEW_SETS[settings.view_set]) + for cv in settings.custom_views: + if cv.enabled: + views.append((cv.name, "ortho" if cv.is_ortho else "persp", + (cv.dir_x, cv.dir_y, cv.dir_z))) + return views + + def build_cameras(context): settings = context.scene.mv_settings coll = ensure_collection(CAMERA_COLL) @@ -1103,10 +2315,7 @@ def build_cameras(context): tan_x, tan_y = sensor_tangents(rx, ry) angle = FRONT_AXIS_ANGLES[settings.front_axis] - views = list(VIEW_SETS[settings.view_set]) - for cv in settings.custom_views: - views.append((cv.name, "ortho" if cv.is_ortho else "persp", - (cv.dir_x, cv.dir_y, cv.dir_z))) + views = planned_views(settings) # Pass 1: measure every view and take the worst case, so one ortho framing # and one perspective framing cover the whole set. Sizing each camera to its @@ -1149,44 +2358,211 @@ def build_cameras(context): cam_data.lens = PERSP_LENS distance = persp_dist cam_data.clip_start = max(distance * 0.01, 1e-5) - cam_data.clip_end = (distance + diag) * 4.0 + # Far enough for the ground, which runs fifty product sizes out; any + # shorter and a perspective view sees the floor stop before the horizon. + cam_data.clip_end = (distance + diag) * 100.0 cam_obj = bpy.data.objects.new(f"MV_Cam_{name}", cam_data) cam_obj.matrix_world = look_at(center + d * distance, center) coll.objects.link(cam_obj) + # What this framing was measured from, for the pre-flight's stale check. + coll["mv_framing"] = framing_signature(context) return skipped +def batch_extent(): + """Centre and largest dimension of the whole batch, which is what the light + rig and the ground are sized from.""" + objs = [] + for pc in get_products(): + objs.extend(collection_meshes(pc)) + if not objs: + objs = [o for o in bpy.data.objects + if o.type == 'MESH' and o.name != GROUND_NAME] + center, size = bbox_of(objs) + return center, max(size.x, size.y, size.z, 1e-3) + + +def front_basis(settings, center): + """Rotation of the Front camera, which every preset is laid out against.""" + d = Vector(rotate_dir_z((0.0, -1.0, 0.0), FRONT_AXIS_ANGLES[settings.front_axis])) + return look_at(center + d, center).to_3x3() + + def build_lighting(context, preset_key): + """Rebuild MV_Lighting from a preset. False if the preset cannot be read. + + Every light hangs off one empty at the batch centre, MV_LightRig. Turning + that empty turns the whole rig about the product, which is how the lights + follow the camera without anything being rebuilt per view. + """ settings = context.scene.mv_settings + specs = lighting_specs(preset_key) + if specs is None: + return False coll = ensure_collection(LIGHT_COLL) clear_collection_objects(coll) - objs = [] - for pc in get_products(): - objs.extend(list(pc.all_objects)) - if not objs: - objs = [o for o in bpy.data.objects if o.type == 'MESH'] - center, size = bbox_of(objs) - max_dim = max(size.x, size.y, size.z, 1e-3) + center, max_dim = batch_extent() angle = FRONT_AXIS_ANGLES[settings.front_axis] + turn = Matrix.Rotation(math.radians(angle), 4, 'Z') intensity = settings.lighting_intensity - for spec in LIGHTING_PRESETS[preset_key]["lights"]: - ld = bpy.data.lights.new(f"MV_Light_{spec['name']}", type='AREA') - ld.size = spec["size"] * max_dim * 0.5 - ld.energy = spec["energy"] * max_dim * intensity - ld.color = spec["color"] - obj = bpy.data.objects.new(f"MV_Light_{spec['name']}", ld) - pos = Vector(rotate_dir_z(spec["pos"], angle)) * max_dim - obj.matrix_world = look_at(center + pos, center) + rig = bpy.data.objects.new(LIGHT_RIG, None) + rig.empty_display_type = 'SPHERE' + rig.empty_display_size = max_dim * 0.15 + rig.matrix_world = Matrix.Translation(center) + coll.objects.link(rig) + + for spec in specs: + kind = spec.get("type", 'AREA') + name = f"MV_Light_{spec.get('name', 'Light')}" + ld = bpy.data.lights.new(name, type=kind) + size = spec.get("size", 1.0) * max_dim * 0.5 + if kind == 'AREA': + ld.size = size + if "shape" in spec: + ld.shape = spec["shape"] + ld.size_y = spec.get("size_y", spec.get("size", 1.0)) * max_dim * 0.5 + elif kind in {'POINT', 'SPOT'}: + ld.shadow_soft_size = size + if kind == 'SPOT': + ld.spot_size = spec.get("spot_size", ld.spot_size) + ld.spot_blend = spec.get("spot_blend", ld.spot_blend) + if kind == 'SUN': + ld.angle = spec.get("angle", ld.angle) + # Sun strength is irradiance, which does not grow with the product. + ld.energy = spec["energy"] * intensity + else: + ld.energy = spec["energy"] * max_dim * intensity + ld.color = spec.get("color", (1.0, 1.0, 1.0)) + + obj = bpy.data.objects.new(name, ld) coll.objects.link(obj) + obj.parent = rig + pos = center + Vector(rotate_dir_z(spec["pos"], angle)) * max_dim + if "rot" in spec: + rot = turn @ Quaternion(spec["rot"]).to_matrix().to_4x4() + obj.matrix_world = Matrix.Translation(pos) @ rot + else: + obj.matrix_world = look_at(pos, center) + return True + + +def orient_light_rig(context, camera): + """Turn the rig so it stands to camera the way it stands to Front.""" + rig = bpy.data.objects.get(LIGHT_RIG) + if rig is None or camera is None: + return + center = rig.matrix_world.translation.copy() + turn = camera.matrix_world.to_3x3().normalized() @ front_basis( + context.scene.mv_settings, center).transposed() + rig.matrix_world = Matrix.Translation(center) @ turn.to_4x4() + + +def reset_light_rig(): + rig = bpy.data.objects.get(LIGHT_RIG) + if rig is not None: + rig.matrix_world = Matrix.Translation(rig.matrix_world.translation.copy()) + + +def capture_lighting(context): + """MV_Lighting's lights as preset specs, in the units presets are written in: + positions and sizes as multiples of the batch's size, laid out against the + Front view with Front Axis taken back out, energy before Intensity. + + Read back through build_lighting, a rig saved here comes out the same on a + product of any size and any front axis. + """ + settings = context.scene.mv_settings + coll = bpy.data.collections.get(LIGHT_COLL) + lights = [o for o in coll.objects if o.type == 'LIGHT'] if coll else [] + if not lights: + return [] + center, max_dim = batch_extent() + rig = bpy.data.objects.get(LIGHT_RIG) + # Measured against the rig rather than the world, so saving straight after + # a render, with the rig still turned for the last view, still works. + frame = rig.matrix_world.copy() if rig else Matrix.Translation(center) + untilt = Matrix.Rotation(-math.radians(FRONT_AXIS_ANGLES[settings.front_axis]), 4, 'Z') + intensity = settings.lighting_intensity or 1.0 + + specs = [] + for obj in lights: + local = untilt @ frame.inverted() @ obj.matrix_world + ld = obj.data + spec = { + "name": obj.name.replace("MV_Light_", "") or "Light", + "type": ld.type, + "pos": [round(v / max_dim, 5) for v in local.translation], + "rot": [round(v, 6) for v in local.to_quaternion().normalized()], + "color": [round(v, 5) for v in ld.color], + "energy": round(ld.energy / (intensity * (1.0 if ld.type == 'SUN' else max_dim)), 5), + } + if ld.type == 'AREA': + spec["size"] = round(ld.size / (max_dim * 0.5), 5) + spec["shape"] = ld.shape + spec["size_y"] = round(ld.size_y / (max_dim * 0.5), 5) + elif ld.type in {'POINT', 'SPOT'}: + spec["size"] = round(ld.shadow_soft_size / (max_dim * 0.5), 5) + if ld.type == 'SPOT': + spec["spot_size"] = round(ld.spot_size, 5) + spec["spot_blend"] = round(ld.spot_blend, 5) + if ld.type == 'SUN': + spec["angle"] = round(ld.angle, 5) + specs.append(spec) + return specs + + +def node_tree_of(idblock): + """The node tree of a material or world, switching nodes on only where + that is still a setting. Blender 5 always uses nodes and warns about the + flag, which it plans to remove.""" + if idblock.node_tree is None or (bpy.app.version < (5, 0, 0) and not idblock.use_nodes): + idblock.use_nodes = True + return idblock.node_tree + + +def build_ground(context): + """Rebuild MV_Stage's ground to match the settings, or clear it for None.""" + s = context.scene.mv_settings + stage = bpy.data.collections.get(STAGE_COLL) + if s.ground_mode == 'NONE': + if stage: + clear_collection_objects(stage) + return None + stage = ensure_collection(STAGE_COLL) + clear_collection_objects(stage) + + center, max_dim = batch_extent() + half = max_dim * 50.0 + mesh = bpy.data.meshes.new(GROUND_NAME) + mesh.from_pydata([(-half, -half, 0), (half, -half, 0), (half, half, 0), (-half, half, 0)], + [], [(0, 1, 2, 3)]) + mat = bpy.data.materials.get(GROUND_NAME) or bpy.data.materials.new(GROUND_NAME) + tree = node_tree_of(mat) + tree.nodes.clear() + bsdf = tree.nodes.new("ShaderNodeBsdfPrincipled") + out = tree.nodes.new("ShaderNodeOutputMaterial") + out.location = (300, 0) + tree.links.new(bsdf.outputs[0], out.inputs["Surface"]) + bsdf.inputs["Base Color"].default_value = (*s.ground_color, 1.0) + bsdf.inputs["Roughness"].default_value = s.ground_roughness + mesh.materials.append(mat) + + ground = bpy.data.objects.new(GROUND_NAME, mesh) + ground.location = (center.x, center.y, center.z) + ground.is_shadow_catcher = s.ground_mode == 'SHADOW' + stage.objects.link(ground) + return ground def apply_world(context): """Point the scene at the add-on's own world and rewrite that one. + Returns an error message, or None. + This used to reuse whatever world the scene already had and clear its node tree, so pressing Apply Background on a file with a hand-built HDRI setup destroyed it with no warning and no undo. The add-on now owns a single @@ -1195,26 +2571,150 @@ def apply_world(context): """ settings = context.scene.mv_settings scene = context.scene + image = None + if settings.world_mode == 'HDRI': + path = bpy.path.abspath(settings.hdri_path) + if not os.path.isfile(path): + return f"HDRI file not found: '{settings.hdri_path}'" + try: + image = bpy.data.images.load(path, check_existing=True) + except RuntimeError as exc: + return f"Cannot read HDRI '{path}': {exc}" + world = bpy.data.worlds.get(WORLD_NAME) or bpy.data.worlds.new(WORLD_NAME) scene.world = world - world.use_nodes = True - nodes = world.node_tree.nodes - links = world.node_tree.links + tree = node_tree_of(world) + nodes = tree.nodes + links = tree.links nodes.clear() - bg = nodes.new("ShaderNodeBackground") out = nodes.new("ShaderNodeOutputWorld") - bg.inputs["Color"].default_value = settings.bg_color - bg.inputs["Strength"].default_value = 1.0 - links.new(bg.outputs["Background"], out.inputs["Surface"]) + out.location = (600, 0) + flat = nodes.new("ShaderNodeBackground") + flat.location = (100, -200) + flat.inputs["Color"].default_value = settings.bg_color + flat.inputs["Strength"].default_value = 1.0 scene.render.film_transparent = settings.transparent_bg + if image is None: + links.new(flat.outputs["Background"], out.inputs["Surface"]) + return None -class MV_OT_AddStepFile(Operator, ImportHelper): - bl_idname = "mv.add_step_file" - bl_label = "Add CAD File" - filter_glob: StringProperty( - default="*.step;*.stp;*.STEP;*.STP;*.stl;*.STL", options={'HIDDEN'}, - ) + coords = nodes.new("ShaderNodeTexCoord") + coords.location = (-700, 150) + mapping = nodes.new("ShaderNodeMapping") + mapping.location = (-500, 150) + mapping.inputs["Rotation"].default_value[2] = settings.hdri_rotation + env = nodes.new("ShaderNodeTexEnvironment") + env.location = (-250, 150) + env.image = image + hdri = nodes.new("ShaderNodeBackground") + hdri.location = (100, 100) + hdri.inputs["Strength"].default_value = settings.hdri_strength + links.new(coords.outputs["Generated"], mapping.inputs["Vector"]) + links.new(mapping.outputs["Vector"], env.inputs["Vector"]) + links.new(env.outputs["Color"], hdri.inputs["Color"]) + if settings.hdri_visible: + links.new(hdri.outputs["Background"], out.inputs["Surface"]) + return None + + # The camera sees the flat colour; everything else (lighting, reflections) + # sees the HDRI. + path = nodes.new("ShaderNodeLightPath") + path.location = (100, 400) + mix = nodes.new("ShaderNodeMixShader") + mix.location = (350, 0) + links.new(path.outputs["Is Camera Ray"], mix.inputs["Fac"]) + links.new(hdri.outputs["Background"], mix.inputs[1]) + links.new(flat.outputs["Background"], mix.inputs[2]) + links.new(mix.outputs["Shader"], out.inputs["Surface"]) + return None + + +def product_material_choice(settings, coll): + """(preset, colour) a product renders with, or None to keep its own.""" + preset = getattr(coll, "mv_material", 'DEFAULT') + if preset == 'DEFAULT': + preset, color = settings.material_preset, settings.material_color + else: + color = coll.mv_material_color + if preset not in MATERIAL_PRESETS: + return None + return preset, tuple(color) + + +def preset_material(name, preset, color): + mat = bpy.data.materials.get(name) or bpy.data.materials.new(name) + tree = node_tree_of(mat) + tree.nodes.clear() + bsdf = tree.nodes.new("ShaderNodeBsdfPrincipled") + out = tree.nodes.new("ShaderNodeOutputMaterial") + out.location = (300, 0) + tree.links.new(bsdf.outputs[0], out.inputs["Surface"]) + bsdf.inputs["Base Color"].default_value = (*color, 1.0) + for key, value in MATERIAL_PRESETS[preset]["inputs"].items(): + socket = bsdf.inputs.get(key) + if socket is not None: + socket.default_value = value + return mat + + +def material_node_group(): + """A Geometry Nodes group that sets one material on everything. + + A modifier carrying it swaps the material for rendering without touching + the product's own material slots, so removing the modifier is a complete + undo, and a mesh with no slots at all (every STL) still gets the material. + """ + group = bpy.data.node_groups.get(MATERIAL_GROUP) + if group is not None and group.bl_idname == 'GeometryNodeTree': + return group + group = bpy.data.node_groups.new(MATERIAL_GROUP, 'GeometryNodeTree') + group.interface.new_socket("Geometry", in_out='INPUT', socket_type='NodeSocketGeometry') + group.interface.new_socket("Material", in_out='INPUT', socket_type='NodeSocketMaterial') + group.interface.new_socket("Geometry", in_out='OUTPUT', socket_type='NodeSocketGeometry') + gin = group.nodes.new("NodeGroupInput") + gout = group.nodes.new("NodeGroupOutput") + gout.location = (400, 0) + setter = group.nodes.new("GeometryNodeSetMaterial") + setter.location = (200, 0) + group.links.new(gin.outputs[0], setter.inputs["Geometry"]) + group.links.new(gin.outputs[1], setter.inputs["Material"]) + group.links.new(setter.outputs["Geometry"], gout.inputs[0]) + return group + + +def apply_product_material(settings, coll): + """Give a product its preset material for rendering. False when it keeps its own.""" + choice = product_material_choice(settings, coll) + if choice is None: + return False + mat = preset_material(f"MV_Material_{coll.name}", *choice) + group = material_node_group() + socket = next(item.identifier for item in group.interface.items_tree + if getattr(item, "in_out", None) == 'INPUT' + and getattr(item, "socket_type", None) == 'NodeSocketMaterial') + for obj in collection_meshes(coll): + if obj.type != 'MESH' or obj.library is not None: + continue + mod = obj.modifiers.new(MATERIAL_MOD, 'NODES') + mod.node_group = group + mod[socket] = mat + return True + + +def remove_product_materials(): + """Take every preset material back off, wherever it was left.""" + for obj in bpy.data.objects: + for mod in [m for m in obj.modifiers + if m.type == 'NODES' and m.name.startswith(MATERIAL_MOD)]: + obj.modifiers.remove(mod) + + +class MV_OT_AddStepFile(Operator, ImportHelper): + """Add CAD or mesh files to the import list""" + bl_idname = "mv.add_step_file" + bl_label = "Add Files" + filter_glob: StringProperty(default=FILE_FILTER, options={'HIDDEN'}) files: CollectionProperty(type=bpy.types.OperatorFileListElement) directory: StringProperty(subtype='DIR_PATH') @@ -1253,12 +2753,64 @@ def execute(self, context): return {'FINISHED'} +class MV_OT_AddFolder(Operator): + """Add every CAD and mesh file in a folder to the import list""" + bl_idname = "mv.add_folder" + bl_label = "Add Folder" + + directory: StringProperty(subtype='DIR_PATH') + filter_folder: BoolProperty(default=True, options={'HIDDEN'}) + recursive: BoolProperty( + name="Include Subfolders", + description="Also add files from every folder inside this one", + default=True, + ) + + def invoke(self, context, event): + context.window_manager.fileselect_add(self) + return {'RUNNING_MODAL'} + + def execute(self, context): + s = context.scene.mv_settings + folder = bpy.path.abspath(self.directory) + if not os.path.isdir(folder): + self.report({'ERROR'}, f"Not a folder: '{folder}'") + return {'CANCELLED'} + found = [] + for current, dirs, files in os.walk(folder): + dirs.sort() + found.extend(os.path.join(current, f) for f in sorted(files) + if file_ext(f) in SUPPORTED_EXTENSIONS) + if not self.recursive: + break + # A folder added twice, or overlapping one added before, would import + # the same file twice as two products. + listed = {os.path.normcase(os.path.abspath(bpy.path.abspath(i.path))) + for i in s.step_files} + added = 0 + for path in found: + key = os.path.normcase(os.path.abspath(path)) + if key in listed: + continue + listed.add(key) + s.step_files.add().path = path + added += 1 + if not found: + self.report({'WARNING'}, "No supported files in that folder.") + else: + skipped = len(found) - added + self.report({'INFO'}, f"Added {added} file(s)" + + (f", {skipped} already in the list." if skipped else ".")) + return {'FINISHED'} + + class MV_OT_ImportAllSteps(Operator): """Import every file in the list, one at a time. - STEP files are converted first, by STEPper or by FreeCAD. STL files are - meshes already, so they skip both and go straight through Blender's own - importer; from the product collection onwards the two are indistinguishable. + STEP files are converted first, by STEPper or by FreeCAD, and IGES by + FreeCAD. Mesh files (STL, OBJ, PLY, glTF, FBX, 3MF) skip both and go + straight through Blender's own importers; from the product collection + onwards they are all indistinguishable. Runs modally: the conversion subprocess is polled between timer ticks instead of being waited on, so Blender keeps redrawing, the status bar @@ -1266,7 +2818,7 @@ class MV_OT_ImportAllSteps(Operator): falls back to running straight through. """ bl_idname = "mv.import_all_steps" - bl_label = "Import CAD Files" + bl_label = "Import Files" bl_options = {'REGISTER', 'UNDO'} def _setup(self, context): @@ -1287,6 +2839,7 @@ def _setup(self, context): self._before = set() self._timer = None self._cancelled = False + self._rescale_batch = False return True # -- one file --------------------------------------------------------- @@ -1307,7 +2860,7 @@ def _discard(self, objs): except (ReferenceError, RuntimeError): pass - def _link_new(self, context, path, new_objs): + def _link_new(self, context, path, new_objs, units): # Always a fresh collection. Looking the name up first meant two # files sharing a basename -- partA/housing.step and partB/housing.step # -- landed in one collection and rendered as a single product, and an @@ -1323,15 +2876,41 @@ def _link_new(self, context, path, new_objs): c.objects.unlink(obj) product_coll.objects.link(obj) + # Measured before fitting, which is the last moment the real size is + # known. The manifest reports it next to the renders. + context.view_layer.update() s = context.scene.mv_settings + product_coll.mv_source = path + record_source_size(product_coll, units or s.mesh_units) + if s.fit_on_import: - context.view_layer.update() - if fit_collection(product_coll, s.target_size, recenter=True): + if s.scale_mode == 'RELATIVE': + # The shared scale depends on the biggest product, which may + # not have arrived yet; the whole batch is scaled together at + # the end instead. + fit_collection(product_coll, None, recenter=True, floor=s.rest_on_floor) + self._rescale_batch = True + elif fit_collection(product_coll, s.target_size, recenter=True, + floor=s.rest_on_floor): _, unbaked = bake_product(product_coll) self._unbaked.update(unbaked) + context.view_layer.update() + remember_home(product_coll) self._imported += 1 self._done += 1 + def _import_ready_file(self, context, path, units, what): + """Link what a finished import added, or reject it as empty.""" + new = self._new_objects() + strays = [o for o in new if o.type in NON_PRODUCT_TYPES] + self._discard(strays) + new = [o for o in new if o not in strays] + if has_geometry(new): + self._link_new(context, path, new, units) + else: + self._discard(new) + self._fail(path, f"{what} contained no geometry") + def _new_objects(self): return [o for o in bpy.data.objects if o not in self._before] @@ -1341,28 +2920,33 @@ def _start_file(self, context, path): self._fail(path, "file not found") return self._before = set(bpy.data.objects) + ext = file_ext(path) + label = ext.lstrip(".").upper() - if is_stl(path): + if ext in MESH_EXTENSIONS: # Nothing to convert, so no importer preference applies and no # subprocess is started: the file lands now and _advance moves on # to the next one on the same tick. - if not import_stl_file(path): - self._fail(path, "STL import failed") + ok, units = import_mesh_file(path) + if not ok: + self._discard(self._new_objects()) + self._fail(path, f"{label} import failed") return - new = self._new_objects() - if has_geometry(new): - self._link_new(context, path, new) - else: - self._discard(new) - self._fail(path, "STL contained no geometry") + self._import_ready_file(context, path, units, label) + return + if ext not in CAD_EXTENSIONS: + self._fail(path, "not a supported file type") return mode = self._prefs.step_import_mode + is_step = ext in STEP_EXTENSIONS - if mode in {'AUTO', 'STEPPER'} and try_stepper_import(path): + if is_step and mode in {'AUTO', 'STEPPER'} and try_stepper_import(path): new = self._new_objects() if has_geometry(new): - self._link_new(context, path, new) + # STEPper does its own unit handling, so the size it lands at + # is not known to be in any particular unit. + self._link_new(context, path, new, "") return self._discard(new) if mode == 'STEPPER': @@ -1370,11 +2954,13 @@ def _start_file(self, context, path): return if mode == 'STEPPER': - self._fail(path, "STEPper import failed") + self._fail(path, "STEPper import failed" if is_step + else "IGES needs FreeCAD; STEPper reads STEP only") return job = freecad_prepare(path, self._prefs.freecad_path, - self._prefs.tessellation_deflection) + self._prefs.tessellation_deflection, + step_colors(path) if is_step else None) if job is None: self._fail(path, "no importer available (set the FreeCAD path in preferences)") return @@ -1385,14 +2971,11 @@ def _start_file(self, context, path): def _collect_job(self, context): job, self._job = self._job, None path = self._job_path - obj_path, log = freecad_collect(job) - if obj_path and import_obj_file(obj_path): - new = self._new_objects() - if has_geometry(new): - self._link_new(context, path, new) - else: - self._discard(new) - self._fail(path, "converted mesh contained no geometry") + result, log = freecad_collect(job) + if result: + import_freecad_result(result) + # FreeCAD tessellates in millimetres whatever the file was drawn in. + self._import_ready_file(context, path, "mm", "converted mesh") else: tail = " ".join(log.split())[-160:] self._fail(path, "FreeCAD conversion failed" @@ -1437,6 +3020,11 @@ def _status(self): return f"MultiView: {self._done}/{self._total} {name}{waiting} - Esc to cancel" def _report_result(self): + if self._rescale_batch: + self._rescale_batch = False + unbaked = standardize_products(bpy.context, get_products(), scale=True, + recenter=False) + self._unbaked.update(unbaked) if self._cancelled: level, msg = {'WARNING'}, f"Import cancelled after {self._imported} file(s)." elif not self._imported: @@ -1514,11 +3102,19 @@ class MV_OT_ApplySceneSetup(Operator): def execute(self, context): s = context.scene.mv_settings - apply_world(context) - build_lighting(context, s.lighting_preset) + problems = [] + error = apply_world(context) + if error: + problems.append(error) + if not build_lighting(context, s.lighting_preset): + problems.append("Lighting preset could not be read.") + build_ground(context) if get_products(): build_cameras(context) - self.report({'INFO'}, "Lighting, background, and cameras updated.") + if problems: + self.report({'WARNING'}, " ".join(problems)) + else: + self.report({'INFO'}, "Lighting, background, ground and cameras updated.") return {'FINISHED'} @@ -1527,6 +3123,9 @@ class MV_OT_BuildCameras(Operator): bl_label = "Build Cameras" def execute(self, context): + if not planned_views(context.scene.mv_settings): + self.report({'ERROR'}, "No views selected. Tick at least one view.") + return {'CANCELLED'} skipped = build_cameras(context) if skipped: self.report({'WARNING'}, @@ -1540,7 +3139,9 @@ class MV_OT_BuildLighting(Operator): bl_label = "Rebuild Lighting" def execute(self, context): - build_lighting(context, context.scene.mv_settings.lighting_preset) + if not build_lighting(context, context.scene.mv_settings.lighting_preset): + self.report({'ERROR'}, "Lighting preset could not be read.") + return {'CANCELLED'} return {'FINISHED'} @@ -1549,7 +3150,69 @@ class MV_OT_ApplyWorld(Operator): bl_label = "Apply Background" def execute(self, context): - apply_world(context) + error = apply_world(context) + if error: + self.report({'ERROR'}, error) + return {'CANCELLED'} + build_ground(context) + return {'FINISHED'} + + +class MV_OT_SaveLightingPreset(Operator): + """Save the lights in MV_Lighting, including any you have added or moved, + as a preset available in every file""" + bl_idname = "mv.save_lighting_preset" + bl_label = "Save Lighting Preset" + + name: StringProperty(name="Name", default="My Studio") + + def invoke(self, context, event): + return context.window_manager.invoke_props_dialog(self) + + def execute(self, context): + specs = capture_lighting(context) + if not specs: + self.report({'ERROR'}, f"No lights in '{LIGHT_COLL}' to save.") + return {'CANCELLED'} + stem = safe_name(self.name, fallback="Preset") + folder = user_preset_dir(create=True) + path = os.path.join(folder, stem + ".json") + try: + with open(path, "w", encoding="utf-8") as f: + json.dump({"label": self.name, "lights": specs}, f, indent=2) + except OSError as exc: + self.report({'ERROR'}, f"Could not save preset: {exc}") + return {'CANCELLED'} + context.scene.mv_settings.lighting_preset = USER_PRESET_PREFIX + stem + self.report({'INFO'}, f"Saved {len(specs)} light(s) as '{stem}'.") + return {'FINISHED'} + + +class MV_OT_DeleteLightingPreset(Operator): + """Delete the selected saved lighting preset. Built-in presets cannot be deleted""" + bl_idname = "mv.delete_lighting_preset" + bl_label = "Delete Lighting Preset" + + @classmethod + def poll(cls, context): + return context.scene.mv_settings.lighting_preset.startswith(USER_PRESET_PREFIX) + + def invoke(self, context, event): + return context.window_manager.invoke_confirm(self, event) + + def execute(self, context): + s = context.scene.mv_settings + entry = user_lighting_presets().get(s.lighting_preset) + if entry is None: + self.report({'ERROR'}, "That preset is not on disk any more.") + return {'CANCELLED'} + try: + os.remove(entry[1]) + except OSError as exc: + self.report({'ERROR'}, f"Could not delete preset: {exc}") + return {'CANCELLED'} + s.lighting_preset = 'STUDIO_SOFT' + self.report({'INFO'}, f"Deleted lighting preset '{entry[0]}'.") return {'FINISHED'} @@ -1592,21 +3255,13 @@ def execute(self, context): return {'CANCELLED'} context.view_layer.update() - done = 0 + done = sum(1 for pc in products if collection_meshes(pc)) unbaked = set() smoothed = 0 + if s.recenter or s.rescale: + unbaked.update(standardize_products(context, products, scale=s.rescale, + recenter=s.recenter)) for pc in products: - if s.recenter or s.rescale: - if fit_collection(pc, - target_size=s.target_size if s.rescale else None, - recenter=s.recenter): - done += 1 - _, skipped_objs = bake_product(pc, - apply_scale=s.rescale, - origin_to_world=s.recenter) - unbaked.update(skipped_objs) - elif collection_meshes(pc): - done += 1 if s.auto_smooth: shaded, skipped_objs = smooth_product(pc, s.smooth_angle) smoothed += len(shaded) @@ -1652,6 +3307,8 @@ def execute(self, context): isolate_product(context, coll.name) context.view_layer.update() step = s.rotate_step + # The way it faced before the first turn is what Reset goes back to. + remember_home(coll) if not rotate_product(context, coll, self.direction, step): self.report({'WARNING'}, f"'{coll.name}' has no geometry to rotate.") return {'CANCELLED'} @@ -1661,6 +3318,76 @@ def execute(self, context): return {'FINISHED'} +def _orient_targets(context, all_products): + """The products an orientation button acts on, made visible to act on.""" + s = context.scene.mv_settings + if all_products: + isolate_product(context, None) + context.view_layer.update() + return get_products() + coll, _ = active_product(context) + if coll is None: + return [] + if s.isolate_active: + isolate_product(context, coll.name) + context.view_layer.update() + return [coll] + + +class MV_OT_AutoOrient(Operator): + """Turn products to a resting orientation: on their largest flat face, or + along their principal axes, with the longest side across the Front view""" + bl_idname = "mv.auto_orient" + bl_label = "Auto Orient" + bl_options = {'REGISTER', 'UNDO'} + + all_products: BoolProperty(name="All Products", default=False) + + def execute(self, context): + s = context.scene.mv_settings + products = _orient_targets(context, self.all_products) + if not products: + self.report({'ERROR'}, f"No product collections under '{PRODUCT_COLL}'.") + return {'CANCELLED'} + turned = sum(1 for pc in products if auto_orient(context, pc, s.orient_method)) + if self.all_products: + # Showing everything was only so they could be measured. + if s.isolate_active: + coll, _ = active_product(context) + isolate_product(context, coll.name if coll else None) + if not turned: + self.report({'WARNING'}, "No mesh geometry to orient.") + return {'CANCELLED'} + self.report({'INFO'}, f"Oriented {turned} product(s).") + return {'FINISHED'} + + +class MV_OT_ResetOrientation(Operator): + """Turn products back to the orientation they had when imported""" + bl_idname = "mv.reset_orientation" + bl_label = "Reset Orientation" + bl_options = {'REGISTER', 'UNDO'} + + all_products: BoolProperty(name="All Products", default=False) + + def execute(self, context): + s = context.scene.mv_settings + products = _orient_targets(context, self.all_products) + if not products: + self.report({'ERROR'}, f"No product collections under '{PRODUCT_COLL}'.") + return {'CANCELLED'} + reset = sum(1 for pc in products if reset_orientation(pc)) + context.view_layer.update() + if self.all_products and s.isolate_active: + coll, _ = active_product(context) + isolate_product(context, coll.name if coll else None) + if not reset: + self.report({'WARNING'}, "Nothing to reset: not turned since import.") + return {'CANCELLED'} + self.report({'INFO'}, f"Reset {reset} product(s).") + return {'FINISHED'} + + class MV_OT_ProductStep(Operator): """Show the next or previous product on its own, hiding the others""" bl_idname = "mv.product_step" @@ -1723,6 +3450,20 @@ def execute(self, context): return {'FINISHED'} +class MV_OT_SetProductsRender(Operator): + """Tick or untick every product for the next batch render""" + bl_idname = "mv.set_products_render" + bl_label = "Set Products Render" + bl_options = {'REGISTER', 'UNDO'} + + state: BoolProperty(default=True) + + def execute(self, context): + for pc in get_products(): + pc.mv_render = self.state + return {'FINISHED'} + + def _do_full_render(scene, filepath): scene.render.filepath = filepath bpy.ops.render.render(write_still=True) @@ -1758,244 +3499,2129 @@ def _do_clay_render(scene, filepath, clay_color): scene.display.shading.show_cavity = saved['show_cavity'] -class _BatchAborted(Exception): - """Stop the batch early without dumping a traceback on the user.""" +NORMAL_MATCAP = "check_normal+y.exr" -class MV_OT_RenderAll(Operator): - bl_idname = "mv.render_all" - bl_label = "Render All" +@contextmanager +def temporarily(*changes): + """Set (owner, attribute, value) triples, and put every one back on the way + out, in reverse, even when the body raises. Settings that depend on each + other (file format before colour depth, light before studio light) are + set in the order given and restored in the opposite one.""" + saved = [] + try: + for owner, attr, value in changes: + old = getattr(owner, attr) + if not isinstance(old, (str, bytes)) and hasattr(old, "__len__") \ + and not isinstance(old, bpy.types.ID): + old = tuple(old) + saved.append((owner, attr, old)) + setattr(owner, attr, value) + yield + finally: + for owner, attr, old in reversed(saved): + try: + setattr(owner, attr, old) + except (AttributeError, TypeError, ValueError): + pass - def _fail(self, failures, done, path, exc): - """Record one failed write, aborting the batch if nothing has landed. - Collected and reported at the end rather than raised, so a single - unwritable file cannot throw away a long batch that is otherwise fine. - Before the first success there is nothing to salvage and the cause is - almost always the setup, so that case stops immediately instead of - failing the same way for every remaining image. - """ - note = f"{os.path.basename(path)}: {exc}" - failures.append(note) - print(f"[MultiView] FAILED {note}") - if not done: - raise _BatchAborted from exc +def _png_output(scene): + image = scene.render.image_settings + return ((image, "file_format", 'PNG'), (image, "color_mode", 'RGBA'), + (image, "color_depth", '8')) - def execute(self, context): - scene = context.scene - s = scene.mv_settings - cam_coll = bpy.data.collections.get(CAMERA_COLL) - cameras = [o for o in cam_coll.objects if o.type == 'CAMERA'] if cam_coll else [] - if not cameras: - self.report({'ERROR'}, "No cameras. Build cameras first.") - return {'CANCELLED'} +def _neutral_view(scene): + """Display values straight through, so white is white and a normal reads + back as the number it was.""" + view = scene.view_settings + return ((view, "view_transform", 'Standard'), (view, "look", 'None'), + (view, "exposure", 0.0), (view, "gamma", 1.0)) - products = get_products() - if not products: - self.report({'ERROR'}, f"No product collections under '{PRODUCT_COLL}'.") - return {'CANCELLED'} - out_root = bpy.path.abspath(s.render_output_dir) - try: - os.makedirs(out_root, exist_ok=True) - except OSError as exc: - self.report({'ERROR'}, f"Cannot use output folder '{out_root}': {exc}") - return {'CANCELLED'} +def _render_to(scene, path): + scene.render.filepath = path + bpy.ops.render.render(write_still=True) + return written_path(scene, path) - saved_hide = {} - for pc in products: - lc = find_layer_coll(context.view_layer.layer_collection, pc.name) - if lc: - saved_hide[pc.name] = lc.exclude - saved_cam = scene.camera - saved_path = scene.render.filepath - mode = s.render_mode - both = mode == 'BOTH' - total_per_pair = (2 if both else 1) * len(cameras) - total = len(products) * total_per_pair - done = 0 - failures = [] +def _save_like_render(scene, px, filepath): + """Write a computed image where, and in the format, a render would have.""" + image = scene.render.image_settings + save_pixels(px, written_path(scene, filepath), image.file_format, image.quality) + scene.render.filepath = filepath - try: - for i, pc in enumerate(products): - for other in products: - lc = find_layer_coll(context.view_layer.layer_collection, other.name) - if lc: - lc.exclude = (other.name != pc.name) - - label = product_label(s.start_prefix, s.start_number, i, s.products_per_letter) - product_dir = os.path.join(out_root, safe_name(f"{label}_{pc.name}")) - try: - os.makedirs(product_dir, exist_ok=True) - except OSError as exc: - self._fail(failures, done, product_dir, exc) - continue - for cam in cameras: - scene.camera = cam - view_name = cam.name.replace("MV_Cam_", "") - base = safe_name(f"{label}_{pc.name}_{view_name}") - - clay = (tuple(s.clay_color),) - if mode == 'FULL': - attempts = ((_do_full_render, base, ()),) - elif mode == 'CLAY': - attempts = ((_do_clay_render, base + "_clay", clay),) - else: - attempts = ((_do_full_render, base + "_render", ()), - (_do_clay_render, base + "_clay", clay)) - - for render, name, extra in attempts: - path = os.path.join(product_dir, name) - try: - render(scene, path, *extra) - except RuntimeError as exc: - # bpy.ops.render.render raises when the image cannot - # be written. Letting that escape put a Python - # traceback in front of the user instead of the name - # of the file that failed. - self._fail(failures, done, path, exc) - continue - done += 1 - print(f"[MultiView] {done}/{total}: {scene.render.filepath}") - except _BatchAborted: - pass - finally: - for pc in products: - lc = find_layer_coll(context.view_layer.layer_collection, pc.name) - if lc and pc.name in saved_hide: - lc.exclude = saved_hide[pc.name] - scene.camera = saved_cam - scene.render.filepath = saved_path - - if not failures: - self.report({'INFO'}, f"Wrote {done} image(s).") - return {'FINISHED'} +def srgb_decode(rgb): + rgb = np.clip(rgb, 0.0, 1.0) + return np.where(rgb <= 0.04045, rgb / 12.92, np.power((rgb + 0.055) / 1.055, 2.4)) - if not done: - # Nothing landed at all, so the batch stopped at the first failure - # rather than burning render time failing the same way every time. - self.report({'ERROR'}, f"Nothing rendered. {failures[0]}") - return {'CANCELLED'} - self.report({'WARNING'}, - f"Wrote {done} of {total} image(s). {len(failures)} failed, " - f"first: {failures[0]}. Full list in the System Console.") - return {'FINISHED'} +def line_mask(normals, crease_angle, width): + """Where to draw lines, from a render of view-space normals. + A line goes wherever neighbouring normals differ by more than the crease + angle, and along every silhouette. Pixels two apart are compared as well + as adjacent ones, because antialiasing blends one pixel between two faces + and halves the step each side of it: a 40 degree crease would otherwise + read as two 20 degree ones and slip under a 30 degree threshold. + """ + inside = normals[..., 3] > 0.5 + n = srgb_decode(normals[..., :3]) * 2.0 - 1.0 + n /= np.maximum(np.linalg.norm(n, axis=-1, keepdims=True), 1e-6) + limit = math.cos(crease_angle) + edges = np.zeros(inside.shape, bool) + for step in (1, 2): + for axis in (0, 1): + a = [slice(None)] * 2 + b = [slice(None)] * 2 + a[axis], b[axis] = slice(None, -step), slice(step, None) + a, b = tuple(a), tuple(b) + crease = inside[a] & inside[b] & ((n[a] * n[b]).sum(axis=-1) < limit) + edges[b] |= crease + if step == 1: + rim = inside[a] != inside[b] + edges[a] |= rim & inside[a] + edges[b] |= rim & inside[b] + thick = edges.copy() + for i in range(1, width): + thick[i:, :] |= edges[:-i, :] + thick[:, i:] |= edges[:, :-i] + return thick, inside + + +def _do_lines_render(scene, filepath, options): + """Technical-drawing pass: outlines and creases as lines. + + Workbench renders the product with a matcap that colours each pixel by its + view-space normal, plus object outlines so separate parts that touch still + get a line between them. The lines are found from that image rather than + with Freestyle or Line Art, which keeps the pass a fast Workbench render + and independent of the engine the full render uses. + """ + angle, width, color, fill, clay_color = options + work = tempfile.mkdtemp(prefix="mv_lines_") + try: + base = None + if fill == 'CLAY': + clay = os.path.join(work, "clay") + _do_clay_render(scene, clay, clay_color) + base = load_pixels(written_path(scene, clay)) + shading = scene.display.shading + transparent = scene.render.film_transparent + with temporarily( + (scene.render, "engine", 'BLENDER_WORKBENCH'), + (shading, "type", 'SOLID'), + (shading, "light", 'MATCAP'), + (shading, "studio_light", NORMAL_MATCAP), + (shading, "show_cavity", False), + (shading, "show_shadows", False), + (shading, "show_specular_highlight", False), + (shading, "show_object_outline", True), + (shading, "object_outline_color", (0.0, 0.0, 0.0)), + (scene.render, "film_transparent", True), + (scene.render, "filepath", scene.render.filepath), + *_neutral_view(scene), *_png_output(scene)): + normals = load_pixels(_render_to(scene, os.path.join(work, "normals"))) + finally: + shutil.rmtree(work, ignore_errors=True) + + lines, inside = line_mask(normals, angle, width) + if base is None: + base = np.ones(normals.shape, np.float32) + if fill == 'NONE': + base[..., 3] = 0.0 + elif transparent: + base[..., 3] = inside | lines + ink = np.zeros(normals.shape, np.float32) + ink[..., :3] = color + ink[..., 3] = lines + paste_over(base, ink, 0, 0) + _save_like_render(scene, base, filepath) + + +def _do_mask_render(scene, filepath): + """Silhouette pass: white where the product is, black elsewhere, with + antialiased edges so it works as a matte.""" + work = tempfile.mkdtemp(prefix="mv_mask_") + shading = scene.display.shading + try: + with temporarily( + (scene.render, "engine", 'BLENDER_WORKBENCH'), + (shading, "type", 'SOLID'), + (shading, "light", 'FLAT'), + (shading, "color_type", 'SINGLE'), + (shading, "single_color", (1.0, 1.0, 1.0)), + (shading, "show_cavity", False), + (shading, "show_shadows", False), + (shading, "show_object_outline", False), + (scene.render, "film_transparent", True), + (scene.render, "filepath", scene.render.filepath), + *_png_output(scene)): + px = load_pixels(_render_to(scene, os.path.join(work, "mask"))) + finally: + shutil.rmtree(work, ignore_errors=True) + out = np.empty_like(px) + out[..., :3] = px[..., 3:4] + out[..., 3] = 1.0 + _save_like_render(scene, out, filepath) + + +def _ao_material(distance): + mat = bpy.data.materials.get("MV_AO") or bpy.data.materials.new("MV_AO") + tree = node_tree_of(mat) + tree.nodes.clear() + ao = tree.nodes.new("ShaderNodeAmbientOcclusion") + ao.samples = 16 + ao.inputs["Color"].default_value = (1.0, 1.0, 1.0, 1.0) + ao.inputs["Distance"].default_value = distance + emit = tree.nodes.new("ShaderNodeEmission") + emit.location = (200, 0) + out = tree.nodes.new("ShaderNodeOutputMaterial") + out.location = (400, 0) + tree.links.new(ao.outputs["Color"], emit.inputs["Color"]) + tree.links.new(emit.outputs["Emission"], out.inputs["Surface"]) + return mat + + +def _ao_world(): + world = bpy.data.worlds.get("MV_AO_World") or bpy.data.worlds.new("MV_AO_World") + tree = node_tree_of(world) + tree.nodes.clear() + bg = tree.nodes.new("ShaderNodeBackground") + bg.inputs["Color"].default_value = (1.0, 1.0, 1.0, 1.0) + out = tree.nodes.new("ShaderNodeOutputWorld") + tree.links.new(bg.outputs["Background"], out.inputs["Surface"]) + return world + + +def _do_ao_render(scene, filepath, options): + """Ambient occlusion pass: Cycles with every surface swapped for a white + emitter darkened by its own occlusion, so light and materials play no part + and only the shape shows.""" + samples, distance = options + if not hasattr(scene, "cycles"): + raise RuntimeError("the AO pass needs the Cycles add-on enabled") + with temporarily( + (scene.render, "engine", 'CYCLES'), + (scene.cycles, "samples", samples), + (bpy.context.view_layer, "material_override", _ao_material(distance)), + (scene, "world", _ao_world()), + *_neutral_view(scene)): + scene.render.filepath = filepath + bpy.ops.render.render(write_still=True) -class MV_UL_StepFiles(UIList): - def draw_item(self, context, layout, data, item, icon, active_data, active_propname, index): - row = layout.row(align=True) - row.label(text=os.path.basename(item.path) or "(empty)", icon='FILE') - row.operator("mv.remove_step_file", text="", icon='X').index = index +def written_path(scene, path): + """Where render.render(write_still=True) puts an image asked for at path. -class MV_UL_CustomViews(UIList): - def draw_item(self, context, layout, data, item, icon, active_data, active_propname, index): - row = layout.row(align=True) - row.prop(item, "name", text="") - row.prop(item, "dir_x", text="X") - row.prop(item, "dir_y", text="Y") - row.prop(item, "dir_z", text="Z") - row.prop(item, "is_ortho", text="", icon='VIEW_ORTHO' if item.is_ortho else 'VIEW_PERSPECTIVE') - row.operator("mv.remove_custom_view", text="", icon='X').index = index + Blender appends the format's extension itself while File Extensions is on, + so a skip check that looked for the bare path would never find anything and + a resumed batch would quietly render everything again. + """ + r = scene.render + return path + r.file_extension if r.use_file_extension else path -class MV_PT_Guide(Panel): - bl_label = "Guide" - bl_space_type = 'VIEW_3D' - bl_region_type = 'UI' - bl_category = "MultiView" - bl_options = {'DEFAULT_CLOSED'} +def file_ok(path): + """True for a file worth keeping. An empty one is what a crash mid-write + leaves behind, and is exactly the file a resume is for.""" + return os.path.isfile(path) and os.path.getsize(path) > 0 - def draw(self, context): - layout = self.layout - box = layout.box() - for line in GUIDE_LINES: - box.label(text=line) +def format_duration(seconds): + seconds = max(int(round(seconds)), 0) + if seconds < 60: + return f"{seconds}s" + minutes, seconds = divmod(seconds, 60) + if minutes < 60: + return f"{minutes}m {seconds:02d}s" + hours, minutes = divmod(minutes, 60) + return f"{hours}h {minutes:02d}m" -class MV_PT_SceneSetup(Panel): - bl_label = "Scene Setup" - bl_space_type = 'VIEW_3D' - bl_region_type = 'UI' - bl_category = "MultiView" - def draw(self, context): - layout = self.layout - s = context.scene.mv_settings +# -- naming --------------------------------------------------------------- - col = layout.column(align=True) - col.label(text="Lighting:") - col.prop(s, "lighting_preset", text="") - col.prop(s, "lighting_intensity", slider=True) +DEFAULT_FILE_TEMPLATE = "{label}_{product}_{view}_{mode}" +DEFAULT_FOLDER_TEMPLATE = "{label}_{product}" +TEMPLATE_TOKEN = re.compile(r"\{(\w+)\}") +TEMPLATE_SEPARATORS = "_-. " - col = layout.column(align=True) - col.label(text="Background:") - col.prop(s, "transparent_bg") - row = col.row() - row.enabled = not s.transparent_bg - row.prop(s, "bg_color", text="") - layout.separator() - layout.operator("mv.apply_scene_setup", icon='SCENE_DATA') +def expand_template(template, values): + """Fill {token}s from values. - row = layout.row(align=True) - row.operator("mv.build_lighting", text="Lighting Only", icon='LIGHT') - row.operator("mv.apply_world", text="Background Only", icon='WORLD') + An empty value takes one neighbouring separator with it, so the blank + {mode} of a Full Render turns 'AA_01_Widget_Front_{mode}' into + 'AA_01_Widget_Front' rather than leaving a dangling underscore. Unknown + tokens are left exactly as typed, so a typo shows up in the panel preview + instead of silently vanishing from every filename. + """ + out = "" + pos = 0 + eat_next = False + for m in TEMPLATE_TOKEN.finditer(template): + literal = template[pos:m.start()] + if eat_next and literal and literal[0] in TEMPLATE_SEPARATORS: + literal = literal[1:] + eat_next = False + out += literal + key = m.group(1) + if key not in values: + out += m.group(0) + elif values[key]: + out += values[key] + elif out and out[-1] in TEMPLATE_SEPARATORS: + out = out[:-1] + else: + eat_next = True + pos = m.end() + literal = template[pos:] + if eat_next and literal and literal[0] in TEMPLATE_SEPARATORS: + literal = literal[1:] + return out + literal -class MV_PT_Import(Panel): - bl_label = "Import Products" - bl_space_type = 'VIEW_3D' - bl_region_type = 'UI' - bl_category = "MultiView" +def name_values(settings, index, product, view, mode): + label = product_label(settings.start_prefix, settings.start_number, index, + settings.products_per_letter) + prefix, _, number = label.rpartition("_") + return {"label": label, "prefix": prefix, "number": number, + "product": product, "view": view, "mode": mode} - def draw(self, context): - layout = self.layout - s = context.scene.mv_settings - prefs = context.preferences.addons[__name__].preferences - box = layout.box() - box.label(text="STEP Importer:") - box.prop(prefs, "step_import_mode", text="Mode") - col = box.column() - col.enabled = prefs.step_import_mode in {'AUTO', 'FREECAD'} - col.prop(prefs, "freecad_path", text="FreeCAD") - box.label(text="STL files ignore this and import directly.", icon='INFO') +def output_folder(settings, values): + """Folder under the output directory, from the folder template. - layout.separator() - row = layout.row() - row.template_list("MV_UL_StepFiles", "", s, "step_files", s, "step_files_index", rows=3) - col = row.column(align=True) - col.operator("mv.add_step_file", text="", icon='ADD') - col.operator("mv.clear_step_files", text="", icon='TRASH') + Each / separated level is sanitized on its own, so a template can nest + folders but a product name can never climb out of the output directory: + '..' sanitizes to nothing and the level is dropped. + """ + levels = [] + for part in re.split(r"[\\/]+", settings.folder_template or ""): + level = safe_name(expand_template(part, values), fallback="") + if level: + levels.append(level) + return os.path.join(*levels) if levels else "" - layout.operator("mv.import_all_steps", icon='IMPORT') - layout.operator("mv.add_product_collection", icon='OUTLINER_COLLECTION') - prods = get_products() - header = layout.row(align=True) - header.prop(s, "show_products_list", text="", - icon='TRIA_DOWN' if s.show_products_list else 'TRIA_RIGHT', emboss=False) - header.label(text=f"Products under '{PRODUCT_COLL}' ({len(prods)})") - if s.show_products_list: - box = layout.box() - if not prods: - box.label(text="(none)", icon='INFO') - else: - for pc in prods: - box.label(text=f"{pc.name} ({len(pc.all_objects)} objs)", icon='OUTLINER_COLLECTION') +def output_basename(settings, values): + return safe_name(expand_template(settings.file_template or DEFAULT_FILE_TEMPLATE, values)) -class MV_PT_Standardize(Panel): - bl_label = "Standardize" - bl_space_type = 'VIEW_3D' - bl_region_type = 'UI' - bl_category = "MultiView" +class _NameClash(Exception): + """Two outputs of one batch would be written to the same file.""" + + +# -- image buffers ---------------------------------------------------------- +# +# Everything below works on float32 RGBA arrays shaped (height, width, 4), in +# 0-1, with rows bottom to top as Blender stores them. Values are the encoded +# 8-bit display values straight out of the file, never linearized, so an image +# read and written back is byte for byte what Render wrote. + +POST_FORMATS = {'PNG': ".png", 'JPEG': ".jpg", 'WEBP': ".webp"} + + +def srgb_encode(rgb): + rgb = np.clip(rgb, 0.0, None) + return np.where(rgb <= 0.0031308, rgb * 12.92, 1.055 * np.power(rgb, 1 / 2.4) - 0.055) + + +def load_pixels(path): + img = bpy.data.images.load(path, check_existing=False) + try: + w, h = img.size + if not w or not h: + raise RuntimeError("Blender could not read the image") + px = np.empty(w * h * 4, dtype=np.float32) + img.pixels.foreach_get(px) + px = px.reshape(h, w, 4) + if img.is_float: + # 16-bit PNG and EXR come in as scene-linear floats. Encode them + # back to display values so they sit with everything else; EXR + # skips the view transform a PNG would have had baked in. + px[..., :3] = srgb_encode(px[..., :3]) + return np.clip(px, 0.0, 1.0) + finally: + bpy.data.images.remove(img) + + +def image_size(path): + try: + img = bpy.data.images.load(path, check_existing=False) + except RuntimeError: + return None, None + try: + w, h = img.size + return (w, h) if w and h else (None, None) + finally: + bpy.data.images.remove(img) + + +def save_pixels(px, path, file_format, quality=90): + h, w = px.shape[:2] + img = bpy.data.images.new("MV_Output", w, h, alpha=True, float_buffer=False) + try: + img.pixels.foreach_set(np.ascontiguousarray(np.clip(px, 0.0, 1.0), dtype=np.float32).ravel()) + img.filepath_raw = path + img.file_format = file_format + try: + img.save(quality=quality) + except TypeError: + img.save() + finally: + bpy.data.images.remove(img) + + +def bilinear(px, new_w, new_h): + h, w = px.shape[:2] + ys = (np.arange(new_h) + 0.5) * h / new_h - 0.5 + xs = (np.arange(new_w) + 0.5) * w / new_w - 0.5 + y0 = np.floor(ys).astype(int) + x0 = np.floor(xs).astype(int) + wy = np.clip(ys - y0, 0.0, 1.0)[:, None, None] + wx = np.clip(xs - x0, 0.0, 1.0)[None, :, None] + y1 = np.clip(y0 + 1, 0, h - 1) + x1 = np.clip(x0 + 1, 0, w - 1) + y0 = np.clip(y0, 0, h - 1) + x0 = np.clip(x0, 0, w - 1) + top = px[y0][:, x0] * (1 - wx) + px[y0][:, x1] * wx + bottom = px[y1][:, x0] * (1 - wx) + px[y1][:, x1] * wx + return (top * (1 - wy) + bottom * wy).astype(np.float32) + + +def resize_pixels(px, new_w, new_h): + """Resample with alpha premultiplied, so transparent pixels' colour cannot + bleed a dark fringe into the edge of the product.""" + h, w = px.shape[:2] + new_w, new_h = max(int(new_w), 1), max(int(new_h), 1) + if (w, h) == (new_w, new_h): + return px + alpha = px[..., 3:4] + pm = np.concatenate([px[..., :3] * alpha, alpha], axis=-1) + # Box-average whole factors first: bilinear alone only samples four + # pixels, which aliases badly when shrinking a 4K render to a thumbnail. + fy, fx = max(1, h // new_h), max(1, w // new_w) + if fx > 1 or fy > 1: + hh, ww = (h // fy) * fy, (w // fx) * fx + pm = pm[:hh, :ww].reshape(hh // fy, fy, ww // fx, fx, 4).mean(axis=(1, 3)) + pm = bilinear(pm, new_w, new_h) + alpha = pm[..., 3:4] + rgb = pm[..., :3] / np.maximum(alpha, 1e-6) + rgb = np.where(alpha > 1e-6, rgb, 0.0) + return np.concatenate([rgb, alpha], axis=-1).astype(np.float32) + + +def fit_long_edge(px, size): + h, w = px.shape[:2] + scale = size / max(w, h) + return resize_pixels(px, round(w * scale), round(h * scale)) + + +def flatten(px, color): + out = np.empty_like(px) + alpha = px[..., 3:4] + out[..., :3] = px[..., :3] * alpha + np.asarray(color[:3], np.float32) * (1 - alpha) + out[..., 3] = 1.0 + return out + + +def has_transparency(px): + return bool(px[..., 3].min() < 0.99) + + +def background_of(px): + """The colour a product sits on: transparent when there is alpha to use, + otherwise whatever the four corners agree on.""" + if has_transparency(px): + return np.zeros(4, np.float32) + corners = np.stack([px[0, 0], px[0, -1], px[-1, 0], px[-1, -1]]) + return np.median(corners, axis=0).astype(np.float32) + + +def content_box(px, background): + """(x0, y0, x1, y1) around everything that is not background, or None.""" + if has_transparency(px): + mask = px[..., 3] > 0.02 + else: + mask = np.abs(px[..., :3] - background[:3]).max(axis=-1) > 0.02 + rows = np.flatnonzero(mask.any(axis=1)) + cols = np.flatnonzero(mask.any(axis=0)) + if not rows.size: + return None + return int(cols[0]), int(rows[0]), int(cols[-1]) + 1, int(rows[-1]) + 1 + + +def pad_pixels(px, left, right, bottom, top, fill): + h, w = px.shape[:2] + out = np.empty((h + bottom + top, w + left + right, 4), np.float32) + out[:] = fill + out[bottom:bottom + h, left:left + w] = px + return out + + +def paste_over(canvas, src, x, y): + """Composite src onto canvas with its bottom-left corner at (x, y).""" + h, w = src.shape[:2] + big_h, big_w = canvas.shape[:2] + x0, y0 = max(x, 0), max(y, 0) + x1, y1 = min(x + w, big_w), min(y + h, big_h) + if x0 >= x1 or y0 >= y1: + return + s = src[y0 - y:y1 - y, x0 - x:x1 - x] + d = canvas[y0:y1, x0:x1] + sa, da = s[..., 3:4], d[..., 3:4] + out_a = sa + da * (1 - sa) + rgb = (s[..., :3] * sa + d[..., :3] * da * (1 - sa)) / np.maximum(out_a, 1e-6) + canvas[y0:y1, x0:x1] = np.concatenate([rgb, out_a], axis=-1) + + +def post_process(px, settings): + """Trim, pad and resize one image the way the Post-Processing panel asks.""" + background = background_of(px) + if settings.post_trim: + box = content_box(px, background) + if box is not None: + x0, y0, x1, y1 = box + px = px[y0:y1, x0:x1] + margin = int(round(settings.post_margin * max(x1 - x0, y1 - y0))) + if margin: + px = pad_pixels(px, margin, margin, margin, margin, background) + if settings.post_square: + h, w = px.shape[:2] + if w != h: + side = max(w, h) + left, bottom = (side - w) // 2, (side - h) // 2 + px = pad_pixels(px, left, side - w - left, bottom, side - h - bottom, background) + # JPEG has no alpha channel. Dropping it unflattened leaves the RGB that + # transparent pixels happen to hold, which for a render is black. + if settings.post_flatten or settings.post_format == 'JPEG': + px = flatten(px, settings.post_bg_color) + if settings.post_max_size and max(px.shape[:2]) > settings.post_max_size: + px = fit_long_edge(px, settings.post_max_size) + return px + + +def parse_sizes(text): + sizes = {int(v) for v in re.findall(r"\d+", text or "") if int(v) >= 8} + return sorted(sizes, reverse=True) + + +# -- contact sheets ----------------------------------------------------------- + +def text_mask(text, size, tmp_dir): + """Coverage (height, width) of text in Blender's UI font, or None on a + Blender that cannot draw text into an image buffer.""" + if not hasattr(blf, "bind_imbuf"): + return None + import imbuf + font = 0 + blf.size(font, size) + width, _ = blf.dimensions(font, text) + buf = imbuf.new((max(int(width) + 4, 1), int(size * 1.5) + 4)) + with blf.bind_imbuf(font, buf): + blf.color(font, 1.0, 1.0, 1.0, 1.0) + blf.position(font, 2, int(size * 0.35) + 2, 0) + blf.draw_buffer(font, text) + # imbuf has no pixel access from Python, so the text goes through a file. + path = os.path.join(tmp_dir, "mv_text.png") + imbuf.write(buf, filepath=path) + return load_pixels(path)[..., 0] + + +def contact_sheet(cells, title, settings, tmp_dir): + """Lay out [(caption, pixels)] on one canvas, captions under each view and + the product's name across the top. Cells arrive already scaled to the + sheet's cell width.""" + cell_w = settings.sheet_cell_width + cell_h = max(px.shape[0] for _, px in cells) + n = len(cells) + cols = min(settings.sheet_columns or math.ceil(math.sqrt(n)), n) + rows = math.ceil(n / cols) + gap = max(8, cell_w // 24) + caption_size = max(11, cell_w // 20) + title_size = int(caption_size * 1.5) + labels = settings.sheet_labels and hasattr(blf, "bind_imbuf") + caption_h = int(caption_size * 1.8) if labels else 0 + title_h = int(title_size * 2.0) if labels else 0 + + width = cols * cell_w + (cols + 1) * gap + height = title_h + rows * (cell_h + caption_h) + (rows + 1) * gap + canvas = np.empty((height, width, 4), np.float32) + canvas[:] = settings.sheet_bg + ink = np.asarray(settings.sheet_text_color, np.float32) + + def draw_text(text, size, x, y_top, max_w): + mask = text_mask(text, size, tmp_dir) + if mask is None: + return + mask = mask[:, :max_w] + layer = np.empty(mask.shape + (4,), np.float32) + layer[..., :3] = ink + layer[..., 3] = mask + paste_over(canvas, layer, x, height - y_top - mask.shape[0]) + + if labels: + draw_text(title, title_size, gap, int(title_size * 0.4), width - 2 * gap) + for i, (caption, px) in enumerate(cells): + row, col = divmod(i, cols) + x = gap + col * (cell_w + gap) + y_top = title_h + gap + row * (cell_h + caption_h + gap) + offset = (cell_h - px.shape[0]) // 2 + # Rows are stored bottom up, so a top-down layout position flips. + paste_over(canvas, px, x, height - (y_top + offset) - px.shape[0]) + if labels: + draw_text(caption, caption_size, x, y_top + cell_h + int(caption_size * 0.2), cell_w) + return canvas + + +# -- turntables --------------------------------------------------------------- + +TURNTABLE_CAM = "MV_Turntable_Cam" + + +def build_turntable_camera(context): + """A perspective camera that keeps every product in frame at every angle. + + Each product spins about its own centre, so the framing is a cylinder + around the batch centre wide enough for the product that reaches furthest + from it, measured from all sides at once. + """ + scene = context.scene + s = scene.mv_settings + boxes = [] + for pc in get_products(): + objs = collection_meshes(pc) + if objs: + boxes.append(bbox_of(objs)) + if not boxes: + return None + lo = Vector([min(c[i] - sz[i] / 2 for c, sz in boxes) for i in range(3)]) + hi = Vector([max(c[i] + sz[i] / 2 for c, sz in boxes) for i in range(3)]) + center = (lo + hi) * 0.5 + radius = 0.0 + for c, sz in boxes: + reach = Vector((c.x - center.x, c.y - center.y)).length + radius = max(radius, reach + 0.5 * Vector((sz.x, sz.y)).length) + steps = 24 + # A polygon through the circle's points would cut inside it between them. + radius = max(radius, 1e-3) / math.cos(math.pi / steps) + corners = [center + Vector((radius * math.cos(a), radius * math.sin(a), z - center.z)) + for a in (2 * math.pi * k / steps for k in range(steps)) + for z in (lo.z, hi.z)] + + elevation = s.turntable_elevation + d = Vector(rotate_dir_z((0.0, -math.cos(elevation), math.sin(elevation)), + FRONT_AXIS_ANGLES[s.front_axis])).normalized() + _, _, depth, local = view_extents(look_at(center + d, center), center, corners) + rx, ry = image_aspect(scene) + tan_x, tan_y = sensor_tangents(rx, ry) + pad = max(s.frame_padding_factor, 1.0) + diag = max((hi - lo).length * 0.5, 1e-3) + distance = max(persp_distance(local, tan_x, tan_y, pad), depth + diag * 0.1) + + data = bpy.data.cameras.new(TURNTABLE_CAM) + data.lens = PERSP_LENS + data.clip_start = max(distance * 0.01, 1e-5) + data.clip_end = (distance + diag) * 100.0 + cam = bpy.data.objects.new(TURNTABLE_CAM, data) + cam.matrix_world = look_at(center + d * distance, center) + # Linked to the scene, not MV_Cameras, so it is never mistaken for a view. + scene.collection.objects.link(cam) + return cam + + +def remove_object(obj): + data = obj.data + bpy.data.objects.remove(obj, do_unlink=True) + if isinstance(data, bpy.types.Camera) and not data.users: + bpy.data.cameras.remove(data) + + +def encode_mp4(frames, out_path, fps, background, tmp_dir): + """H.264 from a list of frame files, through a throwaway sequencer scene. + + Frames are flattened onto the background first, because H.264 has no alpha + and the sequencer would otherwise put a transparent render on black, and + cropped to even dimensions, which the codec requires. + """ + work = tempfile.mkdtemp(prefix="mv_mp4_", dir=tmp_dir) + names = [] + width = height = None + try: + for i, frame in enumerate(frames): + px = flatten(load_pixels(frame), background) + if width is None: + height, width = px.shape[:2] + width -= width % 2 + height -= height % 2 + name = f"frame_{i + 1:04d}.png" + save_pixels(px[:height, :width], os.path.join(work, name), 'PNG') + names.append(name) + + scene = bpy.data.scenes.new("MV_Encode") + try: + r = scene.render + r.resolution_x, r.resolution_y = width, height + r.resolution_percentage = 100 + r.pixel_aspect_x = r.pixel_aspect_y = 1.0 + r.fps, r.fps_base = int(fps), 1.0 + scene.frame_start, scene.frame_end = 1, len(names) + editor = scene.sequence_editor_create() + # Blender 4.4 renamed sequences to strips. Not getattr(...) or: + # a new editor's strips is empty, which reads as false. + strips = editor.strips if hasattr(editor, "strips") else editor.sequences + strip = strips.new_image(name="MV_Frames", filepath=os.path.join(work, names[0]), + channel=1, frame_start=1) + for name in names[1:]: + strip.elements.append(name) + image = r.image_settings + if hasattr(image, "media_type"): + image.media_type = 'VIDEO' + image.file_format = 'FFMPEG' + r.ffmpeg.format = 'MPEG4' + r.ffmpeg.codec = 'H264' + r.ffmpeg.constant_rate_factor = 'HIGH' + # The frames are display values already; a second view transform + # on the way out would wash them out. + scene.view_settings.view_transform = 'Standard' + scene.view_settings.look = 'None' + r.use_sequencer = True + r.use_compositing = False + r.filepath = os.path.join(work, "turntable") + # Blender 5 takes the range from the operator, and left unset it + # reads the window's scene: 250 frames, 242 of them black. + kwargs = {} + props = bpy.ops.render.render.get_rna_type().properties + if "frame_start" in props: + kwargs = {"frame_start": 1, "frame_end": len(names)} + bpy.ops.render.render(animation=True, scene=scene.name, **kwargs) + finally: + bpy.data.scenes.remove(scene) + + produced = [n for n in os.listdir(work) if n.lower().endswith(".mp4")] + if not produced: + raise RuntimeError("Blender's FFmpeg wrote no video") + os.replace(os.path.join(work, produced[0]), out_path) + finally: + shutil.rmtree(work, ignore_errors=True) + return width, height + + +def gif_lzw(data): + """GIF's variable-width LZW over 8-bit palette indices, packed into the + 255-byte sub-blocks the format wants.""" + clear, end = 256, 257 + out = bytearray() + buf = nbits = 0 + size = 9 + table = {} + next_code = 258 + + buf |= clear << nbits + nbits += size + it = iter(data) + prefix = next(it) + for byte in it: + key = (prefix << 8) | byte + code = table.get(key) + if code is not None: + prefix = code + continue + buf |= prefix << nbits + nbits += size + while nbits >= 8: + out.append(buf & 0xFF) + buf >>= 8 + nbits -= 8 + if next_code < 4096: + table[key] = next_code + next_code += 1 + if next_code > (1 << size) and size < 12: + size += 1 + else: + # Table full: start over rather than coding on with a stale one. + buf |= clear << nbits + nbits += size + table.clear() + next_code = 258 + size = 9 + prefix = byte + for code in (prefix, end): + buf |= code << nbits + nbits += size + while nbits >= 8: + out.append(buf & 0xFF) + buf >>= 8 + nbits -= 8 + if nbits: + out.append(buf & 0xFF) + + blocks = bytearray() + for i in range(0, len(out), 255): + chunk = out[i:i + 255] + blocks.append(len(chunk)) + blocks += chunk + blocks.append(0) + return bytes(blocks) + + +def encode_gif(frames, out_path, fps, max_size, background): + """Looping GIF from frame files. + + One palette for the whole animation, from the colours used most across + every frame, so the background does not shimmer from frame to frame the + way per-frame palettes make it. + """ + images = [] + for frame in frames: + px = flatten(load_pixels(frame), background) + if max_size and max(px.shape[:2]) > max_size: + px = fit_long_edge(px, max_size) + # GIF stores rows top down. + images.append((px[::-1, :, :3] * 255 + 0.5).astype(np.uint8)) + height, width = images[0].shape[:2] + + # 6 bits a channel keeps 64 levels of grey for a clay render's shading. + keys = [] + for rgb in images: + q = (rgb >> 2).astype(np.uint32) + keys.append((q[..., 0] << 12) | (q[..., 1] << 6) | q[..., 2]) + used, counts = np.unique(np.concatenate([k.ravel() for k in keys]), return_counts=True) + top = used[np.argsort(counts)[::-1][:256]] + + def expand(k): + return np.stack([(k >> 12) & 63, (k >> 6) & 63, k & 63], axis=-1).astype(np.int32) * 4 + 2 + + palette = expand(top) + lut = np.empty(len(used), np.uint8) + colours = expand(used) + for i in range(0, len(used), 4096): + dist = ((colours[i:i + 4096, None, :] - palette[None, :, :]) ** 2).sum(axis=-1) + lut[i:i + 4096] = dist.argmin(axis=1) + + table = np.zeros((256, 3), np.uint8) + table[:len(palette)] = np.clip(palette, 0, 255) + out = bytearray(b"GIF89a") + out += struct.pack("i", p[:, 0], np.cross(p[:, 1], p[:, 2])).sum() + volume *= np.sign(obj.matrix_world.to_3x3().determinant()) or 1.0 + return 0, bool(volume < 0) + + +def fix_normals(coll): + """Make every face of a product point outwards. Returns meshes changed.""" + import bmesh + fixed = 0 + for obj in collection_meshes(coll): + if obj.type != 'MESH' or obj.library or obj.data.library: + continue + bm = bmesh.new() + try: + bm.from_mesh(obj.data) + bmesh.ops.recalc_face_normals(bm, faces=bm.faces) + bm.to_mesh(obj.data) + finally: + bm.free() + obj.data.update() + fixed += 1 + return fixed + + +def _names_clash(settings, targets, views, modes): + seen = set() + for idx, pc in targets: + for view in views: + for mode in modes: + values = name_values(settings, idx, pc.name, view, mode) + key = os.path.normcase(os.path.join(output_folder(settings, values), + output_basename(settings, values))) + if key in seen: + return True + seen.add(key) + return False + + +def _issue(issues, severity, message, product="", fix=""): + issues.append({"severity": severity, "message": message, "product": product, "fix": fix}) + + +def preflight(context, only_active=False): + """Everything likely to waste a render, found before any render time is + spent. A list of {"severity", "message", "product", "fix"}. + + ERROR means the batch cannot produce what was asked for and Render All + will not start. WARNING means it will run but some images will come out + wrong. "fix" names a one-click repair the panel offers, if there is one. + """ + scene = context.scene + s = scene.mv_settings + issues = [] + products = get_products() + + # What will be rendered. + if not products: + _issue(issues, 'ERROR', f"No products under {PRODUCT_COLL}") + return issues + if only_active: + coll, idx = active_product(context) + targets = [(idx, coll)] + else: + targets = [(i, pc) for i, pc in enumerate(products) if pc.mv_render] + if not targets: + _issue(issues, 'ERROR', "No products are ticked for rendering") + return issues + + # Output. + image = scene.render.image_settings + if image.file_format == 'FFMPEG' or getattr(image, "media_type", 'IMAGE') == 'VIDEO': + _issue(issues, 'ERROR', "Output format is video, not image") + raw = s.render_output_dir + # Absolute before probing: in an unsaved file // resolves to a relative + # path, whose parent is "" and would read as unwritable. + out = os.path.abspath(bpy.path.abspath(raw)) + if raw.startswith("//") and not bpy.data.filepath: + _issue(issues, 'WARNING', "Unsaved file: // output path", fix='SAVE_HINT') + if os.path.exists(out) and not os.path.isdir(out): + _issue(issues, 'ERROR', "Output path is a file, not a folder") + else: + probe = out + while probe and not os.path.isdir(probe): + parent = os.path.dirname(probe.rstrip("\\/")) + if parent == probe: + break + probe = parent + if not probe or not os.path.isdir(probe) or not os.access(probe, os.W_OK): + _issue(issues, 'ERROR', "Output folder cannot be written") + elif probe != out and not os.path.isdir(out): + _issue(issues, 'WARNING', "Output folder does not exist yet", fix='FOLDER') + + # Cameras and views. + if s.reframe_on_render: + if not [v for v in planned_views(s) if Vector(v[2]).length >= 1e-9]: + _issue(issues, 'ERROR', "No views selected") + else: + cam_coll = bpy.data.collections.get(CAMERA_COLL) + if not cam_coll or not [o for o in cam_coll.objects if o.type == 'CAMERA']: + _issue(issues, 'ERROR', "No cameras (build, or Reframe)", fix='CAMERAS') + elif cam_coll.get("mv_framing") != framing_signature(context): + _issue(issues, 'WARNING', "Cameras out of date: may crop", + fix='CAMERAS') + + # Names. + views = ([v[0] for v in planned_views(s)] if s.reframe_on_render else + [o.name.replace("MV_Cam_", "") for o in + getattr(bpy.data.collections.get(CAMERA_COLL), "objects", []) + if o.type == 'CAMERA']) + if _names_clash(s, targets, views + (["Sheet"] if s.sheet_enabled else []), + [m for m, _, _ in render_modes(s)]): + _issue(issues, 'ERROR', "Template gives two images one name") + + # Engine-specific passes and staging. + if s.pass_ao and not hasattr(scene, "cycles"): + _issue(issues, 'ERROR', "AO pass needs the Cycles add-on") + full = any(fn is _do_full_render for _, fn, _ in render_modes(s)) or ( + s.turntable_enabled and s.turntable_style == 'FULL') + if full and s.ground_mode == 'SHADOW' and scene.render.engine != 'CYCLES': + _issue(issues, 'WARNING', "Shadow Catcher needs Cycles") + if full and s.world_mode == 'HDRI' and not os.path.isfile(bpy.path.abspath(s.hdri_path)): + _issue(issues, 'WARNING', "HDRI file not found") + if full and scene.render.engine != 'BLENDER_WORKBENCH' and s.world_mode != 'HDRI' \ + and not any(o.type == 'LIGHT' for o in context.view_layer.objects): + _issue(issues, 'WARNING', "No lights: Full renders will be dark", fix='LIGHTING') + if s.lights_follow_camera and bpy.data.collections.get(LIGHT_COLL) \ + and bpy.data.objects.get(LIGHT_RIG) is None: + _issue(issues, 'WARNING', "Rebuild lighting to follow camera", + fix='LIGHTING') + + # The products themselves. + extents = [] + for _, pc in targets: + meshes = collection_meshes(pc) + if not meshes: + _issue(issues, 'WARNING', "Empty: will be skipped", pc.name) + continue + center, size = bbox_of(meshes) + extents.append((center, max(size))) + flipped = inside_out = 0 + for obj in meshes: + if obj.type == 'MESH': + disagree, out_flag = normal_problems(obj) + flipped += disagree + inside_out += out_flag + if flipped: + _issue(issues, 'WARNING', f"Normals flipped on {flipped} edge(s)", pc.name, + fix='NORMALS') + if inside_out: + _issue(issues, 'WARNING', f"{inside_out} mesh(es) inside out", pc.name, + fix='NORMALS') + if len(extents) > 1: + # Framing covers the whole batch, so products spread around the scene + # each come out as a speck in the middle of an empty frame. + biggest = max(e for _, e in extents) + spread = max((c - extents[0][0]).length for c, _ in extents) + if spread > biggest * 0.5: + _issue(issues, 'WARNING', "Products not centred: render small", + fix='RECENTER') + + strays = stray_render_objects(context) + if strays: + _issue(issues, 'WARNING', f"{len(strays)} stray object(s) in every shot", + fix='STRAYS') + return issues + + +def store_issues(context, issues): + s = context.scene.mv_settings + s.preflight_issues.clear() + for issue in issues: + item = s.preflight_issues.add() + for key, value in issue.items(): + setattr(item, key, value) + s.preflight_time = time.strftime("%H:%M:%S") + + +class MV_OT_Preflight(Operator): + """Check the batch for anything that would waste a render: empty products, + flipped normals, stale cameras, an unwritable output folder and more""" + bl_idname = "mv.preflight" + bl_label = "Check Batch" + + def execute(self, context): + issues = preflight(context) + store_issues(context, issues) + errors = sum(1 for i in issues if i["severity"] == 'ERROR') + if not issues: + self.report({'INFO'}, "Pre-flight: all clear.") + else: + self.report({'ERROR'} if errors else {'WARNING'}, + f"Pre-flight: {errors} error(s), {len(issues) - errors} warning(s).") + return {'FINISHED'} + + +class MV_OT_PreflightFix(Operator): + """Repair what this pre-flight issue found""" + bl_idname = "mv.preflight_fix" + bl_label = "Fix" + bl_options = {'REGISTER', 'UNDO'} + + fix: StringProperty() + product: StringProperty() + + @classmethod + def description(cls, context, properties): + return { + 'NORMALS': "Recalculate this product's normals to point outwards", + 'CAMERAS': "Rebuild the cameras for the batch as it is now", + 'FOLDER': "Create the output folder", + 'LIGHTING': "Rebuild the lighting rig from the selected preset", + 'RECENTER': "Move every product's centre onto the origin", + 'STRAYS': "Switch off rendering for the objects outside products", + 'SAVE_HINT': "Save the .blend file so // has somewhere to point", + }.get(properties.fix, "Repair this issue") + + def execute(self, context): + s = context.scene.mv_settings + if self.fix == 'NORMALS': + coll = bpy.data.collections.get(self.product) + if coll is not None: + fix_normals(coll) + elif self.fix == 'CAMERAS': + build_cameras(context) + elif self.fix == 'FOLDER': + try: + os.makedirs(bpy.path.abspath(s.render_output_dir), exist_ok=True) + except OSError as exc: + self.report({'ERROR'}, f"Could not create the folder: {exc}") + return {'CANCELLED'} + elif self.fix == 'LIGHTING': + build_lighting(context, s.lighting_preset) + elif self.fix == 'RECENTER': + standardize_products(context, get_products(), scale=False, recenter=True) + elif self.fix == 'STRAYS': + for obj in stray_render_objects(context): + obj.hide_render = True + elif self.fix == 'SAVE_HINT': + # The file browser takes over from here; check again once saved. + bpy.ops.wm.save_as_mainfile('INVOKE_DEFAULT') + return {'FINISHED'} + store_issues(context, preflight(context)) + return {'FINISHED'} + + +class MV_OT_RenderPreflightPrompt(Operator): + """Pre-flight found problems; render anyway?""" + bl_idname = "mv.render_preflight_prompt" + bl_label = "Pre-flight Warnings" + + only_active: BoolProperty(options={'SKIP_SAVE', 'HIDDEN'}) + + def invoke(self, context, event): + try: + return context.window_manager.invoke_props_dialog( + self, width=420, title="Pre-flight found problems", + confirm_text="Render Anyway") + except TypeError: + return context.window_manager.invoke_props_dialog(self, width=420) + + def draw(self, context): + col = self.layout.column(align=True) + issues = context.scene.mv_settings.preflight_issues + for item in list(issues)[:10]: + text = f"{item.product}: {item.message}" if item.product else item.message + col.label(text=text, icon='ERROR') + if len(issues) > 10: + col.label(text=f"+{len(issues) - 10} more in the Pre-flight panel") + + def execute(self, context): + return bpy.ops.mv.render_all('INVOKE_DEFAULT', only_active=self.only_active, + skip_preflight=True) + + +# -- the batch ---------------------------------------------------------------- + +class MV_OT_RenderAll(Operator): + """Render every ticked product from every camera. + + Runs modally, one job per timer tick: Blender redraws between images, the + status bar shows progress and the time left, and Esc stops the batch after + the job in flight. Called from a script it runs straight through. + + A job is one rendered image, one post-process of an image already on disk, + one contact sheet, one turntable frame or one turntable encode. + """ + bl_idname = "mv.render_all" + bl_label = "Render All" + + skip_preflight: BoolProperty(options={'SKIP_SAVE', 'HIDDEN'}) + only_active: BoolProperty( + name="Active Only", + description="Render only the active product, whether it is ticked or not", + default=False, + options={'SKIP_SAVE'}, + ) + + @classmethod + def description(cls, context, properties): + if properties.only_active: + return "Render only the active product, from every camera" + return "Render every ticked product, from every camera" + + # -- state ------------------------------------------------------------ + def _save_state(self, context, products): + scene = context.scene + self._saved_hide = {} + for pc in products: + lc = find_layer_coll(context.view_layer.layer_collection, pc.name) + if lc: + self._saved_hide[pc.name] = lc.exclude + # By name, not by reference. Reframing deletes and rebuilds the rig, so + # a scene looking through MV_Cam_Front would be holding a removed + # object by the time it was put back. + self._saved_cam = scene.camera.name if scene.camera else None + self._saved_path = scene.render.filepath + + def _restore(self, context): + scene = context.scene + self._end_turntable_pose() + cam = bpy.data.objects.get(TURNTABLE_CAM) + if cam is not None: + remove_object(cam) + reset_light_rig() + remove_product_materials() + ground = bpy.data.objects.get(GROUND_NAME) + if ground is not None and getattr(self, "_ground_home", None) is not None: + ground.location = self._ground_home + ground.hide_render = False + for pc in get_products(): + lc = find_layer_coll(context.view_layer.layer_collection, pc.name) + if lc and pc.name in self._saved_hide: + lc.exclude = self._saved_hide[pc.name] + scene.camera = bpy.data.objects.get(self._saved_cam) if self._saved_cam else None + scene.render.filepath = self._saved_path + if self._tmp_dir: + shutil.rmtree(self._tmp_dir, ignore_errors=True) + self._tmp_dir = None + + def _tmp(self): + if not self._tmp_dir: + self._tmp_dir = tempfile.mkdtemp(prefix="mv_render_") + return self._tmp_dir + + def _isolate(self, context, product): + if self._isolated != product: + isolate_product(context, product) + self._isolated = product + # Materials go on with the product rather than per image: the + # clay, line, mask and AO passes ignore them anyway. + remove_product_materials() + coll = bpy.data.collections.get(product) + if coll is not None: + apply_product_material(context.scene.mv_settings, coll) + + def _stage(self, context, job, camera): + """Turn the lights to this camera and stand the ground under this + product, for the image about to be rendered.""" + s = context.scene.mv_settings + if s.lights_follow_camera: + orient_light_rig(context, camera) + ground = bpy.data.objects.get(GROUND_NAME) + if ground is None: + return + floor = self._ground_z.get(job["product"]) + if floor is None: + center, size = bbox_of(collection_meshes(bpy.data.collections[job["product"]])) + floor = self._ground_z[job["product"]] = center.z - size.z * 0.5 + ground.location.z = floor + # Full renders only: clay and the other passes are about the product's + # shape. A camera at or below the floor would see its underside, which + # hides the product altogether from Bottom. + ground.hide_render = (job["render"] is not _do_full_render + or camera.matrix_world.translation.z <= floor + 1e-6) + + def _record(self, rec, kind, path, width=None, height=None): + self._records.append(dict(rec, kind=kind, path=path, width=width, height=height)) + + def _render_size(self, scene): + r = scene.render + return (int(r.resolution_x * r.resolution_percentage / 100), + int(r.resolution_y * r.resolution_percentage / 100)) + + # -- planning --------------------------------------------------------- + def _claim(self, path): + key = os.path.normcase(os.path.abspath(path)) + if key in self._claimed: + raise _NameClash(path) + self._claimed.add(key) + + def _meta(self, idx, pc): + s = bpy.context.scene.mv_settings + size = list(pc.mv_source_size) + return { + "label": product_label(s.start_prefix, s.start_number, idx, s.products_per_letter), + "product": pc.name, + "source_file": pc.mv_source or "", + "source_size": [round(v, 6) for v in size] if any(size) else None, + "source_units": pc.mv_source_units if any(size) else "", + } + + def _post_targets(self, rel_dir, base): + s = bpy.context.scene.mv_settings + folder = os.path.join(self._out_root, safe_name(s.post_folder, "processed"), rel_dir) + ext = POST_FORMATS[s.post_format] + targets = [(os.path.join(folder, base + ext), 0)] + targets += [(os.path.join(folder, f"{base}_{size}{ext}"), size) + for size in parse_sizes(s.post_extra_sizes)] + return targets + + def _plan(self, context, targets, cameras): + """Every job this run will do, in order, minus outputs already on disk.""" + scene = context.scene + s = scene.mv_settings + root = self._out_root + modes = render_modes(s) + + for idx, pc in targets: + meta = self._meta(idx, pc) + self._labels.add(meta["label"]) + name = f"{meta['label']}_{pc.name}" + first_job = len(self._queue) + sheet_cells = {mode: [] for mode, _, _ in modes} + + for cam in cameras: + view = cam.name.replace("MV_Cam_", "") + for mode, render, extra in modes: + values = name_values(s, idx, pc.name, view, mode) + rel_dir = output_folder(s, values) + base = output_basename(s, values) + path = os.path.join(root, rel_dir, base) + self._claim(path) + master = written_path(scene, path) + post = self._post_targets(rel_dir, base) if s.post_enabled else [] + for target, _ in post: + self._claim(target) + rec = dict(meta, view=view, mode=mode) + sheet_cells[mode].append((view, post[0][0] if post else master)) + + have_master = s.skip_existing and file_ok(master) + if have_master: + self._record(rec, "render", master, *self._render_size(scene)) + if all(file_ok(t) for t, _ in post): + self._skipped += 1 + for target, _ in post: + self._record(rec, "processed", target) + continue + self._queue.append({ + "kind": "post" if have_master else "render", + "product": pc.name, "status": f"{name} {view}", + "camera": cam.name, "render": render, "path": path, + "master": master, "post": post, "rec": rec, + "extra": extra, + }) + + if s.sheet_enabled: + touched = len(self._queue) > first_job + for mode, cells in sheet_cells.items(): + values = name_values(s, idx, pc.name, "Sheet", mode) + path = os.path.join(root, output_folder(s, values), + output_basename(s, values) + ".png") + self._claim(path) + rec = dict(meta, view="Sheet", mode=mode) + if s.skip_existing and not touched and file_ok(path): + self._record(rec, "sheet", path) + continue + title = f"{name} ({mode})" if mode else name + self._queue.append({"kind": "sheet", "status": f"{name} contact sheet", + "path": path, "cells": cells, "title": title, + "rec": rec}) + + if s.turntable_enabled: + self._plan_turntable(context, idx, pc, meta) + + def _plan_turntable(self, context, idx, pc, meta): + scene = context.scene + s = scene.mv_settings + root = self._out_root + clay = s.turntable_style == 'CLAY' + mode = "clay" if clay else "" + values = name_values(s, idx, pc.name, "Turntable", mode) + rel_dir = output_folder(s, values) + base = output_basename(s, values) + name = f"{meta['label']}_{pc.name}" + rec = dict(meta, view="Turntable", mode=mode) + + outputs = [] + if s.turntable_mp4: + outputs.append(("mp4", os.path.join(root, rel_dir, base + ".mp4"))) + if s.turntable_gif: + outputs.append(("gif", os.path.join(root, rel_dir, base + ".gif"))) + for _, path in outputs: + self._claim(path) + + n = s.turntable_frames + frames, missing = [], [] + for k in range(n): + path = os.path.join(root, rel_dir, "turntable", f"{base}_{k + 1:03d}") + self._claim(path) + master = written_path(scene, path) + frames.append(master) + if s.skip_existing and file_ok(master): + if not outputs: + self._record(rec, "turntable_frame", master, *self._render_size(scene)) + continue + missing.append((k, path, master)) + + self._skipped += n - len(missing) + if s.skip_existing and not missing and all(file_ok(p) for _, p in outputs): + for _, path in outputs: + self._record(rec, "turntable", path) + return + for k, path, master in missing: + self._queue.append({ + "kind": "turn_frame", "product": pc.name, + "status": f"{name} turntable {k + 1}/{n}", + "angle": 2.0 * math.pi * k / n, "path": path, "master": master, + "render": _do_clay_render if clay else _do_full_render, + "extra": (tuple(s.clay_color),) if clay else (), + "rec": rec if not outputs else None, + }) + if outputs: + self._queue.append({"kind": "turn_encode", "product": pc.name, + "status": f"{name} encoding turntable", + "frames": frames, "outputs": outputs, "rec": rec}) + + def _setup(self, context): + """Build the queue. None when there is work to do, otherwise the + result the operator should return straight away.""" + scene = context.scene + s = scene.mv_settings + self._queue = [] + self._failures = [] + self._notes = [] + self._records = [] + self._labels = set() + self._claimed = set() + self._skipped = 0 + self._done = 0 + self._landed = 0 + self._renders_done = 0 + self._n_sheets = 0 + self._n_post = 0 + self._n_turntables = 0 + self._total = 0 + self._isolated = None + self._render_time = 0.0 + self._started = time.monotonic() + self._timer = None + self._tmp_dir = None + self._turn_pose = None + self._ground_z = {} + self._ground_home = None + self._cancelled = False + self._aborted = False + + products = get_products() + if not products: + self.report({'ERROR'}, f"No product collections under '{PRODUCT_COLL}'.") + return {'CANCELLED'} + + if self.only_active: + coll, idx = active_product(context) + targets = [(idx, coll)] + else: + # Index into the whole batch, not the ticked subset: the label is + # the product's identity on disk, and unticking AA_02 must not turn + # AA_03 into a second AA_02. + targets = [(i, pc) for i, pc in enumerate(products) if pc.mv_render] + if not targets: + self.report({'ERROR'}, "No products are ticked for rendering.") + return {'CANCELLED'} + + # An empty product would only ever render blank frames. + empty = [pc.name for _, pc in targets if not collection_meshes(pc)] + if empty: + targets = [(i, pc) for i, pc in targets if pc.name not in empty] + self._notes.append(f"Skipped {len(empty)} empty product(s): {', '.join(empty[:3])}" + + ("..." if len(empty) > 3 else "") + ".") + if not targets: + self.report({'ERROR'}, "Every product to render is empty.") + return {'CANCELLED'} + + if s.reframe_on_render and not planned_views(s): + self.report({'ERROR'}, "No views selected. Tick at least one view.") + return {'CANCELLED'} + + self._out_root = bpy.path.abspath(s.render_output_dir) + try: + os.makedirs(self._out_root, exist_ok=True) + except OSError as exc: + self.report({'ERROR'}, f"Cannot use output folder '{self._out_root}': {exc}") + return {'CANCELLED'} + + self._save_state(context, products) + # Every product into the view layer before anything is measured. One + # hidden by isolation is not evaluated, so a child part of an assembly + # rotated or fitted while it was excluded can still report where it + # used to be. The restore at the end puts the user's hiding back. + isolate_product(context, None) + # Anything a crashed run left on the products goes before measuring. + remove_product_materials() + context.view_layer.update() + + # Rebuilt every run so it always matches the panel, and so Ground set + # to None clears one an earlier run left behind. + if s.ground_mode != 'NONE' or bpy.data.objects.get(GROUND_NAME): + ground = build_ground(context) + if ground is not None: + self._ground_home = ground.location.copy() + if s.ground_mode == 'SHADOW' and scene.render.engine != 'CYCLES': + self._notes.append("Shadow Catcher needs Cycles; other engines " + "show the ground as a plain floor.") + if s.lights_follow_camera and bpy.data.collections.get(LIGHT_COLL) \ + and bpy.data.objects.get(LIGHT_RIG) is None: + self._notes.append("Lights Follow Camera needs the lighting rebuilt once " + "(Scene Setup > Lighting Only).") + + if s.reframe_on_render: + # Framing covers the whole batch, ticked or not, so re-rendering a + # single product later lands at the same scale as its run. + skipped = build_cameras(context) + if skipped: + self._notes.append("Skipped custom view(s) with a zero direction: " + + ", ".join(skipped) + ".") + + cam_coll = bpy.data.collections.get(CAMERA_COLL) + cameras = [o for o in cam_coll.objects if o.type == 'CAMERA'] if cam_coll else [] + if not cameras: + self._restore(context) + self.report({'ERROR'}, "No cameras. Build cameras first.") + return {'CANCELLED'} + + try: + self._plan(context, targets, cameras) + except _NameClash as clash: + self._restore(context) + self.report({'ERROR'}, f"Two outputs would share the name " + f"'{os.path.basename(str(clash))}'. Add {{view}} " + "and {mode} to the file name template.") + return {'CANCELLED'} + + if any(job["kind"] == "turn_frame" for job in self._queue): + if build_turntable_camera(context) is None: + self._queue = [j for j in self._queue if j["kind"] not in {"turn_frame", "turn_encode"}] + self._notes.append("No geometry to frame a turntable around.") + + self._total = len(self._queue) + if not self._queue: + self._restore(context) + self._write_manifest(context) + self.report({'INFO'}, f"Nothing to render: all {self._skipped} image(s) " + "are already on disk.") + return {'FINISHED'} + return None + + # -- jobs ------------------------------------------------------------- + def _fail(self, path, exc): + """Record one failed job. False when the batch should stop. + + Collected and reported at the end rather than raised, so a single + unwritable file cannot throw away a long batch that is otherwise fine. + Before the first success there is nothing to salvage and the cause is + almost always the setup, so that case stops immediately instead of + failing the same way for every remaining job. + """ + note = f"{os.path.basename(path)}: {exc}" + self._failures.append(note) + print(f"[MultiView] FAILED {note}") + if not self._landed: + self._aborted = True + self._queue.clear() + return False + return bool(self._queue) + + def _render_image(self, context, job, camera): + scene = context.scene + self._isolate(context, job["product"]) + os.makedirs(os.path.dirname(job["path"]), exist_ok=True) + cam = bpy.data.objects.get(camera) + if cam is None or cam.type != 'CAMERA': + raise RuntimeError(f"camera '{camera}' no longer exists") + scene.camera = cam + self._stage(context, job, cam) + started = time.monotonic() + # bpy.ops.render.render raises RuntimeError when the image cannot be + # written; _advance turns that into a named failure, not a traceback. + job["render"](scene, job["path"], *job["extra"]) + self._render_time += time.monotonic() - started + self._renders_done += 1 + self._done += 1 + print(f"[MultiView] {self._processed()}/{self._total}: {scene.render.filepath}") + + def _job_render(self, context, job): + self._render_image(context, job, job["camera"]) + self._record(job["rec"], "render", job["master"], *self._render_size(context.scene)) + if job["post"]: + self._job_post(context, job) + + def _job_post(self, context, job): + s = context.scene.mv_settings + px = post_process(load_pixels(job["master"]), s) + fmt = s.post_format + for target, size in job["post"]: + os.makedirs(os.path.dirname(target), exist_ok=True) + out = fit_long_edge(px, size) if size and max(px.shape[:2]) > size else px + save_pixels(out, target, fmt, s.post_quality) + self._record(job["rec"], "processed", target, out.shape[1], out.shape[0]) + self._n_post += 1 + + def _job_sheet(self, context, job): + s = context.scene.mv_settings + cells = [] + for caption, path in job["cells"]: + if file_ok(path): + px = load_pixels(path) + h, w = px.shape[:2] + cells.append((caption, resize_pixels(px, s.sheet_cell_width, + round(h * s.sheet_cell_width / w)))) + if not cells: + raise RuntimeError("none of its images are on disk") + sheet = contact_sheet(cells, job["title"], s, self._tmp()) + os.makedirs(os.path.dirname(job["path"]), exist_ok=True) + save_pixels(sheet, job["path"], 'PNG') + self._record(job["rec"], "sheet", job["path"], sheet.shape[1], sheet.shape[0]) + self._n_sheets += 1 + + def _pose_turntable(self, context, product, angle): + """Spin one product about its own centre to angle, measured from where + it stood before the turntable began. Absolute, so skipped frames and + rounding never accumulate into drift.""" + if self._turn_pose and self._turn_pose["product"] != product: + self._end_turntable_pose() + if self._turn_pose is None: + coll = bpy.data.collections.get(product) + if coll is None: + raise RuntimeError(f"product '{product}' no longer exists") + center, _ = bbox_of(collection_meshes(coll)) + self._turn_pose = { + "product": product, "center": center, + "saved": {o.name: o.matrix_world.copy() for o in collection_roots(coll)}, + } + pose = self._turn_pose + spin = (Matrix.Translation(pose["center"]) + @ Matrix.Rotation(angle, 4, 'Z') + @ Matrix.Translation(-pose["center"])) + for name, matrix in pose["saved"].items(): + obj = bpy.data.objects.get(name) + if obj is not None: + obj.matrix_world = spin @ matrix + + def _end_turntable_pose(self): + pose, self._turn_pose = getattr(self, "_turn_pose", None), None + if not pose: + return + for name, matrix in pose["saved"].items(): + obj = bpy.data.objects.get(name) + if obj is not None: + obj.matrix_world = matrix + + def _job_turn_frame(self, context, job): + self._isolate(context, job["product"]) + self._pose_turntable(context, job["product"], job["angle"]) + self._render_image(context, job, TURNTABLE_CAM) + if job["rec"] is not None: + self._record(job["rec"], "turntable_frame", job["master"], + *self._render_size(context.scene)) + + def _job_turn_encode(self, context, job): + self._end_turntable_pose() + s = context.scene.mv_settings + frames = [f for f in job["frames"] if file_ok(f)] + if len(frames) < 2: + raise RuntimeError("fewer than two turntable frames on disk") + background = tuple(s.bg_color[:3]) + errors = [] + for kind, path in job["outputs"]: + # Each format on its own, so a video codec problem does not also + # cost the GIF. + try: + os.makedirs(os.path.dirname(path), exist_ok=True) + if kind == "mp4": + size = encode_mp4(frames, path, s.turntable_fps, background, self._tmp()) + else: + size = encode_gif(frames, path, s.turntable_fps, s.turntable_gif_size, + background) + except (RuntimeError, OSError, ValueError) as exc: + errors.append(f"{kind.upper()}: {exc}") + continue + self._record(job["rec"], "turntable", path, *size) + if len(errors) < len(job["outputs"]): + self._n_turntables += 1 + if errors: + raise RuntimeError("; ".join(errors)) + + def _advance(self, context): + """Run the next job. False once the batch is over.""" + if not self._queue: + return False + job = self._queue.pop(0) + handler = { + "render": self._job_render, "post": self._job_post, + "sheet": self._job_sheet, "turn_frame": self._job_turn_frame, + "turn_encode": self._job_turn_encode, + }[job["kind"]] + try: + handler(context, job) + except Exception as exc: + if not isinstance(exc, (RuntimeError, OSError)): + # Not a write failure: a bug, so keep the trace for a report. + traceback.print_exc() + target = job.get("path") or job["outputs"][0][1] + return self._fail(target, exc) + self._landed += 1 + return bool(self._queue) + + # -- progress --------------------------------------------------------- + def _processed(self): + return self._total - len(self._queue) + + def _status(self): + text = f"MultiView: {self._processed()}/{self._total}" + if self._queue: + text += f" {self._queue[0]['status']}" + if self._renders_done: + left = sum(1 for j in self._queue if j["kind"] in {"render", "turn_frame"}) + text += f" ~{format_duration(self._render_time / self._renders_done * left)} left" + return text + " - Esc to cancel" + + def _write_manifest(self, context): + fmt = context.scene.mv_settings.manifest_format + if fmt == 'NONE' or not self._records: + return + try: + write_manifest(self._out_root, self._records, self._labels, fmt) + except (OSError, ValueError) as exc: + self._notes.append(f"Manifest not written: {exc}.") + + def _report_result(self): + elapsed = format_duration(time.monotonic() - self._started) + extras = [] + if self._n_post: + extras.append(f"post-processed {self._n_post}") + if self._n_sheets: + extras.append(f"{self._n_sheets} contact sheet(s)") + if self._n_turntables: + extras.append(f"{self._n_turntables} turntable(s)") + made = f"Wrote {self._done} image(s)" + (f", {', '.join(extras)}" if extras else "") + skipped = (f" Skipped {self._skipped} already on disk." + if self._skipped else "") + notes = (" " + " ".join(self._notes)) if self._notes else "" + + if self._aborted: + # Nothing landed at all, so the batch stopped at the first failure + # rather than burning render time failing the same way every time. + self.report({'ERROR'}, f"Nothing rendered. {self._failures[0]}") + return {'CANCELLED'} + if self._cancelled: + self.report({'WARNING'}, + f"Render cancelled after {self._processed()} of {self._total} " + f"job(s). {made} in {elapsed}.{skipped} Tick Skip Existing " + "to resume.") + return {'FINISHED'} if self._landed else {'CANCELLED'} + if self._failures: + self.report({'WARNING'}, + f"{made} in {elapsed}. {len(self._failures)} failed, first: " + f"{self._failures[0]}. Full list in the System Console." + f"{skipped}{notes}") + return {'FINISHED'} + self.report({'WARNING'} if self._notes else {'INFO'}, + f"{made} in {elapsed}.{skipped}{notes}") + return {'FINISHED'} + + def _finish(self, context): + self._restore(context) + self._write_manifest(context) + return self._report_result() + + def _teardown(self, context): + wm = context.window_manager + if self._timer is not None: + wm.event_timer_remove(self._timer) + self._timer = None + wm.progress_end() + try: + context.workspace.status_text_set(None) + except AttributeError: + pass + return self._finish(context) + + # -- entry points ----------------------------------------------------- + def _preflight_errors(self, context, issues): + errors = [i for i in issues if i["severity"] == 'ERROR'] + if not errors: + return None + first = errors[0] + where = f"{first['product']}: " if first["product"] else "" + more = f" (+{len(errors) - 1} more, see the Pre-flight panel)" if len(errors) > 1 else "" + self.report({'ERROR'}, f"Pre-flight: {where}{first['message']}{more}.") + return {'CANCELLED'} + + def execute(self, context): + # Scripts get the same hard stops as the button but no question: + # warnings go into the report instead. + if not self.skip_preflight and context.scene.mv_settings.preflight_on_render: + issues = preflight(context, self.only_active) + store_issues(context, issues) + stop = self._preflight_errors(context, issues) + if stop: + return stop + result = self._setup(context) + if result is not None: + return result + try: + while self._advance(context): + pass + except BaseException: + self._restore(context) + raise + return self._finish(context) + + def invoke(self, context, event): + if not self.skip_preflight and context.scene.mv_settings.preflight_on_render: + issues = preflight(context, self.only_active) + store_issues(context, issues) + stop = self._preflight_errors(context, issues) + if stop: + return stop + if issues: + # Asked in a dialog of its own, which starts the render again + # with the check skipped if the answer is to go ahead. + bpy.ops.mv.render_preflight_prompt('INVOKE_DEFAULT', only_active=self.only_active) + return {'CANCELLED'} + result = self._setup(context) + if result is not None: + return result + wm = context.window_manager + wm.progress_begin(0, self._total) + # Short, because each tick blocks for a whole render anyway. All the + # interval decides is how long Blender idles between two jobs. + self._timer = wm.event_timer_add(0.05, window=context.window) + wm.modal_handler_add(self) + try: + context.workspace.status_text_set(self._status()) + except AttributeError: + pass + return {'RUNNING_MODAL'} + + def modal(self, context, event): + if event.type == 'ESC' and event.value == 'PRESS': + self._cancelled = True + return self._teardown(context) + if event.type != 'TIMER': + # Swallow everything else: products are being hidden, shown and + # spun under the user's feet, so a click in the outliner mid-batch + # would be saved over by the restore at the end. + return {'RUNNING_MODAL'} + + try: + more = self._advance(context) + except Exception: + # Leave the scene the way it was found before the traceback, rather + # than stuck on one product with the timer still firing. + self._teardown(context) + raise + context.window_manager.progress_update(self._processed()) + try: + context.workspace.status_text_set(self._status()) + except AttributeError: + pass + if not more: + return self._teardown(context) + return {'RUNNING_MODAL'} + + +class MV_UL_StepFiles(UIList): + def draw_item(self, context, layout, data, item, icon, active_data, active_propname, index): + row = layout.row(align=True) + row.label(text=os.path.basename(item.path) or "(empty)", icon='FILE') + row.operator("mv.remove_step_file", text="", icon='X').index = index + + +class MV_UL_CustomViews(UIList): + def draw_item(self, context, layout, data, item, icon, active_data, active_propname, index): + row = layout.row(align=True) + row.prop(item, "enabled", text="") + row = row.row(align=True) + row.active = item.enabled + row.prop(item, "name", text="") + row.prop(item, "dir_x", text="X") + row.prop(item, "dir_y", text="Y") + row.prop(item, "dir_z", text="Z") + row.prop(item, "is_ortho", text="", icon='VIEW_ORTHO' if item.is_ortho else 'VIEW_PERSPECTIVE') + row.operator("mv.remove_custom_view", text="", icon='X').index = index + + +class MV_UL_Products(UIList): + """MV_Products' children, each with its render tick and the label its + files will carry. The label is worked out from the product's place in the + whole batch, which is also what Render All uses, so unticking a product + never renumbers the ones after it.""" + + def draw_item(self, context, layout, data, item, icon, active_data, active_propname, index): + s = context.scene.mv_settings + row = layout.row(align=True) + row.prop(item, "mv_render", text="") + sub = row.row(align=True) + sub.active = item.mv_render + label = product_label(s.start_prefix, s.start_number, index, s.products_per_letter) + sub.label(text=f"{label} {item.name}", icon='OUTLINER_COLLECTION') + + +class MV_PT_Guide(Panel): + bl_label = "Guide" + bl_space_type = 'VIEW_3D' + bl_region_type = 'UI' + bl_category = "MultiView" + bl_options = {'DEFAULT_CLOSED'} + + def draw(self, context): + layout = self.layout + box = layout.box() + for line in GUIDE_LINES: + box.label(text=line) + + +class MV_PT_SceneSetup(Panel): + bl_label = "Scene Setup" + bl_space_type = 'VIEW_3D' + bl_region_type = 'UI' + bl_category = "MultiView" + + def draw(self, context): + layout = self.layout + s = context.scene.mv_settings + + col = layout.column(align=True) + col.label(text="Lighting:") + row = col.row(align=True) + row.prop(s, "lighting_preset", text="") + row.operator("mv.save_lighting_preset", text="", icon='ADD') + row.operator("mv.delete_lighting_preset", text="", icon='REMOVE') + col.prop(s, "lighting_intensity", slider=True) + layout.prop(s, "lights_follow_camera") + + col = layout.column(align=True) + col.label(text="Background:") + col.row(align=True).prop(s, "world_mode", expand=True) + if s.world_mode == 'HDRI': + col.prop(s, "hdri_path", text="") + row = col.row(align=True) + row.prop(s, "hdri_strength") + row.prop(s, "hdri_rotation") + col.prop(s, "hdri_visible") + col.prop(s, "transparent_bg") + row = col.row() + # The colour still shows behind an HDRI the camera is not shown. + row.enabled = not s.transparent_bg and not (s.world_mode == 'HDRI' and s.hdri_visible) + row.prop(s, "bg_color", text="") + + col = layout.column(align=True) + col.label(text="Ground (Full renders):") + col.prop(s, "ground_mode", text="") + if s.ground_mode == 'FLOOR': + row = col.row(align=True) + row.prop(s, "ground_color", text="") + row.prop(s, "ground_roughness") + elif s.ground_mode == 'SHADOW' and context.scene.render.engine != 'CYCLES': + col.label(text="Shadow Catcher needs Cycles.", icon='ERROR') + + layout.separator() + layout.operator("mv.apply_scene_setup", icon='SCENE_DATA') + + row = layout.row(align=True) + row.operator("mv.build_lighting", text="Lighting Only", icon='LIGHT') + row.operator("mv.apply_world", text="Background Only", icon='WORLD') + + +class MV_PT_SceneMaterials(Panel): + bl_label = "Materials" + bl_space_type = 'VIEW_3D' + bl_region_type = 'UI' + bl_category = "MultiView" + bl_parent_id = "MV_PT_SceneSetup" + bl_options = {'DEFAULT_CLOSED'} + + def draw(self, context): + layout = self.layout + s = context.scene.mv_settings + layout.label(text="Whole batch:") + row = layout.row(align=True) + row.prop(s, "material_preset", text="") + sub = row.row(align=True) + sub.active = s.material_preset != 'KEEP' + sub.prop(s, "material_color", text="") + + coll, _ = active_product(context) + if coll is None: + return + layout.label(text=f"{coll.name}:", icon='OUTLINER_COLLECTION') + row = layout.row(align=True) + row.prop(coll, "mv_material", text="") + sub = row.row(align=True) + sub.active = coll.mv_material not in {'DEFAULT', 'KEEP'} + sub.prop(coll, "mv_material_color", text="") + layout.label(text="Full renders only. Pick a product in", icon='INFO') + layout.label(text="Products to render to set its own.") + + +class MV_PT_Import(Panel): + bl_label = "Import Products" + bl_space_type = 'VIEW_3D' + bl_region_type = 'UI' + bl_category = "MultiView" + + def draw(self, context): + layout = self.layout + s = context.scene.mv_settings + addon = context.preferences.addons.get(__name__) + + box = layout.box() + box.label(text="STEP / IGES Importer:") + if addon is not None: + prefs = addon.preferences + box.prop(prefs, "step_import_mode", text="Mode") + col = box.column() + col.enabled = prefs.step_import_mode in {'AUTO', 'FREECAD'} + col.prop(prefs, "freecad_path", text="FreeCAD") + box.label(text="IGES always uses FreeCAD.", icon='INFO') + box.label(text="Mesh files (STL, OBJ, ...) are in:") + box.prop(s, "mesh_units", text="") + + layout.separator() + row = layout.row() + row.template_list("MV_UL_StepFiles", "", s, "step_files", s, "step_files_index", rows=3) + col = row.column(align=True) + col.operator("mv.add_step_file", text="", icon='ADD') + col.operator("mv.add_folder", text="", icon='FILE_FOLDER') + col.operator("mv.clear_step_files", text="", icon='TRASH') + layout.label(text="STEP IGES STL OBJ PLY glTF FBX 3MF", icon='FILE') + + layout.operator("mv.import_all_steps", icon='IMPORT') + layout.operator("mv.add_product_collection", icon='OUTLINER_COLLECTION') + + prods = get_products() + header = layout.row(align=True) + header.prop(s, "show_products_list", text="", + icon='TRIA_DOWN' if s.show_products_list else 'TRIA_RIGHT', emboss=False) + header.label(text=f"Products under '{PRODUCT_COLL}' ({len(prods)})") + if s.show_products_list: + box = layout.box() + if not prods: + box.label(text="(none)", icon='INFO') + else: + for pc in prods: + box.label(text=f"{pc.name} ({len(pc.all_objects)} objs)", icon='OUTLINER_COLLECTION') + + +class MV_PT_Standardize(Panel): + bl_label = "Standardize" + bl_space_type = 'VIEW_3D' + bl_region_type = 'UI' + bl_category = "MultiView" def draw(self, context): layout = self.layout @@ -2008,9 +5634,13 @@ def draw(self, context): row = layout.row(align=True) row.prop(s, "recenter", toggle=True) row.prop(s, "rescale", toggle=True) + col = layout.column(align=True) + col.enabled = s.rescale or s.fit_on_import + col.row(align=True).prop(s, "scale_mode", expand=True) + col.prop(s, "target_size") sub = layout.row() - sub.enabled = s.rescale or s.fit_on_import - sub.prop(s, "target_size") + sub.enabled = s.recenter or s.fit_on_import + sub.prop(s, "rest_on_floor") row = layout.row(align=True) row.prop(s, "auto_smooth", toggle=True) @@ -2021,6 +5651,13 @@ def draw(self, context): layout.separator() layout.operator("mv.standardize", icon='FULLSCREEN_EXIT') + coll, _ = active_product(context) + if coll is not None: + real = format_real_size(coll) + col = layout.column(align=True) + col.label(text=f"{coll.name}, as imported:", icon='OUTLINER_COLLECTION') + col.label(text=real or "size not recorded yet") + class MV_PT_Cameras(Panel): bl_label = "Cameras / Views" @@ -2032,6 +5669,26 @@ def draw(self, context): layout = self.layout s = context.scene.mv_settings layout.prop(s, "view_set") + + # A preset is drawn from VIEW_SETS, not from the checkboxes, because + # that is what build_cameras reads for it. Only Custom edits them. + custom = s.view_set == 'CUSTOM' + preset = set() if custom else {v[0] for v in VIEW_SETS[s.view_set]} + ortho = layout.grid_flow(row_major=True, columns=3, even_columns=True, align=True) + persp = layout.row(align=True) + persp.label(text="Persp:") + for i, (name, kind, _) in enumerate(VIEWS_ALL): + # Perspective names are too long for a third of the sidebar, so + # they get their own row and drop the shared prefix. + cell = (ortho if kind == "ortho" else persp).row() + text = name.replace("Persp_", "") + if custom: + cell.prop(s, "views_enabled", index=i, text=text) + else: + cell.enabled = False + cell.label(text=text, + icon='CHECKBOX_HLT' if name in preset else 'CHECKBOX_DEHLT') + layout.prop(s, "front_axis") layout.prop(s, "frame_padding_factor") @@ -2065,11 +5722,23 @@ def draw(self, context): row.operator("mv.rotate_product", text="Up", icon='TRIA_UP').direction = 'UP' row.operator("mv.rotate_product", text="Down", icon='TRIA_DOWN').direction = 'DOWN' + col = box.column(align=True) + col.prop(s, "orient_method", text="") + row = col.row(align=True) + row.operator("mv.auto_orient", text="Auto Orient", icon='ORIENTATION_GLOBAL').all_products = False + row.operator("mv.auto_orient", text="All").all_products = True + row = col.row(align=True) + row.operator("mv.reset_orientation", text="Reset", icon='LOOP_BACK').all_products = False + row.operator("mv.reset_orientation", text="All").all_products = True + row = box.row(align=True) row.operator("mv.product_step", text="Previous", icon='FRAME_PREV').delta = -1 row.operator("mv.product_step", text="Next", icon='FRAME_NEXT').delta = 1 box.operator("mv.show_all_products", icon='HIDE_OFF') - box.label(text="Rebuild cameras after rotating to reframe.", icon='INFO') + if s.reframe_on_render: + box.label(text="Render reframes the cameras itself.", icon='INFO') + else: + box.label(text="Rebuild cameras after rotating to reframe.", icon='INFO') class MV_PT_Render(Panel): @@ -2083,22 +5752,28 @@ def draw(self, context): s = context.scene.mv_settings layout.prop(s, "render_output_dir") layout.prop(s, "render_mode") - if s.render_mode in {'CLAY', 'BOTH'}: + if s.render_mode in {'CLAY', 'BOTH'} or (s.pass_lines and s.lines_fill == 'CLAY'): layout.prop(s, "clay_color") + layout.label(text="Extra passes:") + row = layout.row(align=True) + row.prop(s, "pass_lines", toggle=True) + row.prop(s, "pass_mask", toggle=True) + row.prop(s, "pass_ao", toggle=True) + col = layout.column(align=True) + col.prop(s, "skip_existing") + col.prop(s, "reframe_on_render") - box = layout.box() - box.label(text="Naming:") - row = box.row(align=True) - row.prop(s, "start_prefix") - row.prop(s, "start_number") - box.prop(s, "products_per_letter") - + # Read-only lookup: draw() must not create the root. + root = bpy.data.collections.get(PRODUCT_COLL) prods = get_products() if prods: - first = product_label(s.start_prefix, s.start_number, 0, s.products_per_letter) - last = product_label(s.start_prefix, s.start_number, len(prods) - 1, s.products_per_letter) - box.label(text=f"First: {first}_{prods[0].name}", icon='FILE') - box.label(text=f"Last: {last}_{prods[-1].name}", icon='FILE') + layout.label(text="Products to render:") + row = layout.row() + row.template_list("MV_UL_Products", "", root, "children", + s, "active_product_index", rows=4) + col = row.column(align=True) + col.operator("mv.set_products_render", text="", icon='CHECKBOX_HLT').state = True + col.operator("mv.set_products_render", text="", icon='CHECKBOX_DEHLT').state = False strays = stray_render_objects(context) if strays: @@ -2114,45 +5789,263 @@ def draw(self, context): box.label(text="product, or switch off their") box.label(text="render visibility (camera icon).") - layout.operator("mv.render_all", icon='RENDER_STILL') - - cam_coll = bpy.data.collections.get(CAMERA_COLL) - n_cams = len([o for o in cam_coll.objects if o.type == 'CAMERA']) if cam_coll else 0 - mult = 2 if s.render_mode == 'BOTH' else 1 - layout.label(text=f"{len(prods)} products x {n_cams} views x {mult} = {len(prods) * n_cams * mult} images") + ticked = sum(1 for pc in prods if pc.mv_render) + row = layout.row(align=True) + row.scale_y = 1.3 + row.operator("mv.render_all", icon='RENDER_STILL', + text="Render All" if ticked == len(prods) else f"Render {ticked} Ticked", + ).only_active = False + row.operator("mv.render_all", text="Render Active", + icon='RESTRICT_RENDER_OFF').only_active = True + + if s.reframe_on_render: + # What Render All is about to build, not what is in MV_Cameras now. + n_cams = sum(1 for v in planned_views(s) if Vector(v[2]).length >= 1e-9) + else: + cam_coll = bpy.data.collections.get(CAMERA_COLL) + n_cams = len([o for o in cam_coll.objects if o.type == 'CAMERA']) if cam_coll else 0 + mult = (2 if s.render_mode == 'BOTH' else 1) + s.pass_lines + s.pass_mask + s.pass_ao + layout.label(text=f"{ticked} products x {n_cams} views x {mult} = {ticked * n_cams * mult} images") + if s.turntable_enabled: + layout.label(text=f"+ {ticked} turntable(s) x {s.turntable_frames} frames") + layout.label(text="Esc stops after the image in progress.", icon='INFO') layout.label(text="Engine/samples/resolution/format: Blender Output props", icon='INFO') +class _RenderSubPanel: + bl_space_type = 'VIEW_3D' + bl_region_type = 'UI' + bl_category = "MultiView" + bl_parent_id = "MV_PT_Render" + bl_options = {'DEFAULT_CLOSED'} + + +class MV_PT_RenderPreflight(_RenderSubPanel, Panel): + bl_label = "Pre-flight" + bl_options = set() + + def draw(self, context): + layout = self.layout + s = context.scene.mv_settings + row = layout.row(align=True) + row.operator("mv.preflight", icon='CHECKMARK') + row.prop(s, "preflight_on_render", text="On Render", toggle=True) + if not s.preflight_time: + layout.label(text="Not checked yet.") + return + if not s.preflight_issues: + layout.label(text=f"All clear ({s.preflight_time})", icon='CHECKMARK') + return + layout.label(text=f"Checked {s.preflight_time}:") + col = layout.column(align=True) + for item in list(s.preflight_issues)[:12]: + row = col.row(align=True) + text = f"{item.product}: {item.message}" if item.product else item.message + row.label(text=text, icon='CANCEL' if item.severity == 'ERROR' else 'ERROR') + if item.fix: + op = row.operator("mv.preflight_fix", text="", + icon='FILE_TICK' if item.fix == 'SAVE_HINT' else 'TOOL_SETTINGS') + op.fix = item.fix + op.product = item.product + if len(s.preflight_issues) > 12: + col.label(text=f"+{len(s.preflight_issues) - 12} more") + + +class MV_PT_RenderNaming(_RenderSubPanel, Panel): + bl_label = "Naming" + bl_options = set() + + def draw(self, context): + layout = self.layout + s = context.scene.mv_settings + row = layout.row(align=True) + row.prop(s, "start_prefix") + row.prop(s, "start_number") + layout.prop(s, "products_per_letter") + layout.prop(s, "folder_template") + layout.prop(s, "file_template") + col = layout.column(align=True) + col.label(text="Tokens: {label} {prefix} {number}") + col.label(text="{product} {view} {mode}") + + # The first image of the batch, as it will actually be written. + prods = get_products() + views = planned_views(s) + mode = {'FULL': "", 'CLAY': "clay", 'BOTH': "render"}[s.render_mode] + values = name_values(s, 0, prods[0].name if prods else "Product", + views[0][0] if views else "Front", mode) + folder = output_folder(s, values).replace(os.sep, "/") + col = layout.column(align=True) + col.label(text=f"{folder}/" if folder else "(output directory)", icon='FILE_FOLDER') + col.label(text=output_basename(s, values) + context.scene.render.file_extension, + icon='FILE_IMAGE') + + +class MV_PT_RenderSheet(_RenderSubPanel, Panel): + bl_label = "Contact Sheet" + + def draw_header(self, context): + self.layout.prop(context.scene.mv_settings, "sheet_enabled", text="") + + def draw(self, context): + layout = self.layout + s = context.scene.mv_settings + layout.active = s.sheet_enabled + row = layout.row(align=True) + row.prop(s, "sheet_columns") + row.prop(s, "sheet_cell_width") + layout.prop(s, "sheet_bg") + layout.prop(s, "sheet_labels") + sub = layout.row() + sub.active = s.sheet_labels + sub.prop(s, "sheet_text_color") + if s.sheet_labels and not hasattr(blf, "bind_imbuf"): + layout.label(text="Labels need a newer Blender.", icon='ERROR') + if s.post_enabled: + layout.label(text="Built from the processed copies.", icon='INFO') + + +class MV_PT_RenderTurntable(_RenderSubPanel, Panel): + bl_label = "Turntable" + + def draw_header(self, context): + self.layout.prop(context.scene.mv_settings, "turntable_enabled", text="") + + def draw(self, context): + layout = self.layout + s = context.scene.mv_settings + layout.active = s.turntable_enabled + layout.prop(s, "turntable_style") + row = layout.row(align=True) + row.prop(s, "turntable_frames") + row.prop(s, "turntable_fps") + layout.prop(s, "turntable_elevation") + row = layout.row(align=True) + row.prop(s, "turntable_mp4", toggle=True) + row.prop(s, "turntable_gif", toggle=True) + sub = layout.row() + sub.active = s.turntable_gif + sub.prop(s, "turntable_gif_size") + layout.label(text="Frames are kept in a turntable folder.", icon='INFO') + + +class MV_PT_RenderPost(_RenderSubPanel, Panel): + bl_label = "Post-Processing" + + def draw_header(self, context): + self.layout.prop(context.scene.mv_settings, "post_enabled", text="") + + def draw(self, context): + layout = self.layout + s = context.scene.mv_settings + layout.active = s.post_enabled + layout.prop(s, "post_folder") + row = layout.row(align=True) + row.prop(s, "post_trim") + sub = row.row(align=True) + sub.active = s.post_trim + sub.prop(s, "post_margin") + layout.prop(s, "post_square") + row = layout.row(align=True) + row.prop(s, "post_flatten", text="Flatten") + sub = row.row(align=True) + sub.active = s.post_flatten or s.post_format == 'JPEG' + sub.prop(s, "post_bg_color", text="") + row = layout.row(align=True) + row.prop(s, "post_format", text="") + sub = row.row(align=True) + sub.active = s.post_format != 'PNG' + sub.prop(s, "post_quality") + layout.prop(s, "post_max_size") + layout.label(text="Extra sizes, e.g. 1024, 256:") + layout.prop(s, "post_extra_sizes", text="") + + +class MV_PT_RenderPasses(_RenderSubPanel, Panel): + bl_label = "Extra Passes" + + @classmethod + def poll(cls, context): + s = context.scene.mv_settings + return s.pass_lines or s.pass_ao + + def draw(self, context): + layout = self.layout + s = context.scene.mv_settings + if s.pass_lines: + layout.label(text="Line Art:") + layout.prop(s, "lines_angle") + row = layout.row(align=True) + row.prop(s, "lines_width") + row.prop(s, "lines_color", text="") + layout.prop(s, "lines_fill") + if s.pass_ao: + layout.label(text="Ambient Occlusion:") + row = layout.row(align=True) + row.prop(s, "ao_samples", text="Samples") + row.prop(s, "ao_distance", text="Distance") + if not hasattr(context.scene, "cycles"): + layout.label(text="Needs the Cycles add-on.", icon='ERROR') + + +class MV_PT_RenderManifest(_RenderSubPanel, Panel): + bl_label = "Manifest" + + def draw(self, context): + layout = self.layout + layout.prop(context.scene.mv_settings, "manifest_format", text="Write") + layout.label(text="Saved in the output directory.", icon='INFO') + + classes = ( MV_AddonPreferences, MV_StepFileItem, + MV_Issue, MV_CustomView, MV_Settings, MV_OT_AddStepFile, MV_OT_RemoveStepFile, MV_OT_ClearStepFiles, + MV_OT_AddFolder, MV_OT_ImportAllSteps, MV_OT_ApplySceneSetup, MV_OT_BuildCameras, MV_OT_BuildLighting, MV_OT_ApplyWorld, + MV_OT_SaveLightingPreset, + MV_OT_DeleteLightingPreset, MV_OT_AddCustomView, MV_OT_RemoveCustomView, MV_OT_AddProductCollection, MV_OT_Standardize, MV_OT_RotateProduct, + MV_OT_AutoOrient, + MV_OT_ResetOrientation, MV_OT_ProductStep, MV_OT_ShowAllProducts, + MV_OT_SetProductsRender, + MV_OT_Preflight, + MV_OT_PreflightFix, + MV_OT_RenderPreflightPrompt, MV_OT_RenderAll, MV_UL_StepFiles, MV_UL_CustomViews, + MV_UL_Products, # Panels draw in the sidebar in the order they are registered. MV_PT_Guide, MV_PT_SceneSetup, + MV_PT_SceneMaterials, MV_PT_Import, MV_PT_Standardize, MV_PT_Cameras, MV_PT_Render, + MV_PT_RenderPreflight, + MV_PT_RenderNaming, + MV_PT_RenderSheet, + MV_PT_RenderTurntable, + MV_PT_RenderPasses, + MV_PT_RenderPost, + MV_PT_RenderManifest, ) @@ -2160,9 +6053,50 @@ def register(): for c in classes: bpy.utils.register_class(c) bpy.types.Scene.mv_settings = PointerProperty(type=MV_Settings) + # Stored on the collection itself, so the tick travels with the product + # when it is renamed, reordered or appended into another file. + bpy.types.Collection.mv_render = BoolProperty( + name="Render", + description="Include this product when rendering. Unticked products keep " + "their label, so the rest of the batch is not renumbered", + default=True, + ) + bpy.types.Collection.mv_material = EnumProperty( + name="Material", + description="Material this product renders with in Full renders", + items=[('DEFAULT', "Batch Default", "Use the batch's material setting"), + ('KEEP', "Keep Own", "Use the materials this product already has")] + + MATERIAL_ITEMS, + default='DEFAULT', + ) + bpy.types.Collection.mv_material_color = FloatVectorProperty( + name="Color", subtype='COLOR', size=3, + min=0.0, max=1.0, default=(0.8, 0.8, 0.8), + ) + bpy.types.Collection.mv_source = StringProperty( + name="Source File", + description="The CAD file this product was imported from", + subtype='FILE_PATH', + ) + bpy.types.Collection.mv_source_size = FloatVectorProperty( + name="Source Size", + description="Bounding box of the product as imported, before Fit on Import " + "scaled it, in the source file's units", + size=3, + ) + bpy.types.Collection.mv_source_units = StringProperty( + name="Source Units", + description="Unit Source Size is in: mm, cm, m, in or ft. Empty when not known", + ) def unregister(): + del bpy.types.Collection.mv_source_units + del bpy.types.Collection.mv_source_size + del bpy.types.Collection.mv_source + del bpy.types.Collection.mv_material_color + del bpy.types.Collection.mv_material + del bpy.types.Collection.mv_render del bpy.types.Scene.mv_settings for c in reversed(classes): bpy.utils.unregister_class(c) From 820b60834889ecbfcf44febb32b226e736af0272 Mon Sep 17 00:00:00 2001 From: Danyal Sarfraz Date: Tue, 27 Oct 2026 00:34:18 +0900 Subject: [PATCH 2/3] Add camera export, data passes, COCO/BOP, headless CLI and tests (1.10.0) Annotations - cameras.json per product: per-view K (perspective) or pixel size (orthographic), c2w in OpenGL and w2c in OpenCV axes, image paths - Track each product's source-to-world transform through every move MultiView makes, so renders map back to CAD coordinates and units - nerfstudio-style transforms.json for perspective views and for turntables (equivalent orbiting cameras) - Depth and camera-space normal passes: single-sample Cycles, 32-bit EXR, no filter blending - Part-ID pass: 16-bit PNG label map numbering a product's parts - COCO export: per-part instance masks (RLE) with part names - BOP export from perspective views: rgb, depth (mm), masks, scene_camera/gt/gt_info, PLY models and models_info, with rigid poses in source-model millimetres Command line - cli.py runs a JSON/TOML job headless without installing the add-on: import, standardize, auto-orient, scene setup, render - --shard i/n with job-wide labels and per-shard lists, --merge to combine them, --check for pre-flight only, --list-settings - Exit codes 0/1/2 and a MULTIVIEW_SUMMARY line Tests - tests/ suite run inside background Blender, with a runner and STEP/IGES fixtures; FreeCAD checks skip when it is absent - GitHub Actions workflow running the suite on Blender 5.0.0 Fixes - JSON outputs are rewritten only when their content changes --- .github/workflows/tests.yml | 45 + README.md | 130 ++- multiview_product_renderer/__init__.py | 848 +++++++++++++++++- multiview_product_renderer/cli.py | 343 +++++++ reports/Multiview renderer literature gap.md | 89 ++ .../cad_datasets_and_ml.md | 166 ++++ .../line_drawings.md | 167 ++++ .../multiview_dataset_rendering.md | 166 ++++ .../product_rendering_tools.md | 153 ++++ .../synthetic_data_pipelines.md | 191 ++++ .../venues_and_impact_features.md | 239 +++++ tests/_harness.py | 73 ++ tests/fixtures/README.md | 9 + tests/fixtures/block.igs | 153 ++++ tests/fixtures/colored_parts.step | 512 +++++++++++ tests/fixtures/plain_parts.step | 479 ++++++++++ tests/run_tests.py | 77 ++ tests/test_annotations.py | 265 ++++++ tests/test_cli.py | 110 +++ tests/test_import.py | 264 ++++++ tests/test_look.py | 289 ++++++ tests/test_output.py | 297 ++++++ tests/test_preflight.py | 177 ++++ tests/test_render.py | 201 +++++ 24 files changed, 5412 insertions(+), 31 deletions(-) create mode 100644 .github/workflows/tests.yml create mode 100644 multiview_product_renderer/cli.py create mode 100644 reports/Multiview renderer literature gap.md create mode 100644 research_notes/Multiview renderer literature gap/cad_datasets_and_ml.md create mode 100644 research_notes/Multiview renderer literature gap/line_drawings.md create mode 100644 research_notes/Multiview renderer literature gap/multiview_dataset_rendering.md create mode 100644 research_notes/Multiview renderer literature gap/product_rendering_tools.md create mode 100644 research_notes/Multiview renderer literature gap/synthetic_data_pipelines.md create mode 100644 research_notes/Multiview renderer literature gap/venues_and_impact_features.md create mode 100644 tests/_harness.py create mode 100644 tests/fixtures/README.md create mode 100644 tests/fixtures/block.igs create mode 100644 tests/fixtures/colored_parts.step create mode 100644 tests/fixtures/plain_parts.step create mode 100644 tests/run_tests.py create mode 100644 tests/test_annotations.py create mode 100644 tests/test_cli.py create mode 100644 tests/test_import.py create mode 100644 tests/test_look.py create mode 100644 tests/test_output.py create mode 100644 tests/test_preflight.py create mode 100644 tests/test_render.py diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml new file mode 100644 index 0000000..1eb0c70 --- /dev/null +++ b/.github/workflows/tests.yml @@ -0,0 +1,45 @@ +name: Tests + +on: + push: + pull_request: + workflow_dispatch: + +jobs: + test: + runs-on: ubuntu-24.04 + timeout-minutes: 90 + env: + # The version the suite is written against. Bump deliberately: Blender + # changes its Python API between releases. + BLENDER_VERSION: "5.0.0" + steps: + - uses: actions/checkout@v4 + + - name: Cache Blender + id: cache-blender + uses: actions/cache@v4 + with: + path: blender + key: blender-${{ env.BLENDER_VERSION }}-linux-x64 + + - name: Download Blender + if: steps.cache-blender.outputs.cache-hit != 'true' + run: | + series="${BLENDER_VERSION%.*}" + curl -fsSL -o blender.tar.xz \ + "https://download.blender.org/release/Blender${series}/blender-${BLENDER_VERSION}-linux-x64.tar.xz" + mkdir blender + tar -xf blender.tar.xz -C blender --strip-components=1 + + - name: Install system libraries + # Blender's Workbench and Cycles renders run headless through EGL; + # Mesa supplies a software GL on a runner without a GPU. + run: | + sudo apt-get update + sudo apt-get install -y --no-install-recommends \ + libegl1 libgl1 libglx-mesa0 libgl1-mesa-dri libxi6 libxxf86vm1 \ + libxfixes3 libxrender1 libxkbcommon0 libsm6 libice6 + + - name: Run tests + run: python3 tests/run_tests.py --blender blender/blender diff --git a/README.md b/README.md index 2d3bb2f..3773bc0 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ every view in one click, with predictable file names like `AA_01_Widget_Front_cl - **Blender:** 4.0 and newer - **Panel location:** `3D Viewport → Sidebar (N key) → MultiView` -- **Version:** 1.9.0 +- **Version:** 1.10.0 --- @@ -605,6 +605,116 @@ since been deleted are dropped. --- +#### Annotations for research + +The **Annotations** section under Batch Render turns a render into a dataset. Everything +here is computed from the same fixed camera rig as the images, so it lines up with them to +the pixel. The test suite checks the camera numbers against Blender's own projection and the +depth against ray casts. + +| Setting | Writes | Notes | +| --- | --- | --- | +| **Cameras** (on by default) | `AA_01_Widget_cameras.json`, `AA_01_Widget_transforms.json`, and `..._Turntable_..._transforms.json` | See below | +| **Depth** | `..._Front_depth.exr` | 32-bit EXR, distance along the view axis in scene units, 0 and alpha 0 off the product | +| **Normals** | `..._Front_normal.exr` | 32-bit EXR, camera space, OpenGL axes (+X right, +Y up, +Z towards the camera), values -1..1 | +| **Part IDs** | `..._Front_parts.png` | 16-bit greyscale PNG: 0 is background, *n* is the *n*-th part in `cameras.json`'s `parts` list. Parts are the product's objects, so a STEP assembly's parts keep the names its CAD file gave them | +| **COCO** | `coco.json` | One instance mask (uncompressed RLE) per visible part per image, with `part_id` and `part_name`. Categories are products. Turns on Part IDs | +| **BOP** | `bop/` | A [BOP](https://github.com/thodan/bop_toolkit/blob/master/docs/bop_datasets_format.md) dataset from the perspective views: `train_pbr//rgb`, `depth` (uint16, mm), `mask`, `mask_visib`, `scene_camera.json`, `scene_gt.json`, `scene_gt_info.json`, plus `models/obj_*.ply` and `models_info.json`. Turns on Depth | + +Depth, Normals and Part IDs are each one extra Cycles render per view: one sample, no +antialiasing filter, every surface replaced by an emitter that outputs the number itself. +Each pixel therefore belongs to exactly one surface. There is no blending of a part's ID with +the background, or of a foreground depth with what is behind it. They need the Cycles add-on +(on by default) whatever engine the Full render uses. They are never post-processed or put +on a contact sheet. + +**`cameras.json`** describes every view of the product: + +- `c2w_opengl`: camera-to-world matrix, Blender/OpenGL axes (looking down -Z, +Y up). +- `w2c_opencv`: world-to-camera matrix, OpenCV axes (looking down +Z, +Y down), as COLMAP + and BOP use. +- `K` for perspective views. Orthographic views get `pixel_size` and `principal_point` + instead, since no K describes them. +- `width`, `height`, `clip`, and the relative paths of every image of that view. +- `model_to_world`: the 4x4 matrix from the product's **source file coordinates** (in + `source_units`) to the scene. MultiView records every move it makes to a product (fitting, + recentring, turning, Auto Orient), so a pixel in any render can be traced back to a point + in the original CAD file. Moves you make by hand in the viewport are not seen. + +**`transforms.json`** is in the nerfstudio/Instant-NGP format, for NeRF and Gaussian +splatting tools. It lists the perspective views only, with per-frame intrinsics and +`mask_path` when the Mask pass is on. A turntable gets its own `transforms.json`: the +product turns and the camera stays still, so each frame is given the camera that would +circle a still product. That is what those tools expect. + +BOP wants rigid poses in millimetres of the model, although the scene was scaled to fit. +The scale is taken back out of the camera translation, the model is exported in its own +source units converted to mm, and depth is written in the same mm, with a `depth_scale` per +image so large parts still fit in 16 bits. Only perspective views go into BOP, because the +format has no orthographic camera. + +## Command line + +Everything the panels do can run without Blender's window, from a job file: + +```bash +blender -b --factory-startup --python multiview_product_renderer/cli.py -- --config job.json +``` + +The add-on does not need to be installed for this: `cli.py` loads it from the folder it sits +in. A job is JSON or TOML, and relative paths in it are relative to the job file: + +```json +{ + "inputs": ["parts/", "extra/housing.step"], + "recursive": true, + "output": "renders/", + "freecad": "C:/Program Files/FreeCAD 1.1/bin/freecadcmd.exe", + "step_import_mode": "FREECAD", + "standardize": false, + "auto_orient": "FLAT", + "scene_setup": true, + "settings": { + "render_mode": "CLAY", + "view_set": "ALL", + "pass_depth": true, + "export_coco": true, + "turntable_elevation": 20 + }, + "render": {"engine": "CYCLES", "samples": 64, "device": "CPU", + "resolution": [1024, 1024], "file_format": "PNG"}, + "save_blend": "renders/job.blend" +} +``` + +| Key | Meaning | +| --- | --- | +| `blend` | Start from this `.blend` instead of an empty scene. With no `inputs`, its existing products are rendered | +| `inputs`, `recursive` | Files and folders to import (folders expanded, sorted) | +| `output` | Output folder; `--output` overrides it | +| `freecad`, `step_import_mode`, `deflection` | STEP/IGES conversion, as in the add-on preferences | +| `standardize`, `auto_orient` | Run Standardize Products / Auto Orient (true, `"FLAT"` or `"PCA"`) on everything after import | +| `scene_setup` | Apply Scene Setup (lighting, background, ground). Default true | +| `settings` | Any panel setting by name; `--list-settings` prints them all with their types. Angles are in degrees | +| `render` | `engine`, `samples`, `device`, `resolution`, `percentage`, `file_format` | +| `save_blend` | Save the set-up scene afterwards, to open and inspect in Blender | + +Other modes: + +- `--check` imports and sets up, runs the pre-flight check, prints its findings and renders + nothing. +- `--shard 2/8` imports and renders only every eighth file, starting with the third. Labels + still count through the whole job, so eight shards started at once, on one machine or + eight, write exactly the files one run would. Each writes its own + `manifest_shard02.json`, `coco_shard02.json` and BOP `models_info_shard02.json`. +- `--merge renders/` then combines those into `manifest.json`/`.csv`, `coco.json` and + `models_info.json`. +- `--list-settings` prints every setting `settings` accepts. + +It prints a `MULTIVIEW_SUMMARY {...}` JSON line at the end. The exit code is 0 when +everything rendered, 1 when some images failed, and 2 when the job could not run: a bad job +file, nothing imported, or pre-flight errors. + ## Collections the add-on creates | Collection | Contents | @@ -670,6 +780,24 @@ again. --- +## Tests + +The `tests/` folder holds the test suite: each `test_*.py` runs inside a background Blender +and checks the add-on end to end, by rendering real images and reading them back. + +```bash +python tests/run_tests.py --blender "C:/Program Files/Blender Foundation/Blender 5.0/blender.exe" +``` + +Any Python 3 runs the runner, including Blender's own (`/5.0/python/bin/python`). +FreeCAD is optional: with `freecadcmd` on `PATH`, or `FREECAD_CMD` set, the STEP and IGES +import checks run too; without it they are skipped (`FREECAD_CMD=none` forces that). GitHub +Actions runs the suite on every push against Blender 5.0.0 (`.github/workflows/tests.yml`), +without FreeCAD. + +The suite is written against **Blender 5.0**. The add-on aims to support 4.0 and newer, but +only 5.0 is tested. + ## License MIT. See [LICENSE](LICENSE). diff --git a/multiview_product_renderer/__init__.py b/multiview_product_renderer/__init__.py index ac16f67..ba9f487 100644 --- a/multiview_product_renderer/__init__.py +++ b/multiview_product_renderer/__init__.py @@ -1,7 +1,7 @@ bl_info = { "name": "MultiView Product Renderer", "author": "Danyal Sarfraz", - "version": (1, 9, 0), + "version": (1, 10, 0), "blender": (4, 0, 0), "location": "View3D > Sidebar > MultiView", "description": "Import CAD and mesh files, set up cameras and studio lighting, batch render products across views.", @@ -154,6 +154,11 @@ " or however the Naming templates say.", " Optional extras under Batch Render: contact sheets, turntables", " (MP4/GIF), post-processed copies, and a CSV/JSON manifest.", + " Annotations adds cameras.json/transforms.json, depth, normals,", + " part IDs, COCO and BOP for building datasets.", + "", + "Headless: blender -b --python multiview_product_renderer/cli.py", + " -- --config job.json (see README, Command line)", " Only products are hidden between shots, so anything outside", " MV_Products lands in every image. The panel warns you first.", "", @@ -196,6 +201,26 @@ def draw(self, context): layout.prop(self, "tessellation_deflection") +# Set by the command line, which can run the add-on straight from its folder +# without it being installed, and so without add-on preferences to read. +PREFS_OVERRIDE = None + + +def addon_prefs(context): + if PREFS_OVERRIDE is not None: + return PREFS_OVERRIDE + addon = context.preferences.addons.get(__name__) + if addon is not None: + return addon.preferences + from types import SimpleNamespace + return SimpleNamespace(step_import_mode='AUTO', freecad_path="", + tessellation_deflection=0.1) + + +# What the last Render All did, for the command line's summary and exit code. +LAST_RENDER = {} + + class MV_StepFileItem(PropertyGroup): path: StringProperty(name="Path", subtype='FILE_PATH') @@ -474,6 +499,50 @@ class MV_Settings(PropertyGroup): ('NONE', "None", "Lines only, on transparency")], default='WHITE', ) + pass_depth: BoolProperty( + name="Depth", + description="Write a depth map per view: 32-bit EXR, distance along the view " + "axis in scene units. Needs Cycles. Mode 'depth'", + default=False, + ) + pass_normal: BoolProperty( + name="Normals", + description="Write a camera-space normal map per view: 32-bit EXR, OpenGL axes. " + "Needs Cycles. Mode 'normal'", + default=False, + ) + pass_parts: BoolProperty( + name="Part IDs", + description="Write a 16-bit PNG per view saying which part each pixel shows: 0 is " + "background, the rest number the parts as listed in cameras.json. " + "Needs Cycles. Mode 'parts'", + default=False, + ) + export_cameras: BoolProperty( + name="Cameras", + description="Write each product's camera intrinsics and poses (cameras.json) and " + "a nerfstudio-style transforms.json for its perspective views and " + "turntable", + default=True, + ) + export_coco: BoolProperty( + name="COCO", + description="Write coco.json with one instance mask per visible part per image. " + "Turns on Part IDs", + default=False, + ) + export_bop: BoolProperty( + name="BOP", + description="Write a BOP-format dataset (bop/) from the perspective views: RGB, " + "depth in mm, masks, object poses and models. Turns on Depth", + default=False, + ) + shard_suffix: StringProperty( + description="Added to manifest, COCO and BOP model-info file names by the " + "command line, so shards rendered in parallel do not overwrite " + "each other's lists", + default="", options={'HIDDEN'}, + ) ao_samples: IntProperty(name="AO Samples", default=32, min=1, max=4096) ao_distance: FloatProperty( name="AO Distance", @@ -918,6 +987,32 @@ def collection_roots(coll): return [o for o in members if o.parent is None or o.parent not in members] +IDENTITY_16 = tuple(float(i % 5 == 0) for i in range(16)) + + +def product_to_world(coll): + """Matrix taking the product's source coordinates (as imported, in its + source units) to where it sits in the scene now.""" + m = coll.mv_to_world + return Matrix([m[i * 4:i * 4 + 4] for i in range(4)]) + + +def move_product(coll, xform): + """Apply a world-space transform to a whole product, and fold it into the + product's source-to-world record. + + Every move MultiView makes goes through here -- fitting, recentring, + turning, auto-orienting -- so the record stays exact, and a pixel in any + render can be traced back to a point in the original CAD file. Baking does + not move geometry, so it needs no entry. Moves made by hand in the + viewport are not seen. + """ + for obj in collection_roots(coll): + obj.matrix_world = xform @ obj.matrix_world + total = xform @ product_to_world(coll) + coll.mv_to_world = [v for row in total for v in row] + + def fit_collection(coll, target_size=None, recenter=True, floor=False): """Uniformly scale a product so its bounding box fits inside a cube of target_size, and optionally move that box onto the world origin. @@ -951,8 +1046,7 @@ def fit_collection(coll, target_size=None, recenter=True, floor=False): xform = (Matrix.Translation(dest) @ Matrix.Scale(factor, 4) @ Matrix.Translation(-center)) - for obj in collection_roots(coll): - obj.matrix_world = xform @ obj.matrix_world + move_product(coll, xform) return True @@ -1046,8 +1140,7 @@ def turn_product(coll, rotation): return False center, _ = bbox_of(objs) xform = (Matrix.Translation(center) @ rotation.to_4x4() @ Matrix.Translation(-center)) - for obj in collection_roots(coll): - obj.matrix_world = xform @ obj.matrix_world + move_product(coll, xform) return True @@ -1426,8 +1519,7 @@ def rotate_product(context, coll, direction, step): xform = (Matrix.Translation(center) @ Matrix.Rotation(angle, 4, axis) @ Matrix.Translation(-center)) - for obj in collection_roots(coll): - obj.matrix_world = xform @ obj.matrix_world + move_product(coll, xform) return True @@ -1451,6 +1543,13 @@ def alpha_pair_to_idx(prefix): return max(0, ord(prefix[0]) - ord('A')) +def label_index(position, coll): + """Where a product's label is counted from: its own recorded index when + the command line has set one, otherwise its place in MV_Products.""" + fixed = getattr(coll, "mv_label_index", -1) + return fixed if fixed >= 0 else position + + def product_label(start_prefix, start_number, offset, per_letter): base = alpha_pair_to_idx(start_prefix) linear = (max(1, start_number) - 1) + offset @@ -2826,7 +2925,7 @@ def _setup(self, context): if not s.step_files: self.report({'ERROR'}, "No files in the list.") return False - self._prefs = context.preferences.addons[__name__].preferences + self._prefs = addon_prefs(context) self._queue = [bpy.path.abspath(item.path) for item in s.step_files] self._total = len(self._queue) self._done = 0 @@ -3713,6 +3812,259 @@ def _do_ao_render(scene, filepath, options): bpy.ops.render.render(write_still=True) +# -- data passes ------------------------------------------------------------ +# +# Depth, normals and part IDs are measurements, not pictures. Each is a Cycles +# render of one sample with every surface swapped for an emitter that outputs +# the number wanted, written as 32-bit EXR so nothing is colour managed, +# clamped or quantized. One sample with a near-zero pixel filter puts every +# pixel on exactly one surface: no antialiasing to average a foreground depth +# with the background, or blend two part IDs into a third that does not exist. + +def write_png16(path, values): + """Write a 2D array of 0-65535 as a 16-bit greyscale PNG, top row first. + + Blender's image API saves 8 bits from Python, which caps a label map at + 255 parts and a depth map at 255 steps, so this writes the file directly. + """ + values = np.ascontiguousarray(np.clip(values, 0, 65535).astype(">u2")) + h, w = values.shape + rows = b"".join(b"\x00" + values[y].tobytes() for y in range(h)) + + def chunk(kind, data): + body = kind + data + return struct.pack(">I", len(data)) + body + struct.pack(">I", zlib.crc32(body)) + + with open(path, "wb") as f: + f.write(b"\x89PNG\r\n\x1a\n") + f.write(chunk(b"IHDR", struct.pack(">IIBBBBB", w, h, 16, 0, 0, 0, 0))) + f.write(chunk(b"IDAT", zlib.compress(rows, 6))) + f.write(chunk(b"IEND", b"")) + + +def read_png16(path): + """Read back a 16-bit greyscale PNG written by write_png16.""" + with open(path, "rb") as f: + data = f.read() + if data[:8] != b"\x89PNG\r\n\x1a\n": + raise ValueError("not a PNG") + pos, idat, w, h = 8, b"", 0, 0 + while pos < len(data): + length, kind = struct.unpack(">I4s", data[pos:pos + 8]) + body = data[pos + 8:pos + 8 + length] + if kind == b"IHDR": + w, h, depth, colour = struct.unpack(">IIBB", body[:10]) + if depth != 16 or colour != 0: + raise ValueError("not a 16-bit greyscale PNG") + elif kind == b"IDAT": + idat += body + pos += 12 + length + raw = np.frombuffer(zlib.decompress(idat), dtype=np.uint8).reshape(h, 1 + w * 2) + if raw[:, 0].any(): + raise ValueError("row filters other than None are not supported") + return raw[:, 1:].copy().view(">u2").astype(np.uint16) + + +def _black_world(): + world = bpy.data.worlds.get("MV_Data_World") or bpy.data.worlds.new("MV_Data_World") + tree = node_tree_of(world) + tree.nodes.clear() + bg = tree.nodes.new("ShaderNodeBackground") + bg.inputs["Color"].default_value = (0.0, 0.0, 0.0, 1.0) + bg.inputs["Strength"].default_value = 0.0 + out = tree.nodes.new("ShaderNodeOutputWorld") + tree.links.new(bg.outputs["Background"], out.inputs["Surface"]) + return world + + +def _data_material(kind): + """An emitter whose colour is the measurement itself.""" + name = f"MV_Data_{kind}" + mat = bpy.data.materials.get(name) or bpy.data.materials.new(name) + tree = node_tree_of(mat) + tree.nodes.clear() + emit = tree.nodes.new("ShaderNodeEmission") + emit.inputs["Strength"].default_value = 1.0 + out = tree.nodes.new("ShaderNodeOutputMaterial") + out.location = (400, 0) + tree.links.new(emit.outputs["Emission"], out.inputs["Surface"]) + if kind == "depth": + cam = tree.nodes.new("ShaderNodeCameraData") + cam.location = (-300, 0) + tree.links.new(cam.outputs["View Z Depth"], emit.inputs["Color"]) + elif kind == "normal": + # World space here; _do_normal_render turns it into camera space with + # the camera's own matrix. Cycles' "camera space" points Z the other + # way from OpenGL, so relying on it would flip every normal. + geo = tree.nodes.new("ShaderNodeNewGeometry") + geo.location = (-300, 0) + tree.links.new(geo.outputs["Normal"], emit.inputs["Color"]) + else: + info = tree.nodes.new("ShaderNodeObjectInfo") + info.location = (-300, 0) + tree.links.new(info.outputs["Object Index"], emit.inputs["Color"]) + return mat + + +def _render_data(scene, filepath, kind, extra=()): + if not hasattr(scene, "cycles"): + raise RuntimeError(f"the {kind} pass needs the Cycles add-on enabled") + image = scene.render.image_settings + with temporarily( + (scene.render, "engine", 'CYCLES'), + (scene.cycles, "samples", 1), + (scene.cycles, "use_denoising", False), + (scene.cycles, "use_adaptive_sampling", False), + (scene.cycles, "filter_width", 0.01), + (scene.render, "use_motion_blur", False), + (scene.render, "film_transparent", True), + (scene.render, "use_file_extension", True), + (bpy.context.view_layer, "material_override", _data_material(kind)), + (scene, "world", _black_world()), + (image, "file_format", 'OPEN_EXR'), + (image, "color_mode", 'RGBA'), + (image, "color_depth", '32'), + *_neutral_view(scene), + *extra): + scene.render.filepath = filepath + bpy.ops.render.render(write_still=True) + return filepath + ".exr" + + +def _do_depth_render(scene, filepath): + """Distance along the camera's view axis, in scene units, in R, G and B; + 0 and alpha 0 where nothing was hit.""" + _render_data(scene, filepath, "depth") + + +def save_exr(px, path): + """Write float RGBA pixels (rows bottom to top) to a 32-bit EXR as is.""" + h, w = px.shape[:2] + img = bpy.data.images.new("MV_Data", w, h, alpha=True, float_buffer=True) + try: + img.pixels.foreach_set(np.ascontiguousarray(px, dtype=np.float32).ravel()) + img.filepath_raw = path + img.file_format = 'OPEN_EXR' + img.save() + finally: + bpy.data.images.remove(img) + + +def _do_normal_render(scene, filepath): + """Shading normals in camera space, OpenGL convention: +X right, +Y up, + +Z towards the camera. Values in -1..1, not remapped.""" + work = tempfile.mkdtemp(prefix="mv_normal_") + try: + px = load_pixels(_render_data(scene, os.path.join(work, "world"), "normal"), raw=True) + finally: + shutil.rmtree(work, ignore_errors=True) + to_cam = np.array(scene.camera.matrix_world.to_3x3().normalized().transposed()) + hit = px[..., 3] > 0.5 + px[..., :3] = np.where(hit[..., None], px[..., :3] @ to_cam.T, 0.0) + save_exr(px, filepath + ".exr") + scene.render.filepath = filepath + + +def _do_parts_render(scene, filepath, part_map): + """Which part every pixel shows, as a 16-bit PNG: 0 is background, n is + the n-th part in the product's part list. Returns the label array. + + Part numbers go in through each object's pass index, set only for the + render and put back afterwards. + """ + extra = [] + for name, part_id in part_map.items(): + obj = bpy.data.objects.get(name) + if obj is not None: + extra.append((obj, "pass_index", part_id)) + work = tempfile.mkdtemp(prefix="mv_parts_") + try: + exr = _render_data(scene, os.path.join(work, "ids"), "ids", extra) + px = load_pixels(exr, raw=True) + finally: + shutil.rmtree(work, ignore_errors=True) + labels = np.where(px[..., 3] > 0.5, np.rint(px[..., 0]), 0).astype(np.int64) + labels = np.clip(labels, 0, 65535)[::-1] + write_png16(filepath + ".png", labels) + scene.render.filepath = filepath + return labels + + +# (name, render function, file extension). Unlike the image modes these are +# never post-processed or put on a contact sheet: they are data. +def data_modes(settings): + s = settings + modes = [] + if s.pass_depth or s.export_bop: + modes.append(("depth", _do_depth_render, ".exr")) + if s.pass_normal: + modes.append(("normal", _do_normal_render, ".exr")) + if s.pass_parts or s.export_coco: + modes.append(("parts", _do_parts_render, ".png")) + return modes + + +def part_list(coll): + """The product's parts, numbered from 1 in name order, as written into + the part-ID images and cameras.json.""" + meshes = sorted((o for o in collection_meshes(coll) if o.type == 'MESH'), + key=lambda o: o.name) + return {o.name: i + 1 for i, o in enumerate(meshes)} + + +# -- cameras -------------------------------------------------------------------- + +def render_resolution(scene): + r = scene.render + return (int(r.resolution_x * r.resolution_percentage / 100), + int(r.resolution_y * r.resolution_percentage / 100)) + + +def camera_record(scene, cam): + """Everything needed to project the scene into one of this camera's images. + + Extrinsics come in both conventions in use: camera-to-world in OpenGL axes + (Blender's own, and nerfstudio's: +Y up, looking down -Z) and + world-to-camera in OpenCV axes (+Y down, looking down +Z; COLMAP and BOP). + Perspective cameras get a pinhole K. Orthographic ones get the size of a + pixel in scene units instead, since no K describes them. + """ + r = scene.render + w, h = render_resolution(scene) + data = cam.data + c2w = cam.matrix_world.normalized() + flip = Matrix(((1, 0, 0, 0), (0, -1, 0, 0), (0, 0, -1, 0), (0, 0, 0, 1))) + w2c_cv = flip @ c2w.inverted() + pax, pay = r.pixel_aspect_x, r.pixel_aspect_y + fit = data.sensor_fit + if fit == 'AUTO': + fit = 'HORIZONTAL' if w * pax >= h * pay else 'VERTICAL' + aspect = pay / pax + across = w if fit == 'HORIZONTAL' else h * aspect + record = { + "camera": cam.name, + "width": w, "height": h, + "c2w_opengl": [list(row) for row in c2w], + "w2c_opencv": [list(row) for row in w2c_cv], + } + if data.type == 'ORTHO': + size = data.ortho_scale / across + record.update(projection="orthographic", ortho_scale=data.ortho_scale, + pixel_size=[size, size * aspect], + principal_point=[w / 2 - data.shift_x * across, + h / 2 + data.shift_y * across / aspect]) + else: + sensor = data.sensor_height if fit == 'VERTICAL' else data.sensor_width + fx = data.lens / sensor * across + fy = fx / aspect + cx = w / 2 - data.shift_x * across + cy = h / 2 + data.shift_y * across / aspect + record.update(projection="perspective", lens_mm=data.lens, + K=[[fx, 0.0, cx], [0.0, fy, cy], [0.0, 0.0, 1.0]]) + record["clip"] = [data.clip_start, data.clip_end] + return record + + def written_path(scene, path): """Where render.render(write_still=True) puts an image asked for at path. @@ -3829,7 +4181,9 @@ def srgb_encode(rgb): return np.where(rgb <= 0.0031308, rgb * 12.92, 1.055 * np.power(rgb, 1 / 2.4) - 0.055) -def load_pixels(path): +def load_pixels(path, raw=False): + """RGBA pixels of an image file. raw keeps float data exactly as stored, + for depth, normal and ID passes where the numbers are measurements.""" img = bpy.data.images.load(path, check_existing=False) try: w, h = img.size @@ -3838,6 +4192,8 @@ def load_pixels(path): px = np.empty(w * h * 4, dtype=np.float32) img.pixels.foreach_get(px) px = px.reshape(h, w, 4) + if raw: + return px if img.is_float: # 16-bit PNG and EXR come in as scene-linear floats. Encode them # back to display values so they sit with everything else; EXR @@ -4322,13 +4678,14 @@ def expand(k): # -- manifest ----------------------------------------------------------------- +MEASURABLE_IMAGES = {".png", ".jpg", ".jpeg", ".webp", ".exr", ".tif", ".tiff", ".bmp"} MANIFEST_FIELDS = ("label", "product", "kind", "view", "mode", "path", "width", "height", "source_file", "source_size", "source_units") -def read_manifest(out_root): - json_path = os.path.join(out_root, "manifest.json") - csv_path = os.path.join(out_root, "manifest.csv") +def read_manifest(out_root, name="manifest"): + json_path = os.path.join(out_root, name + ".json") + csv_path = os.path.join(out_root, name + ".csv") try: if os.path.isfile(json_path): with open(json_path, encoding="utf-8") as f: @@ -4341,19 +4698,19 @@ def read_manifest(out_root): return [] -def write_manifest(out_root, records, labels, fmt): +def write_manifest(out_root, records, labels, fmt, name="manifest"): """Merge this run's outputs into manifest.json / manifest.csv. Rows for products this run did not touch are carried over, so rendering one product does not wipe the rest of the batch from the list. Rows for files that are no longer on disk are dropped. """ - rows = [r for r in read_manifest(out_root) if r.get("label") not in labels] + rows = [r for r in read_manifest(out_root, name) if r.get("label") not in labels] for rec in records: if not file_ok(rec["path"]): continue width, height = rec.get("width"), rec.get("height") - if width is None and not rec["path"].lower().endswith((".mp4", ".gif")): + if width is None and file_ext(rec["path"]) in MEASURABLE_IMAGES: width, height = image_size(rec["path"]) row = {k: rec.get(k) for k in MANIFEST_FIELDS} row.update(path=os.path.relpath(rec["path"], out_root).replace("\\", "/"), @@ -4363,11 +4720,11 @@ def write_manifest(out_root, records, labels, fmt): rows.sort(key=lambda r: (str(r.get("label")), str(r.get("kind")), str(r.get("path")))) if fmt in {'JSON', 'BOTH'}: - with open(os.path.join(out_root, "manifest.json"), "w", encoding="utf-8") as f: + with open(os.path.join(out_root, name + ".json"), "w", encoding="utf-8") as f: json.dump({"generated": datetime.now().isoformat(timespec="seconds"), "images": rows}, f, indent=2) if fmt in {'CSV', 'BOTH'}: - with open(os.path.join(out_root, "manifest.csv"), "w", encoding="utf-8", newline="") as f: + with open(os.path.join(out_root, name + ".csv"), "w", encoding="utf-8", newline="") as f: writer = csv.DictWriter(f, fieldnames=MANIFEST_FIELDS, extrasaction='ignore') writer.writeheader() for row in rows: @@ -4520,9 +4877,9 @@ def preflight(context, only_active=False): return issues if only_active: coll, idx = active_product(context) - targets = [(idx, coll)] + targets = [(label_index(idx, coll), coll)] else: - targets = [(i, pc) for i, pc in enumerate(products) if pc.mv_render] + targets = [(label_index(i, pc), pc) for i, pc in enumerate(products) if pc.mv_render] if not targets: _issue(issues, 'ERROR', "No products are ticked for rendering") return issues @@ -4569,12 +4926,22 @@ def preflight(context, only_active=False): getattr(bpy.data.collections.get(CAMERA_COLL), "objects", []) if o.type == 'CAMERA']) if _names_clash(s, targets, views + (["Sheet"] if s.sheet_enabled else []), - [m for m, _, _ in render_modes(s)]): + [m for m, _, _ in render_modes(s)] + [m for m, _, _ in data_modes(s)]): _issue(issues, 'ERROR', "Template gives two images one name") # Engine-specific passes and staging. if s.pass_ao and not hasattr(scene, "cycles"): _issue(issues, 'ERROR', "AO pass needs the Cycles add-on") + if data_modes(s) and not hasattr(scene, "cycles"): + _issue(issues, 'ERROR', "Depth/normal/part passes need Cycles") + if s.export_bop: + if s.reframe_on_render: + persp = [v for v in planned_views(s) if v[1] == "persp"] + else: + persp = [o for o in getattr(bpy.data.collections.get(CAMERA_COLL), "objects", []) + if o.type == 'CAMERA' and o.data.type == 'PERSP'] + if not persp: + _issue(issues, 'WARNING', "BOP uses perspective views; none selected") full = any(fn is _do_full_render for _, fn, _ in render_modes(s)) or ( s.turntable_enabled and s.turntable_style == 'FULL') if full and s.ground_mode == 'SHADOW' and scene.render.engine != 'CYCLES': @@ -4875,6 +5242,8 @@ def _plan(self, context, targets, cameras): s = scene.mv_settings root = self._out_root modes = render_modes(s) + data = data_modes(s) + coco_sources = [] for idx, pc in targets: meta = self._meta(idx, pc) @@ -4882,9 +5251,30 @@ def _plan(self, context, targets, cameras): name = f"{meta['label']}_{pc.name}" first_job = len(self._queue) sheet_cells = {mode: [] for mode, _, _ in modes} + parts = part_list(pc) + views = [] for cam in cameras: view = cam.name.replace("MV_Cam_", "") + files = {} + views.append({"view": view, "camera": cam.name, "files": files}) + for mode, render, ext in data: + values = name_values(s, idx, pc.name, view, mode) + path = os.path.join(root, output_folder(s, values), output_basename(s, values)) + self._claim(path) + master = path + ext + files[mode] = master + rec = dict(meta, view=view, mode=mode) + if s.skip_existing and file_ok(master): + self._skipped += 1 + self._record(rec, "data", master, *self._render_size(scene)) + continue + self._queue.append({ + "kind": "render", "product": pc.name, "status": f"{name} {view} {mode}", + "camera": cam.name, "render": render, "path": path, "master": master, + "post": [], "rec": rec, "record_as": "data", + "extra": (parts,) if mode == "parts" else (), + }) for mode, render, extra in modes: values = name_values(s, idx, pc.name, view, mode) rel_dir = output_folder(s, values) @@ -4896,6 +5286,7 @@ def _plan(self, context, targets, cameras): for target, _ in post: self._claim(target) rec = dict(meta, view=view, mode=mode) + files[mode] = master sheet_cells[mode].append((view, post[0][0] if post else master)) have_master = s.skip_existing and file_ok(master) @@ -4930,9 +5321,33 @@ def _plan(self, context, targets, cameras): "path": path, "cells": cells, "title": title, "rec": rec}) + if s.export_cameras: + paths = {} + for kind in ("cameras", "transforms"): + values = name_values(s, idx, pc.name, kind, "") + paths[kind] = os.path.join(root, output_folder(s, values), + output_basename(s, values) + ".json") + self._claim(paths[kind]) + self._queue.append({"kind": "cameras", "product": pc.name, + "status": f"{name} cameras", "path": paths["cameras"], + "transforms": paths["transforms"], "views": views, + "parts": parts, "rec": dict(meta, view="Cameras", mode="")}) + if s.export_bop: + self._queue.append({"kind": "bop", "product": pc.name, "index": idx, + "status": f"{name} BOP", "path": os.path.join(root, "bop"), + "views": views, "rec": meta}) + if s.export_coco: + coco_sources.append((idx, pc.name, meta, views, parts)) + if s.turntable_enabled: self._plan_turntable(context, idx, pc, meta) + if coco_sources: + path = os.path.join(root, f"coco{s.shard_suffix}.json") + self._claim(path) + self._queue.append({"kind": "coco", "status": "COCO annotations", "path": path, + "sources": coco_sources}) + def _plan_turntable(self, context, idx, pc, meta): scene = context.scene s = scene.mv_settings @@ -4967,6 +5382,12 @@ def _plan_turntable(self, context, idx, pc, meta): missing.append((k, path, master)) self._skipped += n - len(missing) + if s.export_cameras: + path = os.path.join(root, rel_dir, "turntable", f"{base}_transforms.json") + self._claim(path) + self._queue.append({"kind": "turn_cameras", "product": pc.name, + "status": f"{name} turntable cameras", "path": path, + "frames": frames, "rec": dict(rec, view="Turntable cameras")}) if s.skip_existing and not missing and all(file_ok(p) for _, p in outputs): for _, path in outputs: self._record(rec, "turntable", path) @@ -5022,12 +5443,12 @@ def _setup(self, context): if self.only_active: coll, idx = active_product(context) - targets = [(idx, coll)] + targets = [(label_index(idx, coll), coll)] else: # Index into the whole batch, not the ticked subset: the label is # the product's identity on disk, and unticking AA_02 must not turn # AA_03 into a second AA_02. - targets = [(i, pc) for i, pc in enumerate(products) if pc.mv_render] + targets = [(label_index(i, pc), pc) for i, pc in enumerate(products) if pc.mv_render] if not targets: self.report({'ERROR'}, "No products are ticked for rendering.") return {'CANCELLED'} @@ -5101,9 +5522,10 @@ def _setup(self, context): "and {mode} to the file name template.") return {'CANCELLED'} - if any(job["kind"] == "turn_frame" for job in self._queue): + if any(job["kind"] in {"turn_frame", "turn_cameras"} for job in self._queue): if build_turntable_camera(context) is None: - self._queue = [j for j in self._queue if j["kind"] not in {"turn_frame", "turn_encode"}] + self._queue = [j for j in self._queue + if j["kind"] not in {"turn_frame", "turn_encode", "turn_cameras"}] self._notes.append("No geometry to frame a turntable around.") self._total = len(self._queue) @@ -5154,10 +5576,321 @@ def _render_image(self, context, job, camera): def _job_render(self, context, job): self._render_image(context, job, job["camera"]) - self._record(job["rec"], "render", job["master"], *self._render_size(context.scene)) + self._record(job["rec"], job.get("record_as", "render"), job["master"], + *self._render_size(context.scene)) if job["post"]: self._job_post(context, job) + # -- annotations -------------------------------------------------------- + def _write_json(self, path, payload): + """Write a JSON file, unless it already says exactly this: a resumed + batch then leaves its camera files untouched, as it does its images.""" + text = json.dumps(payload, indent=2) + try: + with open(path, encoding="utf-8") as f: + if f.read() == text: + return + except OSError: + pass + os.makedirs(os.path.dirname(path), exist_ok=True) + with open(path, "w", encoding="utf-8") as f: + f.write(text) + + def _job_cameras(self, context, job): + """cameras.json for every view, and transforms.json for the + perspective ones, which are the only ones a pinhole model describes.""" + scene = context.scene + coll = bpy.data.collections.get(job["product"]) + if coll is None: + raise RuntimeError(f"product '{job['product']}' no longer exists") + here = os.path.dirname(job["path"]) + + def rel(path): + return os.path.relpath(path, here).replace("\\", "/") + + to_world = product_to_world(coll) + meta = job["rec"] + views, frames = [], [] + for entry in job["views"]: + cam = bpy.data.objects.get(entry["camera"]) + if cam is None or cam.type != 'CAMERA': + continue + record = camera_record(scene, cam) + record["view"] = entry["view"] + record["images"] = {mode or "render": rel(p) for mode, p in entry["files"].items()} + views.append(record) + if record["projection"] != "perspective": + continue + image = next((p for m, p in entry["files"].items() + if m in {"", "render", "clay"}), None) + if image is None: + continue + (fx, _, cx), (_, fy, cy), _ = record["K"] + frame = {"file_path": rel(image), "transform_matrix": record["c2w_opengl"], + "fl_x": fx, "fl_y": fy, "cx": cx, "cy": cy, + "w": record["width"], "h": record["height"]} + if "mask" in entry["files"]: + frame["mask_path"] = rel(entry["files"]["mask"]) + frames.append(frame) + + payload = { + "generator": f"MultiView Product Renderer {'.'.join(map(str, bl_info['version']))}", + "blender": bpy.app.version_string, + "label": meta["label"], "product": meta["product"], + "source_file": meta.get("source_file", ""), + "source_units": meta.get("source_units", ""), + "source_size": meta.get("source_size"), + "model_to_world": [list(row) for row in to_world], + "conventions": { + "world": "Blender: +Z up, scene units", + "c2w_opengl": "camera to world, camera looks down -Z with +Y up", + "w2c_opencv": "world to camera, camera looks down +Z with +Y down", + "model_to_world": "source file coordinates (source_units) to world", + "depth": "EXR, distance along the view axis in scene units", + "normal": "EXR, camera space, OpenGL axes", + "parts": "16-bit PNG, 0 = background, n = parts[n-1]", + }, + "parts": [{"id": i, "name": n} for n, i in sorted(job["parts"].items(), + key=lambda p: p[1])], + "views": views, + } + self._write_json(job["path"], payload) + self._record(meta, "cameras", job["path"]) + if frames: + self._write_json(job["transforms"], {"camera_model": "OPENCV", "frames": frames}) + self._record(meta, "cameras", job["transforms"]) + + def _job_turn_cameras(self, context, job): + """transforms.json for a turntable. The product turns and the camera + stays put; the same pictures come from a camera circling a still + product, which is what NeRF-style tools expect, so each frame gets + that equivalent camera.""" + self._end_turntable_pose() + scene = context.scene + cam = bpy.data.objects.get(TURNTABLE_CAM) + coll = bpy.data.collections.get(job["product"]) + if cam is None or coll is None: + raise RuntimeError("turntable camera or product missing") + center, _ = bbox_of(collection_meshes(coll)) + base = camera_record(scene, cam) + (fx, _, cx), (_, fy, cy), _ = base["K"] + here = os.path.dirname(job["path"]) + n = len(job["frames"]) + frames = [] + for k, image in enumerate(job["frames"]): + angle = 2.0 * math.pi * k / n + orbit = (Matrix.Translation(center) @ Matrix.Rotation(-angle, 4, 'Z') + @ Matrix.Translation(-center)) + c2w = orbit @ cam.matrix_world.normalized() + frames.append({"file_path": os.path.relpath(image, here).replace("\\", "/"), + "transform_matrix": [list(row) for row in c2w], + "fl_x": fx, "fl_y": fy, "cx": cx, "cy": cy, + "w": base["width"], "h": base["height"]}) + self._write_json(job["path"], {"camera_model": "OPENCV", "frames": frames}) + self._record(job["rec"], "cameras", job["path"]) + + def _job_bop(self, context, job): + """One BOP scene per product, from its perspective views. + + Poses are rigid and in millimetres of the source model, as BOP wants, + although the scene was scaled to fit: the scale is taken back out of + the camera translation, and the model is exported in its own units. + """ + scene = context.scene + s = scene.mv_settings + coll = bpy.data.collections.get(job["product"]) + if coll is None: + raise RuntimeError(f"product '{job['product']}' no longer exists") + root = job["path"] + obj_id = job["index"] + 1 + units_mm = UNIT_TO_MM.get(coll.mv_source_units, 1.0) + to_world = product_to_world(coll) + scale = abs(to_world.to_3x3().determinant()) ** (1.0 / 3.0) or 1.0 + scene_dir = os.path.join(root, "train_pbr", f"{job['index']:06d}") + for sub in ("rgb", "depth", "mask", "mask_visib"): + os.makedirs(os.path.join(scene_dir, sub), exist_ok=True) + + cams, gts, infos = {}, {}, {} + im_id = 0 + for entry in job["views"]: + cam = bpy.data.objects.get(entry["camera"]) + if cam is None or cam.data.type != 'PERSP': + continue + rgb = next((p for m, p in entry["files"].items() if m in {"", "render", "clay"}), None) + depth_exr = entry["files"].get("depth") + if not rgb or not depth_exr or not file_ok(rgb) or not file_ok(depth_exr): + continue + record = camera_record(scene, cam) + px = load_pixels(depth_exr, raw=True)[::-1] + hit = px[..., 3] > 0.5 + depth_mm = np.where(hit, px[..., 0] / scale * units_mm, 0.0) + depth_scale = max(0.1, math.ceil(depth_mm.max() / 65000.0 * 10) / 10) + write_png16(os.path.join(scene_dir, "depth", f"{im_id:06d}.png"), + np.rint(depth_mm / depth_scale)) + shutil.copyfile(rgb, os.path.join(scene_dir, "rgb", f"{im_id:06d}" + + os.path.splitext(rgb)[1].lower())) + mask = np.zeros(hit.shape + (4,), np.float32) + mask[..., :3] = hit[::-1, :, None] + mask[..., 3] = 1.0 + for sub in ("mask", "mask_visib"): + save_pixels(mask, os.path.join(scene_dir, sub, f"{im_id:06d}_000000.png"), 'PNG') + + m2c = Matrix(record["w2c_opencv"]) @ to_world + rot = m2c.to_3x3() * (1.0 / scale) + t_mm = m2c.translation / scale * units_mm + ys, xs = np.nonzero(hit) + bbox = ([int(xs.min()), int(ys.min()), int(xs.max() - xs.min() + 1), + int(ys.max() - ys.min() + 1)] if len(xs) else [-1, -1, 0, 0]) + count = int(hit.sum()) + key = str(im_id) + cams[key] = {"cam_K": [v for row in record["K"] for v in row], + "depth_scale": depth_scale} + gts[key] = [{"obj_id": obj_id, "cam_R_m2c": [v for row in rot for v in row], + "cam_t_m2c": list(t_mm)}] + infos[key] = [{"bbox_obj": bbox, "bbox_visib": bbox, "px_count_all": count, + "px_count_valid": count, "px_count_visib": count, + "visib_fract": 1.0 if count else 0.0}] + im_id += 1 + if not im_id: + raise RuntimeError("no perspective view with both an image and depth") + self._write_json(os.path.join(scene_dir, "scene_camera.json"), cams) + self._write_json(os.path.join(scene_dir, "scene_gt.json"), gts) + self._write_json(os.path.join(scene_dir, "scene_gt_info.json"), infos) + + first = cams["0"]["cam_K"] + w, h = render_resolution(scene) + camera_json = os.path.join(root, "camera.json") + if not os.path.isfile(camera_json): + self._write_json(camera_json, {"fx": first[0], "fy": first[4], "cx": first[2], + "cy": first[5], "width": w, "height": h, + "depth_scale": 0.1}) + info = self._export_bop_model(coll, to_world, units_mm, + os.path.join(root, "models", f"obj_{obj_id:06d}.ply")) + info_path = os.path.join(root, "models", f"models_info{s.shard_suffix}.json") + try: + with open(info_path, encoding="utf-8") as f: + models = json.load(f) + except (OSError, ValueError): + models = {} + models[str(obj_id)] = info + self._write_json(info_path, models) + self._record(job["rec"], "bop", os.path.join(scene_dir, "scene_gt.json")) + + def _export_bop_model(self, coll, to_world, units_mm, path): + """The product as a binary PLY in source-model millimetres, and its BOP + models_info entry. The diameter is the largest distance between any + two of up to 3000 sampled vertices, so it can fall slightly short on + dense meshes.""" + back = to_world.inverted() + verts, faces, offset = [], [], 0 + for obj in collection_meshes(coll): + if obj.type != 'MESH': + continue + me = obj.data + me.calc_loop_triangles() + co = np.empty(len(me.vertices) * 3, np.float64) + me.vertices.foreach_get("co", co) + m = np.array(back @ obj.matrix_world) + local = (co.reshape(-1, 3) @ m[:3, :3].T + m[:3, 3]) * units_mm + tri = np.empty(len(me.loop_triangles) * 3, np.int64) + me.loop_triangles.foreach_get("vertices", tri) + verts.append(local) + faces.append(tri.reshape(-1, 3) + offset) + offset += len(local) + verts = np.concatenate(verts).astype(np.float32) + faces = np.concatenate(faces).astype(np.int32) + os.makedirs(os.path.dirname(path), exist_ok=True) + with open(path, "wb") as f: + f.write((f"ply\nformat binary_little_endian 1.0\nelement vertex {len(verts)}\n" + "property float x\nproperty float y\nproperty float z\n" + f"element face {len(faces)}\nproperty list uchar int vertex_indices\n" + "end_header\n").encode("ascii")) + f.write(verts.astype("= 1 and 0 <= index < count + except (ValueError, AssertionError): + raise JobError(f"--shard wants INDEX/COUNT with 0 <= INDEX < COUNT, got {args.shard}") + s.shard_suffix = f"_shard{index:02d}" if count > 1 else "" + + files = collect_inputs(mv, job.get("inputs", []), base, job.get("recursive", True)) + if files: + mine = files[index::count] + position = {os.path.normcase(os.path.abspath(f)): i for i, f in enumerate(files)} + say(f"importing {len(mine)} of {len(files)} file(s)") + s.step_files.clear() + for path in mine: + s.step_files.add().path = path + if mine: + try: + bpy.ops.mv.import_all_steps() + except RuntimeError as exc: + raise JobError(f"import: {exc}") from exc + # Labels count through the whole job, not this shard, so shards never + # write the same AA_01. + for pc in mv.get_products(): + key = os.path.normcase(os.path.abspath(bpy.path.abspath(pc.mv_source or ""))) + if key in position: + pc.mv_label_index = position[key] + elif count > 1: + # Shard the products already in the .blend instead. + for i, pc in enumerate(mv.get_products()): + pc.mv_label_index = i + pc.mv_render = pc.mv_render and i % count == index + if not mv.get_products(): + raise JobError("no products to render") + + if job.get("standardize"): + bpy.ops.mv.standardize() + if job.get("auto_orient"): + method = job["auto_orient"] + if isinstance(method, str): + s.orient_method = method.upper() + bpy.ops.mv.auto_orient(all_products=True) + if job.get("scene_setup", True): + bpy.ops.mv.apply_scene_setup() + + issues = mv.preflight(context) + for issue in issues: + where = f"{issue['product']}: " if issue["product"] else "" + say(f"pre-flight {issue['severity'].lower()}: {where}{issue['message']}") + errors = [i for i in issues if i["severity"] == 'ERROR'] + if args.check: + say("pre-flight: " + ("all clear" if not issues else + f"{len(errors)} error(s), {len(issues) - len(errors)} warning(s)")) + return EXIT_FAILED if errors else EXIT_OK + if errors: + raise JobError("pre-flight errors, nothing rendered") + + mv.LAST_RENDER.clear() + try: + bpy.ops.mv.render_all(skip_preflight=True) + except RuntimeError as exc: + say(f"render: {exc}") + summary = dict(mv.LAST_RENDER) + print("MULTIVIEW_SUMMARY " + json.dumps(summary), flush=True) + if job.get("save_blend"): + bpy.ops.wm.save_as_mainfile(filepath=resolve(base, job["save_blend"]), copy=True) + if not summary.get("finished") or summary.get("aborted"): + return EXIT_FAILED + return EXIT_PARTIAL if summary.get("failures") else EXIT_OK + + +def merge(mv, folder): + """Fold every shard's lists into the single files a whole run writes.""" + folder = os.path.abspath(folder) + merged = [] + + shards = sorted(glob.glob(os.path.join(folder, "manifest_shard*.json"))) + if shards: + rows = [] + for path in shards: + with open(path, encoding="utf-8") as f: + rows.extend(json.load(f).get("images", [])) + rows.sort(key=lambda r: (str(r.get("label")), str(r.get("kind")), str(r.get("path")))) + with open(os.path.join(folder, "manifest.json"), "w", encoding="utf-8") as f: + json.dump({"images": rows}, f, indent=2) + with open(os.path.join(folder, "manifest.csv"), "w", encoding="utf-8", newline="") as f: + writer = csv.DictWriter(f, fieldnames=mv.MANIFEST_FIELDS, extrasaction='ignore') + writer.writeheader() + for row in rows: + size = row.get("source_size") + if isinstance(size, list): + row = dict(row, source_size=" x ".join(f"{v:g}" for v in size)) + writer.writerow(row) + merged.append(f"manifest ({len(shards)} shards, {len(rows)} rows)") + + shards = sorted(glob.glob(os.path.join(folder, "coco_shard*.json"))) + if shards: + images, annotations, categories = [], [], {} + for path in shards: + with open(path, encoding="utf-8") as f: + part = json.load(f) + remap = {} + for image in part.get("images", []): + remap[image["id"]] = len(images) + 1 + images.append(dict(image, id=remap[image["id"]])) + for ann in part.get("annotations", []): + annotations.append(dict(ann, id=len(annotations) + 1, + image_id=remap[ann["image_id"]])) + for cat in part.get("categories", []): + categories.setdefault(cat["id"], cat) + with open(os.path.join(folder, "coco.json"), "w", encoding="utf-8") as f: + json.dump({"images": images, "annotations": annotations, + "categories": sorted(categories.values(), key=lambda c: c["id"])}, + f, indent=2) + merged.append(f"COCO ({len(images)} images, {len(annotations)} instances)") + + models = os.path.join(folder, "bop", "models") + shards = sorted(glob.glob(os.path.join(models, "models_info_shard*.json"))) + if shards: + info = {} + for path in shards: + with open(path, encoding="utf-8") as f: + info.update(json.load(f)) + with open(os.path.join(models, "models_info.json"), "w", encoding="utf-8") as f: + json.dump(dict(sorted(info.items(), key=lambda kv: int(kv[0]))), f, indent=2) + merged.append(f"BOP models_info ({len(info)} objects)") + + if not merged: + raise JobError(f"no shard files in {folder}") + for line in merged: + say(f"merged {line}") + return EXIT_OK + + +def main(argv=None): + if argv is None: + argv = sys.argv[sys.argv.index("--") + 1:] if "--" in sys.argv else [] + args = parse_args(argv) + mv = load_addon() + try: + if args.list_settings: + list_settings(mv) + return EXIT_OK + if args.merge: + return merge(mv, args.merge) + return run_job(mv, args) + except JobError as exc: + say(f"error: {exc}") + return EXIT_FAILED + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/reports/Multiview renderer literature gap.md b/reports/Multiview renderer literature gap.md new file mode 100644 index 0000000..84b594f --- /dev/null +++ b/reports/Multiview renderer literature gap.md @@ -0,0 +1,89 @@ +# MultiView Fills a Narrow, Defensible Gap + +MultiView Product Renderer does fill a genuine gap, but the gap lies where several features meet, not in any single feature, and the add-on cannot yet hold that ground in front of a research reviewer. Each of its capabilities exists somewhere else. Paid Blender add-ons sell orthographic sheets, turntables and folder batch rendering. BlenderProc and Kubric produce far richer annotations. KeyShot and SOLIDWORKS Visualize automate rendering through paid tiers. A pythonOCC generator (Ortho2CAD) already makes better engineering drawings. **What no surveyed tool combines** is this: native STEP/IGES import that keeps part names and colours, one deterministic canonical camera rig shared across a mixed catalogue, real-size metadata, a no-code interface, and a free, open licence. That combination matters because the CAD machine-learning community re-renders its data for almost every paper and rarely documents how (Text2CAD and cadrille name no renderer), and because 3D-generation datasets each invent their own rig and normalization. The strongest framing is therefore **a reproducible, CAD-aware multi-view rendering protocol**, not "a product renderer". As shipped, though, the add-on falls short of what reviewers check first. It has no depth, normal, segmentation or camera-pose export. Its line art is a raster image with no hidden lines. It has no headless command line and no committed tests. Its public history is 14 commits since 7 September 2026, which fails JOSS's six-month and research-impact rules. The realistic sequence is a non-archival SynData4CV short paper with an experiment, then JOSS once it is eligible, then a dataset paper (Elsevier *Computer-Aided Design* or NeurIPS Evaluations & Datasets). The additions with the most impact per unit of effort are camera-pose export, a headless CLI with tests in CI, depth and normal passes, part-ID masks taken from STEP part names, and vector line drawings with hidden lines. + +## The gap only appears where four communities fail to meet + +Four groups of tools each cover part of what MultiView does, and each has a characteristic blind spot. **Commercial product-visualization suites** can automate rendering, but only through paid upper tiers and hand-written scripts. KeyShot costs US$1,299–1,599 per seat per year ([KeyShot Pricing](https://www.keyshot.com/pricing/)). Its batch-over-a-folder workflow exists as a community script that imports each CAD file and queues one render per file ([KeyShot forum](https://db_e596cbae_9eea_47cf_9859_c9418b6eabfd.influitive.com/forum/t/keyshot-studio-python-script-batch-import-render-automation/14613)), and headless mode cannot call `renderAnimation` ([KeyShot forum](https://community.keyshot.com/forum/t/headless-hurdles/507)). SOLIDWORKS Visualize locks its render queue and "render all camera angles" behind the Professional tier ([SOLIDWORKS PDF](https://www.solidworks.com/sw/docs/SW_Visualize_Top_Professional-only_Features.pdf)), and VRED keeps its Python API out of the Basic edition ([Autodesk APS](https://aps.autodesk.com/node/4011)). These tools are built around one hero project with many cameras, not a folder of unrelated parts that must come out at one scale. + +**The Blender add-on market** comes closest, and the paper must say so. Shot Studio Pro ships an "Orthographic Sheet (4)" preset, e-commerce camera sets and batch rendering ([Superhive](https://superhivemarket.com/products/shot-studio-pro)). Batch Render Pro loads a folder of models and renders each from selected cameras with render passes ([ArtStation](https://www.artstation.com/marketplace/p/1KBxv/blender-batch-render-pro-addon)). SpinWiz batches turntables across models ([Blender Market](https://blendermarket.com/products/spinwiz/faq)). A free GPL STEP importer is on the official Extensions platform, but it advertises only "basic material support" ([extensions.blender.org](https://extensions.blender.org/add-ons/step-importer/)). None of the listings found advertises STEP part names and colours, one framing shared across a catalogue, a manifest with source size and units, or pre-flight checks. That is an absence of advertising, not proof that no such tool exists, and the paper should word it that way. + +**Synthetic-data pipelines** are the closest academic neighbours, and they optimize for the opposite goal. BlenderProc loads OBJ, PLY, blend, FBX and dataset formats but not STEP. It writes depth, normals, segmentation, COCO and BOP, and is driven by Python scripts through `blenderproc run` ([BlenderProc GitHub](https://github.com/DLR-RM/BlenderProc)). Kubric and Omniverse Replicator are built around "randomize the scene" ([Omniverse Replicator docs](https://docs.omniverse.nvidia.com/extensions/latest/ext_replicator.html)). Unity Perception has been discontinued ([Unity Perception GitHub](https://github.com/Unity-Technologies/com.unity.perception)). Only Omniverse has a CAD path, and its STEP import fails often enough that NVIDIA staff reproduced one such failure on the forums ([NVIDIA forum](https://forums.developer.nvidia.com/t/import-issue-not-able-to-import-step-file-using-isaac-sim-4-2-0/310803)). None of these tools documents fixed canonical orthographic views, studio lighting presets or line-art output: they are designed to cover a distribution of views, not to make outputs comparable. + +**CAD-ML datasets** ship geometry, not images. ABC distributes STEP, Parasolid, OBJ and STL per chunk with no renders ([NYU FDA](https://archive.nyu.edu/jspui/handle/2451/44409?mode=simple)). The PNGs in Fusion 360 Gallery are "a png screen capture" with no documented camera, lighting or resolution ([reconstruction.md](https://raw.githubusercontent.com/AutodeskAILab/Fusion360GalleryDataset/master/docs/reconstruction.md)). Its Assembly set keeps part names and appearance names but no colour values ([assembly.md](https://raw.githubusercontent.com/AutodeskAILab/Fusion360GalleryDataset/master/docs/assembly.md)). + +| Capability | Commercial suites | Blender add-ons | BlenderProc / Kubric | CAD dataset tooling | MultiView today | +|---|---|---|---|---|---| +| STEP/IGES with part names and colours | Yes (paid) | Partial, uneven | No (meshes only) | pythonOCC viewers | **Yes (FreeCAD route)** | +| Fixed canonical ortho and perspective rig, shared across products | Scripted | 4-view sheet | Scripted | Ad hoc | **Yes** | +| Real relative scale and recorded source size | Not advertised | Not advertised | Not documented | Units vary (cm, unit cube) | **Yes** | +| Depth, normals, segmentation, camera pose | Partial | Some passes | **Yes, standard** | Rare | **No** | +| Line drawings with hidden lines, vector | Not checked | Not advertised | No | **Yes (pythonOCC HLR)** | Raster, visible only | +| Headless batch at dataset scale | Paid scripting | Varies | **Yes** | Yes | **No CLI** | +| No-code, free and open | No | Mostly paid | Code-first | Code-first | **Yes (MIT)** | + +The table shows the add-on's real position. It wins on CAD semantics, standardization and accessibility. It loses badly on the annotation and scale features that every research-grade generator treats as basic. + +## CAD-ML research re-renders everything and documents almost none of it + +The strongest evidence for a literature gap comes from the image-consuming CAD papers themselves. Text2CAD renders each part and the final model "from predetermined camera angles" to caption roughly 170K models with LLaVA-NeXT, but names no renderer, resolution or colours. Its limitations section also blames **perspective distortion** in those multi-view images for errors that propagate into the generated prompts ([Text2CAD](https://arxiv.org/html/2409.17106v1)). cadrille tiles four fixed views into a 268×268 grid and says only that it follows "established data generation protocols" ([cadrille](https://arxiv.org/html/2505.22914)). CADTalk found untextured CAD renders unrealistic enough that it routes depth maps from 10 viewpoints through ControlNet ([CADTalk](https://arxiv.org/pdf/2311.16703)). Two different papers called Img2CAD use two different Blender protocols, one of them 36 random views per model ([ResearchGate](https://www.researchgate.net/publication/384680308_Img2CAD_Conditioned_3D_CAD_Model_Generation_from_Single_Image_with_Structured_Visual_Geometry)). Image-to-CAD is growing fast, with OpenRealCAD, GIFT, Ortho2CAD and VisCAD all appearing in 2026 ([awesome-ai4cad](https://github.com/gudo7208/awesome-ai4cad)). Each new paper is another bespoke renderer. + +The contrast case shows what is missing. Amazon Berkeley Objects documents its whole protocol: 91 cameras on an upper icosphere at 60° FOV, three random HDRIs out of 108, canonical front alignment and real-world scale ([ABO](https://ar5iv.labs.arxiv.org/html/2110.06199)). But ABO uses artist meshes, not CAD. **No CAD dataset found matches ABO's protocol discipline**, and none provides colour-preserving, part-distinguishable renders of assemblies. MultiView's FreeCAD route, which reads STEP colours itself and passes them on because FreeCAD's console mode cannot, addresses exactly the step where pipelines lose appearance. OpenCASCADE keeps names and colours only through its XCAF reader ([OCC forum](https://dev.opencascade.org/content/preserving-colours-step-file-when-modifying-geometry-open-cascade)), and one converter vendor reports that common glTF exports collapse to all-white ([Adom Step2GLB](https://wiki.adom.inc/adom/adom-step2glb/files/README.md)). That vendor claim is unverified and should be cited as such. + +The 3D-generation literature has the same problem at larger scale. Zero-1-to-3 samples 12 random views, normalizes to [-0.5, 0.5]³ and uses a 49.1° FOV ([Zero-1-to-3](https://arxiv.org/abs/2303.11328)). LRM normalizes to [-1, 1]³ with 32 random views ([LRM](https://arxiv.org/abs/2311.04400)). SyncDreamer uses a 16-view ring at 30° elevation ([SyncDreamer](https://arxiv.org/abs/2309.03453)). Zero123++ changed its own elevations between v1.1 and v1.2 and had to fix the output FOV at 30° ([Zero123++ README](https://github.com/SUDO-AI-3D/zero123plus)). TRELLIS sends Blender's output to DEVNULL, so failed renders go unnoticed ([TRELLIS render.py](https://github.com/microsoft/TRELLIS/blob/main/dataset_toolkits/render.py)). One coincidence works in MultiView's favour: its default **Fit Each at a 1.0 m target with recentring already yields the [-0.5, 0.5]³ box** used by Zero-1-to-3, MVDream and SyncDreamer. Matching LRM needs only a 2.0 target. The only orthographic precedent among these datasets is Wonder3D's BlenderProc script ([Wonder3D render_codes](https://github.com/xxlong0/Wonder3D/tree/main/render_codes)), so the add-on's six-view orthographic rig is unusual there and standard in engineering. + +On the catalogue side, the gap is starker but carries less research weight. The notes found **no peer-reviewed paper on automated catalogue imagery from CAD**. The only evidence is industry anecdote: Onshape's blog reports that IKEA made about 75% of its 2016 catalogue images from CAD ([Onshape blog](https://www.onshape.com/en/blog/how-to-render-photorealistic-images-like-ikea)). There is a clear normative basis, though. GS1 prefers a 15° plunge for 3D primary images and encodes GTIN, facing and angle in file names such as `09520123456740_A1C1` ([GS1 handout](https://gs1.org/docs/images/Primary-Images-handout.pdf)), and that maps directly onto MultiView's orthographic views, perspective views and naming templates. This angle supports an application section in the paper. It is too thin to carry a research paper alone, because the search did not include Google Scholar or ASME directly. + +## Five weaknesses a reviewer will find in the first hour + +**First, the annotation gap is total.** Depth, normals, instance segmentation, camera intrinsics and extrinsics, and 6D pose are the standard output set for BlenderProc2 ([JOSS](https://joss.theoj.org/papers/10.21105/joss.04901.pdf)), Kubric ([arXiv](https://arxiv.org/pdf/2203.03570)) and BOP ([bop_toolkit](https://github.com/thodan/bop_toolkit/blob/master/docs/bop_datasets_format.md)). Every 3D-generation dataset exports per-view camera matrices, either as Objaverse-XL `.npy` files ([objaverse-xl](https://github.com/allenai/objaverse-xl/tree/main/scripts/rendering)) or as NeRF-style `transforms.json` ([nerfstudio](https://docs.nerf.studio/quickstart/data_conventions.html)). MultiView has a mask and nothing else. Its manifest records source size but not the normalization transform, so outputs cannot be mapped back to CAD units. Any claim about machine-learning data will fail on this point alone. + +**Second, the line art is not an engineering drawing.** The pass thresholds a normal matcap in image space. That captures contours and creases, the two line families Cole et al. found most important to artists ([Princeton](https://gfx.cs.princeton.edu/proj/ld3d/)). It gives no visibility information, aliased raster lines and no vector output. Bénard and Hertzmann treat extraction and visibility as separate problems ([arXiv 1810.01175](https://arxiv.org/abs/1810.01175)). Ortho2CAD's pythonOCC generator already writes first-angle projections with **dashed hidden lines and bounding-box dimensions**, more than a million of them ([Ortho2CAD](https://arxiv.org/abs/2607.08891)). PlankAssembly treats visible-only input as a degraded condition ([PlankAssembly](https://arxiv.org/abs/2308.05744)). The honest position is that the line pass is a fast documentation aid. It is not a drawing-to-CAD data generator. CAD2Program's finding that a raster encoder can match vector input ([CAD2Program](https://arxiv.org/html/2412.11892v2)) only partly softens this point. + +**Third, there is no headless path at dataset scale.** Objaverse-XL's 12-view renders took about 8K T4 GPU-hours ([NeurIPS supplement](https://papers.neurips.cc/paper_files/paper/2023/file/70364304877b5e767de4e9a2a511be0c-Supplemental-Datasets_and_Benchmarks.pdf)), and Kubric ships Docker because of "substantial variations between versions" of Blender ([Kubric](https://arxiv.org/pdf/2203.03570)). The README notes that Render All can be called from a script, but there is no documented `blender -b` entry point, config file or sharding. The resumable Skip Existing batch is a good foundation that the paper should present. + +**Fourth, the engineering hygiene fails JOSS's checklist.** The add-on is one 6,106-line `__init__.py`. The only CI workflow builds a release zip, there are no committed tests, and `test_files/` is gitignored scratch space. JOSS rates "an automated test suite hooked up to continuous integration" as Good and documented manual steps as only OK. It also asks single-author projects for tests, CI, a CONTRIBUTING file and governance statements ([JOSS review criteria](https://joss.readthedocs.io/en/latest/review_criteria.html); [JOSS submitting](https://joss.readthedocs.io/en/latest/submitting.html)). There is no `CITATION.cff`, no Zenodo DOI, and no `blender_manifest.toml` for the Extensions platform. + +**Fifth, the history and the evidence of use are not there yet.** JOSS now requires the repository to have been public for more than six months, with history showing "ongoing iteration, not a single burst of commits", and evidence of actual research use, since "aspirational statements about future use are not sufficient" ([JOSS submitting](https://joss.readthedocs.io/en/latest/submitting.html)). The git log has 14 commits from 7 September to 26 October 2026, several of them large version jumps such as "Add … v1.3.0" and "Expand … (1.9.0)". If the repository has been public since its first commit, the earliest eligible date is about **March 2027**, and only if a documented research use exists by then. + +There is also a design tension the paper should state rather than hide. The add-on's clean, consistent studio lighting is the opposite of what the sim-to-real literature recommends for training. In BOP 2020, PBR images in plausible scenes lifted CosyPose on texture-less T-LESS from 6.1% to 64.0% compared with render-and-paste ([BOP 2020](https://arxiv.org/abs/2009.07378)). SIP-17 found that noisy, randomized renders of industrial parts beat clean ones ([SIP-17](https://arxiv.org/pdf/2404.08778)). MultiView's renders suit benchmarks, retrieval, documentation and VLM conditioning, not detector training as they stand. Post-processing with Trim to Content also breaks the shared scale, so the manifest must flag which outputs keep it. + +## Frame it as a protocol, then publish in sequence + +The paper should present **a documented, CAD-aware rendering protocol and its reference implementation**: STEP-to-image with preserved part semantics, canonical views shared across a catalogue, recorded real scale, and a provenance manifest. It should not claim to be a better renderer than BlenderProc or a first of its kind. A working title might be *"MultiView: Reproducible Canonical Multi-View Rendering of CAD Products with Preserved Part Semantics."* The comparison table reviewers will expect is BlenderProc2's own JOSS table, which positions it against NDDS, NViSII, Habitat, Stillleben and Kubric on segmentation, depth, flow, normals, pose and camera sampling ([BlenderProc2 JOSS](https://joss.theoj.org/papers/10.21105/joss.04901.pdf)). MultiView should reproduce that table honestly and add columns where it wins: CAD import, part names and colours, a canonical shared rig, real scale, no-code use and line art. + +| Venue | Fit | Evidence it requires | Readiness today | +|---|---|---|---| +| **SynData4CV @ CVPR** (non-archival) | Best near-term | 4-page short or 8-page long paper, double-blind via OpenReview. Topics include scalable generation pipelines and "whether it needs to look realistic". 2026 deadline was 17 March ([SynData4CV](https://syndata4cv.github.io/)) | Needs one experiment, e.g. clean canonical vs randomized renders tested on T-LESS or real photos | +| **JOSS** | Best software home | 750–1750 words with Summary, Statement of need, State of the field, Software design, Research impact statement and AI usage disclosure ([JOSS paper](https://joss.readthedocs.io/en/latest/paper.html)); six-month public history; demonstrated research use; tests | Ineligible until about March 2027 plus a documented use | +| **SoftwareX** | Fallback | About 3,000 words (one source says 6 pages), GitHub repository with README and licence; official guide unverified ([Cirad](https://ou-publier.cirad.fr/en/node/5493)) | Feasible once tests and pose export exist; APC not confirmed | +| **JORS** | Fallback | Software metapaper, public repository with persistent ID, "substantive intellectual contribution", APC GBP 824 ([JORS](https://openresearchsoftware.metajnl.com/about/submissions)) | Feasible similarly | +| **Elsevier *Computer-Aided Design*** | Dataset paper | "Dataset papers" type with no algorithmic contribution required; dataset freely usable. Rendering and visualization are not listed topics ([Elsevier](https://shop.elsevier.com/journals/computer-aided-design/0010-4485)) | Needs a released render set of a public corpus (ABC, Fusion 360 Gallery) with part-ID masks and a baseline | +| **NeurIPS Evaluations & Datasets** | High bar | "Data generators" in scope; code required when a tool is the main contribution; Croissant with RAI metadata on approved hosts ([NeurIPS CFP](https://neurips.cc/Conferences/2026/CallForEvaluationsDatasets); [NeurIPS blog](https://blog.neurips.cc/2026/05/04/responsible-ai-metadata-requirements-for-the-evaluations-and-datasets-track-neurips-2026/)) | Needs generator, dataset and baselines; 2026 deadline was May | + +Elsevier's ASME counterparts are weaker targets: the JCISE scope page could not be fetched, and its data-wrangling special issue closed in 2022 ([ASME](https://www.asme.org/topics-resources/society-news/asme-news/special-journal-issue-on-data-wrangling-to-support-research-on-engineering-design-and-manufacturing)). The recommended sequence follows from the gates. Build the top additions now. Use the add-on to render a CAD benchmark subset and write the SynData4CV short paper around one experiment. Being non-archival, that paper blocks nothing later and itself becomes the research-use evidence JOSS asks for. Submit to JOSS once the six-month window closes. Then turn the rendered corpus into a *Computer-Aided Design* dataset paper or a NeurIPS E&D submission. The 2027 dates for SynData4CV and NeurIPS were not published in the sources and are assumed to follow the 2026 pattern. + +## Eight additions ranked by research impact per unit of effort + +The ranking weighs how many communities a feature unlocks against how cheap it is in Blender, given that the rig is already fixed and deterministic. + +| Rank | Addition | Effort | Who it unlocks, and the evidence | +|---|---|---|---| +| 1 | **Per-view camera export**: K, extrinsics, nerfstudio-style `transforms.json`, plus the normalization transform in the manifest | Low | NeRF, Gaussian splatting and 3D generation all consume camera matrices; nerfstudio uses Blender's own OpenGL convention, so no axis flip is needed ([nerfstudio](https://docs.nerf.studio/quickstart/data_conventions.html)) | +| 2 | **Headless CLI, config file, per-product sharding, pinned Blender or Docker, tests in CI** | Medium | Moves JOSS Tests from OK to Good ([JOSS](https://joss.readthedocs.io/en/latest/review_criteria.html)); needed for dataset scale ([Kubric](https://arxiv.org/pdf/2203.03570)) | +| 3 | **Depth (metric, mm) and normal passes** | Low | G-Objaverse ships normal and depth EXRs ([G-Objaverse](https://github.com/modelscope/richdreamer/tree/main/dataset/gobjaverse)); Wonder3D generates normals jointly ([Wonder3D](https://arxiv.org/abs/2310.15008)) | +| 4 | **Part-ID and instance masks from STEP part names, with COCO and BOP writers** | Medium | The unique differentiator: part-level renders for Text2CAD-style captioning ([Text2CAD](https://arxiv.org/html/2409.17106v1)); SIP-17 accuracy fell sharply on assemblies ([SIP-17](https://arxiv.org/pdf/2404.08778)); for a single object the BOP `scene_gt` entry is just the object-to-camera transform ([bop_toolkit](https://github.com/thodan/bop_toolkit/blob/master/docs/bop_datasets_format.md)) | +| 5 | **Engineering-drawing mode**: Line Art occlusion levels for visible and hidden lines, dashed hidden layer, SVG, shared recorded mm-per-pixel scale | Medium | Line Art already computes occlusion levels ([Blender manual](https://docs.blender.org/manual/en/dev/grease_pencil/modifiers/generate/line_art.html)); adding dimensions raised CAD2Program F1 from 62.65 to 82.76 ([CAD2Program](https://arxiv.org/html/2412.11892v2)); TriView2CAD cites the lack of dimensioned drawings ([arXiv 2506.00568](https://arxiv.org/html/2506.00568v1)) | +| 6 | **Named protocol presets**: MVCNN 12-view, Zero123, SyncDreamer, Zero123++ v1.2, MVDream, GS1 15° plunge | Low | Lets users reproduce benchmarks exactly ([MVCNN](https://arxiv.org/abs/1505.00880); [Zero123++](https://github.com/SUDO-AI-3D/zero123plus)); GS1 naming and plunge ([GS1](https://gs1.org/docs/images/Primary-Images-handout.pdf)) | +| 7 | **Provenance manifest and Croissant export**: tool and Blender version, engine, samples, seed, sha256 of inputs, `citeAs` | Low | NeurIPS E&D requires Croissant with RAI metadata ([NeurIPS blog](https://blog.neurips.cc/2026/05/04/responsible-ai-metadata-requirements-for-the-evaluations-and-datasets-track-neurips-2026/)); `citeAs` makes every dataset cite the tool ([Croissant spec](https://docs.mlcommons.org/croissant/docs/croissant-spec.html)) | +| 8 | **Optional randomization mode**: HDRI, materials, camera jitter and distractors layered on the canonical rig | Medium–high | PBR plus plausible context dominates for texture-less parts ([BOP 2020](https://arxiv.org/abs/2009.07378)); domain randomization transfers to real images ([Tobin et al.](https://arxiv.org/abs/1703.06907v1)) | + +The first four items together close most of the gap with BlenderProc while keeping the CAD semantics that BlenderProc lacks. That combination is the paper's core claim. Item 8 is ranked last on purpose: randomization is BlenderProc's home ground, and competing there weakens the standardization story. Exporting a scene that BlenderProc can load is a cheaper bridge than building randomization in. Two packaging tasks cost almost nothing and belong alongside item 2: a `CITATION.cff` with Zenodo release DOIs ([GitHub Docs](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-citation-files)), and splitting the single file into modules. + +The tool also enables two studies that could turn a software paper into a research paper. The notes found **no published study measuring how much rendering-protocol choices (normalization, FOV, lighting, background) change benchmark results**. The evidence so far is per-paper patches, such as Zero123++ fixing FOV and the elevation estimators that "introduce extra error" ([Zero123++](https://arxiv.org/abs/2310.15110)). A tool with named presets is the natural instrument for that ablation. Likewise, no study measures STEP colour and part-name loss across converters at scale. Running MultiView's FreeCAD route against the cascadio-based importer and a glTF pipeline on a public STEP corpus would produce that number. + +## Conclusion + +The real finding is that MultiView's best feature is one it barely advertises: **determinism plus CAD semantics**. Its renders look ordinary. What is rare is that part names, colours, real sizes and a single shared framing survive from STEP to pixels and could be recorded in a machine-readable manifest. That reframes the project from a product-photography convenience, a crowded and academically empty niche, into infrastructure for a field whose papers keep writing throwaway renderers and leaving them undocumented. The weaknesses are real but cheap to fix in the right order. Camera export, depth and normals, and part masks are each a few Blender passes on a rig that is already fixed. + +The binding constraint is time and evidence, not code. JOSS's history and research-use gates cannot be met before roughly spring 2027 however good the software becomes. The quickest way to meet them is to use the tool on a public CAD corpus and publish that use non-archivally first. Done that way, the venue sequence also serves as the validation plan, and the dataset that results is probably a more citable contribution than the add-on itself. diff --git a/research_notes/Multiview renderer literature gap/cad_datasets_and_ml.md b/research_notes/Multiview renderer literature gap/cad_datasets_and_ml.md new file mode 100644 index 0000000..d59de39 --- /dev/null +++ b/research_notes/Multiview renderer literature gap/cad_datasets_and_ml.md @@ -0,0 +1,166 @@ +# CAD / Engineering-Part Datasets and ML Work That Uses Rendered Images of CAD Models + +Scope: how the CAD/engineering ML community gets images out of CAD (B-rep/STEP) data, what each dataset ships, and where a CAD-native multi-view renderer (STEP/IGES via FreeCAD, keeping part names and STEP colours, standardized views) would help. Research date: October 2026. + +## 1. Datasets: formats, whether renders are provided, how renders were made, and whether names, colours and assembly structure survive + +### Takeaway +Most major CAD datasets ship B-rep (STEP/Parasolid/SMT) and sometimes tessellated meshes, but **none of the classic CAD datasets ships a standardized, documented multi-view image set**. Where images exist, they are unspecified thumbnails or screen captures (Fusion 360 Gallery) or narrow, task-specific renders (FllumaOne's "visible-edge" views, Img2CAD's Blender renders). Assembly hierarchy, names and appearance survive mainly in Fusion 360 Gallery Assembly (JSON metadata) and AutoMate (Onshape assemblies). ABO, the product-imagery contrast case, is the only one with a fully documented Blender rendering protocol, PBR materials, canonical orientation and real-world scale, but it has artist meshes rather than CAD. + +### Cited Findings + +**ABC (Koch et al., CVPR 2019)** +- About 1 million CAD models, each "a collection of explicitly parametrized curves and surfaces", with ground truth for differential quantities, patch segmentation, feature detection and reconstruction. The abstract says sampling the parametric descriptions "allows generating data in different formats and resolutions." — [arXiv 1812.06216](https://arxiv.org/abs/1812.06216) +- Each chunk holds about 10k objects, packaged as separate 7z archives per file type: `step`, `para` (Parasolid), `obj`, `feat`, `stat`, `meta`, plus `ofs` and `stl2`. For chunk 0081 the archives run from about 1.1 GB (STEP) and 2.0 GB (Parasolid) to 5.6 GB (OBJ) and 8.7 GB (STL). Released under the MIT License, but "the copyright of these CAD models is owned by their creators", and the authors thank Onshape for providing the models. — [NYU FDA, ABC chunk 0081](https://archive.nyu.edu/jspui/handle/2451/44409?mode=simple); [NYU FDA, chunk 0002](https://archivefda.dlib.nyu.edu/handle/2451/44318) +- No rendered images appear among the listed chunk file types. — [NYU FDA chunk listing](https://archive.nyu.edu/jspui/handle/2451/44409?mode=simple) +- A 2026 Autodesk re-release, "ABC-1M" (Hugging Face ADSKAILab/ABC-1M), is listed as processed ABC solids with geometry and topology attributes and train/val/test splits. — [awesome-ai4cad survey list](https://github.com/gudo7208/awesome-ai4cad) + +**Fusion 360 Gallery (Willis et al., SIGGRAPH 2021, plus follow-ups)** +- Sub-datasets: Assembly (8,251 assemblies, 154,468 parts), Assembly-Joint (32,148 joints, 23,029 parts), Reconstruction (8,625 sequences), Segmentation (35,680 parts; the extended set has 42,912 STEP files). Faces in the Segmentation set are labelled by modelling operation (Extrude, Fillet, Chamfer, and so on). — [Fusion360GalleryDataset repo](https://github.com/AutodeskAILab/Fusion360GalleryDataset) +- **Reconstruction** gives per design `.smt` (native Autodesk Shape Manager B-rep), `.step`, `.obj` (triangulated, each face a named group, "not manifold"), `.json` (construction sequence) and `.png`. The PNG is "a png screen capture of the design at this point in the sequence". The docs give no camera, lighting or resolution. Units are cm and radians. Entity names are mostly auto-generated ("Sketch1", "Body1") and "could also be in a localized version." The split is 6,900 train / 1,725 test. — [reconstruction.md](https://raw.githubusercontent.com/AutodeskAILab/Fusion360GalleryDataset/master/docs/reconstruction.md) +- **Assembly** gives `.smt`, `.step`, `.obj` (meshes "not guaranteed to be manifold"), JSON, and "a thumbnail .png image file together with each body and one for the overall assembly". The thumbnails' rendering method is not described. Bodies, components and occurrences carry `name`, and components carry `part_number`. Each body has a `material` (UUID and name, used for density and mass) and an `appearance` (UUID and name of the user-assigned appearance), **but no colour values are documented**. A `tree` holds the designer-defined hierarchy, and occurrences carry `transform`. All units are cm. — [assembly.md](https://raw.githubusercontent.com/AutodeskAILab/Fusion360GalleryDataset/master/docs/assembly.md) +- The repo's `tools/` directory holds Fusion 360 API tools for reconstruction, B-rep traversal and "conversion to other formats". No dedicated rendering pipeline is named. — [Fusion360GalleryDataset repo](https://github.com/AutodeskAILab/Fusion360GalleryDataset) + +**DeepCAD (Wu et al., ICCV 2021)** +- `cad_json` holds construction sequences "parsed from Onshape public documents with links from [ABC dataset]". `cad_vec` holds vectorized sequences. Utilities include `export2step.py` (STEP export, with pythonocc-core 7.5.1 as a dependency), `json2pc.py` (point clouds) and `show.py`, which visualizes "with opencascade". The dataset ships no images. — [DeepCAD repo](https://github.com/ChrisWu1997/DeepCAD) + +**SketchGraphs (Seff et al., ICML 2020 workshop)** +- "15 million sketches extracted from real-world CAD models" (Onshape), stored as geometric constraint graphs. Raw JSON is 128 zstd tar archives (about 43 GB), plus construction sequences in a custom binary file (15 GB) and filtered splits. The page mentions no rendered images and no 3D or B-rep geometry. — [SketchGraphs repo](https://github.com/PrincetonLIPS/SketchGraphs) + +**MFCAD / MFCAD++ / Hierarchical CADNet** +- MFCAD is "a dataset of 3D CAD models with machining feature labels". The models sit in `dataset/step`, and the pythonocc-core scripts are for exploring and visualizing them. No images are provided. — [hducg/MFCAD](https://github.com/hducg/MFCAD) +- MFCAD++ succeeds MFCAD with more complex models and non-planar features: 24 machining feature classes plus stock, 3 to 10 features per model, stored in STEP (ISO 10303), generated with PythonOCC. Hierarchical CADNet learns directly from B-reps and was evaluated on it. — [Hierarchical CADNet, CAD journal (QUB PDF)](https://pure.qub.ac.uk/files/328098955/1_s2.0_S0010448522000240_main.pdf); [ScienceDirect](https://sciencedirect.com/science/article/pii/S0010448522000240?via=ihub%3D) + +**CADNET (Manda, Bhaskare, Muthuganapathy, IEEE Access 2021)** +- 3,317 engineering models in 43 categories, collected from the Engineering Shape Benchmark (ESB) and the National Design Repository and augmented with newly modelled parts. The best classifier fed Light Field Descriptor-style **rendered images** into a ResNet-inspired CNN. This is an early example of a multi-view, image-based approach to classifying mechanical parts. — [arXiv 2107.06481](https://arxiv.org/abs/2107.06481v1); [ar5iv](https://ar5iv.labs.arxiv.org/html/2107.06481) + +**MCB, the Mechanical Components Benchmark (Kim et al., ECCV 2020)** +- More than 58,000 annotated 3D mechanical parts. Dataset A aggregates TraceParts, 3D Warehouse and GrabCAD, and Dataset B aggregates 3D Warehouse and GrabCAD. TraceParts gave API access to its catalogue. The benchmark covers point-cloud, voxel and view-based (multi-view image) classifiers, and the authors stress "the significant difficulty of collecting 3D CAD models of mechanical components." — [Purdue project page](https://engineering.purdue.edu/cdesign/wp/a-large-scale-annotated-mechanical-components-benchmark-for-classification-and-retrieval-tasks-with-deep-neural-networks/); [TraceParts press release](https://info.traceparts.com/news/traceparts-news/traceparts-supported-purdue-university-for-their-mechanical-components-benchmark-mcb-research/); [ECCV page](https://www.ecva.net/papers/eccv_2020/papers_ECCV/html/2945_ECCV_2020_paper.php) + +**AutoMate (Jones et al., SIGGRAPH Asia 2021): Onshape assemblies** +- Scraped from public Onshape documents via the Onshape API: 255,213 assemblies with mates, deduplicated with geometric "fingerprints" down to 92,529 unique assemblies. The Zenodo release has Parasolid and STEP (34.9 GB). Counts differ between versions: the paper text says 541,635 mates and its table says 514,635, while Dryad lists 451,967 parts, 255,211 assemblies and 1,292,016 mates. Duplicates were "a major problem". — [arXiv 2105.12238](https://arxiv.org/html/2105.12238v2); [Dryad](https://datadryad.org/dataset/doi:10.5061/dryad.2547d7wvw); [Zenodo](https://zenodo.org/records/7776208) + +**Text2CAD (Khan et al., NeurIPS 2024 Spotlight)** +- About 170K DeepCAD models and about 660K text annotations. Annotation stage 1 renders **multi-view images of each individual part and of the final model** from "predetermined camera angles", and LLaVA-NeXT writes shape descriptions from them. Stage 2 has an LLM (Mistral) turn those descriptions into multi-level instructions. The paper names no renderer and gives no resolution or colours. — [arXiv 2409.17106 HTML](https://arxiv.org/html/2409.17106v1) + +**CAD-MLLM / Omni-CAD (2024)** +- 453,220 models after augmentation, each with text, multi-view images, point clouds and command sequences. The authors claim it is "the only dataset ... that simultaneously supports multi-view images, text, and point cloud". The Hugging Face card defers image rendering to the GitHub code, and I could not confirm the render specifics. — [arXiv 2411.04954](https://arxiv.org/pdf/2411.04954); [HF dataset card](https://huggingface.co/datasets/jingwei-xu-00/Omni-CAD/discussions/1/files) + +**CAD-Recode / cadrille (2025)** +- CAD-Recode has about 1 million procedurally generated CadQuery programs and is point-cloud only. cadrille adds images by tessellating the B-rep and rendering **4 fixed views tiled into a 2x2 grid of 268 x 268 px** for the Qwen vision encoder. It says it follows "established data generation protocols" and names no tool, lighting or colour. GT models are normalized to fit [-0.5, 0.5]^3. — [cadrille, arXiv 2505.22914](https://arxiv.org/html/2505.22914) + +**Img2CAD (two different papers share this name)** +- Img2CAD (SIGGRAPH Asia 2025; arXiv 2408.01437) uses a VLM (GPT-4V in v1, Llama 3.2 later) to predict CAD command structure and the TrAssembler transformer for parameters. Its released dataset includes a `blender_renderings/` folder of PNGs. — [GitHub](https://github.com/qq456cvb/Img2CAD); [HF dataset](https://huggingface.co/datasets/qq456cvb/img2cad-dataset); [ACM](https://dl.acm.org/doi/10.1145/3757377.3763891) +- A different "Img2CAD" (IEEE journal) builds "ABC-mono", over 200K DeepCAD-derived models with **36 Blender views per model from random viewpoints**. — [ResearchGate](https://www.researchgate.net/publication/384680308_Img2CAD_Conditioned_3D_CAD_Model_Generation_from_Single_Image_with_Structured_Visual_Geometry) + +**CADTalk (Yuan et al., CVPR 2024)** +- Benchmark of 5,288 machine-made and 45 human-made CAD programs with semantic comments. — [CVPR open access](https://openaccess.thecvf.com/content/CVPR2024/html/Yuan_CADTalk_An_Algorithm_and_Benchmark_for_Semantic_Commenting_of_CAD_CVPR_2024_paper.html) + +**2026 datasets that ship renders** +- FllumaOne-100K (arXiv 2606.17696) has 100K kernel-validated samples from an OpenCASCADE-based CAD system. Each includes the program, feature tree, IR, STEP, point cloud, text, metadata and **"eight canonical visible-edge renderings"**. — [arXiv 2606.17696](https://arxiv.org/pdf/2606.17696) +- OmniMech (arXiv 2608.05539, Aug 2026) has more than 251,000 dimensioned and toleranced 2D drawings paired with native CAD, multi-view renderings, meshes, STEP and B-rep. The data is announced but its release is unconfirmed. — [alphaxiv 2608.05539](https://www.alphaxiv.org/abs/2608.05539) +- Also listed for 2026: STEP-Parts (arXiv 2604.14927, a STEP B-rep toolchain with instance labels for about 180K DeepCAD/ABC models), an ECCV 2026 CAD Challenge "renders and engineering views paired with STEP B-rep targets", and SldprtNet (arXiv 2603.13098, SolidWorks .sldprt-based language-to-CAD). — [awesome-ai4cad](https://github.com/gudo7208/awesome-ai4cad) + +**ABO, Amazon Berkeley Objects (Collins et al., CVPR 2022): the product-imagery contrast** +- 147,702 listings, 398,212 catalogue images, and 8,222 products with 360-degree turntables of 24 or 72 frames. 7,953 artist-made glTF 2.0 models carry PBR materials (base colour, metallic, roughness). Models sit in "a canonical coordinate system where the 'front' (when well defined) ... are aligned", with "a scale corresponding to real world units". — [ar5iv 2110.06199](https://ar5iv.labs.arxiv.org/html/2110.06199); [AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-ygocdgcstr7v2) +- Rendering is fully specified. Material estimation uses 512x512 renders from 91 cameras on an upper icosphere (60-degree FOV) in Blender Cycles, lit by 3 random HDRIs out of 108 indoor ones (2.1M images). Single-view reconstruction uses 30 Blender viewpoints (40-degree FOV) on a blank background. The ABO-MVR retrieval benchmark matches renders against real catalogue images, and Recall@1 is about 30% for rendered queries versus about 40% for catalogue queries. — [ar5iv 2110.06199](https://ar5iv.labs.arxiv.org/html/2110.06199) + +**BOP industrial objects (T-LESS, ITODD)** +- The 7 core BOP datasets (LM-O, T-LESS, ITODD, HB, YCB-V, TUD-L, IC-BIN) come with about 350K photorealistic PBR training images generated and annotated with BlenderProc (BlenderProc4BOP). Objects are physics-dropped into a cube whose walls get random CC0 PBR materials. PBR training clearly beat "render & paste". — [BOP Challenge 2022 report](https://arxiv.org/pdf/2302.13075); [BlenderProc4BOP README](https://github.com/DLR-RM/BlenderProc/blob/main/README_BlenderProc4BOP.md); [BOP 2020](https://bop.felk.cvut.cz/challenges/bop-challenge-2020/) + +### Inferences +- Across the classic CAD datasets (ABC, DeepCAD, SketchGraphs, MFCAD/MFCAD++, AutoMate), images are not a first-class modality. Every image-consuming paper therefore re-renders the data itself, with its own camera rig, resolution and lighting, and these are often undocumented (Text2CAD, cadrille). This is the clearest gap a standardized CAD-to-multi-view tool could fill. +- Fusion 360 Gallery Assembly is the only classic CAD dataset found that keeps part names, hierarchy, material and appearance names. Even there, its PNG thumbnails come with no documented rendering protocol, and appearance colours are given as names/UUIDs rather than RGB values. +- ABO shows what a well-specified render protocol looks like: documented camera rig, HDRI lighting, canonical front orientation and real-world scale. No CAD dataset matches it. A CAD-native renderer that brings ABO-style protocol discipline to STEP data would be a credible positioning. +- Semantic, colour-preserving renders of **assemblies** (each part distinguishable by name and colour) appear to be missing from all of these datasets. + +### Gaps +- I could not open the ABC paper's full text (the CVF PDF returned 403 and the arXiv PDF came back as unreadable binary). The tool used to tessellate the OBJ/STL meshes (Onshape export or Open CASCADE) and any statement on units, colours or multi-body models are therefore unverified. +- I did not verify the exact model and class counts for MFCAD (original) and MCB, or MCB's distributed file format (likely meshes), from primary pages. The MCB PDF fetch failed on a certificate error. +- I did not verify the Fusion 360 Segmentation dataset's per-file contents (e.g. whether PNGs or `.seg` label files are included). +- I did not research CADParser or Traceparts-only industrial datasets beyond MCB's use of TraceParts. +- I could not confirm Omni-CAD's render details or ITODD-specific rendering details. + +## 2. ML tasks that consume CAD renders, and the rendering setups they report + +### Takeaway +Image/sketch-to-CAD, CAD VLM captioning, multi-view part classification and industrial sim-to-real detection all consume CAD renders. Rendering is usually a thin, bespoke pre-processing step: fixed 4 to 36 views, untextured or depth-only, often with no tool named. Several papers explicitly blame render quality for downstream errors: perspective distortion (Text2CAD), unrealistic untextured renders (CADTalk), and the need for material and texture randomization (SIP-17). + +### Cited Findings +- **CAD captioning / VLM annotation.** Text2CAD renders each part and the final model from predetermined angles for LLaVA-NeXT. Its limitations section says LLaVA-NeXT "is sensitive to the perspective distortions in multi-view images", and these errors propagate into the generated prompts. — [Text2CAD HTML](https://arxiv.org/html/2409.17106v1) +- **Semantic part labelling of CAD programs.** CADTalk notes that untextured CAD renders look unrealistic to vision models trained on photos. It renders depth maps from **10 viewpoints** and passes them through ControlNet with the category name as the prompt, using 4 seeds per view (40 images per shape). It reports 83.24% accuracy. — [CADTalk arXiv](https://arxiv.org/pdf/2311.16703); [MSR page](https://www.microsoft.com/en-us/research/?p=1034136) +- **Image-to-CAD reconstruction.** cadrille uses 4 fixed views tiled 2x2 at 268 px total. CAD-MLLM, evaluated on Omni-CAD, reaches mean Chamfer Distance 18.5 from point clouds versus 37.7 from multi-view images, so image conditioning still lags. — [cadrille arXiv 2505.22914](https://arxiv.org/html/2505.22914) +- Img2CAD (SIGGRAPH Asia 2025) trains a VLM plus TrAssembler on Blender renderings. The other Img2CAD (ABC-mono) uses 36 random-view Blender renders per model. — [Img2CAD GitHub](https://github.com/qq456cvb/Img2CAD); [ABC-mono Img2CAD](https://www.researchgate.net/publication/384680308_Img2CAD_Conditioned_3D_CAD_Model_Generation_from_Single_Image_with_Structured_Visual_Geometry) +- FllumaOne provides 8 canonical visible-edge renderings per sample, and MIRAGE-CAD (arXiv 2608.28669) uses these "rendered views" as an input modality for generating CAD programs. — [FllumaOne](https://arxiv.org/pdf/2606.17696); [MIRAGE-CAD](https://arxiv.org/pdf/2608.28669) +- Newer 2026 image-to-CAD work listed in the survey includes OpenRealCAD (real photographs paired with editable CAD sequences, arXiv 2608.30617), GIFT (building image-to-CAD training data, arXiv 2603.27448), Ortho2CAD (arXiv 2607.08891) and VisCAD (arXiv 2609.03811). This shows image-to-CAD is an active and growing task. — [awesome-ai4cad](https://github.com/gudo7208/awesome-ai4cad) +- **Multi-view classification of mechanical parts.** MCB benchmarks view-based classifiers alongside point and voxel methods. CADNET's best result uses LFD rendered views with a CNN. — [MCB project page](https://engineering.purdue.edu/cdesign/wp/a-large-scale-annotated-mechanical-components-benchmark-for-classification-and-retrieval-tasks-with-deep-neural-networks/); [CADNET arXiv](https://arxiv.org/abs/2107.06481v1) +- **Cross-domain retrieval.** ABO-MVR matches rendered queries against real catalogue photos, and rendered queries do worse (about 30% vs about 40% R@1). — [ABO](https://ar5iv.labs.arxiv.org/html/2110.06199) +- **Industrial sim-to-real.** SIP-17 (CVPRW 2023) rendered 17 industrial parts and trained only on synthetic images, testing on 566 real photos. "Noisy" Syn_R (randomized backgrounds and post-processing) beat clean Syn_O, and performance dropped sharply on assemblies, which the authors say shows the need for material and texture randomization. — [arXiv 2404.08778](https://arxiv.org/pdf/2404.08778); [CVPRW](https://openaccess.thecvf.com/content/CVPR2023W/VISION/papers/Zhu_Towards_Sim-to-Real_Industrial_Parts_Classification_With_Synthetic_Dataset_CVPRW_2023_paper.pdf) +- For bin picking, Ojer et al. (2025) randomize lighting intensity, temperature and direction, plus material smoothness and reflectivity. Models trained only on virtual data perform comparably on real scenes. — [Ojer et al. 2025](https://doi.org/10.1177/17298806251374367) +- In BOP, BlenderProc PBR images drive state-of-the-art 6D pose results: 82.7 AR from GDRNPP trained on synthetic data only (2022). — [BOP 2022](https://arxiv.org/pdf/2302.13075) +- Metallic, reflective and low-contrast parts are the hardest to render convincingly, and published randomization setups are often tailored to one line, sensor and part taxonomy. — [roboticstomorrow 2026](https://www.roboticstomorrow.com/article/2026/03/how-bin-picking-ai-models-are-trained/26329); [MVPro Media](https://mvpromedia.com/from-cad-to-ai-how-synthetic-data-is-changing-machine-vision-training/) + +### Inferences +- There are two distinct consumer communities. (a) **CAD-generation and VLM researchers** want clean, consistent, canonical multi-views (fixed orthographic or isometric rigs, edge renders, part-level renders) and suffer from perspective distortion and undocumented rigs. (b) **Industrial vision and robotics practitioners** want photorealistic, randomized renders (BlenderProc, Omniverse). A Blender add-on that does standardized multi-view sets with consistent lighting and camera presets sits closer to (a), while offering a Blender-native path toward (b). +- Text2CAD and CADTalk both render individual parts or semantic regions. A tool that keeps STEP part names and colours could produce per-part renders, per-part masks and name labels directly, which would supply ready-made ground truth for part-aware VLM captioning and segmentation. +- Orthographic or fixed-FOV standardized views would directly address the perspective-distortion failure mode that Text2CAD reports. + +### Gaps +- Most image-to-CAD papers I checked (Text2CAD, cadrille, Omni-CAD) do not name their renderer. I could not determine whether pythonOCC, Open3D, PyVista or Blender is the dominant choice. +- I did not find ITODD-specific rendering details beyond its BOP/BlenderProc inclusion. + +## 3. Pain points: STEP-to-mesh at scale, loss of colours and part structure, units and scale, orientation + +### Takeaway +Pipelines from STEP to images routinely lose appearance and structure. Colours and names survive only along the OCCT XCAF path, and common exporters can flatten them. Datasets mix units (Fusion is cm, ABO uses real-world scale), meshes are often non-manifold, and duplicates and orientation are handled ad hoc: methods typically just rescale into a unit cube. + +### Cited Findings +- Keeping colours and names from STEP in OpenCASCADE requires the XCAF reader (`STEPCAFControl_Reader` with colour and name modes on). An OCC forum thread covers keeping STEP colours when geometry is modified. — [OCC forum](https://dev.opencascade.org/content/preserving-colours-step-file-when-modifying-geometry-open-cascade) +- One converter vendor says it preserves per-face colours by writing glTF with `RWGltf_CafWriter` after tessellating with `BRepMesh_IncrementalMesh`, and claims CadQuery's and KiCad's glTF/GLB exports collapse to all-white. This is a single vendor's wiki and should be treated as a lead, not verified fact. — [Adom Step2GLB](https://wiki.adom.inc/adom/adom-step2glb/files/README.md); contrast [CadQuery import/export docs](https://cadquery.readthedocs.io/en/stable/importexport.html), which show coloured assembly export +- build123d writes colours and names into XCAF labels before export and exposes linear and angular deflection settings for tessellating glTF. This shows that tessellation quality is a user-chosen parameter that varies between pipelines. — [build123d exporters3d](https://build123d.readthedocs.io/en/latest/_modules/exporters3d.html) +- Fusion 360 Gallery OBJ meshes are "not manifold" or "not guaranteed to be manifold". Units are cm, and names can be auto-generated or localized. — [reconstruction.md](https://raw.githubusercontent.com/AutodeskAILab/Fusion360GalleryDataset/master/docs/reconstruction.md); [assembly.md](https://raw.githubusercontent.com/AutodeskAILab/Fusion360GalleryDataset/master/docs/assembly.md) +- ABC distributes STEP, Parasolid, OBJ and STL side by side, and a single chunk of about 10k models takes about 1.1 GB as STEP versus about 8.7 GB as STL. Pre-tessellated meshes therefore cost about 8x the storage of the B-rep. — [NYU FDA chunk 0081](https://archive.nyu.edu/jspui/handle/2451/44409?mode=simple) +- Duplicates: AutoMate went from 255,213 scraped assemblies to 92,529 unique ones using geometric fingerprints. — [AutoMate](https://arxiv.org/html/2105.12238v2) +- Scale normalization is ad hoc. cadrille normalizes ground truth into [-0.5, 0.5]^3, whereas ABO keeps real-world units and a canonical front-facing orientation. — [cadrille](https://arxiv.org/html/2505.22914); [ABO](https://ar5iv.labs.arxiv.org/html/2110.06199) +- MCB's authors cite "the significant difficulty of collecting 3D CAD models of mechanical components" as the main barrier to building such datasets. — [MCB project page](https://engineering.purdue.edu/cdesign/wp/a-large-scale-annotated-mechanical-components-benchmark-for-classification-and-retrieval-tasks-with-deep-neural-networks/) +- FllumaOne keeps samples only after geometry, solid-validity and export checks, which implies that export and validity failures are common enough to need filtering. — [FllumaOne](https://arxiv.org/pdf/2606.17696) + +### Inferences +- A FreeCAD-based importer that reads STEP colours and part names (FreeCAD's import is built on OCCT) and maps them to Blender objects and materials covers exactly the step where colour and structure are most often lost. Its documentation should state the tessellation deflection used and the unit handling (STEP mm vs Fusion cm), since nothing else standardizes them. +- Canonical orientation of mechanical parts (unlike ABO's "front" for products) is unsolved in these datasets. An "orient products" step (e.g. a PCA- or bounding-box-based canonical pose plus fixed view presets) is a differentiator, provided it is documented. + +### Gaps +- I found no systematic study quantifying STEP tessellation failure rates or colour-loss rates at dataset scale. +- I did not verify FreeCAD-specific colour import behaviour from a primary source in this research thread. +- I found no primary source on standard practice for orienting CAD parts in ML datasets, beyond unit-cube normalization. + +## 4. Existing tools for rendering CAD datasets + +### Takeaway +There is no widely adopted, CAD-native, standardized multi-view rendering tool for engineering datasets. Practice is fragmented. Dataset repos ship OpenCASCADE/pythonOCC viewers (DeepCAD, MFCAD), Fusion 360 API scripts, or ad hoc Blender scripts (Img2CAD). BlenderProc is the standard for photoreal industrial training data but takes meshes (PLY/OBJ), not STEP, and ignores CAD part semantics. Newer agentic CAD tools (build123d/CadQuery/FreeCAD MCP servers) render for LLM feedback, not for dataset generation. + +### Cited Findings +- DeepCAD visualizes "with opencascade" (`show.py`) and exports STEP via pythonocc-core. — [DeepCAD repo](https://github.com/ChrisWu1997/DeepCAD) +- MFCAD ships pythonocc-core scripts in `occ/` for exploring and visualizing the STEP models. — [MFCAD repo](https://github.com/hducg/MFCAD) +- Fusion 360 Gallery ships Fusion 360 API-based tools for reconstruction, B-rep traversal and format conversion. These need a Fusion 360 installation, and the repo names no dedicated renderer. — [Fusion360GalleryDataset](https://github.com/AutodeskAILab/Fusion360GalleryDataset) +- Img2CAD ships Blender renderings as data (`blender_renderings/`). — [Img2CAD HF dataset](https://huggingface.co/datasets/qq456cvb/img2cad-dataset) +- BlenderProc4BOP is the de facto photoreal synthetic-data renderer for the BOP industrial-object benchmarks, built on Blender Cycles. — [BlenderProc4BOP README](https://github.com/DLR-RM/BlenderProc/blob/main/README_BlenderProc4BOP.md) +- On the commercial side, NVIDIA Omniverse Replicator and Isaac Sim offer domain-randomized synthetic data, and MVTec describes CAD-generated training data for 3D matching in bin picking. — [MVPro Media](https://mvpromedia.com/from-cad-to-ai-how-synthetic-data-is-changing-machine-vision-training/) +- Onshape's API was the access route for ABC, AutoMate and DeepCAD (scraping public documents), but none of these datasets documents using Onshape-rendered images. — [AutoMate](https://arxiv.org/html/2105.12238v2); [DeepCAD](https://github.com/ChrisWu1997/DeepCAD) +- 2025-2026 agentic CAD tools listed in the awesome-ai4cad survey include build123d-mcp ("rendering; STEP, STL, SVG, DXF export"), agentcad, text-to-cad (STEP/GLB/STL/3MF), CadQuery MCP servers, FreeCAD MCP servers and the Onshape FeatureScript MCP Server. These render as feedback for LLM agents, not as standardized dataset image sets. — [awesome-ai4cad](https://github.com/gudo7208/awesome-ai4cad) +- A commercial Blender STEP import/export add-on (Stepio) exists, which shows STEP-in-Blender is a recognized need in the Blender ecosystem. — [Stepio docs](https://superhivemarket.com/products/stepio---blender-step-import--export-add-on/docs) + +### Inferences +- **Positioning:** The add-on would sit at the intersection that no existing tool covers. It reads STEP/IGES natively (via FreeCAD/OCCT), keeps part names and STEP colours, and outputs standardized, documented multi-view sets (fixed camera rigs, lighting and orientation) inside Blender. Blender then provides the path to photoreal and BlenderProc-style randomization when needed. +- **Concrete value propositions**, each tied to a gap found above: + 1. A reproducible, documented render protocol, which Text2CAD, cadrille and Omni-CAD lack. Ideally it would offer ABO-style metadata: camera intrinsics and extrinsics, lighting, resolution. + 2. Orthographic or low-distortion presets, addressing Text2CAD's perspective-distortion failures. + 3. Per-part renders, masks and name labels from STEP assembly structure, for part-aware VLM captioning (Text2CAD part renders, CADTalk semantics) and industrial assembly recognition (SIP-17's assembly drop). + 4. Colour fidelity from STEP, a step where common glTF pipelines reportedly lose colour. + 5. Explicit unit and scale handling plus canonical orientation, given the cm/mm/unit-cube inconsistency across datasets. +- **Likely competitive objections:** BlenderProc already exists for photoreal work, pythonOCC/OCP offscreen rendering is scriptable at scale without Blender, and large datasets (ABC: 1M models) need headless batch throughput. The add-on's case is strongest on semantic fidelity (names, colours, assemblies) and protocol standardization, not raw throughput. + +### Gaps +- I found no primary source describing an Onshape API "shaded views" rendering pipeline used for an ML dataset. +- I did not check whether any open-source pythonOCC or OCP batch multi-view renderer for ML datasets is widely used; none surfaced in the searches. +- I did not verify the throughput of Blender-based or FreeCAD-based conversion at ABC scale. diff --git a/research_notes/Multiview renderer literature gap/line_drawings.md b/research_notes/Multiview renderer literature gap/line_drawings.md new file mode 100644 index 0000000..d0227be --- /dev/null +++ b/research_notes/Multiview renderer literature gap/line_drawings.md @@ -0,0 +1,167 @@ +# Line drawings from 3D models: methods, datasets, and engineering-view (orthographic three-view) drawings + +Context: these notes are for placing the "MultiView Product Renderer" Blender add-on, which writes a "line art" pass for each view. That pass is silhouettes plus creases found in image space: it renders the model with a normal matcap and applies a crease-angle threshold, optionally over a clay render. The add-on also renders orthographic front, top and side views. + +Source reliability: items marked **[fetched]** were confirmed from the page or search snippet this session. Items marked **[background]** come from well-known primary sources that were not re-fetched this session. Their URLs are the canonical project pages, but the writer should spot-check them before quoting exact numbers. + +--- + +## Q1. Classic NPR line rendering: which line types exist, and which are practical in tools today? + +### Takeaway +The graphics literature splits lines into view-dependent types and view-independent types: +- **View-dependent:** occluding contours and silhouettes, suggestive contours, and apparent ridges. +- **View-independent:** creases, ridges and valleys, and material or part boundaries. + +Each type also needs a separate visibility (hidden-line) computation. In today's tools only contours, creases, boundaries, intersections and visibility are robust. Blender ships two object-space line engines that do this: Freestyle (raster render, with an SVG export add-on) and Grease Pencil Line Art (vector strokes, with selectable occlusion levels). Suggestive contours and apparent ridges remain mostly research code. The add-on's matcap-plus-threshold approach is an image-space approximation of "contour + crease" and has no visibility information. + +### Cited Findings +- Bénard and Hertzmann's tutorial "Line Drawings from 3D Models" (Foundations and Trends in Computer Graphics and Vision, vol. 11 no. 1–2, pp. 1–159, 2019; arXiv:1810.01175) is the standard reference. It covers the geometry of occluding contours on meshes and smooth surfaces, plus algorithms for extracting contours, computing their visibility, and stylized rendering. It compares exact methods with hardware-accelerated fast methods and surveys other line types. **[fetched]** — [arXiv](https://arxiv.org/abs/1810.01175); [NOW Publishers](https://nowpublishers.com/article/Details/CGV-075) +- **Suggestive contours** (DeCarlo, Finkelstein, Rusinkiewicz and Santella, SIGGRAPH 2003): these are lines where the surface would become a contour in a nearby viewpoint, that is, zeros of radial curvature where its directional derivative is positive. They are view-dependent and complement occluding contours. Reference code is the "rtsc" real-time suggestive contours library from Princeton. **[background]** — [Princeton project page](https://gfx.cs.princeton.edu/proj/sugcon/) +- **Apparent ridges** (Judd, Durand and Adelson, SIGGRAPH 2007): view-dependent ridges defined on the surface as projected to the screen. **[background]** — [MIT project page](https://people.csail.mit.edu/tjudd/apparentridges.html) +- **"Where Do People Draw Lines?"** (Cole et al., SIGGRAPH 2008): artists drew the same rendered 3D models, and the study then compared their lines with computed line types. It found that occluding contours plus a combination of other types (suggestive contours, ridges and valleys, apparent ridges) explain most of the artists' lines. This artist dataset later became the benchmark for learned line drawing. **[background]** — [Princeton project page](https://gfx.cs.princeton.edu/proj/ld3d/). Neural Contours confirms that the Cole et al. drawings are the standard evaluation set. **[fetched]** — [Neural Contours, CVPR 2020](https://openaccess.thecvf.com/content_CVPR_2020/html/Liu_Neural_Contours_Learning_to_Draw_Lines_From_3D_Shapes_CVPR_2020_paper.html) +- **Blender Grease Pencil Line Art modifier:** + - The 2.93 and 3.2 manuals list the edge types Contour (the separation line between front- and back-facing faces), Loose, Material Borders, Edge Marks, Intersections and Crease. + - Crease is set by a Crease Threshold: "Edge angles that are smaller than this value will be treated as crease." The current dev manual instead has "Crease on Smooth" and "Crease on Sharp" options. + - An Occlusion subpanel has a Level and a Range. Level 0 gives visible lines; Level 1 selects lines hidden by exactly one layer of faces. This makes it possible to extract hidden lines geometrically. + - The output is Grease Pencil vector strokes. + + **[fetched, via search snippets of the manual]** — [Blender manual (dev)](https://docs.blender.org/manual/en/dev/grease_pencil/modifiers/generate/line_art.html); [2.93 manual](https://docs.blender.org/manual/ja/2.93/grease_pencil/modifiers/generate/line_art.html); [3.2 manual](https://docs.blender.org/manual/fi/3.2/grease_pencil/modifiers/generate/line_art.html) +- **Freestyle SVG Exporter:** this Blender add-on exports Freestyle renders to SVG. It can fill contours with material colours and bundle an animation into one SVG. Per the manual, "Because the representation of Freestyle strokes and SVG path objects is fundamentally different, a one-on-one translation between Freestyle and SVG is not possible." **[fetched, snippets of 4.0/4.1 manual]** — [Blender 4.1 manual](https://docs.blender.org/manual/en/4.1/addons/render/render_freestyle_svg.html). The page returned 404 at `/manual/en/latest/addons/...`. This suggests that from 4.2 onward the add-on is no longer in the bundled-add-ons section of the manual (Blender moved most bundled add-ons to the Extensions platform in 4.2), but this was **not confirmed**. + +### Inferences +- The add-on's line pass recovers occluding contours (from silhouette or depth breaks) and creases (from normal discontinuities above an angle threshold). These are the same two line families that Line Art calls "Contour" and "Crease", and they are the two types the literature finds most important (Cole et al.). It does not produce suggestive contours or apparent ridges. Those matter mostly for smooth organic shapes and much less for CAD-like products. +- Line Art in Blender already gives geometric contour, crease, intersection, material-border and occlusion-level lines as vectors. A natural upgrade path is an optional Line Art-based pass, which adds visible and hidden separation and vector output. Freestyle with the SVG exporter is the other route. + +### Gaps +- I did not fetch the current (Blender 5.x) Line Art manual text, which was truncated by page size. The exact current edge-type list and whether "Light Contour" and "Shadow" types are present are unverified. +- The status of the Freestyle SVG exporter in Blender 4.2+ (extension vs removed) is not confirmed. + +--- + +## Q2. Datasets and ML that use rendered line drawings: how were the training drawings made? + +### Takeaway +Most sketch, line-drawing and CAD learning pipelines **synthesize their training drawings by rendering 3D or CAD data**, because paired human drawings are scarce. The generators fall into three groups: +1. **NPR line extractors on meshes**: contours, suggestive contours, ridges (for example Neural Contours candidates). +2. **CAD-aware stroke renderers**: Sketch2CAD, Free2CAD, CAD2Sketch, which draw feature curves plus construction lines from CAD sequences. +3. **B-rep hidden-line-removal (HLR) projectors**, usually pythonOCC/OpenCascade, that produce orthographic three-view engineering drawings: SPARE3D, Ortho2CAD, OrthoCAD-322K. + +Sketch-based retrieval benchmarks (SHREC'13/'14) use hand-drawn queries against 3D models. Retrieval methods then commonly render line or contour views of the 3D targets to bridge the domain gap. + +### Cited Findings +**Learned line drawing** +- **Neural Contours** (Liu, Nabail, Hertzmann, Kalogerakis, CVPR 2020) has two branches: a differentiable module that operates on geometric features of the 3D model (the classic line types) and an image-based module that operates on view-based shape representations. A neural module combines them at test time. + - Training labels came from **crowdsourced rankings** of candidate drawings by unskilled workers, not from artist drawings. The paper notes that in-person artist collection "does not scale well, and it also produces data without much stylistic consistency." + - Evaluation used the Cole et al. artist drawings plus a new, more diverse extended test set. + - Code and data: https://github.com/DifanLiu/NeuralContours + + **[fetched]** — [arXiv 2003.10333](https://arxiv.org/abs/2003.10333); [project page](https://people.cs.umass.edu/~dliu/projects/NeuralContours) + +**Sketch to CAD (synthetic drawings rendered from CAD)** +- **Sketch2CAD** (Li, Pan, Bousseau, Mitra; ACM TOG / SIGGRAPH Asia 2020): there were no paired sketch and CAD-sequence datasets, so the authors generated **10k procedural CAD protocols yielding about 40k training shapes**. They overlaid sketch curves on the built shapes and rendered them from viewpoints to produce input sketch and context maps with segmentation ground truth. It supports four CAD operations. **[fetched]** — [UCL project page](https://geometry.cs.ucl.ac.uk/projects/2020/sketch2cad); [arXiv 2009.04927](https://arxiv.org/pdf/2009.04927) +- **Free2CAD** (Li, Pan, Bousseau, Mitra; ACM TOG / SIGGRAPH 2022): a Transformer groups freehand strokes into CAD operations. It is **trained only on synthetic data** yet generalizes to sketches of real CAD models and from novice users. Code and data are at github.com/Enigma-li/Free2CAD. **[fetched]** — [UCL Discovery](https://discovery-pp.ucl.ac.uk/id/eprint/10159067); [paper PDF](https://geometry.cs.ucl.ac.uk/projects/2022/free2cad/paper_docs/Free2CAD_SIG_2022.pdf) +- **CAD2Sketch** (Hähnlein, Li, Mitra, Bousseau; ACM TOG 41(6), Art. 279, 2022; DOI 10.1145/3550454.3555488) converts CAD sequences into **concept sketches** in three steps: + 1. Generate construction lines for each CAD operation. + 2. Choose a subset of them by binary optimization, to avoid clutter. + 3. Adjust line opacity and shape so they look hand-drawn. + + It produces large numbers of paired sketches and normal maps for training sketch-to-normal networks. The synthetic sketches resemble real pairs in **OpenSketch**, a dataset of expert concept sketches. **[fetched, via the GDL-for-CAD survey and search snippets]** — [GDL for CAD survey, arXiv 2402.17695](https://arxiv.org/pdf/2402.17695); [UCL Discovery](https://discovery-pp.ucl.ac.uk/id/eprint/10190377) + +**Sketch-based 3D shape retrieval** +- **SHREC'13 Large Scale Sketch-Based 3D Shape Retrieval:** 7,200 hand-drawn sketches queried against 1,258 3D models in 80 classes. Per class, 80 sketches are split 50 for training and 30 for test. **[fetched]** — [Eurographics DL](https://diglib.eg.org/handle/10.2312/3DOR.3DOR13.089-096) +- **SHREC'14 Extended Large Scale SBR:** 13,680 sketches and 8,987 3D models in 171 classes, built on a unified multimodal benchmark. A CVIU 2014 comparison evaluated 15 methods. **[fetched]** — [SHREC'14 site](https://sites.usm.edu/bli/sharp/sharp/contest/2014/SBR/) + +**Three-view / engineering-drawing reconstruction (synthetic orthographic drawings rendered from CAD)** +- **PlankAssembly** (Hu et al., ICCV 2023) converts "2D line drawings from three orthographic views into 3D CAD models" using a Transformer that outputs shape programs. + - Its dataset has **26,707 shape programs derived from parametric CAD models** (furniture/plank models). + - The supplementary material reports separate results for noise-free inputs, noisy inputs, and "inputs with visible lines only". This implies the default input includes **hidden lines**, and that removing them is treated as a degradation. + - Classic back-projection methods are described as sensitive to input errors. + + **[fetched]** — [arXiv 2308.05744](https://arxiv.org/abs/2308.05744); [project page](https://manycore-research.github.io/PlankAssembly/); [ICCV open access](https://openaccess.thecvf.com/content/ICCV2023/html/Hu_PlankAssembly_Robust_3D_Reconstruction_from_Three_Orthographic_Views_with_Learnt_ICCV_2023_paper.html) +- **CAD2Program** ("From 2D CAD Drawings to 3D Parametric Models: A Vision-Language Approach", arXiv 2412.11892): 368K cabinet models with real production engineering drawings from an interior-design platform (364K / 2K / 2K split). + - The drawings contain multiple orthographic views, sometimes section views, and views that may not be aligned, plus an annotation layer with dimensions and symbols. + - It treats drawings as raster images, encoded with a ViT. + - It states that classical methods "all assume that a clean geometry layer with exactly three axis-aligned views" is given, and that PlankAssembly requires exactly three axis-aligned views **in vector format**. + - A raster-encoder variant of PlankAssembly performed comparably to the vector original. + + **[fetched]** — [arXiv HTML](https://arxiv.org/html/2412.11892v2) +- **Ortho2CAD** (Joglekar, Regmi, Shimada, Kara; arXiv 2607.08891, July 2026): a **pythonOCC-based generator** renders **first-angle orthographic projections from STEP models with dashed hidden lines and bounding-box dimensions**. It produced over 1 million drawings, plus 100 drawings with manually dimensioned features. The authors name the "unavailability of large scale orthographic drawing datasets" as their core motivation. **[fetched, abstract only]** — [arXiv 2607.08891](https://arxiv.org/abs/2607.08891) +- **CReFT-CAD / TriView2CAD** (arXiv 2506.00568, 2025) is a benchmark for orthographic-projection reasoning. It argues that existing benchmarks provide **only rasterized drawings without precise dimension annotations**. **[fetched, search snippet]** — [arXiv 2506.00568](https://arxiv.org/html/2506.00568v1) +- **OrthoCAD-322K** (IIT Madras group) provides standardized orthographic and isometric views of about 322K public CAD models, in **DXF and raster** formats, for 3D reconstruction and cross-modal retrieval. **[fetched, search snippet only; PDF too large to fetch]** — [PDF](https://ed.iitm.ac.in/~raman/agcl/OrthoCAD_322K_3DoR.pdf) +- **Zenodo: "Automatic 3D CAD models reconstruction from 2D orthographic drawings"** (Zhang, Pinquié, Polette, Carasi, De Charnace, Pernot; Computers & Graphics 2023) pairs **SVG** 2D drawings with STEP models, based on the Fusion 360 Gallery, plus ABC test cases. It is licensed CC BY 4.0. **[fetched]** — [Zenodo 7785223](https://zenodo.org/records/7785223) +- **SPARE3D** (Han et al., CVPR 2020) is a spatial-reasoning benchmark built from three-view and isometric line drawings of CAD models. As I recall, they are generated with pythonOCC HLR and include hidden lines. **[background — not verified this session]** + +### Inferences +- The field's demand for line drawings rendered from 3D is **very high**: nearly every sketch-to-CAD or drawing-to-CAD paper writes its own renderer. The generators differ by community: + - Graphics groups write NPR/feature-curve renderers. + - CAD and engineering groups use OpenCascade HLR to get exact visible and hidden edges, often dashed and in vector form. +- No standard Blender-based generator appears in this literature. A Blender add-on producing consistent orthographic three-view line drawings would be a convenience tool, not a new research contribution, unless it matched engineering conventions: exact HLR, dashed hidden lines, vector output, and consistent scale. + +### Gaps +- I could not confirm from primary text exactly how PlankAssembly renders its input lines (vector segments tokenized from the B-rep, with or without dash styling); the CVF PDF returned 403. +- ProSketch, Img2CAD, OpenSketch-specific rendering details, and SketchGraphs were not researched this session. SketchGraphs is a dataset of 2D constraint sketches from Onshape, not rendered views, so it is likely out of scope. +- The SHREC sketch-track papers' own target-view rendering methods were not fetched. + +--- + +## Q3. Is there evidence that standardized three-view drawings with consistent scale help orthographic-view-to-CAD reconstruction? + +### Takeaway +Yes, in two ways: +- **Direct:** classic and current methods (PlankAssembly) *require* exactly three axis-aligned views. Recent works build datasets with dimensions because size cues measurably help. +- **Indirect:** several 2025–2026 papers explicitly call large standardized orthographic drawing datasets missing. + +Recent work also pushes **away** from strict standardization (raster input, unaligned views, annotations), so that methods are robust to real drawings. + +### Cited Findings +- Classical three-view reconstruction methods "all assume that a clean geometry layer with exactly three axis-aligned views" is provided. PlankAssembly requires exactly three axis-aligned views in vector format. **[fetched]** — [CAD2Program, arXiv 2412.11892](https://arxiv.org/html/2412.11892v2) +- Adding the dimension/annotation layer raised CAD2Program's F1 from 62.65 to 82.76. The authors attribute this to dimensions giving direct size references, which vector pipelines usually discard as noise. This is evidence that **scale information is highly valuable**. **[fetched]** — [CAD2Program](https://arxiv.org/html/2412.11892v2) +- Ortho2CAD cites the "unavailability of large scale orthographic drawing datasets" and generates first-angle projections with dashed hidden lines and bounding-box dimensions. **[fetched]** — [arXiv 2607.08891](https://arxiv.org/abs/2607.08891) +- TriView2CAD/CReFT-CAD says existing benchmarks have only raster drawings without precise dimension annotations. **[fetched, snippet]** — [arXiv 2506.00568](https://arxiv.org/html/2506.00568v1) +- PlankAssembly argues that classic back-projection is brittle to noisy or incomplete input and evaluates robustness with noisy and visible-only inputs. Hidden lines carry information that visible-only drawings lose. **[fetched]** — [arXiv 2308.05744](https://arxiv.org/abs/2308.05744); [project page](https://manycore-research.github.io/PlankAssembly/) + +### Inferences +- What helps downstream reconstruction is (a) **known view convention** (first or third angle; which view is front, top or side), (b) **consistent known scale** (the same pixels-per-unit in every view, or dimensions, or a bounding box), and (c) **hidden-line information**. The add-on's orthographic front, top and side views already give (a). If they share a single ortho scale and log it in metadata (e.g. ortho_scale and pixels-per-mm), they give (b). They lack (c). +- For product photography and e-commerce, consistent ortho views with known scale are useful as a by-product of the add-on. Positioning it as a drawing-to-CAD data generator would require hidden lines and vector output to match what Ortho2CAD and PlankAssembly-style pipelines consume. + +### Gaps +- I found no controlled study that isolates "consistent scale across views" (versus per-view auto-fit) as a factor in reconstruction accuracy. The CAD2Program annotation ablation is the closest evidence. +- No user-study evidence was found on demand from practitioners, as opposed to ML researchers, for auto-generated three-view drawings from meshes. + +--- + +## Q4. Image-space edge detection vs geometric line extraction: what is needed for engineering-drawing-like output? + +### Takeaway +Image-space detection (normal or depth discontinuities in a rendered buffer, which is what the add-on does) is fast and robust for **visible** silhouettes and creases. It has structural limits: +- no hidden lines; +- resolution-dependent, aliased raster lines; +- thresholds that depend on the view; +- no line identity or semantics for line weights; +- no clean vector output. + +Engineering drawings need object-space extraction with **exact visibility (HLR)**, typed lines (visible solid, hidden dashed, optionally center and phantom lines), line weights, and **vector output (SVG/DXF)** at a known scale. + +### Cited Findings +- The Bénard and Hertzmann tutorial frames contour drawing as two separate problems: **extraction** and **visibility**. It contrasts exact object-space methods with hardware-accelerated (image-space) fast methods and discusses their trade-offs. **[fetched, abstract]** — [arXiv 1810.01175](https://arxiv.org/abs/1810.01175) +- Blender Line Art computes occlusion levels per line segment. Level 0 gives visible lines and Level 1 gives lines hidden by exactly one layer of faces, with a range option. This is the geometric capability needed to draw hidden lines, for example on a separate dashed Grease Pencil layer. **[fetched, manual snippets]** — [Blender manual](https://docs.blender.org/manual/en/dev/grease_pencil/modifiers/generate/line_art.html) +- Freestyle can export SVG, but the stroke-to-path translation is lossy: "a one-on-one translation between Freestyle and SVG is not possible." **[fetched]** — [Blender 4.1 manual](https://docs.blender.org/manual/en/4.1/addons/render/render_freestyle_svg.html) +- CAD-side dataset generators produce engineering conventions directly from B-reps with OpenCascade/pythonOCC: first-angle projection, dashed hidden lines, dimensions. **[fetched]** — [Ortho2CAD](https://arxiv.org/abs/2607.08891). Other datasets ship vector formats: DXF in OrthoCAD-322K **[snippet]** ([PDF](https://ed.iitm.ac.in/~raman/agcl/OrthoCAD_322K_3DoR.pdf)) and SVG in the Zhang et al. Fusion 360-based set **[fetched]** ([Zenodo](https://zenodo.org/records/7785223)). +- Learned approaches (Neural Contours) combine geometric line features with image-based features. Purely image-based cues alone were not considered sufficient for high-quality drawings. **[fetched]** — [Neural Contours](https://arxiv.org/abs/2003.10333) + +### Inferences +Requirements for an "engineering-drawing mode" in the add-on, ordered by value: +1. **Visibility-aware geometric lines.** Use Blender Line Art (Contour + Crease + Intersection + Material Border) with occlusion level 0 for visible lines and level 1..N for hidden lines, on two Grease Pencil layers styled solid and thick vs dashed and thin. Alternatively use Freestyle with QI (quantitative invisibility) line sets. This replaces or augments the matcap-threshold pass. +2. **Vector export.** SVG per view, or DXF. Use the Grease Pencil SVG export operator or the Freestyle SVG exporter, keeping visible and hidden lines in separate groups or layers with stroke-dasharray for hidden lines. +3. **A shared, recorded scale.** Use the same orthographic scale for front, top and side. Write a sidecar JSON with mm-per-pixel or mm-per-SVG-unit, the view convention (first vs third angle) and the bounding-box dimensions. This matches what Ortho2CAD and CAD2Program show is useful. +4. **Line weights:** thicker for silhouettes and outlines, thinner for creases and hidden lines, following drafting convention. +5. **Optional extras:** center lines for cylindrical features, and a crease-angle setting in degrees that matches Line Art's crease threshold. + +Limitations to state honestly: mesh inputs (STL/OBJ product meshes) lack B-rep feature edges, so crease detection depends on the tessellation and an angle threshold. Smooth fillets produce no crease, or spurious ones, unlike exact CAD HLR. Pixel-based outputs remain fine for ML training data that is consumed as rasters (CAD2Program shows that raster encoders can match vector input). + +### Gaps +- Not verified this session: + - whether Blender's current Grease Pencil v3 (4.3+/5.x) SVG export preserves layers and dash styles; + - whether Line Art occlusion works identically with orthographic cameras. +- No primary source was found that quantifies how accurate image-space crease detection is compared with geometric extraction on CAD-like meshes. diff --git a/research_notes/Multiview renderer literature gap/multiview_dataset_rendering.md b/research_notes/Multiview renderer literature gap/multiview_dataset_rendering.md new file mode 100644 index 0000000..2d55461 --- /dev/null +++ b/research_notes/Multiview renderer literature gap/multiview_dataset_rendering.md @@ -0,0 +1,166 @@ +# Multi-view rendering protocols in CV and 3D-generation datasets (positioning for MultiView Product Renderer) + +Verification note: items marked **[fetched]** were confirmed against the primary source during this session (Oct 2026). Items marked **[from paper, not re-fetched]** come from the cited primary paper or repo but could not be re-opened this session because PDF fetches timed out. Re-check those numbers before quoting them in a publication. + +## Q1. Multi-view CNN era (MVCNN, ModelNet/ShapeNet conventions, RotationNet, Render for CNN) + +### Takeaway +Classification-era work used a small set of fixed, hand-picked rigs: 12 views on a 30-degree-elevated ring (which assumes an upright object), or 20/80 views from polyhedron vertices (no upright assumption). Shading was simple and untextured (Phong). "Render for CNN" introduced randomized, Blender-based synthetic rendering with random lighting and backgrounds. Each paper shipped its own scripts, and none produced a reusable rig standard. + +### Cited Findings +- MVCNN (Su et al., ICCV 2015), 12-view setup: assumes upright meshes. 12 virtual cameras are placed every 30 degrees around the vertical axis, elevated 30 degrees from the ground plane and pointing at the centroid. Meshes are rendered with the Phong reflection model, with no colour or texture, after being uniformly scaled to fit the viewing volume. [from paper, not re-fetched] — [MVCNN arXiv 1505.00880](https://arxiv.org/abs/1505.00880) +- MVCNN, 80-view setup: no upright assumption. 20 cameras sit at the vertices of an enclosing polyhedron (the paper says "icosahedron", but an icosahedron has only 12 vertices; 20 vertices is a dodecahedron), and each camera renders 4 in-plane rotations (0/90/180/270 degrees), giving 80 views. [from paper, not re-fetched] — [MVCNN arXiv 1505.00880](https://arxiv.org/abs/1505.00880) +- RotationNet (Kanezaki et al., CVPR 2018) uses two rigs: (i) 12 views around an upright object, like MVCNN, and (ii) 20 views from the vertices of a dodecahedron. It treats viewpoint labels as latent variables. Results are reported on ModelNet10/40. [from paper, not re-fetched] — [RotationNet arXiv 1603.06208](https://arxiv.org/abs/1603.06208) +- Render for CNN (Su et al., ICCV 2015) renders ShapeNet models in Blender with randomized viewpoint, lighting, truncation and backgrounds (SUN397 images), to train viewpoint estimators. It synthesized about 2.4M images. The code is a research repo (ShapeNet/RenderForCNN). [from paper, not re-fetched] — [Render for CNN arXiv 1505.05641](https://arxiv.org/abs/1505.05641); [GitHub ShapeNet/RenderForCNN](https://github.com/ShapeNet/RenderForCNN) + +### Inferences +- The 12-view, 30-degree-elevation ring became a de facto convention for upright objects. The add-on's canonical six orthographic views are a different (engineering-drawing) convention, so offering an "MVCNN 12-view" preset would let users reproduce benchmarks directly. +- Normalization in this era was "scale to fit the view volume" with untextured shading. The add-on's clay pass is the closest analogue. + +### Gaps +- MVCNN image resolution and exact light placement could not be re-verified this session. + +## Q2. 3D-generation era: rigs, normalization, lighting, passes and tools + +### Takeaway +Each major work defines its own rig, and no two are the same. +- Rigs vary from uniform random-sphere sampling (Zero-1-to-3, LRM, TRELLIS) to fixed rings: 16 views at 30 degrees (SyncDreamer), 6 views alternating +20/-10 degrees (Zero123++ v1.2), 4 orthogonal views (MVDream, Instant3D, cadrille), 6 orthographic views (Wonder3D), and 38 views across rings plus top/bottom (G-Objaverse). +- Normalization ranges from [-0.5, 0.5]^3 to [-1, 1]^3. +- Field of view is variously fixed at 30, 40 or 49.1 degrees, or random between 15 and 60. +- Lighting is random area lights, random HDRI, or path-traced. +- Almost all use Blender (raw bpy scripts or BlenderProc), but each project uses its own script. + +### Cited Findings + +**Objaverse / Objaverse-XL (data source plus reference renderer)** +- The Objaverse-XL rendering script is pinned to Blender 3.2.2. It renders 12 views per object by default (`num_renders=12`) at randomized camera locations, with an option such as `--only_northern_hemisphere`. Outputs are PNG images, a 3x4 camera matrix per view as `.npy`, and a `metadata.json` (bbox, mesh/poly counts, lamp_count, sha256). Each object's output is zipped. It runs via `python3 main.py` and needs Xorg on headless servers. [fetched] — [objaverse-xl scripts/rendering README](https://github.com/allenai/objaverse-xl/tree/main/scripts/rendering) + +**Zero-1-to-3 (Liu et al., ICCV 2023)** +- 12 views per object, with cameras sampled uniformly on a sphere and pointing at the object centre. Radius is drawn uniformly from [1.5, 2.2], horizontal FoV is 49.1 degrees, and the camera is a pinhole. Assets are normalized to the unit cube [-0.5, 0.5]^3. Renders are 512x512 (training used 256x256) on a white background, with "randomized area lighting", in Blender Cycles (128 samples plus denoising). About 10M images in total. The rendering code is inherited from the Objaverse rendering repo. [fetched] — [Zero-1-to-3 arXiv 2303.11328](https://arxiv.org/abs/2303.11328) + +**Zero123++ (Shi et al., 2023)** +- Six views tiled in a 3x2 grid, using absolute elevations and azimuths relative to the input view. Trained on Objaverse with "random HDRI environment lighting". [fetched] — [Zero123++ arXiv 2310.15110](https://arxiv.org/abs/2310.15110) +- v1.1 uses azimuths 30/90/150/210/270/330 and elevations alternating 30/-20 degrees. v1.2 uses the same azimuths, elevations alternating 20/-10 degrees, and a fixed output FoV of 30 degrees. v1.2 "always outputs views assuming a normalized object size" and is more robust to input FoV. Outputs have a gray background (the zero value of the SD VAE). [fetched] — [Zero123++ README](https://github.com/SUDO-AI-3D/zero123plus) +- The paper and README disagree on v1.1 elevations: the paper says 30 degrees down and 20 degrees up, the README says 30/-20. The protocol changed between v1.1 and v1.2 within a single project. [fetched] — [paper](https://arxiv.org/abs/2310.15110) vs [README](https://github.com/SUDO-AI-3D/zero123plus) + +**SyncDreamer (Liu et al., ICLR 2024)** +- 16 fixed target views at 30 degrees elevation, with azimuths spread evenly over 0-360 degrees (22.5-degree steps). Each object also gets 16 extra input views at the same azimuths but random elevations. Objects are placed at the origin and normalized inside a cube of side 1. Uses about 800k Objaverse objects, at 256x256 latent-compatible resolution. The paper does not specify the engine, lighting, distance or FoV. Evaluation uses 30 randomly chosen GSO objects. [fetched] — [SyncDreamer arXiv 2309.03453](https://arxiv.org/abs/2309.03453) + +**Wonder3D (Long et al., CVPR 2024)** +- Six views (front, front-right, right, back, left, front-left), with RGB and normal maps generated jointly. Trained on the Objaverse LVIS subset (about 30k+ objects after cleanup), rendered with BlenderProc, with random rotations applied to assets. Objects are centred at unit scale, training is at 256x256, and evaluation uses the same 30 GSO objects as SyncDreamer. [fetched] — [Wonder3D arXiv 2310.15008](https://arxiv.org/abs/2310.15008) +- The repo's `render_codes` provides `blenderProc_ortho.py` (orthographic, `--ortho_scale 1.35`, resolution 512) and `blenderProc_persp.py` (perspective, `--radius 2.0`), plus batch shell scripts. BlenderProc uses Cycles, which the README says can hang on some GPUs (A800). [fetched] — [Wonder3D render_codes README](https://github.com/xxlong0/Wonder3D/tree/main/render_codes) + +**MVDream (Shi et al., ICLR 2024)** +- About 350K Objaverse objects (CLIP-filtered). Each is rendered at 32 uniformly spaced azimuths, twice with different random settings. Objects are centred and fit in [-0.5, 0.5]. FoV is random in [15, 60] degrees and elevation random in [0, 30] degrees. Camera distance is 0.5 times the NDC focal length times a random factor in [0.9, 1.1], which keeps the apparent size constant across FoVs. Lighting is a random Blender HDRI. Renders are saved as RGBA, and the background is filled with random grayscale during training. Training samples 4 orthogonal views out of the 32, at 256x256. [fetched] — [MVDream arXiv 2308.16512](https://arxiv.org/abs/2308.16512) + +**Instant3D (Li et al., ICLR 2024)** +- Generates 4 views in a 2x2 grid at fixed elevation 20 degrees and azimuths 0/90/180/270, then reconstructs with a feed-forward LRM-style model. [from paper, not re-fetched] — [Instant3D arXiv 2311.06214](https://arxiv.org/abs/2311.06214) + +**LRM (Hong et al., ICLR 2024)** +- 730,648 Objaverse assets normalized to [-1, 1]^3. Each gets 32 random views with the camera pointing at the shape, ball radius in [1.5, 3.0] and height in [-0.75, 1.60]. Renders are 1024x1024 on a pure white background. Training also uses 220,219 MVImgNet videos, cropped and resized using predicted masks with background removed (Rembg). Poses are normalized so the input camera sits at [0, -2, 0] with z up, and intrinsics are normalized by image size. The paper does not state lighting or the render tool. Evaluation uses 100 random GSO objects with 20 reference views each. [fetched] — [LRM arXiv 2311.04400](https://arxiv.org/abs/2311.04400) + +**TripoSR (Tochilkin et al., 2024)** +- An LRM-derivative trained on a curated Objaverse subset. The report says its rendering techniques were changed to better match the distribution of real images. [from paper, not re-fetched; exact rig not verified] — [TripoSR arXiv 2403.02151](https://arxiv.org/abs/2403.02151) + +**TRELLIS (Microsoft, 2024–25)** +- `dataset_toolkits/render.py` defaults to 150 views per object, placed on a Hammersley sequence over the sphere with a random per-object offset. Radius is fixed at 2, FoV fixed at 40 degrees, and resolution 512x512. Uses Blender 3.0.1 with Cycles. A `transforms.json` per object marks success, and Blender stdout/stderr go to DEVNULL, so render failures are silent. [fetched] — [TRELLIS render.py](https://github.com/microsoft/TRELLIS/blob/main/dataset_toolkits/render.py) + +**G-Objaverse / Gobjaverse (Alibaba, used by RichDreamer)** +- 38 views per object: a 24-view upper ring (elevation 5–30 degrees, 15-degree azimuth steps), a 12-view lower ring (elevation -5 to 5 degrees, 30-degree steps), plus top and bottom views. Rendered with Alibaba's TIDE renderer, a hybrid that rasterizes the first hit and path-traces indirect lighting, with adaptive sampling, denoising and path guiding. Passes are RGB, albedo, normal and depth. About 30M images, costing about 2,000 A10 GPU hours. [fetched] — [G-Objaverse project page](https://aigc3d.github.io/gobjaverse/) +- The release covers about 280K objects (about 6.5 TB). Per-view files are RGB PNG, `_albedo.png`, `_hdr.exr`, `_mr.png` (metal/roughness) and `_nd.exr` (normal and depth), plus a per-view JSON of camera info. The renderer is described as Unity-based, with poses converted to Blender's world-to-camera convention and normals in a "Normal-Bae" camera frame. [fetched] — [RichDreamer gobjaverse README](https://github.com/modelscope/richdreamer/tree/main/dataset/gobjaverse) +- The project page and the repo disagree on the renderer. The page names TIDE (a hybrid rasterization/path-tracing engine), while the repo README calls it "Unity-based". [fetched] — [project page](https://aigc3d.github.io/gobjaverse/) vs [README](https://github.com/modelscope/richdreamer/tree/main/dataset/gobjaverse) + +**Scanned and real-capture datasets (for contrast)** +- Google Scanned Objects: 1,030 high-quality scanned household items (CC-BY 4.0). Papers use it mainly for evaluation renders. [from paper, not re-fetched] — [GSO arXiv 2204.11918](https://arxiv.org/abs/2204.11918) +- OmniObject3D: about 6,000 scanned objects in 190 categories. Provides Blender multi-view renders, point clouds and real captured videos with COLMAP poses. [from paper, not re-fetched; per-object view count not verified] — [OmniObject3D arXiv 2301.07525](https://arxiv.org/abs/2301.07525) +- CO3D: about 19k turntable-style phone videos (about 1.5M frames) across 50 MS-COCO categories, with poses and point clouds from COLMAP. [from paper, not re-fetched] — [CO3D arXiv 2109.00512](https://arxiv.org/abs/2109.00512) +- MVImgNet: about 6.5M frames from about 219k videos across 238 classes, with COLMAP-annotated poses. [from paper, not re-fetched] — [MVImgNet arXiv 2303.06042](https://arxiv.org/abs/2303.06042). LRM reports pre-processing 220,219 MVImgNet videos ([fetched](https://arxiv.org/abs/2311.04400)), which differs slightly from the count above. + +### Inferences +- **Normalization conflicts.** Works use [-0.5, 0.5]^3 (Zero-1-to-3, MVDream), a cube of side 1 (SyncDreamer), [-1, 1]^3 (LRM) and "unit scale" (Wonder3D). A cube of side 1 and [-0.5, 0.5]^3 are the same box, so the real outlier is LRM's [-1, 1]^3, which is twice as large. Camera radius and FoV pairs also differ: r in [1.5, 2.2] at 49.1 degrees, r = 2 at 40 degrees, fixed 30-degree FoV, and FoV in [15, 60] with distance compensation. So "normalize to a target size at the origin" is universal in spirit but parameterized differently everywhere. A renderer that exposes these as named presets ("Zero123 / MVDream / LRM / TRELLIS / SyncDreamer / Zero123++ v1.2 / G-Objaverse") would be directly useful. +- **Projection.** Orthographic rigs appear only in Wonder3D (an orthographic BlenderProc script) and the classic six-view CAD convention. Most generation models train on perspective views. The add-on's shared orthographic six-view rig matches Wonder3D-style and engineering conventions, and its perspective views cover the rest. +- **Passes.** Normals and depth are standard in the newer datasets (G-Objaverse normal/depth/albedo; Wonder3D normals; Zero123++ depth ControlNet trained on "normalized linear depth"). Every dataset exports per-view camera matrices. The add-on's lack of depth, normals and pose export is therefore the biggest functional gap against 3D-generation use. +- **Lighting.** Generation datasets randomize lighting (random area lights, random HDRI) for augmentation, while the add-on uses consistent studio light that follows the camera. That suits product and catalogue consistency, but an ML user would want an optional randomized or HDRI mode. + +### Gaps +- Instant3D's exact FoV, background and lighting, TripoSR's rig, OmniObject3D's per-object view count, and Wonder3D's normal frame (world vs. input-camera) were not verified this session. +- G-Objaverse image resolution: a folder name ("campos_512_v4") suggests 512, but no source states it. + +## Q3. Are the scripts reusable tools or ad-hoc? Known inconsistencies and reproducibility problems + +### Takeaway +Reusable, generic tooling exists: BlenderProc, Kubric and the Objaverse-XL scripts. But each 3D-generation paper ships its own Blender script, tied to a specific Blender version, rig, normalization and lighting. The protocols differ enough that models need adapters (elevation estimators, FoV normalization), and benchmark numbers on GSO are not comparable across papers. + +### Cited Findings +- Zero-1-to-3 reused the Objaverse rendering repo's code. [fetched] — [Zero-1-to-3](https://arxiv.org/abs/2303.11328) +- Wonder3D used BlenderProc (cited as BlenderProc2) with its own ortho/persp wrapper scripts. [fetched] — [Wonder3D](https://arxiv.org/abs/2310.15008); [render_codes](https://github.com/xxlong0/Wonder3D/tree/main/render_codes) +- BlenderProc is a general, procedural Blender pipeline for photorealistic synthetic data. [from paper, not re-fetched] — [BlenderProc arXiv 1911.01911](https://arxiv.org/abs/1911.01911); [GitHub DLR-RM/BlenderProc](https://github.com/DLR-RM/BlenderProc) +- Kubric is a scalable data generator built on Blender and PyBullet. Its examples include GSO-based multi-object scenes. [from paper, not re-fetched] — [Kubric arXiv 2203.03570](https://arxiv.org/abs/2203.03570) +- Pinned Blender versions differ across projects: 3.2.2 for Objaverse-XL and 3.0.1 for TRELLIS. TRELLIS also discards Blender's output, so failures are silent. [fetched] — [objaverse-xl](https://github.com/allenai/objaverse-xl/tree/main/scripts/rendering); [TRELLIS render.py](https://github.com/microsoft/TRELLIS/blob/main/dataset_toolkits/render.py) +- **Pose-convention problem.** Zero-1-to-3 uses poses relative to the input view, so it needs the input's elevation. Downstream pipelines therefore add an elevation estimator that "introduces extra error". Zero123++ switched to absolute elevations with relative azimuths. [fetched] — [Zero123++](https://arxiv.org/abs/2310.15110) +- **Field-of-view and scale problem.** Zero123++ v1.2 had to "unify" the output FoV to 30 degrees and assume a normalized object size, because inputs come with varying FoVs and croppings. [fetched] — [Zero123++ README](https://github.com/SUDO-AI-3D/zero123plus) +- **Background inconsistency.** Backgrounds are white in Zero-1-to-3 and LRM, gray (the VAE zero value) in Zero123++, and random grayscale in MVDream. [fetched] — [Zero-1-to-3](https://arxiv.org/abs/2303.11328); [LRM](https://arxiv.org/abs/2311.04400); [Zero123++ README](https://github.com/SUDO-AI-3D/zero123plus); [MVDream](https://arxiv.org/abs/2308.16512) +- **Evaluation inconsistency.** SyncDreamer and Wonder3D evaluate on the same 30 GSO objects, and SyncDreamer does not specify how they were rendered. LRM uses 100 random GSO objects with 20 reference views each. [fetched] — [SyncDreamer](https://arxiv.org/abs/2309.03453); [Wonder3D](https://arxiv.org/abs/2310.15008); [LRM](https://arxiv.org/abs/2311.04400) +- **Same project, conflicting descriptions.** G-Objaverse's renderer is described as TIDE on the project page and as Unity-based in the repo README. Zero123++ v1.1 elevations differ between the paper and the README. [fetched] — sources as in Q2. + +### Inferences +- The gap is a standardized, documented, parameterized rig with an exported manifest that records every protocol parameter (normalization box, radius, FoV, elevations, azimuths, lighting, background, Blender version). It is not a missing renderer, since Blender and BlenderProc are already ubiquitous. The add-on's JSON/CSV manifest is a strong base if it records these fields. +- Shared framing for all objects (one camera rig and one scale) is exactly what fixed-ring datasets (SyncDreamer, Zero123++, G-Objaverse) rely on. The add-on's "one framing shared across all objects" maps well to these. +- No reviewed work uses a no-code GUI. All are command-line batch scripts aimed at ML engineers, so a GUI add-on is differentiated for non-ML users (product, CAD and industrial teams), but it must also run headless and in batch to be credible for dataset building. + +### Gaps +- I found no systematic, published study that quantifies how much rendering-protocol differences (normalization, lighting, FoV) change benchmark results. The evidence above is anecdotal: per-paper fixes and adapters. + +## Q4. Standard camera and pose export formats + +### Takeaway +Two families dominate: +- **NeRF-style `transforms.json`:** camera-to-world 4x4 matrices in the OpenGL/Blender convention, with `camera_angle_x` or explicit `fl_x`/`fl_y`/`cx`/`cy`/`w`/`h` and optional depth and mask paths. Used by NeRF, Instant-NGP, nerfstudio, and TRELLIS (which writes `transforms.json`). +- **COLMAP text or binary:** world-to-camera quaternion plus translation in the OpenCV convention. + +Research datasets also use ad-hoc numpy matrices (Objaverse-XL 3x4 `.npy`) or per-view JSON (G-Objaverse). + +### Cited Findings +- **nerfstudio convention.** Uses the OpenGL/Blender/original-NeRF convention: +X right, +Y up, +Z back, so the camera looks down -Z. COLMAP/OpenCV flips Y and Z. + - `transforms.json` may carry global intrinsics: `camera_model` (OPENCV or OPENCV_FISHEYE), `fl_x`, `fl_y`, `cx`, `cy`, `w`, `h`, and distortion `k1`–`k4`, `p1`, `p2`. + - Each frame has a 4x4 camera-to-world `transform_matrix`, a `file_path`, and optional `depth_file_path` (16/32-bit, millimetres, 0 = unknown) and `mask_path`. + + [fetched] — [nerfstudio data conventions](https://docs.nerf.studio/quickstart/data_conventions.html) +- **Original NeRF synthetic "Blender" scenes.** Use `transforms_{train,val,test}.json` holding `camera_angle_x` and per-frame `transform_matrix`, rendered from Blender. [from repo, not re-fetched] — [NeRF GitHub bmild/nerf](https://github.com/bmild/nerf) +- **Instant-NGP.** Its `transforms.json` extends the NeRF format with fields such as explicit intrinsics and `aabb_scale`. [from repo, not re-fetched] — [instant-ngp nerf dataset tips](https://github.com/NVlabs/instant-ngp/blob/master/docs/nerf_dataset_tips.md) +- **COLMAP.** The text format stores `cameras.txt` (intrinsics per camera model) and `images.txt` (per image: QW QX QY QZ TX TY TZ, a world-to-camera rotation and translation). [from docs, not re-fetched] — [COLMAP output format](https://colmap.github.io/format.html) +- **Objaverse-XL.** Writes one 3x4 camera matrix per view as `.npy`. [fetched] — [objaverse-xl rendering](https://github.com/allenai/objaverse-xl/tree/main/scripts/rendering) +- **G-Objaverse.** Writes per-view JSON camera info in Blender's world-to-camera convention, with normal/depth in EXR. [fetched] — [gobjaverse README](https://github.com/modelscope/richdreamer/tree/main/dataset/gobjaverse) +- **BlenderNeRF.** An existing open-source Blender add-on that exports NeRF-compatible camera data (`transforms.json`) from Blender scenes. It is the closest GUI competitor for pose export, but it targets NeRF capture, not normalized object-collection rendering. [from repo, not re-fetched] — [GitHub maximeraafat/BlenderNeRF](https://github.com/maximeraafat/BlenderNeRF) + +### Inferences +- The minimum viable pose export for the add-on is a nerfstudio/Instant-NGP-compatible `transforms.json` per object, written in Blender's own OpenGL convention so no axis flip is needed. Add optional `depth_file_path` and `mask_path` (the add-on already produces masks), and optionally a COLMAP text export. Also store the normalization transform (scale and offset) in the manifest, so poses map back to original CAD units. + +### Gaps +- No single "dataset rendering manifest" standard was found. Each project's metadata schema is bespoke. + +## Q5. Work that targets CAD or engineering parts specifically + +### Takeaway +CAD-specific work (image-to-CAD, CAD reconstruction) renders B-rep models tessellated to meshes, from a few fixed views. Examples are 4 views in a 2x2 grid in cadrille, and a single isometric Blender view per shape in a CADFit benchmark. These are ad-hoc and minimally documented: projection, lighting and tool are often unstated. None of the 3D-generation datasets above target engineering parts. This is a plausible niche for a CAD-aware, documented multi-view renderer with canonical orthographic views. + +### Cited Findings +- **cadrille (ICLR 2026).** Multi-modal CAD reconstruction from point clouds, multi-view images or text. Images are "rendered with fixed camera positions, and concatenated into 2x2 grid" (4 views tiled into one 268x268 image). The B-rep is tessellated to a mesh and then rendered. + - Projection, lighting and tool are not specified. Ground truth is normalized to [-0.5, 0.5]^3 for evaluation. + - Data includes DeepCAD (Text2CAD version, about 160k train / 8,046 test), CAD-Recode (about 1M procedural CadQuery programs), Fusion360 (1,725 test) and CC3D scans. + - It also evaluates on Omni-CAD (CAD-MLLM's dataset), whose rendering protocol cadrille does not describe. + + [fetched] — [cadrille arXiv 2505.22914](https://arxiv.org/html/2505.22914v3) +- **CADFit (2026).** Builds an ABC image-to-CAD benchmark by sampling 150 shapes per complexity window and rendering one isometric Blender view per shape. [search-result summary, not opened] — [CADFit arXiv 2605.01171](https://arxiv.org/pdf/2605.01171) +- **CADBench (2026).** Reportedly a unified multimodal CAD benchmark with 18,000 samples drawn from DeepCAD, Fusion 360, ABC, MCB and Objaverse, with single-view and multi-view render inputs. [search-result summary, not opened; render parameters not verified] — [emergentmind entry 2605.10873](https://www.emergentmind.com/papers/2605.10873) +- **IEEE DataPort.** Lists a "Multi-View-Image-Based CAD Reverse Engineering Dataset": single and multi-view RGB renders from canonical viewpoints, paired with DeepCAD-style sketch/extrude sequences. [search-result summary, not opened] — [IEEE DataPort](https://ieee-dataport.org/documents/multi-view-image-based-cad-reverse-engineering-dataset) +- **UniCAD (2026).** Notes that ABC has over 1M models but provides only B-rep geometry, without modelling histories. [search-result summary, not opened] — [UniCAD arXiv 2606.05058](https://arxiv.org/pdf/2606.05058) +- **ABC dataset.** About 1M CAD models from Onshape, distributed as geometry (B-rep/meshes), not as standardized renders. [from paper, not re-fetched] — [ABC arXiv 1812.06216](https://arxiv.org/abs/1812.06216) + +### Inferences +- Engineering-part rendering is fragmented. Each CAD paper picks a few fixed views (isometric, 4 orthogonal) without publishing a reusable renderer or full parameters. A tool that imports STEP/CAD and renders canonical orthographic front/back/top/bottom/left/right views plus isometric/perspective views, with masks, line-art (edges matter for CAD), depth, normals and pose export, would fill a real gap for CAD-ML benchmarks. +- The add-on's line-art and clay passes are unusually well matched to CAD (texture-less parts), where photoreal RGB with HDRI adds little. + +### Gaps +- Exact rendering parameters for Omni-CAD/CAD-MLLM, CADBench and the IEEE DataPort dataset were not verified. The CADFit/CADBench/UniCAD entries rest on search-result summaries only. +- I did not find a CAD-specific dataset that publishes depth or normal renders with camera poses in NeRF or COLMAP format. diff --git a/research_notes/Multiview renderer literature gap/product_rendering_tools.md b/research_notes/Multiview renderer literature gap/product_rendering_tools.md new file mode 100644 index 0000000..749c339 --- /dev/null +++ b/research_notes/Multiview renderer literature gap/product_rendering_tools.md @@ -0,0 +1,153 @@ +# Product-visualization tools for automated multi-view batch rendering from CAD (landscape and gap analysis, as of Oct 2026) + +Scope note: research done 2026-10-26 in about 16 search/fetch calls. Where only third-party or community sources were found, this is flagged. Prices are as of the date stated. Vendor pages were not checked by hand beyond what is cited. + +## 1. What do commercial product-visualization tools offer for automated multi-view batch rendering, and at what cost? + +### Takeaway +The commercial leaders (KeyShot Studio, SOLIDWORKS Visualize Professional, Autodesk VRED Professional/Design, NVIDIA Omniverse) can all automate rendering. They do it through paid tiers, per-seat subscriptions of roughly US$1,300-1,700+ a year, and either a GUI render queue or a scripting API you program yourself. None of the sources found describes a built-in "drop in a folder of STEP files, get a standardized orthographic six-view set, a turntable, a contact sheet and a manifest" pipeline. Batch multi-file workflows exist only as community scripts or as one-project, many-camera queues. + +### Cited Findings +**KeyShot (Luxion / KeyShot Studio)** +- Official pricing page (seen Oct 2026; page carries 2026 copyright and June 2026 images, no effective date): Professional US$1,299/yr ($108.25/user/mo), Business US$1,599/yr ($133.25/user/mo), 3-year totals $3,897 / $4,797, Enterprise custom-quoted. — [KeyShot Pricing](https://www.keyshot.com/pricing/) +- Add-ons on the same page: Network Rendering $749/yr (32-core tier, Business/Enterprise), NX Plugin $348/yr, Studio VR $1,188/yr, Cloud Rendering by credits ($25 for 100 credits up to $6,875 for 50,000). The Cloud Render Queue requires version 2026.2+. "KeyShotWeb (XR & Configurator)" is listed as an add-on. The page says 34+ CAD formats are supported and includes one-click animation and product reveals. — [KeyShot Pricing](https://www.keyshot.com/pricing/) +- Third-party sources say perpetual licences were discontinued in 2023 and there is no free tier, only a 14-day trial. This is not confirmed on the official page. — [renderahouse.com](https://www.renderahouse.com/blog/keyshot-pricing); [myarchitectai.com](https://www.myarchitectai.com/blog/keyshot-pricing) +- Capterra lists a "$99 KeyShot Pro" figure that conflicts with the official pricing and is likely stale. — [Capterra](https://www.capterra.com/p/173819/KeyShot/pricing/) +- KeyShot has a Python scripting API (the `lux` module). KeyShot Studio 2024/2025 use Python 3.12; KeyShot 11 used 3.10. Scripting is a Pro feature, and a headless mode runs scripts from a terminal without the GUI. — [KeyShot 2024 manual: scripting](https://manuals.keyshot.com/keyshot2024/manual/scripting.html); [KeyShot Studio 2025 manual: scripting](https://manuals.keyshot.com/kss2025/manual/scripting.html); [KeyShot 11 manual](https://manual.keyshot.com/keyshot11/manual/scripting/) +- Batch over many CAD files is done with user scripts. A KeyShot community forum post shares `batch_import_render_queue.py`, which imports every CAD file in a folder, optionally applies a material template, sets up a camera and queues a render per file. It uses a GUI settings dialog. The post notes that `lux.ThumbnailRenderOptions` is for material thumbnails, not scene renders. — [KeyShot community forum](https://db_e596cbae_9eea_47cf_9859_c9418b6eabfd.influitive.com/forum/t/keyshot-studio-python-script-batch-import-render-automation/14613) +- Headless has limits: some functions "can not be exposed in headless scripting", and `renderAnimation` was not available headless. The workaround is to loop `lux.setAnimationFrame()` and `lux.RenderImage()`, then call `lux.encodeVideo()`. The render queue misbehaves with multiple instances open. — [KeyShot forum "Headless hurdles"](https://community.keyshot.com/forum/t/headless-hurdles/507) +- Third-party MCP servers wrap KeyShot headless mode for batch rendering of named cameras. One notes that the `lux` API changes between versions. — [KeyShot MCP (glama.ai)](https://glama.ai/mcp/servers/truman-t3/keyshot-mcp); [glama.ai](https://glama.ai/mcp/servers/fo2aege8yd) + +**SOLIDWORKS Visualize (Dassault Systèmes)** +- The Render Queue and one-click "render all configurations / camera angles" from one project are Visualize **Professional**-only features. A one-click 360° turntable output exists, and turntables can be queued to render unattended. — [SOLIDWORKS: Visualize Professional-only features (PDF)](https://www.solidworks.com/sw/docs/SW_Visualize_Top_Professional-only_Features.pdf); [Javelin: Render Queue (2024)](https://www.javelin-tech.com/blog/2024/06/how-to-use-solidworks-visualize-professionals-rendering-queue-for-streamlined-design/); [Javelin: Professional vs Standard (2023)](https://www.javelin-tech.com/blog/2023/04/advantages-of-using-solidworks-visualize-professional-vs-standard/) +- Visualize Boost (network rendering) is included with each Professional seat. — [Javelin](https://www.javelin-tech.com/blog/?p=35219) +- No current public price was found. Pricing is through resellers. — [GoEngineer](https://www.goengineer.com/visualize-solidworks) + +**Autodesk VRED** +- The Python API is only in VRED Professional and VRED Design, not VRED Basic. Autodesk lists batch-processing files and scaling renders across a cluster as uses. — [Autodesk Platform Services: VRED APIs](https://aps.autodesk.com/node/4011) +- The only price found is an aggregator's "starting from US$1,700" with no edition or term stated, so it is unreliable. — [Capterra UK](https://www.capterra.co.uk/software/182129/vred) + +**Onshape (PTC)** +- The REST API thumbnails are a single isometric identification image per element. For other viewpoints, forum guidance points to the Part Studio/Assembly `/shadedviews` endpoints, which take a view matrix and `pixelSize`. Users report cropped or off-centre framing for large models. Automation needs API keys or OAuth. — [Onshape forum: thumbnails for BOM items](https://forum.onshape.com/discussion/18987/api-trying-to-find-a-more-efficient-way-to-pull-thumbnails-for-each-bom-item); [Onshape forum: auto-export view images](https://forum.onshape.com/discussion/17201/how-to-automatically-export-view-images) +- Onshape's own blog describes IKEA's CAD-to-catalogue workflow ("about 75% of images in the 2016 catalog began as 3D CAD models"). This is a secondary report of a conference talk. — [Onshape blog](https://www.onshape.com/en/blog/how-to-render-photorealistic-images-like-ikea) + +**NVIDIA Omniverse** +- Product configurators are an official workflow: CAD is converted to OpenUSD and authored with the Kit SDK, with web delivery through the App Streaming API. — [Omniverse dev overview](https://docs.omniverse.nvidia.com/dev-overview/latest); [Product configurator reference architecture](https://docs.omniverse.nvidia.com/arch-diagrams/latest/ref-arch-diagrams/prod-config-diagram.html) +- NVIDIA says configurators can produce 2D renders or video "at scale" for marketing, citing up to 40% time savings. Katana, SAP and WPP have built on the stack. — [NVIDIA developer blog: product configurators with OpenUSD](https://developer.nvidia.com/blog/developing-product-configurators-with-openusd/) +- A June 2026 NVIDIA FAQ describes OpenUSD + SimReady + MDL turning master CAD into cloud-streamed car configurators. — [NVIDIA Perspectives FAQ](https://perspectives.nvidia.com/physical-ai/omniverse/task/faq/software-cloud-streamed-photorealistic-3d-car-configurator/) + +### Inferences +- Every commercial automation path needs (a) a paid upper tier (KeyShot Pro-level scripting, Visualize Professional, VRED Professional/Design) and (b) custom scripting for batch-over-files. That puts a recurring cost of about $1,300+/seat/yr before any engineering effort on SMEs. +- The commercial tools are built around one hero project with many cameras or configurations. Batch processing of a catalogue folder of heterogeneous CAD files, with consistent framing and scale, is left to user scripts. This is the gap a free tool can fill. +- Onshape's shaded-views API is the closest thing to a free, headless multi-view generator in commercial CAD. It only works on Onshape-hosted documents, gives OpenGL-style shading, and users report framing problems. + +### Gaps +- No official 2026 prices for SOLIDWORKS Visualize, VRED, Siemens NX Render/Teamcenter Visualization, Fusion rendering or Rhino. Not searched in depth because of the call budget. +- Not verified: whether KeyShot's built-in Studios/Render Queue can render a predefined set of orthographic views across many imported models without scripting. +- Fusion (Autodesk), Rhino (Rhino.Compute or Python batch rendering) and Siemens NX/Teamcenter Visualization were not researched in this pass. +- No official source confirms which KeyShot tier includes scripting or headless in the 2026 lineup. The pricing page does not say. + +## 2. What does the Blender ecosystem offer (product-render add-ons and CAD/STEP importers)? + +### Takeaway +Blender has many cheap ($15-ish to tens of dollars) paid add-ons that set up studio lighting, preset multi-camera rigs (including e-commerce sets and some orthographic sheets) and turntables, plus a free GPL STEP importer on the official extensions platform. The sources found describe single-scene tools. None advertises batch import of STEP/IGES folders with deterministic catalogue naming, contact sheets, trimmed/padded web derivatives and a CSV/JSON manifest. + +### Cited Findings +**Product-render and batch add-ons (vendor listings, features not independently verified)** +- **Shot Studio Pro** (Superhive): five camera presets, "E-commerce (5), Marketplace Gallery (6), Hero Shots (4), Turnaround (8) and Orthographic Sheet (4)", a looping turntable exported as PNG frames or MP4, and batch rendering of all or selected cameras. — [Superhive: Shot Studio Pro](https://superhivemarket.com/products/shot-studio-pro) +- **1-Click Product Render Pro** (v1.2.0): one-click camera, lighting and background setup, batch Front/Side/3/4/Top views auto-named and organised, and a turntable. — [Superhive](https://superhivemarket.com/products/1-click-product-render-setup); [ArtStation](https://www.artstation.com/marketplace/p/1KgGK/1-click-product-render-pro-professional-camera-lighting-batch-tools-for-blender) +- **Studio Shot Builder**: generates 4, 6 or 8 product cameras, batch-renders them to PNG, and makes a 360° turntable rendered to H.264 MP4. — [Superhive: Studio Shot Builder](https://superhivemarket.com/products/studio-shot-builder) +- **OneClick Multi-Camera Render** (v3.2.0, $15 on BlenderKit): a multi-angle rig rendered to timestamped folders, no turntable. — [BlenderKit](https://www.blenderkit.com/addons/31b436e2-7cca-442c-aeee-6f7004275252) +- **SpinWiz**: turntable specialist with batch turntables across multiple models, requires Blender 4.1+. — [Blender Market: SpinWiz FAQ](https://blendermarket.com/products/spinwiz/faq) +- **Batch Render Pro** (Blender 5.0+): loads a folder of models and renders each from the selected cameras with render passes, no turntable. — [ArtStation](https://www.artstation.com/marketplace/p/1KBxv/blender-batch-render-pro-addon); see also [Superhive: Batch Render](https://superhivemarket.com/products/batch-render), [Multi-Camera Render Magic docs](https://superhivemarket.com/products/multi-camera-render-magic/docs) + +**STEP/CAD importers for Blender** +- **STEP Importer** by Clonephaze is on the official Blender Extensions platform: free, GPL-3.0, imports .step/.stp without external tools using the bundled "cascadio" library, with tessellation quality presets, post-import topology cleanup and "basic material support". It is v1.0.2 for Blender 5.1+ on Windows, macOS and Linux. One review reports a Blender 5.2 break ("Module 'cascadio' has no attribute load"), fixed by reinstalling. — [extensions.blender.org: STEP Importer](https://extensions.blender.org/add-ons/step-importer/); [reviews](https://extensions.blender.org/add-ons/step-importer/reviews/) +- Paid OpenCASCADE-based alternatives: **Cadprep** (STEP/IGES/BREP), **MStep** (Windows-only; its listing reports errors on Blender 5.1+), and a paid "STEP Importer" for Blender 4.2+ claiming assembly hierarchy and parallel tessellation (promotional post, unverified). **StepIO** exports STEP rather than importing it. — [Superhive: Cadprep](https://superhivemarket.com/products/cadprep); [Superhive: MStep](https://superhivemarket.com/products/mstep); [BlenderArtists thread](https://blenderartists.org/t/new-step-importer-native-cad-import-for-blender-4-2/1641518); [Superhive: StepIO](https://superhivemarket.com/products/stepio---blender-step-import--export-add-on) + +### Inferences +- The Blender market has the per-scene parts covered: camera rigs, turntables and lighting presets, some including an orthographic sheet. What it lacks is a catalogue pipeline: batch over a folder of CAD files, a fixed standardized view set with consistent scale and framing across products, deterministic naming, image post-processing, contact sheets and a machine-readable manifest. MultiView Product Renderer can claim that niche, provided the claim is worded as "none found advertising" rather than "none exist". +- Most listed add-ons are paid and closed or vendor-distributed. A free, open-source (GPL) add-on on extensions.blender.org would be one of very few. +- STEP import with colours and assembly structure in Blender depends on third-party importers of uneven maturity, and these break across Blender versions. Robust STEP colour and part handling is still a practical risk worth stating as a limitation. + +### Gaps +- Not verified whether the free Clonephaze importer keeps STEP part colours and assembly hierarchy. The listing only says "basic material support". +- The "Product Render", "Photographer" and BlenderKit add-ons named in the brief were not looked up specifically. +- Prices for Shot Studio Pro, 1-Click Product Render Pro, Studio Shot Builder and SpinWiz were not captured from the listings. + +## 3. What open-source or web-based options exist, and how do part distributors (TraceParts, 3D ContentCentral, CADENAS PARTcommunity) produce previews at scale? + +### Takeaway +Open-source tools for STEP-to-image are mostly single-thumbnail utilities or libraries, such as Rust CLIs and OpenCASCADE wrappers, not catalogue renderers. How CADENAS and TraceParts generate preview images for millions of parts is not publicly documented in the sources found. + +### Cited Findings +- **Catwalk (catwalk3d, Rust)** is a CLI that renders 3D models, including STEP, to PNG thumbnails on the CPU with no GPU or display server, headless on Linux, macOS and Windows. The docs.rs build for 2.0.0 failed. — [docs.rs: catwalk3d](https://docs.rs/catwalk3d/) +- **cadrum (Rust)** is a parametric CAD library on headless OpenCASCADE that reads and writes STEP/BRep and exports PNG/glTF/STL. It is a toolkit, not a renderer pipeline. — [docs.rs: cadrum 0.8.12](https://docs.rs/crate/cadrum/0.8.12) +- **Adom chip-thumbnailer** (MIT) makes shaded isometric PNGs from STEP for electronic parts. It depends on a separate step2glb/OCCT service, and one page marks it deprecated. — [Adom wiki README](https://wiki.adom.inc/api/pages/adom/adom-chip-thumbnailer/files/README.md); [deprecated notice](https://wiki.adom.inc/api/pages/adom/chip-thumbnailer/files/README.md) +- CADENAS PARTcommunity offers "millions of 3D CAD models from over 400 product catalogs". Version 7 lets manufacturers pin and update preview images and shows an interactive 3D preview. The press material does not say how previews are generated. — [CADENAS: PARTcommunity 7 press release](https://www.cadenas.de/press/reader/partcommunity-7-this-is-what-the-new-download-portal-version-offers-for-engineers-and-purchasers-pr) +- CADENAS reported 86 million 3D part downloads in 2013, which shows the scale of catalogue CAD distribution. — [PARTsolutions press release](https://partsolutions.com/?p=5716) + +### Inferences +- Distributor portals show interactive WebGL viewers plus static preview images, likely made by their own server-side tessellation pipelines. Because this is undocumented, the paper should not make claims about their methods. It can say that catalogue-scale preview generation exists in proprietary portals while no open, documented equivalent was found. +- Open-source headless options give one shaded thumbnail per file. They do not give photoreal multi-view sets, turntables or manifests. + +### Gaps +- TraceParts, 3D ContentCentral, FreeCAD screenshot/render workbenches, CadQuery/build123d viewers (e.g. ocp_vscode) and three.js thumbnailers were not searched because of the call budget. +- No primary source on CADENAS or TraceParts preview-image generation methods. + +## 4. Which tools support the specific capabilities (STEP with parts/colours, standardized multi-view sets, turntables, line/technical drawings, batch over many files with consistent framing, manifests/metadata, headless)? + +### Takeaway +From the sources found, no single tool combines all of these in a ready-made, free package. Turntables and multi-camera queues are common. Standardized ortho six-view sets are rare (one Blender add-on offers a 4-view orthographic sheet). Batch over files and headless operation need scripting in paid tools. Manifest/metadata output was not advertised by any tool found. + +### Cited Findings +- Turntable one-click: SOLIDWORKS Visualize Professional ([Javelin](https://www.javelin-tech.com/blog/2024/06/how-to-use-solidworks-visualize-professionals-rendering-queue-for-streamlined-design/)); KeyShot animation, with headless animation done frame by frame ([KeyShot pricing](https://www.keyshot.com/pricing/); [KeyShot forum](https://community.keyshot.com/forum/t/headless-hurdles/507)); Blender add-ons Shot Studio Pro, Studio Shot Builder, SpinWiz ([Superhive](https://superhivemarket.com/products/shot-studio-pro); [Superhive](https://superhivemarket.com/products/studio-shot-builder); [Blender Market](https://blendermarket.com/products/spinwiz/faq)). +- Standardized view sets: Shot Studio Pro's "Orthographic Sheet (4)" and e-commerce presets ([Superhive](https://superhivemarket.com/products/shot-studio-pro)); 1-Click Product Render Pro's Front/Side/3/4/Top ([Superhive](https://superhivemarket.com/products/1-click-product-render-setup)); Onshape shaded views via a view matrix ([Onshape forum](https://forum.onshape.com/discussion/17201/how-to-automatically-export-view-images)). +- Batch over many files: KeyShot community script ([KeyShot forum](https://db_e596cbae_9eea_47cf_9859_c9418b6eabfd.influitive.com/forum/t/keyshot-studio-python-script-batch-import-render-automation/14613)); VRED Python batch processing ([Autodesk APS](https://aps.autodesk.com/node/4011)); Blender Batch Render Pro (folder of models) and SpinWiz (batch turntables) ([ArtStation](https://www.artstation.com/marketplace/p/1KBxv/blender-batch-render-pro-addon); [Blender Market](https://blendermarket.com/products/spinwiz/faq)). +- Headless: KeyShot headless scripting, with limitations ([KeyShot 2025 manual](https://manuals.keyshot.com/kss2025/manual/scripting.html)); Onshape REST API ([Onshape forum](https://forum.onshape.com/discussion/18987/api-trying-to-find-a-more-efficient-way-to-pull-thumbnails-for-each-bom-item)); Catwalk CLI ([docs.rs](https://docs.rs/catwalk3d/)). +- Catalogue naming conventions exist in standards. GS1 encodes GTIN, image type, facing, angle and state in file names (e.g. `09520123456740_A1C1`). — [GS1 Primary Images handout](https://gs1.org/docs/images/Primary-Images-handout.pdf) + +### Inferences +- Gap matrix for positioning. The rows are fully supported by no tool found: + 1. a fixed, reproducible standard view set (six ortho + perspective) applied identically across a folder of products, with consistent scale and framing; + 2. deterministic, standards-inspired file naming; + 3. post-processing to web derivatives (trim, pad, resize) and contact sheets; + 4. a CSV/JSON manifest; + 5. all of this free and open source, scriptable and headless (Blender `-b`). +- Line/technical-drawing style output (e.g. Freestyle or Line Art) was not advertised by any of the product-render tools found. It is a possible differentiator if the add-on supports it, otherwise a limitation. + +### Gaps +- Not verified against current docs whether KeyShot or VRED output metadata or manifest files with renders. +- Line-drawing support in KeyShot ("toon"/contour materials) and Visualize was not checked. + +## 5. Is there academic or industry literature on automated product imagery from CAD, and on product-image standards (GS1, Amazon)? + +### Takeaway +Academic work on rendering CAD at scale is almost entirely about **synthetic training data for computer vision** (CAD2Render, SynthiCAD, Blender-based pipelines), not catalogue or e-commerce imagery. No peer-reviewed paper on automated catalogue/PIM product-image generation from CAD was found. Industry evidence is anecdotal (IKEA). Strong normative grounding exists in the **GS1 Product Image Specification** (standard facings, plunge angles, naming) and marketplace rules such as Amazon's white background and 85% fill. + +### Cited Findings +**Academic (synthetic data from CAD; closest neighbours)** +- **CAD2Render** (Moonen et al.) is a GPU-accelerated photoreal synthetic-data toolkit on Unity HDRP that works from manufacturer CAD, aimed at manufacturing. — [ResearchGate](https://www.researchgate.net/publication/365783734_CAD2Render_A_Modular_Toolkit_for_GPU-accelerated_Photorealistic_Synthetic_Data_Generation_for_the_Manufacturing_Industry); [Zenodo](https://zenodo.org/records/15501381) +- **SynthiCAD** automatically creates customised image datasets from CAD models, tested on 10 industrial objects (engine parts, screws). — [ResearchGate](https://www.researchgate.net/publication/374151177_SynthiCAD_Generation_of_Industrial_Image_Data_Sets_for_Resilience_Evaluation_of_Safety-Critical_Classifiers) +- **CAD2DMD-SET** renders CAD in Blender through its Python API with randomised lighting, orientation and camera distance. — [arXiv 2508.21732](https://arxiv.org/pdf/2508.21732) +- A Blender-Python pipeline for object-to-model deep learning in industrial applications composites renders on SUN backgrounds, with code released. — [arXiv 1909.10976](https://arxiv.org/pdf/1909.10976) +- About 100 synthetic images per CAD design, varying colour, texture, light and camera, for additive-manufacturing segmentation. — [arXiv 2210.07466](https://arxiv.org/pdf/2210.07466) +- DepthSynth produces real-time synthetic depth data from CAD. — [ResearchGate](https://www.researchgate.net/publication/314115413_DepthSynth_Real-Time_Realistic_Synthetic_Data_Generation_from_CAD_Models_for_25D_Recognition) + +**Industry case** +- IKEA: about 75% of 2016 catalogue images began as 3D CAD models. This is a secondary report on Onshape's blog. — [Onshape blog](https://www.onshape.com/en/blog/how-to-render-photorealistic-images-like-ikea) + +**Standards and marketplace guidelines** +- The GS1 Product Image Specification Standard is at Release 4.4 (ratified April 2024). Earlier releases are 4.0 (May 2021) and 3.6 (Sep 2020, legacy). There is also a separate GS1 Product Image Sharing Guideline (Release 1.0, Dec 2021). — [GS1 current standard](https://www.gs1.org/standards/gs1-product-image-specification-standard/current-standard) (page returned 403 to the fetcher; release and date are from the search index); [GS1 Release 4.0](https://www.gs1.org/standards/gs1-product-image-specification-standard/40); [GS1 Release 3.6](https://gs1.org/standards/gs1-product-image-specification-standard/36); [GS1 Image Sharing Guideline](https://gs1.org/standards/product-image-sharing-guideline/current-standard) +- GS1 angle guidance: a centre front view at 15° top elevation is preferred for 3D images, with left/right 15° rotation views where warranted. A 0° plunge is used for 2D images and for products of negligible depth. Primary-image file names encode GTIN, image type, facing, orientation (Centre/Left/Right/No-plunge) and state, e.g. `09520123456740_A1C1`. — [GS1 Primary Images handout](https://gs1.org/docs/images/Primary-Images-handout.pdf) +- National variations: GS1 New Zealand requires a 2D front shot with no plunge plus a 3D plunge shot, with exemptions such as wine bottles and flat items ≤10 mm deep. GS1 Belgium/Luxembourg makes a front packshot without plunge mandatory for FMCG and foodservice. GS1 Canada publishes its own image submission standard. — [GS1 NZ: what images to supply](https://support.gs1nz.org/hc/en-us/articles/360019561512-What-kind-of-images-do-I-need-to-supply); [GS1 NZ photography specs](https://support.gs1nz.org/hc/en-us/articles/360018596392-What-are-GS1-New-Zealand-s-photography-specifications); [GS1 Belgium/Luxembourg](https://gs1belu.org/en/media/6531/download); [GS1 Canada image submission standard (PDF)](https://gs1ca.org/gs1ca-components/documents/GS1CA_MKTG_Image_Submission_Standard_EN.pdf) +- Amazon main-image rules (mostly third-party summaries): pure white background (RGB 255,255,255), product filling at least 85% of the frame, no text, logos or props, and the full product visible. Extra angles go in secondary images. Minimum resolution claims vary from 500 to 1,000 px on the longest side, with about 1,600 px+ suggested for zoom. The 85% figure is also in a Seller Central forum post. — [Amazon Seller Forums](https://sellercentral.amazon.com/seller-forums/discussions/t/13af96ea-6b07-4bf9-8dbe-a13292c2e3b1); [Jungle Scout](https://www.junglescout.com/resources/articles/amazon-image-requirements/); [SellerSprite](https://www.sellersprite.com/en/blog/amazon-image-requirements) + +### Inferences +- There is a clear **literature gap**: rendering CAD at scale is well studied as synthetic data for machine learning (randomised views), but deterministic, standardized, catalogue-grade imagery from CAD (fixed views, consistent framing and scale, naming, manifests) has no academic treatment in what was found. The add-on can be positioned as filling that gap and as reusable for synthetic-data work (via its manifest and deterministic views). +- GS1's facing/plunge/naming model maps onto the add-on's features. Orthographic front/back/left/right/top/bottom correspond to GS1 facings at 0° plunge, the perspective view to a 15° plunge, and catalogue naming to GS1-style deterministic file names. The trim/pad/resize step maps to Amazon's white background and 85% fill. A GS1-compatible naming preset and a 15°-plunge preset would strengthen the positioning. This is a suggestion, not something the sources say. + +### Gaps +- The full GS1 4.4 text could not be fetched (HTTP 403). Details on resolution, background, clipping paths, 360° spin images and file format come only from the handout and national-office pages, and were not captured in detail. +- Amazon's official Seller Central image-requirements help page was not retrieved; figures are from third-party guides and a forum post. +- Google Scholar/ASME were not searched directly for "e-commerce product rendering from CAD", "PIM imagery" or "configurator rendering". A targeted Scholar search could still turn up a paper, so state "none found in this search" rather than "none exists". diff --git a/research_notes/Multiview renderer literature gap/synthetic_data_pipelines.md b/research_notes/Multiview renderer literature gap/synthetic_data_pipelines.md new file mode 100644 index 0000000..bdac333 --- /dev/null +++ b/research_notes/Multiview renderer literature gap/synthetic_data_pipelines.md @@ -0,0 +1,191 @@ +# Synthetic data and procedural rendering pipelines: what they cover, for positioning MultiView Product Renderer + +Scope: BlenderProc/BlenderProc2, Kubric, NVIDIA Omniverse Replicator (Isaac Sim), Unity Perception, Infinigen, BlenderSynth, Render for CNN (Su et al. 2015). Researched October 2026. Primary sources were GitHub READMEs, official docs, arXiv and JOSS pages, and NVIDIA forums. + +## Q1. For each tool: inputs (STEP/CAD or mesh only), outputs and annotations, how it is driven, license, intended users + +### Takeaway +Every tool here is a code-first or config-first generator of ML training data. They output dense ground truth (depth, normals, instance/semantic segmentation, flow) and machine-readable annotations (COCO, BOP, camera K matrices, poses). None of the Blender-based tools documents native STEP/IGES import; they all load meshes. Only NVIDIA Isaac Sim/Omniverse ships a CAD-to-USD converter, and users report frequent STEP import failures with it. Unity Perception is discontinued. Kubric is pinned to Blender 2.93. + +### Cited Findings + +**BlenderProc / BlenderProc2 (DLR-RM)** +- License: GPL-3.0. — [BlenderProc GitHub](https://github.com/DLR-RM/BlenderProc) +- Inputs: the README lists "Loading: `*.obj`, `*.ply`, `*.blend`, `*.fbx`, BOP, ShapeNet, Haven, 3D-FRONT, etc." It does not mention STEP, IGES or any other B-rep CAD format. — [BlenderProc GitHub](https://github.com/DLR-RM/BlenderProc) +- Outputs: RGB, stereo, depth, normal and segmentation images or sequences. Writers produce ".hdf5 containers, COCO & BOP annotations". — [BlenderProc GitHub](https://github.com/DLR-RM/BlenderProc) +- The docs list examples for optical flow, NOCS, semantic segmentation, diffuse color images, COCO annotations ("Write COCO annotations to a .json file for selected objects in the scene") and BOP Challenge data ("Generate the pose-annotated data used at the BOP Challenge 2020"). — [BlenderProc docs](https://dlr-rm.github.io/BlenderProc/) +- Cameras: "Set, sample or load camera poses from file". Examples include sampling camera positions inside a shape with rotation constraints. The API call is `bproc.camera.add_camera_pose(cam_pose)`. — [BlenderProc GitHub](https://github.com/DLR-RM/BlenderProc); [BlenderProc docs](https://dlr-rm.github.io/BlenderProc/) +- Driven by Python scripts through a CLI: `blenderproc run