Skip to content

Commit 820b608

Browse files
committed
Add camera export, data passes, COCO/BOP, headless CLI and tests (1.10.0)
Annotations - cameras.json per product: per-view K (perspective) or pixel size (orthographic), c2w in OpenGL and w2c in OpenCV axes, image paths - Track each product's source-to-world transform through every move MultiView makes, so renders map back to CAD coordinates and units - nerfstudio-style transforms.json for perspective views and for turntables (equivalent orbiting cameras) - Depth and camera-space normal passes: single-sample Cycles, 32-bit EXR, no filter blending - Part-ID pass: 16-bit PNG label map numbering a product's parts - COCO export: per-part instance masks (RLE) with part names - BOP export from perspective views: rgb, depth (mm), masks, scene_camera/gt/gt_info, PLY models and models_info, with rigid poses in source-model millimetres Command line - cli.py runs a JSON/TOML job headless without installing the add-on: import, standardize, auto-orient, scene setup, render - --shard i/n with job-wide labels and per-shard lists, --merge to combine them, --check for pre-flight only, --list-settings - Exit codes 0/1/2 and a MULTIVIEW_SUMMARY line Tests - tests/ suite run inside background Blender, with a runner and STEP/IGES fixtures; FreeCAD checks skip when it is absent - GitHub Actions workflow running the suite on Blender 5.0.0 Fixes - JSON outputs are rewritten only when their content changes
1 parent f08ae14 commit 820b608

24 files changed

Lines changed: 5412 additions & 31 deletions

‎.github/workflows/tests.yml‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
name: Tests
2+
3+
on:
4+
push:
5+
pull_request:
6+
workflow_dispatch:
7+
8+
jobs:
9+
test:
10+
runs-on: ubuntu-24.04
11+
timeout-minutes: 90
12+
env:
13+
# The version the suite is written against. Bump deliberately: Blender
14+
# changes its Python API between releases.
15+
BLENDER_VERSION: "5.0.0"
16+
steps:
17+
- uses: actions/checkout@v4
18+
19+
- name: Cache Blender
20+
id: cache-blender
21+
uses: actions/cache@v4
22+
with:
23+
path: blender
24+
key: blender-${{ env.BLENDER_VERSION }}-linux-x64
25+
26+
- name: Download Blender
27+
if: steps.cache-blender.outputs.cache-hit != 'true'
28+
run: |
29+
series="${BLENDER_VERSION%.*}"
30+
curl -fsSL -o blender.tar.xz \
31+
"https://download.blender.org/release/Blender${series}/blender-${BLENDER_VERSION}-linux-x64.tar.xz"
32+
mkdir blender
33+
tar -xf blender.tar.xz -C blender --strip-components=1
34+
35+
- name: Install system libraries
36+
# Blender's Workbench and Cycles renders run headless through EGL;
37+
# Mesa supplies a software GL on a runner without a GPU.
38+
run: |
39+
sudo apt-get update
40+
sudo apt-get install -y --no-install-recommends \
41+
libegl1 libgl1 libglx-mesa0 libgl1-mesa-dri libxi6 libxxf86vm1 \
42+
libxfixes3 libxrender1 libxkbcommon0 libsm6 libice6
43+
44+
- name: Run tests
45+
run: python3 tests/run_tests.py --blender blender/blender

‎README.md‎

Lines changed: 129 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ every view in one click, with predictable file names like `AA_01_Widget_Front_cl
88

99
- **Blender:** 4.0 and newer
1010
- **Panel location:** `3D Viewport → Sidebar (N key) → MultiView`
11-
- **Version:** 1.9.0
11+
- **Version:** 1.10.0
1212

1313
---
1414

@@ -605,6 +605,116 @@ since been deleted are dropped.
605605

606606
---
607607

608+
#### Annotations for research
609+
610+
The **Annotations** section under Batch Render turns a render into a dataset. Everything
611+
here is computed from the same fixed camera rig as the images, so it lines up with them to
612+
the pixel. The test suite checks the camera numbers against Blender's own projection and the
613+
depth against ray casts.
614+
615+
| Setting | Writes | Notes |
616+
| --- | --- | --- |
617+
| **Cameras** (on by default) | `AA_01_Widget_cameras.json`, `AA_01_Widget_transforms.json`, and `..._Turntable_..._transforms.json` | See below |
618+
| **Depth** | `..._Front_depth.exr` | 32-bit EXR, distance along the view axis in scene units, 0 and alpha 0 off the product |
619+
| **Normals** | `..._Front_normal.exr` | 32-bit EXR, camera space, OpenGL axes (+X right, +Y up, +Z towards the camera), values -1..1 |
620+
| **Part IDs** | `..._Front_parts.png` | 16-bit greyscale PNG: 0 is background, *n* is the *n*-th part in `cameras.json`'s `parts` list. Parts are the product's objects, so a STEP assembly's parts keep the names its CAD file gave them |
621+
| **COCO** | `coco.json` | One instance mask (uncompressed RLE) per visible part per image, with `part_id` and `part_name`. Categories are products. Turns on Part IDs |
622+
| **BOP** | `bop/` | A [BOP](https://github.com/thodan/bop_toolkit/blob/master/docs/bop_datasets_format.md) dataset from the perspective views: `train_pbr/<scene>/rgb`, `depth` (uint16, mm), `mask`, `mask_visib`, `scene_camera.json`, `scene_gt.json`, `scene_gt_info.json`, plus `models/obj_*.ply` and `models_info.json`. Turns on Depth |
623+
624+
Depth, Normals and Part IDs are each one extra Cycles render per view: one sample, no
625+
antialiasing filter, every surface replaced by an emitter that outputs the number itself.
626+
Each pixel therefore belongs to exactly one surface. There is no blending of a part's ID with
627+
the background, or of a foreground depth with what is behind it. They need the Cycles add-on
628+
(on by default) whatever engine the Full render uses. They are never post-processed or put
629+
on a contact sheet.
630+
631+
**`cameras.json`** describes every view of the product:
632+
633+
- `c2w_opengl`: camera-to-world matrix, Blender/OpenGL axes (looking down -Z, +Y up).
634+
- `w2c_opencv`: world-to-camera matrix, OpenCV axes (looking down +Z, +Y down), as COLMAP
635+
and BOP use.
636+
- `K` for perspective views. Orthographic views get `pixel_size` and `principal_point`
637+
instead, since no K describes them.
638+
- `width`, `height`, `clip`, and the relative paths of every image of that view.
639+
- `model_to_world`: the 4x4 matrix from the product's **source file coordinates** (in
640+
`source_units`) to the scene. MultiView records every move it makes to a product (fitting,
641+
recentring, turning, Auto Orient), so a pixel in any render can be traced back to a point
642+
in the original CAD file. Moves you make by hand in the viewport are not seen.
643+
644+
**`transforms.json`** is in the nerfstudio/Instant-NGP format, for NeRF and Gaussian
645+
splatting tools. It lists the perspective views only, with per-frame intrinsics and
646+
`mask_path` when the Mask pass is on. A turntable gets its own `transforms.json`: the
647+
product turns and the camera stays still, so each frame is given the camera that would
648+
circle a still product. That is what those tools expect.
649+
650+
BOP wants rigid poses in millimetres of the model, although the scene was scaled to fit.
651+
The scale is taken back out of the camera translation, the model is exported in its own
652+
source units converted to mm, and depth is written in the same mm, with a `depth_scale` per
653+
image so large parts still fit in 16 bits. Only perspective views go into BOP, because the
654+
format has no orthographic camera.
655+
656+
## Command line
657+
658+
Everything the panels do can run without Blender's window, from a job file:
659+
660+
```bash
661+
blender -b --factory-startup --python multiview_product_renderer/cli.py -- --config job.json
662+
```
663+
664+
The add-on does not need to be installed for this: `cli.py` loads it from the folder it sits
665+
in. A job is JSON or TOML, and relative paths in it are relative to the job file:
666+
667+
```json
668+
{
669+
"inputs": ["parts/", "extra/housing.step"],
670+
"recursive": true,
671+
"output": "renders/",
672+
"freecad": "C:/Program Files/FreeCAD 1.1/bin/freecadcmd.exe",
673+
"step_import_mode": "FREECAD",
674+
"standardize": false,
675+
"auto_orient": "FLAT",
676+
"scene_setup": true,
677+
"settings": {
678+
"render_mode": "CLAY",
679+
"view_set": "ALL",
680+
"pass_depth": true,
681+
"export_coco": true,
682+
"turntable_elevation": 20
683+
},
684+
"render": {"engine": "CYCLES", "samples": 64, "device": "CPU",
685+
"resolution": [1024, 1024], "file_format": "PNG"},
686+
"save_blend": "renders/job.blend"
687+
}
688+
```
689+
690+
| Key | Meaning |
691+
| --- | --- |
692+
| `blend` | Start from this `.blend` instead of an empty scene. With no `inputs`, its existing products are rendered |
693+
| `inputs`, `recursive` | Files and folders to import (folders expanded, sorted) |
694+
| `output` | Output folder; `--output` overrides it |
695+
| `freecad`, `step_import_mode`, `deflection` | STEP/IGES conversion, as in the add-on preferences |
696+
| `standardize`, `auto_orient` | Run Standardize Products / Auto Orient (true, `"FLAT"` or `"PCA"`) on everything after import |
697+
| `scene_setup` | Apply Scene Setup (lighting, background, ground). Default true |
698+
| `settings` | Any panel setting by name; `--list-settings` prints them all with their types. Angles are in degrees |
699+
| `render` | `engine`, `samples`, `device`, `resolution`, `percentage`, `file_format` |
700+
| `save_blend` | Save the set-up scene afterwards, to open and inspect in Blender |
701+
702+
Other modes:
703+
704+
- `--check` imports and sets up, runs the pre-flight check, prints its findings and renders
705+
nothing.
706+
- `--shard 2/8` imports and renders only every eighth file, starting with the third. Labels
707+
still count through the whole job, so eight shards started at once, on one machine or
708+
eight, write exactly the files one run would. Each writes its own
709+
`manifest_shard02.json`, `coco_shard02.json` and BOP `models_info_shard02.json`.
710+
- `--merge renders/` then combines those into `manifest.json`/`.csv`, `coco.json` and
711+
`models_info.json`.
712+
- `--list-settings` prints every setting `settings` accepts.
713+
714+
It prints a `MULTIVIEW_SUMMARY {...}` JSON line at the end. The exit code is 0 when
715+
everything rendered, 1 when some images failed, and 2 when the job could not run: a bad job
716+
file, nothing imported, or pre-flight errors.
717+
608718
## Collections the add-on creates
609719

610720
| Collection | Contents |
@@ -670,6 +780,24 @@ again.
670780

671781
---
672782

783+
## Tests
784+
785+
The `tests/` folder holds the test suite: each `test_*.py` runs inside a background Blender
786+
and checks the add-on end to end, by rendering real images and reading them back.
787+
788+
```bash
789+
python tests/run_tests.py --blender "C:/Program Files/Blender Foundation/Blender 5.0/blender.exe"
790+
```
791+
792+
Any Python 3 runs the runner, including Blender's own (`<Blender>/5.0/python/bin/python`).
793+
FreeCAD is optional: with `freecadcmd` on `PATH`, or `FREECAD_CMD` set, the STEP and IGES
794+
import checks run too; without it they are skipped (`FREECAD_CMD=none` forces that). GitHub
795+
Actions runs the suite on every push against Blender 5.0.0 (`.github/workflows/tests.yml`),
796+
without FreeCAD.
797+
798+
The suite is written against **Blender 5.0**. The add-on aims to support 4.0 and newer, but
799+
only 5.0 is tested.
800+
673801
## License
674802

675803
MIT. See [LICENSE](LICENSE).

0 commit comments

Comments
 (0)