# PicoBerry image-to-3D benchmark — 2026-09-12

Same-input, engine-default, textured image-to-3D runs across the six engines named in
https://picoberry.ai/which-ai-3d-model-for-game-assets, executed through the public PicoBerry API
(`POST /v1/models/from-image`) with a paid account's API key. Everything here is machine-read; nothing was scored by eye.

## What was run
- **Inputs** (3): `inputs/stylized-prop.webp`, `inputs/hard-surface.webp`, `inputs/character.webp` — generated once with
  `POST /v1/images` (gpt-image-2.5-flare, 1:1) from the prompts in `results.json → inputs[].prompt`, then reused verbatim for every engine.
- **Engines** (6): tripo (PB Slim, Tripo P1-20260311), tripo-p2 (PB Slim 2, Tripo P2-20260801), tripo-v3.1 (PB Standard, Tripo v3.1-20260211),
  tripo-v3.1-ultra (PB Ultra, Tripo v3.1-20260211 with the 2M default polygon target), meshy6 (Meshy 6), hunyuan-3.1 (Tencent Hunyuan 3D 3.1).
  Vendor labels/versions are the catalog values at run time (`results.json → engines`, also per run in `manifest.json`).
- **Settings**: `texture=true`, no `polycount` (engine default), single image. Nothing else set.
- **Repeats**: 3 per input × engine → 54 runs, submitted with concurrency 6 between 00:52 and 01:24 UTC on 2026-09-12. 54/54 succeeded.

## What is measured
From the API (`results.json`, aggregated in `summary.json`):
- `status` — succeeded / failed.
- `wallSeconds` — submit → terminal status, polled every 5 s (±5 s; includes queue time on PicoBerry's side).
- `stats.vertices / faces / topology` — from the pipeline's `files.stats` JSON of the finished asset.
- `modelBytes` / `previewBytes` — `Content-Length` of `model.glb` and `preview.glb`.
- `thumbnails/*.webp` — the pipeline's standard studio render of each result (512²), for visual comparison only.
- Credits — **image-to-3D list price** at run time from `GET /v1/models?category=image-to-3d` (`cost` mesh + `textureCost`), plus the
  account balance delta for the whole run (`results.json → creditsConsumed`): 188,439 → 166,959 = **21,480** = 9 runs × 2,370 (sum of the six
  engines' mesh+texture prices) + 150 for the three reference images.

From the original 3D files (`mesh-metrics.json`, per-engine medians copied into `summary.json → gltf`), parsed with `mesh-metrics.py`:
- SHA-256 and byte size of `model.glb` (the engine's raw output as delivered by the pipeline — no remesh, no post-processing) and `preview.glb`.
- glTF: generator, mesh/primitive/material counts, texture image count and largest pixel size, presence of `POSITION`/`NORMAL`/`TEXCOORD_0`/`TANGENT`,
  vertex and triangle totals (from accessors — equal to the pipeline's reported faces for all 54 files), bounding box, box-centre offset from the origin, extensions used.

## Follow-up on the same 54 outputs (2026-09-12, second re-evaluation)
Nothing was regenerated. The published outputs were (a) imported into Blender and measured, (b) pushed through PicoBerry's own
post-processing API, and (c) rendered from four fixed cameras and inspected.

**(a) Import + topology/UV metrics — `mesh-quality.json`, `mesh_quality_blender.py` (Blender 5.1.2, headless, free).**
All 54 import without errors. Per file: import seconds; triangles/quads/ngons; non-manifold and boundary edges *as delivered* and *after
welding* coincident vertices (`remove_doubles` 1e-6 — glTF splits vertices at UV/normal seams, so the as-delivered counts are seam artefacts;
the welded counts are the real open edges and loose parts); loose vertices, degenerate faces, duplicate vertices, connected parts (shells);
UV layers, faces without UVs, UV verts outside 0–1, summed UV area, overlapping faces (`uv.select_overlap`), texel-density CV, islands (≤300k faces);
bounds and floor contact. Medians per engine are in `summary.json → quality`. Headline: Meshy 6 and Hunyuan 3.1 deliver one closed shell with
0 open edges after welding; PB Standard/Ultra one shell with 114/5 open edges (median); PB Slim/Slim 2 are multi-part low-poly meshes
(13/32 parts, 148/488 open edges, ~1,200/600 UV islands). UV overlap ≤ 0.5 % of faces everywhere.

**(b) Post-processing through the public API — `postprocess.json`, `postprocess.py` (PAID, small).**
Every output: `pb-remesh` to a 20,000-face target (10 credits) → `pb-uv` unwrap on the remeshed result (10 credits). The 18 character
outputs additionally: `tripo-rig` (15) and `meshy-rig` (10) auto-rig on the raw output, rigging only. Results (`summary.json → postprocess`,
`python3 postprocess.py --summarize` is free): remesh 50/54 (all 4 failures on PB Ultra's ~1.9M-face outputs; medians 34 s Slim, 61 s Meshy/Hunyuan,
77–99 s Standard/Ultra), UV 49/50 (one PB Slim character failed), tripo-rig 17/18 (one Hunyuan 3.1 character failed), meshy-rig 18/18
(47–322 s). Wall time includes PicoBerry's queue. The public API does not expose failure reasons — none are invented. Balance moved
166,769 → 165,334 credits (1,435), consistent with list prices with failed external tasks refunded; the account is shared, so this is a
reconciliation, not an audit. One remesh (hunyuan-3.1 / character / 1) was submitted by hand before the runner to check the request shape and
is recorded without a wall time. Derived assets stay in the account (ids in `postprocess.json`); they are not on the CDN.

**(c) Render sheets + inspection notes — `renders/` on the CDN, `visual-review.json`, `render_sheets.py` (free).**
`mesh_quality_blender.py` renders each model from cameras at world −Y/+Y/−X/+X (EEVEE, the file's own PBR materials, three sun lights,
orthographic; the file names front/back/left/right are camera positions, not the model's front — engines face different axes).
`render_sheets.py` composes one sheet per engine × input (`renders/sheets/<engine>__<input>.jpg`, 3 runs × 4 views + the input image).
`visual-review.json` records, per combination, whether the shape matches the input, whether colour/material reads close, visible defects,
and the forward axis — with the criteria, the reviewer (PicoBerry product team, AI-assisted inspection of the sheets) and the limits.
It is a record, not a grade, and no engine is ranked from it.

## What is still NOT measured
Import into Unity, Unreal or Godot (only Blender was tested); whether a rigged character animates acceptably (only rig success/time);
texture quality beyond the 512 px inspection notes; remesh quality beyond face count. The guide page's criteria table says which row is
*measured*, *inspected*, *instrumented* or *not measured*. n = 9 per engine on three inputs is a limited sample, not a general success-rate claim.

## Files
- `summary.json` — per-engine success, wall-time and face medians/ranges, GLB size, credits (mesh, texture, mesh+texture) and `gltf` metrics (the tables on the guide page are rendered from this).
- `results.json` — every run: asset id, timestamps, wall time, stats, sizes, thumbnail path; engine catalog snapshot (image-to-3D); credit delta and its reconciliation.
- `manifest.json` — integrity/provenance: SHA-256 + bytes of every published file and of all 108 GLBs, per-run engine vendor/version, run id, asset id, list price, and the CDN URL of each `model.glb` / `preview.glb`.
- `mesh-metrics.json` — per-file glTF metrics (the data behind `summary.json → gltf`).
- `mesh-quality.json` — per-file Blender import / topology / UV metrics (the data behind `summary.json → quality`); `mesh_quality_blender.py`.
- `postprocess.json` — per-output remesh / UV unwrap / rig results, requests, asset ids, list prices (the data behind `summary.json → postprocess`); `postprocess.py`.
- `visual-review.json` — inspection notes per engine × input, criteria, reviewer, limits; `render_sheets.py`.
- `contact-sheet.jpg` — all 54 thumbnails, engines × inputs × repeats.
- `inputs/` (3 reference images), `thumbnails/` (54), `bench.py`, `mesh-metrics.py`, `manifest.py` (rebuilds `manifest.json` from the other files), `aggregate_followup.py` (folds (a)+(b) into `summary.json`).
- Original 3D files: `https://cdn.umodeler.com/media/benchmarks/2026-09-12-image-to-3d/<engine>__<input>__<n>.{model,preview}.glb`
  (e.g. `tripo-v3.1__character__2.model.glb`). Verify against `manifest.json → runs[].files.model.sha256`. Sizes range from ~0.9 MB (PB Slim) to ~66 MB (Hunyuan 3.1).
- Renders: `…/renders/<engine>__<input>__<n>__{front,back,left,right}.png` (216, 512²) and review sheets `…/renders/sheets/<engine>__<input>.jpg` (18) + `sheets.json`.

## Reproduce / re-check
Free, offline (no API key, no credits):
```
python3 bench.py --summarize results.json          # re-aggregates the published runs and prints the per-engine table
python3 -c "import json;print(len(json.load(open('manifest.json'))['runs']))"   # 54
curl -sO https://cdn.umodeler.com/media/benchmarks/2026-09-12-image-to-3d/tripo__stylized-prop__1.model.glb && shasum -a 256 tripo__stylized-prop__1.model.glb
```
`bench.py --summarize` accepts the published object form (`{ "runs": [...] }`) as well as the raw list the runner writes.

Re-running the generation (**paid**, needs a paid account's API key, ~21,500 credits and ~35 minutes):
```
export PICOBERRY_API_BASE=https://api.picoberry.ai PICOBERRY_API_KEY=pb_live_...
BENCH_REPEATS=3 BENCH_CONCURRENCY=6 python3 bench.py    # writes inputs/, results/, results.json, run-meta.json
```
With this folder's `inputs/*.webp` present and no `inputs/inputs.json`, the script reuses the published reference images as-is (correct MIME and
filename) instead of generating new ones; with neither, it generates fresh images from the prompts. Interrupted runs resume from `pending.json`
and adopt orphaned assets by engine + input hash instead of resubmitting, so a restart does not double-charge. Refusing to run without a key
is deliberate — the default invocation costs money.

Re-computing the Blender metrics and renders (free; needs Blender 4.2+ and the 54 `model.glb` files, e.g. from the CDN):
```
blender -b -P mesh_quality_blender.py -- ./glb ./quality            # writes quality/mesh-quality.json + quality/renders/*.png
python3 render_sheets.py ./quality ./sheets                           # 18 review sheets
python3 postprocess.py --summarize postprocess.json                   # re-aggregates the published post-processing results
python3 aggregate_followup.py                                          # rebuilds summary.json → quality / postprocess
```
Re-running the post-processing itself is paid (≈1,400 credits for the whole chain): `PICOBERRY_API_KEY=... python3 postprocess.py`.

Re-computing the file metrics from the published CDN copies (free, no key; ~1.8 GB of downloads):
```
python3 mesh-metrics.py results.json ./glb --cdn --keep     # SHA-256s must match manifest.json; triangles must equal results.json faces
```
(`PICOBERRY_API_KEY=... python3 mesh-metrics.py results.json ./glb` fetches the same files through the API's signed URLs instead — account that owns the assets only.)

## Corrections log
- 2026-09-12 02:35Z — the first published `results.json`/`summary.json` listed **text-to-3D** catalog prices (default `GET /v1/models`, `category=3d`)
  in the credits columns; image-to-3D prices differ for the Tripo engines (e.g. PB Standard mesh 120, not 60). Replaced with `?category=image-to-3d`
  values and reconciled to the 21,480-credit balance delta. Nothing else in the results changed.
- 2026-09-12 — added `manifest.json` (+ `manifest.py`), `mesh-metrics.json`, `mesh-metrics.py` (`--cdn` mode), the CDN copies of all 108 GLBs, per-engine `gltf` metrics in `summary.json`, and the free `bench.py --summarize` mode.
- 2026-09-13 — `summary.json → quality.duplicateVertexPctMedian` was duplicate vertices ÷ **faces**; renamed to `duplicateVertexPerFacePctMedian` and added `duplicateVertexPctOfVerticesMedian` (÷ the file's vertex count). Values unchanged otherwise.
- 2026-09-12 (later, second re-evaluation) — added the follow-up section above: Blender 5.1.2 import/topology/UV metrics, API remesh → UV unwrap on all 54 outputs and auto-rig on the 18 characters (paid, credits listed), 4-view renders and review sheets on the CDN, and written inspection notes. No output was regenerated.