On November 12, 2026 the authors Suvaditya Mukherjee, merve and Pedro Cuenca published a technical guide and reusable artifacts to fine-tune TRELLIS.2 on a category-specific 3D dataset. The workflow targets geometry errors in general-purpose generative models by training low-rank adapters (LoRA) on a footwear subset of the Google Scanned Objects (GSO) dataset.
How TRELLIS.2 generates 3D and where adapters are added
The TRELLIS.2 pipeline first removes image backgrounds with BiRefNet, then encodes images using DINOv3-L into feature tokens. A sparse-structure flow DiT predicts coarse shape into a small latent, decoded by a VAE into a 64×64×64 occupancy grid. The shape flow DiT refines geometry into O-Voxel at up to 512×512×512 resolution. A texture flow DiT produces material channels that are decoded into per-voxel color/metallic/roughness/alpha and exported as a textured GLB.
The authors add LoRA adapters to the sparse-structure flow DiT and the shape flow DiT because these modules control object geometry. They leave the texture flow DiT frozen, citing the dataset’s lack of separate material maps and the risk of learning baked lighting as surface properties.
Fine-tuning setup and choices
The report explains why LoRA is preferred for a small training set: the footwear subset has 188 objects (179 for training, 9 held out for evaluation) and LoRA reduces trainable parameters and memory compared with full fine-tuning. The configured adapters total about 9.3 million parameters: rank-8 adapters are used in attention and MLP layers of the sparse-structure flow DiT, and rank-16 adapters in the attention layers of the shape flow DiT.
Preprocessing converts GLB/OBJ meshes into TRELLIS.2 latents and renders 16 views per object in headless Blender; the authors report that preprocessing the 188 objects took about one hour on an a100-large (80 GB) job. Training runs use container images provided for preprocessing and fine-tuning and can run on Hugging Face Jobs or locally with Docker. Examples in the guide show hf jobs and docker commands and recommend storing preprocessed latents for reuse.
The authors also note practical requirements: an HF_TOKEN and account access to gated models such as DINOv3 where applicable. Trained LoRA adapters and related materials are published in the repository suvadityamuk/trellis-2-finetune-gso.
Original source: Hugging Face Blog