a5093b158b
Summary: Fixes https://github.com/facebookresearch/pytorch3d/issues/1934. Two independent bugs in the CPU backend. ### 1. Degenerate-triangle filter discards valid faces `marching_cubes_cpu.cpp`, `marching_cubes.py` `tri.clear()` and `ps.clear()` sit inside the degeneracy check, so the buffers only reset when a triangle is *accepted*. Once a cube's first triangle is degenerate, `ps[0..2]` stay frozen on it, and every subsequent triangle in that cube fails the same stale check and is dropped. Fixed by gating on `ps.size() == 3` and clearing unconditionally. The old code could only ever drop faces, never emit incorrect ones, so this is strictly additive. ### 2. Edge hash computed in float32 `marching_cubes_utils.h` `p[v].x/y/z` hold integral coordinates but are stored as `float`, so `x + y*W + z*W*H` evaluates entirely in float32 before truncating to `int`. float32 is exact only to 2²⁴ − 1 = 16,777,215 — and 256³ maxes out at exactly that value. At 512³ the maximum id is 134,217,727, where float32 spacing is 8, so distinct vertices collide on one id and `uniq_edge_id` merges them. Fixed by widening `W/H/D` to `int64_t` and casting each coordinate before multiplying. Also tightens the stride from `(W + W*H + W*H*D)` to `W*H*D`. Raises the CPU ceiling from 256³ to 1448³. Scope is the CPU path only. `marching_cubes_naive` was never affected (Python ints are arbitrary-precision), and neither was CUDA: `hashVpair` there computes ids in `uint` rather than `float`, so it has no 2²⁴ cliff, and `MarchingCubes` already rejects volumes above 1024³ before the CUDA kernel runs. The new `TORCH_CHECK` bound and the "~1448³" note in the new comments describe `MarchingCubesCpu` only. ### Verification Ellipsoid SDF (0.1, 1, 1), `isolevel=0.0`, identical input tensors on both devices. | Resolution | CUDA V | CUDA F | CPU V | CPU F | Degenerate dropped | | -- | -- | -- | -- | -- | -- | | 32³ | 1,664 | 3,324 | 1,664 | 3,324 | 0 | | 64³ | 7,312 | 14,620 | 7,312 | 14,620 | 0 | | 128³ | 30,168 | 60,332 | 30,168 | 60,332 | 0 | | 256³ | 122,448 | 244,892 | 122,448 | 243,996 | 896 | | 512³ | 491,944 | 983,884 | 491,944 | 976,140 | 7,744 | Machine: Arch Linux, RTX 4080, Ryzen 7 7800X3D Vertex counts now match CUDA exactly at every resolution; 512³ previously produced 176,121. The remaining face gap is entirely degenerate geometry — the CUDA mesh at 512³ contains exactly 7,744 zero-area triangles. ### Tests `test_degenerate_triangle_keeps_later_faces` — a 2×2×2 volume at `isolevel=1` chosen so the cube's four candidate triangles collapse onto its four outside corners: triangles 1 and 4 become degenerate, 2 and 3 stay valid. Pre-fix, the first degeneracy suppresses the rest and the mesh comes back empty; post-fix it is the expected quad. Asserts both `marching_cubes_naive` and the C++ extension. `test_large_grid_edge_ids` — a 2×2×4,200,000 volume (~67MB, ~0.1s) holding 16 isolated interior points on the highest-id grid row, positioned so grid-point ids straddle 2²⁴. Each point cuts exactly four grid edges, so the 64-vertex expectation is derived geometrically rather than copied from output. Pre-fix, a hash collision merges two edges and one vertex is lost. All 26 pre-existing tests in `test_marching_cubes.py` pass **unchanged**. That includes `test_cube_no_duplicate_verts` (`isolevel=1`) and `test_sphere` (`isolevel=64`), which both exercise the degenerate path but whose output is identical before and after the fix — so no existing expectation was edited and `sphere_level64.pickle` does not need regenerating. The imported diff contained no test file. The two tests above were written during import and differ from the tests described in the upstream PR description. *Analysis and write-up done collaboratively with AI, figures from testing are done on my own machine and have been checked.* Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/2043 Test Plan: ``` buck2 test fbcode//vision/fair/pytorch3d:tests -- --regex 'test_marching_cubes' ``` `Pass 28. Fail 0.` — 26 pre-existing tests plus the 2 new regression tests. Reverting all three source hunks to their pre-fix state and re-running the same command: both new tests fail (`test_degenerate_triangle_keeps_later_faces` returns an empty mesh instead of 4 verts / 2 faces; `test_large_grid_edge_ids` returns 31 verts instead of 32) and all 26 pre-existing tests still pass. The new tests are therefore pinned to exactly this change, and the change breaks nothing that was already covered. Reviewed By: MichaelRamamonjisoa Differential Revision: D115424433 Pulled By: bottler fbshipit-source-id: 547a260010b94253f52f3a3c223d4c4fa78a7ee6
Introduction
PyTorch3D provides efficient, reusable components for 3D Computer Vision research with PyTorch.
Key features include:
- Data structure for storing and manipulating triangle meshes
- Efficient operations on triangle meshes (projective transformations, graph convolution, sampling, loss functions)
- A differentiable mesh renderer
- Implicitron, see its README, a framework for new-view synthesis via implicit representations. (blog post)
PyTorch3D is designed to integrate smoothly with deep learning methods for predicting and manipulating 3D data. For this reason, all operators in PyTorch3D:
- Are implemented using PyTorch tensors
- Can handle minibatches of hetereogenous data
- Can be differentiated
- Can utilize GPUs for acceleration
Within FAIR, PyTorch3D has been used to power research projects such as Mesh R-CNN.
See our blog post to see more demos and learn about PyTorch3D.
Installation
For detailed instructions refer to INSTALL.md.
License
PyTorch3D is released under the BSD License.
Tutorials
Get started with PyTorch3D by trying one of the tutorial notebooks.
 |
 |
|---|---|
| Deform a sphere mesh to dolphin | Bundle adjustment |
 |
 |
|---|---|
| Render textured meshes | Camera position optimization |
 |
 |
|---|---|
| Render textured pointclouds | Fit a mesh with texture |
 |
 |
|---|---|
| Render DensePose data | Load & Render ShapeNet data |
 |
 |
|---|---|
| Fit Textured Volume | Fit A Simple Neural Radiance Field |
|
 |
|---|---|
| Fit Textured Volume in Implicitron | Implicitron Config System |
Documentation
Learn more about the API by reading the PyTorch3D documentation.
We also have deep dive notes on several API components:
Overview Video
We have created a short (~14 min) video tutorial providing an overview of the PyTorch3D codebase including several code examples. Click on the image below to watch the video on YouTube:
Development
We welcome new contributions to PyTorch3D and we will be actively maintaining this library! Please refer to CONTRIBUTING.md for full instructions on how to run the code, tests and linter, and submit your pull requests.
Development and Compatibility
mainbranch: actively developed, without any guarantee, Anything can be broken at any time- REMARK: this includes nightly builds which are built from
main - HINT: the commit history can help locate regressions or changes
- REMARK: this includes nightly builds which are built from
- backward-compatibility between releases: no guarantee. Best efforts to communicate breaking changes and facilitate migration of code or data (incl. models).
Contributors
PyTorch3D is written and maintained by the Facebook AI Research Computer Vision Team.
In alphabetical order:
- Amitav Baruah
- Steve Branson
- Krzysztof Chalupka
- Jiali Duan
- Luya Gao
- Georgia Gkioxari
- Taylor Gordon
- Justin Johnson
- Patrick Labatut
- Christoph Lassner
- Wan-Yen Lo
- David Novotny
- Nikhila Ravi
- Jeremy Reizenstein
- Dave Schnizlein
- Roman Shapovalov
- Olivia Wiles
Citation
If you find PyTorch3D useful in your research, please cite our tech report:
@article{ravi2020pytorch3d,
author = {Nikhila Ravi and Jeremy Reizenstein and David Novotny and Taylor Gordon
and Wan-Yen Lo and Justin Johnson and Georgia Gkioxari},
title = {Accelerating 3D Deep Learning with PyTorch3D},
journal = {arXiv:2007.08501},
year = {2020},
}
If you are using the pulsar backend for sphere-rendering (the PulsarPointRenderer or pytorch3d.renderer.points.pulsar.Renderer), please cite the tech report:
@article{lassner2020pulsar,
author = {Christoph Lassner and Michael Zollh\"ofer},
title = {Pulsar: Efficient Sphere-based Neural Rendering},
journal = {arXiv:2004.07484},
year = {2020},
}
News
Please see below for a timeline of the codebase updates in reverse chronological order. We are sharing updates on the releases as well as research projects which are built with PyTorch3D. The changelogs for the releases are available under Releases, and the builds can be installed using conda as per the instructions in INSTALL.md.
[Oct 31st 2023]: PyTorch3D v0.7.5 released.
[May 10th 2023]: PyTorch3D v0.7.4 released.
[Apr 5th 2023]: PyTorch3D v0.7.3 released.
[Dec 19th 2022]: PyTorch3D v0.7.2 released.
[Oct 23rd 2022]: PyTorch3D v0.7.1 released.
[Aug 10th 2022]: PyTorch3D v0.7.0 released with Implicitron and MeshRasterizerOpenGL.
[Apr 28th 2022]: PyTorch3D v0.6.2 released
[Dec 16th 2021]: PyTorch3D v0.6.1 released
[Oct 6th 2021]: PyTorch3D v0.6.0 released
[Aug 5th 2021]: PyTorch3D v0.5.0 released
[Feb 9th 2021]: PyTorch3D v0.4.0 released with support for implicit functions, volume rendering and a reimplementation of NeRF.
[November 2nd 2020]: PyTorch3D v0.3.0 released, integrating the pulsar backend.
[Aug 28th 2020]: PyTorch3D v0.2.5 released
[July 17th 2020]: PyTorch3D tech report published on ArXiv: https://arxiv.org/abs/2007.08501
[April 24th 2020]: PyTorch3D v0.2.0 released
[March 25th 2020]: SynSin codebase released using PyTorch3D: https://github.com/facebookresearch/synsin
[March 8th 2020]: PyTorch3D v0.1.1 bug fix release
[Jan 23rd 2020]: PyTorch3D v0.1.0 released. Mesh R-CNN codebase released: https://github.com/facebookresearch/meshrcnn