mirror of
https://github.com/facebookresearch/pytorch3d.git
synced 2026-09-07 15:55:52 +08:00
e73a7e7bfc
Summary: Fixes the coordinate indexing used by frustum face culling, and keeps the now-live culling from deleting faces that straddle the camera plane. `face_verts` has shape `[F, 3, 3]`, where the last two dimensions are the vertex and the xyz coordinate. `_get_culled_faces` indexed it as `face_verts[:, axis]`, which picks one whole vertex out of every face rather than one coordinate out of all three vertices, so `verts_clipped.sum(1) == 3` was asking whether a single vertex was outside the plane on all three axes at once. Every other access in the file agrees with the documented layout — `clip_faces` reads z as `face_verts_unclipped[:, :, 2]`. The fix selects the coordinate with `face_verts[:, :, axis]` and reduces with `all(dim=1)`, which states the documented condition directly: a face is culled only when all three of its vertices lie outside the same plane. Because the old indexing almost never fired, this is the first time frustum culling does real work, and that exposes a second problem. `_get_culled_faces` runs on the unclipped face verts, and `rasterize_meshes` passes `left`/`right`/`top`/`bottom` in NDC while z stays in world space. The perspective divide mirrors vertices behind the camera through the origin, so a triangle straddling the camera plane can have all 3 projected vertices outside the same xy plane while the part of it in front of the camera still crosses the frustum. At fov 90, for instance, the vertices `(-1.1, 0, 1)`, `(100, 0, -1)` and `(100, 0.1, -1)` all project to x < -1, yet the edge from the first to the second passes through x = 0 while still at z > 0. Culled faces are classified as case 2 and dropped outright, before z clipping could have salvaged them, so that triangle would vanish from the render. xy culling is therefore now applied only to faces lying entirely in front of the clipping plane (`z >= z_clip_value`, or `z > 0` when no clip value is set), and only when `perspective_correct` is set; straddling faces are left to the z clipping step. Orthographic projections keep usable xy coordinates behind the camera and are unaffected, as is culling on the z axis, which uses world coordinates throughout. The `frustum.cull` check also moves out of the per-plane loop into an early return. For blast radius: `RasterizationSettings.cull_to_frustum` defaults to `False`, but `rasterize_meshes_python` defaults it to `True`. Fixes https://github.com/facebookresearch/pytorch3d/issues/1936. Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/2044 Test Plan: `buck2 test fbcode//vision/fair/pytorch3d:tests`, filtered to the culling and rendering tests (`--regex 'test_render_meshes_clipped|test_render_meshes\b|clip'`): 34 passed, 0 failed, 8 skipped. The skips are the OpenGL-only tests. Two new unit tests over `_get_culled_faces`: - `test_cull_faces_uses_coordinate_axis` — for each of the 6 planes, a face whose 3 vertices all lie outside it is culled; and a face intersecting the left plane stays visible for 3 different vertex orders. Fails on the old `face_verts[:, axis]` indexing. - `test_cull_faces_straddling_perspective_camera` — a face with one vertex in front of the camera and two behind it, all 3 projecting outside the left plane, is not culled under a perspective projection (with and without a `z_clip_value`) but is culled under an orthographic one; and z-axis culling still fires for a face lying entirely behind `znear`. Negative control: with the xy gating disabled and nothing else changed, this test fails on 2 assertions, so it pins the new behaviour rather than restating it. `arc lint` on `pytorch3d/renderer/mesh/clip.py` and `tests/test_render_meshes_clipped.py`: no issues. Reviewed By: MichaelRamamonjisoa Differential Revision: D117200371 Pulled By: bottler fbshipit-source-id: 99422fc69f4f5725bd82f6af205a0a333a4aef40