mirror of
https://github.com/facebookresearch/pytorch3d.git
synced 2025-12-14 19:36:23 +08:00
Fix coordinate system conventions in renderer
Summary:
## Updates
- Defined the world and camera coordinates according to this figure. The world coordinates are defined as having +Y up, +X left and +Z in.
{F230888499}
- Removed all flipping from blending functions.
- Updated the rasterizer to return images with +Y up and +X left.
- Updated all the mesh rasterizer tests
- The expected values are now defined in terms of the default +Y up, +X left
- Added tests where the triangles in the meshes are non symmetrical so that it is clear which direction +X and +Y are
## Questions:
- Should we have **scene settings** instead of raster settings?
- To be more correct we should be [z clipping in the rasterizer based on the far/near clipping planes](https://github.com/ShichenLiu/SoftRas/blob/master/soft_renderer/cuda/soft_rasterize_cuda_kernel.cu#L400) - these values are also required in the blending functions so should we make these scene level parameters and have a scene settings tuple which is available to the rasterizer and shader?
Reviewed By: gkioxari
Differential Revision: D20208604
fbshipit-source-id: 55787301b1bffa0afa9618f0a0886cc681da51f3
This commit is contained in:
committed by
Facebook Github Bot
parent
767d68a3af
commit
15c72be444
Binary file not shown.
|
Before Width: | Height: | Size: 150 KiB After Width: | Height: | Size: 66 KiB |
BIN
docs/notes/assets/world_camera_image.png
Normal file
BIN
docs/notes/assets/world_camera_image.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 62 KiB |
@@ -34,19 +34,22 @@ The differentiable renderer API is experimental and subject to change!.
|
||||
|
||||
### Coordinate transformation conventions
|
||||
|
||||
Rendering requires transformations between several different coordinate frames: world space, view/camera space, NDC space and screen space. At each step it is important to know where the camera is located, how the x,y,z axes are aligned and the possible range of values. The following figure outlines the conventions used PyTorch3d.
|
||||
Rendering requires transformations between several different coordinate frames: world space, view/camera space, NDC space and screen space. At each step it is important to know where the camera is located, how the +X, +Y, +Z axes are aligned and the possible range of values. The following figure outlines the conventions used PyTorch3d.
|
||||
|
||||
<img src="assets/transformations_overview.png" width="1000">
|
||||
|
||||
|
||||
For example, given a teapot mesh, the world coordinate frame, camera coordiante frame and image are show in the figure below. Note that the world and camera coordinate frames have the +z direction pointing in to the page.
|
||||
|
||||
<img src="assets/world_camera_image.png" width="1000">
|
||||
|
||||
---
|
||||
|
||||
**NOTE: PyTorch3d vs OpenGL**
|
||||
|
||||
While we tried to emulate several aspects of OpenGL, the NDC coordinate system in PyTorch3d is **right-handed** compared with a **left-handed** NDC coordinate system in OpenGL (the projection matrix switches the handedness).
|
||||
|
||||
In OpenGL, the camera at the origin is looking along `-z` axis in camera space, but it is looking along the `+z` axis in NDC space.
|
||||
While we tried to emulate several aspects of OpenGL, there are differences in the coordinate frame conventions.
|
||||
- The default world coordinate frame in PyTorch3D has +Z pointing in to the screen whereas in OpenGL, +Z is pointing out of the screen. Both are right handed.
|
||||
- The NDC coordinate system in PyTorch3d is **right-handed** compared with a **left-handed** NDC coordinate system in OpenGL (the projection matrix switches the handedness).
|
||||
|
||||
<img align="center" src="assets/opengl_coordframes.png" width="300">
|
||||
|
||||
@@ -60,7 +63,7 @@ A renderer in PyTorch3d is composed of a **rasterizer** and a **shader**. Create
|
||||
from pytorch3d.renderer import (
|
||||
OpenGLPerspectiveCameras, look_at_view_transform,
|
||||
RasterizationSettings, BlendParams,
|
||||
MeshRenderer, MeshRasterizer, PhongShader
|
||||
MeshRenderer, MeshRasterizer, HardPhongShader
|
||||
)
|
||||
|
||||
# Initialize an OpenGL perspective camera.
|
||||
@@ -81,7 +84,7 @@ raster_settings = RasterizationSettings(
|
||||
# PhongShader, passing in the device on which to initialize the default parameters
|
||||
renderer = MeshRenderer(
|
||||
rasterizer=MeshRasterizer(cameras=cameras, raster_settings=raster_settings),
|
||||
shader=PhongShader(device=device, cameras=cameras)
|
||||
shader=HardPhongShader(device=device, cameras=cameras)
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user