mirror of
https://github.com/facebookresearch/pytorch3d.git
synced 2025-12-24 16:20:34 +08:00
Website and docs updates
Summary: - Added sbranson's fit mesh tutorial to the website - Updated rendering docs with info about texturing and new shader types. TODO: - add pointcloud rendering tutorial to the website as well (https://github.com/facebookresearch/pytorch3d/blob/master/docs/tutorials/render_colored_points.ipynb) - docs for camera - update some tutorials which depended on the Textures from structures. Reviewed By: gkioxari Differential Revision: D23143977 fbshipit-source-id: 6843c9bf3ce11115c459c64da5b0ad778dc92177
This commit is contained in:
committed by
Facebook GitHub Bot
parent
9a50cf800e
commit
d330765847
@@ -9,7 +9,7 @@ sidebar_label: Getting Started
|
||||
|
||||
The renderer is designed to be modular, extensible and support batching and gradients for all inputs. The following figure describes all the components of the rendering pipeline.
|
||||
|
||||
<img src="assets/architecture_overview.png" width="1000">
|
||||
<img src="assets/architecture_renderer.jpg" width="1000">
|
||||
|
||||
##### Fragments
|
||||
|
||||
@@ -36,7 +36,7 @@ The differentiable renderer API is experimental and subject to change!.
|
||||
|
||||
Rendering requires transformations between several different coordinate frames: world space, view/camera space, NDC space and screen space. At each step it is important to know where the camera is located, how the +X, +Y, +Z axes are aligned and the possible range of values. The following figure outlines the conventions used PyTorch3D.
|
||||
|
||||
<img src="assets/transformations_overview.png" width="1000">
|
||||
<img src="assets/transforms_overview.jpg" width="1000">
|
||||
|
||||
|
||||
For example, given a teapot mesh, the world coordinate frame, camera coordiante frame and image are show in the figure below. Note that the world and camera coordinate frames have the +z direction pointing in to the page.
|
||||
@@ -54,6 +54,17 @@ While we tried to emulate several aspects of OpenGL, there are differences in th
|
||||
<img align="center" src="assets/opengl_coordframes.png" width="300">
|
||||
|
||||
---
|
||||
|
||||
### Texturing options
|
||||
|
||||
For mesh texturing we offer several options (in `pytorch3d/renderer/mesh/texturing.py`):
|
||||
|
||||
1. **Vertex Textures**: D dimensional textures for each vertex (for example an RGB color) which can be interpolated across the face. This can be represented as an `(N, V, D)` tensor. This is a fairly simple representation though and cannot model complex textures if the mesh faces are large.
|
||||
2. **UV Textures**: vertex UV coordinates and **one** texture map for the whole face. For a point on a face with given barycentric coordinates, the face color can be computed by interpolating the vertex uv coordinates and then sampling from the texture map. This representation requires two tensors (UVs: `(N, V, 2), Texture map: `(N, H, W, 3)`), and is limited to only support one texture map per mesh.
|
||||
3. **Face Textures**: In more complex cases such as ShapeNet meshes, there are multiple texture maps per mesh and some faces have texture while other do not. For these cases, a more flexible representation is a texture atlas, where each face is represented as an `(RxR)` texture map where R is the texture resolution. For a given point on the face, the texture value can be sampled from the per face texture map using the barycentric coordinates of the point. This representation requires one tensor of shape `(N, F, R, R, 3)`. This texturing method is inspired by the SoftRasterizer implementation. For more details refer to the [`make_material_atlas`](https://github.com/facebookresearch/pytorch3d/blob/master/pytorch3d/io/mtl_io.py#L123) and [`sample_textures`](https://github.com/facebookresearch/pytorch3d/blob/master/pytorch3d/renderer/mesh/textures.py#L452) functions.
|
||||
|
||||
<img src="assets/texturing.jpg" width="1000">
|
||||
|
||||
### A simple renderer
|
||||
|
||||
A renderer in PyTorch3D is composed of a **rasterizer** and a **shader**. Create a renderer in a few simple steps:
|
||||
@@ -77,7 +88,6 @@ raster_settings = RasterizationSettings(
|
||||
image_size=512,
|
||||
blur_radius=0.0,
|
||||
faces_per_pixel=1,
|
||||
bin_size=0
|
||||
)
|
||||
|
||||
# Create a phong renderer by composing a rasterizer and a shader. Here we can use a predefined
|
||||
@@ -99,13 +109,11 @@ A shader can incorporate several steps:
|
||||
|
||||
We have examples of several combinations of these functions based on the texturing/shading/blending support we have currently. These are summarised in this table below. Many other combinations are possible and we plan to expand the options available for texturing, shading and blending.
|
||||
|
||||
|
||||
|Example Shaders | Vertex Textures| Texture Map| Flat Shading| Gouraud Shading| Phong Shading | Hard blending | Soft Blending |
|
||||
| ------------- |:-------------: | :--------------:| :--------------:| :--------------:| :--------------:|:--------------:|:--------------:|
|
||||
| HardPhongShader | :heavy_check_mark: |||| :heavy_check_mark: | :heavy_check_mark:||
|
||||
| SoftPhongShader | :heavy_check_mark: |||| :heavy_check_mark: | | :heavy_check_mark:|
|
||||
| HardGouraudShader | :heavy_check_mark: ||| :heavy_check_mark: || :heavy_check_mark:||
|
||||
| SoftGouraudShader | :heavy_check_mark: ||| :heavy_check_mark: ||| :heavy_check_mark:|
|
||||
| TexturedSoftPhongShader || :heavy_check_mark: ||| :heavy_check_mark: || :heavy_check_mark:|
|
||||
| HardFlatShader | :heavy_check_mark: || :heavy_check_mark: ||| :heavy_check_mark:||
|
||||
| SoftSilhouetteShader ||||||| :heavy_check_mark:|
|
||||
|Example Shaders | Vertex Textures| UV Textures | Textures Atlas | Flat Shading| Gouraud Shading| Phong Shading | Hard blending | Soft Blending |
|
||||
| ------------- |:-------------: | :--------------:| :--------------:| :--------------:| :--------------:| :--------------:|:--------------:|:--------------:|
|
||||
| HardPhongShader | ✔️ |✔️|✔️||| ✔️ | ✔️||
|
||||
| SoftPhongShader | ✔️ |✔️|✔️||| ✔️ | | ✔️|
|
||||
| HardGouraudShader | ✔️ |✔️|✔️|| ✔️ || ✔️||
|
||||
| SoftGouraudShader | ✔️ |✔️|✔️|| ✔️ ||| ✔️|
|
||||
| HardFlatShader | ✔️ |✔️|✔️| ✔️ ||| ✔️||
|
||||
| SoftSilhouetteShader |||||||| ✔️|
|
||||
|
||||
Reference in New Issue
Block a user