Add MeshRasterizerOpenGL

Browse Source

Summary:
Adding MeshRasterizerOpenGL, a faster alternative to MeshRasterizer. The new rasterizer follows the ideas from "Differentiable Surface Rendering via non-Differentiable Sampling".

The new rasterizer 20x faster on a 2M face mesh (try pose optimization on Nefertiti from https://www.cs.cmu.edu/~kmcrane/Projects/ModelRepository/!). The larger the mesh, the larger the speedup.

There are two main disadvantages:
* The new rasterizer works with an OpenGL backend, so requires pycuda.gl and pyopengl installed (though we avoided writing any C++ code, everything is in Python!)
* The new rasterizer is non-differentiable. However, you can still differentiate the rendering function if you use if with the new SplatterPhongShader which we recently added to PyTorch3D (see the original paper cited above).

Reviewed By: patricklabatut, jcjohnson

Differential Revision: D37698816

fbshipit-source-id: 54d120639d3cb001f096237807e54aced0acda25

...

This commit is contained in:

Krzysztof Chalupka

2022-07-22 15:52:50 -07:00

committed by Facebook GitHub Bot

parent 36edf2b302

commit cb49550486

66 changed files with 1556 additions and 337 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

BIN

tests/data/test_cow_image_rectangle.png

Binary file not shown.

Side by Side Before Width:  |  Height:  |  Size: 32 KiB

BIN

tests/data/test_cow_image_rectangle_MeshRasterizer.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 97 KiB

BIN

tests/data/test_cow_image_rectangle_MeshRasterizerOpenGL.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 120 KiB

BIN

tests/data/test_joinatlas_1_MeshRasterizer.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 13 KiB

BIN

tests/data/test_joinatlas_1_MeshRasterizerOpenGL.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 35 KiB

BIN

tests/data/test_joinatlas_2_MeshRasterizer.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 15 KiB

BIN

tests/data/test_joinatlas_2_MeshRasterizerOpenGL.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 40 KiB

0

tests/data/test_joinatlas_final.png → tests/data/test_joinatlas_final_MeshRasterizer.png

Side by Side Swipe Overlay Before Width:  |  Height:  |  Size: 25 KiB After Width:  |  Height:  |  Size: 25 KiB

BIN

tests/data/test_joinatlas_final_MeshRasterizerOpenGL.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 72 KiB

BIN

tests/data/test_joined_spheres_splatter.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 23 KiB

BIN

tests/data/test_joinuvs0_MeshRasterizerOpenGL_final.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 16 KiB

0

tests/data/test_joinuvs0_final.png → tests/data/test_joinuvs0_MeshRasterizer_final.png

Side by Side Swipe Overlay Before Width:  |  Height:  |  Size: 12 KiB After Width:  |  Height:  |  Size: 12 KiB

BIN

tests/data/test_joinuvs1_MeshRasterizerOpenGL_final.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 16 KiB

0

tests/data/test_joinuvs1_final.png → tests/data/test_joinuvs1_MeshRasterizer_final.png

Side by Side Swipe Overlay Before Width:  |  Height:  |  Size: 12 KiB After Width:  |  Height:  |  Size: 12 KiB

BIN

tests/data/test_joinuvs2_MeshRasterizerOpenGL_final.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 16 KiB

0

tests/data/test_joinuvs2_final.png → tests/data/test_joinuvs2_MeshRasterizer_final.png

Side by Side Swipe Overlay Before Width:  |  Height:  |  Size: 12 KiB After Width:  |  Height:  |  Size: 12 KiB

0

tests/data/test_joinverts_final.png → tests/data/test_joinverts_final_MeshRasterizer.png

Side by Side Swipe Overlay Before Width:  |  Height:  |  Size: 11 KiB After Width:  |  Height:  |  Size: 11 KiB

BIN

tests/data/test_joinverts_final_MeshRasterizerOpenGL.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 15 KiB

0

tests/data/test_rasterized_sphere.png → tests/data/test_rasterized_sphere_MeshRasterizer.png

Side by Side Swipe Overlay Before Width:  |  Height:  |  Size: 2.2 KiB After Width:  |  Height:  |  Size: 2.2 KiB

BIN

tests/data/test_rasterized_sphere_MeshRasterizerOpenGL.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 2.2 KiB

BIN

tests/data/test_rasterized_sphere_zoom.png

Binary file not shown.

Side by Side Before Width:  |  Height:  |  Size: 3.0 KiB

BIN

tests/data/test_rasterized_sphere_zoom_MeshRasterizer.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 568 B

BIN

tests/data/test_rasterized_sphere_zoom_MeshRasterizerOpenGL.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 568 B

BIN

tests/data/test_simple_sphere_batched_flat_FoVPerspectiveCameras_0.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 22 KiB

BIN

tests/data/test_simple_sphere_batched_flat_FoVPerspectiveCameras_1.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 11 KiB

BIN

tests/data/test_simple_sphere_batched_flat_FoVPerspectiveCameras_2.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 7.0 KiB

BIN

tests/data/test_simple_sphere_batched_gouraud_FoVPerspectiveCameras_0.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 27 KiB

BIN

tests/data/test_simple_sphere_batched_gouraud_FoVPerspectiveCameras_1.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 10 KiB

BIN

tests/data/test_simple_sphere_batched_gouraud_FoVPerspectiveCameras_2.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 5.9 KiB

BIN

tests/data/test_simple_sphere_batched_phong_FoVPerspectiveCameras_0.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 29 KiB

BIN

tests/data/test_simple_sphere_batched_phong_FoVPerspectiveCameras_1.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 10 KiB

BIN

tests/data/test_simple_sphere_batched_phong_FoVPerspectiveCameras_2.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 5.8 KiB

BIN

tests/data/test_simple_sphere_batched_splatter_FoVPerspectiveCameras_0.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 31 KiB

BIN

tests/data/test_simple_sphere_batched_splatter_FoVPerspectiveCameras_1.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 11 KiB

BIN

tests/data/test_simple_sphere_batched_splatter_FoVPerspectiveCameras_2.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 6.3 KiB

BIN

tests/data/test_simple_sphere_light_splatter_FoVOrthographicCameras.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 25 KiB

BIN

tests/data/test_simple_sphere_light_splatter_FoVPerspectiveCameras.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 19 KiB

BIN

tests/data/test_simple_sphere_light_splatter_elevated_FoVOrthographicCameras.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 18 KiB

BIN

tests/data/test_simple_sphere_light_splatter_elevated_FoVPerspectiveCameras.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 12 KiB

BIN

tests/data/test_simple_sphere_outside_zfar_10000.png

Binary file not shown.

Side by Side Before Width:  |  Height:  |  Size: 5.7 KiB

BIN

tests/data/test_simple_sphere_outside_zfar_10000_MeshRasterizer.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 5.6 KiB

BIN

tests/data/test_simple_sphere_outside_zfar_10000_MeshRasterizerOpenGL.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 6.3 KiB

0

tests/data/test_simple_sphere_outside_zfar_100.png → tests/data/test_simple_sphere_outside_zfar_100_MeshRasterizer.png

Side by Side Swipe Overlay Before Width:  |  Height:  |  Size: 758 B After Width:  |  Height:  |  Size: 758 B

BIN

tests/data/test_simple_sphere_outside_zfar_100_MeshRasterizerOpenGL.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 758 B

0

tests/data/test_texture_atlas_8x8_back.png → tests/data/test_texture_atlas_8x8_back_MeshRasterizer.png

Side by Side Swipe Overlay Before Width:  |  Height:  |  Size: 31 KiB After Width:  |  Height:  |  Size: 31 KiB

BIN

tests/data/test_texture_atlas_8x8_back_MeshRasterizerOpenGL.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 34 KiB

0

tests/data/test_texture_map_back.png → tests/data/test_texture_map_back_MeshRasterizer.png

Side by Side Swipe Overlay Before Width:  |  Height:  |  Size: 31 KiB After Width:  |  Height:  |  Size: 31 KiB

BIN

tests/data/test_texture_map_back_MeshRasterizerOpenGL.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 34 KiB

0

tests/data/test_texture_map_front.png → tests/data/test_texture_map_front_MeshRasterizer.png

Side by Side Swipe Overlay Before Width:  |  Height:  |  Size: 30 KiB After Width:  |  Height:  |  Size: 30 KiB

BIN

tests/data/test_texture_map_front_MeshRasterizerOpenGL.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 32 KiB