Summary: Fixes the index out of bound errors for texture sampling from a texture atlas: when barycentric coordinates are 1.0, the integer index into the (R, R) per face texture map is R (max can only be R-1).
Reviewed By: gkioxari
Differential Revision: D25543803
fbshipit-source-id: 82d0935b981352b49c1d95d5a17f9cc88bad0a82
Summary:
There are a couple of options for supporting non square images:
1) NDC stays at [-1, 1] in both directions with the distance calculations all modified by (W/H). There are a lot of distance based calculations (e.g. triangle areas for barycentric coordinates etc) so this requires changes in many places.
2) NDC is scaled by (W/H) so the smallest side has [-1, 1]. In this case none of the distance calculations need to be updated and only the pixel to NDC calculation needs to be modified.
I decided to go with option 2 after trying option 1!
API Changes:
- Image size can now be specified optionally as a tuple
TODO:
- add a benchmark test for the non square case.
Reviewed By: jcjohnson
Differential Revision: D24404975
fbshipit-source-id: 545efb67c822d748ec35999b35762bce58db2cf4
Summary: As mentioned in a comment on https://github.com/facebookresearch/pytorch3d/issues/438, curl must be told to follow redirects.
Reviewed By: nikhilaravi
Differential Revision: D24870138
fbshipit-source-id: 0c8aeb5146f8699bcea03d4108276fc24e9eab6b
Summary:
We now require CUB for building, here we make the tutorials include it.
Also make the installation cell do nothing if it has already succeeded.
I use curl not wget, and `os.environ` to set the variables not shell methods, because they are more likely to work on Windows.
Reviewed By: nikhilaravi
Differential Revision: D24860574
fbshipit-source-id: 5be86af15e53f8db016ee0e96fb43153bd69adbc
Summary:
Changes to CI and some minor fixes now that pulsar is part of pytorch3d. Most significantly, add CUB to CI builds.
Make CUB_HOME override the CUB already in cudatoolkit (important for cuda11.0 which uses cub 1.9.9 which pulsar doesn't work well with.
Make imageio available for testing.
Lint fixes.
Fix some test verbosity.
Avoid use of atomicAdd_block on older GPUs.
Reviewed By: nikhilaravi, classner
Differential Revision: D24773716
fbshipit-source-id: 2428356bb2e62735f2bc0c15cbe4cff35b1b24b8
Summary: This commit performs pulsar example and test refinements. The examples are fully adjusted to adhere to PEP style guide and additional comments are added.
Reviewed By: nikhilaravi
Differential Revision: D24723391
fbshipit-source-id: 6d289006f080140159731e7f3a8c98b582164f1a
Summary: This diff updates the documentation and tutorials with information about the new pulsar backend. For more information about the pulsar backend, see the release notes and the paper (https://arxiv.org/abs/2004.07484). For information on how to use the backend, see the point cloud rendering notebook and the examples in the folder docs/examples.
Reviewed By: nikhilaravi
Differential Revision: D24498129
fbshipit-source-id: e312b0169a72b13590df6e4db36bfe6190d742f9
Summary:
This diff integrates the pulsar renderer source code into PyTorch3D as an alternative backend for the PyTorch3D point renderer. This diff is the first of a series of three diffs to complete that migration and focuses on the packaging and integration of the source code.
For more information about the pulsar backend, see the release notes and the paper (https://arxiv.org/abs/2004.07484). For information on how to use the backend, see the point cloud rendering notebook and the examples in the folder `docs/examples`.
Tasks addressed in the following diffs:
* Add the PyTorch3D interface,
* Add notebook examples and documentation (or adapt the existing ones to feature both interfaces).
Reviewed By: nikhilaravi
Differential Revision: D23947736
fbshipit-source-id: a5e77b53e6750334db22aefa89b4c079cda1b443
Summary: We envision `pytorch3d.vis` to contain submodules with different dependencies. Allow (and require) them to be imported independently.
Reviewed By: theschnitz
Differential Revision: D24622519
fbshipit-source-id: 44840f70f5fd2bd410405bf09546024e48238744
Summary: New methods to directly plot a TexturesUV map with its used points, using PIL and matplotlib.
Reviewed By: gkioxari
Differential Revision: D23782968
fbshipit-source-id: 692970857b5be13a35a3175dc82ac03963a73555
Summary: We were importing torch, torchvision, PyTorch3D, and sys twice. This is just removing the duplicate (unneeded) imports
Reviewed By: theschnitz
Differential Revision: D24479270
fbshipit-source-id: 1048732f65242eb776c3eef537cb1ae58815c1eb
Summary: Take in a renderer with camera(s) and render the cameras as wireframes in the corresponding plotly plots
Reviewed By: nikhilaravi
Differential Revision: D24151706
fbshipit-source-id: f8e86d61f3d991500bafc0533738c79b96bda630
Summary: Change the two affected tutorials to use plot_scene and plot_batch_individually.
Reviewed By: nikhilaravi
Differential Revision: D24235480
fbshipit-source-id: ca9d73bfb7ccf733efaf16299c15927406d7f2aa
Summary: Importing from pytorch3d.visualization is wordy, so shortened the path to the vis module and updated the relevant imports.
Reviewed By: nikhilaravi
Differential Revision: D24116527
fbshipit-source-id: e0e4da7d48c5afedec07482d7be43362b6822445
Summary: Add markdown note explaining why PyTorch3D has plotly visualizations, examples, and how to save these visualizations as an image.
Reviewed By: nikhilaravi
Differential Revision: D23976283
fbshipit-source-id: cbbaffd1f0ebe3466841e42fdb454d85773152cd
Summary: Add examples of using the Plotly visualization functions to the corresponding tutorial notebooks.
Reviewed By: nikhilaravi
Differential Revision: D23879109
fbshipit-source-id: ea8c45aa6c828eb2f6ea2ae1c8846adc486f92e0
Summary:
Add a notebook demonstrating how to use Pytorch3D to render a textured mesh with the DensePose textures and the SMPL model
{F336408690}
Reviewed By: nikhilaravi
Differential Revision: D23784314
fbshipit-source-id: c92f32fb9b9468eb7ec26bf58dcabb1f26d92e7b
Summary:
Add cells to the rendering_colored_points tutorial showing how to initialize a renderer and compositor that will render pointclouds with a background color.
{F333731292}
{F334136799}
Reviewed By: nikhilaravi
Differential Revision: D23632503
fbshipit-source-id: e9ce0178b41e74baf912bd82ca1db41b680fc68f
Summary: Previously the tutorial code assumed that the reference image had a black background, resulting in an empty silhouette mask. Since the background is white, this change allows the model to find the correct silhouette mask.
Reviewed By: nikhilaravi, sbranson
Differential Revision: D23502202
fbshipit-source-id: c3a570f93efd480323f27cb081db0a9fb54be219
Summary:
Update the installation cells to import torch.
Replace use of deprecated Textures in deform tutorial.
Correct the deform tutorial's idea of its own name.
Fix typo in batched part of render_textured_meshes which meant only one mesh was being rendered with a batch of cameras.
Add an error check in the rasterizer to make the error friendly from such a mistake elsewhere.
Reviewed By: gkioxari
Differential Revision: D23345462
fbshipit-source-id: 1d5bd25db052f7ef687b7168d7aee5cc4dce8952
Summary:
Add a document to explain how to run the tutorials.
Fix API of TexturesVertex in fit_textured_mesh.
Prepare cuda 10.1 wheels (not 10.2) for linux to be available on pypi - this matches what colab has.
Change the tutorials to use these new wheels.
Reviewed By: gkioxari
Differential Revision: D23324479
fbshipit-source-id: 60e92a3f46a2d878f811b7703638f8d1dae143d9
Summary:
Found a bug in extending textures with vertex uv coordinates. This was due to the padded -> list conversion of vertex uv coordinates i.e. The number of vertices in the mesh and in verts_uvs can differ
e.g. if a vertex is shared between 3 faces, it can
have up to 3 different uv coordinates. Therefore we cannot convert directly from padded to list using _num_verts_per_mesh
Reviewed By: bottler
Differential Revision: D23233595
fbshipit-source-id: 0c66d15baae697ead0bdc384f74c27d4c6539fc9
Summary:
- Added sbranson's fit mesh tutorial to the website
- Updated rendering docs with info about texturing and new shader types.
TODO:
- add pointcloud rendering tutorial to the website as well (https://github.com/facebookresearch/pytorch3d/blob/master/docs/tutorials/render_colored_points.ipynb)
- docs for camera
- update some tutorials which depended on the Textures from structures.
Reviewed By: gkioxari
Differential Revision: D23143977
fbshipit-source-id: 6843c9bf3ce11115c459c64da5b0ad778dc92177
Summary:
Refactor cameras
* CamerasBase was enhanced with `transform_points_screen` that transforms projected points from NDC to screen space
* OpenGLPerspective, OpenGLOrthographic -> FoVPerspective, FoVOrthographic
* SfMPerspective, SfMOrthographic -> Perspective, Orthographic
* PerspectiveCamera can optionally be constructred with screen space parameters
* Note on Cameras and coordinate systems was added
Reviewed By: nikhilaravi
Differential Revision: D23168525
fbshipit-source-id: dd138e2b2cc7e0e0d9f34c45b8251c01266a2063
Summary:
Because of the way Sphinx was parsing this link in Markdown, the link wasn't working properly. This should fix it.
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/330
Test Plan: Tested via local Sphinx.
Reviewed By: nikhilaravi
Differential Revision: D23244163
Pulled By: atroyn
fbshipit-source-id: 019712a841d76391a5210dcd98c77a822947204a
Summary:
A fairly big refactor of the texturing API with some breaking changes to how textures are defined.
Main changes:
- There are now 3 types of texture classes: `TexturesUV`, `TexturesAtlas` and `TexturesVertex`. Each class:
- has a `sample_textures` function which accepts the `fragments` from rasterization and returns `texels`. This means that the shaders will not need to know the type of the mesh texture which will resolve several issues people were reporting on GitHub.
- has a `join_batch` method for joining multiple textures of the same type into a batch
Reviewed By: gkioxari
Differential Revision: D21067427
fbshipit-source-id: 4b346500a60181e72fdd1b0dd89b5505c7a33926
Summary: Tutorial showing how to create a synthetic dataset by rendering a cow from multiple views, fit a mesh using a differential silhouette renderer, then fit a mesh using an RGB renderer.
Reviewed By: nikhilaravi
Differential Revision: D22513859
fbshipit-source-id: 24bdaac4ebec6dd01f98e2f5c702065f9335ff33
Summary: It's easier to understand code changes if we don't store the output in the notebooks. This is just a run of nbstripout on all the notebooks.
Reviewed By: nikhilaravi
Differential Revision: D22357375
fbshipit-source-id: bffdb426af2f676d448630d717658d609c224811
Summary: Fix to enable a mesh/point rasterizer to be initialized without having to specify the camera.
Reviewed By: jcjohnson, gkioxari
Differential Revision: D21362359
fbshipit-source-id: 4f84ea18ad9f179c7b7c2289ebf9422a2f5e26de
Summary: Update version number for version 0.2.0.
Reviewed By: nikhilaravi
Differential Revision: D21157358
fbshipit-source-id: 32a5b93e5dc65a31a806a5ce7231f8603fe02e85
Summary:
Fix a bug which resulted in a rendering artifacts if the image size was not a multiple of 16.
Fix: Revert coarse rasterization to original implementation and only update fine rasterization to reverse the ordering of Y and X axis. This is much simpler than the previous approach!
Additional changes:
- updated mesh rendering end-end tests to check outputs from both naive and coarse to fine rasterization.
- added pointcloud rendering end-end tests
Reviewed By: gkioxari
Differential Revision: D21102725
fbshipit-source-id: 2e7e1b013dd6dd12b3a00b79eb8167deddb2e89a
Summary: Remove `bin_size` and `max_faces_per_pixel` from being specified. This means the coarse-to-fine rasterization will be used by default and will help avoid confusion with the naive version.
Reviewed By: jcjohnson
Differential Revision: D20908905
fbshipit-source-id: c181c88e844d888aa81a36870918307961dc1175
Summary:
Add a note about the difference between naive and coarse-to-fine rasterization to all the rendering tutorials.
Update the render pointclouds tutorial to wget the data file.
Reviewed By: gkioxari
Differential Revision: D20575257
fbshipit-source-id: a2806b9452438f97cb754f87e011c6e32e2545e4
Summary:
A few small website updates:
- changed the tutorials to point to the `stable` tag on github so we don't have to update the website each time we want to run them!
- changed the colab button
- re ran notebook cells to update the images for textured meshes
Once these fixes are landed I can build and publish the website.
Reviewed By: bottler
Differential Revision: D20484836
fbshipit-source-id: 603a05e752f631c60d1a3abb9adeb1b9b451ab98
Summary: Use a consistent case for PyTorch3D (matching the logo...): replace all occurrences of PyTorch3d with PyTorch3D across the codebase (including documentation and notebooks)
Reviewed By: wanyenlo, gkioxari
Differential Revision: D20427546
fbshipit-source-id: 8c7697f51434c51e99b7fe271935932c72a1d9b9
Summary: The shebang line `#!<path to interpreter>` is only required for Python scripts, so remove it on source files for class or function definitions. Additionally explicitly mark as executable the actual Python scripts in the codebase.
Reviewed By: nikhilaravi
Differential Revision: D20095778
fbshipit-source-id: d312599fba485e978a243292f88a180d71e1b55a
Summary:
Applying the changes added for mesh rasterization to ensure that +Y is up and +X is left so that the coordinate system is right handed.
Also updated the diagram in the docs to indicate that (0,0) is in the top left hand corner.
Reviewed By: gkioxari
Differential Revision: D20394849
fbshipit-source-id: cfb7c79090eb1f55ad38b92327a74a70a8dc541e
