244 Commits

Author SHA1 Message Date
Nikhila Ravi
ebe2693b11 Support variable size radius for points in rasterizer
Summary:
Support variable size pointclouds in the renderer API to allow compatibility with Pulsar rasterizer.

If radius is provided as a float, it is converted to a tensor of shape (P). Otherwise radius is expected to be an (N, P_padded) dimensional tensor where P_padded is the max number of points in the batch (following the convention from pulsar: https://our.intern.facebook.com/intern/diffusion/FBS/browse/master/fbcode/frl/gemini/pulsar/pulsar/renderer.py?commit=ee0342850210e5df441e14fd97162675c70d147c&lines=50)

Reviewed By: jcjohnson, gkioxari

Differential Revision: D21429400

fbshipit-source-id: 65de7d9cd2472b27fc29f96160c33687e88098a2
2020-09-18 18:48:18 -07:00
z003yctd
e40c2167ae fix incorrect variable naming (#362)
Summary: Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/362

Reviewed By: bottler

Differential Revision: D23712242

Pulled By: nikhilaravi

fbshipit-source-id: 1c4184c8482049991356be7dbc9755b0c2018a1d
2020-09-17 16:50:34 -07:00
OrHayat
de77e426eb typo in docstring (#322)
Summary:
fixed small typo in update_padded function docstring.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/322

Reviewed By: gkioxari

Differential Revision: D23712324

Pulled By: nikhilaravi

fbshipit-source-id: fea3b68039644b236897c6f465cbb322c5c48085
2020-09-16 16:22:28 -07:00
Amitav Baruah
61121b9f2c Add example of rendering pointclouds with a background color.
Summary:
Add cells to the rendering_colored_points tutorial showing how to initialize a renderer and compositor that will render pointclouds with a background color.

{F333731292}
{F334136799}

Reviewed By: nikhilaravi

Differential Revision: D23632503

fbshipit-source-id: e9ce0178b41e74baf912bd82ca1db41b680fc68f
2020-09-14 10:39:08 -07:00
Amitav Baruah
872ff8c796 Add background color support to compositors
Summary: Support rendering different color backgrounds for pointclouds for both compositors

Reviewed By: nikhilaravi

Differential Revision: D23611043

fbshipit-source-id: ab029650d51349340372c5bd66700e6577d48851
2020-09-14 10:39:08 -07:00
David Novotny
dc40adfa24 Bugfix: Wrong default T in SfMCameras
Summary: Fixes a bug that initializes default SfMCameras with a default _R instead of default _T

Reviewed By: gkioxari

Differential Revision: D23654583

fbshipit-source-id: ccfb7235b2fb6df5a2e402b9fb4b194e97d78dc6
2020-09-11 15:32:25 -07:00
Amitav Baruah
f2eb34dc7a Correctly create reference image silhouette
Summary: Previously the tutorial code assumed that the reference image had a black background, resulting in an empty silhouette mask. Since the background is white, this change allows the model to find the correct silhouette mask.

Reviewed By: nikhilaravi, sbranson

Differential Revision: D23502202

fbshipit-source-id: c3a570f93efd480323f27cb081db0a9fb54be219
2020-09-10 12:55:32 -07:00
Amitav Baruah
eb517dd70b Fix look_at corner case
Summary: When the camera is vertically oriented, calculating the look_at x-axis (also known as the "right" vector) does not succeed, resulting in the x-axis being placed at the origin. Adds a check to correctly calculate the x-axis if this case occurs.

Reviewed By: nikhilaravi, sbranson

Differential Revision: D23511859

fbshipit-source-id: ee5145cdbecdbe2f7c7d288588bd0899480cb327
2020-09-10 12:53:08 -07:00
Steve Branson
f8ea5906c0 Fix softmax_rgb_blend() when mesh is outside zfar
Summary:
This fixes two small issues with blending.py:softmax_rgb_blend():
  1) zfar and znear attributes are propagated from the camera settings instead of just using default settings of znear=1.0 and zfar=100.0
  2) A check is added to prevent arithmetic overflow in softmax_rgb_blend()

This is a fix in response to https://github.com/facebookresearch/pytorch3d/issues/334
where meshes rendererd using a SoftPhongShader with faces_per_pixel=1 appear black.  This only occurs when the scale of the mesh is large (vertex values > 100, where 100 is the default value of zfar).  This fix allows the caller to increase the value of cameras.zfar to match the scale of her/his mesh.

Reviewed By: nikhilaravi

Differential Revision: D23517541

fbshipit-source-id: ab8631ce9e5f2149f140b67b13eff857771b8807
2020-09-09 13:22:25 -07:00
generatedunixname89002005307016
6eb158e548 suppress errors in vision/fair/pytorch3d
Differential Revision: D23521117

fbshipit-source-id: 8e11e91aaa2a91b7ca1fa4c00a3db86ab8648f7f
2020-09-03 18:17:49 -07:00
David Novotny
316b77782e Camera alignment
Summary:
adds `corresponding_cameras_alignment` function that estimates a similarity transformation between two sets of cameras.

The function is essential for computing camera errors in SfM pipelines.

```
Benchmark                                                   Avg Time(μs)      Peak Time(μs) Iterations
--------------------------------------------------------------------------------
CORRESPONDING_CAMERAS_ALIGNMENT_10_centers_False                32219           36211             16
CORRESPONDING_CAMERAS_ALIGNMENT_10_centers_True                 32429           36063             16
CORRESPONDING_CAMERAS_ALIGNMENT_10_extrinsics_False              5548            8782             91
CORRESPONDING_CAMERAS_ALIGNMENT_10_extrinsics_True               6153            9752             82
CORRESPONDING_CAMERAS_ALIGNMENT_100_centers_False               33344           40398             16
CORRESPONDING_CAMERAS_ALIGNMENT_100_centers_True                34528           37095             15
CORRESPONDING_CAMERAS_ALIGNMENT_100_extrinsics_False             5576            7187             90
CORRESPONDING_CAMERAS_ALIGNMENT_100_extrinsics_True              6256            9166             80
CORRESPONDING_CAMERAS_ALIGNMENT_1000_centers_False              32020           37247             16
CORRESPONDING_CAMERAS_ALIGNMENT_1000_centers_True               32776           37644             16
CORRESPONDING_CAMERAS_ALIGNMENT_1000_extrinsics_False            5336            8795             94
CORRESPONDING_CAMERAS_ALIGNMENT_1000_extrinsics_True             6266            9929             80
--------------------------------------------------------------------------------
```

Reviewed By: shapovalov

Differential Revision: D22946415

fbshipit-source-id: 8caae7ee365b304d8aa1f8133cf0dd92c35bc0dd
2020-09-03 13:27:14 -07:00
Adly Templeton
14f015d8bf Add pyre typeshed information for Tensor.ndim and nn.ConvTranspose2d
Summary: Adding some appropriate methods into pyre typeshed. Removing corresponding pyre-ignore and pyre-fixme messages.

Differential Revision: D22949138

fbshipit-source-id: add8acdd4611ab698954868832594d062cd58f88
2020-09-02 06:52:30 -07:00
Jeremy Reizenstein
701bbef4f3 version number for version 0.2.5
Reviewed By: jcjohnson

Differential Revision: D23244140

fbshipit-source-id: 8f95c817637162350b61b1b0fc3d2de66087250e
v0.2.5
2020-08-27 17:15:51 -07:00
Jeremy Reizenstein
7414904c90 Installation updates for next release.
Summary: We've made some build updates.

Reviewed By: jcjohnson

Differential Revision: D23244111

fbshipit-source-id: dba6e5dc3a9d7dcb2b62094f7f7b8b5abca190d9
2020-08-27 17:15:51 -07:00
Steve Branson
46c0e83461 Change default settings of clip_barycentric_coords
Summary: When raster_settings.clip_barycentric_coords is unspecified, clip_barycentric_coords is turned on if blur_radius is greater than 0.  This matches the behavior prior to D21705503 (cc70950f40).

Reviewed By: gkioxari

Differential Revision: D23375257

fbshipit-source-id: 4b87588aabb69d4d835d4dcceb11153628121d30
2020-08-27 16:44:12 -07:00
Nikhila Ravi
c25fd83694 small website updates
Summary: Small fixes to website rendering

Reviewed By: jcjohnson

Differential Revision: D23281746

fbshipit-source-id: c9dc8edd5e52f39d4e0e19f10ecc7e035b39feda
2020-08-26 09:21:43 -07:00
Jeremy Reizenstein
9aeb88b48e tutorial fixes. validate #cameras
Summary:
Update the installation cells to import torch.
Replace use of deprecated Textures in deform tutorial.
Correct the deform tutorial's idea of its own name.
Fix typo in batched part of render_textured_meshes which meant only one mesh was being rendered with a batch of cameras.
Add an error check in the rasterizer to make the error friendly from such a mistake elsewhere.

Reviewed By: gkioxari

Differential Revision: D23345462

fbshipit-source-id: 1d5bd25db052f7ef687b7168d7aee5cc4dce8952
2020-08-26 08:27:53 -07:00
Jeremy Reizenstein
32484500be tutorial fixes from #336. Wheels with cuda10.1.
Summary:
Add a document to explain how to run the tutorials.
Fix API of TexturesVertex in fit_textured_mesh.
Prepare cuda 10.1 wheels (not 10.2) for linux to be available on pypi - this matches what colab has.
Change the tutorials to use these new wheels.

Reviewed By: gkioxari

Differential Revision: D23324479

fbshipit-source-id: 60e92a3f46a2d878f811b7703638f8d1dae143d9
2020-08-25 13:11:10 -07:00
Jeremy Reizenstein
909dc83505 amalgamate meshes with texture into a single scene
Summary:
Add a join_scene method to all the textures to allow the join_mesh function to include textures. Rename the join_mesh function to join_meshes_as_scene.

For TexturesAtlas, we now interpolate if the user attempts to have the resolution vary across the batch. This doesn't look great if the resolution is already very low.

For TexturesUV, a rectangle packing function is required, this does something simple.

Reviewed By: gkioxari

Differential Revision: D23188773

fbshipit-source-id: c013db061a04076e13e90ccc168a7913e933a9c5
2020-08-25 11:28:40 -07:00
Jeremy Reizenstein
e25ccab3d9 align_corners and padding for TexturesUV
Summary:
Allow, and make default, align_corners=True for texture maps. Allow changing the padding_mode and set the default to be "border" which produces more logical results. Some new documentation.

The previous behavior corresponds to padding_mode="zeros" and align_corners=False.

Reviewed By: gkioxari

Differential Revision: D23268775

fbshipit-source-id: 58d6229baa591baa69705bcf97471c80ba3651de
2020-08-25 11:28:40 -07:00
Eduardo Henrique Arnold
d0cec028c7 Fix look_at_view_transform when object location is not (0,0,0) (#230)
Summary:
The look_at_view_transform did not give the correct results when the object location `at` was not (0,0,0).

The problem was on computing the cameras' location in world's coordinate `C`. It only took into account the camera position from spherical angles, but ignored the object location in the world's coordinate system. I simply modified the C tensor to take into account the object's location which is not necessarily in the origin.

I ran unit tests and all but 4 failed with the same error message: `RuntimeError: CUDA error: invalid device ordinal`. However the same happens before this patch, so I believe these errors are unrelated.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/230

Reviewed By: gkioxari

Differential Revision: D23278126

Pulled By: nikhilaravi

fbshipit-source-id: c06e891bc46de8222325ee7b37aa43cde44648e8
2020-08-21 23:43:47 -07:00
Nikhila Ravi
778383eef7 Texture loading and rendering in ShapeNetCore and R2N2 data loaders
Summary:
- Add support for loading textures from ShapeNet Obj files as a texture atlas.
- Support textured rendering of shapenet models

Reviewed By: gkioxari

Differential Revision: D23141143

fbshipit-source-id: 26eb81758d4cdbd6d820b072b58f5c6c08cb90bc
2020-08-21 20:42:17 -07:00
Nikhila Ravi
90f6a005b0 Tutorials textures updates and fix bug in extending meshes with uv textures
Summary:
Found a bug in extending textures with vertex uv coordinates. This was due to the padded -> list conversion of vertex uv coordinates i.e.                 The number of vertices in the mesh and in verts_uvs can differ
e.g. if a vertex is shared between 3 faces, it can
have up to 3 different uv coordinates. Therefore we cannot convert directly from padded to list using _num_verts_per_mesh

Reviewed By: bottler

Differential Revision: D23233595

fbshipit-source-id: 0c66d15baae697ead0bdc384f74c27d4c6539fc9
2020-08-21 19:20:09 -07:00
Nikhila Ravi
d330765847 Website and docs updates
Summary:
- Added sbranson's fit mesh tutorial to the website
- Updated rendering docs with info about texturing and new shader types.

TODO:
- add pointcloud rendering tutorial to the website as well (https://github.com/facebookresearch/pytorch3d/blob/master/docs/tutorials/render_colored_points.ipynb)
- docs for camera
- update some tutorials which depended on the Textures from structures.

Reviewed By: gkioxari

Differential Revision: D23143977

fbshipit-source-id: 6843c9bf3ce11115c459c64da5b0ad778dc92177
2020-08-21 19:17:16 -07:00
Jeremy Reizenstein
9a50cf800e Fix batching bug from TexturesUV packed ambiguity, other textures tidyup
Summary:
faces_uvs_packed and verts_uvs_packed were only used in one place and the definition of the former was ambiguous. This meant that the wrong coordinates could be used for meshes other than the first in the batch. I have therefore removed both functions and build their common result inline. Added a test that a simple batch of two meshes is rendered consistently with the rendering of each alone. This test would have failed before.

I hope this fixes https://github.com/facebookresearch/pytorch3d/issues/283.

Some other small improvements to the textures code.

Reviewed By: nikhilaravi

Differential Revision: D23161936

fbshipit-source-id: f99b560a46f6b30262e07028b049812bc04350a7
2020-08-21 05:53:29 -07:00
Steve Branson
9aaba0483c Temporary fix for mesh rasterization bug for traingles partially behind the camera
Summary: A triangle is culled if any vertex in a triangle is behind the camera.  This fixes incorrect rendering of triangles that are partially behind the camera, where screen coordinate calculations are strange.  It doesn't work for triangles that are partially behind the camera but still intersect with the view frustum.

Reviewed By: nikhilaravi

Differential Revision: D22856181

fbshipit-source-id: a9cbaa1327d89601b83d0dfd3e4a04f934a4a213
2020-08-20 22:24:19 -07:00
Georgia Gkioxari
57a22e7306 camera refactoring
Summary:
Refactor cameras
* CamerasBase was enhanced with `transform_points_screen` that transforms projected points from NDC to screen space
* OpenGLPerspective, OpenGLOrthographic -> FoVPerspective, FoVOrthographic
* SfMPerspective, SfMOrthographic -> Perspective, Orthographic
* PerspectiveCamera can optionally be constructred with screen space parameters
* Note on Cameras and coordinate systems was added

Reviewed By: nikhilaravi

Differential Revision: D23168525

fbshipit-source-id: dd138e2b2cc7e0e0d9f34c45b8251c01266a2063
2020-08-20 22:22:06 -07:00
mlygao@devgpu002.atn3.facebook.com
9242e7e65d ShapeNetCore & R2N2 tutorial
Summary: Tutorial for ShapeNetCore & R2N2.

Reviewed By: gkioxari

Differential Revision: D22916882

fbshipit-source-id: 752742be87f44919164ec7eafcc9c09c17a0f8a3
2020-08-20 17:57:06 -07:00
Anton Troynikov
370f1c380c Fix 404 link in Renderer Doc (#330)
Summary:
Because of the way Sphinx was parsing this link in Markdown, the link wasn't working properly. This should fix it.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/330

Test Plan: Tested via local Sphinx.

Reviewed By: nikhilaravi

Differential Revision: D23244163

Pulled By: atroyn

fbshipit-source-id: 019712a841d76391a5210dcd98c77a822947204a
2020-08-20 17:23:30 -07:00
root@sandcastle5743.frc3.facebook.com
f03aa5803b suppress errors in vision/fair/pytorch3d
Differential Revision: D23180198

fbshipit-source-id: cad1fa7ba9935f3ca20410a2575e173999c04be1
2020-08-17 18:08:19 -07:00
Georgia Gkioxari
7f2f95f225 detach for meshes, pointclouds, textures
Summary: Add `detach` for Meshes, Pointclouds, Textures

Reviewed By: nikhilaravi

Differential Revision: D23070418

fbshipit-source-id: 68671124ce114c4495d7ef3c944c9aac3d0db2d8
2020-08-17 14:55:54 -07:00
Nikhila Ravi
5852b74d12 Softmax blending small fix
Summary:
Small fix to the softmax blending function.

To avoid overflow in the exponential for the softmax, the exponent is shifted by the maximum value. In the final calculation of the color there is a weighted sum between the pixel color and the background color - in order for the sum to be correct, the background color also needs to be handled in the same way witt the shifted exponent.

Reviewed By: gkioxari

Differential Revision: D23148301

fbshipit-source-id: 86066586ee7d3ce7bd4a2076b12ce191fbd151a7
2020-08-17 11:59:53 -07:00
Nikhila Ravi
8e9ff15faf datasets.md (documentation for datasets)
Summary: documentation for datasets

Reviewed By: bottler, gkioxari

Differential Revision: D22992266

fbshipit-source-id: 44aaa8227af04c1baa5ea6c31ba131dea2b9675b
2020-08-14 11:27:50 -07:00
Patrick Labatut
6d76336501 Extract more reusable I/O functions
Summary: Continue extracting reusable I/O functions to a separate utils module (and remove duplication).

Reviewed By: nikhilaravi

Differential Revision: D20720433

fbshipit-source-id: e82b19560a5dc8a506c4c4d098da69c202790c4f
2020-08-11 15:51:30 -07:00
Luya Gao
63ba74f1a8 Return R2N2 voxel coordinates
Summary: Return R2N2's voxel coordinates.

Reviewed By: nikhilaravi

Differential Revision: D22462530

fbshipit-source-id: a995cfa0957b2561eb3b0f4591cb1db42170bc68
2020-08-07 13:22:26 -07:00
Luya Gao
326e4ccb5b Return R2N2 R,T,K
Summary: Return rotation, translation and intrinsic matrices necessary to reproduce R2N2's own renderings.

Reviewed By: nikhilaravi

Differential Revision: D22462520

fbshipit-source-id: 46a3859743ebc43c7a24f75827d2be3adf3f486b
2020-08-07 13:22:26 -07:00
Jeremy Reizenstein
c122ccb13c No more cuda 10.0 builds
Summary: Latest pytorch doesn't support cuda 10.0 and now its pytorch/conda-cuda docker image doesn't include it either. Here we remove the pytorch3d builds which use cuda 10.0.

Reviewed By: nikhilaravi

Differential Revision: D22999392

fbshipit-source-id: c834477fc7c812c2d0594dccd9e9471e33a4ec5e
2020-08-07 10:35:18 -07:00
Jeremy Reizenstein
5d9444307f fix graph_conv test
Summary: The recently added part of a test was assuming that the random gpu was gpu 0.

Reviewed By: nikhilaravi

Differential Revision: D22948397

fbshipit-source-id: 88107e19fc3118e763f95be43a614941176a08f9
2020-08-07 01:59:58 -07:00
Jeremy Reizenstein
7944d24d48 gather_scatter on CPU
Summary: CPU implementation of the graph convolution op.

Reviewed By: nikhilaravi, gkioxari

Differential Revision: D21384361

fbshipit-source-id: bc96730e9727bb9aa1b0a232dcb82f0c0d12fe6b
2020-08-05 07:00:20 -07:00
Jeremy Reizenstein
4872a2c4de wheels with cuda
Summary:
The main pytorch wheels on PyPI support CUDA 10.2. Here we make pytorch3d's wheels do the same, instead of being cpu only. This should ultimately make life easier in colab.

Also a little script to count builds, which can be useful for nightly jobs.

Reviewed By: gkioxari

Differential Revision: D22924321

fbshipit-source-id: d6cea9bfbab49bcb0080f65608066c553ea8bb4d
2020-08-04 15:23:28 -07:00
generatedunixname89002005307016
07d7e12644 suppress errors in vision/fair/pytorch3d
Differential Revision: D22834017

fbshipit-source-id: 441124c67a2bbfe086e2983d5c4c845b58fc965c
2020-07-29 18:24:04 -07:00
Nikhila Ravi
a3932960b3 Texturing API updates
Summary:
A fairly big refactor of the texturing API with some breaking changes to how textures are defined.

Main changes:
- There are now 3 types of texture classes: `TexturesUV`, `TexturesAtlas` and `TexturesVertex`. Each class:
   - has a `sample_textures` function which accepts the `fragments` from rasterization and returns `texels`. This means that the shaders will not need to know the type of the mesh texture which will resolve several issues people were reporting on GitHub.
  -  has a `join_batch` method for joining multiple textures of the same type into a batch

Reviewed By: gkioxari

Differential Revision: D21067427

fbshipit-source-id: 4b346500a60181e72fdd1b0dd89b5505c7a33926
2020-07-29 16:11:23 -07:00
Jeremy Reizenstein
b73d3d6ed9 smaller data in specular test
Summary: Reduce the size of the data in this test, so that on circleci it doesn't run out of memory when pytorch (1.6) is used.

Reviewed By: gkioxari

Differential Revision: D22801490

fbshipit-source-id: 9591253c3d47430facd769a2c51a0b1722e0a305
2020-07-29 05:01:07 -07:00
Georgia Gkioxari
42b5b96595 remove composite params from tutorials
Summary: Remove composite params from tutorial

Reviewed By: nikhilaravi

Differential Revision: D22809242

fbshipit-source-id: e5ed4e93fc892baf74e3a464119f0d11973423c3
2020-07-28 22:21:20 -07:00
Jeremy Reizenstein
326f662878 builds for pytorch 1.6
Summary: Make CI builds with pytorch 1.6.

Reviewed By: gkioxari

Differential Revision: D22790325

fbshipit-source-id: 5b0e075f952de9df2de03b65c16fd69140ab6fdd
2020-07-28 10:40:34 -07:00
Steve Branson
9a5341bde3 Add tutorial for fitting textured mesh to multi-view images
Summary: Tutorial showing how to create a synthetic dataset by rendering a cow from multiple views, fit a mesh using a differential silhouette renderer, then fit a mesh using an RGB renderer.

Reviewed By: nikhilaravi

Differential Revision: D22513859

fbshipit-source-id: 24bdaac4ebec6dd01f98e2f5c702065f9335ff33
2020-07-23 12:14:45 -07:00
Luya Gao
722c2b7149 Add BlenderCamera
Summary: Adding BlenderCamera (for rendering with R2N2 Blender transformations in the next diff).

Reviewed By: nikhilaravi

Differential Revision: D22462515

fbshipit-source-id: 4b40ee9bba8b6d56788dd3c723036ec704668153
2020-07-23 10:17:12 -07:00
Luya Gao
483e538dae Test R2N2 loads correct numbers of instances
Summary:
Sample/Get all views at the loading phase instead of returning phase;
Load only views from the split instead of all 24 views;
Test the numbers of views loaded are correct for each category.

Reviewed By: nikhilaravi

Differential Revision: D22631414

fbshipit-source-id: 1c5ce99fe2bdf6618c1aa0b69bb6899473376bc2
2020-07-23 10:17:12 -07:00
generatedunixname89002005307016
7cb9d8ea86 suppress errors in vision - batch 1
Reviewed By: pradeep90

Differential Revision: D22628883

fbshipit-source-id: a19c271b4025c0e3c61408604a7e0f9fbcbdfa5b
2020-07-20 17:13:59 -07:00
Nikhila Ravi
8b70aa1e85 change citation to arxiv paper and fix video in readme
Summary: Github does not support embedding youtube videos! Changing it to a link with a thumbnail.

Reviewed By: gkioxari

Differential Revision: D22601430

fbshipit-source-id: 1087657bc379a228a33eced1bee8492cef8373a7
2020-07-17 19:43:34 -07:00