Summary: Attempt to fix#659, an observation that the rasterizer is nondeterministic, by resolving tied faces by picking those with lower index.
Reviewed By: nikhilaravi, patricklabatut
Differential Revision: D30699039
fbshipit-source-id: 39ed797eb7e9ce7370ae71259ad6b757f9449923
Summary:
CUDA implementation of farthest point sampling algorithm.
## Visual comparison
Compared to random sampling, farthest point sampling gives better coverage of the shape.
{F658631262}
## Reduction
Parallelized block reduction to find the max value at each iteration happens as follows:
1. First split the points into two equal sized parts (e.g. for a list with 8 values):
`[20, 27, 6, 8 | 11, 10, 2, 33]`
2. Use half of the thread (4 threads) to compare pairs of elements from each half (e.g elements [0, 4], [1, 5] etc) and store the result in the first half of the list:
`[20, 27, 6, 33 | 11, 10, 2, 33]`
Now we no longer care about the second part but again divide the first part into two
`[20, 27 | 6, 33| -, -, -, -]`
Now we can use 2 threads to compare the 4 elements
4. Finally we have gotten down to a single pair
`[20 | 33 | -, - | -, -, -, -]`
Use 1 thread to compare the remaining two elements
5. The max will now be at thread id = 0
`[33 | - | -, - | -, -, -, -]`
The reduction will give the farthest point for the selected batch index at this iteration.
Reviewed By: bottler, jcjohnson
Differential Revision: D30401803
fbshipit-source-id: 525bd5ae27c4b13b501812cfe62306bb003827d2
Summary:
This is a naive python implementation of the iterative farthest point sampling algorithm along with associated simple tests. The C++/CUDA implementations will follow in subsequent diffs.
The algorithm is used to subsample a pointcloud with better coverage of the space of the pointcloud.
The function has not been added to `__init__.py`. I will add this after the full C++/CUDA implementations.
Reviewed By: jcjohnson
Differential Revision: D30285716
fbshipit-source-id: 33f4181041fc652776406bcfd67800a6f0c3dd58
Summary: Fix issue #826. This is a correction to the joining of TexturesUV into a single scene.
Reviewed By: nikhilaravi
Differential Revision: D30767092
fbshipit-source-id: 03ba6a1d2f22e569d1b3641cd13ddbb8dcb87ec7
Summary:
* HAT_INV_SKEW_SYMMETRIC_TOL was a global variable and torch script gives an error when compiling that function. Move it to the function scope.
* torch script gives error when compiling acos_linear_extrapolation because bound is a union of tuple and float. The tuple version is kept in this diff.
Reviewed By: patricklabatut
Differential Revision: D30614916
fbshipit-source-id: 34258d200dc6a09fbf8917cac84ba8a269c00aef
Summary: In D30349234 (1b8d86a104) we introduced persistent=False to some register_buffer calls, which depend on PyTorch 1.6. We go back to the old behaviour for PyTorch 1.5.
Reviewed By: nikhilaravi
Differential Revision: D30731327
fbshipit-source-id: ab02ef98ee87440ef02479b72f4872b562ab85b5
Summary: Change cyclic deps test to be independent of test discovery order. Also let it work without plotly.
Reviewed By: nikhilaravi
Differential Revision: D30669614
fbshipit-source-id: 2eadf3f8b56b6096c5466ce53b4f8ac6df27b964
Summary: Fixes GitHub issue #751. The vectorized implementation of bilinear interpolation didn't properly handle the edge cases in the same way as the `grid_sample` method in PyTorch.
Reviewed By: bottler
Differential Revision: D30684208
fbshipit-source-id: edf241ecbd72d46b94ad340a4e601e26c83db88e
Summary:
As suggested in #802. By not persisting the _xy_grid buffer, we can allow (in some cases) a model with one image_size to be loaded from a saved model which was trained at a different resolution.
Also avoid persisting _frequencies in HarmonicEmbedding for similar reasons.
BC-break: This will cause load_state_dict, in strict mode, to complain if you try to load an old model with the new code.
Reviewed By: patricklabatut
Differential Revision: D30349234
fbshipit-source-id: d6061d1e51c9f79a78d61a9f732c9a5dfadbbb47
Summary:
Use PyTorch3D's new faster sample_pdf function instead of local Python implementation.
Also clarify deps for the Python implementation.
Reviewed By: gkioxari
Differential Revision: D30512109
fbshipit-source-id: 84cfdc00313fada37a6b29837de96f6a4646434f
Summary: New test that each subpackage of pytorch3d imports cleanly.
Reviewed By: patricklabatut
Differential Revision: D30001632
fbshipit-source-id: ca8dcac94491fc22f33602b3bbef481cba927094
Summary: Implement the sample_pdf function from the NeRF project as compiled operators.. The binary search (in searchsorted) is replaced with a low tech linear search, but this is not a problem for the envisaged numbers of bins.
Reviewed By: gkioxari
Differential Revision: D26312535
fbshipit-source-id: df1c3119cd63d944380ed1b2657b6ad81d743e49
Summary: Copy the sample_pdf operation from the NeRF project in to PyTorch3D, in preparation for optimizing it.
Reviewed By: gkioxari
Differential Revision: D27117930
fbshipit-source-id: 20286b007f589a4c4d53ed818c4bc5f2abd22833
Summary: Fixes to a couple of comments on points to volumes, make the mask work in round_points_to_volumes, and remove a duplicate rand calculation
Reviewed By: nikhilaravi
Differential Revision: D29522845
fbshipit-source-id: 86770ba37ef3942b909baf63fd73eed1399635b6
Summary: Much of the code is actually available during the conda tests, as long as we look in the right place. We enable some of them.
Reviewed By: nikhilaravi
Differential Revision: D30249357
fbshipit-source-id: 01c57b6b8c04442237965f23eded594aeb90abfb
Summary:
Implementation of ball query from PointNet++. This function is similar to KNN (find the neighbors in p2 for all points in p1). These are the key differences:
- It will return the **first** K neighbors within a specified radius as opposed to the **closest** K neighbors.
- As all the points in p2 do not need to be considered to find the closest K, the algorithm is much faster than KNN when p2 has a large number of points.
- The neighbors are not sorted
- Due to the radius threshold it is not guaranteed that there will be K neighbors even if there are more than K points in p2.
- The padding value for `idx` is -1 instead of 0.
# Note:
- Some of the code is very similar to KNN so it could be possible to modify the KNN forward kernels to support ball query.
- Some users might want to use kNN with ball query - for this we could provide a wrapper function around the current `knn_points` which enables applying the radius threshold afterwards as an alternative. This could be called `ball_query_knn`.
Reviewed By: jcjohnson
Differential Revision: D30261362
fbshipit-source-id: 66b6a7e0114beff7164daf7eba21546ff41ec450
Summary: New test that notes and tutorials are listed in the website metadata, so that they will be included in the website build.
Reviewed By: nikhilaravi
Differential Revision: D30223799
fbshipit-source-id: 2dca9730b54e68da2fd430a7b47cb7e18814d518
Summary: Fix to resolve GitHub issue #796 - the cameras were being passed in the renderer forward pass instead of at initialization. The rasterizer was correctly using the cameras passed in the `kwargs` for the projection, but the `cameras` are still part of the `kwargs` for the `get_screen_to_ndc_transform` and `get_ndc_to_screen_transform` functions which is causing issues about duplicate arguments.
Reviewed By: bottler
Differential Revision: D30175679
fbshipit-source-id: 547e88d8439456e728fa2772722df5fa0fe4584d
Summary:
API fix for NDC/screen cameras and compatibility with PyTorch3D renderers.
With this new fix:
* Users can define cameras and `transform_points` under any coordinate system conventions. The transformation applies the camera K and RT to the input points, not regarding for PyTorch3D conventions. So this makes cameras completely independent from PyTorch3D renderer.
* Cameras can be defined either in NDC space or screen space. For existing ones, FoV cameras are in NDC space. Perspective/Orthographic can be defined in NDC or screen space.
* The interface with PyTorch3D renderers happens through `transform_points_ndc` which transforms points to the NDC space and assumes that input points are provided according to PyTorch3D conventions.
* Similarly, `transform_points_screen` transforms points to screen space and again assumes that input points are under PyTorch3D conventions.
* For Orthographic/Perspective cameras, if they are defined in screen space, the `get_ndc_camera_transform` allows points to be converted to NDC for use for the renderers.
Reviewed By: nikhilaravi
Differential Revision: D26932657
fbshipit-source-id: 1a964e3e7caa54d10c792cf39c4d527ba2fb2e79
Summary: A bad env var check meant these tests were not being run. Fix that, and fix the copyright test for the new message format.
Reviewed By: patricklabatut
Differential Revision: D29734562
fbshipit-source-id: a1a9bb68901b09c71c7b4ff81a04083febca8d50
Summary:
# Background
There is an unstable error during training (it can happen after several minutes or after several hours).
The error is connected to `torch.det()` function in `_check_valid_rotation_matrix()`.
if I remove the function `torch.det()` in `_check_valid_rotation_matrix()` or remove the whole functions `_check_valid_rotation_matrix()` the error is disappeared (D29555876).
# Solution
Replace `torch.det()` with manual implementation for 3x3 matrix.
Reviewed By: patricklabatut
Differential Revision: D29655924
fbshipit-source-id: 41bde1119274a705ab849751ece28873d2c45155
Summary:
Context: in the code we are releasing with CO3D dataset, we use `cuda()` on TensorProperties like Pointclouds and Cameras where we recursively move batch to a GPU. It would be good to push it to a release so we don’t need to depend on the nightly build.
Additionally, I aligned the logic of `.to("cuda")` without device index to the one of `torch.Tensor` where the current device is populated to index. It should not affect any actual use cases but some tests had to be changed.
Reviewed By: bottler
Differential Revision: D29659529
fbshipit-source-id: abe58aeaca14bacc68da3e6cf5ae07df3353e3ce
Summary: This commit adds a new camera conversion function for OpenCV style parameters to Pulsar parameters to the library. Using this function it addresses a bug reported here: https://fb.workplace.com/groups/629644647557365/posts/1079637302558095, by using the PyTorch3D->OpenCV->Pulsar chain instead of the original direct conversion function. Both conversions are well-tested and an additional test for the full chain has been added, resulting in a more reliable solution requiring less code.
Reviewed By: patricklabatut
Differential Revision: D29322106
fbshipit-source-id: 13df13c2e48f628f75d9f44f19ff7f1646fb7ebd
Summary: Use rotation matrices for OpenCV / PyTorch3D conversions: this avoids hiding issues with conversions to / from axis-angle vectors and ensure new conversion functions have a consistent interface.
Reviewed By: bottler, classner
Differential Revision: D29634099
fbshipit-source-id: 40b28357914eb563fedea60a965dcf69e848ccfa
Summary: Enable this benchmark to be run on its own, like others.
Reviewed By: patricklabatut
Differential Revision: D29522846
fbshipit-source-id: c7b3b5c9a0fcdeeb79d8b2ec197684b4380aa547
Summary: solve and lstsq have moved around in torch. Cope with both.
Reviewed By: patricklabatut
Differential Revision: D29302316
fbshipit-source-id: b34f0b923e90a357f20df359635929241eba6e74
Summary: Deprecate the `so3_exponential_map()` function in favor of its alias `so3_exp_map()`: this aligns with the naming of `so3_log_map()` and the recently introduced `se3_exp_map()` / `se3_log_map()` pair.
Reviewed By: bottler
Differential Revision: D29329966
fbshipit-source-id: b6f60b9e86b2995f70b1fbeb16f9feea05c55de9
Summary: Add functionality to to save an `.obj` file with associated UV textures: `.png` image and `.mtl` file as well as saving verts_uvs and faces_uvs to the `.obj` file.
Reviewed By: bottler
Differential Revision: D29337562
fbshipit-source-id: 86829b40dae9224088b328e7f5a16eacf8582eb5
Summary:
Refactor of all functions to compute laplacian matrices in one file.
Support for:
* Standard Laplacian
* Cotangent Laplacian
* Norm Laplacian
Reviewed By: nikhilaravi
Differential Revision: D29297466
fbshipit-source-id: b96b88915ce8ef0c2f5693ec9b179fd27b70abf9
Summary: This diff implements the inverse of D28992470 (8006842f2a): a function to extract OpenCV convention camera parameters from a PyTorch3D `PerspectiveCameras` object. This is the first part of the new PyTorch3d<>OpenCV<>Pulsar conversion functions.
Reviewed By: patricklabatut
Differential Revision: D29278411
fbshipit-source-id: 68d4555b508dbe8685d8239443f839d194cc2484
Summary: Because rotations and (rotation) quaternions live on curved manifolds, it doesn't make sense to optimize them directly. Having a prominent option to require gradient on random ones may cause people to try, and isn't particularly useful.
Reviewed By: theschnitz
Differential Revision: D29160734
fbshipit-source-id: fc9e320672349fe334747c5b214655882a460a62
Summary:
Change the cow gltf loading test to validate the texture values and not to validate the renderer output because it has an unstable pixel.
Also a couple of lints.
Reviewed By: patricklabatut
Differential Revision: D29131260
fbshipit-source-id: 5e11f066a2a638588aacb09776cc842173ef669f
Summary: As noted in #710, save_ply was failing with some values of the faces tensor. It was assuming the faces were contiguous in using view() to change them. Here we avoid doing that.
Reviewed By: patricklabatut
Differential Revision: D29159655
fbshipit-source-id: 47214a7ce915bab8d81f109c2b97cde464fd57d8
Summary: Implements a conversion function between OpenCV and PyTorch3D cameras.
Reviewed By: patricklabatut
Differential Revision: D28992470
fbshipit-source-id: dbcc9f213ec293c2f6938261c704aea09aad3c90
Summary:
Improves so3 functions to make gradient computation stable:
- Instead of `torch.acos`, uses `acos_linear_extrapolation` which has finite gradients of reasonable magnitude for all inputs.
- Adds tests for the latter.
The tests of the finiteness of the gradient in `test_so3_exp_singularity`, `test_so3_exp_singularity`, `test_so3_cos_bound` would fail if the `so3` functions would call `torch.acos` instead of `acos_linear_extrapolation`.
Reviewed By: bottler
Differential Revision: D23326429
fbshipit-source-id: dc296abf2ae3ddfb3942c8146621491a9cb740ee
Summary:
Implements a backprop-safe version of `torch.acos` that linearly extrapolates the function outside bounds.
Below is a plot of the extrapolated acos for different bounds:
{F611339485}
Reviewed By: bottler, nikhilaravi
Differential Revision: D27945714
fbshipit-source-id: fa2e2385b56d6fe534338d5192447c4a3aec540c
Summary:
Fix small face issue for point_mesh distance computation.
The issue lies in the computation of `IsInsideTriangle` which is unstable and non-symmetrical when faces with small areas are given as input. This diff fixes the issue by returning `False` for `IsInsideTriangle` when small faces are given as input.
Reviewed By: bottler
Differential Revision: D29163052
fbshipit-source-id: be297002f26b5e6eded9394fde00553a37406bee
Summary:
As reported on github, `matrix_to_quaternion` was incorrect for rotations by 180˚. We resolved the sign of the component `i` based on the sign of `i*r`, assuming `r > 0`, which is untrue if `r == 0`.
This diff handles special cases and ensures we use the non-zero elements to copy the sign from.
Reviewed By: bottler
Differential Revision: D29149465
fbshipit-source-id: cd508cc31567fc37ea3463dd7e8c8e8d5d64a235
Summary: Increase code coverage of shader and re-include them in code coverage test
Reviewed By: nikhilaravi
Differential Revision: D29097503
fbshipit-source-id: 2791989ee1562cfa193f3addea0ce72d6840614a
Summary:
There was a bug when `z_clip_value` is not None but there are no faces which are actually visible in the image due to culling. In `rasterize_meshes.py` a function `convert_clipped_rasterization_to_original_faces` is called to convert the clipped face indices etc back to the unclipped versions, but the case where there is no clipping was not handled correctly.
Fixes Github Issue #632
Reviewed By: bottler
Differential Revision: D29116150
fbshipit-source-id: fae82a0b4848c84b3ed7c7b04ef5c9848352cf5c
Summary: Fixed multiple issues with shape broadcasting in lighting, shading and blending and updated the tests.
Reviewed By: bottler
Differential Revision: D28997941
fbshipit-source-id: d3ef93f979344076b1d9098a86178b4da63844c8
Summary: Increase code coverage of subdivide_meshes and re-include it in code coverage test
Reviewed By: bottler
Differential Revision: D29097476
fbshipit-source-id: 3403ae38a90c4b53f24188eed11faae202a235b5
Summary:
When `z_clip_value = None` and faces are outside the view frustum the shape of one of the tensors in `clip.py` is incorrect.
`faces_num_clipped_verts` should be (F,) but it was (F,3). Added a new test to ensure this case is handled.
Reviewed By: bottler
Differential Revision: D29051282
fbshipit-source-id: 5f4172ba4d4a75d928404dde9abf48aef18c68bd
