Summary:
Implementation of ball query from PointNet++. This function is similar to KNN (find the neighbors in p2 for all points in p1). These are the key differences:
- It will return the **first** K neighbors within a specified radius as opposed to the **closest** K neighbors.
- As all the points in p2 do not need to be considered to find the closest K, the algorithm is much faster than KNN when p2 has a large number of points.
- The neighbors are not sorted
- Due to the radius threshold it is not guaranteed that there will be K neighbors even if there are more than K points in p2.
- The padding value for `idx` is -1 instead of 0.
# Note:
- Some of the code is very similar to KNN so it could be possible to modify the KNN forward kernels to support ball query.
- Some users might want to use kNN with ball query - for this we could provide a wrapper function around the current `knn_points` which enables applying the radius threshold afterwards as an alternative. This could be called `ball_query_knn`.
Reviewed By: jcjohnson
Differential Revision: D30261362
fbshipit-source-id: 66b6a7e0114beff7164daf7eba21546ff41ec450
Summary: Fix to resolve GitHub issue #796 - the cameras were being passed in the renderer forward pass instead of at initialization. The rasterizer was correctly using the cameras passed in the `kwargs` for the projection, but the `cameras` are still part of the `kwargs` for the `get_screen_to_ndc_transform` and `get_ndc_to_screen_transform` functions which is causing issues about duplicate arguments.
Reviewed By: bottler
Differential Revision: D30175679
fbshipit-source-id: 547e88d8439456e728fa2772722df5fa0fe4584d
Summary: Restore compatibility with PyTorch 1.4 and 1.5, and a few lint fixes.
Reviewed By: patricklabatut
Differential Revision: D30048115
fbshipit-source-id: ee05efa7c625f6079fb06a3cc23be93e48df9433
Summary:
API fix for NDC/screen cameras and compatibility with PyTorch3D renderers.
With this new fix:
* Users can define cameras and `transform_points` under any coordinate system conventions. The transformation applies the camera K and RT to the input points, not regarding for PyTorch3D conventions. So this makes cameras completely independent from PyTorch3D renderer.
* Cameras can be defined either in NDC space or screen space. For existing ones, FoV cameras are in NDC space. Perspective/Orthographic can be defined in NDC or screen space.
* The interface with PyTorch3D renderers happens through `transform_points_ndc` which transforms points to the NDC space and assumes that input points are provided according to PyTorch3D conventions.
* Similarly, `transform_points_screen` transforms points to screen space and again assumes that input points are under PyTorch3D conventions.
* For Orthographic/Perspective cameras, if they are defined in screen space, the `get_ndc_camera_transform` allows points to be converted to NDC for use for the renderers.
Reviewed By: nikhilaravi
Differential Revision: D26932657
fbshipit-source-id: 1a964e3e7caa54d10c792cf39c4d527ba2fb2e79
Summary: This fixes a recently introduced circular import: the problem went unnoticed by having `pytorch3d.renderer` imported first...
Reviewed By: bottler
Differential Revision: D29686235
fbshipit-source-id: 4b9f2faecec2cc8347ee259cfc359dc9e4f67784
Summary: This changes only documentation. We want to be explicit that ray directions are not normalised (nor assumed to be normalised) but their magnitude is used.
Reviewed By: nikhilaravi
Differential Revision: D29845210
fbshipit-source-id: b81fb3da13a42ad20e8721ed5271fd4f3d8f5acb
Summary: Use PathManager for checking file existence, rather than assuming the path is a local file, in a couple of cases.
Reviewed By: patricklabatut
Differential Revision: D29734621
fbshipit-source-id: e2236a7c2c50ba6916936a4d786abd601205b519
Summary: A bad env var check meant these tests were not being run. Fix that, and fix the copyright test for the new message format.
Reviewed By: patricklabatut
Differential Revision: D29734562
fbshipit-source-id: a1a9bb68901b09c71c7b4ff81a04083febca8d50
Summary:
# Background
There is an unstable error during training (it can happen after several minutes or after several hours).
The error is connected to `torch.det()` function in `_check_valid_rotation_matrix()`.
if I remove the function `torch.det()` in `_check_valid_rotation_matrix()` or remove the whole functions `_check_valid_rotation_matrix()` the error is disappeared (D29555876).
# Solution
Replace `torch.det()` with manual implementation for 3x3 matrix.
Reviewed By: patricklabatut
Differential Revision: D29655924
fbshipit-source-id: 41bde1119274a705ab849751ece28873d2c45155
Summary:
# Make `transform3d.py` a little bit better (performance and code quality)
## 1. Add decorator `torch.no_grad()` to the function `_check_valid_rotation_matrix()`
Function `_check_valid_rotation_matrix()` is needed to identify errors during forward pass only, it's not used for gradients.
## 2. Replace two calls `to` with the single one
Reviewed By: bottler
Differential Revision: D29656501
fbshipit-source-id: 4419e24dbf436c1b60abf77bda4376fb87a593be
Summary:
Context: in the code we are releasing with CO3D dataset, we use `cuda()` on TensorProperties like Pointclouds and Cameras where we recursively move batch to a GPU. It would be good to push it to a release so we don’t need to depend on the nightly build.
Additionally, I aligned the logic of `.to("cuda")` without device index to the one of `torch.Tensor` where the current device is populated to index. It should not affect any actual use cases but some tests had to be changed.
Reviewed By: bottler
Differential Revision: D29659529
fbshipit-source-id: abe58aeaca14bacc68da3e6cf5ae07df3353e3ce
Summary: An early-return test for gradient calculation did not include the opacity gradient calculation - hence would also return early without calculating gradients even if opacity gradients are required.
Reviewed By: bottler
Differential Revision: D29505684
fbshipit-source-id: 575e820b8f58b19476b2fe3288702806733e840b
Summary: This commit adds a new camera conversion function for OpenCV style parameters to Pulsar parameters to the library. Using this function it addresses a bug reported here: https://fb.workplace.com/groups/629644647557365/posts/1079637302558095, by using the PyTorch3D->OpenCV->Pulsar chain instead of the original direct conversion function. Both conversions are well-tested and an additional test for the full chain has been added, resulting in a more reliable solution requiring less code.
Reviewed By: patricklabatut
Differential Revision: D29322106
fbshipit-source-id: 13df13c2e48f628f75d9f44f19ff7f1646fb7ebd
Summary: Use rotation matrices for OpenCV / PyTorch3D conversions: this avoids hiding issues with conversions to / from axis-angle vectors and ensure new conversion functions have a consistent interface.
Reviewed By: bottler, classner
Differential Revision: D29634099
fbshipit-source-id: 40b28357914eb563fedea60a965dcf69e848ccfa
Summary: solve and lstsq have moved around in torch. Cope with both.
Reviewed By: patricklabatut
Differential Revision: D29302316
fbshipit-source-id: b34f0b923e90a357f20df359635929241eba6e74
Summary: Deprecate the `so3_exponential_map()` function in favor of its alias `so3_exp_map()`: this aligns with the naming of `so3_log_map()` and the recently introduced `se3_exp_map()` / `se3_log_map()` pair.
Reviewed By: bottler
Differential Revision: D29329966
fbshipit-source-id: b6f60b9e86b2995f70b1fbeb16f9feea05c55de9
Summary: Add functionality to to save an `.obj` file with associated UV textures: `.png` image and `.mtl` file as well as saving verts_uvs and faces_uvs to the `.obj` file.
Reviewed By: bottler
Differential Revision: D29337562
fbshipit-source-id: 86829b40dae9224088b328e7f5a16eacf8582eb5
Summary: Annotate the (return type of the) following dunder functions across the codebase: `__init__()`, `__len__()`, `__getitem__()`
Reviewed By: nikhilaravi
Differential Revision: D29001801
fbshipit-source-id: 928d9e1c417ffe01ab8c0445311287786e997c7c
Summary:
Refactor of all functions to compute laplacian matrices in one file.
Support for:
* Standard Laplacian
* Cotangent Laplacian
* Norm Laplacian
Reviewed By: nikhilaravi
Differential Revision: D29297466
fbshipit-source-id: b96b88915ce8ef0c2f5693ec9b179fd27b70abf9
Summary: This diff implements the inverse of D28992470 (8006842f2a): a function to extract OpenCV convention camera parameters from a PyTorch3D `PerspectiveCameras` object. This is the first part of the new PyTorch3d<>OpenCV<>Pulsar conversion functions.
Reviewed By: patricklabatut
Differential Revision: D29278411
fbshipit-source-id: 68d4555b508dbe8685d8239443f839d194cc2484
Summary:
Get rid of pyre fixmes related to importing a native module:
- add stub file for the `_C` native extension to the internal typeshed
- add initial annotations to the new stub file
- remove the now unnecessary pyre ignores
Reviewed By: nikhilaravi
Differential Revision: D28929467
fbshipit-source-id: 6525e15c8f27215a3ff6f78392925fd0ed6ec2ac
Summary: Add some more validation of a list of texture maps. Move the initialisation of maps_padded to a new function to reduce complexity.
Reviewed By: nikhilaravi
Differential Revision: D29263443
fbshipit-source-id: 153e262d2e9af21090570768020fca019e364024
Summary: The TexturesUV class supports an arbitrary number of channels. In one place in join_scene we unnecessarily assumed the usual value, 3.
Reviewed By: patricklabatut
Differential Revision: D29133477
fbshipit-source-id: de8eb15fdd55675da084634d9d99e2a3f4d35401
Summary: Comments in textures.py were inconsistent in describing the number of channels, sometimes C, sometimes D, sometimes 3. Now always C.
Reviewed By: patricklabatut
Differential Revision: D29263435
fbshipit-source-id: 7c1260c164c52852dc9e14d0e12da4cfb64af408
Summary: Because rotations and (rotation) quaternions live on curved manifolds, it doesn't make sense to optimize them directly. Having a prominent option to require gradient on random ones may cause people to try, and isn't particularly useful.
Reviewed By: theschnitz
Differential Revision: D29160734
fbshipit-source-id: fc9e320672349fe334747c5b214655882a460a62
Summary:
Change the cow gltf loading test to validate the texture values and not to validate the renderer output because it has an unstable pixel.
Also a couple of lints.
Reviewed By: patricklabatut
Differential Revision: D29131260
fbshipit-source-id: 5e11f066a2a638588aacb09776cc842173ef669f
Summary: As noted in #710, save_ply was failing with some values of the faces tensor. It was assuming the faces were contiguous in using view() to change them. Here we avoid doing that.
Reviewed By: patricklabatut
Differential Revision: D29159655
fbshipit-source-id: 47214a7ce915bab8d81f109c2b97cde464fd57d8
Summary: Implements a conversion function between OpenCV and PyTorch3D cameras.
Reviewed By: patricklabatut
Differential Revision: D28992470
fbshipit-source-id: dbcc9f213ec293c2f6938261c704aea09aad3c90
Summary:
Improves so3 functions to make gradient computation stable:
- Instead of `torch.acos`, uses `acos_linear_extrapolation` which has finite gradients of reasonable magnitude for all inputs.
- Adds tests for the latter.
The tests of the finiteness of the gradient in `test_so3_exp_singularity`, `test_so3_exp_singularity`, `test_so3_cos_bound` would fail if the `so3` functions would call `torch.acos` instead of `acos_linear_extrapolation`.
Reviewed By: bottler
Differential Revision: D23326429
fbshipit-source-id: dc296abf2ae3ddfb3942c8146621491a9cb740ee
Summary:
Implements a backprop-safe version of `torch.acos` that linearly extrapolates the function outside bounds.
Below is a plot of the extrapolated acos for different bounds:
{F611339485}
Reviewed By: bottler, nikhilaravi
Differential Revision: D27945714
fbshipit-source-id: fa2e2385b56d6fe534338d5192447c4a3aec540c
Summary:
Fix small face issue for point_mesh distance computation.
The issue lies in the computation of `IsInsideTriangle` which is unstable and non-symmetrical when faces with small areas are given as input. This diff fixes the issue by returning `False` for `IsInsideTriangle` when small faces are given as input.
Reviewed By: bottler
Differential Revision: D29163052
fbshipit-source-id: be297002f26b5e6eded9394fde00553a37406bee
Summary:
As reported on github, `matrix_to_quaternion` was incorrect for rotations by 180˚. We resolved the sign of the component `i` based on the sign of `i*r`, assuming `r > 0`, which is untrue if `r == 0`.
This diff handles special cases and ensures we use the non-zero elements to copy the sign from.
Reviewed By: bottler
Differential Revision: D29149465
fbshipit-source-id: cd508cc31567fc37ea3463dd7e8c8e8d5d64a235
Summary: Increase code coverage of shader and re-include them in code coverage test
Reviewed By: nikhilaravi
Differential Revision: D29097503
fbshipit-source-id: 2791989ee1562cfa193f3addea0ce72d6840614a
Summary:
There was a bug when `z_clip_value` is not None but there are no faces which are actually visible in the image due to culling. In `rasterize_meshes.py` a function `convert_clipped_rasterization_to_original_faces` is called to convert the clipped face indices etc back to the unclipped versions, but the case where there is no clipping was not handled correctly.
Fixes Github Issue #632
Reviewed By: bottler
Differential Revision: D29116150
fbshipit-source-id: fae82a0b4848c84b3ed7c7b04ef5c9848352cf5c
Summary: Fixed multiple issues with shape broadcasting in lighting, shading and blending and updated the tests.
Reviewed By: bottler
Differential Revision: D28997941
fbshipit-source-id: d3ef93f979344076b1d9098a86178b4da63844c8
Summary: Increase code coverage of subdivide_meshes and re-include it in code coverage test
Reviewed By: bottler
Differential Revision: D29097476
fbshipit-source-id: 3403ae38a90c4b53f24188eed11faae202a235b5
