393 Commits

Author SHA1 Message Date
Nikhila Ravi
ff8d4762f4 (new) CUDA IoU for 3D boxes
Summary: CUDA implementation of 3D bounding box overlap calculation.

Reviewed By: gkioxari

Differential Revision: D31157919

fbshipit-source-id: 5dc89805d01fef2d6779f00a33226131e39c43ed
2021-09-29 18:49:09 -07:00
Nikhila Ravi
53266ec9ff C++ IoU for 3D Boxes
Summary: C++ Implementation of algorithm to compute 3D bounding boxes for batches of bboxes of shape (N, 8, 3) and (M, 8, 3).

Reviewed By: gkioxari

Differential Revision: D30905190

fbshipit-source-id: 02e2cf025cd4fa3ff706ce5cf9b82c0fb5443f96
2021-09-29 17:03:43 -07:00
Nikhila Ravi
2293f1fed0 IoU for 3D boxes
Summary:
I have implemented an exact solution for 3D IoU of oriented 3D boxes.

This file includes:
* box3d_overlap: which computes the exact IoU of box1 and box2
* box3d_overlap_sampling: which computes an approximate IoU of box1 and box2 by sampling points within the boxes

Note that both implementations currently do not support batching.

Our exact IoU implementation is based on the fact that the intersecting shape of the two 3D boxes will be formed by segments of the surface of the boxes. Our algorithm computes these segments by reasoning whether triangles of one box are within the second box and vice versa. We deal with intersecting triangles by clipping them.

Reviewed By: gkioxari

Differential Revision: D30667497

fbshipit-source-id: 2f747f410f90b7f854eeaf3036794bc3ac982917
2021-09-29 13:44:10 -07:00
Pyre Bot Jr
5b89c4e3bb suppress errors in vision/fair/pytorch3d
Differential Revision: D31266959

fbshipit-source-id: 878a59ca2cfe1389e42fc338653e8d3314b56b91
2021-09-29 05:07:37 -07:00
Jeremy Reizenstein
9a737da83c More renderer parameter descriptions
Summary:
Copy some descriptions of renderer parameters to more places so they are easier to find.

Also a couple of small corrections, and make RasterizationSettings a dataclass.

Reviewed By: nikhilaravi, patricklabatut

Differential Revision: D30899822

fbshipit-source-id: 805cf366acb7d51cb308fa574deff0657c199673
2021-09-24 09:59:24 -07:00
Jeremy Reizenstein
860b742a02 deterministic rasterization
Summary: Attempt to fix #659, an observation that the rasterizer is nondeterministic, by resolving tied faces by picking those with lower index.

Reviewed By: nikhilaravi, patricklabatut

Differential Revision: D30699039

fbshipit-source-id: 39ed797eb7e9ce7370ae71259ad6b757f9449923
2021-09-23 06:59:48 -07:00
Jeremy Reizenstein
cb170ac024 Avoid torch/extension.h in cuda
Summary: Unlike other cu files, sigmoid_alpha_blend uses torch/extension.h. Avoid for possible build speed win and because of a reported problem #843 on windows with CUDA 11.4.

Reviewed By: nikhilaravi

Differential Revision: D31054121

fbshipit-source-id: 53a1f985a1695a044dfd2ee1a5b0adabdf280595
2021-09-22 15:54:59 -07:00
Jeremy Reizenstein
fe5bfa5994 rename cpp to avoid clash
Summary: Rename sample_farthest_point.cpp to not match its CUDA equivalent.

Reviewed By: nikhilaravi

Differential Revision: D31006645

fbshipit-source-id: 135b511cbde320d2b3e07fc5b027971ef9210aa9
2021-09-22 15:54:59 -07:00
Jeremy Reizenstein
dbfb3a910a remove __restrict__ in cpp
Summary: Remove use of nonstandard C++. Noticed on windows in issue https://github.com/facebookresearch/pytorch3d/issues/843. (We use `__restrict__` in CUDA, where it is fine, even on windows)

Reviewed By: nikhilaravi

Differential Revision: D31006516

fbshipit-source-id: 929ba9b3216cb70fad3ffa3274c910618d83973f
2021-09-22 15:54:59 -07:00
Pyre Bot Jr
526df446c6 suppress errors in vision/fair/pytorch3d
Differential Revision: D31042748

fbshipit-source-id: fffb983bd6765d306a407587ddf64e68e57e9ecc
2021-09-18 12:24:58 -07:00
Nikhila Ravi
bd04ffaf77 Farthest point sampling CUDA
Summary:
CUDA implementation of farthest point sampling algorithm.

## Visual comparison

Compared to random sampling, farthest point sampling gives better coverage of the shape.

{F658631262}

## Reduction

Parallelized block reduction to find the max value at each iteration happens as follows:

1. First split the points into two equal sized parts (e.g. for a list with 8 values):
`[20, 27, 6, 8 | 11, 10, 2, 33]`
2. Use half of the thread (4 threads) to compare pairs of elements from each half (e.g elements [0, 4], [1, 5] etc) and store the result in the first half of the list:
`[20, 27, 6, 33 | 11, 10, 2, 33]`
Now we no longer care about the second part but again divide the first part into two
`[20, 27 | 6, 33| -, -, -, -]`
Now we can use 2 threads to compare the 4 elements
4. Finally we have gotten down to a single pair
`[20 | 33 | -, - | -, -, -, -]`
Use 1 thread to compare the remaining two elements
5. The max will now be at thread id = 0
`[33 | - | -, - | -, -, -, -]`
The reduction will give the farthest point for the selected batch index at this iteration.

Reviewed By: bottler, jcjohnson

Differential Revision: D30401803

fbshipit-source-id: 525bd5ae27c4b13b501812cfe62306bb003827d2
2021-09-15 13:49:22 -07:00
Nikhila Ravi
d9f7611c4b Farthest point sampling C++
Summary: C++ implementation of iterative farthest point sampling.

Reviewed By: jcjohnson

Differential Revision: D30349887

fbshipit-source-id: d25990f857752633859fe00283e182858a870269
2021-09-15 13:49:21 -07:00
Nikhila Ravi
3b7d78c7a7 Farthest point sampling python naive
Summary:
This is a naive python implementation of the iterative farthest point sampling algorithm along with associated simple tests. The C++/CUDA implementations will follow in subsequent diffs.

The algorithm is used to subsample a pointcloud with better coverage of the space of the pointcloud.

The function has not been added to `__init__.py`. I will add this after the full C++/CUDA implementations.

Reviewed By: jcjohnson

Differential Revision: D30285716

fbshipit-source-id: 33f4181041fc652776406bcfd67800a6f0c3dd58
2021-09-15 13:49:21 -07:00
Jeremy Reizenstein
a0d76a7080 join_scene fix for TexturesUV
Summary: Fix issue #826. This is a correction to the joining of TexturesUV into a single scene.

Reviewed By: nikhilaravi

Differential Revision: D30767092

fbshipit-source-id: 03ba6a1d2f22e569d1b3641cd13ddbb8dcb87ec7
2021-09-13 07:08:58 -07:00
Shangchen Han
46f727cb68 make so3_log_map torch script compatible
Summary:
* HAT_INV_SKEW_SYMMETRIC_TOL was a global variable and torch script gives an error when compiling that function. Move it to the function scope.
* torch script gives error when compiling acos_linear_extrapolation because bound is a union of tuple and float. The tuple version is kept in this diff.

Reviewed By: patricklabatut

Differential Revision: D30614916

fbshipit-source-id: 34258d200dc6a09fbf8917cac84ba8a269c00aef
2021-09-10 11:13:26 -07:00
Jeremy Reizenstein
c3d7808868 register_buffer compatibility
Summary: In D30349234 (1b8d86a104) we introduced persistent=False to some register_buffer calls, which depend on PyTorch 1.6. We go back to the old behaviour for PyTorch 1.5.

Reviewed By: nikhilaravi

Differential Revision: D30731327

fbshipit-source-id: ab02ef98ee87440ef02479b72f4872b562ab85b5
2021-09-09 07:37:57 -07:00
Justin Johnson
bbc7573261 Unify coarse rasterization for points and meshes
Summary:
There has historically been a lot of duplication between the coarse rasterization logic for point clouds and meshes. This diff factors out the shared logic, so coarse rasterization of point clouds and meshes share the same core logic.

Previously the only difference between the coarse rasterization kernels for points and meshes was the logic for checking whether a {point / triangle} intersects a tile in the image. We implement a generic coarse rasterization kernel that takes a set of 2D bounding boxes rather than geometric primitives; we then implement separate kernels that compute 2D bounding boxes for points and triangles.

This change does not affect the Python API at all. It also should not change any rasterization behavior, since this diff is just a refactoring of the existing logic.

I see this diff as the first in a few pieces of rasterizer refactoring. Followup diffs should do the following:
- Add a check for bin overflow in the generic coarse rasterizer kernel: allocate a global scalar to flag bin overflow which kernel worker threads can write to in case they detect bin overflow. The C++ launcher function can then check this flag after the kernel returns and issue a warning to the user in case of overflow.
- As a slightly more involved mechanism, if bin overflow is detected then the coarse kernel can continue running in order to count how many elements fall into each bin, without actually writing out their indices to the coarse output tensor. Then the actual number of entries per bin can be used to re-allocate the output tensor and re-run the coarse rasterization kernel so that bin overflow can be automatically avoided.
- The unification of the coarse and fine rasterization kernels also allows us to insert an extra CUDA kernel prior to coarse rasterization that filters out primitives outside the view frustum. This would be helpful for rendering full scenes (e.g. Matterport data) where only a small piece of the mesh is actually visible at any one time.

Reviewed By: bottler

Differential Revision: D25710361

fbshipit-source-id: 9c9dea512cb339c42adb3c92e7733fedd586ce1b
2021-09-08 16:17:30 -07:00
Justin Johnson
eed68f457d Refactor mesh coarse rasterization
Summary: Renaming parts of the mesh coarse rasterization and separating the bounding box calculation. All in preparation for sharing code with point rasterization.

Reviewed By: bottler

Differential Revision: D30369112

fbshipit-source-id: 3508c0b1239b355030cfa4038d5f3d6a945ebbf4
2021-09-08 16:17:30 -07:00
Justin Johnson
62dbf371ae Move coarse rasterization to new file
Summary: In preparation for sharing coarse rasterization between point clouds and meshes, move the functions to a new file. No code changes.

Reviewed By: bottler

Differential Revision: D30367812

fbshipit-source-id: 9e73835a26c4ac91f5c9f61ff682bc8218e36c6a
2021-09-08 16:17:30 -07:00
Nikhila Ravi
fc156b50c0 (bug) Fix exception when creating a TextureAtlas
Summary: Fixes GitHub issue #751. The vectorized implementation of bilinear interpolation didn't properly handle the edge cases in the same way as the `grid_sample` method in PyTorch.

Reviewed By: bottler

Differential Revision: D30684208

fbshipit-source-id: edf241ecbd72d46b94ad340a4e601e26c83db88e
2021-09-01 09:26:44 -07:00
Georgia Gkioxari
835e662fb5 master -> main
Summary: Replace master with main in hard coded paths or mentions in documentation

Reviewed By: bottler

Differential Revision: D30696097

fbshipit-source-id: d5ff67bb026d90d1543d10ab027f916e8361ca69
2021-09-01 05:33:25 -07:00
Jeremy Reizenstein
1b8d86a104 (breaking) image_size-agnostic GridRaySampler
Summary:
As suggested in #802. By not persisting the _xy_grid buffer, we can allow (in some cases) a model with one image_size to be loaded from a saved model which was trained at a different resolution.

Also avoid persisting _frequencies in HarmonicEmbedding for similar reasons.

BC-break: This will cause load_state_dict, in strict mode, to complain if you try to load an old model with the new code.

Reviewed By: patricklabatut

Differential Revision: D30349234

fbshipit-source-id: d6061d1e51c9f79a78d61a9f732c9a5dfadbbb47
2021-08-31 14:30:24 -07:00
Jeremy Reizenstein
1251446383 Use sample_pdf from PyTorch3D in NeRF
Summary:
Use PyTorch3D's new faster sample_pdf function instead of local Python implementation.

Also clarify deps for the Python implementation.

Reviewed By: gkioxari

Differential Revision: D30512109

fbshipit-source-id: 84cfdc00313fada37a6b29837de96f6a4646434f
2021-08-31 11:26:26 -07:00
Jeremy Reizenstein
77fa5987b8 check for cyclic deps
Summary: New test that each subpackage of pytorch3d imports cleanly.

Reviewed By: patricklabatut

Differential Revision: D30001632

fbshipit-source-id: ca8dcac94491fc22f33602b3bbef481cba927094
2021-08-23 06:16:40 -07:00
Pyre Bot Jr
fadec970c9 suppress errors in vision/fair/pytorch3d
Differential Revision: D30479084

fbshipit-source-id: 6b22dd0afe4dfb1be6249e43a56657519f11dcf1
2021-08-22 23:39:37 -07:00
Jeremy Reizenstein
1ea2b7272a sample_pdf CUDA and C++ implementations.
Summary: Implement the sample_pdf function from the NeRF project as compiled operators.. The binary search (in searchsorted) is replaced with a low tech linear search, but this is not a problem for the envisaged numbers of bins.

Reviewed By: gkioxari

Differential Revision: D26312535

fbshipit-source-id: df1c3119cd63d944380ed1b2657b6ad81d743e49
2021-08-17 08:07:55 -07:00
Jeremy Reizenstein
7d7d00f288 Move sample_pdf into PyTorch3D
Summary: Copy the sample_pdf operation from the NeRF project in to PyTorch3D, in preparation for optimizing it.

Reviewed By: gkioxari

Differential Revision: D27117930

fbshipit-source-id: 20286b007f589a4c4d53ed818c4bc5f2abd22833
2021-08-17 08:07:55 -07:00
Jeremy Reizenstein
b481cfbd01 Correct shape for default grid_sizes
Summary: Small fix for omitting this argument.

Reviewed By: nikhilaravi

Differential Revision: D29548610

fbshipit-source-id: f25032fab3faa2f09006f5fcf8628138555f2f20
2021-08-17 05:59:07 -07:00
Jeremy Reizenstein
5491b46511 Points2vols doc fixes
Summary: Fixes to a couple of comments on points to volumes, make the mask work in round_points_to_volumes, and remove a duplicate rand calculation

Reviewed By: nikhilaravi

Differential Revision: D29522845

fbshipit-source-id: 86770ba37ef3942b909baf63fd73eed1399635b6
2021-08-17 05:59:07 -07:00
Nikhila Ravi
103da63393 Ball Query
Summary:
Implementation of ball query from PointNet++.  This function is similar to KNN (find the neighbors in p2 for all points in p1). These are the key differences:
-  It will return the **first** K neighbors within a specified radius as opposed to the **closest** K neighbors.
- As all the points in p2 do not need to be considered to find the closest K, the algorithm is much faster than KNN when p2 has a large number of points.
- The neighbors are not sorted
- Due to the radius threshold it is not guaranteed that there will be K neighbors even if there are more than K points in p2.
- The padding value for `idx` is -1 instead of 0.

# Note:
- Some of the code is very similar to KNN so it could be possible to modify the KNN forward kernels to support ball query.
- Some users might want to use kNN with ball query - for this we could provide a wrapper function around the current `knn_points` which enables applying the radius threshold afterwards as an alternative. This could be called `ball_query_knn`.

Reviewed By: jcjohnson

Differential Revision: D30261362

fbshipit-source-id: 66b6a7e0114beff7164daf7eba21546ff41ec450
2021-08-12 14:06:32 -07:00
Pyre Bot Jr
9db70400d8 suppress errors in fbcode/vision - batch 2
Differential Revision: D30222339

fbshipit-source-id: 97d498df72ef897b8dc2405764e3ffd432082e3c
2021-08-10 10:21:59 -07:00
Nikhila Ravi
804117833e Fix to allow cameras in the renderer forward pass
Summary: Fix to resolve GitHub issue #796 - the cameras were being passed in the renderer forward pass instead of at initialization. The rasterizer was correctly using the cameras passed in the `kwargs` for the projection, but the `cameras` are still part of the `kwargs` for the `get_screen_to_ndc_transform` and `get_ndc_to_screen_transform` functions which is causing issues about duplicate arguments.

Reviewed By: bottler

Differential Revision: D30175679

fbshipit-source-id: 547e88d8439456e728fa2772722df5fa0fe4584d
2021-08-09 11:42:50 -07:00
Jeremy Reizenstein
4046677cf1 version 0.5.0
Summary: PyTorch3D version 0.5.0

Reviewed By: patricklabatut

Differential Revision: D29538174

fbshipit-source-id: 332516faa1d8e7bfa7c74ec3fecddc55439e2550
2021-08-03 08:10:52 -07:00
Jeremy Reizenstein
5ecce83217 PyTorch 1.4 compat
Summary: Restore compatibility with PyTorch 1.4 and 1.5, and a few lint fixes.

Reviewed By: patricklabatut

Differential Revision: D30048115

fbshipit-source-id: ee05efa7c625f6079fb06a3cc23be93e48df9433
2021-08-03 08:10:52 -07:00
CodemodService Bot
55aaec4d83 Daily arc lint --take BLACK
Reviewed By: wynsmart

Differential Revision: D30065248

fbshipit-source-id: 600915ab43d3d6d4846f60f976391f9dc1d77d10
2021-08-03 04:34:21 -07:00
Georgia Gkioxari
0c32f094af NDC/screen cameras API fix, compatibility with renderer
Summary:
API fix for NDC/screen cameras and compatibility with PyTorch3D renderers.

With this new fix:
* Users can define cameras and `transform_points` under any coordinate system conventions. The transformation applies the camera K and RT to the input points, not regarding for PyTorch3D conventions. So this makes cameras completely independent from PyTorch3D renderer.

* Cameras can be defined either in NDC space or screen space. For existing ones, FoV cameras are in NDC space. Perspective/Orthographic can be defined in NDC or screen space.

* The interface with PyTorch3D renderers happens through `transform_points_ndc` which transforms points to the NDC space and assumes that input points are provided according to PyTorch3D conventions.

* Similarly, `transform_points_screen` transforms points to screen space and again assumes that input points are under PyTorch3D conventions.

* For Orthographic/Perspective cameras, if they are defined in screen space, the `get_ndc_camera_transform` allows points to be converted to NDC for use for the renderers.

Reviewed By: nikhilaravi

Differential Revision: D26932657

fbshipit-source-id: 1a964e3e7caa54d10c792cf39c4d527ba2fb2e79
2021-08-02 01:01:10 -07:00
Patrick Labatut
9a14f54e8b Fix circular import
Summary: This fixes a recently introduced circular import: the problem went unnoticed by having `pytorch3d.renderer` imported first...

Reviewed By: bottler

Differential Revision: D29686235

fbshipit-source-id: 4b9f2faecec2cc8347ee259cfc359dc9e4f67784
2021-07-30 03:06:15 -07:00
Roman Shapovalov
e794d062e8 Improving RayBundle docstrings
Summary: This changes only documentation. We want to be explicit that ray directions are not normalised (nor assumed to be normalised) but their magnitude is used.

Reviewed By: nikhilaravi

Differential Revision: D29845210

fbshipit-source-id: b81fb3da13a42ad20e8721ed5271fd4f3d8f5acb
2021-07-23 09:33:17 -07:00
Jeremy Reizenstein
1872e0249c path_manager in obj_io
Summary: Use PathManager for checking file existence, rather than assuming the path is a local file, in a couple of cases.

Reviewed By: patricklabatut

Differential Revision: D29734621

fbshipit-source-id: e2236a7c2c50ba6916936a4d786abd601205b519
2021-07-19 05:44:20 -07:00
Jeremy Reizenstein
9e8d91ebf9 restore build tests
Summary: A bad env var check meant these tests were not being run. Fix that, and fix the copyright test for the new message format.

Reviewed By: patricklabatut

Differential Revision: D29734562

fbshipit-source-id: a1a9bb68901b09c71c7b4ff81a04083febca8d50
2021-07-19 05:44:20 -07:00
Alexey Sidnev
bcee361d04 Replace torch.det() with manual implementation for 3x3 matrix
Summary:
# Background
There is an unstable error during training (it can happen after several minutes or after several hours).
The error is connected to `torch.det()` function in  `_check_valid_rotation_matrix()`.

if I remove the function `torch.det()` in `_check_valid_rotation_matrix()` or remove the whole functions `_check_valid_rotation_matrix()` the error is disappeared (D29555876).

# Solution
Replace `torch.det()` with manual implementation for 3x3 matrix.

Reviewed By: patricklabatut

Differential Revision: D29655924

fbshipit-source-id: 41bde1119274a705ab849751ece28873d2c45155
2021-07-19 05:02:51 -07:00
Alexey Sidnev
2f668ecefe Disable gradient calculation in _check_valid_rotation_matrix()
Summary:
# Make `transform3d.py` a little bit better (performance and code quality)

## 1. Add decorator `torch.no_grad()` to the function `_check_valid_rotation_matrix()`

Function `_check_valid_rotation_matrix()` is needed to identify errors during forward pass only, it's not used for gradients.

## 2. Replace two calls `to` with the single one

Reviewed By: bottler

Differential Revision: D29656501

fbshipit-source-id: 4419e24dbf436c1b60abf77bda4376fb87a593be
2021-07-16 01:58:29 -07:00
Roman Shapovalov
0c02ae907e Adding utility methods to TensorProperties
Summary:
Context: in the code we are releasing with CO3D dataset, we use  `cuda()` on TensorProperties like Pointclouds and Cameras where we recursively move batch to a GPU. It would be good to push it to a release so we don’t need to depend on the nightly build.

Additionally, I aligned the logic of `.to("cuda")` without device index to the one of `torch.Tensor` where the current device is populated to index. It should not affect any actual use cases but some tests had to be changed.

Reviewed By: bottler

Differential Revision: D29659529

fbshipit-source-id: abe58aeaca14bacc68da3e6cf5ae07df3353e3ce
2021-07-13 10:29:26 -07:00
Christoph Lassner
fa44a05567 Fixing a bug that prevents opacity gradient calculation if no other gradients are required.
Summary: An early-return test for gradient calculation did not include the opacity gradient calculation - hence would also return early without calculating gradients even if opacity gradients are required.

Reviewed By: bottler

Differential Revision: D29505684

fbshipit-source-id: 575e820b8f58b19476b2fe3288702806733e840b
2021-07-10 01:06:56 -07:00
Christoph Lassner
75432a0695 Add OpenCV camera conversion; fix bug for camera unified PyTorch3D interface.
Summary: This commit adds a new camera conversion function for OpenCV style parameters to Pulsar parameters to the library. Using this function it addresses a bug reported here: https://fb.workplace.com/groups/629644647557365/posts/1079637302558095, by using the PyTorch3D->OpenCV->Pulsar chain instead of the original direct conversion function. Both conversions are well-tested and an additional test for the full chain has been added, resulting in a more reliable solution requiring less code.

Reviewed By: patricklabatut

Differential Revision: D29322106

fbshipit-source-id: 13df13c2e48f628f75d9f44f19ff7f1646fb7ebd
2021-07-10 01:06:56 -07:00
Patrick Labatut
fef5bcd8f9 Use rotation matrices for OpenCV / PyTorch3D conversions
Summary: Use rotation matrices for OpenCV / PyTorch3D conversions: this avoids hiding issues with conversions to / from axis-angle vectors and ensure new conversion functions have a consistent interface.

Reviewed By: bottler, classner

Differential Revision: D29634099

fbshipit-source-id: 40b28357914eb563fedea60a965dcf69e848ccfa
2021-07-09 10:26:34 -07:00
Pyre Bot Jr
44d2a9b623 suppress errors in vision/fair/pytorch3d
Differential Revision: D29573014

fbshipit-source-id: 87083e30d757fcceb4e380edc9973e07e6da6c76
2021-07-06 17:29:37 -07:00
David Novotny
4426a9d12c RayBundle visualization
Summary: Extends plotly_vis to visualize `RayBundle`s.

Reviewed By: patricklabatut

Differential Revision: D29014098

fbshipit-source-id: 4dee426510a1fa53d4afefbe1bcdd003684c9932
2021-07-01 17:31:01 -07:00
Jeremy Reizenstein
61754b2fac lint fixes
Summary: Fixing recent lint problems.

Reviewed By: patricklabatut

Differential Revision: D29522647

fbshipit-source-id: 9bd89fbfa512ecd7359ec355cf12b16fb7024b47
2021-07-01 16:08:40 -07:00
Patrick Labatut
5615f072d7 Add missing common module to auto-generated documentation
Summary: Add missing common module to auto-generated documentation

Reviewed By: nikhilaravi

Differential Revision: D29429687

fbshipit-source-id: fcbd02bda959b0f5674c344d17ce3f36ac4b85ae
2021-06-29 15:37:33 -07:00