Summary: Demonstrate current behavior of pixels with new tests of all renderers.
Reviewed By: gkioxari
Differential Revision: D32651141
fbshipit-source-id: 3ca30b4274ed2699bc5e1a9c6437eb3f0b738cbf
Summary:
All the renderers in PyTorch3D (pointclouds including pulsar, meshes, raysampling) use align_corners=False style. NDC space goes between the edges of the outer pixels. For a non square image with W>H, the vertical NDC space goes from -1 to 1 and the horizontal from -W/H to W/H.
However it was recently pointed out that functionality which deals with screen space inside the camera classes is inconsistent with this. It unintentionally uses align_corners=True. This fixes that.
This would change behaviour of the following:
- If you create a camera in screen coordinates, i.e. setting in_ndc=False, then anything you do with the camera which touches NDC space may be affected, including trying to use renderers. The transform_points_screen function will not be affected...
- If you call the function “transform_points_screen” on a camera defined in NDC space results will be different. I have illustrated in the diff how to get the old results from the new results but this probably isn’t the right long-term solution..
Reviewed By: gkioxari
Differential Revision: D32536305
fbshipit-source-id: 377325a9137282971dcb7ca11a6cba3fc700c9ce
Summary: Move benchmarks to a separate directory as tests/ is getting big.
Reviewed By: nikhilaravi
Differential Revision: D32885462
fbshipit-source-id: a832662a494ee341ab77d95493c95b0af0a83f43
Summary: As subj. Tests corrected accordingly. Also changed the test to provide a bit better diagnostics.
Reviewed By: bottler
Differential Revision: D32879498
fbshipit-source-id: 0a852e4a13dcb4ca3e54d71c6b263c5d2eeaf4eb
Summary:
- Old NDC convention had xy coords in [-1,1]x[-1,1]
- New NDC convention has xy coords in [-1, 1]x[-u, u] or [-u, u]x[-1, 1]
where u > 1 is the aspect ratio of the image.
This PR fixes the NDC raysampler to use the new convention.
Partial fix for https://github.com/facebookresearch/pytorch3d/issues/868
Pull Request resolved: https://github.com/fairinternal/pytorch3d/pull/29
Reviewed By: davnov134
Differential Revision: D31926148
Pulled By: bottler
fbshipit-source-id: c6c42c60d1473b04e60ceb49c8c10951ddf03c74
Summary:
Make sure the functions from `rotation_conversion` are jittable, and add some type hints.
Add tests to verify this is the case.
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/898
Reviewed By: patricklabatut
Differential Revision: D31926103
Pulled By: bottler
fbshipit-source-id: bff6013c5ca2d452e37e631bd902f0674d5ca091
Summary: Fix#873, that grid_sizes defaults to the wrong dtype in points2volumes code, and mask doesn't have a proper default.
Reviewed By: nikhilaravi
Differential Revision: D31503545
fbshipit-source-id: fa32a1a6074fc7ac7bdb362edfb5e5839866a472
Summary: PyTorch 1.6.0 came out on 28 Jul 2020. Stop builds for 1.5.0 and 1.5.1. Also update the news section of the README for recent releases.
Reviewed By: nikhilaravi
Differential Revision: D31442830
fbshipit-source-id: 20bdd8a07090776d0461240e71c6536d874615f6
Summary: The epsilon value is important for determining whether vertices are inside/outside a plane.
Reviewed By: gkioxari
Differential Revision: D31485247
fbshipit-source-id: 5517575de7c02f1afa277d00e0190a81f44f5761
Summary: Increase some test tolerances so that they pass in more situations, and re-enable two tests.
Reviewed By: nikhilaravi
Differential Revision: D31379717
fbshipit-source-id: 06a25470cc7b6d71cd639d9fd7df500d4b84c079
Summary:
For non square image, the NDC space in pytorch3d is not square [-1, 1]. Instead, it is [-1, 1] for the smallest side, and [-u, u] for the largest side, where u > 1. This behavior is followed by the pytorch3d renderer.
See the function `get_ndc_to_screen_transform` for a example.
Without this fix, the rendering result is not correct using the converted pytorch3d-camera from a opencv-camera on non square images.
This fix also helps the `transform_points_screen` function delivers consistent results with opencv projection for the converted pytorch3d-camera.
Reviewed By: classner
Differential Revision: D31366775
fbshipit-source-id: 8858ae7b5cf5c0a4af5a2af40a1358b2fe4cf74b
Summary: New function to randomly subsample Pointclouds to a maximum size.
Reviewed By: nikhilaravi
Differential Revision: D30936533
fbshipit-source-id: 789eb5004b6a233034ec1c500f20f2d507a303ff
Summary: Added CUDA implementation to match the new, still unused, C++ function for the core of points2vols.
Reviewed By: nikhilaravi
Differential Revision: D29548608
fbshipit-source-id: 16ebb61787fcb4c70461f9215a86ad5f97aecb4e
Summary: Single C++ function for the core of points2vols, not used anywhere yet. Added ability to control align_corners and the weight of each point, which may be useful later.
Reviewed By: nikhilaravi
Differential Revision: D29548607
fbshipit-source-id: a5cda7ec2c14836624e7dfe744c4bbb3f3d3dfe2
Summary: C++ Implementation of algorithm to compute 3D bounding boxes for batches of bboxes of shape (N, 8, 3) and (M, 8, 3).
Reviewed By: gkioxari
Differential Revision: D30905190
fbshipit-source-id: 02e2cf025cd4fa3ff706ce5cf9b82c0fb5443f96
Summary:
I have implemented an exact solution for 3D IoU of oriented 3D boxes.
This file includes:
* box3d_overlap: which computes the exact IoU of box1 and box2
* box3d_overlap_sampling: which computes an approximate IoU of box1 and box2 by sampling points within the boxes
Note that both implementations currently do not support batching.
Our exact IoU implementation is based on the fact that the intersecting shape of the two 3D boxes will be formed by segments of the surface of the boxes. Our algorithm computes these segments by reasoning whether triangles of one box are within the second box and vice versa. We deal with intersecting triangles by clipping them.
Reviewed By: gkioxari
Differential Revision: D30667497
fbshipit-source-id: 2f747f410f90b7f854eeaf3036794bc3ac982917
Summary: Attempt to fix#659, an observation that the rasterizer is nondeterministic, by resolving tied faces by picking those with lower index.
Reviewed By: nikhilaravi, patricklabatut
Differential Revision: D30699039
fbshipit-source-id: 39ed797eb7e9ce7370ae71259ad6b757f9449923
Summary:
CUDA implementation of farthest point sampling algorithm.
## Visual comparison
Compared to random sampling, farthest point sampling gives better coverage of the shape.
{F658631262}
## Reduction
Parallelized block reduction to find the max value at each iteration happens as follows:
1. First split the points into two equal sized parts (e.g. for a list with 8 values):
`[20, 27, 6, 8 | 11, 10, 2, 33]`
2. Use half of the thread (4 threads) to compare pairs of elements from each half (e.g elements [0, 4], [1, 5] etc) and store the result in the first half of the list:
`[20, 27, 6, 33 | 11, 10, 2, 33]`
Now we no longer care about the second part but again divide the first part into two
`[20, 27 | 6, 33| -, -, -, -]`
Now we can use 2 threads to compare the 4 elements
4. Finally we have gotten down to a single pair
`[20 | 33 | -, - | -, -, -, -]`
Use 1 thread to compare the remaining two elements
5. The max will now be at thread id = 0
`[33 | - | -, - | -, -, -, -]`
The reduction will give the farthest point for the selected batch index at this iteration.
Reviewed By: bottler, jcjohnson
Differential Revision: D30401803
fbshipit-source-id: 525bd5ae27c4b13b501812cfe62306bb003827d2
Summary:
This is a naive python implementation of the iterative farthest point sampling algorithm along with associated simple tests. The C++/CUDA implementations will follow in subsequent diffs.
The algorithm is used to subsample a pointcloud with better coverage of the space of the pointcloud.
The function has not been added to `__init__.py`. I will add this after the full C++/CUDA implementations.
Reviewed By: jcjohnson
Differential Revision: D30285716
fbshipit-source-id: 33f4181041fc652776406bcfd67800a6f0c3dd58
Summary: Fix issue #826. This is a correction to the joining of TexturesUV into a single scene.
Reviewed By: nikhilaravi
Differential Revision: D30767092
fbshipit-source-id: 03ba6a1d2f22e569d1b3641cd13ddbb8dcb87ec7
Summary:
* HAT_INV_SKEW_SYMMETRIC_TOL was a global variable and torch script gives an error when compiling that function. Move it to the function scope.
* torch script gives error when compiling acos_linear_extrapolation because bound is a union of tuple and float. The tuple version is kept in this diff.
Reviewed By: patricklabatut
Differential Revision: D30614916
fbshipit-source-id: 34258d200dc6a09fbf8917cac84ba8a269c00aef
Summary: In D30349234 (1b8d86a104) we introduced persistent=False to some register_buffer calls, which depend on PyTorch 1.6. We go back to the old behaviour for PyTorch 1.5.
Reviewed By: nikhilaravi
Differential Revision: D30731327
fbshipit-source-id: ab02ef98ee87440ef02479b72f4872b562ab85b5
Summary: Change cyclic deps test to be independent of test discovery order. Also let it work without plotly.
Reviewed By: nikhilaravi
Differential Revision: D30669614
fbshipit-source-id: 2eadf3f8b56b6096c5466ce53b4f8ac6df27b964
Summary: Fixes GitHub issue #751. The vectorized implementation of bilinear interpolation didn't properly handle the edge cases in the same way as the `grid_sample` method in PyTorch.
Reviewed By: bottler
Differential Revision: D30684208
fbshipit-source-id: edf241ecbd72d46b94ad340a4e601e26c83db88e
Summary:
As suggested in #802. By not persisting the _xy_grid buffer, we can allow (in some cases) a model with one image_size to be loaded from a saved model which was trained at a different resolution.
Also avoid persisting _frequencies in HarmonicEmbedding for similar reasons.
BC-break: This will cause load_state_dict, in strict mode, to complain if you try to load an old model with the new code.
Reviewed By: patricklabatut
Differential Revision: D30349234
fbshipit-source-id: d6061d1e51c9f79a78d61a9f732c9a5dfadbbb47
Summary:
Use PyTorch3D's new faster sample_pdf function instead of local Python implementation.
Also clarify deps for the Python implementation.
Reviewed By: gkioxari
Differential Revision: D30512109
fbshipit-source-id: 84cfdc00313fada37a6b29837de96f6a4646434f
Summary: New test that each subpackage of pytorch3d imports cleanly.
Reviewed By: patricklabatut
Differential Revision: D30001632
fbshipit-source-id: ca8dcac94491fc22f33602b3bbef481cba927094
Summary: Implement the sample_pdf function from the NeRF project as compiled operators.. The binary search (in searchsorted) is replaced with a low tech linear search, but this is not a problem for the envisaged numbers of bins.
Reviewed By: gkioxari
Differential Revision: D26312535
fbshipit-source-id: df1c3119cd63d944380ed1b2657b6ad81d743e49
Summary: Copy the sample_pdf operation from the NeRF project in to PyTorch3D, in preparation for optimizing it.
Reviewed By: gkioxari
Differential Revision: D27117930
fbshipit-source-id: 20286b007f589a4c4d53ed818c4bc5f2abd22833
Summary: Fixes to a couple of comments on points to volumes, make the mask work in round_points_to_volumes, and remove a duplicate rand calculation
Reviewed By: nikhilaravi
Differential Revision: D29522845
fbshipit-source-id: 86770ba37ef3942b909baf63fd73eed1399635b6
Summary: Much of the code is actually available during the conda tests, as long as we look in the right place. We enable some of them.
Reviewed By: nikhilaravi
Differential Revision: D30249357
fbshipit-source-id: 01c57b6b8c04442237965f23eded594aeb90abfb
Summary:
Implementation of ball query from PointNet++. This function is similar to KNN (find the neighbors in p2 for all points in p1). These are the key differences:
- It will return the **first** K neighbors within a specified radius as opposed to the **closest** K neighbors.
- As all the points in p2 do not need to be considered to find the closest K, the algorithm is much faster than KNN when p2 has a large number of points.
- The neighbors are not sorted
- Due to the radius threshold it is not guaranteed that there will be K neighbors even if there are more than K points in p2.
- The padding value for `idx` is -1 instead of 0.
# Note:
- Some of the code is very similar to KNN so it could be possible to modify the KNN forward kernels to support ball query.
- Some users might want to use kNN with ball query - for this we could provide a wrapper function around the current `knn_points` which enables applying the radius threshold afterwards as an alternative. This could be called `ball_query_knn`.
Reviewed By: jcjohnson
Differential Revision: D30261362
fbshipit-source-id: 66b6a7e0114beff7164daf7eba21546ff41ec450
Summary: New test that notes and tutorials are listed in the website metadata, so that they will be included in the website build.
Reviewed By: nikhilaravi
Differential Revision: D30223799
fbshipit-source-id: 2dca9730b54e68da2fd430a7b47cb7e18814d518
Summary: Fix to resolve GitHub issue #796 - the cameras were being passed in the renderer forward pass instead of at initialization. The rasterizer was correctly using the cameras passed in the `kwargs` for the projection, but the `cameras` are still part of the `kwargs` for the `get_screen_to_ndc_transform` and `get_ndc_to_screen_transform` functions which is causing issues about duplicate arguments.
Reviewed By: bottler
Differential Revision: D30175679
fbshipit-source-id: 547e88d8439456e728fa2772722df5fa0fe4584d
Summary:
API fix for NDC/screen cameras and compatibility with PyTorch3D renderers.
With this new fix:
* Users can define cameras and `transform_points` under any coordinate system conventions. The transformation applies the camera K and RT to the input points, not regarding for PyTorch3D conventions. So this makes cameras completely independent from PyTorch3D renderer.
* Cameras can be defined either in NDC space or screen space. For existing ones, FoV cameras are in NDC space. Perspective/Orthographic can be defined in NDC or screen space.
* The interface with PyTorch3D renderers happens through `transform_points_ndc` which transforms points to the NDC space and assumes that input points are provided according to PyTorch3D conventions.
* Similarly, `transform_points_screen` transforms points to screen space and again assumes that input points are under PyTorch3D conventions.
* For Orthographic/Perspective cameras, if they are defined in screen space, the `get_ndc_camera_transform` allows points to be converted to NDC for use for the renderers.
Reviewed By: nikhilaravi
Differential Revision: D26932657
fbshipit-source-id: 1a964e3e7caa54d10c792cf39c4d527ba2fb2e79
Summary: A bad env var check meant these tests were not being run. Fix that, and fix the copyright test for the new message format.
Reviewed By: patricklabatut
Differential Revision: D29734562
fbshipit-source-id: a1a9bb68901b09c71c7b4ff81a04083febca8d50
Summary:
# Background
There is an unstable error during training (it can happen after several minutes or after several hours).
The error is connected to `torch.det()` function in `_check_valid_rotation_matrix()`.
if I remove the function `torch.det()` in `_check_valid_rotation_matrix()` or remove the whole functions `_check_valid_rotation_matrix()` the error is disappeared (D29555876).
# Solution
Replace `torch.det()` with manual implementation for 3x3 matrix.
Reviewed By: patricklabatut
Differential Revision: D29655924
fbshipit-source-id: 41bde1119274a705ab849751ece28873d2c45155
Summary:
Context: in the code we are releasing with CO3D dataset, we use `cuda()` on TensorProperties like Pointclouds and Cameras where we recursively move batch to a GPU. It would be good to push it to a release so we don’t need to depend on the nightly build.
Additionally, I aligned the logic of `.to("cuda")` without device index to the one of `torch.Tensor` where the current device is populated to index. It should not affect any actual use cases but some tests had to be changed.
Reviewed By: bottler
Differential Revision: D29659529
fbshipit-source-id: abe58aeaca14bacc68da3e6cf5ae07df3353e3ce
Summary: This commit adds a new camera conversion function for OpenCV style parameters to Pulsar parameters to the library. Using this function it addresses a bug reported here: https://fb.workplace.com/groups/629644647557365/posts/1079637302558095, by using the PyTorch3D->OpenCV->Pulsar chain instead of the original direct conversion function. Both conversions are well-tested and an additional test for the full chain has been added, resulting in a more reliable solution requiring less code.
Reviewed By: patricklabatut
Differential Revision: D29322106
fbshipit-source-id: 13df13c2e48f628f75d9f44f19ff7f1646fb7ebd
Summary: Use rotation matrices for OpenCV / PyTorch3D conversions: this avoids hiding issues with conversions to / from axis-angle vectors and ensure new conversion functions have a consistent interface.
Reviewed By: bottler, classner
Differential Revision: D29634099
fbshipit-source-id: 40b28357914eb563fedea60a965dcf69e848ccfa
Summary: Enable this benchmark to be run on its own, like others.
Reviewed By: patricklabatut
Differential Revision: D29522846
fbshipit-source-id: c7b3b5c9a0fcdeeb79d8b2ec197684b4380aa547
Summary: solve and lstsq have moved around in torch. Cope with both.
Reviewed By: patricklabatut
Differential Revision: D29302316
fbshipit-source-id: b34f0b923e90a357f20df359635929241eba6e74
