148 Commits

Author SHA1 Message Date
Jeremy Reizenstein
f2d1d2db69 Alternative type names in PLY #205
Summary: Add ability to decode ply files which use types like int32.

Reviewed By: nikhilaravi

Differential Revision: D21639208

fbshipit-source-id: 0ede7d4aa353a6e940446680a18e7ac0c48fafee
2020-05-20 01:13:23 -07:00
Michele Sanna
b4fd9d1d34 fixes to shader error strings (#204)
Summary: Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/204

Reviewed By: gkioxari

Differential Revision: D21621695

Pulled By: nikhilaravi

fbshipit-source-id: 556f297bef4211c331dfde6471e10762a9956f98
2020-05-18 19:37:56 -07:00
Justin Johnson
d8987c6f48 Add benchmark for diffuse and specular lighting
Summary: I was trying to speed up the lighting computations, but my ideas didn't work. Even if that didn't work, we can at least commit the benchmarking script I wrote for diffuse and specular shading.

Reviewed By: nikhilaravi

Differential Revision: D21580171

fbshipit-source-id: 8b60c0284e91ecbe258b6aae839bd5c2bbe788aa
2020-05-16 15:55:10 -07:00
Nikhila Ravi
3fef506895 Make cuda tensors contiguous in host function and remove contiguous check
Summary:
Update the cuda kernels to:
- remove contiguous checks for the grad tensors and for cpu functions which use accessors
- for cuda implementations call `.contiguous()` on all tensors in the host function before invoking the kernel

Reviewed By: gkioxari

Differential Revision: D21598008

fbshipit-source-id: 9b97bda4582fd4269c8a00999874d4552a1aea2d
2020-05-15 15:00:25 -07:00
Roman Shapovalov
a8377f1f06 Numerical stability of ePnP.
Summary: lg-zhang found the problem with the quadratic part of ePnP implementation: n262385 . It was caused by a coefficient returned from the linear equation solver being equal to exactly 0.0, which caused `sign()` to return 0, something I had not anticipated. I also made sure we avoid division by zero by clamping all relevant denominators.

Reviewed By: nikhilaravi, lg-zhang

Differential Revision: D21531200

fbshipit-source-id: 9eb2fa9d4f4f8f5f411d4cf1cffcc44b365b7e51
2020-05-15 01:36:21 -07:00
Georgia Gkioxari
a0e14cae1e flat shading fix
Summary:
Make flat shading differentiable again

Currently test fails with P130944403 which looks weird.

Reviewed By: nikhilaravi

Differential Revision: D21567106

fbshipit-source-id: 65995b64739e08397b3d021b65625e3c377cd1a5
2020-05-14 13:34:09 -07:00
Jeremy Reizenstein
728179e848 avoid converting a TensorOptions from float to integer
Summary: pytorch is adding checks that mean integer tensors with requires_grad=True need to be avoided. Fix accidentally creating them.

Reviewed By: jcjohnson, gkioxari

Differential Revision: D21576712

fbshipit-source-id: 008218997986800a36d93caa1a032ee91f2bffcd
2020-05-14 13:16:05 -07:00
Jeremy Reizenstein
6a365d203f Pointclouds, Meshes and Textures self-references
Summary: Use `self.__class__` when creating new instances, to slightly accommodate inheritance.

Reviewed By: nikhilaravi

Differential Revision: D21504476

fbshipit-source-id: b4600d15462fc1985da95a4cf761c7d794cfb0bb
2020-05-11 12:57:55 -07:00
David Novotny
34a0df0630 SO3 log map fix for singularity at PI
Summary:
Fixes the case where the rotation angle is exactly 0/PI.
Added a test for `so3_log_map(identity_matrix)`.

Reviewed By: nikhilaravi

Differential Revision: D21477078

fbshipit-source-id: adff804da97f6f0d4f50aa1f6904a34832cb8bfe
2020-05-10 13:15:37 -07:00
Nikhila Ravi
17ca6ecd81 allow cameras to be None in rasterizer initialization
Summary: Fix to enable a mesh/point rasterizer to be initialized without having to specify the camera.

Reviewed By: jcjohnson, gkioxari

Differential Revision: D21362359

fbshipit-source-id: 4f84ea18ad9f179c7b7c2289ebf9422a2f5e26de
2020-05-05 22:32:57 -07:00
ywang
9c5ab57156 fix clone type issue (#179)
Summary:
a quick fix for the clone issue

fixes https://github.com/facebookresearch/pytorch3d/issues/178
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/179

Reviewed By: bottler

Differential Revision: D21414309

Pulled By: nikhilaravi

fbshipit-source-id: 359d7724aa5d78bc88a0a9ffc05e6041056e3b3f
2020-05-05 22:32:57 -07:00
Georgia Gkioxari
a61c9376d5 add align modes for cubify
Summary: Add alignment modes for cubify operation.

Reviewed By: nikhilaravi

Differential Revision: D21393199

fbshipit-source-id: 7022044e591229a6ed5efc361fd3215e65f43f86
2020-05-05 11:09:45 -07:00
Jeremy Reizenstein
8fc28baa27 Looser gradient check in test_rasterize_meshes
Summary: This has been failing intermittently

Reviewed By: nikhilaravi

Differential Revision: D21403157

fbshipit-source-id: 51b74d6c813b52effe72d14b565e250fcabbb463
2020-05-05 09:26:47 -07:00
Jeremy Reizenstein
06ad1fb6c4 KNN return order documentation
Summary: Fix documentation of KNN, issue #180

Reviewed By: gkioxari

Differential Revision: D21384761

fbshipit-source-id: 2b36ee496f2060d17827d2fd66c490cdfa766866
2020-05-04 13:37:25 -07:00
Nikhila Ravi
0eca74fa5f lint fixes
Summary:
Ran the linter.
TODO: need to update the linter as per D21353065.

Reviewed By: bottler

Differential Revision: D21362270

fbshipit-source-id: ad0e781de0a29f565ad25c43bc94a19b1828c020
2020-05-04 09:56:44 -07:00
Jeremy Reizenstein
0c595dcf5b Joining mismatched texture maps on CUDA #175
Summary:
Use nn.functional.interpolate instead of a TorchVision transform to resize texture maps to a common value. This works on all devices. This fixes issue #175.

Also fix the condition so it only happens when needed.

Reviewed By: nikhilaravi

Differential Revision: D21324510

fbshipit-source-id: c50eb06514984995bd81f2c44079be6e0b4098e4
2020-05-01 05:20:10 -07:00
Georgia Gkioxari
e64e0d17ef fix self assign for normals est
Summary: Fix self assignment of normals when estimating normals

Reviewed By: nikhilaravi

Differential Revision: D21315980

fbshipit-source-id: 2aa5864c3f066e39e07343f192cc6423ce1ae771
2020-04-30 14:27:08 -07:00
Jeremy Reizenstein
686c8666d3 version 0.2.0
Summary: Update version number for version 0.2.0.

Reviewed By: nikhilaravi

Differential Revision: D21157358

fbshipit-source-id: 32a5b93e5dc65a31a806a5ce7231f8603fe02e85
v0.2.0
2020-04-26 07:18:57 -07:00
Jeremy Reizenstein
232e4a7e3d Driver update for ci, easier diagnosing
Summary: Bump the nvidia driver used in the conda tests. Add an environment variable (unused) to allow building without ninja. Print relative error on assertClose failure.

Reviewed By: nikhilaravi

Differential Revision: D21227373

fbshipit-source-id: 5dd8eb097151da27d3632daa755a1e7b9ac97845
2020-04-25 16:03:42 -07:00
Nikhila Ravi
0cfa6a122b README updates
Summary:
Some updates to the issue templates, readme and install.md

Creating an FAQ for installation similar to: https://github.com/facebookresearch/detectron2/blob/master/INSTALL.md#common-installation-issues

Reviewed By: gkioxari

Differential Revision: D21117899

fbshipit-source-id: d287c3a7a99c2e425b4e0cffca55a7b225d79e11
2020-04-24 17:18:50 -07:00
Nikhila Ravi
cf84dacf2e fix get cuda device test error
Summary:
Cuda test failing on circle with the error `random_ expects 'from' to be less than 'to', but got from=0 >= to=0`

This is because the `high` value in `torch.randint` is 1 more than the highest value in the distribution from which a value is drawn. So if there is only 1 cuda device available then the low and high are 0.

Reviewed By: gkioxari

Differential Revision: D21236669

fbshipit-source-id: 46c312d431c474f1f2c50747b1d5e7afbd7df3a9
2020-04-24 16:12:49 -07:00
Michele Sanna
f8acecb6b3 a formula for bin size for images over 64x64 (#90)
Summary:
Signed-off-by: Michele Sanna <sanna@arrival.com>

fixes the bin_size calculation with a formula for any image_size > 64. Matches the values chosen so far.

simple test:

```
import numpy as np
import matplotlib.pyplot as plt

image_size = np.arange(64, 2048)
bin_size = np.where(image_size <= 64, 8, (2 ** np.maximum(np.ceil(np.log2(image_size)) - 4, 4)).astype(int))

print(image_size)
print(bin_size)

for ims, bins in zip(image_size, bin_size):
    if ims <= 64:
        assert bins == 8
    elif ims <= 256:
        assert bins == 16
    elif ims <= 512:
        assert bins == 32
    elif ims <= 1024:
        assert bins == 64
    elif ims <= 2048:
        assert bins == 128

    assert (ims + bins - 1) // bins < 22

plt.plot(image_size, bin_size)
plt.grid()
plt.show()
```

![img](https://user-images.githubusercontent.com/54891577/75464693-795bcf00-597f-11ea-9061-26440211691c.png)
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/90

Reviewed By: jcjohnson

Differential Revision: D21160372

Pulled By: nikhilaravi

fbshipit-source-id: 660cf5832f4ca5be243c435a6bed969596fc0188
2020-04-24 14:56:41 -07:00
Nikhila Ravi
c3d636dc8c Cuda updates
Summary:
Updates to:
- enable cuda kernel launches on any GPU (not just the default)
- cuda and contiguous checks for all kernels
- checks to ensure all tensors are on the same device
- error reporting in the cuda kernels
- cuda tests now run on a random device not just the default

Reviewed By: jcjohnson, gkioxari

Differential Revision: D21215280

fbshipit-source-id: 1bedc9fe6c35e9e920bdc4d78ed12865b1005519
2020-04-24 09:11:04 -07:00
Nikhila Ravi
c9267ab7af Update load obj and compare with SoftRas
Summary:
Updated the load obj function to support creating of a per face texture map using the information in an .mtl file. Uses the approach from in SoftRasterizer.

Currently I have ported in the SoftRasterizer code but this is only to help with comparison and will  be deleted before landing. The ShapeNet Test data will also be deleted.

Here is the [Design doc](https://docs.google.com/document/d/1AUcLP4QwVSqlfLAUfbjM9ic5vYn9P54Ha8QbcVXW2eI/edit?usp=sharing).

## Added
- texture atlas creation functions in PyTorch based on the SoftRas cuda implementation
- tests to compare SoftRas vs PyTorch3D implementation to verify it matches (using real shapenet data with meshes consisting of multiple textures)
- benchmarks tests

## Remaining todo:
- add more tests for obj io to test the new functions and the two texturing options
- replace the shapenet data with the output from SoftRas saved as a file.

# MAIN FILES TO REVIEW

- `obj_io.py`
- `test_obj_io.py` [still some tests to be added but have comparisons with SoftRas for now]

The reference SoftRas implementations are in `softras_load_obj.py` and `load_textures.cu`.

Reviewed By: gkioxari

Differential Revision: D20754859

fbshipit-source-id: 42ace9dfb73f26e29d800c763f56d5b66c60c5e2
2020-04-23 19:38:35 -07:00
Jeremy Reizenstein
85c396f822 avoid using torch/extension.h in cuda
Summary:
Use aten instead of torch interface in all cuda code. This allows the cuda build to work with pytorch 1.5 with GCC 5 (e.g. the compiler of ubuntu 16.04LTS). This wasn't working. It has been failing with errors like the below, perhaps due to a bug in nvcc.

```
torch/include/torch/csrc/api/include/torch/nn/cloneable.h:68:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >
```

Reviewed By: nikhilaravi

Differential Revision: D21204029

fbshipit-source-id: ca6bdbcecf42493365e1c23a33fe35e1759fe8b6
2020-04-23 10:26:17 -07:00
Roman Shapovalov
54b482bd66 Not normalising control points by X.std()
Summary:
davnov134 found that the algorithm crashes if X is an axis-aligned plane. This is because I implemented scaling control points by `X.std()` as a poor man’s version of PCA whitening.
I checked that it does not bring consistent improvements, so let’s get rid of it.

The algorithm still results in slightly higher errors on the axis aligned planes but at least it does not crash. As a next step, I will experiment with detecting a planar case and using 3-point barycentric coordinates rather than 4-points.

Reviewed By: davnov134

Differential Revision: D21179968

fbshipit-source-id: 1f002fce5541934486b51808be0e910324977222
2020-04-23 06:04:54 -07:00
Justin Johnson
9f31a4fd46 Expose knn_check_version in python
Summary:
We have multiple KNN CUDA implementations. From python, users can currently request a particular implementation via the `version` flag, but they have no way of knowing which implementations can be used for a given problem.

This diff exposes a function `pytorch3d._C.knn_check_version(version, D, K)` that returns whether a particular version can be used.

Reviewed By: nikhilaravi

Differential Revision: D21162573

fbshipit-source-id: 6061960bdcecba454fd920b00036f4e9ff3fdbc0
2020-04-22 14:30:52 -07:00
Nikhila Ravi
e38cbfe5e3 fix comment in point rasterizer
Reviewed By: gkioxari

Differential Revision: D21180328

fbshipit-source-id: 218919c614c1ea54b5147871bd91960b8346524b
2020-04-22 12:06:22 -07:00
Jeremy Reizenstein
9e4bd2e5e0 chamfer test consistency
Summary:
Modify test_chamfer for more robustness. Avoid empty pointclouds, including where point_reduction is mean, for which we currently return nan (*), and so that we aren't looking at an empty gradient. Make sure we aren't using padding as points in the homogenous cases in the tests, which will lead to a tie between closest points and therefore a potential instability in the gradient - see https://github.com/pytorch/pytorch/issues/35699.

(*) This doesn't attempt to fix the nan.

Reviewed By: nikhilaravi, gkioxari

Differential Revision: D21157322

fbshipit-source-id: a609e84e25a24379c8928ff645d587552526e4af
2020-04-22 09:28:51 -07:00
Jeremy Reizenstein
faf0885811 Cuda 10.2 for builds
Summary: cuda 10.2 location on linux. Also remove unused conda test dependencies.

Reviewed By: nikhilaravi

Differential Revision: D21176409

fbshipit-source-id: dd3f339a92233ff16877ba76506ddf8f4418715d
2020-04-22 09:09:59 -07:00
Nikhila Ravi
4bf30593ff back face culling in rasterization
Summary:
Added backface culling as an option to the `raster_settings`. This is needed for the full forward rendering of shapenet meshes with texture (some meshes contain
multiple overlapping segments which have different textures).

For a triangle (v0, v1, v2) define the vectors A = (v1 - v0) and B = (v2 − v0) and use this to calculate the area of the triangle as:
```
area = 0.5 * A  x B
area = 0.5 * ((x1 − x0)(y2 − y0) − (x2 − x0)(y1 − y0))
```
The area will be positive if (v0, v1, v2) are oriented counterclockwise (a front face), and negative if (v0, v1, v2) are oriented clockwise (a back face).

We can reuse the `edge_function` as it already calculates the triangle area.

Reviewed By: jcjohnson

Differential Revision: D20960115

fbshipit-source-id: 2d8a4b9ccfb653df18e79aed8d05c7ec0f057ab1
2020-04-22 08:22:46 -07:00
charleschiu2012
3c6f9220fc fix definition in function pytorch3d.renderer.cameras.look_at_view_transform (#120)
Summary:
fix Args' definition at line 1016, 1018, 1020 in function pytorch3d.renderer.cameras.look_at_view_transform.
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/120

Reviewed By: bottler

Differential Revision: D20597565

Pulled By: nikhilaravi

fbshipit-source-id: e10a221e3dccc0adf20b26808ad67328408a4388
2020-04-21 15:20:55 -07:00
Jeremy Reizenstein
a53a2d3731 Builds with pytorch 1.5
Summary:
Add conda packages for pytorch 1.5. Make wheels be only pytorch 1.5.

Note that pytorch 1.4 has conda packages for cuda 9.2, 10.0 and 10.1, whilst pytorch 1.5 has packages for cuda 9.2, 10.1 and 10.2. We mirror these choices.

Reviewed By: nikhilaravi

Differential Revision: D21157392

fbshipit-source-id: 2f7311e6a83774a6d6c8afb8110b8bd9f37f1454
2020-04-21 12:15:01 -07:00
Georgia Gkioxari
f2b229c1d1 pytorch3d compatibility
Summary: Making meshrcnn compatible with new PyTorch3D features/API changes.

Reviewed By: nikhilaravi

Differential Revision: D21149516

fbshipit-source-id: 1c7b8c1c1f5a2abe7d379fee10ded5d2db21515a
2020-04-20 23:04:24 -07:00
Nikhila Ravi
9ef1ee8455 coarse rasterization bug fix
Summary:
Fix a bug which resulted in a rendering artifacts if the image size was not a multiple of 16.
Fix: Revert coarse rasterization to original implementation and only update fine rasterization to reverse the ordering of Y and X axis. This is much simpler than the previous approach!

Additional changes:
- updated mesh rendering end-end tests to check outputs from both naive and coarse to fine rasterization.
- added pointcloud rendering end-end tests

Reviewed By: gkioxari

Differential Revision: D21102725

fbshipit-source-id: 2e7e1b013dd6dd12b3a00b79eb8167deddb2e89a
2020-04-20 14:54:16 -07:00
Jeremy Reizenstein
1e4749602d skip code tests in conda build
Summary:
None of the current test_build tests make sense during `conda build`.

Also remove the unnecessary dependency on the `six` library.

Reviewed By: nikhilaravi

Differential Revision: D20893852

fbshipit-source-id: 685f0446eaa0bd9151eeee89fc630a1ddc0252ff
2020-04-20 12:19:45 -07:00
Jeremy Reizenstein
6207c359b1 spelling and flake
Summary: mostly recent lintish things

Reviewed By: nikhilaravi

Differential Revision: D21089003

fbshipit-source-id: 028733c1d875268f1879e4481da475b7100ba0b6
2020-04-17 10:50:22 -07:00
Jeremy Reizenstein
9397cd872d torch C API warnings
Summary: This is mostly replacing the old PackedTensorAccessor with the new PackedTensorAccessor64.

Reviewed By: gkioxari

Differential Revision: D21088773

fbshipit-source-id: 5973e5a29d934eafb7c70ec5ec154ca076b64d27
2020-04-17 10:46:31 -07:00
Jeremy Reizenstein
f25af96959 vert_align for Pointclouds object
Reviewed By: gkioxari

Differential Revision: D21088730

fbshipit-source-id: f8c125ac8c8009d45712ae63237ca64acf1faf45
2020-04-17 10:39:43 -07:00
Jeremy Reizenstein
e19df58766 remove final nearest_neighbor files
Summary: A couple of files for the removed nearest_neighbor functionality are left behind.

Reviewed By: nikhilaravi

Differential Revision: D21088624

fbshipit-source-id: 4bb29016b4e5f63102765b384c363733b60032fa
2020-04-17 09:27:17 -07:00
Roman Shapovalov
04d8bf6a43 Efficient PnP.
Summary:
Efficient PnP algorithm to fit 2D to 3D correspondences under perspective assumption.

Benchmarked both variants of nullspace and pick one; SVD takes 7 times longer in the 100K points case.

Reviewed By: davnov134, gkioxari

Differential Revision: D20095754

fbshipit-source-id: 2b4519729630e6373820880272f674829eaed073
2020-04-17 07:44:16 -07:00
David Novotny
7788a38050 Camera inheritance + unprojections
Summary: Made a CameraBase class. Added `unproject_points` method for each camera class.

Reviewed By: nikhilaravi

Differential Revision: D20373602

fbshipit-source-id: 7e3da5ae420091b5fcab400a9884ef29ad7a7343
2020-04-17 04:38:14 -07:00
David Novotny
365945b1fd Pointcloud normals estimation.
Summary: Estimates normals of a point cloud.

Reviewed By: gkioxari

Differential Revision: D20860182

fbshipit-source-id: 652ec2743fa645e02c01ffa37c2971bf27b89cef
2020-04-16 18:36:19 -07:00
David Novotny
8abbe22ffb ICP - point-to-point version
Summary:
The iterative closest point algorithm - point-to-point version.

Output of `bm_iterative_closest_point`:
Argument key: `batch_size dim n_points_X n_points_Y use_pointclouds`

```
Benchmark                                         Avg Time(μs)      Peak Time(μs) Iterations
--------------------------------------------------------------------------------
IterativeClosestPoint_1_3_100_100_False              107569          111323              5
IterativeClosestPoint_1_3_100_1000_False             118972          122306              5
IterativeClosestPoint_1_3_1000_100_False             108576          110978              5
IterativeClosestPoint_1_3_1000_1000_False            331836          333515              2
IterativeClosestPoint_1_20_100_100_False             134387          137842              4
IterativeClosestPoint_1_20_100_1000_False            149218          153405              4
IterativeClosestPoint_1_20_1000_100_False            414248          416595              2
IterativeClosestPoint_1_20_1000_1000_False           374318          374662              2
IterativeClosestPoint_10_3_100_100_False             539852          539852              1
IterativeClosestPoint_10_3_100_1000_False            752784          752784              1
IterativeClosestPoint_10_3_1000_100_False           1070700         1070700              1
IterativeClosestPoint_10_3_1000_1000_False          1164020         1164020              1
IterativeClosestPoint_10_20_100_100_False            374548          377337              2
IterativeClosestPoint_10_20_100_1000_False           472764          476685              2
IterativeClosestPoint_10_20_1000_100_False          1457175         1457175              1
IterativeClosestPoint_10_20_1000_1000_False         2195820         2195820              1
IterativeClosestPoint_1_3_100_100_True               110084          115824              5
IterativeClosestPoint_1_3_100_1000_True              142728          147696              4
IterativeClosestPoint_1_3_1000_100_True              212966          213966              3
IterativeClosestPoint_1_3_1000_1000_True             369130          375114              2
IterativeClosestPoint_10_3_100_100_True              354615          355179              2
IterativeClosestPoint_10_3_100_1000_True             451815          452704              2
IterativeClosestPoint_10_3_1000_100_True             511833          511833              1
IterativeClosestPoint_10_3_1000_1000_True            798453          798453              1
--------------------------------------------------------------------------------
```

Reviewed By: shapovalov, gkioxari

Differential Revision: D19909952

fbshipit-source-id: f77fadc88fb7c53999909d594114b182ee2a3def
2020-04-16 14:02:16 -07:00
Nikhila Ravi
b5eb33b36c bugfix
Summary: It seemed that even though the chamfer diff was rebased on top of the knn autograd diff, some of the final updates did not get applied. I'm really surprised that the sandcastle tests did not fail and prevent the diff from landing.

Reviewed By: gkioxari

Differential Revision: D21066156

fbshipit-source-id: 5216efe95180c1b6082d0bac404fa1920cfb7b02
2020-04-16 11:58:20 -07:00
Nikhila Ravi
b530b0af32 lint fixes
Summary: Resolved trailing whitespace warnings.

Reviewed By: gkioxari

Differential Revision: D21023982

fbshipit-source-id: 14ea2ca372c13cfa987acc260264ca99ce44c461
2020-04-15 21:58:59 -07:00
Nikhila Ravi
3794f6753f remove nearest_neighbors
Summary: knn is more general and faster than the nearest_neighbor code, so remove the latter.

Reviewed By: gkioxari

Differential Revision: D20816424

fbshipit-source-id: 75d6c44d17180752d0c9859814bbdf7892558158
2020-04-15 20:51:41 -07:00
Nikhila Ravi
790eb8c402 Chamfer for Pointclouds object
Summary:
Allow Pointclouds objects and heterogenous data to be provided for Chamfer loss. Remove "none" as an option for point_reduction because it doesn't make sense and in the current implementation is effectively the same as "sum".

Possible improvement: create specialised operations for sum and cosine_similarity of padded tensors, to avoid having to create masks. sum would be useful elsewhere.

Reviewed By: gkioxari

Differential Revision: D20816301

fbshipit-source-id: 0f32073210225d157c029d80de450eecdb64f4d2
2020-04-15 14:10:45 -07:00
Siddhant Ranade
677b0bd5ae SfMPerspectiveCameras projection matrix bug fix (#148)
Summary:
Fixed a bug in creating the projection matrix for the SfMPerspectiveCameras class. Also fixed the corresponding test in test_cameras.py.

The p0x, p0y are 2D coordinates for the principal point in the NDCS, and therefore should be added AFTER the perspective z divide. I.e. we expect

<a href="https://www.codecogs.com/eqnedit.php?latex=\begin{bmatrix}&space;x\&space;y\&space;z&space;\end{bmatrix}_{\text{NDCS}}&space;=&space;\begin{bmatrix}&space;f_xX/Z&space;&plus;&space;p_x\&space;f_yY/Z&space;&plus;&space;p_y\&space;1&space;/&space;Z\&space;\end{bmatrix}&space;=&space;\text{normalize}\left(&space;\begin{bmatrix}&space;f_xX&space;&plus;&space;p_xZ\&space;f_yY&space;&plus;&space;p_yZ\&space;1\&space;Z&space;\end{bmatrix}&space;\right)" target="_blank"><img src="https://latex.codecogs.com/gif.latex?\begin{bmatrix}&space;x\&space;y\&space;z&space;\end{bmatrix}_{\text{NDCS}}&space;=&space;\begin{bmatrix}&space;f_xX/Z&space;&plus;&space;p_x\&space;f_yY/Z&space;&plus;&space;p_y\&space;1&space;/&space;Z\&space;\end{bmatrix}&space;=&space;\text{normalize}\left(&space;\begin{bmatrix}&space;f_xX&space;&plus;&space;p_xZ\&space;f_yY&space;&plus;&space;p_yZ\&space;1\&space;Z&space;\end{bmatrix}&space;\right)" title="\begin{bmatrix} x\ y\ z \end{bmatrix}_{\text{NDCS}} = \begin{bmatrix} f_xX/Z + p_x\ f_yY/Z + p_y\ 1 / Z\ \end{bmatrix} = \text{normalize}\left( \begin{bmatrix} f_xX + p_xZ\ f_yY + p_yZ\ 1\ Z \end{bmatrix} \right)" /></a>

The current behavior is

<a href="https://www.codecogs.com/eqnedit.php?latex=\begin{bmatrix}&space;x\&space;y\&space;z&space;\end{bmatrix}_{\text{NDCS}}&space;=&space;\begin{bmatrix}&space;f_xX/Z&space;&plus;&space;p_x/Z\&space;f_yY/Z&space;&plus;&space;p_y/Z\&space;1&space;/&space;Z\&space;\end{bmatrix}&space;=&space;\text{normalize}\left(&space;\begin{bmatrix}&space;f_xX&space;&plus;&space;p_x\&space;f_yY&space;&plus;&space;p_y\&space;1\&space;Z&space;\end{bmatrix}&space;\right)" target="_blank"><img src="https://latex.codecogs.com/gif.latex?\begin{bmatrix}&space;x\&space;y\&space;z&space;\end{bmatrix}_{\text{NDCS}}&space;=&space;\begin{bmatrix}&space;f_xX/Z&space;&plus;&space;p_x/Z\&space;f_yY/Z&space;&plus;&space;p_y/Z\&space;1&space;/&space;Z\&space;\end{bmatrix}&space;=&space;\text{normalize}\left(&space;\begin{bmatrix}&space;f_xX&space;&plus;&space;p_x\&space;f_yY&space;&plus;&space;p_y\&space;1\&space;Z&space;\end{bmatrix}&space;\right)" title="\begin{bmatrix} x\ y\ z \end{bmatrix}_{\text{NDCS}} = \begin{bmatrix} f_xX/Z + p_x/Z\ f_yY/Z + p_y/Z\ 1 / Z\ \end{bmatrix} = \text{normalize}\left( \begin{bmatrix} f_xX + p_x\ f_yY + p_y\ 1\ Z \end{bmatrix} \right)" /></a>

which is incorrect.
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/148

Reviewed By: gkioxari

Differential Revision: D21039003

Pulled By: davnov134

fbshipit-source-id: 3e19ac22adbcc39b731ae14052a72fd4ddda2af5
2020-04-15 10:20:43 -07:00
Georgia Gkioxari
b2b0c5a442 knn autograd
Summary:
Adds knn backward to return `grad_pts1` and `grad_pts2`. Adds `knn_gather` to return the nearest neighbors in pts2.

The BM tests include backward pass and are ran on an M40.
```
Benchmark                               Avg Time(μs)      Peak Time(μs) Iterations
--------------------------------------------------------------------------------
KNN_SQUARE_32_256_128_3_24_cpu              39558           43485             13
KNN_SQUARE_32_256_128_3_24_cuda:0            1080            1404            463
KNN_SQUARE_32_256_512_3_24_cpu              81950           85781              7
KNN_SQUARE_32_256_512_3_24_cuda:0            1519            1641            330
--------------------------------------------------------------------------------

Benchmark                               Avg Time(μs)      Peak Time(μs) Iterations
--------------------------------------------------------------------------------
KNN_RAGGED_32_256_128_3_24_cpu              13798           14650             37
KNN_RAGGED_32_256_128_3_24_cuda:0            1576            1713            318
KNN_RAGGED_32_256_512_3_24_cpu              31255           32210             16
KNN_RAGGED_32_256_512_3_24_cuda:0            2024            2162            248
--------------------------------------------------------------------------------
```

Reviewed By: jcjohnson

Differential Revision: D20945556

fbshipit-source-id: a16f616029c6b5f8c2afceb5f2bc12c5c20d2f3c
2020-04-14 17:22:56 -07:00