481 Commits

Author SHA1 Message Date
Patrick Labatut
7e43f29d52 Lint codebase
Summary: Lint codebase

Reviewed By: bottler

Differential Revision: D29263057

fbshipit-source-id: ac97f01d2a79fead3b09c2cbb21b50ce688a577d
2021-06-22 03:45:27 -07:00
Jeremy Reizenstein
ce60d4b00e remove requires_grad from random rotations
Summary: Because rotations and (rotation) quaternions live on curved manifolds, it doesn't make sense to optimize them directly. Having a prominent option to require gradient on random ones may cause people to try, and isn't particularly useful.

Reviewed By: theschnitz

Differential Revision: D29160734

fbshipit-source-id: fc9e320672349fe334747c5b214655882a460a62
2021-06-21 11:45:42 -07:00
Jeremy Reizenstein
31c448a95d Test gltf texture without renderer.
Summary:
Change the cow gltf loading test to validate the texture values and not to validate the renderer output because it has an unstable pixel.

Also a couple of lints.

Reviewed By: patricklabatut

Differential Revision: D29131260

fbshipit-source-id: 5e11f066a2a638588aacb09776cc842173ef669f
2021-06-21 08:11:36 -07:00
Jeremy Reizenstein
9c09c0d316 conda timeout in CI
Summary: Conda build command can be very slow at resolving dependencies with PyTorch 1.8. I extended the timeout for this in the tests. Here do the same for the builds.

Reviewed By: patricklabatut

Differential Revision: D29131188

fbshipit-source-id: 554e694f0f8aa7509141016720b1e9019177b214
2021-06-21 07:26:39 -07:00
Jeremy Reizenstein
354a1808ff Fix save_ply with noncontiguous faces
Summary: As noted in #710, save_ply was failing with some values of the faces tensor. It was assuming the faces were contiguous in using view() to change them. Here we avoid doing that.

Reviewed By: patricklabatut

Differential Revision: D29159655

fbshipit-source-id: 47214a7ce915bab8d81f109c2b97cde464fd57d8
2021-06-21 06:05:45 -07:00
David Novotny
8006842f2a Conversion from OpenCV cameras
Summary: Implements a conversion function between OpenCV and PyTorch3D cameras.

Reviewed By: patricklabatut

Differential Revision: D28992470

fbshipit-source-id: dbcc9f213ec293c2f6938261c704aea09aad3c90
2021-06-21 05:03:32 -07:00
David Novotny
b2ac2655b3 SE3 exponential and logarithm maps.
Summary:
Implements the SE3 logarithm and exponential maps.
(this is a second part of the split of D23326429)

Outputs of `bm_se3`:
```
Benchmark         Avg Time(μs)      Peak Time(μs) Iterations
--------------------------------------------------------------------------------
SE3_EXP_1                738             885            678
SE3_EXP_10               717             877            698
SE3_EXP_100              718             847            697
SE3_EXP_1000             729            1181            686
--------------------------------------------------------------------------------

Benchmark          Avg Time(μs)      Peak Time(μs) Iterations
--------------------------------------------------------------------------------
SE3_LOG_1               1451            2267            345
SE3_LOG_10              2185            2453            229
SE3_LOG_100             2217            2448            226
SE3_LOG_1000            2455            2599            204
--------------------------------------------------------------------------------
```

Reviewed By: patricklabatut

Differential Revision: D27852557

fbshipit-source-id: e42ccc9cfffe780e9cad24129de15624ae818472
2021-06-21 04:48:27 -07:00
David Novotny
9f14e82b5a SO3 improvements for stable gradients.
Summary:
Improves so3 functions to make gradient computation stable:
- Instead of `torch.acos`, uses `acos_linear_extrapolation` which has finite gradients of reasonable magnitude for all inputs.
- Adds tests for the latter.

The tests of the finiteness of the gradient in `test_so3_exp_singularity`, `test_so3_exp_singularity`, `test_so3_cos_bound` would fail if the `so3` functions would call `torch.acos` instead of `acos_linear_extrapolation`.

Reviewed By: bottler

Differential Revision: D23326429

fbshipit-source-id: dc296abf2ae3ddfb3942c8146621491a9cb740ee
2021-06-21 04:48:27 -07:00
David Novotny
dd45123f20 Linearly extrapolated acos.
Summary:
Implements a backprop-safe version of `torch.acos` that linearly extrapolates the function outside bounds.

Below is a plot of the extrapolated acos for different bounds:
{F611339485}

Reviewed By: bottler, nikhilaravi

Differential Revision: D27945714

fbshipit-source-id: fa2e2385b56d6fe534338d5192447c4a3aec540c
2021-06-21 04:48:27 -07:00
Georgia Gkioxari
88f5d79088 fix small face issue for ptmeshdist
Summary:
Fix small face issue for point_mesh distance computation.

The issue lies in the computation of `IsInsideTriangle` which is unstable and non-symmetrical when faces with small areas are given as input. This diff fixes the issue by returning `False` for `IsInsideTriangle` when small faces are given as input.

Reviewed By: bottler

Differential Revision: D29163052

fbshipit-source-id: be297002f26b5e6eded9394fde00553a37406bee
2021-06-18 09:29:55 -07:00
Nikhila Ravi
a343cf534c Small fix to tutorial
Summary: Small fix to `fit_textured_mesh.ipynb` tutorial due to a recent change in numpy

Reviewed By: bottler

Differential Revision: D29219990

fbshipit-source-id: f5feeef9eb952720ea7154d066795fbbe64ce7a1
2021-06-18 07:40:51 -07:00
Talmaj Marinc
029a9da00b Fix ShapeNetDataset (#593)
Summary:
- Add MANIFEST.in and `include_package_data=True` to include dataset .json files in the installation
Fix https://github.com/facebookresearch/pytorch3d/issues/435
- Fix `load_textures=False` for ShapeNetDataset with a test
Fix https://github.com/facebookresearch/pytorch3d/issues/450, partly fix https://github.com/facebookresearch/pytorch3d/issues/444. I've set the textures to `None`, if they should be all white instead, let me know.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/593

Reviewed By: patricklabatut

Differential Revision: D29116264

Pulled By: nikhilaravi

fbshipit-source-id: 1fb0198e616b7f834dfeaf7168bb5e6e530810d1
2021-06-18 07:02:20 -07:00
Roman Shapovalov
1b39cebe92 Sign issue about quaternion_to_matrix and matrix_to_quaternion
Summary:
As reported on github, `matrix_to_quaternion` was incorrect for rotations by 180˚. We resolved the sign of the component `i` based on the sign of `i*r`, assuming `r > 0`, which is untrue if `r == 0`.

This diff handles special cases and ensures we use the non-zero elements to copy the sign from.

Reviewed By: bottler

Differential Revision: D29149465

fbshipit-source-id: cd508cc31567fc37ea3463dd7e8c8e8d5d64a235
2021-06-18 06:40:02 -07:00
Patrick Labatut
a8610e9da4 Increase code coverage of shader
Summary: Increase code coverage of shader and re-include them in code coverage test

Reviewed By: nikhilaravi

Differential Revision: D29097503

fbshipit-source-id: 2791989ee1562cfa193f3addea0ce72d6840614a
2021-06-17 01:35:37 -07:00
Nikhila Ravi
c75ca04cf7 Bug fix in rendering clipped meshes
Summary:
There was a bug when `z_clip_value` is not None but there are no faces which are actually visible in the image due to culling.  In `rasterize_meshes.py` a function `convert_clipped_rasterization_to_original_faces` is called to convert the clipped face indices etc back to the unclipped versions, but the case where there is no clipping was not handled correctly.

Fixes Github Issue #632

Reviewed By: bottler

Differential Revision: D29116150

fbshipit-source-id: fae82a0b4848c84b3ed7c7b04ef5c9848352cf5c
2021-06-15 07:51:12 -07:00
Nikhila Ravi
bc8361fa47 Lighting broadcasting bug fix
Summary: Fixed multiple issues with shape broadcasting in lighting, shading and blending and updated the tests.

Reviewed By: bottler

Differential Revision: D28997941

fbshipit-source-id: d3ef93f979344076b1d9098a86178b4da63844c8
2021-06-14 11:48:27 -07:00
Jeremy Reizenstein
9de627e01b PyTorch 1.8 builds
Summary: Nightly builds to support PyTorch 1.8.0 and PyTorch 1.8.1.

Reviewed By: patricklabatut

Differential Revision: D29098081

fbshipit-source-id: fc6b36e919892ea41979a03e64a6fc8003528b78
2021-06-14 10:29:33 -07:00
Patrick Labatut
780e231536 Increase code coverage of subdivide_meshes
Summary: Increase code coverage of subdivide_meshes and re-include it in code coverage test

Reviewed By: bottler

Differential Revision: D29097476

fbshipit-source-id: 3403ae38a90c4b53f24188eed11faae202a235b5
2021-06-14 04:02:59 -07:00
Nikhila Ravi
a0f79318c5 Culling to frustrum bug fix
Summary:
When `z_clip_value = None` and faces are outside the view frustum the shape of one of the tensors in `clip.py` is incorrect.

`faces_num_clipped_verts` should be (F,) but it was (F,3).  Added a new test to ensure this case is handled.

Reviewed By: bottler

Differential Revision: D29051282

fbshipit-source-id: 5f4172ba4d4a75d928404dde9abf48aef18c68bd
2021-06-11 14:33:40 -07:00
Nikhila Ravi
ef16253953 textures dimension check
Summary:
When textures are set on `Meshes` we need to check if the dimensions actually match that of the verts/faces in the mesh. There was a github issue where someone tried to set the attribute after construction of the `Meshes` object and ran into an error when trying to sample textures.

The desired usage is to initialize the class with the textures (not set an attribute afterwards) but in any case we need to check the dimensions match before sampling textures.

Reviewed By: bottler

Differential Revision: D29020249

fbshipit-source-id: 9fb8a5368b83c9ec53652df92b96fc8b2613f591
2021-06-11 13:38:46 -07:00
Patrick Labatut
1cd1436460 Omit specific code from code coverage
Summary: Omit specific code from code coverage computation. This is done to make code coverage test pass again. Test coverage for shader.py and subdivide_meshes.py will be increased in later diffs to re-include them.

Reviewed By: bottler

Differential Revision: D29061105

fbshipit-source-id: addac35a216c96de9f559e2d8fe42496adc85791
2021-06-11 07:36:22 -07:00
Luke Davis
c4fc4666fc Fixes bug when importing from ..common.types (#706)
Summary:
There are a couple times in the code where Device, make_device and get_device are imported by:
`from ..common.types import Device, get_device, make_device`
which will not work without this init file, at least on python 3.8.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/706

Reviewed By: bottler

Differential Revision: D29031835

Pulled By: patricklabatut

fbshipit-source-id: 15743e3c8cffdfcca51b6d2f377b923da9cbe6f9
2021-06-10 10:41:08 -07:00
Patrick Labatut
44508ed0db Make Transform3d.to() not ignore dtype
Summary: Make Transform3d.to() not ignore a different dtype when device is the same and no copy is requested. Fix other methods where dtype is ignored.

Reviewed By: nikhilaravi

Differential Revision: D28981171

fbshipit-source-id: 4528e6092f4a693aecbe8131ede985fca84e84cf
2021-06-09 15:50:09 -07:00
Patrick Labatut
626bf3fe23 Fix type annotations for device type
Summary: Fix type annotations for device type

Reviewed By: nikhilaravi

Differential Revision: D28971179

fbshipit-source-id: 410b673c76dfd65ac51b2d144f17ed86a04a3058
2021-06-09 15:50:09 -07:00
Patrick Labatut
1f9661e150 Tidy uses of torch.device in Volumes
Summary:
Tidy uses of `torch.device` in `Volumes`:
- Allow `str` or `torch.device` in `Volumes.to()` method
- Consistently use `torch.device` for internal type
- Fix comparison of devices

Reviewed By: nikhilaravi

Differential Revision: D28970876

fbshipit-source-id: c640cc22ced684a54cc450ac38a0f4b3435d47be
2021-06-09 15:50:09 -07:00
Patrick Labatut
1db40ac566 Tidy uses of torch.device in Pointclouds
Summary:
Tidy uses of `torch.device` in `Pointclouds`:
- Allow `str` or `torch.device` in `Pointclouds.to()` method
- Consistently use `torch.device` for internal type
- Fix comparison of devices

Reviewed By: nikhilaravi

Differential Revision: D28970221

fbshipit-source-id: 3ca7104d4c0d9b20b0cff4f00e3ce931c5f1528a
2021-06-09 15:50:09 -07:00
Patrick Labatut
633d66f1f0 Tidy uses of torch.device in Meshes
Summary:
Tidy uses of `torch.device` in `Meshes`:
- Allow `str` or `torch.device` in `Meshes.to()` method
- Consistently use `torch.device` for internal type
- Fix comparison of devices

Reviewed By: nikhilaravi

Differential Revision: D28969461

fbshipit-source-id: 16d3c1f5458954bb11fdf0efea88542e94dccd7a
2021-06-09 15:50:09 -07:00
Patrick Labatut
13a0110b69 Tidy uses of torch.device in Transform3d
Summary:
Tidy uses of `torch.device` in `Transforms3d`:
- Allow `str` or `torch.device` in user-facing methods
- Consistently use `torch.device` for internal types
- Fix comparison of devices

Reviewed By: nikhilaravi

Differential Revision: D28929486

fbshipit-source-id: bd1d6cc7ede3d8fd549fd3224a9b07eec53f8164
2021-06-09 15:50:09 -07:00
Patrick Labatut
48faf8eb7e Introduce device type and utility functions
Summary: Introduce device type and utility functions in common types module

Reviewed By: nikhilaravi

Differential Revision: D28970930

fbshipit-source-id: 191ec07390ed66a958c23eb2b43229312492e0b7
2021-06-09 15:50:09 -07:00
Patrick Labatut
07da36d4c8 Add missing dtype parameter type annotations
Summary: Add missing dtype parameter type annotations

Reviewed By: nikhilaravi

Differential Revision: D28943370

fbshipit-source-id: 2a411d78895f3f3aa9ab0e4807c17a13e7f25caf
2021-06-09 15:50:09 -07:00
Patrick Labatut
02650672f6 Improve volumes type annotations
Summary: Improve type annotations for volumes and remove a few pyre fixmes

Reviewed By: nikhilaravi

Differential Revision: D28943371

fbshipit-source-id: ca2b7a50d72a392910e65cee5e564f34523414d2
2021-06-09 15:50:09 -07:00
Nikhila Ravi
a15c33a3cc Alpha channel to return the mask
Summary: Updated the alpha channel in the `hard_rgb_blend` function to return the mask of the pixels which have overlapping mesh faces.

Reviewed By: bottler

Differential Revision: D29001604

fbshipit-source-id: 22a2173d769f2d3ad34892d68ceb628f073bca22
2021-06-09 15:06:53 -07:00
Pyre Bot Jr
ac6c07fa43 suppress errors in fbcode/vision - batch 2
Differential Revision: D28962863

fbshipit-source-id: 7acc8e2f08ea7a94294647431561ff2cfd52a97d
2021-06-08 11:17:24 -07:00
Jeremy Reizenstein
f00ef66727 NeRF training: avoid caching unused visualization data.
Summary: If we are not visualizing the training with visdom, then there are a couple of outputs of the coarse rendering step which are not small and are returned by the renderer but never used. We don't need to bother transferring them to the CPU.

Reviewed By: nikhilaravi

Differential Revision: D28939958

fbshipit-source-id: 7e0d6681d6524f7fb57b6b20164580006120de80
2021-06-08 04:35:22 -07:00
Jeremy Reizenstein
7204a4ca64 cuda 11 problems with test_normal_consistency
Summary:
One test hits problems with CUDA 11.1 and pytorch 1.8. This seems to be a known bug, so we just run that test on the cpu in the problematic cases.

Note - the full test run is much slower with cuda 11.1 than 10.2, but this is known.

Reviewed By: patricklabatut

Differential Revision: D28938933

fbshipit-source-id: cf8ed84cd10a0b52d8f4292edbef7bd4844fea65
2021-06-08 02:52:06 -07:00
Patrick Labatut
c710d8c101 Improve textures type annotations
Summary: Improve type annotations for textures and remove a few pyre fixmes

Reviewed By: nikhilaravi

Differential Revision: D28942630

fbshipit-source-id: 422f2bdf07b435869461ca103d71473aa0c2b814
2021-06-08 02:15:50 -07:00
Patrick Labatut
d76c00721c Remove explicit inheritance from object
Summary:
All classes implicitly inherit from `object` since Python 3, so remove unnecessary explicit inheritance.

From the [official documentation](https://docs.python.org/3/library/functions.html#object):

> `object` is a base for all classes.

Reviewed By: nikhilaravi

Differential Revision: D28942595

fbshipit-source-id: 466c0d19d8a93a6263e7ad734c3e87160cfa6066
2021-06-08 02:15:50 -07:00
Patrick Labatut
f14c0236f0 Remove extra pyre fixmes
Summary: Remove extra pyre fixmes

Reviewed By: bottler

Differential Revision: D28929468

fbshipit-source-id: 175b7986d49b56de7af063e97a9b0423570f9093
2021-06-07 12:05:52 -07:00
Jeremy Reizenstein
36b451a49b Test failures print message in assertNormsClose
Summary: Restore assertNormsClose's printing of its message on failure which I broke in D26233419 (cd9786e787).

Reviewed By: nikhilaravi

Differential Revision: D28799743

fbshipit-source-id: e7a24b2558b68991c731bbd55fb3ca6c1df98f69
2021-06-03 18:30:54 -07:00
Jeremy Reizenstein
070ec550d3 Always print message on test failure
Summary: make assertClose print its failure information even if a message is supplied.

Reviewed By: nikhilaravi

Differential Revision: D28799745

fbshipit-source-id: 787c8c356342420cd8f40fdc0b2aba036142298e
2021-06-03 18:30:54 -07:00
Ben Ahlbrand
7fd7de4451 fix: renderer.look_at_view_transform args docs updated (#693)
Summary:
Caught a silly typo in the docs, figured I'd send in a PR. Thanks for sharing your work!

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/693

Reviewed By: bottler

Differential Revision: D28837851

Pulled By: patricklabatut

fbshipit-source-id: c326189007c3537429a8aa2a3b2850d60b74b16d
2021-06-02 11:11:32 -07:00
Jeremy Reizenstein
280fed3c76 cuda streams for color/density in NeRF
Summary:
Inside the implicit function, the color and density calculations are independent and time is saved by putting them on separate streams.

(In fact, colors is slower than densities, and the raymarcher does some calculation with the densities before the colors. So theoretically we could go further and not join the streams together until the colors are actually needed. The code would be more complicated. But the profile suggests that the raymarcher is quick and so this wouldn't be expected to make a big difference.)

In inference, this might increase memory usage, so it isn't an obvious win. That is why I have added a flag.

Reviewed By: nikhilaravi

Differential Revision: D28648549

fbshipit-source-id: c087de80d8ccfce1dad3a13e71df2f305a36952e
2021-06-02 05:43:13 -07:00
Jeremy Reizenstein
f63e49d245 Let harmonic embedding layer include input (NeRF)
Summary: When harmonic embedding is used, we always cat its input onto its output before proceeding. Avoid an intermediate tensor by making the module do that for itself.

Reviewed By: davnov134

Differential Revision: D28185791

fbshipit-source-id: 98d92c94a918dd42e16cdadcaac71dabbc7de5c3
2021-06-02 05:43:13 -07:00
Jeremy Reizenstein
ab73f8c3fd LinearWithRepeat layer for NeRF
Summary:
Add custom layer to avoid repeating copied data for every ray position.

This should also save time in the backward pass because there are fewer multiplies with the weights.

Reviewed By: theschnitz

Differential Revision: D28382412

fbshipit-source-id: 1ba7356cd8520ebd598568ae503e47d31d3989eb
2021-06-02 05:43:13 -07:00
Pyre Bot Jr
fe39cc7b80 suppress errors in vision/fair/pytorch3d
Differential Revision: D28766465

fbshipit-source-id: 9c5b123ac279e25965ebc8827d9a947bbb6f3ac5
2021-05-27 18:30:15 -07:00
Ignacio Rocco
4efadaecfc Minor fix in HarmonicEmbedding docstring (#13)
Summary:
The multiplicative factors in function embeddings go from `2**0` to `2**(self.n_harmonic_functions-1)`, and not from `2**0` to `2**self.n_harmonic_functions`.

Pull Request resolved: https://github.com/fairinternal/pytorch3d/pull/13

Reviewed By: nikhilaravi

Differential Revision: D28637894

Pulled By: ignacio-rocco

fbshipit-source-id: da20f39eba9aaa09af5b24be1554a3bfd7556281
2021-05-27 03:14:06 -07:00
Jeremy Reizenstein
ed6983ea84 Experimental glTF reading
Summary: Experimental data loader for taking the default scene from a GLB file and converting it to a single mesh in PyTorch3D.

Reviewed By: nikhilaravi

Differential Revision: D25900167

fbshipit-source-id: bff22ac00298b83a0bd071ae5c8923561e1d81d7
2021-05-26 04:54:19 -07:00
Jeremy Reizenstein
0e85652f07 ambient lighting
Summary:
Specific object to represent light which is 100% everywhere. Sometimes lighting is irrelevant, for example when viewing a mesh which has lighting already baked in.

This is not primarily aiming for a performance win but I think the test which has changed might be a bit faster.

Reviewed By: theschnitz

Differential Revision: D26405151

fbshipit-source-id: 82eae55de0bee918548a3eaf031b002cb95e726c
2021-05-26 04:54:19 -07:00
Jeremy Reizenstein
61e38de034 rotate_on_spot
Summary: Function to relatively rotate a camera position. Also document how to relatively translate a camera position.

Reviewed By: theschnitz

Differential Revision: D25900166

fbshipit-source-id: 2ddaf06ee7c5e2a2e973c04d7dee6ccb61c6ff84
2021-05-26 04:54:19 -07:00
Jeremy Reizenstein
e12a08133f Deduplicate texture maps when joining
Summary:
If you join several meshes which have TexturesUV textures using join_meshes_as_scene then we amalgamate all the texture images in to a single one. This now checks if some of the images are equal (i.e. the tensors are the same tensor, in the `is` sense; they have the same `id` in Python) and only uses one copy if they are.

I have an example of a massive scene made of several textured meshes with some shared, where this makes the difference between fitting the data on the GPU and not.

Reviewed By: theschnitz

Differential Revision: D25982364

fbshipit-source-id: a8228805f38475c796302e27328a340d9b56c8ef
2021-05-26 04:54:19 -07:00