562 Commits

Author SHA1 Message Date
Jeremy Reizenstein
d173a2f8da textures device consistency
Summary: Ensure that `mesh2 = mesh.to(device)` doesn't change the device of `mesh.textures`.

Reviewed By: nikhilaravi

Differential Revision: D25978610

fbshipit-source-id: 0558cd62132965d8693ebeea05e42b8c1d16cfbf
2021-01-25 06:09:56 -08:00
Penn
e58a730e6a Fix dimension check (#524)
Summary:
Fixes the assertion that `p1` and `p2` have the same last dimension. The issue was that `D` is set to equal `p2.size(2)`, and then `D` is compared to `p2.size(2)`. The fix instead compares `D` to `p1.size(2).

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/524

Reviewed By: bottler

Differential Revision: D26008688

Pulled By: nikhilaravi

fbshipit-source-id: e32afe9da127d81b1a411d3c223b539a7400597b
2021-01-22 11:05:18 -08:00
imlixinyang
2ee11c7845 Update generate_cow_renders.py (#529)
Summary:
Typo fixed.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/529

Reviewed By: bottler

Differential Revision: D26008651

Pulled By: nikhilaravi

fbshipit-source-id: 76d5baadba7bcd3577397adb842e964ee4490b7d
2021-01-22 11:01:02 -08:00
Jeremy Reizenstein
8eba1684cb Up vector for plotly
Summary: We previously did not send an `up` vector in to plotly when visualization_cameras is supplied. This meant the image would have the default orientation instead of the correct one. Now we use the library function `camera_to_eye_at_up` to calculate the plotly position, which includes the `up` vector.

Reviewed By: nikhilaravi

Differential Revision: D25981183

fbshipit-source-id: abec72b349f3a5519209e0e6c0518133c3750807
2021-01-22 07:33:31 -08:00
Jeremy Reizenstein
cf9bb7c48c (eye, at, up) extraction function
Summary:
Plotly viewing from a specific camera location requires converting that location in to an (eye, at, up) specification. There may be other reasons to want to do this as well. I create a separate utility function for it.

I envisage more such utility functions for manipulating camera information, so I create a separate camera_utils.py file for such things.

Reviewed By: nikhilaravi

Differential Revision: D25981184

fbshipit-source-id: 0947bf98b212676c021f2fddf775bf436dee3487
2021-01-22 07:33:31 -08:00
Jeremy Reizenstein
ddebdfbcd7 Allow single offset in offset_verts
Summary:
It is common when trying things out to want to move a whole mesh or point cloud by the same amount. Here we allow the offset functions to broadcast.

Also add a sanity check to join_meshes_as_scene which it is easy to call wrongly.

Reviewed By: nikhilaravi

Differential Revision: D25980593

fbshipit-source-id: cdf1568e1317e3b81ad94ed4e608ba7eef81290b
2021-01-22 07:33:31 -08:00
Jeremy Reizenstein
d60c52df4a devices for transform3d
Summary: Make `to` on Transform3D carry its member _transforms.

Reviewed By: nikhilaravi

Differential Revision: D25978611

fbshipit-source-id: 12b39e7a657f28d59ca60800bf9f4193a2c08197
2021-01-21 04:58:40 -08:00
Jeremy Reizenstein
4711665edb lint
Summary: Fix recent lint.

Reviewed By: nikhilaravi

Differential Revision: D25900168

fbshipit-source-id: 6b6e8d35b68c8415ef305dc4719f43eda9316c8f
2021-01-20 13:08:35 -08:00
Christoph Lassner
154ace4ba4 Fix #501.
Summary: Gradient calculation flags were not properly routed through the Python interface. This diff fixes this. In particular, gradients for focal length (only if no other camera gradients were calculated) and opacity were not calculated as required.

Reviewed By: gkioxari

Differential Revision: D25921202

fbshipit-source-id: 22cbae3bda886d81bb95878f0be45c2ddd29934c
2021-01-19 19:28:35 -08:00
Jeremy Reizenstein
b24d89a283 Do not set ccbin in torch 1.7.x
Summary:
PyTorch versions 1.7.0 and 1.7.1 are between https://github.com/pytorch/pytorch/pull/43931 and https://github.com/pytorch/pytorch/pull/47404. In this gap, PyTorch always copies CC to nvcc_args, like PyTorch3D does. Newer nvcc versions are not happy with `-ccbin` being specified twice, even if it is specified twice the same. We update PyTorch3D so that it doesn't supply `-ccbin` in these cases.

Also tweak the detection of the current ccbin so that it is aware that `-ccbin foo` and `-ccbin=foo` are equivalent.

Reviewed By: theschnitz

Differential Revision: D25825468

fbshipit-source-id: b04e7718cf01820649518eedda99c399c732e8af
2021-01-11 15:49:55 -08:00
David Novotny
da7884cef1 Volume / NeRF notebook links in README.md
Summary: Adds links to notebooks implementing the Volume / NeRF fitting.

Reviewed By: gkioxari

Differential Revision: D25849756

fbshipit-source-id: 05d7d40589a8559c9bcc43e0d2e22c49f5a92dfd
2021-01-11 04:12:06 -08:00
Jeremy Reizenstein
95707fba1c PLY pointcloud loading
Summary:
Allow PLY files to not contain faces. Allow loading pointclouds with color, at least encoded according to the way of some cloudcompare examples.

TODO: Allow vertex normals to be read, and allow vertex colors to be written. Make the return type of load_ply something more user friendly, like a dict.

Noticed in https://github.com/facebookresearch/pytorch3d/issues/209

Reviewed By: nikhilaravi

Differential Revision: D22573314

fbshipit-source-id: 72ba1f7c6417f5dfc83f2ebf359eff017057635c
2021-01-07 15:40:11 -08:00
Jeremy Reizenstein
3b9fbfc08c Read heterogenous nonlist PLY properties as arrays
Summary:
In the original implementation, I had considered PLY properties where there are mixed types of elements in a property to be rare and basically unimportant, so the implementation is very naive.

If we want to support pointcloud PLY files, we need to handle at least the subcase where there are no lists efficiently because this seems to be very common there.

Reviewed By: nikhilaravi, gkioxari

Differential Revision: D22573315

fbshipit-source-id: db6f29446d4e555a2e2b37d38c8e4450d061465b
2021-01-07 15:40:11 -08:00
Jeremy Reizenstein
89532a876e add existing mesh formats to pluggable
Summary: We already have code for obj and ply formats. Here we actually make it available in `IO.load_mesh` and `IO.save_mesh`.

Reviewed By: theschnitz, nikhilaravi

Differential Revision: D25400650

fbshipit-source-id: f26d6d7fc46c48634a948eea4d255afad13b807b
2021-01-07 15:40:11 -08:00
Jeremy Reizenstein
b183dcb6e8 skeleton of pluggable IO
Summary: Unified interface for loading and saving meshes and pointclouds.

Reviewed By: nikhilaravi

Differential Revision: D25372968

fbshipit-source-id: 6fe57cc3704a89d81d13e959bee707b0c7b57d3b
2021-01-07 15:40:11 -08:00
David Novotny
9fc661f8b3 Tutorial - Fit neural radiance field
Summary: Implements a simple nerf tutorial.

Reviewed By: nikhilaravi

Differential Revision: D24650983

fbshipit-source-id: b3db51c0ed74779ec9b510350d1675b0ae89422c
2021-01-06 08:27:20 -08:00
David Novotny
01f86ddeb1 Tutorial - Fit textured volume.
Summary: Implements a notebook that fits a volume to multiple views of the cow mesh.

Reviewed By: nikhilaravi

Differential Revision: D24553385

fbshipit-source-id: 367ca39e176b40df2c5946c9c05d3be824dc8d1c
2021-01-06 08:27:20 -08:00
David Novotny
b466c381da Implicit/Volume renderer
Summary: Implements the `ImplicitRenderer` and `VolumeRenderer`.

Reviewed By: gkioxari

Differential Revision: D24418791

fbshipit-source-id: 127f21186d8e210895db1dcd0681f09f230d81a4
2021-01-06 06:23:48 -08:00
generatedunixname89002005287564
e6a32bfc37 Daily arc lint --take BLACK
Reviewed By: zertosh

Differential Revision: D25800514

fbshipit-source-id: 191b2753b8fcfbe2386c761241aaeb58939a973e
2021-01-06 04:23:26 -08:00
David Novotny
e6bc960fb5 Raysampling
Summary: Implements 3 basic raysamplers.

Reviewed By: nikhilaravi

Differential Revision: D24110643

fbshipit-source-id: eb67d0e56773c7871ebdcb23e7e520302dc1b3c9
2021-01-06 04:01:29 -08:00
generatedunixname89002005307016
1f9cf91e1b suppress errors in vision/fair/pytorch3d
Differential Revision: D25781183

fbshipit-source-id: e27808a4c2b94bba205756001cb909827182b592
2021-01-05 05:28:14 -08:00
David Novotny
1af1a36bd6 Raymarching
Summary: Implements two basic raymarchers.

Reviewed By: gkioxari

Differential Revision: D24064250

fbshipit-source-id: 18071bd039995336b7410caa403ea29fafb5c66f
2021-01-05 03:59:20 -08:00
David Novotny
aa9bcaf04c Point clouds to volumes
Summary:
Conversion from point clouds to volumes

```
Benchmark                                                        Avg Time(μs)      Peak Time(μs) Iterations
--------------------------------------------------------------------------------
ADD_POINTS_TO_VOLUMES_10_trilinear_[25, 25, 25]_1000                 43219           44067             12
ADD_POINTS_TO_VOLUMES_10_trilinear_[25, 25, 25]_10000                43274           45313             12
ADD_POINTS_TO_VOLUMES_10_trilinear_[25, 25, 25]_100000               46281           47100             11
ADD_POINTS_TO_VOLUMES_10_trilinear_[101, 111, 121]_1000              51224           51912             10
ADD_POINTS_TO_VOLUMES_10_trilinear_[101, 111, 121]_10000             52092           54487             10
ADD_POINTS_TO_VOLUMES_10_trilinear_[101, 111, 121]_100000            59262           60514              9
ADD_POINTS_TO_VOLUMES_10_nearest_[25, 25, 25]_1000                   15998           17237             32
ADD_POINTS_TO_VOLUMES_10_nearest_[25, 25, 25]_10000                  15964           16994             32
ADD_POINTS_TO_VOLUMES_10_nearest_[25, 25, 25]_100000                 16881           19286             30
ADD_POINTS_TO_VOLUMES_10_nearest_[101, 111, 121]_1000                19150           25277             27
ADD_POINTS_TO_VOLUMES_10_nearest_[101, 111, 121]_10000               18746           19999             27
ADD_POINTS_TO_VOLUMES_10_nearest_[101, 111, 121]_100000              22321           24568             23
ADD_POINTS_TO_VOLUMES_100_trilinear_[25, 25, 25]_1000                49693           50288             11
ADD_POINTS_TO_VOLUMES_100_trilinear_[25, 25, 25]_10000               51429           52449             10
ADD_POINTS_TO_VOLUMES_100_trilinear_[25, 25, 25]_100000             237076          237377              3
ADD_POINTS_TO_VOLUMES_100_trilinear_[101, 111, 121]_1000             81875           82597              7
ADD_POINTS_TO_VOLUMES_100_trilinear_[101, 111, 121]_10000           106671          107045              5
ADD_POINTS_TO_VOLUMES_100_trilinear_[101, 111, 121]_100000          483740          484607              2
ADD_POINTS_TO_VOLUMES_100_nearest_[25, 25, 25]_1000                  16667           18143             31
ADD_POINTS_TO_VOLUMES_100_nearest_[25, 25, 25]_10000                 17682           18922             29
ADD_POINTS_TO_VOLUMES_100_nearest_[25, 25, 25]_100000                65463           67116              8
ADD_POINTS_TO_VOLUMES_100_nearest_[101, 111, 121]_1000               48058           48826             11
ADD_POINTS_TO_VOLUMES_100_nearest_[101, 111, 121]_10000              53529           53998             10
ADD_POINTS_TO_VOLUMES_100_nearest_[101, 111, 121]_100000            123684          123901              5
--------------------------------------------------------------------------------
```

Output with `DEBUG=True`
{F338561209}

Reviewed By: nikhilaravi

Differential Revision: D22017500

fbshipit-source-id: ed3e8ed13940c593841d93211623dd533974012f
2021-01-05 03:39:24 -08:00
David Novotny
03ee1dbf82 Volumes data structure.
Summary: Implemented a data structure for volumes.

Reviewed By: gkioxari

Differential Revision: D20342920

fbshipit-source-id: ccc23eaa183ed8a4e9cd7674b4dcf31e8a65c3c6
2021-01-05 03:39:24 -08:00
David Novotny
1e4a2e8624 __getitem__ for Transform3D
Summary: Implements the `__getitem__` method for `Transform3D`

Reviewed By: nikhilaravi

Differential Revision: D23813975

fbshipit-source-id: 5da752ed8ea029ad0af58bb7a7856f0995519b7a
2021-01-05 03:39:24 -08:00
generatedunixname89002005307016
ac3f8dc833 suppress errors in vision/fair/pytorch3d
Differential Revision: D25777275

fbshipit-source-id: ca30fedca118ff22a8be5e29c4c4f21628c94579
2021-01-04 23:09:43 -08:00
David Novotny
b4dea43963 Support for multi-dimensional list_to_padded/padded_to_list.
Summary: Extends `list_to_padded`/`padded_to_list` to work for tensors with an arbitrary number of input dimensions.

Reviewed By: nikhilaravi, gkioxari

Differential Revision: D23813969

fbshipit-source-id: 52c212a2ecdb3c4dfb6ac47217715e07998f37f1
2021-01-04 09:42:52 -08:00
Jeremy Reizenstein
0ba55a83ad fix conda channel in tests
Summary: Now we use iopath, we need to find it from its own channel.

Reviewed By: nikhilaravi

Differential Revision: D25710499

fbshipit-source-id: 1c67eb6d5b009d35b65a3acd3ebff6e0e45fecc4
2020-12-27 13:14:42 -08:00
generatedunixname89002005307016
fc58acb2d4 suppress errors in vision/fair/pytorch3d
Differential Revision: D25702902

fbshipit-source-id: f0d6708ba917df85b575dfc5525c902b2cab7ea0
2020-12-24 11:24:28 -08:00
Jeremy Reizenstein
25c065e9da PathManager passing
Summary:
Make no internal functions inside pytorch3d/io interpret str paths except using a PathManager from iopath which they have been given. This means we no longer use any global PathManager object and we no longer use fvcore's deprecated file_io.

To preserve the APIs, various top level functions create their own default-initialized PathManager object if they are not provided one.

Reviewed By: theschnitz

Differential Revision: D25372969

fbshipit-source-id: c176ee31439645fa54a157d6f1aef18b09501569
2020-12-24 10:16:03 -08:00
Jeremy Reizenstein
b95621573b lint
Summary: Allowing usort, isort and black to coexist without fighting means we can't have imports commented as deprecated from the same module as other imports.

Reviewed By: nikhilaravi

Differential Revision: D25372970

fbshipit-source-id: 637f5a0025c0df9fbec47cba73ce5387f4f8b467
2020-12-24 10:16:03 -08:00
Jeremy Reizenstein
513a6476bc iopath dependency
Summary: Add ioPath as a dependency of PyTorch3D in preparation for using the new PathManager.

Reviewed By: nikhilaravi

Differential Revision: D25372971

fbshipit-source-id: d8aa661d2de975e747dd494edc42bf843990cf68
2020-12-24 10:16:03 -08:00
Jeremy Reizenstein
0a309ec6c7 requirements for readthedocs
Summary: Use a more recent PyTorch to build the documentation.

Reviewed By: nikhilaravi

Differential Revision: D25679756

fbshipit-source-id: 83d647f709337110d39886eaa6aad2565d740c6d
2020-12-22 08:53:36 -08:00
Christoph Lassner
caa3371376 Fix Pulsar backend batched radius handling.
Summary: This fixes a corner case for multi-radius handling for the pulsar backend. The additional dimensionality check ensures that the batched parsing for radiuses is only performed when appropriate.

Reviewed By: bottler

Differential Revision: D25387708

fbshipit-source-id: c486dcf327f812265b7ca8ca5ef5c6a31e6d4549
2020-12-21 13:02:25 -08:00
Nikhila Ravi
ebac66daeb Classic Marching Cubes algorithm implementation
Summary:
Defines a function to run marching cubes algorithm on a single or batch of 3D scalar fields. Returns a mesh's faces and vertices.

UPDATES (12/18)
- Input data is now specified as a (B, D, H, W) tensor as opposed to a (B, W, H, D) tensor. This will now be compatible with the Volumes datastructure.
- Add an option to return output vertices in local coordinates instead of world coordinates.
Also added a small fix to remove the dype for device in Transforms3D - if passing in a torch.device instead of str it causes a pyre error.

Reviewed By: jcjohnson

Differential Revision: D24599019

fbshipit-source-id: 90554a200319fed8736a12371cc349e7108aacd0
2020-12-18 07:25:50 -08:00
Foo Guo Wei
9c6b58c5ad Expose axis-angle/quaternion conversion functions in transforms package (#486)
Summary:
Fixes https://github.com/facebookresearch/pytorch3d/issues/484

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/486

Reviewed By: bottler

Differential Revision: D25620407

Pulled By: nikhilaravi

fbshipit-source-id: 9a29e4e717aebcca46e46ff5e7bb80a183a92836
2020-12-18 05:18:48 -08:00
Nikhila Ravi
01759d8ffb Texture Atlas sampling bug fix
Summary: Fixes the index out of bound errors for texture sampling from a texture atlas: when barycentric coordinates are 1.0, the integer index into the (R, R) per face texture map is R (max can only be R-1).

Reviewed By: gkioxari

Differential Revision: D25543803

fbshipit-source-id: 82d0935b981352b49c1d95d5a17f9cc88bad0a82
2020-12-17 04:10:56 -08:00
Nikhila Ravi
3d769a66cb Non Square image rasterization for pointclouds
Summary:
Similar to non square image rasterization for meshes, apply the same updates to the pointcloud rasterizer.

Main API Change:
- PointRasterizationSettings now accepts a tuple/list of (H, W) for the image size.

Reviewed By: jcjohnson

Differential Revision: D25465206

fbshipit-source-id: 7370d83c431af1b972158cecae19d82364623380
2020-12-15 14:15:32 -08:00
Evgeniy Zheltonozhskiy
569e5229a9 Add check for verts and faces being on same device and also checks for pointclouds/features/normals being on the same device (#384)
Summary: Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/384

Test Plan: `test_meshes` and `test_points`

Reviewed By: gkioxari

Differential Revision: D24730524

Pulled By: nikhilaravi

fbshipit-source-id: acbd35be5d9f1b13b4d56f3db14f6e8c2c0f7596
2020-12-14 16:18:19 -08:00
Nikhila Ravi
19340462e4 Return self in the to method for the renderer classes
Summary: Add `return self` to the `to` function for the renderer classes.

Reviewed By: bottler

Differential Revision: D25534487

fbshipit-source-id: e8dbd35524f0bd40e835439e93184b5a1f1532ca
2020-12-14 15:28:04 -08:00
Evgeniy Zheltonozhskiy
831e64efb0 Pass epsilon value properly to the transformation (#418)
Summary:
As for now, epsilon value is ignored, since `kwargs` are passed to constructor only

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/418

Reviewed By: gkioxari

Differential Revision: D24730500

Pulled By: nikhilaravi

fbshipit-source-id: 7cce820dbe14f8c74d3df4f18c45d50e228c6a45
2020-12-14 15:25:32 -08:00
Elie Michel
f248bfc415 Two minor fixes in doc and tutorial (#254)
Summary:
`INSTALL.md` On Windows, pip install from git does not work with single quotes. I replace them with double quote as those should work for any OS.

`camera_position_optimization_with_differentiable_rendering.ipynb` To extract the silhouette from the reference image, we need to reject white values, not black values, because the background of the ref is white:
![ref](https://user-images.githubusercontent.com/5802849/86496925-b72e9100-bd7f-11ea-9f61-72abb3efe1b5.png)

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/254

Reviewed By: gkioxari

Differential Revision: D23798508

Pulled By: nikhilaravi

fbshipit-source-id: 1fbec291c48c367539a8a4fea389a109aab49e3d
2020-12-14 12:36:01 -08:00
Jeremy Reizenstein
16a0be790b nightlies with pytorch 1.7.1
Summary: Add nightly linux conda builds with the new pytorch 1.7.1. This supports python 3.9.

Reviewed By: nikhilaravi

Differential Revision: D25532757

fbshipit-source-id: b734637063e148389951899450566275f3cf5831
2020-12-14 08:35:06 -08:00
generatedunixname89002005307016
1b82388ab8 suppress errors in vision - batch 1
Differential Revision: D25497975

fbshipit-source-id: 9f7aa8d1dd31fa62a428321394d8c97b2d9d937d
2020-12-11 12:46:33 -08:00
Jeremy Reizenstein
d6412c94dc CUB conda instructions
Summary: We have already uploaded a conda package of the cub 1.10.0 sources to our channel on anaconda. Here is the code we used.

Reviewed By: nikhilaravi

Differential Revision: D25395358

fbshipit-source-id: b58fd473fcafd425c98c9d7a7c32574f651383a0
2020-12-10 09:47:56 -08:00
Nikhila Ravi
d07307a451 Non square image rasterization for meshes
Summary:
There are a couple of options for supporting non square images:
1) NDC stays at [-1, 1] in both directions with the distance calculations all modified by (W/H). There are a lot of distance based calculations (e.g. triangle areas for barycentric coordinates etc) so this requires changes in many places.
2) NDC is scaled by (W/H) so the smallest side has [-1, 1]. In this case none of the distance calculations need to be updated and only the pixel to NDC calculation needs to be modified.

I decided to go with option 2 after trying option 1!

API Changes:
- Image size can now be specified optionally as a tuple

TODO:
- add a benchmark test for the non square case.

Reviewed By: jcjohnson

Differential Revision: D24404975

fbshipit-source-id: 545efb67c822d748ec35999b35762bce58db2cf4
2020-12-09 09:18:11 -08:00
Jeremy Reizenstein
0216e4689a Slow pull from docker
Summary: The "manual" docker pull in the testrun builds has been taking more than 10 minutes recently, and its lack of output causes circleci to timeout. As a quick fix, we enable the output from this operation.

Reviewed By: gkioxari

Differential Revision: D25305127

fbshipit-source-id: 19682bfa0294145457a37df6d6faf7a08dcc63c9
2020-12-03 12:34:17 -08:00
generatedunixname89002005287564
a0cd2506f6 Daily arc lint --take BLACK
Reviewed By: zertosh

Differential Revision: D25241045

fbshipit-source-id: c3a1e1e02ef557b1e82cb672e987e05016e246fd
2020-12-01 04:19:28 -08:00
Georgia Gkioxari
112959e087 taubin smoothing
Summary: Taubin Smoothing for filtering meshes and making them smoother. Taubin smoothing is an iterative approach.

Reviewed By: nikhilaravi

Differential Revision: D24751149

fbshipit-source-id: fb779e955f1a1f6750e704f1b4c6dfa37aebac1a
2020-11-30 11:38:04 -08:00
Amitav Baruah
fc7a4cacc3 Fix plotly pointcloud visualization feature bug
Summary: If a pointcloud had less than pointcloud_max_points, the colors would not render. This diff fixes that.

Reviewed By: bottler

Differential Revision: D25099044

fbshipit-source-id: 47c3ddcdb4e06284b0a7966ffca1b973f394921f
2020-11-19 13:52:46 -08:00