Summary:
Threaded the for loop:
```
for (int yi = 0; yi < H; ++yi) {...}
```
in function `RasterizeMeshesNaiveCpu()`.
Chunk size is approx equal.
Reviewed By: bottler
Differential Revision: D40063604
fbshipit-source-id: 09150269405538119b0f1b029892179501421e68
Summary: Loads the whole dataset and moves it to the device and sends it to for sampling to enable full dataset heterogeneous raysampling.
Reviewed By: bottler
Differential Revision: D39263009
fbshipit-source-id: c527537dfc5f50116849656c9e171e868f6845b1
Summary:
Changed ray_sampler and metrics to be able to use mixed frame raysampling.
Ray_sampler now has a new member which it passes to the pytorch3d raysampler.
If the raybundle is heterogeneous metrics now samples images by padding xys first. This reduces memory consumption.
Reviewed By: bottler, kjchalup
Differential Revision: D39542221
fbshipit-source-id: a6fec23838d3049ae5c2fd2e1f641c46c7c927e3
Summary: new implicitronRayBundle with added cameraIDs and camera counts. Added to enable a single raybundle inside Implicitron and easier extension in the future. Since RayBundle is named tuple and RayBundleHeterogeneous is dataclass and RayBundleHeterogeneous cannot inherit RayBundle. So if there was no ImplicitronRayBundle every function that uses RayBundle now would have to use Union[RayBundle, RaybundleHeterogeneous] which is confusing and unecessary complicated.
Reviewed By: bottler, kjchalup
Differential Revision: D39262999
fbshipit-source-id: ece160e32f6c88c3977e408e966789bf8307af59
Summary:
Added heterogeneous raysampling to pytorch3d raysampler, different cameras are sampled different number of times.
It now returns RayBundle if heterogeneous raysampling is off and new RayBundleHeterogeneous (with added fields `camera_ids` and `camera_counts`). Heterogeneous raysampling is on if `n_rays_total` is not None.
Reviewed By: bottler
Differential Revision: D39542222
fbshipit-source-id: d3d88d822ec7696e856007c088dc36a1cfa8c625
Summary:
This is quite a thin wrapper – not sure we need it. The motivation is that `Transform3d` is not as matrix-centric now, it can be converted to SE(3) logarithm equally easily.
It simplifies things like averaging cameras and getting axis-angle of camera rotation (previously, one would need to call `se3_log_map(cameras.get_world_to_camera_transform().get_matrix())`), now one fewer thing to call / discover.
Reviewed By: bottler
Differential Revision: D39928000
fbshipit-source-id: 85248d5b8af136618f1d08791af5297ea5179d19
Summary:
`get_rotation_to_best_fit_xy` is useful to expose externally, however there was a bug (which we probably did not care about for our use case): it could return a rotation matrix with det(R) == −1.
The diff fixes that, and also makes centroid optional (it can be computed from points).
Reviewed By: bottler
Differential Revision: D39926791
fbshipit-source-id: 5120c7892815b829f3ddcc23e93d4a5ec0ca0013
Summary: Any module can be subscribed to step updates from the training loop. Once the training loop publishes a step the voxel grid changes its dimensions. During the construction of VoxelGridModule and its parameters it does not know which is the resolution that will be loaded from checkpoint, so before the checkpoint loading a hook runs which changes the VoxelGridModule's parameters to match shapes of the loaded checkpoint.
Reviewed By: bottler
Differential Revision: D39026775
fbshipit-source-id: 0d359ea5c8d2eda11d773d79c7513c83585d5f17
Summary:
User reported that cloned cameras fail to save. The error with latest PyTorch is
```
pickle.PicklingError: Can't pickle ~T_destination: attribute lookup T_destination on torch.nn.modules.module failed
```
This fixes it.
Reviewed By: btgraham
Differential Revision: D39692258
fbshipit-source-id: 75bbf3b8dfa0023dc28bf7d4cc253ca96e46a64d
Summary:
We need to make packing/unpacking in 2 places for mixed frame raysampling (metrics and raysampler) but those tensors that need to be unpacked/packed have more than two dimensions.
I could have reshaped and stored dimensions but this seems to just complicate code there with something which packed_to_padded should support.
I could have made a separate function for implicitron but it would confusing to have two different padded_to_packed functions inside pytorch3d codebase one of which does packing for (b, max) and (b, max, f) and the other for (b, max, …)
Reviewed By: bottler
Differential Revision: D39729026
fbshipit-source-id: 2bdebf290dcc6c316b7fe1aeee49bbb5255e508c
Summary: The implicit function and its members and internal working
Reviewed By: kjchalup
Differential Revision: D38829764
fbshipit-source-id: 28394fe7819e311ed52c9defc9a1b29f37fbc495
Summary: Allow using the new `foreach` option on optimizers.
Reviewed By: shapovalov
Differential Revision: D39694843
fbshipit-source-id: 97109c245b669bc6edff0f246893f95b7ae71f90
Summary: Add the ability to process arbitrary point shapes `[n_grids, ..., 3]` instead of only `[n_grids, n_points, 3]`.
Reviewed By: bottler
Differential Revision: D39574373
fbshipit-source-id: 0a9ecafe9ea58cd8f909644de43a1185ecf934f4
Summary:
Added export of UV textures to IO.save_mesh in Pytorch3d
MeshObjFormat now passes verts_uv, faces_uv, and texture_map as input to save_obj
TODO: check if TexturesUV.verts_uv_list or TexturesUV.verts_uv_padded() should be passed to save_obj
IO.save_mesh(obj_file, meshes, decimal_places=2) should be IO().save_mesh(obj_file, meshes, decimal_places=2)
Reviewed By: bottler
Differential Revision: D39617441
fbshipit-source-id: 4628b7f26f70e38c65f235852b990c8edb0ded23
Summary:
A significant speedup (e.g. >2% of a forward pass).
Move NDCMultinomialRaysampler parts of AbstractMaskRaySampler to members instead of living in a dict. The dict was hiding them from the nn.Module system so their _xy_grid members were remaining on the CPU. Therefore they were being copied to the GPU in every forward pass.
(We couldn't easily use a ModuleDict here because the enum keys are not strs.)
Reviewed By: shapovalov
Differential Revision: D39668589
fbshipit-source-id: 719b88e4a08fd7263a284e0ab38189e666bd7e3a
Summary:
- indicate location of OmegaConf.structured failures
- split the data gathering from enable_get_default_args to ease experimenting with it.
- comment fixes.
- nicer error when a_class_type has weird type.
Reviewed By: kjchalup
Differential Revision: D39434447
fbshipit-source-id: b80c7941547ca450e848038ef5be95b7ebbe8f3e
Summary: Various fixes to get visualize_reconstruction running, and an interactive test for it.
Reviewed By: kjchalup
Differential Revision: D39286691
fbshipit-source-id: 88735034cc01736b24735bcb024577e6ab7ed336
Summary: Workaround for oddity with new hydra.
Reviewed By: davnov134
Differential Revision: D39280639
fbshipit-source-id: 76e91947f633589945446db93cf2dbc259642f8a
Summary: Samples batches without replacement if the number of samples is not specified. This makes sure that we always iterate over the whole dataset in each epoch.
Reviewed By: bottler
Differential Revision: D39270786
fbshipit-source-id: 0c983d1f5e0af711463abfb23939bc0d2b5172a0
Summary:
Move the flyaround rendering function into core implicitron.
The unblocks an example in the facebookresearch/co3d repo.
Reviewed By: bottler
Differential Revision: D39257801
fbshipit-source-id: 6841a88a43d4aa364dd86ba83ca2d4c3cf0435a4
Summary:
The self._stratified_sampling attribute is always overridden unless stratified_sampling is explicitly set to None. However, the desired default behavior is that the value of self._stratified_sampling is used unless the argument stratified_sampling is set to True/False. Changing the default to None achieves this
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1324
Reviewed By: bottler
Differential Revision: D39259775
Pulled By: davnov134
fbshipit-source-id: e01bb747ac80c812eb27bf22e67f5e14f29acadd
Summary: On each call of the stats.update the object calculates current average iteration time by getting time elapsed from the time_start and then dividing it by the current number of steps. It saves the result to AverageMeter object which when queried returns the average of things saved, so the time is averaged twice which biases it towards the start value (which is often larger).
Reviewed By: kjchalup
Differential Revision: D39206989
fbshipit-source-id: ccab5233d7aaca1ac4fd626fb329b83c7c0d6af9
Summary: Currently some implicit functions in implicitron take a raybundle, others take ray_points_world. raybundle is what they really need. However, the raybundle is going to become a bit more flexible later, as it will contain different numbers of rays for each camera.
Reviewed By: bottler
Differential Revision: D39173751
fbshipit-source-id: ebc038e426d22e831e67a18ba64655d8a61e1eb9
Summary: Update the docstring for try_get_projection_transform on the API design.
Reviewed By: kjchalup
Differential Revision: D39227333
fbshipit-source-id: c9d0e625735d4972116d1f71865fb9b763e684de
Summary:
1) Update rasterizer/point rasterizer to accommodate fisheyecamera. Specifically, transform_points is in placement of explicit transform compositions.
2) In rasterizer unittests, update corresponding tests for rasterizer and point_rasterizer. Address comments to test fisheye against perspective camera when distortions are turned off.
3) Address comments to add end2end test for fisheyecameras. In test_render_meshes, fisheyecameras are added to camera enuerations whenever possible.
4) Test renderings with fisheyecameras of different params on cow mesh.
5) Use compositions for linear cameras whenever possible.
Reviewed By: kjchalup
Differential Revision: D38932736
fbshipit-source-id: 5b7074fc001f2390f4cf43c7267a8b37fd987547
Summary:
Amend FisheyeCamera by adding tests for all
combination of params and for different batch_sizes.
Reviewed By: kjchalup
Differential Revision: D39176747
fbshipit-source-id: 830d30da24beeb2f0df52db0b17a4303ed53b59c
Summary: A dummy value in test_opengl_utils seems to be able to break tests in test_mesh_renderer_opengl{,_to}.
Reviewed By: kjchalup
Differential Revision: D39173275
fbshipit-source-id: 83b15159f70135ea575d5085c7b6b37badd6e49e
Summary: D38919607 (c4545a7cbc) and D38858887 (06cbba2628) were premature, turns out CUDA 10.2 doesn't support C++17.
Reviewed By: bottler
Differential Revision: D39156205
fbshipit-source-id: 5e2e84cc4a57d1113a915166631651d438540d56
Summary:
Adds yaml configs to train selected methods on CO3Dv2.
Few more updates:
1) moved some fields to base classes so that we can check is_multisequence in experiment.py
2) skip loading all train cameras for multisequence datasets, without this, co3d-fewview is untrainable
3) fix bug in json index dataset provider v2
Reviewed By: kjchalup
Differential Revision: D38952755
fbshipit-source-id: 3edac6fc8e20775aa70400bd73a0e6d52b091e0c
Summary: Address comments to add benchmarkings for cameras and the new fisheye cameras. The dependency functions in test_cameras have been updated in Diff 1. The following two snapshots show benchmarking results.
Reviewed By: kjchalup
Differential Revision: D38991914
fbshipit-source-id: 51fe9bb7237543e4ee112c9f5068a4cf12a9d482
Summary:
1. A Fisheye camera model that generalizes pinhole camera by considering distortions (i.e. radial, tangential and thin-prism distortions).
2. Added tests against perspective cameras when distortions are off and Aria data points when distortions are on.
3. Address comments to test unhandled shapes between points and transforms. Added tests for __FIELDS, shape broadcasts, cuda etc.
4. Address earlier comments for code efficiency (e.g., adopted torch.norm; torch.solve for matrix inverse; removed inplace operations; unnecessary clone; expand in place of repeat etc).
Reviewed By: jcjohnson
Differential Revision: D38407094
fbshipit-source-id: a3ab48c85c496ac87af692d5d461bb3fc2a2db13
Summary: I think there is a typo here could not find any MultiPassEARenderer just MultiPassEmissionAbsorptionRenderer?
Reviewed By: bottler
Differential Revision: D39056641
fbshipit-source-id: 4dd0b123fc795a0083a957786c032e23dc5abac9
Summary: Added replacable decoding functions which will be applied after the voxel grid to get color and density
Reviewed By: bottler
Differential Revision: D38829763
fbshipit-source-id: f21ce206c1c19548206ea2ce97d7ebea3de30a23
Summary: Simple wrapper around voxel grids to make them a module
Reviewed By: bottler
Differential Revision: D38829762
fbshipit-source-id: dfee85088fa3c65e396cc7d3bf7ebaaffaadb646
Summary:
One of the docstrings is a disaster see https://pytorch3d.readthedocs.io/en/latest/modules/ops.html
Also some minor fixes I encountered when browsing the code
Reviewed By: bottler
Differential Revision: D38581595
fbshipit-source-id: 3b6ca97788af380a44df9144a6a4cac782c6eab8
