Summary: Performance improvement: Use torch.lerp to map uv coordinates to the range needed for grid_sample (i.e. map [0, 1] to [-1, 1] and invert the y-axis)
Reviewed By: bottler
Differential Revision: D51961728
fbshipit-source-id: db19a5e3f482e9af7b96b20f88a1e5d0076dac43
Summary: User confusion (https://github.com/facebookresearch/pytorch3d/issues/1579) about how zbuf is used for alpha compositing. Added small description and reference to paper to help give some context.
Reviewed By: bottler
Differential Revision: D51374933
fbshipit-source-id: 8c489a5b5d0a81f0d936c1348b9ade6787c39c9a
Summary: Fixes lint in test_render_points in the PyTorch3D library.
Differential Revision: D51289841
fbshipit-source-id: 1eae621eb8e87b0fe5979f35acd878944f574a6a
Summary:
When the ply format looks as follows:
```
comment TextureFile ***.png
element vertex 892
property double x
property double y
property double z
property double nx
property double ny
property double nz
property double texture_u
property double texture_v
```
`MeshPlyFormat` class will read uv from the ply file and read the uv map as commented as TextureFile.
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1100
Reviewed By: MichaelRamamonjisoa
Differential Revision: D50885176
Pulled By: bottler
fbshipit-source-id: be75b1ec9a17a1ed87dbcf846a9072ea967aec37
Summary: Remove unused argument `mask_points` from `get_rgbd_point_cloud` and fix `get_implicitron_sequence_pointcloud`, which assumed it was used.
Reviewed By: MichaelRamamonjisoa
Differential Revision: D50885848
fbshipit-source-id: c0b834764ad5ef560107bd8eab04952d000489b8
Summary: I think we include more thrust than needed, and maybe removing it will help things like https://github.com/facebookresearch/pytorch3d/issues/1610 with DebugSyncStream errors on Windows.
Reviewed By: shapovalov
Differential Revision: D48949888
fbshipit-source-id: add889c0acf730a039dc9ffd6bbcc24ded20ef27
Summary: Python3 makes the use of `(object)` in class inheritance unnecessary. Let's modernize our code by eliminating this.
Reviewed By: itamaro
Differential Revision: D48673863
fbshipit-source-id: 032d6028371f0350252e6b731c74f0f5933c83cd
Summary:
The `chamfer_distance` function currently allows `"sum"` or `"mean"` reduction, but does not support returning unreduced (per-point) loss terms. Unreduced losses could be useful if the user wishes to inspect individual losses, or perform additional modifications to loss terms before reduction. One example would be implementing a robust kernel over the loss.
This PR adds a `None` option to the `point_reduction` parameter, similar to `batch_reduction`. In case of bi-directional chamfer loss, both the forward and backward distances are returned (a tuple of Tensors of shape `[D, N]` is returned). If normals are provided, similar logic applies to normals as well.
This PR addresses issue https://github.com/facebookresearch/pytorch3d/issues/622.
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1605
Reviewed By: jcjohnson
Differential Revision: D48313857
Pulled By: bottler
fbshipit-source-id: 35c824827a143649b04166c4817449e1341b7fd9
Summary:
Something's wrong with recommonmark/CommonMark/six, let's see if this fixes it.
https://readthedocs.org/projects/pytorch3d/builds/21292632/
```
File "/home/docs/checkouts/readthedocs.org/user_builds/pytorch3d/envs/latest/lib/python3.11/site-packages/sphinx/config.py", line 368, in eval_config_file
execfile_(filename, namespace)
File "/home/docs/checkouts/readthedocs.org/user_builds/pytorch3d/envs/latest/lib/python3.11/site-packages/sphinx/util/pycompat.py", line 150, in execfile_
exec_(code, _globals)
File "/home/docs/checkouts/readthedocs.org/user_builds/pytorch3d/checkouts/latest/docs/conf.py", line 25, in <module>
from recommonmark.parser import CommonMarkParser
File "/home/docs/checkouts/readthedocs.org/user_builds/pytorch3d/envs/latest/lib/python3.11/site-packages/recommonmark/parser.py", line 6, in <module>
from CommonMark import DocParser, HTMLRenderer
File "/home/docs/checkouts/readthedocs.org/user_builds/pytorch3d/envs/latest/lib/python3.11/site-packages/CommonMark/__init__.py", line 3, in <module>
from CommonMark.CommonMark import HTMLRenderer
File "/home/docs/checkouts/readthedocs.org/user_builds/pytorch3d/envs/latest/lib/python3.11/site-packages/CommonMark/CommonMark.py", line 18, in <module>
HTMLunescape = html.parser.HTMLParser().unescape
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'HTMLParser' object has no attribute 'unescape'
```
Reviewed By: shapovalov
Differential Revision: D47471545
fbshipit-source-id: 48e121e20da535b3cc46b6bd2393d28869067b8b
Summary: New versions of cuda etc. I haven't committed recent changes to this for a while
Reviewed By: shapovalov
Differential Revision: D47396136
fbshipit-source-id: d6c27f5056fa8f4a74a628fa1d831159000acf55
Summary: This is needed from september 2023. As a side effect, implicitron docs should build better because typing.get_args exists etc.
Reviewed By: shapovalov
Differential Revision: D47363855
fbshipit-source-id: a954c5b81b1e5a4435fca146a11aea0d2ca96f45
Summary:
Blender uses OpenEXR to dump depth maps, so we have to support it.
OpenCV requires to explicitly accepth the vulnerabilities by setting the env var before exporting.
We can set it but I think it should be user’s responsibility.
OpenCV error reporting is adequate, so I don’t handle the error on our side.
Reviewed By: bottler
Differential Revision: D47403884
fbshipit-source-id: 2fcadd1df9d0efa0aea563bcfb2e3180b3c4d1d7
Summary:
For fg-masking depth, we assumed np.array but passed a Tensor; for defining the default depth_mask, vice versa.
Note that we change the intended behaviour for the latter, assuming that 0s are areas with empty depth. When loading depth masks, we replace NaNs with zeros, so it is sensible. It is not a BC change as that branch would crash if executed. Since there was no reports, I assume no one cared.
Reviewed By: bottler
Differential Revision: D47403588
fbshipit-source-id: 1094104176d7d767a5657b5bbc9f5a0cc9da0ede
Summary:
Convert ImplicitronRayBundle to a "classic" class instead of a dataclass. This change is introduced as a way to preserve the ImplicitronRayBundle interface while allowing two outcomes:
- init lengths arguments is now a Optional[torch.Tensor] instead of torch.Tensor
- lengths is now a property which returns a `torch.Tensor`. The lengths property will either recompute lengths from bins or return the stored _lengths. `_lenghts` is None if bins is set. It saves us a bit of memory.
Reviewed By: shapovalov
Differential Revision: D46686094
fbshipit-source-id: 3c75c0947216476ebff542b6f552d311024a679b
Summary:
## Context
Bins are used in mipnerf to allow to manipulate easily intervals. For example, by doing the following, `bins[..., :-1]` you will obtain all the left coordinates of your intervals, while doing `bins[..., 1:]` is equals to the right coordinates of your intervals.
We introduce here the support of bins like in MipNerf implementation.
## RayPointRefiner
Small changes have been made to modify RayPointRefiner.
- If bins is None
```
mids = torch.lerp(ray_bundle.lengths[..., 1:], ray_bundle.lengths[…, :-1], 0.5)
z_samples = sample_pdf(
mids, # [..., npt]
weights[..., 1:-1], # [..., npt - 1]
….
)
```
- If bins is not None
In the MipNerf implementation the sampling is done on all the bins. It allows us to use the full weights tensor without slashing it.
```
z_samples = sample_pdf(
ray_bundle.bins, # [..., npt + 1]
weights, # [..., npt]
...
)
```
## RayMarcher
Add a ray_deltas optional argument. If None, keep the same deltas computation from ray_lengths.
Reviewed By: shapovalov
Differential Revision: D46389092
fbshipit-source-id: d4f1963310065bd31c1c7fac1adfe11cbeaba606
Summary:
Add blurpool has defined in [MIP-NeRF](https://arxiv.org/abs/2103.13415).
It has been added has an option for RayPointRefiner.
Reviewed By: shapovalov
Differential Revision: D46356189
fbshipit-source-id: ad841bad86d2b591a68e1cb885d4f781cf26c111
Summary: Add a new implicit module Integral Position Encoding based on [MIP-NeRF](https://arxiv.org/abs/2103.13415).
Reviewed By: shapovalov
Differential Revision: D46352730
fbshipit-source-id: c6a56134c975d80052b3a11f5e92fd7d95cbff1e
Summary:
Introduce methods to approximate the radii of conical frustums along rays as described in [MipNerf](https://arxiv.org/abs/2103.13415):
- Two new attributes are added to ImplicitronRayBundle: bins and radii. Bins is of size n_pts_per_ray + 1. It allows us to manipulate easily and n_pts_per_ray intervals. For example we need the intervals coordinates in the radii computation for \(t_{\mu}, t_{\delta}\). Radii are used to store the radii of the conical frustums.
- Add 3 new methods to compute the radii:
- approximate_conical_frustum_as_gaussians: It computes the mean along the ray direction, the variance of the
conical frustum with respect to t and variance of the conical frustum with respect to its radius. This
implementation follows the stable computation defined in the paper.
- compute_3d_diagonal_covariance_gaussian: Will leverage the two previously computed variances to find the
diagonal covariance of the Gaussian.
- conical_frustum_to_gaussian: Mix everything together to compute the means and the diagonal covariances along
the ray of the Gaussians.
- In AbstractMaskRaySampler, introduces the attribute `cast_ray_bundle_as_cone`. If False it won't change the previous behaviour of the RaySampler. However if True, the samplers will sample `n_pts_per_ray +1` instead of `n_pts_per_ray`. This points are then used to set the bins attribute of ImplicitronRayBundle. The support of HeterogeneousRayBundle has not been added since the current code does not allow it. A safeguard has been added to avoid a silent bug in the future.
Reviewed By: shapovalov
Differential Revision: D45269190
fbshipit-source-id: bf22fad12d71d55392f054e3f680013aa0d59b78
Summary: We now use unittest.mock
Reviewed By: shapovalov
Differential Revision: D45868799
fbshipit-source-id: cd1042dc2c49c82c7b9e024f761c496049a31beb
Summary: Make test work in isolation, and when run internally make it not try the sqlalchemy files.
Reviewed By: shapovalov
Differential Revision: D46352513
fbshipit-source-id: 7417a25d7a5347d937631c9f56ae4e3242dd622e
Summary:
Hi,
Not sure this is the best fix. But while running this notebook, I only ever saw a blank canvas when trying to visualize the dolphin. It might be that I have a broken dependency, like plotly. I also don't know what the visualization is "supposed" to look like.
But incase other people have this issue, this one line change solved the whole problem for me. Now I have a happy, rotatable dolphin.
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1549
Reviewed By: shapovalov
Differential Revision: D46350930
Pulled By: bottler
fbshipit-source-id: e19aa71eb05a93e2955262a2c90d1f0d09576228
Summary: Fix for https://github.com/facebookresearch/pytorch3d/issues/1441 where we were indexing with a tensor on the wrong device.
Reviewed By: shapovalov
Differential Revision: D46276449
fbshipit-source-id: 7750ed45ffecefa5d291fd1eadfe515310c2cf0d
Summary: Making it easier for the clients to use these datasets.
Reviewed By: bottler
Differential Revision: D46727179
fbshipit-source-id: cf619aee4c4c0222a74b30ea590cf37f08f014cc
Summary: In D42739669, I forgot to update the API of existing implementations of DatasetBase to take `subset_filter`. Looks like only one was missing.
Reviewed By: bottler
Differential Revision: D46724488
fbshipit-source-id: 13ab7a457f853278cf06955aad0cc2bab5fbcce6
Summary:
Adds stratified sampling of sequences within categories applied after category / sequence filters but before the num sequence limit.
It respects the insertion order into the sequence_annots table, i.e. takes top N sequences within each category.
Reviewed By: bottler
Differential Revision: D46724002
fbshipit-source-id: 597cb2a795c3f3bc07f838fc51b4e95a4f981ad3
Summary: Single directional chamfer distance and option to use non-absolute cosine similarity
Reviewed By: bottler
Differential Revision: D46593980
fbshipit-source-id: b2e591706a0cdde1c2d361614cecebb84a581433
Summary: Fine implicit function was called before the coarse implicit function.
Reviewed By: shapovalov
Differential Revision: D46224224
fbshipit-source-id: 6b1cc00cc823d3ea7a5b42774c9ec3b73a69edb5
