Summary:
For fg-masking depth, we assumed np.array but passed a Tensor; for defining the default depth_mask, vice versa.
Note that we change the intended behaviour for the latter, assuming that 0s are areas with empty depth. When loading depth masks, we replace NaNs with zeros, so it is sensible. It is not a BC change as that branch would crash if executed. Since there was no reports, I assume no one cared.
Reviewed By: bottler
Differential Revision: D47403588
fbshipit-source-id: 1094104176d7d767a5657b5bbc9f5a0cc9da0ede
Summary:
Convert ImplicitronRayBundle to a "classic" class instead of a dataclass. This change is introduced as a way to preserve the ImplicitronRayBundle interface while allowing two outcomes:
- init lengths arguments is now a Optional[torch.Tensor] instead of torch.Tensor
- lengths is now a property which returns a `torch.Tensor`. The lengths property will either recompute lengths from bins or return the stored _lengths. `_lenghts` is None if bins is set. It saves us a bit of memory.
Reviewed By: shapovalov
Differential Revision: D46686094
fbshipit-source-id: 3c75c0947216476ebff542b6f552d311024a679b
Summary:
## Context
Bins are used in mipnerf to allow to manipulate easily intervals. For example, by doing the following, `bins[..., :-1]` you will obtain all the left coordinates of your intervals, while doing `bins[..., 1:]` is equals to the right coordinates of your intervals.
We introduce here the support of bins like in MipNerf implementation.
## RayPointRefiner
Small changes have been made to modify RayPointRefiner.
- If bins is None
```
mids = torch.lerp(ray_bundle.lengths[..., 1:], ray_bundle.lengths[…, :-1], 0.5)
z_samples = sample_pdf(
mids, # [..., npt]
weights[..., 1:-1], # [..., npt - 1]
….
)
```
- If bins is not None
In the MipNerf implementation the sampling is done on all the bins. It allows us to use the full weights tensor without slashing it.
```
z_samples = sample_pdf(
ray_bundle.bins, # [..., npt + 1]
weights, # [..., npt]
...
)
```
## RayMarcher
Add a ray_deltas optional argument. If None, keep the same deltas computation from ray_lengths.
Reviewed By: shapovalov
Differential Revision: D46389092
fbshipit-source-id: d4f1963310065bd31c1c7fac1adfe11cbeaba606
Summary:
Add blurpool has defined in [MIP-NeRF](https://arxiv.org/abs/2103.13415).
It has been added has an option for RayPointRefiner.
Reviewed By: shapovalov
Differential Revision: D46356189
fbshipit-source-id: ad841bad86d2b591a68e1cb885d4f781cf26c111
Summary: Add a new implicit module Integral Position Encoding based on [MIP-NeRF](https://arxiv.org/abs/2103.13415).
Reviewed By: shapovalov
Differential Revision: D46352730
fbshipit-source-id: c6a56134c975d80052b3a11f5e92fd7d95cbff1e
Summary:
Introduce methods to approximate the radii of conical frustums along rays as described in [MipNerf](https://arxiv.org/abs/2103.13415):
- Two new attributes are added to ImplicitronRayBundle: bins and radii. Bins is of size n_pts_per_ray + 1. It allows us to manipulate easily and n_pts_per_ray intervals. For example we need the intervals coordinates in the radii computation for \(t_{\mu}, t_{\delta}\). Radii are used to store the radii of the conical frustums.
- Add 3 new methods to compute the radii:
- approximate_conical_frustum_as_gaussians: It computes the mean along the ray direction, the variance of the
conical frustum with respect to t and variance of the conical frustum with respect to its radius. This
implementation follows the stable computation defined in the paper.
- compute_3d_diagonal_covariance_gaussian: Will leverage the two previously computed variances to find the
diagonal covariance of the Gaussian.
- conical_frustum_to_gaussian: Mix everything together to compute the means and the diagonal covariances along
the ray of the Gaussians.
- In AbstractMaskRaySampler, introduces the attribute `cast_ray_bundle_as_cone`. If False it won't change the previous behaviour of the RaySampler. However if True, the samplers will sample `n_pts_per_ray +1` instead of `n_pts_per_ray`. This points are then used to set the bins attribute of ImplicitronRayBundle. The support of HeterogeneousRayBundle has not been added since the current code does not allow it. A safeguard has been added to avoid a silent bug in the future.
Reviewed By: shapovalov
Differential Revision: D45269190
fbshipit-source-id: bf22fad12d71d55392f054e3f680013aa0d59b78
Summary: Making it easier for the clients to use these datasets.
Reviewed By: bottler
Differential Revision: D46727179
fbshipit-source-id: cf619aee4c4c0222a74b30ea590cf37f08f014cc
Summary: In D42739669, I forgot to update the API of existing implementations of DatasetBase to take `subset_filter`. Looks like only one was missing.
Reviewed By: bottler
Differential Revision: D46724488
fbshipit-source-id: 13ab7a457f853278cf06955aad0cc2bab5fbcce6
Summary:
Adds stratified sampling of sequences within categories applied after category / sequence filters but before the num sequence limit.
It respects the insertion order into the sequence_annots table, i.e. takes top N sequences within each category.
Reviewed By: bottler
Differential Revision: D46724002
fbshipit-source-id: 597cb2a795c3f3bc07f838fc51b4e95a4f981ad3
Summary: Single directional chamfer distance and option to use non-absolute cosine similarity
Reviewed By: bottler
Differential Revision: D46593980
fbshipit-source-id: b2e591706a0cdde1c2d361614cecebb84a581433
Summary: Fine implicit function was called before the coarse implicit function.
Reviewed By: shapovalov
Differential Revision: D46224224
fbshipit-source-id: 6b1cc00cc823d3ea7a5b42774c9ec3b73a69edb5
Summary:
1. We may need to store arrays of unknown shape in the database. It implements and tests serialisation.
2. Previously, when an inexisting metadata file was passed to SqlIndexDataset, it would try to open it and create an empty file, then crash. We now open the file in a read-only mode, so the error message is more intuitive. Note that the implementation is SQLite specific.
Reviewed By: bottler
Differential Revision: D46047857
fbshipit-source-id: 3064ae4f8122b4fc24ad3d6ab696572ebe8d0c26
Summary:
This is mostly a refactoring diff to reduce friction in extending the frame data.
Slight functional changes: dataset getitem now accepts (seq_name, frame_number_as_singleton_tensor) as a non-advertised feature. Otherwise this code crashes:
```
item = dataset[0]
dataset[item.sequence_name, item.frame_number]
```
Reviewed By: bottler
Differential Revision: D45780175
fbshipit-source-id: 75b8e8d3dabed954a804310abdbd8ab44a8dea29
Summary: We don't want to use print directly in stats.print() method. Instead this method will return the output string to the caller.
Reviewed By: shapovalov
Differential Revision: D45356240
fbshipit-source-id: 2cabe3cdfb9206bf09aa7b3cdd2263148a5ba145
Summary:
typing.NamedTuple was simplified in 3.10
These two fields were the same in 3.8, so this should be a no-op
#buildmore
Reviewed By: bottler
Differential Revision: D45373526
fbshipit-source-id: 2b26156f5f65b7be335133e9e705730f7254260d
Summary:
Although we can load per-vertex normals in `load_obj`, saving per-vertex normals is not supported in `save_obj`.
This patch fixes this by allowing passing per-vertex normal data in `save_obj`:
``` python
def save_obj(
f: PathOrStr,
verts,
faces,
decimal_places: Optional[int] = None,
path_manager: Optional[PathManager] = None,
*,
verts_normals: Optional[torch.Tensor] = None,
faces_normals: Optional[torch.Tensor] = None,
verts_uvs: Optional[torch.Tensor] = None,
faces_uvs: Optional[torch.Tensor] = None,
texture_map: Optional[torch.Tensor] = None,
) -> None:
"""
Save a mesh to an .obj file.
Args:
f: File (str or path) to which the mesh should be written.
verts: FloatTensor of shape (V, 3) giving vertex coordinates.
faces: LongTensor of shape (F, 3) giving faces.
decimal_places: Number of decimal places for saving.
path_manager: Optional PathManager for interpreting f if
it is a str.
verts_normals: FloatTensor of shape (V, 3) giving the normal per vertex.
faces_normals: LongTensor of shape (F, 3) giving the index into verts_normals
for each vertex in the face.
verts_uvs: FloatTensor of shape (V, 2) giving the uv coordinate per vertex.
faces_uvs: LongTensor of shape (F, 3) giving the index into verts_uvs for
each vertex in the face.
texture_map: FloatTensor of shape (H, W, 3) representing the texture map
for the mesh which will be saved as an image. The values are expected
to be in the range [0, 1],
"""
```
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1511
Reviewed By: shapovalov
Differential Revision: D45086045
Pulled By: bottler
fbshipit-source-id: 666efb0d2c302df6cf9f2f6601d83a07856bf32f
Summary:
Added a suit of functions and code additions to experimental_gltf_io.py file to enable saving Meshes in TexturesVertex format into .glb file.
Also added a test to tets_io_gltf.py to check the functionality with the test described in Test Plane.
Reviewed By: bottler
Differential Revision: D44969144
fbshipit-source-id: 9ce815a1584b510442fa36cc4dbc8d41cc3786d5
Summary: Remove the need of tuple and reversed in the raysampling xy_grid computation
Reviewed By: bottler
Differential Revision: D45269342
fbshipit-source-id: d0e4c0923b9a2cca674b35e8d64862043a0eab3b
Summary:
Moving SQL dataset to PyTorch3D. It has been extensively tested in pixar_replay.
It requires SQLAlchemy 2.0, which is not supported in fbcode. So I exclude the sources and tests that depend on it from buck TARGETS.
Reviewed By: bottler
Differential Revision: D45086611
fbshipit-source-id: 0285f03e5824c0478c70ad13731525bb5ec7deef
Summary:
We currently support caching bounding boxes in MaskAnnotation. If present, they are not re-computed from the mask. However, the masks need to be loaded for the bbox to be set.
This diff fixes that. Even if load_masks / load_blobs are unset, the bounding box can be picked up from the metadata.
Reviewed By: bottler
Differential Revision: D45144918
fbshipit-source-id: 8a2e2c115e96070b6fcdc29cbe57e1cee606ddcd
Summary: The code does not crash if depth map/mask are not given.
Reviewed By: bottler
Differential Revision: D45082985
fbshipit-source-id: 3610d8beb4ac897fbbe52f56a6dd012a6365b89b
Summary:
The pattern
```
X.Y if hasattr(X, "Y") else Z
```
can be replaced with
```
getattr(X, "Y", Z)
```
The [getattr](https://www.w3schools.com/python/ref_func_getattr.asp) function gives more succinct code than the [hasattr](https://www.w3schools.com/python/ref_func_hasattr.asp) function. Please use it when appropriate.
**This diff is very low risk. Green tests indicate that you can safely Accept & Ship.**
Reviewed By: bottler
Differential Revision: D44886893
fbshipit-source-id: 86ba23e837217e1ebd64bf8e27d286257894839e
Summary: Provide an extension point pre_expand to let a configurable class A make sure another class B is registered before A is expanded. This reduces top level imports.
Reviewed By: bottler
Differential Revision: D44504122
fbshipit-source-id: c418bebbe6d33862d239be592d9751378eee3a62
Summary:
Introduces the OverfitModel for NeRF-style training with overfitting to one scene.
It is a specific case of GenericModel. It has been disentangle to ease usage.
## General modification
1. Modularize a minimum GenericModel to introduce OverfitModel
2. Introduce OverfitModel and ensure through unit testing that it behaves like GenericModel.
## Modularization
The following methods have been extracted from GenericModel to allow modularity with ManyViewModel:
- get_objective is now a call to weighted_sum_losses
- log_loss_weights
- prepare_inputs
The generic methods have been moved to an utils.py file.
Simplify the code to introduce OverfitModel.
Private methods like chunk_generator are now public and can now be used by ManyViewModel.
Reviewed By: shapovalov
Differential Revision: D43771992
fbshipit-source-id: 6102aeb21c7fdd56aa2ff9cd1dd23fd9fbf26315
Summary: For safety checks, make inplace forward operations in cuda and c++ call increment_version.
Reviewed By: davidsonic
Differential Revision: D44302504
fbshipit-source-id: 6ff62251e352d6778cb54399e2e11459e16e77ba
Summary: - Replace all the relative imports for generic models to absolute import: (from . import y => from pytorch3.x import y)
Reviewed By: shapovalov
Differential Revision: D43620682
fbshipit-source-id: 937318b339b5020d17b511a891c7b000ff659328
Summary:
Aligning the logic with the official CO3Dv2 evaluation: 92283c4368/co3d/dataset/utils.py (L7)
This will make the evaluator work with the datasets that do not define known/unseen subsets.
Reviewed By: bottler
Differential Revision: D42803136
fbshipit-source-id: cfac389eab010c32d2e33b40fc7f6ed845c327ef
Summary: If a configurable class inherits torch.nn.Module and is instantiated, automatically call `torch.nn.Module.__init__` on it before doing anything else.
Reviewed By: shapovalov
Differential Revision: D42760349
fbshipit-source-id: 409894911a4252b7987e1fd218ee9ecefbec8e62
Summary: ChainDataset is iterable, and it toes not go along with a custom batch sampler.
Reviewed By: bottler
Differential Revision: D42742315
fbshipit-source-id: 40a715c8d24abe72cb2777634247d7467f628564
Summary: For the new API, filtering iterators over sequences by subsets is quite helpful. The change is backwards compatible.
Reviewed By: bottler
Differential Revision: D42739669
fbshipit-source-id: d150a404aeaf42fd04a81304c63a4cba203f897d
Summary:
V2 dataset does not have the concept of known/unseen frames. Test-time conditining is done with train-set frames, which violates the previous check.
Also fixing a corner case in VideoWriter.
Reviewed By: bottler
Differential Revision: D42706976
fbshipit-source-id: d43be3dd3060d18cb9f46d5dcf6252d9f084110f
