Summary:
GitHub issue #2002 (https://github.com/facebookresearch/pytorch3d/issues/2002)
points out that the near-pi branch of `matrix_to_axis_angle(..., fast=True)`
normalizes with `torch.norm(n)`, which reduces over the whole batch. That is
real, and there were two further problems next to it.
`matrix_to_axis_angle(..., fast=True)`:
- The axis was read from row 0 of `(R + I) / 2`, which is `n_x * n`. That is
exactly zero whenever the axis is perpendicular to x, giving `nan` (for
example for an exact rotation by pi about y), and is dominated by rounding
when `abs(n_x)` is small. It is now read from
`R + R^T - 2*cos(angle)*I == 2*(1 - cos(angle))*nnT`, taking the column whose
diagonal entry is largest so that the multiplier is at least `1/sqrt(3)`.
Symmetrizing also makes the identity exact at every angle rather than only at
pi. The sign, which `nnT` does not determine, comes from `omegas`.
- `torch.norm(n)` becomes a per-row `torch.linalg.vector_norm`, which is the
reported bug: with more than one near-pi rotation in a batch, every one of
them was scaled wrongly.
- The branch threshold was `isclose(angle, pi)`. `omegas` is `2*sin(angle)`
times the axis, so the other branch loses relative precision like
`1/(pi - angle)`, and just outside `isclose` the float32 round-trip error
reached 9.6e-4. The threshold is now `pi - 1e-2`, which leaves under 1% of
uniformly random rotations on the more expensive branch.
- The `torch.isclose(angles, 0)` guard on `omegas` is removed. `torch.sinc(0)`
is 1, so a zero angle was never a special case, and `torch.norm` has a zero
rather than `nan` gradient at zero. The guard only zeroed the answer for
angles below its `atol` of 1e-8, returning 0 instead of an exact 1e-9
rotation, and cost 24% at a batch of 100k.
Worst-case float32 round-trip matrix error over random axes, before -> after:
exactly pi `nan` or 2.0 -> 3.6e-07; `pi - 1e-5` 2.0 -> 4.2e-07; `pi - 1e-4`
9.6e-04 -> 3.6e-07. The worst case at any angle is now 9.9e-06, at the branch
boundary, against 4.8e-07 for `fast=False`.
On the choice of `pi - 1e-2`: the new near-pi branch is accurate at any
threshold, so the threshold only decides how much of the batch takes the
slower branch, and the worst case is always the angle just below it. Measured
on one batch of 100k random float32 rotations on CPU, where `fast=False` took
11.4 ms, threshold against fraction of uniformly random rotations selected,
time, and worst-case round-trip error:
- `isclose(angle, pi)`, the previous behaviour: 0.003%, 8.6 ms, 1e-03
- `pi - 1e-3`: 0.1%, 8.9 ms, 9e-05
- `pi - 1e-2`, chosen: 0.7%, 8.9 ms, 1e-05
- `pi - 0.05`: 3.2%, 10.2 ms, 3e-06
- `3.0` radians: 9.0%, 17.6 ms, 9e-07
So `pi - 1e-2` buys two orders of magnitude of accuracy over the old threshold
for no measurable time, and it is the last threshold that is free; at `3.0`
radians `fast=True` would be slower than `fast=False`. The errors are sampled
over random axes, so they move in the last digit between runs.
`axis_angle_to_matrix(..., fast=True)` is Rodrigues' formula rewritten with
`cross_product_matrix^2 == outer_product - angle^2 * I` and
`(1 - cos(angle)) / angle^2 == sinc(angle / (2*pi))^2 / 2`. The first removes a
batched 3x3 matrix multiplication, which `bmm` serves poorly at that size, and
folds the leftover `-angle^2 * I` into the identity term as `cos(angle) * I`;
the second is defined at zero, so the `angles_sqrd == 0` special case goes
away. Values are unchanged to 8.9e-16.
`matrix_to_axis_angle` also used eight separate boolean mask indexes, each of
which re-runs `nonzero` over the whole batch and, on CUDA, synchronizes. The
cheap branch is now evaluated densely and the near-pi minority is selected
with a single `nonzero`.
Evaluating the near-pi branch densely too, which would remove the last
`nonzero` and so every data-dependent shape, was tried and rejected: against
this version it is 1.24x to 1.60x faster on CUDA at batches of 1k to 100k but
0.88x at 1M, and about 2x slower on CPU, and it does not survive
`torch.jit.script` without further work. It is the version to revisit if
export or `torch.compile` friendliness ever matters more than speed.
Speedups, before -> after:
- `matrix_to_axis_angle`: CUDA 1.88x at a batch of 1k, 1.66x at 100k, 1.47x at
1M; CPU 1.26x at 1k, 2.01x at 100k, 1.71x at 1M.
- `axis_angle_to_matrix`: CUDA 1.26x at 1k, 1.81x at 100k, 5.49x at 1M; CPU
unchanged.
Before this change `matrix_to_axis_angle(..., fast=True)` was slower than
`fast=False` on CUDA at a batch of 100k, 0.587 against 0.463 ms, because the
selection synchronized.
Rewriting `axis_angle_to_matrix` to compute all nine entries in one
`torch.stack`, as `quaternion_to_matrix` does, was also tried and is not
faster: 0.96x, 0.74x and 1.10x on CUDA at 1k, 100k and 1M, because the extra
kernel launches cost about what the saved intermediates gain.
Reviewed By: MichaelRamamonjisoa
Differential Revision: D115714860
fbshipit-source-id: cf19695f67bf2e2e6f8719419e9f902d5c58c309
Summary:
Fixes https://github.com/facebookresearch/pytorch3d/issues/1934. Two independent bugs in the CPU backend.
### 1. Degenerate-triangle filter discards valid faces
`marching_cubes_cpu.cpp`, `marching_cubes.py`
`tri.clear()` and `ps.clear()` sit inside the degeneracy check, so the buffers only reset when a triangle is *accepted*. Once a cube's first triangle is degenerate, `ps[0..2]` stay frozen on it, and every subsequent triangle in that cube fails the same stale check and is dropped.
Fixed by gating on `ps.size() == 3` and clearing unconditionally. The old code could only ever drop faces, never emit incorrect ones, so this is strictly additive.
### 2. Edge hash computed in float32
`marching_cubes_utils.h`
`p[v].x/y/z` hold integral coordinates but are stored as `float`, so `x + y*W + z*W*H` evaluates entirely in float32 before truncating to `int`. float32 is exact only to 2²⁴ − 1 = 16,777,215 — and 256³ maxes out at exactly that value. At 512³ the maximum id is 134,217,727, where float32 spacing is 8, so distinct vertices collide on one id and `uniq_edge_id` merges them.
Fixed by widening `W/H/D` to `int64_t` and casting each coordinate before multiplying. Also tightens the stride from `(W + W*H + W*H*D)` to `W*H*D`. Raises the CPU ceiling from 256³ to 1448³.
Scope is the CPU path only. `marching_cubes_naive` was never affected (Python ints are arbitrary-precision), and neither was CUDA: `hashVpair` there computes ids in `uint` rather than `float`, so it has no 2²⁴ cliff, and `MarchingCubes` already rejects volumes above 1024³ before the CUDA kernel runs. The new `TORCH_CHECK` bound and the "~1448³" note in the new comments describe `MarchingCubesCpu` only.
### Verification
Ellipsoid SDF (0.1, 1, 1), `isolevel=0.0`, identical input tensors on both devices.
| Resolution | CUDA V | CUDA F | CPU V | CPU F | Degenerate dropped |
| -- | -- | -- | -- | -- | -- |
| 32³ | 1,664 | 3,324 | 1,664 | 3,324 | 0 |
| 64³ | 7,312 | 14,620 | 7,312 | 14,620 | 0 |
| 128³ | 30,168 | 60,332 | 30,168 | 60,332 | 0 |
| 256³ | 122,448 | 244,892 | 122,448 | 243,996 | 896 |
| 512³ | 491,944 | 983,884 | 491,944 | 976,140 | 7,744 |
Machine: Arch Linux, RTX 4080, Ryzen 7 7800X3D
Vertex counts now match CUDA exactly at every resolution; 512³ previously produced 176,121. The remaining face gap is entirely degenerate geometry — the CUDA mesh at 512³ contains exactly 7,744 zero-area triangles.
### Tests
`test_degenerate_triangle_keeps_later_faces` — a 2×2×2 volume at `isolevel=1` chosen so the cube's four candidate triangles collapse onto its four outside corners: triangles 1 and 4 become degenerate, 2 and 3 stay valid. Pre-fix, the first degeneracy suppresses the rest and the mesh comes back empty; post-fix it is the expected quad. Asserts both `marching_cubes_naive` and the C++ extension.
`test_large_grid_edge_ids` — a 2×2×4,200,000 volume (~67MB, ~0.1s) holding 16 isolated interior points on the highest-id grid row, positioned so grid-point ids straddle 2²⁴. Each point cuts exactly four grid edges, so the 64-vertex expectation is derived geometrically rather than copied from output. Pre-fix, a hash collision merges two edges and one vertex is lost.
All 26 pre-existing tests in `test_marching_cubes.py` pass **unchanged**. That includes `test_cube_no_duplicate_verts` (`isolevel=1`) and `test_sphere` (`isolevel=64`), which both exercise the degenerate path but whose output is identical before and after the fix — so no existing expectation was edited and `sphere_level64.pickle` does not need regenerating.
The imported diff contained no test file. The two tests above were written during import and differ from the tests described in the upstream PR description.
*Analysis and write-up done collaboratively with AI, figures from testing are done on my own machine and have been checked.*
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/2043
Test Plan:
```
buck2 test fbcode//vision/fair/pytorch3d:tests -- --regex 'test_marching_cubes'
```
`Pass 28. Fail 0.` — 26 pre-existing tests plus the 2 new regression tests.
Reverting all three source hunks to their pre-fix state and re-running the same command: both new tests fail (`test_degenerate_triangle_keeps_later_faces` returns an empty mesh instead of 4 verts / 2 faces; `test_large_grid_edge_ids` returns 31 verts instead of 32) and all 26 pre-existing tests still pass. The new tests are therefore pinned to exactly this change, and the change breaks nothing that was already covered.
Reviewed By: MichaelRamamonjisoa
Differential Revision: D115424433
Pulled By: bottler
fbshipit-source-id: 547a260010b94253f52f3a3c223d4c4fa78a7ee6
Summary:
`_cot_laplacian_python` used in-place tensor operations (`clamp_`, `/=`,
`+=`) on intermediates that participate in autograd. Once the resulting
sparse Laplacian is used in a backward pass, these in-place mutations
raise a runtime error:
```
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.sparse.FloatTensor [4, 4]], which is output 0 of SparseCooTensorWithDimsAndTensors, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True, check_nan=False).
```
Replace the in-place ops with out-of-place equivalents so gradients can
flow back to the input vertices.
- `clamp_` → `clamp`
- `cot /= 4.0` → `cot = cot / 4.0`
- `L += L.t()` → `L = L + L.t()`
Reviewed By: bottler
Differential Revision: D111896032
fbshipit-source-id: 4c36677487bae5d37a9d81cb791400576f4b2c5f
Summary:
Enables building pytorch3d's `_C` extension against a ROCm-built PyTorch and running the test suite on AMD GPUs, including the pulsar subrenderer. Verified on AMD Instinct MI250X (gfx90a, warpSize=64), HIP 7.2, PyTorch 2.13.
## Mechanics
`torch.utils.cpp_extension.BuildExtension` auto-hipifies `.cu` sources of a `CUDAExtension` against a HIP-built torch (`cuda_runtime.h → hip/hip_runtime.h`, `cub:: → hipcub::`, `cudaStream_t → hipStream_t`, etc.), so most of the lift is build-system glue and a small number of CUDA intrinsics that don't have HIP equivalents.
- `setup.py`: detect ROCm via `torch.version.hip is not None`; treat `ROCM_HOME` as the GPU-toolkit-root analogue of `CUDA_HOME` (without this, `CUDA_HOME is None` silently demoted the build to a CPU-only `CppExtension`); skip `CUB_HOME`, CUDA-13 visibility flags, and `-ccbin=` on ROCm.
- `pytorch3d/csrc/pulsar/gpu/commands.h`: CUDA's `_rn`-suffixed FP rounding intrinsics (`__fadd_rn`, `__fdiv_rn`, `__fsqrt_rn`, `__fmaf_rn`, `__frcp_rn`) and `__saturatef` have no HIP equivalents — AMD's GPU ISA has no instruction-level rounding-mode override, so they expand to plain operators / `sqrtf` / `fmaf` / `1.0f/x` / `fmaxf(0,fminf(1,x))` on the `USE_ROCM` arm, which are rounding-mode-equivalent (both round-to-nearest-even). The HIP compiler may fuse `a+b*c` into a single-rounding FMA where CUDA's `_rn` would have prevented it; if FMA-fusion drift ever becomes a numerical issue, add `-ffp-contract=off` to pulsar's HIPCC flags. `__powf` is replaced with `powf`. `atomicAdd_block` has no HIP function-name equivalent — the semantic equivalent is `__hip_atomic_fetch_add(ptr, val, __ATOMIC_RELAXED, __HIP_MEMORY_SCOPE_WORKGROUP)` (plain HIP `atomicAdd` is device-scope, strictly stronger than block-scope and forces L2-coherent atomics).
- `tests/test_point_mesh_distance.py`: loosen `grad_faces` tolerance in `test_point_face_distance` from `5e-7` to `5e-6` to match the sibling `test_face_point_distance`. The backward kernel uses `atomicAdd` and calls `alertNotDeterministic`; FP add order varies by wavefront width.
- The X_t / camera-R/T equality checks in `test_points_alignment.py` and `test_cameras_alignment.py` are now skipped when `n_points <= dim` (resp. `batch_size <= 3` for camera-center alignment in 3D). Mean-centering renders the SVD rank-deficient in those cases, so the rotation around the degenerate axis is non-unique and different BLAS implementations (rocBLAS RDNA vs CDNA, cuBLAS) pick different valid null-space directions. The center-alignment check still runs and verifies the well-defined part of the transformation.
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/2039
Test Plan:
All GPU tests pass on both AMD Instinct MI250X (gfx90a, wave64, HIP 7.2) and AMD Radeon Pro W7800 (gfx1100, wave32, HIP 7.2.53211, torch 2.13.0a0).
| Module | Result |
|---|---|
| knn, ball_query, sample_farthest_points, face_areas_normals | all pass |
| rasterize_points, rasterize_meshes, chamfer, packed_to_padded | all pass |
| interpolate_face_attributes, blending, compositing, sample_pdf, mesh_normal_consistency | all pass |
| point_mesh_distance | 9/9 pass (with tolerance fix in this PR) |
| pulsar/test_forward, test_channels, test_depth, test_hands, test_ortho, test_small_spheres | 10 passed (FB_TEST=1) |
| test_render_points pulsar tests, test_camera_conversions::test_pulsar_conversion | 3 passed |
| points_to_volumes, iou_box3d, marching_cubes | 20 failures, all env-only |
The 20 env-only failures are `torch.inverse()` on CPU tensors in test reference paths; this verification host's PyTorch was built with `USE_LAPACK: 0` (only `mkl-static` `.a` archives in the conda env; PyTorch's `FindBLAS` looks for `libmkl_intel_lp64.so`). Unrelated to the port — re-verifying with a LAPACK-linked PyTorch is left to upstream.
Reviewed By: MichaelRamamonjisoa
Differential Revision: D106825690
Pulled By: bottler
fbshipit-source-id: f7a9b6028e6fb555f3b8c0f9792e88b818327166
Summary:
No one is using these.
(The minify part has been broken for a couple of years, too)
Reviewed By: patricklabatut
Differential Revision: D96977684
fbshipit-source-id: 4708dfd37b14d1930f1370677eb126a61a0d9d3c
Summary:
Added `atol=1e-4` tolerance parameter to the `assertClose` calls on lines 682 and 683 in the `test_inverse` method of `TestTranslate` class.
This is a retry of D90225548
Reviewed By: sgrigory
Differential Revision: D90682979
fbshipit-source-id: ac13f000174dd9962326296e1c3116d0d39c7751
Summary:
## LLM-generated Summary:
Replaces self.assertTrue(torch.allclose(...)) with self.assertClose(...) throughout fbcode/vision/fair/pytorch3d/tests/test_transforms.py. This standardizes numeric closeness assertions for clearer failures and consistency while preserving tolerances and test behavior.
---
Session: DEV34970678
Reviewed By: shapovalov
Differential Revision: D90251428
fbshipit-source-id: cdae842be82f0ba548802e6977be272134e8508c
Summary: Some random seed changes. Skip multigpu tests when there's only one gpu. This is a better fix for what AI is doing in D80600882.
Reviewed By: MichaelRamamonjisoa
Differential Revision: D80625966
fbshipit-source-id: ac3952e7144125fd3a05ad6e4e6e5976ae10a8ef
Summary:
This is an extension of https://github.com/facebookresearch/pytorch3d/issues/1544 with various speed, stability, and readability improvements. (I could not find a way to make a commit to the existing PR). This PR is still based on the [Rodrigues' rotation formula](https://en.wikipedia.org/wiki/Rotation_formalisms_in_three_dimensions#Rotation_matrix_%E2%86%94_Euler_axis/angle).
The motivation is the same; this change speeds up the conversions up to 10x, depending on the device, batch size, etc.
### Notes
- As the angles get very close to `π`, the existing implementation and the proposed one start to differ. However, (my understanding is that) this is not a problem as the axis can not be stably inferred from the rotation matrix in this case in general.
- bottler , I tried to follow similar conventions as existing functions to deal with weird angles, let me know if something needs to be changed to merge this.
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1948
Reviewed By: MichaelRamamonjisoa
Differential Revision: D69193009
Pulled By: bottler
fbshipit-source-id: e5ed34b45b625114ec4419bb89e22a6aefad4eeb
Summary:
This is a somewhat not BC change: some None paths will be replaced by metadata paths, even when they were not used for data loading.
Moreover, removing the legacy fix to the paths in the old CO3D release.
Reviewed By: bottler
Differential Revision: D69048238
fbshipit-source-id: 2a8b26d7b9f5e2adf39c65888b5863a5a9de1996
Summary:
Converts the directory specified to use the Ruff formatter in pyfmt
ruff_dog
If this diff causes merge conflicts when rebasing, please run
`hg status -n -0 --change . -I '**/*.{py,pyi}' | xargs -0 arc pyfmt`
on your diff, and amend any changes before rebasing onto latest.
That should help reduce or eliminate any merge conflicts.
allow-large-files
Reviewed By: bottler
Differential Revision: D66472063
fbshipit-source-id: 35841cb397e4f8e066e2159550d2f56b403b1bef
Summary:
* Adds a "max" option for the point_reduction input to the
chamfer_distance function.
* When combining the x and y directions, maxes the losses instead
of summing them when point_reduction="max".
* Moves batch reduction to happen after the directions are
combined.
* Adds test_chamfer_point_reduction_max and
test_single_directional_chamfer_point_reduction_max tests.
Fixes https://github.com/facebookresearch/pytorch3d/issues/1838
Reviewed By: bottler
Differential Revision: D60614661
fbshipit-source-id: 7879816acfda03e945bada951b931d2c522756eb
Summary:
Currently, it is not possible to access a sub-transform using an indexer for all 3d transforms inheriting the `Transforms3d` class.
For instance:
```python
from pytorch3d import transforms
N = 10
r = transforms.random_rotations(N)
T = transforms.Transform3d().rotate(R=r)
R = transforms.Rotate(r)
x = T[0] # ok
x = R[0] # TypeError: __init__() got an unexpected keyword argument 'matrix'
```
This is because all these classes (namely `Rotate`, `Translate`, `Scale`, `RotateAxisAngle`) inherit the `__getitem__()` method from `Transform3d` which has the [following code on line 201](https://github.com/facebookresearch/pytorch3d/blob/main/pytorch3d/transforms/transform3d.py#L201):
```python
return self.__class__(matrix=self.get_matrix()[index])
```
The four classes inheriting `Transform3d` are not initialized through a matrix argument, hence they error.
I propose to modify the `__getitem__()` method of the `Transform3d` class to fix this behavior. The least invasive way to do it I can think of consists of creating an empty instance of the current class, then setting the `_matrix` attribute manually. Thus, instead of
```python
return self.__class__(matrix=self.get_matrix()[index])
```
I propose to do:
```python
instance = self.__class__.__new__(self.__class__)
instance._matrix = self.get_matrix()[index]
return instance
```
As far as I can tell, this modification occurs no modification whatsoever for the user, except for the ability to index all 3d transforms.
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1801
Reviewed By: MichaelRamamonjisoa
Differential Revision: D58410389
Pulled By: bottler
fbshipit-source-id: f371e4c63d2ae4c927a7ad48c2de8862761078de
Summary: The diff support colors in cubify for align = "center"
Reviewed By: bottler
Differential Revision: D53777011
fbshipit-source-id: ccb2bd1e3d89be3d1ac943eff08f40e50b0540d9
Summary: Add an option to run tests without the OpenGL Renderer.
Reviewed By: patricklabatut
Differential Revision: D53573400
fbshipit-source-id: 54a14e7b2f156d24e0c561fdb279f4a9af01b793
Summary:
Fixes https://github.com/facebookresearch/pytorch3d/issues/1641. The bug was caused by the mistaken downcasting of an int64_t into int, causing issues only on inputs large enough to have hashes that escaped the bounds of an int32.
Also added a test case for this issue.
Reviewed By: bottler
Differential Revision: D53505370
fbshipit-source-id: 0fdd0efc6d259cc3b0263e7ff3a4ab2c648ec521
Summary: Fixed `get_rgbd_point_cloud` to take any number of image input channels.
Reviewed By: bottler
Differential Revision: D52796276
fbshipit-source-id: 3ddc0d1e337a6cc53fc86c40a6ddb136f036f9bc
Summary: Implement `submeshes` for TexturesUV. Fix what Meshes.submeshes passes to the texture's submeshes function to make this possible.
Reviewed By: bottler
Differential Revision: D52192060
fbshipit-source-id: 526734962e3376aaf75654200164cdcebfff6997
Summary: Performance improvement: Use torch.lerp to map uv coordinates to the range needed for grid_sample (i.e. map [0, 1] to [-1, 1] and invert the y-axis)
Reviewed By: bottler
Differential Revision: D51961728
fbshipit-source-id: db19a5e3f482e9af7b96b20f88a1e5d0076dac43
Summary: Fixes lint in test_render_points in the PyTorch3D library.
Differential Revision: D51289841
fbshipit-source-id: 1eae621eb8e87b0fe5979f35acd878944f574a6a
Summary:
When the ply format looks as follows:
```
comment TextureFile ***.png
element vertex 892
property double x
property double y
property double z
property double nx
property double ny
property double nz
property double texture_u
property double texture_v
```
`MeshPlyFormat` class will read uv from the ply file and read the uv map as commented as TextureFile.
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1100
Reviewed By: MichaelRamamonjisoa
Differential Revision: D50885176
Pulled By: bottler
fbshipit-source-id: be75b1ec9a17a1ed87dbcf846a9072ea967aec37
Summary:
The `chamfer_distance` function currently allows `"sum"` or `"mean"` reduction, but does not support returning unreduced (per-point) loss terms. Unreduced losses could be useful if the user wishes to inspect individual losses, or perform additional modifications to loss terms before reduction. One example would be implementing a robust kernel over the loss.
This PR adds a `None` option to the `point_reduction` parameter, similar to `batch_reduction`. In case of bi-directional chamfer loss, both the forward and backward distances are returned (a tuple of Tensors of shape `[D, N]` is returned). If normals are provided, similar logic applies to normals as well.
This PR addresses issue https://github.com/facebookresearch/pytorch3d/issues/622.
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1605
Reviewed By: jcjohnson
Differential Revision: D48313857
Pulled By: bottler
fbshipit-source-id: 35c824827a143649b04166c4817449e1341b7fd9
Summary:
Convert ImplicitronRayBundle to a "classic" class instead of a dataclass. This change is introduced as a way to preserve the ImplicitronRayBundle interface while allowing two outcomes:
- init lengths arguments is now a Optional[torch.Tensor] instead of torch.Tensor
- lengths is now a property which returns a `torch.Tensor`. The lengths property will either recompute lengths from bins or return the stored _lengths. `_lenghts` is None if bins is set. It saves us a bit of memory.
Reviewed By: shapovalov
Differential Revision: D46686094
fbshipit-source-id: 3c75c0947216476ebff542b6f552d311024a679b
Summary:
## Context
Bins are used in mipnerf to allow to manipulate easily intervals. For example, by doing the following, `bins[..., :-1]` you will obtain all the left coordinates of your intervals, while doing `bins[..., 1:]` is equals to the right coordinates of your intervals.
We introduce here the support of bins like in MipNerf implementation.
## RayPointRefiner
Small changes have been made to modify RayPointRefiner.
- If bins is None
```
mids = torch.lerp(ray_bundle.lengths[..., 1:], ray_bundle.lengths[…, :-1], 0.5)
z_samples = sample_pdf(
mids, # [..., npt]
weights[..., 1:-1], # [..., npt - 1]
….
)
```
- If bins is not None
In the MipNerf implementation the sampling is done on all the bins. It allows us to use the full weights tensor without slashing it.
```
z_samples = sample_pdf(
ray_bundle.bins, # [..., npt + 1]
weights, # [..., npt]
...
)
```
## RayMarcher
Add a ray_deltas optional argument. If None, keep the same deltas computation from ray_lengths.
Reviewed By: shapovalov
Differential Revision: D46389092
fbshipit-source-id: d4f1963310065bd31c1c7fac1adfe11cbeaba606
Summary:
Add blurpool has defined in [MIP-NeRF](https://arxiv.org/abs/2103.13415).
It has been added has an option for RayPointRefiner.
Reviewed By: shapovalov
Differential Revision: D46356189
fbshipit-source-id: ad841bad86d2b591a68e1cb885d4f781cf26c111
Summary: Add a new implicit module Integral Position Encoding based on [MIP-NeRF](https://arxiv.org/abs/2103.13415).
Reviewed By: shapovalov
Differential Revision: D46352730
fbshipit-source-id: c6a56134c975d80052b3a11f5e92fd7d95cbff1e
Summary:
Introduce methods to approximate the radii of conical frustums along rays as described in [MipNerf](https://arxiv.org/abs/2103.13415):
- Two new attributes are added to ImplicitronRayBundle: bins and radii. Bins is of size n_pts_per_ray + 1. It allows us to manipulate easily and n_pts_per_ray intervals. For example we need the intervals coordinates in the radii computation for \(t_{\mu}, t_{\delta}\). Radii are used to store the radii of the conical frustums.
- Add 3 new methods to compute the radii:
- approximate_conical_frustum_as_gaussians: It computes the mean along the ray direction, the variance of the
conical frustum with respect to t and variance of the conical frustum with respect to its radius. This
implementation follows the stable computation defined in the paper.
- compute_3d_diagonal_covariance_gaussian: Will leverage the two previously computed variances to find the
diagonal covariance of the Gaussian.
- conical_frustum_to_gaussian: Mix everything together to compute the means and the diagonal covariances along
the ray of the Gaussians.
- In AbstractMaskRaySampler, introduces the attribute `cast_ray_bundle_as_cone`. If False it won't change the previous behaviour of the RaySampler. However if True, the samplers will sample `n_pts_per_ray +1` instead of `n_pts_per_ray`. This points are then used to set the bins attribute of ImplicitronRayBundle. The support of HeterogeneousRayBundle has not been added since the current code does not allow it. A safeguard has been added to avoid a silent bug in the future.
Reviewed By: shapovalov
Differential Revision: D45269190
fbshipit-source-id: bf22fad12d71d55392f054e3f680013aa0d59b78
Summary: Make test work in isolation, and when run internally make it not try the sqlalchemy files.
Reviewed By: shapovalov
Differential Revision: D46352513
fbshipit-source-id: 7417a25d7a5347d937631c9f56ae4e3242dd622e
Summary: Making it easier for the clients to use these datasets.
Reviewed By: bottler
Differential Revision: D46727179
fbshipit-source-id: cf619aee4c4c0222a74b30ea590cf37f08f014cc
Summary:
Adds stratified sampling of sequences within categories applied after category / sequence filters but before the num sequence limit.
It respects the insertion order into the sequence_annots table, i.e. takes top N sequences within each category.
Reviewed By: bottler
Differential Revision: D46724002
fbshipit-source-id: 597cb2a795c3f3bc07f838fc51b4e95a4f981ad3
Summary: Single directional chamfer distance and option to use non-absolute cosine similarity
Reviewed By: bottler
Differential Revision: D46593980
fbshipit-source-id: b2e591706a0cdde1c2d361614cecebb84a581433
Summary: Fine implicit function was called before the coarse implicit function.
Reviewed By: shapovalov
Differential Revision: D46224224
fbshipit-source-id: 6b1cc00cc823d3ea7a5b42774c9ec3b73a69edb5
Summary:
1. We may need to store arrays of unknown shape in the database. It implements and tests serialisation.
2. Previously, when an inexisting metadata file was passed to SqlIndexDataset, it would try to open it and create an empty file, then crash. We now open the file in a read-only mode, so the error message is more intuitive. Note that the implementation is SQLite specific.
Reviewed By: bottler
Differential Revision: D46047857
fbshipit-source-id: 3064ae4f8122b4fc24ad3d6ab696572ebe8d0c26
Summary: I don't know why RE tests sometimes fail here, but maybe it's a race condition. If that's right, this should fix it.
Reviewed By: shapovalov
Differential Revision: D46020054
fbshipit-source-id: 20b746b09ad9bd77c2601ac681047ccc6cc27ed9
