729 Commits

Author SHA1 Message Date
generatedunixname1417043136753450
7b5c78460a fbcode/vision/fair/pytorch3d/pytorch3d/transforms/se3.py
Reviewed By: sgrigory

Differential Revision: D93709801

fbshipit-source-id: e4bae81fe1a88fed547304e6e21b248c5a345277
2026-02-23 14:51:32 -08:00
generatedunixname1417043136753450
e3c80a4368 fbcode/vision/fair/pytorch3d/pytorch3d/renderer/splatter_blend.py
Reviewed By: sgrigory

Differential Revision: D93710022

fbshipit-source-id: 39253258b93a467fbda6b51ef8d6d3975bb49810
2026-02-23 12:43:53 -08:00
generatedunixname1417043136753450
b9b5ea3428 fbcode/vision/fair/pytorch3d/pytorch3d/common/workaround/symeig3x3.py
Reviewed By: sgrigory

Differential Revision: D93715209

fbshipit-source-id: 1880a8dd72e35ce5cc93cdeecf770aab6469ca31
2026-02-23 12:42:24 -08:00
generatedunixname1417043136753450
0e435c297c fbcode/vision/fair/pytorch3d/pytorch3d/ops/points_alignment.py
Reviewed By: sgrigory

Differential Revision: D93712744

fbshipit-source-id: 660560cdef9ff1d2173ae06de54df31766ee537f
2026-02-23 12:28:37 -08:00
generatedunixname1417043136753450
d631b56fba fbcode/vision/fair/pytorch3d/pytorch3d/ops/sample_farthest_points.py
Reviewed By: sgrigory

Differential Revision: D93708653

fbshipit-source-id: 112158092cd64ac8afddf1378b931cb44e19c372
2026-02-23 10:21:52 -08:00
generatedunixname915440834509264
3ba2030aa4 Fix CQS signal readability-braces-around-statements in fbcode/vision/fair
Reviewed By: bottler

Differential Revision: D94068738

fbshipit-source-id: cd47c67d4269ac7461acb73da6de9e4373da9d4c
2026-02-23 05:18:38 -08:00
generatedunixname1262449429094718
79a7fcf02b fbcode/vision/fair/pytorch3d/pytorch3d/csrc/rasterize_meshes/rasterize_meshes_cpu.cpp
Reviewed By: bottler

Differential Revision: D94062914

fbshipit-source-id: 9147dc68d115ce5761ebb7d07c035ac4b664da0b
2026-02-23 05:10:19 -08:00
generatedunixname1417043136753450
e43ed8c76e fbcode/vision/fair/pytorch3d/pytorch3d/transforms/rotation_conversions.py
Reviewed By: bottler

Differential Revision: D93712828

fbshipit-source-id: 3465af450104bb1e5f491e3c0ee0259698cf8ceb
2026-02-22 07:53:20 -08:00
generatedunixname1417043136753450
49f43402c6 fbcode/vision/fair/pytorch3d/pytorch3d/renderer/mesh/textures.py
Reviewed By: bottler

Differential Revision: D93710616

fbshipit-source-id: 599fe7425066bc85c0999765168788f8df7e34ce
2026-02-22 07:13:45 -08:00
generatedunixname1417043136753450
90646d93ab fbcode/vision/fair/pytorch3d/pytorch3d/renderer/mesh/clip.py
Reviewed By: bottler

Differential Revision: D93715239

fbshipit-source-id: 7417015251fe96be72daf4894e946edd43bb9c46
2026-02-22 07:13:09 -08:00
generatedunixname1417043136753450
eabb511410 fbcode/vision/fair/pytorch3d/pytorch3d/loss/mesh_laplacian_smoothing.py
Reviewed By: bottler

Differential Revision: D93709347

fbshipit-source-id: 69710e6082a0785126a121e26f1d96a571360f1d
2026-02-22 07:08:02 -08:00
generatedunixname1417043136753450
e70188ebbc fbcode/vision/fair/pytorch3d/pytorch3d/transforms/transform3d.py
Reviewed By: bottler

Differential Revision: D93713606

fbshipit-source-id: a8aa52328a76d95d3985daec529cdce04ba12bd4
2026-02-22 07:06:34 -08:00
generatedunixname1417043136753450
1bd911d534 fbcode/vision/fair/pytorch3d/pytorch3d/renderer/cameras.py
Reviewed By: bottler

Differential Revision: D93712137

fbshipit-source-id: 3457f0f9fb7d7baa29be2eaf731074a49bdbb0c8
2026-02-22 07:05:45 -08:00
generatedunixname1417043136753450
3aadd19a2b fbcode/vision/fair/pytorch3d/pytorch3d/ops/laplacian_matrices.py
Reviewed By: bottler

Differential Revision: D93708383

fbshipit-source-id: 7576f0c9800ed3d28795e521be5c63799b7e6676
2026-02-22 06:57:57 -08:00
generatedunixname1417043136753450
42d66c1145 fbcode/vision/fair/pytorch3d/pytorch3d/loss/point_mesh_distance.py
Reviewed By: bottler

Differential Revision: D93708351

fbshipit-source-id: 06a877777e4cb72a497a44ff55db0b6222bda83b
2026-02-22 06:55:36 -08:00
generatedunixname1417043136753450
e9ed1cb178 fbcode/vision/fair/pytorch3d/pytorch3d/renderer/utils.py
Reviewed By: bottler

Differential Revision: D93708316

fbshipit-source-id: f8ae2432ad34116278b3f7f7de5146b89c3fe63e
2026-02-22 04:09:20 -08:00
Jeremy Reizenstein
cbcae096a0 Add atol=1e-4 to assertClose calls in test_inverse for Translate
Summary:
Added `atol=1e-4` tolerance parameter to the `assertClose` calls on lines 682 and 683 in the `test_inverse` method of `TestTranslate` class.

This is a retry of D90225548

Reviewed By: sgrigory

Differential Revision: D90682979

fbshipit-source-id: ac13f000174dd9962326296e1c3116d0d39c7751
2026-01-14 08:57:43 -08:00
generatedunixname537391475639613
5b1cce56bc Fix for T251460511 ("Your diff, D90498281, broke one test")
Reviewed By: sgrigory

Differential Revision: D90649493

fbshipit-source-id: 2a77c45ec8e6e5aa0a20437a765fbb9f0b566406
2026-01-14 08:53:26 -08:00
Bowie Chen
0c3b204375 apply Black 25.11.0 style in fbcode (70/92)
Summary:
Formats the covered files with pyfmt.

paintitblack

Reviewed By: itamaro

Differential Revision: D90476295

fbshipit-source-id: 5101d4aae980a9f8955a4cb10bae23997c48837f
2026-01-12 02:54:36 -08:00
Jeremy Reizenstein
6be5e2da06 Replace assertTrue(torch.allclose(...)) with assertClose in test_transforms.py
Summary:
## LLM-generated Summary:
Replaces self.assertTrue(torch.allclose(...)) with self.assertClose(...) throughout fbcode/vision/fair/pytorch3d/tests/test_transforms.py. This standardizes numeric closeness assertions for clearer failures and consistency while preserving tolerances and test behavior.
 ---
Session: DEV34970678

Reviewed By: shapovalov

Differential Revision: D90251428

fbshipit-source-id: cdae842be82f0ba548802e6977be272134e8508c
2026-01-08 04:35:40 -08:00
Guilherme Albertini
f5f6b78e70 Add initial CUDA 13.0 support for pulsar and pycuda modules
Summary:
CUDA 13.0 introduced breaking changes that cause build failures in pytorch3d:

**1. Symbol Visibility Changes (pulsar)**
- NVCC now forces `__global__` functions to have hidden ELF visibility by default
- `__global__` function template stubs now have internal linkage

**Fix:** Added NVCC flags (`--device-entity-has-hidden-visibility=false` and `-static-global-template-stub=false`) for fbcode builds with CUDA 13.0+.

**2. cuCtxCreate API Change (pycuda)**
- CUDA 13.0 changed `cuCtxCreate` from 3 to 4 arguments
- pycuda 2022.2 (current default) uses the old signature and fails to compile
- pycuda 2025.1.2 (D83501913) includes the CUDA 13.0 fix

**Fix:** Added CUDA 13.0 constraint to pycuda alias to auto-select pycuda 2025.1.2.

**NCCL Compatibility Note:**
- Current stable NCCL (2.25) is NOT compatible with CUDA 13.0 (`cudaTypedefs.h` removed)
- NCCL 2.27+ works with CUDA 13.0 and will become stable in early January 2026 (per HPC Comms team)
- Until then, CUDA 13.0 builds require `-c hpc_comms.use_nccl=2.27`

References:
- GitHub issue: https://github.com/facebookresearch/pytorch3d/issues/2011
- NVIDIA blog: https://developer.nvidia.com/blog/cuda-c-compiler-updates-impacting-elf-visibility-and-linkage/
- FBGEMM_GPU fix: D86474263
- pycuda 2025.1.2 buckification: D83501913

Reviewed By: bottler

Differential Revision: D88816596

fbshipit-source-id: 1ba666dab8c0e06d1286b8d5bc5d84cfc55c86e6
2025-12-17 10:02:10 -08:00
Jeremy Reizenstein
33824be3cb version 0.7.9
Reviewed By: shapovalov

Differential Revision: D87984194

fbshipit-source-id: dee8123a2c3f5cc34ada52f4663c9bbb329e03a7
2025-11-27 09:52:08 -08:00
Eugene Park
2d4d345b6f Improve ball_query() runtime for large-scale cases (#2006)
Summary:
### Overview
The current C++ code for `pytorch3d.ops.ball_query()` performs floating point multiplication for every coordinate of every pair of points (up until the maximum number of neighbor points is reached). This PR modifies the code (for both CPU and CUDA versions) to implement idea presented [here](https://stackoverflow.com/a/3939525): a `D`-cube around the `D`-ball is first constructed, and any point pairs falling outside the cube are skipped, without explicitly computing the squared distances. This change is especially useful for when the dimension `D` and the number of points `P2` are large and the radius is much smaller than the overall volume of space occupied by the point clouds; as much as **~2.5x speedup** (CPU case; ~1.8x speedup in CUDA case) is observed when `D = 10` and `radius = 0.01`. In all benchmark cases, points were uniform randomly distributed inside a unit `D`-cube.

The benchmark code used was different from `tests/benchmarks/bm_ball_query.py` (only the forward part is benchmarked, larger input sizes were used) and is stored in `tests/benchmarks/bm_ball_query_large.py`.

### Average time comparisons

<img width="360" height="270" alt="cpu-03-0 01-avg" src="https://github.com/user-attachments/assets/6cc79893-7921-44af-9366-1766c3caf142" />
<img width="360" height="270" alt="cuda-03-0 01-avg" src="https://github.com/user-attachments/assets/5151647d-0273-40a3-aac6-8b9399ede18a" />
<img width="360" height="270" alt="cpu-03-0 10-avg" src="https://github.com/user-attachments/assets/a87bc150-a5eb-47cd-a4ba-83c2ec81edaf" />
<img width="360" height="270" alt="cuda-03-0 10-avg" src="https://github.com/user-attachments/assets/e3699a9f-dfd3-4dd3-b3c9-619296186d43" />
<img width="360" height="270" alt="cpu-10-0 01-avg" src="https://github.com/user-attachments/assets/5ec8c32d-8e4d-4ced-a94e-1b816b1cb0f8" />
<img width="360" height="270" alt="cuda-10-0 01-avg" src="https://github.com/user-attachments/assets/168a3dfc-777a-4fb3-8023-1ac8c13985b8" />
<img width="360" height="270" alt="cpu-10-0 10-avg" src="https://github.com/user-attachments/assets/43a57fd6-1e01-4c5e-87a9-8ef604ef5fa0" />
<img width="360" height="270" alt="cuda-10-0 10-avg" src="https://github.com/user-attachments/assets/a7c7cc69-f273-493e-95b8-3ba2bb2e32da" />

### Peak time comparisons

<img width="360" height="270" alt="cpu-03-0 01-peak" src="https://github.com/user-attachments/assets/5bbbea3f-ef9b-490d-ab0d-ce551711d74f" />
<img width="360" height="270" alt="cuda-03-0 01-peak" src="https://github.com/user-attachments/assets/30b5ab9b-45cb-4057-b69f-bda6e76bd1dc" />
<img width="360" height="270" alt="cpu-03-0 10-peak" src="https://github.com/user-attachments/assets/db69c333-e5ac-4305-8a86-a26a8a9fe80d" />
<img width="360" height="270" alt="cuda-03-0 10-peak" src="https://github.com/user-attachments/assets/82549656-1f12-409e-8160-dd4c4c9d14f7" />
<img width="360" height="270" alt="cpu-10-0 01-peak" src="https://github.com/user-attachments/assets/d0be8ef1-535e-47bc-b773-b87fad625bf0" />
<img width="360" height="270" alt="cuda-10-0 01-peak" src="https://github.com/user-attachments/assets/e308e66e-ae30-400f-8ad2-015517f6e1af" />
<img width="360" height="270" alt="cpu-10-0 10-peak" src="https://github.com/user-attachments/assets/c9b5bf59-9cc2-465c-ad5d-d4e23bdd138a" />
<img width="360" height="270" alt="cuda-10-0 10-peak" src="https://github.com/user-attachments/assets/311354d4-b488-400c-a1dc-c85a21917aa9" />

### Full benchmark logs

[benchmark-before-change.txt](https://github.com/user-attachments/files/22978300/benchmark-before-change.txt)
[benchmark-after-change.txt](https://github.com/user-attachments/files/22978299/benchmark-after-change.txt)

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/2006

Reviewed By: shapovalov

Differential Revision: D85356394

Pulled By: bottler

fbshipit-source-id: 9b3ce5fc87bb73d4323cc5b4190fc38ae42f41b2
2025-10-30 05:01:32 -07:00
Nikita Lutsenko
45df20e9e2 clang-format | Format fbsource with clang-format 21.
Reviewed By: ChristianK275

Differential Revision: D85317706

fbshipit-source-id: b399c5c4b75252999442b7d7d2778e7a241b0025
2025-10-26 23:40:59 -07:00
Jeremy Reizenstein
fc6a6b8951 separate multigpu tests
Reviewed By: MichaelRamamonjisoa

Differential Revision: D83477594

fbshipit-source-id: 5ea67543e288e9a06ee5141f436e879aa5cfb7f3
2025-10-09 08:17:20 -07:00
Kihyuk Sohn
7711bf34a8 fix device error
Summary: When using `sample_farthest_points` with `lengths`, it throws an error because of the device mismatch between `lengths` and `torch.rand(lengths.size())` on GPU.

Reviewed By: bottler

Differential Revision: D82378997

fbshipit-source-id: 8e929256177d543d1dd1249e8488f70e03e4101f
2025-09-15 06:41:00 -07:00
Jeremy Reizenstein
d098beb7a7 allow python 3.12
Summary: Remove use of distutils

Reviewed By: MichaelRamamonjisoa

Differential Revision: D81594552

fbshipit-source-id: 4e979d5e03ea873bd09bc2b674b7e6480b9c6d65
2025-09-04 08:31:32 -07:00
Jeremy Reizenstein
dd068703d1 test fixes
Summary: Some random seed changes. Skip multigpu tests when there's only one gpu. This is a better fix for what AI is doing in D80600882.

Reviewed By: MichaelRamamonjisoa

Differential Revision: D80625966

fbshipit-source-id: ac3952e7144125fd3a05ad6e4e6e5976ae10a8ef
2025-08-27 06:55:50 -07:00
Antoine Dumoulin
50f8efa1cb Use sparse_coo_tensor in laplacian_matrices.py (#1991)
Summary:
update obsolete torch.sparse.FloatTensor to torch.sparse_coo_tensor

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1991

Reviewed By: MichaelRamamonjisoa

Differential Revision: D80084359

Pulled By: bottler

fbshipit-source-id: dc6c7a90211113d1ce5338a92c8c0030bfe12e65
2025-08-13 07:55:57 -07:00
Olga Gerasimova
5043d15361 avoid CPU/GPU sync in sample_farthest_points
Summary:
Optimizing sample_farthest_poinst by reducing CPU/GPU sync:
1. replacing iterative randint for starting indexes for 1 function call, if length is constant
2. Avoid sync in fetching maxumum of sample points, if we sample the same amount
3. Initializing 1 tensor for samples and indixes

compare
https://fburl.com/mlhub/7wk0xi98
Before
{F1980383703}
after
{F1980383707}

Histogram match pretty closely
{F1980464338}

Reviewed By: bottler

Differential Revision: D78731869

fbshipit-source-id: 060528ae7a1e0fbbd005d129c151eaf9405841de
2025-07-23 10:23:40 -07:00
Stone Tao
e3d3a67a89 Clamp matrices in matrix_to_euler_angles function (#1989)
Summary:
Closes https://github.com/facebookresearch/pytorch3d/issues/1988

Credit goes to tylerlum for raising this issue and suggesting this fix in https://github.com/haosulab/ManiSkill/pull/1090

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1989

Reviewed By: MichaelRamamonjisoa

Differential Revision: D78021983

Pulled By: bottler

fbshipit-source-id: d723f1924a399f4d7fd072e96ea740ae73cf280f
2025-07-10 06:08:19 -07:00
Jeremy Reizenstein
e55ea90609 disable import tests
Summary: these tests don't work, aren't needed right now

Reviewed By: MichaelRamamonjisoa

Differential Revision: D78084742

fbshipit-source-id: 9cff2b30427dec314e34e81179816af4073bbe23
2025-07-10 05:20:22 -07:00
Melvin He
3aee2a6005 Fixes bus error hard crashes on Apple Silicon MPS devices
Summary:
Fixes hard crashes (bus errors) when using MPS device (Apple Silicon) by implementing CPU checks throughout files in csrc subdirectories to check if on same mesh on a CPU device.

Note that this is the fourth and ultimate part of a larger change through multiple files & directories.

Reviewed By: bottler

Differential Revision: D77698176

fbshipit-source-id: 5bc9e3c5cea61afd486aed7396f390d92775ec6d
2025-07-03 12:34:37 -07:00
Melvin He
c5ea8fa49e Adds CHECK_CPU macros checks for tensors not on CPU
Summary:
Adds CHECK_CPU macros that checks if a tensor is on the CPU device throughout csrc directories and subdir up to `pulsar`.

Note that this is the third part of a larger change, and to keep diffs better organized, subsequent diffs will update the remaining directories.

Reviewed By: bottler

Differential Revision: D77696998

fbshipit-source-id: 470ca65b23d9965483b5bdd30c712da8e1131787
2025-07-03 08:29:36 -07:00
Melvin He
3ff6c5ab85 Error instead of crash for tensors on exotic devices
Summary:
Adds CHECK_CPU macros that checks if a tensor is on the CPU device throughout csrc directories up to `marching_cubes`. Directories updated include those in `gather_scatter`, `interp_face_attrs`, `iou_box3d`, `knn`, and `marching_cubes`.

Note that this is the second part of a larger change, and to keep diffs better organized, subsequent diffs will update the remaining directories.

Reviewed By: bottler

Differential Revision: D77558550

fbshipit-source-id: 762a0fe88548dc8d0901b198a11c40d0c36e173f
2025-07-01 09:14:38 -07:00
Srivathsan Govindarajan
267bd8ef87 Revert _sqrt_positive_part change
Reviewed By: bottler

Differential Revision: D77549647

fbshipit-source-id: a0ef0bc015c643ad7416c781886e2e23b5105bdd
2025-06-30 14:13:27 -07:00
Melvin He
177eec6378 Error instead of crash for tensors on exotic devices (#1986)
Summary:
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1986

Adds device checks to prevent crashes on unsupported devices in PyTorch3D. Updates the `pytorch3d_cutils.h` file to include new macro CHECK_CPU that checks if a tensor is on the CPU device. This macro is then used in the directories from `ball_query` to `face_area_normals` to ensure that tensors are not on unsupported devices like MPS.

Note that this is the first part of a larger change, and to keep diffs better organized, subsequent diffs will update the remaining directories.

Reviewed By: bottler

Differential Revision: D77473296

fbshipit-source-id: 13dc84620dee667bddebad1dade2d2cb5a59c737
2025-06-30 12:27:38 -07:00
Srivathsan Govindarajan
71db7a0ea2 Removing dynamic shape ops and boolean indexing in matrix_to_quaternion
Summary:
The current implementation of `matrix_to_quaternion` and `_sqrt_positive_part` uses boolean indexing, which can slow down performance and cause incompatibility with `torch.compile` unless `torch._dynamo.config.capture_dynamic_output_shape_ops` is set to `True`.

To enhance performance and compatibility, I recommend using  `torch.gather` to select the best-conditioned quaternions and `F.relu` instead of `x>0` (bottler's suggestion)

For a detailed comparison of the implementation differences when using `torch.compile`, please refer to my Bento notebook
N7438339.

Reviewed By: bottler

Differential Revision: D77176230

fbshipit-source-id: 9a6a2e0015b5865056297d5f45badc3c425b93ce
2025-06-25 01:18:46 -07:00
Grace Cheng
6020323d94 Fix Self-Assignment in CUDA Stream Parameter in renderer.forward.device.h
Summary: Resolved self-assignment warnings in the `renderer.forward.device.h` file by removing redundant assignments of the `stream` variable to itself in `cub::DeviceSelect::Flagged` function calls. This change eliminates compiler errors and ensures cleaner, more efficient code execution.

Reviewed By: bottler

Differential Revision: D76554140

fbshipit-source-id: 28eae0186246f51a8ac8002644f184349aa49560
2025-06-13 11:00:16 -07:00
Emmanuel Ferdman
182e845c19 Resolve logger warnings (#1981)
Summary:
# PR Summary
This small PR resolves the annoying deprecation warnings of the `logger` library:
```python
DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
```

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1981

Reviewed By: MichaelRamamonjisoa

Differential Revision: D75287169

Pulled By: bottler

fbshipit-source-id: 9ff9f5dd648aca8d8bb5d33577909da711d18647
2025-06-10 02:27:54 -07:00
generatedunixname89002005287564
f315ac131b Fix CQS signal facebook-unused-include-check in fbcode/vision/fair/pytorch3d/pytorch3d/csrc
Reviewed By: dtolnay

Differential Revision: D75938951

fbshipit-source-id: 8e4f9ce82ec988a30e4c8d54881b78560ceab0e0
2025-06-04 13:09:58 -07:00
Nick Riasanovsky
fc08621879 Fix distutils failure in Triton Beta testing
Summary: Fixes the distutils issues similar to D73934713

Reviewed By: bottler

Differential Revision: D75631611

fbshipit-source-id: 09c354d8cc51ff2c46f4688d7f674370e3f48f1e
2025-05-29 18:18:49 -07:00
generatedunixname89002005287564
3f327a516b Fix CQS signal facebook-unused-include-check in fbcode/vision/fair/pytorch3d/pytorch3d/csrc/pulsar
Reviewed By: dtolnay

Differential Revision: D75209078

fbshipit-source-id: 6b67d3354091d18b8171a6f4b38465ffcc9e17c5
2025-05-26 19:14:57 -07:00
Ting Xu
366eff21d9 Fix PyTorch3D build failure on windows
Summary: Replace #defines by typedefs by following the instructions at https://github.com/facebookresearch/pytorch3d/issues/1970?fbclid=IwY2xjawKZqMJleHRuA2FlbQIxMQBicmlkETFyWFczV2hMVmdOczJWellIAR7jxI6zGQiC5ag-FUXjSK12ljn7rmbMKc3HsLX-BC1TMpOUTJy-bsZxmfKzmw_aem_MIG_nc3eg7LL1o2fSAbl0A#issuecomment-2894339456

Reviewed By: bottler

Differential Revision: D75083182

fbshipit-source-id: 7131fe555bb0da615b341e77ddd8761ebce9d7eb
2025-05-21 07:46:49 -07:00
Jeff Daily
0a59450f0e remove IntWrapper (#1964)
Summary:
I could not access https://github.com/NVlabs/cub/issues/172 to understand whether IntWrapper was still necessary but the comment is from 5 years ago and causes problems for the ROCm build.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1964

Reviewed By: MichaelRamamonjisoa

Differential Revision: D71937895

Pulled By: bottler

fbshipit-source-id: 5e0351e1bd8599b670436cd3464796eca33156f6
2025-03-28 08:16:54 -07:00
Richard Barnes
3987612062 Fix CUDA kernel index data type in vision/fair/pytorch3d/pytorch3d/csrc/compositing/alpha_composite.cu +10
Summary:
CUDA kernel variables matching the type `(thread|block|grid).(Idx|Dim).(x|y|z)` [have the data type `uint`](https://docs.nvidia.com/cuda/cuda-c-programming-guide/#built-in-variables).

Many programmers mistakenly use implicit casts to turn these data types into `int`. In fact, the [CUDA Programming Guide](https://docs.nvidia.com/cuda/cuda-c-programming-guide/) it self is inconsistent and incorrect in its use of data types in programming examples.

The result of these implicit casts is that our kernels may give unexpected results when exposed to large datasets, i.e., those exceeding >~2B items.

While we now have linters in place to prevent simple mistakes (D71236150), our codebase has many problematic instances. This diff fixes some of them.

Reviewed By: dtolnay

Differential Revision: D71355356

fbshipit-source-id: cea44891416d9efd2f466d6c45df4e36008fa036
2025-03-19 13:21:43 -07:00
Alexandros Benetatos
06a76ef8dd Correct "fast" matrix_to_axis_angle near pi (#1953)
Summary:
A continuation of https://github.com/facebookresearch/pytorch3d/issues/1948 -- this commit fixes a small numerical issue with `matrix_to_axis_angle(..., fast=True)` near `pi`.
bottler feel free to check this out, it's a single-line change.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1953

Reviewed By: MichaelRamamonjisoa

Differential Revision: D70088251

Pulled By: bottler

fbshipit-source-id: 54cc7f946283db700cec2cd5575cf918456b7f32
2025-03-11 12:25:59 -07:00
Richard Barnes
21205730d9 Fix unused-variable issues, mostly relating to AMD/HIP
Reviewed By: meyering

Differential Revision: D70845538

fbshipit-source-id: 8e52b5e1f1d96b86404fc3b8cbc6fb952e2cb1a6
2025-03-08 13:03:17 -08:00
Richard Barnes
7e09505538 Enable -Wunused-value in vision/PACKAGE +1
Summary:
This diff enables compilation warning flags for the directory in question. Further details are in [this workplace post](https://fb.workplace.com/permalink.php?story_fbid=pfbid02XaWNiCVk69r1ghfvDVpujB8Hr9Y61uDvNakxiZFa2jwiPHscVdEQwCBHrmWZSyMRl&id=100051201402394).

This is a low-risk diff. There are **no run-time effects** and the diff has already been observed to compile locally. **If the code compiles, it work; test errors are spurious.**

Differential Revision: D70282347

fbshipit-source-id: e2fa55c002d7124b13450c812165d244b8a53f4e
2025-03-04 17:49:30 -08:00
Nicholas Ormrod
20bd8b33f6 facebook-unused-include-check in fbcode/vision
Summary:
Remove headers flagged by facebook-unused-include-check over fbcode.vision.

+ format and autodeps

This is a codemod. It was automatically generated and will be landed once it is approved and tests are passing in sandcastle.
You have been added as a reviewer by Sentinel or Butterfly.

Autodiff project: uiv
Autodiff partition: fbcode.vision
Autodiff bookmark: ad.uiv.fbcode.vision

Reviewed By: dtolnay

Differential Revision: D70403619

fbshipit-source-id: d109c15774eeb3d809875f75fa2a26ed20d7f9a6
2025-02-28 18:08:12 -08:00
alex-bene
7a3c0cbc9d Increase performance for conversions including axis angles (#1948)
Summary:
This is an extension of https://github.com/facebookresearch/pytorch3d/issues/1544 with various speed, stability, and readability improvements. (I could not find a way to make a commit to the existing PR). This PR is still based on the [Rodrigues' rotation formula](https://en.wikipedia.org/wiki/Rotation_formalisms_in_three_dimensions#Rotation_matrix_%E2%86%94_Euler_axis/angle).

The motivation is the same; this change speeds up the conversions up to 10x, depending on the device, batch size, etc.

### Notes
- As the angles get very close to `π`, the existing implementation and the proposed one start to differ. However, (my understanding is that) this is not a problem as the axis can not be stably inferred from the rotation matrix in this case in general.
- bottler , I tried to follow similar conventions as existing functions to deal with weird angles, let me know if something needs to be changed to merge this.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1948

Reviewed By: MichaelRamamonjisoa

Differential Revision: D69193009

Pulled By: bottler

fbshipit-source-id: e5ed34b45b625114ec4419bb89e22a6aefad4eeb
2025-02-07 07:37:42 -08:00
Roman Shapovalov
215590b497 In FrameDataBuilder, set all path even if we don’t load blobs
Summary:
This is a somewhat not BC change: some None paths will be replaced by metadata paths, even when they were not used for data loading.

Moreover, removing the legacy fix to the paths in the old CO3D release.

Reviewed By: bottler

Differential Revision: D69048238

fbshipit-source-id: 2a8b26d7b9f5e2adf39c65888b5863a5a9de1996
2025-02-06 09:41:44 -08:00
Antoine Toisoul
43cd681d4f Updates to Implicitron dataset, metrics and tools
Summary: Update Pytorch3D to be able to run assetgen (see later diffs in the stack)

Reviewed By: shapovalov

Differential Revision: D65942513

fbshipit-source-id: 1d01141c9f7e106608fa591be6e0d3262cb5944f
2025-01-27 09:43:42 -08:00
Roman Shapovalov
42a4a7d432 Generalising SqlIndexDataset to support subtypes of SqlSequenceAnnotation
Summary: We did not often extend sequence-level metadata but now for applications like text-to-3D/video, we need to store captions and similar.

Reviewed By: bottler

Differential Revision: D68269926

fbshipit-source-id: f8af308adce51863d719a335d85cd2558943bd4c
2025-01-20 03:39:06 -08:00
generatedunixname89002005307016
699bc671ca Add missing Pyre mode headers] [batch:3/1531] [shard:41/N]
Differential Revision: D68316763

fbshipit-source-id: fb3e1e1a17786f6f681f1b11b48b4efd7a8ac311
2025-01-17 12:41:56 -08:00
Roman Shapovalov
49cf5a0f37 Loading fg probability from the alpha channel of image_rgb
Summary:
It is often easier to store the mask together with RGB, especially for renders. The logic in this diff:
* if load_mask and mask_path provided, take the mask from mask_path,
* otherwise, check if the image has the alpha channel and take it as a mask.

Reviewed By: antoinetlc

Differential Revision: D68160212

fbshipit-source-id: d9b6779f90027a4987ba96800983f441edff9c74
2025-01-15 11:53:30 -08:00
Roman Shapovalov
89b851e64c Refactor a utility function for bbox conversion
Summary: This function makes it easier to extend FrameData class with new channels; brushing it up a bit.

Reviewed By: bottler

Differential Revision: D67816470

fbshipit-source-id: 6575415c864d0f539e283889760cd2331bf226a7
2025-01-06 04:17:57 -08:00
Roman Shapovalov
5247f6ad74 Fixing type hints in FrameData
Summary: As subj

Reviewed By: bottler

Differential Revision: D67791200

fbshipit-source-id: c2db01c94718102618f4c8bc5c5130c65ee1d81f
2025-01-06 04:17:57 -08:00
Roman Shapovalov
e41aff47db Adding default values to FrameData for internal usage
Summary: Ensuring all fields in FrameData have defaults.

Reviewed By: bottler

Differential Revision: D67762780

fbshipit-source-id: b680d29a1a11689850905978df544cdb4eb7ddcd
2025-01-06 04:17:57 -08:00
Roman Shapovalov
64a5bfadc8 Adding SQL Dataset related files to the build script
Summary: Now that we have SQLAlchemy 2.0, we can fully use them.

Reviewed By: bottler

Differential Revision: D66920096

fbshipit-source-id: 25c0ea1c4f7361e66348035519627dc961b9e6e6
2024-12-23 16:05:26 -08:00
Thomas Polasek
055ab3a2e3 Convert directory fbcode/vision to use the Ruff Formatter
Summary:
Converts the directory specified to use the Ruff formatter in pyfmt

ruff_dog

If this diff causes merge conflicts when rebasing, please run
`hg status -n -0 --change . -I '**/*.{py,pyi}' | xargs -0 arc pyfmt`
on your diff, and amend any changes before rebasing onto latest.
That should help reduce or eliminate any merge conflicts.

allow-large-files

Reviewed By: bottler

Differential Revision: D66472063

fbshipit-source-id: 35841cb397e4f8e066e2159550d2f56b403b1bef
2024-11-26 02:38:20 -08:00
Edward Yang
f6c2ca6bfc Prepare for "Fix type-safety of torch.nn.Module instances": wave 2
Summary: See D52890934

Reviewed By: malfet, r-barnes

Differential Revision: D66245100

fbshipit-source-id: 019058106ac7eaacf29c1c55912922ea55894d23
2024-11-21 11:08:51 -08:00
Jeremy Reizenstein
e20cbe9b0e test fixes and lints
Summary:
- followup recent pyre change D63415925
- make tests remove temporary files
- weights_only=True in torch.load
- lint fixes

3 test fixes from VRehnberg in https://github.com/facebookresearch/pytorch3d/issues/1914
- imageio channels fix
- frozen decorator in test_config
- load_blobs positional

Reviewed By: MichaelRamamonjisoa

Differential Revision: D66162167

fbshipit-source-id: 7737e174691b62f1708443a4fae07343cec5bfeb
2024-11-20 09:15:51 -08:00
Jeremy Reizenstein
c17e6f947a run CI tests on main
Reviewed By: MichaelRamamonjisoa

Differential Revision: D66162168

fbshipit-source-id: 90268c1925fa9439b876df143035c9d3c3a74632
2024-11-20 05:06:52 -08:00
Yann Noutary
91c9f34137 Add safeguard in case num_tris diverges
Summary:
This PR fixes adds a safeguard preventing num_tris to overflow in `MAX_TRIS`-length arrays. The update rule of `num_tris` is bounded :

 - max(num_tris(t)) = 2*num_tris(t-1)
 - num_tris(0) = 12
 - t <= 6

So :
 - max(num_tris) = 2^6*12
 - max(num_tris) = 768

Reviewed By: bottler

Differential Revision: D66162573

fbshipit-source-id: e269a79c75c6cc33306986b1f1256cffbe96c730
2024-11-20 01:24:28 -08:00
Jeremy Reizenstein
81d82980bc Fix ogl test hang
Summary: See https://github.com/facebookresearch/pytorch3d/issues/1908

Reviewed By: MichaelRamamonjisoa

Differential Revision: D65280253

fbshipit-source-id: ec05902c5f2f7eb9ddd92bda0045cc3564b8c091
2024-11-06 11:40:42 -08:00
Jeremy Reizenstein
8fe6934885 fix subdivide_meshes with empty mesh #1788
Summary:
Simplify code

fixes https://github.com/facebookresearch/pytorch3d/issues/1788

Reviewed By: MichaelRamamonjisoa

Differential Revision: D61847675

fbshipit-source-id: 48400875d1d885bb3615bc9f4b3c7c3d822b67e7
2024-11-06 11:40:26 -08:00
bottler
c434957b2a Run tests in github action (#1896)
Summary: Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1896

Reviewed By: MichaelRamamonjisoa

Differential Revision: D65272512

Pulled By: bottler

fbshipit-source-id: 3bcfab43acd2d6be5444ff25178381510ddac015
2024-11-06 11:15:34 -08:00
Jeremy Reizenstein
dd2a11b5fc Fix OFF for new numpy errors
Summary: Error messages have changed around numpy version 2, making existing code fail.

Reviewed By: MichaelRamamonjisoa

Differential Revision: D65280674

fbshipit-source-id: b3ae613ea8f0f4ae20fb6e5e816314b8c10e6c65
2024-11-06 11:13:59 -08:00
Richard Barnes
9563ef79ca c10::optional -> std::optional in some files
Reviewed By: jermenkoo

Differential Revision: D65425234

fbshipit-source-id: 1e7707d6b6aab640cc1fdd3bd71a3b50f77a0909
2024-11-04 12:03:51 -08:00
generatedunixname89002005287564
008c7ab58c Pre-silence Pyre Errors for upcoming upgrade] [batch:67/603] [shard:3/N]
Reviewed By: MaggieMoss

Differential Revision: D65290095

fbshipit-source-id: ced87d096aa8939700de5599ce6984cd7ae93912
2024-10-31 16:26:25 -07:00
Jeremy Reizenstein
9eaed4c495 Fix K>1 in multimap UV sampling
Summary:
Fixes https://github.com/facebookresearch/pytorch3d/issues/1897
"Wrong dimension on gather".

Reviewed By: cijose

Differential Revision: D65280675

fbshipit-source-id: 1d587036887972bb2a2ea56d40df19cbf1aeb6cc
2024-10-31 16:05:10 -07:00
Richard Barnes
e13848265d at::optional -> std::optional (#1170)
Summary: Pull Request resolved: https://github.com/pytorch/ao/pull/1170

Reviewed By: gineshidalgo99

Differential Revision: D64938040

fbshipit-source-id: 57f98b90676ad0164a6975ea50e4414fd85ae6c4
2024-10-25 06:37:57 -07:00
generatedunixname89002005307016
58566963d6 Add type error suppressions for upcoming upgrade
Reviewed By: MaggieMoss

Differential Revision: D64502797

fbshipit-source-id: cee9a54dfa8a005d5912b895d0bd094f352c5c6f
2024-10-16 19:22:01 -07:00
Suresh Babu Kolla
e17ed5cd50 Hipify Pulsar for PyTorch3D
Summary:
- Hipified Pytorch Pulsar
   - Created separate target for Pulsar tests and enabled RE testing
   - Pytorch3D full test suite requires additional work like fixing EGL
     dependencies on AMD

Reviewed By: danzimm

Differential Revision: D61339912

fbshipit-source-id: 0d10bc966e4de4a959f3834a386bad24e449dc1f
2024-10-09 14:38:42 -07:00
Richard Barnes
8ed0c7a002 c10::optional -> std::optional
Summary: `c10::optional` is an alias for `std::optional`. Let's remove the alias and use the real thing.

Reviewed By: meyering

Differential Revision: D63402341

fbshipit-source-id: 241383e7ca4b2f3f1f9cac3af083056123dfd02b
2024-10-03 14:38:37 -07:00
Richard Barnes
2da913c7e6 c10::optional -> std::optional
Summary: `c10::optional` is an alias for `std::optional`. Let's remove the alias and use the real thing.

Reviewed By: palmje

Differential Revision: D63409387

fbshipit-source-id: fb6db59a14db9e897e2e6b6ad378f33bf2af86e8
2024-10-02 11:09:29 -07:00
generatedunixname89002005307016
fca83e6369 Convert .pyre_configuration.local to fast by default architecture] [batch:23/263] [shard:3/N] [A]
Reviewed By: connernilsen

Differential Revision: D63415925

fbshipit-source-id: c3e28405c70f9edcf8c21457ac4faf7315b07322
2024-09-25 17:34:03 -07:00
Jeremy Reizenstein
75ebeeaea0 update version to 0.7.8
Summary: as title

Reviewed By: das-intensity

Differential Revision: D62588556

fbshipit-source-id: 55bae19dd1df796e83179cd29d805fcd871b6d23
2024-09-13 02:31:49 -07:00
Jeremy Reizenstein
ab793177c6 remove pytorch2.0 builds
Summary: these are failing in ci

Reviewed By: das-intensity

Differential Revision: D62594666

fbshipit-source-id: 5e3a7441be2978803dc2d3e361365e0fffa7ad3b
2024-09-13 02:07:25 -07:00
Jeremy Reizenstein
9acdd67b83 fix obj material indexing bug #1368
Summary:
Make the negative index actually not an error

fixes https://github.com/facebookresearch/pytorch3d/issues/1368

Reviewed By: das-intensity

Differential Revision: D62177991

fbshipit-source-id: e5ed433bde1f54251c4d4b6db073c029cbe87343
2024-09-13 02:00:49 -07:00
Nicholas Dahm
3f428d9981 pytorch 2.4.0 + 2.4.1
Summary:
Apparently pytorch 2.4 is now supported as per [this closed issue](https://github.com/facebookresearch/pytorch3d/issues/1863).

Added the `2.4.0` & `2.4.1` versions to `regenerate.py` then ran that as per the `README_fb.md` which generated `config.yml` changes.

Reviewed By: bottler

Differential Revision: D62517831

fbshipit-source-id: 002e276dfe2fa078136ff2f6c747d937abbadd1a
2024-09-11 15:09:43 -07:00
Josh Fromm
05cbea115a Hipify Pytorch3D (#1851)
Summary:
X-link: https://github.com/pytorch/pytorch/pull/133343

X-link: https://github.com/fairinternal/pytorch3d/pull/45

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1851

Enables pytorch3d to build on AMD. An important part of enabling this was not compiling the Pulsar backend when the target is AMD. There are simply too many kernel incompatibilites to make it work (I tried haha). Fortunately, it doesnt seem like most modern applications of pytorch3d rely on Pulsar. We should be able to unlock most of pytorch3d's goodness on AMD without it.

Reviewed By: bottler, houseroad

Differential Revision: D61171993

fbshipit-source-id: fd4aee378a3568b22676c5bf2b727c135ff710af
2024-08-15 16:18:22 -07:00
generatedunixname89002005307016
38afdcfc68 upgrade pyre version in fbcode/vision - batch 2
Reviewed By: bottler

Differential Revision: D60992234

fbshipit-source-id: 899db6ed590ef966ff651c11027819e59b8401a3
2024-08-09 02:07:45 -07:00
Christine Sun
1e0b1d9c72 Remove Python versions from Install.md
Summary: To avoid the installation instructions for PyTorch3D becoming out-of-date, instead of specifying certain Python versions, update to just `Python`. Reader will understand it has to be a Python version compatible with GitHub.

Reviewed By: bottler

Differential Revision: D60919848

fbshipit-source-id: 5e974970a0db3d3d32fae44e5dd30cbc1ce237a9
2024-08-07 13:46:31 -07:00
Rebecca Chen (Python)
44702fdb4b Add "max" point reduction for chamfer distance
Summary:
* Adds a "max" option for the point_reduction input to the
  chamfer_distance function.
* When combining the x and y directions, maxes the losses instead
  of summing them when point_reduction="max".
* Moves batch reduction to happen after the directions are
  combined.
* Adds test_chamfer_point_reduction_max and
  test_single_directional_chamfer_point_reduction_max tests.

Fixes  https://github.com/facebookresearch/pytorch3d/issues/1838

Reviewed By: bottler

Differential Revision: D60614661

fbshipit-source-id: 7879816acfda03e945bada951b931d2c522756eb
2024-08-02 10:46:07 -07:00
Jeremy Reizenstein
7edaee71a9 allow matrix_to_quaternion onnx export
Summary: Attempt to allow torch.onnx.dynamo_export(matrix_to_quaternion) to work.

Differential Revision: D59812279

fbshipit-source-id: 4497e5b543bec9d5c2bdccfb779d154750a075ad
2024-07-16 11:30:20 -07:00
Roman Shapovalov
d0d0e02007 Fix: setting FrameData.crop_bbox_xywh for backwards compatibility
Summary: This diff is fixing a backwards compatibility issue in PyTorch3D's dataset API. The code ensures that the `crop_bbox_xywh` attribute is set when box_crop flag is on. This is an implementation detail that people should not really use, however some people depend on this behaviour.

Reviewed By: bottler

Differential Revision: D59777449

fbshipit-source-id: b875e9eb909038b8629ccdade87661bb2c39d529
2024-07-16 02:21:13 -07:00
Jeremy Reizenstein
4df110b0a9 remove fvcore dependency
Summary: This is not actually needed and is causing a conda-forge confusion to do with python_abi - which needs users to have `-c conda-forge` when they install pytorch3d.

Reviewed By: patricklabatut

Differential Revision: D59587930

fbshipit-source-id: 961ae13a62e1b2b2ce6d8781db38bd97eca69e65
2024-07-11 04:35:38 -07:00
Huy Do
51fd114d8b Forward fix internal pyre failure from D58983461
Summary:
X-link: https://github.com/pytorch/pytorch/pull/129525

Somehow, using underscore alias of some builtin types breaks pyre

Reviewed By: malfet, clee2000

Differential Revision: D59029768

fbshipit-source-id: cfa2171b66475727b9545355e57a8297c1dc0bc6
2024-06-27 07:35:18 -07:00
Jeremy Reizenstein
89653419d0 version 0.7.7
Summary: New version

Reviewed By: MichaelRamamonjisoa

Differential Revision: D58668979

fbshipit-source-id: 195eaf83e4da51a106ef72e38dbb98c51c51724c
2024-06-25 06:59:24 -07:00
Jeremy Reizenstein
7980854d44 require pytorch 2.0+
Summary: Problems with timeouts on old builds.

Reviewed By: MichaelRamamonjisoa

Differential Revision: D58819435

fbshipit-source-id: e1976534a102ad3841f3b297c772e916aeea12cb
2024-06-21 08:15:17 -07:00
Jeremy Reizenstein
51d7c06ddd MKL version fix in CI (#1820)
Summary:
Fix for "undefined symbol: iJIT_NotifyEvent" build issue,

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1820

Reviewed By: MichaelRamamonjisoa

Differential Revision: D58685326

fbshipit-source-id: 48b54367c00851cc6fbb111ca98d69a2ace8361b
2024-06-21 08:15:17 -07:00
Sergii Dymchenko
00c36ec01c Update deprecated PyTorch functions in fbcode/vision
Reviewed By: bottler

Differential Revision: D58762015

fbshipit-source-id: a0d05fe63a88d33e3f7783b5a7b2a476dd3a7449
2024-06-20 14:06:28 -07:00
vedrenne
b0462d8079 Allow indexing for classes inheriting Transform3d (#1801)
Summary:
Currently, it is not possible to access a sub-transform using an indexer for all 3d transforms inheriting the `Transforms3d` class.
For instance:

```python
from pytorch3d import transforms

N = 10
r = transforms.random_rotations(N)
T = transforms.Transform3d().rotate(R=r)
R = transforms.Rotate(r)

x = T[0]  # ok
x = R[0]  # TypeError: __init__() got an unexpected keyword argument 'matrix'
```

This is because all these classes (namely `Rotate`, `Translate`, `Scale`, `RotateAxisAngle`) inherit the `__getitem__()` method from `Transform3d` which has the [following code on line 201](https://github.com/facebookresearch/pytorch3d/blob/main/pytorch3d/transforms/transform3d.py#L201):

```python
return self.__class__(matrix=self.get_matrix()[index])
```

The four classes inheriting `Transform3d` are not initialized through a matrix argument, hence they error.
I propose to modify the `__getitem__()` method of the `Transform3d` class to fix this behavior. The least invasive way to do it I can think of consists of creating an empty instance of the current class, then setting the `_matrix` attribute manually. Thus, instead of
```python
return self.__class__(matrix=self.get_matrix()[index])
```
I propose to do:
```python
instance = self.__class__.__new__(self.__class__)
instance._matrix = self.get_matrix()[index]
return instance
```

As far as I can tell, this modification occurs no modification whatsoever for the user, except for the ability to index all 3d transforms.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1801

Reviewed By: MichaelRamamonjisoa

Differential Revision: D58410389

Pulled By: bottler

fbshipit-source-id: f371e4c63d2ae4c927a7ad48c2de8862761078de
2024-06-17 07:48:18 -07:00
Jeremy Reizenstein
b66d17a324 Undo c10=>std optional rename
Summary: Undoes the pytorch3d changes in D57294278 because they break builds for for PyTorch<2.1 .

Reviewed By: MichaelRamamonjisoa

Differential Revision: D57379779

fbshipit-source-id: 47a12511abcec4c3f4e2f62eff5ba99deb2fab4c
2024-06-17 07:09:30 -07:00
Kyle Vedder
717493cb79 Fixed last dimension size check so that it doesn't trivially pass. (#1815)
Summary:
Currently, it checks that the `2`th dimension of `p2` is the same size as the `2`th dimension of `p2` instead of `p1`.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1815

Reviewed By: MichaelRamamonjisoa

Differential Revision: D58586966

Pulled By: bottler

fbshipit-source-id: d4f723fa264f90fe368c10825c1acdfdc4c406dc
2024-06-17 06:00:13 -07:00
Jeremy Reizenstein
302da69461 builds for PyTorch 2.2.1 2.2.2 2.3.0 2.3.1
Summary: Build for new pytorch versions

Reviewed By: MichaelRamamonjisoa

Differential Revision: D58668956

fbshipit-source-id: 7fdfb377b370448d6147daded6a21b8db87586fb
2024-06-17 05:57:59 -07:00
Roman Shapovalov
4ae25bfce7 Moving ray bundle to float dtype
Summary: We can now move ray bundle to float dtype (e.g. from fp16 like types).

Reviewed By: bottler

Differential Revision: D57493109

fbshipit-source-id: 4e18a427e968b646fe5feafbff653811cd007981
2024-05-30 10:06:38 -07:00
Richard Barnes
bd52f4a408 c10::optional -> std::optional in tensorboard/adhoc/Adhoc.h +9
Summary: `c10::optional` was switched to be `std::optional` after PyTorch moved to C++17. Let's eliminate `c10::optional`, if we can.

Reviewed By: albanD

Differential Revision: D57294278

fbshipit-source-id: f6f26133c43f8d92a4588f59df7d689e7909a0cd
2024-05-13 16:40:34 -07:00
generatedunixname89002005307016
17117106e4 upgrade pyre version in fbcode/vision - batch 2
Differential Revision: D57183103

fbshipit-source-id: 7e2f42ddc6a1fa02abc27a451987d67a00264cbb
2024-05-10 01:18:43 -07:00
Richard Barnes
aec76bb4c8 Remove unused-but-set variables in vision/fair/pytorch3d/pytorch3d/csrc/pulsar/include/renderer.render.device.h +1
Summary:
This diff removes a variable that was set, but which was not used.

LLVM-15 has a warning `-Wunused-but-set-variable` which we treat as an error because it's so often diagnostic of a code issue. Unused but set variables often indicate a programming mistake, but can also just be unnecessary cruft that harms readability and performance.

Removing this variable will not change how your code works, but the unused variable may indicate your code isn't working the way you thought it was. I've gone through each of these by hand, but mistakes may have slipped through. If you feel the diff needs changes before landing, **please commandeer** and make appropriate changes: there are hundreds of these and responding to them individually is challenging.

For questions/comments, contact r-barnes.

 - If you approve of this diff, please use the "Accept & Ship" button :-)

Reviewed By: bottler

Differential Revision: D56886956

fbshipit-source-id: 0c515ed98b812b1c106a59e19ec90751ce32e8c0
2024-05-02 13:58:05 -07:00
Andres Suarez
47d5dc8824 Apply clang-format 18
Summary: Previously this code conformed from clang-format 12.

Reviewed By: igorsugak

Differential Revision: D56065247

fbshipit-source-id: f5a985dd8f8b84f2f9e1818b3719b43c5a1b05b3
2024-04-14 11:28:32 -07:00
generatedunixname89002005307016
fe0b1bae49 upgrade pyre version in fbcode/vision - batch 2
Differential Revision: D55650177

fbshipit-source-id: d5faa4d805bb40fe3dea70b0601e7a1382b09f3a
2024-04-02 18:11:50 -07:00
Ruishen Lyu
ccf22911d4 Optimize list_to_packed to avoid for loop (#1737)
Summary:
For larger N and Mi value (e.g. N=154, Mi=238) I notice list_to_packed() has become a bottleneck for my application. By removing the for loop and running on GPU, i see a 10-20 x speedup.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1737

Reviewed By: MichaelRamamonjisoa

Differential Revision: D54187993

Pulled By: bottler

fbshipit-source-id: 16399a24cb63b48c30460c7d960abef603b115d0
2024-04-02 07:50:25 -07:00
Ashim Dahal
128be02fc0 feat: adjusted sample_nums (#1768)
Summary:
adjusted sample_nums to match the number of columns in the image grid. It originally produced image grid with 5 axes but only 3 images and after this fix, the block would work as intended.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1768

Reviewed By: MichaelRamamonjisoa

Differential Revision: D55632872

Pulled By: bottler

fbshipit-source-id: 44d633a8068076889e49d49b8a7910dba0db37a7
2024-04-02 06:02:48 -07:00
Roeia Kishk
31e3488a51 Changed tutorials' pip searching
Summary:
### Generalise tutorials' pip searching:
## Required Information:
This diff contains changes to several PyTorch3D tutorials.

**Purpose of this diff:**
Replace the current installation code with a more streamlined approach that tries to install the wheel first and falls back to installing from source if the wheel is not found.

**Why this diff is required:**
This diff makes it easier to cope with new PyTorch releases and reduce the need for manual intervention, as the current process involves checking the version of PyTorch in Colab and building a new wheel if it doesn't match the expected version, which generates additional work each time there is a a new PyTorch version in Colab.

**Changes:**
Before:
```
    if torch.__version__.startswith("2.1.") and sys.platform.startswith("linux"):
        # We try to install PyTorch3D via a released wheel.
        pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
        version_str="".join([
            f"py3{sys.version_info.minor}_cu",
            torch.version.cuda.replace(".",""),
            f"_pyt{pyt_version_str}"
        ])
        !pip install fvcore iopath
        !pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/{version_str}/download.html
    else:
        # We try to install PyTorch3D from source.
        !pip install 'git+https://github.com/facebookresearch/pytorch3d.git@stable'
```
After:
```
    pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
    version_str="".join([
        f"py3{sys.version_info.minor}_cu",
        torch.version.cuda.replace(".",""),
        f"_pyt{pyt_version_str}"
    ])
    !pip install fvcore iopath
    if sys.platform.startswith("linux"):
      # We try to install PyTorch3D via a released wheel.
      !pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/{version_str}/download.html
      pip_list = !pip freeze
      need_pytorch3d = not any(i.startswith("pytorch3d==") for  i in pip_list)

    if need_pytorch3d:
        # We try to install PyTorch3D from source.
        !pip install 'git+https://github.com/facebookresearch/pytorch3d.git@stable'
```

Reviewed By: bottler

Differential Revision: D55431832

fbshipit-source-id: a8de9162470698320241ae8401427dcb1ce17c37
2024-03-28 11:24:43 -07:00
generatedunixname89002005307016
b215776f2d upgrade pyre version in fbcode/vision - batch 2
Differential Revision: D55395614

fbshipit-source-id: 71677892b5d6f219f6df25b4efb51fb0f6b1441b
2024-03-26 22:02:22 -07:00
Cijo Jose
38cf0dc1c5 TexturesUV multiple maps
Summary: Implements the  the TexturesUV with multiple map ids.

Reviewed By: bottler

Differential Revision: D53944063

fbshipit-source-id: 06c25eb6d69f72db0484f16566dd2ca32a560b82
2024-03-12 06:59:31 -07:00
Jaap Suter
7566530669 CUDA marching_cubes fix
Summary:
Fix an inclusive vs exclusive scan mix-up that was accidentally introduced when removing the Thrust dependency (`Thrust::exclusive_scan`) and reimplementing it using `at::cumsum` (which does an inclusive scan).

This fixes two Github reported issues:

 * https://github.com/facebookresearch/pytorch3d/issues/1731
 * https://github.com/facebookresearch/pytorch3d/issues/1751

Reviewed By: bottler

Differential Revision: D54605545

fbshipit-source-id: da9e92f3f8a9a35f7b7191428d0b9a9ca03e0d4d
2024-03-07 15:38:24 -08:00
Conner Nilsen
a27755db41 Pyre Configurationless migration for] [batch:85/112] [shard:6/N]
Reviewed By: inseokhwang

Differential Revision: D54438157

fbshipit-source-id: a6acfe146ed29fff82123b5e458906d4b4cee6a2
2024-03-04 18:30:37 -08:00
Amethyst Reese
3da7703c5a apply Black 2024 style in fbcode (4/16)
Summary:
Formats the covered files with pyfmt.

paintitblack

Reviewed By: aleivag

Differential Revision: D54447727

fbshipit-source-id: 8844b1caa08de94d04ac4df3c768dbf8c865fd2f
2024-03-02 17:31:19 -08:00
Jeremy Reizenstein
f34104cf6e version 0.7.6
Summary: New version

Reviewed By: cijose

Differential Revision: D53852987

fbshipit-source-id: 962ab9f61153883df9da0601356bd6b108dc5df7
2024-02-19 03:28:54 -08:00
Jeremy Reizenstein
f247c86dc0 Update tutorials for 0.7.6
Summary:
version number changed with
`sed -i "s/2.1\./2.2./" *b`

Reviewed By: cijose

Differential Revision: D53852986

fbshipit-source-id: 1662c8e6d671321887a3263bc3880d5c33d1f866
2024-02-19 03:28:54 -08:00
Cijo Jose
ae9d8787ce Support color in cubify
Summary: The diff support colors in cubify for align = "center"

Reviewed By: bottler

Differential Revision: D53777011

fbshipit-source-id: ccb2bd1e3d89be3d1ac943eff08f40e50b0540d9
2024-02-16 08:19:12 -08:00
Jeremy Reizenstein
8772fe0de8 Make OpenGL optional in tests
Summary: Add an option to run tests without the OpenGL Renderer.

Reviewed By: patricklabatut

Differential Revision: D53573400

fbshipit-source-id: 54a14e7b2f156d24e0c561fdb279f4a9af01b793
2024-02-13 07:43:42 -08:00
Ada Martin
c292c71c1a c++ marching cubes fix
Summary:
Fixes https://github.com/facebookresearch/pytorch3d/issues/1641. The bug was caused by the mistaken downcasting of an int64_t into int, causing issues only on inputs large enough to have hashes that escaped the bounds of an int32.

Also added a test case for this issue.

Reviewed By: bottler

Differential Revision: D53505370

fbshipit-source-id: 0fdd0efc6d259cc3b0263e7ff3a4ab2c648ec521
2024-02-08 11:13:15 -08:00
Jeremy Reizenstein
d0d9cae9cd builds for PyTorch 2.1.1 2.1.2 2.2.0
Summary: Build for new pytorch versions

Reviewed By: shapovalov

Differential Revision: D53266104

fbshipit-source-id: f7aaacaf39cab3839b24f45361c36f087d0ea7c9
2024-02-07 11:56:52 -08:00
generatedunixname89002005287564
1f92c4e9d2 vision/fair
Reviewed By: zsol

Differential Revision: D53258682

fbshipit-source-id: 3f006b5f31a2b1ffdc6323d3a3b08ac46c3162ce
2024-01-31 07:43:49 -08:00
generatedunixname89002005307016
9b981f2c7e suppress errors in vision/fair/pytorch3d
Differential Revision: D53152021

fbshipit-source-id: 78be99b00abe4d992db844ff5877a89d42d468af
2024-01-26 19:10:37 -08:00
generatedunixname89002005307016
85eccbbf77 suppress errors in vision/fair/pytorch3d
Differential Revision: D53111480

fbshipit-source-id: 0f506bf29cf908e40b058ae72f51e828cd597825
2024-01-25 21:13:30 -08:00
generatedunixname89002005307016
b80ab0caf0 upgrade pyre version in fbcode/vision - batch 1
Differential Revision: D53059851

fbshipit-source-id: f5d0951186c858f90ddf550323a163e4b6d42b68
2024-01-24 23:56:06 -08:00
Dimitris Prountzos
1e817914b3 Fix compiler warning in knn.ku
Summary: This change updates the type of p2_idx from size_t to int64_t to address compiler warnings related to signed/unsigned comparison.

Reviewed By: bottler

Differential Revision: D52879393

fbshipit-source-id: de5484d78a907fccdaae3ce036b5e4a1a0a4de70
2024-01-18 12:27:16 -08:00
Ido Zachevsky
799c1cd21b Allow get_rgbd_point_cloud to take any #channels
Summary: Fixed `get_rgbd_point_cloud` to take any number of image input channels.

Reviewed By: bottler

Differential Revision: D52796276

fbshipit-source-id: 3ddc0d1e337a6cc53fc86c40a6ddb136f036f9bc
2024-01-16 03:38:26 -08:00
Abdelrahman Selim
292acc71a3 Update so3 operations for numerical stability
Summary: Replace implementations of `so3_exp_map` and `so3_log_map` in so3.py with existing more-stable implementations.

Reviewed By: bottler

Differential Revision: D52513319

fbshipit-source-id: fbfc039643fef284d8baa11bab61651964077afe
2024-01-04 02:26:56 -08:00
Jeremy Reizenstein
3621a36494 mac build fix
Summary: Fix for https://github.com/facebookresearch/pytorch3d/issues/1708

Reviewed By: patricklabatut

Differential Revision: D52480756

fbshipit-source-id: 530c0f9413970fba042eec354e28318c96e35f42
2024-01-03 07:46:54 -08:00
Abdelrahman Selim
3087ab7f62 Standardize matrix_to_quaternion output
Summary:
An OSS user has pointed out in https://github.com/facebookresearch/pytorch3d/issues/1703 that the output of matrix_to_quaternion (in that file) can be non standardized.

This diff solves the issue by adding a line of standardize at the end of the function

Reviewed By: bottler

Differential Revision: D52368721

fbshipit-source-id: c8d0426307fcdb7fd165e032572382d5ae360cde
2023-12-21 13:43:29 -08:00
Tony Tan
e46ab49a34 Submeshing TexturesAtlas for PyTorch3D 3D Rendering
Summary: Implement submeshing for TexturesAtlas and add associated test

Reviewed By: bottler

Differential Revision: D52334053

fbshipit-source-id: d54080e9af1f0c01551702736e858e3bd439ac58
2023-12-21 11:08:01 -08:00
Hassan Lotfi
8a27590c5f Submeshing TexturesUV
Summary: Implement `submeshes` for TexturesUV. Fix what Meshes.submeshes passes to the texture's submeshes function to make this possible.

Reviewed By: bottler

Differential Revision: D52192060

fbshipit-source-id: 526734962e3376aaf75654200164cdcebfff6997
2023-12-19 06:48:06 -08:00
Eric Young
06cdc313a7 PyTorch3D - Avoid flip in TexturesAtlas
Summary: Performance improvement: Use torch.lerp to map uv coordinates to the range needed for grid_sample (i.e. map [0, 1] to [-1, 1] and invert the y-axis)

Reviewed By: bottler

Differential Revision: D51961728

fbshipit-source-id: db19a5e3f482e9af7b96b20f88a1e5d0076dac43
2023-12-11 12:49:17 -08:00
Roman Shapovalov
94da8841af Align_corners switch in Volumes
Summary:
Porting this commit by davnov134 .
93a3a62800 (diff-a8e107ebe039de52ca112ac6ddfba6ebccd53b4f53030b986e13f019fe57a378)

Capability to interpret world/local coordinates with various align_corners semantics.

Reviewed By: bottler

Differential Revision: D51855420

fbshipit-source-id: 834cd220c25d7f0143d8a55ba880da5977099dd6
2023-12-07 03:07:41 -08:00
generatedunixname89002005307016
fbc6725f03 upgrade pyre version in fbcode/vision - batch 2
Differential Revision: D51902460

fbshipit-source-id: 3ffc5d7d2da5c5d4e971ee8275bd999c709e0b12
2023-12-06 20:53:53 -08:00
Jeremy Reizenstein
6b8766080d Use cuda's make_float3 in pulsar
Summary: Fixes github.com/facebookresearch/pytorch3d/issues/1680

Reviewed By: MichaelRamamonjisoa

Differential Revision: D51587889

fbshipit-source-id: e68ae32d7041fb9ea3e981cf2bde47f947a41ca2
2023-12-05 03:15:02 -08:00
sewon.jeon
c373a84400 Use updated naming to remove warning (#1687)
Summary:
diag_suppress is  deprecated from cuda

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1687

Reviewed By: MichaelRamamonjisoa

Differential Revision: D51495875

Pulled By: bottler

fbshipit-source-id: 6543a15e666238365719117bfcf5f7dac532aec1
2023-12-05 03:15:02 -08:00
sewon.jeon
7606854ff7 Fix windows build (#1689)
Summary:
Change the data type usage in the code to ensure cross-platform compatibility
long -> int64_t

<img width="628" alt="image" src="https://github.com/facebookresearch/pytorch3d/assets/6214316/40041f7f-3c09-4571-b9ff-676c625802e9">

Tested under
Win 11 and Ubuntu 22.04
with
CUDA 12.1.1 torch 2.1.1

Related issues & PR

https://github.com/facebookresearch/pytorch3d/pull/9

https://github.com/facebookresearch/pytorch3d/issues/1679

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1689

Reviewed By: MichaelRamamonjisoa

Differential Revision: D51521562

Pulled By: bottler

fbshipit-source-id: d8ea81e223c642e0e9fb283f5d7efc9d6ac00d93
2023-12-05 03:14:06 -08:00
Jeremy Reizenstein
83bacda8fb lint
Summary: Fix recent flake complaints

Reviewed By: MichaelRamamonjisoa

Differential Revision: D51811912

fbshipit-source-id: 65183f5bc7058da910e4d5a63b2250ce8637f1cc
2023-12-04 13:43:34 -08:00
generatedunixname89002005307016
f74fc450e8 suppress errors in vision/fair/pytorch3d
Differential Revision: D51645956

fbshipit-source-id: 1ae7279efa0a27bb9bc5255527bafebb84fdafd0
2023-11-28 19:10:06 -08:00
Dan Johnson
3b4f8a4980 Adding reference and context to PointsRenderer
Summary: User confusion (https://github.com/facebookresearch/pytorch3d/issues/1579) about how zbuf is used for alpha compositing. Added small description and reference to paper to help give some context.

Reviewed By: bottler

Differential Revision: D51374933

fbshipit-source-id: 8c489a5b5d0a81f0d936c1348b9ade6787c39c9a
2023-11-16 08:58:08 -08:00
Aleksandrs Ecins
79b46734cb Fix lint in test_render_points
Summary: Fixes lint in test_render_points in the PyTorch3D library.

Differential Revision: D51289841

fbshipit-source-id: 1eae621eb8e87b0fe5979f35acd878944f574a6a
2023-11-14 11:07:28 -08:00
YangHai
55638f3bae Support reading uv and uv map for ply format if texture_uv exists in ply file (#1100)
Summary:
When the ply format looks as follows:
  ```
comment TextureFile ***.png
element vertex 892
property double x
property double y
property double z
property double nx
property double ny
property double nz
property double texture_u
property double texture_v
```
`MeshPlyFormat` class will read uv from the ply file and read the uv map as commented as TextureFile.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1100

Reviewed By: MichaelRamamonjisoa

Differential Revision: D50885176

Pulled By: bottler

fbshipit-source-id: be75b1ec9a17a1ed87dbcf846a9072ea967aec37
2023-11-14 07:44:14 -08:00
Jeremy Reizenstein
f4f2209271 Fix for mask_points=False
Summary: Remove unused argument `mask_points` from `get_rgbd_point_cloud` and fix `get_implicitron_sequence_pointcloud`, which assumed it was used.

Reviewed By: MichaelRamamonjisoa

Differential Revision: D50885848

fbshipit-source-id: c0b834764ad5ef560107bd8eab04952d000489b8
2023-11-14 07:42:18 -08:00
Jeremy Reizenstein
f613682551 marching_cubes type fix
Summary: fixes https://github.com/facebookresearch/pytorch3d/issues/1679

Reviewed By: MichaelRamamonjisoa

Differential Revision: D50949933

fbshipit-source-id: 5c467de8bf84dd2a3d61748b3846678582d24ea3
2023-11-14 07:38:54 -08:00
Jeremy Reizenstein
2f11ddc5ee version 0.7.5
Summary: update

Reviewed By: MichaelRamamonjisoa

Differential Revision: D50806966

fbshipit-source-id: 95fd341c9e5e4e07b689eb71b3a729baff3b8192
2023-10-31 09:27:49 -07:00
Jeremy Reizenstein
650cc09d22 update notebooks for 0.7.5
Summary:
```
sed -i 's/startswith((\"1.13.\", \"2.0.\"))/startswith\(\"2.1.\"\)/' *b
```

Reviewed By: shapovalov

Differential Revision: D50806967

fbshipit-source-id: df19462564edb1f840753efeae96b516c7a9f764
2023-10-31 09:27:49 -07:00
Jeremy Reizenstein
8c15afe71d fix warning from cross
Summary: Don't call tensor.cross() without dim, to avoid new warning.

Reviewed By: MichaelRamamonjisoa

Differential Revision: D49879590

fbshipit-source-id: e9ba83923b6dc3bcface6782b8b26729ab5b0a4c
2023-10-30 11:50:23 -07:00
Jeremy Reizenstein
6b437e21a6 CI fixes (#1676)
Summary:
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1676

Remove CUDA 10.2 build, remove sm35 from cuda12.

Reviewed By: MichaelRamamonjisoa

Differential Revision: D50790929

fbshipit-source-id: 2b8cd34493b633a97b4066e0fd61aff077f7ce0c
2023-10-30 09:50:04 -07:00
Jeremy Reizenstein
03f17ca1ea missing cudaGetLastError
Summary: Investigating a reported problem.

Reviewed By: MichaelRamamonjisoa

Differential Revision: D50791296

fbshipit-source-id: 8dc162b87d02debf05d16c2b4816fcd57234d7e0
2023-10-30 09:35:08 -07:00
Aniket Patil
a8c70161a1 Fixed typo in cameras.py (#1656)
Summary:
coodinates -> coordinates

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1656

Reviewed By: MichaelRamamonjisoa

Differential Revision: D50325515

Pulled By: bottler

fbshipit-source-id: 406d2e286ead4fa5e9080092b4918a748495ee23
2023-10-17 06:07:02 -07:00
Jeremy Reizenstein
28f914bf3b pytorch 2.1, drop mac builds
Summary: Build updates for PyTorch 2.1

Reviewed By: MichaelRamamonjisoa

Differential Revision: D50345762

fbshipit-source-id: 89bf4edf1c21566aa86a3abca9b4df7c4d1d17a2
2023-10-17 06:04:06 -07:00
generatedunixname89002005307016
eaf0709d6a suppress errors in vision/fair/pytorch3d
Differential Revision: D49531589

fbshipit-source-id: 61c28ae33d2e5f75fd1695f35dc99931a3aaf7d3
2023-09-22 03:33:05 -07:00
Tristan Rice
b7f4ba097c VolumeSampler: expose padding_mode for inside out rendering (#1638)
Summary:
This exposes a setting on VolumeSampler so you can change the padding_mode. This is very useful when using cameras inside a volume that doesn't cover the entire world. By setting the value to `border` you can get much better behavior than `zeros` which causes edge effects for things like the sky. Border emulates infinitely tall buildings instead.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1638

Test Plan:
Tested with torchdrive

Example before:
![image](https://github.com/facebookresearch/pytorch3d/assets/909104/e99ffb7c-c4ba-40f8-b15c-ad5d1b53f0df)

Example after:
![image](https://github.com/facebookresearch/pytorch3d/assets/909104/f8d9821b-93d5-44b5-b9d4-c1670711ddce)

Reviewed By: MichaelRamamonjisoa

Differential Revision: D49384383

Pulled By: bottler

fbshipit-source-id: 202b526e07320a18944c39a148beec94c0f5d68c
2023-09-20 08:00:02 -07:00
Jeremy Reizenstein
6f2212da46 use less thrust, maybe help Windows
Summary: I think we include more thrust than needed, and maybe removing it will help things like  https://github.com/facebookresearch/pytorch3d/issues/1610 with DebugSyncStream errors on Windows.

Reviewed By: shapovalov

Differential Revision: D48949888

fbshipit-source-id: add889c0acf730a039dc9ffd6bbcc24ded20ef27
2023-09-05 04:40:24 -07:00
Richard Barnes
a3d99cab6b Del (object) from 200 inc vision/fair/mae_st/util/meters.py
Summary: Python3 makes the use of `(object)` in class inheritance unnecessary. Let's modernize our code by eliminating this.

Reviewed By: itamaro

Differential Revision: D48673863

fbshipit-source-id: 032d6028371f0350252e6b731c74f0f5933c83cd
2023-08-26 13:55:56 -07:00
Haritha Jayasinghe
d84f274a08 add None option for chamfer distance point reduction (#1605)
Summary:
The `chamfer_distance` function currently allows `"sum"` or `"mean"` reduction, but does not support returning unreduced (per-point) loss terms. Unreduced losses could be useful if the user wishes to inspect individual losses, or perform additional modifications to loss terms before reduction. One example would be implementing a robust kernel over the loss.

This PR adds a `None` option to the `point_reduction` parameter, similar to `batch_reduction`. In case of bi-directional chamfer loss, both the forward and backward distances are returned (a tuple of Tensors of shape `[D, N]` is returned). If normals are provided, similar logic applies to normals as well.

This PR addresses issue https://github.com/facebookresearch/pytorch3d/issues/622.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1605

Reviewed By: jcjohnson

Differential Revision: D48313857

Pulled By: bottler

fbshipit-source-id: 35c824827a143649b04166c4817449e1341b7fd9
2023-08-15 10:36:06 -07:00
generatedunixname89002005307016
099fc069fb suppress errors in vision/fair/pytorch3d
Differential Revision: D47643182

fbshipit-source-id: 598de1526e0c717f2f7038c3f4873ac119c65bba
2023-07-20 12:37:46 -07:00
Jeremy Reizenstein
57f6e79280 attempt to fix readthedocs
Summary:
Something's wrong with recommonmark/CommonMark/six, let's see if this fixes it.

https://readthedocs.org/projects/pytorch3d/builds/21292632/

```
  File "/home/docs/checkouts/readthedocs.org/user_builds/pytorch3d/envs/latest/lib/python3.11/site-packages/sphinx/config.py", line 368, in eval_config_file
    execfile_(filename, namespace)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pytorch3d/envs/latest/lib/python3.11/site-packages/sphinx/util/pycompat.py", line 150, in execfile_
    exec_(code, _globals)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pytorch3d/checkouts/latest/docs/conf.py", line 25, in <module>
    from recommonmark.parser import CommonMarkParser
  File "/home/docs/checkouts/readthedocs.org/user_builds/pytorch3d/envs/latest/lib/python3.11/site-packages/recommonmark/parser.py", line 6, in <module>
    from CommonMark import DocParser, HTMLRenderer
  File "/home/docs/checkouts/readthedocs.org/user_builds/pytorch3d/envs/latest/lib/python3.11/site-packages/CommonMark/__init__.py", line 3, in <module>
    from CommonMark.CommonMark import HTMLRenderer
  File "/home/docs/checkouts/readthedocs.org/user_builds/pytorch3d/envs/latest/lib/python3.11/site-packages/CommonMark/CommonMark.py", line 18, in <module>
    HTMLunescape = html.parser.HTMLParser().unescape
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'HTMLParser' object has no attribute 'unescape'
```

Reviewed By: shapovalov

Differential Revision: D47471545

fbshipit-source-id: 48e121e20da535b3cc46b6bd2393d28869067b8b
2023-07-14 09:49:09 -07:00
Jeremy Reizenstein
2883a07bfe update wheel scripts
Summary: New versions of cuda etc. I haven't committed recent changes to this for a while

Reviewed By: shapovalov

Differential Revision: D47396136

fbshipit-source-id: d6c27f5056fa8f4a74a628fa1d831159000acf55
2023-07-14 09:09:46 -07:00
Jeremy Reizenstein
6462aa60ea switch to readthedocs.yaml
Summary: This is needed from september 2023. As a side effect, implicitron docs should build better because typing.get_args exists etc.

Reviewed By: shapovalov

Differential Revision: D47363855

fbshipit-source-id: a954c5b81b1e5a4435fca146a11aea0d2ca96f45
2023-07-13 06:56:19 -07:00
Roman Shapovalov
d851bc3173 Read depth maps from OpenXR files
Summary:
Blender uses OpenEXR to dump depth maps, so we have to support it.
OpenCV requires to explicitly accepth the vulnerabilities by setting the env var before exporting.
We can set it but I think it should be user’s responsibility.
OpenCV error reporting is adequate, so I don’t handle the error on our side.

Reviewed By: bottler

Differential Revision: D47403884

fbshipit-source-id: 2fcadd1df9d0efa0aea563bcfb2e3180b3c4d1d7
2023-07-13 04:50:03 -07:00
Roman Shapovalov
8164ac4081 Fix: tensor vs array type correctness
Summary:
For fg-masking depth, we assumed np.array but passed a Tensor; for defining the default depth_mask, vice versa.

Note that we change the intended behaviour for the latter, assuming that 0s are areas with empty depth. When loading depth masks, we replace NaNs with zeros, so it is sensible. It is not a BC change as that branch would crash if executed. Since there was no reports, I assume no one cared.

Reviewed By: bottler

Differential Revision: D47403588

fbshipit-source-id: 1094104176d7d767a5657b5bbc9f5a0cc9da0ede
2023-07-13 04:50:03 -07:00
Emilien Garreau
9446d91fae Avoid to keep in memory lengths and bins for ImplicitronRayBundle
Summary:
Convert ImplicitronRayBundle to a "classic" class instead of a dataclass. This change is introduced as a way to preserve the ImplicitronRayBundle interface while allowing two outcomes:
- init lengths arguments is now a Optional[torch.Tensor] instead of torch.Tensor
- lengths is now a property which returns a `torch.Tensor`. The lengths property will either recompute lengths from bins or return the stored _lengths. `_lenghts` is None if bins is set. It saves us a bit of memory.

Reviewed By: shapovalov

Differential Revision: D46686094

fbshipit-source-id: 3c75c0947216476ebff542b6f552d311024a679b
2023-07-06 02:41:15 -07:00
Emilien Garreau
3d011a9198 Adapt RayPointRefiner and RayMarcher to support bins.
Summary:
## Context

Bins are used in mipnerf to allow to manipulate easily intervals. For example, by doing the following, `bins[..., :-1]` you will obtain all the left coordinates of your intervals, while doing `bins[..., 1:]` is equals to the right coordinates of your intervals.

We introduce here the support of bins like in MipNerf implementation.

## RayPointRefiner

Small changes have been made to modify RayPointRefiner.
- If bins is None

```
mids = torch.lerp(ray_bundle.lengths[..., 1:], ray_bundle.lengths[…, :-1], 0.5)
z_samples = sample_pdf(
		mids, # [..., npt]
		weights[..., 1:-1], # [..., npt - 1]
               ….
            )
```

- If bins is not None
In the MipNerf implementation the sampling is done on all the bins. It allows us to use the full weights tensor without slashing it.

```
z_samples = sample_pdf(
		ray_bundle.bins, # [..., npt + 1]
		weights, # [..., npt]
               ...
            )
```

## RayMarcher

Add a ray_deltas optional argument. If None, keep the same deltas computation from ray_lengths.

Reviewed By: shapovalov

Differential Revision: D46389092

fbshipit-source-id: d4f1963310065bd31c1c7fac1adfe11cbeaba606
2023-07-06 02:41:15 -07:00
Emilien Garreau
5910d81b7b Add blurpool following MIPNerf paper.
Summary:
Add blurpool has defined in [MIP-NeRF](https://arxiv.org/abs/2103.13415).
It has been added has an option for RayPointRefiner.

Reviewed By: shapovalov

Differential Revision: D46356189

fbshipit-source-id: ad841bad86d2b591a68e1cb885d4f781cf26c111
2023-07-06 02:20:53 -07:00
Emilien Garreau
ccf860f1db Add integrated position encoding based on MIPNerf implementation.
Summary: Add a new implicit module Integral Position Encoding based on [MIP-NeRF](https://arxiv.org/abs/2103.13415).

Reviewed By: shapovalov

Differential Revision: D46352730

fbshipit-source-id: c6a56134c975d80052b3a11f5e92fd7d95cbff1e
2023-07-06 02:20:53 -07:00
Emilien Garreau
29b8ebd802 Add utils to approximate the conical frustums as multivariate gaussians.
Summary:
Introduce methods to approximate the radii of conical frustums along rays as described in [MipNerf](https://arxiv.org/abs/2103.13415):

- Two new attributes are added to ImplicitronRayBundle: bins and radii. Bins is of size n_pts_per_ray + 1. It allows us to manipulate easily and n_pts_per_ray intervals. For example we need the intervals coordinates in the radii computation for \(t_{\mu}, t_{\delta}\). Radii are used to store the radii of the conical frustums.

- Add 3 new methods to compute the radii:
   - approximate_conical_frustum_as_gaussians: It computes the mean along the ray direction, the variance of the
      conical frustum  with respect to t and variance of the conical frustum with respect to its radius. This
      implementation follows the stable computation defined in the paper.
   - compute_3d_diagonal_covariance_gaussian: Will leverage the two previously computed variances to find the
     diagonal covariance of the Gaussian.
   - conical_frustum_to_gaussian: Mix everything together to compute the means and the diagonal covariances along
     the ray of the Gaussians.

- In AbstractMaskRaySampler, introduces the attribute `cast_ray_bundle_as_cone`. If False it won't change the previous behaviour of the RaySampler. However if True, the samplers will sample `n_pts_per_ray +1` instead of `n_pts_per_ray`. This points are then used to set the bins attribute of ImplicitronRayBundle. The support of HeterogeneousRayBundle has not been added since the current code does not allow it. A safeguard has been added to avoid a silent bug in the future.

Reviewed By: shapovalov

Differential Revision: D45269190

fbshipit-source-id: bf22fad12d71d55392f054e3f680013aa0d59b78
2023-07-06 01:55:41 -07:00
Emilien Garreau
4e7715ce66 Remove unused pyre-ignore or pyre-fixme
Reviewed By: bottler

Differential Revision: D47223471

fbshipit-source-id: 8bdabf2a69dd7aec7202141122a9c69220ba7ef1
2023-07-05 02:58:47 -07:00
Jeremy Reizenstein
f68371d398 lints
Summary: simple

Reviewed By: shapovalov

Differential Revision: D46438865

fbshipit-source-id: 0f41cb3ddd7e7aca4513267d33299531f7e8d373
2023-06-16 06:48:04 -07:00
Jeremy Reizenstein
dc2c7e489f docs build remove mock after D45600232
Summary: We now use unittest.mock

Reviewed By: shapovalov

Differential Revision: D45868799

fbshipit-source-id: cd1042dc2c49c82c7b9e024f761c496049a31beb
2023-06-16 04:50:30 -07:00
Jeremy Reizenstein
42e7de418c fix test_build internal
Summary: Make test work in isolation, and when run internally make it not try the sqlalchemy files.

Reviewed By: shapovalov

Differential Revision: D46352513

fbshipit-source-id: 7417a25d7a5347d937631c9f56ae4e3242dd622e
2023-06-16 04:49:02 -07:00
Richard Higgins
88429853b9 1-line tutorial notebook plotting change to fix blank figure problem (#1549)
Summary:
Hi,

Not sure this is the best fix. But while running this notebook, I only ever saw a blank canvas when trying to visualize the dolphin. It might be that I have a broken dependency, like plotly. I also don't know what the visualization is "supposed" to look like.

But incase other people have this issue, this one line change solved the whole problem for me. Now I have a happy, rotatable dolphin.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1549

Reviewed By: shapovalov

Differential Revision: D46350930

Pulled By: bottler

fbshipit-source-id: e19aa71eb05a93e2955262a2c90d1f0d09576228
2023-06-16 04:35:15 -07:00
Jeremy Reizenstein
009a3d3b3c projects/nerf subsampling fix for newish pytorch #1441
Summary: Fix for https://github.com/facebookresearch/pytorch3d/issues/1441 where we were indexing with a tensor on the wrong device.

Reviewed By: shapovalov

Differential Revision: D46276449

fbshipit-source-id: 7750ed45ffecefa5d291fd1eadfe515310c2cf0d
2023-06-16 04:33:24 -07:00
Roman Shapovalov
cd5db076d5 Adding SQL dataset classes to ImplicitronDataSource imports
Summary: Making it easier for the clients to use these datasets.

Reviewed By: bottler

Differential Revision: D46727179

fbshipit-source-id: cf619aee4c4c0222a74b30ea590cf37f08f014cc
2023-06-14 10:51:47 -07:00
Roman Shapovalov
3d886c32d5 Fix: subset filter in DatasetBase implementation
Summary: In D42739669, I forgot to update the API of existing implementations of DatasetBase to take `subset_filter`. Looks like only one was missing.

Reviewed By: bottler

Differential Revision: D46724488

fbshipit-source-id: 13ab7a457f853278cf06955aad0cc2bab5fbcce6
2023-06-14 08:48:14 -07:00
Roman Shapovalov
5592d25f68 Fix: changed deprecated matplotlib parameter
Summary:
The parameter was renamed in MPL 3.5.0 in 2021, and the deprecated version is no longer supported.

https://matplotlib.org/stable/api/prev_api_changes/api_changes_3.5.0.html#the-first-parameter-of-axes-grid-and-axis-grid-has-been-renamed-to-visible

Reviewed By: bottler

Differential Revision: D46724242

fbshipit-source-id: 04e2ab6d63369d939ea4f0ce7d64693e0b95ee91
2023-06-14 08:48:14 -07:00
Roman Shapovalov
09a99f2e6d Support limiting num sequences per category.
Summary:
Adds stratified sampling of sequences within categories applied after category / sequence filters but before the num sequence limit.
It respects the insertion order into the sequence_annots table, i.e. takes top N sequences within each category.

Reviewed By: bottler

Differential Revision: D46724002

fbshipit-source-id: 597cb2a795c3f3bc07f838fc51b4e95a4f981ad3
2023-06-14 07:12:02 -07:00
Norman Mueller
5ffeb4d580 Single directional chamfer distance and non-absolute cosine similarity
Summary: Single directional chamfer distance and option to use non-absolute cosine similarity

Reviewed By: bottler

Differential Revision: D46593980

fbshipit-source-id: b2e591706a0cdde1c2d361614cecebb84a581433
2023-06-13 09:09:15 -07:00
generatedunixname89002005307016
573a42cd5f suppress errors in vision/fair/pytorch3d
Differential Revision: D46685078

fbshipit-source-id: daf2e75f24b68d2eab74cddca8ab9446e96951e7
2023-06-13 07:14:48 -07:00
Jeremy Reizenstein
928efdd640 fix camera plot for new matplotlib
Summary: fixes https://github.com/facebookresearch/pytorch3d/issues/1554 , needed for Matplotlib 3.6+

Reviewed By: patricklabatut

Differential Revision: D46438822

fbshipit-source-id: f3c06ad5d8e881a635edd14f96d498dca73c169f
2023-06-07 05:10:10 -07:00
Emilien Garreau
35badc0892 Fix inversion between fine and coarse implicit_functions
Summary: Fine implicit function was called before the coarse implicit function.

Reviewed By: shapovalov

Differential Revision: D46224224

fbshipit-source-id: 6b1cc00cc823d3ea7a5b42774c9ec3b73a69edb5
2023-05-26 08:33:54 -07:00
generatedunixname89002005307016
e0c3ca97ff upgrade pyre version in fbcode/vision - batch 2
Differential Revision: D46119066

fbshipit-source-id: b766118b9dcc07075d328fba73f272419dc9fc38
2023-05-23 20:34:54 -07:00
Roman Shapovalov
d2119c285f Serialising dynamic arrays in SQL; read-only SQLite connection in SQL Dataset
Summary:
1. We may need to store arrays of unknown shape in the database. It implements and tests serialisation.

2. Previously, when an inexisting metadata file was passed to SqlIndexDataset, it would try to open it and create an empty file, then crash. We now open the file in a read-only mode, so the error message is more intuitive. Note that the implementation is SQLite specific.

Reviewed By: bottler

Differential Revision: D46047857

fbshipit-source-id: 3064ae4f8122b4fc24ad3d6ab696572ebe8d0c26
2023-05-22 02:24:49 -07:00
Jeremy Reizenstein
ff80183fdb resources fix
Summary: I don't know why RE tests sometimes fail here, but maybe it's a race condition. If that's right, this should fix it.

Reviewed By: shapovalov

Differential Revision: D46020054

fbshipit-source-id: 20b746b09ad9bd77c2601ac681047ccc6cc27ed9
2023-05-19 06:41:33 -07:00
Roman Shapovalov
b0462598ac Refactor: FrameDataBuilder is more extensible.
Summary:
This is mostly a refactoring diff to reduce friction in extending the frame data.

Slight functional changes: dataset getitem now accepts (seq_name, frame_number_as_singleton_tensor) as a non-advertised feature. Otherwise this code crashes:
```
item = dataset[0]
dataset[item.sequence_name, item.frame_number]
```

Reviewed By: bottler

Differential Revision: D45780175

fbshipit-source-id: 75b8e8d3dabed954a804310abdbd8ab44a8dea29
2023-05-17 10:38:34 -07:00
Virendra Kumar Pathak
d08fe6d45a Softly deprecate the get_str=False flag.
Summary: We don't want to use print directly in stats.print() method. Instead this method will return the output string to the caller.

Reviewed By: shapovalov

Differential Revision: D45356240

fbshipit-source-id: 2cabe3cdfb9206bf09aa7b3cdd2263148a5ba145
2023-05-14 01:24:31 -07:00
Jeremy Reizenstein
297020a4b1 version 0.7.4
Summary: version number

Reviewed By: shapovalov

Differential Revision: D45704549

fbshipit-source-id: d63867f305b07c30ed9ea104f1494d23710fdbb7
2023-05-10 04:42:58 -07:00
Jeremy Reizenstein
062e6c54ae builds for PyTorch 2.0.1; drop 1.9
Summary: Drop support for PyTorch 1.9.0 and 1.9.1.

Reviewed By: shapovalov

Differential Revision: D45704329

fbshipit-source-id: c0fe3ecf6a1eb9bcd4163785c0cb4bf4f5060f50
2023-05-10 02:38:47 -07:00
Roman Shapovalov
c80180c96e Fix: FrameDataBuilder working with PathManager
Summary: In refactoring, we lost path manager here, which broke manifold storage. Fixing this.

Reviewed By: bottler

Differential Revision: D45574940

fbshipit-source-id: 579349eaa654215a09e057be57b56b46769c986a
2023-05-09 04:56:39 -07:00
Jason Fried
23cd19fbc7 typing.NamedTuple.field_types removed in favor of __annotations__
Summary:
typing.NamedTuple was simplified in 3.10
These two fields were the same in 3.8,  so this should be a no-op

#buildmore

Reviewed By: bottler

Differential Revision: D45373526

fbshipit-source-id: 2b26156f5f65b7be335133e9e705730f7254260d
2023-05-08 13:53:16 -07:00
dhb
092400f1e7 allow saving vertex normal in save_obj (#1511)
Summary:
Although we can load per-vertex normals in `load_obj`, saving per-vertex normals is not supported in `save_obj`.

This patch fixes this by allowing passing per-vertex normal data in `save_obj`:
``` python
def save_obj(
    f: PathOrStr,
    verts,
    faces,
    decimal_places: Optional[int] = None,
    path_manager: Optional[PathManager] = None,
    *,
    verts_normals: Optional[torch.Tensor] = None,
    faces_normals: Optional[torch.Tensor] = None,
    verts_uvs: Optional[torch.Tensor] = None,
    faces_uvs: Optional[torch.Tensor] = None,
    texture_map: Optional[torch.Tensor] = None,
) -> None:
    """
    Save a mesh to an .obj file.

    Args:
        f: File (str or path) to which the mesh should be written.
        verts: FloatTensor of shape (V, 3) giving vertex coordinates.
        faces: LongTensor of shape (F, 3) giving faces.
        decimal_places: Number of decimal places for saving.
        path_manager: Optional PathManager for interpreting f if
            it is a str.
        verts_normals: FloatTensor of shape (V, 3) giving the normal per vertex.
        faces_normals: LongTensor of shape (F, 3) giving the index into verts_normals
            for each vertex in the face.
        verts_uvs: FloatTensor of shape (V, 2) giving the uv coordinate per vertex.
        faces_uvs: LongTensor of shape (F, 3) giving the index into verts_uvs for
            each vertex in the face.
        texture_map: FloatTensor of shape (H, W, 3) representing the texture map
            for the mesh which will be saved as an image. The values are expected
            to be in the range [0, 1],
    """
```

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1511

Reviewed By: shapovalov

Differential Revision: D45086045

Pulled By: bottler

fbshipit-source-id: 666efb0d2c302df6cf9f2f6601d83a07856bf32f
2023-05-07 06:32:02 -07:00
generatedunixname89002005287564
ec87284c4b Replace third-party mock with unittest.mock] vision/fair
Reviewed By: bottler

Differential Revision: D45600232

fbshipit-source-id: f41b95c6fca86d241666b54755a128cd33f6dd32
2023-05-05 09:36:30 -07:00
Xiao Xuan
f5a117c74b fix: correct typo in cameras.md (#1501)
Summary:
If my understanding is right, prp_screen[1] should be 32 rather than 48.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1501

Reviewed By: shapovalov

Differential Revision: D45044406

Pulled By: bottler

fbshipit-source-id: 7dd93312db4986f4701e642ba82d94333466b921
2023-05-05 08:13:39 -07:00
Jeremy Reizenstein
b921efae3e CUB usage fix for sample_farthest_points
Summary: Fix for https://github.com/facebookresearch/pytorch3d/issues/1529

Reviewed By: shapovalov

Differential Revision: D45569211

fbshipit-source-id: 8c485f26cd409cafac53d4d982a03cde81a1d853
2023-05-05 05:59:14 -07:00
Roman Shapovalov
c8d6cd427e Fix test_data_source in OSS.
Summary: Import generic path; avoiding incorrect path patching.

Reviewed By: bottler

Differential Revision: D45573976

fbshipit-source-id: e6ff4d759deb936e3b636defa1e0851fb0127b46
2023-05-05 02:05:50 -07:00
Jeremy Reizenstein
ef5f620263 nondeterminism warnings
Summary: do like xformers.

Reviewed By: shapovalov

Differential Revision: D44541873

fbshipit-source-id: 2c23160591cd9026fcd4972998d1bc90adba1356
2023-05-04 12:50:41 -07:00
Roman Shapovalov
3e3644e534 More tests for SQL Dataset
Summary:
I forgot to include these tests to D45086611 when transferring code from pixar_replay repo.

They test the new ORM types used in SQL dataset and are SQL Alchemy 2.0 specific.

An important test for extending types is a proof of concept for generality of SQL Dataset. The idea is to extend FrameAnnotation and FrameData in parallel.

Reviewed By: bottler

Differential Revision: D45529284

fbshipit-source-id: 2a634e518f580c312602107c85fc320db43abcf5
2023-05-04 03:32:27 -07:00
Ilia Vitsnudel
178a7774d4 Adding save mesh into glb file in TexturesVertex format
Summary:
Added a suit of functions and code additions to experimental_gltf_io.py file to enable saving Meshes in TexturesVertex format into .glb file.
Also added a test to tets_io_gltf.py to check the functionality with the test described in Test Plane.

Reviewed By: bottler

Differential Revision: D44969144

fbshipit-source-id: 9ce815a1584b510442fa36cc4dbc8d41cc3786d5
2023-05-01 00:41:47 -07:00
Emilien Garreau
823ab75d27 Simplify _xy_grid computation in raysampling
Summary: Remove the need of tuple and reversed in the raysampling xy_grid computation

Reviewed By: bottler

Differential Revision: D45269342

fbshipit-source-id: d0e4c0923b9a2cca674b35e8d64862043a0eab3b
2023-04-27 03:07:37 -07:00
Roman Shapovalov
32e1992924 SQL Index Dataset
Summary:
Moving SQL dataset to PyTorch3D. It has been extensively tested in pixar_replay.

It requires SQLAlchemy 2.0, which is not supported in fbcode. So I exclude the sources and tests that depend on it from buck TARGETS.

Reviewed By: bottler

Differential Revision: D45086611

fbshipit-source-id: 0285f03e5824c0478c70ad13731525bb5ec7deef
2023-04-25 09:56:15 -07:00
Roman Shapovalov
7aeedd17a4 When bounding boxes are cached in metadata, don’t crash on load_masks=False
Summary:
We currently support caching bounding boxes in MaskAnnotation. If present, they are not re-computed from the mask. However, the masks need to be loaded for the bbox to be set.

This diff fixes that. Even if load_masks / load_blobs are unset, the bounding box can be picked up from the metadata.

Reviewed By: bottler

Differential Revision: D45144918

fbshipit-source-id: 8a2e2c115e96070b6fcdc29cbe57e1cee606ddcd
2023-04-20 07:28:45 -07:00
Roman Shapovalov
0e3138eca8 Optional ground-truth depth maps in visualiser
Summary: The code does not crash if depth map/mask are not given.

Reviewed By: bottler

Differential Revision: D45082985

fbshipit-source-id: 3610d8beb4ac897fbbe52f56a6dd012a6365b89b
2023-04-18 07:00:17 -07:00
Richard Barnes
1af6bf4768 Replace hasattr with getattr in vision/fair/pytorch3d/pytorch3d/renderer/cameras.py
Summary:
The pattern
```
X.Y if hasattr(X, "Y") else Z
```
can be replaced with
```
getattr(X, "Y", Z)
```

The [getattr](https://www.w3schools.com/python/ref_func_getattr.asp) function gives more succinct code than the [hasattr](https://www.w3schools.com/python/ref_func_hasattr.asp) function. Please use it when appropriate.

**This diff is very low risk. Green tests indicate that you can safely Accept & Ship.**

Reviewed By: bottler

Differential Revision: D44886893

fbshipit-source-id: 86ba23e837217e1ebd64bf8e27d286257894839e
2023-04-14 04:24:54 -07:00
generatedunixname89002005307016
355d6332cb upgrade pyre version in fbcode/vision - batch 2
Differential Revision: D44881859

fbshipit-source-id: 4ed410724a14d580f811c1288f51a71ce8fb0c9a
2023-04-11 17:15:12 -07:00
Jeremy Reizenstein
e245560abb version 0.7.3
Summary: update version number

Reviewed By: davidsonic

Differential Revision: D44343297

fbshipit-source-id: 763a25fbe0c880e8b7ad851b8e4b57787e449cab
2023-04-04 07:48:02 -07:00
Jeremy Reizenstein
33bf10f961 INSTALL updates for 0.7.3
Summary: PyTorch 2.0 now supported

Reviewed By: davidsonic

Differential Revision: D44343298

fbshipit-source-id: c81556dc872141e692a97845da0fb50fe82f62da
2023-04-04 07:48:02 -07:00
Jeremy Reizenstein
274b6df918 update notebooks for 0.7.3
Summary:
Allow pytorch2.0 download:
```
sed -i 's/startswith(\\"1.13.\\")/startswith\(\(\\"1.13.\\", \\"2.0.\\"\)\)/' *b
```

Remove lines which download and install CUB:
```
sed -i.bak '/1.10\.0/d' *b
rm *.bak
```

Reviewed By: davidsonic

Differential Revision: D44343299

fbshipit-source-id: e8399b5dd10068c717178ba9ffb0630bacca3253
2023-04-04 07:48:02 -07:00
Ildar Salakhiev
ebdbfde0ce Extract BlobLoader class from JsonIndexDataset and moving crop_by_bbox to FrameData
Summary:
extracted blob loader
added documentation for blob_loader
did some refactoring on fields
for detailed steps and discussions see:
https://github.com/facebookresearch/pytorch3d/pull/1463
https://github.com/fairinternal/pixar_replay/pull/160

Reviewed By: bottler

Differential Revision: D44061728

fbshipit-source-id: eefb21e9679003045d73729f96e6a93a1d4d2d51
2023-04-04 07:17:43 -07:00
Dejan Kovachev
c759fc560f Hard population of registry system with pre_expand
Summary: Provide an extension point pre_expand to let a configurable class A make sure another class B is registered before A is expanded. This reduces top level imports.

Reviewed By: bottler

Differential Revision: D44504122

fbshipit-source-id: c418bebbe6d33862d239be592d9751378eee3a62
2023-03-31 07:44:38 -07:00
Emilien Garreau
813e941de5 Add the OverfitModel
Summary:
Introduces the OverfitModel for NeRF-style training with overfitting to one scene.
It is a specific case of GenericModel. It has been disentangle to ease usage.

## General modification

1. Modularize a minimum GenericModel to introduce OverfitModel
2. Introduce OverfitModel and ensure through unit testing that it behaves like GenericModel.

## Modularization

The following methods have been extracted from GenericModel to allow modularity with ManyViewModel:
- get_objective is now a call to weighted_sum_losses
- log_loss_weights
- prepare_inputs

The generic methods have been moved to an utils.py file.

Simplify the code to introduce OverfitModel.

Private methods like chunk_generator are now public and can now be used by ManyViewModel.

Reviewed By: shapovalov

Differential Revision: D43771992

fbshipit-source-id: 6102aeb21c7fdd56aa2ff9cd1dd23fd9fbf26315
2023-03-24 07:27:39 -07:00
Jeremy Reizenstein
7d8b029aae increment_version for inplace ops
Summary: For safety checks, make inplace forward operations in cuda and c++ call increment_version.

Reviewed By: davidsonic

Differential Revision: D44302504

fbshipit-source-id: 6ff62251e352d6778cb54399e2e11459e16e77ba
2023-03-23 11:48:36 -07:00
Jeremy Reizenstein
9437768687 conda pytorch2.0.0 builds (#1480)
Summary:
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1480

Nightly builds with PyTorch 2.0.

Reviewed By: shapovalov

Differential Revision: D44135997

fbshipit-source-id: 12b363e2eadbda7a9b6ba9d8db376f41b96d551c
2023-03-21 04:41:37 -07:00
generatedunixname89002005307016
8c8004853a upgrade pyre version in fbcode/vision - batch 2
Differential Revision: D44182699

fbshipit-source-id: bdc5f495eaaee8ce461b91271d476d7b3ec3f8a2
2023-03-17 18:33:54 -07:00
Jeremy Reizenstein
013ff4fd90 doc fix load_point_cloud => load_pointcloud
Summary:
spelling errors in io.md

see https://github.com/facebookresearch/pytorch3d/discussions/1450

Reviewed By: davidsonic

Differential Revision: D43269978

fbshipit-source-id: 4bbe1f00bdeda4c51b7620e7b9cc065840303530
2023-03-09 10:11:13 -08:00
Jeremy Reizenstein
a123815f40 join_pointclouds_as_scene
Summary: New function

Reviewed By: davidsonic

Differential Revision: D42776590

fbshipit-source-id: 2a6e73480bcf2d1749f86bcb22d1942e3e8d3167
2023-03-09 06:51:13 -08:00
Emilien Garreau
d388881f2c Replace relative imports in generic_model.py with absolute ones
Summary: - Replace all the relative imports for generic models to absolute import: (from . import y => from pytorch3.x import y)

Reviewed By: shapovalov

Differential Revision: D43620682

fbshipit-source-id: 937318b339b5020d17b511a891c7b000ff659328
2023-02-28 05:41:37 -08:00
generatedunixname89002005287564
33b49cebf0 fbcode//vision/fair/pytorch3d
Reviewed By: bottler

Differential Revision: D43432438

fbshipit-source-id: 58159b2febb67febb533511eb2d1f47d40dad032
2023-02-20 07:19:15 -08:00
generatedunixname89002005307016
8c2b0b01f8 upgrade pyre version in fbcode/vision - batch 2
Differential Revision: D43044534

fbshipit-source-id: dc841b6704ccd562f5a40e7b2834e26063a9f7ae
2023-02-06 09:05:41 -08:00
generatedunixname89002005307016
d8471b26f2 upgrade pyre version in fbcode/vision - batch 2
Differential Revision: D42947615

fbshipit-source-id: 47b078fdf68567220e15993ab643f85771b0d340
2023-02-01 20:28:20 -08:00
David Novotny
18c38ad600 Fix opencv camera convention docs
Summary: see title

Reviewed By: bottler

Differential Revision: D42920739

fbshipit-source-id: 87f3d052657880b2ef58a1219bb7d4f283ab0830
2023-02-01 05:05:23 -08:00
Jeremy Reizenstein
c8af1c45ca fix broken config for GenericModel
Summary: D42760349 (9540c29023) (make Module.__init__ automatic) didn't account properly for inheritance.

Reviewed By: shapovalov

Differential Revision: D42834466

fbshipit-source-id: 53ee4c788985c1678ad905c06ccf12b2b41361e9
2023-01-29 08:19:57 -08:00
Jeremy Reizenstein
7dfa6918ee transform test fix
Summary: Indexing with a big matrix now fails with a ValueError, possibly because of pytorch improvements. Remove the testcase for it.

Reviewed By: davidsonic

Differential Revision: D42609741

fbshipit-source-id: 0a5a6632ed199cb942bfc4cc4ed347b72e491125
2023-01-28 17:00:41 -08:00
Roman Shapovalov
a7256e4034 Consider the first frame as target ignoring subset labels in evaluator
Summary:
Aligning the logic with the official CO3Dv2 evaluation: 92283c4368/co3d/dataset/utils.py (L7)

This will make the evaluator work with the datasets that do not define known/unseen subsets.

Reviewed By: bottler

Differential Revision: D42803136

fbshipit-source-id: cfac389eab010c32d2e33b40fc7f6ed845c327ef
2023-01-27 08:05:42 -08:00
Jeremy Reizenstein
9540c29023 Make Module.__init__ automatic
Summary: If a configurable class inherits torch.nn.Module and is instantiated, automatically call `torch.nn.Module.__init__` on it before doing anything else.

Reviewed By: shapovalov

Differential Revision: D42760349

fbshipit-source-id: 409894911a4252b7987e1fd218ee9ecefbec8e62
2023-01-27 07:07:46 -08:00
Jeremy Reizenstein
97f8f9bf47 test fix
Reviewed By: shapovalov

Differential Revision: D42780711

fbshipit-source-id: 075fcae5097147b782f7ffc935f5430b824f58fd
2023-01-27 03:56:53 -08:00
Nikhila Ravi
7e750a3786 Update README with links to blog posts (#43)
Summary: Pull Request resolved: https://github.com/fairinternal/pytorch3d/pull/43

Reviewed By: bottler

Differential Revision: D42791756

Pulled By: nikhilaravi

fbshipit-source-id: 498399c1ce30bb095579c4d66b6314a6aa846df3
2023-01-27 01:57:54 -08:00
generatedunixname89002005307016
0b11a5dc6d suppress errors in vision/fair/pytorch3d
Differential Revision: D42775736

fbshipit-source-id: 8425305cd99d50ebc155502d56c0deeee1d078ab
2023-01-26 07:04:18 -08:00
Roman Shapovalov
3239594f78 Fix: Correct concatenation of datasets in train conditioning
Summary: ChainDataset is iterable, and it toes not go along with a custom batch sampler.

Reviewed By: bottler

Differential Revision: D42742315

fbshipit-source-id: 40a715c8d24abe72cb2777634247d7467f628564
2023-01-26 03:00:46 -08:00
Roman Shapovalov
11959e0b24 Subsets in dataset iterators
Summary: For the new API, filtering iterators over sequences by subsets is quite helpful. The change is backwards compatible.

Reviewed By: bottler

Differential Revision: D42739669

fbshipit-source-id: d150a404aeaf42fd04a81304c63a4cba203f897d
2023-01-26 03:00:46 -08:00
Roman Shapovalov
54eb76d48c Loosening the checks in eval script for CO3Dv2 style eval
Summary:
V2 dataset does not have the concept of known/unseen frames. Test-time conditining is done with train-set frames, which violates the previous check.

Also fixing a corner case in VideoWriter.

Reviewed By: bottler

Differential Revision: D42706976

fbshipit-source-id: d43be3dd3060d18cb9f46d5dcf6252d9f084110f
2023-01-26 03:00:46 -08:00
David Novotny
9dc28f5dd5 Fixes for RayBundle plotting
Summary:
Fixes some issues with RayBundle plotting:
- allows plotting raybundles on gpu
- view -> reshape since we do not require contiguous raybundle tensors as input

Reviewed By: bottler, shapovalov

Differential Revision: D42665923

fbshipit-source-id: e9c6c7810428365dca4cb5ec80ef15ff28644163
2023-01-25 01:56:36 -08:00
Jeremy Reizenstein
a12612a48f doc rgbd point cloud
Summary: docstring and shape fix

Reviewed By: shapovalov

Differential Revision: D42609661

fbshipit-source-id: fd50234872ad61b5452821eeb89d51344f70c957
2023-01-24 15:26:52 -08:00
Roman Shapovalov
d561f1913e Cleaning up camera difficulty
Summary: We don’t see much value in reporting metrics by camera difficulty while supporting that in new datasets is quite painful, hence deprecating training cameras in the data API and ignoring in evaluation.

Reviewed By: bottler

Differential Revision: D42678879

fbshipit-source-id: aad511f6cb2ca82745f31c19594e1d80594b61d7
2023-01-23 10:38:56 -08:00
David Novotny
1de2d0c820 render_flyaround allow forwarding args from visualize_reconstruction
Summary: Allows to send kwargs to render_flyaround from visualize_reconstruction

Reviewed By: bottler, shapovalov

Differential Revision: D41157683

fbshipit-source-id: 74d8d7de4e991a31b14e72d76de5efdb8ab4b2c5
2023-01-23 06:52:48 -08:00
myla
b95535c8b7 Fixing bug in rasterizer.py caused by duplicate line (#1421)
Summary:
The file [rasterizer.py](de3a474d2b/pytorch3d/renderer/mesh/rasterizer.py (L201)) contains a duplicate line before the check if the projection_transform exists. This causes an exception in the case that a projection transform matrix is already provided. The corresponding lines should be (and are already) in the else case of the if-statement.

Removing these lines fixes the bug and produces the desired behavior.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1421

Reviewed By: shapovalov

Differential Revision: D42450999

Pulled By: bottler

fbshipit-source-id: f7464e87ec9ff8768455656324b0b008132c8a54
2023-01-19 05:49:56 -08:00
Jeremy Reizenstein
dcced4fa29 test fixes
Reviewed By: shapovalov

Differential Revision: D42545069

fbshipit-source-id: e25fb4049dcebd715df43bab3ce813ecb5f85abe
2023-01-17 06:11:56 -08:00
Jeremy Reizenstein
84851c8312 camera iteration #1408
Summary: Use IndexError so that a camera object is an iterable

Reviewed By: shapovalov

Differential Revision: D42312021

fbshipit-source-id: 67c417d5f1398e8b30a6944468eda057b4ceb444
2023-01-16 08:52:52 -08:00
Jeremy Reizenstein
b7e3b7b16c rendered_mesh_dataset improvements
Summary: Allow choosing the device and the distance

Reviewed By: shapovalov

Differential Revision: D42451605

fbshipit-source-id: 214f02d09da94eb127b3cc308d5bae800dc7b9e2
2023-01-16 07:46:41 -08:00
Jeremy Reizenstein
acc60db4f4 INSTALL.md clarification
Summary: Add fvcore and iopath explicitly. As mentioned in #1416.

Reviewed By: shapovalov

Differential Revision: D42451365

fbshipit-source-id: 0d8d2ead3f15dea6abef221fd5df2b4774abc83f
2023-01-16 07:41:46 -08:00
Moritz Kampelmuehler
aa5b31d857 Add nvidia channel for cudatoolkit (#1402)
Summary:
nvidia channel is required for installing cudatoolkit

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1402

Reviewed By: davidsonic

Differential Revision: D42208583

Pulled By: bottler

fbshipit-source-id: 83d0e63efcf7772b778ca909fd9b14d28829c5b6
2023-01-15 08:57:18 -08:00
yurimalheiros
1b4b67fd7f Minor fixes to the Fit Mesh tutorial (#1423)
Summary:
- Fix the numbers in the headers. Currently, there are no header number 2, the tutorial jump from 1 to 3.
- Clean some unnecessary code.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1423

Reviewed By: shapovalov

Differential Revision: D42478609

Pulled By: bottler

fbshipit-source-id: c49fc10b7d38c3573c92fea737101e6c06bbea38
2023-01-15 08:56:12 -08:00
Jeremy Reizenstein
9fb452db5c builds for pytorch 1.13.1
Reviewed By: davidsonic

Differential Revision: D42315355

fbshipit-source-id: f5ced5270e5a7b4a162ee930fc51dee0469d51fc
2023-01-12 14:27:43 -08:00
Jeremy Reizenstein
3cf3998ea5 assume Python 3.8+
Summary: Remove workarounds for Python 3.7

Reviewed By: davidsonic

Differential Revision: D42451534

fbshipit-source-id: 461dd311f3bccf7bef120ffe0b97fbbd173d95be
2023-01-12 14:25:40 -08:00
Jeremy Reizenstein
d71105f5e5 lints
Summary: lint fixes

Reviewed By: davidsonic

Differential Revision: D42451530

fbshipit-source-id: 120bdd58fc074a713895df15df4e9efa9ea0a420
2023-01-12 14:23:39 -08:00
Jeremy Reizenstein
3388d3f0aa Windows fix for marching cubes #1398
Summary: See https://github.com/facebookresearch/pytorch3d/issues/1398 .

Reviewed By: davidsonic

Differential Revision: D42139493

fbshipit-source-id: 972fc33b9c3017554ce704f2f10190eba406b7c8
2022-12-20 04:07:04 -08:00
Jeremy Reizenstein
3145dd4d16 version 0.7.2
Summary: version number

Reviewed By: davidsonic

Differential Revision: D42072729

fbshipit-source-id: 50adaac4daca7786479d33e218d15e941a725bb6
2022-12-15 16:15:45 -08:00
Jeremy Reizenstein
6c053a6cea Update notebooks for 0.7.2
Summary: Expect pytorch 1.13

Reviewed By: davidsonic

Differential Revision: D42072731

fbshipit-source-id: 13ba4e7de18060dc4e5c0e10f850f17a88750fc9
2022-12-15 16:15:45 -08:00
Jeremy Reizenstein
3bbe26124a README.md and INSTALL.md updates
Summary: misc updates

Reviewed By: davidsonic

Differential Revision: D42072730

fbshipit-source-id: fc885d14346083bd6493f18e9d965b35a8d4bdfd
2022-12-15 16:15:45 -08:00
Jeremy Reizenstein
c773830b88 fix and pythonify conda build (#1394)
Summary:
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1394

The bash logic for building conda packages became fiddly to edit. We need to switch cuda-toolkit to pytorch-cuda when PyTorch>=1.12 which was going to be a pain, so here I rewrite the code in python and do it.

Reviewed By: shapovalov

Differential Revision: D42036406

fbshipit-source-id: 8bb80c2f7545477182b23fc97c8514dcafcee176
2022-12-15 03:22:07 -08:00
Jeremy Reizenstein
b7a316232a fix saved glb length
Summary: Make GLB files report their own length correctly. They were off by 28.

Reviewed By: davidsonic

Differential Revision: D41838340

fbshipit-source-id: 9cd66e8337c142298d5ae1d7c27e51fd812d5c7b
2022-12-13 05:34:45 -08:00
generatedunixname89002005307016
de3a474d2b suppress errors in vision/fair/pytorch3d
Differential Revision: D41897914

fbshipit-source-id: 675f4ad6938bc12d295bbb63a69e3ff98b319a9a
2022-12-09 18:50:58 -08:00
Jeremy Reizenstein
a4a29b013b remove python 3.7 builds
Summary: Python 3.7 not needed any more

Reviewed By: shapovalov

Differential Revision: D41841033

fbshipit-source-id: c0cfd048c70e6b9e47224ab8cddcd6b5f4fc5597
2022-12-09 07:31:39 -08:00
Jeremy Reizenstein
e7851f7c79 Mac builds: python 3.10, pytorch 1.13
Summary: All mac builds now pytorch 1.13

Reviewed By: shapovalov

Differential Revision: D41841035

fbshipit-source-id: b932eb2fefed77ae22f9757f9bd628ce12b11fad
2022-12-09 07:31:39 -08:00
Jeremy Reizenstein
20a774e266 builds for PyTorch 1.13.0
Summary: Add builds for current pytorch

Reviewed By: shapovalov

Differential Revision: D41841034

fbshipit-source-id: 4a58515bd5c83b26f14763a3cec9279e905391d2
2022-12-09 07:31:39 -08:00
Jeremy Reizenstein
91b4c04c64 small readme update
Summary: followup D41534315 (5add065f8a)

Reviewed By: shapovalov

Differential Revision: D41842250

fbshipit-source-id: 54413e30880717f083fdc7acbce2eb3e646ffb94
2022-12-08 09:46:09 -08:00
Jiali Duan
cc2840eb44 Write meshes to GLB
Summary: Write the amalgamated mesh from the Mesh module to glb. In this version, the json header and the binary data specified by the buffer are merged into glb. The image texture attributes are added.

Reviewed By: bottler

Differential Revision: D41489778

fbshipit-source-id: 3af0e9a8f9e9098e73737a254177802e0fb6bd3c
2022-12-05 01:25:43 -08:00
David Novotny
dba48fb410 Add tutorial links to main README.md
Summary: <See title>

Reviewed By: bottler

Differential Revision: D41534604

fbshipit-source-id: 20111db87083b2cce7374cc2bd223ae220d7b010
2022-11-30 07:02:01 -08:00
David Novotny
5add065f8a Readme updates
Summary:
Running:
- clearly points users to experiment.py/visualize_reconstruction.py
Reproducing:
- Adds NeRF training on Blender
- Adds CO3Dv2 configs

Reviewed By: bottler

Differential Revision: D41534315

fbshipit-source-id: e85f5f1eafed8c35c9e91d748a04f238509cf8ec
2022-11-30 07:02:01 -08:00
Jeremy Reizenstein
a2c6af9250 more code blocks for readthedocs
Summary: More rst syntax fixes

Reviewed By: davidsonic

Differential Revision: D40977328

fbshipit-source-id: a3a3accbf2ba7cd9c84a0a82d0265010764a9d61
2022-11-29 03:13:40 -08:00
David Novotny
94f321fa3d render_flyaround bugfix
Summary: Fixes a bug which would crash render_flyaround anytime  visualize_preds_keys is adjusted

Reviewed By: shapovalov

Differential Revision: D41124462

fbshipit-source-id: 127045a91a055909f8bd56c8af81afac02c00f60
2022-11-28 04:36:41 -08:00
David Novotny
35f8cb9430 Downgrade "Assigning param_group " msg to DEBUG
Summary: <See title>

Reviewed By: bottler

Differential Revision: D41534524

fbshipit-source-id: 9c39198b9b8d5fc95f857b03ad39bfe0bd720cbb
2022-11-28 02:58:15 -08:00
David Novotny
c3a6ab02da Bug-fix visualize reconstruction
Summary:
Addresses the following issue:
https://github.com/facebookresearch/pytorch3d/issues/1345#issuecomment-1272881244

I.e., when installed from conda, `pytorch3d_implicitron_visualizer` crashes since it invokes `main()` while `main` requires a single positional arg `argv`.

Reviewed By: shapovalov

Differential Revision: D41533497

fbshipit-source-id: e53a923eb8b2f0f9c0e92e9c0866d9cb310c4799
2022-11-25 10:30:01 -08:00
Jeremy Reizenstein
60ab1cdb72 make x_enabled compulsory
Summary: Optional[some_configurable] won't autogenerate the enabled flag

Reviewed By: shapovalov

Differential Revision: D41522104

fbshipit-source-id: 555ff6b343faf6f18aad2f92fbb7c341f5e991c6
2022-11-24 09:38:02 -08:00
Jiali Duan
1706eb8216 Simplify MC C++ hashing logic
Summary: To be consistent with CUDA hashing, the diff replaces boost hasher with a simplified hasher for storing unique global edge_ids.

Reviewed By: kjchalup

Differential Revision: D41140382

fbshipit-source-id: 2ce598e5edcf6369fe13bd15d1f5e014b252027b
2022-11-15 19:42:04 -08:00
Jiali Duan
8b8291830e Marching Cubes cuda extension
Summary:
Torch CUDA extension for Marching Cubes
- MC involving 3 steps:
  - 1st forward pass to collect vertices and occupied state for each voxel
  - Compute compactVoxelArray to skip non-empty voxels
  - 2nd pass to genereate interpolated vertex positions and faces by marching through the grid
- In contrast to existing MC:
   - Bind each interpolated vertex with a global edge_id to address floating-point precision
   - Added deduplication process to remove redundant vertices and faces

Benchmarks (ms):

| N / V(^3)      | python          | C++             |   CUDA   | Speedup |
| 2 / 20          |    12176873  |       24338     |     4363   | 2790x/5x|
| 1 / 100          |     -             |    3070511     |   27126   |    113x    |
| 2 / 100          |     -             |    5968934     |   53129   |    112x    |
| 1 / 256          |     -             |  61278092     | 430900   |    142x    |
| 2 / 256          |     -             |125687930     | 856941   |    146x   |

Reviewed By: kjchalup

Differential Revision: D39644248

fbshipit-source-id: d679c0c79d67b98b235d12296f383d760a00042a
2022-11-15 19:42:04 -08:00
Jeremy Reizenstein
9a0b0c2e74 renderer and vis readthedocs #1363
Summary: Autogenerate docs for the renderer too. This will be helpful but make a slightly ugly TOC

Reviewed By: kjchalup

Differential Revision: D40977315

fbshipit-source-id: 10831de3ced68080cb5671c5dc31d4da8500f761
2022-11-15 14:10:22 -08:00
Daniel L. Lu
d0fbe2cb63 fix typo pucuda.gl --> pycuda.gl (#1379)
Summary:
Every time I try to run code, I get this warning:

```
  warnings.warn("Can't import pucuda.gl, not importing MeshRasterizerOpenGL.")
```

Of course, `pucuda` is a typo of `pycuda`.

This PR fixes the typo

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1379

Reviewed By: kjchalup

Differential Revision: D41295562

Pulled By: bottler

fbshipit-source-id: 2bfa2a2dbe20a5347861d36fbff5094994c1253d
2022-11-15 12:02:44 -08:00
Roman Shapovalov
719c33a7f9 Fix: visualisation with Enums fields in Configurables
Summary:
Enum fields cause the following to crash since they are loaded as strings:
```
config = OmegaConf.load(autodumped_cfg_file)
Experiment(**config)
```

It would be good to come up with the general solution but for now just fixing the visualisation script.

Reviewed By: bottler

Differential Revision: D41140426

fbshipit-source-id: 71c1c6b1fffe3b5ab1ca0114cfa3f0d81160278f
2022-11-08 15:58:41 -08:00
Roman Shapovalov
f3c1e0837c MC rasterize supports heterogeneous bundle; refactoring of bundle-to-padded
Summary:
Rasterize MC was not adapted to heterogeneous bundles.

There are some caveats though:
1) on CO3D, we get up to 18 points per image, which is too few for a reasonable visualisation (see below);
2) rasterising for a batch of 100 is slow.

I also moved the unpacking code close to the bundle to be able to reuse it.

{F789678778}

Reviewed By: bottler, davnov134

Differential Revision: D41008600

fbshipit-source-id: 9f10f1f9f9a174cf8c534b9b9859587d69832b71
2022-11-07 13:43:31 -08:00
Jeremy Reizenstein
7be49bf46f allow dots in param_groups
Summary:
Allow a module's param_group member to specify overrides to the param groups of its members or their members.
Also logging for param group assignments.

This allows defining `params.basis_matrix` in the param_groups of a voxel_grid.

Reviewed By: shapovalov

Differential Revision: D41080667

fbshipit-source-id: 49f3b0e5b36e496f78701db0699cbb8a7e20c51e
2022-11-07 06:41:40 -08:00
Jeremy Reizenstein
a1f2ded58a voxel_grid_implicit_function scaffold fixes
Summary: Fix indexing of directions after filtering of points by scaffold.

Reviewed By: shapovalov

Differential Revision: D40853482

fbshipit-source-id: 9cfdb981e97cb82edcd27632c5848537ed2c6837
2022-11-03 05:46:31 -07:00
David Novotny
e4a3298149 CO3Dv2 multi-category extension
Summary:
Allows loading of multiple categories.
Multiple categories are provided in a comma-separated list of category names.

Reviewed By: bottler, shapovalov

Differential Revision: D40803297

fbshipit-source-id: 863938be3aa6ffefe9e563aede4a2e9e66aeeaa8
2022-11-02 13:55:25 -07:00
Jeremy Reizenstein
c54e048666 more readthedocs
Summary: Quote formats, spelling

Reviewed By: shapovalov

Differential Revision: D40913734

fbshipit-source-id: d6dea65d5204b3c463c656a07ef9b447b7be6a0a
2022-11-02 05:19:15 -07:00
David Novotny
f7ac7b604a readthedocs fixes
Summary: Fixes readthedocs. Sphinx build looks good.

Reviewed By: bottler

Differential Revision: D40893196

fbshipit-source-id: bf00384b921d4ef54e64745ed39172358c2f9bb3
2022-11-01 05:46:13 -07:00
Jeremy Reizenstein
cd113efd98 Add implicitron to readthedocs
Summary: Try to document implicitron. Most of this is autogenerated.

Reviewed By: shapovalov

Differential Revision: D40623742

fbshipit-source-id: 453508277903b7d987b1703656ba1ee09bc2c570
2022-10-31 14:44:41 -07:00
Jeremy Reizenstein
322c8f2f50 make lpips optional
Summary: So that lpips isn't needed in readthedocs

Reviewed By: shapovalov

Differential Revision: D40859351

fbshipit-source-id: 58933bd3b71e78e658fbce56c3663b11ee2331f6
2022-10-31 14:44:41 -07:00
David Novotny
eff0aad15a Bugfix - normalize emitted ray directions after substraction
Summary: The bug lead to non-coinciding origins of the rays emitted from perspective cameras when unit_directions=True

Reviewed By: bottler

Differential Revision: D40865610

fbshipit-source-id: 398598e9e919b53e6bea179f0400e735bbb5b625
2022-10-31 14:00:49 -07:00
David Novotny
bea84a6fcd voxel_grid_if -> remove union type from the settings.
Summary: see title

Reviewed By: shapovalov

Differential Revision: D40803670

fbshipit-source-id: 211189167837af577d6502a698e2f3fb3aec3e30
2022-10-31 11:37:43 -07:00
Roman Shapovalov
f711c4bfe9 Fix parameters not wrapped with nn.Parameter, antialiasing compatibility
Summary: Some things fail if a parameter is not wraped; in particular, it prevented other tensors moving to GPU.

Reviewed By: bottler

Differential Revision: D40819932

fbshipit-source-id: a23b38ceacd7f0dc131cb0355fef1178e3e2f7fd
2022-10-31 01:43:00 -07:00
Jeremy Reizenstein
88620b6847 attempt to fix doc build #1363
Summary: installing from git: is failing

Reviewed By: shapovalov

Differential Revision: D40635668

fbshipit-source-id: 604ba5425e14caeabe4e178bf7f851f2163419bd
2022-10-27 07:26:36 -07:00
generatedunixname89002005307016
db7c80bf76 upgrade pyre version in fbcode/vision - batch 2
Differential Revision: D40695068

fbshipit-source-id: 9944a872d93a9dac348b252a0f8bf9c7e6f929f3
2022-10-25 22:42:06 -07:00
Jeremy Reizenstein
995b60e3b9 version 0.7.1
Summary: update version number

Reviewed By: shapovalov

Differential Revision: D40622583

fbshipit-source-id: 87fa55e1b02fc114f65ee8a5c3d998ba50226ab1
2022-10-23 07:16:40 -07:00
Jeremy Reizenstein
ca58863347 yaml test fix
Summary: Yaml bool case fix

Reviewed By: shapovalov

Differential Revision: D40623031

fbshipit-source-id: 29b2fba171c2cbebfa03834e38b614d07275c997
2022-10-23 07:09:26 -07:00
Jeremy Reizenstein
74754bbf17 voxel_grid_implicit_function
Reviewed By: shapovalov

Differential Revision: D40622304

fbshipit-source-id: 277515a55c46d9b8300058b439526539a7fe00a0
2022-10-23 05:36:34 -07:00
Jeremy Reizenstein
611aba9a20 replicate_last_interval in raymarcher
Summary: Add option to flat pad the last delta. Might to help when training on rgb only.

Reviewed By: shapovalov

Differential Revision: D40587475

fbshipit-source-id: c763fa38948600ea532c730538dc4ff29d2c3e0a
2022-10-23 02:47:09 -07:00
Jeremy Reizenstein
ff933ab82b make visdom optional
Summary: Make Implicitron run without visdom installed.

Reviewed By: shapovalov

Differential Revision: D40587974

fbshipit-source-id: dc319596c7a4d10a4c54c556dabc89ad9d25c2fb
2022-10-22 15:51:22 -07:00
Jiali Duan
46cb5aaaae Omit _check_valid_rotation_matrix by default
Summary:
According to the profiler trace D40326775, _check_valid_rotation_matrix is slow because of aten::all_close operation and _safe_det_3x3 bottlenecks. Disable the check by default unless environment variable PYTORCH3D_CHECK_ROTATION_MATRICES is set to 1.

Comparison after applying the change:
```
Profiling/Function    get_world_to_view (ms)   Transform_points(ms)    specular(ms)
before                12.751                    18.577                  21.384
after                 4.432 (34.7%)             9.248 (49.8%)           11.507 (53.8%)
```

Profiling trace:
https://pxl.cl/2h687
More details in https://docs.google.com/document/d/1kfhEQfpeQToikr5OH9ZssM39CskxWoJ2p8DO5-t6eWk/edit?usp=sharing

Reviewed By: kjchalup

Differential Revision: D40442503

fbshipit-source-id: 954b58de47de235c9d93af441643c22868b547d0
2022-10-20 16:05:22 -07:00
Jeremy Reizenstein
8339cf2610 hydra/OC.structured: don't hide error
Summary: Keep the cause of hydra errors visible in some more cases.

Reviewed By: shapovalov

Differential Revision: D40516202

fbshipit-source-id: 8d214be5cc808a37738add77cc305fe099788546
2022-10-20 03:43:05 -07:00
Jeremy Reizenstein
9535c576e0 test fix for param_groups
Summary: param_groups only expected on MLPDecoder, not ElementwiseDecoder

Reviewed By: shapovalov

Differential Revision: D40508539

fbshipit-source-id: ea040ad6f7e26bd7d87e5de2eaadae2cf4b04faf
2022-10-19 04:08:30 -07:00
Jeremy Reizenstein
fe5bdb2fb5 different learning rate for different parts
Summary:
Adds the ability to have different learning rates for different parts of the model. The trainable parts of the implicitron have a new member

       param_groups: dictionary where keys are names of individual parameters,
            or module’s members and values are the parameter group where the
            parameter/member will be sorted to. "self" key is used to denote the
            parameter group at the module level. Possible keys, including the "self" key
            do not have to be defined. By default all parameters are put into "default"
            parameter group and have the learning rate defined in the optimizer,
            it can be overriden at the:
                - module level with “self” key, all the parameters and child
                    module s parameters will be put to that parameter group
                - member level, which is the same as if the `param_groups` in that
                    member has key=“self” and value equal to that parameter group.
                    This is useful if members do not have `param_groups`, for
                    example torch.nn.Linear.
                - parameter level, parameter with the same name as the key
                    will be put to that parameter group.

And in the optimizer factory, parameters and their learning rates are recursively gathered.

Reviewed By: shapovalov

Differential Revision: D40145802

fbshipit-source-id: 631c02b8d79ee1c0eb4c31e6e42dbd3d2882078a
2022-10-18 15:58:18 -07:00
Jeremy Reizenstein
a819ecb00b MLP last layer config
Summary:
Added initialization configuration for the last layer of the MLP decoding function. You can now set:
- last activation function (tensorf uses sigmoid)
- last bias init (tensorf uses 0, because of sigmoid ofc)
- option to use xavier initialization (we use relu so this should not be set)

Reviewed By: davnov134

Differential Revision: D40304981

fbshipit-source-id: ec398eb2235164ae85cb7c09b9660e843490ea04
2022-10-18 15:58:18 -07:00
Ji Hou
a2659e1730 Update iou3d.md (#1351)
Summary:
fix a typo

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1351

Reviewed By: shapovalov

Differential Revision: D40209834

Pulled By: bottler

fbshipit-source-id: 118133e0eab2df211e5c4f04371f2c695a9ceced
2022-10-15 21:41:02 -07:00
Jeremy Reizenstein
3b3306f9b4 suppress ffmpeg output
Summary: Restore the suppression of ffmpeg output.

Reviewed By: shapovalov

Differential Revision: D40296595

fbshipit-source-id: 41b2c14b6f6245f77e0ef6cc94fa7b41fbb83e33
2022-10-13 14:56:09 -07:00
Jeremy Reizenstein
f13086779d avoid torch.range
Summary: Avoid unintended use of torch.range.

Reviewed By: kjchalup

Differential Revision: D40341396

fbshipit-source-id: 108295983afdec0ca9e43178fef9c65695150bc1
2022-10-13 14:37:46 -07:00
Jeremy Reizenstein
4d9215b3b4 fix to get_default_args(instance)
Summary:
Small config system fix. Allows get_default_args to work on an instance which has been created with a dict (instead of a DictConfig) as an args field. E.g.

```
gm = GenericModel(
        raysampler_AdaptiveRaySampler_args={"scene_extent": 4.0}
    )
    OmegaConf.structured(gm1)
```

Reviewed By: shapovalov

Differential Revision: D40341047

fbshipit-source-id: 587d0e8262e271df442a80858949a48e5d6db3df
2022-10-13 06:05:07 -07:00
Darijan Gudelj
76cddd90be Elementwise decoder
Summary: Tensorf does relu or softmax after the density grid. This diff adds the ability to replicate that.

Reviewed By: bottler

Differential Revision: D40023228

fbshipit-source-id: 9f19868cd68460af98ab6e61c7f708158c26dc08
2022-10-13 05:59:22 -07:00
Jeremy Reizenstein
a607dd063e better implicit function #channels errors
Summary: More helpful errors when the output channels aren't 1 for density and 3 for color

Reviewed By: shapovalov

Differential Revision: D40341088

fbshipit-source-id: 6074bf7fefe11c8e60fee4db2760b776419bcfee
2022-10-13 05:43:36 -07:00
Krzysztof Chalupka
17bc043a0c Remove structured binding
Summary: Couldn't build p3d on devfair because C++17 is unsupported. Two structured bindings sneaked in.

Reviewed By: bottler

Differential Revision: D40280967

fbshipit-source-id: 9627f3f9f76247a6cefbeac067fdead67c6f4e14
2022-10-12 10:34:21 -07:00
Darijan Gudelj
f55d37f07d volume cropping
Summary:
TensoRF at step 2000 does volume croping and resizing.
At those steps it calculates part of the voxel grid which has density big enough to have objects and resizes the grid to fit that object.
Change is done on 3 levels:
- implicit function subscribes to epochs and at specific epochs finds the bounding box of the object and calls resizing of the color and density voxel grids to fit it
- VoxelGrid module calls cropping of the underlaying voxel grid and resizing to fit previous size it also adjusts its extends and translation to match wanted size
- Each voxel grid has its own way of cropping the underlaying data

Reviewed By: kjchalup

Differential Revision: D39854548

fbshipit-source-id: 5435b6e599aef1eaab980f5421d3369ee4829c50
2022-10-12 08:31:51 -07:00
Jeremy Reizenstein
0b5def5257 avoid numpy warning in split
Summary:
avoid creating a numpy array of random things just to split it: this can now generate a warning e.g. if the list contains lists of varying lengths. There might also be a performance win here, and we could do more of the same if we care about that.

(The vanilla way to avoid the new warning is to replace `np.split(a,` with `np.split(np.array(a, dtype=object), ` btw.)

Reviewed By: shapovalov

Differential Revision: D40209308

fbshipit-source-id: daae33a23ceb444e8e7241f72ce1525593e2f239
2022-10-11 14:49:51 -07:00
Darijan Gudelj
56d3465b09 scaffold
Summary: Forward method is sped up using the scaffold, a low resolution voxel grid which is used to filter out the points in empty space. These points will be predicted as having 0 density and (0, 0, 0) color. The points which were not evaluated as empty space will be passed through the steps outlined above.

Reviewed By: kjchalup

Differential Revision: D39579671

fbshipit-source-id: 8eab8bb43ef77c2a73557efdb725e99a6c60d415
2022-10-10 11:01:00 -07:00
Jeremy Reizenstein
95a2acf763 Co3Dv2 point cloud fix
Summary: Avoid certain hardcoded paths in co3dv2 data

Reviewed By: davnov134

Differential Revision: D40209309

fbshipit-source-id: 0e83a15baa47d5bd07d2d23c6048cb4522c1ccba
2022-10-09 05:06:49 -07:00
Jiali Duan
9df875bb5e Fix Circleci build failure: Add != operator for Marching Cubes Vertex struct
Summary: Fix Circleci error: https://app.circleci.com/pipelines/github/facebookresearch/pytorch3d/1066/workflows/94df137b-4882-4959-8fe4-738af447db23/jobs/56560.

Reviewed By: kjchalup

Differential Revision: D40185409

fbshipit-source-id: 7121b0cae66bd60f718df2a5d9ef5d2ac3bc658c
2022-10-07 12:14:24 -07:00
David Novotny
73ba66e4ab Bugfix in reduction feature aggr.
Summary: Bugfix

Reviewed By: bottler

Differential Revision: D40146840

fbshipit-source-id: a8415c361ed3cd939999b87311aff4d3bb1bcfe1
2022-10-07 01:45:24 -07:00
Kyle Hundman
4c8338b00f Improve memory efficiency in VolumeSampler
Summary: Avoids use of `torch.cat` operation when rendering a volume by instead issuing multiple calls to `torch.nn.functional.grid_sample`. Density and color tensors can be large.

Reviewed By: bottler

Differential Revision: D40072399

fbshipit-source-id: eb4cd34f6171d54972bbf2877065f973db497de0
2022-10-06 12:10:26 -07:00
Jiali Duan
0d8608b9f9 Marching Cubes C++ torch extension
Summary:
Torch C++ extension for Marching Cubes

- Add torch C++ extension for marching cubes. Observe a speed up of ~255x-324x speed up (over varying batch sizes and spatial resolutions)

- Add C++ impl in existing unit-tests.

(Note: this ignores all push blocking failures!)

Reviewed By: kjchalup

Differential Revision: D39590638

fbshipit-source-id: e44d2852a24c2c398e5ea9db20f0dfaa1817e457
2022-10-06 11:13:53 -07:00
Jiali Duan
850efdf706 Python marching cubes improvements
Summary: Overhaul of marching_cubes_naive for better performance and to avoid relying on unstable hashing. In particular, instead of hashing vertex positions, we index each interpolated vertex with its corresponding edge in the 3d grid.

Reviewed By: kjchalup

Differential Revision: D39419642

fbshipit-source-id: b5fede3525c545d1d374198928dfb216262f0ec0
2022-10-06 11:08:49 -07:00
Gavin Peng
6471893f59 Multithread CPU naive mesh rasterization
Summary:
Threaded the for loop:
```
for (int yi = 0; yi < H; ++yi) {...}
```
in function `RasterizeMeshesNaiveCpu()`.
Chunk size is approx equal.

Reviewed By: bottler

Differential Revision: D40063604

fbshipit-source-id: 09150269405538119b0f1b029892179501421e68
2022-10-06 06:42:58 -07:00
Darijan Gudelj
37bd280d19 load whole dataset in train loop
Summary: Loads the whole dataset and moves it to the device and sends it to for sampling to enable full dataset heterogeneous raysampling.

Reviewed By: bottler

Differential Revision: D39263009

fbshipit-source-id: c527537dfc5f50116849656c9e171e868f6845b1
2022-10-03 08:36:47 -07:00
Darijan Gudelj
c311a4cbb9 Enable mixed frame raysampling
Summary:
Changed ray_sampler and metrics to be able to use mixed frame raysampling.

Ray_sampler now has a new member which it passes to the pytorch3d raysampler.
If the raybundle is heterogeneous metrics now samples images by padding xys first. This reduces memory consumption.

Reviewed By: bottler, kjchalup

Differential Revision: D39542221

fbshipit-source-id: a6fec23838d3049ae5c2fd2e1f641c46c7c927e3
2022-10-03 08:36:47 -07:00
Darijan Gudelj
ad8907d373 ImplicitronRayBundle
Summary: new implicitronRayBundle with added cameraIDs and camera counts. Added to enable a single raybundle inside Implicitron and easier extension in the future. Since RayBundle is named tuple and RayBundleHeterogeneous is dataclass and RayBundleHeterogeneous cannot inherit RayBundle. So if there was no ImplicitronRayBundle every function that uses RayBundle now would have to use Union[RayBundle, RaybundleHeterogeneous] which is confusing and unecessary complicated.

Reviewed By: bottler, kjchalup

Differential Revision: D39262999

fbshipit-source-id: ece160e32f6c88c3977e408e966789bf8307af59
2022-10-03 08:36:47 -07:00
Darijan Gudelj
6ae863f301 Heterogeneous raysampling -> RayBundleHeterogeneous
Summary:
Added heterogeneous raysampling to pytorch3d raysampler, different cameras are sampled different number of times.

 It now returns RayBundle if heterogeneous raysampling is off and new RayBundleHeterogeneous (with added fields `camera_ids` and `camera_counts`).  Heterogeneous raysampling is on if `n_rays_total` is not None.

Reviewed By: bottler

Differential Revision: D39542222

fbshipit-source-id: d3d88d822ec7696e856007c088dc36a1cfa8c625
2022-09-30 04:03:01 -07:00
Roman Shapovalov
9a0f9ae572 Extending the API of Transform3d with SE(3) log
Summary:
This is quite a thin wrapper – not sure we need it. The motivation is that `Transform3d` is not as matrix-centric now, it can be converted to SE(3) logarithm equally easily.

It simplifies things like averaging cameras and getting axis-angle of camera rotation (previously, one would need to call `se3_log_map(cameras.get_world_to_camera_transform().get_matrix())`), now one fewer thing to call / discover.

Reviewed By: bottler

Differential Revision: D39928000

fbshipit-source-id: 85248d5b8af136618f1d08791af5297ea5179d19
2022-09-29 11:56:14 -07:00
Roman Shapovalov
74bbd6fd76 Fix returning a proper rotation in levelling; supporting batches and default centroid
Summary:
`get_rotation_to_best_fit_xy` is useful to expose externally, however there was a bug (which we probably did not care about for our use case): it could return a rotation matrix with det(R) == −1.
The diff fixes that, and also makes centroid optional (it can be computed from points).

Reviewed By: bottler

Differential Revision: D39926791

fbshipit-source-id: 5120c7892815b829f3ddcc23e93d4a5ec0ca0013
2022-09-29 11:56:14 -07:00
generatedunixname89002005307016
de98c9cc2f upgrade pyre version in fbcode/vision - batch 2
Differential Revision: D39894833

fbshipit-source-id: 95a32d9cb352c5fea345c6947194ad52971d4fe9
2022-09-28 20:07:00 -07:00
Darijan Gudelj
5005f09118 Add rescaling to voxel grids
Summary: Any module can be subscribed to step updates from the training loop. Once the training loop publishes a step the voxel grid changes its dimensions. During the construction of VoxelGridModule and its parameters it does not know which is the resolution that will be loaded from checkpoint, so before the checkpoint loading a hook runs which changes the VoxelGridModule's parameters to match shapes of the loaded checkpoint.

Reviewed By: bottler

Differential Revision: D39026775

fbshipit-source-id: 0d359ea5c8d2eda11d773d79c7513c83585d5f17
2022-09-28 05:23:22 -07:00
Jeremy Reizenstein
efea540bbc Fix camera clone() with torch.save
Summary:
User reported that cloned cameras fail to save. The error with latest PyTorch is

```
pickle.PicklingError: Can't pickle ~T_destination: attribute lookup T_destination on torch.nn.modules.module failed
```

This fixes it.

Reviewed By: btgraham

Differential Revision: D39692258

fbshipit-source-id: 75bbf3b8dfa0023dc28bf7d4cc253ca96e46a64d
2022-09-22 14:52:09 -07:00
Michaël Ramamonjisoa
ce3fce49d7 Adding a Checkerboard mesh utility to Pytorch3d
Summary: Adding a checkerboard mesh utility to Pytorch3d.

Reviewed By: bottler

Differential Revision: D39718916

fbshipit-source-id: d43cd30e566b5db068bae6eed0388057634428c8
2022-09-22 12:51:37 -07:00
Darijan Gudelj
f34da3d3b6 packed_to_padded now accepts all sizes
Summary:
We need to make packing/unpacking in 2 places for mixed frame raysampling (metrics and raysampler) but those tensors that need to be unpacked/packed have more than two dimensions.
I could have reshaped and stored dimensions but this seems to just complicate code there with something which packed_to_padded should support.
I could have made a separate function for implicitron but it would confusing to have two different padded_to_packed functions inside pytorch3d codebase one of which does packing for (b, max) and (b, max, f) and the other for (b, max, …)

Reviewed By: bottler

Differential Revision: D39729026

fbshipit-source-id: 2bdebf290dcc6c316b7fe1aeee49bbb5255e508c
2022-09-22 11:27:43 -07:00
Darijan Gudelj
c2d876c9e8 voxel grid implicit function
Summary: The implicit function and its members and internal working

Reviewed By: kjchalup

Differential Revision: D38829764

fbshipit-source-id: 28394fe7819e311ed52c9defc9a1b29f37fbc495
2022-09-22 10:56:00 -07:00
Jeremy Reizenstein
d6a197be36 make expand_args_fields optional
Summary: Call expand_args_field when instantiating an object.

Reviewed By: shapovalov

Differential Revision: D39541931

fbshipit-source-id: de8e1038927ff0112463394412d5d8c26c4a1e17
2022-09-22 08:36:09 -07:00
Jeremy Reizenstein
209c160a20 foreach optimizers
Summary: Allow using the new `foreach` option on optimizers.

Reviewed By: shapovalov

Differential Revision: D39694843

fbshipit-source-id: 97109c245b669bc6edff0f246893f95b7ae71f90
2022-09-22 05:11:56 -07:00
Darijan Gudelj
db3c12abfb arbitrary shape input to voxel_grids
Summary: Add the ability to process arbitrary point shapes `[n_grids, ..., 3]` instead of  only `[n_grids, n_points, 3]`.

Reviewed By: bottler

Differential Revision: D39574373

fbshipit-source-id: 0a9ecafe9ea58cd8f909644de43a1185ecf934f4
2022-09-22 03:35:11 -07:00
Michaël Ramamonjisoa
6ae6ff9cf7 include TexturesUV in IO.save_mesh(x.obj)
Summary:
Added export of UV textures to IO.save_mesh in Pytorch3d
MeshObjFormat now passes verts_uv, faces_uv, and texture_map as input to save_obj

TODO: check if TexturesUV.verts_uv_list or TexturesUV.verts_uv_padded() should be passed to save_obj

IO.save_mesh(obj_file, meshes, decimal_places=2) should be IO().save_mesh(obj_file, meshes, decimal_places=2)

Reviewed By: bottler

Differential Revision: D39617441

fbshipit-source-id: 4628b7f26f70e38c65f235852b990c8edb0ded23
2022-09-21 06:16:48 -07:00
Jeremy Reizenstein
305cf32f6b Avoid raysampler dict
Summary:
A significant speedup (e.g. >2% of a forward pass).

Move NDCMultinomialRaysampler parts of AbstractMaskRaySampler to members instead of living in a dict. The dict was hiding them from the nn.Module system so their _xy_grid members were remaining on the CPU. Therefore they were being copied to the GPU in every forward pass.

(We couldn't easily use a ModuleDict here because the enum keys are not strs.)

Reviewed By: shapovalov

Differential Revision: D39668589

fbshipit-source-id: 719b88e4a08fd7263a284e0ab38189e666bd7e3a
2022-09-21 04:29:44 -07:00
Jeremy Reizenstein
da7fe2854e small fixes to config
Summary:
- indicate location of OmegaConf.structured failures
- split the data gathering from enable_get_default_args to ease experimenting with it.
- comment fixes.
- nicer error when a_class_type has weird type.

Reviewed By: kjchalup

Differential Revision: D39434447

fbshipit-source-id: b80c7941547ca450e848038ef5be95b7ebbe8f3e
2022-09-15 03:03:52 -07:00
Jeremy Reizenstein
cb7bd33e7f validate lengths in chamfer and farthest_points
Summary: Fixes #1326

Reviewed By: kjchalup

Differential Revision: D39259697

fbshipit-source-id: 51392f4cc4a956165a62901cb115fcefe0e17277
2022-09-08 15:03:36 -07:00
Jeremy Reizenstein
6e25fe8cb3 visualize_reconstruction fixes
Summary: Various fixes to get visualize_reconstruction running, and an interactive test for it.

Reviewed By: kjchalup

Differential Revision: D39286691

fbshipit-source-id: 88735034cc01736b24735bcb024577e6ab7ed336
2022-09-07 20:10:07 -07:00
Jeremy Reizenstein
34ad77b841 trainer test completion
Summary: allow TESTIT to complete properly

Reviewed By: kjchalup

Differential Revision: D39280546

fbshipit-source-id: 38fe69988a736e32dbe78d1d05e6d8421353854a
2022-09-07 14:21:15 -07:00
Jeremy Reizenstein
90b758f725 hydra fix
Summary: Workaround for oddity with new hydra.

Reviewed By: davnov134

Differential Revision: D39280639

fbshipit-source-id: 76e91947f633589945446db93cf2dbc259642f8a
2022-09-07 03:25:39 -07:00
Jeremy Reizenstein
df36223ddf raysampling stratified_sampling doc
Summary: Followup to D39259775 (438c194ec6)

Reviewed By: davnov134

Differential Revision: D39271753

fbshipit-source-id: 5cf11c1210369e1762ef0d5d0c7b60336711b261
2022-09-06 10:28:45 -07:00
David Novotny
73ea4187de SimpleDataLoaderMapProvider sample batches without replacement if num_samples is not specified
Summary: Samples batches without replacement if the number of samples is not specified. This makes sure that we always iterate over the whole dataset in each epoch.

Reviewed By: bottler

Differential Revision: D39270786

fbshipit-source-id: 0c983d1f5e0af711463abfb23939bc0d2b5172a0
2022-09-06 05:59:01 -07:00
David Novotny
f6d43eaa62 Bugfixes in render_flyaround
Summary: Fixes bugs in render_flyaround

Reviewed By: bottler

Differential Revision: D39271932

fbshipit-source-id: 07e6c9ee07ba91feb437b725af0a8942fd98db0b
2022-09-06 05:45:31 -07:00
David Novotny
c79c954dea Rename and move render_flyaround into core implicitron
Summary:
Move the flyaround rendering function into core implicitron.
The unblocks an example in the facebookresearch/co3d repo.

Reviewed By: bottler

Differential Revision: D39257801

fbshipit-source-id: 6841a88a43d4aa364dd86ba83ca2d4c3cf0435a4
2022-09-06 03:03:08 -07:00
Matthias Treder
438c194ec6 stratified_sampling argument: set default to None (#1324)
Summary:
The self._stratified_sampling attribute is always overridden unless stratified_sampling is explicitly set to None. However, the desired default behavior is that the value of self._stratified_sampling is used unless the argument stratified_sampling is set to True/False. Changing the default to None achieves this

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1324

Reviewed By: bottler

Differential Revision: D39259775

Pulled By: davnov134

fbshipit-source-id: e01bb747ac80c812eb27bf22e67f5e14f29acadd
2022-09-05 11:16:25 -07:00
Darijan Gudelj
dd58ded73d stats object counting time wrong
Summary: On each call of the stats.update the object calculates current average iteration time by getting time elapsed from the time_start and then dividing it by the current number of steps. It saves the result to AverageMeter object which when queried returns the average of things saved, so the time is averaged twice which biases it towards the start value (which is often larger).

Reviewed By: kjchalup

Differential Revision: D39206989

fbshipit-source-id: ccab5233d7aaca1ac4fd626fb329b83c7c0d6af9
2022-09-05 06:27:37 -07:00
Darijan Gudelj
72c3a0ebe5 raybundle input to ImplicitFunctions -> api unification
Summary: Currently some implicit functions in implicitron take a raybundle, others take ray_points_world. raybundle is what they really need. However, the raybundle is going to become a bit more flexible later, as it will contain different numbers of rays for each camera.

Reviewed By: bottler

Differential Revision: D39173751

fbshipit-source-id: ebc038e426d22e831e67a18ba64655d8a61e1eb9
2022-09-05 06:26:06 -07:00
Pyre Bot Jr
70dc9c451a suppress errors in vision/fair/pytorch3d
Reviewed By: stroxler

Differential Revision: D39230408

fbshipit-source-id: dce7a461507ee7199f588341773096c06051b2dc
2022-09-03 09:22:49 -07:00
Jiali Duan
8a96770dc2 Update the docstring for try_get_projection_transform
Summary: Update the docstring for try_get_projection_transform on the API design.

Reviewed By: kjchalup

Differential Revision: D39227333

fbshipit-source-id: c9d0e625735d4972116d1f71865fb9b763e684de
2022-09-02 13:44:08 -07:00
Jiali Duan
84fa966643 Fix the typing for a Camera helper function
Summary: Fixed the typing for try_get_projection_transform.

Reviewed By: kjchalup

Differential Revision: D39211811

fbshipit-source-id: ef05c6b545831d1a9f3c754aeb02fb9776e360ed
2022-09-01 12:37:02 -07:00
Pyre Bot Jr
c80e5fd07a suppress errors in vision/fair/pytorch3d
Reviewed By: kjchalup

Differential Revision: D39198333

fbshipit-source-id: 3f4ebcf625215f21d165073837578ff69b05f72d
2022-09-01 11:46:55 -07:00
Jiali Duan
d19e6243d0 Update Rasterizer and add end2end fisheye integration test
Summary:
1) Update rasterizer/point rasterizer to accommodate fisheyecamera. Specifically, transform_points is in placement of explicit transform compositions.

2) In rasterizer unittests, update corresponding tests for rasterizer and point_rasterizer. Address comments to test fisheye against perspective camera when distortions are turned off.

3) Address comments to add end2end test for fisheyecameras. In test_render_meshes, fisheyecameras are added to camera enuerations whenever possible.

4) Test renderings with fisheyecameras of different params on cow mesh.

5) Use compositions for linear cameras whenever possible.

Reviewed By: kjchalup

Differential Revision: D38932736

fbshipit-source-id: 5b7074fc001f2390f4cf43c7267a8b37fd987547
2022-08-31 16:50:41 -07:00
Jiali Duan
b0515e1461 Amend D38407094: Fisheye Camera for PyTorch3D
Summary:
Amend FisheyeCamera by adding tests for all
combination of params and for different batch_sizes.

Reviewed By: kjchalup

Differential Revision: D39176747

fbshipit-source-id: 830d30da24beeb2f0df52db0b17a4303ed53b59c
2022-08-31 13:50:50 -07:00
Chris Lambert
d4a1051e0f Remove pytorch3d's wrappers for eigh, solve, lstsq, qr
Summary: Remove the compat functions eigh, solve, lstsq, and qr. Migrate callers to use torch.linalg directly.

Reviewed By: bottler

Differential Revision: D39172949

fbshipit-source-id: 484230a553237808f06ee5cdfde64651cba91c4c
2022-08-31 13:04:07 -07:00
Jeremy Reizenstein
9a1213e0e5 test order fix
Summary: A dummy value in test_opengl_utils seems to be able to break tests in test_mesh_renderer_opengl{,_to}.

Reviewed By: kjchalup

Differential Revision: D39173275

fbshipit-source-id: 83b15159f70135ea575d5085c7b6b37badd6e49e
2022-08-31 11:47:51 -07:00
Krzysztof Chalupka
95771985b7 Make PyTorch3D C++17 incompatible again :(
Summary: D38919607 (c4545a7cbc) and D38858887 (06cbba2628) were premature, turns out CUDA 10.2 doesn't support C++17.

Reviewed By: bottler

Differential Revision: D39156205

fbshipit-source-id: 5e2e84cc4a57d1113a915166631651d438540d56
2022-08-31 02:38:31 -07:00
Sergii Dymchenko
1530a66469 Update deprecated torch.symeig in vision/fair/pytorch3d/projects/nerf/nerf/eval_video_utils.py
Summary:
torch.symeig is deprecated for a long time and is being removed by https://github.com/pytorch/pytorch/pull/70988.

Created from CodeHub with https://fburl.com/edit-in-codehub

Reviewed By: bottler

Differential Revision: D39153103

fbshipit-source-id: 3a1397b6d86fb3e45e4777e06a4da3ee76591b32
2022-08-30 17:15:11 -07:00
David Novotny
1163eaab43 CO3Dv2 trainer configs
Summary:
Adds yaml configs to train selected methods on CO3Dv2.

Few more updates:
1) moved some fields to base classes so that we can check is_multisequence in experiment.py
2) skip loading all train cameras for multisequence datasets, without this, co3d-fewview is untrainable
3) fix bug in json index dataset provider v2

Reviewed By: kjchalup

Differential Revision: D38952755

fbshipit-source-id: 3edac6fc8e20775aa70400bd73a0e6d52b091e0c
2022-08-30 13:42:19 -07:00
Jiali Duan
03562d87f5 Benchmark Cameras
Summary: Address comments to add benchmarkings for cameras and the new fisheye cameras. The dependency functions in test_cameras have been updated in Diff 1. The following two snapshots show benchmarking results.

Reviewed By: kjchalup

Differential Revision: D38991914

fbshipit-source-id: 51fe9bb7237543e4ee112c9f5068a4cf12a9d482
2022-08-28 11:43:46 -07:00
Jiali Duan
2283c292a9 Fisheye Camera for PyTorch3D
Summary:
1. A Fisheye camera model that generalizes pinhole camera by considering distortions (i.e. radial, tangential and thin-prism distortions).

2. Added tests against perspective cameras when distortions are off and Aria data points when distortions are on.

3. Address comments to test unhandled shapes between points and transforms. Added tests for __FIELDS, shape broadcasts, cuda etc.

4. Address earlier comments for code efficiency (e.g., adopted torch.norm; torch.solve for matrix inverse; removed inplace operations; unnecessary clone; expand in place of repeat etc).

Reviewed By: jcjohnson

Differential Revision: D38407094

fbshipit-source-id: a3ab48c85c496ac87af692d5d461bb3fc2a2db13
2022-08-28 11:17:20 -07:00
Darijan Gudelj
4711d12a09 MultiPassEmissionAbsorptionRenderer
Summary: I think there is a typo here could not find any MultiPassEARenderer just MultiPassEmissionAbsorptionRenderer?

Reviewed By: bottler

Differential Revision: D39056641

fbshipit-source-id: 4dd0b123fc795a0083a957786c032e23dc5abac9
2022-08-26 09:23:49 -07:00
Darijan Gudelj
e7c609f198 Decoding functions
Summary: Added replacable decoding functions which will be applied after the voxel grid to get color and density

Reviewed By: bottler

Differential Revision: D38829763

fbshipit-source-id: f21ce206c1c19548206ea2ce97d7ebea3de30a23
2022-08-26 08:47:30 -07:00
Darijan Gudelj
24f5f4a3e7 VoxelGridModule
Summary: Simple wrapper around voxel grids to make them a module

Reviewed By: bottler

Differential Revision: D38829762

fbshipit-source-id: dfee85088fa3c65e396cc7d3bf7ebaaffaadb646
2022-08-25 09:40:43 -07:00
Krzysztof Chalupka
6653f4400b Fix up docstrings
Summary:
One of the docstrings is a disaster see https://pytorch3d.readthedocs.io/en/latest/modules/ops.html

Also some minor fixes I encountered when browsing the code

Reviewed By: bottler

Differential Revision: D38581595

fbshipit-source-id: 3b6ca97788af380a44df9144a6a4cac782c6eab8
2022-08-23 14:58:49 -07:00
Krzysztof Chalupka
c4545a7cbc Add structured bindings to iou to prove that we're C++17-friendly. Also other minor improvements to bbox iou
Summary: Recently we removed C++14-only compilation, should work.

Reviewed By: bottler

Differential Revision: D38919607

fbshipit-source-id: 6a26fa7713f7ba2163364ccc673ad774aa3a5adb
2022-08-23 13:35:21 -07:00
Jeremy Reizenstein
5e7707b157 Update SMPL url
Summary: Fix issue #1306

Reviewed By: kjchalup

Differential Revision: D38941342

fbshipit-source-id: 306ea698ab6af22b874df6e2abdaa9021b65e1ef
2022-08-23 13:19:18 -07:00
Krzysztof Chalupka
a65928dcb9 Remove spurious compute in _compute_vertex_normals
Summary: https://github.com/facebookresearch/pytorch3d/issues/736

Reviewed By: bottler

Differential Revision: D38881935

fbshipit-source-id: 62aa3575513ab752a5afda4a257a985032bc7f6d
2022-08-23 10:08:14 -07:00
Darijan Gudelj
898ba5c53b Moved MLP and Transformer
Summary: Moved the MLP and transformer from nerf to a new file to be reused.

Reviewed By: bottler

Differential Revision: D38828150

fbshipit-source-id: 8ff77b18b3aeeda398d90758a7bcb2482edce66f
2022-08-23 07:22:41 -07:00
Darijan Gudelj
edee25a1e5 voxel grids with interpolation
Summary: Added voxel grid classes from TensoRF, both in their factorized (CP and VM) and full form.

Reviewed By: bottler

Differential Revision: D38465419

fbshipit-source-id: 8b306338af58dc50ef47a682616022a0512c0047
2022-08-23 07:22:41 -07:00
Jeremy Reizenstein
af799facdd silence stalebot
Summary: Stop stale workflow

Reviewed By: kjchalup

Differential Revision: D38858730

fbshipit-source-id: 25a5c00a0295739bac841ca6f0d5ff8230e689d0
2022-08-23 06:57:54 -07:00
Georgia Gkioxari
1bfe6bf20a eps fix for iou3d
Summary: Fix EPS issue that causes numerical instabilities when boxes are very close

Reviewed By: kjchalup

Differential Revision: D38661465

fbshipit-source-id: d2b6753cba9dc2f0072ace5289c9aa815a1a29f6
2022-08-22 04:26:19 -07:00
Jeremy Reizenstein
06cbba2628 Don't force C++14
Summary: Remove compiler arguments which insist on C++14.

Reviewed By: kjchalup

Differential Revision: D38858887

fbshipit-source-id: 542173ec97cacfa724d14c8a4b9ce9dc2457c5d5
2022-08-21 12:10:51 -07:00
Jeremy Reizenstein
7ce31b4e0f updates to CONTRIBUTING
Summary: Fix outdated info

Reviewed By: kjchalup

Differential Revision: D38858766

fbshipit-source-id: 52b120f355e8f9e86d777875627af02c80ee45b6
2022-08-21 11:51:46 -07:00
Max Wasylow
8ea4da2938 Replacing custom CUDA block reductions with CUB in sample_farthest_points
Summary: Removing hardcoded block reduction operation from `sample_farthest_points.cu` code, and replace it with `cub::BlockReduce` reducing complexity of the code, and letting established libraries do the thinking for us.

Reviewed By: bottler

Differential Revision: D38617147

fbshipit-source-id: b230029c55f05cda0aab1648d3105a8d3e92d27b
2022-08-19 09:17:19 -07:00
David Novotny
597bc7c7f6 Blender config fixes
Summary: Fixes the blender synthetic configs.

Reviewed By: kjchalup

Differential Revision: D38786095

fbshipit-source-id: 6d0784ced41a3f2904f074221108cdb56bd20e7f
2022-08-19 06:03:34 -07:00
Darijan Gudelj
f825f7e42c Split Volumes class to data and location part
Summary: Split Volumes class to data and location part so that location part can be reused in planned VoxelGrid classes.

Reviewed By: bottler

Differential Revision: D38782015

fbshipit-source-id: 489da09c5c236f3b81961ce9b09edbd97afaa7c8
2022-08-18 08:12:33 -07:00
Jeremy Reizenstein
fdaaa299a7 remove stray "generic_model_args" references
Summary:
generic_model_args no longer exists. Update some references to it, mostly in doc.

This fixes the testing of all the yaml files in test_forward pass.

Reviewed By: shapovalov

Differential Revision: D38789202

fbshipit-source-id: f11417efe772d7f86368b3598aa66c52b1309dbf
2022-08-18 07:18:55 -07:00
Jeremy Reizenstein
d42e0d3d86 1GPU for implicitron tests
Reviewed By: shapovalov

Differential Revision: D38794764

fbshipit-source-id: 140c8a935d760bab8569d903cc52ac3dd73cd553
2022-08-18 03:23:21 -07:00
Christoph Lassner
7623457686 Removing LOGGER.debug statements for performance gain. (#1260)
Summary:
We identified that these logging statements can deteriorate performance in certain cases. I propose removing them from the regular renderer implementation and letting individuals re-insert debug logging wherever needed on a case-by-case basis.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1260

Reviewed By: kjchalup

Differential Revision: D38737439

Pulled By: bottler

fbshipit-source-id: cf9dcbbeae4dbf214c2e17d5bafa00b2ff796393
2022-08-17 12:22:05 -07:00
Roman Shapovalov
d281f8efd1 Filtering outlier input cameras in trajectory estimation
Summary: Useful for visualising colmap output where some frames are not correctly registered.

Reviewed By: bottler

Differential Revision: D38743191

fbshipit-source-id: e823df2997870dc41d76784e112d4349f904d311
2022-08-17 03:47:31 -07:00
Roman Shapovalov
b7c826b786 Boolean indexing of cameras
Summary: Reasonable to expect bool indexing.

Reviewed By: bottler, kjchalup

Differential Revision: D38741446

fbshipit-source-id: 22b607bf13110043c5624196c66ca1484fdbce6c
2022-08-16 15:19:39 -07:00
Ashish Sinha
60808972b8 typo fix (#1293)
Summary:
fixes typo of the docstring.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1293

Differential Revision: D38737416

Pulled By: bottler

fbshipit-source-id: 3f9da3e97b55c2acd858263de9e85eaee272a98f
2022-08-16 06:46:11 -07:00
David Novotny
d35781f2d7 Rename psnr -> psnr_masked to avoid confusion
Summary: Previously, "psnr" was evaluated between the masked g.t. image and the render. To avoid confusion, "psnr" is now renamed to "psnr_masked".

Reviewed By: bottler

Differential Revision: D38707511

fbshipit-source-id: 8ee881ab1a05453d6692dde9782333a47d8c1234
2022-08-15 09:11:52 -07:00
Jeremy Reizenstein
b6771233e0 builds for PyTorch 1.12.1, drop 1.8
Summary: Builds for new PyTorch 1.12.1. Drop builds for PyTorch 1.8.0 and 1.8.1.

Reviewed By: kjchalup

Differential Revision: D38658991

fbshipit-source-id: 6192e226c2154cd051eeee98498d9a395cfd6fd5
2022-08-15 08:51:12 -07:00
David Novotny
2ff2c7c836 Enable additional test-time source views for json dataset provider v2
Summary: Adds additional source views to the eval batches for evaluating many-view models on CO3D Challenge

Reviewed By: bottler

Differential Revision: D38705904

fbshipit-source-id: cf7d00dc7db926fbd1656dd97a729674e9ff5adb
2022-08-15 08:40:09 -07:00
Jeremy Reizenstein
e8616cc8ba news to README
Summary: 0.6.2 and 0.7.0

Reviewed By: kjchalup

Differential Revision: D38659393

fbshipit-source-id: eb99f3fbe381039b391419c878b7bbac4a88d619
2022-08-12 22:55:34 -07:00
David Novotny
7b985702bb Add full-image PSNR metric
Summary: Reports also the PSNR between the unmasked G.T. image and the render.

Reviewed By: bottler

Differential Revision: D38655943

fbshipit-source-id: 1603a2d02116ea1ce037e5530abe1afc65a2ba93
2022-08-12 07:56:44 -07:00
Luca Di Grazia
a91f15f24e Incompatible variable type fixed (#1288)
Summary:
**"filename"**: "projects/nerf/nerf/implicit_function.py"
**"warning_type"**: "Incompatible variable type [9]",
**"warning_message"**: " input_skips is declared to have type `Tuple[int]` but is used as type `Tuple[]`.",
**"warning_line"**: 256,
**"fix"**: input_skips: Tuple[int,...] = ()

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1288

Reviewed By: kjchalup

Differential Revision: D38615188

Pulled By: bottler

fbshipit-source-id: a014344dd6cf2125f564f948a3c905ceb84cf994
2022-08-11 08:54:37 -07:00
Jeremy Reizenstein
276c9a8acb version 0.7.0
Summary: new version number

Reviewed By: patricklabatut

Differential Revision: D38426307

fbshipit-source-id: 60c6ab53bb46bdece6f20c8fed14f2351f1c606e
2022-08-09 20:48:51 -07:00
Jeremy Reizenstein
791a068183 avoid math.prod for python 3.7
Summary: This makes the new volumes tutorial work on google colab.

Reviewed By: kjchalup

Differential Revision: D38501906

fbshipit-source-id: a606a357e929dae903dc4d9067bd1519f05b1458
2022-08-09 20:48:51 -07:00
Jeremy Reizenstein
c49ebad249 tutorial fix: add visdom
Summary: need to pip install visdom in new volumes tutorial.

Reviewed By: kjchalup

Differential Revision: D38501905

fbshipit-source-id: 534bf097e41f05b3389e9420e6dd2b61a4517861
2022-08-09 20:48:51 -07:00
Jeremy Reizenstein
1cd0cbffb8 implicitron readme updates
Summary: add link in main readme

Reviewed By: kjchalup

Differential Revision: D38560053

fbshipit-source-id: 0814febb67d0580394cfa2664e49e31ff7254bd4
2022-08-09 20:48:51 -07:00
Jeremy Reizenstein
af48430ec0 IT readme updates
Summary: Updates for recent replaceables.

Reviewed By: kjchalup

Differential Revision: D38437370

fbshipit-source-id: 00d600aa451e5849ba48107cd7a4319e9fc8549f
2022-08-09 20:48:51 -07:00
Jeremy Reizenstein
a39cad40f4 LinearExponential LR
Summary: Linear followed by exponential LR progression. Needed for making Blender scenes converge.

Reviewed By: kjchalup

Differential Revision: D38557007

fbshipit-source-id: ad630dbc5b8fabcb33eeb5bdeed5e4f31360bac2
2022-08-09 18:18:46 -07:00
Jeremy Reizenstein
65e5bb3ea1 update notebooks for 0.7.0
Summary: We now expect pytorch 1.12

Reviewed By: patricklabatut

Differential Revision: D38425758

fbshipit-source-id: a22e672fcb0dc18f7d6424323d9cc6aaf5fcb4c6
2022-08-09 17:52:37 -07:00
Krzysztof Chalupka
c83ec3555d Mods and bugfixes for LLFF and Blender repros
Summary:
LLFF (and most/all non-synth datasets) will have no background/foreground distinction. Add support for data with no fg mask.

Also, we had a bug in stats loading, like this:
  * Load stats
  * One of the stats has a history of length 0
  * That's fine, e.g. maybe it's fg_error but the dataset has no notion of fg/bg. So leave it as len 0
  * Check whether all the stats have the same history length as an arbitrarily chosen "reference-stat"
  * Ooops the reference-stat happened to be the stat with length 0
  * assert (legit_stat_len == reference_stat_len (=0)) ---> failed assert

Also some minor fixes (from Jeremy's other diff) to support LLFF

Reviewed By: davnov134

Differential Revision: D38475272

fbshipit-source-id: 5b35ac86d1d5239759f537621f41a3aa4eb3bd68
2022-08-09 15:04:44 -07:00
Pyre Bot Jr
624bc5a274 suppress errors in vision/fair/pytorch3d
Differential Revision: D38529199

fbshipit-source-id: 4bc0574493b60f13e08d9ea7bded862778b3d171
2022-08-08 23:59:04 -07:00
Jeremy Reizenstein
e924de4bf2 add implicitron tutorials to website
Summary: Adding the implicitron tutorials to the website

Reviewed By: kjchalup

Differential Revision: D38508673

fbshipit-source-id: b38d43174acf6e1355b14bf6d58513f6696e7fa0
2022-08-08 14:26:58 -07:00
Roman Shapovalov
a393cccf6a Make multisequence evaluator use only cameras within a batch (fixes OOM in v2)
Summary: In a multisequence (fewview) scenario, it does not make sense to use all cameras for evaluating the difficulty as they come from different scenes. Using only this batch’s source (known) cameras instead.

Reviewed By: bottler

Differential Revision: D38491070

fbshipit-source-id: d6312d8fbb125b28a33db9f53d4215bcd1ca28a8
2022-08-08 04:43:21 -07:00
Jeremy Reizenstein
0e913b1c2e PYTORCH3D_NO_EXTENSION
Summary:
Request in https://github.com/facebookresearch/pytorch3d/issues/1233 for option to disable CUDA build.

Also option to disable binary build completely. This could be useful e.g. in the config tutorial where we need a small python-only part of pytorch3d.

Reviewed By: kjchalup

Differential Revision: D38458624

fbshipit-source-id: 421a0b1cc31306d7e322d3e743e30a7533d7f034
2022-08-08 04:16:26 -07:00
Sergey Prokudin
bd93e9ce21 small fix to the llff dataloader (#1281)
Summary:
simply fixing a typo in _ls function

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1281

Reviewed By: patricklabatut

Differential Revision: D38457259

Pulled By: bottler

fbshipit-source-id: 5204a57cb4d1fe1c804d8af3301b8ea2945443e8
2022-08-05 11:22:34 -07:00
Jeremy Reizenstein
da9584357e circleci fixes
Summary:
Misc fixes.

- most important: the mac image is gone so switch to a newer one.
- torch.concat is new; was used accidentally
- remove lpips from testing in meta.yaml as it is breaking the conda test. Better to leave the relevant tests failing in OSS.
- TypedDict usage is breaking implicitron on Python 3.7.

Reviewed By: patricklabatut

Differential Revision: D38458164

fbshipit-source-id: b16c26453a743b9a771e2a6787b9a4d2a52e41c2
2022-08-05 08:58:17 -07:00
Jeremy Reizenstein
5b8a9b34a0 clarify GM.bg_color
Summary: This field is specific to one purpose.

Reviewed By: patricklabatut

Differential Revision: D38424891

fbshipit-source-id: e017304497012430c30e436da7052b9ad6fc7614
2022-08-05 03:39:39 -07:00
Jeremy Reizenstein
02c0254f7f more globalencoder followup
Summary: remove n_instances==0 special case, standardise args for GlobalEncoderBase's forward.

Reviewed By: shapovalov

Differential Revision: D37817340

fbshipit-source-id: 0aac5fbc7c336d09be9d412cffff5712bda27290
2022-08-05 03:33:30 -07:00
Jeremy Reizenstein
9d888f1332 install trainer inside pytorch3d
Summary: One way to tidy the installation so we don't install files in site-packages/projects. Fixes https://github.com/facebookresearch/pytorch3d/issues/1279

Reviewed By: shapovalov, davnov134

Differential Revision: D38426772

fbshipit-source-id: ac1a54fbf230adb53904701e1f38bf9567f647ce
2022-08-05 02:49:42 -07:00
Pyre Bot Jr
a0f786f4cf upgrade pyre version in fbcode/vision - batch 2
Differential Revision: D38448021

fbshipit-source-id: 966708035115b4870a74f0ca0bbf8ea88b853527
2022-08-04 22:18:46 -07:00
Jeremy Reizenstein
b28754f8e1 config system tutorial
Summary: new

Reviewed By: kjchalup

Differential Revision: D38425731

fbshipit-source-id: 0fd8f524df6b29ceb8c7c9a674022412c1efc3b5
2022-08-04 13:24:15 -07:00
Jeremy Reizenstein
89d8eebc5a volumes tutorial
Summary: new

Reviewed By: kjchalup

Differential Revision: D38425739

fbshipit-source-id: 23e7a1dd01c034b231511acecf40ea0deb9a067f
2022-08-04 13:19:38 -07:00
David Novotny
326716781b - fail if CO3Dv2 path is not given even if path_manager is None
Summary: ... see title

Reviewed By: bottler

Differential Revision: D38430103

fbshipit-source-id: fd0696da82c367b0f4d9c9d92c0cba0230834137
2022-08-04 11:10:48 -07:00
Jeremy Reizenstein
46e82efb4e clean IF args
Summary: continued - avoid duplicate inputs

Reviewed By: davnov134

Differential Revision: D38248827

fbshipit-source-id: 91ed398e304496a936f66e7a70ab3d189eeb5c70
2022-08-03 12:37:31 -07:00
Jeremy Reizenstein
078846d166 clean renderer args
Summary: continued - don't duplicate inputs

Reviewed By: kjchalup

Differential Revision: D38248829

fbshipit-source-id: 2d56418ecbec9cc597c3cf0c122199e274661516
2022-08-03 12:37:31 -07:00
Jeremy Reizenstein
f45893b845 clean raysampler args
Summary: Don't copy from one part of config to another, rather do the copy within GenericModel.

Reviewed By: davnov134

Differential Revision: D38248828

fbshipit-source-id: ff8af985c37ea1f7df9e0aa0a45a58df34c3f893
2022-08-03 12:37:31 -07:00
Darijan Gudelj
5f069dbb7e open_dict for tweaking
Summary: Made the config system call open_dict when it calls the tweak function.

Reviewed By: shapovalov

Differential Revision: D38315334

fbshipit-source-id: 5924a92d8d0bf399bbf3788247f81fc990e265e7
2022-08-03 05:33:46 -07:00
David Novotny
c3f8dad55c Move load_stats to TrainingLoop
Summary:
Stats are logically connected to the training loop, not to the model. Hence, moving to the training loop.

Also removing resume_epoch from OptimizerFactory in favor of a single place - ModelFactory. This removes the need for config consistency checks etc.

Reviewed By: kjchalup

Differential Revision: D38313475

fbshipit-source-id: a1d188a63e28459df381ff98ad8acdcdb14887b7
2022-08-02 15:40:53 -07:00
Krzysztof Chalupka
760305e044 Fix test evaluation for Blender data
Summary: Blender data doesn't have depths or crops.

Reviewed By: shapovalov

Differential Revision: D38345583

fbshipit-source-id: a19300daf666bbfd799d0038aeefa14641c559d7
2022-08-02 12:40:21 -07:00
Jeremy Reizenstein
3a063f5976 SimpleDataLoaderMapProvider
Summary: Simple DataLoaderMapProvider instance

Reviewed By: davnov134

Differential Revision: D38326719

fbshipit-source-id: 58556833e76fae5790d25a59bea0aac4ce046bf1
2022-08-02 12:10:05 -07:00
Darijan Gudelj
c63ec81750 fix eval_batches in V1
Summary: fix to the D38275943 (597e0259dc).

Reviewed By: bottler

Differential Revision: D38355683

fbshipit-source-id: f326f45279fafa57f24b9211ebd3fda18a518937
2022-08-02 09:09:47 -07:00
Krzysztof Chalupka
b7b188bf54 Fix train_stats.pdf: they now work by default
Summary: Before this diff, train_stats.py would not be created by default, EXCEPT when resuming training. This makes them appear from start.

Reviewed By: shapovalov

Differential Revision: D38320341

fbshipit-source-id: 8ea5b99ec81c377ae129f58e78dc2eaff94821ad
2022-08-02 08:50:50 -07:00
Jeremy Reizenstein
f8bf528043 remove get_task
Summary: Remove the dataset's need to provide the task type.

Reviewed By: davnov134, kjchalup

Differential Revision: D38314000

fbshipit-source-id: 3805d885b5d4528abdc78c0da03247edb9abf3f7
2022-08-02 07:55:42 -07:00
Darijan Gudelj
37250a4326 Add forbidden fields to map_provider_v2
Summary:
Added _NEED_CONTROL
 to JsonIndexDatasetMapProviderV2 and made dataset_tweak_args use it.

Reviewed By: bottler

Differential Revision: D38313914

fbshipit-source-id: 529847571065dfba995b609a66737bd91e002cfe
2022-08-02 06:38:51 -07:00
Jeremy Reizenstein
3b7ab22d10 MeshRasterizerOpenGL import fixes
Summary: Only import it if you ask for it.

Reviewed By: kjchalup

Differential Revision: D38327167

fbshipit-source-id: 3f05231f26eda582a63afc71b669996342b0c6f9
2022-08-02 04:32:32 -07:00
David Novotny
5bf6d532f7 Better error message when dataset root set wrongly in JsonIndexDatasetMapProviderV2
Summary: <see title>

Reviewed By: bottler

Differential Revision: D38314727

fbshipit-source-id: 7178b816a22b06af938a35c5f7bb88404fb1b1c4
2022-08-01 14:08:19 -07:00
Darijan Gudelj
597e0259dc Made eval_batches be set inside the __init__
Summary: Made eval_batches be set in call to `__init__` not after the construction as they were before

Reviewed By: bottler

Differential Revision: D38275943

fbshipit-source-id: 32737401d1ddd16c284e1851b7a91f8b041c406f
2022-08-01 10:23:34 -07:00
David Novotny
80fc0ee0b6 Better seeding of random engines
Summary: Currently, seeds are set only inside the train loop. But this does not ensure that the model weights are initialized the same way everywhere which makes all experiments irreproducible. This diff fixes it.

Reviewed By: bottler

Differential Revision: D38315840

fbshipit-source-id: 3d2ecebbc36072c2b68dd3cd8c5e30708e7dd808
2022-08-01 10:03:09 -07:00
David Novotny
0c3599e8ee Correct MC rasterization pt. radius
Summary: Fixes the MC rasterization bug

Reviewed By: bottler

Differential Revision: D38312234

fbshipit-source-id: 910cf809ef3faff3de7a8d905b0821f395a52edf
2022-08-01 04:53:59 -07:00
Jeremy Reizenstein
14bd5e28e8 provide cow dataset
Summary: Make a dummy single-scene dataset using the code from generate_cow_renders (used in existing NeRF tutorials)

Reviewed By: kjchalup

Differential Revision: D38116910

fbshipit-source-id: 8db6df7098aa221c81d392e5cd21b0e67f65bd70
2022-08-01 01:52:12 -07:00
Krzysztof Chalupka
1b0584f7bd Replace pluggable components to create a proper Configurable hierarchy.
Summary:
This large diff rewrites a significant portion of Implicitron's config hierarchy. The new hierarchy, and some of the default implementation classes, are as follows:
```
Experiment
    data_source: ImplicitronDataSource
        dataset_map_provider
        data_loader_map_provider
    model_factory: ImplicitronModelFactory
        model: GenericModel
    optimizer_factory: ImplicitronOptimizerFactory
    training_loop: ImplicitronTrainingLoop
        evaluator: ImplicitronEvaluator
```

1) Experiment (used to be ExperimentConfig) is now a top-level Configurable and contains as members mainly (mostly new) high-level factory Configurables.
2) Experiment's job is to run factories, do some accelerate setup and then pass the results to the main training loop.
3) ImplicitronOptimizerFactory and ImplicitronModelFactory are new high-level factories that create the optimizer, scheduler, model, and stats objects.
4) TrainingLoop is a new configurable that runs the main training loop and the inner train-validate step.
5) Evaluator is a new configurable that TrainingLoop uses to run validation/test steps.
6) GenericModel is not the only model choice anymore. Instead, ImplicitronModelBase (by default instantiated with GenericModel) is a member of Experiment and can be easily replaced by a custom implementation by the user.

All the new Configurables are children of ReplaceableBase, and can be easily replaced with custom implementations.

In addition, I added support for the exponential LR schedule, updated the config files and the test, as well as added a config file that reproduces NERF results and a test to run the repro experiment.

Reviewed By: bottler

Differential Revision: D37723227

fbshipit-source-id: b36bee880d6aa53efdd2abfaae4489d8ab1e8a27
2022-07-29 17:32:51 -07:00
Jeremy Reizenstein
6b481595f0 redefinition -> defaults kept in config
Summary:
This is an internal change in the config systen. It allows redefining a pluggable implementation with new default values. This is useful in notebooks / interactive use. For example, this now works.

        class A(ReplaceableBase):
            pass

        registry.register
        class B(A):
            i: int = 4

        class C(Configurable):
            a: A
            a_class_type: str = "B"

            def __post_init__(self):
                run_auto_creation(self)

        expand_args_fields(C)

        registry.register
        class B(A):
            i: int = 5

        c = C()

        assert c.a.i == 5

Reviewed By: shapovalov

Differential Revision: D38219371

fbshipit-source-id: 72911a9bd3426d3359cf8802cc016fc7f6d7713b
2022-07-28 09:39:18 -07:00
Krzysztof Chalupka
cb49550486 Add MeshRasterizerOpenGL
Summary:
Adding MeshRasterizerOpenGL, a faster alternative to MeshRasterizer. The new rasterizer follows the ideas from "Differentiable Surface Rendering via non-Differentiable Sampling".

The new rasterizer 20x faster on a 2M face mesh (try pose optimization on Nefertiti from https://www.cs.cmu.edu/~kmcrane/Projects/ModelRepository/!). The larger the mesh, the larger the speedup.

There are two main disadvantages:
* The new rasterizer works with an OpenGL backend, so requires pycuda.gl and pyopengl installed (though we avoided writing any C++ code, everything is in Python!)
* The new rasterizer is non-differentiable. However, you can still differentiate the rendering function if you use if with the new SplatterPhongShader which we recently added to PyTorch3D (see the original paper cited above).

Reviewed By: patricklabatut, jcjohnson

Differential Revision: D37698816

fbshipit-source-id: 54d120639d3cb001f096237807e54aced0acda25
2022-07-22 15:52:50 -07:00
Krzysztof Chalupka
36edf2b302 Add .to methods to the splatter and SplatterPhongShader.
Summary: Needed to properly change devices during OpenGL rasterization.

Reviewed By: jcjohnson

Differential Revision: D37698568

fbshipit-source-id: 38968149d577322e662d3b5d04880204b0a7be29
2022-07-22 14:36:22 -07:00
Krzysztof Chalupka
78bb6d17fa Add EGLContext and DeviceContextManager
Summary:
EGLContext is a utility to render with OpenGL without an attached display (that is, without a monitor).

DeviceContextManager allows us to avoid unnecessary context creations and releases. See docstrings for more info.

Reviewed By: jcjohnson

Differential Revision: D36562551

fbshipit-source-id: eb0d2a2f85555ee110e203d435a44ad243281d2c
2022-07-22 09:43:05 -07:00
Jeremy Reizenstein
54c75b4114 GM error for unbatched inputs
Summary: Error when sending an unbatched FrameData through GM.

Reviewed By: shapovalov

Differential Revision: D38036286

fbshipit-source-id: b8d280c61fbbefdc112c57ccd630ab3ccce7b44e
2022-07-21 15:10:24 -07:00
Jeremy Reizenstein
3783437d2f lazy all_train_cameras
Summary: Avoid calculating all_train_cameras before it is needed, because it is slow in some datasets.

Reviewed By: shapovalov

Differential Revision: D38037157

fbshipit-source-id: 95461226655cde2626b680661951ab17ebb0ec75
2022-07-21 15:04:00 -07:00
Jeremy Reizenstein
b2dc520210 lints
Summary: lint issues (mostly flake) in implicitron

Reviewed By: patricklabatut

Differential Revision: D37920948

fbshipit-source-id: 8cb3c2a2838d111c80a211c98a404c210d4649ed
2022-07-21 13:33:49 -07:00
Jeremy Reizenstein
8597d4c5c1 dependencies for testing
Summary: We especially need omegaconf when testing impicitron.

Reviewed By: patricklabatut

Differential Revision: D37921440

fbshipit-source-id: 4e66fde35aa29f60eabd92bf459cd584cfd7e5ca
2022-07-21 13:22:19 -07:00
Jeremy Reizenstein
38fd8380f7 fix ndc/screen problem in blender/llff (#39)
Summary:
X-link: https://github.com/fairinternal/pytorch3d/pull/39

Blender and LLFF cameras were sending screen space focal length and principal point to a camera init function expecting NDC

Reviewed By: shapovalov

Differential Revision: D37788686

fbshipit-source-id: 2ddf7436248bc0d174eceb04c288b93858138582
2022-07-19 10:38:13 -07:00
Jeremy Reizenstein
67840f8320 multiseq conditioning type
Summary: Add the conditioning types to the repro yaml files. In particular, this fixes test_conditioning_type.

Reviewed By: shapovalov

Differential Revision: D37914537

fbshipit-source-id: 621390f329d9da662d915eb3b7bc709206a20552
2022-07-18 03:11:40 -07:00
Jeremy Reizenstein
9b2e570536 option to avoid accelerate
Summary: For debugging, introduce PYTORCH3D_NO_ACCELERATE env var.

Reviewed By: shapovalov

Differential Revision: D37885393

fbshipit-source-id: de080080c0aa4b6d874028937083a0113bb97c23
2022-07-17 13:15:59 -07:00
Iurii Makarov
0f966217e5 Fixed typing to have compatibility with OmegaConf 2.2.2 in Pytorch3D
Summary:
I tried to run `experiment.py` and `pytorch3d_implicitron_runner` and faced the failure with this traceback: https://www.internalfb.com/phabricator/paste/view/P515734086

It seems to be due to the new release of OmegaConf (version=2.2.2) which requires different typing. This fix helped to overcome it.

Reviewed By: bottler

Differential Revision: D37881644

fbshipit-source-id: be0cd4ced0526f8382cea5bdca9b340e93a2fba2
2022-07-15 05:55:03 -07:00
Jiali Duan
379c8b2780 Fix Pytorch3D PnP test
Summary:
EPnP fails the test when the number of points is below 6. As suggested, quadratic option is in theory to deal with as few as 4 points (so num_pts_thresh=3 is set). And when num_pts > num_pts_thresh=4, skip_q is False.

To avoid bumping num_pts_thresh while passing all the original tests, check_output is set to False when num_pts < 6, similar to the logic in Line 123-127.  It makes sure that the algo doesn't crash.

Reviewed By: shapovalov

Differential Revision: D37804438

fbshipit-source-id: 74576d63a9553e25e3ec344677edb6912b5f9354
2022-07-14 09:50:39 -07:00
Jeremy Reizenstein
8e0c82b89a lint fix: raise from None
Summary: New linter warning is complaining about `raise` inside `except`.

Reviewed By: kjchalup

Differential Revision: D37819264

fbshipit-source-id: 56ad5d0558ea39e1125f3c76b43b7376aea2bc7c
2022-07-14 04:21:44 -07:00
David Novotny
8ba9a694ee Remove -1 from crop mask
Summary: Removing 1 from the crop mask does not seem sensible.

Reviewed By: bottler, shapovalov

Differential Revision: D37843680

fbshipit-source-id: 70cec80f9ea26deac63312da62b9c8af27d2a010
2022-07-14 03:30:51 -07:00
Roman Shapovalov
36ba079bef Fixes to CO3Dv2 provider.
Summary:
1. Random sampling of num batches without replacement not supported.
2.Providers should implement the interface for the training loop to work.

Reviewed By: bottler, davnov134

Differential Revision: D37815388

fbshipit-source-id: 8a2795b524e733f07346ffdb20a9c0eb1a2b8190
2022-07-13 09:45:29 -07:00
Jeremy Reizenstein
b95ec190af followups to D37592429
Summary: Fixing comments on D37592429 (0dce883241)

Reviewed By: shapovalov

Differential Revision: D37752367

fbshipit-source-id: 40aa7ee4dc0c5b8b7a84a09d13a3933a9e3afedd
2022-07-13 06:07:02 -07:00
Jeremy Reizenstein
55f67b0d18 add accelerate dependency
Summary: Accelerate is an additional implicitron dependency, so document it.

Reviewed By: shapovalov

Differential Revision: D37786933

fbshipit-source-id: 11024fe604107881f8ca29e17cb5cbfe492fc7f9
2022-07-13 06:00:05 -07:00
Roman Shapovalov
4261e59f51 Fix: making visualisation work again
Summary:
1. Respecting `visdom_show_preds` parameter when it is False.
2. Clipping the images pre-visualisation, which is important for methods like SRN that are not arare of pixel value range.

Reviewed By: bottler

Differential Revision: D37786439

fbshipit-source-id: 8dbb5104290bcc5c2829716b663cae17edc911bd
2022-07-13 05:29:09 -07:00
David Novotny
af55ba01f8 Fix for box_crop=True
Summary: one more bugfix in JsonIndexDataset

Reviewed By: bottler

Differential Revision: D37789138

fbshipit-source-id: 2fb2bda7448674091ff6b279175f0bbd16ff7a62
2022-07-12 10:03:58 -07:00
Jeremy Reizenstein
d3b7f5f421 fix trainer test
Summary: After recent accelerate change D37543870 (aa8b03f31d), update interactive trainer test.

Reviewed By: shapovalov

Differential Revision: D37785932

fbshipit-source-id: 9211374323b6cfd80f6c5ff3a4fc1c0ca04b54ba
2022-07-12 07:20:21 -07:00
Tristan Rice
4ecc9ea89d shader: fix HardDepthShader sizes + tests (#1252)
Summary:
This fixes a indexing bug in HardDepthShader and adds proper unit tests for both of the DepthShaders. This bug was introduced when updating the shader sizes and discovered when I switched my local model onto pytorch3d trunk instead of the patched copy.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1252

Test Plan:
Unit test + custom model code

```
pytest tests/test_shader.py
```

![image](https://user-images.githubusercontent.com/909104/178397456-f478d0e0-9f6c-467a-a85b-adb4c47adfee.png)

Reviewed By: bottler

Differential Revision: D37775767

Pulled By: d4l3k

fbshipit-source-id: 5f001903985976d7067d1fa0a3102d602790e3e8
2022-07-12 04:38:33 -07:00
Tristan Rice
8d10ba52b2 renderer: add support for rendering high dimensional textures for classification/segmentation use cases (#1248)
Summary:
For 3D segmentation problems it's really useful to be able to train the models from multiple viewpoints using Pytorch3D as the renderer. Currently due to hardcoded assumptions in a few spots the mesh renderer only supports rendering RGB (3 dimensional) data. You can encode the classification information as 3 channel data but if you have more than 3 classes you're out of luck.

This relaxes the assumptions to make rendering semantic classes work with `HardFlatShader` and `AmbientLights` with no diffusion/specular. The other shaders/lights don't make any sense for classification since they mutate the texture values in some way.

This only requires changes in `Materials` and `AmbientLights`. The bulk of the code is the unit test.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1248

Test Plan: Added unit test that renders a 5 dimensional texture and compare dimensions 2-5 to a stored picture.

Reviewed By: bottler

Differential Revision: D37764610

Pulled By: d4l3k

fbshipit-source-id: 031895724d9318a6f6bab5b31055bb3f438176a5
2022-07-11 21:22:45 -07:00
Nikhila Ravi
aa8b03f31d Updates to support Accelerate and multigpu training (#37)
Summary:
## Changes:
- Added Accelerate Library and refactored experiment.py to use it
- Needed to move `init_optimizer` and `ExperimentConfig` to a separate file to be compatible with submitit/hydra
- Needed to make some modifications to data loaders etc to work well with the accelerate ddp wrappers
- Loading/saving checkpoints incorporates an unwrapping step so remove the ddp wrapped model

## Tests

Tested with both `torchrun` and `submitit/hydra` on two gpus locally. Here are the commands:

**Torchrun**

Modules loaded:
```sh
1) anaconda3/2021.05   2) cuda/11.3   3) NCCL/2.9.8-3-cuda.11.3   4) gcc/5.2.0. (but unload gcc when using submit)
```

```sh
torchrun --nnodes=1 --nproc_per_node=2 experiment.py --config-path ./configs --config-name repro_singleseq_nerf_test
```

**Submitit/Hydra Local test**

```sh
~/pytorch3d/projects/implicitron_trainer$ HYDRA_FULL_ERROR=1 python3.9 experiment.py --config-name repro_singleseq_nerf_test --multirun --config-path ./configs  hydra/launcher=submitit_local hydra.launcher.gpus_per_node=2 hydra.launcher.tasks_per_node=2 hydra.launcher.nodes=1
```

**Submitit/Hydra distributed test**

```sh
~/implicitron/pytorch3d$ python3.9 experiment.py --config-name repro_singleseq_nerf_test --multirun --config-path ./configs  hydra/launcher=submitit_slurm hydra.launcher.gpus_per_node=8 hydra.launcher.tasks_per_node=8 hydra.launcher.nodes=1 hydra.launcher.partition=learnlab hydra.launcher.timeout_min=4320
```

## TODOS:
- Fix distributed evaluation: currently this doesn't work as the input format to the evaluation function is not suitable for gathering across gpus (needs to be nested list/tuple/dicts of objects that satisfy `is_torch_tensor`) and currently `frame_data`  contains `Cameras` type.
- Refactor the `accelerator` object to be accessible by all functions instead of needing to pass it around everywhere? Maybe have a `Trainer` class and add it as a method?
- Update readme with installation instructions for accelerate and also commands for running jobs with torchrun and submitit/hydra

X-link: https://github.com/fairinternal/pytorch3d/pull/37

Reviewed By: davnov134, kjchalup

Differential Revision: D37543870

Pulled By: bottler

fbshipit-source-id: be9eb4e91244d4fe3740d87dafec622ae1e0cf76
2022-07-11 19:29:58 -07:00
Jeremy Reizenstein
57a40b3688 fix test
Summary: remove erroneous RandomDataLoaderMapProvider

Reviewed By: davnov134

Differential Revision: D37751116

fbshipit-source-id: cf3b555dc1e6304425914d1522b4f70407b498bf
2022-07-11 06:17:48 -07:00
David Novotny
522e5f0644 Bugfix - wrong mask bounds passed to box clamping
Summary: Fixes a bug

Reviewed By: bottler

Differential Revision: D37743350

fbshipit-source-id: d68e680d6027ae2b9814b2241fb72d3b74df77c1
2022-07-10 16:01:33 -07:00
David Novotny
e8390d3500 JsonIndexDatasetProviderV2
Summary: A new version of json index dataset provider supporting CO3Dv2

Reviewed By: shapovalov

Differential Revision: D37690918

fbshipit-source-id: bf2d5fc9d0f1220259e08661dafc69cdbe6b7f94
2022-07-09 17:16:24 -07:00
David Novotny
4300030d7a Changes for CO3Dv2 release [part1]
Summary:
Implements several changes needed for the CO3Dv2 release:
- FrameData contains crop_bbox_xywh which defines the outline of the image crop corresponding to the image-shaped tensors in FrameData
- revised the definition of a bounding box inside JsonDatasetIndex: bbox_xyxy is [xmin, ymin, xmax, ymax], where xmax, ymax are not inclusive; bbox_xywh = [xmin, ymain, xmax-xmin, ymax-ymin]
- is_filtered for detecting whether the entries of the dataset were somehow filtered
- seq_frame_index_to_dataset_index allows to skip entries that are not present in the dataset

Reviewed By: shapovalov

Differential Revision: D37687547

fbshipit-source-id: 7842756b0517878cc0964fc0935d3c0769454d78
2022-07-09 17:16:24 -07:00
Jeremy Reizenstein
00acf0b0c7 cu116 docker image
Summary: cu116 builds need to happen in a specific image.

Reviewed By: patricklabatut

Differential Revision: D37680352

fbshipit-source-id: 81bef0642ad832e83e4eba6321287759b3229303
2022-07-07 23:23:37 -07:00
Jeremy Reizenstein
a94f3f4c4b Add pytorch 1.12, drop pytorch 1.7
Summary: change deps

Reviewed By: kjchalup

Differential Revision: D37612290

fbshipit-source-id: 51af55159605b0edd89ffa9e177238466fc2d993
2022-07-06 14:36:45 -07:00
Jeremy Reizenstein
efb721320a extract camera_difficulty_bin_breaks
Summary: As part of removing Task, move camera difficulty bin breaks from hard code to the top level.

Reviewed By: davnov134

Differential Revision: D37491040

fbshipit-source-id: f2d6775ebc490f6f75020d13f37f6b588cc07a0b
2022-07-06 07:13:41 -07:00
Jeremy Reizenstein
40fb189c29 typing for trainer
Summary: Enable pyre checking of the trainer code.

Reviewed By: shapovalov

Differential Revision: D36545438

fbshipit-source-id: db1ea8d1ade2da79a2956964eb0c7ba302fa40d1
2022-07-06 07:13:41 -07:00
Jeremy Reizenstein
4e87c2b7f1 get_all_train_cameras
Summary: As part of removing Task, make the dataset code generate the source cameras for itself. There's a small optimization available here, in that the JsonIndexDataset could avoid loading images.

Reviewed By: shapovalov

Differential Revision: D37313423

fbshipit-source-id: 3e5e0b2aabbf9cc51f10547a3523e98c72ad8755
2022-07-06 07:13:41 -07:00
Jeremy Reizenstein
771cf8a328 more padding options in Dataloader
Summary: Add facilities for dataloading non-sequential scenes.

Reviewed By: shapovalov

Differential Revision: D37291277

fbshipit-source-id: 0a33e3727b44c4f0cba3a2abe9b12f40d2a20447
2022-07-06 07:13:41 -07:00
David Novotny
0dce883241 Refactor autodecoders
Summary: Refactors autodecoders. Tests pass.

Reviewed By: bottler

Differential Revision: D37592429

fbshipit-source-id: 8f5c9eac254e1fdf0704d5ec5f69eb42f6225113
2022-07-04 07:18:03 -07:00
Krzysztof Chalupka
ae35824f21 Refactor ViewMetrics
Summary:
Make ViewMetrics easy to replace by putting them into an OmegaConf dataclass.

Also, re-word a few variable names and fix minor TODOs.

Reviewed By: bottler

Differential Revision: D37327157

fbshipit-source-id: 78d8e39bbb3548b952f10abbe05688409fb987cc
2022-06-30 09:22:01 -07:00
Brian Hirsh
f4dd151037 fix internal index.Tensor test on wrong device
Summary: After landing https://github.com/pytorch/pytorch/pull/69607, that made it an error to use indexing with `cpu_tensor[cuda_indices]`. There was one outstanding test in fbcode that incorrectly used indexing in that way, which is fixed here

Reviewed By: bottler, osalpekar

Differential Revision: D37128838

fbshipit-source-id: 611b6f717b5b5d89fa61fd9ebeb513ad7e65a656
2022-06-29 09:30:37 -07:00
Roman Shapovalov
7ce8ed55e1 Fix: typo in dict processing
Summary:
David had his code crashed when using frame_annot["meta"] dictionary. Turns out we had a typo.
The tests were passing by chance since all the keys were single-character strings.

Reviewed By: bottler

Differential Revision: D37503987

fbshipit-source-id: c12b0df21116cfbbc4675a0182b9b9e6d62bad2e
2022-06-28 16:11:49 -07:00
Tristan Rice
7e0146ece4 shader: add SoftDepthShader and HardDepthShader for rendering depth maps (#36)
Summary:
X-link: https://github.com/fairinternal/pytorch3d/pull/36

This adds two shaders for rendering depth maps for meshes. This is useful for structure from motion applications that learn depths based off of camera pair disparities.

There's two shaders, one hard which just returns the distances and then a second that does a cumsum on the probabilities of the points with a weighted sum. Areas that don't have any z faces are set to the zfar distance.

Output from this renderer is `[N, H, W]` since it's just depth no need for channels.

I haven't tested this in an ML model yet just in a notebook.

hard:
![hardzshader](https://user-images.githubusercontent.com/909104/170190363-ef662c97-0bd2-488c-8675-0557a3c7dd06.png)

soft:
![softzshader](https://user-images.githubusercontent.com/909104/170190365-65b08cd7-0c49-4119-803e-d33c1d8c676e.png)

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1208

Reviewed By: bottler

Differential Revision: D36682194

Pulled By: d4l3k

fbshipit-source-id: 5d4e10c6fb0fff5427be4ddd3bd76305a7ccc1e2
2022-06-26 04:01:29 -07:00
Ignacio Rocco
0e4c53c612 Fix link in generic_model.py (#38)
Summary: Pull Request resolved: https://github.com/fairinternal/pytorch3d/pull/38

Reviewed By: ignacio-rocco

Differential Revision: D37415027

Pulled By: bottler

fbshipit-source-id: 9b17049e4762506cd5c152fd6e244d5f0d97855b
2022-06-24 06:41:06 -07:00
Jeremy Reizenstein
879495d38f omegaconf 2.2.2 compatibility
Summary: OmegaConf 2.2.2 doesn't like heterogenous tuples or Sequence or Set members. Workaround this.

Reviewed By: shapovalov

Differential Revision: D37278736

fbshipit-source-id: 123e6657947f5b27514910e4074c92086a457a2a
2022-06-24 04:18:01 -07:00
Jeremy Reizenstein
5c1ca757bb srn/idr followups
Summary: small followup to D37172537 (cba26506b6) and D37209012 (81d63c6382): changing default #harmonics and improving a test

Reviewed By: shapovalov

Differential Revision: D37412357

fbshipit-source-id: 1af1005a129425fd24fa6dd213d69c71632099a0
2022-06-24 04:07:15 -07:00
Jeremy Reizenstein
3e4fb0b9d9 provide fg_probability for blender data
Summary: The blender synthetic dataset contains object masks in the alpha channel. Provide these in the corresponding dataset.

Reviewed By: shapovalov

Differential Revision: D37344380

fbshipit-source-id: 3ddacad9d667c0fa0ae5a61fb1d2ffc806c9abf3
2022-06-22 06:11:50 -07:00
Jeremy Reizenstein
731ea53c80 Llff & blender convention fix
Summary: Images were coming out in the wrong format.

Reviewed By: shapovalov

Differential Revision: D37291278

fbshipit-source-id: c10871c37dd186982e7abf2071ac66ed583df2e6
2022-06-22 05:54:54 -07:00
Jeremy Reizenstein
2e42ef793f register ImplicitronDataSource
Summary: Just register ImplicitronDataSource. We don't use it as pluggable yet here.

Reviewed By: shapovalov

Differential Revision: D37315698

fbshipit-source-id: ac41153383f9ab6b14ac69a3dfdc44aca0d94995
2022-06-22 04:24:14 -07:00
Jeremy Reizenstein
81d63c6382 idr harmonic_fns and doc
Summary: Document the inputs of idr functions and distinguish n_harmonic_functions to be 0 (simple embedding) versus -1 (no embedding).

Reviewed By: davnov134

Differential Revision: D37209012

fbshipit-source-id: 6e5c3eae54c4e5e8c3f76cad1caf162c6c222d52
2022-06-20 13:48:34 -07:00
Jeremy Reizenstein
28c1afaa9d nesting n_known_frames_for_test
Summary: Use generator.permutation instead of choice so that different options for n_known_frames_for_test are nested.

Reviewed By: davnov134

Differential Revision: D37210906

fbshipit-source-id: fd0d34ce62260417c3f63354a3f750aae9998b0d
2022-06-20 13:47:47 -07:00
Jeremy Reizenstein
cba26506b6 bg_color for lstm renderer
Summary: Allow specifying a color for non-opaque pixels in LSTMRenderer.

Reviewed By: davnov134

Differential Revision: D37172537

fbshipit-source-id: 6039726678bb7947f7d8cd04035b5023b2d5398c
2022-06-20 13:46:35 -07:00
Jeremy Reizenstein
65f667fd2e loading llff and blender datasets
Summary: Copy code from NeRF for loading LLFF data and blender synthetic data, and create dataset objects for them

Reviewed By: shapovalov

Differential Revision: D35581039

fbshipit-source-id: af7a6f3e9a42499700693381b5b147c991f57e5d
2022-06-16 03:09:15 -07:00
Pyre Bot Jr
7978ffd1e4 suppress errors in vision/fair/pytorch3d
Differential Revision: D37172764

fbshipit-source-id: a2ec367e56de2781a17f5e708eb5832ec9d7e6b4
2022-06-15 06:27:35 -07:00
John Reese
ea4f3260e4 apply new formatting config
Summary:
pyfmt now specifies a target Python version of 3.8 when formatting
with black. With this new config, black adds trailing commas to all
multiline function calls. This applies the new formatting as part
of rolling out the linttool-integration for pyfmt.

paintitblack

Reviewed By: zertosh, lisroach

Differential Revision: D37084377

fbshipit-source-id: 781a1b883a381a172e54d6e447137657977876b4
2022-06-10 16:04:56 -07:00
Jeremy Reizenstein
023a2369ae test configs are loadable
Summary: Add test that the yaml files deserialize.

Reviewed By: davnov134

Differential Revision: D36830673

fbshipit-source-id: b785d8db97b676686036760bfa2dd3fa638bda57
2022-06-10 12:22:46 -07:00
Jeremy Reizenstein
c0f88e04a0 make ExperimentConfig Configurable
Summary: Preparing for pluggables in experiment.py

Reviewed By: davnov134

Differential Revision: D36830674

fbshipit-source-id: eab499d1bc19c690798fbf7da547544df7e88fa5
2022-06-10 12:22:46 -07:00
Jeremy Reizenstein
6275283202 pluggable JsonIndexDataset
Summary: Make dataset type and args configurable on JsonIndexDatasetMapProvider.

Reviewed By: davnov134

Differential Revision: D36666705

fbshipit-source-id: 4d0a3781d9a956504f51f1c7134c04edf1eb2846
2022-06-10 12:22:46 -07:00
Jeremy Reizenstein
1d43251391 PathManagerFactory
Summary: Allow access to manifold internally by default.

Reviewed By: davnov134

Differential Revision: D36760481

fbshipit-source-id: 2a16bd40e81ef526085ac1b3f4606b63c1841428
2022-06-10 12:22:46 -07:00
Jeremy Reizenstein
1fb268dea6 allow get_default_args(JsonIndexDataset)
Summary: Changes to JsonIndexDataset to make it fit with OmegaConf.structured. Also match some default values to what the provider defaults to.

Reviewed By: davnov134

Differential Revision: D36666704

fbshipit-source-id: 65b059a1dbaa240ce85c3e8762b7c3db3b5a6e75
2022-06-10 12:22:46 -07:00
Jeremy Reizenstein
8bc0a04e86 hooks and allow registering base class
Summary: Allow a class to modify its subparts in get_default_args by defining the special function provide_config_hook.

Reviewed By: davnov134

Differential Revision: D36671081

fbshipit-source-id: 3e5b73880cb846c494a209c4479835f6352f45cf
2022-06-10 12:22:46 -07:00
Jeremy Reizenstein
5cd70067e2 Fix tests for OSS
Summary: New paths.

Reviewed By: patricklabatut

Differential Revision: D36734929

fbshipit-source-id: c0ce7ee9145ddca07ef3758d31cc3c261b088e7d
2022-06-01 13:52:26 -07:00
Krzysztof Chalupka
5b74a2cc27 Remove use of torch.tile to fix CI
Summary: Our tests fail (https://fburl.com/jmoqo9bz) because test_splatter_blend uses torch.tile, which is not supported in earlier torch versions. Replace it with tensor.extend.

Reviewed By: bottler

Differential Revision: D36796098

fbshipit-source-id: 38d5b40667f98f3163b33f44e53e96b858cfeba2
2022-06-01 08:47:26 -07:00
Roman Shapovalov
49ed7b07b1 Adapting configs.
Summary: As subj.

Reviewed By: bottler

Differential Revision: D36705775

fbshipit-source-id: 7370710e863025dc07a140b41f77a7c752e3159f
2022-05-27 02:31:47 -07:00
Jeremy Reizenstein
c6519f29f0 chamfer for empty pointclouds #1174
Summary: Fix divide by zero for empty pointcloud in chamfer. Also for empty batches. In process, needed to regularize num_points_per_cloud for empty batches.

Reviewed By: kjchalup

Differential Revision: D36311330

fbshipit-source-id: 3378ab738bee77ecc286f2110a5c8dc445960340
2022-05-26 14:56:22 -07:00
Krzysztof Chalupka
a42a89a5ba SplatterBlender follow-ups
Summary: A few minor additions I didn't fit into the SplatterBlender diffs, as requested by reviewers.

Reviewed By: jcjohnson

Differential Revision: D36682437

fbshipit-source-id: 57af995e766dfd2674b3984a3ba00aef7ca7db80
2022-05-26 13:03:57 -07:00
Jeremy Reizenstein
c31bf85a23 test runner for experiment.py
Summary: Add simple interactive testrunner for experiment.py

Reviewed By: shapovalov

Differential Revision: D35316221

fbshipit-source-id: d424bcba632eef89eefb56e18e536edb58ec6f85
2022-05-26 05:33:03 -07:00
Jeremy Reizenstein
fbd3c679ac rename ImplicitronDataset to JsonIndexDataset
Summary: The ImplicitronDataset class corresponds to JsonIndexDatasetMapProvider

Reviewed By: shapovalov

Differential Revision: D36661396

fbshipit-source-id: 80ca2ff81ef9ecc2e3d1f4e1cd14b6f66a7ec34d
2022-05-25 10:16:59 -07:00
Jeremy Reizenstein
34f648ede0 move targets
Summary: Move testing targets from pytorch3d/tests/TARGETS to pytorch3d/TARGETS.

Reviewed By: shapovalov

Differential Revision: D36186940

fbshipit-source-id: a4c52c4d99351f885e2b0bf870532d530324039b
2022-05-25 06:16:03 -07:00
Jeremy Reizenstein
f625fe1f8b further test fix
Summary: test_viewpool was inactive so missed being fixed in D36547815 (2d1c6d5d93)

Reviewed By: kjchalup

Differential Revision: D36625587

fbshipit-source-id: e7224eadfa5581fe61f10f67d2221071783de04a
2022-05-25 04:22:38 -07:00
Krzysztof Chalupka
7c25d34d22 SplatterPhongShader Benchmarks
Summary:
Benchmarking. We only use num_faces=2 for splatter, because as far as I can see one would never need to use more. Pose optimization and mesh optimization experiments (see next two diffs) showed that Splatter with 2 faces beats Softmax with 50 and 100 faces in terms of accuracy.

Results: We're slower at 64px^2. At 128px and 256px, we're slower than Softmax+50faces, but faster than Softmax+100faces. We're also slower at 10 faces/pix, but expectation as well as results show that more then 2 faces shouldn't be necessary. See also more results in .https://fburl.com/gdoc/ttv7u7hp

Reviewed By: jcjohnson

Differential Revision: D36210575

fbshipit-source-id: c8de28c8a59ce5fe21a47263bd43d2757b15d123
2022-05-24 22:31:12 -07:00
Krzysztof Chalupka
c5a83f46ef SplatterBlender
Summary: Splatting shader. See code comments for details. Same API as SoftPhongShader.

Reviewed By: jcjohnson

Differential Revision: D36354301

fbshipit-source-id: 71ee37f7ff6bb9ce028ba42a65741424a427a92d
2022-05-24 21:04:11 -07:00
Jeremy Reizenstein
1702c85bec avoid warning in ndc_grid_sample
Summary: If you miss grid_sample in recent pytorch, it gives a warning, so stop doing this.

Reviewed By: kjchalup

Differential Revision: D36410619

fbshipit-source-id: 41dd4455298645c926f4d96c2084093b3f64ee2c
2022-05-24 18:18:21 -07:00
Jeremy Reizenstein
90d00f1b2b PLY heterogenous faces fix
Summary: PLY with mixture of triangle and quadrilateral faces was failing.

Reviewed By: gkioxari

Differential Revision: D36592981

fbshipit-source-id: 5373edb2f38389ac646a75fd2e1fa7300eb8d054
2022-05-24 01:40:22 -07:00
Jeremy Reizenstein
d27ef14ec7 test_forward_pass: speedup and RE fix
Summary: Use small image size for test_all_gm_configs

Reviewed By: shapovalov

Differential Revision: D36511528

fbshipit-source-id: 2c65f518a4f23626850343a62d103f85abfabd88
2022-05-22 15:23:17 -07:00
Jeremy Reizenstein
2d1c6d5d93 simplify image_feature_extractor control
Summary: If no view pooling, don't disable image_feature_extractor. Make image_feature_extractor default to absent.

Reviewed By: davnov134

Differential Revision: D36547815

fbshipit-source-id: e51718e1bcbf65b8b365a6e894d4324f136635e9
2022-05-20 08:32:19 -07:00
Jeremy Reizenstein
9fe15da3cd ImplicitronDatasetBase -> DatasetBase
Summary: Just a rename

Reviewed By: shapovalov

Differential Revision: D36516885

fbshipit-source-id: 2126e3aee26d89a95afdb31e06942d61cbe88d5a
2022-05-20 07:50:30 -07:00
Jeremy Reizenstein
0f12c51646 data_loader_map_provider
Summary: replace dataloader_zoo with a pluggable DataLoaderMapProvider.

Reviewed By: shapovalov

Differential Revision: D36475441

fbshipit-source-id: d16abb190d876940434329928f2e3f2794a25416
2022-05-20 07:50:30 -07:00
Jeremy Reizenstein
79c61a2d86 dataset_map_provider
Summary: replace dataset_zoo with a pluggable DatasetMapProvider. The logic is now in annotated_file_dataset_map_provider.

Reviewed By: shapovalov

Differential Revision: D36443965

fbshipit-source-id: 9087649802810055e150b2fbfcc3c197a761f28a
2022-05-20 07:50:30 -07:00
Jeremy Reizenstein
69c6d06ed8 New file for ImplicitronDatasetBase
Summary: Separate ImplicitronDatasetBase and FrameData (to be used by all data sources) from ImplicitronDataset (which is specific).

Reviewed By: shapovalov

Differential Revision: D36413111

fbshipit-source-id: 3725744cde2e08baa11aff4048237ba10c7efbc6
2022-05-20 07:50:30 -07:00
Jeremy Reizenstein
73dc109dba data_source
Summary:
Move dataset_args and dataloader_args from ExperimentConfig into a new member called datasource so that it can contain replaceables.

Also add enum Task for task type.

Reviewed By: shapovalov

Differential Revision: D36201719

fbshipit-source-id: 47d6967bfea3b7b146b6bbd1572e0457c9365871
2022-05-20 07:50:30 -07:00
Jeremy Reizenstein
9ec9d057cc Make feature extractor pluggable
Summary: Make ResNetFeatureExtractor be an implementation of FeatureExtractorBase.

Reviewed By: davnov134

Differential Revision: D35433098

fbshipit-source-id: 0664a9166a88e150231cfe2eceba017ae55aed3a
2022-05-18 08:50:18 -07:00
Jeremy Reizenstein
cd7b885169 don't check black version
Summary: skip checking the version of black because `black --version` looks different in different versions.

Reviewed By: kjchalup

Differential Revision: D36441262

fbshipit-source-id: a2d9a5cad4f5433909fb85bc9a584e91a2b72601
2022-05-17 09:08:06 -07:00
Jeremy Reizenstein
f632c423ef FrameAnnotation.meta, Optional in _dataclass_from_dict
Summary: Allow extra data in a FrameAnnotation. Therefore allow Optional[T] systematically in _dataclass_from_dict

Reviewed By: davnov134

Differential Revision: D36442691

fbshipit-source-id: ba70f6491574c08b0d9c9acb63f35514d29de214
2022-05-17 08:16:29 -07:00
Jeremy Reizenstein
f36b11fe49 allow Optional[Dict]=None in config
Summary: Fix recently observed case where enable_get_default_args was missing things declared as Optional[something mutable]=None.

Reviewed By: davnov134

Differential Revision: D36440492

fbshipit-source-id: 192ec07564c325b3b24ccc49b003788f67c63a3d
2022-05-17 05:06:18 -07:00
Krzysztof Chalupka
ea5df60d72 In blending, pull common functionality into get_background_color
Summary: A small refactor, originally intended for use with the splatter.

Reviewed By: bottler

Differential Revision: D36210393

fbshipit-source-id: b3372f7cc7690ee45dd3059b2d4be1c8dfa63180
2022-05-16 18:23:51 -07:00
Krzysztof Chalupka
4372001981 Make transform_points_screen's with_xyflip configurable
Summary: We'll need non-flipped screen coords in splatter.

Reviewed By: bottler

Differential Revision: D36337027

fbshipit-source-id: 897f88e8854bab215d2d0e502b25d15526ee86f1
2022-05-16 18:23:51 -07:00
Krzysztof Chalupka
61e2b87019 Add ability for phong_shading to return pixel_coords
Summary: The splatter can re-use pixel coords computed by the shader.

Reviewed By: bottler

Differential Revision: D36332530

fbshipit-source-id: b28e7abe22cca4f48b4108ad397aafc0f1347901
2022-05-16 18:23:51 -07:00
Roman Shapovalov
0143d63ba8 Correcting recent bugs code after debugging on devfair.
Summary:
1. Typo in the dataset path in the config.
2. Typo in num_frames.
3. Pick sequence was cached before it was modified for single-sequence.

Reviewed By: bottler

Differential Revision: D36417329

fbshipit-source-id: 6dcd75583de510412e1ae58f63db04bb4447403e
2022-05-16 12:17:08 -07:00
Jeremy Reizenstein
899a3192b6 create_x_impl
Summary: Make create_x delegate to create_x_impl so that users can rely on create_x_impl in their overrides of create_x.

Reviewed By: shapovalov, davnov134

Differential Revision: D35929810

fbshipit-source-id: 80595894ee93346b881729995775876b016fc08e
2022-05-16 04:42:03 -07:00
John Reese
3b2300641a apply import merging for fbcode (11 of 11)
Summary:
Applies new import merging and sorting from µsort v1.0.

When merging imports, µsort will make a best-effort to move associated
comments to match merged elements, but there are known limitations due to
the diynamic nature of Python and developer tooling. These changes should
not produce any dangerous runtime changes, but may require touch-ups to
satisfy linters and other tooling.

Note that µsort uses case-insensitive, lexicographical sorting, which
results in a different ordering compared to isort. This provides a more
consistent sorting order, matching the case-insensitive order used when
sorting import statements by module name, and ensures that "frog", "FROG",
and "Frog" always sort next to each other.

For details on µsort's sorting and merging semantics, see the user guide:
https://usort.readthedocs.io/en/stable/guide.html#sorting

Reviewed By: lisroach

Differential Revision: D36402260

fbshipit-source-id: 7cb52f09b740ccc580e61e6d1787d27381a8ce00
2022-05-15 12:53:03 -07:00
Jeremy Reizenstein
b5f3d3ce12 fix test_config_use
Summary: Fixes to reenable test_create_gm_overrides. Followup from D35852367 (47d06c8924) using logic from D36349361 (9e57b994ca).

Reviewed By: shapovalov

Differential Revision: D36371762

fbshipit-source-id: ad5fbbb4b5729fac41980d118f17a2589f7e6aba
2022-05-13 07:15:26 -07:00
Jeremy Reizenstein
2c1901522a return types for dataset_zoo, dataloader_zoo
Summary: Stronger typing for these functions

Reviewed By: shapovalov

Differential Revision: D36170489

fbshipit-source-id: a2104b29dbbbcfcf91ae1d076cd6b0e3d2030c0b
2022-05-13 05:38:14 -07:00
Jeremy Reizenstein
90ab219d88 clarify expand_args_fields
Summary: Fix doc and add a call to expand_args_fields for each implicit function.

Reviewed By: shapovalov

Differential Revision: D35929811

fbshipit-source-id: 8c3cfa56b8d8908fd2165614960e3d34b54717bb
2022-05-13 03:26:47 -07:00
Jeremy Reizenstein
9e57b994ca resnet34 weights for remote executor
Summary: Like vgg16 for lpips, internally we need resnet34 weights for coming feature extractor tests.

Reviewed By: davnov134

Differential Revision: D36349361

fbshipit-source-id: 1c33009c904766fcc15e7e31cd15d0f820c57354
2022-05-12 16:57:16 -07:00
David Novotny
e767c4b548 Raysampler as pluggable
Summary:
This converts raysamplers to ReplaceableBase so that users can hack their own raysampling impls.

Context: Andrea tried to implement TensoRF within implicitron but could not due to the need to implement his own raysampler.

Reviewed By: shapovalov

Differential Revision: D36016318

fbshipit-source-id: ef746f3365282bdfa9c15f7b371090a5aae7f8da
2022-05-12 15:39:35 -07:00
David Novotny
e85fa03c5a Generic Raymarcher refactor
Summary: Uses the GenericRaymarcher only as an ABC and derives two common implementations - EA raymarcher and Cumsum raymarcher (from neural volumes)

Reviewed By: shapovalov

Differential Revision: D35927653

fbshipit-source-id: f7e6776e71f8a4e99eefc018a47f29ae769895ee
2022-05-12 14:57:50 -07:00
David Novotny
47d06c8924 ViewPooler class
Summary: Implements a ViewPooler that groups ViewSampler and FeatureAggregator.

Reviewed By: shapovalov

Differential Revision: D35852367

fbshipit-source-id: c1bcaf5a1f826ff94efce53aa5836121ad9c50ec
2022-05-12 12:50:03 -07:00
John Reese
bef959c755 formatting changes from black 22.3.0
Summary:
Applies the black-fbsource codemod with the new build of pyfmt.

paintitblack

Reviewed By: lisroach

Differential Revision: D36324783

fbshipit-source-id: 280c09e88257e5e569ab729691165d8dedd767bc
2022-05-11 19:55:56 -07:00
Krzysztof Chalupka
c21ba144e7 Add Fragments.detach()
Summary: Add a capability to detach all detachable tensors in Fragments.

Reviewed By: bottler

Differential Revision: D35918133

fbshipit-source-id: 03b5d4491a3a6791b0a7bc9119f26c1a7aa43196
2022-05-11 18:50:24 -07:00
Christian Kauten
d737a05e55 Update INSTALL.md (#1194)
Summary:
Resolve https://github.com/facebookresearch/pytorch3d/issues/1186 by fixing the minimal version of CUDA for installing from a wheel

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1194

Reviewed By: patricklabatut

Differential Revision: D36279396

Pulled By: bottler

fbshipit-source-id: 2371256a5451ec33c01d6fa9616c5b24fa83f7f8
2022-05-11 07:03:12 -07:00
David Novotny
2374d19da5 Test all CO3D model configs in test_forward_pass
Summary: Tests all possible model configs in test_forward_pass.py

Reviewed By: shapovalov

Differential Revision: D35851507

fbshipit-source-id: 4860ee1d37cf17a2faab5fc14d4b2ba0b96c4b8b
2022-05-11 05:40:05 -07:00
Pyre Bot Jr
1f3953795c suppress errors in vision/fair/pytorch3d
Differential Revision: D36269817

fbshipit-source-id: 47b8a77747e8297af3731fd0a388d4c5432dc1ff
2022-05-09 19:10:01 -07:00
Roman Shapovalov
a6dada399d Extracted ImplicitronModelBase and unified API for GenericModel and ModelDBIR
Summary:
To avoid model_zoo, we need to make GenericModel pluggable.
I also align creation APIs for convenience.

Reviewed By: bottler, davnov134

Differential Revision: D35933093

fbshipit-source-id: 8228926528eb41a795fbfbe32304b8019197e2b1
2022-05-09 15:23:07 -07:00
David Novotny
5c59841863 Add **kwargs to ViewMetrics.forward
Summary: GenericModel crashes in case the `aux` field of any Renderer is populated. This is because the `rendered.aux` is unpacked to  ViewMetrics.forward whose signature does not contain **kwargs. Hence, the contents of `aux` are unknown to forward's signature resulting in a crash.

Reviewed By: bottler

Differential Revision: D36166118

fbshipit-source-id: 906a067ea02a3648a69667422466451bc219ebf6
2022-05-09 03:04:34 -07:00
Krzysztof Chalupka
2c64635daa Add type hints to MeshRenderer(WithFragments)
Reviewed By: bottler

Differential Revision: D36148049

fbshipit-source-id: 87ca3ea8d5b5a315418cc597b36fd0a1dffb1e00
2022-05-06 14:48:26 -07:00
Jeremy Reizenstein
ec9580a1d4 test runner for eval_demo
Summary:
Create a test runner for the eval_demo code.  Debugging this is useful for understanding datasets.

Introduces an environment variable INTERACTIVE_TESTING for ignoring tests which are not intended for use in regular test runs.

Reviewed By: shapovalov

Differential Revision: D35964016

fbshipit-source-id: ab0f93aff66b6cfeca942b14466cf81f7feb2224
2022-05-06 08:31:19 -07:00
Jeremy Reizenstein
44cb00e468 lstsq fix in circle fitting for old PyTorch
Summary: the pytorch3d.compat.lstsq function needs a 2D rhs.

Reviewed By: patricklabatut

Differential Revision: D36195826

fbshipit-source-id: 9dbafea2057035cc04973f56729dc97b47dcac83
2022-05-06 04:12:51 -07:00
Jeremy Reizenstein
44ca5f95d9 Add vis to readthedocs
Summary: pytorch3d/vis has been missing. Reduce prominence of common.

Reviewed By: patricklabatut

Differential Revision: D36008733

fbshipit-source-id: bbc9fbb031c8dc95870087fa48df29410ae69e35
2022-05-06 04:07:43 -07:00
Pyre Bot Jr
a51a300827 suppress errors in fbcode/vision - batch 2
Differential Revision: D36120486

fbshipit-source-id: bddbf47957f4476f826ad20c2d6e146c98ee73e1
2022-05-03 20:29:21 -07:00
Jeremy Reizenstein
2bd65027ca version 0.6.2
Summary: Update PyTorch3D version number

Differential Revision: D35980555

fbshipit-source-id: 637ccd33eef22d909985d2fce3958c78f3d0d551
2022-04-28 04:48:24 -07:00
Jeremy Reizenstein
11635fbd7d INSTALL/README updates
Summary: Updates for version 0.6.2

Differential Revision: D35980557

fbshipit-source-id: e677a22d4f8a323376310dfb536133bee8045f1f
2022-04-28 04:48:24 -07:00
Jeremy Reizenstein
a268b18e07 update tutorials for version 0.6.2
Summary: colab is now 1.11.0

Differential Revision: D35980556

fbshipit-source-id: 988a06c652518fb61ccbef2e7197e3422a706250
2022-04-28 04:48:24 -07:00
Krzysztof Chalupka
7ea0756b05 fit_textured_mesh tutorial fixes
Summary: Updated to FoV cameras and added perspective_correct=False, otherwise it'll nan out.

Reviewed By: bottler

Differential Revision: D35970900

fbshipit-source-id: 569b8de0b124d415f4b841924ddc85585cee2dda
2022-04-27 12:18:03 -07:00
Krzysztof Chalupka
96889deab9 SplatterPhongShader 1: Pull out common Shader functionality into ShaderBase
Summary: Most of the shaders copypaste exactly the same code into `__init__` and `to`. I will be adding a new shader in the next diff, so let's make it a bit easier.

Reviewed By: bottler

Differential Revision: D35767884

fbshipit-source-id: 0057e3e2ae3be4eaa49ae7e2bf3e4176953dde9d
2022-04-27 12:07:51 -07:00
Jeremy Reizenstein
9f443ed26b isort->usort
Summary: Move from isort to usort now that usort supports sorting within lines.

Reviewed By: patricklabatut

Differential Revision: D35893280

fbshipit-source-id: 621c1cd285199d785408504430ee0bdf8683b21e
2022-04-26 08:34:54 -07:00
Jeremy Reizenstein
9320100abc object_mask only if required
Summary: New function to check if a renderer needs the object mask.

Reviewed By: davnov134

Differential Revision: D35254009

fbshipit-source-id: 4c99e8a1c0f6641d910eb32bfd6cfae9d3463d50
2022-04-26 08:01:45 -07:00
Jeremy Reizenstein
2edb93d184 chunked_inputs
Summary: Make method for SDF's use of object mask more general, so that a renderer can be given per-pixel values.

Reviewed By: shapovalov

Differential Revision: D35247412

fbshipit-source-id: 6aeccb1d0b5f1265a3f692a1453407a07e51a33c
2022-04-26 08:01:45 -07:00
Jeremy Reizenstein
41c594ca37 fix entry points in setup.py
Summary: For `pip install` without -e, we need to name the entry point functions in setup.py.

Reviewed By: patricklabatut

Differential Revision: D35933037

fbshipit-source-id: be15ae1a4bb7c5305ea2ba992d07f3279c452250
2022-04-26 07:59:15 -07:00
Krzysztof Chalupka
c3c4495c7a Fix image links in renderer documentation
Summary: Repo has jpgs but docs/website want pngs.

Reviewed By: nikhilaravi

Differential Revision: D35596475

fbshipit-source-id: 4cafd405c06c0eb339001a8db2422dbbd1f8f28a
2022-04-14 16:37:07 -07:00
Tim Hatch
34bbb3ad32 apply import merging for fbcode/vision/fair (2 of 2)
Summary:
Applies new import merging and sorting from µsort v1.0.

When merging imports, µsort will make a best-effort to move associated
comments to match merged elements, but there are known limitations due to
the diynamic nature of Python and developer tooling. These changes should
not produce any dangerous runtime changes, but may require touch-ups to
satisfy linters and other tooling.

Note that µsort uses case-insensitive, lexicographical sorting, which
results in a different ordering compared to isort. This provides a more
consistent sorting order, matching the case-insensitive order used when
sorting import statements by module name, and ensures that "frog", "FROG",
and "Frog" always sort next to each other.

For details on µsort's sorting and merging semantics, see the user guide:
https://usort.readthedocs.io/en/stable/guide.html#sorting

Reviewed By: bottler

Differential Revision: D35553814

fbshipit-source-id: be49bdb6a4c25264ff8d4db3a601f18736d17be1
2022-04-13 06:51:33 -07:00
Jeremy Reizenstein
df08ea8eb4 Fix inferred typing
Summary: D35513897 (4b94649f7b) was a pyre infer job which got some things wrong. Correct by adding the correct types, so these things shouldn't need worrying about again.

Reviewed By: patricklabatut

Differential Revision: D35546144

fbshipit-source-id: 89f6ea2b67be27aa0b0b14afff4347cccf23feb7
2022-04-13 04:40:56 -07:00
Jeremy Reizenstein
78fd5af1a6 make points2volumes feature rescaling optional
Summary: Add option to not rescale the features, giving more control. https://github.com/facebookresearch/pytorch3d/issues/1137

Reviewed By: nikhilaravi

Differential Revision: D35219577

fbshipit-source-id: cbbb643b91b71bc908cedc6dac0f63f6d1355c85
2022-04-13 04:39:47 -07:00
h5jam
0a7c354dc1 fix typo on NeRF tutorial (#1163)
Summary:
Hello, I'm Seungoh from South Korea.

I'm finding typo while I'm learning tutorials.
Wrong numbers are changed to right numbers.

Thank you.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1163

Reviewed By: patricklabatut

Differential Revision: D35546843

Pulled By: bottler

fbshipit-source-id: b6e70cdf821fd4a108dfd416e8f4bcb3ecbeb449
2022-04-13 04:35:05 -07:00
Pyre Bot Jr
b79764ea69 suppress errors in fbcode/vision - batch 2
Differential Revision: D35590813

fbshipit-source-id: 0f35d7193f839a41f3cac18bf20236b815368f19
2022-04-12 15:56:12 -07:00
Krzysztof Chalupka
b1ff9d9fd4 Disallow None vertex/face lists in texture submeshing
Summary: In order to simplify the interface, we disallow passing None as vertex/face lists to textures.submeshes. This function would only ever get called from within meshes.submeshes where we can provide both arguments, even if they're not necessary for a specific submesh type.

Reviewed By: bottler

Differential Revision: D35581161

fbshipit-source-id: aeab99308a319b144e141ca85ca7515f855116da
2022-04-12 10:46:48 -07:00
Krzysztof Chalupka
22f86072ca Submesh 4/n: TexturesVertex submeshing
Summary: Add submeshing capability for meshes with TexturesVertex.

Reviewed By: bottler

Differential Revision: D35448534

fbshipit-source-id: 6d16a31a5bfb24ce122cf3c300a7616bc58353d1
2022-04-11 16:27:53 -07:00
Krzysztof Chalupka
050f650ae8 Submesh 3/n: Add submeshing functionality
Summary:
Copypasting the docstring:
```
        Split a mesh into submeshes, defined by face indices of the original Meshes object.

        Args:
          face_indices:
            Let the original mesh have verts_list() of length N.
            Can be either
              - List of length N. The n-th element is a list of length num_submeshes_n
                (empty lists are allowed). Each element of the n-th sublist is a LongTensor
                of length num_faces.
              - List of length N. The n-th element is a possibly empty padded LongTensor of
                shape (num_submeshes_n, max_num_faces).

        Returns:
          Meshes object with selected submeshes. The submesh tensors are cloned.

        Currently submeshing only works with no textures or with the TexturesVertex texture.

        Example:

        Take a Meshes object `cubes` with 4 meshes, each a translated cube. Then:
            * len(cubes) is 4, len(cubes.verts_list()) is 4, len(cubes.faces_list()) is 4,
            * [cube_verts.size for cube_verts in cubes.verts_list()] is [8, 8, 8, 8],
            * [cube_faces.size for cube_faces in cubes.faces_list()] if [6, 6, 6, 6],

        Now let front_facet, top_and_bottom, all_facets be LongTensors of
        sizes (2), (4), and (12), each picking up a number of facets of a cube by specifying
        the appropriate triangular faces.

        Then let `subcubes = cubes.submeshes([[front_facet, top_and_bottom], [], [all_facets], []])`.
            * len(subcubes) is 3.
            * subcubes[0] is the front facet of the cube contained in cubes[0].
            * subcubes[1] is a mesh containing the (disconnected) top and bottom facets of cubes[0].
            * subcubes[2] is a clone of cubes[2].
            * There are no submeshes of cubes[1] and cubes[3] in subcubes.
            * subcubes[0] and subcubes[1] are not watertight. subcubes[2] is.
```

Reviewed By: bottler

Differential Revision: D35440657

fbshipit-source-id: 8a6d2d300ce226b5b9eb440688528b5e795195a1
2022-04-11 16:27:53 -07:00
Krzysztof Chalupka
8596fcacd2 Submesh 2/n: to_sorted
Summary:
Sort a mesh's vertices in alphabetical order, and resort the face coords accordingly. Textured meshes are not supported yet, but will be added down the stack.

This, togehter with mesh equality, can be used to compare two meshes in a way invariant to vertex permutations, as shown in the unit tests.

We do not want the submeshing mechanism to guarantee any particular vertex order, leaving that up to the implementation, so we need this function for testing.

Reviewed By: bottler

Differential Revision: D35440656

fbshipit-source-id: 5a4dd921fdb00625a33da08b5fea79e20ac6402c
2022-04-11 16:27:53 -07:00
Krzysztof Chalupka
7f097b064b Submesh 1/n: Implement mesh equality
Summary: Adding a mesh equality operator. Two Meshes objects m1, m2 are equal iff their vertex lists, face lists, and normals lists are equal. Textures meshes are not supported yet, but will be added for vertex textures down the stack.

Reviewed By: bottler, nikhilaravi

Differential Revision: D35440655

fbshipit-source-id: 69974a59c091416afdb2892896859a189f5ebf3a
2022-04-11 16:27:53 -07:00
Krzysztof Chalupka
aab95575a6 Submesh 0/n: Default to empty Meshes
Summary:
The default behavior of Meshes (with verts=None, faces=None) throws an exception:
```
meshes = Meshes()
> ValueError: Verts and Faces must be either a list or a tensor with shape (batch_size, N, 3) where N is either the maximum number of verts or faces respectively.
```

Instead, let's default to an empty mesh, following e.g. PyTorch:
```
empty_tensor = torch.FloatTensor()
> torch.tensor([])
```

this change is backwards-compatible (you can still init with verts=[], faces=[]).

Reviewed By: bottler, nikhilaravi

Differential Revision: D35443453

fbshipit-source-id: d638a8fef49a089bf0da6dd2201727b94ceb21ec
2022-04-11 16:27:53 -07:00
Georgia Gkioxari
67fff956a2 add L1 support for KNN & Chamfer
Summary:
Added L1 norm for KNN and chamfer op
* The norm is now specified with a variable `norm` which can only be 1 or 2

Reviewed By: bottler

Differential Revision: D35419637

fbshipit-source-id: 77813fec650b30c28342af90d5ed02c89133e136
2022-04-10 10:27:20 -07:00
Pyre Bot Jr
4b94649f7b Add annotations to vision/fair/pytorch3d
Reviewed By: shannonzhu

Differential Revision: D35513897

fbshipit-source-id: 1ca12671df1bd6608a7dce9193c145d5985c0b45
2022-04-08 18:23:41 -07:00
Pyre Bot Jr
3809b6094c suppress errors in vision/fair/pytorch3d
Differential Revision: D35455033

fbshipit-source-id: c4fe9577edd7beb9c40be1cb387f125d53a6a577
2022-04-06 18:53:08 -07:00
Jeremy Reizenstein
722646863c Optional[Configurable] in config
Summary: A new type of auto-expanded member of a Configurable: something of type Optional[X] where X is a Configurable. This works like X but its construction is controlled by a boolean membername_enabled.

Reviewed By: davnov134

Differential Revision: D35368269

fbshipit-source-id: 7e0c8a3e8c4930b0aa942fa1b325ce65336ebd5f
2022-04-06 05:56:14 -07:00
Jeremy Reizenstein
e10a90140d enable_get_default_args to allow pickling get_default_args(f)
Summary:
Try again to solve https://github.com/facebookresearch/pytorch3d/issues/1144 pickling problem.
D35258561 (24260130ce) didn't work.

When writing a function or vanilla class C which you want people to be able to call get_default_args on, you must add the line enable_get_default_args(C) to it. This causes autogeneration of a hidden dataclass in the module.

Reviewed By: davnov134

Differential Revision: D35364410

fbshipit-source-id: 53f6e6fff43e7142ae18ca3b06de7d0c849ef965
2022-04-06 03:32:31 -07:00
yaookyie
4c48beb226 Fix scatter_ error in cubify (#1067)
Summary:
Error Reproduction:

python=3.8.12
pytorch=1.9.1
pytorch3d=0.6.1
cudatoolkit=11.1.74

test.py:
```python
import torch
from pytorch3d.ops import cubify
voxels = torch.Tensor([[[[0,1], [0,0]], [[0,1], [0,0]]]]).float()
meshes = cubify(voxels, 0.5, device="cpu")
```

The error appears when `device="cpu"` and `pytorch=1.9.1` (works fine with pytorch=1.10.2)

Error message:
```console
/home/kyle/anaconda3/envs/adapt-net/lib/python3.8/site-packages/torch/_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at  /opt/conda/conda-bld/pytorch_1631630839582/work/aten/src/ATen/native/BinaryOps.cpp:467.)
  return torch.floor_divide(self, other)
Traceback (most recent call last):
  File "test.py", line 5, in <module>
    meshes = cubify(voxels, 0.5, device="cpu")
  File "/home/kyle/anaconda3/envs/adapt-net/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "/home/kyle/Desktop/pytorch3d/pytorch3d/ops/cubify.py", line 227, in cubify
    idleverts.scatter_(0, grid_faces.flatten(), 0)
RuntimeError: Expected index [60] to be smaller than self [27] apart from dimension 0 and to be smaller size than src [27]
```

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1067

Reviewed By: nikhilaravi

Differential Revision: D34893567

Pulled By: bottler

fbshipit-source-id: aa95980f7319302044141f7821ef48129cfa37a6
2022-04-05 13:16:36 -07:00
David Novotny
4db9fc11d2 Allow setting bin_size for render_point_clouds_pytorch3d
Summary: This is required to suppress a huge stdout full of warnings about overflown bins.

Reviewed By: bottler

Differential Revision: D35359824

fbshipit-source-id: 39214b1bdcb4a5d5debf8ed498b2ca81fa43d210
2022-04-04 09:26:54 -07:00
Jeremy Reizenstein
3b8a33e9c5 store original declared types in Configurable
Summary: Aid reflection by adding the original declared types of replaced members of a configurable as values in _processed_members.

Reviewed By: davnov134

Differential Revision: D35358422

fbshipit-source-id: 80ef3266144c51c1c2105f349e0dd3464e230429
2022-04-04 07:19:56 -07:00
Jeremy Reizenstein
199309fcf7 logging
Summary: Use logging instead of printing in the internals of implicitron.

Reviewed By: davnov134

Differential Revision: D35247581

fbshipit-source-id: be5ddad5efe1409adbae0575d35ade6112b3be63
2022-04-04 06:53:16 -07:00
Jeremy Reizenstein
6473aa316c avoid visdom import in tests
Summary: This might make the implicitron tests work better on RE.

Reviewed By: davnov134

Differential Revision: D35283131

fbshipit-source-id: 4dda9684f632ab6e9cebcbf1e6e4a8243ec00c85
2022-04-04 04:43:33 -07:00
Jeremy Reizenstein
2802fd9398 fix Optional[List] in Configurable
Summary: Optional[not_a_type] was causing errors.

Reviewed By: davnov134

Differential Revision: D35355530

fbshipit-source-id: e9b52cfd6347ffae0fe688ef30523a4092ccf9fd
2022-04-04 04:28:17 -07:00
Roman Shapovalov
a999fc22ee Type safety fixes
Summary: Pyre expects Mapping for ** operator.

Reviewed By: bottler

Differential Revision: D35288632

fbshipit-source-id: 34d6f26ad912b3a5046f440922bb6ed2fd86f533
2022-04-01 04:24:46 -07:00
Jeremy Reizenstein
24260130ce _allow_untyped for get_default_args
Summary:
ListConfig and DictConfig members of get_default_args(X) when X is a callable will contain references to a temporary dataclass and therefore be unpicklable. Avoid this in a few cases.

Fixes https://github.com/facebookresearch/pytorch3d/issues/1144

Reviewed By: shapovalov

Differential Revision: D35258561

fbshipit-source-id: e52186825f52accee9a899e466967a4ff71b3d25
2022-03-31 06:31:45 -07:00
Roman Shapovalov
a54ad2b912 get_default_args for callables respects non-class type annotations and Optionals
Summary: as subj

Reviewed By: davnov134

Differential Revision: D35194863

fbshipit-source-id: c8e8f234083d4f0f93dca8d93e090ca0e1e1972d
2022-03-29 11:36:11 -07:00
janEbert
b602edccc4 Fix dtype propagation (#1141)
Summary:
Previously, dtypes were not propagated correctly in composed transforms, resulting in errors when different dtypes were mixed. Even specifying a dtype in the constructor does not fix this. Neither does specifying the dtype for each composition function invocation (e.g. as a `kwarg` in `rotate_axis_angle`).

With the change, I also had to modify the default dtype of `RotateAxisAngle`, which was `torch.float64`; it is now `torch.float32` like for all other transforms. This was required because the fix in propagation broke some tests due to dtype mismatches.

This change in default dtype in turn broke two tests due to precision changes (calculations that were previously done in `torch.float64` were now done in `torch.float32`), so I changed the precision tolerances to be less strict. I chose the lowest power of ten that passed the tests here.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1141

Reviewed By: patricklabatut

Differential Revision: D35192970

Pulled By: bottler

fbshipit-source-id: ba0293e8b3595dfc94b3cf8048e50b7a5e5ed7cf
2022-03-29 08:57:42 -07:00
Jeremy Reizenstein
21262e38c7 Optional ReplaceableBase
Summary: Allow things like `renderer:Optional[BaseRenderer]` in configurables.

Reviewed By: davnov134

Differential Revision: D35118339

fbshipit-source-id: 1219321b2817ed4b26fe924c6d6f73887095c985
2022-03-29 08:43:46 -07:00
Jeremy Reizenstein
e332f9ffa4 test_build for implicitron
Summary: To ensure that tests outside implicitron/ don't use implicitron, split the test for recursive includes in to two. License header checking is not needed here any more.

Reviewed By: shapovalov

Differential Revision: D35077830

fbshipit-source-id: 2ebe7436a6dcc5d21a116434f6ddd08705dfab34
2022-03-29 05:09:27 -07:00
Jeremy Reizenstein
0c3bed55be setup.py for implicitron_trainer
Summary: Enable `pytorch3d_implicitron_runner` executable

Reviewed By: shapovalov

Differential Revision: D34754902

fbshipit-source-id: 213f3e9183e3f7dd7b4df16ad77d95fbc971d625
2022-03-28 04:50:26 -07:00
Jeremy Reizenstein
97894fb37b Reinforce test skipping
Summary: Attempt to solve an internal issue

Reviewed By: shapovalov

Differential Revision: D35143263

fbshipit-source-id: b4fd9ee441d85f0a3ee08f2f1e7febd1c1ccbe86
2022-03-25 07:25:54 -07:00
Roman Shapovalov
645a47d054 Return a typed structured config from default_args for callables
Summary:
Before the fix, running get_default_args(C: Callable) returns an unstructured DictConfig which causes Enums to be handled incorrectly. This is a fix.

WIP update: Currently tests still fail whenever a function signature contains an untyped argument: This needs to be somehow fixed.

Reviewed By: bottler

Differential Revision: D34932124

fbshipit-source-id: ecdc45c738633cfea5caa7480ba4f790ece931e8
2022-03-25 07:08:01 -07:00
Jeremy Reizenstein
8ac5e8f083 add missing __init__.py files
Summary: Some directories in implicitron were missing __init__.py files.

Reviewed By: patricklabatut

Differential Revision: D35076364

fbshipit-source-id: f74442766efe8694fdd47954ac4882e7c4daac60
2022-03-24 07:04:38 -07:00
Jeremy Reizenstein
92f9dfe9d6 overflow warning typo
Summary: bin_size should be 0 not -1 for naive rasterization. See https://github.com/facebookresearch/pytorch3d/issues/1129

Reviewed By: patricklabatut

Differential Revision: D35077115

fbshipit-source-id: b81ff74f47c78429977802f7dcadfd1b96676f8c
2022-03-24 06:53:35 -07:00
Jeremy Reizenstein
f2cf9d4d0b windows fix
Summary: Attempt to reduce nvcc trouble on windows by (1) avoiding flag for c++14 and (2) avoiding `torch/extension.h`, which introduces pybind11, in `.cu` files.

Reviewed By: patricklabatut

Differential Revision: D34969868

fbshipit-source-id: f3878d6a2ba9d644e87ae7b6377cb5008b4b6ce3
2022-03-24 06:52:05 -07:00
Roman Shapovalov
e2622d79c0 Using the new dataset idx API everywhere.
Summary: Using the API from D35012121 everywhere.

Reviewed By: bottler

Differential Revision: D35045870

fbshipit-source-id: dab112b5e04160334859bbe8fa2366344b6e0f70
2022-03-24 05:33:25 -07:00
Roman Shapovalov
c0bb49b5f6 API for accessing frames in order in Implicitron dataset.
Summary: We often want to iterate over frames in the sequence in temporal order. This diff provides the API to do that. `seq_to_idx` should probably be considered to have `protected` visibility.

Reviewed By: davnov134

Differential Revision: D35012121

fbshipit-source-id: 41896672ec35cd62f3ed4be3aa119efd33adada1
2022-03-24 05:33:25 -07:00
Jeremy Francis Reizenstein
05f656c01f Update ShipIt Sync
fbshipit-source-id: d20e2f3d7ae6ca8c4a1e72002c1be8d75217939d
2022-03-23 11:07:56 -07:00
Jeremy Francis Reizenstein
4c22855a23 Update ShipIt Sync
fbshipit-source-id: 29b8a643c0218375bf90b9c1fb8853dedd0906fe
2022-03-23 08:50:53 -07:00
Jeremy Reizenstein
cdd2142dd5 implicitron v0 (#1133)
Co-authored-by: Jeremy Francis Reizenstein <bottler@users.noreply.github.com>
2022-03-21 13:20:10 -07:00
Roman Shapovalov
0e377c6850 Monte-Carlo rasterisation; arbitrary dimensionality of AlphaCompositor blending
Summary:
Fixes required for MC rasterisation to work.
1) Wrong number of channels for background was used (derived from points dimensions, not features dimensions;
2) split of the results on the wrong dimension was done;
3) CORE CHANGE: blending in alpha compositor was assuming RGBA input.

Reviewed By: davnov134

Differential Revision: D34933673

fbshipit-source-id: a5cc9f201ea21e114639ab9e291a10888d495206
2022-03-17 05:12:39 -07:00
Roman Shapovalov
e64f25c255 README file.
Summary: as subj

Reviewed By: davnov134

Differential Revision: D34758227

fbshipit-source-id: c22e7c4c6e69e9ef872b46c99ece901c58c23d71
2022-03-16 07:42:52 -07:00
Jeremy Reizenstein
c85673c626 PyTorch 1.11.0
Summary: Add builds for PyTorch 1.11.0.

Reviewed By: nikhilaravi

Differential Revision: D34861021

fbshipit-source-id: 1a1c46fac48719bc66c81872e65531a48ff538ed
2022-03-16 05:44:40 -07:00
Xie Fangyuan
3de3c13a0f R2n2 (#1124)
Summary:
1. Fix https://github.com/facebookresearch/pytorch3d/issues/1115 Change the type annatations of three arguments in the initializer of `R2N2` class.
2. Fix https://github.com/facebookresearch/pytorch3d/issues/1118 Override two functions in `BlenderCamera` class reruired by subclassing `CamerasBase` class.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1124

Reviewed By: nikhilaravi

Differential Revision: D34890900

Pulled By: bottler

fbshipit-source-id: 65c385369a5964ecbb17ab28f279d5284614b487
2022-03-16 05:44:17 -07:00
Jeremy Reizenstein
9b5a3ffa6c PLY with uint face data (#1104)
Summary: Fix assumption that face indices are signed in the PLY file, as reported in #1104.

Reviewed By: nikhilaravi

Differential Revision: D34892598

fbshipit-source-id: a8b23bfac1357bdc11bbbf752098319142239804
2022-03-16 05:42:34 -07:00
Jeremy Reizenstein
1701b76a31 another meshgrid fix for old pytorch
Summary: Try to fix circleci again.

Reviewed By: nikhilaravi

Differential Revision: D34752188

fbshipit-source-id: 5966c585b61d77df1d8dd97c24383cf74dfb1fae
2022-03-11 01:14:59 -08:00
Jeremy Reizenstein
57a33b25c1 add MeshRendererWithFragments to __init__s
Summary: As noticed in https://github.com/facebookresearch/pytorch3d/issues/1098 , it would be useful to make this more available.

Reviewed By: nikhilaravi

Differential Revision: D34752526

fbshipit-source-id: 5a127bd557a0cd626f36bf194f22bc0a0a6a2436
2022-03-11 01:14:39 -08:00
Jeremy Reizenstein
c371a9a6cc rasterizer.to without cameras
Summary: As reported in https://github.com/facebookresearch/pytorch3d/pull/1100, a rasterizer couldn't be moved if it was missing the optional cameras member. Fix that. This matters because the renderer.to calls rasterizer.to, so this to() could be called even by a user who never sets a cameras member.

Reviewed By: nikhilaravi

Differential Revision: D34643841

fbshipit-source-id: 7e26e32e8bc585eb1ee533052754a7b59bc7467a
2022-03-08 23:54:38 -08:00
Jeremy Reizenstein
4a1f176054 fix _num_faces_per_mesh in join_batch
Summary: As reported in https://github.com/facebookresearch/pytorch3d/pull/1100, _num_faces_per_mesh was changing in the source mesh in join_batch. This affects both TexturesUV and TexturesAtlas

Reviewed By: nikhilaravi

Differential Revision: D34643675

fbshipit-source-id: d67bdaca7278f18a76cfb15ba59d0ea85575bd36
2022-03-08 23:54:37 -08:00
dmitryvinn
16d0aa82c1 docs: add social banner in support of Ukraine (#1101)
Summary:
Our mission at [Meta Open Source](https://opensource.facebook.com/) is to empower communities through open source, and we believe that it means building a welcoming and safe environment for all. As a part of this work, we are adding this banner in support for Ukraine during this crisis.

![image](https://user-images.githubusercontent.com/12485205/156670302-756fac7c-51ba-4e24-b463-f79730dbaba6.png)

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1101

Reviewed By: bottler

Differential Revision: D34628257

Pulled By: dmitryvinn-fb

fbshipit-source-id: 5863afb59a2b9431e8e2ebc5856254ab0cdfcfe8
2022-03-04 01:51:16 -08:00
Jeremy Reizenstein
69b27d160e reallow scalar background color for point rendering
Summary: A scalar background color is not meant to be allowed for the point renderer. It used to be ignored with a warning, but a recent code change made it an error. It was being used, at least in the black (value=0.0) case. Re-enable it.

Reviewed By: nikhilaravi

Differential Revision: D34519651

fbshipit-source-id: d37dcf145bb7b8999c9265cf8fc39b084059dd18
2022-03-01 05:12:55 -08:00
Andres Suarez
84a569c0aa Fix unnecessary LICENSELINT suppressions
Reviewed By: zsol

Differential Revision: D34526295

fbshipit-source-id: f511370dc3186bc396d68a2e6d5e0931facbeb42
2022-02-28 11:53:40 -08:00
Winnie Lin
471b126818 add min_triangle_area argument to IsInsideTriangle
Summary:
1. changed IsInsideTriangle in geometry_utils to take in min_triangle_area parameter instead of hardcoded value
2. updated point_mesh_cpu.cpp and point_mesh_cuda.[h/cu] to adapt to changes in geometry_utils function signatures
3. updated point_mesh_distance.py and test_point_mesh_distance.py to modify _C. calls

Reviewed By: bottler

Differential Revision: D34459764

fbshipit-source-id: 0549e78713c6d68f03d85fb597a13dd88e09b686
2022-02-25 12:43:04 -08:00
Jeremy Reizenstein
4d043fc9ac PyTorch 1.7 compatibility
Summary: Small changes discovered based on circleCI failures.

Reviewed By: patricklabatut

Differential Revision: D34426807

fbshipit-source-id: 819860f34b2f367dd24057ca7490284204180a13
2022-02-25 07:53:34 -08:00
Jeremy Reizenstein
f816568735 rename types to avoid clash
Summary: There are cases where importing pytorch3d seems to fail (internally at Meta) because of a clash between the builtin types module and ours, so rename ours.

Reviewed By: patricklabatut

Differential Revision: D34426817

fbshipit-source-id: f175448db6a4967a9a3f7bb6f595aad2ffb36455
2022-02-25 07:53:34 -08:00
Jeremy Reizenstein
0e88b21de6 Use newer circleci image
Summary:
Run the circleci tests with a non depracated circleci image. Small fix for PyTorch 1.7.
We no longer need to manually install nvidia-docker or the CUDA driver.

Reviewed By: patricklabatut

Differential Revision: D34426816

fbshipit-source-id: d6c67bfb0ff86dff8d8f7fe7b8801657c2e80030
2022-02-25 07:53:34 -08:00
Theo-Cheynel
1cbf80dab6 Added matrix_to_axis_angle to the exports of transforms (#1085)
Summary:
# Changelist
- `matrix_to_axis_angle` was declared in `pytorch3d/transforms/rotation_conversions.py` but never exported from the `__init__` file.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1085

Reviewed By: patricklabatut

Differential Revision: D34379935

Pulled By: bottler

fbshipit-source-id: 993c12a176630f91d0f107f298f458b2b35032e5
2022-02-21 11:27:13 -08:00
Georgia Gkioxari
ee71c7c447 small numerical fix to point_mesh
Summary: Small fix by adjusting the area `eps` to account for really small faces when computing point to face distances

Reviewed By: bottler

Differential Revision: D34331336

fbshipit-source-id: 51c4888ea46fefa4e31d5b0bb494a9f9d77813cd
2022-02-21 09:26:38 -08:00
Georgia Gkioxari
3de41223dd lower eps
Summary: Lower the epsilon value in the IoU3D calculation to fix small numerical issue from GH#1082

Reviewed By: bottler

Differential Revision: D34371597

fbshipit-source-id: 12443fa359b7755ef4ae60e9adf83734a1a295ae
2022-02-21 09:26:38 -08:00
Jeremy Reizenstein
967a099231 Use dataclasses inside ply_io.
Summary: Refactor ply_io to make it easier to add new features. Mostly taken from the starting code I attached to https://github.com/facebookresearch/pytorch3d/issues/904.

Reviewed By: patricklabatut

Differential Revision: D34375978

fbshipit-source-id: ec017d31f07c6f71ba6d97a0623bb10be1e81212
2022-02-21 07:24:21 -08:00
Jeremy Reizenstein
feb5d36394 points2vols test fix
Summary: Fix tests which depended on output tensors being identical to input ones, which now fail in main PyTorch branch because of some change in autograd. The functions still work in-place.

Reviewed By: patricklabatut

Differential Revision: D34375817

fbshipit-source-id: 295ae195f75eab6c7abab412c997470d8de8add1
2022-02-21 07:24:21 -08:00
Jeremy Reizenstein
db1f7c4506 avoid symeig
Summary: Use the newer eigh to avoid deprecation warnings in newer pytorch.

Reviewed By: patricklabatut

Differential Revision: D34375784

fbshipit-source-id: 40efe0d33fdfa071fba80fc97ed008cbfd2ef249
2022-02-21 07:24:21 -08:00
Alex Greene
59972b121d flexible background color for point compositing
Summary:
Modified the compositor background color tests to account for either a 3rd or 4th channel. Also replaced hard coding of channel value with C.

Implemented changes to alpha channel appending logic, and cleaned up extraneous warnings and checks, per task instructions.

Fixes https://github.com/facebookresearch/pytorch3d/issues/1048

Reviewed By: bottler

Differential Revision: D34305312

fbshipit-source-id: 2176c3bdd897d1a2ba6ff4c6fa801fea889e4f02
2022-02-18 07:01:22 -08:00
Jeremy Reizenstein
c8f3d6bc0b Fix Transform3d.stack of compositions
Summary:
Add a test for Transform3d.stack, and make it work with composed transformations.

Fixes https://github.com/facebookresearch/pytorch3d/issues/1072 .

Reviewed By: patricklabatut

Differential Revision: D34211920

fbshipit-source-id: bfbd0895494ca2ad3d08a61bc82ba23637e168cc
2022-02-15 06:52:41 -08:00
Jeremy Reizenstein
2a1de3b610 move LinearWithRepeat to pytorch3d
Summary: Move this simple layer from the NeRF project into pytorch3d.

Reviewed By: shapovalov

Differential Revision: D34126972

fbshipit-source-id: a9c6d6c3c1b662c1b844ea5d1b982007d4df83e6
2022-02-14 04:52:30 -08:00
Sergei Ovchinnikov
ef21a6f6aa Importing obj files without usemtl
Summary:
When there is no "usemtl" statement in the .obj file use material from .mtl if there is one.
https://github.com/facebookresearch/pytorch3d/issues/1068

Reviewed By: bottler

Differential Revision: D34141152

fbshipit-source-id: 7a5b5cc3f0bb287dc617f68de2cd085db8f7ad94
2022-02-10 09:39:44 -08:00
David Novotny
12f20d799e Convert from Pytorch3D NDC coordinates to grid_sample coordinates.
Summary: Implements a utility function to convert from 2D coordinates in Pytorch3D NDC space to the coordinates in grid_sample.

Reviewed By: shapovalov

Differential Revision: D33741394

fbshipit-source-id: 88981653356588fe646e6dea48fe7f7298738437
2022-02-09 12:49:55 -08:00
Jeremy Reizenstein
47c0997227 Followup D33970393 (auto typing)
Summary: D33970393 (e9fb6c27e3) ran an inference to add some typing. Remove some where it was a bit too confident. (Also fix some pyre errors in plotly_vis caused by new mismatch.)

Reviewed By: patricklabatut

Differential Revision: D34004689

fbshipit-source-id: 430182b0ff0b91be542a3120da6d6b1d2b247c59
2022-02-09 12:42:56 -08:00
Pyre Bot Jr
e9fb6c27e3 Add annotations to vision/fair/pytorch3d
Reviewed By: shannonzhu

Differential Revision: D33970393

fbshipit-source-id: 9b4dfaccfc3793fd37705a923d689cb14c9d26ba
2022-02-03 01:46:32 -08:00
Jeremy Reizenstein
c2862ff427 use workaround for points_normals
Summary:
Use existing workaround for batched 3x3 symeig because it is faster than torch.symeig.

Added benchmark showing speedup. True = workaround.
```
Benchmark                Avg Time(μs)      Peak Time(μs) Iterations
--------------------------------------------------------------------------------
normals_True_3000            16237           17233             31
normals_True_6000            33028           33391             16
normals_False_3000        18623069        18623069              1
normals_False_6000        36535475        36535475              1
```

Should help https://github.com/facebookresearch/pytorch3d/issues/988

Reviewed By: nikhilaravi

Differential Revision: D33660585

fbshipit-source-id: d1162b277f5d61ed67e367057a61f25e03888dce
2022-01-24 11:41:55 -08:00
Jeremy Reizenstein
5053142363 typing for unproject_points
Summary: Fix the base class annotation for unproject_points.

Reviewed By: patricklabatut

Differential Revision: D33281586

fbshipit-source-id: 1c34e8c4b30b359fcb9307507bc778ad3fecf290
2022-01-24 10:52:24 -08:00
Jeremy Reizenstein
67778caee8 avoid deprecated raysamplers
Summary: Migrate away from NDCGridRaysampler and GridRaysampler to their more flexible replacements.

Reviewed By: patricklabatut

Differential Revision: D33281584

fbshipit-source-id: 65f8702e700a32d38f7cd6bda3924bb1707a0633
2022-01-24 10:52:23 -08:00
Jeremy Reizenstein
3eb4233844 New raysamplers
Summary: New MultinomialRaysampler succeeds GridRaysampler bringing masking and subsampling. Correspondingly, NDCMultinomialRaysampler succeeds NDCGridRaysampler.

Reviewed By: nikhilaravi, shapovalov

Differential Revision: D33256897

fbshipit-source-id: cd80ec6f35b110d1d20a75c62f4e889ba8fa5d45
2022-01-24 10:52:23 -08:00
Jeremy Reizenstein
174738c33e safer pip install in doc
Summary: Add --no-cache and --no-index to all commands which try to download wheels from S3, to avoid hitting pypi.

Reviewed By: nikhilaravi

Differential Revision: D33507975

fbshipit-source-id: ee796e43cc1864e475cd73c248e9900487012f25
2022-01-21 06:28:32 -08:00
Jeremy Reizenstein
45d096e219 cameras_from_opencv_projection device #1021
Summary: Fix https://github.com/facebookresearch/pytorch3d/issues/1021 that cameras_from_opencv_projection always creates on CPU.

Reviewed By: nikhilaravi

Differential Revision: D33508211

fbshipit-source-id: fadebd45cacafd633af6a58094cf6f654529992c
2022-01-21 05:32:20 -08:00
Jeremy Reizenstein
39bb2ce063 Join cameras as batch
Summary:
Function to join a list of cameras objects into a single batched object.

FB: In the next diff I will remove the `concatenate_cameras` function in implicitron and update the callsites.

Reviewed By: nikhilaravi

Differential Revision: D33198209

fbshipit-source-id: 0c9f5f5df498a0def9dba756c984e6a946618158
2022-01-21 05:29:43 -08:00
Jeremy Reizenstein
9e2bc3a17f ambient lights batching #1043
Summary:
convert_to_tensors_and_broadcast had a special case for a single input, which is not used anywhere except fails to do the right thing if a TensorProperties has only one kwarg. At the moment AmbientLights may be the only way to hit the problem. Fix by removing the special case.

Fixes https://github.com/facebookresearch/pytorch3d/issues/1043

Reviewed By: nikhilaravi

Differential Revision: D33638345

fbshipit-source-id: 7a6695f44242e650504320f73b6da74254d49ac7
2022-01-20 09:44:38 -08:00
Jeremy Reizenstein
fddd6a700f drop builds for PyTorch 1.6.0
Summary: PyTorch 1.7.0 was in Oct 2020 and 1.7.1 was in Dec 2020. We shouldn't need older than them, maybe not even 1.7.0.

Reviewed By: patricklabatut

Differential Revision: D33507967

fbshipit-source-id: d3de09c20c44870cbe5522705f2293acc0e62af3
2022-01-10 10:04:02 -08:00
Jeremy Reizenstein
85cdcc252d builds for PyTorch 1.10.1
Summary: Adds 1.10.1 to the nightly builds

Reviewed By: patricklabatut

Differential Revision: D33507966

fbshipit-source-id: af88b155adbc4e3236107f709323bd46a1819013
2022-01-10 10:04:02 -08:00
Jeremy Reizenstein
fc4dd80208 initialize pointcloud from list containing Nones
Summary:
The following snippet should work in more cases.
     point_cloud = Pointclouds(
         [pcl.points_packed() for pcl in point_clouds],
         features=[pcl.features_packed() for pcl in point_clouds],
     )

We therefore allow features and normals inputs to be lists which contain some (but not all) Nones.

The initialization of a Pointclouds from empty data is also made a bit better now at working out how many feature channels there are.

Reviewed By: davnov134

Differential Revision: D31795089

fbshipit-source-id: 54bf941ba80672d699ffd5ac28927740e830f8ab
2022-01-07 05:54:44 -08:00
Jeremy Reizenstein
9640560541 test listing
Summary: Quick script to list tests to help completion of test command.

Reviewed By: patricklabatut

Differential Revision: D33279584

fbshipit-source-id: acb463106d311498449a14c1daf52434878722bf
2022-01-06 02:56:32 -08:00
Jeremy Reizenstein
6726500ad3 simple warning for bin overflow
Summary: Since coarse rasterization on cuda can overflow bins, we detect when this happens for memory safety. See https://github.com/facebookresearch/pytorch3d/issues/348 . Also try to print a warning.

Reviewed By: patricklabatut

Differential Revision: D33065604

fbshipit-source-id: 99b3c576d01b78e6d77776cf1a3e95984506c93a
2022-01-06 02:30:49 -08:00
Jeremy Reizenstein
d6a12afbe7 Pointclouds.subsample on Windows
Summary: Fix https://github.com/facebookresearch/pytorch3d/issues/1015. Stop relying on the fact that the dtype returned by np.random.choice (int64 on Linux, int32 on Windows) matches the dtype used by pytorch for indexing (int64 everywhere).

Reviewed By: patricklabatut

Differential Revision: D33428680

fbshipit-source-id: 716c857502cd54c563cb256f0eaca7dccd535c10
2022-01-06 02:22:44 -08:00
Jeremy Reizenstein
49f93b6388 remove Python3.6 builds
Summary: Python 3.6 was EOL on 2021-12-23.

Reviewed By: patricklabatut

Differential Revision: D33428708

fbshipit-source-id: 37a73898df49a4a49266839278fc8be56597405d
2022-01-05 07:25:02 -08:00
Jeremy Reizenstein
741777b5b5 More company name & License
Summary: Manual adjustments for license changes.

Reviewed By: patricklabatut

Differential Revision: D33405657

fbshipit-source-id: 8a21735726f3aece9f9164da9e3b272b27db8032
2022-01-04 11:43:38 -08:00
Jeremy Reizenstein
9eeb456e82 Update license for company name
Summary: Update all FB license strings to the new format.

Reviewed By: patricklabatut

Differential Revision: D33403538

fbshipit-source-id: 97a4596c5c888f3c54f44456dc07e718a387a02c
2022-01-04 11:43:38 -08:00
Pyre Bot Jr
7660ed1876 suppress errors in fbcode/vision - batch 2
Differential Revision: D33338085

fbshipit-source-id: fdb207864718c56dfa0d20530b59349c93af11bd
2021-12-28 11:28:48 -08:00
Nikhila Ravi
52c71b8816 Update Harmonic embedding in NeRF
Summary: Removed harmonic embedding function from projects/nerf and changed import to be from core pytorch3d.

Reviewed By: patricklabatut

Differential Revision: D33142358

fbshipit-source-id: 3004247d50392dbd04ea72e9cd4bace0dc03606b
2021-12-21 15:05:33 -08:00
Nikhila Ravi
f9a26a22fc Move Harmonic embedding to core pytorch3d
Summary:
Moved `HarmonicEmbedding` function in core PyTorch3D.
In the next diff will update the NeRF project.

Reviewed By: bottler

Differential Revision: D32833808

fbshipit-source-id: 0a12ccd1627c0ce024463c796544c91eb8d4d122
2021-12-21 15:05:33 -08:00
Nikhila Ravi
d67662d13c Update use of select_cameras
Summary: Removed `select_cameras.py` from implicitron and updated all callsites to directly index the cameras.

Reviewed By: bottler

Differential Revision: D33187605

fbshipit-source-id: aaf5b36aef9d72db0c7e89dec519f23646f6aa05
2021-12-21 05:46:38 -08:00
Nikhila Ravi
28ccdb7328 Enable __getitem__ for Cameras to return an instance of Cameras
Summary:
Added a custom `__getitem__` method to `CamerasBase` which returns an instance of the appropriate camera instead of the `TensorAccessor` class.

Long term we should deprecate the `TensorAccessor` and the `__getitem__` method on `TensorProperties`

FB: In the next diff I will update the uses of `select_cameras` in implicitron.

Reviewed By: bottler

Differential Revision: D33185885

fbshipit-source-id: c31995d0eb126981e91ba61a6151d5404b263f67
2021-12-21 05:46:38 -08:00
Jeremy Reizenstein
cc3259ba93 update linux wheel builds
Summary:
* Add PyTorch 1.10 + CUDA 11.1 combination.
* Change the CUDA 11.3 builds to happen in a separate docker image.
* Update connection to AWS to use the official `aws` commands instead of the wrapper which is now gone.

Reviewed By: patricklabatut

Differential Revision: D33235489

fbshipit-source-id: 56401f27c002a512ae121b3ec5911d020bfab885
2021-12-21 05:07:41 -08:00
Jeremy Reizenstein
b51be58f63 validate sampling_mode
Summary: New sampling mode option in TexturesUV mush match when collating meshes.

Reviewed By: patricklabatut

Differential Revision: D33235901

fbshipit-source-id: f457473d90bf022e65fe122ef45bf5efad134345
2021-12-21 04:48:14 -08:00
Jeremy Reizenstein
7449951850 force old mistune for website
Summary: When parsing tutorials for building the website, we fix a few libraries at old versions. They need mistune 0.8.4, not the new version 2+, so this gets added to the list of fixed-version libraries.

Reviewed By: patricklabatut

Differential Revision: D33236031

fbshipit-source-id: 2b152b64043edffc59fa909012eab5794c7e8844
2021-12-21 04:44:36 -08:00
Nikhila Ravi
262c1bfcd4 Join points as batch
Summary: Function to join a list of pointclouds as a batch similar to the corresponding function for Meshes.

Reviewed By: bottler

Differential Revision: D33145906

fbshipit-source-id: 160639ebb5065e4fae1a1aa43117172719f3871b
2021-12-21 04:44:36 -08:00
Jeremy Reizenstein
eb2bbf8433 screen space docstrings fix
Summary: Fix some comments to match the recent change to transform_points_screen.

Reviewed By: patricklabatut

Differential Revision: D33243697

fbshipit-source-id: dc8d182667a9413bca2c2e3657f97b2f7a47c795
2021-12-21 04:31:33 -08:00
Jeremy Reizenstein
1152a93b72 PyTorch>1.9 version str
Summary: Make code for downloading linux wheels robust to double-digit PyTorch minor version.

Reviewed By: nikhilaravi

Differential Revision: D33170562

fbshipit-source-id: 559a97cc98ff8411e235a9f9e29f84b7a400c716
2021-12-18 15:49:17 -08:00
Pyre Bot Jr
315f2487db suppress errors in vision/fair/pytorch3d
Differential Revision: D33202801

fbshipit-source-id: d4cb0f4f4a8ad5a6519ce4b8c640e8f96fbeaccb
2021-12-17 19:23:58 -08:00
Georgia Gkioxari
ccfb72cc50 small fix for iou3d
Summary:
A small numerical fix for IoU for 3D boxes, fixes GH #992

* Adds a check for boxes with zero side areas (invalid boxes)
* Fixes numerical issue when two boxes have coplanar sides

Reviewed By: nikhilaravi

Differential Revision: D33195691

fbshipit-source-id: 8a34b4d1f1e5ec2edb6d54143930da44bdde0906
2021-12-17 16:13:51 -08:00
Jeremy Reizenstein
069c9fd759 pytorch TORCH_CHECK_ARG version compatibility
Summary: Restore compatibility with old C++ after recent torch change. https://github.com/facebookresearch/pytorch3d/issues/995

Reviewed By: patricklabatut

Differential Revision: D33093174

fbshipit-source-id: 841202fb875d601db265e93dcf9cfa4249d02b25
2021-12-15 08:34:10 -08:00
CodemodService FBSourceClangFormatLinterBot
9eec430f1c Daily arc lint --take CLANGFORMAT
Reviewed By: zertosh

Differential Revision: D33090919

fbshipit-source-id: 78efa486776014a27f280a01a21f9e0af6742e3e
2021-12-14 08:06:14 -08:00
Peter Bell
f8fe9a2be1 Remove THGeneral (#69041)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/69041

`TH_CONCAT_{N}` is still being used by THP so I've moved that into
it's own header but all the compiled code is gone.

Test Plan: Imported from OSS

Reviewed By: anjali411

Differential Revision: D32872477

Pulled By: ngimel

fbshipit-source-id: 06c82d8f96dbcee0715be407c61dfc7d7e8be47a
2021-12-13 16:14:17 -08:00
Jeremy Reizenstein
d049cd2e01 PyTorch 1.10 + CUDA 11.1 builds
Summary: Although the PyTorch website, which describes the current version 1.10, suggests CUDA 10.2 and 11.3 are supported, it would appear that we need to include builds for CUDA 11.1 to avoid surprises. This is because these builds are on anaconda, and this combination is used on Google Colab.

Reviewed By: nikhilaravi

Differential Revision: D33063932

fbshipit-source-id: 1b22d1f06e22bd18fb53ceecb58e78ac6a5d1693
2021-12-13 10:11:00 -08:00
Jeremy Reizenstein
1edc624d82 version 0.6.1
Summary: Update version number

Reviewed By: patricklabatut

Differential Revision: D33016833

fbshipit-source-id: ee3b0997887ab3bc5779503b13fa2014df41eaed
2021-12-13 04:38:16 -08:00
Jeremy Reizenstein
6ea6314792 [pytorch3d install.md for 0.6.1
Summary: Update references to dependencies

Reviewed By: patricklabatut

Differential Revision: D33016832

fbshipit-source-id: aa41c7ccc6acd19654303bc18bfd734dc29d88a3
2021-12-13 04:38:16 -08:00
Jeremy Reizenstein
093999e71f update tutorials for release
Summary: Pre 0.6.1 release, make the tutorials expect wheels with PyTorch 1.10.0

Reviewed By: patricklabatut

Differential Revision: D33016834

fbshipit-source-id: b8c5c1c6158f806c3e55ec668117fa762fa4b75f
2021-12-13 04:38:16 -08:00
Jeremy Reizenstein
a22b1e32a4 linux builds for PyTorch 1.10.0
Summary: Build the wheels with latest PyTorch

Reviewed By: patricklabatut

Differential Revision: D33016835

fbshipit-source-id: 0ec42f31f1e4d4055562f18790f929b34bb13c52
2021-12-13 04:38:16 -08:00
CodemodService FBSourceClangFormatLinterBot
9c9d9440f9 Daily arc lint --take CLANGFORMAT
Reviewed By: zertosh

Differential Revision: D32975574

fbshipit-source-id: 66856595c7bc29921f24a2c5c00c72892f262aa1
2021-12-09 00:10:21 -08:00
Peter Bell
c65af9ef5a Remove remaining THC code (#69039)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/69039

Test Plan: Imported from OSS

Reviewed By: anjali411

Differential Revision: D32872476

Pulled By: ngimel

fbshipit-source-id: 7972aacc24aef9450fb59b707ed6396c501bcb31
2021-12-08 12:15:58 -08:00
Jeremy Reizenstein
70acb3e415 new tests demonstrating pixel matching
Summary: Demonstrate current behavior of pixels with new tests of all renderers.

Reviewed By: gkioxari

Differential Revision: D32651141

fbshipit-source-id: 3ca30b4274ed2699bc5e1a9c6437eb3f0b738cbf
2021-12-07 15:04:20 -08:00
Jeremy Reizenstein
bf3bc6f8e3 screen cameras lose -1
Summary:
All the renderers in PyTorch3D (pointclouds including pulsar, meshes, raysampling) use align_corners=False style. NDC space goes between the edges of the outer pixels. For a non square image with W>H, the vertical NDC space goes from -1 to 1 and the horizontal from -W/H to W/H.

However it was recently pointed out that functionality which deals with screen space inside the camera classes is inconsistent with this. It unintentionally uses align_corners=True. This fixes that.

This would change behaviour of the following:
- If you create a camera in screen coordinates, i.e. setting in_ndc=False, then anything you do with the camera which touches NDC space may be affected, including trying to use renderers. The transform_points_screen function will not be affected...
- If you call the function “transform_points_screen” on a camera defined in NDC space results will be different. I have illustrated in the diff how to get the old results from the new results but this probably isn’t the right long-term solution..

Reviewed By: gkioxari

Differential Revision: D32536305

fbshipit-source-id: 377325a9137282971dcb7ca11a6cba3fc700c9ce
2021-12-07 15:04:20 -08:00
Jeremy Reizenstein
cff4876131 add from_ndc to unproject_points
Summary: Give unproject_points an argument letting it bypass screen space. use it to let the raysampler work for cameras defined in screen space.

Reviewed By: gkioxari

Differential Revision: D32596600

fbshipit-source-id: 2fe585dcd138cdbc65dd1c70e1957fd894512d3d
2021-12-07 15:04:20 -08:00
Jeremy Reizenstein
a0e2d2e3c3 move benchmarks to separate directory
Summary: Move benchmarks to a separate directory as tests/ is getting big.

Reviewed By: nikhilaravi

Differential Revision: D32885462

fbshipit-source-id: a832662a494ee341ab77d95493c95b0af0a83f43
2021-12-07 10:26:50 -08:00
Roman Shapovalov
a6508ac3df Fix: Pointclouds.inside_box reducing over spatial dimensions.
Summary: As subj. Tests corrected accordingly. Also changed the test to provide a bit better diagnostics.

Reviewed By: bottler

Differential Revision: D32879498

fbshipit-source-id: 0a852e4a13dcb4ca3e54d71c6b263c5d2eeaf4eb
2021-12-06 07:45:46 -08:00
Ana Dodik
d9f709599b Adding the option to choose the texture sampling mode in TexturesUV.
Summary:
This diff adds the `sample_mode` parameter to `TexturesUV` to control the interpolation mode during texture sampling. It simply gets forwarded to `torch.nn.funcitonal.grid_sample`.

This option was requested in this [GitHub issue](https://github.com/facebookresearch/pytorch3d/issues/805).

Reviewed By: patricklabatut

Differential Revision: D32665185

fbshipit-source-id: ac0bc66a018bd4cb20d75fec2d7c11145dd20199
2021-11-29 07:01:28 -08:00
Patrick Labatut
e4456dba2f Facebook -> Meta Platforms on website footer + docs
Summary: Update company copyright on website footer + documentation pages, see [guidelines](https://www.internalfb.com/intern/wiki/Open_Source/Launch_an_OSS_Project/Launch_Preparation/Automated_Checkup/Terms_Of_Use_&_Privacy_Policy/).

Reviewed By: bottler

Differential Revision: D32649563

fbshipit-source-id: f285be79c185496832c5d41b839ee974234a8fa5
2021-11-24 10:07:15 -08:00
Jeremy Reizenstein
7fa333f632 Fix some Transform3D -> Transform3d
Summary: Fix some typos in comments.

Reviewed By: patricklabatut

Differential Revision: D32596645

fbshipit-source-id: 09b6d8c49f4f0301b80df626c6f9a2d5b5d9b1a7
2021-11-23 11:31:11 -08:00
Jeremy Reizenstein
a0247ea6bd pulsar image_size validation
Summary:
For a non-square image, the image_size in PointsRasterizationSettings is now (H,W) not (W,H). A part of pulsar's validation code wasn't updated for this.

The following now works.
```
H, W = 249, 125
image_size = (H, W)
camera = PerspectiveCameras(focal_length=1.0, image_size=(image_size,), in_ndc=True)
points_rasterizer = PointsRasterizer(cameras=camera, raster_settings=PointsRasterizationSettings(image_size=image_size, radius=0.0000001))
pulsar_renderer = PulsarPointsRenderer(rasterizer=points_rasterizer)
pulsar_renderer(Pointclouds(...), gamma = (0.1,), znear = (0.1,), zfar = (70,))
```

Reviewed By: nikhilaravi, classner

Differential Revision: D32316322

fbshipit-source-id: 8405a49acecb1c95d37ee368c3055868b797208a
2021-11-17 15:18:01 -08:00
Pyre Bot Jr
a8cb7fa862 suppress errors in fbcode/vision - batch 2
Differential Revision: D32509948

fbshipit-source-id: 762ad27c7e6c76c30eb97fd44f1739295f63b98b
2021-11-17 14:38:11 -08:00
Jeremy Reizenstein
7ce18f38cd TexturesAtlas in plotly
Summary:
Lets a K=1 textures atlas be viewed in plotly. Fixes https://github.com/facebookresearch/pytorch3d/issues/916 .

Test: Now get colored faces in
```
import torch
from pytorch3d.utils import ico_sphere
from pytorch3d.vis.plotly_vis import plot_batch_individually
from pytorch3d.renderer import TexturesAtlas

b = ico_sphere()
face_colors = torch.rand(b.faces_padded().shape)
tex = TexturesAtlas(face_colors[:,:,None,None])
b.textures=tex
plot_batch_individually(b)
```

Reviewed By: gkioxari

Differential Revision: D32190470

fbshipit-source-id: 258d30b7e9d79751a79db44684b5540657a2eff5
2021-11-11 02:15:33 -08:00
Jeremy Reizenstein
5fbdb99aec builds for pytorch 1.10.0
Summary:
Add builds corresponding to the new pytorch 1.10.0. We omit CUDA 11.3 testing because it fails with current hardware, and omit the main build too for the moment.

Also move to the newer GPU circle CI executors.

Reviewed By: gkioxari

Differential Revision: D32335934

fbshipit-source-id: 416d92a8eecd06ef7fc742664a5f2d46f93415f8
2021-11-11 02:03:37 -08:00
Pyre Bot Jr
1836c786fe suppress errors in vision/fair/pytorch3d
Differential Revision: D32310496

fbshipit-source-id: fa1809bbe0b999baee6d07fad3890dc8c2a2157b
2021-11-10 02:52:00 -08:00
Ignacio Rocco
cac6cb1b78 Update NDC raysampler for non-square convention (#29)
Summary:
- Old NDC convention had xy coords in [-1,1]x[-1,1]
- New NDC convention has xy coords in [-1, 1]x[-u, u] or [-u, u]x[-1, 1]

where u > 1 is the aspect ratio of the image.

This PR fixes the NDC raysampler to use the new convention.

Partial fix for https://github.com/facebookresearch/pytorch3d/issues/868

Pull Request resolved: https://github.com/fairinternal/pytorch3d/pull/29

Reviewed By: davnov134

Differential Revision: D31926148

Pulled By: bottler

fbshipit-source-id: c6c42c60d1473b04e60ceb49c8c10951ddf03c74
2021-11-05 10:36:19 -07:00
Jeremy Reizenstein
bfeb82efa3 some pointcloud typing
Summary: Make clear that features_padded() etc can return None

Reviewed By: patricklabatut

Differential Revision: D31795088

fbshipit-source-id: 7b0bbb6f3b7ad7f7b6e6a727129537af1d1873af
2021-10-28 04:54:20 -07:00
Jeremy Reizenstein
73a14d7266 dataparallel fix
Summary: Attempt to overcome flaky test

Reviewed By: patricklabatut

Differential Revision: D31895560

fbshipit-source-id: 1ecbb1782b0eafe132f88425c48487c2d0e10d2d
2021-10-26 14:35:30 -07:00
una-dinosauria
bee31c48d3 Make some matrix conversion jittable (#898)
Summary:
Make sure the functions from `rotation_conversion` are jittable, and add some type hints.

Add tests to verify this is the case.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/898

Reviewed By: patricklabatut

Differential Revision: D31926103

Pulled By: bottler

fbshipit-source-id: bff6013c5ca2d452e37e631bd902f0674d5ca091
2021-10-26 14:31:46 -07:00
RWL
29417d1f9b NaN (divide by zero) fix for issue #561 and #790 (#891)
Summary:
https://github.com/facebookresearch/pytorch3d/issues/561
https://github.com/facebookresearch/pytorch3d/issues/790
Divide by zero fix (NaN fix).  When perspective_correct=True, BarycentricPerspectiveCorrectionForward and BarycentricPerspectiveCorrectionBackward in ../csrc/utils/geometry_utils.cuh are called.  The denominator (denom) values should not be allowed to go to zero. I'm able to resolve this issue locally with this PR and submit it for the team's review.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/891

Reviewed By: patricklabatut

Differential Revision: D31829695

Pulled By: bottler

fbshipit-source-id: a3517b8362f6e60d48c35731258d8ce261b1d912
2021-10-22 04:52:06 -07:00
Peter Bell
57b9c729b8 Remove THCGeneral.cpp (#66766)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/66766

Test Plan: Imported from OSS

Reviewed By: zou3519

Differential Revision: D31721647

Pulled By: ngimel

fbshipit-source-id: 5033a2800871c8745a1a92e379c9f97c98af212e
2021-10-19 16:08:39 -07:00
Pyre Bot Jr
7c111f7379 suppress errors in vision/fair/pytorch3d
Differential Revision: D31737477

fbshipit-source-id: 2590548c1b7a65c277ccddd405276c244fde0961
2021-10-18 12:18:08 -07:00
Jeremy Reizenstein
3953de47ee remove torch from cuda
Summary: Keep using at:: instead of torch:: so we don't need torch/extension.h and can keep other compilers happy.

Reviewed By: patricklabatut

Differential Revision: D31688436

fbshipit-source-id: 1825503da0104acaf1558d17300c02ef663bf538
2021-10-18 03:38:11 -07:00
Jeremy Reizenstein
1a7442a483 windows compatibility
Summary: Few tweaks to make CUDA build on windows happier, as remarked in #876.

Reviewed By: patricklabatut

Differential Revision: D31688188

fbshipit-source-id: 20816d6215f2e3ec898f81ae4221b1c2ff24b64f
2021-10-18 03:38:11 -07:00
Ignacio Rocco
16ebf54e69 NDC doc fix (#28)
Summary:
- Added clarifications about NDC coordinate system for square and non-square images.

Pull Request resolved: https://github.com/fairinternal/pytorch3d/pull/28

Reviewed By: nikhilaravi

Differential Revision: D31681444

Pulled By: bottler

fbshipit-source-id: f71eabe9b3dd54b9372cef617e08f837f316555b
2021-10-17 07:41:57 -07:00
Jeremy Reizenstein
14dd2611ee Remove version number from docs title
Summary: Small docs fixes: spelling. Avoid things which get out of date quickly: year, version.

Reviewed By: patricklabatut

Differential Revision: D31659927

fbshipit-source-id: b0111140bdaf3c6cadc09f70621bf5656909ca02
2021-10-16 15:06:23 -07:00
Jeremy Reizenstein
34b1b4ab8b defaulted grid_sizes in points2vols
Summary: Fix #873, that grid_sizes defaults to the wrong dtype in points2volumes code, and mask doesn't have a proper default.

Reviewed By: nikhilaravi

Differential Revision: D31503545

fbshipit-source-id: fa32a1a6074fc7ac7bdb362edfb5e5839866a472
2021-10-16 14:41:59 -07:00
Nikhila Ravi
2f2466f472 Update eps for coplanar check in 3D IoU
Summary: Make eps=1e-4 by default for coplanar check and also enable it to be set by the user in call to `box3d_overlap`.

Reviewed By: gkioxari

Differential Revision: D31596836

fbshipit-source-id: b57fe603fd136cfa58fddf836922706d44fe894e
2021-10-13 13:29:47 -07:00
Jeremy Reizenstein
53d99671bd remove PyTorch 1.5 builds
Summary: PyTorch 1.6.0 came out on 28 Jul 2020. Stop builds for 1.5.0 and 1.5.1. Also update the news section of the README for recent releases.

Reviewed By: nikhilaravi

Differential Revision: D31442830

fbshipit-source-id: 20bdd8a07090776d0461240e71c6536d874615f6
2021-10-11 06:13:01 -07:00
Pyre Bot Jr
6d36c1e2b0 suppress errors in vision/fair/pytorch3d
Differential Revision: D31496551

fbshipit-source-id: 705fd88f319875db3f7938a2946c48a51ea225f5
2021-10-07 21:58:08 -07:00
Nikhila Ravi
6dfa326922 IOU box3d epsilon fix
Summary: The epsilon value is important for determining whether vertices are inside/outside a plane.

Reviewed By: gkioxari

Differential Revision: D31485247

fbshipit-source-id: 5517575de7c02f1afa277d00e0190a81f44f5761
2021-10-07 18:42:09 -07:00
Jeremy Reizenstein
b26f4bc33a test tolerance loosenings
Summary: Increase some test tolerances so that they pass in more situations, and re-enable two tests.

Reviewed By: nikhilaravi

Differential Revision: D31379717

fbshipit-source-id: 06a25470cc7b6d71cd639d9fd7df500d4b84c079
2021-10-07 10:48:12 -07:00
Ruilong Li
8fa438cbda Fix camera conversion between opencv and pytorch3d
Summary:
For non square image, the NDC space in pytorch3d is not square [-1, 1]. Instead, it is [-1, 1] for the smallest side, and [-u, u] for the largest side, where u > 1. This behavior is followed by the pytorch3d renderer.

See the function `get_ndc_to_screen_transform` for a example.

Without this fix, the rendering result is not correct using the converted pytorch3d-camera from a opencv-camera on non square images.

This fix also helps the `transform_points_screen` function delivers consistent results with opencv projection for the converted pytorch3d-camera.

Reviewed By: classner

Differential Revision: D31366775

fbshipit-source-id: 8858ae7b5cf5c0a4af5a2af40a1358b2fe4cf74b
2021-10-07 10:15:31 -07:00
CodemodService Bot
815a93ce89 Daily arc lint --take BLACK
Reviewed By: zertosh

Differential Revision: D31464988

fbshipit-source-id: 2eaf28d6869ccb70fd4df4f7de15d959cdaba0be
2021-10-06 21:19:23 -07:00
Jeremy Reizenstein
23ef666db1 build website in docker container
Summary: Do the website building in a docker container to avoid worrying about dependencies.

Reviewed By: nikhilaravi

Differential Revision: D30223892

fbshipit-source-id: 77b7b4630188167316891381f6ca9e9fbe7f0a05
2021-10-06 18:09:45 -07:00
Nikita Smetanin
d7d740abe9 Symmetric eigen 3x3 implementation + benchmark & tests
Summary:
Symmetric eigenvalues 3x3 implementation from https://github.com/fairinternal/denseposeslim/blob/roman_c3dpo/tools/functions.py#L612

based on https://en.wikipedia.org/wiki/Eigenvalue_algorithm#3.C3.973_matrices and https://www.geometrictools.com/Documentation/RobustEigenSymmetric3x3.pdf

Benchmarks show significant outperformance of symeig3x3 in comparison with torch implementations (torch.symeig and torch.linalg.eigh) on GPU (P100), especially for large batches: 70-280ns per sample vs 3400ns per sample for torch_linalg_eigh_1048576_cpu

It's worth mentioning that torch.linalg.eigh is still comparably fast for batches up to 8192 on CPU.

Some tests are still failing as the error thresholds need to be adjusted appropriately.

Reviewed By: patricklabatut

Differential Revision: D29915453

fbshipit-source-id: 7c1b062da631c57c4e22a42dd0027ea5e205f1b5
2021-10-06 10:57:07 -07:00
Jeremy Reizenstein
9585a58d10 version number 0.6.0
Summary: update

Reviewed By: patricklabatut

Differential Revision: D31338002

fbshipit-source-id: 90ed6c2ea411c0384dd233ee88e51b5f608eef88
2021-10-05 16:25:25 -07:00
Jeremy Reizenstein
364a7dcaf4 Install.md for next release.
Summary: now supporting PyTorch 1.9.1

Reviewed By: patricklabatut

Differential Revision: D31338001

fbshipit-source-id: 11140819d10af388d31905a39f1da136cf9c5ff2
2021-10-05 16:25:25 -07:00
Georgia Gkioxari
1360d69ffb minor note fix
Summary: A small fix for the iou3d note

Reviewed By: bottler

Differential Revision: D31370686

fbshipit-source-id: 6c97302b5c78de52915f31be70f234179c4b246d
2021-10-03 17:17:47 -07:00
Jeremy Reizenstein
4281df19ce subsample pointclouds
Summary: New function to randomly subsample Pointclouds to a maximum size.

Reviewed By: nikhilaravi

Differential Revision: D30936533

fbshipit-source-id: 789eb5004b6a233034ec1c500f20f2d507a303ff
2021-10-02 13:40:16 -07:00
Jeremy Reizenstein
ee2b2feb98 Use C++/CUDA in points2vols
Summary:
Move the core of add_points_to_volumes to the new C++/CUDA implementation. Add new flag to let the user stop this happening. Avoids copies. About a 30% speedup on the larger cases, up to 50% on the smaller cases.

New timings
```
Benchmark                                                               Avg Time(μs)      Peak Time(μs) Iterations
--------------------------------------------------------------------------------
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[25, 25, 25]_1000                     4575           12591            110
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[25, 25, 25]_10000                   25468           29186             20
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[25, 25, 25]_100000                 202085          209897              3
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[101, 111, 121]_1000                 46059           48188             11
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[101, 111, 121]_10000                83759           95669              7
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[101, 111, 121]_100000              326056          339393              2
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[25, 25, 25]_1000                       2379            4738            211
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[25, 25, 25]_10000                     12100           63099             42
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[25, 25, 25]_100000                    63323           63737              8
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[101, 111, 121]_1000                   45216           45479             12
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[101, 111, 121]_10000                  57205           58524              9
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[101, 111, 121]_100000                139499          139926              4
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[25, 25, 25]_1000                   40129           40431             13
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[25, 25, 25]_10000                 204949          239293              3
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[25, 25, 25]_100000               1664541         1664541              1
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[101, 111, 121]_1000               391573          395108              2
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[101, 111, 121]_10000              674869          674869              1
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[101, 111, 121]_100000            2713632         2713632              1
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[25, 25, 25]_1000                     12726           13506             40
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[25, 25, 25]_10000                    73103           73299              7
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[25, 25, 25]_100000                  598634          598634              1
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[101, 111, 121]_1000                 398742          399256              2
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[101, 111, 121]_10000                543129          543129              1
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[101, 111, 121]_100000              1242956         1242956              1
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[25, 25, 25]_1000                  1814            8884            276
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[25, 25, 25]_10000                 1996            8851            251
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[25, 25, 25]_100000                4608           11529            109
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[101, 111, 121]_1000               5183           12508             97
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[101, 111, 121]_10000              7106           14077             71
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[101, 111, 121]_100000            25914           31818             20
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[25, 25, 25]_1000                    1778            8823            282
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[25, 25, 25]_10000                   1825            8613            274
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[25, 25, 25]_100000                  3154           10161            159
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[101, 111, 121]_1000                 4888            9404            103
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[101, 111, 121]_10000                5194            9963             97
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[101, 111, 121]_100000               8109           14933             62
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[25, 25, 25]_1000                 3320           10306            151
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[25, 25, 25]_10000                7003            8595             72
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[25, 25, 25]_100000              49140           52957             11
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[101, 111, 121]_1000             35890           36918             14
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[101, 111, 121]_10000            58890           59337              9
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[101, 111, 121]_100000          286878          287600              2
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[25, 25, 25]_1000                   2484            8805            202
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[25, 25, 25]_10000                  3967            9090            127
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[25, 25, 25]_100000                19423           19799             26
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[101, 111, 121]_1000               33228           33329             16
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[101, 111, 121]_10000              37292           37370             14
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[101, 111, 121]_100000             73550           74017              7
--------------------------------------------------------------------------------
```
Previous timings
```
Benchmark                                                               Avg Time(μs)      Peak Time(μs) Iterations
--------------------------------------------------------------------------------
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[25, 25, 25]_1000                    10100           46422             50
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[25, 25, 25]_10000                   28442           32100             18
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[25, 25, 25]_100000                 241127          254269              3
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[101, 111, 121]_1000                 54149           79480             10
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[101, 111, 121]_10000               125459          212734              4
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[101, 111, 121]_100000              512739          512739              1
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[25, 25, 25]_1000                       2866           13365            175
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[25, 25, 25]_10000                      7026           12604             72
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[25, 25, 25]_100000                    48822           55607             11
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[101, 111, 121]_1000                   38098           38576             14
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[101, 111, 121]_10000                  48006           54120             11
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[101, 111, 121]_100000                131563          138536              4
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[25, 25, 25]_1000                   64615           91735              8
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[25, 25, 25]_10000                 228815          246095              3
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[25, 25, 25]_100000               3086615         3086615              1
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[101, 111, 121]_1000               464298          465292              2
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[101, 111, 121]_10000             1053440         1053440              1
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[101, 111, 121]_100000            6736236         6736236              1
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[25, 25, 25]_1000                     11940           12440             42
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[25, 25, 25]_10000                    56641           58051              9
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[25, 25, 25]_100000                  711492          711492              1
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[101, 111, 121]_1000                 326437          329846              2
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[101, 111, 121]_10000                418514          427911              2
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[101, 111, 121]_100000              1524285         1524285              1
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[25, 25, 25]_1000                  5949           13602             85
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[25, 25, 25]_10000                 5817           13001             86
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[25, 25, 25]_100000               23833           25971             21
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[101, 111, 121]_1000               9029           16178             56
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[101, 111, 121]_10000             11595           18601             44
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[101, 111, 121]_100000            46986           47344             11
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[25, 25, 25]_1000                    2554            9747            196
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[25, 25, 25]_10000                   2676            9537            187
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[25, 25, 25]_100000                  6567           14179             77
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[101, 111, 121]_1000                 5840           12811             86
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[101, 111, 121]_10000                6102           13128             82
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[101, 111, 121]_100000              11945           11995             42
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[25, 25, 25]_1000                 7642           13671             66
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[25, 25, 25]_10000               25190           25260             20
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[25, 25, 25]_100000             212018          212134              3
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[101, 111, 121]_1000             40421           45692             13
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[101, 111, 121]_10000            92078           92132              6
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[101, 111, 121]_100000          457211          457229              2
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[25, 25, 25]_1000                   3574           10377            140
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[25, 25, 25]_10000                  7222           13023             70
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[25, 25, 25]_100000                48127           48165             11
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[101, 111, 121]_1000               34732           35295             15
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[101, 111, 121]_10000              43050           51064             12
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[101, 111, 121]_100000            106028          106058              5
--------------------------------------------------------------------------------
```

Reviewed By: nikhilaravi

Differential Revision: D29548609

fbshipit-source-id: 7026e832ea299145c3f6b55687f3c1601294f5c0
2021-10-01 11:58:24 -07:00
Jeremy Reizenstein
9ad98c87c3 Cuda function for points2vols
Summary: Added CUDA implementation to match the new, still unused, C++ function for the core of points2vols.

Reviewed By: nikhilaravi

Differential Revision: D29548608

fbshipit-source-id: 16ebb61787fcb4c70461f9215a86ad5f97aecb4e
2021-10-01 11:58:24 -07:00
Jeremy Reizenstein
0dfc6e0eb8 CPU function for points2vols
Summary: Single C++ function for the core of points2vols, not used anywhere yet. Added ability to control align_corners and the weight of each point, which may be useful later.

Reviewed By: nikhilaravi

Differential Revision: D29548607

fbshipit-source-id: a5cda7ec2c14836624e7dfe744c4bbb3f3d3dfe2
2021-10-01 11:58:24 -07:00
Jeremy Reizenstein
c7c6deab86 compatibility statement in README
Summary: Statement about compatibility.

Reviewed By: nikhilaravi

Differential Revision: D30697072

fbshipit-source-id: aeb5e3e0a08c1797033d8c00b24484c8a699cb02
2021-09-30 10:50:11 -07:00
Jeremy Reizenstein
4ad8576541 rasterization header comment fixes
Summary: Fix some missing or misplaced argument descriptions.

Reviewed By: nikhilaravi

Differential Revision: D31305132

fbshipit-source-id: af4fcee9766682b2b7f7f16327e839090e377be2
2021-09-30 10:41:50 -07:00
Simon Moisselin
a5cbb624c1 Fix typo in chamfer loss docstring (#862)
Summary:
y_lengths is about `y`, not `x`.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/862

Reviewed By: bottler

Differential Revision: D31304434

Pulled By: patricklabatut

fbshipit-source-id: 1db4cd57677fc018c229e02172f95ffa903d75eb
2021-09-30 05:10:18 -07:00
Theo-Cheynel
720bdf60f5 Removed typos 'f' from the f-string error messages (#851)
Summary:
Changed mistake in Python f-strings causing an additional letter "f" to appear in the error messages.
The error messages would read something like :
```
raise ValueError(f"Invalid rotation matrix  shape f{matrix.shape}.")
ValueError: Invalid rotation matrix  shape ftorch.Size([4, 4]).
```
(with an additional f, probably a mistake)

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/851

Reviewed By: nikhilaravi

Differential Revision: D31238831

Pulled By: patricklabatut

fbshipit-source-id: 0ba3e61e488e467e997954278097889be606d4f8
2021-09-30 03:26:14 -07:00
Jeremy Reizenstein
1aab192706 Linter when only python3 exists
Reviewed By: nikhilaravi

Differential Revision: D31289856

fbshipit-source-id: 5a522a69537a873bacacf2a178e5f30771aef35f
2021-09-30 00:55:38 -07:00
Jeremy Reizenstein
dd76b41014 save colors as uint8 in PLY
Summary: Allow saving colors as 8bit when writing .ply files.

Reviewed By: patricklabatut, nikitos9000

Differential Revision: D30905312

fbshipit-source-id: 44500982c9ed6d6ee901e04f9623e22792a0e7f7
2021-09-30 00:48:52 -07:00
Georgia Gkioxari
1b1ba5612f Note for iou3d
Summary:
A note for our new algorithm for IoU of oriented 3D boxes. It includes
* A description of the algorithm
* A comparison with Objectron

Reviewed By: nikhilaravi

Differential Revision: D31288066

fbshipit-source-id: 0ea8da887bc5810bf4a3e0848223dd3590df1538
2021-09-29 19:15:19 -07:00
Nikhila Ravi
ff8d4762f4 (new) CUDA IoU for 3D boxes
Summary: CUDA implementation of 3D bounding box overlap calculation.

Reviewed By: gkioxari

Differential Revision: D31157919

fbshipit-source-id: 5dc89805d01fef2d6779f00a33226131e39c43ed
2021-09-29 18:49:09 -07:00
Nikhila Ravi
53266ec9ff C++ IoU for 3D Boxes
Summary: C++ Implementation of algorithm to compute 3D bounding boxes for batches of bboxes of shape (N, 8, 3) and (M, 8, 3).

Reviewed By: gkioxari

Differential Revision: D30905190

fbshipit-source-id: 02e2cf025cd4fa3ff706ce5cf9b82c0fb5443f96
2021-09-29 17:03:43 -07:00
Nikhila Ravi
2293f1fed0 IoU for 3D boxes
Summary:
I have implemented an exact solution for 3D IoU of oriented 3D boxes.

This file includes:
* box3d_overlap: which computes the exact IoU of box1 and box2
* box3d_overlap_sampling: which computes an approximate IoU of box1 and box2 by sampling points within the boxes

Note that both implementations currently do not support batching.

Our exact IoU implementation is based on the fact that the intersecting shape of the two 3D boxes will be formed by segments of the surface of the boxes. Our algorithm computes these segments by reasoning whether triangles of one box are within the second box and vice versa. We deal with intersecting triangles by clipping them.

Reviewed By: gkioxari

Differential Revision: D30667497

fbshipit-source-id: 2f747f410f90b7f854eeaf3036794bc3ac982917
2021-09-29 13:44:10 -07:00
Pyre Bot Jr
5b89c4e3bb suppress errors in vision/fair/pytorch3d
Differential Revision: D31266959

fbshipit-source-id: 878a59ca2cfe1389e42fc338653e8d3314b56b91
2021-09-29 05:07:37 -07:00
Jeremy Reizenstein
d0ca3b9e0c builds for PyTorch 1.9.1
Summary: Add conda builds for the newly released PyTorch version 1.9.1.

Reviewed By: patricklabatut

Differential Revision: D31140206

fbshipit-source-id: 697549a3ef0db8248f4f9b5c00cf1407296b5022
2021-09-27 04:17:13 -07:00
Jeremy Reizenstein
9a737da83c More renderer parameter descriptions
Summary:
Copy some descriptions of renderer parameters to more places so they are easier to find.

Also a couple of small corrections, and make RasterizationSettings a dataclass.

Reviewed By: nikhilaravi, patricklabatut

Differential Revision: D30899822

fbshipit-source-id: 805cf366acb7d51cb308fa574deff0657c199673
2021-09-24 09:59:24 -07:00
Jeremy Reizenstein
860b742a02 deterministic rasterization
Summary: Attempt to fix #659, an observation that the rasterizer is nondeterministic, by resolving tied faces by picking those with lower index.

Reviewed By: nikhilaravi, patricklabatut

Differential Revision: D30699039

fbshipit-source-id: 39ed797eb7e9ce7370ae71259ad6b757f9449923
2021-09-23 06:59:48 -07:00
Jeremy Reizenstein
cb170ac024 Avoid torch/extension.h in cuda
Summary: Unlike other cu files, sigmoid_alpha_blend uses torch/extension.h. Avoid for possible build speed win and because of a reported problem #843 on windows with CUDA 11.4.

Reviewed By: nikhilaravi

Differential Revision: D31054121

fbshipit-source-id: 53a1f985a1695a044dfd2ee1a5b0adabdf280595
2021-09-22 15:54:59 -07:00
Jeremy Reizenstein
fe5bfa5994 rename cpp to avoid clash
Summary: Rename sample_farthest_point.cpp to not match its CUDA equivalent.

Reviewed By: nikhilaravi

Differential Revision: D31006645

fbshipit-source-id: 135b511cbde320d2b3e07fc5b027971ef9210aa9
2021-09-22 15:54:59 -07:00
Jeremy Reizenstein
dbfb3a910a remove __restrict__ in cpp
Summary: Remove use of nonstandard C++. Noticed on windows in issue https://github.com/facebookresearch/pytorch3d/issues/843. (We use `__restrict__` in CUDA, where it is fine, even on windows)

Reviewed By: nikhilaravi

Differential Revision: D31006516

fbshipit-source-id: 929ba9b3216cb70fad3ffa3274c910618d83973f
2021-09-22 15:54:59 -07:00
Pyre Bot Jr
526df446c6 suppress errors in vision/fair/pytorch3d
Differential Revision: D31042748

fbshipit-source-id: fffb983bd6765d306a407587ddf64e68e57e9ecc
2021-09-18 12:24:58 -07:00
Nikhila Ravi
bd04ffaf77 Farthest point sampling CUDA
Summary:
CUDA implementation of farthest point sampling algorithm.

## Visual comparison

Compared to random sampling, farthest point sampling gives better coverage of the shape.

{F658631262}

## Reduction

Parallelized block reduction to find the max value at each iteration happens as follows:

1. First split the points into two equal sized parts (e.g. for a list with 8 values):
`[20, 27, 6, 8 | 11, 10, 2, 33]`
2. Use half of the thread (4 threads) to compare pairs of elements from each half (e.g elements [0, 4], [1, 5] etc) and store the result in the first half of the list:
`[20, 27, 6, 33 | 11, 10, 2, 33]`
Now we no longer care about the second part but again divide the first part into two
`[20, 27 | 6, 33| -, -, -, -]`
Now we can use 2 threads to compare the 4 elements
4. Finally we have gotten down to a single pair
`[20 | 33 | -, - | -, -, -, -]`
Use 1 thread to compare the remaining two elements
5. The max will now be at thread id = 0
`[33 | - | -, - | -, -, -, -]`
The reduction will give the farthest point for the selected batch index at this iteration.

Reviewed By: bottler, jcjohnson

Differential Revision: D30401803

fbshipit-source-id: 525bd5ae27c4b13b501812cfe62306bb003827d2
2021-09-15 13:49:22 -07:00
Nikhila Ravi
d9f7611c4b Farthest point sampling C++
Summary: C++ implementation of iterative farthest point sampling.

Reviewed By: jcjohnson

Differential Revision: D30349887

fbshipit-source-id: d25990f857752633859fe00283e182858a870269
2021-09-15 13:49:21 -07:00
Nikhila Ravi
3b7d78c7a7 Farthest point sampling python naive
Summary:
This is a naive python implementation of the iterative farthest point sampling algorithm along with associated simple tests. The C++/CUDA implementations will follow in subsequent diffs.

The algorithm is used to subsample a pointcloud with better coverage of the space of the pointcloud.

The function has not been added to `__init__.py`. I will add this after the full C++/CUDA implementations.

Reviewed By: jcjohnson

Differential Revision: D30285716

fbshipit-source-id: 33f4181041fc652776406bcfd67800a6f0c3dd58
2021-09-15 13:49:21 -07:00
Jeremy Reizenstein
a0d76a7080 join_scene fix for TexturesUV
Summary: Fix issue #826. This is a correction to the joining of TexturesUV into a single scene.

Reviewed By: nikhilaravi

Differential Revision: D30767092

fbshipit-source-id: 03ba6a1d2f22e569d1b3641cd13ddbb8dcb87ec7
2021-09-13 07:08:58 -07:00
Shangchen Han
46f727cb68 make so3_log_map torch script compatible
Summary:
* HAT_INV_SKEW_SYMMETRIC_TOL was a global variable and torch script gives an error when compiling that function. Move it to the function scope.
* torch script gives error when compiling acos_linear_extrapolation because bound is a union of tuple and float. The tuple version is kept in this diff.

Reviewed By: patricklabatut

Differential Revision: D30614916

fbshipit-source-id: 34258d200dc6a09fbf8917cac84ba8a269c00aef
2021-09-10 11:13:26 -07:00
Jeremy Reizenstein
c3d7808868 register_buffer compatibility
Summary: In D30349234 (1b8d86a104) we introduced persistent=False to some register_buffer calls, which depend on PyTorch 1.6. We go back to the old behaviour for PyTorch 1.5.

Reviewed By: nikhilaravi

Differential Revision: D30731327

fbshipit-source-id: ab02ef98ee87440ef02479b72f4872b562ab85b5
2021-09-09 07:37:57 -07:00
Justin Johnson
bbc7573261 Unify coarse rasterization for points and meshes
Summary:
There has historically been a lot of duplication between the coarse rasterization logic for point clouds and meshes. This diff factors out the shared logic, so coarse rasterization of point clouds and meshes share the same core logic.

Previously the only difference between the coarse rasterization kernels for points and meshes was the logic for checking whether a {point / triangle} intersects a tile in the image. We implement a generic coarse rasterization kernel that takes a set of 2D bounding boxes rather than geometric primitives; we then implement separate kernels that compute 2D bounding boxes for points and triangles.

This change does not affect the Python API at all. It also should not change any rasterization behavior, since this diff is just a refactoring of the existing logic.

I see this diff as the first in a few pieces of rasterizer refactoring. Followup diffs should do the following:
- Add a check for bin overflow in the generic coarse rasterizer kernel: allocate a global scalar to flag bin overflow which kernel worker threads can write to in case they detect bin overflow. The C++ launcher function can then check this flag after the kernel returns and issue a warning to the user in case of overflow.
- As a slightly more involved mechanism, if bin overflow is detected then the coarse kernel can continue running in order to count how many elements fall into each bin, without actually writing out their indices to the coarse output tensor. Then the actual number of entries per bin can be used to re-allocate the output tensor and re-run the coarse rasterization kernel so that bin overflow can be automatically avoided.
- The unification of the coarse and fine rasterization kernels also allows us to insert an extra CUDA kernel prior to coarse rasterization that filters out primitives outside the view frustum. This would be helpful for rendering full scenes (e.g. Matterport data) where only a small piece of the mesh is actually visible at any one time.

Reviewed By: bottler

Differential Revision: D25710361

fbshipit-source-id: 9c9dea512cb339c42adb3c92e7733fedd586ce1b
2021-09-08 16:17:30 -07:00
Justin Johnson
eed68f457d Refactor mesh coarse rasterization
Summary: Renaming parts of the mesh coarse rasterization and separating the bounding box calculation. All in preparation for sharing code with point rasterization.

Reviewed By: bottler

Differential Revision: D30369112

fbshipit-source-id: 3508c0b1239b355030cfa4038d5f3d6a945ebbf4
2021-09-08 16:17:30 -07:00
Justin Johnson
62dbf371ae Move coarse rasterization to new file
Summary: In preparation for sharing coarse rasterization between point clouds and meshes, move the functions to a new file. No code changes.

Reviewed By: bottler

Differential Revision: D30367812

fbshipit-source-id: 9e73835a26c4ac91f5c9f61ff682bc8218e36c6a
2021-09-08 16:17:30 -07:00
Jeremy Reizenstein
f2c44e3540 update test_build for robustness
Summary: Change cyclic deps test to be independent of test discovery order. Also let it work without plotly.

Reviewed By: nikhilaravi

Differential Revision: D30669614

fbshipit-source-id: 2eadf3f8b56b6096c5466ce53b4f8ac6df27b964
2021-09-02 09:32:29 -07:00
Jeremy Reizenstein
a9b0d50baf Restore missing linux conda builds
Summary: Regenerate config.yml after a recent bad merge which lost a few builds.

Reviewed By: nikhilaravi

Differential Revision: D30696918

fbshipit-source-id: 3ecdfca8682baed13692ec710aa7c25dbd24dd44
2021-09-01 10:29:05 -07:00
Nikhila Ravi
fc156b50c0 (bug) Fix exception when creating a TextureAtlas
Summary: Fixes GitHub issue #751. The vectorized implementation of bilinear interpolation didn't properly handle the edge cases in the same way as the `grid_sample` method in PyTorch.

Reviewed By: bottler

Differential Revision: D30684208

fbshipit-source-id: edf241ecbd72d46b94ad340a4e601e26c83db88e
2021-09-01 09:26:44 -07:00
Georgia Gkioxari
835e662fb5 master -> main
Summary: Replace master with main in hard coded paths or mentions in documentation

Reviewed By: bottler

Differential Revision: D30696097

fbshipit-source-id: d5ff67bb026d90d1543d10ab027f916e8361ca69
2021-09-01 05:33:25 -07:00
Jeremy Reizenstein
1b8d86a104 (breaking) image_size-agnostic GridRaySampler
Summary:
As suggested in #802. By not persisting the _xy_grid buffer, we can allow (in some cases) a model with one image_size to be loaded from a saved model which was trained at a different resolution.

Also avoid persisting _frequencies in HarmonicEmbedding for similar reasons.

BC-break: This will cause load_state_dict, in strict mode, to complain if you try to load an old model with the new code.

Reviewed By: patricklabatut

Differential Revision: D30349234

fbshipit-source-id: d6061d1e51c9f79a78d61a9f732c9a5dfadbbb47
2021-08-31 14:30:24 -07:00
Jeremy Reizenstein
1251446383 Use sample_pdf from PyTorch3D in NeRF
Summary:
Use PyTorch3D's new faster sample_pdf function instead of local Python implementation.

Also clarify deps for the Python implementation.

Reviewed By: gkioxari

Differential Revision: D30512109

fbshipit-source-id: 84cfdc00313fada37a6b29837de96f6a4646434f
2021-08-31 11:26:26 -07:00
Alex Naumann
d2bbd0cdb7 Fix link to render textured meshes example (#818)
Summary:
Great work! :)
Just found a link in the examples that is not working. This will fix it.

Best,
Alex

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/818

Reviewed By: nikhilaravi

Differential Revision: D30637532

Pulled By: patricklabatut

fbshipit-source-id: ed6c52375d1e760cb0fb2c0a66648dfeb0c6ed46
2021-08-30 13:11:53 -07:00
Jeremy Reizenstein
6c416b319c remove PyTorch 1.4 builds
Summary: We won't support PyTorch 1.4 in the next release. PyTorch 1.5.0 came out in June 2020, more than a year ago.

Reviewed By: patricklabatut

Differential Revision: D30424388

fbshipit-source-id: 25499096066c9a2b909a0550394f5210409f0d74
2021-08-23 08:32:41 -07:00
Jeremy Reizenstein
77fa5987b8 check for cyclic deps
Summary: New test that each subpackage of pytorch3d imports cleanly.

Reviewed By: patricklabatut

Differential Revision: D30001632

fbshipit-source-id: ca8dcac94491fc22f33602b3bbef481cba927094
2021-08-23 06:16:40 -07:00
Pyre Bot Jr
fadec970c9 suppress errors in vision/fair/pytorch3d
Differential Revision: D30479084

fbshipit-source-id: 6b22dd0afe4dfb1be6249e43a56657519f11dcf1
2021-08-22 23:39:37 -07:00
Jeremy Reizenstein
1ea2b7272a sample_pdf CUDA and C++ implementations.
Summary: Implement the sample_pdf function from the NeRF project as compiled operators.. The binary search (in searchsorted) is replaced with a low tech linear search, but this is not a problem for the envisaged numbers of bins.

Reviewed By: gkioxari

Differential Revision: D26312535

fbshipit-source-id: df1c3119cd63d944380ed1b2657b6ad81d743e49
2021-08-17 08:07:55 -07:00
Jeremy Reizenstein
7d7d00f288 Move sample_pdf into PyTorch3D
Summary: Copy the sample_pdf operation from the NeRF project in to PyTorch3D, in preparation for optimizing it.

Reviewed By: gkioxari

Differential Revision: D27117930

fbshipit-source-id: 20286b007f589a4c4d53ed818c4bc5f2abd22833
2021-08-17 08:07:55 -07:00
Jeremy Reizenstein
b481cfbd01 Correct shape for default grid_sizes
Summary: Small fix for omitting this argument.

Reviewed By: nikhilaravi

Differential Revision: D29548610

fbshipit-source-id: f25032fab3faa2f09006f5fcf8628138555f2f20
2021-08-17 05:59:07 -07:00
Jeremy Reizenstein
46cf1970ac cpu benchmarks for points to volumes
Summary:
Add a CPU version to the benchmarks.

```
Benchmark                                                               Avg Time(μs)      Peak Time(μs) Iterations
--------------------------------------------------------------------------------
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[25, 25, 25]_1000                    10100           46422             50
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[25, 25, 25]_10000                   28442           32100             18
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[25, 25, 25]_100000                 241127          254269              3
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[101, 111, 121]_1000                 54149           79480             10
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[101, 111, 121]_10000               125459          212734              4
ADD_POINTS_TO_VOLUMES_cpu_10_trilinear_[101, 111, 121]_100000              512739          512739              1
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[25, 25, 25]_1000                       2866           13365            175
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[25, 25, 25]_10000                      7026           12604             72
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[25, 25, 25]_100000                    48822           55607             11
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[101, 111, 121]_1000                   38098           38576             14
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[101, 111, 121]_10000                  48006           54120             11
ADD_POINTS_TO_VOLUMES_cpu_10_nearest_[101, 111, 121]_100000                131563          138536              4
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[25, 25, 25]_1000                   64615           91735              8
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[25, 25, 25]_10000                 228815          246095              3
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[25, 25, 25]_100000               3086615         3086615              1
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[101, 111, 121]_1000               464298          465292              2
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[101, 111, 121]_10000             1053440         1053440              1
ADD_POINTS_TO_VOLUMES_cpu_100_trilinear_[101, 111, 121]_100000            6736236         6736236              1
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[25, 25, 25]_1000                     11940           12440             42
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[25, 25, 25]_10000                    56641           58051              9
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[25, 25, 25]_100000                  711492          711492              1
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[101, 111, 121]_1000                 326437          329846              2
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[101, 111, 121]_10000                418514          427911              2
ADD_POINTS_TO_VOLUMES_cpu_100_nearest_[101, 111, 121]_100000              1524285         1524285              1
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[25, 25, 25]_1000                  5949           13602             85
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[25, 25, 25]_10000                 5817           13001             86
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[25, 25, 25]_100000               23833           25971             21
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[101, 111, 121]_1000               9029           16178             56
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[101, 111, 121]_10000             11595           18601             44
ADD_POINTS_TO_VOLUMES_cuda:0_10_trilinear_[101, 111, 121]_100000            46986           47344             11
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[25, 25, 25]_1000                    2554            9747            196
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[25, 25, 25]_10000                   2676            9537            187
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[25, 25, 25]_100000                  6567           14179             77
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[101, 111, 121]_1000                 5840           12811             86
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[101, 111, 121]_10000                6102           13128             82
ADD_POINTS_TO_VOLUMES_cuda:0_10_nearest_[101, 111, 121]_100000              11945           11995             42
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[25, 25, 25]_1000                 7642           13671             66
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[25, 25, 25]_10000               25190           25260             20
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[25, 25, 25]_100000             212018          212134              3
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[101, 111, 121]_1000             40421           45692             13
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[101, 111, 121]_10000            92078           92132              6
ADD_POINTS_TO_VOLUMES_cuda:0_100_trilinear_[101, 111, 121]_100000          457211          457229              2
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[25, 25, 25]_1000                   3574           10377            140
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[25, 25, 25]_10000                  7222           13023             70
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[25, 25, 25]_100000                48127           48165             11
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[101, 111, 121]_1000               34732           35295             15
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[101, 111, 121]_10000              43050           51064             12
ADD_POINTS_TO_VOLUMES_cuda:0_100_nearest_[101, 111, 121]_100000            106028          106058              5
--------------------------------------------------------------------------------
```

Reviewed By: patricklabatut

Differential Revision: D29522830

fbshipit-source-id: 1e857db03613b0c6afcb68a58cdd7ba032e1a874
2021-08-17 05:59:07 -07:00
Jeremy Reizenstein
5491b46511 Points2vols doc fixes
Summary: Fixes to a couple of comments on points to volumes, make the mask work in round_points_to_volumes, and remove a duplicate rand calculation

Reviewed By: nikhilaravi

Differential Revision: D29522845

fbshipit-source-id: 86770ba37ef3942b909baf63fd73eed1399635b6
2021-08-17 05:59:07 -07:00
Jeremy Reizenstein
ae1387b523 let build tests run in conda
Summary: Much of the code is actually available during the conda tests, as long as we look in the right place. We enable some of them.

Reviewed By: nikhilaravi

Differential Revision: D30249357

fbshipit-source-id: 01c57b6b8c04442237965f23eded594aeb90abfb
2021-08-17 04:26:27 -07:00
Jeremy Reizenstein
b0dd0c8821 rename master branch to main
Summary: Change doc references to master branch to its new name main.

Reviewed By: nikhilaravi

Differential Revision: D30303018

fbshipit-source-id: cfdbb207dfe3366de7e0ca759ed56f4b8dd894d1
2021-08-16 04:06:53 -07:00
Nikhila Ravi
103da63393 Ball Query
Summary:
Implementation of ball query from PointNet++.  This function is similar to KNN (find the neighbors in p2 for all points in p1). These are the key differences:
-  It will return the **first** K neighbors within a specified radius as opposed to the **closest** K neighbors.
- As all the points in p2 do not need to be considered to find the closest K, the algorithm is much faster than KNN when p2 has a large number of points.
- The neighbors are not sorted
- Due to the radius threshold it is not guaranteed that there will be K neighbors even if there are more than K points in p2.
- The padding value for `idx` is -1 instead of 0.

# Note:
- Some of the code is very similar to KNN so it could be possible to modify the KNN forward kernels to support ball query.
- Some users might want to use kNN with ball query - for this we could provide a wrapper function around the current `knn_points` which enables applying the radius threshold afterwards as an alternative. This could be called `ball_query_knn`.

Reviewed By: jcjohnson

Differential Revision: D30261362

fbshipit-source-id: 66b6a7e0114beff7164daf7eba21546ff41ec450
2021-08-12 14:06:32 -07:00
Jeremy Reizenstein
e5c58a8a8b Test website metadata
Summary: New test that notes and tutorials are listed in the website metadata, so that they will be included in the website build.

Reviewed By: nikhilaravi

Differential Revision: D30223799

fbshipit-source-id: 2dca9730b54e68da2fd430a7b47cb7e18814d518
2021-08-12 05:07:55 -07:00
Jeremy Reizenstein
64faedfd57 Add new doc and new tutorials to website
Summary: Recent additions need to be included.

Reviewed By: nikhilaravi

Differential Revision: D30223717

fbshipit-source-id: 4b29a4132ea6fb7c1a530aac5d1e36aa61c663bb
2021-08-12 05:07:55 -07:00
Pyre Bot Jr
9db70400d8 suppress errors in fbcode/vision - batch 2
Differential Revision: D30222339

fbshipit-source-id: 97d498df72ef897b8dc2405764e3ffd432082e3c
2021-08-10 10:21:59 -07:00
Nikhila Ravi
804117833e Fix to allow cameras in the renderer forward pass
Summary: Fix to resolve GitHub issue #796 - the cameras were being passed in the renderer forward pass instead of at initialization. The rasterizer was correctly using the cameras passed in the `kwargs` for the projection, but the `cameras` are still part of the `kwargs` for the `get_screen_to_ndc_transform` and `get_ndc_to_screen_transform` functions which is causing issues about duplicate arguments.

Reviewed By: bottler

Differential Revision: D30175679

fbshipit-source-id: 547e88d8439456e728fa2772722df5fa0fe4584d
2021-08-09 11:42:50 -07:00
889 changed files with 80399 additions and 6946 deletions

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) Facebook, Inc. and its affiliates.
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the

View File

@@ -1,5 +1,5 @@
#!/bin/bash -e
# Copyright (c) Facebook, Inc. and its affiliates.
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the

View File

@@ -18,20 +18,13 @@ setupcuda: &setupcuda
working_directory: ~/
command: |
# download and install nvidia drivers, cuda, etc
wget --no-verbose --no-clobber -P ~/nvidia-downloads https://developer.download.nvidia.com/compute/cuda/11.2.2/local_installers/cuda_11.2.2_460.32.03_linux.run
sudo sh ~/nvidia-downloads/cuda_11.2.2_460.32.03_linux.run --silent
wget --no-verbose --no-clobber -P ~/nvidia-downloads https://developer.download.nvidia.com/compute/cuda/11.3.1/local_installers/cuda_11.3.1_465.19.01_linux.run
sudo sh ~/nvidia-downloads/cuda_11.3.1_465.19.01_linux.run --silent
echo "Done installing CUDA."
pyenv versions
nvidia-smi
pyenv global 3.9.1
gpu: &gpu
environment:
CUDA_VERSION: "10.2"
machine:
image: default
resource_class: gpu.medium # tesla m60
binary_common: &binary_common
parameters:
# Edit these defaults to do a release`
@@ -54,42 +47,41 @@ binary_common: &binary_common
description: "Wheel only: what docker image to use"
type: string
default: "pytorch/manylinux-cuda101"
conda_docker_image:
description: "what docker image to use for docker"
type: string
default: "pytorch/conda-cuda"
environment:
PYTHON_VERSION: << parameters.python_version >>
BUILD_VERSION: << parameters.build_version >>
PYTORCH_VERSION: << parameters.pytorch_version >>
CU_VERSION: << parameters.cu_version >>
TESTRUN_DOCKER_IMAGE: << parameters.conda_docker_image >>
jobs:
main:
<<: *gpu
environment:
CUDA_VERSION: "11.3"
resource_class: gpu.nvidia.small.multi
machine:
image: ubuntu-2004:202101-01
image: linux-cuda-11:default
steps:
- checkout
- <<: *setupcuda
- run: pip3 install --progress-bar off imageio wheel matplotlib 'pillow<7'
- run: pip3 install --progress-bar off torch torchvision
- run: pip3 install --progress-bar off torch==1.10.0+cu113 torchvision==0.11.1+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html
# - run: conda create -p ~/conda_env python=3.7 numpy
# - run: conda activate ~/conda_env
# - run: conda install -c pytorch pytorch torchvision
- run: pip3 install --progress-bar off 'git+https://github.com/facebookresearch/fvcore'
- run: pip3 install --progress-bar off 'git+https://github.com/facebookresearch/iopath'
- run:
name: get cub
command: |
cd ..
wget --no-verbose https://github.com/NVIDIA/cub/archive/1.10.0.tar.gz
tar xzf 1.10.0.tar.gz
# This expands to a directory called cub-1.10.0
- run:
name: build
command: |
export LD_LIBRARY_PATH=$LD_LIBARY_PATH:/usr/local/cuda-11.2/lib64
export CUB_HOME=$(realpath ../cub-1.10.0)
export LD_LIBRARY_PATH=$LD_LIBARY_PATH:/usr/local/cuda-11.3/lib64
python3 setup.py build_ext --inplace
- run: LD_LIBRARY_PATH=$LD_LIBARY_PATH:/usr/local/cuda-11.2/lib64 python -m unittest discover -v -s tests
- run: LD_LIBRARY_PATH=$LD_LIBARY_PATH:/usr/local/cuda-11.3/lib64 python -m unittest discover -v -s tests -t .
- run: python3 setup.py bdist_wheel
binary_linux_wheel:
@@ -113,7 +105,7 @@ jobs:
binary_linux_conda:
<<: *binary_common
docker:
- image: "pytorch/conda-cuda"
- image: "<< parameters.conda_docker_image >>"
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
@@ -124,8 +116,8 @@ jobs:
# so we aren't running the tests.
- run:
name: build
no_output_timeout: 20m
command: MAX_JOBS=15 TEST_FLAG=--no-test packaging/build_conda.sh
no_output_timeout: 40m
command: MAX_JOBS=15 TEST_FLAG=--no-test python3 packaging/build_conda.py
- store_artifacts:
path: /opt/conda/conda-bld/linux-64
- persist_to_workspace:
@@ -136,142 +128,44 @@ jobs:
binary_linux_conda_cuda:
<<: *binary_common
machine:
image: ubuntu-1604:201903-01
resource_class: gpu.medium
image: linux-cuda-11:default
resource_class: gpu.nvidia.small.multi
steps:
- checkout
- run:
name: Setup environment
command: |
set -e
curl -L https://packagecloud.io/circleci/trusty/gpgkey | sudo apt-key add -
curl -L https://dl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
sudo apt-get update
sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
gnupg-agent \
software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
sudo apt-get update
export DOCKER_VERSION="5:19.03.2~3-0~ubuntu-xenial"
sudo apt-get install docker-ce=${DOCKER_VERSION} docker-ce-cli=${DOCKER_VERSION} containerd.io=1.2.6-3
# Add the package repositories
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
export NVIDIA_CONTAINER_VERSION="1.0.3-1"
sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit=${NVIDIA_CONTAINER_VERSION}
sudo systemctl restart docker
DRIVER_FN="NVIDIA-Linux-x86_64-460.84.run"
wget "https://us.download.nvidia.com/XFree86/Linux-x86_64/460.84/$DRIVER_FN"
sudo /bin/bash "$DRIVER_FN" -s --no-drm || (sudo cat /var/log/nvidia-installer.log && false)
nvidia-smi
- run:
name: Pull docker image
command: |
nvidia-smi
set -e
{ docker login -u="$DOCKERHUB_USERNAME" -p="$DOCKERHUB_TOKEN" ; } 2> /dev/null
DOCKER_IMAGE=pytorch/conda-cuda
echo Pulling docker image $DOCKER_IMAGE
docker pull $DOCKER_IMAGE
echo Pulling docker image $TESTRUN_DOCKER_IMAGE
docker pull $TESTRUN_DOCKER_IMAGE
- run:
name: Build and run tests
no_output_timeout: 20m
no_output_timeout: 40m
command: |
set -e
cd ${HOME}/project/
DOCKER_IMAGE=pytorch/conda-cuda
export JUST_TESTRUN=1
VARS_TO_PASS="-e PYTHON_VERSION -e BUILD_VERSION -e PYTORCH_VERSION -e CU_VERSION -e JUST_TESTRUN"
docker run --gpus all --ipc=host -v $(pwd):/remote -w /remote ${VARS_TO_PASS} ${DOCKER_IMAGE} ./packaging/build_conda.sh
binary_macos_wheel:
<<: *binary_common
macos:
xcode: "12.0"
steps:
- checkout
- run:
# Cannot easily deduplicate this as source'ing activate
# will set environment variables which we need to propagate
# to build_wheel.sh
command: |
curl -o conda.sh https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
sh conda.sh -b
source $HOME/miniconda3/bin/activate
packaging/build_wheel.sh
- store_artifacts:
path: dist
docker run --gpus all --ipc=host -v $(pwd):/remote -w /remote ${VARS_TO_PASS} ${TESTRUN_DOCKER_IMAGE} python3 ./packaging/build_conda.py
workflows:
version: 2
build_and_test:
jobs:
- main:
context: DOCKERHUB_TOKEN
# - main:
# context: DOCKERHUB_TOKEN
{{workflows()}}
- binary_linux_conda_cuda:
name: testrun_conda_cuda_py36_cu101_pyt14
name: testrun_conda_cuda_py310_cu117_pyt201
context: DOCKERHUB_TOKEN
python_version: "3.6"
pytorch_version: "1.4"
cu_version: "cu101"
- binary_linux_conda_cuda:
name: testrun_conda_cuda_py37_cu102_pyt190
context: DOCKERHUB_TOKEN
python_version: "3.7"
pytorch_version: '1.9.0'
cu_version: "cu102"
- binary_linux_conda_cuda:
name: testrun_conda_cuda_py37_cu110_pyt170
context: DOCKERHUB_TOKEN
python_version: "3.7"
pytorch_version: '1.7.0'
cu_version: "cu110"
- binary_linux_conda_cuda:
name: testrun_conda_cuda_py39_cu111_pyt181
context: DOCKERHUB_TOKEN
python_version: "3.9"
pytorch_version: '1.8.1'
cu_version: "cu111"
- binary_macos_wheel:
cu_version: cpu
name: macos_wheel_py36_cpu
python_version: '3.6'
pytorch_version: '1.9.0'
- binary_macos_wheel:
cu_version: cpu
name: macos_wheel_py37_cpu
python_version: '3.7'
pytorch_version: '1.9.0'
- binary_macos_wheel:
cu_version: cpu
name: macos_wheel_py38_cpu
python_version: '3.8'
pytorch_version: '1.9.0'
- binary_macos_wheel:
cu_version: cpu
name: macos_wheel_py39_cpu
python_version: '3.9'
pytorch_version: '1.9.0'
python_version: "3.10"
pytorch_version: '2.0.1'
cu_version: "cu117"

File diff suppressed because it is too large Load Diff

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) Facebook, Inc. and its affiliates.
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
@@ -13,35 +13,56 @@ import os.path
import jinja2
import yaml
from packaging import version
# The CUDA versions which have pytorch conda packages available for linux for each
# version of pytorch.
# Pytorch 1.4 also supports cuda 10.0 but we no longer build for cuda 10.0 at all.
CONDA_CUDA_VERSIONS = {
"1.4": ["cu92", "cu101"],
"1.5.0": ["cu92", "cu101", "cu102"],
"1.5.1": ["cu92", "cu101", "cu102"],
"1.6.0": ["cu92", "cu101", "cu102"],
"1.7.0": ["cu101", "cu102", "cu110"],
"1.7.1": ["cu101", "cu102", "cu110"],
"1.8.0": ["cu101", "cu102", "cu111"],
"1.8.1": ["cu101", "cu102", "cu111"],
"1.9.0": ["cu102", "cu111"],
"2.1.0": ["cu118", "cu121"],
"2.1.1": ["cu118", "cu121"],
"2.1.2": ["cu118", "cu121"],
"2.2.0": ["cu118", "cu121"],
"2.2.2": ["cu118", "cu121"],
"2.3.1": ["cu118", "cu121"],
"2.4.0": ["cu118", "cu121"],
"2.4.1": ["cu118", "cu121"],
}
def conda_docker_image_for_cuda(cuda_version):
if len(cuda_version) != 5:
raise ValueError("Unknown cuda version")
return "pytorch/conda-builder:cuda" + cuda_version[2:]
def pytorch_versions_for_python(python_version):
if python_version in ["3.6", "3.7", "3.8"]:
if python_version in ["3.8", "3.9"]:
return list(CONDA_CUDA_VERSIONS)
pytorch_without_py39 = ["1.4", "1.5.0", "1.5.1", "1.6.0", "1.7.0"]
return [i for i in CONDA_CUDA_VERSIONS if i not in pytorch_without_py39]
if python_version == "3.10":
return [
i
for i in CONDA_CUDA_VERSIONS
if version.Version(i) >= version.Version("1.11.0")
]
if python_version == "3.11":
return [
i
for i in CONDA_CUDA_VERSIONS
if version.Version(i) >= version.Version("2.1.0")
]
if python_version == "3.12":
return [
i
for i in CONDA_CUDA_VERSIONS
if version.Version(i) >= version.Version("2.2.0")
]
def workflows(prefix="", filter_branch=None, upload=False, indentation=6):
w = []
for btype in ["conda"]:
for python_version in ["3.6", "3.7", "3.8", "3.9"]:
for python_version in ["3.8", "3.9", "3.10", "3.11", "3.12"]:
for pytorch_version in pytorch_versions_for_python(python_version):
for cu_version in CONDA_CUDA_VERSIONS[pytorch_version]:
w += workflow_pair(
@@ -67,7 +88,6 @@ def workflow_pair(
upload=False,
filter_branch,
):
w = []
py = python_version.replace(".", "")
pyt = pytorch_version.replace(".", "")
@@ -106,7 +126,6 @@ def generate_base_workflow(
btype,
filter_branch=None,
):
d = {
"name": base_workflow_name,
"python_version": python_version,
@@ -115,6 +134,10 @@ def generate_base_workflow(
"context": "DOCKERHUB_TOKEN",
}
conda_docker_image = conda_docker_image_for_cuda(cu_version)
if conda_docker_image is not None:
d["conda_docker_image"] = conda_docker_image
if filter_branch is not None:
d["filters"] = {"branches": {"only": filter_branch}}

View File

@@ -1,5 +1,8 @@
[flake8]
ignore = E203, E266, E501, W503, E221
# B028 No explicit stacklevel argument found.
# B907 'foo' is manually surrounded by quotes, consider using the `!r` conversion flag.
# B905 `zip()` without an explicit `strict=` parameter.
ignore = E203, E266, E501, W503, E221, B028, B905, B907
max-line-length = 88
max-complexity = 18
select = B,C,E,F,W,T4,B9

View File

@@ -16,14 +16,13 @@ We do not always accept new features, and we take the following factors into con
When sending a PR, please ensure you complete the following steps:
1. Fork the repo and create your branch from `master`. Follow the instructions
1. Fork the repo and create your branch from `main`. Follow the instructions
in [INSTALL.md](../INSTALL.md) to build the repo.
2. If you've added code that should be tested, add tests.
3. If you've changed any APIs, please update the documentation.
4. Ensure the test suite passes:
4. Ensure the test suite passes, by running this from the project root:
```
cd pytorch3d/tests
python -m unittest -v
python -m unittest discover -v -s tests -t .
```
5. Make sure your code lints by running `dev/linter.sh` from the project root.
6. If a PR contains multiple orthogonal changes, split it into multiple separate PRs.
@@ -46,7 +45,7 @@ outlined on that page and do not file a public issue.
## Coding Style
We follow these [python](http://google.github.io/styleguide/pyguide.html) and [C++](https://google.github.io/styleguide/cppguide.html) style guides.
For the linter to work, you will need to install `black`, `flake`, `isort` and `clang-format`, and
For the linter to work, you will need to install `black`, `flake`, `usort` and `clang-format`, and
they need to be fairly up to date.
## License

BIN
.github/implicitron_config.gif vendored Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.0 MiB

23
.github/workflows/build.yml vendored Normal file
View File

@@ -0,0 +1,23 @@
name: facebookresearch/pytorch3d/build_and_test
on:
pull_request:
branches:
- main
push:
branches:
- main
jobs:
binary_linux_conda_cuda:
runs-on: 4-core-ubuntu-gpu-t4
env:
PYTHON_VERSION: "3.12"
BUILD_VERSION: "${{ github.run_number }}"
PYTORCH_VERSION: "2.4.1"
CU_VERSION: "cu121"
JUST_TESTRUN: 1
steps:
- uses: actions/checkout@v4
- name: Build and run tests
run: |-
conda create --name env --yes --quiet conda-build
conda run --no-capture-output --name env python3 ./packaging/build_conda.py --use-conda-cuda

View File

@@ -1,27 +0,0 @@
name: Mark stale issues and pull requests
on:
schedule:
- cron: '31 5 * * *'
jobs:
stale:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v3
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
exempt-issue-labels: 'enhancement,how-to'
stale-issue-message: 'This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.'
stale-pr-message: 'This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.'
close-issue-message: 'This issue was closed because it has been stalled for 5 days with no activity.'
close-pr-message: 'This PR was closed because it has been stalled for 10 days with no activity.'
days-before-issue-stale: 30
days-before-pr-stale: 45
days-before-issue-close: 5
days-before-pr-close: 10

View File

@@ -8,24 +8,23 @@
The core library is written in PyTorch. Several components have underlying implementation in CUDA for improved performance. A subset of these components have CPU implementations in C++/PyTorch. It is advised to use PyTorch3D with GPU support in order to use all the features.
- Linux or macOS or Windows
- Python 3.6, 3.7, 3.8 or 3.9
- PyTorch 1.4, 1.5.0, 1.5.1, 1.6.0, 1.7.0, 1.7.1, 1.8.0, 1.8.1 or 1.9.0.
- Python
- PyTorch 2.1.0, 2.1.1, 2.1.2, 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.3.1, 2.4.0 or 2.4.1.
- torchvision that matches the PyTorch installation. You can install them together as explained at pytorch.org to make sure of this.
- gcc & g++ ≥ 4.9
- [fvcore](https://github.com/facebookresearch/fvcore)
- [ioPath](https://github.com/facebookresearch/iopath)
- If CUDA is to be used, use a version which is supported by the corresponding pytorch version and at least version 9.2.
- If CUDA is to be used and you are building from source, the CUB library must be available. We recommend version 1.10.0.
- If CUDA older than 11.7 is to be used and you are building from source, the CUB library must be available. We recommend version 1.10.0.
The runtime dependencies can be installed by running:
```
conda create -n pytorch3d python=3.8
conda create -n pytorch3d python=3.9
conda activate pytorch3d
conda install -c pytorch pytorch=1.7.1 torchvision cudatoolkit=10.2
conda install -c fvcore -c iopath -c conda-forge fvcore iopath
conda install pytorch=1.13.0 torchvision pytorch-cuda=11.6 -c pytorch -c nvidia
conda install -c iopath iopath
```
For the CUB build time dependency, if you are using conda, you can continue with
For the CUB build time dependency, which you only need if you have CUDA older than 11.7, if you are using conda, you can continue with
```
conda install -c bottler nvidiacub
```
@@ -43,12 +42,13 @@ export CUB_HOME=$PWD/cub-1.10.0
For developing on top of PyTorch3D or contributing, you will need to run the linter and tests. If you want to run any of the notebook tutorials as `docs/tutorials` or the examples in `docs/examples` you will also need matplotlib and OpenCV.
- scikit-image
- black
- isort
- usort
- flake8
- matplotlib
- tdqm
- jupyter
- imageio
- fvcore
- plotly
- opencv-python
@@ -59,7 +59,8 @@ conda install jupyter
pip install scikit-image matplotlib imageio plotly opencv-python
# Tests/Linting
pip install black 'isort<5' flake8 flake8-bugbear flake8-comprehensions
conda install -c fvcore -c conda-forge fvcore
pip install black usort flake8 flake8-bugbear flake8-comprehensions
```
## Installing prebuilt binaries for PyTorch3D
@@ -77,31 +78,28 @@ Or, to install a nightly (non-official, alpha) build:
# Anaconda Cloud
conda install pytorch3d -c pytorch3d-nightly
```
### 2. Install from PyPI, on Mac only.
This works with pytorch 1.9.0 only. The build is CPU only.
```
pip install pytorch3d
```
### 3. Install wheels for Linux
We have prebuilt wheels with CUDA for Linux for PyTorch 1.9.0, for each of the CUDA versions that they support,
for Python 3.7, 3.8 and 3.9.
### 2. Install wheels for Linux
We have prebuilt wheels with CUDA for Linux for PyTorch 1.11.0, for each of the supported CUDA versions,
for Python 3.8 and 3.9. This is for ease of use on Google Colab.
These are installed in a special way.
For example, to install for Python 3.8, PyTorch 1.9.0 and CUDA 10.2
For example, to install for Python 3.8, PyTorch 1.11.0 and CUDA 11.3
```
pip install pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/py38_cu102_pyt190/download.html
pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/py38_cu113_pyt1110/download.html
```
In general, from inside IPython, or in Google Colab or a jupyter notebook, you can install with
```
import sys
import torch
pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
version_str="".join([
f"py3{sys.version_info.minor}_cu",
torch.version.cuda.replace(".",""),
f"_pyt{torch.__version__[0:5:2]}"
f"_pyt{pyt_version_str}"
])
!pip install pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/{version_str}/download.html
!pip install iopath
!pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/{version_str}/download.html
```
## Building / installing from source.
@@ -146,10 +144,10 @@ After any necessary patching, you can go to "x64 Native Tools Command Prompt for
cd pytorch3d
python3 setup.py install
```
After installing, verify whether all unit tests have passed
After installing, you can run **unit tests**
```
cd tests
python3 -m unittest discover -p *.py
python3 -m unittest discover -v -s tests -t .
```
# FAQ

View File

@@ -2,7 +2,7 @@ BSD License
For PyTorch3D software
Copyright (c) Facebook, Inc. and its affiliates. All rights reserved.
Copyright (c) Meta Platforms, Inc. and affiliates. All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
@@ -14,7 +14,7 @@ are permitted provided that the following conditions are met:
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name Facebook nor the names of its contributors may be used to
* Neither the name Meta nor the names of its contributors may be used to
endorse or promote products derived from this software without specific
prior written permission.

71
LICENSE-3RD-PARTY Normal file
View File

@@ -0,0 +1,71 @@
SRN license ( https://github.com/vsitzmann/scene-representation-networks/ ):
MIT License
Copyright (c) 2019 Vincent Sitzmann
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
IDR license ( github.com/lioryariv/idr ):
MIT License
Copyright (c) 2020 Lior Yariv
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
NeRF https://github.com/bmild/nerf/
Copyright (c) 2020 bmild
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -1,4 +1,4 @@
<img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/master/.github/pytorch3dlogo.png" width="900"/>
<img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/pytorch3dlogo.png" width="900"/>
[![CircleCI](https://circleci.com/gh/facebookresearch/pytorch3d.svg?style=svg)](https://circleci.com/gh/facebookresearch/pytorch3d)
[![Anaconda-Server Badge](https://anaconda.org/pytorch3d/pytorch3d/badges/version.svg)](https://anaconda.org/pytorch3d/pytorch3d)
@@ -12,6 +12,7 @@ Key features include:
- Data structure for storing and manipulating triangle meshes
- Efficient operations on triangle meshes (projective transformations, graph convolution, sampling, loss functions)
- A differentiable mesh renderer
- Implicitron, see [its README](projects/implicitron_trainer), a framework for new-view synthesis via implicit representations. ([blog post](https://ai.facebook.com/blog/implicitron-a-new-modular-extensible-framework-for-neural-implicit-representations-in-pytorch3d/))
PyTorch3D is designed to integrate smoothly with deep learning methods for predicting and manipulating 3D data.
For this reason, all operators in PyTorch3D:
@@ -23,6 +24,8 @@ For this reason, all operators in PyTorch3D:
Within FAIR, PyTorch3D has been used to power research projects such as [Mesh R-CNN](https://arxiv.org/abs/1906.02739).
See our [blog post](https://ai.facebook.com/blog/-introducing-pytorch3d-an-open-source-library-for-3d-deep-learning/) to see more demos and learn about PyTorch3D.
## Installation
For detailed instructions refer to [INSTALL.md](INSTALL.md).
@@ -35,25 +38,30 @@ PyTorch3D is released under the [BSD License](LICENSE).
Get started with PyTorch3D by trying one of the tutorial notebooks.
|<img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/master/.github/dolphin_deform.gif" width="310"/>|<img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/master/.github/bundle_adjust.gif" width="310"/>|
|<img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/dolphin_deform.gif" width="310"/>|<img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/bundle_adjust.gif" width="310"/>|
|:-----------------------------------------------------------------------------------------------------------:|:--------------------------------------------------:|
| [Deform a sphere mesh to dolphin](https://github.com/facebookresearch/pytorch3d/blob/master/docs/tutorials/deform_source_mesh_to_target_mesh.ipynb)| [Bundle adjustment](https://github.com/facebookresearch/pytorch3d/blob/master/docs/tutorials/bundle_adjustment.ipynb) |
| [Deform a sphere mesh to dolphin](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/deform_source_mesh_to_target_mesh.ipynb)| [Bundle adjustment](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/bundle_adjustment.ipynb) |
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/master/.github/render_textured_mesh.gif" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/master/.github/camera_position_teapot.gif" width="310" height="310"/>
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/render_textured_mesh.gif" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/camera_position_teapot.gif" width="310" height="310"/>
|:------------------------------------------------------------:|:--------------------------------------------------:|
| [Render textured meshes](https://github.com/facebookresearch/pytorch3d/blob/master/docs/tutorials/render_textured_meshes.ipynb)| [Camera position optimization](https://github.com/facebookresearch/pytorch3d/blob/master/docs/tutorials/camera_position_optimization_with_differentiable_rendering.ipynb)|
| [Render textured meshes](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/render_textured_meshes.ipynb)| [Camera position optimization](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/camera_position_optimization_with_differentiable_rendering.ipynb)|
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/master/.github/pointcloud_render.png" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/master/.github/cow_deform.gif" width="310" height="310"/>
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/pointcloud_render.png" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/cow_deform.gif" width="310" height="310"/>
|:------------------------------------------------------------:|:--------------------------------------------------:|
| [Render textured pointclouds](https://github.com/facebookresearch/pytorch3d/blob/master/docs/tutorials/render_colored_points.ipynb)| [Fit a mesh with texture](https://github.com/facebookresearch/pytorch3d/blob/master/docs/tutorials/fit_textured_mesh.ipynb)|
| [Render textured pointclouds](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/render_colored_points.ipynb)| [Fit a mesh with texture](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/fit_textured_mesh.ipynb)|
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/master/.github/densepose_render.png" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/master/.github/shapenet_render.png" width="310" height="310"/>
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/densepose_render.png" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/shapenet_render.png" width="310" height="310"/>
|:------------------------------------------------------------:|:--------------------------------------------------:|
| [Render DensePose data](https://github.com/facebookresearch/pytorch3d/blob/master/docs/tutorials/render_densepose.ipynb)| [Load & Render ShapeNet data](https://github.com/facebookresearch/pytorch3d/blob/master/docs/tutorials/dataloaders_ShapeNetCore_R2N2.ipynb)|
| [Render DensePose data](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/render_densepose.ipynb)| [Load & Render ShapeNet data](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/dataloaders_ShapeNetCore_R2N2.ipynb)|
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/master/.github/fit_textured_volume.gif" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/master/.github/fit_nerf.gif" width="310" height="310"/>
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/fit_textured_volume.gif" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/fit_nerf.gif" width="310" height="310"/>
|:------------------------------------------------------------:|:--------------------------------------------------:|
| [Fit Textured Volume](https://github.com/facebookresearch/pytorch3d/blob/master/docs/tutorials/fit_textured_volume.ipynb)| [Fit A Simple Neural Radiance Field](https://github.com/facebookresearch/pytorch3d/blob/master/docs/tutorials/fit_simple_neural_radiance_field.ipynb)|
| [Fit Textured Volume](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/fit_textured_volume.ipynb)| [Fit A Simple Neural Radiance Field](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/fit_simple_neural_radiance_field.ipynb)|
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/fit_textured_volume.gif" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/implicitron_config.gif" width="310" height="310"/>
|:------------------------------------------------------------:|:--------------------------------------------------:|
| [Fit Textured Volume in Implicitron](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/implicitron_volumes.ipynb)| [Implicitron Config System](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/implicitron_config_system.ipynb)|
@@ -64,9 +72,9 @@ Learn more about the API by reading the PyTorch3D [documentation](https://pytorc
We also have deep dive notes on several API components:
- [Heterogeneous Batching](https://github.com/facebookresearch/pytorch3d/tree/master/docs/notes/batching.md)
- [Mesh IO](https://github.com/facebookresearch/pytorch3d/tree/master/docs/notes/meshes_io.md)
- [Differentiable Rendering](https://github.com/facebookresearch/pytorch3d/tree/master/docs/notes/renderer_getting_started.md)
- [Heterogeneous Batching](https://github.com/facebookresearch/pytorch3d/tree/main/docs/notes/batching.md)
- [Mesh IO](https://github.com/facebookresearch/pytorch3d/tree/main/docs/notes/meshes_io.md)
- [Differentiable Rendering](https://github.com/facebookresearch/pytorch3d/tree/main/docs/notes/renderer_getting_started.md)
### Overview Video
@@ -78,6 +86,13 @@ We have created a short (~14 min) video tutorial providing an overview of the Py
We welcome new contributions to PyTorch3D and we will be actively maintaining this library! Please refer to [CONTRIBUTING.md](./.github/CONTRIBUTING.md) for full instructions on how to run the code, tests and linter, and submit your pull requests.
## Development and Compatibility
- `main` branch: actively developed, without any guarantee, Anything can be broken at any time
- REMARK: this includes nightly builds which are built from `main`
- HINT: the commit history can help locate regressions or changes
- backward-compatibility between releases: no guarantee. Best efforts to communicate breaking changes and facilitate migration of code or data (incl. models).
## Contributors
PyTorch3D is written and maintained by the Facebook AI Research Computer Vision Team.
@@ -86,11 +101,13 @@ In alphabetical order:
* Amitav Baruah
* Steve Branson
* Krzysztof Chalupka
* Jiali Duan
* Luya Gao
* Georgia Gkioxari
* Taylor Gordon
* Justin Johnson
* Patrick Labtut
* Patrick Labatut
* Christoph Lassner
* Wan-Yen Lo
* David Novotny
@@ -129,7 +146,27 @@ If you are using the pulsar backend for sphere-rendering (the `PulsarPointRender
Please see below for a timeline of the codebase updates in reverse chronological order. We are sharing updates on the releases as well as research projects which are built with PyTorch3D. The changelogs for the releases are available under [`Releases`](https://github.com/facebookresearch/pytorch3d/releases), and the builds can be installed using `conda` as per the instructions in [INSTALL.md](INSTALL.md).
**[Feb 9th 2021]:** PyTorch3D [v0.4.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.4.0) released with support for implicit functions, volume rendering and a [reimplementation of NeRF](https://github.com/facebookresearch/pytorch3d/tree/master/projects/nerf).
**[Oct 31st 2023]:** PyTorch3D [v0.7.5](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.7.5) released.
**[May 10th 2023]:** PyTorch3D [v0.7.4](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.7.4) released.
**[Apr 5th 2023]:** PyTorch3D [v0.7.3](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.7.3) released.
**[Dec 19th 2022]:** PyTorch3D [v0.7.2](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.7.2) released.
**[Oct 23rd 2022]:** PyTorch3D [v0.7.1](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.7.1) released.
**[Aug 10th 2022]:** PyTorch3D [v0.7.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.7.0) released with Implicitron and MeshRasterizerOpenGL.
**[Apr 28th 2022]:** PyTorch3D [v0.6.2](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.6.2) released
**[Dec 16th 2021]:** PyTorch3D [v0.6.1](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.6.1) released
**[Oct 6th 2021]:** PyTorch3D [v0.6.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.6.0) released
**[Aug 5th 2021]:** PyTorch3D [v0.5.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.5.0) released
**[Feb 9th 2021]:** PyTorch3D [v0.4.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.4.0) released with support for implicit functions, volume rendering and a [reimplementation of NeRF](https://github.com/facebookresearch/pytorch3d/tree/main/projects/nerf).
**[November 2nd 2020]:** PyTorch3D [v0.3.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.3.0) released, integrating the pulsar backend.

View File

@@ -1,5 +1,5 @@
#!/bin/bash -e
# Copyright (c) Facebook, Inc. and its affiliates.
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
@@ -7,20 +7,17 @@
# Run this script at project root by "./dev/linter.sh" before you commit
{
V=$(black --version|cut '-d ' -f3)
code='import distutils.version; assert "19.3" < distutils.version.LooseVersion("'$V'")'
python -c "${code}" 2> /dev/null
} || {
echo "Linter requires black 19.3b0 or higher!"
exit 1
}
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
DIR=$(dirname "${DIR}")
echo "Running isort..."
isort -y -sp "${DIR}"
if [[ -f "${DIR}/BUCK" ]]
then
pyfmt "${DIR}"
else
# run usort externally only
echo "Running usort..."
usort "${DIR}"
fi
echo "Running black..."
black "${DIR}"
@@ -33,11 +30,11 @@ clangformat=$(command -v clang-format-8 || echo clang-format)
find "${DIR}" -regex ".*\.\(cpp\|c\|cc\|cu\|cuh\|cxx\|h\|hh\|hpp\|hxx\|tcc\|mm\|m\)" -print0 | xargs -0 "${clangformat}" -i
# Run arc and pyre internally only.
if [[ -f "${DIR}/tests/TARGETS" ]]
if [[ -f "${DIR}/TARGETS" ]]
then
(cd "${DIR}"; command -v arc > /dev/null && arc lint) || true
echo "Running pyre..."
echo "To restart/kill pyre server, run 'pyre restart' or 'pyre kill' in fbcode/"
( cd ~/fbsource/fbcode; pyre -l vision/fair/pytorch3d/ )
( cd ~/fbsource/fbcode; arc pyre check //vision/fair/pytorch3d/... )
fi

View File

@@ -1,5 +1,5 @@
#!/usr/bin/bash
# Copyright (c) Facebook, Inc. and its affiliates.
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
@@ -23,7 +23,7 @@ conda init bash
source ~/.bashrc
conda create -y -n myenv python=3.8 matplotlib ipython ipywidgets nbconvert
conda activate myenv
conda install -y -c fvcore -c iopath -c conda-forge fvcore iopath
conda install -y -c iopath iopath
conda install -y -c pytorch pytorch=1.6.0 cudatoolkit=10.1 torchvision
conda install -y -c pytorch3d-nightly pytorch3d
pip install plotly scikit-image

65
dev/test_list.py Normal file
View File

@@ -0,0 +1,65 @@
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
# LICENSE file in the root directory of this source tree.
import ast
from pathlib import Path
from typing import List
"""
This module outputs a list of tests for completion.
It has no dependencies.
"""
def get_test_files() -> List[Path]:
root = Path(__file__).parent.parent
dirs = ["tests", "projects/implicitron_trainer"]
return [i for dir in dirs for i in (root / dir).glob("**/test*.py")]
def tests_from_file(path: Path, base: str) -> List[str]:
"""
Returns all the tests in the given file, in format
expected as arguments when running the tests.
e.g.
file_stem
file_stem.TestFunctionality
file_stem.TestFunctionality.test_f
file_stem.TestFunctionality.test_g
"""
with open(path) as f:
node = ast.parse(f.read())
out = [base]
for cls in node.body:
if not isinstance(cls, ast.ClassDef):
continue
if not cls.name.startswith("Test"):
continue
class_base = base + "." + cls.name
out.append(class_base)
for method in cls.body:
if not isinstance(method, ast.FunctionDef):
continue
if not method.name.startswith("test"):
continue
out.append(class_base + "." + method.name)
return out
def main() -> None:
files = get_test_files()
test_root = Path(__file__).parent.parent
all_tests = []
for f in files:
file_base = str(f.relative_to(test_root))[:-3].replace("/", ".")
all_tests.extend(tests_from_file(f, file_base))
for test in sorted(all_tests):
print(test)
if __name__ == "__main__":
main()

27
docs/.readthedocs.yaml Normal file
View File

@@ -0,0 +1,27 @@
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
# LICENSE file in the root directory of this source tree.
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
# Required
version: 2
# Set the version of Python and other tools you might need
build:
os: ubuntu-22.04
tools:
python: "3.11"
# Build documentation in the docs/ directory with Sphinx
sphinx:
configuration: docs/conf.py
# We recommend specifying your dependencies to enable reproducible builds:
# https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html
python:
install:
- requirements: docs/requirements.txt

View File

@@ -1,4 +1,8 @@
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
# LICENSE file in the root directory of this source tree.
# Minimal makefile for Sphinx documentation

View File

@@ -3,7 +3,7 @@
### Install dependencies
```
pip install -U recommonmark mock sphinx sphinx_rtd_theme sphinx_markdown_tables
pip install -U recommonmark sphinx sphinx_rtd_theme sphinx_markdown_tables
```
### Add symlink to the root README.md

View File

@@ -1,5 +1,5 @@
# -*- coding: utf-8 -*-
# Copyright (c) Facebook, Inc. and its affiliates.
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
@@ -19,8 +19,8 @@
#
import os
import sys
import unittest.mock as mock
import mock
from recommonmark.parser import CommonMarkParser
from recommonmark.states import DummyStateMachine
from sphinx.builders.html import StandaloneHTMLBuilder
@@ -82,11 +82,11 @@ for m in ["cv2", "scipy", "numpy", "pytorch3d._C", "np.eye", "np.zeros"]:
# -- Project information -----------------------------------------------------
project = "PyTorch3D"
copyright = "2019, facebookresearch"
copyright = "Meta Platforms, Inc"
author = "facebookresearch"
# The short X.Y version
version = "0.2.0"
version = ""
# The full version, including alpha/beta/rc tags
release = version
@@ -159,7 +159,7 @@ html_theme_options = {"collapse_navigation": True}
def url_resolver(url):
if ".html" not in url:
url = url.replace("../", "")
return "https://github.com/facebookresearch/pytorch3d/blob/master/" + url
return "https://github.com/facebookresearch/pytorch3d/blob/main/" + url
else:
if DEPLOY:
return "http://pytorch3d.readthedocs.io/" + url

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) Facebook, Inc. and its affiliates.
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
@@ -10,6 +10,7 @@ This example demonstrates the most trivial, direct interface of the pulsar
sphere renderer. It renders and saves an image with 10 random spheres.
Output: basic.png.
"""
import logging
import math
from os import path

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) Facebook, Inc. and its affiliates.
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
@@ -11,6 +11,7 @@ interface for sphere renderering. It renders and saves an image with
10 random spheres.
Output: basic-pt3d.png.
"""
import logging
from os import path

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) Facebook, Inc. and its affiliates.
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
@@ -14,6 +14,7 @@ distorted. Gradient-based optimization is used to converge towards the
original camera parameters.
Output: cam.gif.
"""
import logging
import math
from os import path

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) Facebook, Inc. and its affiliates.
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
@@ -14,6 +14,7 @@ distorted. Gradient-based optimization is used to converge towards the
original camera parameters.
Output: cam-pt3d.gif
"""
import logging
from os import path

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) Facebook, Inc. and its affiliates.
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
@@ -18,6 +18,7 @@ This example is not available yet through the 'unified' interface,
because opacity support has not landed in PyTorch3D for general data
structures yet.
"""
import logging
import math
from os import path

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) Facebook, Inc. and its affiliates.
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
@@ -13,6 +13,7 @@ The scene is initialized with random spheres. Gradient-based
optimization is used to converge towards a faithful
scene representation.
"""
import logging
import math

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) Facebook, Inc. and its affiliates.
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
@@ -13,6 +13,7 @@ The scene is initialized with random spheres. Gradient-based
optimization is used to converge towards a faithful
scene representation.
"""
import logging
import math

162
docs/generate_stubs.py Normal file
View File

@@ -0,0 +1,162 @@
#!/usr/bin/env python3
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
# LICENSE file in the root directory of this source tree.
"""
This script makes the stubs for implicitron in docs/modules.
"""
from pathlib import Path
ROOT_DIR = Path(__file__).resolve().parent.parent
def paths_to_modules(paths):
"""
Given an iterable of paths, return equivalent list of modules.
"""
return [
str(i.relative_to(ROOT_DIR))[:-3].replace("/", ".")
for i in paths
if "__pycache__" not in str(i)
]
def create_one_file(title, description, sources, dest_file):
with open(dest_file, "w") as f:
print(title, file=f)
print("=" * len(title), file=f)
print(file=f)
print(description, file=f)
for source in sources:
if source.find("._") != -1:
# ignore internal modules including __init__.py
continue
print(f"\n.. automodule:: {source}", file=f)
print(" :members:", file=f)
print(" :undoc-members:", file=f)
print(" :show-inheritance:", file=f)
def iterate_directory(directory_path, dest):
"""
Create a file for each module in the given path
"""
toc = []
if not dest.exists():
dest.mkdir()
for file in sorted(directory_path.glob("*.py")):
if file.stem.startswith("_"):
continue
module = paths_to_modules([file])
create_one_file(module[0], file.stem, module, dest / f"{file.stem}.rst")
toc.append(file.stem)
for subdir in directory_path.iterdir():
if not subdir.is_dir():
continue
if subdir.name == "fb":
continue
if subdir.name.startswith("_"):
continue
iterate_directory(subdir, dest / (subdir.name))
toc.append(f"{subdir.name}/index")
paths_to_modules_ = paths_to_modules([directory_path.with_suffix(".XX")])
if len(paths_to_modules_) == 0:
return
title = paths_to_modules_[0]
with open(dest / "index.rst", "w") as f:
print(title, file=f)
print("=" * len(title), file=f)
print("\n.. toctree::\n", file=f)
for item in toc:
print(f" {item}", file=f)
def make_directory_index(title: str, directory_path: Path):
index_file = directory_path / "index.rst"
directory_rsts = sorted(directory_path.glob("*.rst"))
subdirs = sorted([f for f in directory_path.iterdir() if f.is_dir()])
with open(index_file, "w") as f:
print(title, file=f)
print("=" * len(title), file=f)
print("\n.. toctree::\n", file=f)
for subdir in subdirs:
print(f" {subdir.stem}/index.rst", file=f)
for rst in directory_rsts:
if rst.stem == "index":
continue
print(f" {rst.stem}", file=f)
def do_implicitron():
DEST_DIR = Path(__file__).resolve().parent / "modules/implicitron"
iterate_directory(ROOT_DIR / "pytorch3d/implicitron/models", DEST_DIR / "models")
unwanted_tools = ["configurable", "depth_cleanup", "utils"]
tools_sources = sorted(ROOT_DIR.glob("pytorch3d/implicitron/tools/*.py"))
tools_modules = [
str(i.relative_to(ROOT_DIR))[:-3].replace("/", ".")
for i in tools_sources
if i.stem not in unwanted_tools
]
create_one_file(
"pytorch3d.implicitron.tools",
"Tools for implicitron",
tools_modules,
DEST_DIR / "tools.rst",
)
dataset_files = sorted(ROOT_DIR.glob("pytorch3d/implicitron/dataset/*.py"))
basic_dataset = [
"dataset_base",
"dataset_map_provider",
"data_loader_map_provider",
"data_source",
"scene_batch_sampler",
]
basic_dataset_modules = [
f"pytorch3d.implicitron.dataset.{i}" for i in basic_dataset
]
create_one_file(
"pytorch3d.implicitron.dataset in general",
"Basics of data for implicitron",
basic_dataset_modules,
DEST_DIR / "data_basics.rst",
)
specific_dataset_files = [
i for i in dataset_files if i.stem.find("_dataset_map_provider") != -1
]
create_one_file(
"pytorch3d.implicitron.dataset specific datasets",
"specific datasets",
paths_to_modules(specific_dataset_files),
DEST_DIR / "datasets.rst",
)
evaluation_files = sorted(ROOT_DIR.glob("pytorch3d/implicitron/evaluation/*.py"))
create_one_file(
"pytorch3d.implicitron.evaluation",
"evaluation",
paths_to_modules(evaluation_files),
DEST_DIR / "evaluation.rst",
)
make_directory_index("pytorch3d.implicitron", DEST_DIR)
def iterate_toplevel_module(name: str) -> None:
dest_dir = Path(__file__).resolve().parent / "modules" / name
iterate_directory(ROOT_DIR / "pytorch3d" / name, dest_dir)
do_implicitron()
iterate_toplevel_module("renderer")
iterate_toplevel_module("vis")

View File

@@ -0,0 +1,29 @@
pytorch3d.implicitron.dataset in general
========================================
Basics of data for implicitron
.. automodule:: pytorch3d.implicitron.dataset.dataset_base
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.dataset.dataset_map_provider
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.dataset.data_loader_map_provider
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.dataset.data_source
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.dataset.scene_batch_sampler
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,29 @@
pytorch3d.implicitron.dataset specific datasets
===============================================
specific datasets
.. automodule:: pytorch3d.implicitron.dataset.blender_dataset_map_provider
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.dataset.json_index_dataset_map_provider
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.dataset.json_index_dataset_map_provider_v2
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.dataset.llff_dataset_map_provider
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.dataset.rendered_mesh_dataset_map_provider
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,14 @@
pytorch3d.implicitron.evaluation
================================
evaluation
.. automodule:: pytorch3d.implicitron.evaluation.evaluate_new_view_synthesis
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.evaluation.evaluator
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,10 @@
pytorch3d.implicitron
=====================
.. toctree::
models/index.rst
data_basics
datasets
evaluation
tools

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.base_model
=======================================
base_model
.. automodule:: pytorch3d.implicitron.models.base_model
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.feature_extractor.feature_extractor
================================================================
feature_extractor
.. automodule:: pytorch3d.implicitron.models.feature_extractor.feature_extractor
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,7 @@
pytorch3d.implicitron.models.feature_extractor
==============================================
.. toctree::
feature_extractor
resnet_feature_extractor

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.feature_extractor.resnet_feature_extractor
=======================================================================
resnet_feature_extractor
.. automodule:: pytorch3d.implicitron.models.feature_extractor.resnet_feature_extractor
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.generic_model
==========================================
generic_model
.. automodule:: pytorch3d.implicitron.models.generic_model
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.global_encoder.autodecoder
=======================================================
autodecoder
.. automodule:: pytorch3d.implicitron.models.global_encoder.autodecoder
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.global_encoder.global_encoder
==========================================================
global_encoder
.. automodule:: pytorch3d.implicitron.models.global_encoder.global_encoder
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,7 @@
pytorch3d.implicitron.models.global_encoder
===========================================
.. toctree::
autodecoder
global_encoder

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.implicit_function.base
===================================================
base
.. automodule:: pytorch3d.implicitron.models.implicit_function.base
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.implicit_function.decoding_functions
=================================================================
decoding_functions
.. automodule:: pytorch3d.implicitron.models.implicit_function.decoding_functions
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.implicit_function.idr_feature_field
================================================================
idr_feature_field
.. automodule:: pytorch3d.implicitron.models.implicit_function.idr_feature_field
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,13 @@
pytorch3d.implicitron.models.implicit_function
==============================================
.. toctree::
base
decoding_functions
idr_feature_field
neural_radiance_field
scene_representation_networks
utils
voxel_grid
voxel_grid_implicit_function

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.implicit_function.neural_radiance_field
====================================================================
neural_radiance_field
.. automodule:: pytorch3d.implicitron.models.implicit_function.neural_radiance_field
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.implicit_function.scene_representation_networks
============================================================================
scene_representation_networks
.. automodule:: pytorch3d.implicitron.models.implicit_function.scene_representation_networks
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.implicit_function.utils
====================================================
utils
.. automodule:: pytorch3d.implicitron.models.implicit_function.utils
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.implicit_function.voxel_grid
=========================================================
voxel_grid
.. automodule:: pytorch3d.implicitron.models.implicit_function.voxel_grid
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.implicit_function.voxel_grid_implicit_function
===========================================================================
voxel_grid_implicit_function
.. automodule:: pytorch3d.implicitron.models.implicit_function.voxel_grid_implicit_function
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,15 @@
pytorch3d.implicitron.models
============================
.. toctree::
base_model
generic_model
metrics
model_dbir
feature_extractor/index
global_encoder/index
implicit_function/index
renderer/index
view_pooler/index
visualization/index

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.metrics
====================================
metrics
.. automodule:: pytorch3d.implicitron.models.metrics
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.model_dbir
=======================================
model_dbir
.. automodule:: pytorch3d.implicitron.models.model_dbir
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.renderer.base
==========================================
base
.. automodule:: pytorch3d.implicitron.models.renderer.base
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,14 @@
pytorch3d.implicitron.models.renderer
=====================================
.. toctree::
base
lstm_renderer
multipass_ea
ray_point_refiner
ray_sampler
ray_tracing
raymarcher
rgb_net
sdf_renderer

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.renderer.lstm_renderer
===================================================
lstm_renderer
.. automodule:: pytorch3d.implicitron.models.renderer.lstm_renderer
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.renderer.multipass_ea
==================================================
multipass_ea
.. automodule:: pytorch3d.implicitron.models.renderer.multipass_ea
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.renderer.ray_point_refiner
=======================================================
ray_point_refiner
.. automodule:: pytorch3d.implicitron.models.renderer.ray_point_refiner
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.renderer.ray_sampler
=================================================
ray_sampler
.. automodule:: pytorch3d.implicitron.models.renderer.ray_sampler
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.renderer.ray_tracing
=================================================
ray_tracing
.. automodule:: pytorch3d.implicitron.models.renderer.ray_tracing
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.renderer.raymarcher
================================================
raymarcher
.. automodule:: pytorch3d.implicitron.models.renderer.raymarcher
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.renderer.rgb_net
=============================================
rgb_net
.. automodule:: pytorch3d.implicitron.models.renderer.rgb_net
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.renderer.sdf_renderer
==================================================
sdf_renderer
.. automodule:: pytorch3d.implicitron.models.renderer.sdf_renderer
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.view_pooler.feature_aggregator
===========================================================
feature_aggregator
.. automodule:: pytorch3d.implicitron.models.view_pooler.feature_aggregator
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,8 @@
pytorch3d.implicitron.models.view_pooler
========================================
.. toctree::
feature_aggregator
view_pooler
view_sampler

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.view_pooler.view_pooler
====================================================
view_pooler
.. automodule:: pytorch3d.implicitron.models.view_pooler.view_pooler
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.view_pooler.view_sampler
=====================================================
view_sampler
.. automodule:: pytorch3d.implicitron.models.view_pooler.view_sampler
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,6 @@
pytorch3d.implicitron.models.visualization
==========================================
.. toctree::
render_flyaround

View File

@@ -0,0 +1,9 @@
pytorch3d.implicitron.models.visualization.render_flyaround
===========================================================
render_flyaround
.. automodule:: pytorch3d.implicitron.models.visualization.render_flyaround
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,64 @@
pytorch3d.implicitron.tools
===========================
Tools for implicitron
.. automodule:: pytorch3d.implicitron.tools.camera_utils
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.tools.circle_fitting
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.tools.config
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.tools.eval_video_trajectory
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.tools.image_utils
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.tools.metric_utils
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.tools.model_io
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.tools.point_cloud_utils
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.tools.rasterize_mc
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.tools.stats
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.tools.video_writer
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.tools.vis_utils
:members:
:undoc-members:
:show-inheritance:

View File

@@ -3,7 +3,6 @@ API Documentation
.. toctree::
common
structures
io
loss
@@ -12,3 +11,6 @@ API Documentation
transforms
utils
datasets
common
vis/index
implicitron/index

View File

@@ -1,7 +1,9 @@
blending
pytorch3d.renderer.blending
===========================
blending
.. automodule:: pytorch3d.renderer.blending
:members:
:undoc-members:
:show-inheritance:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.camera_conversions
=====================================
camera_conversions
.. automodule:: pytorch3d.renderer.camera_conversions
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.camera_utils
===============================
camera_utils
.. automodule:: pytorch3d.renderer.camera_utils
:members:
:undoc-members:
:show-inheritance:

View File

@@ -1,7 +1,9 @@
pytorch3d.renderer.cameras
==========================
cameras
===========================
.. automodule:: pytorch3d.renderer.cameras
:members:
:undoc-members:
:show-inheritance:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.compositing
==============================
compositing
.. automodule:: pytorch3d.renderer.compositing
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.fisheyecameras
=================================
fisheyecameras
.. automodule:: pytorch3d.renderer.fisheyecameras
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.implicit.harmonic_embedding
==============================================
harmonic_embedding
.. automodule:: pytorch3d.renderer.implicit.harmonic_embedding
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,11 @@
pytorch3d.renderer.implicit
===========================
.. toctree::
harmonic_embedding
raymarching
raysampling
renderer
sample_pdf
utils

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.implicit.raymarching
=======================================
raymarching
.. automodule:: pytorch3d.renderer.implicit.raymarching
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.implicit.raysampling
=======================================
raysampling
.. automodule:: pytorch3d.renderer.implicit.raysampling
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.implicit.renderer
====================================
renderer
.. automodule:: pytorch3d.renderer.implicit.renderer
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.implicit.sample_pdf
======================================
sample_pdf
.. automodule:: pytorch3d.renderer.implicit.sample_pdf
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.implicit.utils
=================================
utils
.. automodule:: pytorch3d.renderer.implicit.utils
:members:
:undoc-members:
:show-inheritance:

View File

@@ -1,15 +1,19 @@
pytorch3d.renderer
===========================
pytorch3d.renderer
==================
.. toctree::
rasterizer
blending
camera_conversions
camera_utils
cameras
compositing
fisheyecameras
lighting
materials
texturing
blending
shading
shader
renderer
utils
splatter_blend
utils
implicit/index
mesh/index
opengl/index
points/index

View File

@@ -1,6 +1,9 @@
lighting
pytorch3d.renderer.lighting
===========================
lighting
.. automodule:: pytorch3d.renderer.lighting
:members:
:undoc-members:
:undoc-members:
:show-inheritance:

View File

@@ -1,7 +1,9 @@
pytorch3d.renderer.materials
============================
materials
===========================
.. automodule:: pytorch3d.renderer.materials
:members:
:undoc-members:
:show-inheritance:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.mesh.clip
============================
clip
.. automodule:: pytorch3d.renderer.mesh.clip
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,13 @@
pytorch3d.renderer.mesh
=======================
.. toctree::
clip
rasterize_meshes
rasterizer
renderer
shader
shading
textures
utils

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.mesh.rasterize_meshes
========================================
rasterize_meshes
.. automodule:: pytorch3d.renderer.mesh.rasterize_meshes
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.mesh.rasterizer
==================================
rasterizer
.. automodule:: pytorch3d.renderer.mesh.rasterizer
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.mesh.renderer
================================
renderer
.. automodule:: pytorch3d.renderer.mesh.renderer
:members:
:undoc-members:
:show-inheritance:

View File

@@ -1,7 +1,9 @@
pytorch3d.renderer.mesh.shader
==============================
shader
===========================
.. automodule:: pytorch3d.renderer.mesh.shader
:members:
:undoc-members:
:show-inheritance:

View File

@@ -1,7 +1,9 @@
pytorch3d.renderer.mesh.shading
===============================
shading
===========================
.. automodule:: pytorch3d.renderer.mesh.shading
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.mesh.textures
================================
textures
.. automodule:: pytorch3d.renderer.mesh.textures
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.mesh.utils
=============================
utils
.. automodule:: pytorch3d.renderer.mesh.utils
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,7 @@
pytorch3d.renderer.opengl
=========================
.. toctree::
opengl_utils
rasterizer_opengl

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.opengl.opengl_utils
======================================
opengl_utils
.. automodule:: pytorch3d.renderer.opengl.opengl_utils
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.opengl.rasterizer_opengl
===========================================
rasterizer_opengl
.. automodule:: pytorch3d.renderer.opengl.rasterizer_opengl
:members:
:undoc-members:
:show-inheritance:

View File

@@ -0,0 +1,9 @@
pytorch3d.renderer.points.compositor
====================================
compositor
.. automodule:: pytorch3d.renderer.points.compositor
:members:
:undoc-members:
:show-inheritance:

Some files were not shown because too many files have changed in this diff Show More