39 Commits

Author SHA1 Message Date
Itamar Oren
3143b3baf8 Read implicitron's own annotations via inspect.get_annotations
Summary:
`expand_args_fields` gated its whole member-processing pass on
`"__annotations__" in some_class.__dict__`. Under PEP 649 that entry is not in the
class dict until something materializes it, so on 3.14 the check was simply False,
no member was processed, and the pass that moves defaultable members to the end of
`__annotations__` never ran. The class then reached
`dataclasses.dataclass(eq=False)` with a defaulted `<name>_class_type` field ahead
of the non-defaulted `<name>` it replaces, which is a hard `TypeError` at class
construction.

`inspect.get_annotations` returns a class's own annotations — not a base's — which
is exactly what the `__dict__` lookup was expressing, and it behaves identically on
3.12. The in-place `del` / re-add on `some_class.__annotations__` further down is
unaffected: attribute access materializes and caches the dict on 3.14, so the
mutations stick and `dataclasses` sees them (verified on both interpreters).

325 canary failures in V58.

Reviewed By: bottler, ambv

Differential Revision: D115343017

fbshipit-source-id: 33f38dfac9f10c46b1f6b0e57f7005c9d7750dfa
2026-08-13 07:45:35 -07:00
generatedunixname89002005307016
9381c40163 Suppress type errors for Pyre upgrade
Summary:
This diff was automatically generated by the Pyre per-target upgrade tool.

It adds `# pyre-fixme` or `pyrefly: ignore` comments to suppress type errors that will be introduced by an upcoming Pyre or Pyrefly release. These suppressions allow the upgrade to proceed without breaking existing code.

#pyreupgrade

Differential Revision: D113955432

fbshipit-source-id: e586f1f97f2a9256a8da6431ec9c17231788995e
2026-07-28 14:57:23 -07:00
generatedunixname89002005307016
32a33e2442 Enable Pyrefly in fbcode/vision/fair/pytorch3d
Summary:
Automated migration to enable Pyrefly type checking for `fbcode/vision/fair/pytorch3d`.

- Added `python.set_pyrefly(True)` to PACKAGE file
- Suppressed pre-existing type errors

Pyrefly is Meta's next-generation Python type checker, replacing Pyre.

If you encounter issues, you can revert the PACKAGE change by removing
the `python.set_pyrefly(True)` line.
#pyreupgrade

Differential Revision: D113596106

fbshipit-source-id: a4caffc6c5622ea030fe296c3a64e45fa803f77e
2026-07-24 16:50:19 -07:00
generatedunixname1587093422349604
4daa00b41c Fix CQS signal facebook-unused-include-check in fbcode/vision/fair
Reviewed By: bottler

Differential Revision: D112297528

fbshipit-source-id: 592f5408e70b98361c79c0cff652e3bacccbb163
2026-07-16 04:21:36 -07:00
Roman Shapovalov
3b5ab51de5 Fix backprop through cot_laplacian
Summary:
`_cot_laplacian_python` used in-place tensor operations (`clamp_`, `/=`,
`+=`) on intermediates that participate in autograd. Once the resulting
sparse Laplacian is used in a backward pass, these in-place mutations
raise a runtime error:

```
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.sparse.FloatTensor [4, 4]], which is output 0 of SparseCooTensorWithDimsAndTensors, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True, check_nan=False).
```

Replace the in-place ops with out-of-place equivalents so gradients can
flow back to the input vertices.

- `clamp_` → `clamp`
- `cot /= 4.0` → `cot = cot / 4.0`
- `L += L.t()` → `L = L + L.t()`

Reviewed By: bottler

Differential Revision: D111896032

fbshipit-source-id: 4c36677487bae5d37a9d81cb791400576f4b2c5f
2026-07-14 12:07:08 -07:00
generatedunixname1587093422349604
c8fcd83ff9 Fix CQS signal facebook-unused-include-check in fbcode/vision/fair [A] [A] [A]
Reviewed By: bottler

Differential Revision: D109432550

fbshipit-source-id: 88b38ad0bffee13696cc2d8825f6314ce5e0c07b
2026-06-23 07:53:17 -07:00
Kasra Ghodsi
7f8a8a142f Image.ANTIALIAS -> Image.Resampling.LANCZOS
Summary:
Pillow 10 removed `Image.ANTIALIAS` (the built 10.4.0 wheel raises `AttributeError`). Replace it with `Image.Resampling.LANCZOS`, the documented successor, which is dual-compat across Pillow 9.4.0 / 10.4.0 / 11.3.0 / 12.2.0. This lands cleanly under the current 9.4.0 pin ahead of the fleet pin bump. Pure constant rename, no behavior change.

PyTorch3D is Meta-authored OSS (fbcode is the source of truth); this change is exported to github.com/facebookresearch/pytorch3d, where the `ANTIALIAS` -> `Resampling.LANCZOS` modernization is equally valid.

Part of the Pillow 9.x -> 10.x migration; see `third-party/pypi/pillow/.agents/migrate-9-to-10.md`.

___

Differential Revision: D108493452

fbshipit-source-id: bb1588e9b2057c6cc27a3d6c382faa4b2ac65f7f
2026-06-12 21:53:41 -07:00
generatedunixname89002005307016
f137c87cf9 Suppress type errors for Pyre upgrade
Summary:
This diff was automatically generated by the Pyre per-target upgrade tool.

It adds `# pyre-fixme` or `pyrefly: ignore` comments to suppress type errors that will be introduced by an upcoming Pyre or Pyrefly release. These suppressions allow the upgrade to proceed without breaking existing code.

wed - upgrade new suppression fix

#pyreupgrade

Differential Revision: D108188975

fbshipit-source-id: 65fb6fb0dbd6ade15bd8d85da912413eab2e41f9
2026-06-10 19:55:02 -07:00
Jeremy Reizenstein
d34e87ce52 validate input and edges dtype in GatherScatter python wrapper
Summary:
Add explicit dtype checks for input (torch.float32) and edges (torch.int64) in GatherScatter.forward and gather_scatter_python to match C++ TensorAccessor<float,2> and TensorAccessor<int64_t,2> expectations.

Python previously validated ndim, shape, and input dtype in forward but not edges dtype, and gather_scatter_python lacked dtype checks entirely, relying on ATen error from accessor. This makes errors python-friendly and guards C++ accessor before TensorAccessor construction.

___

Differential Revision: D108140422

fbshipit-source-id: ba54e857279a480a02e2c8f27e316f2e23cc6092
2026-06-10 11:29:55 -07:00
generatedunixname89002005232357
1f7f85c0a3 Revert D107142434: Enable Pyrefly in fbcode/vision/fair
Differential Revision:
D107142434

Original commit changeset: 25929bb3d5a3

Original Phabricator Diff: D107142434

fbshipit-source-id: 0aecaeba28d7d8db8f9273406a080e41aa77c4a7
2026-06-02 16:19:57 -07:00
generatedunixname89002005307016
05025bf005 Enable Pyrefly in fbcode/vision/fair
Summary:
Automated migration to enable Pyrefly type checking for `fbcode/vision/fair`.

- Added `python.set_pyrefly(True)` to PACKAGE file
- Suppressed pre-existing type errors

Pyrefly is Meta's next-generation Python type checker, replacing Pyre.

If you encounter issues, you can revert the PACKAGE change by removing
the `python.set_pyrefly(True)` line.
#pyreupgrade

Differential Revision: D107142434

fbshipit-source-id: 25929bb3d5a310d00dab11a46c5395df94357feb
2026-06-02 06:18:19 -07:00
Jeff Daily
b73d735ecf Port pytorch3d (#2039)
Summary:
Enables building pytorch3d's `_C` extension against a ROCm-built PyTorch and running the test suite on AMD GPUs, including the pulsar subrenderer. Verified on AMD Instinct MI250X (gfx90a, warpSize=64), HIP 7.2, PyTorch 2.13.

## Mechanics

`torch.utils.cpp_extension.BuildExtension` auto-hipifies `.cu` sources of a `CUDAExtension` against a HIP-built torch (`cuda_runtime.h → hip/hip_runtime.h`, `cub:: → hipcub::`, `cudaStream_t → hipStream_t`, etc.), so most of the lift is build-system glue and a small number of CUDA intrinsics that don't have HIP equivalents.

- `setup.py`: detect ROCm via `torch.version.hip is not None`; treat `ROCM_HOME` as the GPU-toolkit-root analogue of `CUDA_HOME` (without this, `CUDA_HOME is None` silently demoted the build to a CPU-only `CppExtension`); skip `CUB_HOME`, CUDA-13 visibility flags, and `-ccbin=` on ROCm.
- `pytorch3d/csrc/pulsar/gpu/commands.h`: CUDA's `_rn`-suffixed FP rounding intrinsics (`__fadd_rn`, `__fdiv_rn`, `__fsqrt_rn`, `__fmaf_rn`, `__frcp_rn`) and `__saturatef` have no HIP equivalents — AMD's GPU ISA has no instruction-level rounding-mode override, so they expand to plain operators / `sqrtf` / `fmaf` / `1.0f/x` / `fmaxf(0,fminf(1,x))` on the `USE_ROCM` arm, which are rounding-mode-equivalent (both round-to-nearest-even). The HIP compiler may fuse `a+b*c` into a single-rounding FMA where CUDA's `_rn` would have prevented it; if FMA-fusion drift ever becomes a numerical issue, add `-ffp-contract=off` to pulsar's HIPCC flags. `__powf` is replaced with `powf`. `atomicAdd_block` has no HIP function-name equivalent — the semantic equivalent is `__hip_atomic_fetch_add(ptr, val, __ATOMIC_RELAXED, __HIP_MEMORY_SCOPE_WORKGROUP)` (plain HIP `atomicAdd` is device-scope, strictly stronger than block-scope and forces L2-coherent atomics).
- `tests/test_point_mesh_distance.py`: loosen `grad_faces` tolerance in `test_point_face_distance` from `5e-7` to `5e-6` to match the sibling `test_face_point_distance`. The backward kernel uses `atomicAdd` and calls `alertNotDeterministic`; FP add order varies by wavefront width.
- The X_t / camera-R/T equality checks in `test_points_alignment.py` and `test_cameras_alignment.py` are now skipped when `n_points <= dim` (resp. `batch_size <= 3` for camera-center alignment in 3D). Mean-centering renders the SVD rank-deficient in those cases, so the rotation around the degenerate axis is non-unique and different BLAS implementations (rocBLAS RDNA vs CDNA, cuBLAS) pick different valid null-space directions. The center-alignment check still runs and verifies the well-defined part of the transformation.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/2039

Test Plan:
All GPU tests pass on both AMD Instinct MI250X (gfx90a, wave64, HIP 7.2) and AMD Radeon Pro W7800 (gfx1100, wave32, HIP 7.2.53211, torch 2.13.0a0).

| Module | Result |
|---|---|
| knn, ball_query, sample_farthest_points, face_areas_normals | all pass |
| rasterize_points, rasterize_meshes, chamfer, packed_to_padded | all pass |
| interpolate_face_attributes, blending, compositing, sample_pdf, mesh_normal_consistency | all pass |
| point_mesh_distance | 9/9 pass (with tolerance fix in this PR) |
| pulsar/test_forward, test_channels, test_depth, test_hands, test_ortho, test_small_spheres | 10 passed (FB_TEST=1) |
| test_render_points pulsar tests, test_camera_conversions::test_pulsar_conversion | 3 passed |
| points_to_volumes, iou_box3d, marching_cubes | 20 failures, all env-only |

The 20 env-only failures are `torch.inverse()` on CPU tensors in test reference paths; this verification host's PyTorch was built with `USE_LAPACK: 0` (only `mkl-static` `.a` archives in the conda env; PyTorch's `FindBLAS` looks for `libmkl_intel_lp64.so`). Unrelated to the port — re-verifying with a LAPACK-linked PyTorch is left to upstream.

Reviewed By: MichaelRamamonjisoa

Differential Revision: D106825690

Pulled By: bottler

fbshipit-source-id: f7a9b6028e6fb555f3b8c0f9792e88b818327166
2026-06-01 06:08:12 -07:00
generatedunixname89002005307016
c307c64c70 Suppress type errors for Pyre upgrade
Summary:
This diff was automatically generated by the Pyre per-target upgrade tool.

It adds `# pyre-fixme` or `pyrefly: ignore` comments to suppress type errors that will be introduced by an upcoming Pyre or Pyrefly release. These suppressions allow the upgrade to proceed without breaking existing code.

Pyrefly Upgrade - f-string fix

#pyreupgrade

Differential Revision: D105268300

fbshipit-source-id: 2f19758e20755944509fe14fc256002c652052a5
2026-05-14 20:27:36 -07:00
Jeremy Reizenstein
b6a77ad7aa [pytorch3d[ Remove LlffDatasetMapProvider and BlenderDatasetMapProvider
Summary:
No one is using these.

(The minify part has been broken for a couple of years, too)

Reviewed By: patricklabatut

Differential Revision: D96977684

fbshipit-source-id: 4708dfd37b14d1930f1370677eb126a61a0d9d3c
2026-03-18 10:09:59 -07:00
Dmitry Vinnik
52164b8324 Remove Support Ukraine banner from PyTorch3D website
Summary: Remove the Support Ukraine banner component and its usage from the PyTorch3D website homepage.

Reviewed By: bottler

Differential Revision: D96559642

fbshipit-source-id: fd716cde7145d5c0105b2d2fb569375395b9b5de
2026-03-16 06:36:16 -07:00
Jeremy Reizenstein
61cc79aa34 Make _sqrt_positive_part ONNX-exportable
Summary:
Replace boolean indexing and torch.is_grad_enabled() control flow in _sqrt_positive_part with a pure torch.where implementation. The old code used ret[positive_mask] = torch.sqrt(x[positive_mask]) which produces an incorrect ONNX Where/index_put node with mismatched broadcast shapes when the model is exported via torch.onnx.export.

The new implementation substitutes 1.0 for non-positive values before sqrt (avoiding infinite gradient at sqrt(0)) and masks the result back to 0, preserving the zero-subgradient-at-zero property.

Fixes https://github.com/facebookresearch/pytorch3d/issues/2020

Reviewed By: sgrigory

Differential Revision: D94365479

fbshipit-source-id: a1ebe8dc077573f83efc262520b6669159b83ef0
2026-03-06 05:23:55 -08:00
generatedunixname2645487282517272
7a6157e38e Fix CQS signal modernize-use-using in fbcode/vision/fair
Reviewed By: bottler

Differential Revision: D94879733

fbshipit-source-id: fc35eaaa723a2a035b3b204732add7ba8b225c57
2026-03-02 05:59:34 -08:00
generatedunixname1417043136753450
d9839a95f2 fbcode/vision/fair/pytorch3d/pytorch3d/ops/cameras_alignment.py
Reviewed By: sgrigory

Differential Revision: D93710806

fbshipit-source-id: da6c1e1e5b7a1c5cdfbf5026993c42c7ec387415
2026-02-23 15:52:03 -08:00
generatedunixname1417043136753450
7b5c78460a fbcode/vision/fair/pytorch3d/pytorch3d/transforms/se3.py
Reviewed By: sgrigory

Differential Revision: D93709801

fbshipit-source-id: e4bae81fe1a88fed547304e6e21b248c5a345277
2026-02-23 14:51:32 -08:00
generatedunixname1417043136753450
e3c80a4368 fbcode/vision/fair/pytorch3d/pytorch3d/renderer/splatter_blend.py
Reviewed By: sgrigory

Differential Revision: D93710022

fbshipit-source-id: 39253258b93a467fbda6b51ef8d6d3975bb49810
2026-02-23 12:43:53 -08:00
generatedunixname1417043136753450
b9b5ea3428 fbcode/vision/fair/pytorch3d/pytorch3d/common/workaround/symeig3x3.py
Reviewed By: sgrigory

Differential Revision: D93715209

fbshipit-source-id: 1880a8dd72e35ce5cc93cdeecf770aab6469ca31
2026-02-23 12:42:24 -08:00
generatedunixname1417043136753450
0e435c297c fbcode/vision/fair/pytorch3d/pytorch3d/ops/points_alignment.py
Reviewed By: sgrigory

Differential Revision: D93712744

fbshipit-source-id: 660560cdef9ff1d2173ae06de54df31766ee537f
2026-02-23 12:28:37 -08:00
generatedunixname1417043136753450
d631b56fba fbcode/vision/fair/pytorch3d/pytorch3d/ops/sample_farthest_points.py
Reviewed By: sgrigory

Differential Revision: D93708653

fbshipit-source-id: 112158092cd64ac8afddf1378b931cb44e19c372
2026-02-23 10:21:52 -08:00
generatedunixname915440834509264
3ba2030aa4 Fix CQS signal readability-braces-around-statements in fbcode/vision/fair
Reviewed By: bottler

Differential Revision: D94068738

fbshipit-source-id: cd47c67d4269ac7461acb73da6de9e4373da9d4c
2026-02-23 05:18:38 -08:00
generatedunixname1262449429094718
79a7fcf02b fbcode/vision/fair/pytorch3d/pytorch3d/csrc/rasterize_meshes/rasterize_meshes_cpu.cpp
Reviewed By: bottler

Differential Revision: D94062914

fbshipit-source-id: 9147dc68d115ce5761ebb7d07c035ac4b664da0b
2026-02-23 05:10:19 -08:00
generatedunixname1417043136753450
e43ed8c76e fbcode/vision/fair/pytorch3d/pytorch3d/transforms/rotation_conversions.py
Reviewed By: bottler

Differential Revision: D93712828

fbshipit-source-id: 3465af450104bb1e5f491e3c0ee0259698cf8ceb
2026-02-22 07:53:20 -08:00
generatedunixname1417043136753450
49f43402c6 fbcode/vision/fair/pytorch3d/pytorch3d/renderer/mesh/textures.py
Reviewed By: bottler

Differential Revision: D93710616

fbshipit-source-id: 599fe7425066bc85c0999765168788f8df7e34ce
2026-02-22 07:13:45 -08:00
generatedunixname1417043136753450
90646d93ab fbcode/vision/fair/pytorch3d/pytorch3d/renderer/mesh/clip.py
Reviewed By: bottler

Differential Revision: D93715239

fbshipit-source-id: 7417015251fe96be72daf4894e946edd43bb9c46
2026-02-22 07:13:09 -08:00
generatedunixname1417043136753450
eabb511410 fbcode/vision/fair/pytorch3d/pytorch3d/loss/mesh_laplacian_smoothing.py
Reviewed By: bottler

Differential Revision: D93709347

fbshipit-source-id: 69710e6082a0785126a121e26f1d96a571360f1d
2026-02-22 07:08:02 -08:00
generatedunixname1417043136753450
e70188ebbc fbcode/vision/fair/pytorch3d/pytorch3d/transforms/transform3d.py
Reviewed By: bottler

Differential Revision: D93713606

fbshipit-source-id: a8aa52328a76d95d3985daec529cdce04ba12bd4
2026-02-22 07:06:34 -08:00
generatedunixname1417043136753450
1bd911d534 fbcode/vision/fair/pytorch3d/pytorch3d/renderer/cameras.py
Reviewed By: bottler

Differential Revision: D93712137

fbshipit-source-id: 3457f0f9fb7d7baa29be2eaf731074a49bdbb0c8
2026-02-22 07:05:45 -08:00
generatedunixname1417043136753450
3aadd19a2b fbcode/vision/fair/pytorch3d/pytorch3d/ops/laplacian_matrices.py
Reviewed By: bottler

Differential Revision: D93708383

fbshipit-source-id: 7576f0c9800ed3d28795e521be5c63799b7e6676
2026-02-22 06:57:57 -08:00
generatedunixname1417043136753450
42d66c1145 fbcode/vision/fair/pytorch3d/pytorch3d/loss/point_mesh_distance.py
Reviewed By: bottler

Differential Revision: D93708351

fbshipit-source-id: 06a877777e4cb72a497a44ff55db0b6222bda83b
2026-02-22 06:55:36 -08:00
generatedunixname1417043136753450
e9ed1cb178 fbcode/vision/fair/pytorch3d/pytorch3d/renderer/utils.py
Reviewed By: bottler

Differential Revision: D93708316

fbshipit-source-id: f8ae2432ad34116278b3f7f7de5146b89c3fe63e
2026-02-22 04:09:20 -08:00
Jeremy Reizenstein
cbcae096a0 Add atol=1e-4 to assertClose calls in test_inverse for Translate
Summary:
Added `atol=1e-4` tolerance parameter to the `assertClose` calls on lines 682 and 683 in the `test_inverse` method of `TestTranslate` class.

This is a retry of D90225548

Reviewed By: sgrigory

Differential Revision: D90682979

fbshipit-source-id: ac13f000174dd9962326296e1c3116d0d39c7751
2026-01-14 08:57:43 -08:00
generatedunixname537391475639613
5b1cce56bc Fix for T251460511 ("Your diff, D90498281, broke one test")
Reviewed By: sgrigory

Differential Revision: D90649493

fbshipit-source-id: 2a77c45ec8e6e5aa0a20437a765fbb9f0b566406
2026-01-14 08:53:26 -08:00
Bowie Chen
0c3b204375 apply Black 25.11.0 style in fbcode (70/92)
Summary:
Formats the covered files with pyfmt.

paintitblack

Reviewed By: itamaro

Differential Revision: D90476295

fbshipit-source-id: 5101d4aae980a9f8955a4cb10bae23997c48837f
2026-01-12 02:54:36 -08:00
Jeremy Reizenstein
6be5e2da06 Replace assertTrue(torch.allclose(...)) with assertClose in test_transforms.py
Summary:
## LLM-generated Summary:
Replaces self.assertTrue(torch.allclose(...)) with self.assertClose(...) throughout fbcode/vision/fair/pytorch3d/tests/test_transforms.py. This standardizes numeric closeness assertions for clearer failures and consistency while preserving tolerances and test behavior.
 ---
Session: DEV34970678

Reviewed By: shapovalov

Differential Revision: D90251428

fbshipit-source-id: cdae842be82f0ba548802e6977be272134e8508c
2026-01-08 04:35:40 -08:00
Guilherme Albertini
f5f6b78e70 Add initial CUDA 13.0 support for pulsar and pycuda modules
Summary:
CUDA 13.0 introduced breaking changes that cause build failures in pytorch3d:

**1. Symbol Visibility Changes (pulsar)**
- NVCC now forces `__global__` functions to have hidden ELF visibility by default
- `__global__` function template stubs now have internal linkage

**Fix:** Added NVCC flags (`--device-entity-has-hidden-visibility=false` and `-static-global-template-stub=false`) for fbcode builds with CUDA 13.0+.

**2. cuCtxCreate API Change (pycuda)**
- CUDA 13.0 changed `cuCtxCreate` from 3 to 4 arguments
- pycuda 2022.2 (current default) uses the old signature and fails to compile
- pycuda 2025.1.2 (D83501913) includes the CUDA 13.0 fix

**Fix:** Added CUDA 13.0 constraint to pycuda alias to auto-select pycuda 2025.1.2.

**NCCL Compatibility Note:**
- Current stable NCCL (2.25) is NOT compatible with CUDA 13.0 (`cudaTypedefs.h` removed)
- NCCL 2.27+ works with CUDA 13.0 and will become stable in early January 2026 (per HPC Comms team)
- Until then, CUDA 13.0 builds require `-c hpc_comms.use_nccl=2.27`

References:
- GitHub issue: https://github.com/facebookresearch/pytorch3d/issues/2011
- NVIDIA blog: https://developer.nvidia.com/blog/cuda-c-compiler-updates-impacting-elf-visibility-and-linkage/
- FBGEMM_GPU fix: D86474263
- pycuda 2025.1.2 buckification: D83501913

Reviewed By: bottler

Differential Revision: D88816596

fbshipit-source-id: 1ba666dab8c0e06d1286b8d5bc5d84cfc55c86e6
2025-12-17 10:02:10 -08:00
163 changed files with 804 additions and 1410 deletions

View File

@@ -21,3 +21,45 @@ jobs:
run: |-
conda create --name env --yes --quiet conda-build
conda run --no-capture-output --name env python3 ./packaging/build_conda.py --use-conda-cuda
# Build-only verification for the ROCm/HIP code paths. Runs in an AMD ROCm
# dev container on a CPU-only GitHub runner; we don't need an AMD GPU just
# to compile, and not running tests keeps the CI cost low. Catches build
# regressions in the ROCm code paths (USE_ROCM guards, hipify-touched sources,
# the pulsar HIP intrinsic replacements, etc.).
linux_rocm_build:
runs-on: ubuntu-latest
container:
# `-complete` tag bundles the full ROCm math stack (rocThrust, hipCUB,
# rocPRIM, ...). The plain `7.2.3` tag is HIP-runtime-only and fails to
# find <thrust/complex.h> when including PyTorch headers.
image: rocm/dev-ubuntu-22.04:7.2.3-complete
env:
PYTORCH_VERSION: "2.11.0"
ROCM_INDEX: "rocm7.2"
steps:
- uses: actions/checkout@v4
- name: Install Python and torch+rocm
run: |-
apt-get update
apt-get install -y --no-install-recommends python3 python3-dev python3-pip git
python3 -m pip install --upgrade pip
python3 -m pip install --index-url https://download.pytorch.org/whl/${ROCM_INDEX} torch==${PYTORCH_VERSION}
- name: Verify torch is ROCm-built
run: |-
python3 -c "import torch; assert torch.version.hip is not None, 'torch is not HIP-built'; print('torch.version.hip:', torch.version.hip)"
- name: Build pytorch3d _C extension (build only, no tests)
env:
# CPU-only runner: torch.cuda.is_available() is False, so force the
# CUDAExtension path. ROCM_HOME is auto-detected from /opt/rocm in
# the rocm/dev-ubuntu container.
FORCE_CUDA: "1"
run: |-
python3 -m pip install --no-build-isolation -v .
- name: Smoke import
# cd out of the checkout root so the source-tree pytorch3d/ directory
# (which has no _C.so since the build doesn't install in-place) doesn't
# shadow the site-packages install via sys.path[0] for `python -c`.
run: |-
cd /tmp
python3 -c "import torch; from pytorch3d import _C; print('PulsarRenderer:', hasattr(_C, 'PulsarRenderer')); print('n_symbols:', len([s for s in dir(_C) if not s.startswith('_')]))"

11
.gitignore vendored
View File

@@ -6,6 +6,17 @@ dist/
**/.ipynb_checkpoints
**/.ipynb_checkpoints/**
# Build artifacts produced in-place by torch.utils.cpp_extension auto-hipify
# when pytorch3d is built against a ROCm PyTorch.
*.so
pytorch3d/csrc/**/*.hip
pytorch3d/csrc/**/*_hip.cpp
pytorch3d/csrc/**/*_hip.h
pytorch3d/csrc/**/*_hip.cuh
# Debug PNG dumps written by pulsar tests when FB_TEST is unset.
tests/pulsar/test_out/
# Docusaurus site
website/yarn.lock

View File

@@ -19,7 +19,6 @@
#
import os
import sys
import unittest.mock as mock
from recommonmark.parser import CommonMarkParser

View File

@@ -3,11 +3,6 @@ pytorch3d.implicitron.dataset specific datasets
specific datasets
.. automodule:: pytorch3d.implicitron.dataset.blender_dataset_map_provider
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.dataset.json_index_dataset_map_provider
:members:
:undoc-members:
@@ -18,11 +13,6 @@ specific datasets
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.dataset.llff_dataset_map_provider
:members:
:undoc-members:
:show-inheritance:
.. automodule:: pytorch3d.implicitron.dataset.rendered_mesh_dataset_map_provider
:members:
:undoc-members:

View File

@@ -1,56 +0,0 @@
defaults:
- overfit_singleseq_base
- _self_
exp_dir: "./data/overfit_nerf_blender_repro/${oc.env:BLENDER_SINGLESEQ_CLASS}"
data_source_ImplicitronDataSource_args:
data_loader_map_provider_SequenceDataLoaderMapProvider_args:
dataset_length_train: 100
dataset_map_provider_class_type: BlenderDatasetMapProvider
dataset_map_provider_BlenderDatasetMapProvider_args:
base_dir: ${oc.env:BLENDER_DATASET_ROOT}/${oc.env:BLENDER_SINGLESEQ_CLASS}
n_known_frames_for_test: null
object_name: ${oc.env:BLENDER_SINGLESEQ_CLASS}
path_manager_factory_class_type: PathManagerFactory
path_manager_factory_PathManagerFactory_args:
silence_logs: true
model_factory_ImplicitronModelFactory_args:
model_class_type: "OverfitModel"
model_OverfitModel_args:
mask_images: false
raysampler_class_type: AdaptiveRaySampler
raysampler_AdaptiveRaySampler_args:
n_pts_per_ray_training: 64
n_pts_per_ray_evaluation: 64
n_rays_per_image_sampled_from_mask: 4096
stratified_point_sampling_training: true
stratified_point_sampling_evaluation: false
scene_extent: 2.0
scene_center:
- 0.0
- 0.0
- 0.0
renderer_MultiPassEmissionAbsorptionRenderer_args:
density_noise_std_train: 0.0
n_pts_per_ray_fine_training: 128
n_pts_per_ray_fine_evaluation: 128
raymarcher_EmissionAbsorptionRaymarcher_args:
blend_output: false
loss_weights:
loss_rgb_mse: 1.0
loss_prev_stage_rgb_mse: 1.0
loss_mask_bce: 0.0
loss_prev_stage_mask_bce: 0.0
loss_autodecoder_norm: 0.00
optimizer_factory_ImplicitronOptimizerFactory_args:
exponential_lr_step_size: 3001
lr_policy: LinearExponential
linear_exponential_lr_milestone: 200
training_loop_ImplicitronTrainingLoop_args:
max_epochs: 6000
metric_print_interval: 10
store_checkpoints_purge: 3
test_when_finished: true
validation_interval: 100

View File

@@ -1,55 +0,0 @@
defaults:
- repro_singleseq_base
- _self_
exp_dir: "./data/nerf_blender_repro/${oc.env:BLENDER_SINGLESEQ_CLASS}"
data_source_ImplicitronDataSource_args:
data_loader_map_provider_SequenceDataLoaderMapProvider_args:
dataset_length_train: 100
dataset_map_provider_class_type: BlenderDatasetMapProvider
dataset_map_provider_BlenderDatasetMapProvider_args:
base_dir: ${oc.env:BLENDER_DATASET_ROOT}/${oc.env:BLENDER_SINGLESEQ_CLASS}
n_known_frames_for_test: null
object_name: ${oc.env:BLENDER_SINGLESEQ_CLASS}
path_manager_factory_class_type: PathManagerFactory
path_manager_factory_PathManagerFactory_args:
silence_logs: true
model_factory_ImplicitronModelFactory_args:
model_GenericModel_args:
mask_images: false
raysampler_class_type: AdaptiveRaySampler
raysampler_AdaptiveRaySampler_args:
n_pts_per_ray_training: 64
n_pts_per_ray_evaluation: 64
n_rays_per_image_sampled_from_mask: 4096
stratified_point_sampling_training: true
stratified_point_sampling_evaluation: false
scene_extent: 2.0
scene_center:
- 0.0
- 0.0
- 0.0
renderer_MultiPassEmissionAbsorptionRenderer_args:
density_noise_std_train: 0.0
n_pts_per_ray_fine_training: 128
n_pts_per_ray_fine_evaluation: 128
raymarcher_EmissionAbsorptionRaymarcher_args:
blend_output: false
loss_weights:
loss_rgb_mse: 1.0
loss_prev_stage_rgb_mse: 1.0
loss_mask_bce: 0.0
loss_prev_stage_mask_bce: 0.0
loss_autodecoder_norm: 0.00
optimizer_factory_ImplicitronOptimizerFactory_args:
exponential_lr_step_size: 3001
lr_policy: LinearExponential
linear_exponential_lr_milestone: 200
training_loop_ImplicitronTrainingLoop_args:
max_epochs: 6000
metric_print_interval: 10
store_checkpoints_purge: 3
test_when_finished: true
validation_interval: 100

View File

@@ -48,22 +48,18 @@ The outputs of the experiment are saved and logged in multiple ways:
import logging
import os
import warnings
from dataclasses import field
import hydra
import torch
from accelerate import Accelerator
from omegaconf import DictConfig, OmegaConf
from packaging import version
from pytorch3d.implicitron.dataset.data_source import (
DataSourceBase,
ImplicitronDataSource,
)
from pytorch3d.implicitron.models.base_model import ImplicitronModelBase
from pytorch3d.implicitron.models.renderer.multipass_ea import (
MultiPassEmissionAbsorptionRenderer,
)
@@ -197,6 +193,7 @@ class Experiment(Configurable):
last_epoch=start_epoch,
model=model,
resume=self.model_factory.resume,
# pyrefly: ignore [missing-attribute]
resume_epoch=self.model_factory.resume_epoch,
)
@@ -216,6 +213,7 @@ class Experiment(Configurable):
# Enter the main training loop.
self.training_loop.run(
# pyrefly: ignore [bad-argument-type]
train_loader=train_loader,
val_loader=val_loader,
test_loader=test_loader,

View File

@@ -11,7 +11,6 @@ import os
from typing import Optional
import torch.optim
from accelerate import Accelerator
from pytorch3d.implicitron.models.base_model import ImplicitronModelBase
from pytorch3d.implicitron.tools import model_io

View File

@@ -14,9 +14,7 @@ from dataclasses import field
from typing import Any, Dict, List, Optional, Tuple
import torch.optim
from accelerate import Accelerator
from pytorch3d.implicitron.models.base_model import ImplicitronModelBase
from pytorch3d.implicitron.tools import model_io
from pytorch3d.implicitron.tools.config import (
@@ -171,9 +169,11 @@ class ImplicitronOptimizerFactory(OptimizerFactoryBase):
gamma=self.gamma,
)
elif self.lr_policy.casefold() == "Exponential".casefold():
# pyre-fixme[28]: Unexpected keyword argument `verbose`.
scheduler = torch.optim.lr_scheduler.LambdaLR(
optimizer,
lambda epoch: self.gamma ** (epoch / self.exponential_lr_step_size),
# pyrefly: ignore [unexpected-keyword]
verbose=False,
)
elif self.lr_policy.casefold() == "LinearExponential".casefold():
@@ -190,8 +190,13 @@ class ImplicitronOptimizerFactory(OptimizerFactoryBase):
gamma = self.gamma ** (epoch_rest / self.exponential_lr_step_size)
return gamma
# pyre-fixme[28]: Unexpected keyword argument `verbose`.
scheduler = torch.optim.lr_scheduler.LambdaLR(
optimizer, _get_lr, verbose=False
# pyrefly: ignore [unexpected-keyword]
optimizer,
_get_lr,
# pyrefly: ignore [unexpected-keyword]
verbose=False,
)
else:
raise ValueError("no such lr policy %s" % self.lr_policy)

View File

@@ -199,6 +199,7 @@ class ImplicitronTrainingLoop(TrainingLoopBase):
and self.test_interval > 0
and epoch % self.test_interval == 0
):
# pyrefly: ignore [missing-attribute]
self.evaluator.run(
device=device,
dataloader=test_loader,
@@ -215,6 +216,7 @@ class ImplicitronTrainingLoop(TrainingLoopBase):
if self.test_when_finished:
if test_loader is not None:
# pyrefly: ignore [missing-attribute]
self.evaluator.run(
device=device,
dump_to_json=True,

View File

@@ -13,13 +13,6 @@ hydra:
data_source_ImplicitronDataSource_args:
dataset_map_provider_class_type: ???
data_loader_map_provider_class_type: SequenceDataLoaderMapProvider
dataset_map_provider_BlenderDatasetMapProvider_args:
base_dir: ???
object_name: ???
path_manager_factory_class_type: PathManagerFactory
n_known_frames_for_test: null
path_manager_factory_PathManagerFactory_args:
silence_logs: true
dataset_map_provider_JsonIndexDatasetMapProvider_args:
category: ???
task_str: singlesequence
@@ -91,14 +84,6 @@ data_source_ImplicitronDataSource_args:
sort_frames: false
path_manager_factory_PathManagerFactory_args:
silence_logs: true
dataset_map_provider_LlffDatasetMapProvider_args:
base_dir: ???
object_name: ???
path_manager_factory_class_type: PathManagerFactory
n_known_frames_for_test: null
path_manager_factory_PathManagerFactory_args:
silence_logs: true
downscale_factor: 4
dataset_map_provider_RenderedMeshDatasetMapProvider_args:
num_views: 40
data_file: null

View File

@@ -12,7 +12,6 @@ import unittest
from pathlib import Path
import torch
from hydra import compose, initialize_config_dir
from omegaconf import OmegaConf
from projects.implicitron_trainer.impl.optimizer_factory import (

View File

@@ -135,6 +135,7 @@ def visualize_reconstruction(
"device": device,
**render_flyaround_kwargs,
}
# pyrefly: ignore [bad-argument-type]
render_flyaround(**render_kwargs)

View File

@@ -82,12 +82,15 @@ class _SymEig3x3(nn.Module):
q = inputs_trace / 3.0
# Calculate squared sum of elements outside the main diagonal / 2
# pyre-fixme[58]: `**` is not supported for operand types `Tensor` and `int`.
p1 = ((inputs**2).sum(dim=(-1, -2)) - (inputs_diag**2).sum(-1)) / 2
# pyre-fixme[58]: `**` is not supported for operand types `Tensor` and `int`.
p2 = ((inputs_diag - q[..., None]) ** 2).sum(dim=-1) + 2.0 * p1.clamp(self._eps)
p1 = (
torch.square(inputs).sum(dim=(-1, -2)) - torch.square(inputs_diag).sum(-1)
) / 2
p2 = torch.square(inputs_diag - q[..., None]).sum(dim=-1) + 2.0 * p1.clamp(
self._eps
)
p = torch.sqrt(p2 / 6.0)
# pyrefly: ignore [unsupported-operation]
B = (inputs - q[..., None, None] * self._identity) / p[..., None, None]
r = torch.det(B) / 2.0
@@ -104,7 +107,9 @@ class _SymEig3x3(nn.Module):
# Soft dispatch between the degenerate case (diagonal A) and general.
# diag_soft_cond -> 1.0 when p1 < 6 * eps and diag_soft_cond -> 0.0 otherwise.
# We use 6 * eps to take into account the error accumulated during the p1 summation
diag_soft_cond = torch.exp(-((p1 / (6 * self._eps)) ** 2)).detach()[..., None]
diag_soft_cond = torch.exp(-torch.square(p1 / (6 * self._eps))).detach()[
..., None
]
# Eigenvalues are the ordered elements of main diagonal in the degenerate case
diag_eigenvals, _ = torch.sort(inputs_diag, dim=-1)
@@ -170,8 +175,10 @@ class _SymEig3x3(nn.Module):
"""
# Find the eigenvector corresponding to alpha0, its eigenvalue is distinct
# pyrefly: ignore [unsupported-operation]
ev0 = self._get_ev0(inputs - alpha0[..., None, None] * self._identity)
u, v = self._get_uv(ev0)
# pyrefly: ignore [unsupported-operation]
ev1 = self._get_ev1(inputs - alpha1[..., None, None] * self._identity, u, v)
# Third eigenvector is computed as the cross-product of the other two
ev2 = torch.cross(ev0, ev1, dim=-1)
@@ -199,8 +206,7 @@ class _SymEig3x3(nn.Module):
cross_products[..., :1, :]
)
# pyre-fixme[58]: `**` is not supported for operand types `Tensor` and `int`.
norms_sq = (cross_products**2).sum(dim=-1)
norms_sq = torch.square(cross_products).sum(dim=-1)
max_norms_index = norms_sq.argmax(dim=-1)
# Pick only the cross-product with highest squared norm for each input
@@ -247,6 +253,7 @@ class _SymEig3x3(nn.Module):
"""
min_idx = w.abs().argmin(dim=-1)
# pyrefly: ignore [bad-index]
rotation_2d = self._rotations_3d[min_idx].to(w)
u = F.normalize((rotation_2d @ w[..., None])[..., 0], dim=-1)

View File

@@ -98,7 +98,6 @@ PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) {
m.def("marching_cubes", &MarchingCubes);
// Pulsar.
// Pulsar not enabled on AMD.
#ifdef PULSAR_LOGGING_ENABLED
c10::ShowLogInfoToStderr();
#endif

View File

@@ -7,7 +7,6 @@
*/
#include <torch/extension.h>
#include <list>
#include <tuple>
#include "iou_box3d/iou_utils.h"

View File

@@ -125,29 +125,45 @@ INLINE DEVICE float3 WARP_SUM_FLOAT3(
// Floating point.
// #define FMUL(a, b) __fmul_rn((a), (b))
#define FMUL(a, b) ((a) * (b))
#define FDIV(a, b) __fdiv_rn((a), (b))
// #define FSUB(a, b) __fsub_rn((a), (b))
#define FSUB(a, b) ((a) - (b))
// HIP has no per-instruction rounding-mode override (round-to-nearest-even is
// the default at the hardware level), so the CUDA *_rn intrinsics have no
// runtime equivalent and we use plain operators. The HIP/clang compiler may
// fuse a+b*c into a single-rounding FMA where CUDA's _rn would have prevented
// it; if that becomes a numerical issue, add `-ffp-contract=off` to the pulsar
// nvcc_args in setup.py.
#if defined(USE_ROCM)
#define FDIV(a, b) ((a) / (b))
#define FADD(a, b) ((a) + (b))
#define FSQRT(a) sqrtf(a)
#define FPOW(a, b) powf((a), (b))
#define FSATURATE(x) fmaxf(0.0f, fminf(1.0f, (x)))
#define FMA(x, y, z) fmaf((x), (y), (z))
#define FRCP(x) (1.0f / (x))
#else
#define FDIV(a, b) __fdiv_rn((a), (b))
#define FADD(a, b) __fadd_rn((a), (b))
#define FSQRT(a) __fsqrt_rn(a)
#define FPOW(a, b) __powf((a), (b))
#define FSATURATE(x) __saturatef(x)
/** Calculates x*y+z. */
#define FMA(x, y, z) __fmaf_rn((x), (y), (z))
#define FRCP(x) __frcp_rn(x)
#endif
#define FEXP(a) fasterexp(a)
#define FLN(a) fasterlog(a)
#define FPOW(a, b) __powf((a), (b))
#define FMAX(a, b) fmax((a), (b))
#define FMIN(a, b) fmin((a), (b))
#define FCEIL(a) ceilf(a)
#define FFLOOR(a) floorf(a)
#define FROUND(x) nearbyintf(x)
#define FSATURATE(x) __saturatef(x)
#define FABS(a) abs(a)
#define IASF(a, loc) (loc) = __int_as_float(a)
#define FASI(a, loc) (loc) = __float_as_int(a)
#define FABSLEQAS(a, b, c) \
((a) <= (b) ? FSUB((b), (a)) <= (c) : FSUB((a), (b)) < (c))
/** Calculates x*y+z. */
#define FMA(x, y, z) __fmaf_rn((x), (y), (z))
#define I2F(a) __int2float_rn(a)
#define FRCP(x) __frcp_rn(x)
#if !defined(USE_ROCM)
__device__ static float atomicMax(float* address, float val) {
int* address_as_i = (int*)address;
@@ -201,8 +217,16 @@ __device__ static float atomicMin(float* address, float val) {
ATOMICADD(&((PTR)->x), VAL.x); \
ATOMICADD(&((PTR)->y), VAL.y); \
ATOMICADD(&((PTR)->z), VAL.z);
#if (CUDART_VERSION >= 10000) && (__CUDA_ARCH__ >= 600)
#if !defined(USE_ROCM) && (CUDART_VERSION >= 10000) && (__CUDA_ARCH__ >= 600)
#define ATOMICADD_B(PTR, VAL) atomicAdd_block((PTR), (VAL))
#elif defined(USE_ROCM)
// HIP has no atomicAdd_block, but the semantic equivalent is a relaxed
// fetch_add scoped to the workgroup (HIP's name for a CUDA thread block).
// This avoids device-wide L2-coherent atomics for what are block-local
// counters in pulsar's inner sphere-loading loop.
#define ATOMICADD_B(PTR, VAL) \
__hip_atomic_fetch_add( \
(PTR), (VAL), __ATOMIC_RELAXED, __HIP_MEMORY_SCOPE_WORKGROUP)
#else
#define ATOMICADD_B(PTR, VAL) ATOMICADD(PTR, VAL)
#endif

View File

@@ -18,68 +18,89 @@ namespace Renderer {
template <bool DEV>
HOST void destruct(Renderer* self) {
if (self->result_d != NULL)
if (self->result_d != NULL) {
FREE(self->result_d);
}
self->result_d = NULL;
if (self->min_depth_d != NULL)
if (self->min_depth_d != NULL) {
FREE(self->min_depth_d);
}
self->min_depth_d = NULL;
if (self->min_depth_sorted_d != NULL)
if (self->min_depth_sorted_d != NULL) {
FREE(self->min_depth_sorted_d);
}
self->min_depth_sorted_d = NULL;
if (self->ii_d != NULL)
if (self->ii_d != NULL) {
FREE(self->ii_d);
}
self->ii_d = NULL;
if (self->ii_sorted_d != NULL)
if (self->ii_sorted_d != NULL) {
FREE(self->ii_sorted_d);
}
self->ii_sorted_d = NULL;
if (self->ids_d != NULL)
if (self->ids_d != NULL) {
FREE(self->ids_d);
}
self->ids_d = NULL;
if (self->ids_sorted_d != NULL)
if (self->ids_sorted_d != NULL) {
FREE(self->ids_sorted_d);
}
self->ids_sorted_d = NULL;
if (self->workspace_d != NULL)
if (self->workspace_d != NULL) {
FREE(self->workspace_d);
}
self->workspace_d = NULL;
if (self->di_d != NULL)
if (self->di_d != NULL) {
FREE(self->di_d);
}
self->di_d = NULL;
if (self->di_sorted_d != NULL)
if (self->di_sorted_d != NULL) {
FREE(self->di_sorted_d);
}
self->di_sorted_d = NULL;
if (self->region_flags_d != NULL)
if (self->region_flags_d != NULL) {
FREE(self->region_flags_d);
}
self->region_flags_d = NULL;
if (self->num_selected_d != NULL)
if (self->num_selected_d != NULL) {
FREE(self->num_selected_d);
}
self->num_selected_d = NULL;
if (self->forw_info_d != NULL)
if (self->forw_info_d != NULL) {
FREE(self->forw_info_d);
}
self->forw_info_d = NULL;
if (self->min_max_pixels_d != NULL)
if (self->min_max_pixels_d != NULL) {
FREE(self->min_max_pixels_d);
}
self->min_max_pixels_d = NULL;
if (self->grad_pos_d != NULL)
if (self->grad_pos_d != NULL) {
FREE(self->grad_pos_d);
}
self->grad_pos_d = NULL;
if (self->grad_col_d != NULL)
if (self->grad_col_d != NULL) {
FREE(self->grad_col_d);
}
self->grad_col_d = NULL;
if (self->grad_rad_d != NULL)
if (self->grad_rad_d != NULL) {
FREE(self->grad_rad_d);
}
self->grad_rad_d = NULL;
if (self->grad_cam_d != NULL)
if (self->grad_cam_d != NULL) {
FREE(self->grad_cam_d);
}
self->grad_cam_d = NULL;
if (self->grad_cam_buf_d != NULL)
if (self->grad_cam_buf_d != NULL) {
FREE(self->grad_cam_buf_d);
}
self->grad_cam_buf_d = NULL;
if (self->grad_opy_d != NULL)
if (self->grad_opy_d != NULL) {
FREE(self->grad_opy_d);
}
self->grad_opy_d = NULL;
if (self->n_grad_contributions_d != NULL)
if (self->n_grad_contributions_d != NULL) {
FREE(self->n_grad_contributions_d);
}
self->n_grad_contributions_d = NULL;
}

View File

@@ -64,8 +64,9 @@ GLOBAL void norm_sphere_gradients(Renderer renderer, const int num_balls) {
// The sphere only contributes to the camera gradients if it is
// large enough in screen space.
if (renderer.ids_sorted_d[idx] > 0 && ii.max.x >= ii.min.x + 3 &&
ii.max.y >= ii.min.y + 3)
ii.max.y >= ii.min.y + 3) {
renderer.ids_sorted_d[idx] = 1;
}
END_PARALLEL_NORET();
};

View File

@@ -139,8 +139,9 @@ GLOBAL void render(
coord_y < cam_norm.film_border_top + cam_norm.film_height) {
// Initialize the result.
if (mode == 0u) {
for (uint c_id = 0; c_id < cam_norm.n_channels; ++c_id)
for (uint c_id = 0; c_id < cam_norm.n_channels; ++c_id) {
result[c_id] = bg_col[c_id];
}
} else {
result[0] = 0.f;
}
@@ -190,20 +191,22 @@ GLOBAL void render(
"render|found intersection with sphere %u.\n",
sphere_id_l[write_idx]);
}
if (ii.min.x == MAX_USHORT)
if (ii.min.x == MAX_USHORT) {
// This is an invalid sphere (out of image). These spheres have
// maximum depth. Since we ordered the spheres by earliest possible
// intersection depth we re certain that there will no other sphere
// that is relevant after this one.
loading_done = true;
}
}
// Reset n_pixels_done.
n_pixels_done = 0;
thread_block.sync(); // Make sure n_loaded is updated.
if (n_loaded > RENDER_BUFFER_LOAD_THRESH) {
// The load buffer is full enough. Draw.
if (thread_block.thread_rank() == 0)
if (thread_block.thread_rank() == 0) {
n_balls_loaded += n_loaded;
}
max_closest_possible_intersection = 0.f;
// This excludes threads outside of the image boundary. Also, it reduces
// block artifacts.
@@ -290,8 +293,9 @@ GLOBAL void render(
uint warp_done = thread_warp.ballot(done);
int warp_done_bit_cnt = POPC(warp_done);
#endif //__CUDACC__ && __HIP_PLATFORM_AMD__
if (thread_warp.thread_rank() == 0)
if (thread_warp.thread_rank() == 0) {
ATOMICADD_B(&n_pixels_done, warp_done_bit_cnt);
}
// This sync is necessary to keep n_loaded until all threads are done with
// painting.
thread_block.sync();
@@ -299,8 +303,9 @@ GLOBAL void render(
}
thread_block.sync();
}
if (thread_block.thread_rank() == 0)
if (thread_block.thread_rank() == 0) {
n_balls_loaded += n_loaded;
}
PULSAR_LOG_DEV_PIX(
PULSAR_LOG_RENDER_PIX,
"render|loaded %d balls in total.\n",
@@ -386,8 +391,9 @@ GLOBAL void render(
static_cast<float>(tracker.get_n_hits());
} else {
float sm_d_normfac = FRCP(FMAX(sm_d, FEPS));
for (uint c_id = 0; c_id < cam_norm.n_channels; ++c_id)
for (uint c_id = 0; c_id < cam_norm.n_channels; ++c_id) {
result[c_id] *= sm_d_normfac;
}
int write_loc = (coord_y - cam_norm.film_border_top) * cam_norm.film_width *
(3 + 2 * n_track) +
(coord_x - cam_norm.film_border_left) * (3 + 2 * n_track);

View File

@@ -7,13 +7,10 @@
*/
#include "./renderer.h"
#include "../include/commands.h"
#include "./camera.h"
#include "./util.h"
#include <ATen/ATen.h>
#ifdef WITH_CUDA
#include <ATen/cuda/CUDAContext.h>
#include <c10/cuda/CUDAGuard.h>
#endif
@@ -860,8 +857,9 @@ std::tuple<torch::Tensor, torch::Tensor> Renderer::forward(
? (cudaStream_t) nullptr
#endif
: (cudaStream_t) nullptr);
if (mode == 1)
if (mode == 1) {
results[batch_i] = results[batch_i].slice(2, 0, 1, 1);
}
forw_infos[batch_i] = from_blob(
this->renderer_vec[batch_i].forw_info_d,
{this->renderer_vec[0].cam.film_height,

View File

@@ -128,8 +128,9 @@ struct Renderer {
stream << "pulsar::Renderer[";
// Device info.
stream << self.device_type;
if (self.device_index != -1)
if (self.device_index != -1) {
stream << ", ID " << self.device_index;
}
stream << "]";
return stream;
}

View File

@@ -8,7 +8,6 @@
#include <torch/extension.h>
#include <algorithm>
#include <list>
#include <thread>
#include <tuple>
#include "ATen/core/TensorAccessor.h"
@@ -106,6 +105,8 @@ auto ComputeFaceAreas(const torch::Tensor& face_verts) {
return face_areas;
}
namespace {
// Helper function to use with std::find_if to find the index of any
// values in the top k struct which match a given idx.
struct IsNeighbor {
@@ -118,7 +119,6 @@ struct IsNeighbor {
int neighbor_idx;
};
namespace {
void RasterizeMeshesNaiveCpu_worker(
const int start_yi,
const int end_yi,

View File

@@ -19,7 +19,7 @@ template <
std::is_same<T, double>::value || std::is_same<T, float>::value>>
struct vec2 {
T x, y;
typedef T scalar_t;
using scalar_t = T;
vec2(T x, T y) : x(x), y(y) {}
};

View File

@@ -18,7 +18,7 @@ template <
std::is_same<T, double>::value || std::is_same<T, float>::value>>
struct vec3 {
T x, y, z;
typedef T scalar_t;
using scalar_t = T;
vec3(T x, T y, T z) : x(x), y(y), z(z) {}
};

View File

@@ -10,10 +10,20 @@
#include <math.h>
#include <cstdio>
// Helper functions WarpReduceMin and WarpReduceMax used in .cu files
// Starting in Volta, instructions are no longer synchronous within a warp.
// We need to call __syncwarp() to sync the 32 threads in the warp
// instead of all the threads in the block.
// Helper functions WarpReduceMin and WarpReduceMax used in .cu files.
// Starting in Volta, instructions are no longer synchronous within a warp,
// so on CUDA __syncwarp() is required between dependent shared-memory
// accesses in the unrolled tail reduction.
//
// On AMD/HIP no __syncwarp() is needed here: all wavefront lanes execute
// in lockstep (AMD has no equivalent of NVIDIA's Independent Thread
// Scheduling), and the AMDGPU memory model guarantees that LDS operations
// issued by the same wavefront are observed in program order without an
// explicit s_waitcnt — see the LLVM AMDGPU backend memory-model rules
// (https://llvm.org/docs/AMDGPUUsage.html) and the HIP hardware-
// implementation docs. This holds for both wave32 (RDNA, gfx10xx/11xx/
// 12xx) and wave64 (CDNA, gfx9xx), so the USE_ROCM skip is
// architecture-independent.
template <typename scalar_t>
__device__ void
@@ -23,8 +33,6 @@ WarpReduceMin(scalar_t* min_dists, int64_t* min_idxs, const size_t tid) {
min_idxs[tid] = min_idxs[tid + 32];
min_dists[tid] = min_dists[tid + 32];
}
// AMD does not use explicit syncwarp and instead automatically inserts memory
// fences during compilation.
#if !defined(USE_ROCM)
__syncwarp();
#endif

View File

@@ -200,6 +200,7 @@ class R2N2(ShapeNetBase): # pragma: no cover
) % (shapenet_dir, ", ".join(synset_not_present))
warnings.warn(msg)
# pyrefly: ignore [bad-override]
def __getitem__(self, model_idx, view_idxs: Optional[List[int]] = None) -> Dict:
"""
Read a model by the given index.
@@ -370,6 +371,7 @@ class R2N2(ShapeNetBase): # pragma: no cover
T = RT[3, :3]
return R, T
# pyrefly: ignore [bad-override]
def render(
self,
model_ids: Optional[List[str]] = None,

View File

@@ -62,8 +62,10 @@ def collate_batched_R2N2(batch: List[Dict]): # pragma: no cover
# all models have the same number of views V, stack the batches of
# views of each model into a new batch of shape (N, V, H, W, 3).
# Otherwise leave it as a list.
# pyrefly: ignore [not-iterable]
if "images" in collated_dict:
try:
# pyrefly: ignore [unsupported-operation]
collated_dict["images"] = torch.stack(collated_dict["images"])
except RuntimeError:
print(
@@ -75,10 +77,14 @@ def collate_batched_R2N2(batch: List[Dict]): # pragma: no cover
# matrices and that all models have the same number of views V, stack each
# type of matrices into a new batch of shape (N, V, ...).
# Otherwise leave them as lists.
# pyrefly: ignore [not-iterable]
if all(x in collated_dict for x in ["R", "T", "K"]):
try:
# pyrefly: ignore [unsupported-operation]
collated_dict["R"] = torch.stack(collated_dict["R"]) # (N, V, 3, 3)
# pyrefly: ignore [unsupported-operation]
collated_dict["T"] = torch.stack(collated_dict["T"]) # (N, V, 3)
# pyrefly: ignore [unsupported-operation]
collated_dict["K"] = torch.stack(collated_dict["K"]) # (N, V, 4, 4)
except RuntimeError:
print(
@@ -89,8 +95,10 @@ def collate_batched_R2N2(batch: List[Dict]): # pragma: no cover
# If collate_batched_meshes receives voxels and all models have the same
# number of views V, stack the batches of voxels into a new batch of shape
# (N, V, S, S, S), where S is the voxel size.
# pyrefly: ignore [not-iterable]
if "voxels" in collated_dict:
try:
# pyrefly: ignore [unsupported-operation]
collated_dict["voxels"] = torch.stack(collated_dict["voxels"])
except RuntimeError:
print(
@@ -458,6 +466,8 @@ class BlenderCamera(CamerasBase): # pragma: no cover
def get_projection_transform(self, **kwargs) -> Transform3d:
transform = Transform3d(device=self.device)
# pyre-fixme[29]: `Union[(self: TensorBase, dim0: int, dim1: int) -> Tensor,
# Tensor, Module]` is not a function.
transform._matrix = self.K.transpose(1, 2).contiguous()
return transform

View File

@@ -52,6 +52,7 @@ class ShapeNetBase(torch.utils.data.Dataset): # pragma: no cover
"""
return len(self.model_ids)
# pyrefly: ignore [bad-override-param-name]
def __getitem__(self, idx) -> Dict:
"""
Read a model by the given index. Need to be implemented for every child class
@@ -147,12 +148,17 @@ class ShapeNetBase(torch.utils.data.Dataset): # pragma: no cover
idxs = self._handle_render_inputs(model_ids, categories, sample_nums, idxs)
# Use the getitem method which loads mesh + texture
models = [self[idx] for idx in idxs]
# pyrefly: ignore [unsupported-operation]
meshes = collate_batched_meshes(models)["mesh"]
# pyrefly: ignore [missing-attribute]
if meshes.textures is None:
# pyrefly: ignore [missing-attribute]
meshes.textures = TexturesVertex(
# pyrefly: ignore [missing-attribute]
verts_features=torch.ones_like(meshes.verts_padded(), device=device)
)
# pyrefly: ignore [missing-attribute]
meshes = meshes.to(device)
cameras = kwargs.get("cameras", FoVPerspectiveCameras()).to(device)
if len(cameras) != 1 and len(cameras) % len(meshes) != 0:

View File

@@ -34,12 +34,14 @@ def collate_batched_meshes(batch: List[Dict]): # pragma: no cover
for k in batch[0].keys():
collated_dict[k] = [d[k] for d in batch]
# pyrefly: ignore [unsupported-operation]
collated_dict["mesh"] = None
if {"verts", "faces"}.issubset(collated_dict.keys()):
textures = None
if "textures" in collated_dict:
textures = TexturesAtlas(atlas=collated_dict["textures"])
# pyrefly: ignore [unsupported-operation]
collated_dict["mesh"] = Meshes(
verts=collated_dict["verts"],
faces=collated_dict["faces"],

View File

@@ -1,55 +0,0 @@
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
# LICENSE file in the root directory of this source tree.
# pyre-unsafe
import torch
from pytorch3d.implicitron.tools.config import registry
from .load_blender import load_blender_data
from .single_sequence_dataset import (
_interpret_blender_cameras,
SingleSceneDatasetMapProviderBase,
)
@registry.register
class BlenderDatasetMapProvider(SingleSceneDatasetMapProviderBase):
"""
Provides data for one scene from Blender synthetic dataset.
Uses the code in load_blender.py
Members:
base_dir: directory holding the data for the scene.
object_name: The name of the scene (e.g. "lego"). This is just used as a label.
It will typically be equal to the name of the directory self.base_dir.
path_manager_factory: Creates path manager which may be used for
interpreting paths.
n_known_frames_for_test: If set, training frames are included in the val
and test datasets, and this many random training frames are added to
each test batch. If not set, test batches each contain just a single
testing frame.
"""
def _load_data(self) -> None:
path_manager = self.path_manager_factory.get()
images, poses, _, hwf, i_split = load_blender_data(
self.base_dir,
testskip=1,
path_manager=path_manager,
)
H, W, focal = hwf
images_masks = torch.from_numpy(images).permute(0, 3, 1, 2)
# pyre-ignore[16]
self.poses = _interpret_blender_cameras(poses, focal)
# pyre-ignore[16]
self.images = images_masks[:, :3]
# pyre-ignore[16]
self.fg_probabilities = images_masks[:, 3:4]
# pyre-ignore[16]
self.i_split = i_split

View File

@@ -64,16 +64,12 @@ class ImplicitronDataSource(DataSourceBase):
def pre_expand(cls) -> None:
# use try/finally to bypass cinder's lazy imports
try:
from .blender_dataset_map_provider import ( # noqa: F401
BlenderDatasetMapProvider,
)
from .json_index_dataset_map_provider import ( # noqa: F401
JsonIndexDatasetMapProvider,
)
from .json_index_dataset_map_provider_v2 import ( # noqa: F401
JsonIndexDatasetMapProviderV2,
)
from .llff_dataset_map_provider import LlffDatasetMapProvider # noqa: F401
from .rendered_mesh_dataset_map_provider import ( # noqa: F401
RenderedMeshDatasetMapProvider,
)

View File

@@ -21,7 +21,6 @@ from typing import (
)
import torch
from pytorch3d.implicitron.dataset.frame_data import FrameData
from pytorch3d.implicitron.dataset.utils import GenericWorkaround
@@ -102,6 +101,7 @@ class DatasetBase(GenericWorkaround, torch.utils.data.Dataset[FrameData]):
# crashes without overriding __getitem__
sequence_category = self[first_frame_idx].sequence_category
c2seq[sequence_category].append(sequence_name)
# pyrefly: ignore [bad-return]
return dict(c2seq)
def sequence_frames_in_order(

View File

@@ -25,7 +25,6 @@ from typing import (
import numpy as np
import torch
from pytorch3d.implicitron.dataset import orm_types, types
from pytorch3d.implicitron.dataset.utils import (
adjust_camera_to_bbox_crop_,
@@ -298,7 +297,11 @@ class FrameData(Mapping[str, Any]):
depth_map = self.depth_map
if depth_map is not None:
clamp_bbox_xyxy_depth = rescale_bbox(
clamp_bbox_xyxy, tuple(depth_map.shape[-2:]), effective_image_size_hw
# pyrefly: ignore [bad-argument-type]
clamp_bbox_xyxy,
# pyrefly: ignore [bad-argument-type]
tuple(depth_map.shape[-2:]),
effective_image_size_hw,
).long()
self.depth_map = crop_around_box(
depth_map,
@@ -309,7 +312,11 @@ class FrameData(Mapping[str, Any]):
depth_mask = self.depth_mask
if depth_mask is not None:
clamp_bbox_xyxy_depth = rescale_bbox(
clamp_bbox_xyxy, tuple(depth_mask.shape[-2:]), effective_image_size_hw
# pyrefly: ignore [bad-argument-type]
clamp_bbox_xyxy,
# pyrefly: ignore [bad-argument-type]
tuple(depth_mask.shape[-2:]),
effective_image_size_hw,
).long()
self.depth_mask = crop_around_box(
depth_mask,
@@ -454,6 +461,7 @@ class FrameDataBuilderBase(ReplaceableBase, Generic[FrameDataSubtype], ABC):
"""
# To be initialised to FrameDataSubtype
# pyrefly: ignore [invalid-annotation]
frame_data_type: ClassVar[Type[FrameDataSubtype]]
@abstractmethod

View File

@@ -38,7 +38,6 @@ from pytorch3d.implicitron.dataset.utils import is_known_frame_scalar
from pytorch3d.implicitron.tools.config import registry, ReplaceableBase
from pytorch3d.renderer.camera_utils import join_cameras_as_batch
from pytorch3d.renderer.cameras import CamerasBase
from tqdm import tqdm
@@ -327,9 +326,9 @@ class JsonIndexDataset(DatasetBase, ReplaceableBase):
assert os.path.normpath(
# pyre-ignore[16]
self.frame_annots[idx]["frame_annotation"].image.path
) == os.path.normpath(
path
), f"Inconsistent frame indices {seq_name, frame_no, path}."
) == os.path.normpath(path), (
f"Inconsistent frame indices {seq_name, frame_no, path}."
)
return idx
dataset_idx = [
@@ -426,6 +425,7 @@ class JsonIndexDataset(DatasetBase, ReplaceableBase):
raise ValueError("subsets not loaded")
if is_known_frame_scalar(frame_type):
cameras.append(self[frame_idx].camera)
# pyrefly: ignore [bad-argument-type]
return join_cameras_as_batch(cameras)
def __getitem__(self, index) -> FrameData:

View File

@@ -21,7 +21,6 @@ from pytorch3d.renderer.cameras import CamerasBase
from .dataset_map_provider import DatasetMap, DatasetMapProviderBase, PathManagerFactory
from .json_index_dataset import JsonIndexDataset
from .utils import (
DATASET_TYPE_KNOWN,
DATASET_TYPE_TEST,

View File

@@ -18,7 +18,6 @@ from typing import Dict, List, Optional, Tuple, Type, Union
import numpy as np
from iopath.common.file_io import PathManager
from omegaconf import DictConfig
from pytorch3d.implicitron.dataset.dataset_map_provider import (
DatasetMap,
@@ -31,7 +30,6 @@ from pytorch3d.implicitron.tools.config import (
registry,
run_auto_creation,
)
from pytorch3d.renderer.cameras import CamerasBase
from tqdm import tqdm
@@ -313,6 +311,7 @@ class JsonIndexDatasetMapProviderV2(DatasetMapProviderBase):
subset_mapping["test"],
) = self._extend_test_data_with_known_views(
subset_mapping,
# pyrefly: ignore [bad-argument-type]
eval_batch_index,
)
@@ -324,6 +323,7 @@ class JsonIndexDatasetMapProviderV2(DatasetMapProviderBase):
try:
test_dataset.eval_batches = (
test_dataset.seq_frame_index_to_dataset_index(
# pyrefly: ignore [bad-argument-type]
eval_batch_index,
)
)
@@ -337,6 +337,7 @@ class JsonIndexDatasetMapProviderV2(DatasetMapProviderBase):
)
test_dataset.eval_batches = (
test_dataset.seq_frame_index_to_dataset_index(
# pyrefly: ignore [bad-argument-type]
eval_batch_index,
allow_missing_indices=True,
remove_missing_indices=True,

View File

@@ -1,69 +0,0 @@
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
# LICENSE file in the root directory of this source tree.
# pyre-unsafe
import numpy as np
import torch
from pytorch3d.implicitron.tools.config import registry
from .load_llff import load_llff_data
from .single_sequence_dataset import (
_interpret_blender_cameras,
SingleSceneDatasetMapProviderBase,
)
@registry.register
class LlffDatasetMapProvider(SingleSceneDatasetMapProviderBase):
"""
Provides data for one scene from the LLFF dataset.
Members:
base_dir: directory holding the data for the scene.
object_name: The name of the scene (e.g. "fern"). This is just used as a label.
It will typically be equal to the name of the directory self.base_dir.
path_manager_factory: Creates path manager which may be used for
interpreting paths.
n_known_frames_for_test: If set, training frames are included in the val
and test datasets, and this many random training frames are added to
each test batch. If not set, test batches each contain just a single
testing frame.
downscale_factor: determines image sizes.
"""
downscale_factor: int = 4
def _load_data(self) -> None:
path_manager = self.path_manager_factory.get()
images, poses, _ = load_llff_data(
self.base_dir, factor=self.downscale_factor, path_manager=path_manager
)
hwf = poses[0, :3, -1]
poses = poses[:, :3, :4]
llffhold = 8
i_test = np.arange(images.shape[0])[::llffhold]
i_test_index = set(i_test.tolist())
i_train = np.array(
[i for i in np.arange(images.shape[0]) if i not in i_test_index]
)
i_split = (i_train, i_test, i_test)
H, W, focal = hwf
focal_ndc = 2 * focal / min(H, W)
images = torch.from_numpy(images).permute(0, 3, 1, 2)
poses = torch.from_numpy(poses)
# pyre-ignore[16]
self.poses = _interpret_blender_cameras(poses, focal_ndc)
# pyre-ignore[16]
self.images = images
# pyre-ignore[16]
self.fg_probabilities = None
# pyre-ignore[16]
self.i_split = i_split

View File

@@ -1,143 +0,0 @@
# @lint-ignore-every LICENSELINT
# Adapted from https://github.com/bmild/nerf/blob/master/load_blender.py
# Copyright (c) 2020 bmild
# pyre-unsafe
import json
import os
import numpy as np
import torch
from PIL import Image
def translate_by_t_along_z(t):
tform = np.eye(4).astype(np.float32)
tform[2][3] = t
return tform
def rotate_by_phi_along_x(phi):
tform = np.eye(4).astype(np.float32)
tform[1, 1] = tform[2, 2] = np.cos(phi)
tform[1, 2] = -np.sin(phi)
tform[2, 1] = -tform[1, 2]
return tform
def rotate_by_theta_along_y(theta):
tform = np.eye(4).astype(np.float32)
tform[0, 0] = tform[2, 2] = np.cos(theta)
tform[0, 2] = -np.sin(theta)
tform[2, 0] = -tform[0, 2]
return tform
def pose_spherical(theta, phi, radius):
c2w = translate_by_t_along_z(radius)
c2w = rotate_by_phi_along_x(phi / 180.0 * np.pi) @ c2w
c2w = rotate_by_theta_along_y(theta / 180 * np.pi) @ c2w
c2w = np.array([[-1, 0, 0, 0], [0, 0, 1, 0], [0, 1, 0, 0], [0, 0, 0, 1]]) @ c2w
return c2w
def _local_path(path_manager, path):
if path_manager is None:
return path
return path_manager.get_local_path(path)
def load_blender_data(
basedir,
half_res=False,
testskip=1,
debug=False,
path_manager=None,
focal_length_in_screen_space=False,
):
splits = ["train", "val", "test"]
metas = {}
for s in splits:
path = os.path.join(basedir, f"transforms_{s}.json")
with open(_local_path(path_manager, path)) as fp:
metas[s] = json.load(fp)
all_imgs = []
all_poses = []
counts = [0]
for s in splits:
meta = metas[s]
imgs = []
poses = []
if s == "train" or testskip == 0:
skip = 1
else:
skip = testskip
for frame in meta["frames"][::skip]:
fname = os.path.join(basedir, frame["file_path"] + ".png")
imgs.append(np.array(Image.open(_local_path(path_manager, fname))))
poses.append(np.array(frame["transform_matrix"]))
imgs = (np.array(imgs) / 255.0).astype(np.float32)
poses = np.array(poses).astype(np.float32)
counts.append(counts[-1] + imgs.shape[0])
all_imgs.append(imgs)
all_poses.append(poses)
i_split = [np.arange(counts[i], counts[i + 1]) for i in range(3)]
imgs = np.concatenate(all_imgs, 0)
poses = np.concatenate(all_poses, 0)
H, W = imgs[0].shape[:2]
camera_angle_x = float(meta["camera_angle_x"])
if focal_length_in_screen_space:
focal = 0.5 * W / np.tan(0.5 * camera_angle_x)
else:
focal = 1 / np.tan(0.5 * camera_angle_x)
render_poses = torch.stack(
[
torch.from_numpy(pose_spherical(angle, -30.0, 4.0))
for angle in np.linspace(-180, 180, 40 + 1)[:-1]
],
0,
)
# In debug mode, return extremely tiny images
if debug:
import cv2
H = H // 32
W = W // 32
if focal_length_in_screen_space:
focal = focal / 32.0
imgs = [
torch.from_numpy(
cv2.resize(imgs[i], dsize=(25, 25), interpolation=cv2.INTER_AREA)
)
for i in range(imgs.shape[0])
]
imgs = torch.stack(imgs, 0)
poses = torch.from_numpy(poses)
return imgs, poses, render_poses, [H, W, focal], i_split
if half_res:
import cv2
# TODO: resize images using INTER_AREA (cv2)
H = H // 2
W = W // 2
if focal_length_in_screen_space:
focal = focal / 2.0
imgs = [
torch.from_numpy(
cv2.resize(imgs[i], dsize=(400, 400), interpolation=cv2.INTER_AREA)
)
for i in range(imgs.shape[0])
]
imgs = torch.stack(imgs, 0)
poses = torch.from_numpy(poses)
return imgs, poses, render_poses, [H, W, focal], i_split

View File

@@ -1,336 +0,0 @@
# @lint-ignore-every LICENSELINT
# Adapted from https://github.com/bmild/nerf/blob/master/load_llff.py
# Copyright (c) 2020 bmild
# pyre-unsafe
import logging
import os
import warnings
import numpy as np
from PIL import Image
# Slightly modified version of LLFF data loading code
# see https://github.com/Fyusion/LLFF for original
logger = logging.getLogger(__name__)
def _minify(basedir, path_manager, factors=(), resolutions=()):
needtoload = False
for r in factors:
imgdir = os.path.join(basedir, "images_{}".format(r))
if not _exists(path_manager, imgdir):
needtoload = True
for r in resolutions:
imgdir = os.path.join(basedir, "images_{}x{}".format(r[1], r[0]))
if not _exists(path_manager, imgdir):
needtoload = True
if not needtoload:
return
assert path_manager is None
from subprocess import check_output
imgdir = os.path.join(basedir, "images")
imgs = [os.path.join(imgdir, f) for f in sorted(_ls(path_manager, imgdir))]
imgs = [f for f in imgs if f.endswith("JPG", "jpg", "png", "jpeg", "PNG")]
imgdir_orig = imgdir
wd = os.getcwd()
for r in factors + resolutions:
if isinstance(r, int):
name = "images_{}".format(r)
resizearg = "{}%".format(100.0 / r)
else:
name = "images_{}x{}".format(r[1], r[0])
resizearg = "{}x{}".format(r[1], r[0])
imgdir = os.path.join(basedir, name)
if os.path.exists(imgdir):
continue
logger.info(f"Minifying {r}, {basedir}")
os.makedirs(imgdir)
check_output("cp {}/* {}".format(imgdir_orig, imgdir), shell=True)
ext = imgs[0].split(".")[-1]
args = " ".join(
["mogrify", "-resize", resizearg, "-format", "png", "*.{}".format(ext)]
)
logger.info(args)
os.chdir(imgdir)
check_output(args, shell=True)
os.chdir(wd)
if ext != "png":
check_output("rm {}/*.{}".format(imgdir, ext), shell=True)
logger.info("Removed duplicates")
logger.info("Done")
def _load_data(
basedir, factor=None, width=None, height=None, load_imgs=True, path_manager=None
):
poses_arr = np.load(
_local_path(path_manager, os.path.join(basedir, "poses_bounds.npy"))
)
poses = poses_arr[:, :-2].reshape([-1, 3, 5]).transpose([1, 2, 0])
bds = poses_arr[:, -2:].transpose([1, 0])
img0 = [
os.path.join(basedir, "images", f)
for f in sorted(_ls(path_manager, os.path.join(basedir, "images")))
if f.endswith("JPG") or f.endswith("jpg") or f.endswith("png")
][0]
def imread(f):
return np.array(Image.open(f))
sh = imread(_local_path(path_manager, img0)).shape
sfx = ""
if factor is not None:
sfx = "_{}".format(factor)
_minify(basedir, path_manager, factors=[factor])
factor = factor
elif height is not None:
factor = sh[0] / float(height)
width = int(sh[1] / factor)
_minify(basedir, path_manager, resolutions=[[height, width]])
sfx = "_{}x{}".format(width, height)
elif width is not None:
factor = sh[1] / float(width)
height = int(sh[0] / factor)
_minify(basedir, path_manager, resolutions=[[height, width]])
sfx = "_{}x{}".format(width, height)
else:
factor = 1
imgdir = os.path.join(basedir, "images" + sfx)
if not _exists(path_manager, imgdir):
raise ValueError(f"{imgdir} does not exist, returning")
imgfiles = [
_local_path(path_manager, os.path.join(imgdir, f))
for f in sorted(_ls(path_manager, imgdir))
if f.endswith("JPG") or f.endswith("jpg") or f.endswith("png")
]
if poses.shape[-1] != len(imgfiles):
raise ValueError(
"Mismatch between imgs {} and poses {} !!!!".format(
len(imgfiles), poses.shape[-1]
)
)
sh = imread(imgfiles[0]).shape
poses[:2, 4, :] = np.array(sh[:2]).reshape([2, 1])
poses[2, 4, :] = poses[2, 4, :] * 1.0 / factor
if not load_imgs:
return poses, bds
imgs = imgs = [imread(f)[..., :3] / 255.0 for f in imgfiles]
imgs = np.stack(imgs, -1)
logger.info(f"Loaded image data, shape {imgs.shape}")
return poses, bds, imgs
def normalize(x):
denom = np.linalg.norm(x)
if denom < 0.001:
warnings.warn("unsafe normalize()")
return x / denom
def viewmatrix(z, up, pos):
vec2 = normalize(z)
vec1_avg = up
vec0 = normalize(np.cross(vec1_avg, vec2))
vec1 = normalize(np.cross(vec2, vec0))
m = np.stack([vec0, vec1, vec2, pos], 1)
return m
def ptstocam(pts, c2w):
tt = np.matmul(c2w[:3, :3].T, (pts - c2w[:3, 3])[..., np.newaxis])[..., 0]
return tt
def poses_avg(poses):
hwf = poses[0, :3, -1:]
center = poses[:, :3, 3].mean(0)
vec2 = normalize(poses[:, :3, 2].sum(0))
up = poses[:, :3, 1].sum(0)
c2w = np.concatenate([viewmatrix(vec2, up, center), hwf], 1)
return c2w
def render_path_spiral(c2w, up, rads, focal, zdelta, zrate, rots, N):
render_poses = []
rads = np.array(list(rads) + [1.0])
hwf = c2w[:, 4:5]
for theta in np.linspace(0.0, 2.0 * np.pi * rots, N + 1)[:-1]:
c = np.dot(
c2w[:3, :4],
np.array([np.cos(theta), -np.sin(theta), -np.sin(theta * zrate), 1.0])
* rads,
)
z = normalize(c - np.dot(c2w[:3, :4], np.array([0, 0, -focal, 1.0])))
render_poses.append(np.concatenate([viewmatrix(z, up, c), hwf], 1))
return render_poses
def recenter_poses(poses):
poses_ = poses + 0
bottom = np.reshape([0, 0, 0, 1.0], [1, 4])
c2w = poses_avg(poses)
c2w = np.concatenate([c2w[:3, :4], bottom], -2)
bottom = np.tile(np.reshape(bottom, [1, 1, 4]), [poses.shape[0], 1, 1])
poses = np.concatenate([poses[:, :3, :4], bottom], -2)
poses = np.linalg.inv(c2w) @ poses
poses_[:, :3, :4] = poses[:, :3, :4]
poses = poses_
return poses
def spherify_poses(poses, bds):
def add_row_to_homogenize_transform(p):
r"""Add the last row to homogenize 3 x 4 transformation matrices."""
return np.concatenate(
[p, np.tile(np.reshape(np.eye(4)[-1, :], [1, 1, 4]), [p.shape[0], 1, 1])], 1
)
# p34_to_44 = lambda p: np.concatenate(
# [p, np.tile(np.reshape(np.eye(4)[-1, :], [1, 1, 4]), [p.shape[0], 1, 1])], 1
# )
p34_to_44 = add_row_to_homogenize_transform
rays_d = poses[:, :3, 2:3]
rays_o = poses[:, :3, 3:4]
def min_line_dist(rays_o, rays_d):
A_i = np.eye(3) - rays_d * np.transpose(rays_d, [0, 2, 1])
b_i = -A_i @ rays_o
pt_mindist = np.squeeze(
-np.linalg.inv((np.transpose(A_i, [0, 2, 1]) @ A_i).mean(0)) @ (b_i).mean(0)
)
return pt_mindist
pt_mindist = min_line_dist(rays_o, rays_d)
center = pt_mindist
up = (poses[:, :3, 3] - center).mean(0)
vec0 = normalize(up)
vec1 = normalize(np.cross([0.1, 0.2, 0.3], vec0))
vec2 = normalize(np.cross(vec0, vec1))
pos = center
c2w = np.stack([vec1, vec2, vec0, pos], 1)
poses_reset = np.linalg.inv(p34_to_44(c2w[None])) @ p34_to_44(poses[:, :3, :4])
rad = np.sqrt(np.mean(np.sum(np.square(poses_reset[:, :3, 3]), -1)))
sc = 1.0 / rad
poses_reset[:, :3, 3] *= sc
bds *= sc
rad *= sc
centroid = np.mean(poses_reset[:, :3, 3], 0)
zh = centroid[2]
radcircle = np.sqrt(rad**2 - zh**2)
new_poses = []
for th in np.linspace(0.0, 2.0 * np.pi, 120):
camorigin = np.array([radcircle * np.cos(th), radcircle * np.sin(th), zh])
up = np.array([0, 0, -1.0])
vec2 = normalize(camorigin)
vec0 = normalize(np.cross(vec2, up))
vec1 = normalize(np.cross(vec2, vec0))
pos = camorigin
p = np.stack([vec0, vec1, vec2, pos], 1)
new_poses.append(p)
new_poses = np.stack(new_poses, 0)
new_poses = np.concatenate(
[new_poses, np.broadcast_to(poses[0, :3, -1:], new_poses[:, :3, -1:].shape)], -1
)
poses_reset = np.concatenate(
[
poses_reset[:, :3, :4],
np.broadcast_to(poses[0, :3, -1:], poses_reset[:, :3, -1:].shape),
],
-1,
)
return poses_reset, new_poses, bds
def _local_path(path_manager, path):
if path_manager is None:
return path
return path_manager.get_local_path(path)
def _ls(path_manager, path):
if path_manager is None:
return os.listdir(path)
return path_manager.ls(path)
def _exists(path_manager, path):
if path_manager is None:
return os.path.exists(path)
return path_manager.exists(path)
def load_llff_data(
basedir,
factor=8,
recenter=True,
bd_factor=0.75,
spherify=False,
path_zflat=False,
path_manager=None,
):
poses, bds, imgs = _load_data(
basedir, factor=factor, path_manager=path_manager
) # factor=8 downsamples original imgs by 8x
logger.info(f"Loaded {basedir}, {bds.min()}, {bds.max()}")
# Correct rotation matrix ordering and move variable dim to axis 0
poses = np.concatenate([poses[:, 1:2, :], -poses[:, 0:1, :], poses[:, 2:, :]], 1)
poses = np.moveaxis(poses, -1, 0).astype(np.float32)
imgs = np.moveaxis(imgs, -1, 0).astype(np.float32)
images = imgs
bds = np.moveaxis(bds, -1, 0).astype(np.float32)
# Rescale if bd_factor is provided
sc = 1.0 if bd_factor is None else 1.0 / (bds.min() * bd_factor)
poses[:, :3, 3] *= sc
bds *= sc
if recenter:
poses = recenter_poses(poses)
if spherify:
poses, render_poses, bds = spherify_poses(poses, bds)
images = images.astype(np.float32)
poses = poses.astype(np.float32)
return images, poses, bds

View File

@@ -13,7 +13,6 @@ import struct
from typing import Optional, Tuple
import numpy as np
from pytorch3d.implicitron.dataset.types import (
DepthAnnotation,
ImageAnnotation,
@@ -22,7 +21,6 @@ from pytorch3d.implicitron.dataset.types import (
VideoAnnotation,
ViewpointAnnotation,
)
from sqlalchemy import LargeBinary
from sqlalchemy.orm import (
composite,
@@ -92,6 +90,7 @@ def TupleTypeFactory(dtype=float, shape: Tuple[int, ...] = (2,)):
impl = LargeBinary
_format = format_symbol * math.prod(shape)
# pyrefly: ignore [bad-override-param-name]
def process_bind_param(self, value, _):
if value is None:
return None
@@ -101,6 +100,7 @@ def TupleTypeFactory(dtype=float, shape: Tuple[int, ...] = (2,)):
return struct.pack(TupleType._format, *value)
# pyrefly: ignore [bad-override-param-name]
def process_result_value(self, value, _):
if value is None:
return None

View File

@@ -121,11 +121,17 @@ class RenderedMeshDatasetMapProvider(DatasetMapProviderBase):
self.poses = poses.cpu()
# pyre-ignore[16]
self.train_dataset = SingleSceneDataset( # pyre-ignore[28]
# pyrefly: ignore [unexpected-keyword]
object_name="cow",
# pyrefly: ignore [unexpected-keyword]
images=list(images.permute(0, 3, 1, 2).cpu()),
# pyrefly: ignore [unexpected-keyword]
fg_probabilities=list(masks[:, None].cpu()),
# pyrefly: ignore [unexpected-keyword]
poses=[self.poses[i] for i in range(len(poses))],
# pyrefly: ignore [unexpected-keyword]
frame_types=[DATASET_TYPE_KNOWN] * len(poses),
# pyrefly: ignore [unexpected-keyword]
eval_batches=None,
)

View File

@@ -85,7 +85,7 @@ class SingleSceneDataset(DatasetBase, Configurable):
class SingleSceneDatasetMapProviderBase(DatasetMapProviderBase):
"""
Base for provider of data for one scene from LLFF or blender datasets.
Base for provider of data for one scene.
Members:
base_dir: directory holding the data for the scene.
@@ -131,10 +131,11 @@ class SingleSceneDatasetMapProviderBase(DatasetMapProviderBase):
None
# pyre-ignore[16]
if self.fg_probabilities is None
else self.fg_probabilities[split]
else self.fg_probabilities[split] # pyrefly: ignore [bad-index]
)
eval_batches = [[i] for i in range(len(split))]
if split_idx != 0 and self.n_known_frames_for_test is not None:
# pyrefly: ignore [missing-attribute]
train_split = self.i_split[0]
if set_eval_batches:
generator = np.random.default_rng(seed=0)
@@ -171,40 +172,3 @@ class SingleSceneDatasetMapProviderBase(DatasetMapProviderBase):
# pyre-ignore[16]
cameras = [self.poses[i] for i in self.i_split[0]]
return join_cameras_as_batch(cameras)
def _interpret_blender_cameras(
poses: torch.Tensor, focal: float
) -> List[PerspectiveCameras]:
"""
Convert 4x4 matrices representing cameras in blender format
to PyTorch3D format.
Args:
poses: N x 3 x 4 camera matrices
focal: ndc space focal length
"""
pose_target_cameras = []
for pose_target in poses:
pose_target = pose_target[:3, :4]
mtx = torch.eye(4, dtype=pose_target.dtype)
mtx[:3, :3] = pose_target[:3, :3].t()
mtx[3, :3] = pose_target[:, 3]
mtx = mtx.inverse()
# flip the XZ coordinates.
mtx[:, [0, 2]] *= -1.0
Rpt3, Tpt3 = mtx[:, :3].split([3, 1], dim=0)
focal_length_pt3 = torch.FloatTensor([[focal, focal]])
principal_point_pt3 = torch.FloatTensor([[0.0, 0.0]])
cameras = PerspectiveCameras(
focal_length=focal_length_pt3,
principal_point=principal_point_pt3,
R=Rpt3[None],
T=Tpt3,
)
pose_target_cameras.append(cameras)
return pose_target_cameras

View File

@@ -10,7 +10,6 @@ import hashlib
import json
import logging
import os
import urllib
from dataclasses import dataclass, Field, field
from typing import (
@@ -32,13 +31,11 @@ import pandas as pd
import sqlalchemy as sa
import torch
from pytorch3d.implicitron.dataset.dataset_base import DatasetBase
from pytorch3d.implicitron.dataset.frame_data import (
FrameData,
FrameDataBuilder, # noqa
FrameDataBuilderBase,
)
from pytorch3d.implicitron.tools.config import (
registry,
ReplaceableBase,
@@ -224,6 +221,7 @@ class SqlIndexDataset(DatasetBase, ReplaceableBase):
def __len__(self) -> int:
return len(self._index)
# pyrefly: ignore [bad-override-param-name]
def __getitem__(self, frame_idx: Union[int, Tuple[str, int]]) -> FrameData:
"""
Fetches FrameData by either iloc in the index or by (sequence, frame_no) pair
@@ -427,6 +425,7 @@ class SqlIndexDataset(DatasetBase, ReplaceableBase):
# override
@property
# pyrefly: ignore [bad-override]
def frame_data_type(self) -> Type[FrameData]:
return self.frame_data_builder.frame_data_type
@@ -486,9 +485,10 @@ class SqlIndexDataset(DatasetBase, ReplaceableBase):
*self._get_pick_filters(),
*self._get_exclude_filters(),
]
if self.pick_sequences_sql_clause:
if pick_sequences_sql_clause := self.pick_sequences_sql_clause:
print("Applying the custom SQL clause.")
where_conditions.append(sa.text(self.pick_sequences_sql_clause))
# pyre-ignore[6]: TextClause is compatible with where conditions
where_conditions.append(sa.text(pick_sequences_sql_clause))
def add_where(stmt):
return stmt.where(*where_conditions) if where_conditions else stmt
@@ -508,6 +508,7 @@ class SqlIndexDataset(DatasetBase, ReplaceableBase):
subquery = add_where(subquery).subquery()
stmt = sa.select(subquery.c.sequence_name).where(
# pyre-ignore[6]: SQLAlchemy column comparison returns ColumnElement, not bool
subquery.c.row_number <= self.limit_sequences_per_category_to
)
@@ -631,14 +632,17 @@ class SqlIndexDataset(DatasetBase, ReplaceableBase):
# dev load: 94 s / 23 s (3.1M / 500K)
pick_frames_criteria.append(
sa.or_(
# pyrefly: ignore [missing-attribute]
self.frame_annotations_type._mask_mass.is_(None),
# pyrefly: ignore [missing-attribute]
self.frame_annotations_type._mask_mass != 0,
)
)
if self.pick_frames_sql_clause:
if pick_frames_sql_clause := self.pick_frames_sql_clause:
logger.info("Applying the custom SQL clause.")
pick_frames_criteria.append(sa.text(self.pick_frames_sql_clause))
# pyre-ignore[6]: TextClause is compatible with where conditions
pick_frames_criteria.append(sa.text(pick_frames_sql_clause))
if pick_frames_criteria:
index = self._pick_frames_by_criteria(index, pick_frames_criteria)
@@ -695,15 +699,18 @@ class SqlIndexDataset(DatasetBase, ReplaceableBase):
if self.remove_empty_masks:
logger.info(" excluding samples with empty masks")
where_conditions.append(
# pyrefly: ignore [bad-argument-type]
sa.or_(
self.frame_annotations_type._mask_mass.is_(None), # pyre-ignore[16]
# pyrefly: ignore [missing-attribute]
self.frame_annotations_type._mask_mass != 0,
)
)
if self.pick_frames_sql_clause:
if pick_frames_sql_clause := self.pick_frames_sql_clause:
logger.info(" applying custom SQL clause")
where_conditions.append(sa.text(self.pick_frames_sql_clause))
# pyre-ignore[6]: TextClause is compatible with where conditions
where_conditions.append(sa.text(pick_frames_sql_clause))
if where_conditions:
stmt = stmt.where(*where_conditions)

View File

@@ -12,9 +12,7 @@ import os
from typing import List, Optional, Tuple, Type
import numpy as np
from omegaconf import DictConfig, OmegaConf
from pytorch3d.implicitron.dataset.dataset_map_provider import (
DatasetMap,
DatasetMapProviderBase,

View File

@@ -18,7 +18,6 @@ from pytorch3d.implicitron.dataset.dataset_base import DatasetBase
from pytorch3d.implicitron.dataset.dataset_map_provider import DatasetMap
from pytorch3d.implicitron.dataset.frame_data import FrameData
from pytorch3d.implicitron.tools.config import registry, run_auto_creation
from torch.utils.data import DataLoader
logger = logging.getLogger(__name__)

View File

@@ -162,6 +162,7 @@ def load_dataclass(f: IO, cls: Type[_X], binary: bool = False) -> _X:
else:
res = _dataclass_from_dict(asdict, cls)
# pyrefly: ignore [bad-return]
return res

View File

@@ -15,7 +15,6 @@ from typing import List, Optional, Tuple, TypeVar, Union
import numpy as np
import torch
from PIL import Image
from pytorch3d.io import IO
from pytorch3d.renderer.cameras import PerspectiveCameras
from pytorch3d.structures.pointclouds import Pointclouds
@@ -118,6 +117,7 @@ def crop_around_box(
# bbox is xyxy, where the upper bound is corrected with +1
bbox = clamp_box_to_image_bounds_and_round(
bbox,
# pyrefly: ignore [bad-argument-type]
image_size_hw=tuple(tensor.shape[-2:]),
)
tensor = tensor[..., bbox[1] : bbox[3], bbox[0] : bbox[2]]

View File

@@ -312,6 +312,7 @@ def eval_batch(
)
if visualize:
# pyrefly: ignore [unbound-name]
visualizer.show_rgb(
results[metric_name].item(), metric_name, loss_mask_now
)
@@ -330,6 +331,7 @@ def eval_batch(
results["depth_abs" + name_postfix] = abs_.mean()
if visualize:
# pyrefly: ignore [unbound-name]
visualizer.show_depth(abs_.mean().item(), name_postfix, loss_mask_now)
if break_after_visualising:
breakpoint() # noqa: B601

View File

@@ -14,7 +14,6 @@ import warnings
from typing import Any, Dict, List, Optional, Tuple
import torch
import tqdm
from pytorch3d.implicitron.evaluation import evaluate_new_view_synthesis as evaluate
from pytorch3d.implicitron.models.base_model import EvaluationMode, ImplicitronModelBase

View File

@@ -10,7 +10,6 @@ from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional
import torch
from pytorch3d.implicitron.models.renderer.base import EvaluationMode
from pytorch3d.implicitron.tools.config import ReplaceableBase
from pytorch3d.renderer.cameras import CamerasBase

View File

@@ -16,7 +16,6 @@ from typing import Any, Dict, List, Optional, Tuple, TYPE_CHECKING, Union
import torch
from omegaconf import DictConfig
from pytorch3d.implicitron.models.base_model import (
ImplicitronModelBase,
ImplicitronRender,
@@ -28,7 +27,6 @@ from pytorch3d.implicitron.models.metrics import (
RegularizationMetricsBase,
ViewMetricsBase,
)
from pytorch3d.implicitron.models.renderer.base import (
BaseRenderer,
EvaluationMode,
@@ -38,7 +36,6 @@ from pytorch3d.implicitron.models.renderer.base import (
RenderSamplingMode,
)
from pytorch3d.implicitron.models.renderer.ray_sampler import RaySamplerBase
from pytorch3d.implicitron.models.utils import (
apply_chunked,
chunk_generator,
@@ -53,7 +50,6 @@ from pytorch3d.implicitron.tools.config import (
registry,
run_auto_creation,
)
from pytorch3d.implicitron.tools.rasterize_mc import rasterize_sparse_ray_bundle
from pytorch3d.renderer import utils as rend_utils
from pytorch3d.renderer.cameras import CamerasBase
@@ -476,6 +472,7 @@ class GenericModel(ImplicitronModelBase):
sequence_name=safe_slice_targets(sequence_name),
frame_timestamp=safe_slice_targets(frame_timestamp),
)
# pyrefly: ignore [unsupported-operation]
custom_args["global_code"] = global_code
# pyre-fixme[29]: `Union[(self: Tensor) -> Any, Tensor, Module]` is not a

View File

@@ -10,7 +10,6 @@ from abc import ABC, abstractmethod
from typing import Optional
from pytorch3d.implicitron.models.renderer.base import ImplicitronRayBundle
from pytorch3d.implicitron.tools.config import ReplaceableBase
from pytorch3d.renderer.cameras import CamerasBase

View File

@@ -16,14 +16,11 @@ This file contains
import logging
from dataclasses import field
from enum import Enum
from typing import Dict, Optional, Tuple
import torch
from omegaconf import DictConfig
from pytorch3d.implicitron.tools.config import (
Configurable,
registry,

View File

@@ -11,7 +11,6 @@ import torch
from pytorch3d.implicitron.models.renderer.base import ImplicitronRayBundle
from pytorch3d.implicitron.tools.config import registry
from pytorch3d.renderer.implicit import HarmonicEmbedding
from torch import nn
from .base import ImplicitFunctionBase

View File

@@ -21,7 +21,6 @@ from pytorch3d.renderer.implicit import HarmonicEmbedding
from pytorch3d.renderer.implicit.utils import ray_bundle_to_ray_points
from .base import ImplicitFunctionBase
from .decoding_functions import ( # noqa
_xavier_init,
MLPWithInputSkips,

View File

@@ -9,7 +9,6 @@
from typing import Callable, Optional
import torch
import torch.nn.functional as F
from pytorch3d.common.compat import prod
from pytorch3d.implicitron.models.renderer.base import ImplicitronRayBundle

View File

@@ -907,6 +907,7 @@ class VoxelGridModule(Configurable, torch.nn.Module):
else:
# Torch Module to hold parameters since they can only be registered
# at object level.
# pyrefly: ignore [bad-assignment]
self.params = _RegistratedBufferDict(vars(params))
@staticmethod

View File

@@ -13,9 +13,7 @@ from dataclasses import fields
from typing import Callable, Dict, Optional, Tuple
import torch
from omegaconf import DictConfig
from pytorch3d.implicitron.models.implicit_function.base import ImplicitFunctionBase
from pytorch3d.implicitron.models.implicit_function.decoding_functions import (
DecoderFunctionBase,
@@ -478,6 +476,7 @@ class VoxelGridImplicitFunction(ImplicitFunctionBase, torch.nn.Module):
call_epochs = list(
set(self.scaffold_calculating_epochs) | set(self.volume_cropping_epochs)
)
# pyrefly: ignore [bad-return]
return call_epochs, callback
def _crop(self, epoch: int) -> bool:
@@ -583,6 +582,7 @@ class VoxelGridImplicitFunction(ImplicitFunctionBase, torch.nn.Module):
)
cls = registry.get(DecoderFunctionBase, type_)
# pyrefly: ignore [bad-argument-type]
need_input_dim = any(field.name == "input_dim" for field in fields(cls))
if need_input_dim:
self.decoder_density = cls(input_dim=input_dim, **args)
@@ -623,6 +623,7 @@ class VoxelGridImplicitFunction(ImplicitFunctionBase, torch.nn.Module):
input_dim = input_dim0 + input_dim1
cls = registry.get(DecoderFunctionBase, type_)
# pyrefly: ignore [bad-argument-type]
need_input_dim = any(field.name == "input_dim" for field in fields(cls))
if need_input_dim:
self.decoder_color = cls(input_dim=input_dim, **args)

View File

@@ -17,7 +17,6 @@ from typing import Any, Callable, Dict, List, Optional, Tuple, TYPE_CHECKING, Un
import torch
from omegaconf import DictConfig
from pytorch3d.implicitron.models.base_model import (
ImplicitronModelBase,
ImplicitronRender,
@@ -28,7 +27,6 @@ from pytorch3d.implicitron.models.metrics import (
RegularizationMetricsBase,
ViewMetricsBase,
)
from pytorch3d.implicitron.models.renderer.base import (
BaseRenderer,
EvaluationMode,
@@ -50,7 +48,6 @@ from pytorch3d.implicitron.tools.config import (
registry,
run_auto_creation,
)
from pytorch3d.implicitron.tools.rasterize_mc import rasterize_sparse_ray_bundle
from pytorch3d.renderer import utils as rend_utils
from pytorch3d.renderer.cameras import CamerasBase

View File

@@ -110,6 +110,7 @@ class ImplicitronRayBundle:
# equivalent to: 0.5 * (bins[..., 1:] + bins[..., :-1]) but more efficient
# pyre-ignore
return torch.lerp(self.bins[..., :-1], self.bins[..., 1:], 0.5)
# pyrefly: ignore [bad-return]
return self._lengths
@lengths.setter
@@ -166,6 +167,7 @@ class ImplicitronRayBundle:
)
num_inputs = camera_counts.sum().item()
max_size = torch.max(camera_counts).item()
# pyrefly: ignore [bad-argument-type]
xys = packed_to_padded(self.xys, first_idxs, max_size)
# pyre-ignore [7] pytorch typeshed inaccuracy
return xys, first_idxs, num_inputs

View File

@@ -11,7 +11,6 @@ import copy
import torch
from pytorch3d.implicitron.models.renderer.base import ImplicitronRayBundle
from pytorch3d.implicitron.tools.config import Configurable, expand_args_fields
from pytorch3d.renderer.implicit.sample_pdf import sample_pdf

View File

@@ -198,6 +198,7 @@ class AccumulativeRaymarcherBase(RaymarcherBase, torch.nn.Module):
depth = (weights * ray_lengths)[..., None].sum(dim=-2)
alpha = opacities if self.blend_output else 1
# pyrefly: ignore [bad-index]
if self._bg_color.shape[-1] not in [1, features.shape[-1]]:
raise ValueError("Wrong number of background color channels.")
# pyre-fixme[58]: `*` is not supported for operand types `int` and

View File

@@ -12,7 +12,6 @@ import torch
from pytorch3d.implicitron.models.renderer.base import ImplicitronRayBundle
from pytorch3d.implicitron.tools.config import enable_get_default_args
from pytorch3d.renderer.implicit import HarmonicEmbedding
from torch import nn

View File

@@ -17,11 +17,8 @@ from typing import Any, Dict, Optional, Tuple
import torch
import tqdm
from pytorch3d.common.compat import prod
from pytorch3d.implicitron.models.renderer.base import ImplicitronRayBundle
from pytorch3d.implicitron.tools import image_utils
from pytorch3d.implicitron.tools.utils import cat_dataclass
@@ -83,9 +80,9 @@ def preprocess_input(
if mask_depths and fg_mask is not None and depth_map is not None:
# mask the depths
assert (
mask_threshold > 0.0
), "Depths should be masked only with thresholded masks"
assert mask_threshold > 0.0, (
"Depths should be masked only with thresholded masks"
)
warnings.warn("Masking depths!")
depth_map = depth_map * fg_mask
@@ -121,6 +118,7 @@ def weighted_sum_losses(
return None
loss = sum(losses_weighted)
assert torch.is_tensor(loss)
# pyrefly: ignore [bad-return]
return loss

View File

@@ -231,7 +231,9 @@ class Configurable:
return obj
# pyrefly: ignore [invalid-type-var]
_X = TypeVar("X", bound=ReplaceableBase)
# pyrefly: ignore [invalid-type-var]
_Y = TypeVar("Y", bound=Union[ReplaceableBase, Configurable])
@@ -623,6 +625,7 @@ def _field_annotations_for_default_args(
field_ = dataclasses.field(default=default)
field_annotations.append((pname, defval.annotation, field_))
# pyrefly: ignore [bad-return]
return field_annotations
@@ -890,20 +893,27 @@ def expand_args_fields(
continue
expand_args_fields(base, _do_not_process=_do_not_process)
if "_creation_functions" in base.__dict__:
# pyrefly: ignore [missing-attribute]
creation_functions.extend(base._creation_functions)
if "_known_implementations" in base.__dict__:
# pyrefly: ignore [missing-attribute]
known_implementations.update(base._known_implementations)
if "_processed_members" in base.__dict__:
# pyrefly: ignore [missing-attribute]
processed_members.update(base._processed_members)
to_process: List[Tuple[str, Type, _ProcessType]] = []
if "__annotations__" in some_class.__dict__:
for name, type_ in some_class.__annotations__.items():
underlying_and_process_type = _get_type_to_process(type_)
if underlying_and_process_type is None:
continue
underlying_type, process_type = underlying_and_process_type
to_process.append((name, underlying_type, process_type))
# Only this class's own annotations, never a base's. Reading
# some_class.__dict__["__annotations__"] used to express that, but under
# PEP 649 the entry is not in the class dict until something materializes
# it, so on 3.14 the lookup silently found nothing and no member was
# processed at all.
for name, type_ in inspect.get_annotations(some_class).items():
underlying_and_process_type = _get_type_to_process(type_)
if underlying_and_process_type is None:
continue
underlying_type, process_type = underlying_and_process_type
to_process.append((name, underlying_type, process_type))
for name, underlying_type, process_type in to_process:
processed_members[name] = some_class.__annotations__[name]

View File

@@ -10,7 +10,6 @@ import math
from typing import Optional, Tuple
import pytorch3d
import torch
from pytorch3d.ops import packed_to_padded
from pytorch3d.renderer import PerspectiveCameras
@@ -63,9 +62,14 @@ def rasterize_sparse_ray_bundle(
max_size = torch.max(camera_counts).item()
features_depth_ras = packed_to_padded(
features_depth_ras[:, 0], first_idxs, max_size
# pyrefly: ignore [bad-argument-type]
features_depth_ras[:, 0],
first_idxs,
# pyrefly: ignore [bad-argument-type]
max_size,
)
if masks is not None:
# pyrefly: ignore [bad-argument-type]
padded_mask = packed_to_padded(masks.flatten(1, -1), first_idxs, max_size)
masks_ras = padded_mask * masks_ras

View File

@@ -499,7 +499,7 @@ class StatsJSONEncoder(json.JSONEncoder):
return enc
else:
raise TypeError(
f"Object of type {o.__class__.__name__} " f"is not JSON serializable"
f"Object of type {o.__class__.__name__} is not JSON serializable"
)

View File

@@ -17,7 +17,6 @@ import matplotlib
import matplotlib.pyplot as plt
import numpy as np
import torch
from PIL import Image
_NO_TORCHVISION = False
@@ -114,6 +113,7 @@ class VideoWriter:
elif isinstance(frame, np.ndarray):
if frame.dtype in (np.float64, np.float32, float):
frame = (np.transpose(frame, (1, 2, 0)) * 255.0).astype(np.uint8)
# pyrefly: ignore [bad-argument-type]
im = Image.fromarray(frame)
elif isinstance(frame, Image.Image):
im = frame
@@ -125,13 +125,16 @@ class VideoWriter:
if im is not None:
if resize is not None:
if isinstance(resize, float):
# pyrefly: ignore [bad-assignment]
resize = [int(resize * s) for s in im.size]
else:
resize = im.size
# make sure size is divisible by 2
# pyrefly: ignore [bad-assignment, bad-index, unsupported-operation]
resize = tuple([resize[i] + resize[i] % 2 for i in (0, 1)])
im = im.resize(resize, Image.ANTIALIAS)
# pyrefly: ignore [bad-argument-type]
im = im.resize(resize, Image.Resampling.LANCZOS)
im.save(outfile)
self.frames.append(outfile)

View File

@@ -120,6 +120,7 @@ class _TargetType(IntEnum):
class OurEncoder(json.JSONEncoder):
# pyrefly: ignore [bad-override-param-name]
def default(self, obj):
if isinstance(obj, np.int64):
return str(obj)
@@ -242,6 +243,7 @@ class _GLTFLoader:
by _get_texture_map_image which caches it.
"""
# pyrefly: ignore [unsupported-operation]
image_json = self._json_data["images"][image_index]
buffer_view = self._buffer_views[image_json["bufferView"]]
if "byteStride" in buffer_view:
@@ -407,10 +409,12 @@ class _GLTFLoader:
verts_uvs[:, 1] = 1 - verts_uvs[:, -1]
faces_uvs = indices
material_index = primitive.get("material", 0)
# pyrefly: ignore [unsupported-operation]
material = self._json_data["materials"][material_index]
material_roughness = material["pbrMetallicRoughness"]
if "baseColorTexture" in material_roughness:
texture_index = material_roughness["baseColorTexture"]["index"]
# pyrefly: ignore [unsupported-operation]
texture_json = self._json_data["textures"][texture_index]
# Todo - include baseColorFactor when also given
# Todo - look at the sampler
@@ -555,6 +559,7 @@ class _GLTFWriter:
# pyre-fixme[6]: Incompatible parameter type
self._json_data["scene"] = scene_index
self._json_data["scenes"].append({"nodes": [scene_index]})
# pyrefly: ignore [unsupported-operation]
self._json_data["asset"] = {"version": "2.0"}
node = {"name": "Node", "mesh": 0}
self._json_data["nodes"].append(node)
@@ -621,6 +626,7 @@ class _GLTFWriter:
byte_per_element = 3 * _DTYPE_BYTES[_ITEM_TYPES[_ComponentType.FLOAT]]
elif key == "texcoords":
component_type = _ComponentType.FLOAT
# pyrefly: ignore [missing-attribute]
data = self.mesh.textures.verts_uvs_list()[0].cpu().numpy()
data[:, 1] = 1 - data[:, -1] # flip y tex-coordinate
element_type = "VEC2"
@@ -630,6 +636,7 @@ class _GLTFWriter:
byte_per_element = 2 * _DTYPE_BYTES[_ITEM_TYPES[_ComponentType.FLOAT]]
elif key == "texvertices":
component_type = _ComponentType.FLOAT
# pyrefly: ignore [missing-attribute]
data = self.mesh.textures.verts_features_list()[0].cpu().numpy()
element_type = "VEC3"
buffer_view = 2
@@ -700,11 +707,14 @@ class _GLTFWriter:
target = _TargetType.ELEMENT_ARRAY_BUFFER
bufferview["target"] = target
# pyrefly: ignore [bad-assignment]
bufferview["byteOffset"] = kwargs.get("offset")
# pyrefly: ignore [bad-assignment]
bufferview["byteLength"] = kwargs.get("byte_length")
self._json_data["bufferViews"].append(bufferview)
def _write_image_buffer(self, **kwargs) -> Tuple[int, bytes]:
# pyrefly: ignore [missing-attribute]
image_np = self.mesh.textures.maps_list()[0].cpu().numpy()
image_array = (image_np * 255.0).astype(np.uint8)
im = Image.fromarray(image_array)
@@ -716,6 +726,7 @@ class _GLTFWriter:
bufferview_image = {
"buffer": 0,
}
# pyrefly: ignore [bad-assignment]
bufferview_image["byteOffset"] = kwargs.get("offset")
bufferview_image["byteLength"] = image_data_byte_length
self._json_data["bufferViews"].append(bufferview_image)

View File

@@ -226,6 +226,7 @@ def load_obj(
with _open_file(f, path_manager, "r") as f:
return _load_obj(
f,
# pyrefly: ignore [bad-argument-type]
data_dir=data_dir,
load_textures=load_textures,
create_texture_atlas=create_texture_atlas,
@@ -641,6 +642,7 @@ def _load_obj(
material_names.append(next(iter(material_colors.keys())))
# replace all -1 by 0 material idx
if torch.is_tensor(faces_materials_idx):
# pyrefly: ignore [missing-attribute]
faces_materials_idx.clamp_(min=0)
if create_texture_atlas:
@@ -649,14 +651,18 @@ def _load_obj(
# Create an array of strings of material names for each face.
# If faces_materials_idx == -1 then that face doesn't have a material.
# pyrefly: ignore [missing-attribute]
idx = faces_materials_idx.cpu().numpy()
face_material_names = np.array([""] + material_names)[idx + 1] # (F,)
# Construct the atlas.
texture_atlas = make_mesh_texture_atlas(
# pyrefly: ignore [bad-argument-type]
material_colors,
# pyrefly: ignore [bad-argument-type]
texture_images,
face_material_names,
# pyrefly: ignore [bad-argument-type]
faces_textures_idx,
verts_uvs,
texture_atlas_size,
@@ -796,7 +802,7 @@ def save_obj(
# Create .mtl file with the material name and texture map filename
# TODO: enable material properties to also be saved.
with _open_file(mtl_path, path_manager, "w") as f_mtl:
lines = f"newmtl mesh\n" f"map_Kd {output_path.stem}.png\n"
lines = f"newmtl mesh\nmap_Kd {output_path.stem}.png\n"
f_mtl.write(lines)

View File

@@ -587,6 +587,7 @@ def _read_ply_element_binary_nolists(f, definition: _PlyElementType, big_endian:
# piece = data[:, offset:end_offset].view(_PLY_TYPES[dtype].np_type)
# but it fails in the general case
# because of https://github.com/numpy/numpy/issues/9496.
# pyre-fixme[16]: Module `stride_tricks` has no attribute `as_strided`.
piece = np.lib.stride_tricks.as_strided(
data[:1, offset:end_offset].view(_PLY_TYPES[dtype].np_type),
shape=(definition.count, count),
@@ -877,10 +878,14 @@ def _get_verts_column_indices(
):
color_scale = 1.0 / 255
return _VertsColumnIndices(
# pyrefly: ignore [bad-argument-type]
point_idxs=point_idxs,
# pyrefly: ignore [bad-argument-type]
color_idxs=None if None in color_idxs else color_idxs,
color_scale=color_scale,
# pyrefly: ignore [bad-argument-type]
normal_idxs=None if None in normal_idxs else normal_idxs,
# pyrefly: ignore [bad-argument-type]
texture_uv_idxs=None if None in texture_uv_idxs else texture_uv_idxs,
)

View File

@@ -8,11 +8,8 @@
from .chamfer import chamfer_distance
from .mesh_edge_loss import mesh_edge_loss
from .mesh_laplacian_smoothing import mesh_laplacian_smoothing
from .mesh_normal_consistency import mesh_normal_consistency
from .point_mesh_distance import point_mesh_edge_distance, point_mesh_face_distance

View File

@@ -114,9 +114,7 @@ def mesh_laplacian_smoothing(meshes, method: str = "uniform"):
if method == "cot":
norm_w = torch.sparse.sum(L, dim=1).to_dense().view(-1, 1)
idx = norm_w > 0
# pyre-fixme[58]: `/` is not supported for operand types `float` and
# `Tensor`.
norm_w[idx] = 1.0 / norm_w[idx]
norm_w[idx] = torch.reciprocal(norm_w[idx])
else:
L_sum = torch.sparse.sum(L, dim=1).to_dense().view(-1, 1)
norm_w = 0.25 * inv_areas

View File

@@ -6,6 +6,7 @@
# pyre-unsafe
import torch
from pytorch3d import _C
from pytorch3d.structures import Meshes, Pointclouds
from torch.autograd import Function
@@ -80,6 +81,7 @@ class _PointFaceDistance(Function):
@staticmethod
@once_differentiable
# pyrefly: ignore [bad-override]
def backward(ctx, grad_dists):
grad_dists = grad_dists.contiguous()
points, tris, idxs = ctx.saved_tensors
@@ -142,6 +144,7 @@ class _FacePointDistance(Function):
@staticmethod
@once_differentiable
# pyrefly: ignore [bad-override]
def backward(ctx, grad_dists):
grad_dists = grad_dists.contiguous()
points, tris, idxs = ctx.saved_tensors
@@ -193,6 +196,7 @@ class _PointEdgeDistance(Function):
@staticmethod
@once_differentiable
# pyrefly: ignore [bad-override]
def backward(ctx, grad_dists):
grad_dists = grad_dists.contiguous()
points, segms, idxs = ctx.saved_tensors
@@ -243,6 +247,7 @@ class _EdgePointDistance(Function):
@staticmethod
@once_differentiable
# pyrefly: ignore [bad-override]
def backward(ctx, grad_dists):
grad_dists = grad_dists.contiguous()
points, segms, idxs = ctx.saved_tensors
@@ -302,8 +307,7 @@ def point_mesh_edge_distance(meshes: Meshes, pcls: Pointclouds):
point_to_cloud_idx = pcls.packed_to_cloud_idx() # (sum(P_i), )
num_points_per_cloud = pcls.num_points_per_cloud() # (N,)
weights_p = num_points_per_cloud.gather(0, point_to_cloud_idx)
# pyre-fixme[58]: `/` is not supported for operand types `float` and `Tensor`.
weights_p = 1.0 / weights_p.float()
weights_p = torch.reciprocal(weights_p.float())
point_to_edge = point_to_edge * weights_p
point_dist = point_to_edge.sum() / N
@@ -377,8 +381,7 @@ def point_mesh_face_distance(
point_to_cloud_idx = pcls.packed_to_cloud_idx() # (sum(P_i),)
num_points_per_cloud = pcls.num_points_per_cloud() # (N,)
weights_p = num_points_per_cloud.gather(0, point_to_cloud_idx)
# pyre-fixme[58]: `/` is not supported for operand types `float` and `Tensor`.
weights_p = 1.0 / weights_p.float()
weights_p = torch.reciprocal(weights_p.float())
point_to_face = point_to_face * weights_p
point_dist = point_to_face.sum() / N

View File

@@ -8,17 +8,14 @@
from .ball_query import ball_query
from .cameras_alignment import corresponding_cameras_alignment
from .cubify import cubify
from .graph_conv import GraphConv
from .interp_face_attrs import interpolate_face_attributes
from .iou_box3d import box3d_overlap
from .knn import knn_gather, knn_points
from .laplacian_matrices import cot_laplacian, laplacian, norm_laplacian
from .mesh_face_areas_normals import mesh_face_areas_normals
from .mesh_filtering import taubin_smoothing
from .packed_to_padded import packed_to_padded, padded_to_packed
from .perspective_n_points import efficient_pnp
from .points_alignment import corresponding_points_alignment, iterative_closest_point
@@ -30,9 +27,7 @@ from .points_to_volumes import (
add_pointclouds_to_volumes,
add_points_features_to_volume_densities_features,
)
from .sample_farthest_points import sample_farthest_points
from .sample_points_from_meshes import sample_points_from_meshes
from .subdivide_meshes import SubdivideMeshes
from .utils import (
@@ -42,7 +37,6 @@ from .utils import (
is_pointclouds,
wmean,
)
from .vert_align import vert_align

View File

@@ -36,6 +36,7 @@ class _ball_query(Function):
@staticmethod
@once_differentiable
# pyrefly: ignore [bad-override]
def backward(ctx, grad_dists, grad_idx):
p1, p2, lengths1, lengths2, idx = ctx.saved_tensors
# TODO(gkioxari) Change cast to floats once we add support for doubles.

View File

@@ -181,6 +181,8 @@ def _align_camera_extrinsics(
```
"""
# pyre-fixme[6]: For 1st argument expected `Tensor` but got `Union[Tensor, Module]`.
# pyre-fixme[29]: `Union[(self: TensorBase, dim0: int, dim1: int) -> Tensor,
# Tensor, Module]` is not a function.
RRcov = torch.bmm(cameras_src.R, cameras_tgt.R.transpose(2, 1)).mean(0)
U, _, V = torch.svd(RRcov)
align_t_R = V @ U.t()
@@ -223,8 +225,7 @@ def _align_camera_extrinsics(
# of centered A and centered B
Ac = A - Amu
Bc = B - Bmu
# pyre-fixme[58]: `**` is not supported for operand types `Tensor` and `int`.
align_t_s = (Ac * Bc).mean() / (Ac**2).mean().clamp(eps)
align_t_s = (Ac * Bc).mean() / torch.square(Ac).mean().clamp(eps)
else:
# set the scale to identity
align_t_s = 1.0

View File

@@ -11,9 +11,7 @@ from typing import Optional
import torch
import torch.nn.functional as F
from pytorch3d.common.compat import meshgrid_ij
from pytorch3d.structures import Meshes

View File

@@ -114,6 +114,10 @@ def gather_scatter_python(input, edges, directed: bool = False):
raise ValueError("edges can only have 2 dimensions.")
if not (edges.shape[1] == 2):
raise ValueError("edges must be of shape (num_edges, 2).")
if not (input.dtype == torch.float32):
raise ValueError("input has to be of type torch.float32.")
if not (edges.dtype == torch.int64):
raise ValueError("edges has to be of type torch.int64.")
num_vertices, input_feature_dim = input.shape
num_edges = edges.shape[0]
@@ -152,6 +156,8 @@ class GatherScatter(Function):
raise ValueError("edges must be of shape (num_edges, 2).")
if not (input.dtype == torch.float32):
raise ValueError("input has to be of type torch.float32.")
if not (edges.dtype == torch.int64):
raise ValueError("edges has to be of type torch.int64.")
ctx.directed = directed
input, edges = input.contiguous(), edges.contiguous()
@@ -162,6 +168,7 @@ class GatherScatter(Function):
@staticmethod
@once_differentiable
# pyrefly: ignore [bad-override]
def backward(ctx, grad_output):
grad_output = grad_output.contiguous()
edges = ctx.saved_tensors[0]

View File

@@ -72,6 +72,7 @@ class _InterpFaceAttrs(Function):
@staticmethod
@once_differentiable
# pyrefly: ignore [bad-override]
def backward(ctx, grad_pix_attrs):
args = ctx.saved_tensors
args = args + (grad_pix_attrs,)

View File

@@ -106,6 +106,7 @@ class _box3d_overlap(Function):
return vol, iou
@staticmethod
# pyrefly: ignore [bad-override]
def backward(ctx, grad_vol, grad_iou):
raise ValueError("box3d_overlap backward is not supported")

View File

@@ -95,6 +95,7 @@ class _knn_points(Function):
@staticmethod
@once_differentiable
# pyrefly: ignore [bad-override]
def backward(ctx, grad_dists, grad_idx):
p1, p2, lengths1, lengths2, idx = ctx.saved_tensors
norm = ctx.norm

View File

@@ -55,11 +55,9 @@ def laplacian(verts: torch.Tensor, edges: torch.Tensor) -> torch.Tensor:
# We construct the Laplacian matrix by adding the non diagonal values
# i.e. L[i, j] = 1 ./ deg(i) if (i, j) is an edge
deg0 = deg[e0]
# pyre-fixme[58]: `/` is not supported for operand types `float` and `Tensor`.
deg0 = torch.where(deg0 > 0.0, 1.0 / deg0, deg0)
deg0 = torch.where(deg0 > 0.0, torch.reciprocal(deg0), deg0)
deg1 = deg[e1]
# pyre-fixme[58]: `/` is not supported for operand types `float` and `Tensor`.
deg1 = torch.where(deg1 > 0.0, 1.0 / deg1, deg1)
deg1 = torch.where(deg1 > 0.0, torch.reciprocal(deg1), deg1)
val = torch.cat([deg0, deg1])
L = torch.sparse_coo_tensor(idx, val, (V, V), dtype=torch.float32)
@@ -105,8 +103,8 @@ def cot_laplacian(
s = 0.5 * (A + B + C)
# note that the area can be negative (close to 0) causing nans after sqrt()
# we clip it to a small positive value
# pyre-fixme[16]: `float` has no attribute `clamp_`.
area = (s * (s - A) * (s - B) * (s - C)).clamp_(min=eps).sqrt()
# pyre-fixme[16]: `float` has no attribute `clamp`.
area = (s * (s - A) * (s - B) * (s - C)).clamp(min=eps).sqrt()
# Compute cotangents of angles, of shape (sum(F_n), 3)
A2, B2, C2 = A * A, B * B, C * C
@@ -114,7 +112,7 @@ def cot_laplacian(
cotb = (A2 + C2 - B2) / area
cotc = (A2 + B2 - C2) / area
cot = torch.stack([cota, cotb, cotc], dim=1)
cot /= 4.0
cot = cot / 4.0
# Construct a sparse matrix by basically doing:
# L[v1, v2] = cota
@@ -129,7 +127,7 @@ def cot_laplacian(
# L[v2, v1] = cota
# L[v0, v2] = cotb
# L[v1, v0] = cotc
L += L.t()
L = L + L.t()
# For each vertex, compute the sum of areas for triangles containing it.
idx = faces.view(-1)
@@ -137,8 +135,7 @@ def cot_laplacian(
val = torch.stack([area] * 3, dim=1).view(-1)
inv_areas.scatter_add_(0, idx, val)
idx = inv_areas > 0
# pyre-fixme[58]: `/` is not supported for operand types `float` and `Tensor`.
inv_areas[idx] = 1.0 / inv_areas[idx]
inv_areas[idx] = torch.reciprocal(inv_areas[idx])
inv_areas = inv_areas.view(-1, 1)
return L, inv_areas

View File

@@ -247,6 +247,7 @@ class _marching_cubes(Function):
return verts, faces, ids
@staticmethod
# pyrefly: ignore [bad-override]
def backward(ctx, grad_verts, grad_faces):
raise ValueError("marching_cubes backward is not supported")

View File

@@ -50,6 +50,7 @@ class _MeshFaceAreasNormals(Function):
@staticmethod
@once_differentiable
# pyrefly: ignore [bad-override]
def backward(ctx, grad_areas, grad_normals):
grad_areas = grad_areas.contiguous()
grad_normals = grad_normals.contiguous()

View File

@@ -54,6 +54,7 @@ class _PackedToPadded(Function):
@staticmethod
@once_differentiable
# pyrefly: ignore [bad-override]
def backward(ctx, grad_output):
grad_output = grad_output.contiguous()
first_idxs = ctx.saved_tensors[0]
@@ -143,6 +144,7 @@ class _PaddedToPacked(Function):
@staticmethod
@once_differentiable
# pyrefly: ignore [bad-override]
def backward(ctx, grad_output):
grad_output = grad_output.contiguous()
first_idxs = ctx.saved_tensors[0]

View File

@@ -103,14 +103,17 @@ def iterative_closest_point(
Xt, num_points_X = oputil.convert_pointclouds_to_tensor(X)
Yt, num_points_Y = oputil.convert_pointclouds_to_tensor(Y)
# pyrefly: ignore [missing-attribute]
b, size_X, dim = Xt.shape
# pyrefly: ignore [missing-attribute]
if (Xt.shape[2] != Yt.shape[2]) or (Xt.shape[0] != Yt.shape[0]):
raise ValueError(
"Point sets X and Y have to have the same "
+ "number of batches and data dimensions."
)
# pyrefly: ignore [missing-attribute]
if ((num_points_Y < Yt.shape[1]).any() or (num_points_X < Xt.shape[1]).any()) and (
num_points_Y != num_points_X
).any():
@@ -121,6 +124,7 @@ def iterative_closest_point(
< num_points_X[:, None]
).type_as(Xt)
else:
# pyrefly: ignore [missing-attribute]
mask_X = Xt.new_ones(b, size_X)
# clone the initial point cloud
@@ -145,11 +149,15 @@ def iterative_closest_point(
"of scalars of shape (minibatch,)."
) from None
# apply the init transform to the input point cloud
# pyrefly: ignore [bad-argument-type]
Xt = _apply_similarity_transform(Xt, R, T, s)
else:
# initialize the transformation with identity
# pyrefly: ignore [missing-attribute]
R = oputil.eyes(dim, b, device=Xt.device, dtype=Xt.dtype)
# pyrefly: ignore [missing-attribute]
T = Xt.new_zeros((b, dim))
# pyrefly: ignore [missing-attribute]
s = Xt.new_ones(b)
prev_rmse = None
@@ -163,7 +171,14 @@ def iterative_closest_point(
# the main loop over ICP iterations
for iteration in range(max_iterations):
Xt_nn_points = knn_points(
Xt, Yt, lengths1=num_points_X, lengths2=num_points_Y, K=1, return_nn=True
# pyrefly: ignore [bad-argument-type]
Xt,
# pyrefly: ignore [bad-argument-type]
Yt,
lengths1=num_points_X,
lengths2=num_points_Y,
K=1,
return_nn=True,
).knn[:, :, 0, :]
# get the alignment of the nearest neighbors from Yt with Xt_init
@@ -182,8 +197,7 @@ def iterative_closest_point(
t_history.append(SimilarityTransform(R, T, s))
# compute the root mean squared error
# pyre-fixme[58]: `**` is not supported for operand types `Tensor` and `int`.
Xt_sq_diff = ((Xt - Xt_nn_points) ** 2).sum(2)
Xt_sq_diff = torch.square(Xt - Xt_nn_points).sum(2)
rmse = oputil.wmean(Xt_sq_diff[:, :, None], mask_X).sqrt()[:, 0, 0]
# compute the relative rmse
@@ -217,6 +231,7 @@ def iterative_closest_point(
if oputil.is_pointclouds(X):
Xt = X.update_padded(Xt) # type: ignore
# pyrefly: ignore [bad-argument-type]
return ICPSolution(converged, rmse, Xt, SimilarityTransform(R, T, s), t_history)
@@ -277,6 +292,7 @@ def corresponding_points_alignment(
Xt, num_points = oputil.convert_pointclouds_to_tensor(X)
Yt, num_points_Y = oputil.convert_pointclouds_to_tensor(Y)
# pyrefly: ignore [missing-attribute]
if (Xt.shape != Yt.shape) or (num_points != num_points_Y).any():
raise ValueError(
"Point sets X and Y have to have the same \
@@ -292,25 +308,33 @@ def corresponding_points_alignment(
weights = [w[..., None] for w in weights]
weights = strutil.list_to_padded(weights)[..., 0]
# pyrefly: ignore [bad-index]
if Xt.shape[:2] != weights.shape:
raise ValueError("weights should have the same first two dimensions as X.")
# pyrefly: ignore [not-iterable]
b, n, dim = Xt.shape
# pyrefly: ignore [bad-index, missing-attribute]
if (num_points < Xt.shape[1]).any() or (num_points < Yt.shape[1]).any():
# in case we got Pointclouds as input, mask the unused entries in Xc, Yc
mask = (
torch.arange(n, dtype=torch.int64, device=Xt.device)[None]
< num_points[:, None]
).type_as(Xt)
# pyrefly: ignore [bad-argument-type]
weights = mask if weights is None else mask * weights.type_as(Xt)
# compute the centroids of the point sets
# pyrefly: ignore [bad-argument-type]
Xmu = oputil.wmean(Xt, weight=weights, eps=eps)
# pyrefly: ignore [bad-argument-type]
Ymu = oputil.wmean(Yt, weight=weights, eps=eps)
# mean-center the point sets
# pyrefly: ignore [unsupported-operation]
Xc = Xt - Xmu
# pyrefly: ignore [unsupported-operation]
Yc = Yt - Ymu
total_weight = torch.clamp(num_points, 1)

View File

@@ -119,6 +119,7 @@ def estimate_pointcloud_local_coord_frames(
points_padded, num_points = convert_pointclouds_to_tensor(pointclouds)
# pyrefly: ignore [missing-attribute]
ba, N, dim = points_padded.shape
if dim != 3:
raise ValueError(
@@ -133,6 +134,7 @@ def estimate_pointcloud_local_coord_frames(
# undo global mean for stability
# TODO: replace with tutil.wmean once landed
# pyrefly: ignore [missing-attribute]
pcl_mean = points_padded.sum(1) / num_points[:, None]
points_centered = points_padded - pcl_mean[:, None, :]
@@ -154,17 +156,26 @@ def estimate_pointcloud_local_coord_frames(
if disambiguate_directions:
# disambiguate normal
n = _disambiguate_vector_directions(
points_centered, knns, local_coord_frames[:, :, :, 0]
# pyrefly: ignore [unsupported-operation]
points_centered,
knns,
# pyrefly: ignore [unsupported-operation]
local_coord_frames[:, :, :, 0],
)
# disambiguate the main curvature
z = _disambiguate_vector_directions(
points_centered, knns, local_coord_frames[:, :, :, 2]
# pyrefly: ignore [unsupported-operation]
points_centered,
knns,
# pyrefly: ignore [unsupported-operation]
local_coord_frames[:, :, :, 2],
)
# the secondary curvature is just a cross between n and z
y = torch.cross(n, z, dim=2)
# cat to form the set of principal directions
local_coord_frames = torch.stack((n, y, z), dim=3)
# pyrefly: ignore [bad-return]
return curvatures, local_coord_frames

View File

@@ -141,6 +141,7 @@ class _points_to_volumes_function(Function):
@staticmethod
@once_differentiable
# pyrefly: ignore [bad-override]
def backward(ctx, grad_volume_densities, grad_volume_features):
splat = ctx.splat
N, C = grad_volume_features.shape[:2]
@@ -377,6 +378,7 @@ def add_points_features_to_volume_densities_features(
if grid_sizes is None:
# grid sizes shape (minibatch, 3)
grid_sizes = (
# pyrefly: ignore [bad-assignment]
torch.LongTensor(list(volume_densities.shape[2:]))
.to(volume_densities.device)
.expand(volume_densities.shape[0], 3)

View File

@@ -141,6 +141,7 @@ def sample_farthest_points_naive(
for n in range(N):
# Initialize an array for the sampled indices, shape: (max_K,)
# pyrefly: ignore [no-matching-overload]
sample_idx_batch = torch.full(
# pyre-fixme[6]: For 1st param expected `Union[List[int], Size,
# typing.Tuple[int, ...]]` but got `Tuple[Tensor]`.
@@ -179,9 +180,7 @@ def sample_farthest_points_naive(
# and all the other points. If a point has already been selected
# it's distance will be 0.0 so it will not be selected again as the max.
dist = points[n, selected_idx, :] - points[n, : lengths[n], :]
# pyre-fixme[58]: `**` is not supported for operand types `Tensor` and
# `int`.
dist_to_last_selected = (dist**2).sum(-1) # (P - i)
dist_to_last_selected = torch.square(dist).sum(-1) # (P - i)
# If closer than currently saved distance to one of the selected
# points, then updated closest_dists

View File

@@ -16,9 +16,7 @@ import sys
from typing import Tuple, Union
import torch
from pytorch3d.ops.mesh_face_areas_normals import mesh_face_areas_normals
from pytorch3d.ops.packed_to_padded import packed_to_padded
from pytorch3d.renderer.mesh.rasterizer import Fragments as MeshFragments

View File

@@ -143,7 +143,7 @@ def convert_pointclouds_to_tensor(pcl: Union[torch.Tensor, "Pointclouds"]):
elif torch.is_tensor(pcl):
X = pcl
num_points = X.shape[1] * torch.ones( # type: ignore
X.shape[0],
X.shape[0], # pyrefly: ignore [missing-attribute]
device=X.device,
dtype=torch.int64,
)

View File

@@ -69,7 +69,6 @@ from .mesh import (
TexturesUV,
TexturesVertex,
)
from .points import (
AlphaCompositor,
NormWeightedCompositor,

View File

@@ -101,6 +101,7 @@ class _SigmoidAlphaBlend(torch.autograd.Function):
return alphas
@staticmethod
# pyrefly: ignore [bad-override]
def backward(ctx, grad_alphas):
dists, pix_to_face, alphas = ctx.saved_tensors
sigma = ctx.sigma
@@ -212,8 +213,10 @@ def softmax_rgb_blend(
# Reshape to be compatible with (N, H, W, K) values in fragments
if torch.is_tensor(zfar):
# pyrefly: ignore [bad-index]
zfar = zfar[:, None, None, None]
if torch.is_tensor(znear):
# pyrefly: ignore [bad-index]
znear = znear[:, None, None, None]
# pyre-fixme[6]: Expected `float` but got `Union[float, Tensor]`

View File

@@ -86,6 +86,7 @@ def _opencv_from_cameras_projection(
scale = scale.expand(-1, 2)
c0 = image_size_wh / 2.0
# pyrefly: ignore [unsupported-operation]
principal_point = -p0_pytorch3d * scale + c0
focal_length = focal_pytorch3d * scale
@@ -153,12 +154,12 @@ def _pulsar_from_opencv_projection(
# Check image sizes.
image_w = image_size_wh[0, 0]
image_h = image_size_wh[0, 1]
assert torch.all(
image_size_wh[:, 0] == image_w
), "All images in a batch must have the same width!"
assert torch.all(
image_size_wh[:, 1] == image_h
), "All images in a batch must have the same height!"
assert torch.all(image_size_wh[:, 0] == image_w), (
"All images in a batch must have the same width!"
)
assert torch.all(image_size_wh[:, 1] == image_h), (
"All images in a batch must have the same height!"
)
# Focal length.
fx = camera_matrix[:, 0, 0].unsqueeze(1)
fy = camera_matrix[:, 1, 1].unsqueeze(1)

View File

@@ -202,8 +202,10 @@ def join_cameras_as_batch(cameras_list: Sequence[CamerasBase]) -> CamerasBase:
# In the init, all inputs will be converted to
# batched tensors before set as attributes
# Join as a tensor along the batch dimension
# pyrefly: ignore [unsupported-operation]
kwargs[field] = torch.cat(attrs_list, dim=0)
else:
raise ValueError(f"Field {field} type is not supported for batching")
# pyrefly: ignore [bad-argument-type]
return c0.__class__(**kwargs)

View File

@@ -362,6 +362,7 @@ class CamerasBase(TensorProperties):
self, with_xyflip=with_xyflip, image_size=image_size
).transform_points(points_ndc, eps=eps)
# pyrefly: ignore [bad-override]
def clone(self):
"""
Returns a copy of `self`.
@@ -390,6 +391,7 @@ class CamerasBase(TensorProperties):
"""
return getattr(self, "image_size", None)
# pyrefly: ignore [bad-override]
def __getitem__(
self, index: Union[int, List[int], torch.BoolTensor, torch.LongTensor]
) -> "CamerasBase":
@@ -455,11 +457,14 @@ class CamerasBase(TensorProperties):
elif isinstance(val, torch.Tensor):
# In the init, all inputs will be converted to
# tensors before setting as attributes
# pyrefly: ignore [unsupported-operation]
kwargs[field] = val[index]
else:
raise ValueError(f"Field {field} type is not supported for indexing")
# pyrefly: ignore [unsupported-operation]
kwargs["device"] = self.device
# pyrefly: ignore [bad-argument-type]
return self.__class__(**kwargs)
@@ -629,10 +634,8 @@ class FoVPerspectiveCameras(CamerasBase):
# so the so the z sign is 1.0.
z_sign = 1.0
# pyre-fixme[58]: `/` is not supported for operand types `float` and `Tensor`.
K[:, 0, 0] = 2.0 * znear / (max_x - min_x)
# pyre-fixme[58]: `/` is not supported for operand types `float` and `Tensor`.
K[:, 1, 1] = 2.0 * znear / (max_y - min_y)
K[:, 0, 0] = torch.div(2.0 * znear, max_x - min_x)
K[:, 1, 1] = torch.div(2.0 * znear, max_y - min_y)
K[:, 0, 2] = (max_x + min_x) / (max_x - min_x)
K[:, 1, 2] = (max_y + min_y) / (max_y - min_y)
K[:, 3, 2] = z_sign * ones
@@ -1178,9 +1181,7 @@ class PerspectiveCameras(CamerasBase):
xy_inv_depth = torch.cat(
# pyre-fixme[6]: For 1st argument expected `Union[List[Tensor],
# tuple[Tensor, ...]]` but got `Tuple[Tensor, float]`.
# pyre-fixme[58]: `/` is not supported for operand types `float` and
# `Tensor`.
(xy_depth[..., :2], 1.0 / xy_depth[..., 2:3]),
(xy_depth[..., :2], torch.reciprocal(xy_depth[..., 2:3])),
dim=-1, # type: ignore
)
return unprojection_transform.transform_points(xy_inv_depth)
@@ -1745,7 +1746,11 @@ def look_at_view_transform(
dist, elev, azim, at, up = broadcasted_args
C = (
camera_position_from_spherical_angles(
dist, elev, azim, degrees=degrees, device=device
dist, # pyrefly: ignore [bad-argument-type]
elev, # pyrefly: ignore [bad-argument-type]
azim, # pyrefly: ignore [bad-argument-type]
degrees=degrees,
device=device, # pyrefly: ignore [bad-argument-type]
)
+ at
)
@@ -1791,6 +1796,7 @@ def get_ndc_to_screen_transform(
K = torch.zeros((cameras._N, 4, 4), device=cameras.device, dtype=torch.float32)
if not torch.is_tensor(image_size):
image_size = torch.tensor(image_size, device=cameras.device)
# pyrefly: ignore [missing-attribute]
image_size = image_size.view(-1, 2) # of shape (1 or B)x2
height, width = image_size.unbind(1)

View File

@@ -51,6 +51,7 @@ class _CompositeAlphaPoints(torch.autograd.Function):
return pt_cld
@staticmethod
# pyrefly: ignore [bad-override]
def backward(ctx, grad_output):
grad_features = None
grad_alphas = None
@@ -130,6 +131,7 @@ class _CompositeNormWeightedSumPoints(torch.autograd.Function):
return pt_cld
@staticmethod
# pyrefly: ignore [bad-override]
def backward(ctx, grad_output):
grad_features = None
grad_alphas = None
@@ -208,6 +210,7 @@ class _CompositeWeightedSumPoints(torch.autograd.Function):
return pt_cld
@staticmethod
# pyrefly: ignore [bad-override]
def backward(ctx, grad_output):
grad_features = None
grad_alphas = None

View File

@@ -124,6 +124,7 @@ class FishEyeCameras(CamerasBase):
else:
self.image_size = None
# pyrefly: ignore [bad-assignment]
self.device = device
self.focal = focal_length.to(self.device)
self.principal_point = principal_point.to(self.device)

Some files were not shown because too many files have changed in this diff Show More