Summary:
EGLContext is a utility to render with OpenGL without an attached display (that is, without a monitor).
DeviceContextManager allows us to avoid unnecessary context creations and releases. See docstrings for more info.
Reviewed By: jcjohnson
Differential Revision: D36562551
fbshipit-source-id: eb0d2a2f85555ee110e203d435a44ad243281d2c
Summary: Avoid calculating all_train_cameras before it is needed, because it is slow in some datasets.
Reviewed By: shapovalov
Differential Revision: D38037157
fbshipit-source-id: 95461226655cde2626b680661951ab17ebb0ec75
Summary: We especially need omegaconf when testing impicitron.
Reviewed By: patricklabatut
Differential Revision: D37921440
fbshipit-source-id: 4e66fde35aa29f60eabd92bf459cd584cfd7e5ca
Summary:
X-link: https://github.com/fairinternal/pytorch3d/pull/39
Blender and LLFF cameras were sending screen space focal length and principal point to a camera init function expecting NDC
Reviewed By: shapovalov
Differential Revision: D37788686
fbshipit-source-id: 2ddf7436248bc0d174eceb04c288b93858138582
Summary: Add the conditioning types to the repro yaml files. In particular, this fixes test_conditioning_type.
Reviewed By: shapovalov
Differential Revision: D37914537
fbshipit-source-id: 621390f329d9da662d915eb3b7bc709206a20552
Summary:
I tried to run `experiment.py` and `pytorch3d_implicitron_runner` and faced the failure with this traceback: https://www.internalfb.com/phabricator/paste/view/P515734086
It seems to be due to the new release of OmegaConf (version=2.2.2) which requires different typing. This fix helped to overcome it.
Reviewed By: bottler
Differential Revision: D37881644
fbshipit-source-id: be0cd4ced0526f8382cea5bdca9b340e93a2fba2
Summary:
EPnP fails the test when the number of points is below 6. As suggested, quadratic option is in theory to deal with as few as 4 points (so num_pts_thresh=3 is set). And when num_pts > num_pts_thresh=4, skip_q is False.
To avoid bumping num_pts_thresh while passing all the original tests, check_output is set to False when num_pts < 6, similar to the logic in Line 123-127. It makes sure that the algo doesn't crash.
Reviewed By: shapovalov
Differential Revision: D37804438
fbshipit-source-id: 74576d63a9553e25e3ec344677edb6912b5f9354
Summary: Removing 1 from the crop mask does not seem sensible.
Reviewed By: bottler, shapovalov
Differential Revision: D37843680
fbshipit-source-id: 70cec80f9ea26deac63312da62b9c8af27d2a010
Summary:
1. Random sampling of num batches without replacement not supported.
2.Providers should implement the interface for the training loop to work.
Reviewed By: bottler, davnov134
Differential Revision: D37815388
fbshipit-source-id: 8a2795b524e733f07346ffdb20a9c0eb1a2b8190
Summary: Accelerate is an additional implicitron dependency, so document it.
Reviewed By: shapovalov
Differential Revision: D37786933
fbshipit-source-id: 11024fe604107881f8ca29e17cb5cbfe492fc7f9
Summary:
1. Respecting `visdom_show_preds` parameter when it is False.
2. Clipping the images pre-visualisation, which is important for methods like SRN that are not arare of pixel value range.
Reviewed By: bottler
Differential Revision: D37786439
fbshipit-source-id: 8dbb5104290bcc5c2829716b663cae17edc911bd
Summary: one more bugfix in JsonIndexDataset
Reviewed By: bottler
Differential Revision: D37789138
fbshipit-source-id: 2fb2bda7448674091ff6b279175f0bbd16ff7a62
Summary:
This fixes a indexing bug in HardDepthShader and adds proper unit tests for both of the DepthShaders. This bug was introduced when updating the shader sizes and discovered when I switched my local model onto pytorch3d trunk instead of the patched copy.
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1252
Test Plan:
Unit test + custom model code
```
pytest tests/test_shader.py
```
Reviewed By: bottler
Differential Revision: D37775767
Pulled By: d4l3k
fbshipit-source-id: 5f001903985976d7067d1fa0a3102d602790e3e8
Summary:
For 3D segmentation problems it's really useful to be able to train the models from multiple viewpoints using Pytorch3D as the renderer. Currently due to hardcoded assumptions in a few spots the mesh renderer only supports rendering RGB (3 dimensional) data. You can encode the classification information as 3 channel data but if you have more than 3 classes you're out of luck.
This relaxes the assumptions to make rendering semantic classes work with `HardFlatShader` and `AmbientLights` with no diffusion/specular. The other shaders/lights don't make any sense for classification since they mutate the texture values in some way.
This only requires changes in `Materials` and `AmbientLights`. The bulk of the code is the unit test.
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1248
Test Plan: Added unit test that renders a 5 dimensional texture and compare dimensions 2-5 to a stored picture.
Reviewed By: bottler
Differential Revision: D37764610
Pulled By: d4l3k
fbshipit-source-id: 031895724d9318a6f6bab5b31055bb3f438176a5
Summary:
## Changes:
- Added Accelerate Library and refactored experiment.py to use it
- Needed to move `init_optimizer` and `ExperimentConfig` to a separate file to be compatible with submitit/hydra
- Needed to make some modifications to data loaders etc to work well with the accelerate ddp wrappers
- Loading/saving checkpoints incorporates an unwrapping step so remove the ddp wrapped model
## Tests
Tested with both `torchrun` and `submitit/hydra` on two gpus locally. Here are the commands:
**Torchrun**
Modules loaded:
```sh
1) anaconda3/2021.05 2) cuda/11.3 3) NCCL/2.9.8-3-cuda.11.3 4) gcc/5.2.0. (but unload gcc when using submit)
```
```sh
torchrun --nnodes=1 --nproc_per_node=2 experiment.py --config-path ./configs --config-name repro_singleseq_nerf_test
```
**Submitit/Hydra Local test**
```sh
~/pytorch3d/projects/implicitron_trainer$ HYDRA_FULL_ERROR=1 python3.9 experiment.py --config-name repro_singleseq_nerf_test --multirun --config-path ./configs hydra/launcher=submitit_local hydra.launcher.gpus_per_node=2 hydra.launcher.tasks_per_node=2 hydra.launcher.nodes=1
```
**Submitit/Hydra distributed test**
```sh
~/implicitron/pytorch3d$ python3.9 experiment.py --config-name repro_singleseq_nerf_test --multirun --config-path ./configs hydra/launcher=submitit_slurm hydra.launcher.gpus_per_node=8 hydra.launcher.tasks_per_node=8 hydra.launcher.nodes=1 hydra.launcher.partition=learnlab hydra.launcher.timeout_min=4320
```
## TODOS:
- Fix distributed evaluation: currently this doesn't work as the input format to the evaluation function is not suitable for gathering across gpus (needs to be nested list/tuple/dicts of objects that satisfy `is_torch_tensor`) and currently `frame_data` contains `Cameras` type.
- Refactor the `accelerator` object to be accessible by all functions instead of needing to pass it around everywhere? Maybe have a `Trainer` class and add it as a method?
- Update readme with installation instructions for accelerate and also commands for running jobs with torchrun and submitit/hydra
X-link: https://github.com/fairinternal/pytorch3d/pull/37
Reviewed By: davnov134, kjchalup
Differential Revision: D37543870
Pulled By: bottler
fbshipit-source-id: be9eb4e91244d4fe3740d87dafec622ae1e0cf76
Summary: A new version of json index dataset provider supporting CO3Dv2
Reviewed By: shapovalov
Differential Revision: D37690918
fbshipit-source-id: bf2d5fc9d0f1220259e08661dafc69cdbe6b7f94
Summary:
Implements several changes needed for the CO3Dv2 release:
- FrameData contains crop_bbox_xywh which defines the outline of the image crop corresponding to the image-shaped tensors in FrameData
- revised the definition of a bounding box inside JsonDatasetIndex: bbox_xyxy is [xmin, ymin, xmax, ymax], where xmax, ymax are not inclusive; bbox_xywh = [xmin, ymain, xmax-xmin, ymax-ymin]
- is_filtered for detecting whether the entries of the dataset were somehow filtered
- seq_frame_index_to_dataset_index allows to skip entries that are not present in the dataset
Reviewed By: shapovalov
Differential Revision: D37687547
fbshipit-source-id: 7842756b0517878cc0964fc0935d3c0769454d78
Summary: cu116 builds need to happen in a specific image.
Reviewed By: patricklabatut
Differential Revision: D37680352
fbshipit-source-id: 81bef0642ad832e83e4eba6321287759b3229303
Summary: As part of removing Task, move camera difficulty bin breaks from hard code to the top level.
Reviewed By: davnov134
Differential Revision: D37491040
fbshipit-source-id: f2d6775ebc490f6f75020d13f37f6b588cc07a0b
Summary: As part of removing Task, make the dataset code generate the source cameras for itself. There's a small optimization available here, in that the JsonIndexDataset could avoid loading images.
Reviewed By: shapovalov
Differential Revision: D37313423
fbshipit-source-id: 3e5e0b2aabbf9cc51f10547a3523e98c72ad8755
Summary:
Make ViewMetrics easy to replace by putting them into an OmegaConf dataclass.
Also, re-word a few variable names and fix minor TODOs.
Reviewed By: bottler
Differential Revision: D37327157
fbshipit-source-id: 78d8e39bbb3548b952f10abbe05688409fb987cc
Summary: After landing https://github.com/pytorch/pytorch/pull/69607, that made it an error to use indexing with `cpu_tensor[cuda_indices]`. There was one outstanding test in fbcode that incorrectly used indexing in that way, which is fixed here
Reviewed By: bottler, osalpekar
Differential Revision: D37128838
fbshipit-source-id: 611b6f717b5b5d89fa61fd9ebeb513ad7e65a656
Summary:
David had his code crashed when using frame_annot["meta"] dictionary. Turns out we had a typo.
The tests were passing by chance since all the keys were single-character strings.
Reviewed By: bottler
Differential Revision: D37503987
fbshipit-source-id: c12b0df21116cfbbc4675a0182b9b9e6d62bad2e
Summary: small followup to D37172537 (cba26506b6) and D37209012 (81d63c6382): changing default #harmonics and improving a test
Reviewed By: shapovalov
Differential Revision: D37412357
fbshipit-source-id: 1af1005a129425fd24fa6dd213d69c71632099a0
Summary: The blender synthetic dataset contains object masks in the alpha channel. Provide these in the corresponding dataset.
Reviewed By: shapovalov
Differential Revision: D37344380
fbshipit-source-id: 3ddacad9d667c0fa0ae5a61fb1d2ffc806c9abf3
Summary: Images were coming out in the wrong format.
Reviewed By: shapovalov
Differential Revision: D37291278
fbshipit-source-id: c10871c37dd186982e7abf2071ac66ed583df2e6
Summary: Just register ImplicitronDataSource. We don't use it as pluggable yet here.
Reviewed By: shapovalov
Differential Revision: D37315698
fbshipit-source-id: ac41153383f9ab6b14ac69a3dfdc44aca0d94995
Summary: Document the inputs of idr functions and distinguish n_harmonic_functions to be 0 (simple embedding) versus -1 (no embedding).
Reviewed By: davnov134
Differential Revision: D37209012
fbshipit-source-id: 6e5c3eae54c4e5e8c3f76cad1caf162c6c222d52
Summary: Use generator.permutation instead of choice so that different options for n_known_frames_for_test are nested.
Reviewed By: davnov134
Differential Revision: D37210906
fbshipit-source-id: fd0d34ce62260417c3f63354a3f750aae9998b0d
Summary: Allow specifying a color for non-opaque pixels in LSTMRenderer.
Reviewed By: davnov134
Differential Revision: D37172537
fbshipit-source-id: 6039726678bb7947f7d8cd04035b5023b2d5398c
Summary: Copy code from NeRF for loading LLFF data and blender synthetic data, and create dataset objects for them
Reviewed By: shapovalov
Differential Revision: D35581039
fbshipit-source-id: af7a6f3e9a42499700693381b5b147c991f57e5d
Summary:
pyfmt now specifies a target Python version of 3.8 when formatting
with black. With this new config, black adds trailing commas to all
multiline function calls. This applies the new formatting as part
of rolling out the linttool-integration for pyfmt.
paintitblack
Reviewed By: zertosh, lisroach
Differential Revision: D37084377
fbshipit-source-id: 781a1b883a381a172e54d6e447137657977876b4
