Summary: Don't copy from one part of config to another, rather do the copy within GenericModel.
Reviewed By: davnov134
Differential Revision: D38248828
fbshipit-source-id: ff8af985c37ea1f7df9e0aa0a45a58df34c3f893
Summary:
Stats are logically connected to the training loop, not to the model. Hence, moving to the training loop.
Also removing resume_epoch from OptimizerFactory in favor of a single place - ModelFactory. This removes the need for config consistency checks etc.
Reviewed By: kjchalup
Differential Revision: D38313475
fbshipit-source-id: a1d188a63e28459df381ff98ad8acdcdb14887b7
Summary: Before this diff, train_stats.py would not be created by default, EXCEPT when resuming training. This makes them appear from start.
Reviewed By: shapovalov
Differential Revision: D38320341
fbshipit-source-id: 8ea5b99ec81c377ae129f58e78dc2eaff94821ad
Summary: Remove the dataset's need to provide the task type.
Reviewed By: davnov134, kjchalup
Differential Revision: D38314000
fbshipit-source-id: 3805d885b5d4528abdc78c0da03247edb9abf3f7
Summary:
Added _NEED_CONTROL
to JsonIndexDatasetMapProviderV2 and made dataset_tweak_args use it.
Reviewed By: bottler
Differential Revision: D38313914
fbshipit-source-id: 529847571065dfba995b609a66737bd91e002cfe
Summary: Only import it if you ask for it.
Reviewed By: kjchalup
Differential Revision: D38327167
fbshipit-source-id: 3f05231f26eda582a63afc71b669996342b0c6f9
Summary: Currently, seeds are set only inside the train loop. But this does not ensure that the model weights are initialized the same way everywhere which makes all experiments irreproducible. This diff fixes it.
Reviewed By: bottler
Differential Revision: D38315840
fbshipit-source-id: 3d2ecebbc36072c2b68dd3cd8c5e30708e7dd808
Summary: Make a dummy single-scene dataset using the code from generate_cow_renders (used in existing NeRF tutorials)
Reviewed By: kjchalup
Differential Revision: D38116910
fbshipit-source-id: 8db6df7098aa221c81d392e5cd21b0e67f65bd70
Summary:
This large diff rewrites a significant portion of Implicitron's config hierarchy. The new hierarchy, and some of the default implementation classes, are as follows:
```
Experiment
data_source: ImplicitronDataSource
dataset_map_provider
data_loader_map_provider
model_factory: ImplicitronModelFactory
model: GenericModel
optimizer_factory: ImplicitronOptimizerFactory
training_loop: ImplicitronTrainingLoop
evaluator: ImplicitronEvaluator
```
1) Experiment (used to be ExperimentConfig) is now a top-level Configurable and contains as members mainly (mostly new) high-level factory Configurables.
2) Experiment's job is to run factories, do some accelerate setup and then pass the results to the main training loop.
3) ImplicitronOptimizerFactory and ImplicitronModelFactory are new high-level factories that create the optimizer, scheduler, model, and stats objects.
4) TrainingLoop is a new configurable that runs the main training loop and the inner train-validate step.
5) Evaluator is a new configurable that TrainingLoop uses to run validation/test steps.
6) GenericModel is not the only model choice anymore. Instead, ImplicitronModelBase (by default instantiated with GenericModel) is a member of Experiment and can be easily replaced by a custom implementation by the user.
All the new Configurables are children of ReplaceableBase, and can be easily replaced with custom implementations.
In addition, I added support for the exponential LR schedule, updated the config files and the test, as well as added a config file that reproduces NERF results and a test to run the repro experiment.
Reviewed By: bottler
Differential Revision: D37723227
fbshipit-source-id: b36bee880d6aa53efdd2abfaae4489d8ab1e8a27
Summary: Avoid calculating all_train_cameras before it is needed, because it is slow in some datasets.
Reviewed By: shapovalov
Differential Revision: D38037157
fbshipit-source-id: 95461226655cde2626b680661951ab17ebb0ec75
Summary: Add the conditioning types to the repro yaml files. In particular, this fixes test_conditioning_type.
Reviewed By: shapovalov
Differential Revision: D37914537
fbshipit-source-id: 621390f329d9da662d915eb3b7bc709206a20552
Summary:
I tried to run `experiment.py` and `pytorch3d_implicitron_runner` and faced the failure with this traceback: https://www.internalfb.com/phabricator/paste/view/P515734086
It seems to be due to the new release of OmegaConf (version=2.2.2) which requires different typing. This fix helped to overcome it.
Reviewed By: bottler
Differential Revision: D37881644
fbshipit-source-id: be0cd4ced0526f8382cea5bdca9b340e93a2fba2
Summary:
1. Random sampling of num batches without replacement not supported.
2.Providers should implement the interface for the training loop to work.
Reviewed By: bottler, davnov134
Differential Revision: D37815388
fbshipit-source-id: 8a2795b524e733f07346ffdb20a9c0eb1a2b8190
Summary: Accelerate is an additional implicitron dependency, so document it.
Reviewed By: shapovalov
Differential Revision: D37786933
fbshipit-source-id: 11024fe604107881f8ca29e17cb5cbfe492fc7f9
Summary:
1. Respecting `visdom_show_preds` parameter when it is False.
2. Clipping the images pre-visualisation, which is important for methods like SRN that are not arare of pixel value range.
Reviewed By: bottler
Differential Revision: D37786439
fbshipit-source-id: 8dbb5104290bcc5c2829716b663cae17edc911bd
Summary:
## Changes:
- Added Accelerate Library and refactored experiment.py to use it
- Needed to move `init_optimizer` and `ExperimentConfig` to a separate file to be compatible with submitit/hydra
- Needed to make some modifications to data loaders etc to work well with the accelerate ddp wrappers
- Loading/saving checkpoints incorporates an unwrapping step so remove the ddp wrapped model
## Tests
Tested with both `torchrun` and `submitit/hydra` on two gpus locally. Here are the commands:
**Torchrun**
Modules loaded:
```sh
1) anaconda3/2021.05 2) cuda/11.3 3) NCCL/2.9.8-3-cuda.11.3 4) gcc/5.2.0. (but unload gcc when using submit)
```
```sh
torchrun --nnodes=1 --nproc_per_node=2 experiment.py --config-path ./configs --config-name repro_singleseq_nerf_test
```
**Submitit/Hydra Local test**
```sh
~/pytorch3d/projects/implicitron_trainer$ HYDRA_FULL_ERROR=1 python3.9 experiment.py --config-name repro_singleseq_nerf_test --multirun --config-path ./configs hydra/launcher=submitit_local hydra.launcher.gpus_per_node=2 hydra.launcher.tasks_per_node=2 hydra.launcher.nodes=1
```
**Submitit/Hydra distributed test**
```sh
~/implicitron/pytorch3d$ python3.9 experiment.py --config-name repro_singleseq_nerf_test --multirun --config-path ./configs hydra/launcher=submitit_slurm hydra.launcher.gpus_per_node=8 hydra.launcher.tasks_per_node=8 hydra.launcher.nodes=1 hydra.launcher.partition=learnlab hydra.launcher.timeout_min=4320
```
## TODOS:
- Fix distributed evaluation: currently this doesn't work as the input format to the evaluation function is not suitable for gathering across gpus (needs to be nested list/tuple/dicts of objects that satisfy `is_torch_tensor`) and currently `frame_data` contains `Cameras` type.
- Refactor the `accelerator` object to be accessible by all functions instead of needing to pass it around everywhere? Maybe have a `Trainer` class and add it as a method?
- Update readme with installation instructions for accelerate and also commands for running jobs with torchrun and submitit/hydra
X-link: https://github.com/fairinternal/pytorch3d/pull/37
Reviewed By: davnov134, kjchalup
Differential Revision: D37543870
Pulled By: bottler
fbshipit-source-id: be9eb4e91244d4fe3740d87dafec622ae1e0cf76
Summary: As part of removing Task, move camera difficulty bin breaks from hard code to the top level.
Reviewed By: davnov134
Differential Revision: D37491040
fbshipit-source-id: f2d6775ebc490f6f75020d13f37f6b588cc07a0b
Summary: As part of removing Task, make the dataset code generate the source cameras for itself. There's a small optimization available here, in that the JsonIndexDataset could avoid loading images.
Reviewed By: shapovalov
Differential Revision: D37313423
fbshipit-source-id: 3e5e0b2aabbf9cc51f10547a3523e98c72ad8755
Summary:
Make ViewMetrics easy to replace by putting them into an OmegaConf dataclass.
Also, re-word a few variable names and fix minor TODOs.
Reviewed By: bottler
Differential Revision: D37327157
fbshipit-source-id: 78d8e39bbb3548b952f10abbe05688409fb987cc
Summary: small followup to D37172537 (cba26506b6) and D37209012 (81d63c6382): changing default #harmonics and improving a test
Reviewed By: shapovalov
Differential Revision: D37412357
fbshipit-source-id: 1af1005a129425fd24fa6dd213d69c71632099a0
Summary: Document the inputs of idr functions and distinguish n_harmonic_functions to be 0 (simple embedding) versus -1 (no embedding).
Reviewed By: davnov134
Differential Revision: D37209012
fbshipit-source-id: 6e5c3eae54c4e5e8c3f76cad1caf162c6c222d52
Summary: Allow specifying a color for non-opaque pixels in LSTMRenderer.
Reviewed By: davnov134
Differential Revision: D37172537
fbshipit-source-id: 6039726678bb7947f7d8cd04035b5023b2d5398c
Summary: Copy code from NeRF for loading LLFF data and blender synthetic data, and create dataset objects for them
Reviewed By: shapovalov
Differential Revision: D35581039
fbshipit-source-id: af7a6f3e9a42499700693381b5b147c991f57e5d
Summary: Make dataset type and args configurable on JsonIndexDatasetMapProvider.
Reviewed By: davnov134
Differential Revision: D36666705
fbshipit-source-id: 4d0a3781d9a956504f51f1c7134c04edf1eb2846
Summary: The ImplicitronDataset class corresponds to JsonIndexDatasetMapProvider
Reviewed By: shapovalov
Differential Revision: D36661396
fbshipit-source-id: 80ca2ff81ef9ecc2e3d1f4e1cd14b6f66a7ec34d
Summary: replace dataset_zoo with a pluggable DatasetMapProvider. The logic is now in annotated_file_dataset_map_provider.
Reviewed By: shapovalov
Differential Revision: D36443965
fbshipit-source-id: 9087649802810055e150b2fbfcc3c197a761f28a
Summary: Separate ImplicitronDatasetBase and FrameData (to be used by all data sources) from ImplicitronDataset (which is specific).
Reviewed By: shapovalov
Differential Revision: D36413111
fbshipit-source-id: 3725744cde2e08baa11aff4048237ba10c7efbc6
Summary:
Move dataset_args and dataloader_args from ExperimentConfig into a new member called datasource so that it can contain replaceables.
Also add enum Task for task type.
Reviewed By: shapovalov
Differential Revision: D36201719
fbshipit-source-id: 47d6967bfea3b7b146b6bbd1572e0457c9365871
Summary: Make ResNetFeatureExtractor be an implementation of FeatureExtractorBase.
Reviewed By: davnov134
Differential Revision: D35433098
fbshipit-source-id: 0664a9166a88e150231cfe2eceba017ae55aed3a
Summary:
1. Typo in the dataset path in the config.
2. Typo in num_frames.
3. Pick sequence was cached before it was modified for single-sequence.
Reviewed By: bottler
Differential Revision: D36417329
fbshipit-source-id: 6dcd75583de510412e1ae58f63db04bb4447403e
Summary:
This converts raysamplers to ReplaceableBase so that users can hack their own raysampling impls.
Context: Andrea tried to implement TensoRF within implicitron but could not due to the need to implement his own raysampler.
Reviewed By: shapovalov
Differential Revision: D36016318
fbshipit-source-id: ef746f3365282bdfa9c15f7b371090a5aae7f8da
Summary: Implements a ViewPooler that groups ViewSampler and FeatureAggregator.
Reviewed By: shapovalov
Differential Revision: D35852367
fbshipit-source-id: c1bcaf5a1f826ff94efce53aa5836121ad9c50ec
Summary:
Applies the black-fbsource codemod with the new build of pyfmt.
paintitblack
Reviewed By: lisroach
Differential Revision: D36324783
fbshipit-source-id: 280c09e88257e5e569ab729691165d8dedd767bc
