Summary: Currently some implicit functions in implicitron take a raybundle, others take ray_points_world. raybundle is what they really need. However, the raybundle is going to become a bit more flexible later, as it will contain different numbers of rays for each camera.
Reviewed By: bottler
Differential Revision: D39173751
fbshipit-source-id: ebc038e426d22e831e67a18ba64655d8a61e1eb9
Summary: Added replacable decoding functions which will be applied after the voxel grid to get color and density
Reviewed By: bottler
Differential Revision: D38829763
fbshipit-source-id: f21ce206c1c19548206ea2ce97d7ebea3de30a23
Summary: Simple wrapper around voxel grids to make them a module
Reviewed By: bottler
Differential Revision: D38829762
fbshipit-source-id: dfee85088fa3c65e396cc7d3bf7ebaaffaadb646
Summary: Added voxel grid classes from TensoRF, both in their factorized (CP and VM) and full form.
Reviewed By: bottler
Differential Revision: D38465419
fbshipit-source-id: 8b306338af58dc50ef47a682616022a0512c0047
Summary:
generic_model_args no longer exists. Update some references to it, mostly in doc.
This fixes the testing of all the yaml files in test_forward pass.
Reviewed By: shapovalov
Differential Revision: D38789202
fbshipit-source-id: f11417efe772d7f86368b3598aa66c52b1309dbf
Summary: Previously, "psnr" was evaluated between the masked g.t. image and the render. To avoid confusion, "psnr" is now renamed to "psnr_masked".
Reviewed By: bottler
Differential Revision: D38707511
fbshipit-source-id: 8ee881ab1a05453d6692dde9782333a47d8c1234
Summary: Reports also the PSNR between the unmasked G.T. image and the render.
Reviewed By: bottler
Differential Revision: D38655943
fbshipit-source-id: 1603a2d02116ea1ce037e5530abe1afc65a2ba93
Summary:
LLFF (and most/all non-synth datasets) will have no background/foreground distinction. Add support for data with no fg mask.
Also, we had a bug in stats loading, like this:
* Load stats
* One of the stats has a history of length 0
* That's fine, e.g. maybe it's fg_error but the dataset has no notion of fg/bg. So leave it as len 0
* Check whether all the stats have the same history length as an arbitrarily chosen "reference-stat"
* Ooops the reference-stat happened to be the stat with length 0
* assert (legit_stat_len == reference_stat_len (=0)) ---> failed assert
Also some minor fixes (from Jeremy's other diff) to support LLFF
Reviewed By: davnov134
Differential Revision: D38475272
fbshipit-source-id: 5b35ac86d1d5239759f537621f41a3aa4eb3bd68
Summary: Don't copy from one part of config to another, rather do the copy within GenericModel.
Reviewed By: davnov134
Differential Revision: D38248828
fbshipit-source-id: ff8af985c37ea1f7df9e0aa0a45a58df34c3f893
Summary: Made the config system call open_dict when it calls the tweak function.
Reviewed By: shapovalov
Differential Revision: D38315334
fbshipit-source-id: 5924a92d8d0bf399bbf3788247f81fc990e265e7
Summary:
Stats are logically connected to the training loop, not to the model. Hence, moving to the training loop.
Also removing resume_epoch from OptimizerFactory in favor of a single place - ModelFactory. This removes the need for config consistency checks etc.
Reviewed By: kjchalup
Differential Revision: D38313475
fbshipit-source-id: a1d188a63e28459df381ff98ad8acdcdb14887b7
Summary:
Added _NEED_CONTROL
to JsonIndexDatasetMapProviderV2 and made dataset_tweak_args use it.
Reviewed By: bottler
Differential Revision: D38313914
fbshipit-source-id: 529847571065dfba995b609a66737bd91e002cfe
Summary: Make a dummy single-scene dataset using the code from generate_cow_renders (used in existing NeRF tutorials)
Reviewed By: kjchalup
Differential Revision: D38116910
fbshipit-source-id: 8db6df7098aa221c81d392e5cd21b0e67f65bd70
Summary:
This is an internal change in the config systen. It allows redefining a pluggable implementation with new default values. This is useful in notebooks / interactive use. For example, this now works.
class A(ReplaceableBase):
pass
registry.register
class B(A):
i: int = 4
class C(Configurable):
a: A
a_class_type: str = "B"
def __post_init__(self):
run_auto_creation(self)
expand_args_fields(C)
registry.register
class B(A):
i: int = 5
c = C()
assert c.a.i == 5
Reviewed By: shapovalov
Differential Revision: D38219371
fbshipit-source-id: 72911a9bd3426d3359cf8802cc016fc7f6d7713b
Summary:
Adding MeshRasterizerOpenGL, a faster alternative to MeshRasterizer. The new rasterizer follows the ideas from "Differentiable Surface Rendering via non-Differentiable Sampling".
The new rasterizer 20x faster on a 2M face mesh (try pose optimization on Nefertiti from https://www.cs.cmu.edu/~kmcrane/Projects/ModelRepository/!). The larger the mesh, the larger the speedup.
There are two main disadvantages:
* The new rasterizer works with an OpenGL backend, so requires pycuda.gl and pyopengl installed (though we avoided writing any C++ code, everything is in Python!)
* The new rasterizer is non-differentiable. However, you can still differentiate the rendering function if you use if with the new SplatterPhongShader which we recently added to PyTorch3D (see the original paper cited above).
Reviewed By: patricklabatut, jcjohnson
Differential Revision: D37698816
fbshipit-source-id: 54d120639d3cb001f096237807e54aced0acda25
Summary:
EGLContext is a utility to render with OpenGL without an attached display (that is, without a monitor).
DeviceContextManager allows us to avoid unnecessary context creations and releases. See docstrings for more info.
Reviewed By: jcjohnson
Differential Revision: D36562551
fbshipit-source-id: eb0d2a2f85555ee110e203d435a44ad243281d2c
Summary: Avoid calculating all_train_cameras before it is needed, because it is slow in some datasets.
Reviewed By: shapovalov
Differential Revision: D38037157
fbshipit-source-id: 95461226655cde2626b680661951ab17ebb0ec75
Summary:
X-link: https://github.com/fairinternal/pytorch3d/pull/39
Blender and LLFF cameras were sending screen space focal length and principal point to a camera init function expecting NDC
Reviewed By: shapovalov
Differential Revision: D37788686
fbshipit-source-id: 2ddf7436248bc0d174eceb04c288b93858138582
Summary: A new version of json index dataset provider supporting CO3Dv2
Reviewed By: shapovalov
Differential Revision: D37690918
fbshipit-source-id: bf2d5fc9d0f1220259e08661dafc69cdbe6b7f94
Summary:
Implements several changes needed for the CO3Dv2 release:
- FrameData contains crop_bbox_xywh which defines the outline of the image crop corresponding to the image-shaped tensors in FrameData
- revised the definition of a bounding box inside JsonDatasetIndex: bbox_xyxy is [xmin, ymin, xmax, ymax], where xmax, ymax are not inclusive; bbox_xywh = [xmin, ymain, xmax-xmin, ymax-ymin]
- is_filtered for detecting whether the entries of the dataset were somehow filtered
- seq_frame_index_to_dataset_index allows to skip entries that are not present in the dataset
Reviewed By: shapovalov
Differential Revision: D37687547
fbshipit-source-id: 7842756b0517878cc0964fc0935d3c0769454d78
Summary: As part of removing Task, make the dataset code generate the source cameras for itself. There's a small optimization available here, in that the JsonIndexDataset could avoid loading images.
Reviewed By: shapovalov
Differential Revision: D37313423
fbshipit-source-id: 3e5e0b2aabbf9cc51f10547a3523e98c72ad8755
Summary:
Make ViewMetrics easy to replace by putting them into an OmegaConf dataclass.
Also, re-word a few variable names and fix minor TODOs.
Reviewed By: bottler
Differential Revision: D37327157
fbshipit-source-id: 78d8e39bbb3548b952f10abbe05688409fb987cc
Summary:
David had his code crashed when using frame_annot["meta"] dictionary. Turns out we had a typo.
The tests were passing by chance since all the keys were single-character strings.
Reviewed By: bottler
Differential Revision: D37503987
fbshipit-source-id: c12b0df21116cfbbc4675a0182b9b9e6d62bad2e
Summary: small followup to D37172537 (cba26506b6) and D37209012 (81d63c6382): changing default #harmonics and improving a test
Reviewed By: shapovalov
Differential Revision: D37412357
fbshipit-source-id: 1af1005a129425fd24fa6dd213d69c71632099a0
Summary: The blender synthetic dataset contains object masks in the alpha channel. Provide these in the corresponding dataset.
Reviewed By: shapovalov
Differential Revision: D37344380
fbshipit-source-id: 3ddacad9d667c0fa0ae5a61fb1d2ffc806c9abf3
Summary: Images were coming out in the wrong format.
Reviewed By: shapovalov
Differential Revision: D37291278
fbshipit-source-id: c10871c37dd186982e7abf2071ac66ed583df2e6
Summary: Allow specifying a color for non-opaque pixels in LSTMRenderer.
Reviewed By: davnov134
Differential Revision: D37172537
fbshipit-source-id: 6039726678bb7947f7d8cd04035b5023b2d5398c
Summary: Copy code from NeRF for loading LLFF data and blender synthetic data, and create dataset objects for them
Reviewed By: shapovalov
Differential Revision: D35581039
fbshipit-source-id: af7a6f3e9a42499700693381b5b147c991f57e5d
Summary: Make dataset type and args configurable on JsonIndexDatasetMapProvider.
Reviewed By: davnov134
Differential Revision: D36666705
fbshipit-source-id: 4d0a3781d9a956504f51f1c7134c04edf1eb2846
Summary: Changes to JsonIndexDataset to make it fit with OmegaConf.structured. Also match some default values to what the provider defaults to.
Reviewed By: davnov134
Differential Revision: D36666704
fbshipit-source-id: 65b059a1dbaa240ce85c3e8762b7c3db3b5a6e75
Summary: Allow a class to modify its subparts in get_default_args by defining the special function provide_config_hook.
Reviewed By: davnov134
Differential Revision: D36671081
fbshipit-source-id: 3e5b73880cb846c494a209c4479835f6352f45cf
Summary: The ImplicitronDataset class corresponds to JsonIndexDatasetMapProvider
Reviewed By: shapovalov
Differential Revision: D36661396
fbshipit-source-id: 80ca2ff81ef9ecc2e3d1f4e1cd14b6f66a7ec34d
Summary: test_viewpool was inactive so missed being fixed in D36547815 (2d1c6d5d93)
Reviewed By: kjchalup
Differential Revision: D36625587
fbshipit-source-id: e7224eadfa5581fe61f10f67d2221071783de04a
Summary: Use small image size for test_all_gm_configs
Reviewed By: shapovalov
Differential Revision: D36511528
fbshipit-source-id: 2c65f518a4f23626850343a62d103f85abfabd88
