17 Commits

Author SHA1 Message Date
Jeremy Reizenstein
f00ef66727 NeRF training: avoid caching unused visualization data.
Summary: If we are not visualizing the training with visdom, then there are a couple of outputs of the coarse rendering step which are not small and are returned by the renderer but never used. We don't need to bother transferring them to the CPU.

Reviewed By: nikhilaravi

Differential Revision: D28939958

fbshipit-source-id: 7e0d6681d6524f7fb57b6b20164580006120de80
2021-06-08 04:35:22 -07:00
Patrick Labatut
d76c00721c Remove explicit inheritance from object
Summary:
All classes implicitly inherit from `object` since Python 3, so remove unnecessary explicit inheritance.

From the [official documentation](https://docs.python.org/3/library/functions.html#object):

> `object` is a base for all classes.

Reviewed By: nikhilaravi

Differential Revision: D28942595

fbshipit-source-id: 466c0d19d8a93a6263e7ad734c3e87160cfa6066
2021-06-08 02:15:50 -07:00
Jeremy Reizenstein
280fed3c76 cuda streams for color/density in NeRF
Summary:
Inside the implicit function, the color and density calculations are independent and time is saved by putting them on separate streams.

(In fact, colors is slower than densities, and the raymarcher does some calculation with the densities before the colors. So theoretically we could go further and not join the streams together until the colors are actually needed. The code would be more complicated. But the profile suggests that the raymarcher is quick and so this wouldn't be expected to make a big difference.)

In inference, this might increase memory usage, so it isn't an obvious win. That is why I have added a flag.

Reviewed By: nikhilaravi

Differential Revision: D28648549

fbshipit-source-id: c087de80d8ccfce1dad3a13e71df2f305a36952e
2021-06-02 05:43:13 -07:00
Jeremy Reizenstein
f63e49d245 Let harmonic embedding layer include input (NeRF)
Summary: When harmonic embedding is used, we always cat its input onto its output before proceeding. Avoid an intermediate tensor by making the module do that for itself.

Reviewed By: davnov134

Differential Revision: D28185791

fbshipit-source-id: 98d92c94a918dd42e16cdadcaac71dabbc7de5c3
2021-06-02 05:43:13 -07:00
Jeremy Reizenstein
ab73f8c3fd LinearWithRepeat layer for NeRF
Summary:
Add custom layer to avoid repeating copied data for every ray position.

This should also save time in the backward pass because there are fewer multiplies with the weights.

Reviewed By: theschnitz

Differential Revision: D28382412

fbshipit-source-id: 1ba7356cd8520ebd598568ae503e47d31d3989eb
2021-06-02 05:43:13 -07:00
Jeremy Reizenstein
097b0ef2c6 use no_grad for sample_pdf in NeRF project
Summary: We don't use gradents of sample_pdf. Here we disable gradient calculation around calling it, instead of calling detach later. There's a theoretical speedup, but mainly this enables using sample_pdf implementations which don't support gradients.

Reviewed By: nikhilaravi

Differential Revision: D28057284

fbshipit-source-id: 8a9d5e73f18b34e1e4291028008e02973023638d
2021-04-28 09:34:50 -07:00
Jeremy Reizenstein
124bb5e391 spelling
Summary: Collection of spelling things, mostly in docs / tutorials.

Reviewed By: gkioxari

Differential Revision: D26101323

fbshipit-source-id: 652f62bc9d71a4ff872efa21141225e43191353a
2021-04-09 09:58:54 -07:00
David Novotny
51de308b80 Readme
Summary: Adds the readme file.

Reviewed By: nikhilaravi

Differential Revision: D25684459

fbshipit-source-id: f1aaa621a2a67c98d5fcfe33fe9bbfea8f95b537
2021-02-02 05:45:41 -08:00
David Novotny
dc28b615ae Generation of test camera trajectories
Summary: Implements methods for generating trajectories of test cameras.

Reviewed By: nikhilaravi

Differential Revision: D26100869

fbshipit-source-id: cf2b61a34d4c749cd8cba881e97f6c388e57d1f8
2021-02-02 05:45:41 -08:00
David Novotny
9751f1f185 Main training script
Summary: Implements the training script of NeRF.

Reviewed By: nikhilaravi

Differential Revision: D25684439

fbshipit-source-id: 8b19b6dc282eb6bf6e46ec4476bb0f13a84c90dd
2021-02-02 05:45:40 -08:00
David Novotny
5b74911881 NeRF training stats logger.
Summary: Implements the `Stats` class that handles logging of the training statistics.

Reviewed By: nikhilaravi

Differential Revision: D25684430

fbshipit-source-id: 920a1c65917ab5d047988494d92173da60cfd64b
2021-02-02 05:45:40 -08:00
David Novotny
0666848338 NeRF dataloader.
Summary: Implements the dataloader for NeRF.

Reviewed By: nikhilaravi

Differential Revision: D25684424

fbshipit-source-id: 4f7092ce23135bd418186833a087e243433babc7
2021-02-02 05:45:40 -08:00
David Novotny
eb908487b8 Radiance field renderer
Summary: Implements the main NeRF model class that controls the radiance field and its renderer

Reviewed By: nikhilaravi

Differential Revision: D25684419

fbshipit-source-id: fae45572daa6748c6234bd212f3e68110f778238
2021-02-02 05:45:39 -08:00
David Novotny
bf633ab556 Implicit function
Summary: Implements the radiance field function of NeRF

Reviewed By: nikhilaravi

Differential Revision: D25684413

fbshipit-source-id: 4bf6dd5d22e6134a09f7b9f314536ec16670f737
2021-02-02 05:45:39 -08:00
David Novotny
1e82341da7 Harmonic embedding
Summary: Implements the positional embedding of NeRF

Reviewed By: nikhilaravi

Differential Revision: D25684406

fbshipit-source-id: 9f3b657babacff48bd6a0497d7a859607ffa5f89
2021-02-02 05:45:39 -08:00
David Novotny
7cbda3ec17 NeRF Raysampler
Summary: Implements the NeRF raysampler.

Reviewed By: nikhilaravi

Differential Revision: D25684403

fbshipit-source-id: 616a60f047c79479f60a6a75d214f87cbfb06d28
2021-02-02 05:45:39 -08:00
David Novotny
fba419b7f7 NeRF Raymarcher
Summary: An initial NeRF diff which sets up the folder structure and implements the raymarching algorithm of NeRF.

Reviewed By: nikhilaravi

Differential Revision: D25623990

fbshipit-source-id: ac6b05a9b866358bd4bbf44858f06859d8a6ebd1
2021-02-02 05:45:39 -08:00