997 Commits

Author SHA1 Message Date
Jeremy Reizenstein
a2c6af9250 more code blocks for readthedocs
Summary: More rst syntax fixes

Reviewed By: davidsonic

Differential Revision: D40977328

fbshipit-source-id: a3a3accbf2ba7cd9c84a0a82d0265010764a9d61
2022-11-29 03:13:40 -08:00
David Novotny
94f321fa3d render_flyaround bugfix
Summary: Fixes a bug which would crash render_flyaround anytime  visualize_preds_keys is adjusted

Reviewed By: shapovalov

Differential Revision: D41124462

fbshipit-source-id: 127045a91a055909f8bd56c8af81afac02c00f60
2022-11-28 04:36:41 -08:00
David Novotny
35f8cb9430 Downgrade "Assigning param_group " msg to DEBUG
Summary: <See title>

Reviewed By: bottler

Differential Revision: D41534524

fbshipit-source-id: 9c39198b9b8d5fc95f857b03ad39bfe0bd720cbb
2022-11-28 02:58:15 -08:00
David Novotny
c3a6ab02da Bug-fix visualize reconstruction
Summary:
Addresses the following issue:
https://github.com/facebookresearch/pytorch3d/issues/1345#issuecomment-1272881244

I.e., when installed from conda, `pytorch3d_implicitron_visualizer` crashes since it invokes `main()` while `main` requires a single positional arg `argv`.

Reviewed By: shapovalov

Differential Revision: D41533497

fbshipit-source-id: e53a923eb8b2f0f9c0e92e9c0866d9cb310c4799
2022-11-25 10:30:01 -08:00
Jeremy Reizenstein
60ab1cdb72 make x_enabled compulsory
Summary: Optional[some_configurable] won't autogenerate the enabled flag

Reviewed By: shapovalov

Differential Revision: D41522104

fbshipit-source-id: 555ff6b343faf6f18aad2f92fbb7c341f5e991c6
2022-11-24 09:38:02 -08:00
Jiali Duan
1706eb8216 Simplify MC C++ hashing logic
Summary: To be consistent with CUDA hashing, the diff replaces boost hasher with a simplified hasher for storing unique global edge_ids.

Reviewed By: kjchalup

Differential Revision: D41140382

fbshipit-source-id: 2ce598e5edcf6369fe13bd15d1f5e014b252027b
2022-11-15 19:42:04 -08:00
Jiali Duan
8b8291830e Marching Cubes cuda extension
Summary:
Torch CUDA extension for Marching Cubes
- MC involving 3 steps:
  - 1st forward pass to collect vertices and occupied state for each voxel
  - Compute compactVoxelArray to skip non-empty voxels
  - 2nd pass to genereate interpolated vertex positions and faces by marching through the grid
- In contrast to existing MC:
   - Bind each interpolated vertex with a global edge_id to address floating-point precision
   - Added deduplication process to remove redundant vertices and faces

Benchmarks (ms):

| N / V(^3)      | python          | C++             |   CUDA   | Speedup |
| 2 / 20          |    12176873  |       24338     |     4363   | 2790x/5x|
| 1 / 100          |     -             |    3070511     |   27126   |    113x    |
| 2 / 100          |     -             |    5968934     |   53129   |    112x    |
| 1 / 256          |     -             |  61278092     | 430900   |    142x    |
| 2 / 256          |     -             |125687930     | 856941   |    146x   |

Reviewed By: kjchalup

Differential Revision: D39644248

fbshipit-source-id: d679c0c79d67b98b235d12296f383d760a00042a
2022-11-15 19:42:04 -08:00
Jeremy Reizenstein
9a0b0c2e74 renderer and vis readthedocs #1363
Summary: Autogenerate docs for the renderer too. This will be helpful but make a slightly ugly TOC

Reviewed By: kjchalup

Differential Revision: D40977315

fbshipit-source-id: 10831de3ced68080cb5671c5dc31d4da8500f761
2022-11-15 14:10:22 -08:00
Daniel L. Lu
d0fbe2cb63 fix typo pucuda.gl --> pycuda.gl (#1379)
Summary:
Every time I try to run code, I get this warning:

```
  warnings.warn("Can't import pucuda.gl, not importing MeshRasterizerOpenGL.")
```

Of course, `pucuda` is a typo of `pycuda`.

This PR fixes the typo

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1379

Reviewed By: kjchalup

Differential Revision: D41295562

Pulled By: bottler

fbshipit-source-id: 2bfa2a2dbe20a5347861d36fbff5094994c1253d
2022-11-15 12:02:44 -08:00
Roman Shapovalov
719c33a7f9 Fix: visualisation with Enums fields in Configurables
Summary:
Enum fields cause the following to crash since they are loaded as strings:
```
config = OmegaConf.load(autodumped_cfg_file)
Experiment(**config)
```

It would be good to come up with the general solution but for now just fixing the visualisation script.

Reviewed By: bottler

Differential Revision: D41140426

fbshipit-source-id: 71c1c6b1fffe3b5ab1ca0114cfa3f0d81160278f
2022-11-08 15:58:41 -08:00
Roman Shapovalov
f3c1e0837c MC rasterize supports heterogeneous bundle; refactoring of bundle-to-padded
Summary:
Rasterize MC was not adapted to heterogeneous bundles.

There are some caveats though:
1) on CO3D, we get up to 18 points per image, which is too few for a reasonable visualisation (see below);
2) rasterising for a batch of 100 is slow.

I also moved the unpacking code close to the bundle to be able to reuse it.

{F789678778}

Reviewed By: bottler, davnov134

Differential Revision: D41008600

fbshipit-source-id: 9f10f1f9f9a174cf8c534b9b9859587d69832b71
2022-11-07 13:43:31 -08:00
Jeremy Reizenstein
7be49bf46f allow dots in param_groups
Summary:
Allow a module's param_group member to specify overrides to the param groups of its members or their members.
Also logging for param group assignments.

This allows defining `params.basis_matrix` in the param_groups of a voxel_grid.

Reviewed By: shapovalov

Differential Revision: D41080667

fbshipit-source-id: 49f3b0e5b36e496f78701db0699cbb8a7e20c51e
2022-11-07 06:41:40 -08:00
Jeremy Reizenstein
a1f2ded58a voxel_grid_implicit_function scaffold fixes
Summary: Fix indexing of directions after filtering of points by scaffold.

Reviewed By: shapovalov

Differential Revision: D40853482

fbshipit-source-id: 9cfdb981e97cb82edcd27632c5848537ed2c6837
2022-11-03 05:46:31 -07:00
David Novotny
e4a3298149 CO3Dv2 multi-category extension
Summary:
Allows loading of multiple categories.
Multiple categories are provided in a comma-separated list of category names.

Reviewed By: bottler, shapovalov

Differential Revision: D40803297

fbshipit-source-id: 863938be3aa6ffefe9e563aede4a2e9e66aeeaa8
2022-11-02 13:55:25 -07:00
Jeremy Reizenstein
c54e048666 more readthedocs
Summary: Quote formats, spelling

Reviewed By: shapovalov

Differential Revision: D40913734

fbshipit-source-id: d6dea65d5204b3c463c656a07ef9b447b7be6a0a
2022-11-02 05:19:15 -07:00
David Novotny
f7ac7b604a readthedocs fixes
Summary: Fixes readthedocs. Sphinx build looks good.

Reviewed By: bottler

Differential Revision: D40893196

fbshipit-source-id: bf00384b921d4ef54e64745ed39172358c2f9bb3
2022-11-01 05:46:13 -07:00
Jeremy Reizenstein
cd113efd98 Add implicitron to readthedocs
Summary: Try to document implicitron. Most of this is autogenerated.

Reviewed By: shapovalov

Differential Revision: D40623742

fbshipit-source-id: 453508277903b7d987b1703656ba1ee09bc2c570
2022-10-31 14:44:41 -07:00
Jeremy Reizenstein
322c8f2f50 make lpips optional
Summary: So that lpips isn't needed in readthedocs

Reviewed By: shapovalov

Differential Revision: D40859351

fbshipit-source-id: 58933bd3b71e78e658fbce56c3663b11ee2331f6
2022-10-31 14:44:41 -07:00
David Novotny
eff0aad15a Bugfix - normalize emitted ray directions after substraction
Summary: The bug lead to non-coinciding origins of the rays emitted from perspective cameras when unit_directions=True

Reviewed By: bottler

Differential Revision: D40865610

fbshipit-source-id: 398598e9e919b53e6bea179f0400e735bbb5b625
2022-10-31 14:00:49 -07:00
David Novotny
bea84a6fcd voxel_grid_if -> remove union type from the settings.
Summary: see title

Reviewed By: shapovalov

Differential Revision: D40803670

fbshipit-source-id: 211189167837af577d6502a698e2f3fb3aec3e30
2022-10-31 11:37:43 -07:00
Roman Shapovalov
f711c4bfe9 Fix parameters not wrapped with nn.Parameter, antialiasing compatibility
Summary: Some things fail if a parameter is not wraped; in particular, it prevented other tensors moving to GPU.

Reviewed By: bottler

Differential Revision: D40819932

fbshipit-source-id: a23b38ceacd7f0dc131cb0355fef1178e3e2f7fd
2022-10-31 01:43:00 -07:00
Jeremy Reizenstein
88620b6847 attempt to fix doc build #1363
Summary: installing from git: is failing

Reviewed By: shapovalov

Differential Revision: D40635668

fbshipit-source-id: 604ba5425e14caeabe4e178bf7f851f2163419bd
2022-10-27 07:26:36 -07:00
generatedunixname89002005307016
db7c80bf76 upgrade pyre version in fbcode/vision - batch 2
Differential Revision: D40695068

fbshipit-source-id: 9944a872d93a9dac348b252a0f8bf9c7e6f929f3
2022-10-25 22:42:06 -07:00
Jeremy Reizenstein
995b60e3b9 version 0.7.1
Summary: update version number

Reviewed By: shapovalov

Differential Revision: D40622583

fbshipit-source-id: 87fa55e1b02fc114f65ee8a5c3d998ba50226ab1
v0.7.1
2022-10-23 07:16:40 -07:00
Jeremy Reizenstein
ca58863347 yaml test fix
Summary: Yaml bool case fix

Reviewed By: shapovalov

Differential Revision: D40623031

fbshipit-source-id: 29b2fba171c2cbebfa03834e38b614d07275c997
2022-10-23 07:09:26 -07:00
Jeremy Reizenstein
74754bbf17 voxel_grid_implicit_function
Reviewed By: shapovalov

Differential Revision: D40622304

fbshipit-source-id: 277515a55c46d9b8300058b439526539a7fe00a0
2022-10-23 05:36:34 -07:00
Jeremy Reizenstein
611aba9a20 replicate_last_interval in raymarcher
Summary: Add option to flat pad the last delta. Might to help when training on rgb only.

Reviewed By: shapovalov

Differential Revision: D40587475

fbshipit-source-id: c763fa38948600ea532c730538dc4ff29d2c3e0a
2022-10-23 02:47:09 -07:00
Jeremy Reizenstein
ff933ab82b make visdom optional
Summary: Make Implicitron run without visdom installed.

Reviewed By: shapovalov

Differential Revision: D40587974

fbshipit-source-id: dc319596c7a4d10a4c54c556dabc89ad9d25c2fb
2022-10-22 15:51:22 -07:00
Jiali Duan
46cb5aaaae Omit _check_valid_rotation_matrix by default
Summary:
According to the profiler trace D40326775, _check_valid_rotation_matrix is slow because of aten::all_close operation and _safe_det_3x3 bottlenecks. Disable the check by default unless environment variable PYTORCH3D_CHECK_ROTATION_MATRICES is set to 1.

Comparison after applying the change:
```
Profiling/Function    get_world_to_view (ms)   Transform_points(ms)    specular(ms)
before                12.751                    18.577                  21.384
after                 4.432 (34.7%)             9.248 (49.8%)           11.507 (53.8%)
```

Profiling trace:
https://pxl.cl/2h687
More details in https://docs.google.com/document/d/1kfhEQfpeQToikr5OH9ZssM39CskxWoJ2p8DO5-t6eWk/edit?usp=sharing

Reviewed By: kjchalup

Differential Revision: D40442503

fbshipit-source-id: 954b58de47de235c9d93af441643c22868b547d0
2022-10-20 16:05:22 -07:00
Jeremy Reizenstein
8339cf2610 hydra/OC.structured: don't hide error
Summary: Keep the cause of hydra errors visible in some more cases.

Reviewed By: shapovalov

Differential Revision: D40516202

fbshipit-source-id: 8d214be5cc808a37738add77cc305fe099788546
2022-10-20 03:43:05 -07:00
Jeremy Reizenstein
9535c576e0 test fix for param_groups
Summary: param_groups only expected on MLPDecoder, not ElementwiseDecoder

Reviewed By: shapovalov

Differential Revision: D40508539

fbshipit-source-id: ea040ad6f7e26bd7d87e5de2eaadae2cf4b04faf
2022-10-19 04:08:30 -07:00
Jeremy Reizenstein
fe5bdb2fb5 different learning rate for different parts
Summary:
Adds the ability to have different learning rates for different parts of the model. The trainable parts of the implicitron have a new member

       param_groups: dictionary where keys are names of individual parameters,
            or module’s members and values are the parameter group where the
            parameter/member will be sorted to. "self" key is used to denote the
            parameter group at the module level. Possible keys, including the "self" key
            do not have to be defined. By default all parameters are put into "default"
            parameter group and have the learning rate defined in the optimizer,
            it can be overriden at the:
                - module level with “self” key, all the parameters and child
                    module s parameters will be put to that parameter group
                - member level, which is the same as if the `param_groups` in that
                    member has key=“self” and value equal to that parameter group.
                    This is useful if members do not have `param_groups`, for
                    example torch.nn.Linear.
                - parameter level, parameter with the same name as the key
                    will be put to that parameter group.

And in the optimizer factory, parameters and their learning rates are recursively gathered.

Reviewed By: shapovalov

Differential Revision: D40145802

fbshipit-source-id: 631c02b8d79ee1c0eb4c31e6e42dbd3d2882078a
2022-10-18 15:58:18 -07:00
Jeremy Reizenstein
a819ecb00b MLP last layer config
Summary:
Added initialization configuration for the last layer of the MLP decoding function. You can now set:
- last activation function (tensorf uses sigmoid)
- last bias init (tensorf uses 0, because of sigmoid ofc)
- option to use xavier initialization (we use relu so this should not be set)

Reviewed By: davnov134

Differential Revision: D40304981

fbshipit-source-id: ec398eb2235164ae85cb7c09b9660e843490ea04
2022-10-18 15:58:18 -07:00
Ji Hou
a2659e1730 Update iou3d.md (#1351)
Summary:
fix a typo

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1351

Reviewed By: shapovalov

Differential Revision: D40209834

Pulled By: bottler

fbshipit-source-id: 118133e0eab2df211e5c4f04371f2c695a9ceced
2022-10-15 21:41:02 -07:00
Jeremy Reizenstein
3b3306f9b4 suppress ffmpeg output
Summary: Restore the suppression of ffmpeg output.

Reviewed By: shapovalov

Differential Revision: D40296595

fbshipit-source-id: 41b2c14b6f6245f77e0ef6cc94fa7b41fbb83e33
2022-10-13 14:56:09 -07:00
Jeremy Reizenstein
f13086779d avoid torch.range
Summary: Avoid unintended use of torch.range.

Reviewed By: kjchalup

Differential Revision: D40341396

fbshipit-source-id: 108295983afdec0ca9e43178fef9c65695150bc1
2022-10-13 14:37:46 -07:00
Jeremy Reizenstein
4d9215b3b4 fix to get_default_args(instance)
Summary:
Small config system fix. Allows get_default_args to work on an instance which has been created with a dict (instead of a DictConfig) as an args field. E.g.

```
gm = GenericModel(
        raysampler_AdaptiveRaySampler_args={"scene_extent": 4.0}
    )
    OmegaConf.structured(gm1)
```

Reviewed By: shapovalov

Differential Revision: D40341047

fbshipit-source-id: 587d0e8262e271df442a80858949a48e5d6db3df
2022-10-13 06:05:07 -07:00
Darijan Gudelj
76cddd90be Elementwise decoder
Summary: Tensorf does relu or softmax after the density grid. This diff adds the ability to replicate that.

Reviewed By: bottler

Differential Revision: D40023228

fbshipit-source-id: 9f19868cd68460af98ab6e61c7f708158c26dc08
2022-10-13 05:59:22 -07:00
Jeremy Reizenstein
a607dd063e better implicit function #channels errors
Summary: More helpful errors when the output channels aren't 1 for density and 3 for color

Reviewed By: shapovalov

Differential Revision: D40341088

fbshipit-source-id: 6074bf7fefe11c8e60fee4db2760b776419bcfee
2022-10-13 05:43:36 -07:00
Krzysztof Chalupka
17bc043a0c Remove structured binding
Summary: Couldn't build p3d on devfair because C++17 is unsupported. Two structured bindings sneaked in.

Reviewed By: bottler

Differential Revision: D40280967

fbshipit-source-id: 9627f3f9f76247a6cefbeac067fdead67c6f4e14
2022-10-12 10:34:21 -07:00
Darijan Gudelj
f55d37f07d volume cropping
Summary:
TensoRF at step 2000 does volume croping and resizing.
At those steps it calculates part of the voxel grid which has density big enough to have objects and resizes the grid to fit that object.
Change is done on 3 levels:
- implicit function subscribes to epochs and at specific epochs finds the bounding box of the object and calls resizing of the color and density voxel grids to fit it
- VoxelGrid module calls cropping of the underlaying voxel grid and resizing to fit previous size it also adjusts its extends and translation to match wanted size
- Each voxel grid has its own way of cropping the underlaying data

Reviewed By: kjchalup

Differential Revision: D39854548

fbshipit-source-id: 5435b6e599aef1eaab980f5421d3369ee4829c50
2022-10-12 08:31:51 -07:00
Jeremy Reizenstein
0b5def5257 avoid numpy warning in split
Summary:
avoid creating a numpy array of random things just to split it: this can now generate a warning e.g. if the list contains lists of varying lengths. There might also be a performance win here, and we could do more of the same if we care about that.

(The vanilla way to avoid the new warning is to replace `np.split(a,` with `np.split(np.array(a, dtype=object), ` btw.)

Reviewed By: shapovalov

Differential Revision: D40209308

fbshipit-source-id: daae33a23ceb444e8e7241f72ce1525593e2f239
2022-10-11 14:49:51 -07:00
Darijan Gudelj
56d3465b09 scaffold
Summary: Forward method is sped up using the scaffold, a low resolution voxel grid which is used to filter out the points in empty space. These points will be predicted as having 0 density and (0, 0, 0) color. The points which were not evaluated as empty space will be passed through the steps outlined above.

Reviewed By: kjchalup

Differential Revision: D39579671

fbshipit-source-id: 8eab8bb43ef77c2a73557efdb725e99a6c60d415
2022-10-10 11:01:00 -07:00
Jeremy Reizenstein
95a2acf763 Co3Dv2 point cloud fix
Summary: Avoid certain hardcoded paths in co3dv2 data

Reviewed By: davnov134

Differential Revision: D40209309

fbshipit-source-id: 0e83a15baa47d5bd07d2d23c6048cb4522c1ccba
2022-10-09 05:06:49 -07:00
Jiali Duan
9df875bb5e Fix Circleci build failure: Add != operator for Marching Cubes Vertex struct
Summary: Fix Circleci error: https://app.circleci.com/pipelines/github/facebookresearch/pytorch3d/1066/workflows/94df137b-4882-4959-8fe4-738af447db23/jobs/56560.

Reviewed By: kjchalup

Differential Revision: D40185409

fbshipit-source-id: 7121b0cae66bd60f718df2a5d9ef5d2ac3bc658c
2022-10-07 12:14:24 -07:00
David Novotny
73ba66e4ab Bugfix in reduction feature aggr.
Summary: Bugfix

Reviewed By: bottler

Differential Revision: D40146840

fbshipit-source-id: a8415c361ed3cd939999b87311aff4d3bb1bcfe1
2022-10-07 01:45:24 -07:00
Kyle Hundman
4c8338b00f Improve memory efficiency in VolumeSampler
Summary: Avoids use of `torch.cat` operation when rendering a volume by instead issuing multiple calls to `torch.nn.functional.grid_sample`. Density and color tensors can be large.

Reviewed By: bottler

Differential Revision: D40072399

fbshipit-source-id: eb4cd34f6171d54972bbf2877065f973db497de0
2022-10-06 12:10:26 -07:00
Jiali Duan
0d8608b9f9 Marching Cubes C++ torch extension
Summary:
Torch C++ extension for Marching Cubes

- Add torch C++ extension for marching cubes. Observe a speed up of ~255x-324x speed up (over varying batch sizes and spatial resolutions)

- Add C++ impl in existing unit-tests.

(Note: this ignores all push blocking failures!)

Reviewed By: kjchalup

Differential Revision: D39590638

fbshipit-source-id: e44d2852a24c2c398e5ea9db20f0dfaa1817e457
2022-10-06 11:13:53 -07:00
Jiali Duan
850efdf706 Python marching cubes improvements
Summary: Overhaul of marching_cubes_naive for better performance and to avoid relying on unstable hashing. In particular, instead of hashing vertex positions, we index each interpolated vertex with its corresponding edge in the 3d grid.

Reviewed By: kjchalup

Differential Revision: D39419642

fbshipit-source-id: b5fede3525c545d1d374198928dfb216262f0ec0
2022-10-06 11:08:49 -07:00
Gavin Peng
6471893f59 Multithread CPU naive mesh rasterization
Summary:
Threaded the for loop:
```
for (int yi = 0; yi < H; ++yi) {...}
```
in function `RasterizeMeshesNaiveCpu()`.
Chunk size is approx equal.

Reviewed By: bottler

Differential Revision: D40063604

fbshipit-source-id: 09150269405538119b0f1b029892179501421e68
2022-10-06 06:42:58 -07:00