Add utils to approximate the conical frustums as multivariate gaussians.

Summary:
Introduce methods to approximate the radii of conical frustums along rays as described in [MipNerf](https://arxiv.org/abs/2103.13415):

- Two new attributes are added to ImplicitronRayBundle: bins and radii. Bins is of size n_pts_per_ray + 1. It allows us to manipulate easily and n_pts_per_ray intervals. For example we need the intervals coordinates in the radii computation for \(t_{\mu}, t_{\delta}\). Radii are used to store the radii of the conical frustums.

- Add 3 new methods to compute the radii:
   - approximate_conical_frustum_as_gaussians: It computes the mean along the ray direction, the variance of the
      conical frustum  with respect to t and variance of the conical frustum with respect to its radius. This
      implementation follows the stable computation defined in the paper.
   - compute_3d_diagonal_covariance_gaussian: Will leverage the two previously computed variances to find the
     diagonal covariance of the Gaussian.
   - conical_frustum_to_gaussian: Mix everything together to compute the means and the diagonal covariances along
     the ray of the Gaussians.

- In AbstractMaskRaySampler, introduces the attribute `cast_ray_bundle_as_cone`. If False it won't change the previous behaviour of the RaySampler. However if True, the samplers will sample `n_pts_per_ray +1` instead of `n_pts_per_ray`. This points are then used to set the bins attribute of ImplicitronRayBundle. The support of HeterogeneousRayBundle has not been added since the current code does not allow it. A safeguard has been added to avoid a silent bug in the future.

Reviewed By: shapovalov

Differential Revision: D45269190

fbshipit-source-id: bf22fad12d71d55392f054e3f680013aa0d59b78
This commit is contained in:
Emilien Garreau
2023-07-06 01:55:41 -07:00
committed by Facebook GitHub Bot
parent 4e7715ce66
commit 29b8ebd802
10 changed files with 977 additions and 65 deletions

View File

@@ -216,6 +216,7 @@ model_factory_ImplicitronModelFactory_args:
n_rays_total_training: null
stratified_point_sampling_training: true
stratified_point_sampling_evaluation: false
cast_ray_bundle_as_cone: false
scene_extent: 8.0
scene_center:
- 0.0
@@ -228,6 +229,7 @@ model_factory_ImplicitronModelFactory_args:
n_rays_total_training: null
stratified_point_sampling_training: true
stratified_point_sampling_evaluation: false
cast_ray_bundle_as_cone: false
min_depth: 0.1
max_depth: 8.0
renderer_LSTMRenderer_args:
@@ -642,6 +644,7 @@ model_factory_ImplicitronModelFactory_args:
n_rays_total_training: null
stratified_point_sampling_training: true
stratified_point_sampling_evaluation: false
cast_ray_bundle_as_cone: false
scene_extent: 8.0
scene_center:
- 0.0
@@ -654,6 +657,7 @@ model_factory_ImplicitronModelFactory_args:
n_rays_total_training: null
stratified_point_sampling_training: true
stratified_point_sampling_evaluation: false
cast_ray_bundle_as_cone: false
min_depth: 0.1
max_depth: 8.0
renderer_LSTMRenderer_args: