423A35C7/pytorch3d
1
0
Fork 0
mirror of https://github.com/facebookresearch/pytorch3d.git synced 2025-12-22 23:30:35 +08:00
Code Issues Packages Projects Releases Wiki Activity

spelling

Browse Source 
Summary: Collection of spelling things, mostly in docs / tutorials.

Reviewed By: gkioxari

Differential Revision: D26101323

fbshipit-source-id: 652f62bc9d71a4ff872efa21141225e43191353a
This commit is contained in:
Jeremy Reizenstein
2021-04-09 09:57:55 -07:00
committed by Facebook GitHub Bot
parent c2e62a5087
commit 124bb5e391
75 changed files with 220 additions and 217 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
docs/tutorials/fit_simple_neural_radiance_field.ipynb
View File

@@ -17,7 +17,7 @@
"\n",
"This tutorial shows how to fit Neural Radiance Field given a set of views of a scene using differentiable implicit function rendering.\n",
"\n",
"More specificially, this tutorial will explain how to:\n",
"More specifically, this tutorial will explain how to:\n",
"1. Create a differentiable implicit function renderer with either image-grid or Monte Carlo ray sampling.\n",
"2. Create an Implicit model of a scene.\n",
"3. Fit the implicit function (Neural Radiance Field) based on input images using the differentiable implicit renderer. \n",
@@ -158,9 +158,9 @@
"The following initializes an implicit renderer that emits a ray from each pixel of a target image and samples a set of uniformly-spaced points along the ray. At each ray-point, the corresponding density and color value is obtained by querying the corresponding location in the neural model of the scene (the model is described & instantiated in a later cell).\n",
"\n",
"The renderer is composed of a *raymarcher* and a *raysampler*.\n",
"- The *raysampler* is responsible for emiting rays from image pixels and sampling the points along them. Here, we use two different raysamplers:\n",
"- The *raysampler* is responsible for emitting rays from image pixels and sampling the points along them. Here, we use two different raysamplers:\n",
" - `MonteCarloRaysampler` is used to generate rays from a random subset of pixels of the image plane. The random subsampling of pixels is carried out during **training** to decrease the memory consumption of the implicit model.\n",
" - `NDCGridRaysampler` which follows the standard PyTorch3d coordinate grid convention (+X from right to left; +Y from bottom to top; +Z away from the user). In combination with the implicit model of the scene, `NDCGridRaysampler` consumes a large amount of memory and, hence, is only used for visualizing the results of the training at **test** time.\n",
" - `NDCGridRaysampler` which follows the standard PyTorch3D coordinate grid convention (+X from right to left; +Y from bottom to top; +Z away from the user). In combination with the implicit model of the scene, `NDCGridRaysampler` consumes a large amount of memory and, hence, is only used for visualizing the results of the training at **test** time.\n",
"- The *raymarcher* takes the densities and colors sampled along each ray and renders each ray into a color and an opacity value of the ray's source pixel. Here we use the `EmissionAbsorptionRaymarcher` which implements the standard Emission-Absorption raymarching algorithm."
]
},
@@ -186,7 +186,7 @@
"# 1) Instantiate the raysamplers.\n",
"\n",
"# Here, NDCGridRaysampler generates a rectangular image\n",
"# grid of rays whose coordinates follow the PyTorch3d\n",
"# grid of rays whose coordinates follow the PyTorch3D\n",
"# coordinate conventions.\n",
"raysampler_grid = NDCGridRaysampler(\n",
" image_height=render_size,\n",
@@ -236,7 +236,7 @@
"\n",
"The `forward` function of `NeuralRadianceField` (NeRF) receives as input a set of tensors that parametrize a bundle of rendering rays. The ray bundle is later converted to 3D ray points in the world coordinates of the scene. Each 3D point is then mapped to a harmonic representation using the `HarmonicEmbedding` layer (defined in the next cell). The harmonic embeddings then enter the _color_ and _opacity_ branches of the NeRF model in order to label each ray point with a 3D vector and a 1D scalar ranging in [0-1] which define the point's RGB color and opacity respectively.\n",
"\n",
"Since NeRF has a large memory footprint, we also implement the `NeuralRadianceField.forward_batched` method. The method splits the input rays into batches and executes the `forward` function for each batch separately in a for loop. This allows to render a large set of rays without running out of GPU memory. Standardly, `forward_batched` would be used to render rays emitted from all pixels of an image in order to produce a full-sized render of a scene.\n"
"Since NeRF has a large memory footprint, we also implement the `NeuralRadianceField.forward_batched` method. The method splits the input rays into batches and executes the `forward` function for each batch separately in a for loop. This lets us render a large set of rays without running out of GPU memory. Standardly, `forward_batched` would be used to render rays emitted from all pixels of an image in order to produce a full-sized render of a scene.\n"
]
},
{
@@ -266,7 +266,7 @@
" ]\n",
" \n",
" Note that `x` is also premultiplied by `omega0` before\n",
" evaluting the harmonic functions.\n",
" evaluating the harmonic functions.\n",
" \"\"\"\n",
" super().__init__()\n",
" self.register_buffer(\n",
@@ -417,7 +417,7 @@
"\n",
" Returns:\n",
" rays_densities: A tensor of shape `(minibatch, ..., num_points_per_ray, 1)`\n",
" denoting the opacitiy of each ray point.\n",
" denoting the opacity of each ray point.\n",
" rays_colors: A tensor of shape `(minibatch, ..., num_points_per_ray, 3)`\n",
" denoting the color of each ray point.\n",
" \"\"\"\n",
@@ -457,11 +457,11 @@
" This function is used to allow for memory efficient processing\n",
" of input rays. The input rays are first split to `n_batches`\n",
" chunks and passed through the `self.forward` function one at a time\n",
" in a for loop. Combined with disabling Pytorch gradient caching\n",
" in a for loop. Combined with disabling PyTorch gradient caching\n",
" (`torch.no_grad()`), this allows for rendering large batches\n",
" of rays that do not all fit into GPU memory in a single forward pass.\n",
" In our case, batched_forward is used to export a fully-sized render\n",
" of the radiance field for visualisation purposes.\n",
" of the radiance field for visualization purposes.\n",
" \n",
" Args:\n",
" ray_bundle: A RayBundle object containing the following variables:\n",
@@ -477,7 +477,7 @@
"\n",
" Returns:\n",
" rays_densities: A tensor of shape `(minibatch, ..., num_points_per_ray, 1)`\n",
" denoting the opacitiy of each ray point.\n",
" denoting the opacity of each ray point.\n",
" rays_colors: A tensor of shape `(minibatch, ..., num_points_per_ray, 3)`\n",
" denoting the color of each ray point.\n",
"\n",
@@ -576,12 +576,12 @@
" intermediate results of the learning. \n",
" \n",
" Since the `NeuralRadianceField` suffers from\n",
" a large memory footprint, which does not allow to\n",
" a large memory footprint, which does not let us\n",
" render the full image grid in a single forward pass,\n",
" we utilize the `NeuralRadianceField.batched_forward`\n",
" function in combination with disabling the gradient caching.\n",
" This chunks the set of emitted rays to batches and \n",
" evaluates the implicit function on one-batch at a time\n",
" evaluates the implicit function on one batch at a time\n",
" to prevent GPU memory overflow.\n",
" \"\"\"\n",
" \n",
@@ -720,7 +720,7 @@
" rendered_images_silhouettes.split([3, 1], dim=-1)\n",
" )\n",
" \n",
" # Compute the silhoutte error as the mean huber\n",
" # Compute the silhouette error as the mean huber\n",
" # loss between the predicted masks and the\n",
" # sampled target silhouettes.\n",
" silhouettes_at_rays = sample_images_at_mc_locs(\n",
@@ -818,7 +818,7 @@
" fov=target_cameras.fov[0],\n",
" device=device,\n",
" )\n",
" # Note that we again render with `NDCGridSampler`\n",
" # Note that we again render with `NDCGridRaySampler`\n",
" # and the batched_forward function of neural_radiance_field.\n",
" frames.append(\n",
" renderer_grid(\n",
@@ -841,7 +841,7 @@
"source": [
"## 6. Conclusion\n",
"\n",
"In this tutorial, we have shown how to optimize an implicit representation of a scene such that the renders of the scene from known viewpoints match the observed images for each viewpoint. The rendering was carried out using the Pytorch3D's implicit function renderer composed of either a `MonteCarloRaysampler` or `NDCGridRaysampler`, and an `EmissionAbsorptionRaymarcher`."
"In this tutorial, we have shown how to optimize an implicit representation of a scene such that the renders of the scene from known viewpoints match the observed images for each viewpoint. The rendering was carried out using the PyTorch3D's implicit function renderer composed of either a `MonteCarloRaysampler` or `NDCGridRaysampler`, and an `EmissionAbsorptionRaymarcher`."
]
}
],