examples and docs.

Summary: This diff updates the documentation and tutorials with information about the new pulsar backend. For more information about the pulsar backend, see the release notes and the paper (https://arxiv.org/abs/2004.07484). For information on how to use the backend, see the point cloud rendering notebook and the examples in the folder docs/examples.

Reviewed By: nikhilaravi

Differential Revision: D24498129

fbshipit-source-id: e312b0169a72b13590df6e4db36bfe6190d742f9

docs/tutorials/render_colored_points.ipynb

@@ -73,6 +73,7 @@
     "    FoVOrthographicCameras, \n",
     "    PointsRasterizationSettings,\n",
     "    PointsRenderer,\n",
+    "    PulsarPointsRenderer,\n",
     "    PointsRasterizer,\n",
     "    AlphaCompositor,\n",
     "    NormWeightedCompositor\n",
@@ -169,10 +170,11 @@
     "\n",
     "# Create a points renderer by compositing points using an alpha compositor (nearer points\n",
     "# are weighted more heavily). See [1] for an explanation.\n",
+    "rasterizer = PointsRasterizer(cameras=cameras, raster_settings=raster_settings)\n",
     "renderer = PointsRenderer(\n",
-    "    rasterizer=PointsRasterizer(cameras=cameras, raster_settings=raster_settings),\n",
+    "    rasterizer=rasterizer,\n",
     "    compositor=AlphaCompositor()\n",
-    ")\n"
+    ")"
    ]
   },
   {
@@ -202,12 +204,13 @@
    "outputs": [],
    "source": [
     "renderer = PointsRenderer(\n",
-    "    rasterizer=PointsRasterizer(cameras=cameras, raster_settings=raster_settings),\n",
+    "    rasterizer=rasterizer,\n",
     "    # Pass in background_color to the alpha compositor, setting the background color \n",
     "    # to the 3 item tuple, representing rgb on a scale of 0 -> 1, in this case blue\n",
     "    compositor=AlphaCompositor(background_color=(0, 0, 1))\n",
     ")\n",
     "images = renderer(point_cloud)\n",
+    "\n",
     "plt.figure(figsize=(10, 10))\n",
     "plt.imshow(images[0, ..., :3].cpu().numpy())\n",
     "plt.grid(\"off\")\n",
@@ -288,6 +291,39 @@
     "plt.axis(\"off\");"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Using the pulsar backend\n",
+    "\n",
+    "Switching to the pulsar backend is easy! The pulsar backend has a compositor built-in, so the `compositor` argument is not required when creating it (a warning will be displayed if you provide it nevertheless). It pre-allocates memory on the rendering device, that's why it needs the `n_channels` at construction time.\n",
+    "\n",
+    "All parameters for the renderer forward function are batch-wise except the background color (in this example, `gamma`) and you have to provide as many values as you have examples in your batch. The background color is optional and by default set to all zeros. You can find a detailed explanation of how gamma influences the rendering function here in the paper [Fast Differentiable Raycasting for Neural Rendering using\n",
+    "Sphere-based Representations](https://arxiv.org/pdf/2004.07484.pdf).\n",
+    "\n",
+    "You can also use the `native` backend for the pulsar backend which already provides access to point opacity. The native backend can be imported from `pytorch3d.renderer.points.pulsar`; you can find examples for this in the folder `docs/examples`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "renderer = PulsarPointsRenderer(\n",
+    "    rasterizer=PointsRasterizer(cameras=cameras, raster_settings=raster_settings),\n",
+    "    n_channels=4\n",
+    ").to(device)\n",
+    "\n",
+    "images = renderer(point_cloud, gamma=(1e-4,),\n",
+    "                  bg_col=torch.tensor([0.0, 1.0, 0.0, 1.0], dtype=torch.float32, device=device))\n",
+    "plt.figure(figsize=(10, 10))\n",
+    "plt.imshow(images[0, ..., :3].cpu().numpy())\n",
+    "plt.grid(\"off\")\n",
+    "plt.axis(\"off\");"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -412,9 +448,9 @@
    "bento/extensions/theme/main.css": true
   },
   "kernelspec": {
-   "display_name": "pytorch3d_etc (local)",
+   "display_name": "Python 3",
    "language": "python",
-   "name": "pytorch3d_etc_local"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -426,7 +462,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.5+"
+   "version": "3.6.8"
   }
  },
  "nbformat": 4,