camera refactoring
Summary: Refactor cameras * CamerasBase was enhanced with `transform_points_screen` that transforms projected points from NDC to screen space * OpenGLPerspective, OpenGLOrthographic -> FoVPerspective, FoVOrthographic * SfMPerspective, SfMOrthographic -> Perspective, Orthographic * PerspectiveCamera can optionally be constructred with screen space parameters * Note on Cameras and coordinate systems was added Reviewed By: nikhilaravi Differential Revision: D23168525 fbshipit-source-id: dd138e2b2cc7e0e0d9f34c45b8251c01266a2063
							
								
								
									
										63
									
								
								docs/notes/cameras.md
									
									
									
									
									
										Normal file
									
								
							
							
						
						@ -0,0 +1,63 @@
 | 
			
		||||
# Cameras
 | 
			
		||||
 | 
			
		||||
## Camera Coordinate Systems
 | 
			
		||||
 | 
			
		||||
When working with 3D data, there are 4 coordinate systems users need to know
 | 
			
		||||
* **World coordinate system**
 | 
			
		||||
This is the system the object/scene lives - the world.
 | 
			
		||||
* **Camera view coordinate system**
 | 
			
		||||
This is the system that has its origin on the image plane and the `Z`-axis perpendicular to the image plane. In PyTorch3D, we assume that `+X` points left, and `+Y` points up and `+Z` points out from the image plane. The transformation from world to view happens after applying a rotation (`R`) and translation (`T`). 
 | 
			
		||||
* **NDC coordinate system**
 | 
			
		||||
This is the normalized coordinate system that confines in a volume the renderered part of the object/scene. Also known as view volume. Under the PyTorch3D convention, `(+1, +1, znear)` is the top left near corner, and `(-1, -1, zfar)` is the bottom right far corner of the volume. The transformation from view to NDC happens after applying the camera projection matrix (`P`).
 | 
			
		||||
* **Screen coordinate system**
 | 
			
		||||
This is another representation of the view volume with the `XY` coordinates defined in pixel space instead of a normalized space.
 | 
			
		||||
 | 
			
		||||
An illustration of the 4 coordinate systems is shown below
 | 
			
		||||

 | 
			
		||||
 | 
			
		||||
## Defining Cameras in PyTorch3D
 | 
			
		||||
 | 
			
		||||
Cameras in PyTorch3D transform an object/scene from world to NDC by first transforming the object/scene to view (via transforms `R` and `T`) and then projecting the 3D object/scene to NDC (via the projection matrix `P`, else known as camera matrix). Thus, the camera parameters in `P` are assumed to be in NDC space. If the user has camera parameters in screen space, which is a common use case, the parameters should transformed to NDC (see below for an example)
 | 
			
		||||
 | 
			
		||||
We describe the camera types in PyTorch3D and the convention for the camera parameters provided at construction time. 
 | 
			
		||||
 | 
			
		||||
### Camera Types
 | 
			
		||||
 | 
			
		||||
All cameras inherit from `CamerasBase` which is a base class for all cameras. PyTorch3D provides four different camera types. The `CamerasBase` defines methods that are common to all camera models:
 | 
			
		||||
* `get_camera_center` that returns the optical center of the camera in world coordinates
 | 
			
		||||
* `get_world_to_view_transform` which returns a 3D transform from world coordinates to the camera view coordinates (R, T)
 | 
			
		||||
* `get_full_projection_transform` which composes the projection transform (P) with the world-to-view transform (R, T)
 | 
			
		||||
* `transform_points` which takes a set of input points in world coordinates and projects to NDC coordinates ranging from [-1, -1, znear] to  [+1, +1, zfar].
 | 
			
		||||
* `transform_points_screen` which takes a set of input points in world coordinates and projects them to the screen coordinates ranging from [0, 0, znear] to [W-1, H-1, zfar] 
 | 
			
		||||
 | 
			
		||||
Users can easily customize their own cameras. For each new camera, users should implement the `get_projection_transform` routine that returns the mapping `P` from camera view coordinates to NDC coordinates.
 | 
			
		||||
 | 
			
		||||
#### FoVPerspectiveCameras, FoVOrthographicCameras
 | 
			
		||||
These two cameras follow the OpenGL convention for perspective and orthographic cameras respectively. The user provides the near `znear` and far `zfar` field which confines the view volume in the `Z` axis. The view volume in the `XY` plane is defined by field of view angle (`fov`) in the case of `FoVPerspectiveCameras` and by `min_x, min_y, max_x, max_y` in the case of `FoVOrthographicCameras`. 
 | 
			
		||||
 | 
			
		||||
#### PerspectiveCameras, OrthographicCameras
 | 
			
		||||
These two cameras follow the Multi-View Geometry convention for cameras. The user provides the focal length (`fx`, `fy`) and the principal point (`px`, `py`). For example, `camera = PerspectiveCameras(focal_length=((fx, fy),), principal_point=((px, py),))`
 | 
			
		||||
 | 
			
		||||
As mentioned above, the focal length and principal point are used to convert a point `(X, Y, Z)` from view coordinates to NDC coordinates, as follows
 | 
			
		||||
 | 
			
		||||
```
 | 
			
		||||
# for perspective
 | 
			
		||||
x_ndc = fx * X / Z + px
 | 
			
		||||
y_ndc = fy * Y / Z + py
 | 
			
		||||
z_ndc = 1 / Z
 | 
			
		||||
 | 
			
		||||
# for orthographic
 | 
			
		||||
x_ndc = fx * X + px
 | 
			
		||||
y_ndc = fy * Y + py
 | 
			
		||||
z_ndc = Z
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
Commonly, users have access to the focal length (`fx_screen`, `fy_screen`) and the principal point (`px_screen`, `py_screen`) in screen space. In that case, to construct the camera the user needs to additionally provide the `image_size = ((image_width, image_height),)`. More precisely, `camera = PerspectiveCameras(focal_length=((fx_screen, fy_screen),), principal_point=((px_screen, py_screen),), image_size = ((image_width, image_height),))`. Internally, the camera parameters are converted from screen to NDC as follows:
 | 
			
		||||
 | 
			
		||||
```
 | 
			
		||||
fx = fx_screen * 2.0 / image_width
 | 
			
		||||
fy = fy_screen * 2.0 / image_height
 | 
			
		||||
 | 
			
		||||
px = - (px_screen - image_width / 2.0) * 2.0 / image_width
 | 
			
		||||
py = - (py_screen - image_height / 2.0) * 2.0/ image_height
 | 
			
		||||
```
 | 
			
		||||
@ -39,7 +39,7 @@ Rendering requires transformations between several different coordinate frames:
 | 
			
		||||
<img src="assets/transformations_overview.png" width="1000">
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
For example, given a teapot mesh, the world coordinate frame, camera coordiante frame and image are show in the figure below. Note that the world and camera coordinate frames have the +z direction pointing in to the page. 
 | 
			
		||||
For example, given a teapot mesh, the world coordinate frame, camera coordiante frame and image are show in the figure below. Note that the world and camera coordinate frames have the +z direction pointing in to the page.
 | 
			
		||||
 | 
			
		||||
<img src="assets/world_camera_image.png" width="1000">
 | 
			
		||||
 | 
			
		||||
@ -47,8 +47,8 @@ For example, given a teapot mesh, the world coordinate frame, camera coordiante
 | 
			
		||||
 | 
			
		||||
**NOTE: PyTorch3D vs OpenGL**
 | 
			
		||||
 | 
			
		||||
While we tried to emulate several aspects of OpenGL, there are differences in the coordinate frame conventions. 
 | 
			
		||||
- The default world coordinate frame in PyTorch3D has +Z pointing in to the screen whereas in OpenGL, +Z is pointing out of the screen.  Both are right handed. 
 | 
			
		||||
While we tried to emulate several aspects of OpenGL, there are differences in the coordinate frame conventions.
 | 
			
		||||
- The default world coordinate frame in PyTorch3D has +Z pointing in to the screen whereas in OpenGL, +Z is pointing out of the screen.  Both are right handed.
 | 
			
		||||
- The NDC coordinate system in PyTorch3D is **right-handed** compared with a **left-handed** NDC coordinate system in OpenGL (the projection matrix switches the handedness).
 | 
			
		||||
 | 
			
		||||
<img align="center" src="assets/opengl_coordframes.png" width="300">
 | 
			
		||||
@ -61,14 +61,14 @@ A renderer in PyTorch3D is composed of a **rasterizer** and a **shader**. Create
 | 
			
		||||
```
 | 
			
		||||
# Imports
 | 
			
		||||
from pytorch3d.renderer import (
 | 
			
		||||
    OpenGLPerspectiveCameras, look_at_view_transform,
 | 
			
		||||
    FoVPerspectiveCameras, look_at_view_transform,
 | 
			
		||||
    RasterizationSettings, BlendParams,
 | 
			
		||||
    MeshRenderer, MeshRasterizer, HardPhongShader
 | 
			
		||||
)
 | 
			
		||||
 | 
			
		||||
# Initialize an OpenGL perspective camera.
 | 
			
		||||
R, T = look_at_view_transform(2.7, 10, 20)
 | 
			
		||||
cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
cameras = FoVPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
 | 
			
		||||
# Define the settings for rasterization and shading. Here we set the output image to be of size
 | 
			
		||||
# 512x512. As we are rendering images for visualization purposes only we will set faces_per_pixel=1
 | 
			
		||||
 | 
			
		||||
@ -102,7 +102,7 @@
 | 
			
		||||
    "\n",
 | 
			
		||||
    "# rendering components\n",
 | 
			
		||||
    "from pytorch3d.renderer import (\n",
 | 
			
		||||
    "    OpenGLPerspectiveCameras, look_at_view_transform, look_at_rotation, \n",
 | 
			
		||||
    "    FoVPerspectiveCameras, look_at_view_transform, look_at_rotation, \n",
 | 
			
		||||
    "    RasterizationSettings, MeshRenderer, MeshRasterizer, BlendParams,\n",
 | 
			
		||||
    "    SoftSilhouetteShader, HardPhongShader, PointLights\n",
 | 
			
		||||
    ")"
 | 
			
		||||
@ -217,8 +217,8 @@
 | 
			
		||||
   },
 | 
			
		||||
   "outputs": [],
 | 
			
		||||
   "source": [
 | 
			
		||||
    "# Initialize an OpenGL perspective camera.\n",
 | 
			
		||||
    "cameras = OpenGLPerspectiveCameras(device=device)\n",
 | 
			
		||||
    "# Initialize a perspective camera.\n",
 | 
			
		||||
    "cameras = FoVPerspectiveCameras(device=device)\n",
 | 
			
		||||
    "\n",
 | 
			
		||||
    "# To blend the 100 faces we set a few parameters which control the opacity and the sharpness of \n",
 | 
			
		||||
    "# edges. Refer to blending.py for more details. \n",
 | 
			
		||||
 | 
			
		||||
@ -129,7 +129,7 @@
 | 
			
		||||
        "from pytorch3d.structures import Meshes, Textures\n",
 | 
			
		||||
        "from pytorch3d.renderer import (\n",
 | 
			
		||||
        "    look_at_view_transform,\n",
 | 
			
		||||
        "    OpenGLPerspectiveCameras, \n",
 | 
			
		||||
        "    FoVPerspectiveCameras, \n",
 | 
			
		||||
        "    PointLights, \n",
 | 
			
		||||
        "    DirectionalLights, \n",
 | 
			
		||||
        "    Materials, \n",
 | 
			
		||||
@ -309,16 +309,16 @@
 | 
			
		||||
        "# the cow is facing the -z direction. \n",
 | 
			
		||||
        "lights = PointLights(device=device, location=[[0.0, 0.0, -3.0]])\n",
 | 
			
		||||
        "\n",
 | 
			
		||||
        "# Initialize an OpenGL perspective camera that represents a batch of different \n",
 | 
			
		||||
        "# Initialize a camera that represents a batch of different \n",
 | 
			
		||||
        "# viewing angles. All the cameras helper methods support mixed type inputs and \n",
 | 
			
		||||
        "# broadcasting. So we can view the camera from the a distance of dist=2.7, and \n",
 | 
			
		||||
        "# then specify elevation and azimuth angles for each viewpoint as tensors. \n",
 | 
			
		||||
        "R, T = look_at_view_transform(dist=2.7, elev=elev, azim=azim)\n",
 | 
			
		||||
        "cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)\n",
 | 
			
		||||
        "cameras = FoVPerspectiveCameras(device=device, R=R, T=T)\n",
 | 
			
		||||
        "\n",
 | 
			
		||||
        "# We arbitrarily choose one particular view that will be used to visualize \n",
 | 
			
		||||
        "# results\n",
 | 
			
		||||
        "camera = OpenGLPerspectiveCameras(device=device, R=R[None, 1, ...], \n",
 | 
			
		||||
        "camera = FoVPerspectiveCameras(device=device, R=R[None, 1, ...], \n",
 | 
			
		||||
        "                                  T=T[None, 1, ...]) \n",
 | 
			
		||||
        "\n",
 | 
			
		||||
        "# Define the settings for rasterization and shading. Here we set the output \n",
 | 
			
		||||
@ -361,7 +361,7 @@
 | 
			
		||||
        "# Our multi-view cow dataset will be represented by these 2 lists of tensors,\n",
 | 
			
		||||
        "# each of length num_views.\n",
 | 
			
		||||
        "target_rgb = [target_images[i, ..., :3] for i in range(num_views)]\n",
 | 
			
		||||
        "target_cameras = [OpenGLPerspectiveCameras(device=device, R=R[None, i, ...], \n",
 | 
			
		||||
        "target_cameras = [FoVPerspectiveCameras(device=device, R=R[None, i, ...], \n",
 | 
			
		||||
        "                                           T=T[None, i, ...]) for i in range(num_views)]"
 | 
			
		||||
      ],
 | 
			
		||||
      "execution_count": null,
 | 
			
		||||
@ -925,4 +925,4 @@
 | 
			
		||||
      ]
 | 
			
		||||
    }
 | 
			
		||||
  ]
 | 
			
		||||
}
 | 
			
		||||
}
 | 
			
		||||
 | 
			
		||||
@ -64,7 +64,7 @@
 | 
			
		||||
    "from pytorch3d.structures import Pointclouds\n",
 | 
			
		||||
    "from pytorch3d.renderer import (\n",
 | 
			
		||||
    "    look_at_view_transform,\n",
 | 
			
		||||
    "    OpenGLOrthographicCameras, \n",
 | 
			
		||||
    "    FoVOrthographicCameras, \n",
 | 
			
		||||
    "    PointsRasterizationSettings,\n",
 | 
			
		||||
    "    PointsRenderer,\n",
 | 
			
		||||
    "    PointsRasterizer,\n",
 | 
			
		||||
@ -147,9 +147,9 @@
 | 
			
		||||
   "metadata": {},
 | 
			
		||||
   "outputs": [],
 | 
			
		||||
   "source": [
 | 
			
		||||
    "# Initialize an OpenGL perspective camera.\n",
 | 
			
		||||
    "# Initialize a camera.\n",
 | 
			
		||||
    "R, T = look_at_view_transform(20, 10, 0)\n",
 | 
			
		||||
    "cameras = OpenGLOrthographicCameras(device=device, R=R, T=T, znear=0.01)\n",
 | 
			
		||||
    "cameras = FoVOrthographicCameras(device=device, R=R, T=T, znear=0.01)\n",
 | 
			
		||||
    "\n",
 | 
			
		||||
    "# Define the settings for rasterization and shading. Here we set the output image to be of size\n",
 | 
			
		||||
    "# 512x512. As we are rendering images for visualization purposes only we will set faces_per_pixel=1\n",
 | 
			
		||||
@ -195,9 +195,9 @@
 | 
			
		||||
   "metadata": {},
 | 
			
		||||
   "outputs": [],
 | 
			
		||||
   "source": [
 | 
			
		||||
    "# Initialize an OpenGL perspective camera.\n",
 | 
			
		||||
    "# Initialize a camera.\n",
 | 
			
		||||
    "R, T = look_at_view_transform(20, 10, 0)\n",
 | 
			
		||||
    "cameras = OpenGLOrthographicCameras(device=device, R=R, T=T, znear=0.01)\n",
 | 
			
		||||
    "cameras = FoVOrthographicCameras(device=device, R=R, T=T, znear=0.01)\n",
 | 
			
		||||
    "\n",
 | 
			
		||||
    "# Define the settings for rasterization and shading. Here we set the output image to be of size\n",
 | 
			
		||||
    "# 512x512. As we are rendering images for visualization purposes only we will set faces_per_pixel=1\n",
 | 
			
		||||
 | 
			
		||||
@ -90,7 +90,7 @@
 | 
			
		||||
    "from pytorch3d.structures import Meshes, Textures\n",
 | 
			
		||||
    "from pytorch3d.renderer import (\n",
 | 
			
		||||
    "    look_at_view_transform,\n",
 | 
			
		||||
    "    OpenGLPerspectiveCameras, \n",
 | 
			
		||||
    "    FoVPerspectiveCameras, \n",
 | 
			
		||||
    "    PointLights, \n",
 | 
			
		||||
    "    DirectionalLights, \n",
 | 
			
		||||
    "    Materials, \n",
 | 
			
		||||
@ -286,11 +286,11 @@
 | 
			
		||||
   },
 | 
			
		||||
   "outputs": [],
 | 
			
		||||
   "source": [
 | 
			
		||||
    "# Initialize an OpenGL perspective camera.\n",
 | 
			
		||||
    "# Initialize a camera.\n",
 | 
			
		||||
    "# With world coordinates +Y up, +X left and +Z in, the front of the cow is facing the -Z direction. \n",
 | 
			
		||||
    "# So we move the camera by 180 in the azimuth direction so it is facing the front of the cow. \n",
 | 
			
		||||
    "R, T = look_at_view_transform(2.7, 0, 180) \n",
 | 
			
		||||
    "cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)\n",
 | 
			
		||||
    "cameras = FoVPerspectiveCameras(device=device, R=R, T=T)\n",
 | 
			
		||||
    "\n",
 | 
			
		||||
    "# Define the settings for rasterization and shading. Here we set the output image to be of size\n",
 | 
			
		||||
    "# 512x512. As we are rendering images for visualization purposes only we will set faces_per_pixel=1\n",
 | 
			
		||||
@ -444,7 +444,7 @@
 | 
			
		||||
   "source": [
 | 
			
		||||
    "# Rotate the object by increasing the elevation and azimuth angles\n",
 | 
			
		||||
    "R, T = look_at_view_transform(dist=2.7, elev=10, azim=-150)\n",
 | 
			
		||||
    "cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)\n",
 | 
			
		||||
    "cameras = FoVPerspectiveCameras(device=device, R=R, T=T)\n",
 | 
			
		||||
    "\n",
 | 
			
		||||
    "# Move the light location so the light is shining on the cow's face.  \n",
 | 
			
		||||
    "lights.location = torch.tensor([[2.0, 2.0, -2.0]], device=device)\n",
 | 
			
		||||
@ -519,7 +519,7 @@
 | 
			
		||||
    "# view the camera from the same distance and specify dist=2.7 as a float,\n",
 | 
			
		||||
    "# and then specify elevation and azimuth angles for each viewpoint as tensors. \n",
 | 
			
		||||
    "R, T = look_at_view_transform(dist=2.7, elev=elev, azim=azim)\n",
 | 
			
		||||
    "cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)\n",
 | 
			
		||||
    "cameras = FoVPerspectiveCameras(device=device, R=R, T=T)\n",
 | 
			
		||||
    "\n",
 | 
			
		||||
    "# Move the light back in front of the cow which is facing the -z direction.\n",
 | 
			
		||||
    "lights.location = torch.tensor([[0.0, 0.0, -3.0]], device=device)"
 | 
			
		||||
 | 
			
		||||
@ -10,7 +10,7 @@ from pytorch3d.renderer import (
 | 
			
		||||
    HardPhongShader,
 | 
			
		||||
    MeshRasterizer,
 | 
			
		||||
    MeshRenderer,
 | 
			
		||||
    OpenGLPerspectiveCameras,
 | 
			
		||||
    FoVPerspectiveCameras,
 | 
			
		||||
    PointLights,
 | 
			
		||||
    RasterizationSettings,
 | 
			
		||||
    TexturesVertex,
 | 
			
		||||
@ -125,7 +125,7 @@ class ShapeNetBase(torch.utils.data.Dataset):
 | 
			
		||||
        meshes.textures = TexturesVertex(
 | 
			
		||||
            verts_features=torch.ones_like(meshes.verts_padded(), device=device)
 | 
			
		||||
        )
 | 
			
		||||
        cameras = kwargs.get("cameras", OpenGLPerspectiveCameras()).to(device)
 | 
			
		||||
        cameras = kwargs.get("cameras", FoVPerspectiveCameras()).to(device)
 | 
			
		||||
        if len(cameras) != 1 and len(cameras) % len(meshes) != 0:
 | 
			
		||||
            raise ValueError("Mismatch between batch dims of cameras and meshes.")
 | 
			
		||||
        if len(cameras) > 1:
 | 
			
		||||
 | 
			
		||||
@ -6,11 +6,15 @@ from .blending import (
 | 
			
		||||
    sigmoid_alpha_blend,
 | 
			
		||||
    softmax_rgb_blend,
 | 
			
		||||
)
 | 
			
		||||
from .cameras import OpenGLOrthographicCameras  # deprecated
 | 
			
		||||
from .cameras import OpenGLPerspectiveCameras  # deprecated
 | 
			
		||||
from .cameras import SfMOrthographicCameras  # deprecated
 | 
			
		||||
from .cameras import SfMPerspectiveCameras  # deprecated
 | 
			
		||||
from .cameras import (
 | 
			
		||||
    OpenGLOrthographicCameras,
 | 
			
		||||
    OpenGLPerspectiveCameras,
 | 
			
		||||
    SfMOrthographicCameras,
 | 
			
		||||
    SfMPerspectiveCameras,
 | 
			
		||||
    FoVOrthographicCameras,
 | 
			
		||||
    FoVPerspectiveCameras,
 | 
			
		||||
    OrthographicCameras,
 | 
			
		||||
    PerspectiveCameras,
 | 
			
		||||
    camera_position_from_spherical_angles,
 | 
			
		||||
    get_world_to_view_transform,
 | 
			
		||||
    look_at_rotation,
 | 
			
		||||
 | 
			
		||||
@ -1,6 +1,7 @@
 | 
			
		||||
# Copyright (c) Facebook, Inc. and its affiliates. All rights reserved.
 | 
			
		||||
 | 
			
		||||
import math
 | 
			
		||||
import warnings
 | 
			
		||||
from typing import Optional, Sequence, Tuple
 | 
			
		||||
 | 
			
		||||
import numpy as np
 | 
			
		||||
@ -20,23 +21,43 @@ class CamerasBase(TensorProperties):
 | 
			
		||||
    """
 | 
			
		||||
    `CamerasBase` implements a base class for all cameras.
 | 
			
		||||
 | 
			
		||||
    For cameras, there are four different coordinate systems (or spaces)
 | 
			
		||||
    - World coordinate system: This is the system the object lives - the world.
 | 
			
		||||
    - Camera view coordinate system: This is the system that has its origin on the image plane
 | 
			
		||||
        and the and the Z-axis perpendicular to the image plane.
 | 
			
		||||
        In PyTorch3D, we assume that +X points left, and +Y points up and
 | 
			
		||||
        +Z points out from the image plane.
 | 
			
		||||
        The transformation from world -> view happens after applying a rotation (R)
 | 
			
		||||
        and translation (T)
 | 
			
		||||
    - NDC coordinate system: This is the normalized coordinate system that confines
 | 
			
		||||
        in a volume the renderered part of the object or scene. Also known as view volume.
 | 
			
		||||
        Given the PyTorch3D convention, (+1, +1, znear) is the top left near corner,
 | 
			
		||||
        and (-1, -1, zfar) is the bottom right far corner of the volume.
 | 
			
		||||
        The transformation from view -> NDC happens after applying the camera
 | 
			
		||||
        projection matrix (P).
 | 
			
		||||
    - Screen coordinate system: This is another representation of the view volume with
 | 
			
		||||
        the XY coordinates defined in pixel space instead of a normalized space.
 | 
			
		||||
 | 
			
		||||
    A better illustration of the coordinate systems can be found in pytorch3d/docs/notes/cameras.md.
 | 
			
		||||
 | 
			
		||||
    It defines methods that are common to all camera models:
 | 
			
		||||
        - `get_camera_center` that returns the optical center of the camera in
 | 
			
		||||
    world coordinates
 | 
			
		||||
        - `get_world_to_view_transform` which returns a 3D transform from
 | 
			
		||||
    world coordinates to the camera coordinates
 | 
			
		||||
    world coordinates to the camera view coordinates (R, T)
 | 
			
		||||
        - `get_full_projection_transform` which composes the projection
 | 
			
		||||
    transform with the world-to-view transform
 | 
			
		||||
        - `transform_points` which takes a set of input points and
 | 
			
		||||
    projects them onto a 2D camera plane.
 | 
			
		||||
    transform (P) with the world-to-view transform (R, T)
 | 
			
		||||
        - `transform_points` which takes a set of input points in world coordinates and
 | 
			
		||||
    projects to NDC coordinates ranging from [-1, -1, znear] to [+1, +1, zfar].
 | 
			
		||||
        - `transform_points_screen` which takes a set of input points in world coordinates and
 | 
			
		||||
    projects them to the screen coordinates ranging from [0, 0, znear] to [W-1, H-1, zfar]
 | 
			
		||||
 | 
			
		||||
    For each new camera, one should implement the `get_projection_transform`
 | 
			
		||||
    routine that returns the mapping from camera coordinates in world units
 | 
			
		||||
    to the screen coordinates.
 | 
			
		||||
    routine that returns the mapping from camera view coordinates to NDC coordinates.
 | 
			
		||||
 | 
			
		||||
    Another useful function that is specific to each camera model is
 | 
			
		||||
    `unproject_points` which sends points from screen coordinates back to
 | 
			
		||||
    camera or world coordinates depending on the `world_coordinates`
 | 
			
		||||
    `unproject_points` which sends points from NDC coordinates back to
 | 
			
		||||
    camera view or world coordinates depending on the `world_coordinates`
 | 
			
		||||
    boolean argument of the function.
 | 
			
		||||
    """
 | 
			
		||||
 | 
			
		||||
@ -56,7 +77,7 @@ class CamerasBase(TensorProperties):
 | 
			
		||||
 | 
			
		||||
    def unproject_points(self):
 | 
			
		||||
        """
 | 
			
		||||
        Transform input points in screen coodinates
 | 
			
		||||
        Transform input points from NDC coodinates
 | 
			
		||||
        to the world / camera coordinates.
 | 
			
		||||
 | 
			
		||||
        Each of the input points `xy_depth` of shape (..., 3) is
 | 
			
		||||
@ -74,7 +95,7 @@ class CamerasBase(TensorProperties):
 | 
			
		||||
 | 
			
		||||
            cameras = # camera object derived from CamerasBase
 | 
			
		||||
            xyz = # 3D points of shape (batch_size, num_points, 3)
 | 
			
		||||
            # transform xyz to the camera coordinates
 | 
			
		||||
            # transform xyz to the camera view coordinates
 | 
			
		||||
            xyz_cam = cameras.get_world_to_view_transform().transform_points(xyz)
 | 
			
		||||
            # extract the depth of each point as the 3rd coord of xyz_cam
 | 
			
		||||
            depth = xyz_cam[:, :, 2:]
 | 
			
		||||
@ -94,7 +115,7 @@ class CamerasBase(TensorProperties):
 | 
			
		||||
            world_coordinates: If `True`, unprojects the points back to world
 | 
			
		||||
                coordinates using the camera extrinsics `R` and `T`.
 | 
			
		||||
                `False` ignores `R` and `T` and unprojects to
 | 
			
		||||
                the camera coordinates.
 | 
			
		||||
                the camera view coordinates.
 | 
			
		||||
 | 
			
		||||
        Returns
 | 
			
		||||
            new_points: unprojected points with the same shape as `xy_depth`.
 | 
			
		||||
@ -141,7 +162,7 @@ class CamerasBase(TensorProperties):
 | 
			
		||||
        lighting calculations.
 | 
			
		||||
 | 
			
		||||
        Returns:
 | 
			
		||||
            T: a Transform3d object which represents a batch of transforms
 | 
			
		||||
            A Transform3d object which represents a batch of transforms
 | 
			
		||||
            of shape (N, 3, 3)
 | 
			
		||||
        """
 | 
			
		||||
        self.R = kwargs.get("R", self.R)  # pyre-ignore[16]
 | 
			
		||||
@ -151,8 +172,8 @@ class CamerasBase(TensorProperties):
 | 
			
		||||
 | 
			
		||||
    def get_full_projection_transform(self, **kwargs) -> Transform3d:
 | 
			
		||||
        """
 | 
			
		||||
        Return the full world-to-screen transform composing the
 | 
			
		||||
        world-to-view and view-to-screen transforms.
 | 
			
		||||
        Return the full world-to-NDC transform composing the
 | 
			
		||||
        world-to-view and view-to-NDC transforms.
 | 
			
		||||
 | 
			
		||||
        Args:
 | 
			
		||||
            **kwargs: parameters for the projection transforms can be passed in
 | 
			
		||||
@ -164,26 +185,26 @@ class CamerasBase(TensorProperties):
 | 
			
		||||
        lighting calculations.
 | 
			
		||||
 | 
			
		||||
        Returns:
 | 
			
		||||
            T: a Transform3d object which represents a batch of transforms
 | 
			
		||||
            a Transform3d object which represents a batch of transforms
 | 
			
		||||
            of shape (N, 3, 3)
 | 
			
		||||
        """
 | 
			
		||||
        self.R = kwargs.get("R", self.R)  # pyre-ignore[16]
 | 
			
		||||
        self.T = kwargs.get("T", self.T)  # pyre-ignore[16]
 | 
			
		||||
        world_to_view_transform = self.get_world_to_view_transform(R=self.R, T=self.T)
 | 
			
		||||
        view_to_screen_transform = self.get_projection_transform(**kwargs)
 | 
			
		||||
        return world_to_view_transform.compose(view_to_screen_transform)
 | 
			
		||||
        view_to_ndc_transform = self.get_projection_transform(**kwargs)
 | 
			
		||||
        return world_to_view_transform.compose(view_to_ndc_transform)
 | 
			
		||||
 | 
			
		||||
    def transform_points(
 | 
			
		||||
        self, points, eps: Optional[float] = None, **kwargs
 | 
			
		||||
    ) -> torch.Tensor:
 | 
			
		||||
        """
 | 
			
		||||
        Transform input points from world to screen space.
 | 
			
		||||
        Transform input points from world to NDC space.
 | 
			
		||||
 | 
			
		||||
        Args:
 | 
			
		||||
            points: torch tensor of shape (..., 3).
 | 
			
		||||
            eps: If eps!=None, the argument is used to clamp the
 | 
			
		||||
                divisor in the homogeneous normalization of the points
 | 
			
		||||
                transformed to the screen space. Plese see
 | 
			
		||||
                transformed to the ndc space. Please see
 | 
			
		||||
                `transforms.Transform3D.transform_points` for details.
 | 
			
		||||
 | 
			
		||||
                For `CamerasBase.transform_points`, setting `eps > 0`
 | 
			
		||||
@ -194,8 +215,50 @@ class CamerasBase(TensorProperties):
 | 
			
		||||
        Returns
 | 
			
		||||
            new_points: transformed points with the same shape as the input.
 | 
			
		||||
        """
 | 
			
		||||
        world_to_screen_transform = self.get_full_projection_transform(**kwargs)
 | 
			
		||||
        return world_to_screen_transform.transform_points(points, eps=eps)
 | 
			
		||||
        world_to_ndc_transform = self.get_full_projection_transform(**kwargs)
 | 
			
		||||
        return world_to_ndc_transform.transform_points(points, eps=eps)
 | 
			
		||||
 | 
			
		||||
    def transform_points_screen(
 | 
			
		||||
        self, points, image_size, eps: Optional[float] = None, **kwargs
 | 
			
		||||
    ) -> torch.Tensor:
 | 
			
		||||
        """
 | 
			
		||||
        Transform input points from world to screen space.
 | 
			
		||||
 | 
			
		||||
        Args:
 | 
			
		||||
            points: torch tensor of shape (N, V, 3).
 | 
			
		||||
            image_size: torch tensor of shape (N, 2)
 | 
			
		||||
            eps: If eps!=None, the argument is used to clamp the
 | 
			
		||||
                divisor in the homogeneous normalization of the points
 | 
			
		||||
                transformed to the ndc space. Please see
 | 
			
		||||
                `transforms.Transform3D.transform_points` for details.
 | 
			
		||||
 | 
			
		||||
                For `CamerasBase.transform_points`, setting `eps > 0`
 | 
			
		||||
                stabilizes gradients since it leads to avoiding division
 | 
			
		||||
                by excessivelly low numbers for points close to the
 | 
			
		||||
                camera plane.
 | 
			
		||||
 | 
			
		||||
        Returns
 | 
			
		||||
            new_points: transformed points with the same shape as the input.
 | 
			
		||||
        """
 | 
			
		||||
 | 
			
		||||
        ndc_points = self.transform_points(points, eps=eps, **kwargs)
 | 
			
		||||
 | 
			
		||||
        if not torch.is_tensor(image_size):
 | 
			
		||||
            image_size = torch.tensor(
 | 
			
		||||
                image_size, dtype=torch.int64, device=points.device
 | 
			
		||||
            )
 | 
			
		||||
        if (image_size < 1).any():
 | 
			
		||||
            raise ValueError("Provided image size is invalid.")
 | 
			
		||||
 | 
			
		||||
        image_width, image_height = image_size.unbind(1)
 | 
			
		||||
        image_width = image_width.view(-1, 1)  # (N, 1)
 | 
			
		||||
        image_height = image_height.view(-1, 1)  # (N, 1)
 | 
			
		||||
 | 
			
		||||
        ndc_z = ndc_points[..., 2]
 | 
			
		||||
        screen_x = (image_width - 1.0) / 2.0 * (1.0 - ndc_points[..., 0])
 | 
			
		||||
        screen_y = (image_height - 1.0) / 2.0 * (1.0 - ndc_points[..., 1])
 | 
			
		||||
 | 
			
		||||
        return torch.stack((screen_x, screen_y, ndc_z), dim=2)
 | 
			
		||||
 | 
			
		||||
    def clone(self):
 | 
			
		||||
        """
 | 
			
		||||
@ -206,21 +269,56 @@ class CamerasBase(TensorProperties):
 | 
			
		||||
        return super().clone(other)
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
########################
 | 
			
		||||
# Specific camera classes
 | 
			
		||||
########################
 | 
			
		||||
############################################################
 | 
			
		||||
#             Field of View Camera Classes                 #
 | 
			
		||||
############################################################
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class OpenGLPerspectiveCameras(CamerasBase):
 | 
			
		||||
def OpenGLPerspectiveCameras(
 | 
			
		||||
    znear=1.0,
 | 
			
		||||
    zfar=100.0,
 | 
			
		||||
    aspect_ratio=1.0,
 | 
			
		||||
    fov=60.0,
 | 
			
		||||
    degrees: bool = True,
 | 
			
		||||
    R=r,
 | 
			
		||||
    T=t,
 | 
			
		||||
    device="cpu",
 | 
			
		||||
):
 | 
			
		||||
    """
 | 
			
		||||
    OpenGLPerspectiveCameras has been DEPRECATED. Use FoVPerspectiveCameras instead.
 | 
			
		||||
    Preserving OpenGLPerspectiveCameras for backward compatibility.
 | 
			
		||||
    """
 | 
			
		||||
 | 
			
		||||
    warnings.warn(
 | 
			
		||||
        """OpenGLPerspectiveCameras is deprecated,
 | 
			
		||||
        Use FoVPerspectiveCameras instead.
 | 
			
		||||
        OpenGLPerspectiveCameras will be removed in future releases.""",
 | 
			
		||||
        PendingDeprecationWarning,
 | 
			
		||||
    )
 | 
			
		||||
 | 
			
		||||
    return FoVPerspectiveCameras(
 | 
			
		||||
        znear=znear,
 | 
			
		||||
        zfar=zfar,
 | 
			
		||||
        aspect_ratio=aspect_ratio,
 | 
			
		||||
        fov=fov,
 | 
			
		||||
        degrees=degrees,
 | 
			
		||||
        R=R,
 | 
			
		||||
        T=T,
 | 
			
		||||
        device=device,
 | 
			
		||||
    )
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class FoVPerspectiveCameras(CamerasBase):
 | 
			
		||||
    """
 | 
			
		||||
    A class which stores a batch of parameters to generate a batch of
 | 
			
		||||
    projection matrices using the OpenGL convention for a perspective camera.
 | 
			
		||||
    projection matrices by specifiying the field of view.
 | 
			
		||||
    The definition of the parameters follow the OpenGL perspective camera.
 | 
			
		||||
 | 
			
		||||
    The extrinsics of the camera (R and T matrices) can also be set in the
 | 
			
		||||
    initializer or passed in to `get_full_projection_transform` to get
 | 
			
		||||
    the full transformation from world -> screen.
 | 
			
		||||
    the full transformation from world -> ndc.
 | 
			
		||||
 | 
			
		||||
    The `transform_points` method calculates the full world -> screen transform
 | 
			
		||||
    The `transform_points` method calculates the full world -> ndc transform
 | 
			
		||||
    and then applies it to the input points.
 | 
			
		||||
 | 
			
		||||
    The transforms can also be returned separately as Transform3d objects.
 | 
			
		||||
@ -267,8 +365,11 @@ class OpenGLPerspectiveCameras(CamerasBase):
 | 
			
		||||
 | 
			
		||||
    def get_projection_transform(self, **kwargs) -> Transform3d:
 | 
			
		||||
        """
 | 
			
		||||
        Calculate the OpenGL perpective projection matrix with a symmetric
 | 
			
		||||
        Calculate the perpective projection matrix with a symmetric
 | 
			
		||||
        viewing frustrum. Use column major order.
 | 
			
		||||
        The viewing frustrum will be projected into ndc, s.t.
 | 
			
		||||
        (max_x, max_y) -> (+1, +1)
 | 
			
		||||
        (min_x, min_y) -> (-1, -1)
 | 
			
		||||
 | 
			
		||||
        Args:
 | 
			
		||||
            **kwargs: parameters for the projection can be passed in as keyword
 | 
			
		||||
@ -276,14 +377,14 @@ class OpenGLPerspectiveCameras(CamerasBase):
 | 
			
		||||
 | 
			
		||||
        Return:
 | 
			
		||||
            P: a Transform3d object which represents a batch of projection
 | 
			
		||||
            matrices of shape (N, 3, 3)
 | 
			
		||||
            matrices of shape (N, 4, 4)
 | 
			
		||||
 | 
			
		||||
        .. code-block:: python
 | 
			
		||||
 | 
			
		||||
            f1 = -(far + near)/(far−near)
 | 
			
		||||
            f2 = -2*far*near/(far-near)
 | 
			
		||||
            h1 = (top + bottom)/(top - bottom)
 | 
			
		||||
            w1 = (right + left)/(right - left)
 | 
			
		||||
            h1 = (max_y + min_y)/(max_y - min_y)
 | 
			
		||||
            w1 = (max_x + min_x)/(max_x - min_x)
 | 
			
		||||
            tanhalffov = tan((fov/2))
 | 
			
		||||
            s1 = 1/tanhalffov
 | 
			
		||||
            s2 = 1/(tanhalffov * (aspect_ratio))
 | 
			
		||||
@ -310,10 +411,10 @@ class OpenGLPerspectiveCameras(CamerasBase):
 | 
			
		||||
        if not torch.is_tensor(fov):
 | 
			
		||||
            fov = torch.tensor(fov, device=self.device)
 | 
			
		||||
        tanHalfFov = torch.tan((fov / 2))
 | 
			
		||||
        top = tanHalfFov * znear
 | 
			
		||||
        bottom = -top
 | 
			
		||||
        right = top * aspect_ratio
 | 
			
		||||
        left = -right
 | 
			
		||||
        max_y = tanHalfFov * znear
 | 
			
		||||
        min_y = -max_y
 | 
			
		||||
        max_x = max_y * aspect_ratio
 | 
			
		||||
        min_x = -max_x
 | 
			
		||||
 | 
			
		||||
        # NOTE: In OpenGL the projection matrix changes the handedness of the
 | 
			
		||||
        # coordinate frame. i.e the NDC space postive z direction is the
 | 
			
		||||
@ -323,28 +424,19 @@ class OpenGLPerspectiveCameras(CamerasBase):
 | 
			
		||||
        # so the so the z sign is 1.0.
 | 
			
		||||
        z_sign = 1.0
 | 
			
		||||
 | 
			
		||||
        P[:, 0, 0] = 2.0 * znear / (right - left)
 | 
			
		||||
        P[:, 1, 1] = 2.0 * znear / (top - bottom)
 | 
			
		||||
        P[:, 0, 2] = (right + left) / (right - left)
 | 
			
		||||
        P[:, 1, 2] = (top + bottom) / (top - bottom)
 | 
			
		||||
        P[:, 0, 0] = 2.0 * znear / (max_x - min_x)
 | 
			
		||||
        P[:, 1, 1] = 2.0 * znear / (max_y - min_y)
 | 
			
		||||
        P[:, 0, 2] = (max_x + min_x) / (max_x - min_x)
 | 
			
		||||
        P[:, 1, 2] = (max_y + min_y) / (max_y - min_y)
 | 
			
		||||
        P[:, 3, 2] = z_sign * ones
 | 
			
		||||
 | 
			
		||||
        # NOTE: This part of the matrix is for z renormalization in OpenGL
 | 
			
		||||
        # which maps the z to [-1, 1]. This won't work yet as the torch3d
 | 
			
		||||
        # rasterizer ignores faces which have z < 0.
 | 
			
		||||
        # P[:, 2, 2] = z_sign * (far + near) / (far - near)
 | 
			
		||||
        # P[:, 2, 3] = -2.0 * far * near / (far - near)
 | 
			
		||||
        # P[:, 3, 2] = z_sign * torch.ones((N))
 | 
			
		||||
 | 
			
		||||
        # NOTE: This maps the z coordinate from [0, 1] where z = 0 if the point
 | 
			
		||||
        # is at the near clipping plane and z = 1 when the point is at the far
 | 
			
		||||
        # clipping plane. This replaces the OpenGL z normalization to [-1, 1]
 | 
			
		||||
        # until rasterization is changed to clip at z = -1.
 | 
			
		||||
        # clipping plane.
 | 
			
		||||
        P[:, 2, 2] = z_sign * zfar / (zfar - znear)
 | 
			
		||||
        P[:, 2, 3] = -(zfar * znear) / (zfar - znear)
 | 
			
		||||
 | 
			
		||||
        # OpenGL uses column vectors so need to transpose the projection matrix
 | 
			
		||||
        # as torch3d uses row vectors.
 | 
			
		||||
        # Transpose the projection matrix as PyTorch3d transforms use row vectors.
 | 
			
		||||
        transform = Transform3d(device=self.device)
 | 
			
		||||
        transform._matrix = P.transpose(1, 2).contiguous()
 | 
			
		||||
        return transform
 | 
			
		||||
@ -357,7 +449,7 @@ class OpenGLPerspectiveCameras(CamerasBase):
 | 
			
		||||
        **kwargs
 | 
			
		||||
    ) -> torch.Tensor:
 | 
			
		||||
        """>!
 | 
			
		||||
        OpenGL cameras further allow for passing depth in world units
 | 
			
		||||
        FoV cameras further allow for passing depth in world units
 | 
			
		||||
        (`scaled_depth_input=False`) or in the [0, 1]-normalized units
 | 
			
		||||
        (`scaled_depth_input=True`)
 | 
			
		||||
 | 
			
		||||
@ -367,11 +459,11 @@ class OpenGLPerspectiveCameras(CamerasBase):
 | 
			
		||||
                the world units.
 | 
			
		||||
        """
 | 
			
		||||
 | 
			
		||||
        # obtain the relevant transformation to screen
 | 
			
		||||
        # obtain the relevant transformation to ndc
 | 
			
		||||
        if world_coordinates:
 | 
			
		||||
            to_screen_transform = self.get_full_projection_transform()
 | 
			
		||||
            to_ndc_transform = self.get_full_projection_transform()
 | 
			
		||||
        else:
 | 
			
		||||
            to_screen_transform = self.get_projection_transform()
 | 
			
		||||
            to_ndc_transform = self.get_projection_transform()
 | 
			
		||||
 | 
			
		||||
        if scaled_depth_input:
 | 
			
		||||
            # the input is scaled depth, so we don't have to do anything
 | 
			
		||||
@ -390,45 +482,84 @@ class OpenGLPerspectiveCameras(CamerasBase):
 | 
			
		||||
            xy_sdepth = torch.cat((xy_depth[..., 0:2], sdepth), dim=-1)
 | 
			
		||||
 | 
			
		||||
        # unproject with inverse of the projection
 | 
			
		||||
        unprojection_transform = to_screen_transform.inverse()
 | 
			
		||||
        unprojection_transform = to_ndc_transform.inverse()
 | 
			
		||||
        return unprojection_transform.transform_points(xy_sdepth)
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class OpenGLOrthographicCameras(CamerasBase):
 | 
			
		||||
def OpenGLOrthographicCameras(
 | 
			
		||||
    znear=1.0,
 | 
			
		||||
    zfar=100.0,
 | 
			
		||||
    top=1.0,
 | 
			
		||||
    bottom=-1.0,
 | 
			
		||||
    left=-1.0,
 | 
			
		||||
    right=1.0,
 | 
			
		||||
    scale_xyz=((1.0, 1.0, 1.0),),  # (1, 3)
 | 
			
		||||
    R=r,
 | 
			
		||||
    T=t,
 | 
			
		||||
    device="cpu",
 | 
			
		||||
):
 | 
			
		||||
    """
 | 
			
		||||
    OpenGLOrthographicCameras has been DEPRECATED. Use FoVOrthographicCameras instead.
 | 
			
		||||
    Preserving OpenGLOrthographicCameras for backward compatibility.
 | 
			
		||||
    """
 | 
			
		||||
 | 
			
		||||
    warnings.warn(
 | 
			
		||||
        """OpenGLOrthographicCameras is deprecated,
 | 
			
		||||
        Use FoVOrthographicCameras instead.
 | 
			
		||||
        OpenGLOrthographicCameras will be removed in future releases.""",
 | 
			
		||||
        PendingDeprecationWarning,
 | 
			
		||||
    )
 | 
			
		||||
 | 
			
		||||
    return FoVOrthographicCameras(
 | 
			
		||||
        znear=znear,
 | 
			
		||||
        zfar=zfar,
 | 
			
		||||
        max_y=top,
 | 
			
		||||
        min_y=bottom,
 | 
			
		||||
        max_x=right,
 | 
			
		||||
        min_x=left,
 | 
			
		||||
        scale_xyz=scale_xyz,
 | 
			
		||||
        R=R,
 | 
			
		||||
        T=T,
 | 
			
		||||
        device=device,
 | 
			
		||||
    )
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class FoVOrthographicCameras(CamerasBase):
 | 
			
		||||
    """
 | 
			
		||||
    A class which stores a batch of parameters to generate a batch of
 | 
			
		||||
    transformation matrices using the OpenGL convention for orthographic camera.
 | 
			
		||||
    projection matrices by specifiying the field of view.
 | 
			
		||||
    The definition of the parameters follow the OpenGL orthographic camera.
 | 
			
		||||
    """
 | 
			
		||||
 | 
			
		||||
    def __init__(
 | 
			
		||||
        self,
 | 
			
		||||
        znear=1.0,
 | 
			
		||||
        zfar=100.0,
 | 
			
		||||
        top=1.0,
 | 
			
		||||
        bottom=-1.0,
 | 
			
		||||
        left=-1.0,
 | 
			
		||||
        right=1.0,
 | 
			
		||||
        max_y=1.0,
 | 
			
		||||
        min_y=-1.0,
 | 
			
		||||
        max_x=1.0,
 | 
			
		||||
        min_x=-1.0,
 | 
			
		||||
        scale_xyz=((1.0, 1.0, 1.0),),  # (1, 3)
 | 
			
		||||
        R=r,
 | 
			
		||||
        T=t,
 | 
			
		||||
        device="cpu",
 | 
			
		||||
    ):
 | 
			
		||||
        """
 | 
			
		||||
        __init__(self, znear, zfar, top, bottom, left, right, scale_xyz, R, T, device) -> None  # noqa
 | 
			
		||||
        __init__(self, znear, zfar, max_y, min_y, max_x, min_x, scale_xyz, R, T, device) -> None  # noqa
 | 
			
		||||
 | 
			
		||||
        Args:
 | 
			
		||||
            znear: near clipping plane of the view frustrum.
 | 
			
		||||
            zfar: far clipping plane of the view frustrum.
 | 
			
		||||
            top: position of the top of the screen.
 | 
			
		||||
            bottom: position of the bottom of the screen.
 | 
			
		||||
            left: position of the left of the screen.
 | 
			
		||||
            right: position of the right of the screen.
 | 
			
		||||
            max_y: maximum y coordinate of the frustrum.
 | 
			
		||||
            min_y: minimum y coordinate of the frustrum.
 | 
			
		||||
            max_x: maximum x coordinate of the frustrum.
 | 
			
		||||
            min_x: minumum x coordinage of the frustrum
 | 
			
		||||
            scale_xyz: scale factors for each axis of shape (N, 3).
 | 
			
		||||
            R: Rotation matrix of shape (N, 3, 3).
 | 
			
		||||
            T: Translation of shape (N, 3).
 | 
			
		||||
            device: torch.device or string.
 | 
			
		||||
 | 
			
		||||
        Only need to set left, right, top, bottom for viewing frustrums
 | 
			
		||||
        Only need to set min_x, max_x, min_y, max_y for viewing frustrums
 | 
			
		||||
        which are non symmetric about the origin.
 | 
			
		||||
        """
 | 
			
		||||
        # The initializer formats all inputs to torch tensors and broadcasts
 | 
			
		||||
@ -437,10 +568,10 @@ class OpenGLOrthographicCameras(CamerasBase):
 | 
			
		||||
            device=device,
 | 
			
		||||
            znear=znear,
 | 
			
		||||
            zfar=zfar,
 | 
			
		||||
            top=top,
 | 
			
		||||
            bottom=bottom,
 | 
			
		||||
            left=left,
 | 
			
		||||
            right=right,
 | 
			
		||||
            max_y=max_y,
 | 
			
		||||
            min_y=min_y,
 | 
			
		||||
            max_x=max_x,
 | 
			
		||||
            min_x=min_x,
 | 
			
		||||
            scale_xyz=scale_xyz,
 | 
			
		||||
            R=R,
 | 
			
		||||
            T=T,
 | 
			
		||||
@ -448,7 +579,7 @@ class OpenGLOrthographicCameras(CamerasBase):
 | 
			
		||||
 | 
			
		||||
    def get_projection_transform(self, **kwargs) -> Transform3d:
 | 
			
		||||
        """
 | 
			
		||||
        Calculate the OpenGL orthographic projection matrix.
 | 
			
		||||
        Calculate the orthographic projection matrix.
 | 
			
		||||
        Use column major order.
 | 
			
		||||
 | 
			
		||||
        Args:
 | 
			
		||||
@ -456,16 +587,16 @@ class OpenGLOrthographicCameras(CamerasBase):
 | 
			
		||||
                      override the default values set in __init__.
 | 
			
		||||
        Return:
 | 
			
		||||
            P: a Transform3d object which represents a batch of projection
 | 
			
		||||
               matrices of shape (N, 3, 3)
 | 
			
		||||
               matrices of shape (N, 4, 4)
 | 
			
		||||
 | 
			
		||||
        .. code-block:: python
 | 
			
		||||
 | 
			
		||||
            scale_x = 2/(right - left)
 | 
			
		||||
            scale_y = 2/(top - bottom)
 | 
			
		||||
            scale_z = 2/(far-near)
 | 
			
		||||
            mid_x = (right + left)/(right - left)
 | 
			
		||||
            mix_y = (top + bottom)/(top - bottom)
 | 
			
		||||
            mid_z = (far + near)/(far−near)
 | 
			
		||||
            scale_x = 2 / (max_x - min_x)
 | 
			
		||||
            scale_y = 2 / (max_y - min_y)
 | 
			
		||||
            scale_z = 2 / (far-near)
 | 
			
		||||
            mid_x = (max_x + min_x) / (max_x - min_x)
 | 
			
		||||
            mix_y = (max_y + min_y) / (max_y - min_y)
 | 
			
		||||
            mid_z = (far + near) / (far−near)
 | 
			
		||||
 | 
			
		||||
            P = [
 | 
			
		||||
                    [scale_x,        0,         0,  -mid_x],
 | 
			
		||||
@ -476,10 +607,10 @@ class OpenGLOrthographicCameras(CamerasBase):
 | 
			
		||||
        """
 | 
			
		||||
        znear = kwargs.get("znear", self.znear)  # pyre-ignore[16]
 | 
			
		||||
        zfar = kwargs.get("zfar", self.zfar)  # pyre-ignore[16]
 | 
			
		||||
        left = kwargs.get("left", self.left)  # pyre-ignore[16]
 | 
			
		||||
        right = kwargs.get("right", self.right)  # pyre-ignore[16]
 | 
			
		||||
        top = kwargs.get("top", self.top)  # pyre-ignore[16]
 | 
			
		||||
        bottom = kwargs.get("bottom", self.bottom)  # pyre-ignore[16]
 | 
			
		||||
        max_x = kwargs.get("max_x", self.max_x)  # pyre-ignore[16]
 | 
			
		||||
        min_x = kwargs.get("min_x", self.min_x)  # pyre-ignore[16]
 | 
			
		||||
        max_y = kwargs.get("max_y", self.max_y)  # pyre-ignore[16]
 | 
			
		||||
        min_y = kwargs.get("min_y", self.min_y)  # pyre-ignore[16]
 | 
			
		||||
        scale_xyz = kwargs.get("scale_xyz", self.scale_xyz)  # pyre-ignore[16]
 | 
			
		||||
 | 
			
		||||
        P = torch.zeros((self._N, 4, 4), dtype=torch.float32, device=self.device)
 | 
			
		||||
@ -489,10 +620,10 @@ class OpenGLOrthographicCameras(CamerasBase):
 | 
			
		||||
        # right handed coordinate system throughout.
 | 
			
		||||
        z_sign = +1.0
 | 
			
		||||
 | 
			
		||||
        P[:, 0, 0] = (2.0 / (right - left)) * scale_xyz[:, 0]
 | 
			
		||||
        P[:, 1, 1] = (2.0 / (top - bottom)) * scale_xyz[:, 1]
 | 
			
		||||
        P[:, 0, 3] = -(right + left) / (right - left)
 | 
			
		||||
        P[:, 1, 3] = -(top + bottom) / (top - bottom)
 | 
			
		||||
        P[:, 0, 0] = (2.0 / (max_x - min_x)) * scale_xyz[:, 0]
 | 
			
		||||
        P[:, 1, 1] = (2.0 / (max_y - min_y)) * scale_xyz[:, 1]
 | 
			
		||||
        P[:, 0, 3] = -(max_x + min_x) / (max_x - min_x)
 | 
			
		||||
        P[:, 1, 3] = -(max_y + min_y) / (max_y - min_y)
 | 
			
		||||
        P[:, 3, 3] = ones
 | 
			
		||||
 | 
			
		||||
        # NOTE: This maps the z coordinate to the range [0, 1] and replaces the
 | 
			
		||||
@ -500,12 +631,6 @@ class OpenGLOrthographicCameras(CamerasBase):
 | 
			
		||||
        P[:, 2, 2] = z_sign * (1.0 / (zfar - znear)) * scale_xyz[:, 2]
 | 
			
		||||
        P[:, 2, 3] = -znear / (zfar - znear)
 | 
			
		||||
 | 
			
		||||
        # NOTE: This part of the matrix is for z renormalization in OpenGL.
 | 
			
		||||
        # The z is mapped to the range [-1, 1] but this won't work yet in
 | 
			
		||||
        # pytorch3d as the rasterizer ignores faces which have z < 0.
 | 
			
		||||
        # P[:, 2, 2] = z_sign * (2.0 / (far - near)) * scale[:, 2]
 | 
			
		||||
        # P[:, 2, 3] = -(far + near) / (far - near)
 | 
			
		||||
 | 
			
		||||
        transform = Transform3d(device=self.device)
 | 
			
		||||
        transform._matrix = P.transpose(1, 2).contiguous()
 | 
			
		||||
        return transform
 | 
			
		||||
@ -518,7 +643,7 @@ class OpenGLOrthographicCameras(CamerasBase):
 | 
			
		||||
        **kwargs
 | 
			
		||||
    ) -> torch.Tensor:
 | 
			
		||||
        """>!
 | 
			
		||||
        OpenGL cameras further allow for passing depth in world units
 | 
			
		||||
        FoV cameras further allow for passing depth in world units
 | 
			
		||||
        (`scaled_depth_input=False`) or in the [0, 1]-normalized units
 | 
			
		||||
        (`scaled_depth_input=True`)
 | 
			
		||||
 | 
			
		||||
@ -529,9 +654,9 @@ class OpenGLOrthographicCameras(CamerasBase):
 | 
			
		||||
        """
 | 
			
		||||
 | 
			
		||||
        if world_coordinates:
 | 
			
		||||
            to_screen_transform = self.get_full_projection_transform(**kwargs.copy())
 | 
			
		||||
            to_ndc_transform = self.get_full_projection_transform(**kwargs.copy())
 | 
			
		||||
        else:
 | 
			
		||||
            to_screen_transform = self.get_projection_transform(**kwargs.copy())
 | 
			
		||||
            to_ndc_transform = self.get_projection_transform(**kwargs.copy())
 | 
			
		||||
 | 
			
		||||
        if scaled_depth_input:
 | 
			
		||||
            # the input depth is already scaled
 | 
			
		||||
@ -547,22 +672,88 @@ class OpenGLOrthographicCameras(CamerasBase):
 | 
			
		||||
            # cat xy and scaled depth
 | 
			
		||||
            xy_sdepth = torch.cat((xy_depth[..., :2], scaled_depth), dim=-1)
 | 
			
		||||
        # finally invert the transform
 | 
			
		||||
        unprojection_transform = to_screen_transform.inverse()
 | 
			
		||||
        unprojection_transform = to_ndc_transform.inverse()
 | 
			
		||||
        return unprojection_transform.transform_points(xy_sdepth)
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class SfMPerspectiveCameras(CamerasBase):
 | 
			
		||||
############################################################
 | 
			
		||||
#             MultiView Camera Classes                     #
 | 
			
		||||
############################################################
 | 
			
		||||
"""
 | 
			
		||||
Note that the MultiView Cameras accept  parameters in both
 | 
			
		||||
screen and NDC space.
 | 
			
		||||
If the user specifies `image_size` at construction time then
 | 
			
		||||
we assume the parameters are in screen space.
 | 
			
		||||
"""
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
def SfMPerspectiveCameras(
 | 
			
		||||
    focal_length=1.0, principal_point=((0.0, 0.0),), R=r, T=t, device="cpu"
 | 
			
		||||
):
 | 
			
		||||
    """
 | 
			
		||||
    SfMPerspectiveCameras has been DEPRECATED. Use PerspectiveCameras instead.
 | 
			
		||||
    Preserving SfMPerspectiveCameras for backward compatibility.
 | 
			
		||||
    """
 | 
			
		||||
 | 
			
		||||
    warnings.warn(
 | 
			
		||||
        """SfMPerspectiveCameras is deprecated,
 | 
			
		||||
        Use PerspectiveCameras instead.
 | 
			
		||||
        SfMPerspectiveCameras will be removed in future releases.""",
 | 
			
		||||
        PendingDeprecationWarning,
 | 
			
		||||
    )
 | 
			
		||||
 | 
			
		||||
    return PerspectiveCameras(
 | 
			
		||||
        focal_length=focal_length,
 | 
			
		||||
        principal_point=principal_point,
 | 
			
		||||
        R=R,
 | 
			
		||||
        T=T,
 | 
			
		||||
        device=device,
 | 
			
		||||
    )
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class PerspectiveCameras(CamerasBase):
 | 
			
		||||
    """
 | 
			
		||||
    A class which stores a batch of parameters to generate a batch of
 | 
			
		||||
    transformation matrices using the multi-view geometry convention for
 | 
			
		||||
    perspective camera.
 | 
			
		||||
 | 
			
		||||
    Parameters for this camera can be specified in NDC or in screen space.
 | 
			
		||||
    If you wish to provide parameters in screen space, you NEED to provide
 | 
			
		||||
    the image_size = (imwidth, imheight).
 | 
			
		||||
    If you wish to provide parameters in NDC space, you should NOT provide
 | 
			
		||||
    image_size. Providing valid image_size will triger a screen space to
 | 
			
		||||
    NDC space transformation in the camera.
 | 
			
		||||
 | 
			
		||||
    For example, here is how to define cameras on the two spaces.
 | 
			
		||||
 | 
			
		||||
    .. code-block:: python
 | 
			
		||||
        # camera defined in screen space
 | 
			
		||||
        cameras = PerspectiveCameras(
 | 
			
		||||
            focal_length=((22.0, 15.0),),  # (fx_screen, fy_screen)
 | 
			
		||||
            principal_point=((192.0, 128.0),),  # (px_screen, py_screen)
 | 
			
		||||
            image_size=((256, 256),),  # (imwidth, imheight)
 | 
			
		||||
        )
 | 
			
		||||
 | 
			
		||||
        # the equivalent camera defined in NDC space
 | 
			
		||||
        cameras = PerspectiveCameras(
 | 
			
		||||
            focal_length=((0.17875, 0.11718),),  # fx = fx_screen / half_imwidth,
 | 
			
		||||
                                                # fy = fy_screen / half_imheight
 | 
			
		||||
            principal_point=((-0.5, 0),),  # px = - (px_screen - half_imwidth) / half_imwidth,
 | 
			
		||||
                                           # py = - (py_screen - half_imheight) / half_imheight
 | 
			
		||||
        )
 | 
			
		||||
    """
 | 
			
		||||
 | 
			
		||||
    def __init__(
 | 
			
		||||
        self, focal_length=1.0, principal_point=((0.0, 0.0),), R=r, T=t, device="cpu"
 | 
			
		||||
        self,
 | 
			
		||||
        focal_length=1.0,
 | 
			
		||||
        principal_point=((0.0, 0.0),),
 | 
			
		||||
        R=r,
 | 
			
		||||
        T=t,
 | 
			
		||||
        device="cpu",
 | 
			
		||||
        image_size=((-1, -1),),
 | 
			
		||||
    ):
 | 
			
		||||
        """
 | 
			
		||||
        __init__(self, focal_length, principal_point, R, T, device) -> None
 | 
			
		||||
        __init__(self, focal_length, principal_point, R, T, device, image_size) -> None
 | 
			
		||||
 | 
			
		||||
        Args:
 | 
			
		||||
            focal_length: Focal length of the camera in world units.
 | 
			
		||||
@ -574,6 +765,11 @@ class SfMPerspectiveCameras(CamerasBase):
 | 
			
		||||
            R: Rotation matrix of shape (N, 3, 3)
 | 
			
		||||
            T: Translation matrix of shape (N, 3)
 | 
			
		||||
            device: torch.device or string
 | 
			
		||||
            image_size: If image_size = (imwidth, imheight) with imwidth, imheight > 0
 | 
			
		||||
                is provided, the camera parameters are assumed to be in screen
 | 
			
		||||
                space. They will be converted to NDC space.
 | 
			
		||||
                If image_size is not provided, the parameters are assumed to
 | 
			
		||||
                be in NDC space.
 | 
			
		||||
        """
 | 
			
		||||
        # The initializer formats all inputs to torch tensors and broadcasts
 | 
			
		||||
        # all the inputs to have the same batch dimension where necessary.
 | 
			
		||||
@ -583,6 +779,7 @@ class SfMPerspectiveCameras(CamerasBase):
 | 
			
		||||
            principal_point=principal_point,
 | 
			
		||||
            R=R,
 | 
			
		||||
            T=T,
 | 
			
		||||
            image_size=image_size,
 | 
			
		||||
        )
 | 
			
		||||
 | 
			
		||||
    def get_projection_transform(self, **kwargs) -> Transform3d:
 | 
			
		||||
@ -615,9 +812,20 @@ class SfMPerspectiveCameras(CamerasBase):
 | 
			
		||||
        principal_point = kwargs.get("principal_point", self.principal_point)
 | 
			
		||||
        # pyre-ignore[16]
 | 
			
		||||
        focal_length = kwargs.get("focal_length", self.focal_length)
 | 
			
		||||
        # pyre-ignore[16]
 | 
			
		||||
        image_size = kwargs.get("image_size", self.image_size)
 | 
			
		||||
 | 
			
		||||
        # if imwidth > 0, parameters are in screen space
 | 
			
		||||
        in_screen = image_size[0][0] > 0
 | 
			
		||||
        image_size = image_size if in_screen else None
 | 
			
		||||
 | 
			
		||||
        P = _get_sfm_calibration_matrix(
 | 
			
		||||
            self._N, self.device, focal_length, principal_point, False
 | 
			
		||||
            self._N,
 | 
			
		||||
            self.device,
 | 
			
		||||
            focal_length,
 | 
			
		||||
            principal_point,
 | 
			
		||||
            orthographic=False,
 | 
			
		||||
            image_size=image_size,
 | 
			
		||||
        )
 | 
			
		||||
 | 
			
		||||
        transform = Transform3d(device=self.device)
 | 
			
		||||
@ -628,29 +836,83 @@ class SfMPerspectiveCameras(CamerasBase):
 | 
			
		||||
        self, xy_depth: torch.Tensor, world_coordinates: bool = True, **kwargs
 | 
			
		||||
    ) -> torch.Tensor:
 | 
			
		||||
        if world_coordinates:
 | 
			
		||||
            to_screen_transform = self.get_full_projection_transform(**kwargs)
 | 
			
		||||
            to_ndc_transform = self.get_full_projection_transform(**kwargs)
 | 
			
		||||
        else:
 | 
			
		||||
            to_screen_transform = self.get_projection_transform(**kwargs)
 | 
			
		||||
            to_ndc_transform = self.get_projection_transform(**kwargs)
 | 
			
		||||
 | 
			
		||||
        unprojection_transform = to_screen_transform.inverse()
 | 
			
		||||
        unprojection_transform = to_ndc_transform.inverse()
 | 
			
		||||
        xy_inv_depth = torch.cat(
 | 
			
		||||
            (xy_depth[..., :2], 1.0 / xy_depth[..., 2:3]), dim=-1  # type: ignore
 | 
			
		||||
        )
 | 
			
		||||
        return unprojection_transform.transform_points(xy_inv_depth)
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class SfMOrthographicCameras(CamerasBase):
 | 
			
		||||
def SfMOrthographicCameras(
 | 
			
		||||
    focal_length=1.0, principal_point=((0.0, 0.0),), R=r, T=t, device="cpu"
 | 
			
		||||
):
 | 
			
		||||
    """
 | 
			
		||||
    SfMOrthographicCameras has been DEPRECATED. Use OrthographicCameras instead.
 | 
			
		||||
    Preserving SfMOrthographicCameras for backward compatibility.
 | 
			
		||||
    """
 | 
			
		||||
 | 
			
		||||
    warnings.warn(
 | 
			
		||||
        """SfMOrthographicCameras is deprecated,
 | 
			
		||||
        Use OrthographicCameras instead.
 | 
			
		||||
        SfMOrthographicCameras will be removed in future releases.""",
 | 
			
		||||
        PendingDeprecationWarning,
 | 
			
		||||
    )
 | 
			
		||||
 | 
			
		||||
    return OrthographicCameras(
 | 
			
		||||
        focal_length=focal_length,
 | 
			
		||||
        principal_point=principal_point,
 | 
			
		||||
        R=R,
 | 
			
		||||
        T=T,
 | 
			
		||||
        device=device,
 | 
			
		||||
    )
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class OrthographicCameras(CamerasBase):
 | 
			
		||||
    """
 | 
			
		||||
    A class which stores a batch of parameters to generate a batch of
 | 
			
		||||
    transformation matrices using the multi-view geometry convention for
 | 
			
		||||
    orthographic camera.
 | 
			
		||||
 | 
			
		||||
    Parameters for this camera can be specified in NDC or in screen space.
 | 
			
		||||
    If you wish to provide parameters in screen space, you NEED to provide
 | 
			
		||||
    the image_size = (imwidth, imheight).
 | 
			
		||||
    If you wish to provide parameters in NDC space, you should NOT provide
 | 
			
		||||
    image_size. Providing valid image_size will triger a screen space to
 | 
			
		||||
    NDC space transformation in the camera.
 | 
			
		||||
 | 
			
		||||
    For example, here is how to define cameras on the two spaces.
 | 
			
		||||
 | 
			
		||||
    .. code-block:: python
 | 
			
		||||
        # camera defined in screen space
 | 
			
		||||
        cameras = OrthographicCameras(
 | 
			
		||||
            focal_length=((22.0, 15.0),),  # (fx, fy)
 | 
			
		||||
            principal_point=((192.0, 128.0),),  # (px, py)
 | 
			
		||||
            image_size=((256, 256),),  # (imwidth, imheight)
 | 
			
		||||
        )
 | 
			
		||||
 | 
			
		||||
        # the equivalent camera defined in NDC space
 | 
			
		||||
        cameras = OrthographicCameras(
 | 
			
		||||
            focal_length=((0.17875, 0.11718),),  # := (fx / half_imwidth, fy / half_imheight)
 | 
			
		||||
            principal_point=((-0.5, 0),),  # := (- (px - half_imwidth) / half_imwidth,
 | 
			
		||||
                                                 - (py - half_imheight) / half_imheight)
 | 
			
		||||
        )
 | 
			
		||||
    """
 | 
			
		||||
 | 
			
		||||
    def __init__(
 | 
			
		||||
        self, focal_length=1.0, principal_point=((0.0, 0.0),), R=r, T=t, device="cpu"
 | 
			
		||||
        self,
 | 
			
		||||
        focal_length=1.0,
 | 
			
		||||
        principal_point=((0.0, 0.0),),
 | 
			
		||||
        R=r,
 | 
			
		||||
        T=t,
 | 
			
		||||
        device="cpu",
 | 
			
		||||
        image_size=((-1, -1),),
 | 
			
		||||
    ):
 | 
			
		||||
        """
 | 
			
		||||
        __init__(self, focal_length, principal_point, R, T, device) -> None
 | 
			
		||||
        __init__(self, focal_length, principal_point, R, T, device, image_size) -> None
 | 
			
		||||
 | 
			
		||||
        Args:
 | 
			
		||||
            focal_length: Focal length of the camera in world units.
 | 
			
		||||
@ -662,6 +924,11 @@ class SfMOrthographicCameras(CamerasBase):
 | 
			
		||||
            R: Rotation matrix of shape (N, 3, 3)
 | 
			
		||||
            T: Translation matrix of shape (N, 3)
 | 
			
		||||
            device: torch.device or string
 | 
			
		||||
            image_size: If image_size = (imwidth, imheight) with imwidth, imheight > 0
 | 
			
		||||
                is provided, the camera parameters are assumed to be in screen
 | 
			
		||||
                space. They will be converted to NDC space.
 | 
			
		||||
                If image_size is not provided, the parameters are assumed to
 | 
			
		||||
                be in NDC space.
 | 
			
		||||
        """
 | 
			
		||||
        # The initializer formats all inputs to torch tensors and broadcasts
 | 
			
		||||
        # all the inputs to have the same batch dimension where necessary.
 | 
			
		||||
@ -671,6 +938,7 @@ class SfMOrthographicCameras(CamerasBase):
 | 
			
		||||
            principal_point=principal_point,
 | 
			
		||||
            R=R,
 | 
			
		||||
            T=T,
 | 
			
		||||
            image_size=image_size,
 | 
			
		||||
        )
 | 
			
		||||
 | 
			
		||||
    def get_projection_transform(self, **kwargs) -> Transform3d:
 | 
			
		||||
@ -703,9 +971,20 @@ class SfMOrthographicCameras(CamerasBase):
 | 
			
		||||
        principal_point = kwargs.get("principal_point", self.principal_point)
 | 
			
		||||
        # pyre-ignore[16]
 | 
			
		||||
        focal_length = kwargs.get("focal_length", self.focal_length)
 | 
			
		||||
        # pyre-ignore[16]
 | 
			
		||||
        image_size = kwargs.get("image_size", self.image_size)
 | 
			
		||||
 | 
			
		||||
        # if imwidth > 0, parameters are in screen space
 | 
			
		||||
        in_screen = image_size[0][0] > 0
 | 
			
		||||
        image_size = image_size if in_screen else None
 | 
			
		||||
 | 
			
		||||
        P = _get_sfm_calibration_matrix(
 | 
			
		||||
            self._N, self.device, focal_length, principal_point, True
 | 
			
		||||
            self._N,
 | 
			
		||||
            self.device,
 | 
			
		||||
            focal_length,
 | 
			
		||||
            principal_point,
 | 
			
		||||
            orthographic=True,
 | 
			
		||||
            image_size=image_size,
 | 
			
		||||
        )
 | 
			
		||||
 | 
			
		||||
        transform = Transform3d(device=self.device)
 | 
			
		||||
@ -716,17 +995,26 @@ class SfMOrthographicCameras(CamerasBase):
 | 
			
		||||
        self, xy_depth: torch.Tensor, world_coordinates: bool = True, **kwargs
 | 
			
		||||
    ) -> torch.Tensor:
 | 
			
		||||
        if world_coordinates:
 | 
			
		||||
            to_screen_transform = self.get_full_projection_transform(**kwargs)
 | 
			
		||||
            to_ndc_transform = self.get_full_projection_transform(**kwargs)
 | 
			
		||||
        else:
 | 
			
		||||
            to_screen_transform = self.get_projection_transform(**kwargs)
 | 
			
		||||
            to_ndc_transform = self.get_projection_transform(**kwargs)
 | 
			
		||||
 | 
			
		||||
        unprojection_transform = to_screen_transform.inverse()
 | 
			
		||||
        unprojection_transform = to_ndc_transform.inverse()
 | 
			
		||||
        return unprojection_transform.transform_points(xy_depth)
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
# SfMCameras helper
 | 
			
		||||
################################################
 | 
			
		||||
#       Helper functions for cameras           #
 | 
			
		||||
################################################
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
def _get_sfm_calibration_matrix(
 | 
			
		||||
    N, device, focal_length, principal_point, orthographic: bool
 | 
			
		||||
    N,
 | 
			
		||||
    device,
 | 
			
		||||
    focal_length,
 | 
			
		||||
    principal_point,
 | 
			
		||||
    orthographic: bool = False,
 | 
			
		||||
    image_size=None,
 | 
			
		||||
) -> torch.Tensor:
 | 
			
		||||
    """
 | 
			
		||||
    Returns a calibration matrix of a perspective/orthograpic camera.
 | 
			
		||||
@ -736,6 +1024,10 @@ def _get_sfm_calibration_matrix(
 | 
			
		||||
        focal_length: Focal length of the camera in world units.
 | 
			
		||||
        principal_point: xy coordinates of the center of
 | 
			
		||||
            the principal point of the camera in pixels.
 | 
			
		||||
        orthographic: Boolean specifying if the camera is orthographic or not
 | 
			
		||||
        image_size: (Optional) Specifying the image_size = (imwidth, imheight).
 | 
			
		||||
            If not None, the camera parameters are assumed to be in screen space
 | 
			
		||||
            and are transformed to NDC space.
 | 
			
		||||
 | 
			
		||||
        The calibration matrix `K` is set up as follows:
 | 
			
		||||
 | 
			
		||||
@ -769,7 +1061,7 @@ def _get_sfm_calibration_matrix(
 | 
			
		||||
    if not torch.is_tensor(focal_length):
 | 
			
		||||
        focal_length = torch.tensor(focal_length, device=device)
 | 
			
		||||
 | 
			
		||||
    if len(focal_length.shape) in (0, 1) or focal_length.shape[1] == 1:
 | 
			
		||||
    if focal_length.ndim in (0, 1) or focal_length.shape[1] == 1:
 | 
			
		||||
        fx = fy = focal_length
 | 
			
		||||
    else:
 | 
			
		||||
        fx, fy = focal_length.unbind(1)
 | 
			
		||||
@ -779,6 +1071,22 @@ def _get_sfm_calibration_matrix(
 | 
			
		||||
 | 
			
		||||
    px, py = principal_point.unbind(1)
 | 
			
		||||
 | 
			
		||||
    if image_size is not None:
 | 
			
		||||
        if not torch.is_tensor(image_size):
 | 
			
		||||
            image_size = torch.tensor(image_size, device=device)
 | 
			
		||||
        imwidth, imheight = image_size.unbind(1)
 | 
			
		||||
        # make sure imwidth, imheight are valid (>0)
 | 
			
		||||
        if (imwidth < 1).any() or (imheight < 1).any():
 | 
			
		||||
            raise ValueError(
 | 
			
		||||
                "Camera parameters provided in screen space. Image width or height invalid."
 | 
			
		||||
            )
 | 
			
		||||
        half_imwidth = imwidth / 2.0
 | 
			
		||||
        half_imheight = imheight / 2.0
 | 
			
		||||
        fx = fx / half_imwidth
 | 
			
		||||
        fy = fy / half_imheight
 | 
			
		||||
        px = -(px - half_imwidth) / half_imwidth
 | 
			
		||||
        py = -(py - half_imheight) / half_imheight
 | 
			
		||||
 | 
			
		||||
    K = fx.new_zeros(N, 4, 4)
 | 
			
		||||
    K[:, 0, 0] = fx
 | 
			
		||||
    K[:, 1, 1] = fy
 | 
			
		||||
 | 
			
		||||
@ -4,7 +4,7 @@ from itertools import product
 | 
			
		||||
 | 
			
		||||
import torch
 | 
			
		||||
from fvcore.common.benchmark import benchmark
 | 
			
		||||
from pytorch3d.renderer.cameras import OpenGLPerspectiveCameras, look_at_view_transform
 | 
			
		||||
from pytorch3d.renderer.cameras import FoVPerspectiveCameras, look_at_view_transform
 | 
			
		||||
from pytorch3d.renderer.mesh.rasterizer import (
 | 
			
		||||
    Fragments,
 | 
			
		||||
    MeshRasterizer,
 | 
			
		||||
@ -28,7 +28,7 @@ def baryclip_cuda(
 | 
			
		||||
    sphere_meshes = ico_sphere(ico_level, device).extend(num_meshes)
 | 
			
		||||
    # Init transform
 | 
			
		||||
    R, T = look_at_view_transform(1.0, 0.0, 0.0)
 | 
			
		||||
    cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
    cameras = FoVPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
    # Init rasterizer
 | 
			
		||||
    raster_settings = RasterizationSettings(
 | 
			
		||||
        image_size=image_size,
 | 
			
		||||
@ -58,7 +58,7 @@ def baryclip_pytorch(
 | 
			
		||||
    sphere_meshes = ico_sphere(ico_level, device).extend(num_meshes)
 | 
			
		||||
    # Init transform
 | 
			
		||||
    R, T = look_at_view_transform(1.0, 0.0, 0.0)
 | 
			
		||||
    cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
    cameras = FoVPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
    # Init rasterizer
 | 
			
		||||
    raster_settings = RasterizationSettings(
 | 
			
		||||
        image_size=image_size,
 | 
			
		||||
 | 
			
		||||
@ -5,7 +5,7 @@ from itertools import product
 | 
			
		||||
 | 
			
		||||
import torch
 | 
			
		||||
from fvcore.common.benchmark import benchmark
 | 
			
		||||
from pytorch3d.renderer.cameras import OpenGLPerspectiveCameras, look_at_view_transform
 | 
			
		||||
from pytorch3d.renderer.cameras import FoVPerspectiveCameras, look_at_view_transform
 | 
			
		||||
from pytorch3d.renderer.mesh.rasterizer import MeshRasterizer
 | 
			
		||||
from pytorch3d.utils.ico_sphere import ico_sphere
 | 
			
		||||
 | 
			
		||||
@ -15,7 +15,7 @@ def rasterize_transform_with_init(num_meshes: int, ico_level: int = 5, device="c
 | 
			
		||||
    sphere_meshes = ico_sphere(ico_level, device).extend(num_meshes)
 | 
			
		||||
    # Init transform
 | 
			
		||||
    R, T = look_at_view_transform(1.0, 0.0, 0.0)
 | 
			
		||||
    cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
    cameras = FoVPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
    # Init rasterizer
 | 
			
		||||
    rasterizer = MeshRasterizer(cameras=cameras)
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
							
								
								
									
										
											BIN
										
									
								
								tests/data/test_FoVOrthographicCameras_silhouette.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 48 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								tests/data/test_FoVPerspectiveCameras_silhouette.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 8.7 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								tests/data/test_OrthographicCameras_silhouette.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 48 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								tests/data/test_PerspectiveCameras_silhouette.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 5.5 KiB  | 
| 
		 Before Width: | Height: | Size: 8.8 KiB  | 
| 
		 After Width: | Height: | Size: 26 KiB  | 
| 
		 After Width: | Height: | Size: 18 KiB  | 
| 
		 After Width: | Height: | Size: 18 KiB  | 
| 
		 Before Width: | Height: | Size: 3.4 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								tests/data/test_simple_sphere_dark_FoVOrthographicCameras.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 3.7 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								tests/data/test_simple_sphere_dark_FoVPerspectiveCameras.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 2.8 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								tests/data/test_simple_sphere_dark_OrthographicCameras.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 3.7 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								tests/data/test_simple_sphere_dark_PerspectiveCameras.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 2.0 KiB  | 
| 
		 After Width: | Height: | Size: 7.8 KiB  | 
| 
		 After Width: | Height: | Size: 3.8 KiB  | 
| 
		 After Width: | Height: | Size: 7.8 KiB  | 
| 
		 After Width: | Height: | Size: 2.5 KiB  | 
| 
		 Before Width: | Height: | Size: 3.8 KiB  | 
| 
		 Before Width: | Height: | Size: 26 KiB  | 
| 
		 After Width: | Height: | Size: 34 KiB  | 
| 
		 After Width: | Height: | Size: 26 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								tests/data/test_simple_sphere_light_flat_OrthographicCameras.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 34 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								tests/data/test_simple_sphere_light_flat_PerspectiveCameras.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 13 KiB  | 
| 
		 After Width: | Height: | Size: 25 KiB  | 
| 
		 After Width: | Height: | Size: 15 KiB  | 
| 
		 After Width: | Height: | Size: 25 KiB  | 
| 
		 After Width: | Height: | Size: 8.5 KiB  | 
| 
		 Before Width: | Height: | Size: 15 KiB  | 
| 
		 Before Width: | Height: | Size: 18 KiB  | 
| 
		 After Width: | Height: | Size: 23 KiB  | 
| 
		 After Width: | Height: | Size: 18 KiB  | 
| 
		 After Width: | Height: | Size: 23 KiB  | 
| 
		 After Width: | Height: | Size: 9.1 KiB  | 
| 
		 After Width: | Height: | Size: 17 KiB  | 
| 
		 After Width: | Height: | Size: 10 KiB  | 
| 
		 After Width: | Height: | Size: 17 KiB  | 
| 
		 After Width: | Height: | Size: 5.8 KiB  | 
| 
		 Before Width: | Height: | Size: 10 KiB  | 
| 
		 Before Width: | Height: | Size: 21 KiB  | 
| 
		 After Width: | Height: | Size: 22 KiB  | 
| 
		 After Width: | Height: | Size: 18 KiB  | 
| 
		 After Width: | Height: | Size: 22 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								tests/data/test_simple_sphere_light_phong_PerspectiveCameras.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 8.8 KiB  | 
| 
		 After Width: | Height: | Size: 16 KiB  | 
| 
		 After Width: | Height: | Size: 10 KiB  | 
| 
		 After Width: | Height: | Size: 16 KiB  | 
| 
		 After Width: | Height: | Size: 5.6 KiB  | 
| 
		 Before Width: | Height: | Size: 10 KiB  | 
@ -31,12 +31,16 @@ import unittest
 | 
			
		||||
import numpy as np
 | 
			
		||||
import torch
 | 
			
		||||
from common_testing import TestCaseMixin
 | 
			
		||||
from pytorch3d.renderer.cameras import OpenGLOrthographicCameras  # deprecated
 | 
			
		||||
from pytorch3d.renderer.cameras import OpenGLPerspectiveCameras  # deprecated
 | 
			
		||||
from pytorch3d.renderer.cameras import SfMOrthographicCameras  # deprecated
 | 
			
		||||
from pytorch3d.renderer.cameras import SfMPerspectiveCameras  # deprecated
 | 
			
		||||
from pytorch3d.renderer.cameras import (
 | 
			
		||||
    CamerasBase,
 | 
			
		||||
    OpenGLOrthographicCameras,
 | 
			
		||||
    OpenGLPerspectiveCameras,
 | 
			
		||||
    SfMOrthographicCameras,
 | 
			
		||||
    SfMPerspectiveCameras,
 | 
			
		||||
    FoVOrthographicCameras,
 | 
			
		||||
    FoVPerspectiveCameras,
 | 
			
		||||
    OrthographicCameras,
 | 
			
		||||
    PerspectiveCameras,
 | 
			
		||||
    camera_position_from_spherical_angles,
 | 
			
		||||
    get_world_to_view_transform,
 | 
			
		||||
    look_at_rotation,
 | 
			
		||||
@ -109,6 +113,25 @@ def orthographic_project_naive(points, scale_xyz=(1.0, 1.0, 1.0)):
 | 
			
		||||
    return points
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
def ndc_to_screen_points_naive(points, imsize):
 | 
			
		||||
    """
 | 
			
		||||
    Transforms points from PyTorch3D's NDC space to screen space
 | 
			
		||||
    Args:
 | 
			
		||||
        points: (N, V, 3) representing padded points
 | 
			
		||||
        imsize: (N, 2) image size = (width, height)
 | 
			
		||||
    Returns:
 | 
			
		||||
        (N, V, 3) tensor of transformed points
 | 
			
		||||
    """
 | 
			
		||||
    imwidth, imheight = imsize.unbind(1)
 | 
			
		||||
    imwidth = imwidth.view(-1, 1)
 | 
			
		||||
    imheight = imheight.view(-1, 1)
 | 
			
		||||
 | 
			
		||||
    x, y, z = points.unbind(2)
 | 
			
		||||
    x = (1.0 - x) * (imwidth - 1) / 2.0
 | 
			
		||||
    y = (1.0 - y) * (imheight - 1) / 2.0
 | 
			
		||||
    return torch.stack((x, y, z), dim=2)
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class TestCameraHelpers(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
    def setUp(self) -> None:
 | 
			
		||||
        super().setUp()
 | 
			
		||||
@ -359,6 +382,10 @@ class TestCamerasCommon(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
            OpenGLOrthographicCameras,
 | 
			
		||||
            SfMOrthographicCameras,
 | 
			
		||||
            SfMPerspectiveCameras,
 | 
			
		||||
            FoVOrthographicCameras,
 | 
			
		||||
            FoVPerspectiveCameras,
 | 
			
		||||
            OrthographicCameras,
 | 
			
		||||
            PerspectiveCameras,
 | 
			
		||||
        ):
 | 
			
		||||
            cam = cam_type(R=R, T=T)
 | 
			
		||||
            RT_class = cam.get_world_to_view_transform()
 | 
			
		||||
@ -374,6 +401,10 @@ class TestCamerasCommon(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
            OpenGLOrthographicCameras,
 | 
			
		||||
            SfMOrthographicCameras,
 | 
			
		||||
            SfMPerspectiveCameras,
 | 
			
		||||
            FoVOrthographicCameras,
 | 
			
		||||
            FoVPerspectiveCameras,
 | 
			
		||||
            OrthographicCameras,
 | 
			
		||||
            PerspectiveCameras,
 | 
			
		||||
        ):
 | 
			
		||||
            cam = cam_type(R=R, T=T)
 | 
			
		||||
            C = cam.get_camera_center()
 | 
			
		||||
@ -398,13 +429,53 @@ class TestCamerasCommon(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
                cam_params["bottom"] = -(torch.rand(batch_size)) * 0.2 - 0.9
 | 
			
		||||
                cam_params["left"] = -(torch.rand(batch_size)) * 0.2 - 0.9
 | 
			
		||||
                cam_params["right"] = torch.rand(batch_size) * 0.2 + 0.9
 | 
			
		||||
        elif cam_type in (SfMOrthographicCameras, SfMPerspectiveCameras):
 | 
			
		||||
        elif cam_type in (FoVPerspectiveCameras, FoVOrthographicCameras):
 | 
			
		||||
            cam_params["znear"] = torch.rand(batch_size) * 10 + 0.1
 | 
			
		||||
            cam_params["zfar"] = torch.rand(batch_size) * 4 + 1 + cam_params["znear"]
 | 
			
		||||
            if cam_type == FoVPerspectiveCameras:
 | 
			
		||||
                cam_params["fov"] = torch.rand(batch_size) * 60 + 30
 | 
			
		||||
                cam_params["aspect_ratio"] = torch.rand(batch_size) * 0.5 + 0.5
 | 
			
		||||
            else:
 | 
			
		||||
                cam_params["max_y"] = torch.rand(batch_size) * 0.2 + 0.9
 | 
			
		||||
                cam_params["min_y"] = -(torch.rand(batch_size)) * 0.2 - 0.9
 | 
			
		||||
                cam_params["min_x"] = -(torch.rand(batch_size)) * 0.2 - 0.9
 | 
			
		||||
                cam_params["max_x"] = torch.rand(batch_size) * 0.2 + 0.9
 | 
			
		||||
        elif cam_type in (
 | 
			
		||||
            SfMOrthographicCameras,
 | 
			
		||||
            SfMPerspectiveCameras,
 | 
			
		||||
            OrthographicCameras,
 | 
			
		||||
            PerspectiveCameras,
 | 
			
		||||
        ):
 | 
			
		||||
            cam_params["focal_length"] = torch.rand(batch_size) * 10 + 0.1
 | 
			
		||||
            cam_params["principal_point"] = torch.randn((batch_size, 2))
 | 
			
		||||
 | 
			
		||||
        else:
 | 
			
		||||
            raise ValueError(str(cam_type))
 | 
			
		||||
        return cam_type(**cam_params)
 | 
			
		||||
 | 
			
		||||
    @staticmethod
 | 
			
		||||
    def init_equiv_cameras_ndc_screen(cam_type: CamerasBase, batch_size: int):
 | 
			
		||||
        T = torch.randn(batch_size, 3) * 0.03
 | 
			
		||||
        T[:, 2] = 4
 | 
			
		||||
        R = so3_exponential_map(torch.randn(batch_size, 3) * 3.0)
 | 
			
		||||
        screen_cam_params = {"R": R, "T": T}
 | 
			
		||||
        ndc_cam_params = {"R": R, "T": T}
 | 
			
		||||
        if cam_type in (OrthographicCameras, PerspectiveCameras):
 | 
			
		||||
            ndc_cam_params["focal_length"] = torch.rand((batch_size, 2)) * 3.0
 | 
			
		||||
            ndc_cam_params["principal_point"] = torch.randn((batch_size, 2))
 | 
			
		||||
 | 
			
		||||
            image_size = torch.randint(low=2, high=64, size=(batch_size, 2))
 | 
			
		||||
            screen_cam_params["image_size"] = image_size
 | 
			
		||||
            screen_cam_params["focal_length"] = (
 | 
			
		||||
                ndc_cam_params["focal_length"] * image_size / 2.0
 | 
			
		||||
            )
 | 
			
		||||
            screen_cam_params["principal_point"] = (
 | 
			
		||||
                (1.0 - ndc_cam_params["principal_point"]) * image_size / 2.0
 | 
			
		||||
            )
 | 
			
		||||
        else:
 | 
			
		||||
            raise ValueError(str(cam_type))
 | 
			
		||||
        return cam_type(**ndc_cam_params), cam_type(**screen_cam_params)
 | 
			
		||||
 | 
			
		||||
    def test_unproject_points(self, batch_size=50, num_points=100):
 | 
			
		||||
        """
 | 
			
		||||
        Checks that an unprojection of a randomly projected point cloud
 | 
			
		||||
@ -416,6 +487,10 @@ class TestCamerasCommon(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
            OpenGLPerspectiveCameras,
 | 
			
		||||
            OpenGLOrthographicCameras,
 | 
			
		||||
            SfMPerspectiveCameras,
 | 
			
		||||
            FoVOrthographicCameras,
 | 
			
		||||
            FoVPerspectiveCameras,
 | 
			
		||||
            OrthographicCameras,
 | 
			
		||||
            PerspectiveCameras,
 | 
			
		||||
        ):
 | 
			
		||||
            # init the cameras
 | 
			
		||||
            cameras = TestCamerasCommon.init_random_cameras(cam_type, batch_size)
 | 
			
		||||
@ -437,9 +512,14 @@ class TestCamerasCommon(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
                else:
 | 
			
		||||
                    matching_xyz = xyz_cam
 | 
			
		||||
 | 
			
		||||
                # if we have OpenGL cameras
 | 
			
		||||
                # if we have FoV (= OpenGL) cameras
 | 
			
		||||
                # test for scaled_depth_input=True/False
 | 
			
		||||
                if cam_type in (OpenGLPerspectiveCameras, OpenGLOrthographicCameras):
 | 
			
		||||
                if cam_type in (
 | 
			
		||||
                    OpenGLPerspectiveCameras,
 | 
			
		||||
                    OpenGLOrthographicCameras,
 | 
			
		||||
                    FoVPerspectiveCameras,
 | 
			
		||||
                    FoVOrthographicCameras,
 | 
			
		||||
                ):
 | 
			
		||||
                    for scaled_depth_input in (True, False):
 | 
			
		||||
                        if scaled_depth_input:
 | 
			
		||||
                            xy_depth_ = xyz_proj
 | 
			
		||||
@ -459,6 +539,56 @@ class TestCamerasCommon(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
                    )
 | 
			
		||||
                    self.assertTrue(torch.allclose(xyz_unproj, matching_xyz, atol=1e-4))
 | 
			
		||||
 | 
			
		||||
    def test_project_points_screen(self, batch_size=50, num_points=100):
 | 
			
		||||
        """
 | 
			
		||||
        Checks that an unprojection of a randomly projected point cloud
 | 
			
		||||
        stays the same.
 | 
			
		||||
        """
 | 
			
		||||
 | 
			
		||||
        for cam_type in (
 | 
			
		||||
            OpenGLOrthographicCameras,
 | 
			
		||||
            OpenGLPerspectiveCameras,
 | 
			
		||||
            SfMOrthographicCameras,
 | 
			
		||||
            SfMPerspectiveCameras,
 | 
			
		||||
            FoVOrthographicCameras,
 | 
			
		||||
            FoVPerspectiveCameras,
 | 
			
		||||
            OrthographicCameras,
 | 
			
		||||
            PerspectiveCameras,
 | 
			
		||||
        ):
 | 
			
		||||
 | 
			
		||||
            # init the cameras
 | 
			
		||||
            cameras = TestCamerasCommon.init_random_cameras(cam_type, batch_size)
 | 
			
		||||
            # xyz - the ground truth point cloud
 | 
			
		||||
            xyz = torch.randn(batch_size, num_points, 3) * 0.3
 | 
			
		||||
            # image size
 | 
			
		||||
            image_size = torch.randint(low=2, high=64, size=(batch_size, 2))
 | 
			
		||||
            # project points
 | 
			
		||||
            xyz_project_ndc = cameras.transform_points(xyz)
 | 
			
		||||
            xyz_project_screen = cameras.transform_points_screen(xyz, image_size)
 | 
			
		||||
            # naive
 | 
			
		||||
            xyz_project_screen_naive = ndc_to_screen_points_naive(
 | 
			
		||||
                xyz_project_ndc, image_size
 | 
			
		||||
            )
 | 
			
		||||
            self.assertClose(xyz_project_screen, xyz_project_screen_naive)
 | 
			
		||||
 | 
			
		||||
    def test_equiv_project_points(self, batch_size=50, num_points=100):
 | 
			
		||||
        """
 | 
			
		||||
        Checks that NDC and screen cameras project points to ndc correctly.
 | 
			
		||||
        Applies only to OrthographicCameras and PerspectiveCameras.
 | 
			
		||||
        """
 | 
			
		||||
        for cam_type in (OrthographicCameras, PerspectiveCameras):
 | 
			
		||||
            # init the cameras
 | 
			
		||||
            (
 | 
			
		||||
                ndc_cameras,
 | 
			
		||||
                screen_cameras,
 | 
			
		||||
            ) = TestCamerasCommon.init_equiv_cameras_ndc_screen(cam_type, batch_size)
 | 
			
		||||
            # xyz - the ground truth point cloud
 | 
			
		||||
            xyz = torch.randn(batch_size, num_points, 3) * 0.3
 | 
			
		||||
            # project points
 | 
			
		||||
            xyz_ndc_cam = ndc_cameras.transform_points(xyz)
 | 
			
		||||
            xyz_screen_cam = screen_cameras.transform_points(xyz)
 | 
			
		||||
            self.assertClose(xyz_ndc_cam, xyz_screen_cam, atol=1e-6)
 | 
			
		||||
 | 
			
		||||
    def test_clone(self, batch_size: int = 10):
 | 
			
		||||
        """
 | 
			
		||||
        Checks the clone function of the cameras.
 | 
			
		||||
@ -468,6 +598,10 @@ class TestCamerasCommon(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
            OpenGLPerspectiveCameras,
 | 
			
		||||
            OpenGLOrthographicCameras,
 | 
			
		||||
            SfMPerspectiveCameras,
 | 
			
		||||
            FoVOrthographicCameras,
 | 
			
		||||
            FoVPerspectiveCameras,
 | 
			
		||||
            OrthographicCameras,
 | 
			
		||||
            PerspectiveCameras,
 | 
			
		||||
        ):
 | 
			
		||||
            cameras = TestCamerasCommon.init_random_cameras(cam_type, batch_size)
 | 
			
		||||
            cameras = cameras.to(torch.device("cpu"))
 | 
			
		||||
@ -483,11 +617,16 @@ class TestCamerasCommon(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
                    self.assertTrue(val == val_clone)
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class TestPerspectiveProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
############################################################
 | 
			
		||||
#                FoVPerspective Camera                     #
 | 
			
		||||
############################################################
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class TestFoVPerspectiveProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
    def test_perspective(self):
 | 
			
		||||
        far = 10.0
 | 
			
		||||
        near = 1.0
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(znear=near, zfar=far, fov=60.0)
 | 
			
		||||
        cameras = FoVPerspectiveCameras(znear=near, zfar=far, fov=60.0)
 | 
			
		||||
        P = cameras.get_projection_transform()
 | 
			
		||||
        # vertices are at the far clipping plane so z gets mapped to 1.
 | 
			
		||||
        vertices = torch.tensor([1, 2, far], dtype=torch.float32)
 | 
			
		||||
@ -512,7 +651,7 @@ class TestPerspectiveProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        self.assertClose(v1.squeeze(), projected_verts)
 | 
			
		||||
 | 
			
		||||
    def test_perspective_kwargs(self):
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(znear=5.0, zfar=100.0, fov=0.0)
 | 
			
		||||
        cameras = FoVPerspectiveCameras(znear=5.0, zfar=100.0, fov=0.0)
 | 
			
		||||
        # Override defaults by passing in values to get_projection_transform
 | 
			
		||||
        far = 10.0
 | 
			
		||||
        P = cameras.get_projection_transform(znear=1.0, zfar=far, fov=60.0)
 | 
			
		||||
@ -528,7 +667,7 @@ class TestPerspectiveProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        far = torch.tensor([10.0, 20.0], dtype=torch.float32)
 | 
			
		||||
        near = 1.0
 | 
			
		||||
        fov = torch.tensor(60.0)
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(znear=near, zfar=far, fov=fov)
 | 
			
		||||
        cameras = FoVPerspectiveCameras(znear=near, zfar=far, fov=fov)
 | 
			
		||||
        P = cameras.get_projection_transform()
 | 
			
		||||
        vertices = torch.tensor([1, 2, 10], dtype=torch.float32)
 | 
			
		||||
        z1 = 1.0  # vertices at far clipping plane so z = 1.0
 | 
			
		||||
@ -550,7 +689,7 @@ class TestPerspectiveProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        far = torch.tensor([10.0])
 | 
			
		||||
        near = 1.0
 | 
			
		||||
        fov = torch.tensor(60.0, requires_grad=True)
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(znear=near, zfar=far, fov=fov)
 | 
			
		||||
        cameras = FoVPerspectiveCameras(znear=near, zfar=far, fov=fov)
 | 
			
		||||
        P = cameras.get_projection_transform()
 | 
			
		||||
        vertices = torch.tensor([1, 2, 10], dtype=torch.float32)
 | 
			
		||||
        vertices_batch = vertices[None, None, :]
 | 
			
		||||
@ -566,7 +705,7 @@ class TestPerspectiveProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
 | 
			
		||||
    def test_camera_class_init(self):
 | 
			
		||||
        device = torch.device("cuda:0")
 | 
			
		||||
        cam = OpenGLPerspectiveCameras(znear=10.0, zfar=(100.0, 200.0))
 | 
			
		||||
        cam = FoVPerspectiveCameras(znear=10.0, zfar=(100.0, 200.0))
 | 
			
		||||
 | 
			
		||||
        # Check broadcasting
 | 
			
		||||
        self.assertTrue(cam.znear.shape == (2,))
 | 
			
		||||
@ -585,7 +724,7 @@ class TestPerspectiveProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        self.assertTrue(new_cam.device == device)
 | 
			
		||||
 | 
			
		||||
    def test_get_full_transform(self):
 | 
			
		||||
        cam = OpenGLPerspectiveCameras()
 | 
			
		||||
        cam = FoVPerspectiveCameras()
 | 
			
		||||
        T = torch.tensor([0.0, 0.0, 1.0]).view(1, -1)
 | 
			
		||||
        R = look_at_rotation(T)
 | 
			
		||||
        P = cam.get_full_projection_transform(R=R, T=T)
 | 
			
		||||
@ -597,7 +736,7 @@ class TestPerspectiveProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        # Check transform_points methods works with default settings for
 | 
			
		||||
        # RT and P
 | 
			
		||||
        far = 10.0
 | 
			
		||||
        cam = OpenGLPerspectiveCameras(znear=1.0, zfar=far, fov=60.0)
 | 
			
		||||
        cam = FoVPerspectiveCameras(znear=1.0, zfar=far, fov=60.0)
 | 
			
		||||
        points = torch.tensor([1, 2, far], dtype=torch.float32)
 | 
			
		||||
        points = points.view(1, 1, 3).expand(5, 10, -1)
 | 
			
		||||
        projected_points = torch.tensor(
 | 
			
		||||
@ -608,11 +747,16 @@ class TestPerspectiveProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        self.assertClose(new_points, projected_points)
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class TestOpenGLOrthographicProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
############################################################
 | 
			
		||||
#                FoVOrthographic Camera                    #
 | 
			
		||||
############################################################
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class TestFoVOrthographicProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
    def test_orthographic(self):
 | 
			
		||||
        far = 10.0
 | 
			
		||||
        near = 1.0
 | 
			
		||||
        cameras = OpenGLOrthographicCameras(znear=near, zfar=far)
 | 
			
		||||
        cameras = FoVOrthographicCameras(znear=near, zfar=far)
 | 
			
		||||
        P = cameras.get_projection_transform()
 | 
			
		||||
 | 
			
		||||
        vertices = torch.tensor([1, 2, far], dtype=torch.float32)
 | 
			
		||||
@ -637,7 +781,7 @@ class TestOpenGLOrthographicProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        # applying the scale puts the z coordinate at the far clipping plane
 | 
			
		||||
        # so the z is mapped to 1.0
 | 
			
		||||
        projected_verts = torch.tensor([2, 1, 1], dtype=torch.float32)
 | 
			
		||||
        cameras = OpenGLOrthographicCameras(znear=1.0, zfar=10.0, scale_xyz=scale)
 | 
			
		||||
        cameras = FoVOrthographicCameras(znear=1.0, zfar=10.0, scale_xyz=scale)
 | 
			
		||||
        P = cameras.get_projection_transform()
 | 
			
		||||
        v1 = P.transform_points(vertices)
 | 
			
		||||
        v2 = orthographic_project_naive(vertices, scale)
 | 
			
		||||
@ -645,7 +789,7 @@ class TestOpenGLOrthographicProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        self.assertClose(v1, projected_verts[None, None])
 | 
			
		||||
 | 
			
		||||
    def test_orthographic_kwargs(self):
 | 
			
		||||
        cameras = OpenGLOrthographicCameras(znear=5.0, zfar=100.0)
 | 
			
		||||
        cameras = FoVOrthographicCameras(znear=5.0, zfar=100.0)
 | 
			
		||||
        far = 10.0
 | 
			
		||||
        P = cameras.get_projection_transform(znear=1.0, zfar=far)
 | 
			
		||||
        vertices = torch.tensor([1, 2, far], dtype=torch.float32)
 | 
			
		||||
@ -657,7 +801,7 @@ class TestOpenGLOrthographicProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
    def test_orthographic_mixed_inputs_broadcast(self):
 | 
			
		||||
        far = torch.tensor([10.0, 20.0])
 | 
			
		||||
        near = 1.0
 | 
			
		||||
        cameras = OpenGLOrthographicCameras(znear=near, zfar=far)
 | 
			
		||||
        cameras = FoVOrthographicCameras(znear=near, zfar=far)
 | 
			
		||||
        P = cameras.get_projection_transform()
 | 
			
		||||
        vertices = torch.tensor([1.0, 2.0, 10.0], dtype=torch.float32)
 | 
			
		||||
        z2 = 1.0 / (20.0 - 1.0) * 10.0 + -1.0 / (20.0 - 1.0)
 | 
			
		||||
@ -674,7 +818,7 @@ class TestOpenGLOrthographicProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        far = torch.tensor([10.0])
 | 
			
		||||
        near = 1.0
 | 
			
		||||
        scale = torch.tensor([[1.0, 1.0, 1.0]], requires_grad=True)
 | 
			
		||||
        cameras = OpenGLOrthographicCameras(znear=near, zfar=far, scale_xyz=scale)
 | 
			
		||||
        cameras = FoVOrthographicCameras(znear=near, zfar=far, scale_xyz=scale)
 | 
			
		||||
        P = cameras.get_projection_transform()
 | 
			
		||||
        vertices = torch.tensor([1.0, 2.0, 10.0], dtype=torch.float32)
 | 
			
		||||
        vertices_batch = vertices[None, None, :]
 | 
			
		||||
@ -694,9 +838,14 @@ class TestOpenGLOrthographicProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        self.assertClose(scale_grad, grad_scale)
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class TestSfMOrthographicProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
############################################################
 | 
			
		||||
#                Orthographic Camera                       #
 | 
			
		||||
############################################################
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class TestOrthographicProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
    def test_orthographic(self):
 | 
			
		||||
        cameras = SfMOrthographicCameras()
 | 
			
		||||
        cameras = OrthographicCameras()
 | 
			
		||||
        P = cameras.get_projection_transform()
 | 
			
		||||
 | 
			
		||||
        vertices = torch.randn([3, 4, 3], dtype=torch.float32)
 | 
			
		||||
@ -711,9 +860,7 @@ class TestSfMOrthographicProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        focal_length_x = 10.0
 | 
			
		||||
        focal_length_y = 15.0
 | 
			
		||||
 | 
			
		||||
        cameras = SfMOrthographicCameras(
 | 
			
		||||
            focal_length=((focal_length_x, focal_length_y),)
 | 
			
		||||
        )
 | 
			
		||||
        cameras = OrthographicCameras(focal_length=((focal_length_x, focal_length_y),))
 | 
			
		||||
        P = cameras.get_projection_transform()
 | 
			
		||||
 | 
			
		||||
        vertices = torch.randn([3, 4, 3], dtype=torch.float32)
 | 
			
		||||
@ -730,9 +877,7 @@ class TestSfMOrthographicProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        self.assertClose(v1, projected_verts)
 | 
			
		||||
 | 
			
		||||
    def test_orthographic_kwargs(self):
 | 
			
		||||
        cameras = SfMOrthographicCameras(
 | 
			
		||||
            focal_length=5.0, principal_point=((2.5, 2.5),)
 | 
			
		||||
        )
 | 
			
		||||
        cameras = OrthographicCameras(focal_length=5.0, principal_point=((2.5, 2.5),))
 | 
			
		||||
        P = cameras.get_projection_transform(
 | 
			
		||||
            focal_length=2.0, principal_point=((2.5, 3.5),)
 | 
			
		||||
        )
 | 
			
		||||
@ -745,9 +890,14 @@ class TestSfMOrthographicProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        self.assertClose(v1, projected_verts)
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class TestSfMPerspectiveProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
############################################################
 | 
			
		||||
#                Perspective Camera                        #
 | 
			
		||||
############################################################
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
class TestPerspectiveProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
    def test_perspective(self):
 | 
			
		||||
        cameras = SfMPerspectiveCameras()
 | 
			
		||||
        cameras = PerspectiveCameras()
 | 
			
		||||
        P = cameras.get_projection_transform()
 | 
			
		||||
 | 
			
		||||
        vertices = torch.randn([3, 4, 3], dtype=torch.float32)
 | 
			
		||||
@ -761,7 +911,7 @@ class TestSfMPerspectiveProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        p0x = 15.0
 | 
			
		||||
        p0y = 30.0
 | 
			
		||||
 | 
			
		||||
        cameras = SfMPerspectiveCameras(
 | 
			
		||||
        cameras = PerspectiveCameras(
 | 
			
		||||
            focal_length=((focal_length_x, focal_length_y),),
 | 
			
		||||
            principal_point=((p0x, p0y),),
 | 
			
		||||
        )
 | 
			
		||||
@ -777,7 +927,7 @@ class TestSfMPerspectiveProjection(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        self.assertClose(v3[..., :2], v2[..., :2])
 | 
			
		||||
 | 
			
		||||
    def test_perspective_kwargs(self):
 | 
			
		||||
        cameras = SfMPerspectiveCameras(focal_length=5.0, principal_point=((2.5, 2.5),))
 | 
			
		||||
        cameras = PerspectiveCameras(focal_length=5.0, principal_point=((2.5, 2.5),))
 | 
			
		||||
        P = cameras.get_projection_transform(
 | 
			
		||||
            focal_length=2.0, principal_point=((2.5, 3.5),)
 | 
			
		||||
        )
 | 
			
		||||
 | 
			
		||||
@ -18,7 +18,7 @@ from pytorch3d.datasets import (
 | 
			
		||||
    render_cubified_voxels,
 | 
			
		||||
)
 | 
			
		||||
from pytorch3d.renderer import (
 | 
			
		||||
    OpenGLPerspectiveCameras,
 | 
			
		||||
    FoVPerspectiveCameras,
 | 
			
		||||
    PointLights,
 | 
			
		||||
    RasterizationSettings,
 | 
			
		||||
    look_at_view_transform,
 | 
			
		||||
@ -211,7 +211,7 @@ class TestR2N2(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
 | 
			
		||||
        # Render first three models in the dataset.
 | 
			
		||||
        R, T = look_at_view_transform(1.0, 1.0, 90)
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(R=R, T=T, device=device)
 | 
			
		||||
        cameras = FoVPerspectiveCameras(R=R, T=T, device=device)
 | 
			
		||||
        raster_settings = RasterizationSettings(image_size=512)
 | 
			
		||||
        lights = PointLights(
 | 
			
		||||
            location=torch.tensor([0.0, 1.0, -2.0], device=device)[None],
 | 
			
		||||
 | 
			
		||||
@ -7,7 +7,7 @@ from pathlib import Path
 | 
			
		||||
import numpy as np
 | 
			
		||||
import torch
 | 
			
		||||
from PIL import Image
 | 
			
		||||
from pytorch3d.renderer.cameras import OpenGLPerspectiveCameras, look_at_view_transform
 | 
			
		||||
from pytorch3d.renderer.cameras import FoVPerspectiveCameras, look_at_view_transform
 | 
			
		||||
from pytorch3d.renderer.mesh.rasterizer import MeshRasterizer, RasterizationSettings
 | 
			
		||||
from pytorch3d.renderer.points.rasterizer import (
 | 
			
		||||
    PointsRasterizationSettings,
 | 
			
		||||
@ -43,7 +43,7 @@ class TestMeshRasterizer(unittest.TestCase):
 | 
			
		||||
 | 
			
		||||
        # Init rasterizer settings
 | 
			
		||||
        R, T = look_at_view_transform(2.7, 0, 0)
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        cameras = FoVPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        raster_settings = RasterizationSettings(
 | 
			
		||||
            image_size=512, blur_radius=0.0, faces_per_pixel=1, bin_size=0
 | 
			
		||||
        )
 | 
			
		||||
@ -148,7 +148,7 @@ class TestPointRasterizer(unittest.TestCase):
 | 
			
		||||
        verts_padded[..., 0] += 0.2
 | 
			
		||||
        pointclouds = Pointclouds(points=verts_padded)
 | 
			
		||||
        R, T = look_at_view_transform(2.7, 0.0, 0.0)
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        cameras = FoVPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        raster_settings = PointsRasterizationSettings(
 | 
			
		||||
            image_size=256, radius=5e-2, points_per_pixel=1
 | 
			
		||||
        )
 | 
			
		||||
 | 
			
		||||
@ -4,6 +4,7 @@
 | 
			
		||||
"""
 | 
			
		||||
Sanity checks for output images from the renderer.
 | 
			
		||||
"""
 | 
			
		||||
import os
 | 
			
		||||
import unittest
 | 
			
		||||
from pathlib import Path
 | 
			
		||||
 | 
			
		||||
@ -12,7 +13,13 @@ import torch
 | 
			
		||||
from common_testing import TestCaseMixin, load_rgb_image
 | 
			
		||||
from PIL import Image
 | 
			
		||||
from pytorch3d.io import load_obj
 | 
			
		||||
from pytorch3d.renderer.cameras import OpenGLPerspectiveCameras, look_at_view_transform
 | 
			
		||||
from pytorch3d.renderer.cameras import (
 | 
			
		||||
    FoVOrthographicCameras,
 | 
			
		||||
    FoVPerspectiveCameras,
 | 
			
		||||
    OrthographicCameras,
 | 
			
		||||
    PerspectiveCameras,
 | 
			
		||||
    look_at_view_transform,
 | 
			
		||||
)
 | 
			
		||||
from pytorch3d.renderer.lighting import PointLights
 | 
			
		||||
from pytorch3d.renderer.materials import Materials
 | 
			
		||||
from pytorch3d.renderer.mesh import TexturesAtlas, TexturesUV, TexturesVertex
 | 
			
		||||
@ -60,78 +67,94 @@ class TestRenderMeshes(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        if elevated_camera:
 | 
			
		||||
            # Elevated and rotated camera
 | 
			
		||||
            R, T = look_at_view_transform(dist=2.7, elev=45.0, azim=45.0)
 | 
			
		||||
            postfix = "_elevated_camera"
 | 
			
		||||
            postfix = "_elevated_"
 | 
			
		||||
            # If y axis is up, the spot of light should
 | 
			
		||||
            # be on the bottom left of the sphere.
 | 
			
		||||
        else:
 | 
			
		||||
            # No elevation or azimuth rotation
 | 
			
		||||
            R, T = look_at_view_transform(2.7, 0.0, 0.0)
 | 
			
		||||
            postfix = ""
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
            postfix = "_"
 | 
			
		||||
        for cam_type in (
 | 
			
		||||
            FoVPerspectiveCameras,
 | 
			
		||||
            FoVOrthographicCameras,
 | 
			
		||||
            PerspectiveCameras,
 | 
			
		||||
            OrthographicCameras,
 | 
			
		||||
        ):
 | 
			
		||||
            cameras = cam_type(device=device, R=R, T=T)
 | 
			
		||||
 | 
			
		||||
        # Init shader settings
 | 
			
		||||
        materials = Materials(device=device)
 | 
			
		||||
        lights = PointLights(device=device)
 | 
			
		||||
        lights.location = torch.tensor([0.0, 0.0, +2.0], device=device)[None]
 | 
			
		||||
            # Init shader settings
 | 
			
		||||
            materials = Materials(device=device)
 | 
			
		||||
            lights = PointLights(device=device)
 | 
			
		||||
            lights.location = torch.tensor([0.0, 0.0, +2.0], device=device)[None]
 | 
			
		||||
 | 
			
		||||
        raster_settings = RasterizationSettings(
 | 
			
		||||
            image_size=512, blur_radius=0.0, faces_per_pixel=1
 | 
			
		||||
        )
 | 
			
		||||
        rasterizer = MeshRasterizer(cameras=cameras, raster_settings=raster_settings)
 | 
			
		||||
        blend_params = BlendParams(1e-4, 1e-4, (0, 0, 0))
 | 
			
		||||
            raster_settings = RasterizationSettings(
 | 
			
		||||
                image_size=512, blur_radius=0.0, faces_per_pixel=1
 | 
			
		||||
            )
 | 
			
		||||
            rasterizer = MeshRasterizer(
 | 
			
		||||
                cameras=cameras, raster_settings=raster_settings
 | 
			
		||||
            )
 | 
			
		||||
            blend_params = BlendParams(1e-4, 1e-4, (0, 0, 0))
 | 
			
		||||
 | 
			
		||||
        # Test several shaders
 | 
			
		||||
        shaders = {
 | 
			
		||||
            "phong": HardPhongShader,
 | 
			
		||||
            "gouraud": HardGouraudShader,
 | 
			
		||||
            "flat": HardFlatShader,
 | 
			
		||||
        }
 | 
			
		||||
        for (name, shader_init) in shaders.items():
 | 
			
		||||
            shader = shader_init(
 | 
			
		||||
            # Test several shaders
 | 
			
		||||
            shaders = {
 | 
			
		||||
                "phong": HardPhongShader,
 | 
			
		||||
                "gouraud": HardGouraudShader,
 | 
			
		||||
                "flat": HardFlatShader,
 | 
			
		||||
            }
 | 
			
		||||
            for (name, shader_init) in shaders.items():
 | 
			
		||||
                shader = shader_init(
 | 
			
		||||
                    lights=lights,
 | 
			
		||||
                    cameras=cameras,
 | 
			
		||||
                    materials=materials,
 | 
			
		||||
                    blend_params=blend_params,
 | 
			
		||||
                )
 | 
			
		||||
                renderer = MeshRenderer(rasterizer=rasterizer, shader=shader)
 | 
			
		||||
                images = renderer(sphere_mesh)
 | 
			
		||||
                rgb = images[0, ..., :3].squeeze().cpu()
 | 
			
		||||
                filename = "simple_sphere_light_%s%s%s.png" % (
 | 
			
		||||
                    name,
 | 
			
		||||
                    postfix,
 | 
			
		||||
                    cam_type.__name__,
 | 
			
		||||
                )
 | 
			
		||||
 | 
			
		||||
                image_ref = load_rgb_image("test_%s" % filename, DATA_DIR)
 | 
			
		||||
                self.assertClose(rgb, image_ref, atol=0.05)
 | 
			
		||||
 | 
			
		||||
                if DEBUG:
 | 
			
		||||
                    filename = "DEBUG_%s" % filename
 | 
			
		||||
                    Image.fromarray((rgb.numpy() * 255).astype(np.uint8)).save(
 | 
			
		||||
                        DATA_DIR / filename
 | 
			
		||||
                    )
 | 
			
		||||
 | 
			
		||||
            ########################################################
 | 
			
		||||
            # Move the light to the +z axis in world space so it is
 | 
			
		||||
            # behind the sphere. Note that +Z is in, +Y up,
 | 
			
		||||
            # +X left for both world and camera space.
 | 
			
		||||
            ########################################################
 | 
			
		||||
            lights.location[..., 2] = -2.0
 | 
			
		||||
            phong_shader = HardPhongShader(
 | 
			
		||||
                lights=lights,
 | 
			
		||||
                cameras=cameras,
 | 
			
		||||
                materials=materials,
 | 
			
		||||
                blend_params=blend_params,
 | 
			
		||||
            )
 | 
			
		||||
            renderer = MeshRenderer(rasterizer=rasterizer, shader=shader)
 | 
			
		||||
            images = renderer(sphere_mesh)
 | 
			
		||||
            filename = "simple_sphere_light_%s%s.png" % (name, postfix)
 | 
			
		||||
            image_ref = load_rgb_image("test_%s" % filename, DATA_DIR)
 | 
			
		||||
            phong_renderer = MeshRenderer(rasterizer=rasterizer, shader=phong_shader)
 | 
			
		||||
            images = phong_renderer(sphere_mesh, lights=lights)
 | 
			
		||||
            rgb = images[0, ..., :3].squeeze().cpu()
 | 
			
		||||
 | 
			
		||||
            if DEBUG:
 | 
			
		||||
                filename = "DEBUG_%s" % filename
 | 
			
		||||
                filename = "DEBUG_simple_sphere_dark%s%s.png" % (
 | 
			
		||||
                    postfix,
 | 
			
		||||
                    cam_type.__name__,
 | 
			
		||||
                )
 | 
			
		||||
                Image.fromarray((rgb.numpy() * 255).astype(np.uint8)).save(
 | 
			
		||||
                    DATA_DIR / filename
 | 
			
		||||
                )
 | 
			
		||||
            self.assertClose(rgb, image_ref, atol=0.05)
 | 
			
		||||
 | 
			
		||||
        ########################################################
 | 
			
		||||
        # Move the light to the +z axis in world space so it is
 | 
			
		||||
        # behind the sphere. Note that +Z is in, +Y up,
 | 
			
		||||
        # +X left for both world and camera space.
 | 
			
		||||
        ########################################################
 | 
			
		||||
        lights.location[..., 2] = -2.0
 | 
			
		||||
        phong_shader = HardPhongShader(
 | 
			
		||||
            lights=lights,
 | 
			
		||||
            cameras=cameras,
 | 
			
		||||
            materials=materials,
 | 
			
		||||
            blend_params=blend_params,
 | 
			
		||||
        )
 | 
			
		||||
        phong_renderer = MeshRenderer(rasterizer=rasterizer, shader=phong_shader)
 | 
			
		||||
        images = phong_renderer(sphere_mesh, lights=lights)
 | 
			
		||||
        rgb = images[0, ..., :3].squeeze().cpu()
 | 
			
		||||
        if DEBUG:
 | 
			
		||||
            filename = "DEBUG_simple_sphere_dark%s.png" % postfix
 | 
			
		||||
            Image.fromarray((rgb.numpy() * 255).astype(np.uint8)).save(
 | 
			
		||||
                DATA_DIR / filename
 | 
			
		||||
            image_ref_phong_dark = load_rgb_image(
 | 
			
		||||
                "test_simple_sphere_dark%s%s.png" % (postfix, cam_type.__name__),
 | 
			
		||||
                DATA_DIR,
 | 
			
		||||
            )
 | 
			
		||||
 | 
			
		||||
        # Load reference image
 | 
			
		||||
        image_ref_phong_dark = load_rgb_image(
 | 
			
		||||
            "test_simple_sphere_dark%s.png" % postfix, DATA_DIR
 | 
			
		||||
        )
 | 
			
		||||
        self.assertClose(rgb, image_ref_phong_dark, atol=0.05)
 | 
			
		||||
            self.assertClose(rgb, image_ref_phong_dark, atol=0.05)
 | 
			
		||||
 | 
			
		||||
    def test_simple_sphere_elevated_camera(self):
 | 
			
		||||
        """
 | 
			
		||||
@ -142,6 +165,60 @@ class TestRenderMeshes(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        """
 | 
			
		||||
        self.test_simple_sphere(elevated_camera=True)
 | 
			
		||||
 | 
			
		||||
    def test_simple_sphere_screen(self):
 | 
			
		||||
 | 
			
		||||
        """
 | 
			
		||||
        Test output when rendering with PerspectiveCameras & OrthographicCameras
 | 
			
		||||
        in NDC vs screen space.
 | 
			
		||||
        """
 | 
			
		||||
        device = torch.device("cuda:0")
 | 
			
		||||
 | 
			
		||||
        # Init mesh
 | 
			
		||||
        sphere_mesh = ico_sphere(5, device)
 | 
			
		||||
        verts_padded = sphere_mesh.verts_padded()
 | 
			
		||||
        faces_padded = sphere_mesh.faces_padded()
 | 
			
		||||
        feats = torch.ones_like(verts_padded, device=device)
 | 
			
		||||
        textures = TexturesVertex(verts_features=feats)
 | 
			
		||||
        sphere_mesh = Meshes(verts=verts_padded, faces=faces_padded, textures=textures)
 | 
			
		||||
 | 
			
		||||
        R, T = look_at_view_transform(2.7, 0.0, 0.0)
 | 
			
		||||
 | 
			
		||||
        # Init shader settings
 | 
			
		||||
        materials = Materials(device=device)
 | 
			
		||||
        lights = PointLights(device=device)
 | 
			
		||||
        lights.location = torch.tensor([0.0, 0.0, +2.0], device=device)[None]
 | 
			
		||||
 | 
			
		||||
        raster_settings = RasterizationSettings(
 | 
			
		||||
            image_size=512, blur_radius=0.0, faces_per_pixel=1
 | 
			
		||||
        )
 | 
			
		||||
        for cam_type in (PerspectiveCameras, OrthographicCameras):
 | 
			
		||||
            cameras = cam_type(
 | 
			
		||||
                device=device,
 | 
			
		||||
                R=R,
 | 
			
		||||
                T=T,
 | 
			
		||||
                principal_point=((256.0, 256.0),),
 | 
			
		||||
                focal_length=((256.0, 256.0),),
 | 
			
		||||
                image_size=((512, 512),),
 | 
			
		||||
            )
 | 
			
		||||
            rasterizer = MeshRasterizer(
 | 
			
		||||
                cameras=cameras, raster_settings=raster_settings
 | 
			
		||||
            )
 | 
			
		||||
            blend_params = BlendParams(1e-4, 1e-4, (0, 0, 0))
 | 
			
		||||
 | 
			
		||||
            shader = HardPhongShader(
 | 
			
		||||
                lights=lights,
 | 
			
		||||
                cameras=cameras,
 | 
			
		||||
                materials=materials,
 | 
			
		||||
                blend_params=blend_params,
 | 
			
		||||
            )
 | 
			
		||||
            renderer = MeshRenderer(rasterizer=rasterizer, shader=shader)
 | 
			
		||||
            images = renderer(sphere_mesh)
 | 
			
		||||
            rgb = images[0, ..., :3].squeeze().cpu()
 | 
			
		||||
            filename = "test_simple_sphere_light_phong_%s.png" % cam_type.__name__
 | 
			
		||||
 | 
			
		||||
            image_ref = load_rgb_image(filename, DATA_DIR)
 | 
			
		||||
            self.assertClose(rgb, image_ref, atol=0.05)
 | 
			
		||||
 | 
			
		||||
    def test_simple_sphere_batched(self):
 | 
			
		||||
        """
 | 
			
		||||
        Test a mesh with vertex textures can be extended to form a batch, and
 | 
			
		||||
@ -165,7 +242,7 @@ class TestRenderMeshes(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        elev = torch.zeros_like(dist)
 | 
			
		||||
        azim = torch.zeros_like(dist)
 | 
			
		||||
        R, T = look_at_view_transform(dist, elev, azim)
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        cameras = FoVPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        raster_settings = RasterizationSettings(
 | 
			
		||||
            image_size=512, blur_radius=0.0, faces_per_pixel=1
 | 
			
		||||
        )
 | 
			
		||||
@ -193,12 +270,16 @@ class TestRenderMeshes(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
            renderer = MeshRenderer(rasterizer=rasterizer, shader=shader)
 | 
			
		||||
            images = renderer(sphere_meshes)
 | 
			
		||||
            image_ref = load_rgb_image(
 | 
			
		||||
                "test_simple_sphere_light_%s.png" % name, DATA_DIR
 | 
			
		||||
                "test_simple_sphere_light_%s_%s.png" % (name, type(cameras).__name__),
 | 
			
		||||
                DATA_DIR,
 | 
			
		||||
            )
 | 
			
		||||
            for i in range(batch_size):
 | 
			
		||||
                rgb = images[i, ..., :3].squeeze().cpu()
 | 
			
		||||
                if i == 0 and DEBUG:
 | 
			
		||||
                    filename = "DEBUG_simple_sphere_batched_%s.png" % name
 | 
			
		||||
                    filename = "DEBUG_simple_sphere_batched_%s_%s.png" % (
 | 
			
		||||
                        name,
 | 
			
		||||
                        type(cameras).__name__,
 | 
			
		||||
                    )
 | 
			
		||||
                    Image.fromarray((rgb.numpy() * 255).astype(np.uint8)).save(
 | 
			
		||||
                        DATA_DIR / filename
 | 
			
		||||
                    )
 | 
			
		||||
@ -209,8 +290,6 @@ class TestRenderMeshes(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        Test silhouette blending. Also check that gradient calculation works.
 | 
			
		||||
        """
 | 
			
		||||
        device = torch.device("cuda:0")
 | 
			
		||||
        ref_filename = "test_silhouette.png"
 | 
			
		||||
        image_ref_filename = DATA_DIR / ref_filename
 | 
			
		||||
        sphere_mesh = ico_sphere(5, device)
 | 
			
		||||
        verts, faces = sphere_mesh.get_mesh_verts_faces(0)
 | 
			
		||||
        sphere_mesh = Meshes(verts=[verts], faces=[faces])
 | 
			
		||||
@ -225,32 +304,45 @@ class TestRenderMeshes(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
 | 
			
		||||
        # Init rasterizer settings
 | 
			
		||||
        R, T = look_at_view_transform(2.7, 0, 0)
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        for cam_type in (
 | 
			
		||||
            FoVPerspectiveCameras,
 | 
			
		||||
            FoVOrthographicCameras,
 | 
			
		||||
            PerspectiveCameras,
 | 
			
		||||
            OrthographicCameras,
 | 
			
		||||
        ):
 | 
			
		||||
            cameras = cam_type(device=device, R=R, T=T)
 | 
			
		||||
 | 
			
		||||
        # Init renderer
 | 
			
		||||
        renderer = MeshRenderer(
 | 
			
		||||
            rasterizer=MeshRasterizer(cameras=cameras, raster_settings=raster_settings),
 | 
			
		||||
            shader=SoftSilhouetteShader(blend_params=blend_params),
 | 
			
		||||
        )
 | 
			
		||||
        images = renderer(sphere_mesh)
 | 
			
		||||
        alpha = images[0, ..., 3].squeeze().cpu()
 | 
			
		||||
        if DEBUG:
 | 
			
		||||
            Image.fromarray((alpha.numpy() * 255).astype(np.uint8)).save(
 | 
			
		||||
                DATA_DIR / "DEBUG_silhouette.png"
 | 
			
		||||
            # Init renderer
 | 
			
		||||
            renderer = MeshRenderer(
 | 
			
		||||
                rasterizer=MeshRasterizer(
 | 
			
		||||
                    cameras=cameras, raster_settings=raster_settings
 | 
			
		||||
                ),
 | 
			
		||||
                shader=SoftSilhouetteShader(blend_params=blend_params),
 | 
			
		||||
            )
 | 
			
		||||
            images = renderer(sphere_mesh)
 | 
			
		||||
            alpha = images[0, ..., 3].squeeze().cpu()
 | 
			
		||||
            if DEBUG:
 | 
			
		||||
                filename = os.path.join(
 | 
			
		||||
                    DATA_DIR, "DEBUG_%s_silhouette.png" % (cam_type.__name__)
 | 
			
		||||
                )
 | 
			
		||||
                Image.fromarray((alpha.detach().numpy() * 255).astype(np.uint8)).save(
 | 
			
		||||
                    filename
 | 
			
		||||
                )
 | 
			
		||||
 | 
			
		||||
        with Image.open(image_ref_filename) as raw_image_ref:
 | 
			
		||||
            image_ref = torch.from_numpy(np.array(raw_image_ref))
 | 
			
		||||
            ref_filename = "test_%s_silhouette.png" % (cam_type.__name__)
 | 
			
		||||
            image_ref_filename = DATA_DIR / ref_filename
 | 
			
		||||
            with Image.open(image_ref_filename) as raw_image_ref:
 | 
			
		||||
                image_ref = torch.from_numpy(np.array(raw_image_ref))
 | 
			
		||||
 | 
			
		||||
        image_ref = image_ref.to(dtype=torch.float32) / 255.0
 | 
			
		||||
        self.assertClose(alpha, image_ref, atol=0.055)
 | 
			
		||||
            image_ref = image_ref.to(dtype=torch.float32) / 255.0
 | 
			
		||||
            self.assertClose(alpha, image_ref, atol=0.055)
 | 
			
		||||
 | 
			
		||||
        # Check grad exist
 | 
			
		||||
        verts.requires_grad = True
 | 
			
		||||
        sphere_mesh = Meshes(verts=[verts], faces=[faces])
 | 
			
		||||
        images = renderer(sphere_mesh)
 | 
			
		||||
        images[0, ...].sum().backward()
 | 
			
		||||
        self.assertIsNotNone(verts.grad)
 | 
			
		||||
            # Check grad exist
 | 
			
		||||
            verts.requires_grad = True
 | 
			
		||||
            sphere_mesh = Meshes(verts=[verts], faces=[faces])
 | 
			
		||||
            images = renderer(sphere_mesh)
 | 
			
		||||
            images[0, ...].sum().backward()
 | 
			
		||||
            self.assertIsNotNone(verts.grad)
 | 
			
		||||
 | 
			
		||||
    def test_texture_map(self):
 | 
			
		||||
        """
 | 
			
		||||
@ -274,7 +366,7 @@ class TestRenderMeshes(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
 | 
			
		||||
        # Init rasterizer settings
 | 
			
		||||
        R, T = look_at_view_transform(2.7, 0, 0)
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        cameras = FoVPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
 | 
			
		||||
        raster_settings = RasterizationSettings(
 | 
			
		||||
            image_size=512, blur_radius=0.0, faces_per_pixel=1
 | 
			
		||||
@ -337,7 +429,7 @@ class TestRenderMeshes(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        ##########################################
 | 
			
		||||
 | 
			
		||||
        R, T = look_at_view_transform(2.7, 0, 180)
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        cameras = FoVPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
 | 
			
		||||
        # Move light to the front of the cow in world space
 | 
			
		||||
        lights.location = torch.tensor([0.0, 0.0, -2.0], device=device)[None]
 | 
			
		||||
@ -367,7 +459,7 @@ class TestRenderMeshes(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        # Add blurring to rasterization
 | 
			
		||||
        #################################
 | 
			
		||||
        R, T = look_at_view_transform(2.7, 0, 180)
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        cameras = FoVPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        blend_params = BlendParams(sigma=5e-4, gamma=1e-4)
 | 
			
		||||
        raster_settings = RasterizationSettings(
 | 
			
		||||
            image_size=512,
 | 
			
		||||
@ -429,7 +521,7 @@ class TestRenderMeshes(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
 | 
			
		||||
        # Init rasterizer settings
 | 
			
		||||
        R, T = look_at_view_transform(2.7, 0.0, 0.0)
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        cameras = FoVPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        raster_settings = RasterizationSettings(
 | 
			
		||||
            image_size=512, blur_radius=0.0, faces_per_pixel=1
 | 
			
		||||
        )
 | 
			
		||||
@ -490,7 +582,7 @@ class TestRenderMeshes(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
 | 
			
		||||
        # Init rasterizer settings
 | 
			
		||||
        R, T = look_at_view_transform(2.7, 0, 0)
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        cameras = FoVPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
 | 
			
		||||
        raster_settings = RasterizationSettings(
 | 
			
		||||
            image_size=512, blur_radius=0.0, faces_per_pixel=1, cull_backfaces=True
 | 
			
		||||
 | 
			
		||||
@ -14,8 +14,8 @@ import torch
 | 
			
		||||
from common_testing import TestCaseMixin, load_rgb_image
 | 
			
		||||
from PIL import Image
 | 
			
		||||
from pytorch3d.renderer.cameras import (
 | 
			
		||||
    OpenGLOrthographicCameras,
 | 
			
		||||
    OpenGLPerspectiveCameras,
 | 
			
		||||
    FoVOrthographicCameras,
 | 
			
		||||
    FoVPerspectiveCameras,
 | 
			
		||||
    look_at_view_transform,
 | 
			
		||||
)
 | 
			
		||||
from pytorch3d.renderer.points import (
 | 
			
		||||
@ -47,7 +47,7 @@ class TestRenderPoints(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
            points=verts_padded, features=torch.ones_like(verts_padded)
 | 
			
		||||
        )
 | 
			
		||||
        R, T = look_at_view_transform(2.7, 0.0, 0.0)
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        cameras = FoVPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        raster_settings = PointsRasterizationSettings(
 | 
			
		||||
            image_size=256, radius=5e-2, points_per_pixel=1
 | 
			
		||||
        )
 | 
			
		||||
@ -97,7 +97,7 @@ class TestRenderPoints(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        point_cloud = Pointclouds(points=[verts], features=[rgb_feats])
 | 
			
		||||
 | 
			
		||||
        R, T = look_at_view_transform(20, 10, 0)
 | 
			
		||||
        cameras = OpenGLOrthographicCameras(device=device, R=R, T=T, znear=0.01)
 | 
			
		||||
        cameras = FoVOrthographicCameras(device=device, R=R, T=T, znear=0.01)
 | 
			
		||||
 | 
			
		||||
        raster_settings = PointsRasterizationSettings(
 | 
			
		||||
            # Set image_size so it is not a multiple of 16 (min bin_size)
 | 
			
		||||
@ -150,7 +150,7 @@ class TestRenderPoints(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
        batch_size = 20
 | 
			
		||||
        pointclouds = pointclouds.extend(batch_size)
 | 
			
		||||
        R, T = look_at_view_transform(2.7, 0.0, 0.0)
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        cameras = FoVPerspectiveCameras(device=device, R=R, T=T)
 | 
			
		||||
        raster_settings = PointsRasterizationSettings(
 | 
			
		||||
            image_size=256, radius=5e-2, points_per_pixel=1
 | 
			
		||||
        )
 | 
			
		||||
 | 
			
		||||
@ -12,7 +12,7 @@ from common_testing import TestCaseMixin, load_rgb_image
 | 
			
		||||
from PIL import Image
 | 
			
		||||
from pytorch3d.datasets import ShapeNetCore, collate_batched_meshes
 | 
			
		||||
from pytorch3d.renderer import (
 | 
			
		||||
    OpenGLPerspectiveCameras,
 | 
			
		||||
    FoVPerspectiveCameras,
 | 
			
		||||
    PointLights,
 | 
			
		||||
    RasterizationSettings,
 | 
			
		||||
    look_at_view_transform,
 | 
			
		||||
@ -174,7 +174,7 @@ class TestShapenetCore(TestCaseMixin, unittest.TestCase):
 | 
			
		||||
 | 
			
		||||
        # Rendering settings.
 | 
			
		||||
        R, T = look_at_view_transform(1.0, 1.0, 90)
 | 
			
		||||
        cameras = OpenGLPerspectiveCameras(R=R, T=T, device=device)
 | 
			
		||||
        cameras = FoVPerspectiveCameras(R=R, T=T, device=device)
 | 
			
		||||
        raster_settings = RasterizationSettings(image_size=512)
 | 
			
		||||
        lights = PointLights(
 | 
			
		||||
            location=torch.tensor([0.0, 1.0, -2.0], device=device)[None],
 | 
			
		||||
 | 
			
		||||