423A35C7/pytorch3d
1
0
Fork 0
mirror of https://github.com/facebookresearch/pytorch3d.git synced 2025-12-19 05:40:34 +08:00
Code Issues Packages Projects Releases Wiki Activity

Fix camera conversion between opencv and pytorch3d

Browse Source 
Summary:
For non square image, the NDC space in pytorch3d is not square [-1, 1]. Instead, it is [-1, 1] for the smallest side, and [-u, u] for the largest side, where u > 1. This behavior is followed by the pytorch3d renderer.

See the function `get_ndc_to_screen_transform` for a example.

Without this fix, the rendering result is not correct using the converted pytorch3d-camera from a opencv-camera on non square images.

This fix also helps the `transform_points_screen` function delivers consistent results with opencv projection for the converted pytorch3d-camera.

Reviewed By: classner

Differential Revision: D31366775

fbshipit-source-id: 8858ae7b5cf5c0a4af5a2af40a1358b2fe4cf74b
This commit is contained in:
Ruilong Li
2021-10-07 10:14:23 -07:00
committed by Facebook GitHub Bot
parent 815a93ce89
commit 8fa438cbda
2 changed files with 24 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
tests/test_camera_conversions.py
View File

@@ -24,14 +24,6 @@ from pytorch3d.utils import (
DATA_DIR = get_tests_dir() / "data"
def _coords_opencv_screen_to_pytorch3d_ndc(xy_opencv, image_size):
"""
Converts the OpenCV screen coordinates `xy_opencv` to PyTorch3D NDC coordinates.
"""
xy_pytorch3d = -(2.0 * xy_opencv / image_size.flip(dims=(1,))[:, None] - 1.0)
return xy_pytorch3d
def cv2_project_points(pts, rvec, tvec, camera_matrix):
"""
Reproduces the `cv2.projectPoints` function from OpenCV using PyTorch.
@@ -145,18 +137,13 @@ class TestCameraConversions(TestCaseMixin, unittest.TestCase):
R, tvec, camera_matrix, image_size
)
# project the 3D points with converted cameras
pts_proj_pytorch3d = cameras_opencv_to_pytorch3d.transform_points(pts)[..., :2]
# convert the opencv-projected points to pytorch3d screen coords
pts_proj_opencv_in_pytorch3d_screen = _coords_opencv_screen_to_pytorch3d_ndc(
pts_proj_opencv, image_size
)
# project the 3D points with converted cameras to screen space.
pts_proj_pytorch3d_screen = cameras_opencv_to_pytorch3d.transform_points_screen(
pts
)[..., :2]
# compare to the cached projected points
self.assertClose(
pts_proj_opencv_in_pytorch3d_screen, pts_proj_pytorch3d, atol=1e-5
)
self.assertClose(pts_proj_opencv, pts_proj_pytorch3d_screen, atol=1e-5)
# Check the inverse.
R_i, tvec_i, camera_matrix_i = opencv_from_cameras_projection(