Summary:
Allow PLY files to not contain faces. Allow loading pointclouds with color, at least encoded according to the way of some cloudcompare examples.
TODO: Allow vertex normals to be read, and allow vertex colors to be written. Make the return type of load_ply something more user friendly, like a dict.
Noticed in https://github.com/facebookresearch/pytorch3d/issues/209
Reviewed By: nikhilaravi
Differential Revision: D22573314
fbshipit-source-id: 72ba1f7c6417f5dfc83f2ebf359eff017057635c
Summary:
In the original implementation, I had considered PLY properties where there are mixed types of elements in a property to be rare and basically unimportant, so the implementation is very naive.
If we want to support pointcloud PLY files, we need to handle at least the subcase where there are no lists efficiently because this seems to be very common there.
Reviewed By: nikhilaravi, gkioxari
Differential Revision: D22573315
fbshipit-source-id: db6f29446d4e555a2e2b37d38c8e4450d061465b
Summary: We already have code for obj and ply formats. Here we actually make it available in `IO.load_mesh` and `IO.save_mesh`.
Reviewed By: theschnitz, nikhilaravi
Differential Revision: D25400650
fbshipit-source-id: f26d6d7fc46c48634a948eea4d255afad13b807b
Summary: Implemented a data structure for volumes.
Reviewed By: gkioxari
Differential Revision: D20342920
fbshipit-source-id: ccc23eaa183ed8a4e9cd7674b4dcf31e8a65c3c6
Summary: Extends `list_to_padded`/`padded_to_list` to work for tensors with an arbitrary number of input dimensions.
Reviewed By: nikhilaravi, gkioxari
Differential Revision: D23813969
fbshipit-source-id: 52c212a2ecdb3c4dfb6ac47217715e07998f37f1
Summary:
Make no internal functions inside pytorch3d/io interpret str paths except using a PathManager from iopath which they have been given. This means we no longer use any global PathManager object and we no longer use fvcore's deprecated file_io.
To preserve the APIs, various top level functions create their own default-initialized PathManager object if they are not provided one.
Reviewed By: theschnitz
Differential Revision: D25372969
fbshipit-source-id: c176ee31439645fa54a157d6f1aef18b09501569
Summary: Allowing usort, isort and black to coexist without fighting means we can't have imports commented as deprecated from the same module as other imports.
Reviewed By: nikhilaravi
Differential Revision: D25372970
fbshipit-source-id: 637f5a0025c0df9fbec47cba73ce5387f4f8b467
Summary:
Defines a function to run marching cubes algorithm on a single or batch of 3D scalar fields. Returns a mesh's faces and vertices.
UPDATES (12/18)
- Input data is now specified as a (B, D, H, W) tensor as opposed to a (B, W, H, D) tensor. This will now be compatible with the Volumes datastructure.
- Add an option to return output vertices in local coordinates instead of world coordinates.
Also added a small fix to remove the dype for device in Transforms3D - if passing in a torch.device instead of str it causes a pyre error.
Reviewed By: jcjohnson
Differential Revision: D24599019
fbshipit-source-id: 90554a200319fed8736a12371cc349e7108aacd0
Summary: Fixes the index out of bound errors for texture sampling from a texture atlas: when barycentric coordinates are 1.0, the integer index into the (R, R) per face texture map is R (max can only be R-1).
Reviewed By: gkioxari
Differential Revision: D25543803
fbshipit-source-id: 82d0935b981352b49c1d95d5a17f9cc88bad0a82
Summary:
Similar to non square image rasterization for meshes, apply the same updates to the pointcloud rasterizer.
Main API Change:
- PointRasterizationSettings now accepts a tuple/list of (H, W) for the image size.
Reviewed By: jcjohnson
Differential Revision: D25465206
fbshipit-source-id: 7370d83c431af1b972158cecae19d82364623380
Summary: Add `return self` to the `to` function for the renderer classes.
Reviewed By: bottler
Differential Revision: D25534487
fbshipit-source-id: e8dbd35524f0bd40e835439e93184b5a1f1532ca
Summary:
There are a couple of options for supporting non square images:
1) NDC stays at [-1, 1] in both directions with the distance calculations all modified by (W/H). There are a lot of distance based calculations (e.g. triangle areas for barycentric coordinates etc) so this requires changes in many places.
2) NDC is scaled by (W/H) so the smallest side has [-1, 1]. In this case none of the distance calculations need to be updated and only the pixel to NDC calculation needs to be modified.
I decided to go with option 2 after trying option 1!
API Changes:
- Image size can now be specified optionally as a tuple
TODO:
- add a benchmark test for the non square case.
Reviewed By: jcjohnson
Differential Revision: D24404975
fbshipit-source-id: 545efb67c822d748ec35999b35762bce58db2cf4
Summary: Taubin Smoothing for filtering meshes and making them smoother. Taubin smoothing is an iterative approach.
Reviewed By: nikhilaravi
Differential Revision: D24751149
fbshipit-source-id: fb779e955f1a1f6750e704f1b4c6dfa37aebac1a
Summary: This fixes issues with `pulsar.test.TestDepth` that we are encountering on CircleCI. The ID equality test is removed, which seems to give different results on different hardware (which is okay, because the exact order of spheres can slightly vary if they're close due to numerical instabilities). The depth map validity test stays in place.
Reviewed By: bottler
Differential Revision: D24840776
fbshipit-source-id: 2f38ea4880abf202c84d2987fdd71a84c5ef3b05
Summary:
Changes to CI and some minor fixes now that pulsar is part of pytorch3d. Most significantly, add CUB to CI builds.
Make CUB_HOME override the CUB already in cudatoolkit (important for cuda11.0 which uses cub 1.9.9 which pulsar doesn't work well with.
Make imageio available for testing.
Lint fixes.
Fix some test verbosity.
Avoid use of atomicAdd_block on older GPUs.
Reviewed By: nikhilaravi, classner
Differential Revision: D24773716
fbshipit-source-id: 2428356bb2e62735f2bc0c15cbe4cff35b1b24b8
Summary: Users want to be able to obtain the depth from the renderer. Current work-around requires running the rasterizer and extra time. This change creates a new renderer class that also returns the fragments from the rasterizer.
Reviewed By: nikhilaravi
Differential Revision: D24432381
fbshipit-source-id: 6552e8a6bfee646791afb34bdb7452fbc4094aed
Summary: This commit performs pulsar example and test refinements. The examples are fully adjusted to adhere to PEP style guide and additional comments are added.
Reviewed By: nikhilaravi
Differential Revision: D24723391
fbshipit-source-id: 6d289006f080140159731e7f3a8c98b582164f1a
Summary:
This diff builds on top of the `pulsar integration` diff to provide a unified interface for the existing PyTorch3D point renderer and Pulsar. For more information about the pulsar backend, see the release notes and the paper (https://arxiv.org/abs/2004.07484). For information on how to use the backend, see the point cloud rendering notebook and the examples in the folder docs/examples.
The unified interfaces are completely consistent. Switching the render backend is as easy as using `renderer = PulsarPointsRenderer(rasterizer=rasterizer).to(device)` instead of `renderer = PointsRenderer(rasterizer=rasterizer, compositor=compositor)` and adding the `gamma` parameter to the forward function. All PyTorch3D camera types are supported as far as possible; keyword arguments are properly forwarded to the camera. The `PerspectiveCamera` and `OrthographicCamera` require znear and zfar as additional parameters for the forward pass.
Reviewed By: nikhilaravi
Differential Revision: D21421443
fbshipit-source-id: 4aa0a83a419592d9a0bb5d62486a1cdea9d73ce6
Summary:
This diff integrates the pulsar renderer source code into PyTorch3D as an alternative backend for the PyTorch3D point renderer. This diff is the first of a series of three diffs to complete that migration and focuses on the packaging and integration of the source code.
For more information about the pulsar backend, see the release notes and the paper (https://arxiv.org/abs/2004.07484). For information on how to use the backend, see the point cloud rendering notebook and the examples in the folder `docs/examples`.
Tasks addressed in the following diffs:
* Add the PyTorch3D interface,
* Add notebook examples and documentation (or adapt the existing ones to feature both interfaces).
Reviewed By: nikhilaravi
Differential Revision: D23947736
fbshipit-source-id: a5e77b53e6750334db22aefa89b4c079cda1b443
Summary: To initialize the Cameras class currently we require the principal point, focal length and other parameters to be specified from which we calculate the intrinsic matrix. In some cases the matrix might be directly available e.g. from a dataset and the associated metadata for an image.
Reviewed By: nikhilaravi
Differential Revision: D24489509
fbshipit-source-id: 1b411f19c5f6c8074bcfbf613f3339d5e242c119
Summary: This recently added test is sensitive to the version of PIL because of different algorithms to draw ellipses/circles. Remove it as there is no obvious safe way to test this. Replace with a test for the underlying centres_for_image().
Reviewed By: theschnitz
Differential Revision: D24622465
fbshipit-source-id: e46d7384df491c71ac87ba8bbbce89507ac40080
Summary: New methods to directly plot a TexturesUV map with its used points, using PIL and matplotlib.
Reviewed By: gkioxari
Differential Revision: D23782968
fbshipit-source-id: 692970857b5be13a35a3175dc82ac03963a73555
Summary: We can represent a rotation as a vector in the axis direction, whose length is the rotation anticlockwise in radians around that axis.
Reviewed By: gkioxari
Differential Revision: D24306293
fbshipit-source-id: 2e0f138eda8329f6cceff600a6e5f17a00e4deb7
Summary:
Small fix and updated tests for multigpu rendering case.
This resolves the issue seen in: https://github.com/facebookresearch/pytorch3d/issues/401
Reviewed By: gkioxari
Differential Revision: D24314681
fbshipit-source-id: 84c5a5359844c77518b48044001daa9a86f3c43a
Summary: Issue #119. The function `sqrt(max(x, 0))` is not convex and has infinite gradient at 0, but 0 is a subgradient at 0. Here we implement it in such a way as to give 0 as the gradient.
Reviewed By: gkioxari
Differential Revision: D24306294
fbshipit-source-id: 48d136faca083babad4d64970be7ea522dbe9e09
Summary:
Fix for GitHub issue #381.
The example mesh provided in the issue only had material properties but no texture image. The current implementation of texture atlassing generated an atlas using both the material properties and the texture image but only worked if there was a texture image and associated vertex uv coordinates. I have now modified the texture atlas creation so that it doesn't require an image and can work with materials which only have material properties.
Reviewed By: gkioxari
Differential Revision: D24153068
fbshipit-source-id: 63e9d325db09a84b336b83369d5342ce588a9932
Summary: Enhance every texture type with `faces_verts_textures_packed` that allows users to query the texture of each vertex in mesh
Reviewed By: nikhilaravi
Differential Revision: D24058778
fbshipit-source-id: 19d0e3a244fa96aae462c47bf52e07dfd3b7c6f0
Summary:
Enhanced `sample_points_from_meshes` with texture sampling
* This new feature is used to return textures corresponding to the sampled points in `sample_points_from_meshes`
Reviewed By: nikhilaravi
Differential Revision: D24031525
fbshipit-source-id: 8e5d8f784cc38aa391aa8e84e54423bd9fad7ad1
Summary:
Support for moving all the tensors of the renderer to another device by calling `renderer.to(new_device)`
Currently the `MeshRenderer`, `MeshRasterizer` and `SoftPhongShader` (and other shaders) are all of type `nn.Module` which already supports easily moving tensors of submodules (defined as class attributes) to a different device. However the class attributes of the rasterizer and shader (e.g. cameras, lights, materials), are of type `TensorProperties`, not nn.Module so we need to explicity create a `to` method to move these tensors to device. Note that the `TensorProperties` class already has a `to` method so we only need to call `cameras.to(device)` and don't need to worry about the internal tensors.
The other option is of course making these other classes (cameras, lights etc) also of type nn.Module.
Reviewed By: gkioxari
Differential Revision: D23885107
fbshipit-source-id: d71565c442181f739de4d797076ed5d00fb67f8e
Summary:
I'm constantly encountering 3D models with resources that have spaces in their filenames (especially on Windows) and therefore they can't be loaded in pytorch3d. Let me know what you think.
Thanks
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/358
Reviewed By: bottler
Differential Revision: D23798492
Pulled By: nikhilaravi
fbshipit-source-id: 4d85b7ee05339486d2e5ef53a531f8e6052251c5
Summary:
Make save_ply save to binary instead of ascii. An option makes the previous functionality available. save_ply's API accepts a stream, but this is undocumented; that stream must now be a binary stream not a text stream.
Avoiding warnings about making tensors from immutable numpy arrays.
Possible performance improvement when reading binary files.
Fix reading zero-length binary lists.
Reviewed By: nikhilaravi
Differential Revision: D22333118
fbshipit-source-id: b423dfd3da46e047bead200255f47a7707306811
Summary: Support rendering different color backgrounds for pointclouds for both compositors
Reviewed By: nikhilaravi
Differential Revision: D23611043
fbshipit-source-id: ab029650d51349340372c5bd66700e6577d48851
Summary: When the camera is vertically oriented, calculating the look_at x-axis (also known as the "right" vector) does not succeed, resulting in the x-axis being placed at the origin. Adds a check to correctly calculate the x-axis if this case occurs.
Reviewed By: nikhilaravi, sbranson
Differential Revision: D23511859
fbshipit-source-id: ee5145cdbecdbe2f7c7d288588bd0899480cb327
Summary:
This fixes two small issues with blending.py:softmax_rgb_blend():
1) zfar and znear attributes are propagated from the camera settings instead of just using default settings of znear=1.0 and zfar=100.0
2) A check is added to prevent arithmetic overflow in softmax_rgb_blend()
This is a fix in response to https://github.com/facebookresearch/pytorch3d/issues/334
where meshes rendererd using a SoftPhongShader with faces_per_pixel=1 appear black. This only occurs when the scale of the mesh is large (vertex values > 100, where 100 is the default value of zfar). This fix allows the caller to increase the value of cameras.zfar to match the scale of her/his mesh.
Reviewed By: nikhilaravi
Differential Revision: D23517541
fbshipit-source-id: ab8631ce9e5f2149f140b67b13eff857771b8807
Summary:
Add a join_scene method to all the textures to allow the join_mesh function to include textures. Rename the join_mesh function to join_meshes_as_scene.
For TexturesAtlas, we now interpolate if the user attempts to have the resolution vary across the batch. This doesn't look great if the resolution is already very low.
For TexturesUV, a rectangle packing function is required, this does something simple.
Reviewed By: gkioxari
Differential Revision: D23188773
fbshipit-source-id: c013db061a04076e13e90ccc168a7913e933a9c5
Summary:
Allow, and make default, align_corners=True for texture maps. Allow changing the padding_mode and set the default to be "border" which produces more logical results. Some new documentation.
The previous behavior corresponds to padding_mode="zeros" and align_corners=False.
Reviewed By: gkioxari
Differential Revision: D23268775
fbshipit-source-id: 58d6229baa591baa69705bcf97471c80ba3651de
