mirror of
https://github.com/facebookresearch/pytorch3d.git
synced 2025-08-02 03:42:50 +08:00
updates for v0.7.0
This commit is contained in:
Jeremy Francis Reizenstein
parent
af1c31a02b
commit
27dc9dc21d
@ -75,7 +75,7 @@
|
||||
<p>In order to experiment with different approaches, we wanted a modular implementation that is easy to use and extend, and supports <a href="/docs/batching">heterogeneous batching</a>. Taking inspiration from existing work [<a href="#1">1</a>, <a href="#2">2</a>], we have created a new, modular, differentiable renderer with <strong>parallel implementations in PyTorch, C++ and CUDA</strong>, as well as comprehensive documentation and tests, with the aim of helping to further research in this field.</p>
|
||||
<p>Our implementation decouples the rasterization and shading steps of rendering. The core rasterization step (based on <a href="#2">[2]</a>) returns several intermediate variables and has an optimized implementation in CUDA. The rest of the pipeline is implemented purely in PyTorch, and is designed to be customized and extended. With this approach, the PyTorch3D differentiable renderer can be imported as a library.</p>
|
||||
<h2><a class="anchor" aria-hidden="true" id="uget-startedu"></a><a href="#uget-startedu" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><u>Get started</u></h2>
|
||||
<p>To learn about more the implementation and start using the renderer refer to <a href="/docs/renderer_getting_started">getting started with renderer</a>, which also contains the <a href="/docs/assets/architecture_overview.png">architecture overview</a> and <a href="/docs/assets/transformations_overview.png">coordinate transformation conventions</a>.</p>
|
||||
<p>To learn about more the implementation and start using the renderer refer to <a href="/docs/renderer_getting_started">getting started with renderer</a>, which also contains the <a href="/docs/assets/architecture_renderer.jpg">architecture overview</a> and <a href="/docs/assets/transforms_overview.jpg">coordinate transformation conventions</a>.</p>
|
||||
<h2><a class="anchor" aria-hidden="true" id="utech-reportu"></a><a href="#utech-reportu" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><u>Tech Report</u></h2>
|
||||
<p>For an in depth explanation of the renderer design, key features and benchmarks please refer to the PyTorch3D Technical Report on ArXiv: <a href="https://arxiv.org/abs/2007.08501">Accelerating 3D Deep Learning with PyTorch3D</a>, for the pulsar backend see here: <a href="https://arxiv.org/abs/2004.07484">Fast Differentiable Raycasting for Neural Rendering using Sphere-based Representations</a>.</p>
|
||||
<hr>
|
||||
@ -109,4 +109,4 @@ total_memory = memory_forward_pass + memory_backward_pass
|
||||
<p><a id="6">[6]</a> Yifan et al, 'Differentiable Surface Splatting for Point-based Geometry Processing', SIGGRAPH Asia 2019</p>
|
||||
<p><a id="7">[7]</a> Loubet et al, 'Reparameterizing Discontinuous Integrands for Differentiable Rendering', SIGGRAPH Asia 2019</p>
|
||||
<p><a id="8">[8]</a> Chen et al, 'Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer', NeurIPS 2019</p>
|
||||
</span></div></article></div><div class="docLastUpdate"><em>Last updated by Christoph Lassner</em></div><div class="docs-prevnext"><a class="docs-prev button" href="/docs/visualization"><span class="arrow-prev">← </span><span>Plotly Visualization</span></a><a class="docs-next button" href="/docs/renderer_getting_started"><span>Getting Started</span><span class="arrow-next"> →</span></a></div></div></div><nav class="onPageNav"><ul class="toc-headings"><li><a href="#uget-startedu"><u>Get started</u></a></li><li><a href="#utech-reportu"><u>Tech Report</u></a><ul class="toc-headings"><li><a href="#references">References</a></li></ul></li></ul></nav></div><footer class="nav-footer" id="footer"><section class="sitemap"><div class="footerSection"><div class="social"><a class="github-button" href="https://github.com/facebookresearch/pytorch3d" data-count-href="https://github.com/facebookresearch/pytorch3d/stargazers" data-show-count="true" data-count-aria-label="# stargazers on GitHub" aria-label="Star PyTorch3D on GitHub">pytorch3d</a></div></div></section><a href="https://opensource.facebook.com/" target="_blank" rel="noreferrer noopener" class="fbOpenSource"><img src="/img/oss_logo.png" alt="Facebook Open Source" width="170" height="45"/></a><section class="copyright">Copyright © 2022 Meta Platforms, Inc<br/>Legal:<a href="https://opensource.facebook.com/legal/privacy/" target="_blank" rel="noreferrer noopener">Privacy</a><a href="https://opensource.facebook.com/legal/terms/" target="_blank" rel="noreferrer noopener">Terms</a></section></footer></div></body></html>
|
||||
</span></div></article></div><div class="docLastUpdate"><em>Last updated by Krzysztof Chalupka</em></div><div class="docs-prevnext"><a class="docs-prev button" href="/docs/visualization"><span class="arrow-prev">← </span><span>Plotly Visualization</span></a><a class="docs-next button" href="/docs/renderer_getting_started"><span>Getting Started</span><span class="arrow-next"> →</span></a></div></div></div><nav class="onPageNav"><ul class="toc-headings"><li><a href="#uget-startedu"><u>Get started</u></a></li><li><a href="#utech-reportu"><u>Tech Report</u></a><ul class="toc-headings"><li><a href="#references">References</a></li></ul></li></ul></nav></div><footer class="nav-footer" id="footer"><section class="sitemap"><div class="footerSection"><div class="social"><a class="github-button" href="https://github.com/facebookresearch/pytorch3d" data-count-href="https://github.com/facebookresearch/pytorch3d/stargazers" data-show-count="true" data-count-aria-label="# stargazers on GitHub" aria-label="Star PyTorch3D on GitHub">pytorch3d</a></div></div></section><a href="https://opensource.facebook.com/" target="_blank" rel="noreferrer noopener" class="fbOpenSource"><img src="/img/oss_logo.png" alt="Facebook Open Source" width="170" height="45"/></a><section class="copyright">Copyright © 2022 Meta Platforms, Inc<br/>Legal:<a href="https://opensource.facebook.com/legal/privacy/" target="_blank" rel="noreferrer noopener">Privacy</a><a href="https://opensource.facebook.com/legal/terms/" target="_blank" rel="noreferrer noopener">Terms</a></section></footer></div></body></html>
|
||||
@ -75,7 +75,7 @@
|
||||
<p>In order to experiment with different approaches, we wanted a modular implementation that is easy to use and extend, and supports <a href="/docs/batching">heterogeneous batching</a>. Taking inspiration from existing work [<a href="#1">1</a>, <a href="#2">2</a>], we have created a new, modular, differentiable renderer with <strong>parallel implementations in PyTorch, C++ and CUDA</strong>, as well as comprehensive documentation and tests, with the aim of helping to further research in this field.</p>
|
||||
<p>Our implementation decouples the rasterization and shading steps of rendering. The core rasterization step (based on <a href="#2">[2]</a>) returns several intermediate variables and has an optimized implementation in CUDA. The rest of the pipeline is implemented purely in PyTorch, and is designed to be customized and extended. With this approach, the PyTorch3D differentiable renderer can be imported as a library.</p>
|
||||
<h2><a class="anchor" aria-hidden="true" id="uget-startedu"></a><a href="#uget-startedu" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><u>Get started</u></h2>
|
||||
<p>To learn about more the implementation and start using the renderer refer to <a href="/docs/renderer_getting_started">getting started with renderer</a>, which also contains the <a href="/docs/assets/architecture_overview.png">architecture overview</a> and <a href="/docs/assets/transformations_overview.png">coordinate transformation conventions</a>.</p>
|
||||
<p>To learn about more the implementation and start using the renderer refer to <a href="/docs/renderer_getting_started">getting started with renderer</a>, which also contains the <a href="/docs/assets/architecture_renderer.jpg">architecture overview</a> and <a href="/docs/assets/transforms_overview.jpg">coordinate transformation conventions</a>.</p>
|
||||
<h2><a class="anchor" aria-hidden="true" id="utech-reportu"></a><a href="#utech-reportu" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><u>Tech Report</u></h2>
|
||||
<p>For an in depth explanation of the renderer design, key features and benchmarks please refer to the PyTorch3D Technical Report on ArXiv: <a href="https://arxiv.org/abs/2007.08501">Accelerating 3D Deep Learning with PyTorch3D</a>, for the pulsar backend see here: <a href="https://arxiv.org/abs/2004.07484">Fast Differentiable Raycasting for Neural Rendering using Sphere-based Representations</a>.</p>
|
||||
<hr>
|
||||
@ -109,4 +109,4 @@ total_memory = memory_forward_pass + memory_backward_pass
|
||||
<p><a id="6">[6]</a> Yifan et al, 'Differentiable Surface Splatting for Point-based Geometry Processing', SIGGRAPH Asia 2019</p>
|
||||
<p><a id="7">[7]</a> Loubet et al, 'Reparameterizing Discontinuous Integrands for Differentiable Rendering', SIGGRAPH Asia 2019</p>
|
||||
<p><a id="8">[8]</a> Chen et al, 'Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer', NeurIPS 2019</p>
|
||||
</span></div></article></div><div class="docLastUpdate"><em>Last updated by Christoph Lassner</em></div><div class="docs-prevnext"><a class="docs-prev button" href="/docs/visualization"><span class="arrow-prev">← </span><span>Plotly Visualization</span></a><a class="docs-next button" href="/docs/renderer_getting_started"><span>Getting Started</span><span class="arrow-next"> →</span></a></div></div></div><nav class="onPageNav"><ul class="toc-headings"><li><a href="#uget-startedu"><u>Get started</u></a></li><li><a href="#utech-reportu"><u>Tech Report</u></a><ul class="toc-headings"><li><a href="#references">References</a></li></ul></li></ul></nav></div><footer class="nav-footer" id="footer"><section class="sitemap"><div class="footerSection"><div class="social"><a class="github-button" href="https://github.com/facebookresearch/pytorch3d" data-count-href="https://github.com/facebookresearch/pytorch3d/stargazers" data-show-count="true" data-count-aria-label="# stargazers on GitHub" aria-label="Star PyTorch3D on GitHub">pytorch3d</a></div></div></section><a href="https://opensource.facebook.com/" target="_blank" rel="noreferrer noopener" class="fbOpenSource"><img src="/img/oss_logo.png" alt="Facebook Open Source" width="170" height="45"/></a><section class="copyright">Copyright © 2022 Meta Platforms, Inc<br/>Legal:<a href="https://opensource.facebook.com/legal/privacy/" target="_blank" rel="noreferrer noopener">Privacy</a><a href="https://opensource.facebook.com/legal/terms/" target="_blank" rel="noreferrer noopener">Terms</a></section></footer></div></body></html>
|
||||
</span></div></article></div><div class="docLastUpdate"><em>Last updated by Krzysztof Chalupka</em></div><div class="docs-prevnext"><a class="docs-prev button" href="/docs/visualization"><span class="arrow-prev">← </span><span>Plotly Visualization</span></a><a class="docs-next button" href="/docs/renderer_getting_started"><span>Getting Started</span><span class="arrow-next"> →</span></a></div></div></div><nav class="onPageNav"><ul class="toc-headings"><li><a href="#uget-startedu"><u>Get started</u></a></li><li><a href="#utech-reportu"><u>Tech Report</u></a><ul class="toc-headings"><li><a href="#references">References</a></li></ul></li></ul></nav></div><footer class="nav-footer" id="footer"><section class="sitemap"><div class="footerSection"><div class="social"><a class="github-button" href="https://github.com/facebookresearch/pytorch3d" data-count-href="https://github.com/facebookresearch/pytorch3d/stargazers" data-show-count="true" data-count-aria-label="# stargazers on GitHub" aria-label="Star PyTorch3D on GitHub">pytorch3d</a></div></div></section><a href="https://opensource.facebook.com/" target="_blank" rel="noreferrer noopener" class="fbOpenSource"><img src="/img/oss_logo.png" alt="Facebook Open Source" width="170" height="45"/></a><section class="copyright">Copyright © 2022 Meta Platforms, Inc<br/>Legal:<a href="https://opensource.facebook.com/legal/privacy/" target="_blank" rel="noreferrer noopener">Privacy</a><a href="https://opensource.facebook.com/legal/terms/" target="_blank" rel="noreferrer noopener">Terms</a></section></footer></div></body></html>
|
||||
@ -89,7 +89,7 @@
|
||||
"except ModuleNotFoundError:\n",
|
||||
" need_pytorch3d=True\n",
|
||||
"if need_pytorch3d:\n",
|
||||
" if torch.__version__.startswith(\"1.10.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" if torch.__version__.startswith(\"1.12.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" # We try to install PyTorch3D via a released wheel.\n",
|
||||
" pyt_version_str=torch.__version__.split(\"+\")[0].replace(\".\", \"\")\n",
|
||||
" version_str=\"\".join([\n",
|
||||
|
||||
@ -49,7 +49,7 @@ try:
|
||||
except ModuleNotFoundError:
|
||||
need_pytorch3d=True
|
||||
if need_pytorch3d:
|
||||
if torch.__version__.startswith("1.10.") and sys.platform.startswith("linux"):
|
||||
if torch.__version__.startswith("1.12.") and sys.platform.startswith("linux"):
|
||||
# We try to install PyTorch3D via a released wheel.
|
||||
pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
|
||||
version_str="".join([
|
||||
|
||||
@ -76,7 +76,7 @@
|
||||
"except ModuleNotFoundError:\n",
|
||||
" need_pytorch3d=True\n",
|
||||
"if need_pytorch3d:\n",
|
||||
" if torch.__version__.startswith(\"1.10.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" if torch.__version__.startswith(\"1.12.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" # We try to install PyTorch3D via a released wheel.\n",
|
||||
" pyt_version_str=torch.__version__.split(\"+\")[0].replace(\".\", \"\")\n",
|
||||
" version_str=\"\".join([\n",
|
||||
|
||||
@ -36,7 +36,7 @@ try:
|
||||
except ModuleNotFoundError:
|
||||
need_pytorch3d=True
|
||||
if need_pytorch3d:
|
||||
if torch.__version__.startswith("1.10.") and sys.platform.startswith("linux"):
|
||||
if torch.__version__.startswith("1.12.") and sys.platform.startswith("linux"):
|
||||
# We try to install PyTorch3D via a released wheel.
|
||||
pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
|
||||
version_str="".join([
|
||||
|
||||
@ -51,7 +51,7 @@
|
||||
"except ModuleNotFoundError:\n",
|
||||
" need_pytorch3d=True\n",
|
||||
"if need_pytorch3d:\n",
|
||||
" if torch.__version__.startswith(\"1.10.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" if torch.__version__.startswith(\"1.12.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" # We try to install PyTorch3D via a released wheel.\n",
|
||||
" pyt_version_str=torch.__version__.split(\"+\")[0].replace(\".\", \"\")\n",
|
||||
" version_str=\"\".join([\n",
|
||||
|
||||
@ -31,7 +31,7 @@ try:
|
||||
except ModuleNotFoundError:
|
||||
need_pytorch3d=True
|
||||
if need_pytorch3d:
|
||||
if torch.__version__.startswith("1.10.") and sys.platform.startswith("linux"):
|
||||
if torch.__version__.startswith("1.12.") and sys.platform.startswith("linux"):
|
||||
# We try to install PyTorch3D via a released wheel.
|
||||
pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
|
||||
version_str="".join([
|
||||
|
||||
@ -90,7 +90,7 @@
|
||||
"except ModuleNotFoundError:\n",
|
||||
" need_pytorch3d=True\n",
|
||||
"if need_pytorch3d:\n",
|
||||
" if torch.__version__.startswith(\"1.10.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" if torch.__version__.startswith(\"1.12.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" # We try to install PyTorch3D via a released wheel.\n",
|
||||
" pyt_version_str=torch.__version__.split(\"+\")[0].replace(\".\", \"\")\n",
|
||||
" version_str=\"\".join([\n",
|
||||
|
||||
@ -48,7 +48,7 @@ try:
|
||||
except ModuleNotFoundError:
|
||||
need_pytorch3d=True
|
||||
if need_pytorch3d:
|
||||
if torch.__version__.startswith("1.10.") and sys.platform.startswith("linux"):
|
||||
if torch.__version__.startswith("1.12.") and sys.platform.startswith("linux"):
|
||||
# We try to install PyTorch3D via a released wheel.
|
||||
pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
|
||||
version_str="".join([
|
||||
|
||||
@ -56,7 +56,7 @@
|
||||
"except ModuleNotFoundError:\n",
|
||||
" need_pytorch3d=True\n",
|
||||
"if need_pytorch3d:\n",
|
||||
" if torch.__version__.startswith(\"1.10.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" if torch.__version__.startswith(\"1.12.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" # We try to install PyTorch3D via a released wheel.\n",
|
||||
" pyt_version_str=torch.__version__.split(\"+\")[0].replace(\".\", \"\")\n",
|
||||
" version_str=\"\".join([\n",
|
||||
@ -815,7 +815,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 5. Visualizing the optimized neural radiance field\n",
|
||||
"## 6. Visualizing the optimized neural radiance field\n",
|
||||
"\n",
|
||||
"Finally, we visualize the neural radiance field by rendering from multiple viewpoints that rotate around the volume's y-axis."
|
||||
]
|
||||
@ -865,7 +865,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 6. Conclusion\n",
|
||||
"## 7. Conclusion\n",
|
||||
"\n",
|
||||
"In this tutorial, we have shown how to optimize an implicit representation of a scene such that the renders of the scene from known viewpoints match the observed images for each viewpoint. The rendering was carried out using the PyTorch3D's implicit function renderer composed of either a `MonteCarloRaysampler` or `NDCMultinomialRaysampler`, and an `EmissionAbsorptionRaymarcher`."
|
||||
]
|
||||
|
||||
@ -42,7 +42,7 @@ try:
|
||||
except ModuleNotFoundError:
|
||||
need_pytorch3d=True
|
||||
if need_pytorch3d:
|
||||
if torch.__version__.startswith("1.10.") and sys.platform.startswith("linux"):
|
||||
if torch.__version__.startswith("1.12.") and sys.platform.startswith("linux"):
|
||||
# We try to install PyTorch3D via a released wheel.
|
||||
pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
|
||||
version_str="".join([
|
||||
@ -744,7 +744,7 @@ for iteration in range(n_iter):
|
||||
)
|
||||
|
||||
|
||||
# ## 5. Visualizing the optimized neural radiance field
|
||||
# ## 6. Visualizing the optimized neural radiance field
|
||||
#
|
||||
# Finally, we visualize the neural radiance field by rendering from multiple viewpoints that rotate around the volume's y-axis.
|
||||
|
||||
@ -786,6 +786,6 @@ image_grid(rotating_nerf_frames.clamp(0., 1.).cpu().numpy(), rows=3, cols=5, rgb
|
||||
plt.show()
|
||||
|
||||
|
||||
# ## 6. Conclusion
|
||||
# ## 7. Conclusion
|
||||
#
|
||||
# In this tutorial, we have shown how to optimize an implicit representation of a scene such that the renders of the scene from known viewpoints match the observed images for each viewpoint. The rendering was carried out using the PyTorch3D's implicit function renderer composed of either a `MonteCarloRaysampler` or `NDCMultinomialRaysampler`, and an `EmissionAbsorptionRaymarcher`.
|
||||
|
||||
@ -68,7 +68,7 @@
|
||||
"except ModuleNotFoundError:\n",
|
||||
" need_pytorch3d=True\n",
|
||||
"if need_pytorch3d:\n",
|
||||
" if torch.__version__.startswith(\"1.10.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" if torch.__version__.startswith(\"1.12.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" # We try to install PyTorch3D via a released wheel.\n",
|
||||
" pyt_version_str=torch.__version__.split(\"+\")[0].replace(\".\", \"\")\n",
|
||||
" version_str=\"\".join([\n",
|
||||
@ -118,7 +118,7 @@
|
||||
"from pytorch3d.structures import Meshes\n",
|
||||
"from pytorch3d.renderer import (\n",
|
||||
" look_at_view_transform,\n",
|
||||
" OpenGLPerspectiveCameras, \n",
|
||||
" FoVPerspectiveCameras, \n",
|
||||
" PointLights, \n",
|
||||
" DirectionalLights, \n",
|
||||
" Materials, \n",
|
||||
@ -304,11 +304,11 @@
|
||||
"# broadcasting. So we can view the camera from the a distance of dist=2.7, and \n",
|
||||
"# then specify elevation and azimuth angles for each viewpoint as tensors. \n",
|
||||
"R, T = look_at_view_transform(dist=2.7, elev=elev, azim=azim)\n",
|
||||
"cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)\n",
|
||||
"cameras = FoVPerspectiveCameras(device=device, R=R, T=T)\n",
|
||||
"\n",
|
||||
"# We arbitrarily choose one particular view that will be used to visualize \n",
|
||||
"# results\n",
|
||||
"camera = OpenGLPerspectiveCameras(device=device, R=R[None, 1, ...], \n",
|
||||
"camera = FoVPerspectiveCameras(device=device, R=R[None, 1, ...], \n",
|
||||
" T=T[None, 1, ...]) \n",
|
||||
"\n",
|
||||
"# Define the settings for rasterization and shading. Here we set the output \n",
|
||||
@ -351,7 +351,7 @@
|
||||
"# Our multi-view cow dataset will be represented by these 2 lists of tensors,\n",
|
||||
"# each of length num_views.\n",
|
||||
"target_rgb = [target_images[i, ..., :3] for i in range(num_views)]\n",
|
||||
"target_cameras = [OpenGLPerspectiveCameras(device=device, R=R[None, i, ...], \n",
|
||||
"target_cameras = [FoVPerspectiveCameras(device=device, R=R[None, i, ...], \n",
|
||||
" T=T[None, i, ...]) for i in range(num_views)]"
|
||||
]
|
||||
},
|
||||
@ -708,6 +708,7 @@
|
||||
" image_size=128, \n",
|
||||
" blur_radius=np.log(1. / 1e-4 - 1.)*sigma, \n",
|
||||
" faces_per_pixel=50, \n",
|
||||
" perspective_correct=False, \n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Differentiable soft renderer using per vertex RGB colors for texture\n",
|
||||
|
||||
@ -31,7 +31,7 @@ try:
|
||||
except ModuleNotFoundError:
|
||||
need_pytorch3d=True
|
||||
if need_pytorch3d:
|
||||
if torch.__version__.startswith("1.10.") and sys.platform.startswith("linux"):
|
||||
if torch.__version__.startswith("1.12.") and sys.platform.startswith("linux"):
|
||||
# We try to install PyTorch3D via a released wheel.
|
||||
pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
|
||||
version_str="".join([
|
||||
@ -74,7 +74,7 @@ from pytorch3d.loss import (
|
||||
from pytorch3d.structures import Meshes
|
||||
from pytorch3d.renderer import (
|
||||
look_at_view_transform,
|
||||
OpenGLPerspectiveCameras,
|
||||
FoVPerspectiveCameras,
|
||||
PointLights,
|
||||
DirectionalLights,
|
||||
Materials,
|
||||
@ -185,11 +185,11 @@ lights = PointLights(device=device, location=[[0.0, 0.0, -3.0]])
|
||||
# broadcasting. So we can view the camera from the a distance of dist=2.7, and
|
||||
# then specify elevation and azimuth angles for each viewpoint as tensors.
|
||||
R, T = look_at_view_transform(dist=2.7, elev=elev, azim=azim)
|
||||
cameras = OpenGLPerspectiveCameras(device=device, R=R, T=T)
|
||||
cameras = FoVPerspectiveCameras(device=device, R=R, T=T)
|
||||
|
||||
# We arbitrarily choose one particular view that will be used to visualize
|
||||
# results
|
||||
camera = OpenGLPerspectiveCameras(device=device, R=R[None, 1, ...],
|
||||
camera = FoVPerspectiveCameras(device=device, R=R[None, 1, ...],
|
||||
T=T[None, 1, ...])
|
||||
|
||||
# Define the settings for rasterization and shading. Here we set the output
|
||||
@ -232,7 +232,7 @@ target_images = renderer(meshes, cameras=cameras, lights=lights)
|
||||
# Our multi-view cow dataset will be represented by these 2 lists of tensors,
|
||||
# each of length num_views.
|
||||
target_rgb = [target_images[i, ..., :3] for i in range(num_views)]
|
||||
target_cameras = [OpenGLPerspectiveCameras(device=device, R=R[None, i, ...],
|
||||
target_cameras = [FoVPerspectiveCameras(device=device, R=R[None, i, ...],
|
||||
T=T[None, i, ...]) for i in range(num_views)]
|
||||
|
||||
|
||||
@ -461,6 +461,7 @@ raster_settings_soft = RasterizationSettings(
|
||||
image_size=128,
|
||||
blur_radius=np.log(1. / 1e-4 - 1.)*sigma,
|
||||
faces_per_pixel=50,
|
||||
perspective_correct=False,
|
||||
)
|
||||
|
||||
# Differentiable soft renderer using per vertex RGB colors for texture
|
||||
|
||||
@ -47,7 +47,7 @@
|
||||
"except ModuleNotFoundError:\n",
|
||||
" need_pytorch3d=True\n",
|
||||
"if need_pytorch3d:\n",
|
||||
" if torch.__version__.startswith(\"1.10.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" if torch.__version__.startswith(\"1.12.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" # We try to install PyTorch3D via a released wheel.\n",
|
||||
" pyt_version_str=torch.__version__.split(\"+\")[0].replace(\".\", \"\")\n",
|
||||
" version_str=\"\".join([\n",
|
||||
|
||||
@ -32,7 +32,7 @@ try:
|
||||
except ModuleNotFoundError:
|
||||
need_pytorch3d=True
|
||||
if need_pytorch3d:
|
||||
if torch.__version__.startswith("1.10.") and sys.platform.startswith("linux"):
|
||||
if torch.__version__.startswith("1.12.") and sys.platform.startswith("linux"):
|
||||
# We try to install PyTorch3D via a released wheel.
|
||||
pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
|
||||
version_str="".join([
|
||||
|
||||
1252
files/implicitron_config_system.ipynb
Normal file
1252
files/implicitron_config_system.ipynb
Normal file
File diff suppressed because it is too large
Load Diff
519
files/implicitron_config_system.py
Normal file
519
files/implicitron_config_system.py
Normal file
@ -0,0 +1,519 @@
|
||||
|
||||
# coding: utf-8
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# Copyright (c) Meta Platforms, Inc. and affiliates. All rights reserved.
|
||||
|
||||
|
||||
# # Implicitron's config system
|
||||
|
||||
# Implicitron's components are all based on a unified hierarchical configuration system.
|
||||
# This allows configurable variables and all defaults to be defined separately for each new component.
|
||||
# All configs relevant to an experiment are then automatically composed into a single configuration file that fully specifies the experiment.
|
||||
# An especially important feature is extension points where users can insert their own sub-classes of Implicitron's base components.
|
||||
#
|
||||
# The file which defines this system is [here](https://github.com/facebookresearch/pytorch3d/blob/main/pytorch3d/implicitron/tools/config.py) in the PyTorch3D repo.
|
||||
# The Implicitron volumes tutorial contains a simple example of using the config system.
|
||||
# This tutorial provides detailed hands-on experience in using and modifying Implicitron's configurable components.
|
||||
#
|
||||
|
||||
# ## 0. Install and import modules
|
||||
#
|
||||
# Ensure `torch` and `torchvision` are installed. If `pytorch3d` is not installed, install it using the following cell:
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
import os
|
||||
import sys
|
||||
import torch
|
||||
need_pytorch3d=False
|
||||
try:
|
||||
import pytorch3d
|
||||
except ModuleNotFoundError:
|
||||
need_pytorch3d=True
|
||||
if need_pytorch3d:
|
||||
if torch.__version__.startswith("1.12.") and sys.platform.startswith("linux"):
|
||||
# We try to install PyTorch3D via a released wheel.
|
||||
pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
|
||||
version_str="".join([
|
||||
f"py3{sys.version_info.minor}_cu",
|
||||
torch.version.cuda.replace(".",""),
|
||||
f"_pyt{pyt_version_str}"
|
||||
])
|
||||
get_ipython().system('pip install fvcore iopath')
|
||||
get_ipython().system('pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/{version_str}/download.html')
|
||||
else:
|
||||
# We try to install PyTorch3D from source.
|
||||
get_ipython().system('curl -LO https://github.com/NVIDIA/cub/archive/1.10.0.tar.gz')
|
||||
get_ipython().system('tar xzf 1.10.0.tar.gz')
|
||||
os.environ["CUB_HOME"] = os.getcwd() + "/cub-1.10.0"
|
||||
get_ipython().system("pip install 'git+https://github.com/facebookresearch/pytorch3d.git@stable'")
|
||||
|
||||
|
||||
# Ensure omegaconf is installed. If not, run this cell. (It should not be necessary to restart the runtime.)
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
get_ipython().system('pip install omegaconf')
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional, Tuple
|
||||
|
||||
import torch
|
||||
from omegaconf import DictConfig, OmegaConf
|
||||
from pytorch3d.implicitron.tools.config import (
|
||||
Configurable,
|
||||
ReplaceableBase,
|
||||
expand_args_fields,
|
||||
get_default_args,
|
||||
registry,
|
||||
run_auto_creation,
|
||||
)
|
||||
|
||||
|
||||
# ## 1. Introducing dataclasses
|
||||
#
|
||||
# [Type hints](https://docs.python.org/3/library/typing.html) give a taxonomy of types in Python. [Dataclasses](https://docs.python.org/3/library/dataclasses.html) let you create a class based on a list of members which have names, types and possibly default values. The `__init__` function is created automatically, and calls a `__post_init__` function if present as a final step. For example
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
@dataclass
|
||||
class MyDataclass:
|
||||
a: int
|
||||
b: int = 8
|
||||
c: Optional[Tuple[int, ...]] = None
|
||||
|
||||
def __post_init__(self):
|
||||
print(f"created with a = {self.a}")
|
||||
self.d = 2 * self.b
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
my_dataclass_instance = MyDataclass(a=18)
|
||||
assert my_dataclass_instance.d == 16
|
||||
|
||||
|
||||
# 👷 Note that the `dataclass` decorator here is function which modifies the definition of the class itself.
|
||||
# It runs immediately after the definition.
|
||||
# Our config system requires that implicitron library code contains classes whose modified versions need to be aware of user-defined implementations.
|
||||
# Therefore we need the modification of the class to be delayed. We don't use a decorator.
|
||||
#
|
||||
|
||||
# ## 2. Introducing omegaconf and OmegaConf.structured
|
||||
#
|
||||
# The [omegaconf](https://github.com/omry/omegaconf/) library provides a DictConfig class which is like a `dict` with str keys, but with extra features for ease-of-use as a configuration system.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
dc = DictConfig({"a": 2, "b": True, "c": None, "d": "hello"})
|
||||
assert dc.a == dc["a"] == 2
|
||||
|
||||
|
||||
# OmegaConf has a serialization to and from yaml. The [Hydra](https://hydra.cc/) library relies on this for its configuration files.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
print(OmegaConf.to_yaml(dc))
|
||||
assert OmegaConf.create(OmegaConf.to_yaml(dc)) == dc
|
||||
|
||||
|
||||
# OmegaConf.structured provides a DictConfig from a dataclass or instance of a dataclass. Unlike a normal DictConfig, it is type-checked and only known keys can be added.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
structured = OmegaConf.structured(MyDataclass)
|
||||
assert isinstance(structured, DictConfig)
|
||||
print(structured)
|
||||
print()
|
||||
print(OmegaConf.to_yaml(structured))
|
||||
|
||||
|
||||
# `structured` knows it is missing a value for `a`.
|
||||
|
||||
# Such an object has members compatible with the dataclass, so an initialisation can be performed as follows.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
structured.a = 21
|
||||
my_dataclass_instance2 = MyDataclass(**structured)
|
||||
print(my_dataclass_instance2)
|
||||
|
||||
|
||||
# You can also call OmegaConf.structured on an instance.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
structured_from_instance = OmegaConf.structured(my_dataclass_instance)
|
||||
my_dataclass_instance3 = MyDataclass(**structured_from_instance)
|
||||
print(my_dataclass_instance3)
|
||||
|
||||
|
||||
# ## 3. Our approach to OmegaConf.structured
|
||||
#
|
||||
# We provide functions which are equivalent to `OmegaConf.structured` but support more features.
|
||||
# To achieve the above using our functions, the following is used.
|
||||
# Note that we indicate configurable classes using a special base class `Configurable`, not a decorator.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
class MyConfigurable(Configurable):
|
||||
a: int
|
||||
b: int = 8
|
||||
c: Optional[Tuple[int, ...]] = None
|
||||
|
||||
def __post_init__(self):
|
||||
print(f"created with a = {self.a}")
|
||||
self.d = 2 * self.b
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# expand_args_fields must be called on an object before it is instantiated.
|
||||
# A warning is raised if this is missed, but it is possible to not notice the warning.
|
||||
# It modifies the class like @dataclass
|
||||
expand_args_fields(MyConfigurable)
|
||||
my_configurable_instance = MyConfigurable(a=18)
|
||||
assert my_configurable_instance.d == 16
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# get_default_args calls expand_args_fields automatically
|
||||
our_structured = get_default_args(MyConfigurable)
|
||||
assert isinstance(our_structured, DictConfig)
|
||||
print(OmegaConf.to_yaml(our_structured))
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
our_structured.a = 21
|
||||
print(MyConfigurable(**our_structured))
|
||||
|
||||
|
||||
# ## 4. First enhancement: nested types 🪺
|
||||
#
|
||||
# Our system allows Configurable classes to contain each other.
|
||||
# One thing to remember: add a call to `run_auto_creation` in `__post_init__`.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
class Inner(Configurable):
|
||||
a: int = 8
|
||||
b: bool = True
|
||||
c: Tuple[int, ...] = (2, 3, 4, 6)
|
||||
|
||||
|
||||
class Outer(Configurable):
|
||||
inner: Inner
|
||||
x: str = "hello"
|
||||
xx: bool = False
|
||||
|
||||
def __post_init__(self):
|
||||
run_auto_creation(self)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
outer_dc = get_default_args(Outer)
|
||||
print(OmegaConf.to_yaml(outer_dc))
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
outer = Outer(**outer_dc)
|
||||
assert isinstance(outer, Outer)
|
||||
assert isinstance(outer.inner, Inner)
|
||||
print(vars(outer))
|
||||
print(outer.inner)
|
||||
|
||||
|
||||
# Note how inner_args is an extra member of outer. `run_auto_creation(self)` is equivalent to
|
||||
# ```
|
||||
# self.inner = Inner(**self.inner_args)
|
||||
# ```
|
||||
|
||||
# ## 5. Second enhancement: pluggable/replaceable components 🔌
|
||||
#
|
||||
# If a class uses `ReplaceableBase` as a base class instead of `Configurable`, we call it a replaceable.
|
||||
# It indicates that it is designed for child classes to use in its place.
|
||||
# We might use `NotImplementedError` to indicate functionality which subclasses are expected to implement.
|
||||
# The system maintains a global `registry` containing subclasses of each ReplaceableBase.
|
||||
# The subclasses register themselves with it with a decorator.
|
||||
#
|
||||
# A configurable class (i.e. a class which uses our system, i.e. a child of `Configurable` or `ReplaceableBase`) which contains a ReplaceableBase must also
|
||||
# contain a corresponding class_type field of type `str` which indicates which concrete child class to use.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
class InnerBase(ReplaceableBase):
|
||||
def say_something(self):
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
@registry.register
|
||||
class Inner1(InnerBase):
|
||||
a: int = 1
|
||||
b: str = "h"
|
||||
|
||||
def say_something(self):
|
||||
print("hello from an Inner1")
|
||||
|
||||
|
||||
@registry.register
|
||||
class Inner2(InnerBase):
|
||||
a: int = 2
|
||||
|
||||
def say_something(self):
|
||||
print("hello from an Inner2")
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
class Out(Configurable):
|
||||
inner: InnerBase
|
||||
inner_class_type: str = "Inner1"
|
||||
x: int = 19
|
||||
|
||||
def __post_init__(self):
|
||||
run_auto_creation(self)
|
||||
|
||||
def talk(self):
|
||||
self.inner.say_something()
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
Out_dc = get_default_args(Out)
|
||||
print(OmegaConf.to_yaml(Out_dc))
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
Out_dc.inner_class_type = "Inner2"
|
||||
out = Out(**Out_dc)
|
||||
print(out.inner)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
out.talk()
|
||||
|
||||
|
||||
# Note in this case there are many `args` members. It is usually fine to ignore them in the code. They are needed for the config.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
print(vars(out))
|
||||
|
||||
|
||||
# ## 6. Example with torch.nn.Module 🔥
|
||||
# Typically in implicitron, we use this system in combination with [`Module`](https://pytorch.org/docs/stable/generated/torch.nn.Module.html)s.
|
||||
# Note in this case it is necessary to call `Module.__init__` explicitly in `__post_init__`.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
class MyLinear(torch.nn.Module, Configurable):
|
||||
d_in: int = 2
|
||||
d_out: int = 200
|
||||
|
||||
def __post_init__(self):
|
||||
super().__init__()
|
||||
self.linear = torch.nn.Linear(in_features=self.d_in, out_features=self.d_out)
|
||||
|
||||
def forward(self, x):
|
||||
return self.linear.forward(x)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
expand_args_fields(MyLinear)
|
||||
my_linear = MyLinear()
|
||||
input = torch.zeros(2)
|
||||
output = my_linear(input)
|
||||
print("output shape:", output.shape)
|
||||
|
||||
|
||||
# `my_linear` has all the usual features of a Module.
|
||||
# E.g. it can be saved and loaded with `torch.save` and `torch.load`.
|
||||
# It has parameters:
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
for name, value in my_linear.named_parameters():
|
||||
print(name, value.shape)
|
||||
|
||||
|
||||
# ## 7. Example of implementing your own pluggable component
|
||||
# Let's say I am using a library with `Out` like in section **5** but I want to implement my own child of InnerBase.
|
||||
# All I need to do is register its definition, but I need to do this before expand_args_fields is explicitly or implicitly called on Out.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
@registry.register
|
||||
class UserImplementedInner(InnerBase):
|
||||
a: int = 200
|
||||
|
||||
def say_something(self):
|
||||
print("hello from the user")
|
||||
|
||||
|
||||
# At this point, we need to redefine the class Out.
|
||||
# Otherwise if it has already been expanded without UserImplementedInner, then the following would not work,
|
||||
# because the implementations known to a class are fixed when it is expanded.
|
||||
#
|
||||
# If you are running experiments from a script, the thing to remember here is that you must import your own modules, which register your own implementations,
|
||||
# before you *use* the library classes.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
class Out(Configurable):
|
||||
inner: InnerBase
|
||||
inner_class_type: str = "Inner1"
|
||||
x: int = 19
|
||||
|
||||
def __post_init__(self):
|
||||
run_auto_creation(self)
|
||||
|
||||
def talk(self):
|
||||
self.inner.say_something()
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
expand_args_fields(Out)
|
||||
out2 = Out(inner_class_type="UserImplementedInner")
|
||||
print(out2.inner)
|
||||
|
||||
|
||||
# ## 8: Example of making a subcomponent pluggable
|
||||
#
|
||||
# Let's look what needs to happen if we have a subcomponent which we make pluggable, to allow users to supply their own.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
class SubComponent(Configurable):
|
||||
x: float = 0.25
|
||||
|
||||
def apply(self, a: float) -> float:
|
||||
return a + self.x
|
||||
|
||||
|
||||
class LargeComponent(Configurable):
|
||||
repeats: int = 4
|
||||
subcomponent: SubComponent
|
||||
|
||||
def __post_init__(self):
|
||||
run_auto_creation(self)
|
||||
|
||||
def apply(self, a: float) -> float:
|
||||
for _ in range(self.repeats):
|
||||
a = self.subcomponent.apply(a)
|
||||
return a
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
expand_args_fields(LargeComponent)
|
||||
large_component = LargeComponent()
|
||||
assert large_component.apply(3) == 4
|
||||
print(OmegaConf.to_yaml(LargeComponent))
|
||||
|
||||
|
||||
# Made generic:
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
class SubComponentBase(ReplaceableBase):
|
||||
def apply(self, a: float) -> float:
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
@registry.register
|
||||
class SubComponent(SubComponentBase):
|
||||
x: float = 0.25
|
||||
|
||||
def apply(self, a: float) -> float:
|
||||
return a + self.x
|
||||
|
||||
|
||||
class LargeComponent(Configurable):
|
||||
repeats: int = 4
|
||||
subcomponent: SubComponentBase
|
||||
subcomponent_class_type: str = "SubComponent"
|
||||
|
||||
def __post_init__(self):
|
||||
run_auto_creation(self)
|
||||
|
||||
def apply(self, a: float) -> float:
|
||||
for _ in range(self.repeats):
|
||||
a = self.subcomponent.apply(a)
|
||||
return a
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
expand_args_fields(LargeComponent)
|
||||
large_component = LargeComponent()
|
||||
assert large_component.apply(3) == 4
|
||||
print(OmegaConf.to_yaml(LargeComponent))
|
||||
|
||||
|
||||
# The following things had to change:
|
||||
# * The base class SubComponentBase was defined.
|
||||
# * SubComponent gained a `@registry.register` decoration and had its base class changed to the new one.
|
||||
# * `subcomponent_class_type` was added as a member of the outer class.
|
||||
# * In any saved configuration yaml files, the key `subcomponent_args` had to be changed to `subcomponent_SubComponent_args`.
|
||||
|
||||
# ## Appendix: gotchas ⚠️
|
||||
#
|
||||
# * Omitting to define `__post_init__` or not calling `run_auto_creation` in it.
|
||||
# * Using a configurable class without calling get_default_args or expand_args_fields on it.
|
||||
# * Omitting a type annotation on a field. For example, writing
|
||||
# ```
|
||||
# subcomponent_class_type = "SubComponent"
|
||||
# ```
|
||||
# instead of
|
||||
# ```
|
||||
# subcomponent_class_type: str = "SubComponent"
|
||||
# ```
|
||||
#
|
||||
#
|
||||
913
files/implicitron_volumes.ipynb
Normal file
913
files/implicitron_volumes.ipynb
Normal file
@ -0,0 +1,913 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659619824914,
|
||||
"executionStopTime": 1659619825485,
|
||||
"originalKey": "d38652e8-200a-413c-a36a-f4d349b78a9d",
|
||||
"requestMsgId": "641de8aa-0e42-4446-9304-c160a2d226bf",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright (c) Meta Platforms, Inc. and affiliates. All rights reserved."
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"customInput": null,
|
||||
"originalKey": "a48a9dcf-e80f-474b-a0c4-2c9a765b15c5",
|
||||
"showInput": false
|
||||
},
|
||||
"source": [
|
||||
"# A simple model using Implicitron\n",
|
||||
"\n",
|
||||
"In this demo, we use the VolumeRenderer from PyTorch3D as a custom implicit function in Implicitron. We will see\n",
|
||||
"* some of the main objects in Implicitron\n",
|
||||
"* how to plug in a custom part of a model"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"customInput": null,
|
||||
"originalKey": "51337c0e-ad27-4b75-ad6a-737dca5d7b95",
|
||||
"showInput": false
|
||||
},
|
||||
"source": [
|
||||
"## 0. Install and import modules\n",
|
||||
"\n",
|
||||
"Ensure `torch` and `torchvision` are installed. If `pytorch3d` is not installed, install it using the following cell:\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659619898147,
|
||||
"executionStopTime": 1659619898274,
|
||||
"originalKey": "76f1ecd4-6b73-4214-81b0-118ef8d86872",
|
||||
"requestMsgId": "deb6a860-6923-4227-abef-d31388b5142d",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"import torch\n",
|
||||
"need_pytorch3d=False\n",
|
||||
"try:\n",
|
||||
" import pytorch3d\n",
|
||||
"except ModuleNotFoundError:\n",
|
||||
" need_pytorch3d=True\n",
|
||||
"if need_pytorch3d:\n",
|
||||
" if torch.__version__.startswith(\"1.12.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" # We try to install PyTorch3D via a released wheel.\n",
|
||||
" pyt_version_str=torch.__version__.split(\"+\")[0].replace(\".\", \"\")\n",
|
||||
" version_str=\"\".join([\n",
|
||||
" f\"py3{sys.version_info.minor}_cu\",\n",
|
||||
" torch.version.cuda.replace(\".\",\"\"),\n",
|
||||
" f\"_pyt{pyt_version_str}\"\n",
|
||||
" ])\n",
|
||||
" !pip install fvcore iopath\n",
|
||||
" !pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/{version_str}/download.html\n",
|
||||
" else:\n",
|
||||
" # We try to install PyTorch3D from source.\n",
|
||||
" !curl -LO https://github.com/NVIDIA/cub/archive/1.10.0.tar.gz\n",
|
||||
" !tar xzf 1.10.0.tar.gz\n",
|
||||
" os.environ[\"CUB_HOME\"] = os.getcwd() + \"/cub-1.10.0\"\n",
|
||||
" !pip install 'git+https://github.com/facebookresearch/pytorch3d.git@stable'"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"customInput": null,
|
||||
"originalKey": "2c1020e6-eb4a-4644-9719-9147500d8e4f",
|
||||
"showInput": false
|
||||
},
|
||||
"source": [
|
||||
"Ensure omegaconf and visdom are installed. If not, run this cell. (It should not be necessary to restart the runtime.)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"originalKey": "9e751931-a38d-44c9-9ff1-ac2f7d3a3f99",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install omegaconf visdom"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659612480556,
|
||||
"executionStopTime": 1659612480644,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "86807e4a-1675-4520-a033-c7af85b233ec",
|
||||
"requestMsgId": "880a7e20-4a90-4b37-a5eb-bccc0b23cac6"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import logging\n",
|
||||
"from typing import Tuple\n",
|
||||
"\n",
|
||||
"import matplotlib.animation as animation\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"import numpy as np\n",
|
||||
"import torch\n",
|
||||
"import tqdm\n",
|
||||
"from IPython.display import HTML\n",
|
||||
"from omegaconf import OmegaConf\n",
|
||||
"from PIL import Image\n",
|
||||
"from pytorch3d.implicitron.dataset.dataset_base import FrameData\n",
|
||||
"from pytorch3d.implicitron.dataset.rendered_mesh_dataset_map_provider import RenderedMeshDatasetMapProvider\n",
|
||||
"from pytorch3d.implicitron.models.generic_model import GenericModel\n",
|
||||
"from pytorch3d.implicitron.models.implicit_function.base import ImplicitFunctionBase\n",
|
||||
"from pytorch3d.implicitron.models.renderer.base import EvaluationMode\n",
|
||||
"from pytorch3d.implicitron.tools.config import expand_args_fields, get_default_args, registry, remove_unused_components\n",
|
||||
"from pytorch3d.renderer import RayBundle\n",
|
||||
"from pytorch3d.renderer.implicit.renderer import VolumeSampler\n",
|
||||
"from pytorch3d.structures import Volumes\n",
|
||||
"from pytorch3d.vis.plotly_vis import plot_batch_individually, plot_scene"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659610929375,
|
||||
"executionStopTime": 1659610929383,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "b2d9f5bd-a9d4-4f78-b21e-92f2658e0fe9",
|
||||
"requestMsgId": "7e43e623-4030-438b-af4e-b96170c9a052",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"output_resolution = 80"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659610930042,
|
||||
"executionStopTime": 1659610930050,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "0b0c2087-4c86-4c57-b0ee-6f48a70a9c78",
|
||||
"requestMsgId": "46883aad-f00b-4fd4-ac17-eec0b2ac272a",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"torch.set_printoptions(sci_mode=False)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"customInput": null,
|
||||
"originalKey": "37809d0d-b02e-42df-85b6-cdd038373653",
|
||||
"showInput": false
|
||||
},
|
||||
"source": [
|
||||
"## 1. Load renders of a mesh (the cow mesh) as a dataset\n",
|
||||
"\n",
|
||||
"A dataset's train, val and test parts in Implicitron are represented as a `dataset_map`, and provided by an implementation of `DatasetMapProvider`. \n",
|
||||
"`RenderedMeshDatasetMapProvider` is one which generates a single-scene dataset with only a train component by taking a mesh and rendering it.\n",
|
||||
"We use it with the cow mesh."
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659620739780,
|
||||
"executionStopTime": 1659620739914,
|
||||
"originalKey": "cc68cb9c-b8bf-4e9e-bef1-2cfafdf6caa2",
|
||||
"requestMsgId": "398cfcae-5d43-4b6f-9c75-db3d297364d4",
|
||||
"showInput": false
|
||||
},
|
||||
"source": [
|
||||
"If running this notebook using **Google Colab**, run the following cell to fetch the mesh obj and texture files and save it at the path data/cow_mesh.\n",
|
||||
"If running locally, the data is already available at the correct path."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"originalKey": "2c55e002-a885-4169-8fdc-af9078b05968",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!mkdir -p data/cow_mesh\n",
|
||||
"!wget -P data/cow_mesh https://dl.fbaipublicfiles.com/pytorch3d/data/cow_mesh/cow.obj\n",
|
||||
"!wget -P data/cow_mesh https://dl.fbaipublicfiles.com/pytorch3d/data/cow_mesh/cow.mtl\n",
|
||||
"!wget -P data/cow_mesh https://dl.fbaipublicfiles.com/pytorch3d/data/cow_mesh/cow_texture.png"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"customInput": null,
|
||||
"originalKey": "2a976be8-01bf-4a1c-a6e7-61d5d08c3dbd",
|
||||
"showInput": false
|
||||
},
|
||||
"source": [
|
||||
"If we want to instantiate one of Implicitron's configurable objects, such as `RenderedMeshDatasetMapProvider`, without using the OmegaConf initialisation (get_default_args), we need to call `expand_args_fields` on the class first."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659621652237,
|
||||
"executionStopTime": 1659621652903,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "eb77aaec-048c-40bd-bd69-0e66b6ab60b1",
|
||||
"requestMsgId": "09b9975c-ff86-41c9-b4a9-975d23afc562",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"expand_args_fields(RenderedMeshDatasetMapProvider)\n",
|
||||
"cow_provider = RenderedMeshDatasetMapProvider(\n",
|
||||
" data_file=\"data/cow_mesh/cow.obj\",\n",
|
||||
" use_point_light=False,\n",
|
||||
" resolution=output_resolution,\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659610966145,
|
||||
"executionStopTime": 1659610966255,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "8210e15b-da48-4306-a49a-41c4e7e7d42f",
|
||||
"requestMsgId": "c243edd2-a106-4fba-8471-dfa4f99a2088",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"dataset_map = cow_provider.get_dataset_map()\n",
|
||||
"tr_cameras = [training_frame.camera for training_frame in dataset_map.train]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659610967703,
|
||||
"executionStopTime": 1659610967848,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "458d72ad-d9a7-4f13-b5b7-90d2aec61c16",
|
||||
"requestMsgId": "7f9431f3-8717-4d89-a7fe-1420dd0e00c4",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# The cameras are all in the XZ plane, in a circle about 2.7 from the origin\n",
|
||||
"centers = torch.cat([i.get_camera_center() for i in tr_cameras])\n",
|
||||
"print(centers.min(0).values)\n",
|
||||
"print(centers.max(0).values)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659552920194,
|
||||
"executionStopTime": 1659552923122,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "931e712b-b141-437a-97fb-dc2a07ce3458",
|
||||
"requestMsgId": "931e712b-b141-437a-97fb-dc2a07ce3458",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# visualization of the cameras\n",
|
||||
"plot = plot_scene({\"k\": {i: camera for i, camera in enumerate(tr_cameras)}}, camera_scale=0.25)\n",
|
||||
"plot.layout.scene.aspectmode = \"data\"\n",
|
||||
"plot"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"customInput": null,
|
||||
"originalKey": "afa9c02d-f76b-4f68-83e9-9733c615406b",
|
||||
"showInput": false
|
||||
},
|
||||
"source": [
|
||||
"## 2. Custom implicit function 🧊\n",
|
||||
"\n",
|
||||
"At the core of neural rendering methods are functions of spatial coordinates called implicit functions, which are used in some kind of rendering process.\n",
|
||||
"(Often those functions can additionally take other data as well, such as view direction.)\n",
|
||||
"A common rendering process is ray marching over densities and colors provided by an implicit function.\n",
|
||||
"In our case, taking samples from a 3D volume grid is a very simple function of spatial coordinates. \n",
|
||||
"\n",
|
||||
"Here we define our own implicit function, which uses PyTorch3D's existing functionality for sampling from a volume grid.\n",
|
||||
"We do this by subclassing `ImplicitFunctionBase`.\n",
|
||||
"We need to register our subclass with a special decorator.\n",
|
||||
"We use Python's dataclass annotations for configuring the module."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659613575850,
|
||||
"executionStopTime": 1659613575940,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "61b55043-dc52-4de7-992e-e2195edd2123",
|
||||
"requestMsgId": "dfaace3c-098c-4ffe-9240-6a7ae0ff271e",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"@registry.register\n",
|
||||
"class MyVolumes(ImplicitFunctionBase, torch.nn.Module):\n",
|
||||
" grid_resolution: int = 50 # common HWD of volumes, the number of voxels in each direction\n",
|
||||
" extent: float = 1.0 # In world coordinates, the volume occupies is [-extent, extent] along each axis\n",
|
||||
"\n",
|
||||
" def __post_init__(self):\n",
|
||||
" # We have to call this explicitly if there are other base classes like Module\n",
|
||||
" super().__init__()\n",
|
||||
"\n",
|
||||
" # We define parameters like other torch.nn.Module objects.\n",
|
||||
" # In this case, both our parameter tensors are trainable; they govern the contents of the volume grid.\n",
|
||||
" density = torch.full((self.grid_resolution, self.grid_resolution, self.grid_resolution), -2.0)\n",
|
||||
" self.density = torch.nn.Parameter(density)\n",
|
||||
" color = torch.full((3, self.grid_resolution, self.grid_resolution, self.grid_resolution), 0.0)\n",
|
||||
" self.color = torch.nn.Parameter(color)\n",
|
||||
" self.density_activation = torch.nn.Softplus()\n",
|
||||
"\n",
|
||||
" def forward(\n",
|
||||
" self,\n",
|
||||
" ray_bundle: RayBundle,\n",
|
||||
" fun_viewpool=None,\n",
|
||||
" global_code=None,\n",
|
||||
" ):\n",
|
||||
" densities = self.density_activation(self.density[None, None])\n",
|
||||
" voxel_size = 2.0 * float(self.extent) / self.grid_resolution\n",
|
||||
" features = self.color.sigmoid()[None]\n",
|
||||
"\n",
|
||||
" # Like other PyTorch3D structures, the actual Volumes object should only exist as long\n",
|
||||
" # as one iteration of training. It is local to this function.\n",
|
||||
"\n",
|
||||
" volume = Volumes(densities=densities, features=features, voxel_size=voxel_size)\n",
|
||||
" sampler = VolumeSampler(volumes=volume)\n",
|
||||
" densities, features = sampler(ray_bundle)\n",
|
||||
"\n",
|
||||
" # When an implicit function is used for raymarching, i.e. for MultiPassEmissionAbsorptionRenderer,\n",
|
||||
" # it must return (densities, features, an auxiliary tuple)\n",
|
||||
" return densities, features, {}\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"customInput": null,
|
||||
"originalKey": "abaf2cd6-1b68-400e-a142-8fb9f49953f3",
|
||||
"showInput": false
|
||||
},
|
||||
"source": [
|
||||
"## 3. Construct the model object.\n",
|
||||
"\n",
|
||||
"The main model object in PyTorch3D is `GenericModel`, which has pluggable components for the major steps, including the renderer and the implicit function(s).\n",
|
||||
"There are two ways to construct it which are equivalent here."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659621267561,
|
||||
"executionStopTime": 1659621267938,
|
||||
"originalKey": "f26c3dce-fbae-4592-bd0e-e4a8abc57c2c",
|
||||
"requestMsgId": "9213687e-1caf-46a8-a4e5-a9c531530092",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"CONSTRUCT_MODEL_FROM_CONFIG = True\n",
|
||||
"if CONSTRUCT_MODEL_FROM_CONFIG:\n",
|
||||
" # Via a DictConfig - this is how our training loop with hydra works\n",
|
||||
" cfg = get_default_args(GenericModel)\n",
|
||||
" cfg.implicit_function_class_type = \"MyVolumes\"\n",
|
||||
" cfg.render_image_height=output_resolution\n",
|
||||
" cfg.render_image_width=output_resolution\n",
|
||||
" cfg.loss_weights={\"loss_rgb_huber\": 1.0}\n",
|
||||
" cfg.tqdm_trigger_threshold=19000\n",
|
||||
" cfg.raysampler_AdaptiveRaySampler_args.scene_extent= 4.0\n",
|
||||
" gm = GenericModel(**cfg)\n",
|
||||
"else:\n",
|
||||
" # constructing GenericModel directly\n",
|
||||
" expand_args_fields(GenericModel)\n",
|
||||
" gm = GenericModel(\n",
|
||||
" implicit_function_class_type=\"MyVolumes\",\n",
|
||||
" render_image_height=output_resolution,\n",
|
||||
" render_image_width=output_resolution,\n",
|
||||
" loss_weights={\"loss_rgb_huber\": 1.0},\n",
|
||||
" tqdm_trigger_threshold=19000,\n",
|
||||
" raysampler_AdaptiveRaySampler_args = {\"scene_extent\": 4.0}\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" # In this case we can get the equivalent DictConfig cfg object to the way gm is configured as follows\n",
|
||||
" cfg = OmegaConf.structured(gm)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659611214689,
|
||||
"executionStopTime": 1659611214748,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "4e659f7d-ce66-4999-83de-005eb09d7705",
|
||||
"requestMsgId": "7b815b2b-cf19-44d0-ae89-76fde6df35ec",
|
||||
"showInput": false
|
||||
},
|
||||
"source": [
|
||||
" The default renderer is an emission-absorbtion raymarcher. We keep that default."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659621268007,
|
||||
"executionStopTime": 1659621268190,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "d37ae488-c57c-44d3-9def-825dc1a6495b",
|
||||
"requestMsgId": "71143ec1-730f-4876-8a14-e46eea9d6dd1",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# We can display the configuration in use as follows.\n",
|
||||
"remove_unused_components(cfg)\n",
|
||||
"yaml = OmegaConf.to_yaml(cfg, sort_keys=False)\n",
|
||||
"%page -r yaml"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659621268727,
|
||||
"executionStopTime": 1659621268776,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "52e53179-3c6e-4c1f-a38a-3a6d803687bb",
|
||||
"requestMsgId": "05de9bc3-3f74-4a6f-851c-9ec919b59506",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"device = torch.device(\"cuda:0\")\n",
|
||||
"gm.to(device)\n",
|
||||
"assert next(gm.parameters()).is_cuda"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"customInput": null,
|
||||
"originalKey": "528a7d53-c645-49c2-9021-09adbb18cd23",
|
||||
"showInput": false
|
||||
},
|
||||
"source": [
|
||||
"## 4. train the model "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659621270236,
|
||||
"executionStopTime": 1659621270446,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "953280bd-3161-42ba-8dcb-0c8ef2d5cc25",
|
||||
"requestMsgId": "9bba424b-7bfd-4e5a-9d79-ae316e20bab0",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"train_data_collated = [FrameData.collate([frame.to(device)]) for frame in dataset_map.train]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659621270815,
|
||||
"executionStopTime": 1659621270948,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "2fcf07f0-0c28-49c7-8c76-1c9a9d810167",
|
||||
"requestMsgId": "821deb43-6084-4ece-83c3-dee214562c47",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"gm.train()\n",
|
||||
"optimizer = torch.optim.Adam(gm.parameters(), lr=0.1)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659621271875,
|
||||
"executionStopTime": 1659621298146,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "105099f7-ed0c-4e7f-a976-61a93fd0a8fe",
|
||||
"requestMsgId": "0c87c108-83e3-4129-ad02-85e0140f1368",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"iterator = tqdm.tqdm(range(2000))\n",
|
||||
"for n_batch in iterator:\n",
|
||||
" optimizer.zero_grad()\n",
|
||||
"\n",
|
||||
" frame = train_data_collated[n_batch % len(dataset_map.train)]\n",
|
||||
" out = gm(**frame, evaluation_mode=EvaluationMode.TRAINING)\n",
|
||||
" out[\"objective\"].backward()\n",
|
||||
" if n_batch % 100 == 0:\n",
|
||||
" iterator.set_postfix_str(f\"loss: {float(out['objective']):.5f}\")\n",
|
||||
" optimizer.step()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659535024768,
|
||||
"executionStopTime": 1659535024906,
|
||||
"originalKey": "e3cd494a-536b-48bc-8290-c048118c82eb",
|
||||
"requestMsgId": "e3cd494a-536b-48bc-8290-c048118c82eb",
|
||||
"showInput": false
|
||||
},
|
||||
"source": [
|
||||
"## 5. Evaluate the module\n",
|
||||
"\n",
|
||||
"We generate complete images from all the viewpoints to see how they look."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659621299859,
|
||||
"executionStopTime": 1659621311133,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "fbe1b2ea-cc24-4b20-a2d7-0249185e34a5",
|
||||
"requestMsgId": "771ef1f8-5eee-4932-9e81-33604bf0512a",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def to_numpy_image(image):\n",
|
||||
" # Takes an image of shape (C, H, W) in [0,1], where C=3 or 1\n",
|
||||
" # to a numpy uint image of shape (H, W, 3)\n",
|
||||
" return (image * 255).to(torch.uint8).permute(1, 2, 0).detach().cpu().expand(-1, -1, 3).numpy()\n",
|
||||
"def resize_image(image):\n",
|
||||
" # Takes images of shape (B, C, H, W) to (B, C, output_resolution, output_resolution)\n",
|
||||
" return torch.nn.functional.interpolate(image, size=(output_resolution, output_resolution))\n",
|
||||
"\n",
|
||||
"gm.eval()\n",
|
||||
"images = []\n",
|
||||
"expected = []\n",
|
||||
"masks = []\n",
|
||||
"masks_expected = []\n",
|
||||
"for frame in tqdm.tqdm(train_data_collated):\n",
|
||||
" with torch.no_grad():\n",
|
||||
" out = gm(**frame, evaluation_mode=EvaluationMode.EVALUATION)\n",
|
||||
"\n",
|
||||
" image_rgb = to_numpy_image(out[\"images_render\"][0])\n",
|
||||
" mask = to_numpy_image(out[\"masks_render\"][0])\n",
|
||||
" expd = to_numpy_image(resize_image(frame.image_rgb)[0])\n",
|
||||
" mask_expected = to_numpy_image(resize_image(frame.fg_probability)[0])\n",
|
||||
"\n",
|
||||
" images.append(image_rgb)\n",
|
||||
" masks.append(mask)\n",
|
||||
" expected.append(expd)\n",
|
||||
" masks_expected.append(mask_expected)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659614622542,
|
||||
"executionStopTime": 1659614622757,
|
||||
"originalKey": "24953039-9780-40fd-bd81-5d63e9f40069",
|
||||
"requestMsgId": "7af895a3-dfe4-4c28-ac3b-4ff0fbb40c7f",
|
||||
"showInput": false
|
||||
},
|
||||
"source": [
|
||||
"We draw a grid showing predicted image and expected image, followed by predicted mask and expected mask, from each viewpoint. \n",
|
||||
"This is a grid of four rows of images, wrapped in to several large rows, i.e..\n",
|
||||
"<small><center>\n",
|
||||
"```\n",
|
||||
"┌────────┬────────┐ ┌────────┐\n",
|
||||
"│pred │pred │ │pred │\n",
|
||||
"│image │image │ │image │\n",
|
||||
"│1 │2 │ │n │\n",
|
||||
"├────────┼────────┤ ├────────┤\n",
|
||||
"│expected│expected│ │expected│\n",
|
||||
"│image │image │ ... │image │\n",
|
||||
"│1 │2 │ │n │\n",
|
||||
"├────────┼────────┤ ├────────┤\n",
|
||||
"│pred │pred │ │pred │\n",
|
||||
"│mask │mask │ │mask │\n",
|
||||
"│1 │2 │ │n │\n",
|
||||
"├────────┼────────┤ ├────────┤\n",
|
||||
"│expected│expected│ │expected│\n",
|
||||
"│mask │mask │ │mask │\n",
|
||||
"│1 │2 │ │n │\n",
|
||||
"├────────┼────────┤ ├────────┤\n",
|
||||
"│pred │pred │ │pred │\n",
|
||||
"│image │image │ │image │\n",
|
||||
"│n+1 │n+1 │ │2n │\n",
|
||||
"├────────┼────────┤ ├────────┤\n",
|
||||
"│expected│expected│ │expected│\n",
|
||||
"│image │image │ ... │image │\n",
|
||||
"│n+1 │n+2 │ │2n │\n",
|
||||
"├────────┼────────┤ ├────────┤\n",
|
||||
"│pred │pred │ │pred │\n",
|
||||
"│mask │mask │ │mask │\n",
|
||||
"│n+1 │n+2 │ │2n │\n",
|
||||
"├────────┼────────┤ ├────────┤\n",
|
||||
"│expected│expected│ │expected│\n",
|
||||
"│mask │mask │ │mask │\n",
|
||||
"│n+1 │n+2 │ │2n │\n",
|
||||
"└────────┴────────┘ └────────┘\n",
|
||||
" ...\n",
|
||||
"```\n",
|
||||
"</center></small>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659621313894,
|
||||
"executionStopTime": 1659621314042,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "c488a34a-e46d-4649-93fb-4b1bb5a0e439",
|
||||
"requestMsgId": "4221e632-fca1-4fe5-b2e3-f92c37aa40e4",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"images_to_display = [images.copy(), expected.copy(), masks.copy(), masks_expected.copy()]\n",
|
||||
"n_rows = 4\n",
|
||||
"n_images = len(images)\n",
|
||||
"blank_image = images[0] * 0\n",
|
||||
"n_per_row = 1+(n_images-1)//n_rows\n",
|
||||
"for _ in range(n_per_row*n_rows - n_images):\n",
|
||||
" for group in images_to_display:\n",
|
||||
" group.append(blank_image)\n",
|
||||
"\n",
|
||||
"images_to_display_listed = [[[i] for i in j] for j in images_to_display]\n",
|
||||
"split = []\n",
|
||||
"for row in range(n_rows):\n",
|
||||
" for group in images_to_display_listed:\n",
|
||||
" split.append(group[row*n_per_row:(row+1)*n_per_row]) \n",
|
||||
"\n",
|
||||
"Image.fromarray(np.block(split))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659621323795,
|
||||
"executionStopTime": 1659621323820,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "49eab9e1-4fe2-4fbe-b4f3-7b6953340170",
|
||||
"requestMsgId": "85b402ad-f903-431f-a13e-c2d697e869bb",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Print the maximum channel intensity in the first image.\n",
|
||||
"print(images[1].max()/255)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"code_folding": [],
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659621408642,
|
||||
"executionStopTime": 1659621409559,
|
||||
"hidden_ranges": [],
|
||||
"originalKey": "137d2c43-d39d-4266-ac5e-2b714da5e0ee",
|
||||
"requestMsgId": "8e27ec57-c2d6-4ae0-be69-b63b6af929ff",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"plt.ioff()\n",
|
||||
"fig, ax = plt.subplots(figsize=(3,3))\n",
|
||||
"\n",
|
||||
"ax.grid(None)\n",
|
||||
"ims = [[ax.imshow(im, animated=True)] for im in images]\n",
|
||||
"ani = animation.ArtistAnimation(fig, ims, interval=80, blit=True)\n",
|
||||
"ani_html = ani.to_jshtml()\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659621409620,
|
||||
"executionStopTime": 1659621409725,
|
||||
"originalKey": "783e70d6-7cf1-4d76-a126-ba11ffc2f5be",
|
||||
"requestMsgId": "b6843506-c5fa-4508-80fc-8ecae51a934a",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"HTML(ani_html)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"customInput": null,
|
||||
"customOutput": null,
|
||||
"executionStartTime": 1659614670081,
|
||||
"executionStopTime": 1659614670168,
|
||||
"originalKey": "0286c350-2362-4f47-8181-2fc2ba51cfcf",
|
||||
"requestMsgId": "976f4db9-d4c7-466c-bcfd-218234400226",
|
||||
"showInput": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# If you want to see the output of the model with the volume forced to opaque white, run this and re-evaluate\n",
|
||||
"# with torch.no_grad():\n",
|
||||
"# gm._implicit_functions[0]._fn.density.fill_(9.0)\n",
|
||||
"# gm._implicit_functions[0]._fn.color.fill_(9.0)\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"bento_stylesheets": {
|
||||
"bento/extensions/flow/main.css": true,
|
||||
"bento/extensions/kernel_selector/main.css": true,
|
||||
"bento/extensions/kernel_ui/main.css": true,
|
||||
"bento/extensions/new_kernel/main.css": true,
|
||||
"bento/extensions/system_usage/main.css": true,
|
||||
"bento/extensions/theme/main.css": true
|
||||
},
|
||||
"captumWidgetMessage": {},
|
||||
"dataExplorerConfig": {},
|
||||
"kernelspec": {
|
||||
"display_name": "pytorch3d",
|
||||
"language": "python",
|
||||
"metadata": {
|
||||
"cinder_runtime": false,
|
||||
"fbpkg_supported": true,
|
||||
"is_prebuilt": true,
|
||||
"kernel_name": "bento_kernel_pytorch3d",
|
||||
"nightly_builds": true
|
||||
},
|
||||
"name": "bento_kernel_pytorch3d"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3"
|
||||
},
|
||||
"last_base_url": "https://9177.od.fbinfra.net:443/",
|
||||
"last_kernel_id": "bb33cd83-7924-489a-8bd8-2d9d62eb0126",
|
||||
"last_msg_id": "99f7088e-d22b355b859660479ef0574e_5743",
|
||||
"last_server_session_id": "2944b203-9ea8-4c0e-9634-645dfea5f26b",
|
||||
"outputWidgetContext": {}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
420
files/implicitron_volumes.py
Normal file
420
files/implicitron_volumes.py
Normal file
@ -0,0 +1,420 @@
|
||||
|
||||
# coding: utf-8
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# Copyright (c) Meta Platforms, Inc. and affiliates. All rights reserved.
|
||||
|
||||
|
||||
# # A simple model using Implicitron
|
||||
#
|
||||
# In this demo, we use the VolumeRenderer from PyTorch3D as a custom implicit function in Implicitron. We will see
|
||||
# * some of the main objects in Implicitron
|
||||
# * how to plug in a custom part of a model
|
||||
|
||||
# ## 0. Install and import modules
|
||||
#
|
||||
# Ensure `torch` and `torchvision` are installed. If `pytorch3d` is not installed, install it using the following cell:
|
||||
#
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
import os
|
||||
import sys
|
||||
import torch
|
||||
need_pytorch3d=False
|
||||
try:
|
||||
import pytorch3d
|
||||
except ModuleNotFoundError:
|
||||
need_pytorch3d=True
|
||||
if need_pytorch3d:
|
||||
if torch.__version__.startswith("1.12.") and sys.platform.startswith("linux"):
|
||||
# We try to install PyTorch3D via a released wheel.
|
||||
pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
|
||||
version_str="".join([
|
||||
f"py3{sys.version_info.minor}_cu",
|
||||
torch.version.cuda.replace(".",""),
|
||||
f"_pyt{pyt_version_str}"
|
||||
])
|
||||
get_ipython().system('pip install fvcore iopath')
|
||||
get_ipython().system('pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/{version_str}/download.html')
|
||||
else:
|
||||
# We try to install PyTorch3D from source.
|
||||
get_ipython().system('curl -LO https://github.com/NVIDIA/cub/archive/1.10.0.tar.gz')
|
||||
get_ipython().system('tar xzf 1.10.0.tar.gz')
|
||||
os.environ["CUB_HOME"] = os.getcwd() + "/cub-1.10.0"
|
||||
get_ipython().system("pip install 'git+https://github.com/facebookresearch/pytorch3d.git@stable'")
|
||||
|
||||
|
||||
# Ensure omegaconf and visdom are installed. If not, run this cell. (It should not be necessary to restart the runtime.)
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
get_ipython().system('pip install omegaconf visdom')
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
import logging
|
||||
from typing import Tuple
|
||||
|
||||
import matplotlib.animation as animation
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
import torch
|
||||
import tqdm
|
||||
from IPython.display import HTML
|
||||
from omegaconf import OmegaConf
|
||||
from PIL import Image
|
||||
from pytorch3d.implicitron.dataset.dataset_base import FrameData
|
||||
from pytorch3d.implicitron.dataset.rendered_mesh_dataset_map_provider import RenderedMeshDatasetMapProvider
|
||||
from pytorch3d.implicitron.models.generic_model import GenericModel
|
||||
from pytorch3d.implicitron.models.implicit_function.base import ImplicitFunctionBase
|
||||
from pytorch3d.implicitron.models.renderer.base import EvaluationMode
|
||||
from pytorch3d.implicitron.tools.config import expand_args_fields, get_default_args, registry, remove_unused_components
|
||||
from pytorch3d.renderer import RayBundle
|
||||
from pytorch3d.renderer.implicit.renderer import VolumeSampler
|
||||
from pytorch3d.structures import Volumes
|
||||
from pytorch3d.vis.plotly_vis import plot_batch_individually, plot_scene
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
output_resolution = 80
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
torch.set_printoptions(sci_mode=False)
|
||||
|
||||
|
||||
# ## 1. Load renders of a mesh (the cow mesh) as a dataset
|
||||
#
|
||||
# A dataset's train, val and test parts in Implicitron are represented as a `dataset_map`, and provided by an implementation of `DatasetMapProvider`.
|
||||
# `RenderedMeshDatasetMapProvider` is one which generates a single-scene dataset with only a train component by taking a mesh and rendering it.
|
||||
# We use it with the cow mesh.
|
||||
|
||||
# If running this notebook using **Google Colab**, run the following cell to fetch the mesh obj and texture files and save it at the path data/cow_mesh.
|
||||
# If running locally, the data is already available at the correct path.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
get_ipython().system('mkdir -p data/cow_mesh')
|
||||
get_ipython().system('wget -P data/cow_mesh https://dl.fbaipublicfiles.com/pytorch3d/data/cow_mesh/cow.obj')
|
||||
get_ipython().system('wget -P data/cow_mesh https://dl.fbaipublicfiles.com/pytorch3d/data/cow_mesh/cow.mtl')
|
||||
get_ipython().system('wget -P data/cow_mesh https://dl.fbaipublicfiles.com/pytorch3d/data/cow_mesh/cow_texture.png')
|
||||
|
||||
|
||||
# If we want to instantiate one of Implicitron's configurable objects, such as `RenderedMeshDatasetMapProvider`, without using the OmegaConf initialisation (get_default_args), we need to call `expand_args_fields` on the class first.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
expand_args_fields(RenderedMeshDatasetMapProvider)
|
||||
cow_provider = RenderedMeshDatasetMapProvider(
|
||||
data_file="data/cow_mesh/cow.obj",
|
||||
use_point_light=False,
|
||||
resolution=output_resolution,
|
||||
)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
dataset_map = cow_provider.get_dataset_map()
|
||||
tr_cameras = [training_frame.camera for training_frame in dataset_map.train]
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# The cameras are all in the XZ plane, in a circle about 2.7 from the origin
|
||||
centers = torch.cat([i.get_camera_center() for i in tr_cameras])
|
||||
print(centers.min(0).values)
|
||||
print(centers.max(0).values)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# visualization of the cameras
|
||||
plot = plot_scene({"k": {i: camera for i, camera in enumerate(tr_cameras)}}, camera_scale=0.25)
|
||||
plot.layout.scene.aspectmode = "data"
|
||||
plot
|
||||
|
||||
|
||||
# ## 2. Custom implicit function 🧊
|
||||
#
|
||||
# At the core of neural rendering methods are functions of spatial coordinates called implicit functions, which are used in some kind of rendering process.
|
||||
# (Often those functions can additionally take other data as well, such as view direction.)
|
||||
# A common rendering process is ray marching over densities and colors provided by an implicit function.
|
||||
# In our case, taking samples from a 3D volume grid is a very simple function of spatial coordinates.
|
||||
#
|
||||
# Here we define our own implicit function, which uses PyTorch3D's existing functionality for sampling from a volume grid.
|
||||
# We do this by subclassing `ImplicitFunctionBase`.
|
||||
# We need to register our subclass with a special decorator.
|
||||
# We use Python's dataclass annotations for configuring the module.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
@registry.register
|
||||
class MyVolumes(ImplicitFunctionBase, torch.nn.Module):
|
||||
grid_resolution: int = 50 # common HWD of volumes, the number of voxels in each direction
|
||||
extent: float = 1.0 # In world coordinates, the volume occupies is [-extent, extent] along each axis
|
||||
|
||||
def __post_init__(self):
|
||||
# We have to call this explicitly if there are other base classes like Module
|
||||
super().__init__()
|
||||
|
||||
# We define parameters like other torch.nn.Module objects.
|
||||
# In this case, both our parameter tensors are trainable; they govern the contents of the volume grid.
|
||||
density = torch.full((self.grid_resolution, self.grid_resolution, self.grid_resolution), -2.0)
|
||||
self.density = torch.nn.Parameter(density)
|
||||
color = torch.full((3, self.grid_resolution, self.grid_resolution, self.grid_resolution), 0.0)
|
||||
self.color = torch.nn.Parameter(color)
|
||||
self.density_activation = torch.nn.Softplus()
|
||||
|
||||
def forward(
|
||||
self,
|
||||
ray_bundle: RayBundle,
|
||||
fun_viewpool=None,
|
||||
global_code=None,
|
||||
):
|
||||
densities = self.density_activation(self.density[None, None])
|
||||
voxel_size = 2.0 * float(self.extent) / self.grid_resolution
|
||||
features = self.color.sigmoid()[None]
|
||||
|
||||
# Like other PyTorch3D structures, the actual Volumes object should only exist as long
|
||||
# as one iteration of training. It is local to this function.
|
||||
|
||||
volume = Volumes(densities=densities, features=features, voxel_size=voxel_size)
|
||||
sampler = VolumeSampler(volumes=volume)
|
||||
densities, features = sampler(ray_bundle)
|
||||
|
||||
# When an implicit function is used for raymarching, i.e. for MultiPassEmissionAbsorptionRenderer,
|
||||
# it must return (densities, features, an auxiliary tuple)
|
||||
return densities, features, {}
|
||||
|
||||
|
||||
# ## 3. Construct the model object.
|
||||
#
|
||||
# The main model object in PyTorch3D is `GenericModel`, which has pluggable components for the major steps, including the renderer and the implicit function(s).
|
||||
# There are two ways to construct it which are equivalent here.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
CONSTRUCT_MODEL_FROM_CONFIG = True
|
||||
if CONSTRUCT_MODEL_FROM_CONFIG:
|
||||
# Via a DictConfig - this is how our training loop with hydra works
|
||||
cfg = get_default_args(GenericModel)
|
||||
cfg.implicit_function_class_type = "MyVolumes"
|
||||
cfg.render_image_height=output_resolution
|
||||
cfg.render_image_width=output_resolution
|
||||
cfg.loss_weights={"loss_rgb_huber": 1.0}
|
||||
cfg.tqdm_trigger_threshold=19000
|
||||
cfg.raysampler_AdaptiveRaySampler_args.scene_extent= 4.0
|
||||
gm = GenericModel(**cfg)
|
||||
else:
|
||||
# constructing GenericModel directly
|
||||
expand_args_fields(GenericModel)
|
||||
gm = GenericModel(
|
||||
implicit_function_class_type="MyVolumes",
|
||||
render_image_height=output_resolution,
|
||||
render_image_width=output_resolution,
|
||||
loss_weights={"loss_rgb_huber": 1.0},
|
||||
tqdm_trigger_threshold=19000,
|
||||
raysampler_AdaptiveRaySampler_args = {"scene_extent": 4.0}
|
||||
)
|
||||
|
||||
# In this case we can get the equivalent DictConfig cfg object to the way gm is configured as follows
|
||||
cfg = OmegaConf.structured(gm)
|
||||
|
||||
|
||||
# The default renderer is an emission-absorbtion raymarcher. We keep that default.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# We can display the configuration in use as follows.
|
||||
remove_unused_components(cfg)
|
||||
yaml = OmegaConf.to_yaml(cfg, sort_keys=False)
|
||||
get_ipython().run_line_magic('page', '-r yaml')
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
device = torch.device("cuda:0")
|
||||
gm.to(device)
|
||||
assert next(gm.parameters()).is_cuda
|
||||
|
||||
|
||||
# ## 4. train the model
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
train_data_collated = [FrameData.collate([frame.to(device)]) for frame in dataset_map.train]
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
gm.train()
|
||||
optimizer = torch.optim.Adam(gm.parameters(), lr=0.1)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
iterator = tqdm.tqdm(range(2000))
|
||||
for n_batch in iterator:
|
||||
optimizer.zero_grad()
|
||||
|
||||
frame = train_data_collated[n_batch % len(dataset_map.train)]
|
||||
out = gm(**frame, evaluation_mode=EvaluationMode.TRAINING)
|
||||
out["objective"].backward()
|
||||
if n_batch % 100 == 0:
|
||||
iterator.set_postfix_str(f"loss: {float(out['objective']):.5f}")
|
||||
optimizer.step()
|
||||
|
||||
|
||||
# ## 5. Evaluate the module
|
||||
#
|
||||
# We generate complete images from all the viewpoints to see how they look.
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
def to_numpy_image(image):
|
||||
# Takes an image of shape (C, H, W) in [0,1], where C=3 or 1
|
||||
# to a numpy uint image of shape (H, W, 3)
|
||||
return (image * 255).to(torch.uint8).permute(1, 2, 0).detach().cpu().expand(-1, -1, 3).numpy()
|
||||
def resize_image(image):
|
||||
# Takes images of shape (B, C, H, W) to (B, C, output_resolution, output_resolution)
|
||||
return torch.nn.functional.interpolate(image, size=(output_resolution, output_resolution))
|
||||
|
||||
gm.eval()
|
||||
images = []
|
||||
expected = []
|
||||
masks = []
|
||||
masks_expected = []
|
||||
for frame in tqdm.tqdm(train_data_collated):
|
||||
with torch.no_grad():
|
||||
out = gm(**frame, evaluation_mode=EvaluationMode.EVALUATION)
|
||||
|
||||
image_rgb = to_numpy_image(out["images_render"][0])
|
||||
mask = to_numpy_image(out["masks_render"][0])
|
||||
expd = to_numpy_image(resize_image(frame.image_rgb)[0])
|
||||
mask_expected = to_numpy_image(resize_image(frame.fg_probability)[0])
|
||||
|
||||
images.append(image_rgb)
|
||||
masks.append(mask)
|
||||
expected.append(expd)
|
||||
masks_expected.append(mask_expected)
|
||||
|
||||
|
||||
# We draw a grid showing predicted image and expected image, followed by predicted mask and expected mask, from each viewpoint.
|
||||
# This is a grid of four rows of images, wrapped in to several large rows, i.e..
|
||||
# <small><center>
|
||||
# ```
|
||||
# ┌────────┬────────┐ ┌────────┐
|
||||
# │pred │pred │ │pred │
|
||||
# │image │image │ │image │
|
||||
# │1 │2 │ │n │
|
||||
# ├────────┼────────┤ ├────────┤
|
||||
# │expected│expected│ │expected│
|
||||
# │image │image │ ... │image │
|
||||
# │1 │2 │ │n │
|
||||
# ├────────┼────────┤ ├────────┤
|
||||
# │pred │pred │ │pred │
|
||||
# │mask │mask │ │mask │
|
||||
# │1 │2 │ │n │
|
||||
# ├────────┼────────┤ ├────────┤
|
||||
# │expected│expected│ │expected│
|
||||
# │mask │mask │ │mask │
|
||||
# │1 │2 │ │n │
|
||||
# ├────────┼────────┤ ├────────┤
|
||||
# │pred │pred │ │pred │
|
||||
# │image │image │ │image │
|
||||
# │n+1 │n+1 │ │2n │
|
||||
# ├────────┼────────┤ ├────────┤
|
||||
# │expected│expected│ │expected│
|
||||
# │image │image │ ... │image │
|
||||
# │n+1 │n+2 │ │2n │
|
||||
# ├────────┼────────┤ ├────────┤
|
||||
# │pred │pred │ │pred │
|
||||
# │mask │mask │ │mask │
|
||||
# │n+1 │n+2 │ │2n │
|
||||
# ├────────┼────────┤ ├────────┤
|
||||
# │expected│expected│ │expected│
|
||||
# │mask │mask │ │mask │
|
||||
# │n+1 │n+2 │ │2n │
|
||||
# └────────┴────────┘ └────────┘
|
||||
# ...
|
||||
# ```
|
||||
# </center></small>
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
images_to_display = [images.copy(), expected.copy(), masks.copy(), masks_expected.copy()]
|
||||
n_rows = 4
|
||||
n_images = len(images)
|
||||
blank_image = images[0] * 0
|
||||
n_per_row = 1+(n_images-1)//n_rows
|
||||
for _ in range(n_per_row*n_rows - n_images):
|
||||
for group in images_to_display:
|
||||
group.append(blank_image)
|
||||
|
||||
images_to_display_listed = [[[i] for i in j] for j in images_to_display]
|
||||
split = []
|
||||
for row in range(n_rows):
|
||||
for group in images_to_display_listed:
|
||||
split.append(group[row*n_per_row:(row+1)*n_per_row])
|
||||
|
||||
Image.fromarray(np.block(split))
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# Print the maximum channel intensity in the first image.
|
||||
print(images[1].max()/255)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
plt.ioff()
|
||||
fig, ax = plt.subplots(figsize=(3,3))
|
||||
|
||||
ax.grid(None)
|
||||
ims = [[ax.imshow(im, animated=True)] for im in images]
|
||||
ani = animation.ArtistAnimation(fig, ims, interval=80, blit=True)
|
||||
ani_html = ani.to_jshtml()
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
HTML(ani_html)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# If you want to see the output of the model with the volume forced to opaque white, run this and re-evaluate
|
||||
# with torch.no_grad():
|
||||
# gm._implicit_functions[0]._fn.density.fill_(9.0)
|
||||
# gm._implicit_functions[0]._fn.color.fill_(9.0)
|
||||
|
||||
@ -50,7 +50,7 @@
|
||||
"except ModuleNotFoundError:\n",
|
||||
" need_pytorch3d=True\n",
|
||||
"if need_pytorch3d:\n",
|
||||
" if torch.__version__.startswith(\"1.10.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" if torch.__version__.startswith(\"1.12.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" # We try to install PyTorch3D via a released wheel.\n",
|
||||
" pyt_version_str=torch.__version__.split(\"+\")[0].replace(\".\", \"\")\n",
|
||||
" version_str=\"\".join([\n",
|
||||
|
||||
@ -30,7 +30,7 @@ try:
|
||||
except ModuleNotFoundError:
|
||||
need_pytorch3d=True
|
||||
if need_pytorch3d:
|
||||
if torch.__version__.startswith("1.10.") and sys.platform.startswith("linux"):
|
||||
if torch.__version__.startswith("1.12.") and sys.platform.startswith("linux"):
|
||||
# We try to install PyTorch3D via a released wheel.
|
||||
pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
|
||||
version_str="".join([
|
||||
|
||||
@ -57,7 +57,7 @@
|
||||
"except ModuleNotFoundError:\n",
|
||||
" need_pytorch3d=True\n",
|
||||
"if need_pytorch3d:\n",
|
||||
" if torch.__version__.startswith(\"1.10.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" if torch.__version__.startswith(\"1.12.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" # We try to install PyTorch3D via a released wheel.\n",
|
||||
" pyt_version_str=torch.__version__.split(\"+\")[0].replace(\".\", \"\")\n",
|
||||
" version_str=\"\".join([\n",
|
||||
|
||||
@ -34,7 +34,7 @@ try:
|
||||
except ModuleNotFoundError:
|
||||
need_pytorch3d=True
|
||||
if need_pytorch3d:
|
||||
if torch.__version__.startswith("1.10.") and sys.platform.startswith("linux"):
|
||||
if torch.__version__.startswith("1.12.") and sys.platform.startswith("linux"):
|
||||
# We try to install PyTorch3D via a released wheel.
|
||||
pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
|
||||
version_str="".join([
|
||||
|
||||
@ -73,7 +73,7 @@
|
||||
"except ModuleNotFoundError:\n",
|
||||
" need_pytorch3d=True\n",
|
||||
"if need_pytorch3d:\n",
|
||||
" if torch.__version__.startswith(\"1.10.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" if torch.__version__.startswith(\"1.12.\") and sys.platform.startswith(\"linux\"):\n",
|
||||
" # We try to install PyTorch3D via a released wheel.\n",
|
||||
" pyt_version_str=torch.__version__.split(\"+\")[0].replace(\".\", \"\")\n",
|
||||
" version_str=\"\".join([\n",
|
||||
|
||||
@ -32,7 +32,7 @@ try:
|
||||
except ModuleNotFoundError:
|
||||
need_pytorch3d=True
|
||||
if need_pytorch3d:
|
||||
if torch.__version__.startswith("1.10.") and sys.platform.startswith("linux"):
|
||||
if torch.__version__.startswith("1.12.") and sys.platform.startswith("linux"):
|
||||
# We try to install PyTorch3D via a released wheel.
|
||||
pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
|
||||
version_str="".join([
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -133,7 +133,7 @@ where $d(g_i, g_j)$ is a suitable metric that compares the extrinsics of cameras
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -133,7 +133,7 @@ where $d(g_i, g_j)$ is a suitable metric that compares the extrinsics of cameras
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -127,7 +127,7 @@
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -127,7 +127,7 @@
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -126,7 +126,7 @@
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -126,7 +126,7 @@
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -146,7 +146,7 @@ the predicted mesh is closer to the target mesh at each optimization step. To ac
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -146,7 +146,7 @@ the predicted mesh is closer to the target mesh at each optimization step. To ac
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -127,7 +127,7 @@
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
@ -900,7 +900,7 @@ described in the previous cell.</p>
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="5.-Visualizing-the-optimized-neural-radiance-field">5. Visualizing the optimized neural radiance field<a class="anchor-link" href="#5.-Visualizing-the-optimized-neural-radiance-field">¶</a></h2><p>Finally, we visualize the neural radiance field by rendering from multiple viewpoints that rotate around the volume's y-axis.</p>
|
||||
<h2 id="6.-Visualizing-the-optimized-neural-radiance-field">6. Visualizing the optimized neural radiance field<a class="anchor-link" href="#6.-Visualizing-the-optimized-neural-radiance-field">¶</a></h2><p>Finally, we visualize the neural radiance field by rendering from multiple viewpoints that rotate around the volume's y-axis.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@ -951,7 +951,7 @@ described in the previous cell.</p>
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="6.-Conclusion">6. Conclusion<a class="anchor-link" href="#6.-Conclusion">¶</a></h2><p>In this tutorial, we have shown how to optimize an implicit representation of a scene such that the renders of the scene from known viewpoints match the observed images for each viewpoint. The rendering was carried out using the PyTorch3D's implicit function renderer composed of either a <code>MonteCarloRaysampler</code> or <code>NDCMultinomialRaysampler</code>, and an <code>EmissionAbsorptionRaymarcher</code>.</p>
|
||||
<h2 id="7.-Conclusion">7. Conclusion<a class="anchor-link" href="#7.-Conclusion">¶</a></h2><p>In this tutorial, we have shown how to optimize an implicit representation of a scene such that the renders of the scene from known viewpoints match the observed images for each viewpoint. The rendering was carried out using the PyTorch3D's implicit function renderer composed of either a <code>MonteCarloRaysampler</code> or <code>NDCMultinomialRaysampler</code>, and an <code>EmissionAbsorptionRaymarcher</code>.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -127,7 +127,7 @@
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
@ -900,7 +900,7 @@ described in the previous cell.</p>
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="5.-Visualizing-the-optimized-neural-radiance-field">5. Visualizing the optimized neural radiance field<a class="anchor-link" href="#5.-Visualizing-the-optimized-neural-radiance-field">¶</a></h2><p>Finally, we visualize the neural radiance field by rendering from multiple viewpoints that rotate around the volume's y-axis.</p>
|
||||
<h2 id="6.-Visualizing-the-optimized-neural-radiance-field">6. Visualizing the optimized neural radiance field<a class="anchor-link" href="#6.-Visualizing-the-optimized-neural-radiance-field">¶</a></h2><p>Finally, we visualize the neural radiance field by rendering from multiple viewpoints that rotate around the volume's y-axis.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@ -951,7 +951,7 @@ described in the previous cell.</p>
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="6.-Conclusion">6. Conclusion<a class="anchor-link" href="#6.-Conclusion">¶</a></h2><p>In this tutorial, we have shown how to optimize an implicit representation of a scene such that the renders of the scene from known viewpoints match the observed images for each viewpoint. The rendering was carried out using the PyTorch3D's implicit function renderer composed of either a <code>MonteCarloRaysampler</code> or <code>NDCMultinomialRaysampler</code>, and an <code>EmissionAbsorptionRaymarcher</code>.</p>
|
||||
<h2 id="7.-Conclusion">7. Conclusion<a class="anchor-link" href="#7.-Conclusion">¶</a></h2><p>In this tutorial, we have shown how to optimize an implicit representation of a scene such that the renders of the scene from known viewpoints match the observed images for each viewpoint. The rendering was carried out using the PyTorch3D's implicit function renderer composed of either a <code>MonteCarloRaysampler</code> or <code>NDCMultinomialRaysampler</code>, and an <code>EmissionAbsorptionRaymarcher</code>.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -125,7 +125,7 @@
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
@ -173,7 +173,7 @@
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.structures</span> <span class="kn">import</span> <span class="n">Meshes</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.renderer</span> <span class="kn">import</span> <span class="p">(</span>
|
||||
<span class="n">look_at_view_transform</span><span class="p">,</span>
|
||||
<span class="n">OpenGLPerspectiveCameras</span><span class="p">,</span>
|
||||
<span class="n">FoVPerspectiveCameras</span><span class="p">,</span>
|
||||
<span class="n">PointLights</span><span class="p">,</span>
|
||||
<span class="n">DirectionalLights</span><span class="p">,</span>
|
||||
<span class="n">Materials</span><span class="p">,</span>
|
||||
@ -331,11 +331,11 @@ If running locally, the data is already available at the correct path.</p>
|
||||
<span class="c1"># broadcasting. So we can view the camera from the a distance of dist=2.7, and </span>
|
||||
<span class="c1"># then specify elevation and azimuth angles for each viewpoint as tensors. </span>
|
||||
<span class="n">R</span><span class="p">,</span> <span class="n">T</span> <span class="o">=</span> <span class="n">look_at_view_transform</span><span class="p">(</span><span class="n">dist</span><span class="o">=</span><span class="mf">2.7</span><span class="p">,</span> <span class="n">elev</span><span class="o">=</span><span class="n">elev</span><span class="p">,</span> <span class="n">azim</span><span class="o">=</span><span class="n">azim</span><span class="p">)</span>
|
||||
<span class="n">cameras</span> <span class="o">=</span> <span class="n">OpenGLPerspectiveCameras</span><span class="p">(</span><span class="n">device</span><span class="o">=</span><span class="n">device</span><span class="p">,</span> <span class="n">R</span><span class="o">=</span><span class="n">R</span><span class="p">,</span> <span class="n">T</span><span class="o">=</span><span class="n">T</span><span class="p">)</span>
|
||||
<span class="n">cameras</span> <span class="o">=</span> <span class="n">FoVPerspectiveCameras</span><span class="p">(</span><span class="n">device</span><span class="o">=</span><span class="n">device</span><span class="p">,</span> <span class="n">R</span><span class="o">=</span><span class="n">R</span><span class="p">,</span> <span class="n">T</span><span class="o">=</span><span class="n">T</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># We arbitrarily choose one particular view that will be used to visualize </span>
|
||||
<span class="c1"># results</span>
|
||||
<span class="n">camera</span> <span class="o">=</span> <span class="n">OpenGLPerspectiveCameras</span><span class="p">(</span><span class="n">device</span><span class="o">=</span><span class="n">device</span><span class="p">,</span> <span class="n">R</span><span class="o">=</span><span class="n">R</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span>
|
||||
<span class="n">camera</span> <span class="o">=</span> <span class="n">FoVPerspectiveCameras</span><span class="p">(</span><span class="n">device</span><span class="o">=</span><span class="n">device</span><span class="p">,</span> <span class="n">R</span><span class="o">=</span><span class="n">R</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span>
|
||||
<span class="n">T</span><span class="o">=</span><span class="n">T</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="o">...</span><span class="p">])</span>
|
||||
|
||||
<span class="c1"># Define the settings for rasterization and shading. Here we set the output </span>
|
||||
@ -378,7 +378,7 @@ If running locally, the data is already available at the correct path.</p>
|
||||
<span class="c1"># Our multi-view cow dataset will be represented by these 2 lists of tensors,</span>
|
||||
<span class="c1"># each of length num_views.</span>
|
||||
<span class="n">target_rgb</span> <span class="o">=</span> <span class="p">[</span><span class="n">target_images</span><span class="p">[</span><span class="n">i</span><span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="p">:</span><span class="mi">3</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">num_views</span><span class="p">)]</span>
|
||||
<span class="n">target_cameras</span> <span class="o">=</span> <span class="p">[</span><span class="n">OpenGLPerspectiveCameras</span><span class="p">(</span><span class="n">device</span><span class="o">=</span><span class="n">device</span><span class="p">,</span> <span class="n">R</span><span class="o">=</span><span class="n">R</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="n">i</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span>
|
||||
<span class="n">target_cameras</span> <span class="o">=</span> <span class="p">[</span><span class="n">FoVPerspectiveCameras</span><span class="p">(</span><span class="n">device</span><span class="o">=</span><span class="n">device</span><span class="p">,</span> <span class="n">R</span><span class="o">=</span><span class="n">R</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="n">i</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span>
|
||||
<span class="n">T</span><span class="o">=</span><span class="n">T</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="n">i</span><span class="p">,</span> <span class="o">...</span><span class="p">])</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">num_views</span><span class="p">)]</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
@ -697,6 +697,7 @@ If running locally, the data is already available at the correct path.</p>
|
||||
<span class="n">image_size</span><span class="o">=</span><span class="mi">128</span><span class="p">,</span>
|
||||
<span class="n">blur_radius</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="mf">1.</span> <span class="o">/</span> <span class="mf">1e-4</span> <span class="o">-</span> <span class="mf">1.</span><span class="p">)</span><span class="o">*</span><span class="n">sigma</span><span class="p">,</span>
|
||||
<span class="n">faces_per_pixel</span><span class="o">=</span><span class="mi">50</span><span class="p">,</span>
|
||||
<span class="n">perspective_correct</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
<span class="p">)</span>
|
||||
|
||||
<span class="c1"># Differentiable soft renderer using per vertex RGB colors for texture</span>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -125,7 +125,7 @@
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
@ -173,7 +173,7 @@
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.structures</span> <span class="kn">import</span> <span class="n">Meshes</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.renderer</span> <span class="kn">import</span> <span class="p">(</span>
|
||||
<span class="n">look_at_view_transform</span><span class="p">,</span>
|
||||
<span class="n">OpenGLPerspectiveCameras</span><span class="p">,</span>
|
||||
<span class="n">FoVPerspectiveCameras</span><span class="p">,</span>
|
||||
<span class="n">PointLights</span><span class="p">,</span>
|
||||
<span class="n">DirectionalLights</span><span class="p">,</span>
|
||||
<span class="n">Materials</span><span class="p">,</span>
|
||||
@ -331,11 +331,11 @@ If running locally, the data is already available at the correct path.</p>
|
||||
<span class="c1"># broadcasting. So we can view the camera from the a distance of dist=2.7, and </span>
|
||||
<span class="c1"># then specify elevation and azimuth angles for each viewpoint as tensors. </span>
|
||||
<span class="n">R</span><span class="p">,</span> <span class="n">T</span> <span class="o">=</span> <span class="n">look_at_view_transform</span><span class="p">(</span><span class="n">dist</span><span class="o">=</span><span class="mf">2.7</span><span class="p">,</span> <span class="n">elev</span><span class="o">=</span><span class="n">elev</span><span class="p">,</span> <span class="n">azim</span><span class="o">=</span><span class="n">azim</span><span class="p">)</span>
|
||||
<span class="n">cameras</span> <span class="o">=</span> <span class="n">OpenGLPerspectiveCameras</span><span class="p">(</span><span class="n">device</span><span class="o">=</span><span class="n">device</span><span class="p">,</span> <span class="n">R</span><span class="o">=</span><span class="n">R</span><span class="p">,</span> <span class="n">T</span><span class="o">=</span><span class="n">T</span><span class="p">)</span>
|
||||
<span class="n">cameras</span> <span class="o">=</span> <span class="n">FoVPerspectiveCameras</span><span class="p">(</span><span class="n">device</span><span class="o">=</span><span class="n">device</span><span class="p">,</span> <span class="n">R</span><span class="o">=</span><span class="n">R</span><span class="p">,</span> <span class="n">T</span><span class="o">=</span><span class="n">T</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># We arbitrarily choose one particular view that will be used to visualize </span>
|
||||
<span class="c1"># results</span>
|
||||
<span class="n">camera</span> <span class="o">=</span> <span class="n">OpenGLPerspectiveCameras</span><span class="p">(</span><span class="n">device</span><span class="o">=</span><span class="n">device</span><span class="p">,</span> <span class="n">R</span><span class="o">=</span><span class="n">R</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span>
|
||||
<span class="n">camera</span> <span class="o">=</span> <span class="n">FoVPerspectiveCameras</span><span class="p">(</span><span class="n">device</span><span class="o">=</span><span class="n">device</span><span class="p">,</span> <span class="n">R</span><span class="o">=</span><span class="n">R</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span>
|
||||
<span class="n">T</span><span class="o">=</span><span class="n">T</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="o">...</span><span class="p">])</span>
|
||||
|
||||
<span class="c1"># Define the settings for rasterization and shading. Here we set the output </span>
|
||||
@ -378,7 +378,7 @@ If running locally, the data is already available at the correct path.</p>
|
||||
<span class="c1"># Our multi-view cow dataset will be represented by these 2 lists of tensors,</span>
|
||||
<span class="c1"># each of length num_views.</span>
|
||||
<span class="n">target_rgb</span> <span class="o">=</span> <span class="p">[</span><span class="n">target_images</span><span class="p">[</span><span class="n">i</span><span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="p">:</span><span class="mi">3</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">num_views</span><span class="p">)]</span>
|
||||
<span class="n">target_cameras</span> <span class="o">=</span> <span class="p">[</span><span class="n">OpenGLPerspectiveCameras</span><span class="p">(</span><span class="n">device</span><span class="o">=</span><span class="n">device</span><span class="p">,</span> <span class="n">R</span><span class="o">=</span><span class="n">R</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="n">i</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span>
|
||||
<span class="n">target_cameras</span> <span class="o">=</span> <span class="p">[</span><span class="n">FoVPerspectiveCameras</span><span class="p">(</span><span class="n">device</span><span class="o">=</span><span class="n">device</span><span class="p">,</span> <span class="n">R</span><span class="o">=</span><span class="n">R</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="n">i</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span>
|
||||
<span class="n">T</span><span class="o">=</span><span class="n">T</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="n">i</span><span class="p">,</span> <span class="o">...</span><span class="p">])</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">num_views</span><span class="p">)]</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
@ -697,6 +697,7 @@ If running locally, the data is already available at the correct path.</p>
|
||||
<span class="n">image_size</span><span class="o">=</span><span class="mi">128</span><span class="p">,</span>
|
||||
<span class="n">blur_radius</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="mf">1.</span> <span class="o">/</span> <span class="mf">1e-4</span> <span class="o">-</span> <span class="mf">1.</span><span class="p">)</span><span class="o">*</span><span class="n">sigma</span><span class="p">,</span>
|
||||
<span class="n">faces_per_pixel</span><span class="o">=</span><span class="mi">50</span><span class="p">,</span>
|
||||
<span class="n">perspective_correct</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
<span class="p">)</span>
|
||||
|
||||
<span class="c1"># Differentiable soft renderer using per vertex RGB colors for texture</span>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -118,7 +118,7 @@
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -118,7 +118,7 @@
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
|
||||
884
tutorials/implicitron_config_system.html
Normal file
884
tutorials/implicitron_config_system.html
Normal file
@ -0,0 +1,884 @@
|
||||
<!DOCTYPE html><html lang=""><head><meta charSet="utf-8"/><meta http-equiv="X-UA-Compatible" content="IE=edge"/><title>PyTorch3D · A library for deep learning with 3D data</title><meta name="viewport" content="width=device-width, initial-scale=1.0"/><meta name="generator" content="Docusaurus"/><meta name="description" content="A library for deep learning with 3D data"/><meta property="og:title" content="PyTorch3D · A library for deep learning with 3D data"/><meta property="og:type" content="website"/><meta property="og:url" content="https://pytorch3d.org/"/><meta property="og:description" content="A library for deep learning with 3D data"/><meta property="og:image" content="https://pytorch3d.org/img/pytorch3dlogoicon.svg"/><meta name="twitter:card" content="summary"/><meta name="twitter:image" content="https://pytorch3d.org/img/pytorch3dlogoicon.svg"/><link rel="shortcut icon" href="/img/pytorch3dfavicon.png"/><link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/default.min.css"/><script>
|
||||
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
|
||||
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
|
||||
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
|
||||
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
var links = coll[i].nextElementSibling.getElementsByTagName('*');
|
||||
if (checkActiveCategory){
|
||||
for (var j = 0; j < links.length; j++) {
|
||||
if (links[j].classList.contains('navListItemActive')){
|
||||
coll[i].nextElementSibling.classList.toggle('hide');
|
||||
coll[i].childNodes[1].classList.toggle('rotate');
|
||||
checkActiveCategory = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
coll[i].addEventListener('click', function() {
|
||||
var arrow = this.childNodes[1];
|
||||
arrow.classList.toggle('rotate');
|
||||
var content = this.nextElementSibling;
|
||||
content.classList.toggle('hide');
|
||||
});
|
||||
}
|
||||
|
||||
document.addEventListener('DOMContentLoaded', function() {
|
||||
createToggler('#navToggler', '#docsNav', 'docsSliderActive');
|
||||
createToggler('#tocToggler', 'body', 'tocActive');
|
||||
|
||||
var headings = document.querySelector('.toc-headings');
|
||||
headings && headings.addEventListener('click', function(event) {
|
||||
var el = event.target;
|
||||
while(el !== headings){
|
||||
if (el.tagName === 'A') {
|
||||
document.body.classList.remove('tocActive');
|
||||
break;
|
||||
} else{
|
||||
el = el.parentNode;
|
||||
}
|
||||
}
|
||||
}, false);
|
||||
|
||||
function createToggler(togglerSelector, targetSelector, className) {
|
||||
var toggler = document.querySelector(togglerSelector);
|
||||
var target = document.querySelector(targetSelector);
|
||||
|
||||
if (!toggler) {
|
||||
return;
|
||||
}
|
||||
|
||||
toggler.onclick = function(event) {
|
||||
event.preventDefault();
|
||||
|
||||
target.classList.toggle(className);
|
||||
};
|
||||
}
|
||||
});
|
||||
</script></nav></div><div class="container mainContainer"><div class="wrapper"><div class="tutorialButtonsWrapper"><div class="tutorialButtonWrapper buttonWrapper"><a class="tutorialButton button" download="" href="https://colab.research.google.com/github/facebookresearch/pytorch3d/blob/stable/docs/tutorials/implicitron_config_system.ipynb" target="_blank"><img class="colabButton" align="left" src="/img/colab_icon.png"/>Run in Google Colab</a></div><div class="tutorialButtonWrapper buttonWrapper"><a class="tutorialButton button" download="" href="/files/implicitron_config_system.ipynb" target="_blank"><svg aria-hidden="true" focusable="false" data-prefix="fas" data-icon="file-download" class="svg-inline--fa fa-file-download fa-w-12" role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 384 512"><path fill="currentColor" d="M224 136V0H24C10.7 0 0 10.7 0 24v464c0 13.3 10.7 24 24 24h336c13.3 0 24-10.7 24-24V160H248c-13.2 0-24-10.8-24-24zm76.45 211.36l-96.42 95.7c-6.65 6.61-17.39 6.61-24.04 0l-96.42-95.7C73.42 337.29 80.54 320 94.82 320H160v-80c0-8.84 7.16-16 16-16h32c8.84 0 16 7.16 16 16v80h65.18c14.28 0 21.4 17.29 11.27 27.36zM377 105L279.1 7c-4.5-4.5-10.6-7-17-7H256v128h128v-6.1c0-6.3-2.5-12.4-7-16.9z"></path></svg>Download Tutorial Jupyter Notebook</a></div><div class="tutorialButtonWrapper buttonWrapper"><a class="tutorialButton button" download="" href="/files/implicitron_config_system.py" target="_blank"><svg aria-hidden="true" focusable="false" data-prefix="fas" data-icon="file-download" class="svg-inline--fa fa-file-download fa-w-12" role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 384 512"><path fill="currentColor" d="M224 136V0H24C10.7 0 0 10.7 0 24v464c0 13.3 10.7 24 24 24h336c13.3 0 24-10.7 24-24V160H248c-13.2 0-24-10.8-24-24zm76.45 211.36l-96.42 95.7c-6.65 6.61-17.39 6.61-24.04 0l-96.42-95.7C73.42 337.29 80.54 320 94.82 320H160v-80c0-8.84 7.16-16 16-16h32c8.84 0 16 7.16 16 16v80h65.18c14.28 0 21.4 17.29 11.27 27.36zM377 105L279.1 7c-4.5-4.5-10.6-7-17-7H256v128h128v-6.1c0-6.3-2.5-12.4-7-16.9z"></path></svg>Download Tutorial Source Code</a></div></div><div class="tutorialBody">
|
||||
<script
|
||||
src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.1.10/require.min.js">
|
||||
</script>
|
||||
<script
|
||||
src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.min.js">
|
||||
</script>
|
||||
<div class="notebook">
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># Copyright (c) Meta Platforms, Inc. and affiliates. All rights reserved.</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h1 id="Implicitron's-config-system">Implicitron's config system<a class="anchor-link" href="#Implicitron's-config-system">¶</a></h1>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>Implicitron's components are all based on a unified hierarchical configuration system.
|
||||
This allows configurable variables and all defaults to be defined separately for each new component.
|
||||
All configs relevant to an experiment are then automatically composed into a single configuration file that fully specifies the experiment.
|
||||
An especially important feature is extension points where users can insert their own sub-classes of Implicitron's base components.</p>
|
||||
<p>The file which defines this system is <a href="https://github.com/facebookresearch/pytorch3d/blob/main/pytorch3d/implicitron/tools/config.py">here</a> in the PyTorch3D repo.
|
||||
The Implicitron volumes tutorial contains a simple example of using the config system.
|
||||
This tutorial provides detailed hands-on experience in using and modifying Implicitron's configurable components.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="0.-Install-and-import-modules">0. Install and import modules<a class="anchor-link" href="#0.-Install-and-import-modules">¶</a></h2><p>Ensure <code>torch</code> and <code>torchvision</code> are installed. If <code>pytorch3d</code> is not installed, install it using the following cell:</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">os</span>
|
||||
<span class="kn">import</span> <span class="nn">sys</span>
|
||||
<span class="kn">import</span> <span class="nn">torch</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">False</span>
|
||||
<span class="k">try</span><span class="p">:</span>
|
||||
<span class="kn">import</span> <span class="nn">pytorch3d</span>
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
<span class="sa">f</span><span class="s2">"py3</span><span class="si">{</span><span class="n">sys</span><span class="o">.</span><span class="n">version_info</span><span class="o">.</span><span class="n">minor</span><span class="si">}</span><span class="s2">_cu"</span><span class="p">,</span>
|
||||
<span class="n">torch</span><span class="o">.</span><span class="n">version</span><span class="o">.</span><span class="n">cuda</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span><span class="s2">""</span><span class="p">),</span>
|
||||
<span class="sa">f</span><span class="s2">"_pyt</span><span class="si">{</span><span class="n">pyt_version_str</span><span class="si">}</span><span class="s2">"</span>
|
||||
<span class="p">])</span>
|
||||
<span class="o">!</span>pip install fvcore iopath
|
||||
<span class="o">!</span>pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/<span class="o">{</span>version_str<span class="o">}</span>/download.html
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="c1"># We try to install PyTorch3D from source.</span>
|
||||
<span class="o">!</span>curl -LO https://github.com/NVIDIA/cub/archive/1.10.0.tar.gz
|
||||
<span class="o">!</span>tar xzf <span class="m">1</span>.10.0.tar.gz
|
||||
<span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">"CUB_HOME"</span><span class="p">]</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">getcwd</span><span class="p">()</span> <span class="o">+</span> <span class="s2">"/cub-1.10.0"</span>
|
||||
<span class="o">!</span>pip install <span class="s1">'git+https://github.com/facebookresearch/pytorch3d.git@stable'</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>Ensure omegaconf is installed. If not, run this cell. (It should not be necessary to restart the runtime.)</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="o">!</span>pip install omegaconf
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">dataclasses</span> <span class="kn">import</span> <span class="n">dataclass</span>
|
||||
<span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Optional</span><span class="p">,</span> <span class="n">Tuple</span>
|
||||
|
||||
<span class="kn">import</span> <span class="nn">torch</span>
|
||||
<span class="kn">from</span> <span class="nn">omegaconf</span> <span class="kn">import</span> <span class="n">DictConfig</span><span class="p">,</span> <span class="n">OmegaConf</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.implicitron.tools.config</span> <span class="kn">import</span> <span class="p">(</span>
|
||||
<span class="n">Configurable</span><span class="p">,</span>
|
||||
<span class="n">ReplaceableBase</span><span class="p">,</span>
|
||||
<span class="n">expand_args_fields</span><span class="p">,</span>
|
||||
<span class="n">get_default_args</span><span class="p">,</span>
|
||||
<span class="n">registry</span><span class="p">,</span>
|
||||
<span class="n">run_auto_creation</span><span class="p">,</span>
|
||||
<span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="1.-Introducing-dataclasses">1. Introducing dataclasses<a class="anchor-link" href="#1.-Introducing-dataclasses">¶</a></h2><p><a href="https://docs.python.org/3/library/typing.html">Type hints</a> give a taxonomy of types in Python. <a href="https://docs.python.org/3/library/dataclasses.html">Dataclasses</a> let you create a class based on a list of members which have names, types and possibly default values. The <code>__init__</code> function is created automatically, and calls a <code>__post_init__</code> function if present as a final step. For example</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="nd">@dataclass</span>
|
||||
<span class="k">class</span> <span class="nc">MyDataclass</span><span class="p">:</span>
|
||||
<span class="n">a</span><span class="p">:</span> <span class="nb">int</span>
|
||||
<span class="n">b</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">8</span>
|
||||
<span class="n">c</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">Tuple</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o">...</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"created with a = </span><span class="si">{</span><span class="bp">self</span><span class="o">.</span><span class="n">a</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">d</span> <span class="o">=</span> <span class="mi">2</span> <span class="o">*</span> <span class="bp">self</span><span class="o">.</span><span class="n">b</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">my_dataclass_instance</span> <span class="o">=</span> <span class="n">MyDataclass</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="mi">18</span><span class="p">)</span>
|
||||
<span class="k">assert</span> <span class="n">my_dataclass_instance</span><span class="o">.</span><span class="n">d</span> <span class="o">==</span> <span class="mi">16</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>👷 Note that the <code>dataclass</code> decorator here is function which modifies the definition of the class itself.
|
||||
It runs immediately after the definition.
|
||||
Our config system requires that implicitron library code contains classes whose modified versions need to be aware of user-defined implementations.
|
||||
Therefore we need the modification of the class to be delayed. We don't use a decorator.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="2.-Introducing-omegaconf-and-OmegaConf.structured">2. Introducing omegaconf and OmegaConf.structured<a class="anchor-link" href="#2.-Introducing-omegaconf-and-OmegaConf.structured">¶</a></h2><p>The <a href="https://github.com/omry/omegaconf/">omegaconf</a> library provides a DictConfig class which is like a <code>dict</code> with str keys, but with extra features for ease-of-use as a configuration system.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">dc</span> <span class="o">=</span> <span class="n">DictConfig</span><span class="p">({</span><span class="s2">"a"</span><span class="p">:</span> <span class="mi">2</span><span class="p">,</span> <span class="s2">"b"</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span> <span class="s2">"c"</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span> <span class="s2">"d"</span><span class="p">:</span> <span class="s2">"hello"</span><span class="p">})</span>
|
||||
<span class="k">assert</span> <span class="n">dc</span><span class="o">.</span><span class="n">a</span> <span class="o">==</span> <span class="n">dc</span><span class="p">[</span><span class="s2">"a"</span><span class="p">]</span> <span class="o">==</span> <span class="mi">2</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>OmegaConf has a serialization to and from yaml. The <a href="https://hydra.cc/">Hydra</a> library relies on this for its configuration files.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="nb">print</span><span class="p">(</span><span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">dc</span><span class="p">))</span>
|
||||
<span class="k">assert</span> <span class="n">OmegaConf</span><span class="o">.</span><span class="n">create</span><span class="p">(</span><span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">dc</span><span class="p">))</span> <span class="o">==</span> <span class="n">dc</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>OmegaConf.structured provides a DictConfig from a dataclass or instance of a dataclass. Unlike a normal DictConfig, it is type-checked and only known keys can be added.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">structured</span> <span class="o">=</span> <span class="n">OmegaConf</span><span class="o">.</span><span class="n">structured</span><span class="p">(</span><span class="n">MyDataclass</span><span class="p">)</span>
|
||||
<span class="k">assert</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">structured</span><span class="p">,</span> <span class="n">DictConfig</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">structured</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">()</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">structured</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p><code>structured</code> knows it is missing a value for <code>a</code>.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>Such an object has members compatible with the dataclass, so an initialisation can be performed as follows.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">structured</span><span class="o">.</span><span class="n">a</span> <span class="o">=</span> <span class="mi">21</span>
|
||||
<span class="n">my_dataclass_instance2</span> <span class="o">=</span> <span class="n">MyDataclass</span><span class="p">(</span><span class="o">**</span><span class="n">structured</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">my_dataclass_instance2</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>You can also call OmegaConf.structured on an instance.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">structured_from_instance</span> <span class="o">=</span> <span class="n">OmegaConf</span><span class="o">.</span><span class="n">structured</span><span class="p">(</span><span class="n">my_dataclass_instance</span><span class="p">)</span>
|
||||
<span class="n">my_dataclass_instance3</span> <span class="o">=</span> <span class="n">MyDataclass</span><span class="p">(</span><span class="o">**</span><span class="n">structured_from_instance</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">my_dataclass_instance3</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="3.-Our-approach-to-OmegaConf.structured">3. Our approach to OmegaConf.structured<a class="anchor-link" href="#3.-Our-approach-to-OmegaConf.structured">¶</a></h2><p>We provide functions which are equivalent to <code>OmegaConf.structured</code> but support more features.
|
||||
To achieve the above using our functions, the following is used.
|
||||
Note that we indicate configurable classes using a special base class <code>Configurable</code>, not a decorator.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">MyConfigurable</span><span class="p">(</span><span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">a</span><span class="p">:</span> <span class="nb">int</span>
|
||||
<span class="n">b</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">8</span>
|
||||
<span class="n">c</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">Tuple</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o">...</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"created with a = </span><span class="si">{</span><span class="bp">self</span><span class="o">.</span><span class="n">a</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">d</span> <span class="o">=</span> <span class="mi">2</span> <span class="o">*</span> <span class="bp">self</span><span class="o">.</span><span class="n">b</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># expand_args_fields must be called on an object before it is instantiated.</span>
|
||||
<span class="c1"># A warning is raised if this is missed, but it is possible to not notice the warning.</span>
|
||||
<span class="c1"># It modifies the class like @dataclass</span>
|
||||
<span class="n">expand_args_fields</span><span class="p">(</span><span class="n">MyConfigurable</span><span class="p">)</span>
|
||||
<span class="n">my_configurable_instance</span> <span class="o">=</span> <span class="n">MyConfigurable</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="mi">18</span><span class="p">)</span>
|
||||
<span class="k">assert</span> <span class="n">my_configurable_instance</span><span class="o">.</span><span class="n">d</span> <span class="o">==</span> <span class="mi">16</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># get_default_args calls expand_args_fields automatically</span>
|
||||
<span class="n">our_structured</span> <span class="o">=</span> <span class="n">get_default_args</span><span class="p">(</span><span class="n">MyConfigurable</span><span class="p">)</span>
|
||||
<span class="k">assert</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">our_structured</span><span class="p">,</span> <span class="n">DictConfig</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">our_structured</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">our_structured</span><span class="o">.</span><span class="n">a</span> <span class="o">=</span> <span class="mi">21</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">MyConfigurable</span><span class="p">(</span><span class="o">**</span><span class="n">our_structured</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="4.-First-enhancement:-nested-types-🪺">4. First enhancement: nested types 🪺<a class="anchor-link" href="#4.-First-enhancement:-nested-types-🪺">¶</a></h2><p>Our system allows Configurable classes to contain each other.
|
||||
One thing to remember: add a call to <code>run_auto_creation</code> in <code>__post_init__</code>.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">Inner</span><span class="p">(</span><span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">a</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">8</span>
|
||||
<span class="n">b</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
<span class="n">c</span><span class="p">:</span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o">...</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
|
||||
|
||||
|
||||
<span class="k">class</span> <span class="nc">Outer</span><span class="p">(</span><span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">inner</span><span class="p">:</span> <span class="n">Inner</span>
|
||||
<span class="n">x</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">"hello"</span>
|
||||
<span class="n">xx</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="n">run_auto_creation</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">outer_dc</span> <span class="o">=</span> <span class="n">get_default_args</span><span class="p">(</span><span class="n">Outer</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">outer_dc</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">outer</span> <span class="o">=</span> <span class="n">Outer</span><span class="p">(</span><span class="o">**</span><span class="n">outer_dc</span><span class="p">)</span>
|
||||
<span class="k">assert</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">outer</span><span class="p">,</span> <span class="n">Outer</span><span class="p">)</span>
|
||||
<span class="k">assert</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">outer</span><span class="o">.</span><span class="n">inner</span><span class="p">,</span> <span class="n">Inner</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="nb">vars</span><span class="p">(</span><span class="n">outer</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">outer</span><span class="o">.</span><span class="n">inner</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>Note how inner_args is an extra member of outer. <code>run_auto_creation(self)</code> is equivalent to</p>
|
||||
<pre><code> self.inner = Inner(**self.inner_args)</code></pre>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="5.-Second-enhancement:-pluggable/replaceable-components-🔌">5. Second enhancement: pluggable/replaceable components 🔌<a class="anchor-link" href="#5.-Second-enhancement:-pluggable/replaceable-components-🔌">¶</a></h2><p>If a class uses <code>ReplaceableBase</code> as a base class instead of <code>Configurable</code>, we call it a replaceable.
|
||||
It indicates that it is designed for child classes to use in its place.
|
||||
We might use <code>NotImplementedError</code> to indicate functionality which subclasses are expected to implement.
|
||||
The system maintains a global <code>registry</code> containing subclasses of each ReplaceableBase.
|
||||
The subclasses register themselves with it with a decorator.</p>
|
||||
<p>A configurable class (i.e. a class which uses our system, i.e. a child of <code>Configurable</code> or <code>ReplaceableBase</code>) which contains a ReplaceableBase must also
|
||||
contain a corresponding class_type field of type <code>str</code> which indicates which concrete child class to use.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">InnerBase</span><span class="p">(</span><span class="n">ReplaceableBase</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">say_something</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="k">raise</span> <span class="ne">NotImplementedError</span>
|
||||
|
||||
|
||||
<span class="nd">@registry</span><span class="o">.</span><span class="n">register</span>
|
||||
<span class="k">class</span> <span class="nc">Inner1</span><span class="p">(</span><span class="n">InnerBase</span><span class="p">):</span>
|
||||
<span class="n">a</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">1</span>
|
||||
<span class="n">b</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">"h"</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">say_something</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"hello from an Inner1"</span><span class="p">)</span>
|
||||
|
||||
|
||||
<span class="nd">@registry</span><span class="o">.</span><span class="n">register</span>
|
||||
<span class="k">class</span> <span class="nc">Inner2</span><span class="p">(</span><span class="n">InnerBase</span><span class="p">):</span>
|
||||
<span class="n">a</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">2</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">say_something</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"hello from an Inner2"</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">Out</span><span class="p">(</span><span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">inner</span><span class="p">:</span> <span class="n">InnerBase</span>
|
||||
<span class="n">inner_class_type</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">"Inner1"</span>
|
||||
<span class="n">x</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">19</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="n">run_auto_creation</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">talk</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">inner</span><span class="o">.</span><span class="n">say_something</span><span class="p">()</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">Out_dc</span> <span class="o">=</span> <span class="n">get_default_args</span><span class="p">(</span><span class="n">Out</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">Out_dc</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">Out_dc</span><span class="o">.</span><span class="n">inner_class_type</span> <span class="o">=</span> <span class="s2">"Inner2"</span>
|
||||
<span class="n">out</span> <span class="o">=</span> <span class="n">Out</span><span class="p">(</span><span class="o">**</span><span class="n">Out_dc</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">out</span><span class="o">.</span><span class="n">inner</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">out</span><span class="o">.</span><span class="n">talk</span><span class="p">()</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>Note in this case there are many <code>args</code> members. It is usually fine to ignore them in the code. They are needed for the config.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="nb">print</span><span class="p">(</span><span class="nb">vars</span><span class="p">(</span><span class="n">out</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="6.-Example-with-torch.nn.Module--🔥">6. Example with torch.nn.Module 🔥<a class="anchor-link" href="#6.-Example-with-torch.nn.Module--🔥">¶</a></h2><p>Typically in implicitron, we use this system in combination with <a href="https://pytorch.org/docs/stable/generated/torch.nn.Module.html"><code>Module</code></a>s.
|
||||
Note in this case it is necessary to call <code>Module.__init__</code> explicitly in <code>__post_init__</code>.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">MyLinear</span><span class="p">(</span><span class="n">torch</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">Module</span><span class="p">,</span> <span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">d_in</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">2</span>
|
||||
<span class="n">d_out</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">200</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">()</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">linear</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">Linear</span><span class="p">(</span><span class="n">in_features</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">d_in</span><span class="p">,</span> <span class="n">out_features</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">d_out</span><span class="p">)</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span>
|
||||
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">linear</span><span class="o">.</span><span class="n">forward</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">expand_args_fields</span><span class="p">(</span><span class="n">MyLinear</span><span class="p">)</span>
|
||||
<span class="n">my_linear</span> <span class="o">=</span> <span class="n">MyLinear</span><span class="p">()</span>
|
||||
<span class="nb">input</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
|
||||
<span class="n">output</span> <span class="o">=</span> <span class="n">my_linear</span><span class="p">(</span><span class="nb">input</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"output shape:"</span><span class="p">,</span> <span class="n">output</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p><code>my_linear</code> has all the usual features of a Module.
|
||||
E.g. it can be saved and loaded with <code>torch.save</code> and <code>torch.load</code>.
|
||||
It has parameters:</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">my_linear</span><span class="o">.</span><span class="n">named_parameters</span><span class="p">():</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">value</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="7.-Example-of-implementing-your-own-pluggable-component">7. Example of implementing your own pluggable component<a class="anchor-link" href="#7.-Example-of-implementing-your-own-pluggable-component">¶</a></h2><p>Let's say I am using a library with <code>Out</code> like in section <strong>5</strong> but I want to implement my own child of InnerBase.
|
||||
All I need to do is register its definition, but I need to do this before expand_args_fields is explicitly or implicitly called on Out.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="nd">@registry</span><span class="o">.</span><span class="n">register</span>
|
||||
<span class="k">class</span> <span class="nc">UserImplementedInner</span><span class="p">(</span><span class="n">InnerBase</span><span class="p">):</span>
|
||||
<span class="n">a</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">200</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">say_something</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"hello from the user"</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>At this point, we need to redefine the class Out.
|
||||
Otherwise if it has already been expanded without UserImplementedInner, then the following would not work,
|
||||
because the implementations known to a class are fixed when it is expanded.</p>
|
||||
<p>If you are running experiments from a script, the thing to remember here is that you must import your own modules, which register your own implementations,
|
||||
before you <em>use</em> the library classes.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">Out</span><span class="p">(</span><span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">inner</span><span class="p">:</span> <span class="n">InnerBase</span>
|
||||
<span class="n">inner_class_type</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">"Inner1"</span>
|
||||
<span class="n">x</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">19</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="n">run_auto_creation</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">talk</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">inner</span><span class="o">.</span><span class="n">say_something</span><span class="p">()</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">expand_args_fields</span><span class="p">(</span><span class="n">Out</span><span class="p">)</span>
|
||||
<span class="n">out2</span> <span class="o">=</span> <span class="n">Out</span><span class="p">(</span><span class="n">inner_class_type</span><span class="o">=</span><span class="s2">"UserImplementedInner"</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">out2</span><span class="o">.</span><span class="n">inner</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="8:-Example-of-making-a-subcomponent-pluggable">8: Example of making a subcomponent pluggable<a class="anchor-link" href="#8:-Example-of-making-a-subcomponent-pluggable">¶</a></h2><p>Let's look what needs to happen if we have a subcomponent which we make pluggable, to allow users to supply their own.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">SubComponent</span><span class="p">(</span><span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">x</span><span class="p">:</span> <span class="nb">float</span> <span class="o">=</span> <span class="mf">0.25</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">apply</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">a</span><span class="p">:</span> <span class="nb">float</span><span class="p">)</span> <span class="o">-></span> <span class="nb">float</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="n">a</span> <span class="o">+</span> <span class="bp">self</span><span class="o">.</span><span class="n">x</span>
|
||||
|
||||
|
||||
<span class="k">class</span> <span class="nc">LargeComponent</span><span class="p">(</span><span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">repeats</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">4</span>
|
||||
<span class="n">subcomponent</span><span class="p">:</span> <span class="n">SubComponent</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="n">run_auto_creation</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">apply</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">a</span><span class="p">:</span> <span class="nb">float</span><span class="p">)</span> <span class="o">-></span> <span class="nb">float</span><span class="p">:</span>
|
||||
<span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">repeats</span><span class="p">):</span>
|
||||
<span class="n">a</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">subcomponent</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">a</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">expand_args_fields</span><span class="p">(</span><span class="n">LargeComponent</span><span class="p">)</span>
|
||||
<span class="n">large_component</span> <span class="o">=</span> <span class="n">LargeComponent</span><span class="p">()</span>
|
||||
<span class="k">assert</span> <span class="n">large_component</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span> <span class="o">==</span> <span class="mi">4</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">LargeComponent</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>Made generic:</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">SubComponentBase</span><span class="p">(</span><span class="n">ReplaceableBase</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">apply</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">a</span><span class="p">:</span> <span class="nb">float</span><span class="p">)</span> <span class="o">-></span> <span class="nb">float</span><span class="p">:</span>
|
||||
<span class="k">raise</span> <span class="ne">NotImplementedError</span>
|
||||
|
||||
|
||||
<span class="nd">@registry</span><span class="o">.</span><span class="n">register</span>
|
||||
<span class="k">class</span> <span class="nc">SubComponent</span><span class="p">(</span><span class="n">SubComponentBase</span><span class="p">):</span>
|
||||
<span class="n">x</span><span class="p">:</span> <span class="nb">float</span> <span class="o">=</span> <span class="mf">0.25</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">apply</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">a</span><span class="p">:</span> <span class="nb">float</span><span class="p">)</span> <span class="o">-></span> <span class="nb">float</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="n">a</span> <span class="o">+</span> <span class="bp">self</span><span class="o">.</span><span class="n">x</span>
|
||||
|
||||
|
||||
<span class="k">class</span> <span class="nc">LargeComponent</span><span class="p">(</span><span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">repeats</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">4</span>
|
||||
<span class="n">subcomponent</span><span class="p">:</span> <span class="n">SubComponentBase</span>
|
||||
<span class="n">subcomponent_class_type</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">"SubComponent"</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="n">run_auto_creation</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">apply</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">a</span><span class="p">:</span> <span class="nb">float</span><span class="p">)</span> <span class="o">-></span> <span class="nb">float</span><span class="p">:</span>
|
||||
<span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">repeats</span><span class="p">):</span>
|
||||
<span class="n">a</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">subcomponent</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">a</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">expand_args_fields</span><span class="p">(</span><span class="n">LargeComponent</span><span class="p">)</span>
|
||||
<span class="n">large_component</span> <span class="o">=</span> <span class="n">LargeComponent</span><span class="p">()</span>
|
||||
<span class="k">assert</span> <span class="n">large_component</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span> <span class="o">==</span> <span class="mi">4</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">LargeComponent</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>The following things had to change:</p>
|
||||
<ul>
|
||||
<li>The base class SubComponentBase was defined.</li>
|
||||
<li>SubComponent gained a <code>@registry.register</code> decoration and had its base class changed to the new one.</li>
|
||||
<li><code>subcomponent_class_type</code> was added as a member of the outer class.</li>
|
||||
<li>In any saved configuration yaml files, the key <code>subcomponent_args</code> had to be changed to <code>subcomponent_SubComponent_args</code>.</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="Appendix:-gotchas-⚠️">Appendix: gotchas ⚠️<a class="anchor-link" href="#Appendix:-gotchas-⚠️">¶</a></h2><ul>
|
||||
<li>Omitting to define <code>__post_init__</code> or not calling <code>run_auto_creation</code> in it.</li>
|
||||
<li>Using a configurable class without calling get_default_args or expand_args_fields on it.</li>
|
||||
<li>Omitting a type annotation on a field. For example, writing
|
||||
<pre><code> subcomponent_class_type = "SubComponent"</code></pre>
|
||||
instead of
|
||||
<pre><code> subcomponent_class_type: str = "SubComponent"</code></pre>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div></div></div></div></div><footer class="nav-footer" id="footer"><section class="sitemap"><div class="footerSection"><div class="social"><a class="github-button" href="https://github.com/facebookresearch/pytorch3d" data-count-href="https://github.com/facebookresearch/pytorch3d/stargazers" data-show-count="true" data-count-aria-label="# stargazers on GitHub" aria-label="Star PyTorch3D on GitHub">pytorch3d</a></div></div></section><a href="https://opensource.facebook.com/" target="_blank" rel="noreferrer noopener" class="fbOpenSource"><img src="/img/oss_logo.png" alt="Facebook Open Source" width="170" height="45"/></a><section class="copyright">Copyright © 2022 Meta Platforms, Inc<br/>Legal:<a href="https://opensource.facebook.com/legal/privacy/" target="_blank" rel="noreferrer noopener">Privacy</a><a href="https://opensource.facebook.com/legal/terms/" target="_blank" rel="noreferrer noopener">Terms</a></section></footer></div></body></html>
|
||||
884
tutorials/implicitron_config_system/index.html
Normal file
884
tutorials/implicitron_config_system/index.html
Normal file
@ -0,0 +1,884 @@
|
||||
<!DOCTYPE html><html lang=""><head><meta charSet="utf-8"/><meta http-equiv="X-UA-Compatible" content="IE=edge"/><title>PyTorch3D · A library for deep learning with 3D data</title><meta name="viewport" content="width=device-width, initial-scale=1.0"/><meta name="generator" content="Docusaurus"/><meta name="description" content="A library for deep learning with 3D data"/><meta property="og:title" content="PyTorch3D · A library for deep learning with 3D data"/><meta property="og:type" content="website"/><meta property="og:url" content="https://pytorch3d.org/"/><meta property="og:description" content="A library for deep learning with 3D data"/><meta property="og:image" content="https://pytorch3d.org/img/pytorch3dlogoicon.svg"/><meta name="twitter:card" content="summary"/><meta name="twitter:image" content="https://pytorch3d.org/img/pytorch3dlogoicon.svg"/><link rel="shortcut icon" href="/img/pytorch3dfavicon.png"/><link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/default.min.css"/><script>
|
||||
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
|
||||
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
|
||||
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
|
||||
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
var links = coll[i].nextElementSibling.getElementsByTagName('*');
|
||||
if (checkActiveCategory){
|
||||
for (var j = 0; j < links.length; j++) {
|
||||
if (links[j].classList.contains('navListItemActive')){
|
||||
coll[i].nextElementSibling.classList.toggle('hide');
|
||||
coll[i].childNodes[1].classList.toggle('rotate');
|
||||
checkActiveCategory = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
coll[i].addEventListener('click', function() {
|
||||
var arrow = this.childNodes[1];
|
||||
arrow.classList.toggle('rotate');
|
||||
var content = this.nextElementSibling;
|
||||
content.classList.toggle('hide');
|
||||
});
|
||||
}
|
||||
|
||||
document.addEventListener('DOMContentLoaded', function() {
|
||||
createToggler('#navToggler', '#docsNav', 'docsSliderActive');
|
||||
createToggler('#tocToggler', 'body', 'tocActive');
|
||||
|
||||
var headings = document.querySelector('.toc-headings');
|
||||
headings && headings.addEventListener('click', function(event) {
|
||||
var el = event.target;
|
||||
while(el !== headings){
|
||||
if (el.tagName === 'A') {
|
||||
document.body.classList.remove('tocActive');
|
||||
break;
|
||||
} else{
|
||||
el = el.parentNode;
|
||||
}
|
||||
}
|
||||
}, false);
|
||||
|
||||
function createToggler(togglerSelector, targetSelector, className) {
|
||||
var toggler = document.querySelector(togglerSelector);
|
||||
var target = document.querySelector(targetSelector);
|
||||
|
||||
if (!toggler) {
|
||||
return;
|
||||
}
|
||||
|
||||
toggler.onclick = function(event) {
|
||||
event.preventDefault();
|
||||
|
||||
target.classList.toggle(className);
|
||||
};
|
||||
}
|
||||
});
|
||||
</script></nav></div><div class="container mainContainer"><div class="wrapper"><div class="tutorialButtonsWrapper"><div class="tutorialButtonWrapper buttonWrapper"><a class="tutorialButton button" download="" href="https://colab.research.google.com/github/facebookresearch/pytorch3d/blob/stable/docs/tutorials/implicitron_config_system.ipynb" target="_blank"><img class="colabButton" align="left" src="/img/colab_icon.png"/>Run in Google Colab</a></div><div class="tutorialButtonWrapper buttonWrapper"><a class="tutorialButton button" download="" href="/files/implicitron_config_system.ipynb" target="_blank"><svg aria-hidden="true" focusable="false" data-prefix="fas" data-icon="file-download" class="svg-inline--fa fa-file-download fa-w-12" role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 384 512"><path fill="currentColor" d="M224 136V0H24C10.7 0 0 10.7 0 24v464c0 13.3 10.7 24 24 24h336c13.3 0 24-10.7 24-24V160H248c-13.2 0-24-10.8-24-24zm76.45 211.36l-96.42 95.7c-6.65 6.61-17.39 6.61-24.04 0l-96.42-95.7C73.42 337.29 80.54 320 94.82 320H160v-80c0-8.84 7.16-16 16-16h32c8.84 0 16 7.16 16 16v80h65.18c14.28 0 21.4 17.29 11.27 27.36zM377 105L279.1 7c-4.5-4.5-10.6-7-17-7H256v128h128v-6.1c0-6.3-2.5-12.4-7-16.9z"></path></svg>Download Tutorial Jupyter Notebook</a></div><div class="tutorialButtonWrapper buttonWrapper"><a class="tutorialButton button" download="" href="/files/implicitron_config_system.py" target="_blank"><svg aria-hidden="true" focusable="false" data-prefix="fas" data-icon="file-download" class="svg-inline--fa fa-file-download fa-w-12" role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 384 512"><path fill="currentColor" d="M224 136V0H24C10.7 0 0 10.7 0 24v464c0 13.3 10.7 24 24 24h336c13.3 0 24-10.7 24-24V160H248c-13.2 0-24-10.8-24-24zm76.45 211.36l-96.42 95.7c-6.65 6.61-17.39 6.61-24.04 0l-96.42-95.7C73.42 337.29 80.54 320 94.82 320H160v-80c0-8.84 7.16-16 16-16h32c8.84 0 16 7.16 16 16v80h65.18c14.28 0 21.4 17.29 11.27 27.36zM377 105L279.1 7c-4.5-4.5-10.6-7-17-7H256v128h128v-6.1c0-6.3-2.5-12.4-7-16.9z"></path></svg>Download Tutorial Source Code</a></div></div><div class="tutorialBody">
|
||||
<script
|
||||
src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.1.10/require.min.js">
|
||||
</script>
|
||||
<script
|
||||
src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.min.js">
|
||||
</script>
|
||||
<div class="notebook">
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># Copyright (c) Meta Platforms, Inc. and affiliates. All rights reserved.</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h1 id="Implicitron's-config-system">Implicitron's config system<a class="anchor-link" href="#Implicitron's-config-system">¶</a></h1>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>Implicitron's components are all based on a unified hierarchical configuration system.
|
||||
This allows configurable variables and all defaults to be defined separately for each new component.
|
||||
All configs relevant to an experiment are then automatically composed into a single configuration file that fully specifies the experiment.
|
||||
An especially important feature is extension points where users can insert their own sub-classes of Implicitron's base components.</p>
|
||||
<p>The file which defines this system is <a href="https://github.com/facebookresearch/pytorch3d/blob/main/pytorch3d/implicitron/tools/config.py">here</a> in the PyTorch3D repo.
|
||||
The Implicitron volumes tutorial contains a simple example of using the config system.
|
||||
This tutorial provides detailed hands-on experience in using and modifying Implicitron's configurable components.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="0.-Install-and-import-modules">0. Install and import modules<a class="anchor-link" href="#0.-Install-and-import-modules">¶</a></h2><p>Ensure <code>torch</code> and <code>torchvision</code> are installed. If <code>pytorch3d</code> is not installed, install it using the following cell:</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">os</span>
|
||||
<span class="kn">import</span> <span class="nn">sys</span>
|
||||
<span class="kn">import</span> <span class="nn">torch</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">False</span>
|
||||
<span class="k">try</span><span class="p">:</span>
|
||||
<span class="kn">import</span> <span class="nn">pytorch3d</span>
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
<span class="sa">f</span><span class="s2">"py3</span><span class="si">{</span><span class="n">sys</span><span class="o">.</span><span class="n">version_info</span><span class="o">.</span><span class="n">minor</span><span class="si">}</span><span class="s2">_cu"</span><span class="p">,</span>
|
||||
<span class="n">torch</span><span class="o">.</span><span class="n">version</span><span class="o">.</span><span class="n">cuda</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span><span class="s2">""</span><span class="p">),</span>
|
||||
<span class="sa">f</span><span class="s2">"_pyt</span><span class="si">{</span><span class="n">pyt_version_str</span><span class="si">}</span><span class="s2">"</span>
|
||||
<span class="p">])</span>
|
||||
<span class="o">!</span>pip install fvcore iopath
|
||||
<span class="o">!</span>pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/<span class="o">{</span>version_str<span class="o">}</span>/download.html
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="c1"># We try to install PyTorch3D from source.</span>
|
||||
<span class="o">!</span>curl -LO https://github.com/NVIDIA/cub/archive/1.10.0.tar.gz
|
||||
<span class="o">!</span>tar xzf <span class="m">1</span>.10.0.tar.gz
|
||||
<span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">"CUB_HOME"</span><span class="p">]</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">getcwd</span><span class="p">()</span> <span class="o">+</span> <span class="s2">"/cub-1.10.0"</span>
|
||||
<span class="o">!</span>pip install <span class="s1">'git+https://github.com/facebookresearch/pytorch3d.git@stable'</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>Ensure omegaconf is installed. If not, run this cell. (It should not be necessary to restart the runtime.)</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="o">!</span>pip install omegaconf
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">dataclasses</span> <span class="kn">import</span> <span class="n">dataclass</span>
|
||||
<span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Optional</span><span class="p">,</span> <span class="n">Tuple</span>
|
||||
|
||||
<span class="kn">import</span> <span class="nn">torch</span>
|
||||
<span class="kn">from</span> <span class="nn">omegaconf</span> <span class="kn">import</span> <span class="n">DictConfig</span><span class="p">,</span> <span class="n">OmegaConf</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.implicitron.tools.config</span> <span class="kn">import</span> <span class="p">(</span>
|
||||
<span class="n">Configurable</span><span class="p">,</span>
|
||||
<span class="n">ReplaceableBase</span><span class="p">,</span>
|
||||
<span class="n">expand_args_fields</span><span class="p">,</span>
|
||||
<span class="n">get_default_args</span><span class="p">,</span>
|
||||
<span class="n">registry</span><span class="p">,</span>
|
||||
<span class="n">run_auto_creation</span><span class="p">,</span>
|
||||
<span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="1.-Introducing-dataclasses">1. Introducing dataclasses<a class="anchor-link" href="#1.-Introducing-dataclasses">¶</a></h2><p><a href="https://docs.python.org/3/library/typing.html">Type hints</a> give a taxonomy of types in Python. <a href="https://docs.python.org/3/library/dataclasses.html">Dataclasses</a> let you create a class based on a list of members which have names, types and possibly default values. The <code>__init__</code> function is created automatically, and calls a <code>__post_init__</code> function if present as a final step. For example</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="nd">@dataclass</span>
|
||||
<span class="k">class</span> <span class="nc">MyDataclass</span><span class="p">:</span>
|
||||
<span class="n">a</span><span class="p">:</span> <span class="nb">int</span>
|
||||
<span class="n">b</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">8</span>
|
||||
<span class="n">c</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">Tuple</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o">...</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"created with a = </span><span class="si">{</span><span class="bp">self</span><span class="o">.</span><span class="n">a</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">d</span> <span class="o">=</span> <span class="mi">2</span> <span class="o">*</span> <span class="bp">self</span><span class="o">.</span><span class="n">b</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">my_dataclass_instance</span> <span class="o">=</span> <span class="n">MyDataclass</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="mi">18</span><span class="p">)</span>
|
||||
<span class="k">assert</span> <span class="n">my_dataclass_instance</span><span class="o">.</span><span class="n">d</span> <span class="o">==</span> <span class="mi">16</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>👷 Note that the <code>dataclass</code> decorator here is function which modifies the definition of the class itself.
|
||||
It runs immediately after the definition.
|
||||
Our config system requires that implicitron library code contains classes whose modified versions need to be aware of user-defined implementations.
|
||||
Therefore we need the modification of the class to be delayed. We don't use a decorator.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="2.-Introducing-omegaconf-and-OmegaConf.structured">2. Introducing omegaconf and OmegaConf.structured<a class="anchor-link" href="#2.-Introducing-omegaconf-and-OmegaConf.structured">¶</a></h2><p>The <a href="https://github.com/omry/omegaconf/">omegaconf</a> library provides a DictConfig class which is like a <code>dict</code> with str keys, but with extra features for ease-of-use as a configuration system.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">dc</span> <span class="o">=</span> <span class="n">DictConfig</span><span class="p">({</span><span class="s2">"a"</span><span class="p">:</span> <span class="mi">2</span><span class="p">,</span> <span class="s2">"b"</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span> <span class="s2">"c"</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span> <span class="s2">"d"</span><span class="p">:</span> <span class="s2">"hello"</span><span class="p">})</span>
|
||||
<span class="k">assert</span> <span class="n">dc</span><span class="o">.</span><span class="n">a</span> <span class="o">==</span> <span class="n">dc</span><span class="p">[</span><span class="s2">"a"</span><span class="p">]</span> <span class="o">==</span> <span class="mi">2</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>OmegaConf has a serialization to and from yaml. The <a href="https://hydra.cc/">Hydra</a> library relies on this for its configuration files.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="nb">print</span><span class="p">(</span><span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">dc</span><span class="p">))</span>
|
||||
<span class="k">assert</span> <span class="n">OmegaConf</span><span class="o">.</span><span class="n">create</span><span class="p">(</span><span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">dc</span><span class="p">))</span> <span class="o">==</span> <span class="n">dc</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>OmegaConf.structured provides a DictConfig from a dataclass or instance of a dataclass. Unlike a normal DictConfig, it is type-checked and only known keys can be added.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">structured</span> <span class="o">=</span> <span class="n">OmegaConf</span><span class="o">.</span><span class="n">structured</span><span class="p">(</span><span class="n">MyDataclass</span><span class="p">)</span>
|
||||
<span class="k">assert</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">structured</span><span class="p">,</span> <span class="n">DictConfig</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">structured</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">()</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">structured</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p><code>structured</code> knows it is missing a value for <code>a</code>.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>Such an object has members compatible with the dataclass, so an initialisation can be performed as follows.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">structured</span><span class="o">.</span><span class="n">a</span> <span class="o">=</span> <span class="mi">21</span>
|
||||
<span class="n">my_dataclass_instance2</span> <span class="o">=</span> <span class="n">MyDataclass</span><span class="p">(</span><span class="o">**</span><span class="n">structured</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">my_dataclass_instance2</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>You can also call OmegaConf.structured on an instance.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">structured_from_instance</span> <span class="o">=</span> <span class="n">OmegaConf</span><span class="o">.</span><span class="n">structured</span><span class="p">(</span><span class="n">my_dataclass_instance</span><span class="p">)</span>
|
||||
<span class="n">my_dataclass_instance3</span> <span class="o">=</span> <span class="n">MyDataclass</span><span class="p">(</span><span class="o">**</span><span class="n">structured_from_instance</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">my_dataclass_instance3</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="3.-Our-approach-to-OmegaConf.structured">3. Our approach to OmegaConf.structured<a class="anchor-link" href="#3.-Our-approach-to-OmegaConf.structured">¶</a></h2><p>We provide functions which are equivalent to <code>OmegaConf.structured</code> but support more features.
|
||||
To achieve the above using our functions, the following is used.
|
||||
Note that we indicate configurable classes using a special base class <code>Configurable</code>, not a decorator.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">MyConfigurable</span><span class="p">(</span><span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">a</span><span class="p">:</span> <span class="nb">int</span>
|
||||
<span class="n">b</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">8</span>
|
||||
<span class="n">c</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">Tuple</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o">...</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"created with a = </span><span class="si">{</span><span class="bp">self</span><span class="o">.</span><span class="n">a</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">d</span> <span class="o">=</span> <span class="mi">2</span> <span class="o">*</span> <span class="bp">self</span><span class="o">.</span><span class="n">b</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># expand_args_fields must be called on an object before it is instantiated.</span>
|
||||
<span class="c1"># A warning is raised if this is missed, but it is possible to not notice the warning.</span>
|
||||
<span class="c1"># It modifies the class like @dataclass</span>
|
||||
<span class="n">expand_args_fields</span><span class="p">(</span><span class="n">MyConfigurable</span><span class="p">)</span>
|
||||
<span class="n">my_configurable_instance</span> <span class="o">=</span> <span class="n">MyConfigurable</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="mi">18</span><span class="p">)</span>
|
||||
<span class="k">assert</span> <span class="n">my_configurable_instance</span><span class="o">.</span><span class="n">d</span> <span class="o">==</span> <span class="mi">16</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># get_default_args calls expand_args_fields automatically</span>
|
||||
<span class="n">our_structured</span> <span class="o">=</span> <span class="n">get_default_args</span><span class="p">(</span><span class="n">MyConfigurable</span><span class="p">)</span>
|
||||
<span class="k">assert</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">our_structured</span><span class="p">,</span> <span class="n">DictConfig</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">our_structured</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">our_structured</span><span class="o">.</span><span class="n">a</span> <span class="o">=</span> <span class="mi">21</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">MyConfigurable</span><span class="p">(</span><span class="o">**</span><span class="n">our_structured</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="4.-First-enhancement:-nested-types-🪺">4. First enhancement: nested types 🪺<a class="anchor-link" href="#4.-First-enhancement:-nested-types-🪺">¶</a></h2><p>Our system allows Configurable classes to contain each other.
|
||||
One thing to remember: add a call to <code>run_auto_creation</code> in <code>__post_init__</code>.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">Inner</span><span class="p">(</span><span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">a</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">8</span>
|
||||
<span class="n">b</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
<span class="n">c</span><span class="p">:</span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o">...</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
|
||||
|
||||
|
||||
<span class="k">class</span> <span class="nc">Outer</span><span class="p">(</span><span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">inner</span><span class="p">:</span> <span class="n">Inner</span>
|
||||
<span class="n">x</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">"hello"</span>
|
||||
<span class="n">xx</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="n">run_auto_creation</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">outer_dc</span> <span class="o">=</span> <span class="n">get_default_args</span><span class="p">(</span><span class="n">Outer</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">outer_dc</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">outer</span> <span class="o">=</span> <span class="n">Outer</span><span class="p">(</span><span class="o">**</span><span class="n">outer_dc</span><span class="p">)</span>
|
||||
<span class="k">assert</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">outer</span><span class="p">,</span> <span class="n">Outer</span><span class="p">)</span>
|
||||
<span class="k">assert</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">outer</span><span class="o">.</span><span class="n">inner</span><span class="p">,</span> <span class="n">Inner</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="nb">vars</span><span class="p">(</span><span class="n">outer</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">outer</span><span class="o">.</span><span class="n">inner</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>Note how inner_args is an extra member of outer. <code>run_auto_creation(self)</code> is equivalent to</p>
|
||||
<pre><code> self.inner = Inner(**self.inner_args)</code></pre>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="5.-Second-enhancement:-pluggable/replaceable-components-🔌">5. Second enhancement: pluggable/replaceable components 🔌<a class="anchor-link" href="#5.-Second-enhancement:-pluggable/replaceable-components-🔌">¶</a></h2><p>If a class uses <code>ReplaceableBase</code> as a base class instead of <code>Configurable</code>, we call it a replaceable.
|
||||
It indicates that it is designed for child classes to use in its place.
|
||||
We might use <code>NotImplementedError</code> to indicate functionality which subclasses are expected to implement.
|
||||
The system maintains a global <code>registry</code> containing subclasses of each ReplaceableBase.
|
||||
The subclasses register themselves with it with a decorator.</p>
|
||||
<p>A configurable class (i.e. a class which uses our system, i.e. a child of <code>Configurable</code> or <code>ReplaceableBase</code>) which contains a ReplaceableBase must also
|
||||
contain a corresponding class_type field of type <code>str</code> which indicates which concrete child class to use.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">InnerBase</span><span class="p">(</span><span class="n">ReplaceableBase</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">say_something</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="k">raise</span> <span class="ne">NotImplementedError</span>
|
||||
|
||||
|
||||
<span class="nd">@registry</span><span class="o">.</span><span class="n">register</span>
|
||||
<span class="k">class</span> <span class="nc">Inner1</span><span class="p">(</span><span class="n">InnerBase</span><span class="p">):</span>
|
||||
<span class="n">a</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">1</span>
|
||||
<span class="n">b</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">"h"</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">say_something</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"hello from an Inner1"</span><span class="p">)</span>
|
||||
|
||||
|
||||
<span class="nd">@registry</span><span class="o">.</span><span class="n">register</span>
|
||||
<span class="k">class</span> <span class="nc">Inner2</span><span class="p">(</span><span class="n">InnerBase</span><span class="p">):</span>
|
||||
<span class="n">a</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">2</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">say_something</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"hello from an Inner2"</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">Out</span><span class="p">(</span><span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">inner</span><span class="p">:</span> <span class="n">InnerBase</span>
|
||||
<span class="n">inner_class_type</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">"Inner1"</span>
|
||||
<span class="n">x</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">19</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="n">run_auto_creation</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">talk</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">inner</span><span class="o">.</span><span class="n">say_something</span><span class="p">()</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">Out_dc</span> <span class="o">=</span> <span class="n">get_default_args</span><span class="p">(</span><span class="n">Out</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">Out_dc</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">Out_dc</span><span class="o">.</span><span class="n">inner_class_type</span> <span class="o">=</span> <span class="s2">"Inner2"</span>
|
||||
<span class="n">out</span> <span class="o">=</span> <span class="n">Out</span><span class="p">(</span><span class="o">**</span><span class="n">Out_dc</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">out</span><span class="o">.</span><span class="n">inner</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">out</span><span class="o">.</span><span class="n">talk</span><span class="p">()</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>Note in this case there are many <code>args</code> members. It is usually fine to ignore them in the code. They are needed for the config.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="nb">print</span><span class="p">(</span><span class="nb">vars</span><span class="p">(</span><span class="n">out</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="6.-Example-with-torch.nn.Module--🔥">6. Example with torch.nn.Module 🔥<a class="anchor-link" href="#6.-Example-with-torch.nn.Module--🔥">¶</a></h2><p>Typically in implicitron, we use this system in combination with <a href="https://pytorch.org/docs/stable/generated/torch.nn.Module.html"><code>Module</code></a>s.
|
||||
Note in this case it is necessary to call <code>Module.__init__</code> explicitly in <code>__post_init__</code>.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">MyLinear</span><span class="p">(</span><span class="n">torch</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">Module</span><span class="p">,</span> <span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">d_in</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">2</span>
|
||||
<span class="n">d_out</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">200</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">()</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">linear</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">Linear</span><span class="p">(</span><span class="n">in_features</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">d_in</span><span class="p">,</span> <span class="n">out_features</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">d_out</span><span class="p">)</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span>
|
||||
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">linear</span><span class="o">.</span><span class="n">forward</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">expand_args_fields</span><span class="p">(</span><span class="n">MyLinear</span><span class="p">)</span>
|
||||
<span class="n">my_linear</span> <span class="o">=</span> <span class="n">MyLinear</span><span class="p">()</span>
|
||||
<span class="nb">input</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
|
||||
<span class="n">output</span> <span class="o">=</span> <span class="n">my_linear</span><span class="p">(</span><span class="nb">input</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"output shape:"</span><span class="p">,</span> <span class="n">output</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p><code>my_linear</code> has all the usual features of a Module.
|
||||
E.g. it can be saved and loaded with <code>torch.save</code> and <code>torch.load</code>.
|
||||
It has parameters:</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">my_linear</span><span class="o">.</span><span class="n">named_parameters</span><span class="p">():</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">value</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="7.-Example-of-implementing-your-own-pluggable-component">7. Example of implementing your own pluggable component<a class="anchor-link" href="#7.-Example-of-implementing-your-own-pluggable-component">¶</a></h2><p>Let's say I am using a library with <code>Out</code> like in section <strong>5</strong> but I want to implement my own child of InnerBase.
|
||||
All I need to do is register its definition, but I need to do this before expand_args_fields is explicitly or implicitly called on Out.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="nd">@registry</span><span class="o">.</span><span class="n">register</span>
|
||||
<span class="k">class</span> <span class="nc">UserImplementedInner</span><span class="p">(</span><span class="n">InnerBase</span><span class="p">):</span>
|
||||
<span class="n">a</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">200</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">say_something</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"hello from the user"</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>At this point, we need to redefine the class Out.
|
||||
Otherwise if it has already been expanded without UserImplementedInner, then the following would not work,
|
||||
because the implementations known to a class are fixed when it is expanded.</p>
|
||||
<p>If you are running experiments from a script, the thing to remember here is that you must import your own modules, which register your own implementations,
|
||||
before you <em>use</em> the library classes.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">Out</span><span class="p">(</span><span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">inner</span><span class="p">:</span> <span class="n">InnerBase</span>
|
||||
<span class="n">inner_class_type</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">"Inner1"</span>
|
||||
<span class="n">x</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">19</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="n">run_auto_creation</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">talk</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">inner</span><span class="o">.</span><span class="n">say_something</span><span class="p">()</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">expand_args_fields</span><span class="p">(</span><span class="n">Out</span><span class="p">)</span>
|
||||
<span class="n">out2</span> <span class="o">=</span> <span class="n">Out</span><span class="p">(</span><span class="n">inner_class_type</span><span class="o">=</span><span class="s2">"UserImplementedInner"</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">out2</span><span class="o">.</span><span class="n">inner</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="8:-Example-of-making-a-subcomponent-pluggable">8: Example of making a subcomponent pluggable<a class="anchor-link" href="#8:-Example-of-making-a-subcomponent-pluggable">¶</a></h2><p>Let's look what needs to happen if we have a subcomponent which we make pluggable, to allow users to supply their own.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">SubComponent</span><span class="p">(</span><span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">x</span><span class="p">:</span> <span class="nb">float</span> <span class="o">=</span> <span class="mf">0.25</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">apply</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">a</span><span class="p">:</span> <span class="nb">float</span><span class="p">)</span> <span class="o">-></span> <span class="nb">float</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="n">a</span> <span class="o">+</span> <span class="bp">self</span><span class="o">.</span><span class="n">x</span>
|
||||
|
||||
|
||||
<span class="k">class</span> <span class="nc">LargeComponent</span><span class="p">(</span><span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">repeats</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">4</span>
|
||||
<span class="n">subcomponent</span><span class="p">:</span> <span class="n">SubComponent</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="n">run_auto_creation</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">apply</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">a</span><span class="p">:</span> <span class="nb">float</span><span class="p">)</span> <span class="o">-></span> <span class="nb">float</span><span class="p">:</span>
|
||||
<span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">repeats</span><span class="p">):</span>
|
||||
<span class="n">a</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">subcomponent</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">a</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">expand_args_fields</span><span class="p">(</span><span class="n">LargeComponent</span><span class="p">)</span>
|
||||
<span class="n">large_component</span> <span class="o">=</span> <span class="n">LargeComponent</span><span class="p">()</span>
|
||||
<span class="k">assert</span> <span class="n">large_component</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span> <span class="o">==</span> <span class="mi">4</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">LargeComponent</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>Made generic:</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">SubComponentBase</span><span class="p">(</span><span class="n">ReplaceableBase</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">apply</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">a</span><span class="p">:</span> <span class="nb">float</span><span class="p">)</span> <span class="o">-></span> <span class="nb">float</span><span class="p">:</span>
|
||||
<span class="k">raise</span> <span class="ne">NotImplementedError</span>
|
||||
|
||||
|
||||
<span class="nd">@registry</span><span class="o">.</span><span class="n">register</span>
|
||||
<span class="k">class</span> <span class="nc">SubComponent</span><span class="p">(</span><span class="n">SubComponentBase</span><span class="p">):</span>
|
||||
<span class="n">x</span><span class="p">:</span> <span class="nb">float</span> <span class="o">=</span> <span class="mf">0.25</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">apply</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">a</span><span class="p">:</span> <span class="nb">float</span><span class="p">)</span> <span class="o">-></span> <span class="nb">float</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="n">a</span> <span class="o">+</span> <span class="bp">self</span><span class="o">.</span><span class="n">x</span>
|
||||
|
||||
|
||||
<span class="k">class</span> <span class="nc">LargeComponent</span><span class="p">(</span><span class="n">Configurable</span><span class="p">):</span>
|
||||
<span class="n">repeats</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">4</span>
|
||||
<span class="n">subcomponent</span><span class="p">:</span> <span class="n">SubComponentBase</span>
|
||||
<span class="n">subcomponent_class_type</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">"SubComponent"</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="n">run_auto_creation</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">apply</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">a</span><span class="p">:</span> <span class="nb">float</span><span class="p">)</span> <span class="o">-></span> <span class="nb">float</span><span class="p">:</span>
|
||||
<span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">repeats</span><span class="p">):</span>
|
||||
<span class="n">a</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">subcomponent</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">a</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">expand_args_fields</span><span class="p">(</span><span class="n">LargeComponent</span><span class="p">)</span>
|
||||
<span class="n">large_component</span> <span class="o">=</span> <span class="n">LargeComponent</span><span class="p">()</span>
|
||||
<span class="k">assert</span> <span class="n">large_component</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span> <span class="o">==</span> <span class="mi">4</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">LargeComponent</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>The following things had to change:</p>
|
||||
<ul>
|
||||
<li>The base class SubComponentBase was defined.</li>
|
||||
<li>SubComponent gained a <code>@registry.register</code> decoration and had its base class changed to the new one.</li>
|
||||
<li><code>subcomponent_class_type</code> was added as a member of the outer class.</li>
|
||||
<li>In any saved configuration yaml files, the key <code>subcomponent_args</code> had to be changed to <code>subcomponent_SubComponent_args</code>.</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="Appendix:-gotchas-⚠️">Appendix: gotchas ⚠️<a class="anchor-link" href="#Appendix:-gotchas-⚠️">¶</a></h2><ul>
|
||||
<li>Omitting to define <code>__post_init__</code> or not calling <code>run_auto_creation</code> in it.</li>
|
||||
<li>Using a configurable class without calling get_default_args or expand_args_fields on it.</li>
|
||||
<li>Omitting a type annotation on a field. For example, writing
|
||||
<pre><code> subcomponent_class_type = "SubComponent"</code></pre>
|
||||
instead of
|
||||
<pre><code> subcomponent_class_type: str = "SubComponent"</code></pre>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div></div></div></div></div><footer class="nav-footer" id="footer"><section class="sitemap"><div class="footerSection"><div class="social"><a class="github-button" href="https://github.com/facebookresearch/pytorch3d" data-count-href="https://github.com/facebookresearch/pytorch3d/stargazers" data-show-count="true" data-count-aria-label="# stargazers on GitHub" aria-label="Star PyTorch3D on GitHub">pytorch3d</a></div></div></section><a href="https://opensource.facebook.com/" target="_blank" rel="noreferrer noopener" class="fbOpenSource"><img src="/img/oss_logo.png" alt="Facebook Open Source" width="170" height="45"/></a><section class="copyright">Copyright © 2022 Meta Platforms, Inc<br/>Legal:<a href="https://opensource.facebook.com/legal/privacy/" target="_blank" rel="noreferrer noopener">Privacy</a><a href="https://opensource.facebook.com/legal/terms/" target="_blank" rel="noreferrer noopener">Terms</a></section></footer></div></body></html>
|
||||
669
tutorials/implicitron_volumes.html
Normal file
669
tutorials/implicitron_volumes.html
Normal file
@ -0,0 +1,669 @@
|
||||
<!DOCTYPE html><html lang=""><head><meta charSet="utf-8"/><meta http-equiv="X-UA-Compatible" content="IE=edge"/><title>PyTorch3D · A library for deep learning with 3D data</title><meta name="viewport" content="width=device-width, initial-scale=1.0"/><meta name="generator" content="Docusaurus"/><meta name="description" content="A library for deep learning with 3D data"/><meta property="og:title" content="PyTorch3D · A library for deep learning with 3D data"/><meta property="og:type" content="website"/><meta property="og:url" content="https://pytorch3d.org/"/><meta property="og:description" content="A library for deep learning with 3D data"/><meta property="og:image" content="https://pytorch3d.org/img/pytorch3dlogoicon.svg"/><meta name="twitter:card" content="summary"/><meta name="twitter:image" content="https://pytorch3d.org/img/pytorch3dlogoicon.svg"/><link rel="shortcut icon" href="/img/pytorch3dfavicon.png"/><link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/default.min.css"/><script>
|
||||
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
|
||||
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
|
||||
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
|
||||
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
var links = coll[i].nextElementSibling.getElementsByTagName('*');
|
||||
if (checkActiveCategory){
|
||||
for (var j = 0; j < links.length; j++) {
|
||||
if (links[j].classList.contains('navListItemActive')){
|
||||
coll[i].nextElementSibling.classList.toggle('hide');
|
||||
coll[i].childNodes[1].classList.toggle('rotate');
|
||||
checkActiveCategory = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
coll[i].addEventListener('click', function() {
|
||||
var arrow = this.childNodes[1];
|
||||
arrow.classList.toggle('rotate');
|
||||
var content = this.nextElementSibling;
|
||||
content.classList.toggle('hide');
|
||||
});
|
||||
}
|
||||
|
||||
document.addEventListener('DOMContentLoaded', function() {
|
||||
createToggler('#navToggler', '#docsNav', 'docsSliderActive');
|
||||
createToggler('#tocToggler', 'body', 'tocActive');
|
||||
|
||||
var headings = document.querySelector('.toc-headings');
|
||||
headings && headings.addEventListener('click', function(event) {
|
||||
var el = event.target;
|
||||
while(el !== headings){
|
||||
if (el.tagName === 'A') {
|
||||
document.body.classList.remove('tocActive');
|
||||
break;
|
||||
} else{
|
||||
el = el.parentNode;
|
||||
}
|
||||
}
|
||||
}, false);
|
||||
|
||||
function createToggler(togglerSelector, targetSelector, className) {
|
||||
var toggler = document.querySelector(togglerSelector);
|
||||
var target = document.querySelector(targetSelector);
|
||||
|
||||
if (!toggler) {
|
||||
return;
|
||||
}
|
||||
|
||||
toggler.onclick = function(event) {
|
||||
event.preventDefault();
|
||||
|
||||
target.classList.toggle(className);
|
||||
};
|
||||
}
|
||||
});
|
||||
</script></nav></div><div class="container mainContainer"><div class="wrapper"><div class="tutorialButtonsWrapper"><div class="tutorialButtonWrapper buttonWrapper"><a class="tutorialButton button" download="" href="https://colab.research.google.com/github/facebookresearch/pytorch3d/blob/stable/docs/tutorials/implicitron_volumes.ipynb" target="_blank"><img class="colabButton" align="left" src="/img/colab_icon.png"/>Run in Google Colab</a></div><div class="tutorialButtonWrapper buttonWrapper"><a class="tutorialButton button" download="" href="/files/implicitron_volumes.ipynb" target="_blank"><svg aria-hidden="true" focusable="false" data-prefix="fas" data-icon="file-download" class="svg-inline--fa fa-file-download fa-w-12" role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 384 512"><path fill="currentColor" d="M224 136V0H24C10.7 0 0 10.7 0 24v464c0 13.3 10.7 24 24 24h336c13.3 0 24-10.7 24-24V160H248c-13.2 0-24-10.8-24-24zm76.45 211.36l-96.42 95.7c-6.65 6.61-17.39 6.61-24.04 0l-96.42-95.7C73.42 337.29 80.54 320 94.82 320H160v-80c0-8.84 7.16-16 16-16h32c8.84 0 16 7.16 16 16v80h65.18c14.28 0 21.4 17.29 11.27 27.36zM377 105L279.1 7c-4.5-4.5-10.6-7-17-7H256v128h128v-6.1c0-6.3-2.5-12.4-7-16.9z"></path></svg>Download Tutorial Jupyter Notebook</a></div><div class="tutorialButtonWrapper buttonWrapper"><a class="tutorialButton button" download="" href="/files/implicitron_volumes.py" target="_blank"><svg aria-hidden="true" focusable="false" data-prefix="fas" data-icon="file-download" class="svg-inline--fa fa-file-download fa-w-12" role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 384 512"><path fill="currentColor" d="M224 136V0H24C10.7 0 0 10.7 0 24v464c0 13.3 10.7 24 24 24h336c13.3 0 24-10.7 24-24V160H248c-13.2 0-24-10.8-24-24zm76.45 211.36l-96.42 95.7c-6.65 6.61-17.39 6.61-24.04 0l-96.42-95.7C73.42 337.29 80.54 320 94.82 320H160v-80c0-8.84 7.16-16 16-16h32c8.84 0 16 7.16 16 16v80h65.18c14.28 0 21.4 17.29 11.27 27.36zM377 105L279.1 7c-4.5-4.5-10.6-7-17-7H256v128h128v-6.1c0-6.3-2.5-12.4-7-16.9z"></path></svg>Download Tutorial Source Code</a></div></div><div class="tutorialBody">
|
||||
<script
|
||||
src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.1.10/require.min.js">
|
||||
</script>
|
||||
<script
|
||||
src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.min.js">
|
||||
</script>
|
||||
<div class="notebook">
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># Copyright (c) Meta Platforms, Inc. and affiliates. All rights reserved.</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h1 id="A-simple-model-using-Implicitron">A simple model using Implicitron<a class="anchor-link" href="#A-simple-model-using-Implicitron">¶</a></h1><p>In this demo, we use the VolumeRenderer from PyTorch3D as a custom implicit function in Implicitron. We will see</p>
|
||||
<ul>
|
||||
<li>some of the main objects in Implicitron</li>
|
||||
<li>how to plug in a custom part of a model</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="0.-Install-and-import-modules">0. Install and import modules<a class="anchor-link" href="#0.-Install-and-import-modules">¶</a></h2><p>Ensure <code>torch</code> and <code>torchvision</code> are installed. If <code>pytorch3d</code> is not installed, install it using the following cell:</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">os</span>
|
||||
<span class="kn">import</span> <span class="nn">sys</span>
|
||||
<span class="kn">import</span> <span class="nn">torch</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">False</span>
|
||||
<span class="k">try</span><span class="p">:</span>
|
||||
<span class="kn">import</span> <span class="nn">pytorch3d</span>
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
<span class="sa">f</span><span class="s2">"py3</span><span class="si">{</span><span class="n">sys</span><span class="o">.</span><span class="n">version_info</span><span class="o">.</span><span class="n">minor</span><span class="si">}</span><span class="s2">_cu"</span><span class="p">,</span>
|
||||
<span class="n">torch</span><span class="o">.</span><span class="n">version</span><span class="o">.</span><span class="n">cuda</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span><span class="s2">""</span><span class="p">),</span>
|
||||
<span class="sa">f</span><span class="s2">"_pyt</span><span class="si">{</span><span class="n">pyt_version_str</span><span class="si">}</span><span class="s2">"</span>
|
||||
<span class="p">])</span>
|
||||
<span class="o">!</span>pip install fvcore iopath
|
||||
<span class="o">!</span>pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/<span class="o">{</span>version_str<span class="o">}</span>/download.html
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="c1"># We try to install PyTorch3D from source.</span>
|
||||
<span class="o">!</span>curl -LO https://github.com/NVIDIA/cub/archive/1.10.0.tar.gz
|
||||
<span class="o">!</span>tar xzf <span class="m">1</span>.10.0.tar.gz
|
||||
<span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">"CUB_HOME"</span><span class="p">]</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">getcwd</span><span class="p">()</span> <span class="o">+</span> <span class="s2">"/cub-1.10.0"</span>
|
||||
<span class="o">!</span>pip install <span class="s1">'git+https://github.com/facebookresearch/pytorch3d.git@stable'</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>Ensure omegaconf and visdom are installed. If not, run this cell. (It should not be necessary to restart the runtime.)</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="o">!</span>pip install omegaconf visdom
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
||||
<span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Tuple</span>
|
||||
|
||||
<span class="kn">import</span> <span class="nn">matplotlib.animation</span> <span class="k">as</span> <span class="nn">animation</span>
|
||||
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
|
||||
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">import</span> <span class="nn">torch</span>
|
||||
<span class="kn">import</span> <span class="nn">tqdm</span>
|
||||
<span class="kn">from</span> <span class="nn">IPython.display</span> <span class="kn">import</span> <span class="n">HTML</span>
|
||||
<span class="kn">from</span> <span class="nn">omegaconf</span> <span class="kn">import</span> <span class="n">OmegaConf</span>
|
||||
<span class="kn">from</span> <span class="nn">PIL</span> <span class="kn">import</span> <span class="n">Image</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.implicitron.dataset.dataset_base</span> <span class="kn">import</span> <span class="n">FrameData</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.implicitron.dataset.rendered_mesh_dataset_map_provider</span> <span class="kn">import</span> <span class="n">RenderedMeshDatasetMapProvider</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.implicitron.models.generic_model</span> <span class="kn">import</span> <span class="n">GenericModel</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.implicitron.models.implicit_function.base</span> <span class="kn">import</span> <span class="n">ImplicitFunctionBase</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.implicitron.models.renderer.base</span> <span class="kn">import</span> <span class="n">EvaluationMode</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.implicitron.tools.config</span> <span class="kn">import</span> <span class="n">expand_args_fields</span><span class="p">,</span> <span class="n">get_default_args</span><span class="p">,</span> <span class="n">registry</span><span class="p">,</span> <span class="n">remove_unused_components</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.renderer</span> <span class="kn">import</span> <span class="n">RayBundle</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.renderer.implicit.renderer</span> <span class="kn">import</span> <span class="n">VolumeSampler</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.structures</span> <span class="kn">import</span> <span class="n">Volumes</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.vis.plotly_vis</span> <span class="kn">import</span> <span class="n">plot_batch_individually</span><span class="p">,</span> <span class="n">plot_scene</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">output_resolution</span> <span class="o">=</span> <span class="mi">80</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">torch</span><span class="o">.</span><span class="n">set_printoptions</span><span class="p">(</span><span class="n">sci_mode</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="1.-Load-renders-of-a-mesh-(the-cow-mesh)-as-a-dataset">1. Load renders of a mesh (the cow mesh) as a dataset<a class="anchor-link" href="#1.-Load-renders-of-a-mesh-(the-cow-mesh)-as-a-dataset">¶</a></h2><p>A dataset's train, val and test parts in Implicitron are represented as a <code>dataset_map</code>, and provided by an implementation of <code>DatasetMapProvider</code>.
|
||||
<code>RenderedMeshDatasetMapProvider</code> is one which generates a single-scene dataset with only a train component by taking a mesh and rendering it.
|
||||
We use it with the cow mesh.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>If running this notebook using <strong>Google Colab</strong>, run the following cell to fetch the mesh obj and texture files and save it at the path data/cow_mesh.
|
||||
If running locally, the data is already available at the correct path.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="o">!</span>mkdir -p data/cow_mesh
|
||||
<span class="o">!</span>wget -P data/cow_mesh https://dl.fbaipublicfiles.com/pytorch3d/data/cow_mesh/cow.obj
|
||||
<span class="o">!</span>wget -P data/cow_mesh https://dl.fbaipublicfiles.com/pytorch3d/data/cow_mesh/cow.mtl
|
||||
<span class="o">!</span>wget -P data/cow_mesh https://dl.fbaipublicfiles.com/pytorch3d/data/cow_mesh/cow_texture.png
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>If we want to instantiate one of Implicitron's configurable objects, such as <code>RenderedMeshDatasetMapProvider</code>, without using the OmegaConf initialisation (get_default_args), we need to call <code>expand_args_fields</code> on the class first.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">expand_args_fields</span><span class="p">(</span><span class="n">RenderedMeshDatasetMapProvider</span><span class="p">)</span>
|
||||
<span class="n">cow_provider</span> <span class="o">=</span> <span class="n">RenderedMeshDatasetMapProvider</span><span class="p">(</span>
|
||||
<span class="n">data_file</span><span class="o">=</span><span class="s2">"data/cow_mesh/cow.obj"</span><span class="p">,</span>
|
||||
<span class="n">use_point_light</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
<span class="n">resolution</span><span class="o">=</span><span class="n">output_resolution</span><span class="p">,</span>
|
||||
<span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">dataset_map</span> <span class="o">=</span> <span class="n">cow_provider</span><span class="o">.</span><span class="n">get_dataset_map</span><span class="p">()</span>
|
||||
<span class="n">tr_cameras</span> <span class="o">=</span> <span class="p">[</span><span class="n">training_frame</span><span class="o">.</span><span class="n">camera</span> <span class="k">for</span> <span class="n">training_frame</span> <span class="ow">in</span> <span class="n">dataset_map</span><span class="o">.</span><span class="n">train</span><span class="p">]</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># The cameras are all in the XZ plane, in a circle about 2.7 from the origin</span>
|
||||
<span class="n">centers</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">cat</span><span class="p">([</span><span class="n">i</span><span class="o">.</span><span class="n">get_camera_center</span><span class="p">()</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">tr_cameras</span><span class="p">])</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">centers</span><span class="o">.</span><span class="n">min</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">.</span><span class="n">values</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">centers</span><span class="o">.</span><span class="n">max</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">.</span><span class="n">values</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># visualization of the cameras</span>
|
||||
<span class="n">plot</span> <span class="o">=</span> <span class="n">plot_scene</span><span class="p">({</span><span class="s2">"k"</span><span class="p">:</span> <span class="p">{</span><span class="n">i</span><span class="p">:</span> <span class="n">camera</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">camera</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">tr_cameras</span><span class="p">)}},</span> <span class="n">camera_scale</span><span class="o">=</span><span class="mf">0.25</span><span class="p">)</span>
|
||||
<span class="n">plot</span><span class="o">.</span><span class="n">layout</span><span class="o">.</span><span class="n">scene</span><span class="o">.</span><span class="n">aspectmode</span> <span class="o">=</span> <span class="s2">"data"</span>
|
||||
<span class="n">plot</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="2.-Custom-implicit-function-🧊">2. Custom implicit function 🧊<a class="anchor-link" href="#2.-Custom-implicit-function-🧊">¶</a></h2><p>At the core of neural rendering methods are functions of spatial coordinates called implicit functions, which are used in some kind of rendering process.
|
||||
(Often those functions can additionally take other data as well, such as view direction.)
|
||||
A common rendering process is ray marching over densities and colors provided by an implicit function.
|
||||
In our case, taking samples from a 3D volume grid is a very simple function of spatial coordinates.</p>
|
||||
<p>Here we define our own implicit function, which uses PyTorch3D's existing functionality for sampling from a volume grid.
|
||||
We do this by subclassing <code>ImplicitFunctionBase</code>.
|
||||
We need to register our subclass with a special decorator.
|
||||
We use Python's dataclass annotations for configuring the module.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="nd">@registry</span><span class="o">.</span><span class="n">register</span>
|
||||
<span class="k">class</span> <span class="nc">MyVolumes</span><span class="p">(</span><span class="n">ImplicitFunctionBase</span><span class="p">,</span> <span class="n">torch</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">Module</span><span class="p">):</span>
|
||||
<span class="n">grid_resolution</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">50</span> <span class="c1"># common HWD of volumes, the number of voxels in each direction</span>
|
||||
<span class="n">extent</span><span class="p">:</span> <span class="nb">float</span> <span class="o">=</span> <span class="mf">1.0</span> <span class="c1"># In world coordinates, the volume occupies is [-extent, extent] along each axis</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="c1"># We have to call this explicitly if there are other base classes like Module</span>
|
||||
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">()</span>
|
||||
|
||||
<span class="c1"># We define parameters like other torch.nn.Module objects.</span>
|
||||
<span class="c1"># In this case, both our parameter tensors are trainable; they govern the contents of the volume grid.</span>
|
||||
<span class="n">density</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">full</span><span class="p">((</span><span class="bp">self</span><span class="o">.</span><span class="n">grid_resolution</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">grid_resolution</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">grid_resolution</span><span class="p">),</span> <span class="o">-</span><span class="mf">2.0</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">density</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">Parameter</span><span class="p">(</span><span class="n">density</span><span class="p">)</span>
|
||||
<span class="n">color</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">full</span><span class="p">((</span><span class="mi">3</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">grid_resolution</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">grid_resolution</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">grid_resolution</span><span class="p">),</span> <span class="mf">0.0</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">color</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">Parameter</span><span class="p">(</span><span class="n">color</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">density_activation</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">Softplus</span><span class="p">()</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">forward</span><span class="p">(</span>
|
||||
<span class="bp">self</span><span class="p">,</span>
|
||||
<span class="n">ray_bundle</span><span class="p">:</span> <span class="n">RayBundle</span><span class="p">,</span>
|
||||
<span class="n">fun_viewpool</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
|
||||
<span class="n">global_code</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
|
||||
<span class="p">):</span>
|
||||
<span class="n">densities</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">density_activation</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">density</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">])</span>
|
||||
<span class="n">voxel_size</span> <span class="o">=</span> <span class="mf">2.0</span> <span class="o">*</span> <span class="nb">float</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">extent</span><span class="p">)</span> <span class="o">/</span> <span class="bp">self</span><span class="o">.</span><span class="n">grid_resolution</span>
|
||||
<span class="n">features</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">color</span><span class="o">.</span><span class="n">sigmoid</span><span class="p">()[</span><span class="kc">None</span><span class="p">]</span>
|
||||
|
||||
<span class="c1"># Like other PyTorch3D structures, the actual Volumes object should only exist as long</span>
|
||||
<span class="c1"># as one iteration of training. It is local to this function.</span>
|
||||
|
||||
<span class="n">volume</span> <span class="o">=</span> <span class="n">Volumes</span><span class="p">(</span><span class="n">densities</span><span class="o">=</span><span class="n">densities</span><span class="p">,</span> <span class="n">features</span><span class="o">=</span><span class="n">features</span><span class="p">,</span> <span class="n">voxel_size</span><span class="o">=</span><span class="n">voxel_size</span><span class="p">)</span>
|
||||
<span class="n">sampler</span> <span class="o">=</span> <span class="n">VolumeSampler</span><span class="p">(</span><span class="n">volumes</span><span class="o">=</span><span class="n">volume</span><span class="p">)</span>
|
||||
<span class="n">densities</span><span class="p">,</span> <span class="n">features</span> <span class="o">=</span> <span class="n">sampler</span><span class="p">(</span><span class="n">ray_bundle</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># When an implicit function is used for raymarching, i.e. for MultiPassEmissionAbsorptionRenderer,</span>
|
||||
<span class="c1"># it must return (densities, features, an auxiliary tuple)</span>
|
||||
<span class="k">return</span> <span class="n">densities</span><span class="p">,</span> <span class="n">features</span><span class="p">,</span> <span class="p">{}</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="3.-Construct-the-model-object.">3. Construct the model object.<a class="anchor-link" href="#3.-Construct-the-model-object.">¶</a></h2><p>The main model object in PyTorch3D is <code>GenericModel</code>, which has pluggable components for the major steps, including the renderer and the implicit function(s).
|
||||
There are two ways to construct it which are equivalent here.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">CONSTRUCT_MODEL_FROM_CONFIG</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">CONSTRUCT_MODEL_FROM_CONFIG</span><span class="p">:</span>
|
||||
<span class="c1"># Via a DictConfig - this is how our training loop with hydra works</span>
|
||||
<span class="n">cfg</span> <span class="o">=</span> <span class="n">get_default_args</span><span class="p">(</span><span class="n">GenericModel</span><span class="p">)</span>
|
||||
<span class="n">cfg</span><span class="o">.</span><span class="n">implicit_function_class_type</span> <span class="o">=</span> <span class="s2">"MyVolumes"</span>
|
||||
<span class="n">cfg</span><span class="o">.</span><span class="n">render_image_height</span><span class="o">=</span><span class="n">output_resolution</span>
|
||||
<span class="n">cfg</span><span class="o">.</span><span class="n">render_image_width</span><span class="o">=</span><span class="n">output_resolution</span>
|
||||
<span class="n">cfg</span><span class="o">.</span><span class="n">loss_weights</span><span class="o">=</span><span class="p">{</span><span class="s2">"loss_rgb_huber"</span><span class="p">:</span> <span class="mf">1.0</span><span class="p">}</span>
|
||||
<span class="n">cfg</span><span class="o">.</span><span class="n">tqdm_trigger_threshold</span><span class="o">=</span><span class="mi">19000</span>
|
||||
<span class="n">cfg</span><span class="o">.</span><span class="n">raysampler_AdaptiveRaySampler_args</span><span class="o">.</span><span class="n">scene_extent</span><span class="o">=</span> <span class="mf">4.0</span>
|
||||
<span class="n">gm</span> <span class="o">=</span> <span class="n">GenericModel</span><span class="p">(</span><span class="o">**</span><span class="n">cfg</span><span class="p">)</span>
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="c1"># constructing GenericModel directly</span>
|
||||
<span class="n">expand_args_fields</span><span class="p">(</span><span class="n">GenericModel</span><span class="p">)</span>
|
||||
<span class="n">gm</span> <span class="o">=</span> <span class="n">GenericModel</span><span class="p">(</span>
|
||||
<span class="n">implicit_function_class_type</span><span class="o">=</span><span class="s2">"MyVolumes"</span><span class="p">,</span>
|
||||
<span class="n">render_image_height</span><span class="o">=</span><span class="n">output_resolution</span><span class="p">,</span>
|
||||
<span class="n">render_image_width</span><span class="o">=</span><span class="n">output_resolution</span><span class="p">,</span>
|
||||
<span class="n">loss_weights</span><span class="o">=</span><span class="p">{</span><span class="s2">"loss_rgb_huber"</span><span class="p">:</span> <span class="mf">1.0</span><span class="p">},</span>
|
||||
<span class="n">tqdm_trigger_threshold</span><span class="o">=</span><span class="mi">19000</span><span class="p">,</span>
|
||||
<span class="n">raysampler_AdaptiveRaySampler_args</span> <span class="o">=</span> <span class="p">{</span><span class="s2">"scene_extent"</span><span class="p">:</span> <span class="mf">4.0</span><span class="p">}</span>
|
||||
<span class="p">)</span>
|
||||
|
||||
<span class="c1"># In this case we can get the equivalent DictConfig cfg object to the way gm is configured as follows</span>
|
||||
<span class="n">cfg</span> <span class="o">=</span> <span class="n">OmegaConf</span><span class="o">.</span><span class="n">structured</span><span class="p">(</span><span class="n">gm</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>The default renderer is an emission-absorbtion raymarcher. We keep that default.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># We can display the configuration in use as follows.</span>
|
||||
<span class="n">remove_unused_components</span><span class="p">(</span><span class="n">cfg</span><span class="p">)</span>
|
||||
<span class="n">yaml</span> <span class="o">=</span> <span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">cfg</span><span class="p">,</span> <span class="n">sort_keys</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
<span class="o">%</span><span class="k">page</span> -r yaml
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">device</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">device</span><span class="p">(</span><span class="s2">"cuda:0"</span><span class="p">)</span>
|
||||
<span class="n">gm</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">)</span>
|
||||
<span class="k">assert</span> <span class="nb">next</span><span class="p">(</span><span class="n">gm</span><span class="o">.</span><span class="n">parameters</span><span class="p">())</span><span class="o">.</span><span class="n">is_cuda</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="4.-train-the-model">4. train the model<a class="anchor-link" href="#4.-train-the-model">¶</a></h2>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">train_data_collated</span> <span class="o">=</span> <span class="p">[</span><span class="n">FrameData</span><span class="o">.</span><span class="n">collate</span><span class="p">([</span><span class="n">frame</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">)])</span> <span class="k">for</span> <span class="n">frame</span> <span class="ow">in</span> <span class="n">dataset_map</span><span class="o">.</span><span class="n">train</span><span class="p">]</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">gm</span><span class="o">.</span><span class="n">train</span><span class="p">()</span>
|
||||
<span class="n">optimizer</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">optim</span><span class="o">.</span><span class="n">Adam</span><span class="p">(</span><span class="n">gm</span><span class="o">.</span><span class="n">parameters</span><span class="p">(),</span> <span class="n">lr</span><span class="o">=</span><span class="mf">0.1</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">iterator</span> <span class="o">=</span> <span class="n">tqdm</span><span class="o">.</span><span class="n">tqdm</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">2000</span><span class="p">))</span>
|
||||
<span class="k">for</span> <span class="n">n_batch</span> <span class="ow">in</span> <span class="n">iterator</span><span class="p">:</span>
|
||||
<span class="n">optimizer</span><span class="o">.</span><span class="n">zero_grad</span><span class="p">()</span>
|
||||
|
||||
<span class="n">frame</span> <span class="o">=</span> <span class="n">train_data_collated</span><span class="p">[</span><span class="n">n_batch</span> <span class="o">%</span> <span class="nb">len</span><span class="p">(</span><span class="n">dataset_map</span><span class="o">.</span><span class="n">train</span><span class="p">)]</span>
|
||||
<span class="n">out</span> <span class="o">=</span> <span class="n">gm</span><span class="p">(</span><span class="o">**</span><span class="n">frame</span><span class="p">,</span> <span class="n">evaluation_mode</span><span class="o">=</span><span class="n">EvaluationMode</span><span class="o">.</span><span class="n">TRAINING</span><span class="p">)</span>
|
||||
<span class="n">out</span><span class="p">[</span><span class="s2">"objective"</span><span class="p">]</span><span class="o">.</span><span class="n">backward</span><span class="p">()</span>
|
||||
<span class="k">if</span> <span class="n">n_batch</span> <span class="o">%</span> <span class="mi">100</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
|
||||
<span class="n">iterator</span><span class="o">.</span><span class="n">set_postfix_str</span><span class="p">(</span><span class="sa">f</span><span class="s2">"loss: </span><span class="si">{</span><span class="nb">float</span><span class="p">(</span><span class="n">out</span><span class="p">[</span><span class="s1">'objective'</span><span class="p">])</span><span class="si">:</span><span class="s2">.5f</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
<span class="n">optimizer</span><span class="o">.</span><span class="n">step</span><span class="p">()</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="5.-Evaluate-the-module">5. Evaluate the module<a class="anchor-link" href="#5.-Evaluate-the-module">¶</a></h2><p>We generate complete images from all the viewpoints to see how they look.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">to_numpy_image</span><span class="p">(</span><span class="n">image</span><span class="p">):</span>
|
||||
<span class="c1"># Takes an image of shape (C, H, W) in [0,1], where C=3 or 1</span>
|
||||
<span class="c1"># to a numpy uint image of shape (H, W, 3)</span>
|
||||
<span class="k">return</span> <span class="p">(</span><span class="n">image</span> <span class="o">*</span> <span class="mi">255</span><span class="p">)</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">torch</span><span class="o">.</span><span class="n">uint8</span><span class="p">)</span><span class="o">.</span><span class="n">permute</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span><span class="o">.</span><span class="n">detach</span><span class="p">()</span><span class="o">.</span><span class="n">cpu</span><span class="p">()</span><span class="o">.</span><span class="n">expand</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span><span class="o">.</span><span class="n">numpy</span><span class="p">()</span>
|
||||
<span class="k">def</span> <span class="nf">resize_image</span><span class="p">(</span><span class="n">image</span><span class="p">):</span>
|
||||
<span class="c1"># Takes images of shape (B, C, H, W) to (B, C, output_resolution, output_resolution)</span>
|
||||
<span class="k">return</span> <span class="n">torch</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">functional</span><span class="o">.</span><span class="n">interpolate</span><span class="p">(</span><span class="n">image</span><span class="p">,</span> <span class="n">size</span><span class="o">=</span><span class="p">(</span><span class="n">output_resolution</span><span class="p">,</span> <span class="n">output_resolution</span><span class="p">))</span>
|
||||
|
||||
<span class="n">gm</span><span class="o">.</span><span class="n">eval</span><span class="p">()</span>
|
||||
<span class="n">images</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
<span class="n">expected</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
<span class="n">masks</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
<span class="n">masks_expected</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
<span class="k">for</span> <span class="n">frame</span> <span class="ow">in</span> <span class="n">tqdm</span><span class="o">.</span><span class="n">tqdm</span><span class="p">(</span><span class="n">train_data_collated</span><span class="p">):</span>
|
||||
<span class="k">with</span> <span class="n">torch</span><span class="o">.</span><span class="n">no_grad</span><span class="p">():</span>
|
||||
<span class="n">out</span> <span class="o">=</span> <span class="n">gm</span><span class="p">(</span><span class="o">**</span><span class="n">frame</span><span class="p">,</span> <span class="n">evaluation_mode</span><span class="o">=</span><span class="n">EvaluationMode</span><span class="o">.</span><span class="n">EVALUATION</span><span class="p">)</span>
|
||||
|
||||
<span class="n">image_rgb</span> <span class="o">=</span> <span class="n">to_numpy_image</span><span class="p">(</span><span class="n">out</span><span class="p">[</span><span class="s2">"images_render"</span><span class="p">][</span><span class="mi">0</span><span class="p">])</span>
|
||||
<span class="n">mask</span> <span class="o">=</span> <span class="n">to_numpy_image</span><span class="p">(</span><span class="n">out</span><span class="p">[</span><span class="s2">"masks_render"</span><span class="p">][</span><span class="mi">0</span><span class="p">])</span>
|
||||
<span class="n">expd</span> <span class="o">=</span> <span class="n">to_numpy_image</span><span class="p">(</span><span class="n">resize_image</span><span class="p">(</span><span class="n">frame</span><span class="o">.</span><span class="n">image_rgb</span><span class="p">)[</span><span class="mi">0</span><span class="p">])</span>
|
||||
<span class="n">mask_expected</span> <span class="o">=</span> <span class="n">to_numpy_image</span><span class="p">(</span><span class="n">resize_image</span><span class="p">(</span><span class="n">frame</span><span class="o">.</span><span class="n">fg_probability</span><span class="p">)[</span><span class="mi">0</span><span class="p">])</span>
|
||||
|
||||
<span class="n">images</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">image_rgb</span><span class="p">)</span>
|
||||
<span class="n">masks</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">mask</span><span class="p">)</span>
|
||||
<span class="n">expected</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">expd</span><span class="p">)</span>
|
||||
<span class="n">masks_expected</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">mask_expected</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>We draw a grid showing predicted image and expected image, followed by predicted mask and expected mask, from each viewpoint.
|
||||
This is a grid of four rows of images, wrapped in to several large rows, i.e..
|
||||
<small><center></center></small></p>
|
||||
<pre><code>┌────────┬────────┐ ┌────────┐
|
||||
│pred │pred │ │pred │
|
||||
│image │image │ │image │
|
||||
│1 │2 │ │n │
|
||||
├────────┼────────┤ ├────────┤
|
||||
│expected│expected│ │expected│
|
||||
│image │image │ ... │image │
|
||||
│1 │2 │ │n │
|
||||
├────────┼────────┤ ├────────┤
|
||||
│pred │pred │ │pred │
|
||||
│mask │mask │ │mask │
|
||||
│1 │2 │ │n │
|
||||
├────────┼────────┤ ├────────┤
|
||||
│expected│expected│ │expected│
|
||||
│mask │mask │ │mask │
|
||||
│1 │2 │ │n │
|
||||
├────────┼────────┤ ├────────┤
|
||||
│pred │pred │ │pred │
|
||||
│image │image │ │image │
|
||||
│n+1 │n+1 │ │2n │
|
||||
├────────┼────────┤ ├────────┤
|
||||
│expected│expected│ │expected│
|
||||
│image │image │ ... │image │
|
||||
│n+1 │n+2 │ │2n │
|
||||
├────────┼────────┤ ├────────┤
|
||||
│pred │pred │ │pred │
|
||||
│mask │mask │ │mask │
|
||||
│n+1 │n+2 │ │2n │
|
||||
├────────┼────────┤ ├────────┤
|
||||
│expected│expected│ │expected│
|
||||
│mask │mask │ │mask │
|
||||
│n+1 │n+2 │ │2n │
|
||||
└────────┴────────┘ └────────┘
|
||||
...</code></pre>
|
||||
<p></center></small></p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">images_to_display</span> <span class="o">=</span> <span class="p">[</span><span class="n">images</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">expected</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">masks</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">masks_expected</span><span class="o">.</span><span class="n">copy</span><span class="p">()]</span>
|
||||
<span class="n">n_rows</span> <span class="o">=</span> <span class="mi">4</span>
|
||||
<span class="n">n_images</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">images</span><span class="p">)</span>
|
||||
<span class="n">blank_image</span> <span class="o">=</span> <span class="n">images</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">*</span> <span class="mi">0</span>
|
||||
<span class="n">n_per_row</span> <span class="o">=</span> <span class="mi">1</span><span class="o">+</span><span class="p">(</span><span class="n">n_images</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">//</span><span class="n">n_rows</span>
|
||||
<span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">n_per_row</span><span class="o">*</span><span class="n">n_rows</span> <span class="o">-</span> <span class="n">n_images</span><span class="p">):</span>
|
||||
<span class="k">for</span> <span class="n">group</span> <span class="ow">in</span> <span class="n">images_to_display</span><span class="p">:</span>
|
||||
<span class="n">group</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">blank_image</span><span class="p">)</span>
|
||||
|
||||
<span class="n">images_to_display_listed</span> <span class="o">=</span> <span class="p">[[[</span><span class="n">i</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">j</span><span class="p">]</span> <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="n">images_to_display</span><span class="p">]</span>
|
||||
<span class="n">split</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
<span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">n_rows</span><span class="p">):</span>
|
||||
<span class="k">for</span> <span class="n">group</span> <span class="ow">in</span> <span class="n">images_to_display_listed</span><span class="p">:</span>
|
||||
<span class="n">split</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">group</span><span class="p">[</span><span class="n">row</span><span class="o">*</span><span class="n">n_per_row</span><span class="p">:(</span><span class="n">row</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="n">n_per_row</span><span class="p">])</span>
|
||||
|
||||
<span class="n">Image</span><span class="o">.</span><span class="n">fromarray</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">block</span><span class="p">(</span><span class="n">split</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># Print the maximum channel intensity in the first image.</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">images</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">max</span><span class="p">()</span><span class="o">/</span><span class="mi">255</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">plt</span><span class="o">.</span><span class="n">ioff</span><span class="p">()</span>
|
||||
<span class="n">fig</span><span class="p">,</span> <span class="n">ax</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">(</span><span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">))</span>
|
||||
|
||||
<span class="n">ax</span><span class="o">.</span><span class="n">grid</span><span class="p">(</span><span class="kc">None</span><span class="p">)</span>
|
||||
<span class="n">ims</span> <span class="o">=</span> <span class="p">[[</span><span class="n">ax</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">im</span><span class="p">,</span> <span class="n">animated</span><span class="o">=</span><span class="kc">True</span><span class="p">)]</span> <span class="k">for</span> <span class="n">im</span> <span class="ow">in</span> <span class="n">images</span><span class="p">]</span>
|
||||
<span class="n">ani</span> <span class="o">=</span> <span class="n">animation</span><span class="o">.</span><span class="n">ArtistAnimation</span><span class="p">(</span><span class="n">fig</span><span class="p">,</span> <span class="n">ims</span><span class="p">,</span> <span class="n">interval</span><span class="o">=</span><span class="mi">80</span><span class="p">,</span> <span class="n">blit</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
<span class="n">ani_html</span> <span class="o">=</span> <span class="n">ani</span><span class="o">.</span><span class="n">to_jshtml</span><span class="p">()</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">HTML</span><span class="p">(</span><span class="n">ani_html</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># If you want to see the output of the model with the volume forced to opaque white, run this and re-evaluate</span>
|
||||
<span class="c1"># with torch.no_grad():</span>
|
||||
<span class="c1"># gm._implicit_functions[0]._fn.density.fill_(9.0)</span>
|
||||
<span class="c1"># gm._implicit_functions[0]._fn.color.fill_(9.0)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div></div></div></div></div><footer class="nav-footer" id="footer"><section class="sitemap"><div class="footerSection"><div class="social"><a class="github-button" href="https://github.com/facebookresearch/pytorch3d" data-count-href="https://github.com/facebookresearch/pytorch3d/stargazers" data-show-count="true" data-count-aria-label="# stargazers on GitHub" aria-label="Star PyTorch3D on GitHub">pytorch3d</a></div></div></section><a href="https://opensource.facebook.com/" target="_blank" rel="noreferrer noopener" class="fbOpenSource"><img src="/img/oss_logo.png" alt="Facebook Open Source" width="170" height="45"/></a><section class="copyright">Copyright © 2022 Meta Platforms, Inc<br/>Legal:<a href="https://opensource.facebook.com/legal/privacy/" target="_blank" rel="noreferrer noopener">Privacy</a><a href="https://opensource.facebook.com/legal/terms/" target="_blank" rel="noreferrer noopener">Terms</a></section></footer></div></body></html>
|
||||
669
tutorials/implicitron_volumes/index.html
Normal file
669
tutorials/implicitron_volumes/index.html
Normal file
@ -0,0 +1,669 @@
|
||||
<!DOCTYPE html><html lang=""><head><meta charSet="utf-8"/><meta http-equiv="X-UA-Compatible" content="IE=edge"/><title>PyTorch3D · A library for deep learning with 3D data</title><meta name="viewport" content="width=device-width, initial-scale=1.0"/><meta name="generator" content="Docusaurus"/><meta name="description" content="A library for deep learning with 3D data"/><meta property="og:title" content="PyTorch3D · A library for deep learning with 3D data"/><meta property="og:type" content="website"/><meta property="og:url" content="https://pytorch3d.org/"/><meta property="og:description" content="A library for deep learning with 3D data"/><meta property="og:image" content="https://pytorch3d.org/img/pytorch3dlogoicon.svg"/><meta name="twitter:card" content="summary"/><meta name="twitter:image" content="https://pytorch3d.org/img/pytorch3dlogoicon.svg"/><link rel="shortcut icon" href="/img/pytorch3dfavicon.png"/><link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/default.min.css"/><script>
|
||||
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
|
||||
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
|
||||
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
|
||||
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
var links = coll[i].nextElementSibling.getElementsByTagName('*');
|
||||
if (checkActiveCategory){
|
||||
for (var j = 0; j < links.length; j++) {
|
||||
if (links[j].classList.contains('navListItemActive')){
|
||||
coll[i].nextElementSibling.classList.toggle('hide');
|
||||
coll[i].childNodes[1].classList.toggle('rotate');
|
||||
checkActiveCategory = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
coll[i].addEventListener('click', function() {
|
||||
var arrow = this.childNodes[1];
|
||||
arrow.classList.toggle('rotate');
|
||||
var content = this.nextElementSibling;
|
||||
content.classList.toggle('hide');
|
||||
});
|
||||
}
|
||||
|
||||
document.addEventListener('DOMContentLoaded', function() {
|
||||
createToggler('#navToggler', '#docsNav', 'docsSliderActive');
|
||||
createToggler('#tocToggler', 'body', 'tocActive');
|
||||
|
||||
var headings = document.querySelector('.toc-headings');
|
||||
headings && headings.addEventListener('click', function(event) {
|
||||
var el = event.target;
|
||||
while(el !== headings){
|
||||
if (el.tagName === 'A') {
|
||||
document.body.classList.remove('tocActive');
|
||||
break;
|
||||
} else{
|
||||
el = el.parentNode;
|
||||
}
|
||||
}
|
||||
}, false);
|
||||
|
||||
function createToggler(togglerSelector, targetSelector, className) {
|
||||
var toggler = document.querySelector(togglerSelector);
|
||||
var target = document.querySelector(targetSelector);
|
||||
|
||||
if (!toggler) {
|
||||
return;
|
||||
}
|
||||
|
||||
toggler.onclick = function(event) {
|
||||
event.preventDefault();
|
||||
|
||||
target.classList.toggle(className);
|
||||
};
|
||||
}
|
||||
});
|
||||
</script></nav></div><div class="container mainContainer"><div class="wrapper"><div class="tutorialButtonsWrapper"><div class="tutorialButtonWrapper buttonWrapper"><a class="tutorialButton button" download="" href="https://colab.research.google.com/github/facebookresearch/pytorch3d/blob/stable/docs/tutorials/implicitron_volumes.ipynb" target="_blank"><img class="colabButton" align="left" src="/img/colab_icon.png"/>Run in Google Colab</a></div><div class="tutorialButtonWrapper buttonWrapper"><a class="tutorialButton button" download="" href="/files/implicitron_volumes.ipynb" target="_blank"><svg aria-hidden="true" focusable="false" data-prefix="fas" data-icon="file-download" class="svg-inline--fa fa-file-download fa-w-12" role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 384 512"><path fill="currentColor" d="M224 136V0H24C10.7 0 0 10.7 0 24v464c0 13.3 10.7 24 24 24h336c13.3 0 24-10.7 24-24V160H248c-13.2 0-24-10.8-24-24zm76.45 211.36l-96.42 95.7c-6.65 6.61-17.39 6.61-24.04 0l-96.42-95.7C73.42 337.29 80.54 320 94.82 320H160v-80c0-8.84 7.16-16 16-16h32c8.84 0 16 7.16 16 16v80h65.18c14.28 0 21.4 17.29 11.27 27.36zM377 105L279.1 7c-4.5-4.5-10.6-7-17-7H256v128h128v-6.1c0-6.3-2.5-12.4-7-16.9z"></path></svg>Download Tutorial Jupyter Notebook</a></div><div class="tutorialButtonWrapper buttonWrapper"><a class="tutorialButton button" download="" href="/files/implicitron_volumes.py" target="_blank"><svg aria-hidden="true" focusable="false" data-prefix="fas" data-icon="file-download" class="svg-inline--fa fa-file-download fa-w-12" role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 384 512"><path fill="currentColor" d="M224 136V0H24C10.7 0 0 10.7 0 24v464c0 13.3 10.7 24 24 24h336c13.3 0 24-10.7 24-24V160H248c-13.2 0-24-10.8-24-24zm76.45 211.36l-96.42 95.7c-6.65 6.61-17.39 6.61-24.04 0l-96.42-95.7C73.42 337.29 80.54 320 94.82 320H160v-80c0-8.84 7.16-16 16-16h32c8.84 0 16 7.16 16 16v80h65.18c14.28 0 21.4 17.29 11.27 27.36zM377 105L279.1 7c-4.5-4.5-10.6-7-17-7H256v128h128v-6.1c0-6.3-2.5-12.4-7-16.9z"></path></svg>Download Tutorial Source Code</a></div></div><div class="tutorialBody">
|
||||
<script
|
||||
src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.1.10/require.min.js">
|
||||
</script>
|
||||
<script
|
||||
src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.min.js">
|
||||
</script>
|
||||
<div class="notebook">
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># Copyright (c) Meta Platforms, Inc. and affiliates. All rights reserved.</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h1 id="A-simple-model-using-Implicitron">A simple model using Implicitron<a class="anchor-link" href="#A-simple-model-using-Implicitron">¶</a></h1><p>In this demo, we use the VolumeRenderer from PyTorch3D as a custom implicit function in Implicitron. We will see</p>
|
||||
<ul>
|
||||
<li>some of the main objects in Implicitron</li>
|
||||
<li>how to plug in a custom part of a model</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="0.-Install-and-import-modules">0. Install and import modules<a class="anchor-link" href="#0.-Install-and-import-modules">¶</a></h2><p>Ensure <code>torch</code> and <code>torchvision</code> are installed. If <code>pytorch3d</code> is not installed, install it using the following cell:</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">os</span>
|
||||
<span class="kn">import</span> <span class="nn">sys</span>
|
||||
<span class="kn">import</span> <span class="nn">torch</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">False</span>
|
||||
<span class="k">try</span><span class="p">:</span>
|
||||
<span class="kn">import</span> <span class="nn">pytorch3d</span>
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
<span class="sa">f</span><span class="s2">"py3</span><span class="si">{</span><span class="n">sys</span><span class="o">.</span><span class="n">version_info</span><span class="o">.</span><span class="n">minor</span><span class="si">}</span><span class="s2">_cu"</span><span class="p">,</span>
|
||||
<span class="n">torch</span><span class="o">.</span><span class="n">version</span><span class="o">.</span><span class="n">cuda</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span><span class="s2">""</span><span class="p">),</span>
|
||||
<span class="sa">f</span><span class="s2">"_pyt</span><span class="si">{</span><span class="n">pyt_version_str</span><span class="si">}</span><span class="s2">"</span>
|
||||
<span class="p">])</span>
|
||||
<span class="o">!</span>pip install fvcore iopath
|
||||
<span class="o">!</span>pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/<span class="o">{</span>version_str<span class="o">}</span>/download.html
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="c1"># We try to install PyTorch3D from source.</span>
|
||||
<span class="o">!</span>curl -LO https://github.com/NVIDIA/cub/archive/1.10.0.tar.gz
|
||||
<span class="o">!</span>tar xzf <span class="m">1</span>.10.0.tar.gz
|
||||
<span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">"CUB_HOME"</span><span class="p">]</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">getcwd</span><span class="p">()</span> <span class="o">+</span> <span class="s2">"/cub-1.10.0"</span>
|
||||
<span class="o">!</span>pip install <span class="s1">'git+https://github.com/facebookresearch/pytorch3d.git@stable'</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>Ensure omegaconf and visdom are installed. If not, run this cell. (It should not be necessary to restart the runtime.)</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="o">!</span>pip install omegaconf visdom
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
||||
<span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Tuple</span>
|
||||
|
||||
<span class="kn">import</span> <span class="nn">matplotlib.animation</span> <span class="k">as</span> <span class="nn">animation</span>
|
||||
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
|
||||
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">import</span> <span class="nn">torch</span>
|
||||
<span class="kn">import</span> <span class="nn">tqdm</span>
|
||||
<span class="kn">from</span> <span class="nn">IPython.display</span> <span class="kn">import</span> <span class="n">HTML</span>
|
||||
<span class="kn">from</span> <span class="nn">omegaconf</span> <span class="kn">import</span> <span class="n">OmegaConf</span>
|
||||
<span class="kn">from</span> <span class="nn">PIL</span> <span class="kn">import</span> <span class="n">Image</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.implicitron.dataset.dataset_base</span> <span class="kn">import</span> <span class="n">FrameData</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.implicitron.dataset.rendered_mesh_dataset_map_provider</span> <span class="kn">import</span> <span class="n">RenderedMeshDatasetMapProvider</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.implicitron.models.generic_model</span> <span class="kn">import</span> <span class="n">GenericModel</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.implicitron.models.implicit_function.base</span> <span class="kn">import</span> <span class="n">ImplicitFunctionBase</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.implicitron.models.renderer.base</span> <span class="kn">import</span> <span class="n">EvaluationMode</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.implicitron.tools.config</span> <span class="kn">import</span> <span class="n">expand_args_fields</span><span class="p">,</span> <span class="n">get_default_args</span><span class="p">,</span> <span class="n">registry</span><span class="p">,</span> <span class="n">remove_unused_components</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.renderer</span> <span class="kn">import</span> <span class="n">RayBundle</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.renderer.implicit.renderer</span> <span class="kn">import</span> <span class="n">VolumeSampler</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.structures</span> <span class="kn">import</span> <span class="n">Volumes</span>
|
||||
<span class="kn">from</span> <span class="nn">pytorch3d.vis.plotly_vis</span> <span class="kn">import</span> <span class="n">plot_batch_individually</span><span class="p">,</span> <span class="n">plot_scene</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">output_resolution</span> <span class="o">=</span> <span class="mi">80</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">torch</span><span class="o">.</span><span class="n">set_printoptions</span><span class="p">(</span><span class="n">sci_mode</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="1.-Load-renders-of-a-mesh-(the-cow-mesh)-as-a-dataset">1. Load renders of a mesh (the cow mesh) as a dataset<a class="anchor-link" href="#1.-Load-renders-of-a-mesh-(the-cow-mesh)-as-a-dataset">¶</a></h2><p>A dataset's train, val and test parts in Implicitron are represented as a <code>dataset_map</code>, and provided by an implementation of <code>DatasetMapProvider</code>.
|
||||
<code>RenderedMeshDatasetMapProvider</code> is one which generates a single-scene dataset with only a train component by taking a mesh and rendering it.
|
||||
We use it with the cow mesh.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>If running this notebook using <strong>Google Colab</strong>, run the following cell to fetch the mesh obj and texture files and save it at the path data/cow_mesh.
|
||||
If running locally, the data is already available at the correct path.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="o">!</span>mkdir -p data/cow_mesh
|
||||
<span class="o">!</span>wget -P data/cow_mesh https://dl.fbaipublicfiles.com/pytorch3d/data/cow_mesh/cow.obj
|
||||
<span class="o">!</span>wget -P data/cow_mesh https://dl.fbaipublicfiles.com/pytorch3d/data/cow_mesh/cow.mtl
|
||||
<span class="o">!</span>wget -P data/cow_mesh https://dl.fbaipublicfiles.com/pytorch3d/data/cow_mesh/cow_texture.png
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>If we want to instantiate one of Implicitron's configurable objects, such as <code>RenderedMeshDatasetMapProvider</code>, without using the OmegaConf initialisation (get_default_args), we need to call <code>expand_args_fields</code> on the class first.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">expand_args_fields</span><span class="p">(</span><span class="n">RenderedMeshDatasetMapProvider</span><span class="p">)</span>
|
||||
<span class="n">cow_provider</span> <span class="o">=</span> <span class="n">RenderedMeshDatasetMapProvider</span><span class="p">(</span>
|
||||
<span class="n">data_file</span><span class="o">=</span><span class="s2">"data/cow_mesh/cow.obj"</span><span class="p">,</span>
|
||||
<span class="n">use_point_light</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
<span class="n">resolution</span><span class="o">=</span><span class="n">output_resolution</span><span class="p">,</span>
|
||||
<span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">dataset_map</span> <span class="o">=</span> <span class="n">cow_provider</span><span class="o">.</span><span class="n">get_dataset_map</span><span class="p">()</span>
|
||||
<span class="n">tr_cameras</span> <span class="o">=</span> <span class="p">[</span><span class="n">training_frame</span><span class="o">.</span><span class="n">camera</span> <span class="k">for</span> <span class="n">training_frame</span> <span class="ow">in</span> <span class="n">dataset_map</span><span class="o">.</span><span class="n">train</span><span class="p">]</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># The cameras are all in the XZ plane, in a circle about 2.7 from the origin</span>
|
||||
<span class="n">centers</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">cat</span><span class="p">([</span><span class="n">i</span><span class="o">.</span><span class="n">get_camera_center</span><span class="p">()</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">tr_cameras</span><span class="p">])</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">centers</span><span class="o">.</span><span class="n">min</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">.</span><span class="n">values</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">centers</span><span class="o">.</span><span class="n">max</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">.</span><span class="n">values</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># visualization of the cameras</span>
|
||||
<span class="n">plot</span> <span class="o">=</span> <span class="n">plot_scene</span><span class="p">({</span><span class="s2">"k"</span><span class="p">:</span> <span class="p">{</span><span class="n">i</span><span class="p">:</span> <span class="n">camera</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">camera</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">tr_cameras</span><span class="p">)}},</span> <span class="n">camera_scale</span><span class="o">=</span><span class="mf">0.25</span><span class="p">)</span>
|
||||
<span class="n">plot</span><span class="o">.</span><span class="n">layout</span><span class="o">.</span><span class="n">scene</span><span class="o">.</span><span class="n">aspectmode</span> <span class="o">=</span> <span class="s2">"data"</span>
|
||||
<span class="n">plot</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="2.-Custom-implicit-function-🧊">2. Custom implicit function 🧊<a class="anchor-link" href="#2.-Custom-implicit-function-🧊">¶</a></h2><p>At the core of neural rendering methods are functions of spatial coordinates called implicit functions, which are used in some kind of rendering process.
|
||||
(Often those functions can additionally take other data as well, such as view direction.)
|
||||
A common rendering process is ray marching over densities and colors provided by an implicit function.
|
||||
In our case, taking samples from a 3D volume grid is a very simple function of spatial coordinates.</p>
|
||||
<p>Here we define our own implicit function, which uses PyTorch3D's existing functionality for sampling from a volume grid.
|
||||
We do this by subclassing <code>ImplicitFunctionBase</code>.
|
||||
We need to register our subclass with a special decorator.
|
||||
We use Python's dataclass annotations for configuring the module.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="nd">@registry</span><span class="o">.</span><span class="n">register</span>
|
||||
<span class="k">class</span> <span class="nc">MyVolumes</span><span class="p">(</span><span class="n">ImplicitFunctionBase</span><span class="p">,</span> <span class="n">torch</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">Module</span><span class="p">):</span>
|
||||
<span class="n">grid_resolution</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">50</span> <span class="c1"># common HWD of volumes, the number of voxels in each direction</span>
|
||||
<span class="n">extent</span><span class="p">:</span> <span class="nb">float</span> <span class="o">=</span> <span class="mf">1.0</span> <span class="c1"># In world coordinates, the volume occupies is [-extent, extent] along each axis</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__post_init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
<span class="c1"># We have to call this explicitly if there are other base classes like Module</span>
|
||||
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">()</span>
|
||||
|
||||
<span class="c1"># We define parameters like other torch.nn.Module objects.</span>
|
||||
<span class="c1"># In this case, both our parameter tensors are trainable; they govern the contents of the volume grid.</span>
|
||||
<span class="n">density</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">full</span><span class="p">((</span><span class="bp">self</span><span class="o">.</span><span class="n">grid_resolution</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">grid_resolution</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">grid_resolution</span><span class="p">),</span> <span class="o">-</span><span class="mf">2.0</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">density</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">Parameter</span><span class="p">(</span><span class="n">density</span><span class="p">)</span>
|
||||
<span class="n">color</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">full</span><span class="p">((</span><span class="mi">3</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">grid_resolution</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">grid_resolution</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">grid_resolution</span><span class="p">),</span> <span class="mf">0.0</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">color</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">Parameter</span><span class="p">(</span><span class="n">color</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">density_activation</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">Softplus</span><span class="p">()</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">forward</span><span class="p">(</span>
|
||||
<span class="bp">self</span><span class="p">,</span>
|
||||
<span class="n">ray_bundle</span><span class="p">:</span> <span class="n">RayBundle</span><span class="p">,</span>
|
||||
<span class="n">fun_viewpool</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
|
||||
<span class="n">global_code</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
|
||||
<span class="p">):</span>
|
||||
<span class="n">densities</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">density_activation</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">density</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">])</span>
|
||||
<span class="n">voxel_size</span> <span class="o">=</span> <span class="mf">2.0</span> <span class="o">*</span> <span class="nb">float</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">extent</span><span class="p">)</span> <span class="o">/</span> <span class="bp">self</span><span class="o">.</span><span class="n">grid_resolution</span>
|
||||
<span class="n">features</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">color</span><span class="o">.</span><span class="n">sigmoid</span><span class="p">()[</span><span class="kc">None</span><span class="p">]</span>
|
||||
|
||||
<span class="c1"># Like other PyTorch3D structures, the actual Volumes object should only exist as long</span>
|
||||
<span class="c1"># as one iteration of training. It is local to this function.</span>
|
||||
|
||||
<span class="n">volume</span> <span class="o">=</span> <span class="n">Volumes</span><span class="p">(</span><span class="n">densities</span><span class="o">=</span><span class="n">densities</span><span class="p">,</span> <span class="n">features</span><span class="o">=</span><span class="n">features</span><span class="p">,</span> <span class="n">voxel_size</span><span class="o">=</span><span class="n">voxel_size</span><span class="p">)</span>
|
||||
<span class="n">sampler</span> <span class="o">=</span> <span class="n">VolumeSampler</span><span class="p">(</span><span class="n">volumes</span><span class="o">=</span><span class="n">volume</span><span class="p">)</span>
|
||||
<span class="n">densities</span><span class="p">,</span> <span class="n">features</span> <span class="o">=</span> <span class="n">sampler</span><span class="p">(</span><span class="n">ray_bundle</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># When an implicit function is used for raymarching, i.e. for MultiPassEmissionAbsorptionRenderer,</span>
|
||||
<span class="c1"># it must return (densities, features, an auxiliary tuple)</span>
|
||||
<span class="k">return</span> <span class="n">densities</span><span class="p">,</span> <span class="n">features</span><span class="p">,</span> <span class="p">{}</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="3.-Construct-the-model-object.">3. Construct the model object.<a class="anchor-link" href="#3.-Construct-the-model-object.">¶</a></h2><p>The main model object in PyTorch3D is <code>GenericModel</code>, which has pluggable components for the major steps, including the renderer and the implicit function(s).
|
||||
There are two ways to construct it which are equivalent here.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">CONSTRUCT_MODEL_FROM_CONFIG</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">CONSTRUCT_MODEL_FROM_CONFIG</span><span class="p">:</span>
|
||||
<span class="c1"># Via a DictConfig - this is how our training loop with hydra works</span>
|
||||
<span class="n">cfg</span> <span class="o">=</span> <span class="n">get_default_args</span><span class="p">(</span><span class="n">GenericModel</span><span class="p">)</span>
|
||||
<span class="n">cfg</span><span class="o">.</span><span class="n">implicit_function_class_type</span> <span class="o">=</span> <span class="s2">"MyVolumes"</span>
|
||||
<span class="n">cfg</span><span class="o">.</span><span class="n">render_image_height</span><span class="o">=</span><span class="n">output_resolution</span>
|
||||
<span class="n">cfg</span><span class="o">.</span><span class="n">render_image_width</span><span class="o">=</span><span class="n">output_resolution</span>
|
||||
<span class="n">cfg</span><span class="o">.</span><span class="n">loss_weights</span><span class="o">=</span><span class="p">{</span><span class="s2">"loss_rgb_huber"</span><span class="p">:</span> <span class="mf">1.0</span><span class="p">}</span>
|
||||
<span class="n">cfg</span><span class="o">.</span><span class="n">tqdm_trigger_threshold</span><span class="o">=</span><span class="mi">19000</span>
|
||||
<span class="n">cfg</span><span class="o">.</span><span class="n">raysampler_AdaptiveRaySampler_args</span><span class="o">.</span><span class="n">scene_extent</span><span class="o">=</span> <span class="mf">4.0</span>
|
||||
<span class="n">gm</span> <span class="o">=</span> <span class="n">GenericModel</span><span class="p">(</span><span class="o">**</span><span class="n">cfg</span><span class="p">)</span>
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="c1"># constructing GenericModel directly</span>
|
||||
<span class="n">expand_args_fields</span><span class="p">(</span><span class="n">GenericModel</span><span class="p">)</span>
|
||||
<span class="n">gm</span> <span class="o">=</span> <span class="n">GenericModel</span><span class="p">(</span>
|
||||
<span class="n">implicit_function_class_type</span><span class="o">=</span><span class="s2">"MyVolumes"</span><span class="p">,</span>
|
||||
<span class="n">render_image_height</span><span class="o">=</span><span class="n">output_resolution</span><span class="p">,</span>
|
||||
<span class="n">render_image_width</span><span class="o">=</span><span class="n">output_resolution</span><span class="p">,</span>
|
||||
<span class="n">loss_weights</span><span class="o">=</span><span class="p">{</span><span class="s2">"loss_rgb_huber"</span><span class="p">:</span> <span class="mf">1.0</span><span class="p">},</span>
|
||||
<span class="n">tqdm_trigger_threshold</span><span class="o">=</span><span class="mi">19000</span><span class="p">,</span>
|
||||
<span class="n">raysampler_AdaptiveRaySampler_args</span> <span class="o">=</span> <span class="p">{</span><span class="s2">"scene_extent"</span><span class="p">:</span> <span class="mf">4.0</span><span class="p">}</span>
|
||||
<span class="p">)</span>
|
||||
|
||||
<span class="c1"># In this case we can get the equivalent DictConfig cfg object to the way gm is configured as follows</span>
|
||||
<span class="n">cfg</span> <span class="o">=</span> <span class="n">OmegaConf</span><span class="o">.</span><span class="n">structured</span><span class="p">(</span><span class="n">gm</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>The default renderer is an emission-absorbtion raymarcher. We keep that default.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># We can display the configuration in use as follows.</span>
|
||||
<span class="n">remove_unused_components</span><span class="p">(</span><span class="n">cfg</span><span class="p">)</span>
|
||||
<span class="n">yaml</span> <span class="o">=</span> <span class="n">OmegaConf</span><span class="o">.</span><span class="n">to_yaml</span><span class="p">(</span><span class="n">cfg</span><span class="p">,</span> <span class="n">sort_keys</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
<span class="o">%</span><span class="k">page</span> -r yaml
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">device</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">device</span><span class="p">(</span><span class="s2">"cuda:0"</span><span class="p">)</span>
|
||||
<span class="n">gm</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">)</span>
|
||||
<span class="k">assert</span> <span class="nb">next</span><span class="p">(</span><span class="n">gm</span><span class="o">.</span><span class="n">parameters</span><span class="p">())</span><span class="o">.</span><span class="n">is_cuda</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="4.-train-the-model">4. train the model<a class="anchor-link" href="#4.-train-the-model">¶</a></h2>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">train_data_collated</span> <span class="o">=</span> <span class="p">[</span><span class="n">FrameData</span><span class="o">.</span><span class="n">collate</span><span class="p">([</span><span class="n">frame</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">)])</span> <span class="k">for</span> <span class="n">frame</span> <span class="ow">in</span> <span class="n">dataset_map</span><span class="o">.</span><span class="n">train</span><span class="p">]</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">gm</span><span class="o">.</span><span class="n">train</span><span class="p">()</span>
|
||||
<span class="n">optimizer</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">optim</span><span class="o">.</span><span class="n">Adam</span><span class="p">(</span><span class="n">gm</span><span class="o">.</span><span class="n">parameters</span><span class="p">(),</span> <span class="n">lr</span><span class="o">=</span><span class="mf">0.1</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">iterator</span> <span class="o">=</span> <span class="n">tqdm</span><span class="o">.</span><span class="n">tqdm</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">2000</span><span class="p">))</span>
|
||||
<span class="k">for</span> <span class="n">n_batch</span> <span class="ow">in</span> <span class="n">iterator</span><span class="p">:</span>
|
||||
<span class="n">optimizer</span><span class="o">.</span><span class="n">zero_grad</span><span class="p">()</span>
|
||||
|
||||
<span class="n">frame</span> <span class="o">=</span> <span class="n">train_data_collated</span><span class="p">[</span><span class="n">n_batch</span> <span class="o">%</span> <span class="nb">len</span><span class="p">(</span><span class="n">dataset_map</span><span class="o">.</span><span class="n">train</span><span class="p">)]</span>
|
||||
<span class="n">out</span> <span class="o">=</span> <span class="n">gm</span><span class="p">(</span><span class="o">**</span><span class="n">frame</span><span class="p">,</span> <span class="n">evaluation_mode</span><span class="o">=</span><span class="n">EvaluationMode</span><span class="o">.</span><span class="n">TRAINING</span><span class="p">)</span>
|
||||
<span class="n">out</span><span class="p">[</span><span class="s2">"objective"</span><span class="p">]</span><span class="o">.</span><span class="n">backward</span><span class="p">()</span>
|
||||
<span class="k">if</span> <span class="n">n_batch</span> <span class="o">%</span> <span class="mi">100</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
|
||||
<span class="n">iterator</span><span class="o">.</span><span class="n">set_postfix_str</span><span class="p">(</span><span class="sa">f</span><span class="s2">"loss: </span><span class="si">{</span><span class="nb">float</span><span class="p">(</span><span class="n">out</span><span class="p">[</span><span class="s1">'objective'</span><span class="p">])</span><span class="si">:</span><span class="s2">.5f</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
<span class="n">optimizer</span><span class="o">.</span><span class="n">step</span><span class="p">()</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h2 id="5.-Evaluate-the-module">5. Evaluate the module<a class="anchor-link" href="#5.-Evaluate-the-module">¶</a></h2><p>We generate complete images from all the viewpoints to see how they look.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">to_numpy_image</span><span class="p">(</span><span class="n">image</span><span class="p">):</span>
|
||||
<span class="c1"># Takes an image of shape (C, H, W) in [0,1], where C=3 or 1</span>
|
||||
<span class="c1"># to a numpy uint image of shape (H, W, 3)</span>
|
||||
<span class="k">return</span> <span class="p">(</span><span class="n">image</span> <span class="o">*</span> <span class="mi">255</span><span class="p">)</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">torch</span><span class="o">.</span><span class="n">uint8</span><span class="p">)</span><span class="o">.</span><span class="n">permute</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span><span class="o">.</span><span class="n">detach</span><span class="p">()</span><span class="o">.</span><span class="n">cpu</span><span class="p">()</span><span class="o">.</span><span class="n">expand</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span><span class="o">.</span><span class="n">numpy</span><span class="p">()</span>
|
||||
<span class="k">def</span> <span class="nf">resize_image</span><span class="p">(</span><span class="n">image</span><span class="p">):</span>
|
||||
<span class="c1"># Takes images of shape (B, C, H, W) to (B, C, output_resolution, output_resolution)</span>
|
||||
<span class="k">return</span> <span class="n">torch</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">functional</span><span class="o">.</span><span class="n">interpolate</span><span class="p">(</span><span class="n">image</span><span class="p">,</span> <span class="n">size</span><span class="o">=</span><span class="p">(</span><span class="n">output_resolution</span><span class="p">,</span> <span class="n">output_resolution</span><span class="p">))</span>
|
||||
|
||||
<span class="n">gm</span><span class="o">.</span><span class="n">eval</span><span class="p">()</span>
|
||||
<span class="n">images</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
<span class="n">expected</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
<span class="n">masks</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
<span class="n">masks_expected</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
<span class="k">for</span> <span class="n">frame</span> <span class="ow">in</span> <span class="n">tqdm</span><span class="o">.</span><span class="n">tqdm</span><span class="p">(</span><span class="n">train_data_collated</span><span class="p">):</span>
|
||||
<span class="k">with</span> <span class="n">torch</span><span class="o">.</span><span class="n">no_grad</span><span class="p">():</span>
|
||||
<span class="n">out</span> <span class="o">=</span> <span class="n">gm</span><span class="p">(</span><span class="o">**</span><span class="n">frame</span><span class="p">,</span> <span class="n">evaluation_mode</span><span class="o">=</span><span class="n">EvaluationMode</span><span class="o">.</span><span class="n">EVALUATION</span><span class="p">)</span>
|
||||
|
||||
<span class="n">image_rgb</span> <span class="o">=</span> <span class="n">to_numpy_image</span><span class="p">(</span><span class="n">out</span><span class="p">[</span><span class="s2">"images_render"</span><span class="p">][</span><span class="mi">0</span><span class="p">])</span>
|
||||
<span class="n">mask</span> <span class="o">=</span> <span class="n">to_numpy_image</span><span class="p">(</span><span class="n">out</span><span class="p">[</span><span class="s2">"masks_render"</span><span class="p">][</span><span class="mi">0</span><span class="p">])</span>
|
||||
<span class="n">expd</span> <span class="o">=</span> <span class="n">to_numpy_image</span><span class="p">(</span><span class="n">resize_image</span><span class="p">(</span><span class="n">frame</span><span class="o">.</span><span class="n">image_rgb</span><span class="p">)[</span><span class="mi">0</span><span class="p">])</span>
|
||||
<span class="n">mask_expected</span> <span class="o">=</span> <span class="n">to_numpy_image</span><span class="p">(</span><span class="n">resize_image</span><span class="p">(</span><span class="n">frame</span><span class="o">.</span><span class="n">fg_probability</span><span class="p">)[</span><span class="mi">0</span><span class="p">])</span>
|
||||
|
||||
<span class="n">images</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">image_rgb</span><span class="p">)</span>
|
||||
<span class="n">masks</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">mask</span><span class="p">)</span>
|
||||
<span class="n">expected</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">expd</span><span class="p">)</span>
|
||||
<span class="n">masks_expected</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">mask_expected</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
|
||||
</div>
|
||||
<div class="inner_cell">
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<p>We draw a grid showing predicted image and expected image, followed by predicted mask and expected mask, from each viewpoint.
|
||||
This is a grid of four rows of images, wrapped in to several large rows, i.e..
|
||||
<small><center></center></small></p>
|
||||
<pre><code>┌────────┬────────┐ ┌────────┐
|
||||
│pred │pred │ │pred │
|
||||
│image │image │ │image │
|
||||
│1 │2 │ │n │
|
||||
├────────┼────────┤ ├────────┤
|
||||
│expected│expected│ │expected│
|
||||
│image │image │ ... │image │
|
||||
│1 │2 │ │n │
|
||||
├────────┼────────┤ ├────────┤
|
||||
│pred │pred │ │pred │
|
||||
│mask │mask │ │mask │
|
||||
│1 │2 │ │n │
|
||||
├────────┼────────┤ ├────────┤
|
||||
│expected│expected│ │expected│
|
||||
│mask │mask │ │mask │
|
||||
│1 │2 │ │n │
|
||||
├────────┼────────┤ ├────────┤
|
||||
│pred │pred │ │pred │
|
||||
│image │image │ │image │
|
||||
│n+1 │n+1 │ │2n │
|
||||
├────────┼────────┤ ├────────┤
|
||||
│expected│expected│ │expected│
|
||||
│image │image │ ... │image │
|
||||
│n+1 │n+2 │ │2n │
|
||||
├────────┼────────┤ ├────────┤
|
||||
│pred │pred │ │pred │
|
||||
│mask │mask │ │mask │
|
||||
│n+1 │n+2 │ │2n │
|
||||
├────────┼────────┤ ├────────┤
|
||||
│expected│expected│ │expected│
|
||||
│mask │mask │ │mask │
|
||||
│n+1 │n+2 │ │2n │
|
||||
└────────┴────────┘ └────────┘
|
||||
...</code></pre>
|
||||
<p></center></small></p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">images_to_display</span> <span class="o">=</span> <span class="p">[</span><span class="n">images</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">expected</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">masks</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">masks_expected</span><span class="o">.</span><span class="n">copy</span><span class="p">()]</span>
|
||||
<span class="n">n_rows</span> <span class="o">=</span> <span class="mi">4</span>
|
||||
<span class="n">n_images</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">images</span><span class="p">)</span>
|
||||
<span class="n">blank_image</span> <span class="o">=</span> <span class="n">images</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">*</span> <span class="mi">0</span>
|
||||
<span class="n">n_per_row</span> <span class="o">=</span> <span class="mi">1</span><span class="o">+</span><span class="p">(</span><span class="n">n_images</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">//</span><span class="n">n_rows</span>
|
||||
<span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">n_per_row</span><span class="o">*</span><span class="n">n_rows</span> <span class="o">-</span> <span class="n">n_images</span><span class="p">):</span>
|
||||
<span class="k">for</span> <span class="n">group</span> <span class="ow">in</span> <span class="n">images_to_display</span><span class="p">:</span>
|
||||
<span class="n">group</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">blank_image</span><span class="p">)</span>
|
||||
|
||||
<span class="n">images_to_display_listed</span> <span class="o">=</span> <span class="p">[[[</span><span class="n">i</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">j</span><span class="p">]</span> <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="n">images_to_display</span><span class="p">]</span>
|
||||
<span class="n">split</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
<span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">n_rows</span><span class="p">):</span>
|
||||
<span class="k">for</span> <span class="n">group</span> <span class="ow">in</span> <span class="n">images_to_display_listed</span><span class="p">:</span>
|
||||
<span class="n">split</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">group</span><span class="p">[</span><span class="n">row</span><span class="o">*</span><span class="n">n_per_row</span><span class="p">:(</span><span class="n">row</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="n">n_per_row</span><span class="p">])</span>
|
||||
|
||||
<span class="n">Image</span><span class="o">.</span><span class="n">fromarray</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">block</span><span class="p">(</span><span class="n">split</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># Print the maximum channel intensity in the first image.</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">images</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">max</span><span class="p">()</span><span class="o">/</span><span class="mi">255</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">plt</span><span class="o">.</span><span class="n">ioff</span><span class="p">()</span>
|
||||
<span class="n">fig</span><span class="p">,</span> <span class="n">ax</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">(</span><span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">))</span>
|
||||
|
||||
<span class="n">ax</span><span class="o">.</span><span class="n">grid</span><span class="p">(</span><span class="kc">None</span><span class="p">)</span>
|
||||
<span class="n">ims</span> <span class="o">=</span> <span class="p">[[</span><span class="n">ax</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">im</span><span class="p">,</span> <span class="n">animated</span><span class="o">=</span><span class="kc">True</span><span class="p">)]</span> <span class="k">for</span> <span class="n">im</span> <span class="ow">in</span> <span class="n">images</span><span class="p">]</span>
|
||||
<span class="n">ani</span> <span class="o">=</span> <span class="n">animation</span><span class="o">.</span><span class="n">ArtistAnimation</span><span class="p">(</span><span class="n">fig</span><span class="p">,</span> <span class="n">ims</span><span class="p">,</span> <span class="n">interval</span><span class="o">=</span><span class="mi">80</span><span class="p">,</span> <span class="n">blit</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
<span class="n">ani_html</span> <span class="o">=</span> <span class="n">ani</span><span class="o">.</span><span class="n">to_jshtml</span><span class="p">()</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="n">HTML</span><span class="p">(</span><span class="n">ani_html</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell border-box-sizing code_cell rendered">
|
||||
<div class="input">
|
||||
<div class="prompt input_prompt">In [ ]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight hl-ipython3"><pre><span></span><span class="c1"># If you want to see the output of the model with the volume forced to opaque white, run this and re-evaluate</span>
|
||||
<span class="c1"># with torch.no_grad():</span>
|
||||
<span class="c1"># gm._implicit_functions[0]._fn.density.fill_(9.0)</span>
|
||||
<span class="c1"># gm._implicit_functions[0]._fn.color.fill_(9.0)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div></div></div></div></div><footer class="nav-footer" id="footer"><section class="sitemap"><div class="footerSection"><div class="social"><a class="github-button" href="https://github.com/facebookresearch/pytorch3d" data-count-href="https://github.com/facebookresearch/pytorch3d/stargazers" data-show-count="true" data-count-aria-label="# stargazers on GitHub" aria-label="Star PyTorch3D on GitHub">pytorch3d</a></div></div></section><a href="https://opensource.facebook.com/" target="_blank" rel="noreferrer noopener" class="fbOpenSource"><img src="/img/oss_logo.png" alt="Facebook Open Source" width="170" height="45"/></a><section class="copyright">Copyright © 2022 Meta Platforms, Inc<br/>Legal:<a href="https://opensource.facebook.com/legal/privacy/" target="_blank" rel="noreferrer noopener">Privacy</a><a href="https://opensource.facebook.com/legal/terms/" target="_blank" rel="noreferrer noopener">Terms</a></section></footer></div></body></html>
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -124,7 +124,7 @@
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -124,7 +124,7 @@
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -127,7 +127,7 @@ In this tutorial, we provide an example of using DensePose data in PyTorch3D.</p
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -127,7 +127,7 @@ In this tutorial, we provide an example of using DensePose data in PyTorch3D.</p
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -126,7 +126,7 @@
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
|
||||
ga('create', 'UA-157376881-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div></div></section></div><script>
|
||||
</script><script type="text/javascript" src="https://buttons.github.io/buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/pytorch3dfavicon.png" alt="PyTorch3D"/><h2 class="headerTitleWithLogo">PyTorch3D</h2></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/docs/why_pytorch3d" target="_self">Docs</a></li><li class=""><a href="/tutorials" target="_self">Tutorials</a></li><li class=""><a href="https://pytorch3d.readthedocs.io/" target="_self">API</a></li><li class=""><a href="https://github.com/facebookresearch/pytorch3d" target="_self">GitHub</a></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container docsNavContainer" id="docsNav"><nav class="toc"><div class="toggleNav"><section class="navWrapper wrapper"><div class="navBreadcrumb wrapper"><div class="navToggle" id="navToggler"><div class="hamburger-menu"><div class="line1"></div><div class="line2"></div><div class="line3"></div></div></div><h2><i>›</i><span></span></h2><div class="tocToggler" id="tocToggler"><i class="icon-toc"></i></div></div><div class="navGroups"><div class="navGroup"><h3 class="navGroupCategoryTitle">Tutorials</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/">Overview</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">3D operators</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/deform_source_mesh_to_target_mesh">Fit Mesh</a></li><li class="navListItem"><a class="navItem" href="/tutorials/bundle_adjustment">Bundle Adjustment</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Rendering</h3><ul class=""><li class="navListItem navListItemActive"><a class="navItem" href="/tutorials/render_textured_meshes">Render Textured Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_densepose">Render DensePose Meshes</a></li><li class="navListItem"><a class="navItem" href="/tutorials/render_colored_points">Render Colored Pointclouds</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_mesh">Fit a Mesh with Texture via Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/camera_position_optimization_with_differentiable_rendering">Camera Position Optimization with Differentiable Rendering</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_textured_volume">Fit a volume via raymarching</a></li><li class="navListItem"><a class="navItem" href="/tutorials/fit_simple_neural_radiance_field">Fit a simplified NeRF via raymarching</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Dataloaders</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/dataloaders_ShapeNetCore_R2N2">Data loaders for ShapeNetCore and R2N2</a></li></ul></div><div class="navGroup"><h3 class="navGroupCategoryTitle">Implicitron</h3><ul class=""><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_volumes">Training a custom volumes function with implicitron</a></li><li class="navListItem"><a class="navItem" href="/tutorials/implicitron_config_system">Implicitron config system deep dive</a></li></ul></div></div></section></div><script>
|
||||
var coll = document.getElementsByClassName('collapsible');
|
||||
var checkActiveCategory = true;
|
||||
for (var i = 0; i < coll.length; i++) {
|
||||
@ -126,7 +126,7 @@
|
||||
<span class="k">except</span> <span class="ne">ModuleNotFoundError</span><span class="p">:</span>
|
||||
<span class="n">need_pytorch3d</span><span class="o">=</span><span class="kc">True</span>
|
||||
<span class="k">if</span> <span class="n">need_pytorch3d</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.10."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"1.12."</span><span class="p">)</span> <span class="ow">and</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"linux"</span><span class="p">):</span>
|
||||
<span class="c1"># We try to install PyTorch3D via a released wheel.</span>
|
||||
<span class="n">pyt_version_str</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">__version__</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"+"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span>
|
||||
<span class="n">version_str</span><span class="o">=</span><span class="s2">""</span><span class="o">.</span><span class="n">join</span><span class="p">([</span>
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user