The Current State of Generative 3D (Aug 2023)
<h1>What Does the Future Hold?</h1>
<p>At Mirage, we’ve kept our platform at the edge of generative 3D starting with <a href="https://github.com/NasirKhalid24/CLIP-Mesh" rel="noopener ugc nofollow" target="_blank">CLIP-Mesh by Nasir</a> → <a href="https://github.com/nv-tlabs/GET3D" rel="noopener ugc nofollow" target="_blank">GET3D by NVIDIA</a> → <a href="https://github.com/ashawkey/stable-dreamfusion" rel="noopener ugc nofollow" target="_blank">Stable Dreamfusion by Google (implemented by ashawkey)</a> → <a href="https://openai.com/research/point-e" rel="noopener ugc nofollow" target="_blank">Point-E by OpenAI</a> → <a href="https://github.com/openai/shap-e" rel="noopener ugc nofollow" target="_blank">Shap-E by OpenAI</a>.</p>
<p>We’ve spent time optimizing these open-source repos to produce usable game assets in multiple formats (recently GLTF).</p>
<p>Today, we are open-sourcing Mirage3D. This repo is built upon the fantastic work of the folks mentioned above and is optimized to create GLB files easily.</p>
<h2><a href="https://github.com/MirageML/Mirage3D?source=post_page-----b389bd7a65c6--------------------------------" rel="noopener ugc nofollow" target="_blank">GitHub — MirageML/Mirage3D: Open-Source Implementations of 3D Diffusion Models Optimized for GLB…</a></h2>
<h3><a href="https://github.com/MirageML/Mirage3D?source=post_page-----b389bd7a65c6--------------------------------" rel="noopener ugc nofollow" target="_blank">Open-Source Implementations of 3D Diffusion Models Optimized for GLB Output — GitHub — MirageML/Mirage3D: Open-Source…</a></h3>
<p><a href="https://github.com/MirageML/Mirage3D?source=post_page-----b389bd7a65c6--------------------------------" rel="noopener ugc nofollow" target="_blank">github.com</a></p>
<p>Try out the repo yourself and consider contributing to help create a single source of truth for open-source generative 3D models, optimized to create usable 3D assets!</p>
<h1>The Problem</h1>
<p>Generative 3D is progressing slower than other modalities (audio, video, image, etc) due to the lack of 3D data online.</p>
<p>Unlike image models, it seems that closed-source models are going to have a significant quality improvement over open-source models.</p>
<h1>Open-Source is Far Behind</h1>
<p>There are a few notable open-source repositories that have been released recently.</p>
<p><a href="https://github.com/openai/shap-e" rel="noopener ugc nofollow" target="_blank"><strong>Shap-E</strong></a><strong>:</strong> Very efficient model from OpenAI that is able to generate meshes and NeRFs from text or image. Takes about 1 minute to generate.</p>
<p><a href="https://blog.mirageml.com/the-current-state-of-generative-3d-aug-2023-b389bd7a65c6"><strong>Learn More</strong></a></p>