How to Set Up and Use Shap-E for 3D Model Generation: A Complete Guide

What is Shape-E?

Shap-E, a groundbreaking model developed by OpenAI, revolutionizes the world of 3D applications by producing an array of 3D objects using text or images as input. This innovative technology is freely accessible on GitHub, allowing users to run it seamlessly on their local systems without requiring an OpenAI API key or an internet connection. One of the most remarkable attributes of Shap-E is its versatility: the generated 3D objects can be opened in various software such as Microsoft Paint 3D or converted into STL files for 3D printing. With its unique capabilities, Shap-E is redefining the landscape of text-to-3D and image-to-3D generation, opening up new creative possibilities for AI applications.

What We Are Going to Do

In this tutorial, we will learn how to create a Notebook in Google Colab, set up and utilize OpenAI's Shape-E model to generate 3D models, and customize them using Blender Studio.

Prerequisites

• Download Blender Studio compatible with your operating system from Blender.org.

Let's Start!

Step 1 - Setting Up the Project

Begin by creating a new Notebook in Google Colab:

1. Visit Google Colab and create a new Notebook.

Step 2 - Clone the Repository

Next, we need to clone the Shap-E repository into our Google Colab Notebook:

1. Enter the directory and install the necessary requirements.

Step 3 - Implementing and Cooking the 3D Models

1. Add a new code cell. Import all the necessary libraries and click Run or use CMD/CTRL + Enter to execute.
2. Add another code cell to set the device to CUDA if available; otherwise, set it to CPU. Run the cell.
3. Add a new code cell to load the models and weights. Run the cell and be patient, as this process may take some time - approximately 5 minutes depending on your internet connection.
4. Add another code cell to generate the 3D models. Clicking Run will trigger the generation process, which takes about 22 seconds with a batch size of 1.
5. Add another new code cell for rendering the models. Use render_mode = 'nerf' for Neural Radiance Fields (NeRF) or change to render_mode = 'stf' for Style Transfer Function (STF). Execute the cell and enjoy watching the models rotate.
6. Finally, save the 3D models as .ply and .obj files. Remember, we will use the .obj file for customization in Blender Studio later. Download the .obj file to your local machine after refreshing the Files tab.

Step 4 - Customizing the 3D Models in Blender Studio

1. Open Blender Studio and create a new project. Delete the default cube.
2. Click File > Import > Wavefront (.obj). Select the .obj file you downloaded from Google Colab.
3. The 3D model should appear at the center of your Blender workspace, looking amazing! You can customize the model further; for example, I colored the topping pink and the donut brown for demonstration purposes.

Conclusion

In this tutorial, we learned how to set up and use the Shape-E model using Google Colab and explored how to customize the generated 3D model in Blender Studio. Now it's your turn to explore your newly acquired skills and create your unique 3D art. As you experiment, you may discover limitations and want to develop your own Shap-E app.

Participate in AI hackathons to challenge yourself and potentially win exciting prizes! Join the community of AI enthusiasts and learn more about AI and its remarkable potential.

Thank you for reading this tutorial!