GENERATING 3D MODEL ASSETS USING AI FOR SIGHTLAB VR AND VIZARD USING 3D CSM

3D CSM (Common Sense Machines) generates 3D models from either images or videos. You can use their Discord server to directly connect with your Midjourney account (a popular AI image creation tool) or directly upload images. For best results it is suggested that you have images that have a solid color background (can prompt for this if using an image generator like Midjourney, Leonardo.ai, Dall-E 3 or others).

‍

• Start at this page https://3d.csm.ai/

• Click on “login” and then “sign up now”.
• Choose from either the free plan or one of the paid ones. The main difference is the paid versions give you faster response times, Pro 3D model quality, can do video to 3D (free one just does image to 3D) and the “Creative Pro” version keeps your models private instead of sharing publicly.

‍

Once logged in, click on “Upload Image” at the top right. You can then either directly upload an image to generate a 3D model or use the Discord server.

‍

For direct upload, click “Upload Image”.

‍

For Discord, click on the “Discord” link and then click “Verify”.

‍

Click on the “Image to 3D” channel.

‍

You can directly type in a prompt using /get3d or enter the image URL and then the model version (1.0 or 1.1).

‍

Or click on the “+” symbol or drag an image in to upload.

‍

Click on “Open in Browser” or right click to get the image url.

‍

If directly entering a prompt you can click on 360 preview and then “Get 3D Preview”.

‍

Click “Generate Mesh” to generate a model and then your model will begin creation (this can take quite some time to finish (around 2–3 hours on average).

Once the generation of the model is done, click “Download” to download the model as a .glb and bring it into Vizard or SightLab VR.

‍

Download “Refined Mesh”.

‍

‍

To generate an avatar, you will need to upload your own image of a T-Pose as a reference to an AI image generator such as Midjourney, generating an avatar image on a white background, using 3D CSM to generate the mesh and then auto-rig that mesh using an auto-rigger like Mixamo or Accurig.

‍

Once the model is downloaded you can bring it into Inspector (Tools-Inspector or use the shortcut in the SightLab root folder) and do things such as check the scale and adjust lighting.

‍

You can experiment with what looks the best in your scene as far as lighting. For some cases you can just toggle off lighting on the mesh to see the model as it looks from the image, or you can add various lights (point, spot and directional).

‍

With lighting off:

‍

‍

With point light:

‍

‍

For generating 360 images using AI see our article on using Skybox Blockade Labs.

‍

Other tools for generating 3D models from text prompts using AI include the following:

‍

• https://huggingface.co/spaces/shariqfarooq/ZoeDepth
• https://skybox.blockadelabs.com/
• https://www.genmo.ai/
• https://www.meshy.ai/
• https://huggingface.co/spaces/hysts/Shap-E
• https://spline.design/ai
• Luma Labs
• https://3dfy.ai/
• More to come soon…

‍

For more information on getting assets into Vizard and SightLab see this page in the documentation.

‍

For more information on getting access to Vizard or SightLab contact sales@worldviz.com.