This AI System Can Turn an Image Into 3D Interactive Scenes

World Labs, the artificial intelligence (AI) startup, unveiled its first AI system on Monday. The currently unnamed AI system can generate interactive 3D worlds using an image input. These generated worlds turn the 2D visual asset into explorable 3D scenes where users can navigate using a keyboard and mouse. The AI system is currently in preview and has not been made public. However, the startup, which was founded by the computer scientist Fei-Fei Li, stated that it is working to release the full version soon.

World Labs Unveils AI System Capable of Generating 3D Worlds

In a blog post, the San Francisco-based startup showcased the capabilities of its AI model. World Labs highlighted that most generative AI tools today can create 2D content such as images or videos. Although some AI firms do generate 3D models from 2D images or text prompts, the scope is pretty limited. In recent times, only Google DeepMind has unveiled an AI model that generates unique 2D video game levels.

However, based on the interactive assets shared by the startup, the unnamed AI system’s capabilities surpass the generative capabilities seen so far. Put short, the company claims users can add any image that depicts a scene as input, and the AI model can generate a 3D interactive version of the scene. This means users can move forward, backwards, and side-to-side and explore the generated area.

The AI model does not only generate three-dimensional renders of the objects in the image, it also creates unseen details such as new alleyways, ceiling art, new objects and more from scratch. World Labs claim that apart from the initial image, everything is generated by the AI system.

Additionally, the generated scenes can also be modified. Users can change camera angles, depth, and zoom as well as add 3D effects to the background as well as the objects in the foreground.

World Labs’ AI system can also be integrated with other AI tools. The startup said this will allow creators to first generate the starting image using a familiar text-to-image generator such as Ideogram, Dall-E, or Midjourney and then create a 3D world using the startup’s tool. The AI firm is currently working with a few creators to test the AI system’s capabilities and its 3D-native generative AI workflow.

As of now, the AI system is not available in the public domain, and the startup highlighted that it is still working on it to improve the size and the fidelity of the generated worlds. However, interested individuals can join the company’s waitlist to be informed when the AI system is released.