Zuckerberg Unveils AI Projects to Build Metaverse Future

Meta CEO Mark Zuckerberg said the company is working on AI research to generate worlds through speech, improve how people chat to voice assistants, and translate between languages. Speaking at the company's live-streamed event, he said that the key to unlocking a lot of advances is AI.

Zuckerberg said Meta was working on a new class of generative AI models that will allow people to describe a world and generate aspects of it. Zuckerberg showcased an AI concept called Builder Bot, where he appeared as a legless 3D avatar on an island and gave speech commands to create a beach and then add clouds, trees, and even a picnic blanket.

"As we advance this technology further, you'll be able to create nuanced worlds to explore and share experiences with others, with just your voice," said Zuckerberg. He did not set a timeline for these advancements or give more details on how Builder Bot works. Meta CEO said the company is working on an AI solution that will allow people to have more natural conversations with voice assistants. He added that the company's Project CAIRaoke was a fully end-to-end neural model for building on-device assistants.

Zuckerberg also announced that Meta was working on a universal speech translator that should provide an instant translation across all languages. The company previously set a goal for its AI system to translate all written languages. Meta is preparing for how AI could interpret and predict the types of interactions that would occur in the metaverse, by working on self-supervised learning where AI is given raw data rather than being trained on lots of labeled data.

Zuckerberg said Meta is also developing egocentric data, which involves seeing worlds from a first-person perspective. He said it had brought together a global consortium of 13 universities and labs to work on the largest ever egocentric dataset, called Ego4D.