Elon Musk’s XAI is joining the race to build “universal models” for playing video games

Stay informed with free updates

Simply sign up artificial intelligence myFT Digest – delivered straight to your inbox.

Elon Musk’s xAI company seeks to build so-called universal models, joining competitors such as Meta and Google in the race to develop artificial intelligence systems that can navigate and model physical environments.

The San Francisco-based startup hired specialists from Nvidia over the summer to work on this next generation Amnesty International Models, which train on videos and data from robots to understand the real world.

Universal models can push AI capabilities beyond the large, text-trained language models that underpin popular AI tools like ChatGPT and XAI’s Grok.

Two people familiar with the plans said that the company is building global models with the aim of applying them in games, where they can be used to create interactive 3D environments. One person added that it could be applied to artificial intelligence systems for robots.

Khai Nvidia has hired Zeeshan Patel and Ethan He, two Nvidia AI researchers with global modeling expertise. Nvidia has pioneered the development of this technology with its Omniverse platform, which creates and runs simulations.

Some technology groups have high expectations of global models, which could unleash uses of AI beyond software and computers in physical products such as humanoid robots. Nvidia last month He told the Financial Times The potential market for global models could be approximately the size of the current global economy.

Musk said in a post on the X website that xAI will launch “a wonderful game produced by artificial intelligence before the end of next year,” confirming the goal that the billionaire set last year.

On Tuesday, xAI launched its latest image and video generation model, which it said has “huge upgrades” and is free to use.

Existing video generation models, such as Sora from OpenAI, generate image frames for videos by predicting patterns learned from training data.

Universal models will be a major advance because they will have a causal understanding of physics and how things interact in different environments in real time.

The company is advertising for image and video generation technical staff to join its “holistic team,” which “creates magical AI experiences that go beyond text, enabling content to be understood and generated across different modalities, including images, video and audio.”

Salaries for these jobs range from $180,000 to $440,000. It also has an open position for a “video game tutor,” who will train Grok in video game production and enable “users to explore AI-assisted game design,” for $45 to $100 per hour.

Musk follows other leading AI labs, such as Google and Meta, which are also working on these systems.

However, global modeling remains a major technical challenge. Finding enough data to simulate the real world and training such models has proven difficult and expensive.

Michael Doss, Head of Publishing, Larian Studios, Who develops video games Baldur’s Gate 3AI cannot solve the gaming industry’s “big problem”, which is “leadership (and) vision,” he said on Channel X this week.

He added that the industry needs “not more mathematically produced and psychologically trained loops of play (but) more expressions of the worlds people engage with, or want to engage with.”

xAI, Patel and He did not respond to requests for comment.

Additional reporting from Hannah Murphy in San Francisco