AI company Runway, on Monday, August 31, unveiled a new AI system that could potentially change how we interact with software. The new system, Solaris, is described as an ‘Interface World Model’. The idea here is simple; instead of developers building an app first and then deciding what users can do inside it, Solaris can generate the interface itself as a user interacts with it.

According to Runway, Solaris is the first model in its new family of AI systems called Interface World Models. “Solaris starts with a question: what happens when an operating system generates apps and websites as you use them?” the company wrote in a blog post.

As of today, most apps and websites work roughly in a similar manner. A designer creates how an interface should look. Later, developers turn this design into code using HTML, CSS, JavaScript, etc. Eventually, the final product comes with predefined buttons, menus, screens, and interactions. So far, this approach has been reliable and fast, and it also puts limits on what users can do. In this process, developers expect possible interactions and programme them in advance.

Story continues below this ad

How is Solaris different?

Solaris takes a different approach. In place of converting visual design into code, Solaris can generate the visual interface directly. The model generates the screen frame by frame, at the same time respond to what a user does. In simple words, the interface is not simply displaying a program; the generated visual environment becomes the interface. This is more like an app that behaves more like the real world.

In its blog, Runway gives several examples of what this could look like. Picture an online clothing store. Instead of sifting through product pages, users could now enter a virtual showroom. They could pick a shirt, drag and drop it onto an image of themselves and see how it looks on them. In another case, a room-design application can let users simply tell the AI system to move a table or change the colour of a sofa, and it would respond instead of manually selecting options from menus. Another example is making salad; instead of clicking buttons named ‘add tomato’ or ‘add lettuce’, users could simply drag the ingredients into a bowl and watch the environment respond.

Essentially, this makes software feel less like going through numerous web pages and more like interacting in a simulated environment.

How is it different from an AI chatbot?

While most AI chatbots are good at following instructions, they generally respond with text, images or links. Runway’s Solaris has been designed to respond visually, and that too continuously. If a user demands an interactive demonstration of combustion, it could generate the visual scenario and allow the user to experiment inside it.

Story continues below this ad

This may be useful for AI agents. Solaris could help AI agents understand constantly changing interfaces, helping them learn to adapt rather than simply memorise where buttons are.

When it comes to underlying function, Solaris builds on Runway’s Gen-4.5 video model and its earlier GWM-1 world model. It generates the interface one frame at a time. The company said that user actions such as clicks and drags become inputs that help determine what happens in the next frame. Meanwhile, a separate language model helps it understand what the user wants and decides how the scene should change. Solaris later handles the visual rendering.

According to Runway, it had to make the system fast enough for real-time interaction. It changed the video model to generate frames sequentially and reduced the number of processing steps required to create each frame.

The AI system is still experimental and comes with some limitations. Stable, legible text remains a major problem because Solaris is built on video-generation technology, while software interfaces depend heavily on text. Another concern is accuracy, as an AI-generated interface could look convincing on the surface while showing something incorrect. Runway said that it is working on measures to keep Solaris grounded in verified product information and other reference material. Apart from this, it also needs to maintain consistency during long interactions and eventually work with accessibility tech such as screen readers.

Story continues below this ad

The bigger idea here is that instead of opening a separate app for shopping, learning, designing, or booking, an AI-powered operating system could generate the interface needed by the user on the go. The same underlying system could generate a different interface for every person and every task. Now, this does not mean traditional apps are going away. Solaris is still an early research system, and Runway is working with partners toward a public launch.