Google Unveils New AI Assistant Capable of “Seeing” Surrounding Space and Recognizing Objects: Introducing Project Astra at Google I/O 2024

At the annual Google I/O developers conference, Demis Hassabis, head of Google's AI division Deep Mind, unveiled an early version of what the company refers to as a "universal AI assistant." This algorithm, codenamed Project Astra, is a multimodal AI assistant that operates in real-time, capable of "seeing" the environment, recognizing objects, and assisting in various tasks.

Hassabis shared, “I've had this idea for a long time. We will have this universal assistant. It’s multimodal; it's always with you... This assistant is simply useful. You will get used to having it always nearby when needed,” during his presentation.

In conjunction with the announcement, Google released a short video demonstrating some capabilities of Project Astra’s early version. In the clip, a Google employee at their London office activates the AI assistant and requests it to notify her when it "sees" something capable of producing sound. As she rotates her smartphone, it captures a speaker on the table, and the algorithm promptly notifies her. She then asks the assistant to describe the colored chalks in a glass on the table, to which the AI responds that they can be used to create “colorful creations.”

The camera then pans to a section of computer code displayed on a monitor, and when the employee inquires about the specific function of that code segment, Project Astra provides an accurate response almost instantaneously. Furthermore, the AI assistant identified the location of the Google office based on the landscape viewed through a window and accomplished several other tasks in what appeared to be real-time, impressing those present.

According to Hassabis, Project Astra is significantly closer to how a real-time AI assistant should operate compared to previous similar products. The algorithm is built on the large language model Gemini 1.5 Pro, currently Google’s most powerful neural network. However, to enhance the performance of the AI assistant, Google had to optimize it to increase processing speed and reduce response time delays. Hassabis noted that developers have spent the last six months working specifically on speeding up the algorithm, including optimizing the entire infrastructure associated with it.

Looking to the future, it is expected that Project Astra will not only be available on smartphones but also on smart glasses equipped with cameras. As this is still an early version of the AI assistant, exact timelines for its mass rollout have not been announced.