Proactive AI Agents on Smart Glasses

Kaden Pierce from the MIT Media Lab envisions smart glasses as the next big technology after smartphones, but with a caveat: they must offer more than just porting smartphone applications to a wearable format. He emphasizes the need for a new kind of application that is contextual, proactive, and intelligent. These apps would understand the user's environment and offer assistance without being explicitly asked, thus providing a 10x to 100x improvement over traditional smartphone usage. Kaden illustrates this with an example of a smart glasses user landing in a new city late at night, suggesting that the glasses could automatically assist with tasks like booking a ride to a hotel, based on the context of the user's situation.

The speaker delves into the concept of proactive AI agents, explaining that they should be able to listen to the user's surroundings and understand context to provide real-time assistance. He shares an anecdote about a smart glasses prototype that performed an online search to provide information about the caffeine content in dark chocolate during a conversation. This example demonstrates how AI could be integrated into daily life to offer immediate, context-aware assistance, enhancing user experience by preempting their needs and providing information without manual queries.

Kaden discusses the importance of context in making AI applications truly useful. He argues that future apps need to be aware of the user's situation, such as their location, recent activities, and ongoing conversations, to provide relevant assistance. He provides examples of how smart glasses could enhance everyday activities like shopping in a mall or planning outdoor activities by overlaying useful information based on the context of the user's current needs and environment.

The speaker reflects on the evolution of technology and its interaction with humans. He points out that as AI becomes more integrated into our lives, there are instances where we may not know how to leverage it to solve problems because we're not aware of its full capabilities. Kaden suggests that proactive AI agents could act like a human assistant, taking context into account to provide help that we might not have thought to ask for, thus enhancing our ability to utilize technology to its fullest potential.

Kaden introduces the concept of proactive AI in language learning glasses, which could assist users by providing real-time translations and explanations without the need for manual input. He also discusses 'Convos Scope,' a system designed to augment conversations by introducing proactive AI agents that can answer questions, generate new ideas, and even play the role of a devil's advocate to prevent groupthink. These agents aim to make conversations more creative, problem-solving, and connected.

The speaker outlines the technical and conceptual shifts required to make proactive AI a reality. He suggests that the current model of apps running on demand needs to evolve to accommodate always-listening, context-aware systems. Kaden proposes a semantic layer or natural language interface for operating systems and APIs that would allow apps to describe their utility and context of use, enabling the operating system to decide when and how to present information to the user.

In his closing remarks, Kaden Pierce reflects on the potential of proactive AI agents on wearables and the significance of the current technological landscape in making this vision a reality. He anticipates the convergence of lightweight, wearable head-up displays and advanced AI, which will enable a new paradigm of human-computer interaction. Kaden expresses his enthusiasm for the development of human intelligence augmentation technology, which he believes will allow us to learn, understand, and achieve more than ever before.