An LLM journey speed run: Going from Hugging Face to Vertex AI

The video begins with Skander Hannachi and Rajesh Thallam introducing themselves as part of Google Cloud's Vertex AI applied engineering team. They express excitement about discussing the augmentation of a Gemini-based generative AI solution with open source models from Hugging Face, specifically focusing on integrating a time series forecasting model into a large language model. The conversation aims to guide viewers on utilizing the Vertex AI platform to develop generative AI applications, with a spotlight on a recent use case involving Google's open source time series foundation model, TimesFM, launched on Hugging Face. The hosts walk through the steps of integrating TimesFM into a generative AI application, emphasizing the importance of choosing the right models for specific use cases and the benefits of integrating Vertex AI with Hugging Face.

Skander delves into the thought process behind building generative AI applications using the Vertex AI platform. He outlines the challenges of designing a Gemini-based solution, such as deciding whether to fine-tune a model or use a retrieval augmented generation approach. The discussion highlights the importance of not relying on a single model for all tasks, but instead considering a variety of models available through Vertex AI and Hugging Face. Rajesh then distinguishes between Google AI Studio and Vertex AI Studio, explaining that Google AI Studio is a prototyping tool, while Vertex AI is an end-to-end machine learning platform. He emphasizes the full data control, enterprise security, and data governance provided by Vertex AI Studio. Skander further elaborates on the Vertex AI platform's components, including the Vertex Model Garden, Model Builder, and Agent Builder, catering to a wide range of users from business analysts to ML researchers.

The focus shifts to a concrete industry example, where Skander discusses the challenges of managing product catalogs in e-commerce and how the Vertex AI team has addressed these with a solution that uses Vertex Gemini to enhance product catalog data. The solution streamlines the process of generating and enhancing new product content using a category detection module, a filtering process for detailed attributes, and Gemini to generate detailed product descriptions. Skander describes the system design as a single path state machine, where each part of the user journey represents a specific state. He then explores the idea of adding demand forecasting capabilities to the system, noting the distinction between listing new products and forecasting demand, and introduces the concept of an intent detection layer to determine the merchant's goals.

Skander introduces TimesFM, Google's newly released open source time series foundational model, which is trained on synthetic time series data and leverages a decoder-only architecture similar to large language models. He explains that TimesFM is capable of zero-shot learning for various time series forecasting tasks, allowing it to generate forecasts on the fly without prior training on historical data sets. This capability is particularly beneficial for merchants who do not have the operational overhead for traditional forecasting tools. Skander also mentions that TimesFM has been benchmarked against dedicated models and has shown superior performance in terms of accuracy and the ability to generate forecasts instantly.

Rajesh takes over to demonstrate the practical steps of integrating the TimesFM model from Hugging Face with Vertex AI. He outlines the user journey for reviewing item performance by building an agent using foundation models from Vertex AI and Hugging Face. Rajesh explains the concept of a generative AI agent, which is an application that attempts to complete complex tasks or user queries by understanding intent and acting on it with the help of external systems or APIs. He discusses the components needed to build performant Gen AI applications, including models, tools, orchestration, and runtime. Rajesh then walks through the process of deploying the TimesFM model on Vertex AI, defining Python functions as tools, and using LangChain templates to create executable tools for the model to perform information retrieval or transactions via API calls.

The video script details the process of deploying the TimesFM model from Hugging Face onto Vertex AI's prediction endpoint. It includes steps for data preparation, such as downloading a dataset from Kaggle, transforming it for time series forecasting, and uploading it to BigQuery. Rajesh demonstrates how to deploy the model using Vertex AI SDK, create a custom predictor, and test it locally before pushing it to the model registry and deploying it to a Vertex AI endpoint. The script also covers defining functions as tools to interact with BigQuery and call the TimesFM model, and using these tools to fetch data and generate forecasts. The process of defining a LangChain agent, binding the model with these tools, and deploying the agent on Vertex AI's Reasoning Engine is also explained, highlighting the ability to productionize and scale the agent for business-critical applications.

In the concluding part of the video script, Skander and Rajesh summarize the key takeaways from their discussion. They emphasize that there is no single foundational model that can handle all tasks, and the importance of using a variety of models to augment Gemini-based solutions. They also highlight the ease of deploying models from Hugging Face directly to Vertex AI and the potential of combining generative AI with predictive models to unlock the full potential of Gen AI solutions. The hosts conclude by encouraging viewers to explore function calling and Reasoning Engine for building scalable Gen AI agents and invite them to seek additional information on the use case presented.