Using Open Source AI Models with Hugging Face | Build Free AI Models

Alara, a PhD candidate at Imperial College London and a former machine learning engineer at Hugging Face, introduces a code along focused on using open source AI models with the Hugging Face project. She provides an overview of Hugging Face, an AI company dedicated to simplifying access to state-of-the-art AI research through open source tools and libraries. The core of their ecosystem is 'The Hub,' a platform for discovering models and datasets, similar to GitHub. Alara emphasizes the ease of use of Hugging Face's ecosystem, including the ability to clone, update, and store large files for free. She also mentions the Transformers library and other resources like the Hugging Face blog, tutorials, and demo spaces. The session aims to teach how to use these tools to create custom machine learning pipelines for tasks like multilingual text translation and image captioning.

The setup process for the code along is outlined, which includes creating a Hugging Face account and obtaining a token for uploading datasets. Alara instructs on installing necessary libraries such as Transformers, datasets, and the Hugging Face Hub library, ensuring compatibility with the latest versions. She demonstrates how to import these libraries and sets up the environment for the coding session, highlighting the importance of restarting the kernel after installation to ensure the correct versions are loaded.

Alara explains how to load pretrained models from the Hugging Face Hub using the Transformers library. She details the integration of the library with the Hub, which allows for the storage of model checkpoints and configuration files. The use of 'auto' classes like AutoModel and AutoTokenizer is introduced as a convenient way to load models and their corresponding preprocessing tools by simply providing the repository name. The process involves using the 'from_pretrained' method to automatically handle the model architecture and loading. Alara also touches on the structure of model repositories on the Hub and the use of local paths or URLs for model loading.

The discussion shifts to tokenizers, essential for natural language processing (NLP) models to convert text into a mathematical format. Alara demonstrates how to print and understand the tokenizer object, highlighting its capabilities like padding, truncation, and handling unknown words. She then proceeds to load a pretrained model, explaining the warnings that may appear due to the separation of base model classes and task-specific classes in Transformers. Alara advises referring to the model configuration for clarity on the model's architecture and task, using the 'model_config' attribute to retrieve this information.

Alara corrects the model loading process by using the appropriate 'AutoModel' class for the specific task, which in this case is 'RobertaForSequenceClassification'. She contrasts this with the base 'AutoModel' class, emphasizing the importance of using the correct class to avoid loading warnings. The summary also includes a brief on how to use explicit class names for loading models and preprocessors, which provides more control and understanding of the model being used.

The focus is on building NLP pipelines for text translation using the flan-T5 base model by Google. Alara explains that this model, part of the Transformers library, can perform multilingual translation among other tasks. She guides through the process of preparing input text for the model, including specifying source and target languages. The use of tokenizers to preprocess the input text into token IDs and attention masks is detailed, setting the stage for model inference.

Alara demonstrates how to perform inference using the model by disabling unnecessary computations with 'torch.no_grad' and running the prediction using the 'model.generate' method. She explains the output structure, which consists of token IDs, and the need to decode these back into a human-readable format using the tokenizer's 'decode' method. The process results in a translated text, showcasing the model's capabilities in language translation.

Alara introduces the Hugging Face 'datasets' library, which simplifies the process of loading datasets from the Hub. She uses a fashion image captioning dataset as an example, demonstrating how to load and explore the dataset's structure. The summary includes instructions on how to visualize images from the dataset and the flexibility of downloading specific subsets like train, validation, or test.

The session focuses on building an image captioning pipeline using the BLIP model by Salesforce. Alara contrasts BLIP with language models, highlighting its multimodal nature and the need to import a specific processor for conditional generation. She guides through the process of initializing the preprocessor and model, preprocessing the image data, and performing inference to generate captions. The summary also covers decoding the token IDs to produce human-readable captions.

Alara demonstrates creating a mapping function to preprocess and generate new captions for all samples in the dataset. She explains the utility of the 'map' method in the datasets library, which applies a function to all data samples. The 'replace_caption' function is detailed, which preprocesses the image, generates token IDs, decodes them into a caption, and overwrites the original caption. The process is applied to the entire dataset, showcasing the efficiency of batch processing.

The final task involves pushing the updated dataset to the Hugging Face Hub. Alara guides through the process of logging into the Hub using a token and demonstrates how to use the 'push_to_hub' method to upload the dataset. She emphasizes the ease of sharing and collaborating on datasets, models, and other resources on the Hub, encouraging further exploration and experimentation.