I Built a CoPilot+ AI PC (without Windows)

Jeff Geerling
4 Jun 202412:49

TLDRJeff Geerling builds a custom CoPilot PC using a Raspberry Pi AI kit, which includes an M.2 HAT and a 13 TOPS Hailo NPU for machine learning tasks. He explores its capabilities in real-time video processing, compares it to other AI solutions, and attempts to surpass Microsoft's 40 TOPS requirement by combining multiple AI accelerators. The video highlights practical applications of AI in robotics, safety monitoring, and machine vision, emphasizing the kit's potential for developers and enthusiasts.

Takeaways

  • 🤖 The CoPilot PC is a custom Raspberry Pi AI PC, not related to Microsoft's Copilot.
  • 🚀 The PC uses a $70 Raspberry Pi AI kit, which includes an M.2 HAT and a 13 TOPS Hailo NPU for AI processing.
  • 🔢 The goal is to achieve over 40 TOPS of neural compute, which is more than Apple's M4 or Snapdragon X.
  • 👷‍♂️ The AI kit is designed for practical applications like machine vision, not just hype.
  • 📷 Raspberry Pi is focusing on real-time video applications, such as object detection and pose estimation.
  • 🌾 The technology can be used in various industries like farming, manufacturing, and traffic planning for real-world problem-solving.
  • 🤖 The Raspberry Pi does not have its own custom NPU and relies on add-on devices for AI capabilities.
  • 💡 The AI kit is limited by the Pi's power and RAM, making it unsuitable for tasks like model training or running large language models.
  • 🔌 There are challenges with powering multiple AI accelerators from the Pi's PCI Express port, which is limited to 5 watts.
  • 🛠️ The video demonstrates experimenting with various PCI Express devices to increase the neural compute power of the Raspberry Pi.
  • 🔮 The future of Raspberry Pi and AI might include more integrated solutions and easier access to AI capabilities for developers.

Q & A

  • What is the CoPilot PC mentioned in the script?

    -The CoPilot PC is a custom Raspberry Pi AI PC built by the author, not to be confused with Microsoft's Copilot. It is designed to perform AI tasks with a focus on machine learning and neural computing.

  • Why did the author build the CoPilot PC instead of using Microsoft's Copilot?

    -The author built the CoPilot PC because they had reservations about the way Microsoft marketed their Copilot product, although the specific reasons are not detailed in the script.

  • What is the neural compute requirement for Microsoft's Copilot?

    -Microsoft states that you need at least 40 TOPS of neural compute for their Copilot.

  • What is the neural compute performance of the CoPilot PC built by the author?

    -The CoPilot PC initially has 13 TOPS, but the author plans to enhance it to achieve 47 TOPS [or actually 51 TOPS], which is faster than Apple's M4 or the Snapdragon X.

  • What is included in the $70 Raspberry Pi AI kit mentioned in the script?

    -The $70 Raspberry Pi AI kit includes an M.2 HAT and a 13 TOPS Hailo NPU, which functions as a machine learning accelerator for the Raspberry Pi.

  • How does the Coral TPU compare to the Hailo-8L in terms of TOPS and efficiency?

    -The Coral TPU offers 2 TOPS at around 2 TOPS per watt efficiency, while the Hailo-8L provides 13 TOPS with an efficiency of 8 TOPS per watt.

  • What is the main purpose of the AI kit according to the script?

    -The AI kit is designed for accelerating generic machine learning tasks such as object detection, pose estimation, and image segmentation, with a focus on practical applications like machine vision.

  • Why did the author mention Sony's factory in the script?

    -The author mentioned Sony's factory to illustrate a real-world application of AI-powered cameras, which are used for inspection and quality assurance in the production of Raspberry Pis.

  • What are some of the practical applications of the CoPilot PC mentioned in the script?

    -Some practical applications include real-time video processing for security, traffic planning, factory production line monitoring, and farming for spoilage detection.

  • Why did the author attempt to connect multiple AI accelerators to the Raspberry Pi?

    -The author attempted to connect multiple AI accelerators to surpass the 40 TOPS neural compute threshold of Microsoft's Copilot, aiming to demonstrate the potential of the Raspberry Pi with various AI accelerators.

  • What challenges did the author encounter when trying to use multiple AI accelerators with the Raspberry Pi?

    -The author faced challenges such as power limitations of the Raspberry Pi's PCI Express port, which is meant to supply up to 5 watts, and potential issues with multiple PCI Express switches in line.

  • What is the conclusion the author reaches regarding the use of multiple AI accelerators with the Raspberry Pi?

    -The author concludes that while it's technically possible to connect multiple AI accelerators to the Raspberry Pi, it may be more practical to use a single, more powerful NPU if higher performance is required.

Outlines

00:00

🤖 Custom CoPilot PC with Raspberry Pi AI Kit

The script introduces a custom CoPilot PC, a Raspberry Pi AI system, contrasting it with Microsoft's Copilot. The creator criticizes Microsoft's marketing and explains the technical specifications of their setup, which includes a Raspberry Pi with a Hailo NPU for 13 TOPS of neural compute, surpassing the 40 TOPS minimum required by Microsoft's Copilot. The video promises to increase this to 47 TOPS (or 51), outperforming Apple's M4 and Snapdragon X. The script discusses the inclusion of an M.2 HAT and the efficiency of the Hailo-8L NPU, comparing it to the Coral TPU. It emphasizes the practical applications of AI in machine vision and the potential of the Raspberry Pi AI kit for real-world problem-solving, such as in surveillance, traffic planning, and factory monitoring, while also touching on the limitations of AI hype and the channel's preference for practical technology solutions.

05:04

🔍 Exploring AI Capabilities and Limitations with Raspberry Pi

This paragraph delves into the capabilities of the Raspberry Pi AI kit, showcasing its real-time performance in tasks like object identification, pose estimation, and image segmentation. It discusses the benefits of having a dedicated AI coprocessor for handling complex machine learning tasks efficiently, with lower power consumption and less strain on the CPU. The script also mentions the challenges of training models on the Pi due to limited RAM and the time-consuming process compared to modern GPUs. It explores the potential for integrating the AI kit with projects like Frigate and StereoPi and acknowledges the kit's limitations for certain applications, such as image generation or large language models, suggesting that while it may not be suitable for everyone, it offers a valuable alternative for those working in machine vision and robotics.

10:09

🚀 Pushing the Limits with Multiple AI Coprocessors

The final paragraph describes an ambitious attempt to surpass Microsoft's neural compute requirement by connecting multiple AI coprocessors to the Raspberry Pi. It details the process of setting up various PCI Express devices, including the Pineboard's HatBrick! Commander and multiple Hailo and Coral NPUs, to increase the system's TOPS. The script highlights the technical challenges encountered, such as power limitations and potential software incompatibilities, and the unsuccessful attempt to boot the system with all devices connected. It concludes with a reflection on the feasibility of such a setup, suggesting that while it's an interesting experiment, it may be more practical to opt for a more powerful NPU for those who truly require high computational power. The video ends with thoughts on the future of AI integration in devices and the potential for even more specialized and accessible AI solutions.

Mindmap

Keywords

💡CoPilot PC

The term 'CoPilot PC' refers to a custom-built AI personal computer that is not associated with Microsoft's Copilot. In the script, the creator expresses a personal preference for the name 'Co-Pilot' over 'Copilot' due to marketing concerns with Microsoft's product. The CoPilot PC in the video is essentially a Raspberry Pi AI kit, which is a small, low-power device capable of running AI models, and is used to demonstrate the capabilities of neural computing on a budget-friendly platform.

💡TOPS

TOPS stands for trillion operations per second and is a unit of measurement for the performance of AI processors. In the context of the video, the creator discusses the computational power of the CoPilot PC, stating that it has 13 TOPS and later aims to increase this to 47 TOPS [or 51], which is a significant figure in the realm of AI processing. TOPS is crucial for understanding the processing capabilities of AI hardware and is used to compare the performance of different AI systems.

💡Raspberry Pi

Raspberry Pi is a series of small single-board computers that have gained popularity for their versatility and affordability. In the video, the creator uses a Raspberry Pi as the foundation for the CoPilot PC, highlighting its potential for AI applications despite its modest hardware specifications. The Raspberry Pi is central to the video's theme of demonstrating AI capabilities on a budget-friendly platform.

💡AI Kit

The 'AI Kit' mentioned in the script refers to a $70 kit that includes an M.2 HAT and a 13 TOPS Hailo NPU, designed to enhance the AI capabilities of a Raspberry Pi. The kit is described as a 'machine learning jetpack' for the Pi, emphasizing its role in boosting the device's neural computing power for tasks such as object detection and image segmentation.

💡Hailo NPU

The Hailo NPU, or Neural Processing Unit, is a specialized hardware accelerator designed for machine learning tasks. In the video, the Hailo-8L NPU is highlighted for its ability to deliver 13 TOPS of performance at 8 TOPS per watt efficiency. The Hailo NPU is a key component of the CoPilot PC, enabling it to perform AI tasks with greater efficiency and speed compared to the Raspberry Pi's CPU alone.

💡Machine Learning

Machine learning is a subset of artificial intelligence that involves the development of algorithms and statistical models that enable computers to perform tasks without explicit programming. In the context of the video, machine learning is synonymous with AI and is the core technology behind the CoPilot PC's capabilities. The script discusses how the Raspberry Pi AI kit can be used for machine learning tasks such as object detection and image segmentation.

💡Coral TPU

The Coral TPU, or Tensor Processing Unit, is a hardware accelerator designed by Google to speed up machine learning tasks. In the video, the creator mentions using a Coral TPU with a Frigate server for detecting cars, people, and birds. The Coral TPU is an example of the type of hardware that can be used to enhance the AI capabilities of devices like the Raspberry Pi.

💡Object Detection

Object detection is a machine learning task that involves identifying and locating objects in images or video frames. The script describes how the CoPilot PC, with its AI kit, can perform object detection tasks, such as identifying cars or people in real-time video feeds. This capability is a practical application of AI and is showcased as a key feature of the CoPilot PC.

💡Neural Compute

Neural compute refers to the computational power required to run neural networks, which are a type of machine learning model inspired by the human brain. In the video, the creator discusses the need for at least 40 TOPS of neural compute for Microsoft's Copilot and attempts to surpass this with the CoPilot PC. Neural compute is a critical concept for understanding the performance capabilities of AI hardware.

💡Machine Vision

Machine vision is the ability of a machine to 'see' and understand visual information from the world, typically using cameras and image processing algorithms. In the video, machine vision is highlighted as a primary use case for the CoPilot PC, with applications ranging from factory inspections to agricultural monitoring. The script emphasizes the practical benefits of machine vision in solving real-world problems.

💡Pose Estimation

Pose estimation is a computer vision technique used to determine the position and orientation of objects within an image or video. In the context of the video, pose estimation is mentioned as a capability of the CoPilot PC, allowing for real-time analysis of body movements and gestures. This technology can be used to develop gesture-based applications and enhance safety features in vehicles, such as predicting pedestrian behavior.

💡Image Segmentation

Image segmentation is the process of partitioning an image into multiple segments or regions to simplify analysis or enhance processing. In the video, the creator discusses how the CoPilot PC can perform image segmentation, which is used in applications like the iPhone's portrait mode to create depth effects by distinguishing between the subject and the background.

💡HatBrick! Commander

The HatBrick! Commander is a PCI Express board mentioned in the script, designed to adapt one port into two, allowing for the connection of multiple devices to the Raspberry Pi. In the video, the creator uses this board in an attempt to increase the neural compute power of the CoPilot PC by connecting additional AI accelerators, demonstrating the potential for expanding the capabilities of the system.

Highlights

Introduction of a custom CoPilot PC, a Raspberry Pi AI kit, which is different from Microsoft's Copilot.

The CoPilot PC is designed to have 13 TOPS of neural compute, aiming to surpass the 40 TOPS required by Microsoft's Copilot.

The Raspberry Pi AI kit includes an M.2 HAT and a 13 TOPS Hailo NPU, enhancing the Pi's machine learning capabilities.

Comparison of the Coral TPU's 2 TOPS per watt efficiency to the Hailo-8L's 8 TOPS per watt.

The AI kit's purpose is for real-world applications like machine vision, not just buzzword compliance.

Raspberry Pi's preannouncement of an AI camera at Embedded World, indicating a focus on practical applications.

The versatility of the AI kit for accelerating machine learning tasks such as object detection and image segmentation.

Examples of real-world applications, including factory inspections and traffic planning with AI-powered cameras.

Discussion on why Raspberry Pi doesn't integrate an NPU into their main chip and the role of Broadcom in this decision.

Comparison of different approaches to AI hardware, from custom NPUs to AI extensions on the CPU.

The limitations of using a GPU for AI tasks on low-power devices like the Raspberry Pi.

Demonstration of the AI kit's performance提升 with YOLOv5, showing real-time object detection with low CPU usage.

The potential for pose estimation and behavior prediction in automotive safety with the AI kit.

The AI kit's capability for image segmentation, enabling features like iPhone's portrait mode.

Hailo's model zoo and Raspberry Pi's documentation efforts to support users in leveraging the AI kit.

The unsuitability of the AI kit for tasks like image generation and large language model processing due to RAM constraints.

An ambitious attempt to surpass Microsoft's neural compute requirement by connecting multiple AI accelerators to the Pi.

The challenges faced when trying to power and utilize multiple high TOPS devices on a single Raspberry Pi.

Recommendations for those who need higher neural compute performance to consider more powerful NPUs.

Final thoughts on the AI kit's niche appeal and its potential impact on machine vision and robotics.

Casual Browsing

I Built A SECRET ROOM IN SCHOOL

2024-04-14 14:35:00

👉Cài đặt Stable Diffusion AI trên PC Windows thành công 100% | Setups SD on PC

2024-04-04 05:45:00

I Built My Dream Studio

2024-05-17 17:40:01

I Built My Dream Office (Workspace Tour 2023)

2024-05-17 17:50:01

I Built an AI Art Business in 16 Minutes (Step-by-Step Guide)

2024-04-02 21:00:01