Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions content/install-guides/llm-jetson-xavier.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
---
additional_search_terms:
- cloud
- deploy

layout: installtoolsall
minutes_to_complete: 15
author: Fidel Makatia
multi_install: false
multitool_install_part: false
official_docs: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
test_images:
- ubuntu:latest
test_maintenance: true
title: AWS CLI
tool_install: true
weight: 1
---

[AWS CLI](https://docs.aws.amazon.com/cli/index.html) is a cross-platform command-line tool that can be installed on development computers. The AWS Command Line Interface (AWS CLI) is a unified tool that provides a consistent interface for interacting with all parts of Amazon Web Services.

It is available for a variety of operating systems and Linux distributions, supports the Arm architecture and has multiple ways to install it.

## What should I do before installing AWS CLI?

This article provides quick solutions to install the latest version of AWS CLI, version 2, for Ubuntu on Arm.

Confirm you are using an Arm computer with 64-bit Linux by running:

```bash { target="ubuntu:latest" }
uname -m
```

The output should be:

```output
aarch64
```

If you see a different result, you are not using an Arm computer running 64-bit Linux.

## How do I download and install AWS CLI version 2?

The easiest way to install the latest version of the AWS CLI for Ubuntu on Arm is to download and run the installer from AWS.

Before starting, install `unzip`:

```bash { target="ubuntu:latest" }
sudo apt update
sudo apt install unzip -y
```

Download the zip file with `curl`, extract the installer, and run it.

```bash { target="ubuntu:latest" }
curl "https://awscli.amazonaws.com/awscli-exe-linux-aarch64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install --update
```

Confirm the CLI version 2 is available by invoking the `aws` command to print the version.

```bash { target="ubuntu:latest" }
aws --version
```

Review [AWS CLI v2 is now generally available](https://aws.amazon.com/blogs/developer/aws-cli-v2-is-now-generally-available/) to review the new features in version 2.

You now have the latest version of the AWS CLI installed. Follow [this guide](/install-guides/aws_access_keys/) to generate and configure access keys needed to use the AWS CLI.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
---
title: Build a Privacy-First LLM Smart Home on Raspberry Pi 5

minutes_to_complete: 45

who_is_this_for: Anyone who wants a private, cloud-free smart home powered by GenAI on Arm

learning_objectives:
- "Deploy a local Large Language Model (LLM) for smart home control"
- "Integrate natural language processing with GPIO control"
- "Build and run everything on Arm-based single-board computers (no cloud required)"
- "Optimize for speed, privacy, and offline operation"
- "Create an interactive web dashboard for smart home control"
prerequisites:
- "Basic Python knowledge"
- "A text editor (e.g., VS Code, Sublime, Notepad++)"
- "An Arm-based single board computer (e.g., Raspberry Pi 5 with at least 8GB RAM)"
- "Basic electronic components such as LEDs, sensors, and relays"
- "Basic understanding of GPIO pins and electronics"

author: "Fidel Makatia Omusilibwa"

### Tags
skilllevels: "Introductory"
subjects: "ML"
armips:
- "Arm Cortex A"
tools_software_languages:
- "Python"
- "Ollama"
- "gpiozero"
- "lgpio"
- "FastAPI"
- "VS Code or your preferred code editor"
- "Raspberry Pi OS (64-bit)"
operatingsystems:
- "Windows , Linux, MacOS"

further_reading:
- resource:
title: "Raspberry Pi 5 Smart Home Assistant with EdgeAI"
link: "https://github.com/fidel-makatia/EdgeAI_Raspi5"
type: "source"
- resource:
title: "Ollama Python/JavaScript Libraries"
link: "https://ollama.com/blog/python-javascript-libraries"
type: "documentation"
- resource:
title: "gpiozero Documentation for Raspberry Pi"
link: "https://gpiozero.readthedocs.io/en/stable/"
type: "documentation"
- resource:
title: "lgpio Library for Raspberry Pi 5"
link: "https://abyz.me.uk/lg/lgpio.html"
type: "documentation"
- resource:
title: "Raspberry Pi 5 Official Documentation"
link: "https://www.raspberrypi.org/documentation/computers/raspberry-pi.html"
type: "documentation"
- resource:
title: "Ollama Model Library"
link: "https://ollama.com/library"
type: "documentation"

### FIXED, DO NOT MODIFY
# ================================================================================
weight: 1 # _index.md always has weight of 1 to order correctly
layout: "learningpathall" # All files under learning paths have this same wrapper
learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content.
---
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
# ================================================================================
# FIXED, DO NOT MODIFY THIS FILE
# ================================================================================
weight: 21 # Set to always be larger than the content in this path to be at the end of the navigation.
title: "Next Steps" # Always the same, html page title.
layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing.
---
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
---
title: Overview
weight: 2

### FIXED, DO NOT MODIFY
layout: learningpathall
---

## Overview

Control your smart home using natural language with no cloud connection, no third-party servers, and no compromises on privacy. With rapid advances in Generative AI and the power of Arm Cortex-A processors, you can now run large language models (LLMs) directly in your home on the Raspberry Pi 5.

You will create a fully local, privacy-first smart home system that leverages the strengths of Arm Cortex-A architecture. The system achieves 15+ tokens per second inference speeds using optimized models like TinyLlama and Qwen, while maintaining the energy efficiency that makes Arm processors ideal for always-on applications.

## Why Arm Cortex-A for Edge AI?

The Raspberry Pi 5's Arm Cortex-A76 processor excels at high-performance computing tasks like AI inference through:

- Superscalar architecture that executes multiple instructions simultaneously
- Advanced SIMD with 128-bit NEON units for matrix operations
- Multi-level cache hierarchy that reduces memory latency
- Thermal efficiency that maintains performance in compact form factors

Your Arm-powered smart home processes everything locally, providing:

- **Total Privacy**: Conversations and routines never leave your device
- **Lightning Speed**: Sub-100ms response times with optimized processing
- **Rock-Solid Reliability**: Operation continues when internet connectivity fails
- **Unlimited Customization**: Complete control over AI models and automations
- **Future-Proof Performance**: Continued optimization through Arm's roadmap

## Performance Benchmarks on Raspberry Pi 5

| Model | Tokens/Sec | Avg Latency (ms) | Performance Rating |
| ------------------- | ---------- | ---------------- | -------------------- |
| qwen:0.5b | 17.0 | 8,217 | ⭐⭐⭐⭐⭐ Excellent |
| tinyllama:1.1b | 12.3 | 9,429 | ⭐⭐⭐⭐⭐ Excellent |
| deepseek-coder:1.3b | 7.3 | 22,503 | ⭐⭐⭐⭐ Very Good |
| gemma2:2b | 4.1 | 23,758 | ⭐⭐⭐⭐ Very Good |
| deepseek-r1:7b | 1.6 | 64,797 | ⭐⭐⭐ Good |

Performance insights:

- Qwen 0.5B and TinyLlama 1.1B provide optimal speed for real-time smart home commands
- DeepSeek-Coder 1.3B and Gemma2 2B handle complex automation tasks effectively
- DeepSeek-R1 7B offers advanced reasoning capabilities with acceptable latency

## Arm Ecosystem Advantages

The Raspberry Pi 5 benefits from the extensive Arm developer ecosystem:

- Optimized compilers including GCC and Clang with Arm-specific enhancements
- Native libraries such as gpiozero and lgpio optimized for Raspberry Pi
- Community support from millions of developers contributing Arm-optimized code
- Long-term support through Arm's commitment to backward compatibility
- Industrial adoption with the same architecture powering smartphones, servers, and embedded systems

## Supported Arm-Powered Devices

This learning path focuses on the Raspberry Pi 5, but you can adapt the concepts and code to other Arm-powered devices:

### Recommended Platforms

**Raspberry Pi 5 (Primary Focus)**

- Arm Cortex-A76 quad-core @ 2.4GHz
- Up to 16GB RAM for larger models
- Native lgpio support with optimized GPIO performance

**Raspberry Pi 4**

- Arm Cortex-A72 quad-core @ 1.8GHz
- 8GB RAM maximum, suitable for smaller models
- Proven compatibility with gpiozero ecosystem

### Compatibility Requirements

Any Arm device can potentially run this project with:

- Arm Cortex-A processor
- Minimum 4GB RAM (8GB+ recommended)
- GPIO pins for hardware control
- Python 3.8+ support
- Ability to run Ollama

If your Arm device supports Linux, Python, and has GPIO capabilities, you can adapt this learning path to your specific hardware.

## What You Will Build

By completing this learning path, your Raspberry Pi 5 will run:

- Ultra-fast AI processing with 15+ tokens/second performance
- Complete GPIO control for lights, fans, locks, and sensors via gpiozero + lgpio
- Modern web dashboard with FastAPI-powered interface optimized for mobile
- NEON-accelerated performance using custom ARM assembly for critical paths
- Zero-cloud architecture with everything running locally on your Arm processor
- Intelligent automation with scene-based control using natural language

You will build a smart home system that demonstrates why Arm processors represent the future of edge computing, combining efficiency, performance, and complete privacy control.
Loading