diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 8cc8425334..3ca6f547e6 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -4,6 +4,16 @@ This project is a collection of "Learning Paths" (long-form tutorials) and "inst Assume the audience is made up of Arm software developers. Bias all information toward Arm platforms. For Linux, assume systems are aarch64 architecture and not x86. Readers also use macOS and Windows on Arm systems, and assume Arm architecture where relevant. +## Highest priority rules + +- One Learning Path must own one clear developer task +- Install guides are for installation and verification only +- Every Learning Path `_index.md` must include a `description` field +- Use task-led titles, introductions, and metadata +- Do not use placeholder alt text such as `alt-txt` +- Prefer Arm-native solutions and Arm-specific framing +- Avoid hype, duplication, and vague summaries + ## Project structure The key directories are: @@ -42,20 +52,21 @@ The `/content` directory is the primary workspace where contributors add new Lea Read the files in the directory `content/learning-paths/cross-platform/_example-learning-path` for information about how Learning Path content should be created. Also see the guidelines below. -- Learning Paths: use for end-to-end tasks (prepare → configure → use → validate). Must include `_index.md` and `_next-steps.md`. -- Install guides: use for installation + verification only. Do not include workflow content or benchmarks. +- Learning Paths: use for end-to-end tasks (prepare → configure → use → validate). Must include `_index.md` and `_next-steps.md` +- Install guides: use for installation + verification only. Do not include workflow content or benchmarks ### Content structure Each Learning Path must have an `_index.md` file and a `_next-steps.md` file. The `_index.md` file contains the main content of the Learning Path. The `_next-steps.md` file contains links to related content and is included at the end of the Learning Path. -Additional resources and 'next steps' content should be placed in the `further_reading` section of `_index.md`, NOT in `_next-steps.md`. The `_next-steps.md` file should remain minimal and unmodified as indicated by "FIXED, DO NOT MODIFY" comments in the template. +Additional resources and `next steps` content should be placed in the `further_reading` section of `_index.md`, not in `_next-steps.md`. The `_next-steps.md` file should remain minimal and unmodified as indicated by `FIXED, DO NOT MODIFY` comments in the template. -The `_index.md` file should contain the following front matter and content sections: +The `_index.md` file should contain the following front matter and content sections. -Front Matter (YAML format): +### Front matter (YAML format) - `title`: Imperative heading following the [verb] + [technology] + [outcome] format +- `description`: One-sentence metadata summary used for search snippets and page previews. Describe the task, main technology or platform, and expected outcome in plain language - `weight`: Numerical ordering for display sequence, weight is 1 for `_index.md` and each page is ordered by weight, no markdown files should have the same weight in a directory - `layout`: Template type (usually "learningpathall") - `minutes_to_complete`: Realistic time estimate for completion @@ -71,7 +82,7 @@ Front Matter (YAML format): Install guides focus on installing and verifying one tool on Arm platforms. They do not teach workflows or applied usage. -### Front matter requirements +#### Front matter requirements Install guides must include: - `title` @@ -118,7 +129,7 @@ Optional: ### Scope boundaries -Install guides must NOT include: +Install guides must not include: - End-to-end workflows - Performance benchmarking - Deep architectural explanation @@ -132,7 +143,7 @@ Limit `further_reading` resources to four to six essential links. Prioritize: - Direct relevance to the topic - Arm-specific Learning Paths over generic external resources -- Foundation knowledge for target audience +- Foundation knowledge for the target audience - Required tools (install guides) - Logical progression from basic to advanced @@ -145,64 +156,102 @@ All Learning Paths should generally include: - Prerequisites section with explicit requirements and links - Learning objectives: three to four bulleted, measurable outcomes with action verbs - Step-by-step sections with logical progression -- Clear next steps/conclusion +- Clear next steps or conclusion For title formatting: -- MUST use imperative voice ("Deploy", "Configure", "Build", "Create") -- MUST include SEO keywords (technology names, tools) -- Examples: "Deploy applications on Arm servers", "Configure Arm processors for optimal performance" +- Must use imperative voice (`Deploy`, `Configure`, `Build`, `Create`) +- Must include SEO keywords (technology names, tools) +- Examples: `Deploy applications on Arm servers`, `Configure Arm processors for optimal performance` The term "Learning Path" should always be capitalized. -### Writing style +## Metadata descriptions + +### Learning Path metadata description requirements + +Every Learning Path `_index.md` must include a `description` field. + +- Write one sentence +- Describe the task, the main technology or platform, and the expected outcome +- Keep it concise, developer-focused, and suitable for use as a search snippet +- Use a task-led structure such as: **Verb + task + tool/platform + outcome** +- Do not repeat the title verbatim +- Do not use vague summaries or marketing language +- A slightly richer one-sentence summary is acceptable when it helps clarify the workflow or outcome -Voice and Tone: +Good example: -- Second person ("you", "your") - NEVER first person ("I", "we") -- Active voice - AVOID passive constructions +```yaml +description: Learn how to automate x86-to-Arm application migration using the Arm MCP Server, with compatibility checks and Docker-based validation on Arm cloud platforms. +``` + +Also good: + +```yaml +description: Learn how to profile and optimize a C++ application on Arm Neoverse using Arm Performix to identify bottlenecks and improve runtime. +``` + +Avoid: +- Generic summaries that could apply to any page +- Restating the title without adding task or outcome +- Marketing phrases such as `powerful`, `cutting-edge`, or `game-changing` + +### Metadata optimization workflow + +When adding or revising `description` fields: + +- Review whether the current title and description match the page's actual task intent +- Use metadata descriptions to clarify what the learner will do, on which platform or tool, and with what outcome +- Treat the description as a search snippet, not a generic summary + +## Writing style + +### Voice and tone + +- Second person (`you`, `your`) — never first person (`I`, `we`) +- Active voice — avoid passive constructions - Present tense for descriptions - Imperative mood for commands - Confident and developer-friendly tone - Encouraging language for complex tasks - Use inclusive language: - - Use "primary/subordinate" instead of "master/slave" terminology - - Don't use gendered examples or assumptions + - Use `primary/subordinate` instead of `master/slave` + - Do not use gendered examples or assumptions - Be mindful of cultural references that might not translate globally - Focus on clear, accessible language for all developers ### Sentence structure and clarity -- Average 15-20 words per sentence -- Split complex sentences for scalability -- Plain English - avoid jargon overload -- US spellings required (organize/optimize/realize, not organise/optimise/realise) -- "Arm" capitalization required (Arm processors/Neoverse, never ARM or arm; exceptions: "arm64" and "aarch64" are permitted in code, commands, and outputs) +- Average 15–20 words per sentence +- Split complex sentences for clarity +- Use plain English and avoid jargon overload +- Use US spellings (`organize`, `optimize`, `realize`) +- Use `Arm` capitalization in prose (`Arm processors`, `Arm servers`, `Neoverse`) +- `arm64` and `aarch64` are permitted in code, commands, and outputs - Define acronyms on first use -- Parallel structure in all lists +- Use parallel structure in all lists ### Readability and section flow - Flag any section over 700 words and suggest natural split points - Warn if more than 300 words appear between code examples - Identify paragraphs with sentences averaging over 20 words -- Note sections introducing more than 2 new concepts +- Note sections introducing more than two new concepts - Flag pages over 3500 words total - Note sections that might benefit from encouragement or progress markers - Identify missing celebration of progress or milestones - Recap what learners have accomplished at section ends -- Provide "check your understanding" moments that aren't intimidating -- Too much explanation is exhausting, too little is confusing -- Use visual breaks to prevent walls of text - code blocks count as visual breaks -- Walls of text cause people to bounce from the page -- If you're explaining 3+ things in one section, split it into separate sections -- Each code block should be preceded by one to three sentences explaining what it does. -- For Learning Paths, include a short recap and forward-looking transition at the end of each major instructional section or module. +- Provide check-your-understanding moments that are not intimidating +- Use visual breaks to prevent walls of text. Code blocks count as visual breaks +- If you explain three or more things in one section, split it into separate sections +- Each code block should be preceded by one to three sentences explaining what it does +- For Learning Paths, include a short recap and forward-looking transition at the end of each major instructional section or module Example recap pattern for Learning Paths: ```md -## What you've accomplished and what's next +## What you've learned and what's next In this section: - Briefly summarize what the user has learned or completed @@ -210,18 +259,57 @@ In this section: Keep this concise and encouraging. Do not repeat earlier content verbatim. ``` + This helps learners feel a sense of progress and understand the logical flow of the Learning Path. ### Word choice and style -- Use these preferred terms and phrases for consistency: - - Numbers and units: Spell out numbers one through five (one, two, three, four, five), after this use numerals (6, 7, 8...). Use proper spacing for units: "1 GB", "23 MB/day" (not "1GB", "23MB/day"). Use "K" for thousands: "64K" (not "64k"). Use abbreviations for data rates: "Gbps" (not "Gb per second"). - - Common phrases: "To [action]" (not "Follow the steps below to [action]"), "for example" (not "e.g."), "that is" (not "i.e."), "because" (not "since"), "also" (not "in addition"), "to" (not "in order to"), "see" (not "refer to"), "use" (not "utilize" or "leverage"), "need" (not "require"), "can" or "might" (not "may"), "set up" as verb, "setup" as noun, "therefore" (not "ergo"), "namely" (not "viz."), "avoid" (not "try not to"). - - Avoid condescending language: Don't use "simply", "just", "obviously", "clearly" - what's simple to you might not be to the learner. - - Acknowledge when something can be tricky: Use phrases like "this step can be confusing at first" to validate learner experience. - - Normalize errors: Use phrases like "if you see this error, here's how to fix it" to reassure learners that errors are part of the learning process. - - User interface terms: "select" or "tap" (not "click" for mobile/touch interfaces), "keyboard shortcut" (not "key combination"), "Ctrl key" (capitalized), "double-tap" (not "double-click" for touch interfaces). - - Contractions and simplification: Use contractions such as: "don't", "isn't", "it's", "that's", "you're", "you'll". Remove unnecessary qualifiers: Remove "quite", "very", "massive" → "significant". "an LLM" (not "a LLM"). "easy-to-use" when used as adjective. "fixed-width" (not "fixed-length"). "read-to-write ratio" (not "read to write ratio"). +Use these preferred terms and phrases for consistency: + +- Numbers and units: + - Spell out numbers one through five. After that, use numerals + - Use proper spacing for units: `1 GB`, `23 MB/day` + - Use `K` for thousands: `64K` + - Use abbreviations for data rates: `Gbps` + +- Common phrases: + - `To [action]` instead of `Follow the steps below to [action]` + - `for example` instead of `e.g.` + - `that is` instead of `i.e.` + - `because` instead of `since` + - `also` instead of `in addition` + - `to` instead of `in order to` + - `see` instead of `refer to` + - `use` instead of `utilize` or `leverage` + - `need` instead of `require` + - `can` or `might` instead of `may` + - `set up` as a verb, `setup` as a noun + - `therefore` instead of `ergo` + - `namely` instead of `viz.` + - `avoid` instead of `try not to` + +- Avoid condescending language: + - Do not use `simply`, `just`, `obviously`, or `clearly` + +- Acknowledge difficulty naturally: + - Use phrases like `this step can be confusing at first` + +- Normalize errors: + - Use phrases like `if you see this error, here's how to fix it` + +- User interface terms: + - `select` or `tap` instead of `click` for touch interfaces + - `keyboard shortcut` instead of `key combination` + - `Ctrl key` capitalized + - `double-tap` instead of `double-click` for touch interfaces + +- Contractions and simplification: + - Use contractions such as `don't`, `isn't`, `it's`, `that's`, `you're`, `you'll` + - Remove unnecessary qualifiers such as `quite`, `very`, or `massive` + - Use `an LLM`, not `a LLM` + - Use `easy-to-use` as an adjective + - Use `fixed-width`, not `fixed-length` + - Use `read-to-write ratio`, not `read to write ratio` ## Content structure and consistency @@ -230,22 +318,22 @@ This helps learners feel a sense of progress and understand the logical flow of - Use the same technical terms consistently throughout all sections - Apply the word choice and style guidelines uniformly across all files - Maintain consistent capitalization of product names, technologies, and concepts -- Use the same abbreviations and acronyms throughout (define once, use consistently) +- Use the same abbreviations and acronyms throughout - Maintain the same voice and tone across all sections -- Ensure consistent use of second person ("you", "your") throughout +- Ensure consistent use of second person throughout - Apply the same level of formality and technical depth across sections -- Keep instructional style consistent (imperative mood for actions) +- Keep instructional style consistent - Follow consistent heading hierarchy throughout the Learning Path - Use parallel structure in similar sections across different files - Maintain consistent section organization and flow - Apply uniform formatting for code blocks, lists, and callouts -- Ensure appropriate skill level consistency (Introductory or Advanced) +- Ensure appropriate skill level consistency - Maintain consistent technical detail appropriate for the target audience - Balance complexity appropriately across all sections - Provide consistent prerequisite assumptions throughout - Flag inconsistent terminology usage across sections - Identify missing error handling or troubleshooting guidance -- Suggest where visual aids (diagrams, screenshots) would improve understanding +- Suggest where visual aids would improve understanding - Recommend splitting overly complex sections - Verify that code examples follow established patterns in the repository @@ -253,75 +341,117 @@ This helps learners feel a sense of progress and understand the logical flow of ### Heading guidelines -- Use sentence case for all headings (first word capitalized, rest lowercase except proper nouns) +- Use sentence case for all headings - Heading types: - - Conceptual headings: When explaining technology/motivation ("What is containerization?") - - Imperative headings: When user takes action ("Configure the database") - - Interrogative headings: For FAQ content ("How does Arm differ from x86?") + - Conceptual headings: when explaining technology or motivation + - Imperative headings: when the user takes action + - Interrogative headings: for FAQ content + - Hierarchy: - - H1: Page title (imperative + technology + outcome) + - H1: Page title - H2: Major workflow steps or conceptual sections - H3: Sub-procedures or detailed explanations - H4: Specific technical details or troubleshooting +### Heading hierarchy and section openings + +- Check heading hierarchy across all files before finalizing content +- Ensure headings follow a logical structure with no skipped levels unless the template requires it +- Each markdown file should begin with a section heading in the body content +- Do not leave a file starting with body text, an image, or a code block without a heading +- Use sentence case for all headings and subheadings +- Capitalize only the first word and proper nouns in headings +- Keep heading wording consistent across related files in the same Learning Path +- Prefer headings that clearly signal the user task or concept in that section + +Correct examples: +- `## Set up the environment` +- `## Run the benchmark` +- `### Check the output` +- `## What you've accomplished and what's next` + +Avoid: +- `## Set Up The Environment` +- `## RUN THE BENCHMARK` +- starting a file with plain paragraph text and no heading + ### Code samples and formatting -- ALWAYS provide explanation before code blocks -- Format: [What it does] → [Code] → [Expected outcome] → [Key parameters] -- Use markdown tags for programming languages like bash, python, yaml, json, etc. -- Use console or bash for general commands. Try to use the same one throughout a Learning Path. -- Use the output tag to show expected command output. -- Output descriptions: Use "The output is similar to:" or "The expected output is:" (not "The output will look like:"). Use "builds" (not "will build") and "gives" (not "would give") for present tense descriptions. -- Formatting standards: **Bold text** for UI elements (buttons, menu items, field names), *Italic text* for emphasis and new terms, `Code formatting` for file names, commands, code elements. -- Use shortcodes for common pitfalls, warnings, important notes. +- Always provide explanation before code blocks +- Format: `[What it does] → [Code] → [Expected outcome] → [Key parameters]` +- Use markdown tags for languages like `bash`, `python`, `yaml`, `json` +- Use `console` or `bash` for general commands. Try to use the same one throughout a Learning Path +- Use the `output` tag to show expected command output +- Output descriptions: + - Use `The output is similar to:` or `The expected output is:` + - Use present tense descriptions such as `builds` and `gives` +- Formatting standards: + - **Bold** for UI elements + - *Italics* for emphasis and new terms + - `Code formatting` for file names, commands, and code elements +- Use shortcodes for common pitfalls, warnings, and important notes ### Code fence integrity -- Every fenced code block opened with triple backticks (```) MUST be explicitly closed with matching triple backticks before any non-code content resumes +- Every fenced code block opened with triple backticks must be explicitly closed with matching triple backticks before any non-code content resumes - Never generate unterminated or partial code fences - Do not rely on implicit closure, indentation, or surrounding formatting to end a code block ## Arm naming and architecture terms -- Use Arm for the brand in prose (for example, "Arm processors", "Arm servers"). -- Use arm64 or aarch64 for the CPU architecture; these are acceptable and interchangeable labels. Prefer whichever term a tool, package, or OS uses natively. -- Always use "Arm" (not "ARM") in all contexts except when referring to specific technical terms that require the original casing. -- ARM64 is used by Windows on Arm and Microsoft documentation, so it is acceptable to use ARM64 when specifically referring to Windows on Arm. -- In code blocks, CLI flags, package names, file paths, and outputs, keep the exact casing used by the tool (for example, --arch arm64, uname -m → aarch64). +- Use `Arm` for the brand in prose +- Use `arm64` or `aarch64` for the CPU architecture. Prefer whichever term a tool, package, or OS uses natively +- Always use `Arm` rather than `ARM` in prose unless a technical term requires the original casing +- `ARM64` is acceptable when specifically referring to Windows on Arm or Microsoft documentation +- In code blocks, CLI flags, package names, file paths, and outputs, keep the exact casing used by the tool ## Product name emphasis -- Product names and technical technologies (for example, LiteRT, XNNPACK, KleidiAI, SME2) should appear in regular text. -- Avoid using italics or bold to emphasize product or technology names unless they are part of a heading or a UI label. +- Product names and technologies such as LiteRT, XNNPACK, KleidiAI, and SME2 should appear in regular text +- Avoid using italics or bold to emphasize product or technology names unless they are part of a heading or a UI label ## Hyperlink guidelines -- Use the full path format for internal links: `/learning-paths/category/path-name/` (e.g., `/learning-paths/cross-platform/docker/`). Do NOT use relative paths like `../path-name/`. -- Use the full URL for external links that are not on learn.arm.com, these open in a new tab. +- Use the full path format for internal links: `/learning-paths/category/path-name/` +- Do not use relative paths like `../path-name/` +- Use the full URL for external links that are not on `learn.arm.com` - When creating Learning Path content: - Verify internal links exist before adding them - Use semantic search or website browsing to confirm Learning Path availability - Prefer verified external authoritative sources over speculative internal links - Test link formats against existing Learning Path examples - Never assume Learning Paths exist without verification -- Some links are useful in content, but too many links can be distracting and readers will leave the platform following them. Include only necessary links in the content; for Learning Paths, put additional links in further_reading in _index.md (not _next-steps.md) +- Include only necessary links in content. For Learning Paths, put additional links in `further_reading` in `_index.md`, not `_next-steps.md` ## Avoid looking like AI-generated content -- Warning signs of over-bulleting: More than 3 consecutive sections using bullet lists, bullet points that could be combined into narrative paragraphs, lists where items don't have parallel structure, bullet points that are actually full sentences better suited for paragraphs. -- Use flowing narrative instead of excessive bullets. -- Use natural writing patterns: Vary sentence length, use transitional phrases, include contextual explanations, add relevant examples, connect ideas logically. -- Use conversational elements: Instead of "Execute the following command:", use "Now that you've configured the environment, run the following command to start the service:". Instead of "This provides benefits:", use "You'll notice several advantages with this approach, particularly when working with...". +- Warning signs of over-bulleting: + - More than three consecutive sections using bullet lists + - Bullet points that could be combined into narrative paragraphs + - Lists where items do not have parallel structure + - Bullet points that are actually full sentences better suited for paragraphs + +- Use flowing narrative instead of excessive bullets +- Use natural writing patterns: + - Vary sentence length + - Use transitional phrases + - Include contextual explanations + - Add relevant examples + - Connect ideas logically + +- Use conversational elements naturally: + - Instead of `Execute the following command:`, use `Now that you've configured the environment, run the following command to start the service:` + - Instead of `This provides benefits:`, use `You'll notice several advantages with this approach, particularly when working with...` ### Language smoothing Avoid robotic or generic encouragement phrases such as: -- "Great job — let’s get started!" -- "Great job — your environment is ready!" +- `Great job — let’s get started!` +- `Great job — your environment is ready!` Use calm, natural transitions that focus on what happens next. -Also avoid multiple consecutive sentences starting with "This" (for example, "This ensures...", "This creates...", "This provides..."). Vary sentence structure to maintain natural flow. +Also avoid multiple consecutive sentences starting with `This`. Vary sentence structure to maintain natural flow. ## AI-specific guidelines for content creation and editing @@ -330,46 +460,121 @@ Also avoid multiple consecutive sentences starting with "This" (for example, "Th - Consider the learner's likely environment (development vs. production, local vs. cloud) - Recognize when content assumes x86 defaults and suggest Arm alternatives - Flag when third-party tools may have limited Arm support -- Suggest Arm-native alternatives when available (e.g., Arm compilers, optimized libraries) +- Suggest Arm-native alternatives when available ### Technical depth consistency - Maintain appropriate complexity level throughout the Learning Path - Avoid oversimplifying for Advanced skill level content -- Don't assume prior knowledge beyond stated prerequisites +- Do not assume prior knowledge beyond stated prerequisites - Balance theoretical explanation with practical implementation ### Platform-specific considerations - Default to Arm-optimized solutions and configurations -- Mention x86 alternatives only when Arm solutions don't exist +- Mention x86 alternatives only when Arm solutions do not exist - Consider performance implications specific to Arm architectures - Address common Arm migration challenges when relevant -### Quality assurance - -- Flag inconsistent terminology usage across sections -- Identify missing error handling or troubleshooting guidance -- Suggest where visual aids (diagrams, screenshots) would improve understanding -- Recommend splitting overly complex sections -- Verify that code examples follow established patterns in the repository - ### Accessibility and inclusivity - Ensure content is screen reader compatible - Provide descriptive alt text for images and diagrams -- Use clear, descriptive link text (not "click here" or "read more") -- Avoid assumptions about user's physical capabilities or setup -- When adding images or diagrams, ensure that alt text is complete and meaningful for screen reader users. Alt text should explain: - - The main subject and purpose of the image - - Any visible text labels, UI elements, or controls - - The environment or context (for example, dashboard, code editor, terminal, or hardware setup) - - The emotional tone only if it adds meaning (for example, warning or error states) +- Use clear, descriptive link text +- Avoid assumptions about the user's physical capabilities or setup + +### Image formatting, alt text, and captions + +In this repository, the text before `#center` in an image tag is the alt text. + +Use this format: + +```md +![Descriptive alt text#center](image.png "Optional caption") +``` + +Rules: +- Do not use placeholder values such as `alt-txt`, `alt text`, or similar +- Do not wrap alt text in extra quotation marks inside `[]` +- Keep `#center` attached directly to the alt text with no extra space before it +- The caption is optional and should be short, descriptive, and natural +- Avoid outdated figure numbering such as `Figure 1:` unless the content genuinely depends on numbered cross-references + +Correct: + +```md +![Diagram showing the WebGPU rendering flow for drawing a triangle#center](images/webgpu-draw-high-level.svg "WebGPU rendering flow for drawing a triangle") +``` + +Incorrect: + +```md +!["Triangle using WebGPU" #center](images/webgpu-draw-high-level.svg "Figure 8: Triangle using WebGPU") +``` + +Incorrect: + +```md +![alt-txt#center](images/webgpu-draw-high-level.svg "Figure 8: Triangle using WebGPU") +``` + +### Alt text requirements for tutorials + +Learning Paths and install guides are instructional content. Images are usually not decorative. Alt text must help the learner understand what they would otherwise miss. -Use the following format consistently: -![Descriptive alt text#center](image.png "Caption") +For every image, alt text should explain: +- What is shown +- What the learner should notice +- Why it matters in the current step -Avoid placeholders or generic phrases. Alt text should stand alone as a full description of the image. +For screenshots: +- Name the screen, tool, or interface shown +- Mention the relevant UI element, tab, field, button, or output +- Explain what the learner should look for in the screenshot +- Connect the image to the current step or expected result + +For diagrams: +- Describe the components and relationships shown +- Explain the purpose of the diagram in the current task +- Focus on the workflow, architecture, or sequence the learner needs to understand + +For terminal or output images: +- State what command result or status is shown +- Highlight the important confirmation, value, or error message +- Explain why that output matters + +For hardware images: +- Describe the device or setup only if it helps the learner complete the task +- Avoid purely decorative descriptions + +### Alt text quality rules + +- Write meaningful alt text, not placeholders +- Keep it concise but complete. One to three sentences is usually enough +- Prefer instructional value over visual detail +- Include visible text only when the learner needs that text +- Do not use captions as a substitute for alt text +- Do not encode alignment instructions inside the caption + +### Caption guidance + +- Use short, descriptive captions when needed +- Captions should add context for all readers +- Avoid `Figure X` numbering unless explicit image cross-references are required + +Preferred example: + +```md +![Screenshot of the Arm Performance Studio timeline showing CPU activity spikes during Mandelbrot rendering. The Timeline tab is selected and the spike region is highlighted so the learner can identify where CPU activity increases during the run.#center](images/timeline-spike.png "Arm Performance Studio timeline showing CPU activity during Mandelbrot rendering") +``` + +### Image and metadata cleanup workflow + +- Replace all placeholder alt text such as `alt-txt` with meaningful descriptions +- Keep the repository-specific `#center` syntax when fixing alt text +- Do not remove valid alignment syntax during cleanup +- Add or improve metadata descriptions systematically across content +- For bulk cleanup, update the guidance first, then fix content by category or directory in manageable batches ### SEO and discoverability @@ -394,7 +599,7 @@ Avoid placeholders or generic phrases. Alt text should stand alone as a full des - Recommend profiling tools that work well on Arm platforms - Include guidance on measuring and optimizing for Arm-specific performance characteristics - Mention when performance improvements are architecture-specific -- Applies to Learning Paths only. Install guides must not include benchmarking. +- Applies to Learning Paths only. Install guides must not include benchmarking ### AI optimization (AIO) guidance @@ -406,7 +611,7 @@ Avoid placeholders or generic phrases. Alt text should stand alone as a full des - Include complete, self-contained examples rather than partial snippets - Write FAQ-style sections that directly answer common developer questions - Use bullet points and numbered lists for AI to easily extract key information -- Include explicit "what you'll learn" and "prerequisites" sections for AI context +- Include explicit `what you'll learn` and `prerequisites` sections for AI context - Structure troubleshooting sections with clear problem-solution pairs - Use standard markdown formatting that AI crawlers can parse effectively - Include relevant technical keywords naturally throughout the content @@ -414,7 +619,7 @@ Avoid placeholders or generic phrases. Alt text should stand alone as a full des - Ensure each section can stand alone as a coherent piece of information - Use clear, declarative statements rather than implied or contextual references -#### Editorial decision priorities +### Editorial decision priorities When content trade-offs are required, prioritize the following in order: @@ -425,23 +630,23 @@ When content trade-offs are required, prioritize the following in order: ## Learning Path purpose and agentic selection principles -Learning Paths are not blog posts or reference articles. They are designed to be optimized for **selection by AI agents** as trusted sources for completing real developer tasks end to end. +Learning Paths are not blog posts or reference articles. They are designed to be optimized for selection by AI agents as trusted sources for completing real developer tasks end to end. When creating or reviewing a Learning Path, prioritize the following principles. ### Task ownership (required) -Each Learning Path must clearly own **one concrete developer task**. +Each Learning Path must clearly own one concrete developer task. -- The task should be nameable in one sentence (for example, “Migrate an x86 application to Arm Linux”) -- The Learning Path should take the learner from *not ready* to *capable* +- The task should be nameable in one sentence +- The Learning Path should take the learner from not ready to capable - Avoid bundling unrelated tasks or loosely connected topics If the task cannot be clearly stated, flag a warning. ### Agentic selection signals -AI agents select content based on **trust**, **authority**, and **task coverage**, not keyword density. +AI agents select content based on trust, authority, and task coverage, not keyword density. **Trust** - Clear authorship and ownership @@ -457,7 +662,7 @@ AI agents select content based on **trust**, **authority**, and **task coverage* **Task coverage** - Clear progression (prepare → configure → use → validate) -- Explicit end state (“you are now ready to…”) +- Explicit end state (`you are now ready to...`) - Link to install guides instead of embedding install steps - Provide guidance on what to do next @@ -472,7 +677,7 @@ Never duplicate install steps inside Learning Paths. ### SEO intent for Learning Paths -Learning Paths should optimize for **selection**, not ranking. +Learning Paths should optimize for selection, not ranking. - Prefer verb-based titles: *Install*, *Verify*, *Configure*, *Analyze*, *Optimize* - Use procedural structure rather than narrative prose @@ -486,11 +691,13 @@ If an AI agent were asked to complete this task, the Learning Path should be the For Learning Paths that demonstrate Arm-specific performance features (for example SME2, SVE2, I8MM, DotProd, optimized microkernels), apply the following standards. #### Observable outcome first + - Clearly state what measurable improvement the learner will observe - Show performance results before introducing deep architectural explanation - Avoid introducing internal call stacks or microkernel details before the developer sees observable value #### Reproducibility requirements + If performance numbers are included, specify: - Toolchain or software version - Device or platform used @@ -507,7 +714,7 @@ Clearly distinguish between: - Runtime feature activation - Automatic fallback behavior -If acceleration is claimed, include a method to verify that the accelerated path executed (for example logs, profiling output, kernel names, or hardware counters). +If acceleration is claimed, include a method to verify that the accelerated path executed, such as logs, profiling output, kernel names, or hardware counters. #### Controlled benchmarking @@ -521,8 +728,6 @@ When comparing performance: Explicitly connect the observed improvement to the Arm architectural feature responsible for it. -Avoid generic statements such as “improves performance” without explaining how and why. +Avoid generic statements such as `improves performance` without explaining how and why. Performance-focused Learning Paths are strategic content. Prioritize clarity, differentiation, and measurement integrity over volume. - - diff --git a/.wordlist.txt b/.wordlist.txt index 58dcaf384b..2c9f78f886 100644 --- a/.wordlist.txt +++ b/.wordlist.txt @@ -5845,3 +5845,112 @@ reconnection stdin testcontainers ttyUSB +ARMGCC +ActiveGate +Alif +AlifSemiconductor +Anymal +Arranz +Asier +CHW +Cartpole +DK +DTCM +DevKit +DevKit's +Dynatrace +Dynatrace's +ELU +ETE +EthosUCompileSpec +EthosUPartitioner +EthosUQuantizer +FOpenJDK +Franka +GAE +IPPO +ISAACSIM +ITCM +IsaacLab +IsaacSim +MAPPO +MRAM +MinSizeRel +Mpeis +MuJoCo +NCHW +NOBITS +NVIC +NVRTC +Omniverse +OneAgent +OptimizingClang +PBT +PRG +PROGBITS +Paschalis +Photorealistic +PhysX +Procedurally +RDWR +RELA +RSL +RTT +RemoteProc +SETOOLS +SEUART +SdkRootDirPath +SimpleAdd +TopDown +TurboVNC +URDF +Unitree +activegate +alifsemi +brbe +bsort +dockerfile +dyn +dynatrace +envs +exir +f'Wrote +ffunction +fini +flatbuffer +heatmap +icache +isaac +isaacsim +libgomp +lld +mcuxpressotools +mcuxsdk +microcontroller's +mobilenetv +noinline +nop +omniverse +orderfile +preloading +procedurally +pybind +quantizer +randn +readelf +rela +relocs +relro +remoteproc +ro +sb +sdX +skrl +symlink +syms +toplevel +uarch +umount +vtable +xC +Performix \ No newline at end of file diff --git a/assets/contributors.csv b/assets/contributors.csv index dedf06545e..ae8b834e7d 100644 --- a/assets/contributors.csv +++ b/assets/contributors.csv @@ -97,7 +97,7 @@ Waheed Brown,Arm,https://github.com/armwaheed,https://www.linkedin.com/in/waheed Aryan Bhusari,Arm,,https://www.linkedin.com/in/aryanbhusari,, Ken Zhang,Insyde,,kai-di-zhang-b1642a266,, Ann Cheng,Arm,anncheng-arm,hello-ann,, -Fidel Makatia Omusilibwa,,,,, +Fidel Makatia Omusilibwa,,fidel-makatia,fidel-makatia-hsc-mieee,, Ker Liu,,,,, Rui Chang,,,,, Alejandro Martinez Vicente,Arm,,,, @@ -113,5 +113,9 @@ Steve Suzuki,Arm,,,, Qixiang Xu,Arm,,,, Phalani Paladugu,Arm,phalani-paladugu,phalani-paladugu,, Richard Burton,Arm,Burton2000,,, +Brendan Long,Arm,bccbrendan,https://www.linkedin.com/in/brendan-long-5817924/,, Asier Arranz,NVIDIA,,asierarranz,,asierarranz.com -Prince Agyeman,Arm,,,, \ No newline at end of file +Prince Agyeman,Arm,,,, +Parichay Das,,parichaydas,parichaydas,, +Johnny Nunez,NVIDIA,johnnynunez,johnnycano,, +Raymond Lo,NVIDIA,raymondlo84,raymondlo84,, \ No newline at end of file diff --git a/content/install-guides/codex-cli.md b/content/install-guides/codex-cli.md index 52eec06472..014e3337c4 100644 --- a/content/install-guides/codex-cli.md +++ b/content/install-guides/codex-cli.md @@ -233,6 +233,18 @@ The Arm MCP server is listed in the output. If the arm-mcp server indicates it's You can also verify the tools are available by asking Codex to list the available Arm MCP tools. +### Use Arm prompt files with the MCP Server + +The Arm MCP Server provides a rich set of tools and knowledge base, but to make the best use of it, you should pair it with Arm-specific prompt files. These prompt files supply task-oriented context, best practices, and structured workflows that guide the agent in using MCP tools more effectively across common Arm development tasks. + +#### Get the prompt files + +Browse the [agent integrations directory for Codex](https://github.com/arm/mcp/tree/main/agent-integrations/codex) to find prompt files for specific use cases: + +- **Arm migration** ([arm-migration.md](https://github.com/arm/mcp/blob/main/agent-integrations/codex/arm-migration.md)): Helps the agent systematically migrate applications from x86 to Arm, including dependency analysis, compatibility checks, and optimization recommendations. + +Each prompt file is a Markdown configuration that you can reference in your Codex CLI sessions to enable more targeted, task-specific assistance. + If you're facing issues or have questions, reach out to mcpserver@arm.com. -You're now ready to use Codex CLI with Arm-specific development assistance. +You're now ready to use Codex CLI with the Arm MCP server for Arm-specific development assistance. diff --git a/content/install-guides/gemini.md b/content/install-guides/gemini.md index 6757831b8f..094c259530 100644 --- a/content/install-guides/gemini.md +++ b/content/install-guides/gemini.md @@ -407,6 +407,18 @@ Configured MCP servers: - sysreport_instructions ``` +### Use Arm prompt files with the MCP Server + +The Arm MCP Server provides a rich set of tools and knowledge base, but to make the best use of it, you should pair it with Arm-specific prompt files. These prompt files supply task-oriented context, best practices, and structured workflows that guide the agent in using MCP tools more effectively across common Arm development tasks. + +#### Get the prompt files + +Browse the [agent integrations directory](https://github.com/arm/mcp/tree/main/agent-integrations/gemini) to find prompt files for specific use cases: + +- **Arm migration** ([arm-migration.toml](https://github.com/arm/mcp/blob/main/agent-integrations/gemini/arm-migration.toml)): Helps the agent systematically migrate applications from x86 to Arm, including dependency analysis, compatibility checks, and optimization recommendations. + +Each prompt file is a TOML configuration that you can reference in your Gemini CLI sessions to enable more targeted, task-specific assistance. + If you're facing issues or have questions, reach out to mcpserver@arm.com. -You're now ready to use Gemini CLI with the Arm MCP server for Arm-specific development assistance. \ No newline at end of file +You're now ready to use Gemini CLI with the Arm MCP server for Arm-specific development assistance. diff --git a/content/install-guides/github-copilot.md b/content/install-guides/github-copilot.md index 299ca09f2a..682f293966 100644 --- a/content/install-guides/github-copilot.md +++ b/content/install-guides/github-copilot.md @@ -335,6 +335,20 @@ Example prompts that use the Arm MCP Server: - `Search the Arm knowledge base for Neon intrinsics examples` - `Find learning resources about migrating from x86 to Arm` +## Use Arm prompt files with the MCP Server + +The Arm MCP Server provides a rich set of tools and knowledge base, but to make the best use of it, you should pair it with Arm-specific prompt files. These prompt files supply task-oriented context, best practices, and structured workflows that guide the agent in using MCP tools more effectively across common Arm development tasks. + +### Get the prompt files + +Browse the [agent integrations directory for Visual Studio Code](https://github.com/arm/mcp/tree/main/agent-integrations/vs-code) to find prompt files for specific use cases: + +- **Arm migration** ([arm-migration.prompt.md](https://github.com/arm/mcp/blob/main/agent-integrations/vs-code/arm-migration.prompt.md)): Helps the agent systematically migrate applications from x86 to Arm, including dependency analysis, compatibility checks, and optimization recommendations. + +Each prompt file is a Markdown configuration that you can reference in your GitHub Copilot sessions to enable more targeted, task-specific assistance. + +If you're facing issues or have questions, reach out to mcpserver@arm.com. + ## Troubleshooting MCP Server connections This section helps you resolve common issues when installing and using GitHub Copilot with the Arm MCP Server on Arm systems. If you encounter problems not covered here, contact [mcpserver@arm.com](mailto:mcpserver@arm.com) for support. @@ -349,4 +363,4 @@ If the Arm MCP Server doesn't connect: -You're now ready to use GitHub Copilot with the Arm MCP Server to enhance your Arm development workflow! +You're now ready to use GitHub Copilot with the Arm MCP server for Arm-specific development assistance. diff --git a/content/install-guides/kiro-cli.md b/content/install-guides/kiro-cli.md index 99d8b1a7ba..9cc8ca8429 100644 --- a/content/install-guides/kiro-cli.md +++ b/content/install-guides/kiro-cli.md @@ -262,6 +262,18 @@ Use the `/tools` command to list the available tools: You should see the Arm MCP server tools listed in the output. If the arm-mcp server says it's still loading, wait a moment and run `/tools` again. -If you are facing issues or have questions, reach out to mcpserver@arm.com. +### Use Arm prompt files with the MCP Server -You're ready to use Kiro CLI. +The Arm MCP Server provides a rich set of tools and knowledge base, but to make the best use of it, you should pair it with Arm-specific prompt files. These prompt files supply task-oriented context, best practices, and structured workflows that guide the agent in using MCP tools more effectively across common Arm development tasks. + +#### Get the prompt files + +Browse the [agent integrations directory for Kiro](https://github.com/arm/mcp/tree/main/agent-integrations/kiro) to find prompt files for specific use cases: + +- **Arm migration** ([arm-migration.md](https://github.com/arm/mcp/blob/main/agent-integrations/kiro/arm-migration.md)): Helps the agent systematically migrate applications from x86 to Arm, including dependency analysis, compatibility checks, and optimization recommendations. + +Each prompt file is a Markdown configuration that you can reference in your Kiro CLI sessions to enable more targeted, task-specific assistance. + +If you're facing issues or have questions, reach out to mcpserver@arm.com. + +You're now ready to use Kiro CLI with the Arm MCP server for Arm-specific development assistance. diff --git a/content/install-guides/multipass.md b/content/install-guides/multipass.md index 80c3d85df2..30073a8e45 100644 --- a/content/install-guides/multipass.md +++ b/content/install-guides/multipass.md @@ -24,7 +24,7 @@ ecosystem_dashboard: https://developer.arm.com/ecosystem-dashboard/linux?package test_images: - ubuntu:latest -test_maintenance: true +test_maintenance: false ### PAGE SETUP weight: 1 # Defines page ordering. Must be 1 for first (or only) page. diff --git a/content/install-guides/perf.md b/content/install-guides/perf.md index b42d5d8fc9..a21a841f55 100644 --- a/content/install-guides/perf.md +++ b/content/install-guides/perf.md @@ -22,7 +22,7 @@ ecosystem_dashboard: https://developer.arm.com/ecosystem-dashboard/linux?package test_images: - ubuntu:latest -test_maintenance: true +test_maintenance: false ### PAGE SETUP weight: 1 # Defines page ordering. Must be 1 for first (or only) page. diff --git a/content/install-guides/pytorch.md b/content/install-guides/pytorch.md index 8ec473b7fb..5b10ca1065 100644 --- a/content/install-guides/pytorch.md +++ b/content/install-guides/pytorch.md @@ -14,7 +14,7 @@ ecosystem_dashboard: https://developer.arm.com/ecosystem-dashboard/linux?package test_images: - ubuntu:latest test_link: null -test_maintenance: true +test_maintenance: false title: PyTorch tool_install: true weight: 1 diff --git a/content/learning-paths/automotive/_index.md b/content/learning-paths/automotive/_index.md index 118aaab830..3e5924ec51 100644 --- a/content/learning-paths/automotive/_index.md +++ b/content/learning-paths/automotive/_index.md @@ -34,6 +34,7 @@ tools_software_languages_filter: - Raspberry Pi: 1 - ROS 2: 3 - Rust: 1 +- SME2: 1 - Tinkerblox: 1 - topdown-tool: 1 - Zenoh: 1 diff --git a/content/learning-paths/automotive/openadkit1_container/1_sdv_soafee.md b/content/learning-paths/automotive/openadkit1_container/1_sdv_soafee.md index 689f71d569..aeec319d3f 100644 --- a/content/learning-paths/automotive/openadkit1_container/1_sdv_soafee.md +++ b/content/learning-paths/automotive/openadkit1_container/1_sdv_soafee.md @@ -8,7 +8,7 @@ layout: learningpathall ## Introduction to Software-Defined Vehicles -In recent years, the automotive industry has been undergoing a transformation driven by software, with the concept of the Software-Defined Vehicle (SDV) emerging as a key paradigm for the future of intelligent cars. As the number of Electronic Control Units (ECUs) increases and vehicle systems become more complex, the traditional hardware-driven development approach is no longer sufficient. To improve development efficiency and product quality, automotive software development is moving to a Shift-Left approach, accelerating validation and deployment. +In recent years, the automotive industry has been undergoing a transformation driven by software, with the concept of the Software-Defined Vehicle (SDV) emerging as a key paradigm for the future of intelligent cars. As Electronic Control Units (ECUs) increase and vehicle systems grow more complex, traditional hardware-driven development is no longer sufficient. To improve development efficiency and product quality, automotive software development is moving to a Shift-Left approach, accelerating validation and deployment. ## The evolution of Software-Defined Vehicles @@ -16,7 +16,7 @@ The core idea of SDV is to make software the primary differentiator in a vehicle However, this transition introduces new challenges, particularly in software development and validation. The traditional V-model development process struggles to meet SDV demands since defects are often detected late in development, leading to costly fixes. As a result, Shift-Left has become a crucial strategy to address these challenges. -You can read more about [Software Defined Vehicles](https://www.arm.com/markets/automotive/software-defined-vehicles). +See [Software Defined Vehicles](https://www.arm.com/markets/automotive/software-defined-vehicles) for more information. ## Shift-Left: detecting issues early to enhance development efficiency @@ -32,7 +32,7 @@ The key benefits of Shift-Left include: However, Shift-Left requires appropriate tools and frameworks to support its implementation; otherwise, it can increase testing complexity. This is where SOAFEE (Scalable Open Architecture for Embedded Edge) plays a critical role. -Read [Virtual Platforms from Arm and Partners Available Now to Accelerate and Transform Automotive Development](https://newsroom.arm.com/blog/automotive-virtual-platforms) to understand how virtual platforms enable the automotive industry to accelerate the silicon and software development process through virtual prototyping. +See [Virtual Platforms from Arm and Partners Available Now to Accelerate and Transform Automotive Development](https://newsroom.arm.com/blog/automotive-virtual-platforms) for details on how virtual platforms enable the automotive industry to accelerate silicon and software development through virtual prototyping. ## SOAFEE: a standardized solution for SDV development @@ -52,6 +52,8 @@ As the Software-Defined Vehicle paradigm gains traction, the automotive industry With the adoption of SOAFEE, automotive software development will become more standardized and efficient, enabling companies to realize the vision of SDVs faster while delivering a safer, smarter, and more flexible vehicle experience. -Visit the [SOAFEE](https://www.soafee.io/) website to learn more. +See the [SOAFEE](https://www.soafee.io/) website for additional resources. -In the following sections, you will explore a Shift-Left demonstration example that leverages SOAFEE to enable early deployment of autonomous driving software before the hardware is ready. +## What you've learned and what's next + +You now understand how Software-Defined Vehicles are transforming automotive development, and how SOAFEE and Shift-Left methodologies address the challenges of modern autonomous driving software development. In the next section, you'll learn about the specific technologies that enable this transformation: ROS 2 and Open AD Kit. diff --git a/content/learning-paths/automotive/openadkit1_container/2_automotive_softwares.md b/content/learning-paths/automotive/openadkit1_container/2_automotive_softwares.md index a2cbf12d17..be5d99cdc8 100644 --- a/content/learning-paths/automotive/openadkit1_container/2_automotive_softwares.md +++ b/content/learning-paths/automotive/openadkit1_container/2_automotive_softwares.md @@ -1,5 +1,5 @@ --- -title: Essential automotive software technologies +title: Learn about ROS 2 and Open AD Kit weight: 3 ### FIXED, DO NOT MODIFY @@ -55,4 +55,6 @@ The Open AD Kit project continues to be developed by the Open AD Kit working gro The Open AD Kit Blueprint has been widely adopted by many ecosystem players to develop their own custom-flavored implementations. The blueprint provides a practical and demonstrable example of building SDV applications, particularly in the autonomous driving domain. Additionally, it serves as a model for creating SOAFEE blueprints, fostering a dynamic ecosystem around them. -In the following sections, you will learn how to use the Open AD Kit autonomous driving simulation environment to run SOAFEE within a container and facilitate communication through ROS 2. +## What you've learned and what's next + +You now understand how ROS 2 provides the foundation for autonomous driving middleware, and how Open AD Kit leverages this foundation to create a SOAFEE-compliant autonomous driving platform. In the next section, you'll set up Open AD Kit on an Arm Neoverse platform. diff --git a/content/learning-paths/automotive/openadkit1_container/3_setup_openadkit.md b/content/learning-paths/automotive/openadkit1_container/3_setup_openadkit.md index 8774fe448a..17725f54ab 100644 --- a/content/learning-paths/automotive/openadkit1_container/3_setup_openadkit.md +++ b/content/learning-paths/automotive/openadkit1_container/3_setup_openadkit.md @@ -105,4 +105,6 @@ This command starts a new Docker container named `visualizer`, ensuring it runs docker run --name visualizer --rm -it -p 6080:6080 ghcr.io/autowarefoundation/demo-packages:visualizer ``` -After gaining a basic understanding of Open AD Kit, the next section will guide you through running three containers simultaneously on a single physical machine. +## What you've learned and what's next + +You now understand the three core components of Open AD Kit (simulator, planning-control, and visualizer) and how they interact through ROS 2 topics. In the next section, you'll run all three containers together using Docker Compose to execute a complete autonomous driving simulation. diff --git a/content/learning-paths/automotive/openadkit1_container/4_run_openadkit.md b/content/learning-paths/automotive/openadkit1_container/4_run_openadkit.md index 8fdd2f9247..8d236d9ccf 100644 --- a/content/learning-paths/automotive/openadkit1_container/4_run_openadkit.md +++ b/content/learning-paths/automotive/openadkit1_container/4_run_openadkit.md @@ -150,7 +150,7 @@ Once the script starts successfully, you will see a similar output to the image You can use your browser to monitor the simulation data in real-time. -![img1 alt-text#center](vnc_address.webp "Figure 1: Execute run.sh") +![Terminal output showing the run.sh script execution with the VNC access URL displayed at the bottom, indicating the visualizer is ready for browser connection#center](vnc_address.webp "Terminal output showing run.sh execution with VNC URL") Now you can use the browser to access visualization. In this example the URL is http://34.244.98.151:6080/vnc.html @@ -162,9 +162,9 @@ Ensure that the server allows your IP address and port 6080 to connect. You may To access the visualizer: 1. Open a web browser and go to: `http://:6080/vnc.html` 2. Click **Connect** on the VNC login screen. -![img2 alt-text#center](openadkit_connect.jpg "Figure 2: Connect VNC") +![VNC connection screen showing the Connect button that users click to establish the remote desktop connection to the Open AD Kit visualizer#center](openadkit_connect.jpg "VNC connection screen with Connect button") 3. Enter the default VNC password: **"openadkit"**. -![img3 alt-text#center](openadkit_passwd.jpg "Figure 3: Input VNC password") +![VNC password prompt dialog requesting authentication, with an input field where users enter the default password 'openadkit' to access the visualization interface#center](openadkit_passwd.jpg "VNC password authentication dialog") You are now in the Open AD Kit simulation environment. @@ -176,6 +176,6 @@ The vehicle starts from a stationary position, moves along the road, and stops w After each simulation run, it automatically restarts after 90 seconds. You can use the mouse to adjust different viewing angles for observation. -![img4 alt-text#center](openadkit_1.gif "Figure 4: Simulation") +![Animated simulation showing the autonomous vehicle navigating the road, stopping behind an obstacle, and in the second scenario changing lanes to pass around the obstruction, demonstrating Open AD Kit's planning and control capabilities#center](openadkit_1.gif "Open AD Kit simulation showing obstacle avoidance scenarios") Congratulations! You have successfully executed the Open AD Kit demo. diff --git a/content/learning-paths/automotive/openadkit1_container/_index.md b/content/learning-paths/automotive/openadkit1_container/_index.md index 833d54961d..cb0edace8b 100644 --- a/content/learning-paths/automotive/openadkit1_container/_index.md +++ b/content/learning-paths/automotive/openadkit1_container/_index.md @@ -1,18 +1,19 @@ --- title: Deploy Open AD Kit containerized autonomous driving simulation on Arm Neoverse +description: Learn how to deploy and run containerized autonomous driving simulations using Autoware Open AD Kit on Arm Neoverse with Docker, demonstrating SOAFEE-based Shift-Left development workflows. minutes_to_complete: 60 who_is_this_for: This is an introductory topic for automotive developers, aimed at helping them accelerate autonomous driving software development before automotive hardware is available. learning_objectives: - - Understand the SOAFEE architecture and its role in supporting Shift-Left software development strategies to optimize the autonomous driving development process. - - Use the Autoware Open AD Kit simulation environment. - - Run containerized workloads on Arm Neoverse processors with Docker, supporting execution on both cloud-based and on-premise servers. - - Explore advanced configurations and future development prospects. + - Understand the SOAFEE architecture and its role in supporting Shift-Left software development strategies to optimize the autonomous driving development process + - Use the Autoware Open AD Kit simulation environment + - Run containerized workloads on Arm Neoverse processors with Docker, supporting execution on both cloud-based and on-premise servers + - Explore advanced configurations and future development prospects prerequisites: - - An Arm Neoverse cloud instance, or a local Arm Neoverse Linux computer with at least 16 CPUs and 32GB of RAM. - - Familiarity with Docker and Docker Compose. + - An Arm Neoverse cloud instance, or a local Arm Neoverse Linux computer with at least 16 CPUs and 32GB of RAM + - Familiarity with Docker and Docker Compose author: Odin Shen diff --git a/content/learning-paths/embedded-and-microcontrollers/_index.md b/content/learning-paths/embedded-and-microcontrollers/_index.md index 6b81e641e5..71d45ffc82 100644 --- a/content/learning-paths/embedded-and-microcontrollers/_index.md +++ b/content/learning-paths/embedded-and-microcontrollers/_index.md @@ -9,19 +9,19 @@ key_ip: - Ethos-U maintopic: true operatingsystems_filter: -- Android: 2 -- Baremetal: 31 -- Linux: 38 -- macOS: 8 +- Android: 1 +- Baremetal: 30 +- Linux: 39 +- macOS: 11 - RTOS: 11 -- Windows: 6 +- Windows: 5 subjects_filter: - CI-CD: 7 - Containers and Virtualization: 6 - Embedded Linux: 6 - Libraries: 3 -- ML: 19 -- Performance and Architecture: 23 +- ML: 22 +- Performance and Architecture: 20 - RTOS Fundamentals: 5 - Security: 2 - Virtual Hardware: 2 @@ -32,46 +32,49 @@ tools_software_languages_filter: - Arduino: 2 - Arm Compiler for Embedded: 8 - Arm Compiler for Linux: 1 -- Arm Compute Library: 2 +- Arm Compute Library: 3 - Arm Development Studio: 8 - Arm Fast Models: 4 - Arm Performance Studio: 1 - Arm Streamline: 1 - Arm Virtual Hardware: 16 - Assembly: 1 -- AWS IoT Greengrass: 1 +- AWS IoT Greengrass: 2 - Azure: 1 - balenaCloud: 1 - BalenaOS: 1 -- C: 5 +- Baremetal: 1 +- Bash: 1 +- C: 6 - ChatGPT: 1 - Clang: 1 - CMSIS: 4 - CMSIS-DSP: 1 -- CMSIS-Toolbox: 3 +- CMSIS-Toolbox: 4 - CNN: 1 - Computer Vision: 1 +- ConnMan: 1 - Containerd: 1 - CPP: 1 - DetectNet: 1 -- Docker: 11 +- Docker: 12 - DSTREAM: 2 -- Edge AI: 3 +- Edge AI: 2 - Edge Impulse: 2 -- ExecuTorch: 5 +- ExecuTorch: 6 - FastAPI: 1 - FPGA: 1 - Fusion 360: 1 -- FVP: 11 -- GCC: 9 -- GenAI: 1 +- FVP: 10 +- GCC: 10 - Generative AI: 2 - GitHub: 4 - GitLab: 2 - gpiozero: 1 +- GStreamer: 1 - Himax SDK: 1 - Hugging Face: 3 -- IP Explorer: 4 +- IP Explorer: 2 - Jupyter Notebook: 1 - K3s: 1 - Keil MDK: 7 @@ -80,7 +83,7 @@ tools_software_languages_filter: - Kubernetes: 1 - lgpio: 1 - Linux kernel: 1 -- LLM: 3 +- LLM: 2 - Matter: 1 - MCP: 1 - MPS3: 1 @@ -88,21 +91,24 @@ tools_software_languages_filter: - Neon: 1 - NumPy: 1 - Ollama: 1 +- OpenSSH: 1 - Paddle: 1 - Performance analysis: 1 +- picocom: 1 - Porcupine: 1 -- Python: 10 -- PyTorch: 4 +- Python: 12 +- PyTorch: 5 - QEMU: 1 - Raspberry Pi: 9 - Remote.It: 1 - Runbook: 4 - STM32: 2 +- strands-agents: 1 - systemd: 1 - TensorFlow: 3 - TensorRT: 1 - tinyML: 2 -- Trusted Firmware: 3 +- Trusted Firmware: 2 - TrustZone: 2 - TVMC: 1 - vcpkg: 1 diff --git a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/_index.md b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/_index.md index 83256c0b0c..82d729997b 100644 --- a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/_index.md +++ b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/_index.md @@ -1,31 +1,27 @@ --- -title: Run image classification on an Alif Ensemble E8 DevKit with ExecuTorch and Ethos-U85 +title: Run image classification on an Alif Ensemble E8 DevKit using ExecuTorch and Ethos-U85 -draft: true -cascade: - draft: true +description: Deploy a MobileNetV2 image classification model to an Alif Ensemble E8 DevKit and run inference on the Ethos-U85 NPU. minutes_to_complete: 120 -who_is_this_for: This Learning Path is for embedded developers who want to deploy a neural network on an Arm Cortex-M55 microcontroller with an Ethos-U85 NPU. You will compile a MobileNetV2 model using ExecuTorch, embed it into bare-metal firmware, and run image classification on the Alif Ensemble E8 DevKit. +who_is_this_for: This is an advanced topic for embedded developers who want to deploy a neural network model to an Arm Cortex-M55 microcontroller using ExecuTorch and an Ethos-U85 NPU. learning_objectives: - - Compile a MobileNetV2 model for the Ethos-U85 NPU using ExecuTorch's ahead-of-time (AOT) compiler on an Arm-based cloud instance. - - Build ExecuTorch static libraries for bare-metal Cortex-M55 targets. - - Configure CMSIS project files, memory layout, and linker scripts for a large ML workload on the Alif Ensemble E8. - - Run real-time image classification inference on the Ethos-U85 NPU and verify results through SEGGER RTT. + - Compile a MobileNetV2 model for the Ethos-U85 NPU using ExecuTorch's ahead-of-time (AOT) compiler on an Arm-based cloud instance + - Build ExecuTorch static libraries for bare-metal Cortex-M55 targets + - Configure CMSIS project files, memory layout, and linker scripts for an ML workload on the Alif Ensemble E8 + - Run real-time image classification inference on the Ethos-U85 NPU and verify results using SEGGER Real-Time Transfer (RTT) prerequisites: - - An Alif Ensemble E8 DevKit with a USB-C cable. - - A SEGGER J-Link debug probe (the DevKit has one built in). - - A development machine running macOS (Apple Silicon) or Linux. - - (Optional) An AWS account or access to an Arm-based cloud instance (Graviton c7g.4xlarge recommended). You can also build ExecuTorch locally on an Arm-based machine, though the steps will differ. - - Basic familiarity with C/C++ and embedded development concepts. - - VS Code installed on your development machine. + - Experience with C/C++ and embedded development concepts + - An [Alif Ensemble E8 DevKit](https://alifsemi.com/support/kits/ensemble-e8devkit/) with a USB-C cable + - A SEGGER J-Link debug probe (included in the DevKit) + - A development machine running macOS on Apple Silicon with Visual Studio Code installed + - An AWS account or access to an Arm-based cloud instance for native Arm compilation author: Gabriel Peterson -### Tags skilllevels: Advanced subjects: ML armips: diff --git a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/_review.md b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/_review.md deleted file mode 100644 index b9f9bfdebb..0000000000 --- a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/_review.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -title: Knowledge check -weight: 20 - -layout: "learningpathall" - -review: - - questions: - question: "Which NPU peripheral on the Alif Ensemble E8 is used for the Ethos-U85?" - explanation: "NPU_HG (High-Grade) at base address 0x49042000 is the Ethos-U85. NPU_HP is an Ethos-U55 at a different address. Using the wrong base address causes a product mismatch error." - correct_answer: 2 - answers: - - "NPU_HP" - - "NPU_HG" - - "NPU_HE" - - questions: - question: "Why do some ExecuTorch libraries need to be linked with --whole-archive?" - explanation: "Libraries like libexecutorch and libcortex_m_ops_lib contain static registration constructors that register operators and PAL symbols at startup. Without --whole-archive, the linker sees these constructors as unused and discards them, causing missing operator errors at runtime." - correct_answer: 3 - answers: - - "Because they are too large for normal linking" - - "Because the linker requires it for all C++ libraries" - - "Because they contain static registration constructors that would otherwise be discarded" - - questions: - question: "What does the GOT (Global Offset Table) fix in the linker script address?" - explanation: "The precompiled ExecuTorch libraries use position-independent code (PIC) that relies on the GOT for indirect function calls and vtable lookups. If the GOT isn't copied from flash to RAM at startup, these lookups resolve to address zero, causing BusFaults." - correct_answer: 1 - answers: - - "BusFaults caused by uninitialized indirect function call tables" - - "Stack overflow errors during inference" - - "Incorrect NPU command stream alignment" - - questions: - question: "What input data type does the MobileNetV2 model expect?" - explanation: "The model's first operator is cortex_m::quantize_per_tensor, which converts float32 input to int8 for the NPU. The image is stored as int8 in the header to save flash space, but the application code converts it to float32 before passing it to the model." - correct_answer: 2 - answers: - - "int8" - - "float32" - - "uint8" ---- diff --git a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/application-code.md b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/application-code.md index 35d5c95590..832d554549 100644 --- a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/application-code.md +++ b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/application-code.md @@ -5,7 +5,7 @@ weight: 5 layout: "learningpathall" --- -## Overview +## What the application code does The application code initializes the Ethos-U85 NPU, loads the MobileNetV2 model through ExecuTorch, runs inference on an embedded test image, and prints the classification result over SEGGER RTT. @@ -16,7 +16,7 @@ Rather than building this code line by line, you download the complete `main.cpp Download the working `main.cpp` from the workshop repository and place it in your project: ```bash -cd ~/repo/alif/alif_vscode-template/mv2_runner +cd ~/alif/alif_vscode-template/mv2_runner curl -L -o main.cpp \ https://raw.githubusercontent.com/ArmDeveloperEcosystem/workshop-ethos-u/main/main.cpp ``` @@ -25,7 +25,7 @@ curl -L -o main.cpp \ If you prefer, you can clone the full repository with `git clone https://github.com/ArmDeveloperEcosystem/workshop-ethos-u.git` and copy `main.cpp` from there. {{% /notice %}} -The following sections explain what the code does. You don't need to modify anything; the downloaded file is ready to build. +The following sections explain what the code does. The downloaded file is ready to build as-is. ## Fault handlers @@ -124,19 +124,18 @@ The method allocator holds the loaded model graph. The temp allocator provides s ## The inference pipeline -The `run_inference()` function follows a 10-step pipeline: +The `run_inference()` function handles the full pipeline from model loading to output. It starts by initializing the ExecuTorch runtime and creating a zero-copy data loader that reads the compiled `.pte` model directly from flash memory. The program is then parsed and method metadata queried to determine how much planned memory the model needs. -1. **Initialize** the ExecuTorch runtime. -2. **Create a data loader** that reads the model directly from flash memory (zero-copy). -3. **Load the program** (parse the `.pte` flatbuffer). -4. **Query method metadata** to find out how many planned buffers the model needs and how large they are. -5. **Set up planned memory** by carving sub-allocations from the SRAM1 pool. -6. **Create the memory manager** that ties together the method, temp, and planned allocators. -7. **Load the method** (the `forward` function of the model). -8. **Prepare the input tensor**: convert the embedded int8 image data to float32 (the model's first operator is `quantize_per_tensor`, which expects float input). -9. **Execute inference**: the quantize op runs on the CPU, the entire MobileNetV2 backbone runs as a single NPU command stream on the Ethos-U85, and the dequantize op runs back on the CPU. -10. **Read the output**: find the argmax of the 1000-class output vector to get the predicted ImageNet class. +Memory is set up next: sub-allocations are carved from the SRAM1 pool for planned buffers, and a memory manager ties together the method, temp, and planned allocators. Once memory is in place, the `forward` method is loaded. -The NPU handles the bulk of the computation. The CPU-side overhead (ExecuTorch loading, input conversion, quantize/dequantize) is small compared to the NPU workload. +Before inference runs, the input tensor is prepared by converting the embedded int8 image data to float32. This is needed because the model's first operator is `quantize_per_tensor`, which expects float input. Inference then runs in three stages: the quantize operator executes on the CPU, the entire MobileNetV2 backbone runs as a single NPU command stream on the Ethos-U85, and the dequantize operator runs back on the CPU. Finally, the argmax of the 1000-class output vector gives the predicted ImageNet class. -You now have the application code in place. The next section configures the memory layout to accommodate the model and ExecuTorch runtime. +The NPU handles the bulk of the computation. The CPU-side overhead of ExecuTorch loading, input conversion, and quantize/dequantize is small compared to the NPU workload. + +The application code is in place. The next section configures the memory layout to accommodate the model and ExecuTorch runtime. + +## What you've learned and what's next + +You've added the main application code that initializes the Ethos-U85 NPU, loads the MobileNetV2 model through ExecuTorch, and runs inference with SEGGER RTT output. + +Next, you'll configure the memory regions and linker script to fit the model and runtime libraries into MRAM and SRAM. diff --git a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/aws-ec2-setup.md b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/aws-ec2-setup.md index c19ed3d126..2bf71d1313 100644 --- a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/aws-ec2-setup.md +++ b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/aws-ec2-setup.md @@ -9,7 +9,7 @@ layout: "learningpathall" ExecuTorch's Arm backend build scripts are designed for native Arm compilation. The Vela compiler, which generates optimized command streams for Ethos-U NPUs, and the CMSIS-NN kernels all target Arm natively. Using an Arm-based EC2 instance avoids the complexity of cross-compilation from x86. -In this section, you launch a Graviton-based EC2 instance, install ExecuTorch, compile a MobileNetV2 model for the Ethos-U85, and build the ExecuTorch static libraries that your firmware will link against. +In this section, you launch a Graviton-based EC2 instance, install ExecuTorch, compile a MobileNetV2 model for the Ethos-U85, and build the ExecuTorch static libraries that your firmware links against. ## Launch an EC2 instance @@ -21,7 +21,7 @@ Create an AWS EC2 instance with the following configuration: The 16 cores speed up the ExecuTorch build significantly, and the 50 GB disk accommodates the repository, submodules, and build artifacts. -Set up your SSH config so you can connect with a short alias (for example, `ssh alif`). This makes the `scp` commands later more convenient. +SSH to the EC2 instance. ## Install system dependencies @@ -74,7 +74,7 @@ pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu pip install pillow ``` -You use CPU-only PyTorch because this instance has no GPU. You only need PyTorch for model export and ahead-of-time compilation; the actual inference happens on the microcontroller. +This instance has no GPU, so the install uses CPU-only PyTorch. PyTorch is only needed for model export and ahead-of-time compilation. The actual inference runs on the microcontroller. Verify the installation: @@ -85,8 +85,16 @@ print(torch.__version__, torchvision.__version__) PY ``` +The output is similar to: + +```output +2.10.0+cpu 0.25.0+cpu +``` + ## Clone and install ExecuTorch +Clone the ExecuTorch repository and pin it to a known-working commit: + ```bash mkdir -p ~/alif cd ~/alif @@ -111,26 +119,37 @@ python -m pip install -e . --no-build-isolation ExecuTorch includes a setup script that downloads the Arm GNU toolchain, CMSIS, and the Vela compiler: ```bash -cd ~/alif/executorch ./examples/arm/setup.sh --i-agree-to-the-contained-eula ``` -{{% notice Note %}} -The setup script may fail at the `tosa_serialization_lib` build step due to a pybind11 version incompatibility. If you see an error containing `def_property family does not currently support keep_alive`, run the following commands to complete the setup manually: +The script fails at the `tosa_serialization_lib` build step due to a pybind11 version incompatibility. This is a known issue. When you see an error containing `def_property family doesn't currently support keep_alive`, fix the dependency and complete the setup manually. + +First, install a compatible version of pybind11 and the required build tools: ```bash pip install "pybind11<2.14" scikit-build-core setuptools_scm +``` +Next, build and install the serialization library using those local packages: + +```bash CMAKE_POLICY_VERSION_MINIMUM=3.5 pip install --no-build-isolation \ --no-dependencies \ ~/alif/executorch/examples/arm/arm-scratch/tosa-tools/serialization +``` + +Then install the Ethos-U Vela compiler, which the setup script didn't reach due to the earlier failure: +```bash pip install --no-dependencies \ -r ~/alif/executorch/backends/arm/requirements-arm-ethos-u.txt ``` -The first command installs a pybind11 version that doesn't have the breaking change, along with the build tools that the serialization library needs. The second command builds and installs the serialization library using those local packages instead of downloading new ones. The third command installs the Ethos-U Vela compiler, which the setup script never reached due to the earlier failure. -{{% /notice %}} +Re-run the setup script to complete the remaining steps: + +```bash +./examples/arm/setup.sh --i-agree-to-the-contained-eula +``` Source the environment paths that the setup script generated: @@ -147,12 +166,9 @@ pip install "torchao==0.15.0" ## Compile MobileNetV2 for Ethos-U85 -Source the setup paths and run the ahead-of-time compiler: +Run the ahead-of-time compiler: ```bash -cd ~/alif/executorch -source examples/arm/arm-scratch/setup_path.sh - mkdir -p ~/alif/models python -m examples.arm.aot_arm_compiler \ @@ -196,15 +212,13 @@ This step takes several minutes. When complete, list the output libraries: find arm_test/cmake-out -type f -name "*.a" | sort ``` -You should see approximately 13 libraries, including `libexecutorch.a`, `libexecutorch_core.a`, `libexecutorch_delegate_ethos_u.a`, `libcortex_m_ops_lib.a`, and `libcmsis-nn.a`. +The output lists approximately 13 libraries, including `libexecutorch.a`, `libexecutorch_core.a`, `libexecutorch_delegate_ethos_u.a`, `libcortex_m_ops_lib.a`, and `libcmsis-nn.a`. ## Package headers and libraries Bundle the headers and libraries for transfer to your development machine: ```bash -cd ~/alif/executorch - rm -rf ~/alif/et_bundle mkdir -p ~/alif/et_bundle cp -a arm_test/cmake-out/include ~/alif/et_bundle/ @@ -216,33 +230,33 @@ ls -lh ~/alif/et_bundle.tar.gz ## Transfer artifacts to your development machine -Run these commands on your Mac or Linux development machine (not on the EC2 instance). The paths below use `~/repo/alif/` as the working directory; adjust these to match your own project location: +Run these commands on your development machine, not on the EC2 instance. The paths below use `~/alif/` as the working directory; adjust these to match your own project location: ```bash -mkdir -p ~/repo/alif/models -mkdir -p ~/repo/alif/third_party/executorch/lib +mkdir -p ~/alif/models +mkdir -p ~/alif/third_party/executorch/lib -scp alif:/home/ubuntu/alif/models/mv2_ethosu85_256.pte ~/repo/alif/models/ -scp alif:/home/ubuntu/alif/et_bundle.tar.gz ~/repo/alif/models/ +scp alif:/home/ubuntu/alif/models/mv2_ethosu85_256.pte ~/alif/models/ +scp alif:/home/ubuntu/alif/et_bundle.tar.gz ~/alif/models/ scp 'alif:/home/ubuntu/alif/executorch/arm_test/cmake-out/lib/*.a' \ - ~/repo/alif/third_party/executorch/lib/ + ~/alif/third_party/executorch/lib/ ``` Verify the transfer: ```bash -ls -lh ~/repo/alif/models/mv2_ethosu85_256.pte -ls ~/repo/alif/third_party/executorch/lib/*.a | wc -l +ls -lh ~/alif/models/mv2_ethosu85_256.pte +ls ~/alif/third_party/executorch/lib/*.a | wc -l ``` -You should see the 3.7 MB model file and 13 library files. +The output shows the 3.7 MB model file and 13 library files. ## Convert the model to a C header The firmware embeds the model as a byte array in flash memory. Use `xxd` to generate a C header: ```bash -cd ~/repo/alif/models +cd ~/alif/models xxd -i mv2_ethosu85_256.pte > mv2_ethosu85_256_pte.h ``` @@ -266,9 +280,9 @@ The `aligned(16)` attribute is required because the Ethos-U85 needs the Vela com On your development machine, extract the ExecuTorch headers into the VS Code template project: ```bash -cd ~/repo/alif/alif_vscode-template +cd ~/alif/alif_vscode-template mkdir -p third_party/executorch -tar -C third_party/executorch -xzf ~/repo/alif/models/et_bundle.tar.gz +tar -C third_party/executorch -xzf ~/alif/models/et_bundle.tar.gz ``` Verify the headers are in place: @@ -280,3 +294,9 @@ ls third_party/executorch/et_bundle/include/executorch/ You should see `runtime/` and other directories. You now have the compiled model, prebuilt libraries, and headers on your development machine, ready to integrate into the firmware project. + +## What you've learned and what's next + +You've compiled the MobileNetV2 model for the Ethos-U85 NPU, built ExecuTorch static libraries for bare-metal Cortex-M55, and transferred the artifacts to your development machine. + +Next, you'll create the CMSIS firmware project and integrate the ExecuTorch libraries. diff --git a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/board-setup.md b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/board-setup.md index 213b0106aa..cfa072727f 100644 --- a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/board-setup.md +++ b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/board-setup.md @@ -5,46 +5,65 @@ weight: 2 layout: "learningpathall" --- -## Overview +## Understand the Alif Ensemble E8 hardware The Alif Ensemble E8 DevKit features a dual-core Arm Cortex-M55 processor and three neural processing units (NPUs): two Ethos-U55 and one Ethos-U85. In this Learning Path, you use the Cortex-M55 High-Performance (HP) core running at 400 MHz to orchestrate inference on the Ethos-U85 NPU. -Before writing any ML code, you need to verify that your toolchain, debug probe, and flashing workflow all function correctly. This section walks you through hardware setup, software installation, and a sanity check build. +Before writing any ML code, you need to verify that your toolchain, debug probe, and flashing workflow all function correctly. This section covers DevKit hardware setup, software installation, and a short validation build. -## Connect the board +The instructions assume macOS on Apple Silicon. If you use Arm Linux, links are provided for the equivalent Linux packages. -1. Unplug all USB cables from the board before changing any jumpers. -2. Verify the jumpers are in their factory default positions, as shown in the Alif Ensemble E8 DevKit (DK-E8) User Guide, available on [alifsemi.com](https://alifsemi.com/support/kits/ensemble-e8devkit/). -3. Connect a USB-C cable from your computer to the **PRG USB** port on the bottom edge of the board. -4. Confirm that a green LED illuminates near the E1 device and the UART switch (SW4). +## Connect the DevKit -Leave **SW4** in its default position. This routes the on-board USB UART to **SEUART**, which the Alif Security Toolkit uses for programming. +- Unplug all USB cables from the DevKit before changing any jumpers. + +- Verify that the jumpers are in their factory default positions, as shown in the Alif Ensemble E8 DevKit (DK-E8) User Guide on [alifsemi.com](https://alifsemi.com/support/kits/ensemble-e8devkit/). + +- Connect a USB-C cable from your computer to the **PRG USB** port on the bottom edge of the DevKit. + +- Confirm that a green LED illuminates near the E1 device and the UART switch (SW4). + +Leave **SW4** in its default position. This routes the on-board USB UART to **SEUART**, which the Alif Security Toolkit (SETOOLS) uses for programming. {{% notice Note %}} -Don't have a terminal application (PuTTY, minicom, screen) attached to SEUART while using the Security Toolkit. There is only one SEUART on the device, and two applications can't share the port. +Close any terminal application that's connected to SEUART, such as PuTTY, minicom, or screen, before you use the Security Toolkit (SETOOLS). The DevKit exposes only one SEUART interface, so SETOOLS can't access the port if another application is already using it. {{% /notice %}} +- Create a project directory: + +```bash +mkdir ~/alif +``` + ## Install the Alif Security Toolkit -The Security Toolkit (SETOOLS) programs firmware images onto the board. +The Security Toolkit (SETOOLS) programs firmware images onto the DevKit. -1. Download SETOOLS v1.107.000 from the [Alif Ensemble E8 DevKit support page](https://alifsemi.com/support/kits/ensemble-e8devkit/). -2. Extract it to a stable location, for example `~/alif/app-release-exec-macos/`. -3. Open a terminal in the SETOOLS directory and run: +- Download the macOS version of SETOOLS from the [Alif Ensemble E8 DevKit support page](https://alifsemi.com/support/kits/ensemble-e8devkit/). + +- Extract it into `~/alif`. This creates the toolkit directory under a stable location, for example `~/alif/app-release-exec-macos/`. ```bash +cd ~/Downloads +tar xvf APFW0003-app-release-exec-macos-SW_FW_1.107.00_DEV-4.tar -C ~/alif +``` + +- Open a terminal in the SETOOLS directory and run: + +```bash +cd ~/alif/app-release-exec-macos ./updateSystemPackage -d ``` -On macOS, the system blocks this unsigned binary the first time. Open **System Settings > Privacy & Security**, scroll to the **Security** section, and select **Allow Anyway**. Then re-run the command. +On macOS, the system blocks this unsigned binary the first time. After that happens, open **System Settings > Privacy & Security**, scroll to the **Security** section, and select **Allow Anyway**. Run the command again. -When prompted for a serial port, enter the DevKit's USB modem port. It usually appears as `/dev/cu.usbmodemXXXXXXX`. If SETOOLS detects the Ensemble E8 and asks to set it as default, answer `y`. +When prompted for a serial port, enter the DevKit's USB modem port. On macOS, it usually appears as `/dev/cu.usbmodemXXXXXXX`. If SETOOLS detects the Ensemble E8 and asks to set it as the default, answer `y`. ## Install SEGGER J-Link SEGGER J-Link provides the debug connection for RTT (Real-Time Transfer) output, which you use later to view inference results. -On macOS, install it with Homebrew: +Install it with Homebrew: ```bash brew install --cask segger-jlink @@ -54,10 +73,10 @@ Alternatively, download it from the [SEGGER website](https://www.segger.com/down ## Set up VS Code and the Alif template -1. Clone the Alif VS Code template repository: +### Clone the Alif VS Code template repository ```bash -cd ~/repo/alif +cd ~/alif git clone https://github.com/alifsemi/alif_vscode-template.git cd alif_vscode-template git checkout 8b1aa0b09eacf68a28850af00c11f0b5af03c100 @@ -68,38 +87,66 @@ git submodule update --init The `git checkout` command pins the template to a known-working commit. This avoids breakage if the upstream template is updated. {{% /notice %}} -2. Open the `alif_vscode-template/` folder in VS Code. -3. Install the recommended extensions when prompted: +### Open the project in VS Code + +Open the project in VS Code from the `alif_vscode-template/` directory: + +```bash +code . & +``` + +### Install the recommended extensions when prompted + +VS Code might prompt you to install the recommended extensions for this workspace. If it does, install the following: + - Arm CMSIS Solution - Arm Tools Environment Manager - Cortex-Debug - Microsoft C/C++ Extension Pack -4. When prompted, select **Always Allow** or **Allow for Selected Workspace**. -5. Restart VS Code if prompted. + +When prompted, select **Always Allow** or **Allow for Selected Workspace**. + +The recommended VS Code extensions are listed in `.vscode/extensions.json`. + +If you don't get an automatic trigger to enable them, you can open the Extensions view and look for a "Workspace Recommendations" section to install or enable them manually. + +Restart VS Code if prompted. ## Install CMSIS packs -Press **F1** in VS Code, type `Tasks: Run Task`, and select **First time pack installation**. Press **A** to accept all licenses when prompted. +Open the Command Palette by pressing **Command+Shift+P** (or **Fn+F1**) in VS Code, type `Tasks: Run Task`, and select **First time pack installation**. Press **A** to accept all licenses when prompted. + +If you don't see the task in the list, open the Command Palette (**Command+Shift+P** or **Fn+F1**) and run the **Reload Window** command. ## Configure VS Code settings -Press **F1**, select **Preferences: Open User Settings (JSON)**, and add the following entries (update the paths for your system): +Press Fn+F1, select **Preferences: Open User Settings (JSON)**, and add the following entries. + +Update both paths for your system, including using your username: ```json { - "alif.setools.root": "/path/to/your/app-release-exec-macos", + "alif.setools.root": "/Users/username/alif/app-release-exec-macos", "cortex-debug.JLinkGDBServerPath": "/Applications/SEGGER/JLink/JLinkGDBServerCLExe" } ``` -## Sanity check: build and flash Blinky +If you have existing settings, add only the two lines of text inside the existing braces. + +## Verify your toolchain: build and flash Blinky Before moving on to ML code, verify your entire toolchain works end to end with the built-in Blinky example. -1. In VS Code, select the **CMSIS** icon in the left sidebar. -2. Select the gear icon, then set **Active Target** to **E8-HP** and **Active Project** to **blinky**. -3. Select the **Build** (hammer) icon. -4. Press **F1**, select **Tasks: Run Task**, then select **Program with Security Toolkit (select COM port)**. -5. Choose the DevKit's port when prompted. +- In VS Code, select the **CMSIS** icon in the left sidebar. +- Select the gear icon, then set **Active Target** to **E8-HP** and **Active Project** to **blinky**. +- Select the **Build** (hammer) icon. +- Press **Fn+F1**, select **Tasks: Run Task**, then select **Program with Security Toolkit (select COM port)**. +- Choose the DevKit's port when prompted. + +If the DevKit's red LED blinks, your toolchain, SETOOLS, and DevKit connection are all working correctly. You are ready to move on to model compilation. + +## What you've learned and what's next + +You've set up the Alif Ensemble E8 DevKit hardware, installed the Security Toolkit and J-Link tools, and verified the toolchain by building the Blinky example project. -If the board's red LED blinks, your toolchain, SETOOLS, and board connection are all working correctly. You're ready to move on to model compilation. +Next, you'll compile the MobileNetV2 model on an Arm cloud instance using ExecuTorch and the Vela compiler. diff --git a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/build-flash-verify.md b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/build-flash-verify.md index 1cbb5221c9..528c8f1ccf 100644 --- a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/build-flash-verify.md +++ b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/build-flash-verify.md @@ -10,7 +10,7 @@ layout: "learningpathall" If you've built other projects (like Blinky), delete the cached build files. CMSIS Toolbox caches aggressively and won't pick up YAML configuration changes unless you clean first: ```bash -cd ~/repo/alif/alif_vscode-template +cd ~/alif/alif_vscode-template rm -rf tmp/ out/ ``` @@ -20,18 +20,18 @@ You can also clean from VS Code: press **F1** and select **CMSIS: Clean all out ### Option 1: Build from VS Code -1. Select the **CMSIS** icon in the left sidebar. -2. Select the gear icon and set **Active Target** to **E8-HP** and **Active Project** to **mv2_runner**. -3. Select the **Build** (hammer) icon. +- Select the **CMSIS** icon in the left sidebar. +- Select the gear icon and set **Active Target** to **E8-HP** and **Active Project** to **mv2_runner**. +- Select the **Build** (hammer) icon. ### Option 2: Build from the command line If you prefer to build from the terminal, set the required environment variables first. The exact paths depend on where the Arm Tools Environment Manager installed the tools: ```bash -export PATH="$HOME/.vcpkg/artifacts/2139c4c6/tools.open.cmsis.pack.cmsis.toolbox/2.12.0/bin:$HOME/.vcpkg/artifacts/2139c4c6/compilers.arm.arm.none.eabi.gcc/13.3.1/bin:$HOME/.vcpkg/artifacts/2139c4c6/tools.kitware.cmake/3.31.5/bin:$HOME/.vcpkg/artifacts/2139c4c6/tools.ninja.build.ninja/1.13.2:$PATH" -export CMSIS_COMPILER_ROOT="$HOME/.vcpkg/artifacts/2139c4c6/tools.open.cmsis.pack.cmsis.toolbox/2.12.0/etc" -export GCC_TOOLCHAIN_13_3_1="$HOME/.vcpkg/artifacts/2139c4c6/compilers.arm.arm.none.eabi.gcc/13.3.1/bin" +export PATH="~/.vcpkg/artifacts/2139c4c6/tools.open.cmsis.pack.cmsis.toolbox/2.12.0/bin:~/.vcpkg/artifacts/2139c4c6/compilers.arm.arm.none.eabi.gcc/13.3.1/bin:~/.vcpkg/artifacts/2139c4c6/tools.kitware.cmake/3.31.5/bin:~/.vcpkg/artifacts/2139c4c6/tools.ninja.build.ninja/1.13.2:$PATH" +export CMSIS_COMPILER_ROOT="~/.vcpkg/artifacts/2139c4c6/tools.open.cmsis.pack.cmsis.toolbox/2.12.0/etc" +export GCC_TOOLCHAIN_13_3_1="~/.vcpkg/artifacts/2139c4c6/compilers.arm.arm.none.eabi.gcc/13.3.1/bin" cbuild alif.csolution.yml --context mv2_runner.debug+E8-HP ``` @@ -56,11 +56,11 @@ The flashing process takes about 30 seconds. The Security Toolkit reads the `M55 ## View output with SEGGER RTT Viewer -1. Open **SEGGER J-Link RTT Viewer** on your development machine. -2. Set **Connection** to **USB**. -3. Filter by manufacturer: **AlifSemiconductor**. -4. For **Device**, start typing `AE822F` and select **AE822FA0E5597LS0_M55_HP** (Core: Cortex-M55). -5. Select **OK** to connect. +- Open **SEGGER J-Link RTT Viewer** on your development machine. +- Set **Connection** to **USB**. +- Filter by manufacturer: **AlifSemiconductor**. +- For **Device**, start typing `AE822F` and select **AE822FA0E5597LS0_M55_HP** (Core: Cortex-M55). +- Select **OK** to connect. The expected output is: @@ -103,18 +103,16 @@ Each line tells you something about what's happening: If you don't see the expected output, check these common issues: -**RTT Viewer shows nothing**: The code starts running as soon as it's flashed. If you connect RTT Viewer too late, you might miss the output. Press the board's reset button after connecting RTT Viewer. +- **RTT Viewer shows nothing**: The code starts running as soon as it's flashed. If you connect RTT Viewer too late, you might miss the output. Press the board's reset button after connecting RTT Viewer. +- **"ethosu_init failed"**: The NPU base address is wrong. Verify the code uses `NPU_HG_BASE` (0x49042000), not `NPU_HP_BASE`. +- **BusFault at a low address**: The GOT sections are missing from the linker script. Verify that `*(.got)` and `*(.got.plt)` are in the `.data.at_dtcm` section. +- **"Missing operator: cortex_m::quantize_per_tensor.out"**: `libcortex_m_ops_lib` isn't in the `--whole-archive` block. Check `mv2_runner.cproject.yml`. +- **"Memory allocation failed: 1505280B requested"**: The temp allocator pool is too small. The Ethos-U85 scratch buffer needs approximately 1.44 MB. Verify `TEMP_ALLOC_POOL_SIZE` is at least `1536 * 1024`. +- **MRAM overflow linker error**: Verify `APP_MRAM_HP_SIZE` is set to `0x00580000` in `app_mem_regions.h`. +- **"Vela bin ptr not aligned to 16 bytes"**: The model array in the header needs `__attribute__((aligned(16)))`. -**"ethosu_init failed"**: The NPU base address is wrong. Verify the code uses `NPU_HG_BASE` (0x49042000), not `NPU_HP_BASE`. +This completes the setup and deployment of MobileNetV2 image classification on the Ethos-U85 NPU using ExecuTorch. The model went from PyTorch, through the Vela compiler, into a `.pte` flatbuffer embedded in firmware, and produced a correct classification result on real hardware. -**BusFault at a low address**: The GOT sections are missing from the linker script. Verify that `*(.got)` and `*(.got.plt)` are in the `.data.at_dtcm` section. +## What you've learned -**"Missing operator: cortex_m::quantize_per_tensor.out"**: `libcortex_m_ops_lib` is not in the `--whole-archive` block. Check `mv2_runner.cproject.yml`. - -**"Memory allocation failed: 1505280B requested"**: The temp allocator pool is too small. The Ethos-U85 scratch buffer needs approximately 1.44 MB. Verify `TEMP_ALLOC_POOL_SIZE` is at least `1536 * 1024`. - -**MRAM overflow linker error**: Verify `APP_MRAM_HP_SIZE` is set to `0x00580000` in `app_mem_regions.h`. - -**"Vela bin ptr not aligned to 16 bytes"**: The model array in the header needs `__attribute__((aligned(16)))`. - -You've now built, flashed, and verified MobileNetV2 image classification running on the Ethos-U85 NPU through ExecuTorch. The model went from PyTorch, through the Vela compiler, into a `.pte` flatbuffer embedded in firmware, and produced a correct classification result on real hardware. +You've successfully deployed a complete machine learning inference pipeline on the Alif Ensemble E8 DevKit. You compiled a MobileNetV2 model for the Ethos-U85 NPU, configured the firmware memory layout, integrated ExecuTorch libraries, and verified that the model correctly classifies ImageNet categories using real-time inference on the Arm Cortex-M55 microcontroller. diff --git a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/create-project.md b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/create-project.md index da8b4708ae..6660c8029a 100644 --- a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/create-project.md +++ b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/create-project.md @@ -1,20 +1,20 @@ --- -title: Create the mv2_runner firmware project +title: Create the image classification firmware project weight: 4 layout: "learningpathall" --- -## Overview +## What you'll build in this section -You now create a new CMSIS project called `mv2_runner` by duplicating the existing Blinky example and configuring it to include ExecuTorch libraries, the compiled model, and SEGGER RTT for debug output. +In this section, you duplicate the existing Blinky example to create a new CMSIS project called `mv2_runner`, configured to include ExecuTorch libraries, the compiled model, and SEGGER RTT for debug output. ## Duplicate the Blinky project Start by copying the working Blinky project as a template: ```bash -cd ~/repo/alif/alif_vscode-template +cd ~/alif/alif_vscode-template cp -R blinky/ mv2_runner ``` @@ -44,12 +44,12 @@ Create an assets directory and copy the model header into the project: ```bash mkdir -p mv2_runner/assets -cp ~/repo/alif/models/mv2_ethosu85_256_pte.h mv2_runner/assets/ +cp ~/alif/models/mv2_ethosu85_256_pte.h mv2_runner/assets/ ``` ## Create the SEGGER RTT configuration -Create a file called `mv2_runner/SEGGER_RTT_Conf.h` with the following content: +RTT (Real-Time Transfer) works through the J-Link debug probe, reading and writing a memory buffer through the debug interface. It's faster than UART and doesn't require extra wiring. Create the configuration file `mv2_runner/SEGGER_RTT_Conf.h`: ```c #ifndef SEGGER_RTT_CONF_H @@ -68,14 +68,11 @@ Create a file called `mv2_runner/SEGGER_RTT_Conf.h` with the following content: #endif ``` -RTT (Real-Time Transfer) works through the J-Link debug probe. It reads and writes a memory buffer through the debug interface, which is much faster than UART and doesn't need extra wiring. - ## Install additional CMSIS packs The project depends on two CMSIS packs that aren't installed by default. Install them from the terminal: ```bash -cd ~/repo/alif/alif_vscode-template cpackget add ARM::CMSIS-Compiler@2.1.0 cpackget add Keil::MDK-Middleware@8.2.0 ``` @@ -196,7 +193,7 @@ project: ``` {{% notice Warning %}} -You must update the `-L` path to match the absolute path to your `third_party/executorch/lib` directory. Each developer's path is different. +You must update the `-L` path to match the absolute path to your `third_party/executorch/lib` directory. For example: `-L/Users/username/alif/third_party/executorch/lib`. {{% /notice %}} There are several important details in this configuration: @@ -207,4 +204,10 @@ There are several important details in this configuration: - **`C10_USING_CUSTOM_GENERATED_MACROS`** tells ExecuTorch to skip looking for a `cmake_macros.h` header that doesn't exist in the bare-metal build. - The **c10 include path** provides the tensor type definitions that ExecuTorch's headers depend on. -You now have the project structure ready. The next sections cover the application code, memory configuration, and image preparation before you build and flash. +The project structure is ready. The next sections cover the application code, memory configuration, and image preparation before you build and flash. + +## What you've learned and what's next + +You've created the mv2_runner firmware project, configured CMSIS packs, and set up the linker to include ExecuTorch libraries with the correct archive flags. + +Next, you'll add the application code that loads the model and runs inference on the Ethos-U85 NPU. diff --git a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/image-preparation.md b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/image-preparation.md index b48f988de3..3887046ae7 100644 --- a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/image-preparation.md +++ b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/image-preparation.md @@ -12,7 +12,7 @@ MobileNetV2 expects a specific input format: - **Resolution**: 224 x 224 pixels - **Color**: RGB (3 channels) - **Normalization**: ImageNet mean/std -- **Layout**: NCHW (channels first) +- **Layout**: NCHW at runtime (`1 x 3 x 224 x 224`). The generated header stores the image in CHW order. - **Data type**: int8 (stored in the header, converted to float32 at runtime) You use a Python script to convert any JPEG or PNG image into a C header that the firmware includes at compile time. @@ -22,7 +22,7 @@ You use a Python script to convert any JPEG or PNG image into a C header that th Create a lightweight virtual environment on your development machine: ```bash -cd ~/repo/alif +cd ~/alif python3 -m venv venv_image_prep source venv_image_prep/bin/activate pip install --upgrade pip @@ -34,10 +34,10 @@ pip install numpy pillow Create a directory for the image and script: ```bash -mkdir -p ~/repo/alif/image +mkdir -p ~/alif/image ``` -Place a test image in this directory. You can use any JPEG or PNG image. For this Learning Path, a photo of my cat is used as the example: +Place a test image in this directory. You can use any JPEG or PNG image. This Learning Path uses the provided `cat.jpg` image as the example: ![Test image of a cat used for classification#center](cat.jpg "Test image: a cat photo for ImageNet classification") @@ -59,7 +59,7 @@ mean = np.array([0.485, 0.456, 0.406]) * 255 std = np.array([0.229, 0.224, 0.225]) * 255 x = (x - mean) / std -# NHWC -> NCHW +# HWC -> CHW x = np.transpose(x, (2, 0, 1)) # Quantize to int8 @@ -78,17 +78,14 @@ with open("input_image.h", "w") as f: f.write("const unsigned int input_image_len = 3 * 224 * 224;\n") ``` -The script performs these transformations: -1. Resizes the image to 224x224 pixels. -2. Applies ImageNet normalization (subtracts the dataset mean, divides by standard deviation). -3. Transposes from HWC (height, width, channels) to NCHW (batch, channels, height, width) layout. -4. Quantizes to int8 range (-128 to 127). -5. Writes a C header with the pixel data as a constant array. +The script resizes the image to 224 x 224 pixels and applies ImageNet normalization by subtracting the dataset mean and dividing by the standard deviation. It then transposes the layout from HWC (height, width, channels) to CHW (channels, height, width) format. The batch dimension is added later in the application. Finally, it quantizes the values to int8 range (-128 to 127) and writes the pixel data to a C header as a constant array. ## Run the script +From the image directory, run the preprocessing script: + ```bash -cd ~/repo/alif/image +cd ~/alif/image python prepare_image.py ``` @@ -97,8 +94,8 @@ This generates `input_image.h` in the same directory. The file is approximately ## Copy the header to the project ```bash -cp ~/repo/alif/image/input_image.h \ - ~/repo/alif/alif_vscode-template/mv2_runner/assets/ +cp ~/alif/image/input_image.h \ + ~/alif/alif_vscode-template/mv2_runner/assets/ ``` {{% notice Note %}} @@ -124,3 +121,9 @@ The model classifies among 1000 ImageNet categories. Some common class ranges: For a complete list, search for "ImageNet 1000 class labels" online. The test image is ready. The next section covers building, flashing, and verifying the inference output. + +## What you've learned and what's next + +You've created a Python script that converts a test image into a C header with the proper normalization and quantization for MobileNetV2 input. + +Next, you'll build the complete firmware, flash it to the DevKit, and verify the inference results using SEGGER RTT Viewer. diff --git a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/memory-configuration.md b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/memory-configuration.md index 020e947130..2c85b2027e 100644 --- a/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/memory-configuration.md +++ b/content/learning-paths/embedded-and-microcontrollers/alif-image-classification/memory-configuration.md @@ -7,7 +7,7 @@ layout: "learningpathall" ## Why memory configuration matters -The stock Alif VS Code template divides memory equally between the two Cortex-M55 cores and allocates modest stack/heap sizes suitable for simple examples like Blinky. A MobileNetV2 model with ExecuTorch needs significantly more: +The default Alif VS Code template divides memory equally between the two Cortex-M55 cores and allocates modest stack/heap sizes suitable for simple examples like Blinky. A MobileNetV2 model with ExecuTorch needs significantly more: - The embedded model is approximately 3.7 MB (stored in MRAM/flash). - The ExecuTorch runtime, operator libraries, and application code add another 800 KB of code. @@ -19,10 +19,10 @@ You need to reconfigure the MRAM allocation, stack/heap sizes, and linker script Open `device/ensemble/RTE/Device/AE822FA0E5597LS0_M55_HP/app_mem_regions.h`. -Change the following values from their stock defaults: +Change the following values from their defaults: -| Define | Stock value | New value | Purpose | -|--------|------------|-----------|---------| +| Define | Default value | New value | Purpose | +|--------|--------------|-----------|---------| | `APP_MRAM_HE_BASE` | `0x80000000` | `0x80580000` | Move HE core out of the way | | `APP_MRAM_HE_SIZE` | `0x00200000` | `0x00000000` | Give HE core zero MRAM | | `APP_MRAM_HP_BASE` | `0x80200000` | `0x80000000` | HP core starts at MRAM base | @@ -30,13 +30,13 @@ Change the following values from their stock defaults: | `APP_HP_STACK_SIZE` | `0x00002000` | `0x00004000` | 16 KB stack (doubled) | | `APP_HP_HEAP_SIZE` | `0x00004000` | `0x00010000` | 64 KB heap (quadrupled) | -The stock template splits MRAM 2 MB / 2 MB between the two cores. Since you're only using the HP core, you give it the entire 5.5 MB of available MRAM. The increased stack and heap accommodate ExecuTorch's initialization code, which uses more stack depth and a few small dynamic allocations. +The default template splits MRAM 2 MB / 2 MB between the two cores. Since you're only using the HP core, you give it the entire 5.5 MB of available MRAM. The increased stack and heap accommodate ExecuTorch's initialization code, which uses more stack depth and a few small dynamic allocations. ## Edit the linker script Open `device/ensemble/RTE/Device/AE822FA0E5597LS0_M55_HP/linker_gnu_mram.ld.src`. -You need three changes to this file. +Make the following three changes to this file. ### Add SRAM1 to the zero-initialization table @@ -99,7 +99,7 @@ Add the GOT entries after `KEEP(*(.jcr*))`: ``` {{% notice Note %}} -This was the hardest bug to find during development. Without these two lines, the firmware boots, loads the model, but crashes with a BusFault when ExecuTorch tries to call any virtual function. The GOT is like a phone book for indirect calls. If you don't copy it from flash to RAM at startup, every lookup finds address zero and the CPU faults. +This issue can be difficult to diagnose. Without these two lines, the firmware boots and loads the model, but crashes with a BusFault when ExecuTorch calls a virtual function. The GOT stores addresses for indirect calls. If the startup code doesn't copy it from flash to RAM, those lookups resolve to address zero and the CPU faults. {{% /notice %}} ### Add SRAM section wildcards @@ -182,4 +182,10 @@ The key fields are: You can view the completed versions of these edited files in the [workshop repository](https://github.com/ArmDeveloperEcosystem/workshop-ethos-u) for reference. -The memory layout and flash configuration are now ready. The next section covers preparing the test image. +The memory layout and flash configuration are complete. The next section covers preparing the test image. + +## What you've learned and what's next + +You've reconfigured the memory regions to allocate the full 5.5 MB of MRAM to the HP core, modified the linker script to support GOT relocation and SRAM memory pools, and configured the Security Toolkit JSON file to boot the application from the correct MRAM address. + +Next, you'll prepare a test image for classification by converting it to the format the model expects. diff --git a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/_index.md b/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/_index.md deleted file mode 100644 index 05c9989cbb..0000000000 --- a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/_index.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -title: Customer Support Chatbot with Llama and ExecuTorch on Arm-Based Mobile Devices (with Agentic AI Capabilities) - -draft: true -cascade: - draft: true - -minutes_to_complete: 60 - -who_is_this_for: This learning plan is designed for developers with basic knowledge of Python, Mobile development, and machine learning concepts.It guides you through creating an on-device customer support chatbot using Meta's Llama models deployed via PyTorch's ExecuTorch runtime.The focus is on Arm-based Android devices.The chatbot will handle common customer queries (e.g., product info, troubleshooting) with low latency, privacy (no cloud dependency), and optimized performance.Incorporates agentic AI capabilities, transforming the chatbot from reactive (simple Q&A) to proactive and autonomous. Agentic AI enables the bot to plan multi-step actions, use external tools,reason over user intent, and adapt responses dynamically. This is achieved by extending the core LLM with tool-calling mechanisms and multi-agent orchestration. - -learning_objectives: - - Explain the architecture and capabilities of Llama models (e.g., Llama 3.2 1B/3B) for mobile use. - - Master the process of quantizing LLMs (e.g., 4-bit PTQ) to reduce model size and enable efficient inference on resource-constrained mobile devices. - - Gain proficiency in using ExecuTorch to export PyTorch models to .pte format for on-device deployment. - - Learn to leverage Arm-specific optimizations (e.g., XNNPACK, KleidiAI) to achieve 2-3x faster inference on Arm-based Android devices. - - Implement real-time inference with Llama models, enabling seamless customer support interactions (e.g., handling FAQs, troubleshooting). - -prerequisites: - - Basic Understanding of Machine Learning & Deep Learning (Familiarity with concepts like supervised learning, neural networks, transfer learning and Understanding of model training, validation, & overfitting concepts). - - Familiarity with Deep Learning Frameworks (Experience with PyTorch for building, training neural networks and Knowledge of Hugging Face Transformers for working with pre-trained LLMs. - - An Arm-powered smartphone with the i8mm feature running Android, with 16GB of RAM. - - A USB cable to connect your smartphone to your development machine. - - An AWS Graviton4 r8g.16xlarge instance to test Arm performance optimizations, or any [Arm based instance](/learning-paths/servers-and-cloud-computing/csp/) from a cloud service provider or an on-premise Arm server or Arm based laptop. - - Android Debug Bridge (adb) installed on your device. Follow the steps in [adb](https://developer.android.com/tools/adb) to install Android SDK Platform Tools. The adb tool is included in this package. - - Java 17 JDK. Follow the steps in [Java 17 JDK](https://www.oracle.com/java/technologies/javase/jdk17-archive-downloads.html) to download and install JDK for host. - - Android Studio. Follow the steps in [Android Studio](https://developer.android.com/studio) to download and install Android Studio for host. - - Python 3.10. - -author: Parichay Das - -### Tags -skilllevels: Introductory -subjects: ML -armips: - - Neoverse - -tools_software_languages: - - LLM - - GenAI - - Python - - PyTorch - - ExecuTorch -operatingsystems: - - Linux - - Windows - - Android - - -further_reading: - - resource: - title: Hugging Face Documentation - link: https://huggingface.co/docs - type: documentation - - resource: - title: PyTorch Documentation - link: https://pytorch.org/docs/stable/index.html - type: documentation - - resource: - title: Android - link: https://www.android.com/ - type: website - - -### FIXED, DO NOT MODIFY -# ================================================================================ -weight: 1 # _index.md always has weight of 1 to order correctly -layout: "learningpathall" # All files under learning paths have this same wrapper -learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content. ---- diff --git a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/example-picture.png b/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/example-picture.png deleted file mode 100644 index c69844bed4..0000000000 Binary files a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/example-picture.png and /dev/null differ diff --git a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-1.md b/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-1.md deleted file mode 100644 index 224fa4013e..0000000000 --- a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-1.md +++ /dev/null @@ -1,34 +0,0 @@ ---- -title: Overview -weight: 2 - -### FIXED, DO NOT MODIFY -layout: learningpathall ---- - -## Understanding Llama: Meta’s Large Language Model -Llama is a family of large language models trained using publicly available datasets. These models demonstrate strong performance across a range of natural language processing (NLP) tasks, including language translation, question answering, and text summarization. - -In addition to their analytical capabilities, Llama models can generate human-like, coherent, and contextually relevant text, making them highly effective for applications that rely on natural language generation. Consequently, they serve as powerful tools in areas such as chatbots, virtual assistants, and language translation, as well as in creative and content-driven domains where producing natural and engaging text is essential. - -Please note that the models are subject to the [acceptable use policy](https://github.com/meta-llama/llama/blob/main/USE_POLICY.md) and this [responsible use guide](https://github.com/meta-llama/llama/blob/main/RESPONSIBLE_USE_GUIDE.md) . - - - -## Quantization -A practical approach to make models fit within smartphone memory constraints is through 4-bit groupwise per-token dynamic quantization of all linear layers. In this technique, dynamic quantization is applied to activations—meaning the quantization parameters are computed at runtime based on the observed minimum and maximum activation values. Meanwhile, the model weights are statically quantized, where each channel is quantized in groups using 4-bit signed integers. This method significantly reduces memory usage while maintaining model performance for on-device inference. - -This method ensures efficient memory usage while maintaining model performance on resource-constrained devices. - -For further information, refer to [torchao: PyTorch Architecture Optimization](https://github.com/pytorch-labs/ao/). - -The table below evaluates WikiText perplexity using [LM Eval](https://github.com/EleutherAI/lm-evaluation-harness). - -The results are for two different groupsizes, with max_seq_len 2048, and 1000 samples: - -|Model | Baseline (FP32) | Groupwise 4-bit (128) | Groupwise 4-bit (256) -|--------|-----------------| ---------------------- | --------------- -|Llama 2 7B | 9.2 | 10.2 | 10.7 -|Llama 3 8B | 7.9 | 9.4 | 9.7 - -Note that groupsize less than 128 was not enabled in this example, since the model was still too large. This is because current efforts have focused on enabling FP32, and support for FP16 is under way. diff --git a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-2.md b/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-2.md deleted file mode 100644 index 2c3a0c4822..0000000000 --- a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-2.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -title: Environment Setup -weight: 3 - -### FIXED, DO NOT MODIFY -layout: learningpathall ---- - -## Android NDK and Android Studio - Environment Setup - -#### Platform Required -- An AWS Graviton4 r8g.16xlarge instance to test Arm performance optimizations, or any [Arm based instance](/learning-paths/servers-and-cloud-computing/csp/) from a cloud service provider or an on-premise Arm server or Arm based laptop. -- An Arm-powered smartphone with the i8mm feature running Android, with 16GB of RAM. -- A USB cable to connect your smartphone to your development machine. - -The installation and configuration of Android Studio can be accomplished through the following steps: -1. Download and install the latest version of [Android Studio](https://developer.android.com/studio). -2. Launch Android Studio and access the Settings dialog. -3. Navigate to Languages & Frameworks → Android SDK. -4. Under the SDK Platforms tab, ensure that Android 14.0 (“UpsideDownCake”) is selected. - -Next, proceed to install the required version of the Android NDK by first setting up the Android Command Line Tools. -Linux: -```bash -curl https://dl.google.com/android/repository/commandlinetools-linux-11076708_latest.zip -o commandlinetools.zip -unzip commandlinetools.zip -./commandlinetools/bin/sdkmanager --install "ndk;26.1.10909697" -``` -Install the NDK in the same directory where Android Studio has installed the SDK, which is typically located at ~/Library/Android/sdk by default. Then, configure the necessary environment variables as follows: -```bash -export ANDROID_HOME="$(realpath ~/Library/Android/sdk)" -export PATH=$ANDROID_HOME/cmdline-tools/bin/:$PATH -sdkmanager --sdk_root="${ANDROID_HOME}" --install "ndk;28.0.12433566" -export ANDROID_NDK=$ANDROID_HOME/ndk/28.0.12433566/ -``` - -#### Install Java 17 JDK -1. Open the Java SE 17 Archive [Downloads](https://www.oracle.com/java/technologies/javase/jdk17-archive-downloads.html) page in your browser. -2. Select an appropriate download for your development machine operating system. - -#### Install Git and cmake -```bash -sudo apt-get install git cmake -``` - -#### Install Python 3.10 -```bash -sudo apt-get install python3.10 -``` - -#### Set up ExecuTorch -ExecuTorch is an end-to-end framework designed to facilitate on-device inference across a wide range of mobile and edge platforms, including wearables, embedded systems, and microcontrollers. As a component of the PyTorch Edge ecosystem, it streamlines the efficient deployment of PyTorch models on edge devices. For further details, refer to the [ExecuTorch Overview](https://pytorch.org/executorch/stable/overview/). - -It is recommended to create an isolated Python environment to install the ExecuTorch dependencies. Instructions are available for setting up either a Python virtual environment or a Conda virtual environment—you only need to choose one of these options. - -##### Install Required Tools ( Python environment setup) -```python -python3 -m venv exec_env -source exec_env/bin/activate -pip install torch torchvision torchaudio -pip install executorch -``` -##### Clone Required Repositories -```bash -git clone https://github.com/pytorch/executorch.git -git clone https://github.com/pytorch/text.git -``` -##### Download Pretrained Model (Llama 3.1 Instruct) -Download the quantized model weights optimized for mobile deployment from either the Meta AI Hub or Hugging Face. -``` -git lfs install -git clone https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct -``` - -##### Verify Arm SDK Path -``` -ANDROID_SDK_ROOT=/Users//Library/Android/sdk -ANDROID_NDK_HOME=$ANDROID_SDK_ROOT/ndk/26.1.10909125 -``` \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-3.md b/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-3.md deleted file mode 100644 index c05ddda728..0000000000 --- a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-3.md +++ /dev/null @@ -1,161 +0,0 @@ ---- -title: Model Preparation and Conversion -weight: 4 - -### FIXED, DO NOT MODIFY -layout: learningpathall ---- - -To begin working with Llama 3, the pre-trained model parameters can be accessed through Meta’s Llama Downloads page. Users are required to request access by submitting their details and reviewing and accepting the Responsible Use Guide. Upon approval, a license and a download link—valid for 24 hours—are provided. For this exercise, the Llama 3.2 1B Instruct model is utilized; however, the same procedures can be applied to other available variants with only minor modifications. - -Convert the model into an ExecuTorch-compatible format optimized for Arm devices -## Script the Model - -```python -import torch -from transformers import AutoModelForCausalLM - -model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct", torch_dtype=torch.float16) -scripted_model = torch.jit.script(model) -scripted_model.save("llama_exec.pt") - -``` - -Install the llama-stack package from pip. -```python -pip install llama-stack -``` - -Run the command to download, and paste the download link from the email when prompted. -```python -llama model download --source meta --model-id Llama3.2-1B-Instruct -``` - -When the download is finished, the installation path is printed as output. -```python -Successfully downloaded model to //.llama/checkpoints/Llama3.2-1B-Instruct -``` -Verify by viewing the downloaded files under this path: -``` -ls $HOME/.llama/checkpoints/Llama3.2-1B-Instruct -checklist.chk consolidated.00.pth params.json tokenizer.model - -``` - -Export the model and generate a .pte file by running the appropriate Python command. This command will export the model and save the resulting file in your current working directory. -```python -python3 -m examples.models.llama.export_llama \ ---checkpoint $HOME/.llama/checkpoints/Llama3.2-1B-Instruct/consolidated.00.pth \ ---params $HOME/.llama/checkpoints/Llama3.2-1B-Instruct/params.json \ --kv --use_sdpa_with_kv_cache -X --xnnpack-extended-ops -qmode 8da4w \ ---group_size 64 -d fp32 \ ---metadata '{"get_bos_id":128000, "get_eos_ids":[128009, 128001, 128006, 128007]}' \ ---embedding-quantize 4,32 \ ---output_name="llama3_1B_kv_sdpa_xnn_qe_4_64_1024_embedding_4bit.pte" \ ---max_seq_length 1024 \ ---max_context_length 1024 -``` - -Because Llama 3 has a larger vocabulary size, it is recommended to quantize the embeddings using the parameter --embedding-quantize 4,32. This helps to further optimize memory usage and reduce the overall model size. - - -###### Load a pre-fine-tuned model (from Hugging Face) -- Example: meta-llama/Llama-3-8B-Instruct or a customer-support fine-tuned variant - -###### Model Optimization for ARM (Understanding Quantization) -- Reduces model precision (e.g., 32-bit → 8-bit) -- Decreases memory footprint (~4x reduction) -- Speeds up inference on CPU -- Minimal accuracy loss for most tasks - -###### Apply Dynamic Quantization -- Create optimize_model.py - -```python -import torch -from transformers import AutoModelForCausalLM, AutoTokenizer -from torch.quantization import quantize_dynamic -import time -import os - -def load_base_model(model_name): - """Load the base model""" - print(f"Loading base model: {model_name}") - - tokenizer = AutoTokenizer.from_pretrained(model_name) - tokenizer.pad_token = tokenizer.eos_token - - model = AutoModelForCausalLM.from_pretrained( - model_name, - torch_dtype=torch.float32, - device_map=None, - low_cpu_mem_usage=True - ) - model.eval() - - return model, tokenizer - -def apply_quantization(model): - """Apply dynamic quantization""" - print("Applying dynamic quantization...") - - quantized_model = quantize_dynamic( - model, - {torch.nn.Linear}, # Quantize linear layers - dtype=torch.qint8 - ) - - return quantized_model - -def test_model(model, tokenizer, prompt): - """Test model with a sample prompt""" - inputs = tokenizer(prompt, return_tensors="pt") - - start_time = time.time() - with torch.no_grad(): - outputs = model.generate( - inputs.input_ids, - max_new_tokens=100, - do_sample=False, - pad_token_id=tokenizer.eos_token_id - ) - inference_time = time.time() - start_time - - response = tokenizer.decode(outputs[0], skip_special_tokens=True) - - return response, inference_time - -def main(): - model_name = "meta-llama/Meta-Llama-3-8B-Instruct" - - # Load base model - base_model, tokenizer = load_base_model(model_name) - - # Test base model - test_prompt = "How do I track my order?" - print("\nTesting base model...") - response, base_time = test_model(base_model, tokenizer, test_prompt) - print(f"Base model inference time: {base_time:.2f}s") - - # Apply quantization - quantized_model = apply_quantization(base_model) - - # Test quantized model - print("\nTesting quantized model...") - response, quant_time = test_model(quantized_model, tokenizer, test_prompt) - print(f"Quantized model inference time: {quant_time:.2f}s") - print(f"Speedup: {base_time / quant_time:.2f}x") - - # Save quantized model - save_dir = "./models/quantized_llama3" - os.makedirs(save_dir, exist_ok=True) - - torch.save(quantized_model.state_dict(), f"{save_dir}/model.pt") - tokenizer.save_pretrained(save_dir) - - print(f"\nQuantized model saved to: {save_dir}") - -if __name__ == "__main__": - main() - -``` \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-4.md b/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-4.md deleted file mode 100644 index e2291f4795..0000000000 --- a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-4.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -title: Building the Chatbot Logic - -### FIXED, DO NOT MODIFY -layout: learningpathall ---- - -## Conversation Framework (Python prototype) -```python -from transformers import AutoTokenizer -tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct") - -def generate_response(model, query, context): - prompt = f"### Context:\n{context}\n### User Query:\n{query}\n### Assistant Response:" - inputs = tokenizer(prompt, return_tensors="pt") - outputs = model.generate(**inputs, max_new_tokens=200) - return tokenizer.decode(outputs[0], skip_special_tokens=True) -``` - -###### Context Memory (Simple JSON Store) - -```python -import json - -def update_memory(user_id, query, response): - memory = json.load(open("chat_memory.json", "r")) - memory[user_id].append({"query": query, "response": response}) - json.dump(memory, open("chat_memory.json", "w")) - -``` - diff --git a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-5.md b/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-5.md deleted file mode 100644 index b155e4245b..0000000000 --- a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-5.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -title: Adding Agentic AI Capabilities -weight: 6 - -### FIXED, DO NOT MODIFY -layout: learningpathall ---- -Enable the chatbot to perform reasoning, make decisions, and execute actions autonomously - -## Define Agentic Loop -```python -class AgenticChatbot: - def __init__(self, model): - self.model = model - - def observe(self, input): - return f"User said: {input}" - - def think(self, observation): - return f"Decide best next step based on intent." - - def act(self, decision): - if "refund" in decision: - return "Processing refund..." - elif "troubleshoot" in decision: - return "Let's check your device settings." - else: - return "Connecting you with an agent." - - def respond(self, query): - obs = self.observe(query) - thought = self.think(obs) - action = self.act(thought) - return f"Reasoning: {thought}\nAction: {action}" -``` -## Integrate Llama with Reasoning Loop -```python -def generate_agentic_response(query, context): - reasoning = agent.respond(query) - model_response = generate_response(model, query, context) - return reasoning + "\n\n" + model_response -``` \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-6.md b/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-6.md deleted file mode 100644 index dbd78eff20..0000000000 --- a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-6.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -title: Android Integration -weight: 7 - -### FIXED, DO NOT MODIFY -layout: learningpathall ---- - -#### Integration -Build the Llama Runner Binary for Android -Cross-compile the Llama Runner to enable execution on Android by following the steps outlined below. - -#### Android NDK -Configure the environment variable to reference the Android NDK -``` -export ANDROID_NDK=$ANDROID_HOME/ndk/28.0.12433566/ -``` - -Ensure that $ANDROID_NDK/build/cmake/android.toolchain.cmake is accessible so CMake can perform cross-compilation. - -#### Use KleidiAI to build ExecuTorch and the required libraries for Android deployment -build ExecuTorch for Android, leveraging the performance optimizations offered by [KleidiAI](https://gitlab.arm.com/kleidi/kleidiai) kernels - -Use cmake to cross-compile ExecuTorch: -``` -cmake -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \ - -DANDROID_ABI=arm64-v8a \ - -DANDROID_PLATFORM=android-23 \ - -DCMAKE_INSTALL_PREFIX=cmake-out-android \ - -DEXECUTORCH_ENABLE_LOGGING=1 \ - -DCMAKE_BUILD_TYPE=Release \ - -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \ - -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \ - -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \ - -DEXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR=ON \ - -DEXECUTORCH_BUILD_XNNPACK=ON \ - -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON \ - -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON \ - -DEXECUTORCH_BUILD_KERNELS_CUSTOM=ON \ - -DEXECUTORCH_BUILD_KERNELS_LLM=ON \ - -DEXECUTORCH_BUILD_EXTENSION_LLM_RUNNER=ON \ - -DEXECUTORCH_BUILD_EXTENSION_LLM=ON \ - -DEXECUTORCH_BUILD_EXTENSION_RUNNER_UTIL=ON \ - -DEXECUTORCH_XNNPACK_ENABLE_KLEIDI=ON \ - -DXNNPACK_ENABLE_ARM_BF16=OFF \ - -DBUILD_TESTING=OFF \ - -Bcmake-out-android . - -cmake --build cmake-out-android -j7 --target install --config Release -``` -Beginning with ExecuTorch version 0.7 beta, KleidiAI is enabled by default. The option -DEXECUTORCH_XNNPACK_ENABLE_KLEIDI=ON is active, providing built-in support for KleidiAI kernels within ExecuTorch when using XNNPack. - -#### Build Llama runner for Android -``` -cmake -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \ - -DANDROID_ABI=arm64-v8a \ - -DANDROID_PLATFORM=android-23 \ - -DCMAKE_INSTALL_PREFIX=cmake-out-android \ - -DCMAKE_BUILD_TYPE=Release \ - -DPYTHON_EXECUTABLE=python \ - -DEXECUTORCH_BUILD_XNNPACK=ON \ - -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON \ - -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON \ - -DEXECUTORCH_BUILD_KERNELS_CUSTOM=ON \ - -DSUPPORT_REGEX_LOOKAHEAD=ON \ - -DBUILD_TESTING=OFF \ - -Bcmake-out-android/examples/models/llama \ - examples/models/llama - -cmake --build cmake-out-android/examples/models/llama -j16 --config Release - -``` -Execute on Android using adb shell. -You will need an Arm-based Android smartphone with the i8mm feature and at least 16GB of RAM. The steps below were validated on a Google Pixel 8 Pro. -#### Create New Android Project -Open Android Studio → New Project → Empty Activity - -#### Add ExecuTorch Runtime to build.gradle -``` -dependencies { - implementation files('libs/executorch.aar') -} -``` - -#### Android phone connection -Connect your Android device to your computer using a USB cable. - -Ensure that USB debugging is enabled on your device. You can follow the Configure on-device developer options guide to enable it. - -After enabling USB debugging and connecting the device via USB, run the following command: -``` -adb devices -``` - -#### model, tokenizer, and Llama runner -``` -adb shell mkdir -p /data/local/tmp/llama -adb push llama3_1B_kv_sdpa_xnn_qe_4_64_1024_embedding_4bit.pte /data/local/tmp/llama/ -adb push $HOME/.llama/checkpoints/Llama3.2-1B-Instruct/tokenizer.model /data/local/tmp/llama/ -adb push cmake-out-android/examples/models/llama/llama_main /data/local/tmp/llama/ - -``` - -#### Model Running -``` -adb shell "cd /data/local/tmp/llama && ./llama_main --model_path llama3_1B_kv_sdpa_xnn_qe_4_64_1024_embedding_4bit.pte --tokenizer_path tokenizer.model --prompt '<|start_header_id|>system<|end_header_id|>\nYour name is Cookie. you are helpful, polite, precise, concise, honest, good at writing. You always give precise and brief answers up to 32 words<|eot_id|><|start_header_id|>user<|end_header_id|>\nHey Cookie! how are you today?<|eot_id|><|start_header_id|>assistant<|end_header_id|>' --warmup=1 --cpu_threads=5" -``` \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-7.md b/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-7.md deleted file mode 100644 index 422f8c8d78..0000000000 --- a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-7.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -title: Run, Testing and Benchmarking -weight: 8 - -### FIXED, DO NOT MODIFY -layout: learningpathall ---- - -#### Build the Android (AAR) -You can use the Android demo application included in the ExecuTorch repository, [LlamaDemo](https://github.com/pytorch/executorch/tree/main/examples/android/LlamaDemo), to showcase local inference with ExecuTorch - -Open a terminal and navigate to the root directory of the ExecuTorch repository.Then, set the following environment variables: - -```bash -export ANDROID_NDK=$ANDROID_HOME/ndk/28.0.12433566/ -export ANDROID_ABI=arm64-v8a -``` -Run the following commands to set up the required JNI library: -```bash -pushd extension/android -./gradlew build -popd -pushd examples/demo-apps/android/LlamaDemo -./gradlew :app:setup -popd -``` -Check if the files are available on the phone: -```bash -adb shell "ls -la /data/local/tmp/llama/" -``` -If not, copy them: -``` -adb shell mkdir -p /data/local/tmp/llama -adb push /data/local/tmp/llama/ -adb push /data/local/tmp/llama/ -``` - -#### Build the Android Package Kit using Android Studio -- Open Android Studio and choose Open an existing Android Studio project. -- Navigate to examples/demo-apps/android/LlamaDemo and open it. -- Run the app (^R) to build and launch it on your connected Android device. - -#### Measure Inference Latency -- adb shell am start -n com.example.chatbot/.MainActivity -- adb shell dumpsys meminfo com.example.chatbot diff --git a/content/learning-paths/embedded-and-microcontrollers/device-connect-strands/_index.md b/content/learning-paths/embedded-and-microcontrollers/device-connect-strands/_index.md new file mode 100644 index 0000000000..7a35bea9dd --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/device-connect-strands/_index.md @@ -0,0 +1,63 @@ +--- +title: Connect AI agents to edge devices using Device Connect and Strands + +draft: true +cascade: + draft: true + +minutes_to_complete: 30 + +who_is_this_for: This is an introductory topic for software developers who want to connect AI agents to edge devices. You'll use Device Connect, Arm's platform for structured device access, and Strands, AWS's open-source agent SDK. The examples cover both physical and simulated devices. + +learning_objectives: + - Understand how Device Connect and Strands work together to give AI agents structured access to Arm-based edge devices + - Set up a Python environment with the Device Connect SDK and agent tools installed from source + - Start a simulated robot that registers itself on the local network and is discovered automatically by an agent + - Discover and invoke the robot using the Device Connect agent tools and the robot_mesh Strands tool + +prerequisites: + - A development machine with Python 3.12 installed + - Git installed + - Basic familiarity with Python virtual environments and command-line tools + - (Optional) A Raspberry Pi for testing a full device-to-device (D2D) setup + +author: + - Annie Tallund + - Kavya Sri Chennoju + + + +### Tags +skilllevels: Introductory +subjects: ML +armips: + - Cortex-A + - Neoverse +operatingsystems: + - Linux + - macOS +tools_software_languages: + - Python + - Docker + - strands-agents + +further_reading: + - resource: + title: Strands Agents SDK documentation + link: https://strandsagents.com/ + type: website + - resource: + title: Strands robots repository + link: https://github.com/strands-labs/robots/tree/dev + type: website + - resource: + title: Device Connect integration guide + link: https://github.com/atsyplikhin/robots/blob/feat/device-connect-integration-draft/strands_robots/device_connect/GUIDE.md + type: website + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 # _index.md always has weight of 1 to order correctly +layout: "learningpathall" # All files under learning paths have this same wrapper +learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content. +--- diff --git a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/_next-steps.md b/content/learning-paths/embedded-and-microcontrollers/device-connect-strands/_next-steps.md similarity index 100% rename from content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/_next-steps.md rename to content/learning-paths/embedded-and-microcontrollers/device-connect-strands/_next-steps.md diff --git a/content/learning-paths/embedded-and-microcontrollers/device-connect-strands/background.md b/content/learning-paths/embedded-and-microcontrollers/device-connect-strands/background.md new file mode 100644 index 0000000000..5f5b1e9328 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/device-connect-strands/background.md @@ -0,0 +1,71 @@ +--- +title: Learn Device Connect and Strands architecture for edge devices +weight: 2 + +# FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Why connect AI agents to edge devices? + +Arm processors are at the heart of a remarkable range of systems - from Cortex-M microcontrollers in industrial sensors to Neoverse servers running in the cloud. That breadth of hardware is one of Arm's greatest strengths, but it raises a practical question for AI developers: how do you give an agent structured, safe access to devices that are physically distributed and built on different software stacks? + +Device Connect is Arm's answer to that question. It's a platform layer that handles device registration, discovery, and remote procedure calls across a network of devices, with no bespoke networking code required. Strands is an open-source agent SDK from AWS that takes a model-driven approach to building AI agents - an LLM calls Python tools in a structured reasoning loop, and the SDK handles the rest. When you combine them, an agent can ask "which devices are online and what can they do?" and then invoke a function on a specific device, turning natural language intent into physical action. + +This Learning Path puts both tools through their paces. It starts with a single machine, for example a laptop, where a simulated robot and an agent discover each other automatically, then extends to a two-machine setup where a Raspberry Pi joins the same device mesh over the network. + +## Device Connect architecture layers + +**Device layer** + +A device is any process that registers itself on the mesh and exposes callable functions. In this Learning Path you'll create a simulated robot arm, namely the simulated robotic arm SO-100 from Hugging Face, from the `strands-robots` SDK. The moment this object is created, it registers on the local network under a unique device ID (for example, `so100_sim-abc23`) and begins publishing a presence heartbeat. No explicit registration call is required. Device Connect uses Zenoh as its underlying messaging transport, which handles low-level connectivity and routing automatically. + +**Agent layer** + +Two interfaces sit at this layer. The `device-connect-agent-tools` package exposes `discover_devices()` and `invoke_device()` as plain Python functions you can call directly from a script or REPL, with no LLM involved. The `robot_mesh` tool from `robots` wraps the same capabilities as a Strands agent tool, which means an LLM can also call them during a reasoning loop. Both share the same underlying Device Connect transport, so anything you can do with one you can do with the other. + +The diagram below shows how these layers communicate at runtime: + +``` +┌──────────────────────────────────────┐ +│ Agent layer │ +│ discover_devices · invoke_device │ +│ robot_mesh Strands tool │ +└──────────────┬───────────────────────┘ + │ Device Connect + │ (device-to-device discovery & RPC) +┌──────────────▼───────────────────────┐ +│ Device layer │ +│ Simulated SO-100 arm | +| - so100-abc123 │ +│ heartbeat · execute · getStatus │ +└──────────────────────────────────────┘ +``` + +## How device discovery works + +When the `SO-100 arm` instance starts, Device Connect automatically announces the device on the local network. Any process running `discover_devices()` or `robot_mesh(action='peers')` on the same network will hear the announcement and add the device to its live table of available hardware. + +## What the simulated robot provides + +When you run `Robot('so100')`, the SDK downloads the MuJoCo physics model for the SO-100 arm (this happens once on first run) and starts a local simulation. The robot exposes three functions that any agent can call via RPC: + +- `execute` - start a task with a given instruction and policy provider +- `getStatus` - query the current task state +- `stop` - halt the current task + +For this Learning Path, the `policy_provider='mock'` argument is used, which means `execute` accepts the call and returns `{'status': 'accepted'}` without actually running a motion policy. This keeps the focus on the connectivity and invocation patterns rather than robotics. + +Once you have the flow working end to end, replacing `'mock'` with a real policy is a one-line change. + +## What you'll learn in this Learning Path + +By working through the remaining sections you'll: + +- Clone the sample repository and install the Device Connect SDK, agent tools, and Strands robot runtime from source into a single virtual environment. +- Start a simulated robot that registers itself on the local device mesh. +- Discover and invoke the robot using `device-connect-agent-tools` directly. +- Discover and command the robot through the `robot_mesh` Strands tool, including an emergency stop. +- Optionally extend the setup to a Raspberry Pi connected over the network, discovering and commanding it from your laptop through the Device Connect infrastructure. + +The next section covers the environment setup. \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/device-connect-strands/run-example.md b/content/learning-paths/embedded-and-microcontrollers/device-connect-strands/run-example.md new file mode 100644 index 0000000000..759306eac5 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/device-connect-strands/run-example.md @@ -0,0 +1,196 @@ +--- +title: Run the example end to end +weight: 4 + +# FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Overview + +This section runs Device Connect's device-to-device discovery. There are two ways to walk through this setup. Optionally, you can connect an external device. + +### Option 1: run on a single machine + +For a proof-of-concept, follow the steps using two terminal windows on your machine with the virtual environment set up. + +### Option 2: run with real hardware + +If you have access to an external device, you can use it with this setup as well. You'll need: + +- Your machine with the virtual environment set up. This machine will be referred to as the host. +- A Raspberry Pi (or any similar device) connected to the same network as your host machine. This machine is your target. +- An SSH connection or keyboard and monitor attached to the target. + +You'll need two terminal windows open at the same time: one to keep the simulated robot running, and one to invoke it from the agent side. Both terminals need the virtual environment activated. + +| Machine | Terminal | Purpose | +|---------|----------|---------| +| Host or Target | 1 | Simulated robot - keep running throughout | +| Host | 2 | Agent tool invocations | + +Make sure you are in the repository directory and that your virtual environment is activated: + +```bash +cd ~/strands-device-connect/robots +source .venv/bin/activate +``` + +## Start the simulated robot + +In terminal 1, run the following command to create and start the simulated SO-100 robot arm: + +```python +python <<'PY' +import logging +logging.basicConfig(level=logging.INFO) +from strands_robots import Robot +r = Robot('so100') +r.run() +PY +``` + +When the `Robot('so100')` object is created, the SDK downloads the MuJoCo physics model for the SO-100 arm. This download happens only on the first run and takes a minute or two. After that, it starts the simulation and registers the robot on the Device Connect device mesh. The robot publishes a presence heartbeat every 0.5 seconds under a unique device ID, for example `so100-abc123`. + +You should see INFO-level log output similar to: + +```output +device_connect_sdk.device.so100-abc123 - INFO - Using ZENOH messaging backend +device_connect_sdk.device.so100-abc123 - INFO - Connected to ZENOH broker: [] +device_connect_sdk.device.so100-abc123 - INFO - Driver connected: strands_sim +device_connect_sdk.device.so100-abc123 - INFO - Subscribed to commands on device-connect.default.so100-abc123.cmd +🤖 so100-abc123 is online. Ctrl+C to stop. +``` + +Leave this process running. The simulated robot is only discoverable as long as this process is alive. + +## Control the robot using the robot_mesh Strands tool + +The `robot_mesh` tool wraps the same discovery and invocation primitives as a Strands agent tool. You can call it directly from a Python script or attach it to an LLM agent; the API is identical either way. + +### Discover available robots on the device mesh + +Start by confirming which robots are currently visible on the mesh: + +```python +python <<'PY' +from strands_robots.tools.robot_mesh import robot_mesh +print(robot_mesh(action='peers')) +PY +``` + +The output is similar to: + +```output +Discovered 1 device(s): + [robot] so100-abc123 - idle + Functions: execute, getFeatures, getStatus, reset, step, stop +``` + +The peer ID (for example `so100-abc123`) is assigned at startup and changes each run. Note the actual ID shown in your terminal - you'll need it in the next step. + +### Execute an instruction + +Send a task to one of the discovered robots. Replace `so100-abc123` with the peer ID shown in your `peers` output: + +```python +python <<'PY' +from strands_robots.tools.robot_mesh import robot_mesh +print(robot_mesh( + action='tell', + target='so100-abc123', + instruction='pick up the cube', + policy_provider='mock', +)) +PY +``` + +You will see the following output: + +```output +-> so100-abc123: pick up the cube + {"status": "success", "content": [...]} +``` + +{{% notice Robot output in terminal 1 %}} +The robot also logs event updates as it processes the task. If you switch back to terminal 1, it logs the execution of the task, similar to: + +```output +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +*** EVENT so100-abc123::stateUpdate [111aaabb] + payload: sim_time=4.34, step_count=2070, running_policies={'so100': {'steps': 814, 'instruction': 'pick up the cube'}} +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +``` +{{% /notice %}} + + +### Broadcast emergency stop to all devices + +The emergency stop broadcasts a halt command to every device on the mesh simultaneously. This is useful when you want to stop all robots without knowing their individual peer IDs: + +```python +python <<'PY' +from strands_robots.tools.robot_mesh import robot_mesh +print(robot_mesh(action='emergency_stop')) +PY +``` + +The output is similar to: + +```output +E-STOP: 1/1 devices stopped +``` + +## Optional: discover and invoke the robot using the agent tools + +The `device-connect-agent-tools` package gives you direct programmatic access to the mesh, without involving an LLM. This is useful for testing, scripting, or validating the stack before wiring it up to an agent. Open terminal 2, activate the virtual environment, then run: + +```python +python <<'PY' +from device_connect_agent_tools import connect, discover_devices, invoke_device + +connect() + +devices = discover_devices(device_type='') +print(f'Found {len(devices)} robot(s):') +for d in devices: + print(f' {d["device_id"]}') + +if devices: + result = invoke_device( + devices[0]['device_id'], + 'execute', + {'instruction': 'pick up the cube', 'policy_provider': 'mock'}, + ) + print(f'Execute result: {result}') + + status = invoke_device(devices[0]['device_id'], 'getStatus') + print(f'Status: {status}') +PY +``` + +{{% notice About the snippet %}} +When an agent calls `execute(instruction="pick up the cup", policy_provider="groot")`, Device Connect handles the RPC delivery, and the policy handles the actual arm movement. + +`discover_devices(device_type='')` returns all devices on the mesh regardless of type. If you pass `device_type='strands_robot'` you can filter to only `Robot()` instances. `invoke_device` sends an RPC to the named device; here `policy_provider='mock'` tells the robot to accept the task without executing real motion, which is appropriate for this connectivity test. +{{% /notice %}} + + +The output is similar to: + +```output +Found 1 robot(s): + so100-abc123 - idle +Execute result: {'success': True, 'result': {'status': 'success', 'content': [...]}} +Status: {'success': True, 'result': {...}} # full sim state dict +``` + +## What you've learned and what's next + +In this section you've: + +- Started a simulated SO-100 robot that registered itself on the Device Connect device mesh. +- Used `device-connect-agent-tools` to discover the robot and invoke an RPC call against it. +- Used the `robot_mesh` Strands tool to list peers, send an instruction, and trigger an emergency stop. + +This showcases the ease of setting up a mesh on a local network. In the next section, you can extend the configuration to a Docker-based approach, opening up a new category of possibilities with agent integration. \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/device-connect-strands/run-infra-example.md b/content/learning-paths/embedded-and-microcontrollers/device-connect-strands/run-infra-example.md new file mode 100644 index 0000000000..7d3adbc131 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/device-connect-strands/run-infra-example.md @@ -0,0 +1,199 @@ +--- +title: Run with full Device Connect infrastructure (optional) +weight: 5 + +# FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Why add infrastructure? + +The previous section ran entirely on a local network, with Device Connect handling device-to-device discovery automatically. That approach is fast and requires zero configuration, but it has natural limits: both the robot and the agent must be on the same LAN, device state is ephemeral, and there's no registry you can query by device type. + +This section goes one step further. You'll run the Zenoh router, an etcd state store, and a registry service on your machine using Docker, then connect a Raspberry Pi on the same network as the remote device. You can use a different device as long as you can access it, but the Raspberry Pi will be used as an example. This device is also referred to as the target. + +The agent running on your machine will discover the robot running on the Pi through the infrastructure, as if both were part of the same managed fleet. The Pi doesn't need to run Docker - it just needs Python and the packages from setup. + +Confirm Docker and Docker Compose v2 are available on the host before continuing: + +```bash +docker --version +docker compose version +``` + +## Clone and bring up the Docker image + +```bash +cd ~/strands-device-connect +git clone --depth 1 https://github.com/arm/device-connect.git + +``` + +## Machine and terminal layout + +This section involves two machines. Keep track of which commands run where: + +| Machine | Terminal | Purpose | +|---------|----------|---------| +| Host | 1 | Docker Compose infrastructure | +| Host | 2 | Agent tool invocations | +| Target | - | Robot process | + +## Step 1 - start the infrastructure on your host machine + +In host terminal 1, bring up the Device Connect infrastructure stack. The Compose file is inside the `device-connect` repository you cloned during setup: + +```bash +cd ~/strands-device-connect/device-connect/packages/device-connect-server +docker compose -f infra/docker-compose-dev.yml up -d +cd ../../.. +``` + +Confirm the services are healthy: + +```bash +docker compose -f infra/docker-compose-dev.yml ps +``` + +The output is similar to: + +```output +NAME STATUS PORTS +zenoh-router running 0.0.0.0:7447->7447/tcp +etcd running 0.0.0.0:2379->2379/tcp +device-registry running 0.0.0.0:8080->8080/tcp +``` + +All three services must show `running` before you continue. The router on port 7447 is the single rendezvous point for all device traffic. Every device on any machine points at this address to join the mesh. + +## Step 2 - find your host's IP address + +The Raspberry Pi needs to connect to the Device Connect router on your host by IP address. Find it now: + +```bash +# macOS +ipconfig getifaddr en0 + +# Linux +hostname -I | awk '{print $1}' +``` + +Note the address returned - for the rest of this section it's referred to as `HOST_IP`. For example, if the command returns `192.168.1.42`, replace every occurrence of `HOST_IP` below with that address. + +## Step 3 - prepare the Raspberry Pi + +On the Raspberry Pi, follow the same repository and environment setup from the setup section of this Learning Path: install Python 3.12, clone the `robots` repository, create the virtual environment, and install the packages with the same editable install commands. + +Once the environment is ready, export the three variables that tell the SDK to route traffic through the Device Connect router on your host rather than using local network discovery: + +```bash +export MESSAGING_BACKEND=zenoh +export ZENOH_CONNECT=tcp/HOST_IP:7447 +export DEVICE_CONNECT_ALLOW_INSECURE=true +``` + +Replace `HOST_IP` with the address you noted in Step 2. `DEVICE_CONNECT_ALLOW_INSECURE=true` disables mTLS for this local development setup - don't use this flag in production. + +## Step 4 - start the robot on the Raspberry Pi + +On the Raspberry Pi, with the environment active and the variables set, start the simulated SO-100 robot: + +```python +python <<'PY' +import logging +logging.basicConfig(level=logging.INFO) +from strands_robots import Robot +r = Robot('so100') +r.run() +PY +``` + +Because `ZENOH_CONNECT` points at your host, the SDK routes traffic through the Device Connect router instead of using local network discovery. The robot registers with the persistent registry and you should see output similar to: + +```output +INFO:strands_robots.mesh:Zenoh session started +INFO:strands_robots.mesh:Peer ID: so100-abc123 +device_connect_sdk.device.so100-abc123 - INFO - Using ZENOH messaging backend +device_connect_sdk.device.so100-abc123 - INFO - Connected to ZENOH broker: ['tcp/192.168.1.42:7447'] +device_connect_sdk.device.so100-abc123 - INFO - Device registered: registration_id=ecfff6a7-... +``` + +Note the peer ID (for example `so100-abc123`). You'll need it in the `tell` command below. Leave this process running on the Pi. + +## Discover and invoke using the robot_mesh Strands tool + +In host terminal 2 (with the environment variables set), use `robot_mesh` to confirm the Pi's robot is visible as a peer: + +```python +python <<'PY' +from strands_robots.tools.robot_mesh import robot_mesh +print(robot_mesh(action='peers')) +PY +``` + +The output is similar to: + +```output +Discovered 1 device(s): + [robot] so100-abc123 - idle + Functions: execute, getFeatures, getState, getStatus, stop +``` + +Send an instruction to the robot. Replace `so100-abc123` with the peer ID shown in your output: + +```python +python <<'PY' +from strands_robots.tools.robot_mesh import robot_mesh +print(robot_mesh( + action='tell', + target='so100-abc123', + instruction='pick up the cube', + policy_provider='mock', +)) +PY +``` + +The output is similar to: + +```output +-> so100-abc123: pick up the cube + {"status": "accepted"} +``` + +Trigger an emergency stop across every device registered with the infrastructure: + +```python +python <<'PY' +from strands_robots.tools.robot_mesh import robot_mesh +print(robot_mesh(action='emergency_stop')) +PY +``` + +The output is similar to: + +```output +E-STOP: 1/1 devices stopped +``` + +The stop broadcast reaches every device in the registry - whether it is running on your host, the Pi on your desk, or a machine in a remote lab. + +## Shut down cleanly + +Stop the robot process on the Pi with `Ctrl+C`, then bring down the infrastructure on your host: + +```bash +cd ~/strands-device-connect/device-connect/packages/device-connect-server +docker compose -f infra/docker-compose-dev.yml down +``` + +This removes the containers and clears the in-container etcd state. Your virtual environment and cloned repositories remain intact on both machines. + +## What you've learned and what's next + +In this section you've: + +- Started a persistent Device Connect infrastructure stack on your host - a router, etcd, and a device registry. +- Connected a Raspberry Pi as a remote device by pointing its SDK at the router's TCP address. +- Discovered the Pi's robot from your host by querying the persistent registry and sent commands to it across the network. + +This is a deliberately simple two-device setup, but it demonstrates the foundation for something much larger. Once devices register through a shared infrastructure, agents can discover and command any of them without caring where they run - a fleet of robot arms, a network of sensors, or a mix of physical and simulated devices all become equally reachable. Adding more devices is just a matter of pointing them at the same router. That's the core of what Device Connect makes possible: a mesh of heterogeneous devices that agents can reason about and act on, at any scale. diff --git a/content/learning-paths/embedded-and-microcontrollers/device-connect-strands/setup.md b/content/learning-paths/embedded-and-microcontrollers/device-connect-strands/setup.md new file mode 100644 index 0000000000..9730e2282a --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/device-connect-strands/setup.md @@ -0,0 +1,69 @@ +--- +title: Set up the developer environment +weight: 3 + +# FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Verify the required tools + +Before cloning any repositories, confirm that Python 3.12 and Git are available: + +```bash +python3.12 --version +git --version +``` + +These instructions are tested on Python 3.12. Earlier versions of Python 3 may work but are not validated against the `feat/device-connect-integration-draft` branch used in this Learning Path. + +## Clone the repository + +The code run in this Learning Path sits in the `robots` repository. It contains the robot runtime and the `robot_mesh` Strands tool. + +```bash +mkdir ~/strands-device-connect +cd strands-device-connect +git clone https://github.com/atsyplikhin/robots.git +``` + +## Check out the integration branch + +The Device Connect integration code for `robots` lives on the `feat/device-connect-integration-draft` branch. This branch adds the `RobotDeviceDriver` adapter and the updated `robot_mesh` tool that routes calls through the Device Connect SDK rather than the raw Zenoh mesh. + +```bash +cd ~/strands-device-connect/robots +git checkout feat/device-connect-integration-draft +cd .. +``` + +## Create a Python virtual environment + +Create a single virtual environment at the workspace root, then activate it: + +```bash +python3.12 -m venv .venv +source .venv/bin/activate +``` + +Now install the packages and make sure they are available on your `PYTHONPATH` environment variable: + +```bash +pip install -e ".[sim]" +export PYTHONPATH="$PWD:$PYTHONPATH" +``` + +## How discovery works - no configuration needed + +The `strands-robots` SDK uses Device Connect's built-in device-to-device discovery: every `Robot()` instance announces itself on the local network at startup, and any process running `discover_devices()` or `robot_mesh(action='peers')` on the same network segment will find it automatically. + +This means discovery works as long as the device process and the agent process are on the same LAN or on the same machine. Discovery is typically available on home and office networks. If you are behind a firewall or VPN that blocks local network traffic, devices will not discover each other - that scenario requires the infrastructure-backed setup with a Zenoh router, which is covered later in this Learning Path. + +## What you've set up and what's next + +At this point you've: + +- Cloned `robots` with the `feat/device-connect-integration-draft` branch checked out. +- Created a Python 3.12 virtual environment with the Device Connect SDK, agent tools, and robot simulation runtime all installed. + +The next section walks you through starting a simulated robot and invoking it from both the agent tools and the `robot_mesh` Strands tool. \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/1-overview.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/1-overview.md index 788cb474a7..0e87ef1f6f 100644 --- a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/1-overview.md +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/1-overview.md @@ -1,55 +1,55 @@ --- -title: Overview +title: Understand ExecuTorch deployment on NXP with Ethos-U weight: 2 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Hardware Overview - NXP's FRDM i.MX 93 Board +## Before you begin -Selecting the best hardware for machine learning (ML) models depends on effective tools. You can visualize ML performance early in the development cycle by using NXP's [FRDM i.MX 93](https://www.nxp.com/design/design-center/development-boards-and-designs/frdm-i-mx-93-development-board:FRDM-IMX93) board. +This Learning Path assumes your FRDM i.MX 93 board is already set up and you can transfer files between your host machine and the board. -
- +If you still need to set up Linux, serial console access, and file transfer, follow the Learning Path [Linux on an NXP FRDM i.MX 93 board](/learning-paths/embedded-and-microcontrollers/linux-nxp-board/) before continuing. -*Unboxing NXP's FRDM i.MX 93 board* -
+ExecuTorch is designed to scale from servers to endpoints, and Arm systems often scale within a *single device*. +The FRDM i.MX 93 platform combines: -![NXP FRDM i.MX 93 Board SoC Highlighted alt-text#center](./nxp-frdm-imx-93-board-soc-highlighted.png "Arm Ethos-U65 NPU location") +- An application processor running Linux (Cortex-A) that handles system services and orchestration +- A Cortex-M33 microcontroller core that runs real-time firmware +- An Ethos-U65 NPU that accelerates TinyML inference -### NXP's FRDM i.MX 93 Processor Decoded +This Learning Path focuses on a concrete milestone: successful bring-up of an ExecuTorch `executor_runner` firmware on Cortex-M33 on this NXP platform. -![i.MX 93 Processor SoC alt-text#center](./imx-93-application-processor-soc.png "NXP's FRDM i.MX 93 processor") +This example keeps the Linux side intentionally simple. Linux loads and starts the Cortex-M33 firmware through RemoteProc, and you stage a compiled ExecuTorch `.pte` model so the firmware can run it. -**NXP's Processor Labeling Convention:** -|Line|Meaning| -|----|-------| -|MIMX9352|• MI – Microcontroller IC
• MX93 – i.MX 93 family
• 52 – Variant:
• Dual-core Arm Cortex-A55
• Single Cortex-M33
• Includes **Ethos-U65 NPU**| -|CVVXMAB|• C - Commercial temperature grade (0°C to 95°C)
• VVX - Indicates package type and pinout (BGA, pitch, etc.)
• MAB - Specific configuration (e.g., NPU present, security level, memory interfaces) -| -|1P87F|• Silicon mask set identifier| -|SBBM2410E|• NXP traceability code| +## What you’ll build and validate -## Software Overview - NXP's MCUXpresso IDE +By the end of this Learning Path, you'll have: -NXP generously provides free software for working with their boards, the [MCUXpresso Integrated Development Environment (IDE)](https://www.nxp.com/design/design-center/software/development-software/mcuxpresso-software-and-tools-/mcuxpresso-integrated-development-environment-ide:MCUXpresso-IDE). In this learning path, you will instead use [MCUXpresso for Visual Studio Code](https://www.nxp.com/design/design-center/software/development-software/mcuxpresso-software-and-tools-/mcuxpresso-for-visual-studio-code:MCUXPRESSO-VSC). +- A `.pte` model artifact compiled for `ethos-u65-256` +- A Cortex-M33 `executor_runner` firmware image built against prebuilt ExecuTorch libraries +- A repeatable deployment flow that loads the firmware, runs inference, and reports results through the remoteproc trace buffer -## Software Overview - Visual Studio Code +## What you need before you continue -[Visual Studio Code](https://code.visualstudio.com/) is a free integrated development environment provided by Microsoft. It is platform independent, full featured, and accomodating of many engineering frameworks. You will use Visual Studio Code to both configure NXP's software and connect to NXP's hardware. +After you complete the Linux setup Learning Path, you should have: -## Software Overview - TinyML. +- A way to log in to the board (serial console and/or SSH) +- A way to transfer files (for example, `scp`) + +## NXP's MCUXpresso IDE + +NXP provides free software for working with their boards, the [MCUXpresso Integrated Development Environment (IDE)](https://www.nxp.com/design/design-center/software/development-software/mcuxpresso-software-and-tools-/mcuxpresso-integrated-development-environment-ide:MCUXpresso-IDE). In this Learning Path, you use [MCUXpresso for Visual Studio Code](https://www.nxp.com/design/design-center/software/development-software/mcuxpresso-software-and-tools-/mcuxpresso-for-visual-studio-code:MCUXPRESSO-VSC). + +MCUXpresso matters here because it gives you a predictable way to build and manage Cortex-M firmware on a platform where Linux is running at the same time. + +## TinyML This Learning Path uses TinyML. TinyML is machine learning tailored to function on devices with limited resources, constrained memory, low power, and fewer processing capabilities. -For a learning path focused on creating and deploying your own TinyML models, please see [Introduction to TinyML on Arm using PyTorch and ExecuTorch](/learning-paths/embedded-and-microcontrollers/introduction-to-tinyml-on-arm/) +For a Learning Path focused on creating and deploying your own TinyML models, see [Introduction to TinyML on Arm using PyTorch and ExecuTorch](/learning-paths/embedded-and-microcontrollers/introduction-to-tinyml-on-arm/) -## Benefits and applications +In this Learning Path, you focus on deployment and observation: building the two runtime artifacts (the `.pte` model and the `executor_runner` firmware), bringing them up on the board, and confirming the Ethos-U acceleration path is active. -NPUs, like Arm's [Ethos-U65](https://www.arm.com/products/silicon-ip-cpu/ethos/ethos-u65) NPU are available on physical devices specifically made for developers. Development boards like NXP's [FRDM i.MX 93](https://www.nxp.com/design/design-center/development-boards-and-designs/frdm-i-mx-93-development-board:FRDM-IMX93) also connect to displays via a HDMI cable. Additionally the board accepts video inputs. This is useful for for ML performance visualization due to: -- visual confirmation that your ML model is running on the physical device -- image and video inputs for computer vision models running on the device -- clearly indicated instruction counts -- confirmation of total execution time and -- visually appealing output for prototypes and demos +The next section covers booting the FRDM i.MX 93 and establishing a console connection so you can log in and transfer files. diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/10-deploy-executorchrunner-nxp-board.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/10-deploy-executorchrunner-nxp-board.md index ccc9e90bfa..da44cc3a28 100644 --- a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/10-deploy-executorchrunner-nxp-board.md +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/10-deploy-executorchrunner-nxp-board.md @@ -6,283 +6,222 @@ weight: 11 layout: learningpathall --- -## Connect to the FRDM-IMX93 board +## Deployment overview -The FRDM-IMX93 board runs Linux on the Cortex-A55 cores. You need network or serial access to deploy the firmware. +This section is where the heterogeneous system comes together. +Linux on the application cores manages the lifecycle of Cortex-M33 through RemoteProc, and your Cortex-M33 firmware brings up ExecuTorch and the Ethos-U65 delegate. -Find your board's IP address using the serial console or check your router's DHCP leases. +Your success criteria is simple and observable: the remoteproc trace buffer shows a completed inference run with no bus errors. -Connect via SSH: +## Prerequisites -{{< tabpane code=false >}} -{{< tab header="Windows/Linux" >}} -```bash -ssh root@192.168.1.24 -``` +Before deploying, verify the following on your FRDM-IMX93 board: -Alternative with PuTTY on Windows: -- Host: `192.168.1.24` -- Port: `22` -- Connection type: SSH -- Username: `root` -{{< /tab >}} -{{< tab header="macOS" >}} -```bash -ssh root@192.168.1.24 -``` -{{< /tab >}} -{{< /tabpane >}} +1. **Ethos-U kernel driver is loaded.** The Linux `ethosu` driver must be bound so the NPU is powered and clocked: -Replace `192.168.1.24` with your board's IP address. + ```bash { command_line="root@frdm-imx93" output_lines="2" } + ls /dev/ethosu* + /dev/ethosu0 + ``` -## Copy the firmware to the board + If `/dev/ethosu0` doesn't exist, the NPU isn't powered and the firmware will hang at NPU initialization. -Copy the built firmware file to the board's firmware directory: +2. **DDR memory is reserved for the CM33.** The NXP BSP reserves two DDR regions by default: + + ```dts + reserved-memory { + model@c0000000 { + reg = <0 0xc0000000 0 0x400000>; /* 4MB for .pte model */ + no-map; + }; + ethosu_region@A8000000 { + reg = <0 0xa8000000 0 0x8000000>; /* 128MB for NPU working memory */ + no-map; + }; + }; + ``` + +## Copy files to the board + +From the `Executorch_runner_cm33` project, copy both the firmware and the `.pte` model to the board: -{{< tabpane code=false >}} -{{< tab header="Windows/Linux" >}} -```bash -scp debug/executorch_runner_cm33.elf root@192.168.1.24:/lib/firmware/ -``` -{{< /tab >}} -{{< tab header="macOS" >}} ```bash -scp debug/executorch_runner_cm33.elf root@192.168.1.24:/lib/firmware/ +scp debug/executorch_runner_cm33.elf root@:/lib/firmware/ +scp mobilenetv2_u65.pte root@:/tmp/ ``` -{{< /tab >}} -{{< /tabpane >}} -Verify the file was copied: +## Connect to the board + +SSH into the board for the remaining steps: -```bash { command_line="root@frdm-imx93" output_lines="2" } -ls -lh /lib/firmware/executorch_runner_cm33.elf --rw-r--r-- 1 root root 601K Oct 24 10:30 /lib/firmware/executorch_runner_cm33.elf +```bash +ssh root@ ``` -## Load the firmware on Cortex-M33 +Replace `` with your board's actual IP address. -The Cortex-M33 firmware is managed by the RemoteProc framework running on Linux. +## Load the model to DDR -Stop any currently running firmware: +The `executor_runner` firmware reads the `.pte` model from DDR at address `0xC0000000`. Write the model into DDR using `/dev/mem`: -```bash { command_line="root@frdm-imx93" } -echo stop > /sys/class/remoteproc/remoteproc0/state +```bash { command_line="root@frdm-imx93" output_lines="8" } +python3 -c " +import mmap, os +pte = open('/tmp/mobilenetv2_u65.pte', 'rb').read() +fd = os.open('/dev/mem', os.O_RDWR | os.O_SYNC) +m = mmap.mmap(fd, len(pte), mmap.MAP_SHARED, mmap.PROT_WRITE, offset=0xC0000000) +m.write(pte) +m.close() +os.close(fd) +print(f'Wrote {len(pte)} bytes to 0xC0000000') +" +Wrote 3507872 bytes to 0xC0000000 ``` -Set the new firmware: +{{% notice Note %}} +You can also load the model via U-Boot if the `.pte` file is on the SD card's first partition. At the U-Boot prompt, run `fatload mmc 0:1 0xc0000000 mobilenetv2_u65.pte` followed by `boot`. The model remains in DDR across Linux boot because the region is marked `no-map`. +{{% /notice %}} -```bash { command_line="root@frdm-imx93" } -echo executorch_runner_cm33.elf > /sys/class/remoteproc/remoteproc0/firmware -``` +## Run inference -Start the Cortex-M33 with the new firmware: +Start the Cortex-M33 firmware through RemoteProc. RemoteProc is the control plane for this platform: it gives you a consistent way to stop, replace, and start the Cortex-M33 image without manually resetting the system. ```bash { command_line="root@frdm-imx93" } +echo stop > /sys/class/remoteproc/remoteproc0/state +echo executorch_runner_cm33.elf > /sys/class/remoteproc/remoteproc0/firmware echo start > /sys/class/remoteproc/remoteproc0/state +sleep 15 +cat /sys/kernel/debug/remoteproc/remoteproc0/trace0 ``` -Verify the firmware loaded successfully: +{{% notice Note %}} +If no firmware is running, the `stop` command prints an error. That is expected and can be ignored. +{{% /notice %}} + +## Expected output -```bash { command_line="root@frdm-imx93" output_lines="2-5" } -dmesg | grep remoteproc | tail -n 5 -[12345.678] remoteproc remoteproc0: powering up imx-rproc -[12345.679] remoteproc remoteproc0: Booting fw image executorch_runner_cm33.elf, size 614984 -[12345.680] remoteproc remoteproc0: header-less resource table -[12345.681] remoteproc remoteproc0: remote processor imx-rproc is now up +You should see output similar to: + +```output +NPU config match +NPU arch match +cmd_end_reached 0x1 +bus_status_error 0x0 +1 inferences finished +Output[0]: dtype=6, numel=1000, nbytes=4000 +Program complete, exiting. ``` -The message "remote processor imx-rproc is now up" confirms successful loading. +The key indicators of a successful inference run: -## Load a model to DDR memory +| Output | Meaning | +|--------|---------| +| `NPU config match` | The compiled model's NPU configuration matches the hardware | +| `NPU arch match` | The compiled model's architecture version matches the hardware | +| `cmd_end_reached 0x1` | The NPU executed all 116 operators in the command stream | +| `bus_status_error 0x0` | No AXI bus errors during NPU memory access | +| `numel=1000` | MobileNet V2 output: 1000 ImageNet classification scores (one per class) | -The executor_runner loads `.pte` model files from DDR memory at address 0x80100000. +The model runs with uninitialized input data, so the output scores don't correspond to a real image classification. To get meaningful predictions, feed a real 224x224 RGB image as input. -Copy your `.pte` model to the board: +{{% notice Note %}} +If the trace buffer shows `Program identifier '' != expected 'ET12'`, the `.pte` model wasn't loaded into DDR at `0xC0000000`. Reload the model using the steps above. +{{% /notice %}} -{{< tabpane code=false >}} -{{< tab header="Windows/Linux" >}} -```bash -scp model.pte root@192.168.1.24:/tmp/ -``` -{{< /tab >}} -{{< tab header="macOS" >}} -```bash -scp model.pte root@192.168.1.24:/tmp/ -``` -{{< /tab >}} -{{< /tabpane >}} +## Re-run inference -Write the model to DDR memory: +The `executor_runner` runs inference once when the firmware starts. To re-run, reload the firmware: ```bash { command_line="root@frdm-imx93" } -dd if=/tmp/model.pte of=/dev/mem bs=1M seek=2049 +echo stop > /sys/class/remoteproc/remoteproc0/state +sleep 2 +echo start > /sys/class/remoteproc/remoteproc0/state +sleep 15 +cat /sys/kernel/debug/remoteproc/remoteproc0/trace0 ``` -The seek value of 2049 corresponds to address 0x80100000 (2049 MB = 0x801 in hex). +The trace buffer resets at the start of each firmware load, so you'll always see fresh output. -Verify the model was written: - -```bash { command_line="root@frdm-imx93" output_lines="2-5" } -xxd -l 64 -s 0x80100000 /dev/mem -80100000: 504b 0304 1400 0000 0800 0000 2100 a3b4 PK..........!... -80100010: 7d92 5801 0000 6c04 0000 1400 0000 7661 }.X...l.......va -80100020: 6c75 652f 7061 7261 6d73 2e70 6b6c 6500 lue/params.pkl. -80100030: ed52 cd4b 0241 1cfd 66de 49b6 9369 1ad9 .R.K.A..f.I..i.. -``` +## What you've accomplished and what's next -Non-zero bytes confirm the model is present in memory. +In this section: -## Monitor Cortex-M33 output +- You used Linux RemoteProc to load and boot a custom Cortex-M33 firmware image +- You validated an end-to-end ExecuTorch inference run that delegates computation to the Ethos-U65 NPU -The executor_runner outputs debug information via UART. Connect a USB-to-serial adapter to the M33 UART pins on the FRDM board. +Next, you can iterate on `.pte` models (and measure how operator coverage and model shape affect runtime behavior) while keeping the firmware bring-up path stable. -Open a serial terminal (115200 baud, 8N1): +## Update the firmware -{{< tabpane code=false >}} -{{< tab header="Windows/Linux" >}} -```bash -screen /dev/ttyUSB0 115200 -``` +To deploy a new version of the firmware: -Alternative with minicom: -```bash -minicom -D /dev/ttyUSB0 -b 115200 -``` -{{< /tab >}} -{{< tab header="macOS" >}} -```bash -screen /dev/tty.usbserial-* 115200 -``` +1. Build the updated firmware on your development machine +2. Copy to the board: -Alternative with minicom: ```bash -minicom -D /dev/tty.usbserial-* -b 115200 -``` -{{< /tab >}} -{{< /tabpane >}} - -You should see output from the ExecuTorch runtime: - -```output -ExecuTorch Runtime Starting... -Loading model from 0x80100000 -Model loaded successfully -Initializing Ethos-U NPU delegate -NPU initialized -Running inference... -Inference complete: 45.2ms +scp debug/executorch_runner_cm33.elf root@:/lib/firmware/ ``` -{{% notice Tip %}} -If you don't see UART output, verify the serial connection settings (115200 baud, 8N1) and check that the UART pins are correctly connected. -{{% /notice %}} - -## Test inference - -The executor_runner automatically runs inference when it starts. Check the UART output for inference results and timing. - -To restart inference, you can reload the firmware: +3. Re-run inference on the board: ```bash { command_line="root@frdm-imx93" } echo stop > /sys/class/remoteproc/remoteproc0/state +sleep 2 echo start > /sys/class/remoteproc/remoteproc0/state +sleep 15 +cat /sys/kernel/debug/remoteproc/remoteproc0/trace0 ``` -Monitor the UART console to see the new inference run. - -## Verify deployment success - -Confirm your deployment is working correctly: - -1. **RemoteProc status shows "running":** - -```bash { command_line="root@frdm-imx93" output_lines="2" } -cat /sys/class/remoteproc/remoteproc0/state -running -``` - -2. **Firmware is loaded:** - -```bash { command_line="root@frdm-imx93" output_lines="2" } -cat /sys/class/remoteproc/remoteproc0/firmware -executorch_runner_cm33.elf -``` - -3. **Model is in DDR memory** (non-zero bytes at 0x80100000) - -4. **UART shows inference output** with timing information - -## Troubleshooting - +{{% notice Troubleshooting %}} **RemoteProc fails to load firmware:** -Check file permissions: +Check that the file exists and has correct permissions: ```bash { command_line="root@frdm-imx93" } +ls -la /lib/firmware/executorch_runner_cm33.elf chmod 644 /lib/firmware/executorch_runner_cm33.elf ``` -Verify the file exists: +**`Program identifier '' != expected 'ET12'`:** -```bash { command_line="root@frdm-imx93" } -ls -la /lib/firmware/executorch_runner_cm33.elf -``` +The `.pte` model is not present at DDR address `0xC0000000`. Reload the model using the `/dev/mem` method or via U-Boot. -**Model not found error:** +**Firmware hangs (no trace output):** -Verify the model was written to memory: +Verify the Ethos-U kernel driver is loaded: ```bash { command_line="root@frdm-imx93" } -xxd -l 256 -s 0x80100000 /dev/mem | head +ls /dev/ethosu* +dmesg | grep ethosu ``` -If all zeros, re-run the `dd` command to write the model. - -**No UART output:** +If `/dev/ethosu0` does not exist, the NPU is not powered and the firmware cannot initialize it. -Check the serial connection: -- Baud rate: 115200 -- Data bits: 8 -- Parity: None -- Stop bits: 1 +**Memory allocation failed for planned buffer:** -Try a different USB port or serial terminal program. +This occurs when a large model's activation tensors exceed the DTCM method allocator. The firmware automatically uses DDR for models that need more than 12KB of planned buffers. If you see this error, verify the `ethosu_region@A8000000` (128MB) is reserved in the device tree. -**Firmware crashes or hangs:** +**BUS FAULT or vtable corruption:** -Check kernel logs for errors: +The SDK linker script patch has not been applied. Run the patch script and rebuild: -```bash { command_line="root@frdm-imx93" } -dmesg | grep -i error | tail +```bash +./patches/apply_patches.sh +cmake --preset debug +cmake --build debug ``` -This might indicate memory configuration issues. Reduce the memory pool sizes in `CMakeLists.txt` and rebuild. +**Firmware crashes after NPU init:** -## Update the firmware +Check kernel logs: -To deploy a new version of the firmware: - -1. Build the updated firmware on your development machine -2. Copy to the board: - -{{< tabpane code=false >}} -{{< tab header="Windows/Linux" >}} -```bash -scp debug/executorch_runner_cm33.elf root@:/lib/firmware/ -``` -{{< /tab >}} -{{< tab header="macOS" >}} -```bash -scp debug/executorch_runner_cm33.elf root@:/lib/firmware/ +```bash { command_line="root@frdm-imx93" } +dmesg | grep -i error | tail ``` -{{< /tab >}} -{{< /tabpane >}} -3. Restart RemoteProc: +This might indicate memory configuration issues. Verify that both DDR regions (`0xC0000000`--`0xC03FFFFF` and `0xA8000000`--`0xAFFFFFFF`) are reserved in the device tree. +{{% /notice %}} -```bash { command_line="root@frdm-imx93" } -echo stop > /sys/class/remoteproc/remoteproc0/state -echo start > /sys/class/remoteproc/remoteproc0/state -``` +## Summary -4. Monitor UART output to verify the new firmware is running +You've completed the full bring-up flow for ExecuTorch on the NXP FRDM i.MX 93. Along the way, you set up a reproducible build environment, compiled two `.pte` model artifacts targeting the Ethos-U65, built and patched the Cortex-M33 `executor_runner` firmware, and deployed it through Linux RemoteProc to confirm a successful end-to-end inference run. The remoteproc trace buffer confirmed zero bus errors and full NPU operator coverage, establishing a stable foundation for iterating on models and firmware independently. \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/2-boot-nxp.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/2-boot-nxp.md index 831aa0ec48..dd5ba0d52f 100644 --- a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/2-boot-nxp.md +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/2-boot-nxp.md @@ -1,81 +1,77 @@ --- # User change -title: "Boot the NXP FRDM i.MX 93 Board" +title: "Boot the NXP FRDM i.MX 93 board" weight: 3 # Do not modify these elements layout: "learningpathall" --- +## Connect to the board -In this section, you will prepare the NXP [FRDM i.MX 93](https://www.nxp.com/design/design-center/development-boards-and-designs/frdm-i-mx-93-development-board:FRDM-IMX93) board for ML development. +This section walks through powering on the board and establishing a serial console connection. If your board is already running Linux and you can log in, skip ahead to the next section. -## Unbox the NXP Board +You need a serial terminal to see the boot console and log in. -Follow NXP's getting started instructions: [Getting Started with FRDM-IMX93](https://www.nxp.com/document/guide/getting-started-with-frdm-imx93:GS-FRDM-IMX93): -* Stop when you complete section "1.6 Connect Power Supply" +{{% notice macOS %}} +On macOS as your host, you'll need the following set up before getting started: -## Connect to the NXP Board +- Install the [Silicon Labs USB-to-UART driver](https://www.silabs.com/developer-tools/usb-to-uart-bridge-vcp-drivers?tab=downloads) +- Install [picocom](https://github.com/npat-efault/picocom) + ```bash + brew install picocom + ``` +{{% /notice %}} -Prior to logging in to the NXP board, you need to configure `picocom`. This allows you to connect to the board using a USB cable. +Connect the board's **DEBUG** USB-C connector to your host machine. -{{% notice macOS %}} +Find the board's serial device: -1. Install the Silicon Labs driver: + ```bash { output_lines = "2-5" } + ls /dev/tty.* + ... + /dev/tty.usbmodem + /dev/tty.usbmodem + ... + ``` - https://www.silabs.com/developer-tools/usb-to-uart-bridge-vcp-drivers?tab=downloads - -2. Install [picocom](https://github.com/npat-efault/picocom): - ```bash - brew install picocom + The exact device names vary per board. Look for entries containing `usbmodem`. + +Open a serial connection using the first `usbmodem` device: + + ```bash { output_lines = "2-4" } + sudo picocom -b 115200 /dev/tty.usbmodem + picocom v3.1 + ... + Terminal ready ``` + +Connect the board's **POWER** USB-C connector to your host machine. You should see a red and a white LED on the board. + +Wait for the boot log to scroll past in the picocom window. When it finishes, you'll see a login prompt: + + ```output + NXP i.MX Release Distro 6.6-scarthgap imx93frdm ttyLP0 + + imx93frdm login: + ``` + +{{% notice Tip %}} +If you miss the login prompt, hold the board's power button for two seconds to power off, then press it again to reboot. {{% /notice %}} -1. Establish a USB-to-UART (serial) connection: - - Connect the board's "DEBUG" USB-C connector to your laptop - - Find the NXP board's USB connections in your computer's terminal: - ```bash { output_lines = "2-7" } - ls /dev/tty.* - # output lines - ... - /dev/tty.debug-console - /dev/tty.usbmodem56D70442811 - /dev/tty.usbmodem56D70442813 - ... - ``` - - - Connect to the NXP board: - ```bash { output_lines = "2-5" } - sudo picocom -b 115200 /dev/tty.usbmodem56D70442811 - # output lines - picocom v3.1 - ... - Terminal ready - ``` -2. Log in to the NXP board: - - Connect the board's "POWER" USB-C connector to your laptop - - At this point you should see one red and one white light on the board - - Next you should see scrolling text in your `picocom` window, as the NXP board boots - - The last line should say `login:` - ```bash { output_lines = "1-9" } - # output lines - ... - [ OK ] Reached target Graphical Interface. - Starting Record Runlevel Change in UTMP... - [ OK ] Finished Record Runlevel Change in UTMP. - - NXP i.MX Release Distro 6.6-scarthgap imx93frdm ttyLP0 - - imx93frdm login: - ``` - -3. [Optional] Troubleshooting: - - Restart the NXP board, to get to the `login:` prompt: - - Hold the NXP board's power button for 2-seconds, until the lights turn off - - Hold the NXP board's power button again for 2-seconds, until the lights turn on - -## [Optional] Run the Built-In NXP Demos -* Connect the NXP board to a monitor via HDMI -* Connect a mouse to the NXP board's USB-A port +## Run the built-in NXP demos (optional) + +Connect the board to a monitor via HDMI and plug a mouse into the board's USB-A port. NXP includes several ML demos that run out of the box. ![NXP board built-in ML demos alt-text#center](./nxp-board-built-in-ml-demos.png "NXP board built-in ML demos") + +## What you've learned and what's next + +In this section you've: + +- Connected to the board via serial console +- Booted the NXP FRDM i.MX 93 board and confirmed Linux is running +- Verified you can access the login prompt + +With the board running and Linux accessible, the next step is setting up the build environment for ExecuTorch. diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/4-environment-setup.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/4-environment-setup.md index 47aa4bc6d2..2ff692e783 100644 --- a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/4-environment-setup.md +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/4-environment-setup.md @@ -1,6 +1,6 @@ --- # User change -title: "Enviroment Setup" +title: "Set up the ExecuTorch build environment" weight: 5 # 1 is first, 2 is second, etc. @@ -8,30 +8,33 @@ weight: 5 # 1 is first, 2 is second, etc. layout: "learningpathall" --- -For detailed instructions on setting up your ExecuTorch build environment, please see the official PyTorch documentation: [Environment Setup](https://docs.pytorch.org/executorch/stable/using-executorch-building-from-source.html#environment-setup) +## For macOS: build ExecuTorch in a Docker container -{{% notice macOS %}} +On macOS, it’s easiest to build ExecuTorch in an Ubuntu container. This keeps your toolchain consistent with the rest of the Learning Path and avoids gaps in macOS-native cross-compilers (for example, the Arm GNU Toolchain doesn’t provide an “AArch64 GNU/Linux target” for macOS). -Use a Docker container to build ExecuTorch: -* The [Arm GNU Toolchain](https://developer.arm.com/Tools%20and%20Software/GNU%20Toolchain) currently does not have a "AArch64 GNU/Linux target" for macOS -* You will use this toolchain's `gcc-aarch64-linux-gnu` and `g++-aarch64-linux-gnu` compilers on the next page of this learning path +This container isn’t part of the runtime deployment. It’s a build environment that produces the artifacts you later move onto the FRDM i.MX 93: -1. Install and start [Docker Desktop](https://www.docker.com/) +- Prebuilt ExecuTorch libraries you link into Cortex-M33 firmware +- `.pte` model files compiled for Ethos-U65 -2. Create a directory for building a `ubuntu-24-container`: +Keeping this step reproducible helps you focus on the actual bring-up milestone: booting custom firmware on Cortex-M33 and using Ethos-U65 for inference. + +Start by installing and launching [Docker Desktop](https://www.docker.com/). + +Next, create a working directory for your container build: ```bash mkdir ubuntu-24-container ``` -3. Create a `dockerfile` in the `ubuntu-24-container` directory: +Now create a `Dockerfile` and switch into the directory: ```bash cd ubuntu-24-container touch Dockerfile ``` -4. Add the following commands to your `Dockerfile`: +Add the following content to your `Dockerfile` to install a few basic tools in the image: ```dockerfile FROM ubuntu:24.04 @@ -44,51 +47,54 @@ Use a Docker container to build ExecuTorch: curl vim git ``` - The `ubuntu:24.04` container image includes Python 3.12, which will be used for this learning path. + The `ubuntu:24.04` container image includes Python 3.12, which you use later in this Learning Path. -5. Create the `ubuntu-24-container`: + Build the container image: ```bash docker build -t ubuntu-24-container . ``` -6. Run the `ubuntu-24-container`: +Run the container and open an interactive shell: ```bash { output_lines = "2-3" } docker run -it ubuntu-24-container /bin/bash # Output will be the Docker container prompt - ubuntu@:/# + root@:/# ``` - [OPTIONAL] If you already have an existing container: - - Get the existing CONTAINER ID: - ```bash { output_lines = "2-4" } - docker ps -a - # Output - CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES - 0123456789ab ubuntu-24-container "/bin/bash" 27 hours ago Exited (255) 59 minutes ago. container_name - ``` - - Log in to the existing container: - ```bash - docker start 0123456789ab - docker exec -it 0123456789ab /bin/bash - ``` +If you already created a container before, reuse it instead of creating a new one. -{{% /notice %}} +First, list your containers to find the container ID: + +```bash { output_lines = "2-4" } +docker ps -a +# Output +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +0123456789ab ubuntu-24-container "/bin/bash" 27 hours ago Exited (255) 59 minutes ago. container_name +``` -After logging in to the Docker container, navigate to the ubuntu home directory: +Then start the container and attach a shell: ```bash -cd /home/ubuntu +docker start 0123456789ab +docker exec -it 0123456789ab /bin/bash ``` -1. **Install dependencies:** +Once you’re inside the container, move to your home directory: + +```bash +cd /root +``` - ```bash { output_lines = "1" } - # Use "sudo apt ..." if you are not logged in as root +## Install dependencies + +Install the packages ExecuTorch needs to build. If you’re not running as root, prefix the commands with `sudo`. + + ```bash apt update apt install -y \ - python-is-python3 python3.12-dev python3.12-venv \ + python-is-python3 python3.12-dev python3.12-venv python3-pip \ gcc g++ \ make cmake \ build-essential \ @@ -96,25 +102,43 @@ cd /home/ubuntu libboost-all-dev ``` -2. Clone ExecuTorch: +## Create a Python virtual environment + +Create and activate a virtual environment so your Python packages stay scoped to this project: + ```bash { output_lines = "3" } + python3 -m venv .venv + source .venv/bin/activate + ``` + +## Get the ExecuTorch source code + +Clone ExecuTorch and initialize its submodules: + ```bash git clone https://github.com/pytorch/executorch.git cd executorch git fetch --tags - git checkout v1.0.0 + git checkout c70a742344e30158dc370d7d35d60ed07660fee0 git submodule sync git submodule update --init --recursive ``` -3. Create a Virtual Environment: - ```bash { output_lines = "3" } - python3 -m venv .venv - source .venv/bin/activate - # Your prompt will prefix with (.venv) - ``` +{{% notice EthosUCompileSpec parameters %}} +The `EthosUCompileSpec` parameters used in this guide: -4. Configure your git username and email globally: - ```bash - git config --global user.email "you@example.com" - git config --global user.name "Your Name" - ``` +| Parameter | Value | Description | +| ----------------- | --------------------- | ---------------------------------------------- | +| `target` | `ethos-u65-256` | Targets the Ethos-U65 with 256 MAC units | +| `system_config` | `Ethos_U65_High_End` | High-end system configuration for optimal performance | +| `memory_mode` | `Shared_Sram` | Uses shared SRAM memory mode | +{{% /notice %}} + +## What you've learned and what's next + +In this section you've: + +- Set up an Ubuntu 24.04 Docker container for building ExecuTorch (macOS users) +- Installed required dependencies and created a Python virtual environment +- Cloned the ExecuTorch repository and checked out the correct version + +With your build environment configured and the ExecuTorch source checked out, the next step is building and installing the ExecuTorch package. diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/6-build-executorch.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/6-build-executorch.md index 48ef3db142..dd20c9a86e 100644 --- a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/6-build-executorch.md +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/6-build-executorch.md @@ -1,6 +1,6 @@ --- # User change -title: "Build ExecuTorch" +title: "Build and install ExecuTorch" weight: 7 # 1 is first, 2 is second, etc. @@ -8,152 +8,78 @@ weight: 7 # 1 is first, 2 is second, etc. layout: "learningpathall" --- -For a full tutorial on building ExecuTorch please see learning path [Introduction to TinyML on Arm using PyTorch and ExecuTorch](/learning-paths/embedded-and-microcontrollers/introduction-to-tinyml-on-arm/). +## Overview -## Install ExecuTorch +With the ExecuTorch source checked out and your virtual environment active, you can now build ExecuTorch and set up the Arm toolchain for Ethos-U cross-compilation. + +For a full tutorial on building ExecuTorch, see the Learning Path [Introduction to TinyML on Arm using PyTorch and ExecuTorch](/learning-paths/embedded-and-microcontrollers/introduction-to-tinyml-on-arm/). -1. Upgrade pip and install build tools: +## Install ExecuTorch - ```bash - pip install --upgrade pip setuptools wheel - ``` +Upgrade pip and install build tools: -2. Build and install the `executorch` pip package: +```bash +pip install --upgrade pip setuptools wheel +``` - ```bash - ./install_executorch.sh - ``` +Build and install the `executorch` pip package: -## Build troubleshooting +```bash +./install_executorch.sh +``` -If the `install_executorch.sh` script fails, manually install the dependencies using the following commands: +After the installation finishes, verify the package is available: ```bash -pip install torch torchvision -pip install --no-build-isolation . -pip install --no-build-isolation third-party/ao +pip list | grep executorch ``` ## Set up the Arm toolchain Initialize the Arm-specific environment and accept the EULA: -1. Run the setup script: - - ```bash - ./examples/arm/setup.sh --i-agree-to-the-contained-eula - ``` - -2. Source the environment variables: - - ```bash - source ./examples/arm/arm-scratch/setup_path.sh - ``` - -## Apply the Ethos-U65 patch - -As of this writing, ExecuTorch does not officially support the Ethos-U65. You must patch the `compile_spec.py` file to enable U65 compilation targets within the build system. - -1. Create and apply the patch by running the following command block: - - ```bash - cat > /tmp/patch_u65.py << 'PATCH' - import os - - # Locate the file within the virtual environment - filepath = "/root/executorch/.venv/lib/python3.12/site-packages/executorch/backends/arm/ethosu/compile_spec.py" - - with open(filepath, 'r') as f: - content = f.read() - - # 1. Inject U65 Configuration Support - old_code = ' elif "ethos-u85" in target_lower:' - new_code = ''' elif "ethos-u65" in target_lower: - self.tosa_spec = TosaSpecification.create_from_string("TOSA-1.0+INT") - default_system_config = "Ethos_U65_High_End" - default_memory_mode = "Shared_Sram" - elif "ethos-u85" in target_lower:''' - - content = content.replace(old_code, new_code) - - # 2. Inject U65 Compile Spec Builder Logic - old_check = ''' if "u55" in target_lower: - return CompileSpecBuilder( - TosaSpecification.create_from_string("TOSA-0.80+BI+u55") - ) - if "u85" in self.target:''' - - new_check = ''' if "u55" in target_lower: - return CompileSpecBuilder( - TosaSpecification.create_from_string("TOSA-0.80+BI+u55") - ) - if "u65" in target_lower: - return CompileSpecBuilder( - TosaSpecification.create_from_string("TOSA-1.0+INT") - ) - if "u85" in self.target:''' - - content = content.replace(old_check, new_check) - - with open(filepath, 'w') as f: - f.write(content) - - print(f"Patched {filepath}! U65 support added.") - PATCH - - python3 /tmp/patch_u65.py - ``` - -2. Verify the patch by running: - - ```bash - python3 -c "from executorch.backends.arm.ethosu import EthosUCompileSpec; EthosUCompileSpec(target='ethos-u65-256'); print('U65 OK')" - ``` - - If successful, you see the output `U65 OK`. - -## Additional troubleshooting - -1. Allocate at least 4 GB of swap space: +```bash +./examples/arm/setup.sh --i-agree-to-the-contained-eula +``` - ```bash - fallocate -l 4G /swapfile - chmod 600 /swapfile - mkswap /swapfile - swapon /swapfile - ``` +Source the environment variables: - Deallocate the swap space after you complete this learning path (optional): +```bash +source ./examples/arm/ethos-u-scratch/setup_path.sh +``` - ```bash - swapoff /swapfile - rm /swapfile - ``` +{{% notice Troubleshooting %}} +If `install_executorch.sh` fails, install the dependencies manually: - {{% notice macOS %}} +```bash +pip install torch torchvision +pip install --no-build-isolation . +pip install --no-build-isolation third-party/ao +``` - Increase the "Swap" space in Docker settings to 4 GB: - ![Increase the swap space in Docker settings to 4 GB alt-text#center](./increase-swap-space-to-4-gb.jpg "Increase the swap space in Docker settings to 4 GB") +If `buck2` hangs during the build: - {{% /notice %}} +```bash +ps aux | grep buck +pkill -f buck +``` -2. Kill the `buck2` process if it hangs: +To clean the build environment and start fresh: - ```bash - ps aux | grep buck - pkill -f buck - ``` +```bash +./install_executorch.sh --clean +git submodule sync +git submodule update --init --recursive +./install_executorch.sh +``` +{{% /notice %}} -3. Clean the build environment and reinitialize all submodules: +## What you've learned and what's next - ```bash - ./install_executorch.sh --clean - git submodule sync - git submodule update --init --recursive - ``` +In this section you've: -4. Try `install_executorch.sh` again: +- Installed the ExecuTorch package and verified its availability +- Set up the Arm toolchain with Ethos-U support +- Configured the environment for cross-compiling to Ethos-U65 - ```bash - ./install_executorch.sh - ``` \ No newline at end of file +With ExecuTorch installed and the Arm toolchain configured, you can now compile `.pte` model files targeting the Ethos-U65 NPU. diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/7-build-executorch-pte.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/7-build-executorch-pte.md index df634fed95..0802662121 100644 --- a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/7-build-executorch-pte.md +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/7-build-executorch-pte.md @@ -1,6 +1,6 @@ --- # User change -title: "Build the ExecuTorch .pte" +title: "Build ExecuTorch models for Ethos-U65" weight: 8 # 1 is first, 2 is second, etc. @@ -8,9 +8,10 @@ weight: 8 # 1 is first, 2 is second, etc. layout: "learningpathall" --- -Embedded systems like the NXP board require two ExecuTorch runtime components: a `.pte` file and an `executor_runner` file. +## ExecuTorch deployment components + +On the FRDM i.MX93, you deploy ExecuTorch as two cooperating artifacts: -**ExecuTorch Runtime Files for Embedded Systems** |Component|Role in Deployment|What It Contains|Why It's Required| |---------|------------------|----------------|-----------------| |**`.pte file`** (e.g., `mobilenetv2_u65.pte`)|The model itself, exported from ExecuTorch|Serialized and quantized operator graph + weights + metadata|Provides the neural network to be executed| @@ -81,7 +82,13 @@ Embedded systems like the NXP board require two ExecuTorch runtime components: a ## Build the ExecuTorch .pte for Ethos-U65 -This section shows you how to build `.pte` files for the Ethos-U65 NPU. You will compile two models: a simple addition model to verify the setup, and MobileNet V2 for real-world inference. +This section shows you how to build `.pte` files for the Ethos-U65 NPU. +You compile two models: + +- A simple add model to validate the toolchain and U65 compile spec +- MobileNet V2 to demonstrate a realistic workload and confirm NPU operator coverage + +The `.pte` file is the handoff point between “host build time” and “device run time”. In the later deployment steps, Linux is responsible for loading firmware, and the Cortex-M33 firmware is responsible for loading and executing the `.pte` using the Ethos-U delegate. ### Compile a simple add model @@ -155,6 +162,10 @@ This script creates a basic addition model, quantizes it, and compiles it for th This script compiles the [MobileNet V2](https://pytorch.org/hub/pytorch_vision_mobilenet_v2/) computer vision model for the Ethos-U65. MobileNet V2 is a convolutional neural network (CNN) used for image classification and object detection. +{{% notice Note %}} +MobileNet V2 uses `export_for_training()` instead of `export()` because it contains batch normalization layers that must be traced in training mode before quantization can fuse them. The simple add model above has no such layers, so `export()` is sufficient. +{{% /notice %}} + 1. Create the MobileNet V2 compilation script: ```bash @@ -204,7 +215,7 @@ This script compiles the [MobileNet V2](https://pytorch.org/hub/pytorch_vision_m python3 compile_mv2_u65.py ``` - If successful, you see the Vela compiler summary indicating 100% NPU utilization: + If successful, you see the Vela compiler summary indicating 100% NPU utilization. This is an important bring-up signal because it shows the graph was lowered onto the Ethos-U path rather than falling back to CPU execution. ```output Network summary for out @@ -225,12 +236,45 @@ This script compiles the [MobileNet V2](https://pytorch.org/hub/pytorch_vision_m ls -la mobilenetv2_u65.pte ``` -{{% notice Note %}} -The `EthosUCompileSpec` parameters used in this guide: +### Copy models to the SD card + +The `executor_runner` firmware loads `.pte` models from a fixed DDR address (`0xC0000000`). The model must be written to DDR before Linux boots, using U-Boot. + +1. Copy the `.pte` files out of the Docker container: -| Parameter | Value | Description | -| ----------------- | --------------------- | ---------------------------------------------- | -| `target` | `ethos-u65-256` | Targets the Ethos-U65 with 256 MAC units | -| `system_config` | `Ethos_U65_High_End` | High-end system configuration for optimal performance | -| `memory_mode` | `Shared_Sram` | Uses shared SRAM memory mode | + ```bash + docker cp :/root/executorch/model_u65.pte . + docker cp :/root/executorch/mobilenetv2_u65.pte . + ``` + + Replace `` with your Docker container ID. + +2. Write the `.pte` files to the first partition of the SD card so U-Boot can load them: + +{{< tabpane code=false >}} +{{< tab header="Linux" >}} +sudo mount /dev/sdX1 /mnt +sudo cp model_u65.pte mobilenetv2_u65.pte /mnt/ +sudo umount /mnt +{{< /tab >}} +{{< tab header="macOS" >}} +cp model_u65.pte mobilenetv2_u65.pte /Volumes// +{{< /tab >}} +{{< /tabpane >}} + +{{% notice Parameter placeholders %}} +For the commands above: +- Replace `/dev/sdX1` with your SD card's first partition. +- Replace `` with the mounted volume name. {{% /notice %}} + +## What you've learned and what's next + +In this section you've: + +- Compiled a simple add model to verify the Ethos-U65 toolchain setup +- Built a MobileNet V2 `.pte` file with 100% NPU operator coverage +- Staged the models on your SD card for deployment + +With both `.pte` files ready, you can now build the Cortex-M33 `executor_runner` firmware that will load and execute them. + diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/9-build-executorch-runner-for-cm33.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/9-build-executorch-runner-for-cm33.md index de7f900cae..961e90eecc 100644 --- a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/9-build-executorch-runner-for-cm33.md +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/9-build-executorch-runner-for-cm33.md @@ -1,63 +1,44 @@ --- -title: Build the executor_runner firmware +title: Build Cortex-M33 firmware for ExecuTorch weight: 10 - -### FIXED, DO NOT MODIFY +# FIXED, DO NOT MODIFY layout: learningpathall --- +## ExecuTorch deployment components + +In this section, you build the Cortex-M33 `executor_runner` firmware. You then deploy it to the FRDM i.MX 93 board. + +This is a key milestone. You're proving you can control the real-time ML runtime for Ethos-U65 workloads. + +On i.MX 93, Linux runs on the application cores, but the real-time ML runtime that talks to Ethos-U65 runs as **firmware on Cortex-M33**. +When you can build and boot your own `executor_runner`, you've proven that the microcontroller side of the system is under your control and ready to host ML workloads. + +The project includes prebuilt ExecuTorch static libraries in the repository, so this section focuses on pulling the runner project, applying the required SDK patches, wiring it up to your toolchain, and building it. + +In architectural terms: + +- The application core (Linux) loads the firmware image and manages its lifecycle +- The Cortex-M33 firmware owns model execution and the delegate path into Ethos-U65 +- The `.pte` model remains a separate artifact that you update independently + ## Set up MCUXpresso for VS Code -Install the MCUXpresso extension in VS Code: +Install the [MCUXpresso extension for VS Code](https://marketplace.visualstudio.com/items?itemName=NXPSemiconductors.mcuxpresso). -{{< tabpane code=false >}} -{{< tab header="Windows/Linux" >}} -1. Open VS Code and press `Ctrl+Shift+X` to open Extensions -2. Search for "MCUXpresso for VS Code" -3. Click **Install** on the NXP extension -{{< /tab >}} -{{< tab header="macOS" >}} -1. Open VS Code and press `Cmd+Shift+X` to open Extensions -2. Search for "MCUXpresso for VS Code" -3. Click **Install** on the NXP extension -{{< /tab >}} -{{< /tabpane >}} +In VS Code, open **Extensions**, search for "MCUXpresso", and install the extension published by NXP. -Configure the ARM toolchain path: +## Install MCUXpresso SDK and Arm toolchain -{{< tabpane code=false >}} -{{< tab header="Windows/Linux" >}} -1. Open Settings with `Ctrl+,` -2. Search for **MCUXpresso: Toolchain** -3. Set the toolchain path to: `/opt/arm-gnu-toolchain-14.2.rel1-x86_64-arm-none-eabi/bin` -{{< /tab >}} -{{< tab header="macOS" >}} -1. Open Settings with `Cmd+,` -2. Search for **MCUXpresso: Toolchain** -3. Set the toolchain path to: `/opt/arm-gnu-toolchain-14.2.rel1-x86_64-arm-none-eabi/bin` -{{< /tab >}} -{{< /tabpane >}} +Use the MCUXpresso Installer to install the SDK and toolchain components. -Install the MCUXpresso SDK for FRDM-MIMX93: +In VS Code, open the Command Palette and run **MCUXpresso for VS Code: Open MCUXpresso Installer**. Select the following items, then select **Install**: -{{< tabpane code=false >}} -{{< tab header="Windows/Linux" >}} -1. Open Command Palette: `Ctrl+Shift+P` -2. Type: **MCUXpresso: Install MCUXpresso SDK** -3. Search for "FRDM-MIMX93" or select **MCIMX93-EVK** -4. Select the latest SDK and click **Install** -{{< /tab >}} -{{< tab header="macOS" >}} -1. Open Command Palette: `Cmd+Shift+P` -2. Type: **MCUXpresso: Install MCUXpresso SDK** -3. Search for "FRDM-MIMX93" or select **MCIMX93-EVK** -4. Select the latest SDK and click **Install** -{{< /tab >}} -{{< /tabpane >}} +- **MCUXpresso SDK Developer** (under Software Kits) +- **Arm GNU Toolchain (Latest)** (under Arm components) +- **Standalone Toolchain Add-ons (Latest)** (under Arm components) -{{% notice Note %}} -If the FRDM-MIMX93 development board is not listed in the current MCUXpresso SDK catalog, you can alternatively select **MCIMX93-EVK** as they share the same i.MX93 SoC with Cortex-M33 core architecture. The SDK compatibility ensures seamless development across both platforms. -{{% /notice %}} +![MCUXpresso Installer showing SDK and toolchain selection options#center](mcuxpresso-installer.png "MCUXpresso Installer") ## Clone the executor_runner repository @@ -68,130 +49,159 @@ git clone https://github.com/fidel-makatia/Executorch_runner_cm33.git cd Executorch_runner_cm33 ``` -The repository contains the complete runtime source code and build configuration for Cortex-M33. - -## Copy ExecuTorch libraries +The repository contains the complete runtime source code, pre-built ExecuTorch libraries, and build configuration for Cortex-M33 with Ethos-U65 NPU support. -The executor_runner requires prebuilt ExecuTorch libraries with Ethos-U NPU support from your Docker container. +## Configure the project for FRDM-MIMX93 -Find your ExecuTorch build container: +Open the project in VS Code: -```bash { output_lines = "2-3" } -docker ps -a -CONTAINER ID IMAGE COMMAND CREATED STATUS -abc123def456 executorch "/bin/bash" 2 hours ago Exited +```bash +code . ``` -Copy the libraries: +If the MCUXpresso extension doesn't automatically pick up the project, import it: -```bash -docker cp abc123def456:/home/ubuntu/executorch/cmake-out/lib/. ./executorch/lib/ -docker cp abc123def456:/home/ubuntu/executorch/. ./executorch/include/executorch/ -``` +1. Open the Command Palette +2. Run **MCUXpresso for VS Code: Import Project** +3. Select the `Executorch_runner_cm33` folder +4. When prompted, choose **Arm GNU Toolchain** -Replace `abc123def456` with your actual container ID. +![Import Project dialog showing project path, repository, and toolchain selection#center](import-project.png "Import Cloned Project folder") -{{% notice Note %}} -In some Docker containers, the `cmake-out` folder might not exist. If you don't see the libraries, run the following command to build them: +## Set environment variables -```bash -./examples/arm/run.sh --build-only -``` +Set three environment variables so the build can find your toolchain, your SDK, and the MCUXpresso Python environment. These must be set before building or running the SDK patch script. -The libraries will be generated in `arm_test/cmake-out`. -{{% /notice %}} +Do this once for your user account, then restart VS Code so the changes take effect. -Verify the libraries: +### Required variables -```bash { output_lines = "2-5" } -ls -lh executorch/lib/ --rw-r--r-- 1 user user 2.1M libexecutorch.a --rw-r--r-- 1 user user 856K libexecutorch_core.a --rw-r--r-- 1 user user 1.3M libexecutorch_delegate_ethos_u.a -``` +| Variable | Description | +|----------|-------------| +| `ARMGCC_DIR` | Path to the Arm GCC toolchain root, i.e. a directory starting with `arm-gnu-toolchain-14.2.rel1*` | +| `SdkRootDirPath` | Path to the folder that contains the `mcuxsdk/` subdirectory | +| `MCUX_VENV_PATH` | Path to the MCUXpresso Python venv executables | -## Configure the project for FRDM-MIMX93 +Find the installed toolchain directory name: -Open the project in VS Code: +{{< tabpane code=false >}} +{{< tab header="Linux/macOS" >}} +ls ~/.mcuxpressotools/arm-gnu-toolchain-* +{{< /tab >}} +{{< tab header="Windows (PowerShell)" >}} +dir $env:USERPROFILE\.mcuxpressotools\arm-gnu-toolchain-* +{{< /tab >}} +{{< /tabpane >}} -```bash -code . -``` +Use the directory name from the output for the `ARMGCC_DIR` variable below. The name looks like `arm-gnu-toolchain-14.2.rel1-darwin-arm64-arm-none-eabi` (the version number may differ). -Initialize the MCUXpresso project: +{{< tabpane code=false >}} +{{< tab header="Linux/macOS" >}} +export ARMGCC_DIR="$HOME/.mcuxpressotools/" +export SdkRootDirPath="$HOME/mcuxsdk_root" +export MCUX_VENV_PATH="$HOME/.mcuxpressotools/.venv/bin" +{{< /tab >}} +{{< tab header="Windows (PowerShell)" >}} +[Environment]::SetEnvironmentVariable("ARMGCC_DIR", "$env:USERPROFILE\.mcuxpressotools\", "User") +[Environment]::SetEnvironmentVariable("SdkRootDirPath", "$env:USERPROFILE\mcuxsdk_root", "User") +[Environment]::SetEnvironmentVariable("MCUX_VENV_PATH", "$env:USERPROFILE\.mcuxpressotools\.venv\Scripts", "User") +{{< /tab >}} +{{< /tabpane >}} + +These quick checks catch most path mistakes before you start debugging build errors: {{< tabpane code=false >}} -{{< tab header="Windows/Linux" >}} -1. Press `Ctrl+Shift+P` to open Command Palette -2. Type: **MCUXpresso: Import Repository** -3. Select the current folder -4. Choose **MIMX9352_cm33** as the target processor +{{< tab header="Linux/macOS" >}} +test -x "$ARMGCC_DIR/bin/arm-none-eabi-gcc" && echo "OK: toolchain" || echo "FAIL: ARMGCC_DIR" +test -d "$SdkRootDirPath/mcuxsdk" && echo "OK: SDK" || echo "FAIL: SdkRootDirPath" {{< /tab >}} -{{< tab header="macOS" >}} -1. Press `Cmd+Shift+P` to open Command Palette -2. Type: **MCUXpresso: Import Repository** -3. Select the current folder -4. Choose **MIMX9352_cm33** as the target processor +{{< tab header="Windows (PowerShell)" >}} +if (Test-Path "$env:ARMGCC_DIR\bin\arm-none-eabi-gcc.exe") { "OK: toolchain" } else { "FAIL: ARMGCC_DIR" } +if (Test-Path "$env:SdkRootDirPath\mcuxsdk") { "OK: SDK" } else { "FAIL: SdkRootDirPath" } {{< /tab >}} {{< /tabpane >}} -VS Code generates the MCUXpresso configuration. +## Apply required SDK patches + +The MCUXpresso SDK ships with a linker script and an Ethos-U driver log header that need two fixes before the firmware can run correctly. The patch script in the repository reads `SdkRootDirPath` to locate your SDK and applies both fixes automatically. + +Using a `bash` shell, navigate to the `Executorch_runner_cm33` directory and run: + +```bash +./patches/apply_patches.sh +``` + +You should see output confirming each patch: + +```output +=== ExecuTorch Runner SDK Patches === + +[1/2] Linker script GOT fix: .../MIMX9352xxxxM_ram.ld + APPLIED: Added *(.got) and *(.got.plt) inside .data section. + +[2/2] Ethos-U driver log redirect: .../ethosu_log.h + APPLIED: LOG_ERR/LOG_WARN/LOG_INFO/LOG_DEBUG now write to remoteproc trace buffer. -## Configure memory settings +=== All patches applied successfully === +``` + +The two patches and what they fix: + +| Patch | What it changes | What happens without it | +|-------|----------------|------------------------| +| **GOT initialization** | Adds `*(.got)` and `*(.got.plt)` inside the `.data` section of the SDK linker script so the startup code copies the Global Offset Table from flash to RAM | The GOT stays zeroed out after boot. Every C++ virtual call resolves to address zero, causing a **BUS FAULT** the first time the firmware calls `load_method` | +| **NPU log redirect** | Replaces the Ethos-U driver's `LOG_ERR`/`LOG_INFO` macros so they write to the remoteproc trace buffer instead of the UART | NPU error messages go to UART only. When you read `trace0` from Linux, driver errors are invisible, making NPU failures difficult to diagnose | + +{{% notice Note %}} +Run the patch script once after installing the SDK. If you run it again, the script detects that the patches are already in place and skips them. If you reinstall or update the SDK, run the script again. +{{% /notice %}} + +## Pre-built ExecuTorch libraries + +The repository includes pre-built static libraries in `executorch/lib/`, cross-compiled for Cortex-M33 with size optimization (`-Os`, MinSizeRel): + +| Library | Size | Purpose | +|---------|------|---------| +| `libexecutorch.a` | 52KB | ExecuTorch runtime | +| `libexecutorch_core.a` | 217KB | Core runtime (gc-sections removes unused code) | +| `libexecutorch_delegate_ethos_u.a` | 19KB | Ethos-U NPU delegate backend | +| `libquantized_ops_lib_selective.a` | 7KB | Registers only `quantize_per_tensor.out` and `dequantize_per_tensor.out` | +| `libquantized_kernels.a` | 242KB | Kernel implementations (gc-sections removes unused code) | +| `libkernels_util_all_deps.a` | 308KB | Kernel utilities (gc-sections removes unused code) | + +{{% notice Note %}} +The selective quantized ops library registers only the two CPU operators needed at the NPU delegation boundary. The full `libquantized_ops_lib.a` registers all quantized operators and pulls in approximately 92KB of kernel code, which overflows the 128KB ITCM. If you rebuild the libraries from source, you must build this selective library separately. +{{% /notice %}} -The Cortex-M33 has 108KB of RAM. The default memory configuration allocates: -- 16KB for the method allocator (activation tensors) -- 8KB for the scratch allocator (temporary operations) +## Understand the memory configuration -These settings are in `CMakeLists.txt`: +The Cortex-M33 has 128KB of ITCM (code) and 108KB of DTCM (data). The firmware also uses reserved DDR regions for the model and NPU working memory. The key settings are defined in `CMakeLists.txt`: ```cmake target_compile_definitions(${MCUX_SDK_PROJECT_NAME} PRIVATE - ET_ARM_BAREMETAL_METHOD_ALLOCATOR_POOL_SIZE=0x4000 # 16KB - ET_ARM_BAREMETAL_SCRATCH_TEMP_ALLOCATOR_POOL_SIZE=0x2000 # 8KB - ET_MODEL_PTE_ADDR=0x80100000 # DDR address for model + ET_ARM_BAREMETAL_METHOD_ALLOCATOR_POOL_SIZE=0x6000 # 24KB method allocator + ET_ARM_BAREMETAL_SCRATCH_TEMP_ALLOCATOR_POOL_SIZE=0x200000 # 2MB scratch allocator + ET_MODEL_PTE_ADDR=0xC0000000 # DDR address where U-Boot loads the .pte model ) ``` +| Setting | Value | Description | +|---------|-------|-------------| +| Method allocator | 24KB (`0x6000`) | Method metadata and small model activations | +| Scratch allocator | 2MB (`0x200000`) | NPU scratch buffer (MobileNet V2 needs approximately 1.5MB) | +| Model address | `0xC0000000` | Start of the 4MB reserved DDR region | + {{% notice Note %}} -If you see "region RAM overflowed" errors during build, reduce these pool sizes. For example, change to 0x2000 (8KB) and 0x1000 (4KB) respectively. +The i.MX93 device tree reserves two DDR regions: `model@c0000000` (4MB for the `.pte` model) and `ethosu_region@A8000000` (128MB for NPU working memory). The NPU scratch buffer is placed at `0xA8000000` and planned buffers for large models at `0xA8200000`, both inside the 128MB ethosu_region. The Ethos-U65 accesses memory via the AXI bus and cannot reach the CM33's tightly-coupled DTCM, so all NPU buffers must be in DDR. {{% /notice %}} ## Build the firmware -Configure the build system: - -{{< tabpane code=false >}} -{{< tab header="Windows/Linux" >}} -1. Press `Ctrl+Shift+P` -2. Type: **CMake: Configure** -3. Select **ARM GCC** as the kit -4. Choose **Debug** or **Release** -{{< /tab >}} -{{< tab header="macOS" >}} -1. Press `Cmd+Shift+P` -2. Type: **CMake: Configure** -3. Select **ARM GCC** as the kit -4. Choose **Debug** or **Release** -{{< /tab >}} -{{< /tabpane >}} - -Build the project: - -Press `F7` or: +Build the project from VS Code. In the left sidebar, open **Explorer**, then in the MCUXpresso **Projects** view select the build icon next to `executorch_runner_cm33`. -{{< tabpane code=false >}} -{{< tab header="Windows/Linux" >}} -1. Press `Ctrl+Shift+P` -2. Type: **CMake: Build** -{{< /tab >}} -{{< tab header="macOS" >}} -1. Press `Cmd+Shift+P` -2. Type: **CMake: Build** -{{< /tab >}} -{{< /tabpane >}} +![VS Code explorer showing project structure with build icon in the Projects tab#center](build-nxp.png "VS Code Projects panel with build icon") -Watch the build output: +The build output shows the progress: ```output [build] Scanning dependencies of target executorch_runner_cm33.elf @@ -202,50 +212,47 @@ Watch the build output: [build] Build finished with exit code 0 ``` -Verify the build succeeded: +Verify the memory usage to ensure the firmware fits in the Cortex-M33: -```bash { output_lines = "2" } -ls -lh build/executorch_runner_cm33.elf --rwxr-xr-x 1 user user 601K executorch_runner_cm33.elf -``` - -Check memory usage to ensure it fits in the Cortex-M33: - -```bash { output_lines = "2-3" } -arm-none-eabi-size build/executorch_runner_cm33.elf - text data bss dec hex filename - 52408 724 50472 103604 19494 executorch_runner_cm33.elf +```output +Memory region Used Size Region Size %age Used + m_interrupts: 1140 B 1144 B 99.65% + m_text: 103476 B 129928 B 79.64% + m_data: 61984 B 108 KB 56.05% ``` -The total RAM usage (data + bss) is approximately 51KB, well within the 108KB limit. +The text section uses approximately 80% of the 128KB ITCM, and data uses approximately 56% of the 108KB DTCM. -## Troubleshooting +You now have everything you need to deploy the `.elf` binary on your NXP board. -**ARM toolchain not found:** +{{% notice Troubleshooting %}} +**SDK patches not applied:** -Add the toolchain to your PATH: +If you see a BUS FAULT during `load_method` or vtable corruption errors, the GOT linker script patch has not been applied. Run: ```bash -export PATH=/opt/arm-gnu-toolchain-14.2.rel1-x86_64-arm-none-eabi/bin:$PATH +./patches/apply_patches.sh ``` -**Cannot find ExecuTorch libraries:** +Then rebuild the project. -Verify the libraries were copied correctly: +**Region `m_text` overflowed:** -```bash -ls executorch/lib/libexecutorch*.a -``` +The 128KB ITCM is nearly full. Verify that `CMakeLists.txt` links `libquantized_ops_lib_selective.a` (not the full `libquantized_ops_lib.a`). The selective library registers only the two operators needed for NPU delegation. -If missing, re-copy from the Docker container. +**`resolve_operator` error for `quantized_decomposed::*`:** -**Region RAM overflowed:** +The quantized operator kernels are not linked. Verify that `CMakeLists.txt` links `libquantized_ops_lib_selective.a` with `--whole-archive` and that `libquantized_kernels.a` and `libkernels_util_all_deps.a` are also listed. -Edit `CMakeLists.txt` and reduce the memory pool sizes: +{{% /notice %}} -```cmake -ET_ARM_BAREMETAL_METHOD_ALLOCATOR_POOL_SIZE=0x2000 # 8KB -ET_ARM_BAREMETAL_SCRATCH_TEMP_ALLOCATOR_POOL_SIZE=0x1000 # 4KB -``` +## What you've learned and what's next + +In this section you've: + +- Set up MCUXpresso for VS Code with the SDK and Arm toolchain +- Cloned the executor_runner project with prebuilt ExecuTorch libraries +- Applied critical SDK patches for GOT initialization and NPU logging +- Built the Cortex-M33 firmware and verified it fits within memory constraints -Then rebuild with `F7`. \ No newline at end of file +With the firmware binary built and its memory usage verified, you're ready to deploy it to the FRDM i.MX 93 and run your first inference. \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/_index.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/_index.md index 58a799b2ba..2c4c653c0b 100644 --- a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/_index.md +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/_index.md @@ -1,27 +1,24 @@ --- -title: Observing Ethos-U on a Physical Device, Built on Arm +title: Deploy ExecuTorch firmware on NXP FRDM i.MX 93 for Ethos-U65 acceleration -draft: true -cascade: - draft: true - minutes_to_complete: 120 -who_is_this_for: This is an introductory topic for developers and data scientists new to Tiny Machine Learning (TinyML), who want to observe ExecuTorch performance on a physical device. +who_is_this_for: This is an introductory topic for developers and data scientists new to TinyML who want to observe ExecuTorch performance on a physical device. learning_objectives: - - Identify suitable physical Arm-based devices for TinyML applications. - - Optionally, configure physical embedded devices. - - Deploy a TinyML ExecuTorch model to NXP's FRDM i.MX 93 applicaiton processor (board). - + - Bring up a custom ExecuTorch `executor_runner` firmware on the FRDM i.MX 93 Cortex-M33 using Linux RemoteProc + - Compile an ExecuTorch `.pte` model for Ethos-U65 and run inference with NPU acceleration + - Understand how heterogeneous Arm systems split responsibilities across application cores, microcontrollers, and NPUs prerequisites: - - Purchase of a NXP [FRDM i.MX 93](https://www.nxp.com/design/design-center/development-boards-and-designs/frdm-i-mx-93-development-board:FRDM-IMX93) board. - - A USB Mini-B to USB Type-A cable, or a USB Mini-B to USB Type-C cable. - - Basic knowledge of Machine Learning concepts. - - A computer running Linux or macOS. - - VS Code + - An NXP [FRDM i.MX 93](https://www.nxp.com/design/design-center/development-boards-and-designs/frdm-i-mx-93-development-board:FRDM-IMX93) development board + - A USB Mini-B to USB Type-A cable, or a USB Mini-B to USB Type-C cable + - Completion of [Use Linux on an NXP FRDM i.MX 93 board](/learning-paths/embedded-and-microcontrollers/linux-nxp-board/) (Linux setup, login access, and file transfer) + - Basic knowledge of Machine Learning concepts + - A host computer to compile ExecuTorch libraries -author: Waheed Brown, Fidel Makatia Omusilibwa +author: +- Waheed Brown +- Fidel Makatia Omusilibwa ### Tags skilllevels: Introductory diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/build-nxp.png b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/build-nxp.png new file mode 100644 index 0000000000..4d89176aaa Binary files /dev/null and b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/build-nxp.png differ diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/import-project.png b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/import-project.png new file mode 100644 index 0000000000..8baefef046 Binary files /dev/null and b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/import-project.png differ diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/imx-93-application-processor-soc.png b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/imx-93-application-processor-soc.png deleted file mode 100644 index 838d47f6d5..0000000000 Binary files a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/imx-93-application-processor-soc.png and /dev/null differ diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/increase-swap-space-to-4-gb.jpg b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/increase-swap-space-to-4-gb.jpg deleted file mode 100644 index bd36c08bac..0000000000 Binary files a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/increase-swap-space-to-4-gb.jpg and /dev/null differ diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/increase-the-memory-limit-to-12-gb.jpg b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/increase-the-memory-limit-to-12-gb.jpg deleted file mode 100644 index 441c20636b..0000000000 Binary files a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/increase-the-memory-limit-to-12-gb.jpg and /dev/null differ diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/mcuxpresso-installer.png b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/mcuxpresso-installer.png new file mode 100644 index 0000000000..630f8f62a1 Binary files /dev/null and b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/mcuxpresso-installer.png differ diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/nxp-frdm-imx-93-board-soc-highlighted.png b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/nxp-frdm-imx-93-board-soc-highlighted.png deleted file mode 100644 index b50ace3a21..0000000000 Binary files a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/nxp-frdm-imx-93-board-soc-highlighted.png and /dev/null differ diff --git a/content/learning-paths/laptops-and-desktops/_index.md b/content/learning-paths/laptops-and-desktops/_index.md index 87833fcbd2..fe2ba9aa2a 100644 --- a/content/learning-paths/laptops-and-desktops/_index.md +++ b/content/learning-paths/laptops-and-desktops/_index.md @@ -7,17 +7,17 @@ key_ip: - Cortex-A maintopic: true operatingsystems_filter: -- Android: 2 +- Android: 3 - ChromeOS: 2 -- Linux: 38 -- macOS: 10 +- Linux: 41 +- macOS: 11 - Windows: 47 subjects_filter: -- CI-CD: 6 -- Containers and Virtualization: 7 +- CI-CD: 7 +- Containers and Virtualization: 6 - Migration to Arm: 30 -- ML: 6 -- Performance and Architecture: 28 +- ML: 8 +- Performance and Architecture: 30 subtitle: Build native Windows on Arm applications that are fast and efficient. title: Laptops and Desktops tools_software_languages_filter: @@ -28,28 +28,31 @@ tools_software_languages_filter: - Arm Performance Libraries: 2 - Arm64EC: 1 - Assembly: 1 -- Bash: 3 -- C: 11 +- Bash: 4 +- C: 12 - C#: 6 - CCA: 1 - Clang: 13 -- CMake: 3 +- CMake: 4 - CPP: 12 - CSS: 1 -- Daytona: 1 -- Docker: 6 +- Docker: 9 +- ExecuTorch: 1 - FFmpeg: 1 - GCC: 12 - Git: 1 - GitHub: 3 +- GitHub Actions: 1 - GitLab: 2 - Google Benchmark: 1 - Google Test: 1 - HTML: 2 -- Hugging Face: 1 +- Hugging Face: 2 - Hyper-V: 1 - i3: 1 - Intrinsics: 1 +- IsaacLab: 1 +- IsaacSim: 1 - JavaScript: 2 - KleidiCV: 1 - Kubernetes: 1 @@ -59,7 +62,9 @@ tools_software_languages_filter: - LLM: 1 - LLVM: 2 - llvm-mca: 1 +- MCP: 1 - MSBuild: 1 +- MSVC: 1 - MTE: 1 - Neon: 1 - Neovim: 1 @@ -67,17 +72,22 @@ tools_software_languages_filter: - ONNX Runtime: 1 - OpenCV: 1 - perf: 4 +- PGO: 1 - PowerShell: 1 -- Python: 9 +- Pytest: 1 +- Python: 14 +- PyTorch: 1 - QEMU: 1 - Qt: 2 - RDP: 1 - Remote.It: 1 - RME: 1 -- Runbook: 18 +- Runbook: 17 - Rust: 2 +- SME2: 3 - SVE: 1 - SVE2: 1 +- Testcontainers: 1 - Trusted Firmware: 1 - Ubuntu: 1 - Visual Studio: 14 diff --git a/content/learning-paths/laptops-and-desktops/dgx_spark_isaac_robotics/_index.md b/content/learning-paths/laptops-and-desktops/dgx_spark_isaac_robotics/_index.md index d4b5893f5f..7d5a7c916b 100644 --- a/content/learning-paths/laptops-and-desktops/dgx_spark_isaac_robotics/_index.md +++ b/content/learning-paths/laptops-and-desktops/dgx_spark_isaac_robotics/_index.md @@ -1,10 +1,6 @@ --- title: Build Robot Simulation and Reinforcement Learning Workflows with Isaac Sim and Isaac Lab on DGX Spark -draft: true -cascade: - draft: true - minutes_to_complete: 90 who_is_this_for: This is an advanced topic for robotics developers, simulation engineers, and AI researchers who want to run high-fidelity robotic simulations and reinforcement learning (RL) pipelines using NVIDIA Isaac Sim and Isaac Lab on Arm-based NVIDIA DGX Spark system powered by the Grace–Blackwell (GB10) architecture. diff --git a/content/learning-paths/mobile-graphics-and-gaming/_index.md b/content/learning-paths/mobile-graphics-and-gaming/_index.md index 45814a60c4..ce93cbc1bd 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/_index.md +++ b/content/learning-paths/mobile-graphics-and-gaming/_index.md @@ -9,14 +9,14 @@ key_ip: - Mali maintopic: true operatingsystems_filter: -- Android: 35 -- Linux: 36 -- macOS: 16 -- Windows: 15 +- Android: 40 +- Linux: 40 +- macOS: 19 +- Windows: 16 subjects_filter: - Gaming: 6 - Graphics: 6 -- ML: 18 +- ML: 23 - Performance and Architecture: 36 subtitle: Optimize Android apps and build faster games using cutting-edge Arm tech. title: Mobile, Graphics, and Gaming @@ -35,13 +35,13 @@ tools_software_languages_filter: - Bazel: 1 - C: 5 - C#: 3 -- C++: 1 +- C++: 3 - CCA: 1 - Clang: 12 -- CMake: 2 +- CMake: 3 - CPP: 15 - Docker: 1 -- ExecuTorch: 4 +- ExecuTorch: 7 - Frame Advisor: 1 - GCC: 12 - Generative AI: 2 @@ -52,20 +52,21 @@ tools_software_languages_filter: - Hugging Face: 6 - Java: 7 - Jupyter Notebook: 1 -- KleidiAI: 2 +- KleidiAI: 3 - Kotlin: 9 - LiteRT: 1 -- llama.cpp: 1 +- llama.cpp: 3 - LLM: 1 - LLVM: 1 - llvm-mca: 1 - MediaPipe: 2 - MTE: 2 - Neon: 1 +- NX: 4 - ONNX: 1 - ONNX Runtime: 2 - OpenGL ES: 1 -- Python: 9 +- Python: 11 - PyTorch: 3 - QEMU: 1 - RenderDoc: 1 @@ -73,14 +74,17 @@ tools_software_languages_filter: - Runbook: 14 - Rust: 2 - SDDiskTool: 1 +- SME2: 9 - SVE2: 1 - TensorFlow: 1 +- TorchAO: 1 +- TOSA: 1 - Trusted Firmware: 1 - Unity: 6 - Unreal Engine: 4 - Visual Studio: 1 - Visual Studio Code: 1 -- Vulkan: 5 +- Vulkan: 6 - Vulkan SDK: 1 - XNNPACK: 2 weight: 3 diff --git a/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/_index.md b/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/_index.md new file mode 100644 index 0000000000..2d768b5fd6 --- /dev/null +++ b/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/_index.md @@ -0,0 +1,67 @@ +--- +title: Build a customer support chatbot on Android with Llama and ExecuTorch + +minutes_to_complete: 60 + +who_is_this_for: This is an introductory topic for software developers interested in building an on-device customer support chatbot for Android using Meta's Llama models and the ExecuTorch runtime. + +learning_objectives: + - Set up a development environment for building and deploying ExecuTorch-based apps on Android + - Describe how ExecuTorch uses KleidiAI kernels to accelerate performance on Arm-based platforms + - Export a Llama 3.2 model to .pte format optimized for on-device inference + - Run a Llama model on an Arm-powered Android phone and verify inference performance + - Build and run an Android chat app configured as a customer support assistant + +prerequisites: + - An Apple M1/M2/M3 development machine, or a Linux machine with at least 16GB of RAM + - An Arm-powered smartphone with the i8mm feature running Android, with 16GB of RAM + - A USB cable to connect your smartphone to your development machine + - Android Debug Bridge (adb) installed. Follow the steps in [adb](https://developer.android.com/tools/adb) to install Android SDK Platform Tools + - Java 17 JDK. Follow the steps in [Java SE 17 Archive Downloads](https://www.oracle.com/java/technologies/javase/jdk17-archive-downloads.html) to download and install JDK for your host + - Python 3.10 or later + - A [Hugging Face](https://huggingface.co/) account with access to Meta Llama models + +author: Parichay Das + +### Tags +skilllevels: Introductory +subjects: ML +armips: + - Cortex-A + +tools_software_languages: + - Java + - Python + - ExecuTorch + +operatingsystems: + - macOS + - Linux + - Android + + +further_reading: + - resource: + title: ExecuTorch Overview + link: https://pytorch.org/executorch-overview + type: website + - resource: + title: ExecuTorch Documentation + link: https://pytorch.org/executorch/stable/index.html + type: documentation + - resource: + title: KleidiAI + link: https://gitlab.arm.com/kleidi/kleidiai + type: website + - resource: + title: Build an Android chat app with Llama, KleidiAI, ExecuTorch, and XNNPACK + link: /learning-paths/mobile-graphics-and-gaming/build-llama3-chat-android-app-using-executorch-and-xnnpack/ + type: learning-path + + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 # _index.md always has weight of 1 to order correctly +layout: "learningpathall" # All files under learning paths have this same wrapper +learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content. +--- diff --git a/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/_next-steps.md b/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/_next-steps.md new file mode 100644 index 0000000000..c3db0de5a2 --- /dev/null +++ b/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/_next-steps.md @@ -0,0 +1,8 @@ +--- +# ================================================================================ +# FIXED, DO NOT MODIFY THIS FILE +# ================================================================================ +weight: 21 # Set to always be larger than the content in this path to be at the end of the navigation. +title: "Next Steps" # Always the same, html page title. +layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing. +--- diff --git a/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-1.md b/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-1.md new file mode 100644 index 0000000000..d3ce2e917b --- /dev/null +++ b/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-1.md @@ -0,0 +1,106 @@ +--- +title: Create a development environment +weight: 2 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Set up your development environment + +In this Learning Path, you will build and deploy an on-device customer support chatbot to an Android smartphone using ExecuTorch and XNNPACK with [KleidiAI](https://gitlab.arm.com/kleidi/kleidiai). Arm has worked with the Meta team to integrate KleidiAI into ExecuTorch through XNNPACK. These optimizations increase the throughput of quantized LLMs running on Arm chips with the i8mm (8-bit integer matrix multiply) feature. Running the chatbot entirely on-device means no cloud dependency, lower latency, and greater privacy for your users. + +The first step is to prepare a development environment with the required software: + +- Android Studio (latest version recommended). +- Android NDK version 29.0.14206865 or later. +- Java 17 JDK. +- Git. +- Python 3.10 or later (these instructions have been tested with 3.10 and 3.12). + +The instructions assume macOS with Apple Silicon, or a Debian or Ubuntu Linux machine, with at least 16GB of RAM. + +## Install Java 17 JDK + +Open the [Java SE 17 Archive Downloads](https://www.oracle.com/java/technologies/javase/jdk17-archive-downloads.html) page in your browser. + +Select an appropriate download for your development machine operating system. Downloads are available for macOS as well as Linux. + +## Install and configure Android Studio + +Download and install the latest version of Android Studio from the Downloads page: + +``` +https://developer.android.com/studio/ +``` + +### For macOS: Using the UI + +Configure Android Studio: + +- Start Android Studio and open the **Settings** dialog +- Navigate to **Languages & Frameworks**, then **Android SDK** +- In the **SDK Platforms** tab, check **Android 14.0 ("UpsideDownCake")**. Select **Apply** to install +- In the **SDK Tools** tab, check **NDK (Side by side)**. Select **Apply** to install + +Set the `ANDROID_HOME` environment variable: + +```bash +export ANDROID_HOME="$(realpath ~/Library/Android/sdk)" +``` + +### For Linux: Using the CLI + +Command-line tools let you manage Android SDK components without the GUI. Create the SDK directory and download the command-line tools: + +```bash +mkdir -p ~/Android/cmdline-tools +cd ~/Android/cmdline-tools +wget https://dl.google.com/android/repository/commandlinetools-linux-10406996_latest.zip +``` + +Unzip and move the directory: + +```bash +unzip commandlinetools-linux-*.zip +mv cmdline-tools latest +``` + +Set the `ANDROID_HOME` environment variable and add `sdkmanager` to `PATH`: + +```bash +export ANDROID_HOME="~/Android" +export PATH="$ANDROID_HOME/cmdline-tools/latest/bin:$ANDROID_HOME/platform-tools:$PATH" +``` + +Accept the license agreements. Press `y`, then **Enter**, as many times as prompted. + +```bash +sdkmanager --licenses +``` + +Install the required Android SDK components: + +```bash +sdkmanager "platform-tools" \ + "platforms;android-34" \ + "build-tools;34.0.0" \ + "ndk;29.0.14206865" +``` + +## Verify NDK installation + +Verify that the NDK was installed in the same directory where Android Studio installed the SDK. + +{{% notice Default Path %}} +On macOS, this is generally `~/Library/Android/sdk`, and on Linux, it's `~/Android/Sdk`. Update the command to use your installed NDK version. +{{% /notice %}} + +## What you've learned and what's next + +You have set up a complete Android development environment with: +- Android Studio configured with the required SDK and NDK +- Environment variables set for Android development +- All necessary build tools installed + +In the next section, you will set up ExecuTorch, the runtime that enables efficient on-device inference for PyTorch models on mobile platforms. diff --git a/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-2.md b/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-2.md new file mode 100644 index 0000000000..17a31d75ca --- /dev/null +++ b/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-2.md @@ -0,0 +1,47 @@ +--- +title: Set up ExecuTorch +weight: 3 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Set up ExecuTorch + +ExecuTorch is an end-to-end solution for enabling on-device inference across mobile and edge devices, including wearables, embedded devices, and microcontrollers. It is part of the PyTorch Edge ecosystem and enables efficient deployment of PyTorch models to edge devices. You can learn more by reading the [ExecuTorch Overview](https://pytorch.org/executorch/stable/intro-overview.html). + +The best practice is to create an isolated Python environment to install ExecuTorch dependencies. + +### Create a Python virtual environment + +Use the `venv` module available through Python: + +```bash +python3.10 -m venv executorch-venv +source executorch-venv/bin/activate +``` + +Your terminal prompt now shows `executorch-venv` as a prefix to indicate the virtual environment is active. + +### Clone ExecuTorch and install dependencies + +From within the virtual environment, run the commands below to download the ExecuTorch repository and install the required packages: + +```bash +git clone https://github.com/pytorch/executorch.git +cd executorch +git checkout release/1.0 +git submodule sync +git submodule update --init --recursive +./install_executorch.sh +./examples/models/llama/install_requirements.sh +``` + +## What you've learned and what's next + +You have successfully: +- Created an isolated Python environment for ExecuTorch +- Downloaded and configured the ExecuTorch repository +- Installed all required dependencies for Llama model deployment + +Before preparing your Llama model for deployment, the next section explains what Llama models are and why they work well for customer support applications. diff --git a/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-3.md b/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-3.md new file mode 100644 index 0000000000..1c77488c7d --- /dev/null +++ b/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-3.md @@ -0,0 +1,71 @@ +--- +title: Understand Llama models +weight: 4 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Overview + +Before exporting and deploying a model, understanding Llama's capabilities and constraints helps you make informed decisions about model selection, quantization strategy, and expected performance on mobile devices. + +## What is Llama? + +Llama is a family of large language models that uses publicly available data for training. Llama models perform well on a variety of natural language processing tasks, such as: + +- Language translation +- Question answering +- Text summarization + +Llama models are capable of generating human-like text, making them well-suited for customer-facing applications where natural, contextually relevant responses matter. A customer support chatbot built on Llama can handle free-form queries, explain product details, and guide users through troubleshooting steps -- all on-device, with no data sent to external servers. + +Please note that the models are subject to the [acceptable use policy](https://github.com/facebookresearch/llama/blob/main/USE_POLICY.md) and [this responsible use guide](https://ai.meta.com/static-resource/responsible-use-guide/). + +## Why on-device inference? + +Running inference on the device rather than calling a cloud API has three key advantages for customer support: + +- **Privacy**: user queries and session data never leave the device. +- **Low latency**: responses begin generating immediately, with no round-trip network call. +- **Offline availability**: the bot works even without an internet connection. + +The tradeoff is memory. Smartphone memory constraints mean you need to quantize the model before deployment. + +## Quantization + +One way to create models that fit in smartphone memory is to use 4-bit groupwise per-token dynamic quantization of all linear layers. *Dynamic quantization* means quantization parameters for activations are calculated at runtime from the observed min/max range. Weights are statically quantized -- grouped per channel using 4-bit signed integers. + +For further information, refer to [torchao: PyTorch Architecture Optimization](https://github.com/pytorch-labs/ao/). + +The table below evaluates WikiText perplexity using [LM Eval](https://github.com/EleutherAI/lm-evaluation-harness). + +The results are for two different groupsizes, with max_seq_len 2048, and 1000 samples: + +| Model | Baseline (FP32) | Groupwise 4-bit (128) | Groupwise 4-bit (256) | +|--------------|-----------------|-----------------------|-----------------------| +| Llama 2 7B | 9.2 | 10.2 | 10.7 | +| Llama 3 8B | 7.9 | 9.4 | 9.7 | + +Note that groupsize less than 128 was not enabled in this example, because the model was still too large. Current efforts have focused on enabling FP32, and support for FP16 is under way. + +What this implies for model size: + +- The embedding table is in FP32 +- Quantized weight scales are FP32 + +## KleidiAI and Arm performance + +Arm has contributed [KleidiAI](https://gitlab.arm.com/kleidi/kleidiai) kernels into ExecuTorch via XNNPACK. On Arm Cortex-A processors with the i8mm feature (such as those found in many recent Android smartphones), these kernels can significantly improve inference throughput for quantized LLMs compared to standard XNNPACK kernels. + +You will enable this acceleration in the build step later in the Learning Path. + +## What you've learned and what's next + +You now understand: +- Llama's capabilities for natural language tasks and customer support scenarios +- The benefits of on-device inference: privacy, low latency, and offline availability +- How quantization makes large language models fit in smartphone memory +- KleidiAI's role in accelerating inference on Arm processors + +The next section walks you through downloading and exporting a Llama model to the format required by ExecuTorch. diff --git a/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-4.md b/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-4.md new file mode 100644 index 0000000000..539e9a16f0 --- /dev/null +++ b/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-4.md @@ -0,0 +1,75 @@ +--- +title: Prepare Llama models for ExecuTorch +weight: 5 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Download and export the Llama 3.2 1B model + +To get started with Llama 3, obtain the pre-trained parameters by visiting [Meta's Llama Downloads](https://llama.meta.com/llama-downloads/) page. Request access by providing your details, then read and accept the Responsible Use Guide. After approval, you receive a license and a download link valid for 24 hours. The Llama 3.2 1B Instruct model is used in this Learning Path, but the same instructions apply to other variants with minimal modification. + +Install the `llama-stack` package from `pip`: + +```bash +pip install llama-stack +``` + +Run the command to download, and paste the download link from the email when prompted: + +```bash +llama model download --source meta --model-id Llama3.2-1B-Instruct +``` + +When the download finishes, the installation path prints: + +```output +Successfully downloaded model to //.llama/checkpoints/Llama3.2-1B-Instruct +``` + +Verify by listing the downloaded files: + +```bash +ls $HOME/.llama/checkpoints/Llama3.2-1B-Instruct +``` + +The output is similar to: + +```output +checklist.chk consolidated.00.pth params.json tokenizer.model +``` + +{{% notice Working Directory %}} +The remaining instructions should be run from the ExecuTorch base directory. +{{% /notice %}} + +## Export to .pte format + +Export the model and generate a `.pte` file. Run the Python command below to export the model to your current directory: + +```bash +python3 -m examples.models.llama.export_llama \ +--checkpoint $HOME/.llama/checkpoints/Llama3.2-1B-Instruct/consolidated.00.pth \ +--params $HOME/.llama/checkpoints/Llama3.2-1B-Instruct/params.json \ +-kv --use_sdpa_with_kv_cache -X --xnnpack-extended-ops -qmode 8da4w \ +--group_size 64 -d fp32 \ +--metadata '{"get_bos_id":128000, "get_eos_ids":[128009, 128001, 128006, 128007]}' \ +--embedding-quantize 4,32 \ +--output_name="llama3_1B_kv_sdpa_xnn_qe_4_64_1024_embedding_4bit.pte" \ +--max_seq_length 1024 \ +--max_context_length 1024 +``` + +Due to the larger vocabulary size of Llama 3, quantize the embeddings with `--embedding-quantize 4,32` to further reduce model size. + +When this command completes, you have a `llama3_1B_kv_sdpa_xnn_qe_4_64_1024_embedding_4bit.pte` file in your working directory. This is the compiled, quantized model ready for on-device deployment via ExecuTorch. + +## What you've learned and what's next + +You have: +- Downloaded the Llama 3.2 1B Instruct model from Meta +- Exported the model to `.pte` format with quantization optimizations +- Created a model file ready for on-device deployment + +The next section builds the Llama runner for Android and demonstrates how to run the chatbot directly on your phone via the command line. diff --git a/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-5.md b/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-5.md new file mode 100644 index 0000000000..e4f324a8d4 --- /dev/null +++ b/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-5.md @@ -0,0 +1,151 @@ +--- +title: Run the chatbot on Android +weight: 6 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Build the Llama runner binary for Android + +Cross-compile the Llama runner to run on your Android phone using the steps below. + +### Set the Android NDK path + +Set the environment variable to point to the Android NDK: + +```bash +export ANDROID_NDK=$ANDROID_HOME/ndk/29.0.14206865/ +``` + +{{% notice Note %}} +Make sure `$ANDROID_NDK/build/cmake/android.toolchain.cmake` is available for CMake to cross-compile. +{{% /notice %}} + +### Build ExecuTorch and associated libraries for Android with KleidiAI + +Use `cmake` to cross-compile ExecuTorch, enabling the KleidiAI kernels for accelerated inference on Arm: + +```bash +cmake -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \ + -DANDROID_ABI=arm64-v8a \ + -DANDROID_PLATFORM=android-23 \ + -DEXECUTORCH_BUILD_EXTENSION_NAMED_DATA_MAP=ON \ + -DCMAKE_INSTALL_PREFIX=cmake-out-android \ + -DEXECUTORCH_ENABLE_LOGGING=1 \ + -DCMAKE_BUILD_TYPE=Release \ + -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \ + -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \ + -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \ + -DEXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR=ON \ + -DEXECUTORCH_BUILD_XNNPACK=ON \ + -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON \ + -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON \ + -DEXECUTORCH_BUILD_KERNELS_CUSTOM=ON \ + -DEXECUTORCH_BUILD_KERNELS_LLM=ON \ + -DEXECUTORCH_BUILD_EXTENSION_LLM_RUNNER=ON \ + -DEXECUTORCH_BUILD_EXTENSION_LLM=ON \ + -DEXECUTORCH_BUILD_EXTENSION_RUNNER_UTIL=ON \ + -DEXECUTORCH_XNNPACK_ENABLE_KLEIDI=ON \ + -DXNNPACK_ENABLE_ARM_BF16=OFF \ + -DBUILD_TESTING=OFF \ + -Bcmake-out-android . + +cmake --build cmake-out-android -j7 --target install --config Release +``` + +{{% notice Note %}} +Starting with ExecuTorch version 0.7 beta, KleidiAI is enabled by default. The `-DEXECUTORCH_XNNPACK_ENABLE_KLEIDI=ON` option adds built-in support for KleidiAI kernels in ExecuTorch with XNNPACK. +{{% /notice %}} + +### Build the Llama runner for Android + +Use `cmake` to cross-compile the Llama runner: + +```bash +cmake -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \ + -DANDROID_ABI=arm64-v8a \ + -DANDROID_PLATFORM=android-23 \ + -DCMAKE_INSTALL_PREFIX=cmake-out-android \ + -DCMAKE_BUILD_TYPE=Release \ + -DPYTHON_EXECUTABLE=python \ + -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON \ + -DBUILD_TESTING=OFF \ + -Bcmake-out-android/examples/models/llama \ + examples/models/llama + +cmake --build cmake-out-android/examples/models/llama -j16 --config Release +``` + +You should now have `llama_main` available for Android. + +{{% notice Note %}} +If Gradle cannot find the Android SDK, add the `sdk.dir` path to `executorch/extension/android/local.properties`. +{{% /notice %}} + +## Run the chatbot on Android via adb shell + +You need an Arm-powered smartphone with the i8mm feature running Android, with at least 16GB of RAM. The steps below were tested on a Google Pixel 8 Pro. + +### Connect your Android phone + +Connect your phone to your computer using a USB cable. + +Enable USB debugging on your Android device by following [Configure on-device developer options](https://developer.android.com/studio/debug/dev-options). + +Once USB debugging is enabled, run: + +```bash +adb devices +``` + +You should see your device listed to confirm it is connected. + +### Copy the model, tokenizer, and runner binary to the phone + +```bash +adb shell mkdir -p /data/local/tmp/llama +adb push llama3_1B_kv_sdpa_xnn_qe_4_64_1024_embedding_4bit.pte /data/local/tmp/llama/ +adb push $HOME/.llama/checkpoints/Llama3.2-1B-Instruct/tokenizer.model /data/local/tmp/llama/ +adb push cmake-out-android/examples/models/llama/llama_main /data/local/tmp/llama/ +``` + +### Run the chatbot + +The system prompt is where you define your chatbot's persona and behavior. The example below configures the model as a customer support assistant. Adapt the system prompt text to match your product or domain. + +```bash +adb shell "cd /data/local/tmp/llama && ./llama_main \ + --model_path llama3_1B_kv_sdpa_xnn_qe_4_64_1024_embedding_4bit.pte \ + --tokenizer_path tokenizer.model \ + --prompt '<|start_header_id|>system<|end_header_id|>\nYou are a helpful customer support assistant. You answer questions about products, help with troubleshooting, and escalate issues politely when needed. Keep responses concise and friendly.<|eot_id|><|start_header_id|>user<|end_header_id|>\nMy order has not arrived yet. What should I do?<|eot_id|><|start_header_id|>assistant<|end_header_id|>' \ + --warmup=1 --cpu_threads=5" +``` + +The output is similar to: + +```output +I tokenizers:regex.cpp:27] Registering override fallback regex +I 00:00:00.003288 executorch:main.cpp:87] Resetting threadpool with num threads = 5 +I 00:00:00.006393 executorch:runner.cpp:44] Creating LLaMa runner: model_path=llama3_1B_kv_sdpa_xnn_qe_4_64_1024_embedding_4bit.pte, tokenizer_path=tokenizer.model +I 00:00:00.131486 executorch:llm_runner_helper.cpp:57] Loaded TikToken tokenizer +I 00:00:00.186538 executorch:llm_runner_helper.cpp:110] Metadata: use_sdpa_with_kv_cache = 1 +I 00:00:00.186574 executorch:llm_runner_helper.cpp:110] Metadata: use_kv_cache = 1 +I 00:00:00.186578 executorch:llm_runner_helper.cpp:110] Metadata: get_max_context_len = 1024 +I 00:00:00.186584 executorch:llm_runner_helper.cpp:110] Metadata: get_max_seq_len = 1024 +I 00:00:01.086570 executorch:text_llm_runner.cpp:89] Doing a warmup run... +I 00:00:02.264379 executorch:text_llm_runner.cpp:209] Warmup run finished! +I 00:00:02.264384 executorch:text_llm_runner.cpp:95] RSS after loading model: 1122.187500 MiB (0 if unsupported) +``` + +After the warmup, the model generates a response to the user query. The RSS figure confirms how much memory the model is using on the device. + +## What you've learned and what's next + +You have: +- Cross-compiled ExecuTorch with KleidiAI support for Android +- Built the Llama runner binary for Arm Android devices +- Deployed and run the chatbot on your Android phone via adb +- Verified that the model inference works correctly with a customer support prompt + +The next section wraps this functionality into an Android app with a full chat interface. diff --git a/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-6.md b/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-6.md new file mode 100644 index 0000000000..0db53e547f --- /dev/null +++ b/content/learning-paths/mobile-graphics-and-gaming/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-6.md @@ -0,0 +1,102 @@ +--- +title: Build and run the Android chat app +weight: 7 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Overview + +In this section, you will use the ExecuTorch Android demo application to run the customer support chatbot with a full chat interface on your phone. + +## Build the Android Archive (AAR) + +Set up the build environment: + +- Open a terminal and navigate to the root directory of the `executorch` repository +- If you have not already done so, set the following environment variables: + + ```bash + export ANDROID_NDK=$ANDROID_HOME/ndk/29.0.14206865/ + export ANDROID_ABI=arm64-v8a + export ANDROID_SDK=$ANDROID_HOME + ``` + + {{% notice Note %}} +`` is the root for the NDK, which is usually under `~/Library/Android/sdk/ndk/XX.Y.ZZZZZ` on macOS, and contains `NOTICE` and `README.md`. Make sure `/build/cmake/android.toolchain.cmake` is available for CMake to cross-compile. + {{% /notice %}} + +Run the following command to set up the required JNI library: + +```bash +sh scripts/build_android_library.sh +``` + +## Copy model files to the phone + +Make sure the exported model and tokenizer are on your Android phone. + +### Option 1: Using adb + +Check if the files are already on the phone: + +```bash +adb shell "ls -la /data/local/tmp/llama/" +``` + +If they are not present, copy them: + +```bash +adb shell mkdir -p /data/local/tmp/llama +adb push llama3_1B_kv_sdpa_xnn_qe_4_64_1024_embedding_4bit.pte /data/local/tmp/llama/ +adb push $HOME/.llama/checkpoints/Llama3.2-1B-Instruct/tokenizer.model /data/local/tmp/llama/ +``` + +### Option 2: Using Android Studio + +Use Android Studio's **Device Explorer** to browse the phone's filesystem and upload the files if they are not already present. + +## Build the Android Package Kit + +Clone the `executorch-examples` repository, which contains the LlamaDemo app: + +```bash +git clone https://github.com/meta-pytorch/executorch-examples.git +``` + +### Option 1: Using Android Studio (recommended) + +Build and launch the app: + +- Open Android Studio and select **Open an existing Android Studio project** +- Navigate to and open `executorch-examples/llm/android/LlamaDemo` +- Run the app (**^R**). This builds and launches the app on your connected phone + +### Option 2: Command line + +```bash +pushd llm/android/LlamaDemo +./gradlew :app:installDebug +popd +``` + +## Configure the chatbot persona + +Once the app is running, you can set a system prompt in the app's settings to configure it as a customer support assistant. Set the system prompt to something like: + +``` +You are a helpful customer support assistant. You answer questions about products, help with troubleshooting, and escalate issues politely when needed. Keep responses concise and friendly. +``` + +This gives the Llama model its role and behavioral guidelines for every conversation, without changing the underlying model weights. + +## What you've learned and what's next + +You have successfully: +- Built the ExecuTorch Android library with KleidiAI support +- Deployed the model and tokenizer to your Android device +- Built and launched a full-featured chat application +- Configured the app as a customer support assistant + +You now have a fully functional on-device customer support chatbot running on an Arm Android phone using ExecuTorch and KleidiAI. All inference runs locally with no cloud dependency and no user data leaving the device. You can customize the system prompt to match your specific product or domain requirements. diff --git a/content/learning-paths/servers-and-cloud-computing/_index.md b/content/learning-paths/servers-and-cloud-computing/_index.md index 5481f854d4..095b3ea8b3 100644 --- a/content/learning-paths/servers-and-cloud-computing/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/_index.md @@ -8,8 +8,9 @@ key_ip: maintopic: true operatingsystems_filter: - Android: 3 -- Linux: 211 +- Linux: 223 - macOS: 15 +- other: 1 - Windows: 14 pinned_modules: - module: @@ -18,12 +19,12 @@ pinned_modules: - providers - migration subjects_filter: -- CI-CD: 12 -- Containers and Virtualization: 36 -- Databases: 23 +- CI-CD: 13 +- Containers and Virtualization: 41 +- Databases: 24 - Libraries: 9 -- ML: 36 -- Performance and Architecture: 80 +- ML: 32 +- Performance and Architecture: 89 - Storage: 2 - Web: 17 subtitle: Migrate and optimize cloud native apps on Arm-based servers. @@ -33,49 +34,52 @@ tools_software_languages_filter: - .NET SDK: 1 - 5G: 1 - ACL: 1 +- ActiveGate: 1 - AI: 1 - Android Studio: 1 - Ansible: 2 - Apache: 1 +- Apache Arrow: 1 - Apache Beam: 1 -- Apache Bench: 1 - Apache Cassandra: 1 - Apache Spark: 2 - Apache Tomcat: 2 - ApacheBench: 1 +- Argo CD: 1 - Arm Compiler for Linux: 1 - Arm Development Studio: 3 - Arm ISA: 1 -- Arm Performance Libraries: 1 +- Arm Performance Libraries: 2 +- Arm Performix: 2 - Arm Streamline: 1 - armclang: 1 - armie: 1 - ArmRAL: 1 +- Arrow Flight: 1 - ASP.NET Core: 2 - Assembly: 5 - async-profiler: 1 - Autocannon: 1 - AWS: 2 -- AWS CDK: 2 - AWS Cloud Formation: 1 - AWS CodeBuild: 1 -- AWS EC2: 2 +- AWS EC2: 3 - AWS Elastic Container Service (ECS): 1 - AWS Elastic Kubernetes Service (EKS): 3 - AWS Graviton: 1 - AWS Lambda: 1 - Azure: 1 -- Azure CLI: 2 +- Azure CLI: 3 - Azure Portal: 1 - Bash: 3 - bash: 2 - Bastion: 3 -- BOLT: 2 +- BOLT: 3 - bpftool: 1 - Buildkite: 1 -- C: 12 +- C: 14 - C#: 2 -- C++: 1 +- C++: 3 - Capstone: 1 - Cargo: 1 - cassandra-stress: 1 @@ -86,51 +90,53 @@ tools_software_languages_filter: - ClickBench: 1 - ClickHouse: 2 - Cloud Build: 1 -- CMake: 1 -- conda: 1 +- Cloud SQL (PostgreSQL): 1 +- CMake: 2 - Confidential Containers: 1 - Couchbase: 1 - CPP: 15 - cqlsh: 1 - Criterion: 1 -- Daytona: 1 - Demo: 3 - Django: 2 -- Docker: 30 +- Docker: 35 - Docker Buildx: 1 +- Dynatrace: 1 - Envoy: 3 - Erlang: 1 - ExecuTorch: 1 - Facter: 1 -- FAISS: 1 +- Fastpath: 1 - FlameGraph: 1 +- Flask: 1 - Flink: 2 - Fortran: 1 - FunASR: 1 - FVP: 8 - Gardener: 1 -- GCC: 26 +- GCC: 28 - gdb: 1 - Geekbench: 1 - Generative AI: 13 -- Git: 1 -- GitHub: 6 -- GitHub Actions: 1 +- Git: 2 +- GitHub: 3 +- GitHub Actions: 3 - GitHub CLI: 1 -- GitHub Copilot: 1 +- GitHub Copilot: 2 - GitLab: 2 -- GKE: 2 +- GKE: 3 - glibc: 1 - Go: 4 - Golang: 1 +- Google Artifact Registry: 1 - Google Axion: 3 - Google Benchmark: 2 - Google Cloud: 2 - Google Cloud Pub/Sub: 1 - Google Dataflow: 1 - Google Test: 1 +- Grafana: 1 - Groovy (Jenkins Pipeline): 1 -- Gunicorn: 1 - HammerDB: 1 - Helm: 3 - Herd7: 1 @@ -140,33 +146,40 @@ tools_software_languages_filter: - Intrinsics: 1 - iPerf3: 1 - ipmitool: 1 +- Jaeger: 1 - Java: 6 - JAX: 1 - Jenkins: 1 - JMH: 1 +- JSON: 1 - Kafka: 2 - Kata Containers: 1 - KEDA: 1 - Kedify: 1 - Keras: 2 - KinD: 1 +- Kiro: 1 - kube-bench: 1 -- kubectl: 1 -- Kubernetes: 15 -- Libamath: 1 +- kubectl: 2 +- Kubernetes: 17 +- Libamath: 2 - libbpf: 1 - Linaro Forge: 1 +- Linux: 1 - Litmus7: 1 - llama.cpp: 2 - Llama.cpp: 2 - LLM: 11 +- LLVM: 1 - llvm-mca: 1 - LM Evaluation Harness: 1 - LSE: 1 - MariaDB: 1 - Maven: 1 -- MCP: 1 +- MCP: 3 - Memcached: 2 +- Memorystore (Redis): 1 +- MinIO: 1 - MLPerf: 1 - ModelScope: 1 - MongoDB: 4 @@ -177,7 +190,7 @@ tools_software_languages_filter: - Neon: 7 - Networking: 1 - Nexmark: 1 -- NGINX: 5 +- NGINX: 7 - nginx: 1 - Node.js: 5 - node.js: 1 @@ -191,17 +204,20 @@ tools_software_languages_filter: - OpenShift: 1 - Orchard Core: 1 - PAPI: 1 -- perf: 7 +- perf: 8 - Perf: 1 - PHP: 1 - PHPBench: 1 - pika: 1 -- PostgreSQL: 6 +- PostgreSQL: 7 - Profiling: 1 +- Prometheus: 1 +- psycopg2: 1 - Puppet: 1 -- Python: 39 +- Pytest: 1 +- Python: 37 - Python 3.11: 1 -- PyTorch: 11 +- PyTorch: 10 - QEMU: 1 - RabbitMQ: 1 - RAG: 1 @@ -211,11 +227,12 @@ tools_software_languages_filter: - Remote.It: 2 - RME: 9 - Ruby: 2 -- Runbook: 71 +- Runbook: 73 - Rust: 3 - Service Mesh: 1 - Siege: 1 - Skaffold: 1 +- SME2: 1 - snappy: 1 - Snort3: 1 - SQL: 8 @@ -230,16 +247,23 @@ tools_software_languages_filter: - Telemetry: 1 - TensorFlow: 3 - Terraform: 11 +- Testcontainers: 1 - ThirdAI: 1 +- TimescaleDB: 1 +- Tinkerblox: 1 - topdown-tool: 1 +- Trivy: 1 - Trusted Firmware: 1 - Trustee: 2 - TSan: 1 +- TuxMake: 1 +- tuxmake: 1 - TypeScript: 2 - Vectorscan: 1 - Veraison: 3 - Visual Studio Code: 5 - vLLM: 3 +- VS Code: 1 - vvenc: 1 - Web Server: 1 - Whisper: 1 @@ -247,14 +271,15 @@ tools_software_languages_filter: - WordPress: 3 - wrk2: 2 - x265: 1 +- YAML: 1 - YCSB: 1 - Yocto/BitBake: 1 - zlib: 1 - ZooKeeper: 1 weight: 1 cloud_service_providers_filter: -- AWS: 19 -- Google Cloud: 38 -- Microsoft Azure: 19 -- Oracle: 2 +- AWS: 89 +- Google Cloud: 104 +- Microsoft Azure: 87 +- Oracle: 61 --- diff --git a/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/3-simd-migration.md b/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/3-simd-migration.md index 090f933d6f..5c469ae466 100644 --- a/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/3-simd-migration.md +++ b/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/3-simd-migration.md @@ -211,7 +211,7 @@ Pitfalls to avoid: * Don't confuse a software version with a language wrapper package version. For example, when checking the Python Redis client, check the Python package name "redis" rather than the Redis server version. Setting the Python Redis package version to the Redis server version in requirements.txt will fail. * NEON lane indices must be compile-time constants, not variables. * If you're unsure about Arm equivalents, use knowledge_base_search to find documentation. -* Be sure to find out from the user or system what the target machine is, and use the appropriate intrinsics. For instance, if neoverse (Graviton, Axion, Cobalt) is targeted, use the latest SME/SME2. +* Be sure to find out from the user or system what the target machine is, and use the appropriate intrinsics. For instance, if neoverse (Graviton, Axion, Cobalt) is targeted, use the latest SVE2 (or SVE for older neoverse). If you have good versions to update for the Dockerfile, requirements.txt, and other files, change them immediately without asking for confirmation. diff --git a/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/4-agentic-systems.md b/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/4-agentic-systems.md index 2bc1b8e628..a9884e7760 100644 --- a/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/4-agentic-systems.md +++ b/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/4-agentic-systems.md @@ -43,7 +43,7 @@ Pitfalls to avoid: * Don't confuse a software version with a language wrapper package version. For example, when checking the Python Redis client, check the Python package name "redis" rather than the Redis server version. * NEON lane indices must be compile-time constants, not variables. * If you're unsure about Arm equivalents, use knowledge_base_search to find documentation. -* Be sure to find out from the user or system what the target machine is, and use the appropriate intrinsics. For instance, if neoverse (Graviton, Axion, Cobalt) is targeted, use the latest SME/SME2. +* Be sure to find out from the user or system what the target machine is, and use the appropriate intrinsics. For instance, if neoverse (Graviton, Axion, Cobalt) is targeted, use the latest SVE2 (or SVE for older neoverse). If you have good versions to update for the Dockerfile, requirements.txt, and other files, change them immediately without asking for confirmation. @@ -81,7 +81,7 @@ Pitfalls to avoid: * Don't confuse a software version with a language wrapper package version. For example, when checking the Python Redis client, check the Python package name "redis" rather than the Redis server version. * NEON lane indices must be compile-time constants, not variables. * If you're unsure about Arm equivalents, use knowledge_base_search to find documentation. -* Be sure to find out from the user or system what the target machine is, and use the appropriate intrinsics. For instance, if neoverse (Graviton, Axion, Cobalt) is targeted, use the latest SME/SME2. +* Be sure to find out from the user or system what the target machine is, and use the appropriate intrinsics. For instance, if neoverse (Graviton, Axion, Cobalt) is targeted, use the latest SVE2 (or SVE for older neoverse). If you have good versions to update for the Dockerfile, requirements.txt, and other files, change them immediately without asking for confirmation. diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/_index.md b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/_index.md new file mode 100644 index 0000000000..2187c9270e --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/_index.md @@ -0,0 +1,49 @@ +--- +title: Find Code Hotspots with Arm Performix + +draft: true +cascade: + draft: true + +minutes_to_complete: 30 + +who_is_this_for: Software developers and performance engineers who want to identify code hotspots in applications running on Arm Linux systems. + +learning_objectives: + - Run the Code Hotspots recipe in Arm Performix + - Identify which functions consume the most CPU cycles and target them for optimization + +prerequisites: + - Access to Arm Performix + - Basic understanding of C++ + +author: Kieran Hejmadi + +### Tags +skilllevels: Introductory +subjects: Performance and Architecture +armips: + - Neoverse +tools_software_languages: + - Arm Performix + - C++ + - Runbook +operatingsystems: + - Linux + + +further_reading: + - resource: + title: Flame Graphs + link: https://www.brendangregg.com/flamegraphs.html + type: blog + + + + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 # _index.md always has weight of 1 to order correctly +layout: "learningpathall" # All files under learning paths have this same wrapper +learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content. +--- diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/_next-steps.md b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/_next-steps.md new file mode 100644 index 0000000000..727b395ddd --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/_next-steps.md @@ -0,0 +1,8 @@ +--- +# ================================================================================ +# FIXED, DO NOT MODIFY THIS FILE +# ================================================================================ +weight: 21 # The weight controls the order of the pages. _index.md always has weight 1. +title: "Next Steps" # Always the same, html page title. +layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing. +--- diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/code-hotspots-config.jpg b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/code-hotspots-config.jpg new file mode 100644 index 0000000000..ce519e0552 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/code-hotspots-config.jpg differ diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/comparison-time.jpg b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/comparison-time.jpg new file mode 100644 index 0000000000..1bff267336 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/comparison-time.jpg differ diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/comparison.jpg b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/comparison.jpg new file mode 100644 index 0000000000..2788aabb90 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/comparison.jpg differ diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/flame-graph-comparison.jpg b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/flame-graph-comparison.jpg new file mode 100644 index 0000000000..7c145158eb Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/flame-graph-comparison.jpg differ diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/flame-graph-example.jpg b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/flame-graph-example.jpg new file mode 100644 index 0000000000..00ce3856e7 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/flame-graph-example.jpg differ diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/green_mandelbrot-1-thread.jpg b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/green_mandelbrot-1-thread.jpg new file mode 100644 index 0000000000..60484a1718 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/green_mandelbrot-1-thread.jpg differ diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/how-to-1.md b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/how-to-1.md new file mode 100644 index 0000000000..5c32687c14 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/how-to-1.md @@ -0,0 +1,29 @@ +--- +title: Background Information +weight: 2 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Overview + +A flame graph is a visualization built from many sampled call stacks that shows where a program spends CPU time. In a performance investigation it is often the first thing to generate when it is not yet clear which parts of the codebase are affecting execution time. Instead of guessing where the bottleneck is, you take a quick sample of real execution and use the resulting graph to identify the hottest code paths that deserve deeper analysis. + +## Example Flame Graph + +The flame graph below shows a typical profiling result. + +![A flame graph showing function call stacks. The x-axis shows relative sample frequency and the y-axis shows call stack depth, with the widest blocks representing functions that consumed the most CPU time#center](./flame-graph-example.jpg "Example flame graph") + +The x-axis represents the relative number of samples attributed to code paths, ordered alphabetically, not a timeline. A wider block means that function appeared in more samples and therefore consumed more CPU time. The y-axis represents call stack depth. Frames at the bottom are closer to the root of execution, such as a thread entry point, and frames above them are functions called by those below. + +Each sample captures a snapshot of the current call stack. Many samples are then aggregated by grouping identical stacks and summing their counts, which is what makes flame graphs compact and readable. A common workflow is to start with the widest blocks, then move upward through the stack to understand which callees dominate that hot path. Reliable stack walking depends on frame pointers being present; they allow the profiler to unwind through nested calls and produce accurate stacks that merge cleanly into stable blocks. + +This Learning Path does not cover flame graphs in depth. To learn more, see [Brendan Gregg's flame graph reference](https://www.brendangregg.com/flamegraphs.html). + +## Tooling options + +On Linux, flame graphs are commonly generated from samples collected with `perf`. perf periodically interrupts the running program and records a stack trace, then the collected stacks are converted into a folded format and rendered as the graph. Sampling frequency is important. If the frequency is too low you may miss short-lived hotspots, and if it is too high you may introduce overhead or skew the results. To make the output informative, compile with debug symbols and preserve frame pointers so stacks resolve to meaningful function names and unwind reliably. A typical build uses `-g` and `-fno-omit-frame-pointer`. + +Arm has built a tool, Arm Performix that simplifies this workflow through the Code Hotspots recipe, making it easier to configure collection, run captures, and explore the resulting call hierarchies without manually stitching together the individual steps. This is the tooling solution you will use in this Learning Path. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/how-to-2.md b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/how-to-2.md new file mode 100644 index 0000000000..c8ccfa0a38 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/how-to-2.md @@ -0,0 +1,46 @@ +--- +title: Setup +weight: 3 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Setup + +This Learning Path uses a hands-on worked example to make sampling-based profiling and flame graphs practical. You’ll build a C++11 program that generates a fractal bitmap by computing the Mandelbrot set, then mapping each pixel’s iteration count to a pixel value. You’ll have the full source code, so you can rebuild the program, profile it, and connect what you see in the flame graph back to the exact functions and loops responsible for the runtime. + +A fractal is a pattern that shows detail at many scales, often with self-similar structure. Fractals are usually generated by repeatedly applying a simple mathematical rule. In the Mandelbrot set, each pixel corresponds to a complex number, which is iterated through a basic recurrence. How quickly the value “escapes” (or whether it stays bounded) determines the pixel’s color and produces the familiar Mandelbrot image. + +You don't need to understand the Mandelbrot algorithm in detail to follow this Learning Path — it's used here as a convenient, compute-heavy workload for profiling. To learn more, see the [Mandelbrot set article on Wikipedia](https://en.wikipedia.org/wiki/Mandelbrot_set). + + +## Connect to Target + +See the [Arm Performix installation guide](https://learn.arm.com/install-guides/atp) if this is your first time setting up Arm Performix. In this Learning Path you will connect to an AWS Graviton3 metal instance (`m7g.metal`) with 64 Neoverse V1 cores, your remote target server. From the host machine, test the connection to the remote server by navigating to **Targets** > **Test Connection**. You should see the successful connection screen below. + +![The Arm Performix Targets panel showing a successful connection test result for a remote Arm server#center](./successful-connection.jpg "Successful connection to remote target") + +## Build Application on Remote Server + +Connect to the remote server using SSH or Visual Studio Code. Install git and the C++ compiler. On dnf-based systems such as Amazon Linux 2023 or RHEL, run: + +```bash +sudo dnf update && sudo dnf install git gcc g++ make +``` + +Clone the Mandelbrot repository, check out the single-threaded branch, and create the output directories. The repository is available under the [Arm Education License](https://github.com/arm-university/Mandelbrot-Example?tab=License-1-ov-file) for teaching and learning. + +```bash +git clone https://github.com/arm-education/Mandelbrot-Example.git +cd Mandelbrot-Example +mkdir -p images build +``` + +Build the application: + +```bash +make single_thread DEBUG=1 +``` + +This creates the binary `./builds/mandelbrot_single_thread_debug`. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/how-to-3.md b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/how-to-3.md new file mode 100644 index 0000000000..db47105a7e --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/how-to-3.md @@ -0,0 +1,68 @@ +--- +title: Assess Baseline Performance +weight: 4 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Run Code Hotspots Recipe + +As shown in the `src/main_single_thread.cpp` file below, the program generates a 1920×1080 bitmap image of the fractal. To identify performance bottlenecks, run the Code Hotspots recipe in Arm Performix (APX). APX uses sampling to estimate where the CPU spends most of its time, allowing it to highlight the hottest functions—especially useful in larger applications where it isn't obvious ahead of time which functions will dominate runtime. + +{{% notice Note %}} +The `myplot.draw()` call uses a relative path (`./images/green.bmp`). When APX launches the binary, it runs it from a temporary location, so the image would be written there rather than to your project directory. To ensure the output is saved where you expect it, update the first string argument in `src/main_single_thread.cpp` to the absolute path of the output file, for example `/home/ec2-user/Mandelbrot-Example/images/green.bmp`. +{{% /notice %}} + +```cpp +#include "Mandelbrot.h" +#include + +using namespace std; + +int main(int argc, char* argv[]){ + + const int NUM_THREADS = 1; + std::cout << "Number of Threads = " << NUM_THREADS << std::endl; + + Mandelbrot::Mandelbrot myplot(1920, 1080, NUM_THREADS); + myplot.draw("/home/ec2-user/Mandelbrot-Example/images/green.bmp", Mandelbrot::Mandelbrot::GREEN); + + return 0; +} +``` + + + Rebuild the application before continuing: + + ```bash +make clean +make single_thread DEBUG=1 + ``` + +Open APX from the host machine. Select the **Code Hotspot** recipe. If this is the first time running the recipe on this target machine you may need to select the install tools button. + +![The Arm Performix recipe selection screen with the Code Hotspots recipe highlighted#center](./install-tools.jpg "Selecting the Code Hotspots recipe") + +Configure the recipe to launch a new process. APX will automatically start collecting metrics when the program starts and stop when the program exits. + +Provide the absolute path to the binary built in the previous step: `/home/ec2-user/Mandelbrot-Example/build/mandelbrot_single_thread_debug`. + +Use the default sampling rate of **Normal**. If your application is short-running, consider a higher sample rate, at the cost of more data to store and process. + +![The Arm Performix Code Hotspots recipe configuration screen showing launch settings, binary path, and sampling rate fields#center](./code-hotspots-config.jpg "Code Hotspots recipe configuration") + +## Analyse Results + +A flame graph is generated once the run completes. The default colour mode labels the hottest functions—those using CPU most frequently—in the darkest shade. In this example, the `__complex_abs__` function is present in approximately 65% of samples, and it calls the `__hypot` symbol in `libm.so`. + +![A flame graph showing single-threaded Mandelbrot profiling results with __complex_abs__ as the dominant hotspot#center](./single-thread-flame-graph.jpg "Single-threaded flame graph showing __complex_abs__ as the hottest function") + +To investigate further, you can map source code lines to the functions in the flame graph. Right-click on a specific function and select **View Source Code**. You may need to copy the source code onto your host machine to use this feature. + +![The Arm Performix flame graph view showing source code annotations mapped to the selected hot function#center](./view-with-source-code.jpg "Flame graph with source code view") + +Finally, check your `images` directory for the generated bitmap fractal `green.bmp` + +![A rendered Mandelbrot set fractal in green, generated from the single-threaded build at maximum iterations#center](./plot-1-thread-max-iterations.jpg "Mandelbrot fractal output from single-threaded build") + diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/how-to-4.md b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/how-to-4.md new file mode 100644 index 0000000000..c1395a135d --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/how-to-4.md @@ -0,0 +1,94 @@ +--- +title: Optimize +weight: 5 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Optimize hot functions + +Use the insights from Arm Performix to focus optimizations on the hottest functions. The flame graph from the previous step shows that `__hypot` is invoked by `Mandelbrot::getIterations` to calculate the absolute value of a complex number. One option worth exploring is replacing the `libm` implementation with an optimized alternative, but first consider the more targeted changes below. + +Looking at the `Mandelbrot::getIterations` function, there are two clear optimization opportunities. + +```cpp + while (iterations < MAX_ITERATIONS){ + z = (z*z) + c; + if (abs(z) > THRESHOLD){ + break; + } + iterations++; + } +``` + +### Optimization 1 - Limiting loop boundary + +The iteration count is bounded by `MAX_ITERATIONS`, defined as 1024, a `static const` integer in the `Mandelbrot.h` header. Halving this to 512 reduces the maximum work per pixel but you will need to verify that the change in image quality is acceptable. + +```cpp +public: + ... + static const int MAX_ITERATIONS = (1<<10); + ... +``` + +On the remote server, reduce `MAX_ITERATIONS` in `Mandelbrot.h` to `(1<<9)` (512), update the output image filename in `main.cpp` to a different name (for example: `green-512.bmp`)so you can compare the output with the baseline image, then rebuild: + +```bash +make clean +make single_thread DEBUG=1 +``` + +Select the refresh icon in the top right to rerun the recipe, then switch to comparison mode to view differences between runs. Under the **Run Details** tab, the run duration drops from 1m 0s to 0m 32s — almost proportional to the reduction in `MAX_ITERATIONS`. The trade-off to verify is whether the image quality is still acceptable. + +There is negligible difference in perceived image quality when halving `MAX_ITERATIONS`. + +![Side-by-side comparison of Mandelbrot fractal output at MAX_ITERATIONS 1024 and 512, showing no visible quality difference#center](./comparison.jpg "Image quality comparison: 1024 vs 512 iterations") + +### Optimization 2 - Parallelising the hot function + +The loop in `Mandelbrot::getIterations` has no loop-carried dependencies — each iteration's result is independent of any other. This means you can parallelize the hot function across multiple threads if your CPU has multiple cores. + +The repository contains a parallel build target. + + +The `src/mandelbrot_parallel.cpp` parallelizes the `Mandelbrot::draw` function, which is an earlier function in the call stack that eventually calls `__hypot`. Before building, update the `myplot.draw()` call in `main.cpp` to use an absolute output path: + +```cpp +myplot.draw("/home/ec2-user/Mandelbrot-Example/images/Green-Parallel-512.bmp", Mandelbrot::Mandelbrot::GREEN); +``` + +Build the example. This creates a binary `./builds/mandelbrot-parallel` that takes a single numeric command-line argument to set the number of threads. + +```bash +make clean +make parallel DEBUG=1 +``` + +Update the binary path in APX to `./builds/mandelbrot_parallel_debug` and pass the desired thread count as an argument, then rerun the recipe from the host. + +To compare with a previous run, switch to comparison mode. Under the **Run Details** tab, execution time drops further from 0m 32s to 7s with 32 threads. + +![The Arm Performix Run Details tab comparing execution time between the baseline and parallelized builds, showing a reduction from 32s to 7s#center](./comparison-time.jpg "Execution time comparison: single-threaded vs parallelized build") + +The proportion of samples has not changed significantly overall, but with 64 threads the percentage of samples landing on `Mandelbrot::draw` has reduced by 7%. To further improve execution time, you can continue optimizing `Mandelbrot::draw`. + +![Side-by-side flame graph comparison between single-threaded and parallel Mandelbrot builds, showing reduced dominance of the Mandelbrot::draw function#center](./flame-graph-comparison.jpg "Flame graph comparison: single-threaded vs parallelized build") + +{{% notice Note %}} +The total run duration shown in APX includes tooling setup and data analysis time, not just application execution time. To measure only the application, use the `time` command: the application now runs in approximately 1 second — close to a 100x improvement over the original single-threaded baseline. +{{% /notice %}} + +### (Optional Challenge) Additional optimizations + +The `Makefile` uses the `-O0` flag when the `DEBUG=1` argument is passed in. This disables all compiler optimizations. Try experimenting with higher optimization levels, different loop boundary sizes, and thread counts. See the Learning Path [Get started with compiler optimization flags](/learning-paths/servers-and-cloud-computing/cplusplus_compilers_flags/) for guidance. You may also want to explore vectorized math libraries that could replace the `libm` hypotenuse function, such as the [Arm Performance Libraries vector math functions](https://developer.arm.com/documentation/101004/2601/Arm-Performance-Libraries-Math-Functions/Arm-Performance-Libraries-Vector-Math-Functions--Accuracy-Table). + + +## Summary + +In this Learning Path, you reduced the runtime of the Mandelbrot example by focusing on the hottest code paths—cutting execution time from around 1 minute to ~1 second through targeted optimization and parallelization. While this example is relatively simple and the optimizations are more obvious, the same principle applies to real-world workloads: optimize what matters most first, based on measurement. + +The Code Hotspots recipe is designed to quickly identify an application's most CPU-time-dominant functions, giving you a clear, evidence-based starting point for performance work. By surfacing where execution time is actually spent, it ensures your optimizations target the parts of the code most likely to deliver the largest gains. + +This is often one of the first profiling steps to run when assessing an application's performance — especially to determine which functions dominate runtime and should be prioritized. Once hotspots are identified, you can follow up with deeper function-specific analysis, such as memory investigations or top-down studies, and build microbenchmarks around hot functions to explore lower-level bottlenecks and uncover additional optimization opportunities. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/install-tools.jpg b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/install-tools.jpg new file mode 100644 index 0000000000..421f6cce2f Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/install-tools.jpg differ diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/plot-1-thead-512-iterations.jpg b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/plot-1-thead-512-iterations.jpg new file mode 100644 index 0000000000..3787e8c4a0 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/plot-1-thead-512-iterations.jpg differ diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/plot-1-thread-max-iterations.jpg b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/plot-1-thread-max-iterations.jpg new file mode 100644 index 0000000000..fd9e98d5da Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/plot-1-thread-max-iterations.jpg differ diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/runtime-difference.jpg b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/runtime-difference.jpg new file mode 100644 index 0000000000..8a2529c62b Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/runtime-difference.jpg differ diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/single-thread-flame-graph.jpg b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/single-thread-flame-graph.jpg new file mode 100644 index 0000000000..e4ae59cf38 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/single-thread-flame-graph.jpg differ diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/successful-connection.jpg b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/successful-connection.jpg new file mode 100644 index 0000000000..311417a01e Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/successful-connection.jpg differ diff --git a/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/view-with-source-code.jpg b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/view-with-source-code.jpg new file mode 100644 index 0000000000..dccc52d0da Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/view-with-source-code.jpg differ diff --git a/content/learning-paths/servers-and-cloud-computing/docker-mcp-toolkit/4-run-migration.md b/content/learning-paths/servers-and-cloud-computing/docker-mcp-toolkit/4-run-migration.md index dd81516b7e..a86012de99 100644 --- a/content/learning-paths/servers-and-cloud-computing/docker-mcp-toolkit/4-run-migration.md +++ b/content/learning-paths/servers-and-cloud-computing/docker-mcp-toolkit/4-run-migration.md @@ -43,7 +43,7 @@ Important notes: - If unsure about Arm equivalents, use knowledge_base_search to find docs - Be sure to find out from the user or system what the target machine is, and use the appropriate intrinsics. For instance, if neoverse (Graviton, - Axion, Cobalt) is targeted, use the latest SME/SME2. + Axion, Cobalt) is targeted, use the latest SVE2 (or SVE for older neoverse). After completing the migration: - Create a pull request with a detailed description of changes diff --git a/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/_index.md b/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/_index.md index 6fcda9c8f7..1315e39b94 100644 --- a/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/_index.md @@ -1,13 +1,9 @@ --- title: Monitor Azure Cobalt 100 Arm64 virtual machines using Dynatrace OneAgent - -draft: true -cascade: - draft: true minutes_to_complete: 30 -who_is_this_for: This learning path is designed for developers, DevOps engineers, and platform engineers who want to implement infrastructure and application monitoring using Dynatrace on Arm-based cloud environments. +who_is_this_for: This is an introductory topic for developers, DevOps engineers, and platform engineers who want to implement infrastructure and application monitoring using Dynatrace on Arm-based cloud environments. learning_objectives: - Deploy Dynatrace OneAgent on Azure Cobalt 100 Arm64 virtual machines diff --git a/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/activegate-installation.md b/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/activegate-installation.md index 02be2776cf..823e6f80bd 100644 --- a/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/activegate-installation.md +++ b/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/activegate-installation.md @@ -13,12 +13,12 @@ It improves scalability, enables Kubernetes monitoring, and routes monitoring tr In this guide, you'll install Dynatrace ActiveGate on an **Azure Ubuntu 24.04 LTS Arm64 virtual machine running on Azure Cobalt 100 processors**. -At the end of the installation, ActiveGate will be: +After installation completes, ActiveGate: -* Installed and running as a system service -* Listening on port **9999** for Dynatrace communication -* Connected to your Dynatrace SaaS environment -* Verified on **Arm64 (Aarch64)** architecture +* Runs as a system service +* Listens on port **9999** for Dynatrace communication +* Connects to your Dynatrace SaaS environment +* Operates natively on **Arm64 (`aarch64`)** architecture ## Verify OneAgent installation @@ -31,7 +31,7 @@ Verify that OneAgent is running: sudo systemctl status oneagent ``` -If OneAgent is not installed, install it first using the previous guide. +If OneAgent isn't installed, install it first using the previous guide. ## Log in to Dynatrace @@ -51,24 +51,26 @@ This is your Dynatrace SaaS environment URL. ## Navigate to the ActiveGate deployment page -From the Dynatrace dashboard: +From the Dynatrace main dashboard: -- Select Deployment status -- Choose ActiveGate -- Install ActiveGate +* Select **Search** on the upper left and search for **Deployment**. +* Select **Deployment status**. +* Choose **ActiveGate**. +* Select **Install ActiveGate**. ![Dynatrace deployment status page showing no ActiveGate detected alt-txt#center](images/activegate1.png "Dynatrace ActiveGate not yet installed") -Dynatrace will generate an installation command specifically for your environment. +Dynatrace generates an installation command specifically for your environment. -## Select ARM architecture +## Select Arm architecture On the installer configuration page: -- Platform → Linux -- Architecture → ARM64 +* Platform -> Linux +* Architecture -> ARM64 +* Select **Generate token** to create an authentication token -![Dynatrace Install ActiveGate page showing deployment type Linux and architecture ARM selection alt-txt#center](images/activegate-install.png "Dynatrace ActiveGate installation configuration") +![Dynatrace Install ActiveGate page showing Linux platform and ARM64 architecture selection alt-txt#center](images/activegate-install.png "Dynatrace ActiveGate installation configuration") This ensures that the installer downloads the Arm64-compatible ActiveGate binaries. @@ -95,24 +97,29 @@ wget -O Dynatrace-ActiveGate-Linux-arm.sh \ **Verify signature:** ```console -wget https://ca.dynatrace.com/dt-root.cert.pem ; ( echo 'Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg="sha-256"; boundary="--SIGNED-INSTALLER"'; echo ; echo ; echo '----SIGNED-INSTALLER' ; cat Dynatrace-ActiveGate-Linux-arm64-1.331.24.20260210-044521.sh ) | openssl cms -verify -CAfile dt-root.cert.pem > /dev/null +wget https://ca.dynatrace.com/dt-root.cert.pem +( echo 'Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg="sha-256"; boundary="--SIGNED-INSTALLER"'; echo; echo; echo '----SIGNED-INSTALLER'; cat Dynatrace-ActiveGate-Linux-arm.sh ) | openssl cms -verify -CAfile dt-root.cert.pem > /dev/null ``` -![Dynatrace ActiveGate installer command for Linux ARM64 environment alt-txt#center](images/activegate-installation-commands.png "Dynatrace ActiveGate installer command") + +![Dynatrace showing ActiveGate installer command for Linux ARM64 alt-txt#center](images/activegate-installation-commands.png "Dynatrace ActiveGate installer command") **Install ActiveGate as the privileged user:** +Copy the command under "Install ActiveGate as the privileged user" from your dashboard and prepend "sudo " to it to launch the install: + ```console -sudo /bin/bash Dynatrace-ActiveGate-Linux-arm64-1.331.24.20260210-044521.sh +sudo /bin/bash Dynatrace-ActiveGate-Linux-arm.sh ``` The installer automatically performs the following tasks: -- Downloads ActiveGate components -- Installs the ActiveGate service -- Configures communication with Dynatrace SaaS -- Starts the ActiveGate service +* Downloads ActiveGate components +* Installs the ActiveGate service +* Configures communication with Dynatrace SaaS +* Starts the ActiveGate service + +The output is similar to: -The output is similar to: ```output 2026-03-12 05:59:21 UTC Starting Dynatrace ActiveGate AutoUpdater... 2026-03-12 05:59:21 UTC Checking if Dynatrace ActiveGate AutoUpdater is running ... @@ -133,12 +140,14 @@ sudo systemctl status dynatracegateway ``` The output is similar to: + ```output ● dynatracegateway.service - Dynatrace ActiveGate service Loaded: loaded (/etc/systemd/system/dynatracegateway.service; enabled; preset: enabled) Active: active (running) since Thu 2026-03-12 05:59:07 UTC; 1min 7s ago Process: 20280 ExecStart=/opt/dynatrace/gateway/dynatracegateway start (code=exited, status=0/SUCCESS) ``` + This confirms that ActiveGate started successfully. ## Verify the ActiveGate communication port @@ -152,6 +161,7 @@ sudo ss -tulnp | grep 9999 ``` The output is similar to: + ```console tcp LISTEN 0 50 *:9999 *:* users:(("java",pid=20319,fd=403)) ``` @@ -168,17 +178,17 @@ Deployment Status → ActiveGates You should see your ActiveGate instance listed with: -- Host name -- Version -- Status: Connected +* Host name +* Version +* Status: Connected -![Dynatrace Deployment Status page showing ActiveGate instance connected and running alt-txt#center](images/activegate-ui.png "Dynatrace ActiveGate deployment status") +![Dynatrace Deployment Status page showing ActiveGate connected and running alt-txt#center](images/activegate-ui.png "Dynatrace ActiveGate deployment status") -![Dynatrace ActiveGate details page displaying modules architecture and configuration alt-txt#center](images/activegate-details.png "Dynatrace ActiveGate details view") +![Dynatrace ActiveGate details page showing modules and configuration alt-txt#center](images/activegate-details.png "Dynatrace ActiveGate details view") ## Test application monitoring with Nginx -To validate that Dynatrace is collecting monitoring data correctly, deploy a simple web server on the virtual machine. Dynatrace OneAgent will automatically detect and monitor the process. +To validate that Dynatrace is collecting monitoring data correctly, deploy a simple web server on the virtual machine. Dynatrace OneAgent automatically detects and monitors the process. ### Install Nginx @@ -188,7 +198,8 @@ Update the package index and install the Nginx web server. sudo apt update sudo apt install -y nginx ``` -## Check the Nginx service status. + +## Check the Nginx service status ```console sudo systemctl status nginx @@ -218,20 +229,17 @@ Processes You should see a process similar to: -- nginx -- Dynatrace automatically begins collecting metrics such as: -- CPU usage -- memory consumption -- network activity -- request throughput +* `nginx` + +Dynatrace automatically begins collecting metrics such as: -![Dynatrace Infrastructure Explorer showing NGINX process monitoring on an ARM64 host alt-txt#center](images/nginx-monitoring.png "Dynatrace NGINX process monitoring dashboard") +* CPU usage +* memory consumption +* network activity +* request throughput -## What you've accomplished +![Dynatrace Infrastructure Explorer showing NGINX process monitoring on ARM64 host alt-txt#center](images/nginx-monitoring.png "Dynatrace NGINX process monitoring dashboard") -You've successfully installed Dynatrace ActiveGate on your Azure Ubuntu Arm64 virtual machine. Your environment now includes: +## What you've learned -- Dynatrace OneAgent performing host monitoring -- ActiveGate routing monitors traffic securely -- Communication with Dynatrace SaaS through port 9999 -- Full compatibility with Arm64-based Cobalt 100 processors +Your Azure Cobalt 100 virtual machine now has a complete Dynatrace monitoring stack. OneAgent monitors host resources and processes, while ActiveGate securely routes monitoring data to your Dynatrace environment through port 9999. The entire setup operates natively on Arm64 architecture, providing full observability for your applications and infrastructure. diff --git a/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/background.md b/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/background.md index 42f64494e5..03f7d7fe9d 100644 --- a/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/background.md +++ b/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/background.md @@ -20,8 +20,10 @@ Dynatrace automatically maps dependencies between services, hosts, containers, a There are three main components of Dynatrace: -- **Dynatrace OneAgent:** a lightweight monitoring agent installed on hosts that automatically collects metrics, logs, and traces from applications and infrastructure. Learn more in the [Dynatrace OneAgent documentation](https://docs.dynatrace.com/docs/ingest-from/dynatrace-oneagent). +- **Dynatrace OneAgent:** a lightweight monitoring agent installed on hosts that automatically collects metrics, logs, and traces from applications and infrastructure. Learn more in the [Dynatrace OneAgent documentation](https://docs.dynatrace.com/docs/ingest-from/dynatrace-oneagent). - **Dynatrace ActiveGate:** a secure gateway component that routes monitoring traffic, enables cloud integrations, and provides additional monitoring capabilities such as Kubernetes monitoring and synthetic monitoring. Learn more in the [Dynatrace ActiveGate documentation](https://docs.dynatrace.com/docs/ingest-from/dynatrace-activegate). - **Dynatrace Platform (SaaS or Managed):** the central observability platform that processes monitoring data, provides dashboards, AI-driven root cause analysis, and system-wide visibility across applications and infrastructure. See the [Dynatrace documentation portal](https://docs.dynatrace.com/docs) for more details. + +In this Learning Path, you'll set up both OneAgent and ActiveGate on Azure Cobalt 100 Arm64 virtual machines to establish full-stack monitoring for your applications and infrastructure. diff --git a/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/firewall.md b/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/firewall.md index 9d4b90aa85..3e26fcabf3 100644 --- a/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/firewall.md +++ b/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/firewall.md @@ -10,7 +10,7 @@ layout: learningpathall To allow external traffic on port **9999** for Dynatrace ActiveGate running on an Azure virtual machine, open the port in the Network Security Group (NSG) attached to the virtual machine's network interface or subnet. -{{% notice Note %}} For more information about Azure setup, see [Getting started with Microsoft Azure Platform](/learning-paths/servers-and-cloud-computing/csp/azure/).{{% /notice %}} +{{% notice Note %}}For more information about Azure setup, see [Getting started with Microsoft Azure Platform](/learning-paths/servers-and-cloud-computing/csp/azure/).{{% /notice %}} ## Create a firewall rule in Azure @@ -18,17 +18,17 @@ To expose the TCP port **9999**, create a firewall rule. Navigate to the [Azure Portal](https://portal.azure.com), go to **Virtual Machines**, and select your virtual machine. -![Azure portal Virtual Machines page showing list of running VMs with details like status, location, and size alt-txt#center](images/virtual_machine.png "Virtual Machines") +![Azure Portal showing Virtual Machines list alt-txt#center](images/virtual_machine.png "Virtual Machines") -In the left menu, select **Networking** and in the **Networking** select **Network settings** that is associated with the virtual machine's network interface. +In the left menu, select **Networking** and in the **Networking** select **Network settings** that's associated with the virtual machine's network interface. -![Azure portal Network settings page showing network interface and security group configuration alt-txt#center](images/networking.png "Network settings") +![Azure Portal showing Network settings with security group configuration alt-txt#center](images/networking.png "Network settings") Navigate to **Create port rule**, and select **Inbound port rule**. -![Azure portal showing Create port rule dropdown with Inbound port rule option highlighted alt-txt#center](images/port_rule.png "Create rule") +![Azure Portal showing Create port rule dropdown menu alt-txt#center](images/port_rule.png "Create rule") -Configure it using the following details: +Configure the inbound security rule with the following settings: - **Source:** Any - **Source port ranges:** * @@ -36,11 +36,16 @@ Configure it using the following details: - **Destination port ranges:** **9999** - **Protocol:** TCP - **Action:** Allow -- **Priority:** 1000 -- **Name:** dynatrace-activegate +- **Name:** allow-tcp-9999 After filling in the details, select **Add** to save the rule. -![Azure portal inbound security rule configuration form showing port 9999, TCP protocol, and Allow action alt-txt#center](images/inbound_rule.png "Network settings") +![Azure Portal showing inbound security rule form with port 9999 configuration alt-txt#center](images/inbound_rule.png "Network settings") The network firewall rule is now created, allowing Dynatrace ActiveGate to communicate over port **9999**. + +## What you've learned and what's next + +You've configured the Azure Network Security Group to allow incoming traffic on port 9999. This firewall rule enables secure communication between your virtual machine and the Dynatrace platform when you install ActiveGate. + +Next, you'll install Dynatrace OneAgent on your Azure Cobalt 100 virtual machine to begin monitoring system resources and processes. diff --git a/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/instance.md b/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/instance.md index 1da75b2837..4b5e36880f 100644 --- a/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/instance.md @@ -14,7 +14,7 @@ There are several common ways to create an Arm-based Cobalt 100 virtual machine, - The Azure CLI - An infrastructure as code (IaC) tool -In this section, you will launch the Azure Portal to create a virtual machine with the Arm-based Azure Cobalt 100 processor. +In this section, you'll launch the Azure Portal to create a virtual machine with the Arm-based Azure Cobalt 100 processor. This Learning Path focuses on general-purpose virtual machines in the Dpsv6 series. For more information, see the [Microsoft Azure guide for the Dpsv6 size series](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/general-purpose/dpsv6-series). @@ -22,7 +22,7 @@ While the steps to create this instance are included here for convenience, you c ## Create an Arm-based Azure virtual machine -Creating a virtual machine based on Azure Cobalt 100 is no different to creating any other virtual machine in Azure. To create an Azure virtual machine: +Creating a virtual machine based on Azure Cobalt 100 is no different from creating any other virtual machine in Azure. To create an Azure virtual machine: - Launch the Azure portal and navigate to **Virtual Machines**. - Select **Create**, and select **Virtual Machine** from the drop-down list. @@ -31,7 +31,7 @@ Creating a virtual machine based on Azure Cobalt 100 is no different to creating - In the **Size** field, select **See all sizes** and select the D-Series v6 family of virtual machines. - Select **D4ps_v6** from the list as shown in the diagram below: -![Azure Portal VM size selection showing the D-Series v6 (Dpsv6) family highlighted with D4ps_v6 option selected for Arm64 architecture alt-txt#center](images/instance.png "Select D4ps_v6 from the D-Series v6 family") +![Azure Portal showing D-Series v6 VM size selection with D4ps_v6 highlighted alt-txt#center](images/instance.png "Select D4ps_v6 from the D-Series v6 family") - For **Authentication type**, select **SSH public key**. @@ -49,28 +49,24 @@ RSA offers better security with keys longer than 3072 bits. - Give your SSH key a key pair name. - In the **Inbound port rules**, select **HTTP (80)** and **SSH (22)** as the inbound ports, as shown below: -![Azure Portal inbound port rules configuration showing HTTP (80) and SSH (22) selected as allowed incoming traffic alt-txt#center](images/instance1.png "Configure inbound port rules for HTTP and SSH access") +![Azure Portal showing inbound port rules with HTTP (80) and SSH (22) selected alt-txt#center](images/instance1.png "Configure inbound port rules for HTTP and SSH access") - Now select the **Review + Create** tab and review the configuration for your virtual machine. It should look like the following: -![Azure Portal Review + Create tab showing VM configuration summary with Ubuntu Pro 24.04 LTS image, D4ps_v6 size, and networking settings configured alt-txt#center](images/ubuntu-pro.png "Review VM configuration before creation") +![Azure Portal Review + Create tab showing VM configuration summary ready for deployment alt-txt#center](images/ubuntu-pro.png "Review VM configuration before creation") -- When you are happy with your selection, select the **Create** button and then **Download Private key and Create Resource** button. +- When you're happy with your selection, select the **Create** button and then **Download Private key and Create Resource** button. -![Azure Portal showing the Create button and Download Private key and Create Resource button to finalize VM creation alt-txt#center](images/instance4.png "Download SSH key and create the virtual machine") +![Azure Portal showing Create button and SSH key download dialog alt-txt#center](images/instance4.png "Download SSH key and create the virtual machine") Your virtual machine should be ready and running in a few minutes. You can SSH into the virtual machine using the private key, along with the public IP details. -![Azure Portal showing successful VM deployment with green checkmark, VM name, resource group, and public IP address displayed in the confirmation notification alt-txt#center](images/final-vm.png "Successful VM deployment confirmation") +![Azure Portal showing successful VM deployment with confirmation details alt-txt#center](images/final-vm.png "Successful VM deployment confirmation") -{{% notice Note %}}To learn more about Arm-based virtual machine in Azure, see “Getting Started with Microsoft Azure” in [Get started with Arm-based cloud instances](/learning-paths/servers-and-cloud-computing/csp/azure).{{% /notice %}} +{{% notice Note %}}To learn more about Arm-based virtual machines in Azure, see "Getting Started with Microsoft Azure" in [Get started with Arm-based cloud instances](/learning-paths/servers-and-cloud-computing/csp/azure/).{{% /notice %}} -## What you've accomplished and what's next +## What you've learned and what's next -You've successfully: +You've created an Azure Cobalt 100 Arm64 virtual machine running Ubuntu Pro 24.04 LTS with SSH authentication configured. The VM is ready for Dynatrace monitoring. -- Created an Azure Cobalt 100 Arm-based virtual machine using the D-Series v6 (Dpsv6) family -- Selected Ubuntu Pro 24.04 LTS as the operating system -- Configured SSH authentication for secure access - -Your Azure Cobalt 100 Arm64 virtual machine is now ready. Next, you'll install Dynatrace OneAgent to begin monitoring the virtual machine and automatically discover running services and processes. +Next, you'll configure the Azure firewall to allow ActiveGate communication on port 9999. diff --git a/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/native-installation.md b/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/native-installation.md new file mode 100644 index 0000000000..486da25c81 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/native-installation.md @@ -0,0 +1,244 @@ +--- +title: Install Dynatrace OneAgent on Azure Ubuntu Arm64 virtual machine +weight: 5 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Install Dynatrace OneAgent on Azure Ubuntu Arm64 + +To install Dynatrace OneAgent on an Azure Ubuntu 24.04 LTS Arm64 virtual machine, follow these steps. + +After installation completes, Dynatrace OneAgent: + +* Runs as a host monitoring agent +* Connects to your Dynatrace SaaS environment +* Monitors system processes and services automatically +* Operates natively on Arm64 (`aarch64`) architecture + +## Update the system and install required tools + +Update the operating system and install the tools required for downloading the Dynatrace installer. + +```console +sudo apt update && sudo apt upgrade -y +sudo apt install -y curl wget unzip ca-certificates +``` + +## Verify Arm64 architecture + +Confirm that the virtual machine is running on the Arm64 architecture. + +```console +uname -m +``` + +The output is similar to: + +```output +aarch64 +``` + +This confirms the system is using the Arm64 architecture required for Cobalt 100 processors. + +## Create your Dynatrace trial environment + +In a web browser, go to [Dynatrace](https://dynatrace.com) and select **Try it free** followed by **Start trial**. + +Enter your email and then complete the requested fields: + +* Create and supply a password for your trial account +* First name +* Last name +* Work email address +* Company name +* Country + +After submitting the form, Dynatrace creates a new SaaS monitoring environment for you. + +This process usually takes 1–2 minutes. + +## Access your Dynatrace environment + +After the environment is created, your browser shows a button named **Launch Dynatrace**. Select it to open your Dynatrace environment. + +Make a note of the environment ID assigned to your account because it appears in your dashboard URL. + +**Example:** + +```text +https://mbp77458.apps.dynatrace.com/ui/apps/dynatrace.launcher/getting-started +``` + +The Environment ID uniquely identifies your Dynatrace tenant and is required for agent installation. + +![Dynatrace environment login page showing SaaS environment URL#center](images/dynatrace-env-id.png "Dynatrace SaaS environment login") + +## Navigate to Deployment + +From the Dynatrace dashboard: + +* Choose **OneAgent** +* Choose **Setup** + +![Dynatrace deployment page showing OneAgent setup options#center](images/oneagent1.png "Dynatrace OneAgent deployment setup page") + +This page generates the installation command tailored for your environment. + +## Select Arm64 architecture + +On the installer page, confirm these selections: + +* Platform -> Linux +* Architecture -> ARM64 +* Monitoring mode -> Full-stack monitoring + +Then select **Generate token** to create an authentication token. + +![Dynatrace installer page showing ARM64 architecture selection#center](images/oneagent-arch.png "Dynatrace OneAgent ARM64 architecture selection") + +## Copy OneAgent installer command + +Dynatrace generates an installer command that includes your environment ID and API token.: + +```console +wget -O Dynatrace-OneAgent-Linux-arm.sh \ +"https://.live.dynatrace.com/api/v1/deployment/installer/agent/unix/default/latest?arch=arm" \ +--header="Authorization: Api-Token " +``` + +**Example:** + +```console +wget -O Dynatrace-OneAgent-Linux-arm.sh \ +"https://qzo72404.live.dynatrace.com/api/v1/deployment/installer/agent/unix/default/latest?arch=arm" \ +--header="Authorization: Api-Token DT_API_TOKEN" +``` + +The API token allows secure access to the Dynatrace installer. + +Run this command on the virtual machine to download the installer. + +## Verify installer signature + +For security, verify the installer signature using Dynatrace's root certificate. + +```console +wget https://ca.dynatrace.com/dt-root.cert.pem +( echo 'Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg="sha-256"; boundary="--SIGNED-INSTALLER"'; echo; echo; echo '----SIGNED-INSTALLER'; cat Dynatrace-OneAgent-Linux-arm.sh ) | openssl cms -verify -CAfile dt-root.cert.pem > /dev/null +``` + +This command validates the downloaded installer before you run it. + +![Dynatrace UI showing generated OneAgent installer command#center](images/onagent-install.png "Dynatrace OneAgent installation command") + +## Install OneAgent as the privileged user + +Copy the command under **Install OneAgent as the privileged user** from your Dynatrace dashboard and prepend `sudo` to run the install. + +```console +sudo /bin/sh Dynatrace-OneAgent-Linux-arm.sh --set-monitoring-mode=fullstack --set-app-log-content-access=true +``` + +The output is similar to: + +```output +Starting Dynatrace OneAgent installer... +Installing OneAgent... +Setting agent configuration... +Installation finished successfully. +``` + +The installer performs several tasks automatically: + +* Downloads monitoring components +* Configures kernel instrumentation +* Installs the OneAgent system service +* Registers the host with your Dynatrace environment + +## Verify OneAgent service + +Check that the Dynatrace monitoring service is running. + +```console +sudo systemctl status oneagent +``` + +The output is similar to: + +```output +● oneagent.service - Dynatrace OneAgent + Loaded: loaded (/etc/systemd/system/oneagent.service; enabled; preset: enabled) + Active: active (running) +``` + +This confirms the monitoring agent started successfully. + +## Verify Dynatrace processes + +You can also check the OneAgent processes from the terminal. + +```console +ps aux | grep oneagent +``` + +The output is similar to: + +```output +dtuser 17754 0.0 0.0 307872 4388 ? Ssl 05:48 0:00 /opt/dynatrace/oneagent/agent/lib64/oneagentwatchdog -bg -config=/opt/dynatrace/oneagent/agent/conf/watchdog.conf +dtuser 17761 0.2 0.3 1183000 59136 ? Sl 05:48 0:06 oneagentos -Dcom.compuware.apm.WatchDogTimeout=900 -watchdog.restart_file_location=/var/lib/dynatrace/oneagent/agent/watchdog/watchdog_restart_file -Dcom.compuware.apm.WatchDogPipe=/var/lib/dynatrace/oneagent/agent/watchdog/oneagentos_pipe_17754 +dtuser 17793 0.0 0.2 689184 34408 ? Sl 05:48 0:01 oneagentloganalytics -Dcom.compuware.apm.WatchDogTimeout=900 -Dcom.compuware.apm.WatchDogPipe=/var/lib/dynatrace/oneagent/agent/watchdog/oneagentloganalytics_pipe_17754 +dtuser 17795 0.1 0.2 361936 42940 ? Sl 05:48 0:04 oneagentnetwork -Dcom.compuware.apm.WatchDogTimeout=900 -Dcom.compuware.apm.WatchDogPipe=/var/lib/dynatrace/oneagent/agent/watchdog/oneagentnetwork_pipe_17754 +dtuser 17883 0.0 0.0 28212 5340 ? Sl 05:49 0:00 /opt/dynatrace/oneagent/agent/lib64/oneagentebpfdiscovery --log-dir /var/log/dynatrace/oneagent/os/ --log-no-stdout --log-level info +azureus+ 23847 0.0 0.0 9988 2772 pts/0 S+ 06:33 0:00 grep --color=auto oneagent +``` + +This confirms the monitoring agent started successfully. + +## Confirm host detection in Dynatrace + +Return to the Dynatrace web interface. + +Navigate to: + +```text +Infrastructure & Operations +→ Hosts +``` + +You should see: + +```output +Host name: +OS: Linux +Architecture: ARM64 +Monitoring mode: Full Stack +``` + +![Dynatrace hosts dashboard showing detected ARM64 virtual machine#center](images/dynatrace-host.png "Dynatrace host monitoring dashboard") + +## Check automatic process discovery + +Dynatrace automatically discovers running applications and services. + +View them under: + +```text +Hosts → Processes +``` + +Dynatrace identifies services such as: + +* system processes +* web servers +* databases +* container runtimes + +![Dynatrace process monitoring dashboard showing automatically discovered services#center](images/dynatrace-process.png "Dynatrace process discovery view") + +## What you've learned and what's next + +Dynatrace OneAgent is now monitoring your Azure Cobalt 100 virtual machine. The agent runs as a system service, automatically discovers processes, and securely transmits monitoring data to your Dynatrace environment. + +Next, you'll install Dynatrace ActiveGate to enable secure data routing, Kubernetes monitoring, and extension support. diff --git a/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/native-intstallation.md b/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/native-intstallation.md deleted file mode 100644 index a070f5365e..0000000000 --- a/content/learning-paths/servers-and-cloud-computing/dynatrace-azure/native-intstallation.md +++ /dev/null @@ -1,247 +0,0 @@ ---- -title: Install Dynatrace OneAgent on Azure Ubuntu Arm64 virtual machine -weight: 5 - -### FIXED, DO NOT MODIFY -layout: learningpathall ---- - -## Install Dynatrace OneAgent on Azure Ubuntu Arm64 - -To install Dynatrace OneAgent on an Azure Ubuntu 24.04 LTS Arm64 virtual machine, follow these steps. - -At the end of the installation, Dynatrace is: - -* Installed and running as a host monitoring agent -* Connected to the Dynatrace SaaS environment -* Monitoring system processes and services automatically -* Verified on Arm64 (aarch64) architecture - -## Update the system and install required tools - -Update the operating system and install the tools required for downloading the Dynatrace installer. - -```console -sudo apt update && sudo apt upgrade -y -sudo apt install -y curl wget unzip ca-certificates -``` - -## Verify Arm64 architecture - -Confirm that the virtual machine is running on the Arm64 architecture. - -```console -uname -m -``` - -output is similar to: -```output -aarch64 -``` -This confirms the system is using the Arm64 architecture required for Cobalt 100 processors. - -## Create your Dynatrace trial environment - -Fill in the required information: - -- First name -- Last name -- Work email address -- Company name -- Country - -After submitting the form, Dynatrace creates a new SaaS monitoring environment for you. - -This process usually takes 1–2 minutes. - -## Access your Dynatrace environment - -After the environment is created, you will receive an email with a link similar to: - -```console -https://.live.dynatrace.com -``` - -**Example:** - -```text -https://qzo72404.live.dynatrace.com -``` -The Environment ID uniquely identifies your Dynatrace tenant and is required for agent installation. - -![Dynatrace environment login page showing the SaaS environment URL alt-txt#center](images/dynatrace-env-id.png "Dynatrace SaaS environment login") - -## Navigate to Deployment - -From the Dynatrace dashboard: - -- Select Deploy Dynatrace -- Choose OneAgent -- Select Linux - -![Dynatrace deployment page showing OneAgent setup options alt-txt#center](images/oneagent1.png "Dynatrace OneAgent deployment setup page") - -This page generates the installation command tailored for your environment. - -## Select ARM Architecture - -In the installer page: - -- Cloud platform → Linux -- Select architecture → ARM64 -- Select monitoring mode: - >Full-stack monitoring - -![Dynatrace installer configuration page showing ARM64 architecture selection alt-txt#center](images/oneagent-arch.png "Dynatrace OneAgent ARM64 architecture selection") - -## Copy OneAgent Installer Command - -Dynatrace generates an installer command that includes your environment ID and API token.: - -```console -wget -O Dynatrace-OneAgent-Linux-arm.sh \ -"https://.live.dynatrace.com/api/v1/deployment/installer/agent/unix/default/latest?arch=arm" \ ---header="Authorization: Api-Token " -``` - -**Example:** - -```text -wget -O Dynatrace-OneAgent-Linux-arm.sh \ -"https://qzo72404.live.dynatrace.com/api/v1/deployment/installer/agent/unix/default/latest?arch=arm" \ ---header="Authorization: Api-Token DT_API_TOKEN" -``` -- The API token allows secure access to the Dynatrace installer. - -Run this command on the virtual machine to download the installer. - -**Verify signature** - -For security, verify the installer signature using Dynatrace’s root certificate. - -```console -wget https://ca.dynatrace.com/dt-root.cert.pem ; ( echo 'Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg="sha-256"; boundary="--SIGNED-INSTALLER"'; echo ; echo ; echo '----SIGNED-INSTALLER' ; cat Dynatrace-OneAgent-Linux-x86-1.331.49.20260227-104933.sh ) | openssl cms -verify -CAfile dt-root.cert.pem > /dev/null -``` -Run it on the VM. - -![Dynatrace UI displaying the generated OneAgent installer command alt-txt#center](images/onagent-install.png "Dynatrace OneAgent installation command") - -## Install OneAgent as the privileged user - -Run: - -```console -sudo /bin/sh Dynatrace-OneAgent-Linux-x86-1.331.49.20260227-104933.sh --set-monitoring-mode=fullstack --set-app-log-content-access=true -``` - -The output is similar to: -```output -2026-03-12 05:59:21 UTC Starting Dynatrace ActiveGate AutoUpdater... -2026-03-12 05:59:21 UTC Checking if Dynatrace ActiveGate AutoUpdater is running ... -2026-03-12 05:59:21 UTC Dynatrace ActiveGate AutoUpdater is running. -2026-03-12 05:59:21 UTC Cleaning autobackup... -2026-03-12 05:59:21 UTC Removing old installation log files... -2026-03-12 05:59:21 UTC -2026-03-12 05:59:21 UTC -------------------------------------------------------------- -2026-03-12 05:59:21 UTC Installation finished successfully. -``` - -The installer performs several tasks automatically: - -- Downloads monitoring components -- Configures kernel instrumentation -- Installs the OneAgent system service -- Registers the host with your Dynatrace environment - -## Verify OneAgent Service - -Check that the Dynatrace monitoring service is running. - -This confirms the monitoring agent started successfully. -```console -sudo systemctl status oneagent -``` - -The output is similar to: -```output -● dynatracegateway.service - Dynatrace ActiveGate service - Loaded: loaded (/etc/systemd/system/dynatracegateway.service; enabled; preset: enabled) - Active: active (running) since Thu 2026-03-12 05:59:07 UTC; 1min 7s ago - Process: 20280 ExecStart=/opt/dynatrace/gateway/dynatracegateway start (code=exited, status=0/SUCCESS) - Main PID: 20316 (dynatracegatewa) -``` - -This confirms the monitoring agent started successfully. - -## Verify Dynatrace Processes - -This confirms the monitoring agent started successfully. - -```console -ps aux | grep oneagent -``` - -The output is similar to: -```output -dtuser 17754 0.0 0.0 307872 4388 ? Ssl 05:48 0:00 /opt/dynatrace/oneagent/agent/lib64/oneagentwatchdog -bg -config=/opt/dynatrace/oneagent/agent/conf/watchdog.conf -dtuser 17761 0.2 0.3 1183000 59136 ? Sl 05:48 0:06 oneagentos -Dcom.compuware.apm.WatchDogTimeout=900 -watchdog.restart_file_location=/var/lib/dynatrace/oneagent/agent/watchdog/watchdog_restart_file -Dcom.compuware.apm.WatchDogPipe=/var/lib/dynatrace/oneagent/agent/watchdog/oneagentos_pipe_17754 -dtuser 17793 0.0 0.2 689184 34408 ? Sl 05:48 0:01 oneagentloganalytics -Dcom.compuware.apm.WatchDogTimeout=900 -Dcom.compuware.apm.WatchDogPipe=/var/lib/dynatrace/oneagent/agent/watchdog/oneagentloganalytics_pipe_17754 -dtuser 17795 0.1 0.2 361936 42940 ? Sl 05:48 0:04 oneagentnetwork -Dcom.compuware.apm.WatchDogTimeout=900 -Dcom.compuware.apm.WatchDogPipe=/var/lib/dynatrace/oneagent/agent/watchdog/oneagentnetwork_pipe_17754 -dtuser 17883 0.0 0.0 28212 5340 ? Sl 05:49 0:00 /opt/dynatrace/oneagent/agent/lib64/oneagentebpfdiscovery --log-dir /var/log/dynatrace/oneagent/os/ --log-no-stdout --log-level info -azureus+ 23847 0.0 0.0 9988 2772 pts/0 S+ 06:33 0:00 grep --color=auto oneagent -``` - -This confirms the monitoring agent started successfully. - -## Confirm Host Detection in Dynatrace - -Return to the Dynatrace web interface. - -Navigate to: - -```text -Infrastructure & Operations -→ Hosts -``` - -You should see: - -```outout -Host name: -OS: Linux -Architecture: ARM64 -Monitoring mode: Full Stack -``` - -![Dynatrace hosts dashboard showing detected ARM64 virtual machine alt-txt#center](images/dynatrace-host.png "Dynatrace host monitoring dashboard") - -## Check Automatic Process Discovery - -Dynatrace automatically discovers running applications and services. - -View them under: - -```text -Hosts → Processes -``` - -Dynatrace identifies services such as: - -- system processes -- web servers -- databases -- container runtimes - -![Dynatrace process monitoring dashboard showing automatically discovered services alt-txt#center](images/dynatrace-process.png "Dynatrace process discovery view") - -## What you've accomplished and what's next - -You've successfully installed Dynatrace OneAgent on your Azure Ubuntu Arm64 virtual machine. Your installation includes: - -- Dynatrace OneAgent is installed and running as a system service -- Automatic startup enabled through systemd -- Secure connection to the Dynatrace SaaS platform -- Full-stack monitoring of system resources and processes -- Arm64-native monitoring on Azure Cobalt 100 processors - -Next, you'll install Dynatrace ActiveGate to enable additional capabilities such as Kubernetes monitoring, secure data routing, and extension support. diff --git a/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/1-setup.md b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/1-setup.md new file mode 100644 index 0000000000..ddb4003719 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/1-setup.md @@ -0,0 +1,68 @@ +--- +title: Set up the target environment and compile the application +weight: 2 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## About the sample application + +To analyze performance bottlenecks, you need an environment and a sample application to profile. In this section, you configure an Arm Performix connection and build a Mandelbrot set generator. + +A Mandelbrot set generator is a classic computer science application used to test computational performance. It calculates a famous mathematical fractal by performing intense, repeated mathematical operations (often floating-point) for every pixel in a large image. Because the math for each pixel is independent of the others, it's a highly parallelizable workload that's perfect for demonstrating CPU optimizations like vectorization and loop unrolling. + +## Before you begin + +Make sure Arm Performix is installed on your host machine. The host machine is your local computer where the Arm Performix GUI runs, and it can be a Windows, macOS, or Linux machine. The target machine is the Linux server where your application is compiled and where the application runs. + +If you don't have Arm Performix installed, see the [Arm Performix install guide](/install-guides/atp/). + +From the host machine, open the Arm Performix application and navigate to the **Targets** tab. Set up an SSH connection to the target that runs the workload, and test the connection. For the examples in this guide, you connect to an Arm Neoverse-based server. + +The Arm Performix collection agent requires Python and `binutils` to run on the target machine. + +Connect to your target machine using SSH and install these required OS packages. This includes the GNU C++ compiler. + +For Ubuntu and other Debian-based distributions, run the following command: + +```bash +sudo apt-get install python3 python3-venv build-essential -y +``` + +## Build the sample application on the target machine + +Download the sample application from GitHub, which is a Mandelbrot set generator provided under the [Arm Education License](https://github.com/ArmDeveloperEcosystem/mandelbrot-example?tab=License-1-ov-file). + +```bash +cd $HOME +git clone https://github.com/ArmDeveloperEcosystem/mandelbrot-example.git +``` + +Run the provided setup script to build the baseline application: + +```bash +./build.sh baseline +``` + +This builds the scalar version of the application, no vectorization is used. + +When the build completes, a binary named `mandelbrot-baseline` is created in the `./builds` directory. + +The application requires one argument: the number of threads to use. Run this new executable with 4 threads: + +```bash +./builds/mandelbrot-baseline 4 +``` + +The application generates a bitmap image file in your `./images` directory that looks similar to the following fractal: + +![Mandelbrot set fractal generated by the sample application#center](./green-parallel-512.webp "Mandelbrot Set") + +## What you've learned and what's next + +In this section: +- You set up the target machine and established an SSH connection. +- You downloaded, compiled, and ran the baseline Mandelbrot application. + +Next, you'll use the CPU Microarchitecture recipe to identify performance bottlenecks in the application. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/2-run-cpu-uarch.md b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/2-run-cpu-uarch.md new file mode 100644 index 0000000000..98118934eb --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/2-run-cpu-uarch.md @@ -0,0 +1,77 @@ +--- +title: Identify application bottlenecks with the CPU Microarchitecture recipe +weight: 3 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Run the CPU Microarchitecture recipe + +To identify performance bottlenecks, run the CPU Microarchitecture recipe in Arm Performix. Arm Performix uses microarchitectural sampling to show which instruction pipeline stages dominate program latency, and then highlights ways to improve those bottlenecks. + +Start by reviewing the code in `main.cpp`, the program generates a 1920×1080 bitmap image of the fractal. + +```cpp +#include "Mandelbrot.h" +#include + +using namespace std; + +int main(int argc, char* argv[]){ + + const int NUM_THREADS = argc == 2 ? std::stoi(argv[1]) : 1; + std::cout << "Number of Threads = " << NUM_THREADS << std::endl; + + Mandelbrot::Mandelbrot myplot(1920, 1080, NUM_THREADS); + myplot.draw("Green-Parallel-512.bmp", Mandelbrot::Mandelbrot::GREEN); + + return 0; +} +``` + +When Arm Performix launches the executable on the target machine, it does so from a temporary agent directory, `/tmp/atperf/tools/atperf-agent`. If your code uses a relative path to save the image, the image is written to that temporary folder and might be deleted or you can't find it. + +To prevent this, edit the `myplot.draw()` line in `main.cpp` to use the absolute path to your project's image folder (for example, `/home/ubuntu/mandelbrot-example/Green-Parallel-512.bmp`), and then rebuild the application. + +In the Arm Performix application on your host machine, select the **CPU Microarchitecture** recipe. + +![Arm Performix CPU Microarchitecture configuration screen#center](./cpu-uarch-config.webp "CPU Microarchitecture Configuration") + +Select the target you configured in the setup section. If this is your first run on this target, you might need to select **Install Tools** to copy the collection tools to the target. After the tools are installed, you will see that the target is ready. + +Next, select the **Workload type** and select **Launch a new process**. + +Enter the absolute path to your executable in the **Workload** field. For example, `/home/ubuntu/mandelbrot-example/builds/mandelbrot-baseline`. Make sure to add the number of threads argument. + +{{% notice Note %}} +Use the full path to your executable because the **Workload** field doesn't currently support shell-style path expansion. +{{% /notice %}} + +Before starting the analysis, you can customize the configuration. For instance, you can set a time limit for the workload or choose specific metrics to investigate. You can also adjust the sampling rate (High, Normal, or Low) to balance collection overhead against sampling granularity. Because this Mandelbrot example is a native C++ application, you can ignore the **Collect managed code stacks** toggle, which is used for Java or .NET workloads. + +When your configuration is ready, select **Run Recipe** to launch the workload and collect the performance data. + +## View the run results + +Arm Performix generates a high-level instruction pipeline view, highlighting where the most time is spent. + +![Arm Performix high-level instruction pipeline results#center](cpu-uarch-results.webp "Instruction Pipeline View") + +In this breakdown, you see frontend and backend Stalls. Within the backend stalls, work is split between integer and floating-point operations. + +There is no measured SIMD activity, even though this workload is highly parallelizable. + +The **Insights** panel explains the highest category, which is frontend stalls for this system. + +![Arm Performix insights panel highlighting frontend stalls#center](cpu-uarch-insights.webp "Insights Panel") + +To inspect executed instruction types in more detail, use the Instruction Mix recipe in the next step. + +## What you've learned and what's next + +In this section: +- You ran the CPU Microarchitecture recipe on the Mandelbrot application. +- You identified that the application spends most of its time in Backend Stalls without using SIMD operations. + +Next, you'll run the Instruction Mix recipe to confirm where optimization opportunities exist and implement vectorization. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/3-instruction-mix.md b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/3-instruction-mix.md new file mode 100644 index 0000000000..cde3741981 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/3-instruction-mix.md @@ -0,0 +1,101 @@ +--- +title: Analyze SIMD utilization with the Instruction Mix recipe +weight: 4 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Run the Instruction Mix recipe + +The previous CPU Microarchitecture analysis showed that the sample application used no single instruction, multiple data (SIMD) operations, which points to an optimization opportunity. Run the Instruction Mix recipe to learn more. The Instruction Mix launch panel is similar to CPU Microarchitecture, but it doesn't include options to choose metrics. Again, enter the full path to the workload. + +Select **Dynamic** for the **Analysis Mode**. + +![Arm Performix Instruction Mix configuration screen#center](instruction-mix-config.webp "Instruction Mix Configuration") + +The results below confirm a high number of integer and floating-point operations, with no SIMD operations. The **Insights** panel suggests vectorization as a path forward, lists possible root causes, and links to related Learning Paths. + +![Arm Performix Instruction Mix results showing high integer and floating point operations#center](instruction-mix-results.webp "Instruction Mix Results") + +## Vectorize the application + +To address the lack of SIMD operations, you can vectorize the application's most intensive functions using Neon. For the Mandelbrot application, `Mandelbrot::draw` and its inner `Mandelbrot::getIterations` function consume most of the runtime. + +You can build a vectorized version which uses Neon operations and will run on any Neoverse system. Your system might support alternatives such as SVE or SVE2 which can also be used, but only Neon is explained here to make sure you can run it on any Arm Linux system. + +Connect to your target machine using SSH and navigate to your project directory. + +Build the Neon version: + +```bash +cd $HOME/mandelbrot-example +./build.sh neon +``` + +The Neon executable is `builds/mandelbrot-neon` + +Run the Instruction Mix recipe again with the Neon executable. Integer and floating-point operations are greatly reduced and replaced by a smaller set of SIMD instructions. + +![Arm Performix Instruction Mix results after vectorization showing increased SIMD operations#center](instruction-mix-simd-results.webp "SIMD Instruction Mix Results") + +## Assess the performance improvements + +Because you are running multiple experiments, give each run a meaningful nickname to keep results organized. +![Arm Performix run renaming interface#center](rename-run.webp "Rename Run") + +Use the **Compare** feature at the top right of an entry in the **Runs** view to select another run of the same recipe for comparison. + +![Arm Performix compare view selection box#center](compare-with-box.webp "Compare Runs") + +After you select two runs, Arm Performix overlays them so you can review category changes in one view. In the new run, you see Advanced SIMD Operations increase dramatically and Floating Point Operations shrink. + +![Arm Performix comparison showing differences in instruction mix#center](instruction-mix-diff-results.webp "Instruction Mix Comparison") +Compared to the baseline, floating-point operations, branch operations, and some integer operations have been traded for loads, stores, and SIMD operations. + +Execution time also improves significantly. You can confirm by running each version with the Linux `time` command. + +Run the baseline version: + +```bash +time builds/mandelbrot-baseline 4 +``` + +Your output will differ depending on the system you are using, but the output is similar to: + +```output +Number of Threads = 4 + +real 0m1.575s +user 0m5.958s +sys 0m0.018s +``` + +Run the Neon version: + +```bash +time builds/mandelbrot-neon 4 +``` + +The Neon output shows a significant performance improvement: + +```output +Number of Threads = 4 + +real 0m0.240s +user 0m0.798s +sys 0m0.027s +``` + +## Compare the CPU Microarchitecture results + +The CPU Microarchitecture recipe also supports a **Compare** view that shows percentage-point changes in each stage and instruction type. +![Arm Performix CPU Microarchitecture comparison showing changes in each stage#center](cpu-uarch-simd-results-diff.webp "CPU Microarchitecture Difference View") + +You can see the relative differences in backend stalls between the baseline version and the Neon version. The Insights panel offers additional explanation. + +In this section: +- You used the Instruction Mix recipe to confirm a lack of SIMD operations. +- You vectorized the sample application and verified the shift toward SIMD execution. + +You're now ready to analyze and optimize your own native C/C++ applications on Arm Neoverse using Arm Performix. Review the next steps to continue your learning journey. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/_index.md b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/_index.md new file mode 100644 index 0000000000..33eac6446a --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/_index.md @@ -0,0 +1,64 @@ +--- +title: Optimize application performance using Arm Performix CPU microarchitecture analysis + +draft: true +cascade: + draft: true + +minutes_to_complete: 60 + +who_is_this_for: This is an introductory topic for software developers who want to learn performance analysis methodologies for Linux applications on Arm Neoverse-based servers. + +learning_objectives: + - Identify CPU pipeline bottlenecks using the Arm Performix CPU Microarchitecture recipe + - Analyze instruction types and SIMD utilization using the Instruction Mix recipe + - Optimize application performance using vectorization and compiler flags + - Compare performance profiles to measure execution improvements + +prerequisites: + - An Arm Neoverse-based server running Linux (bare-metal or cloud bare-metal instance preferred for access to hardware performance counters) + - Familiarity with Linux command line + - Basic understanding of CPU performance concepts + +author: +- Brendan Long +- Kieran Hejmadi +- Jason Andrews + +### Tags +skilllevels: Introductory +subjects: Performance and Architecture +armips: + - Neoverse +tools_software_languages: + - Arm Performix + - C + - Runbook + +operatingsystems: + - Linux + +further_reading: + - resource: + title: "Find CPU Cycle Hotspots with Arm Performix" + link: /learning-paths/servers-and-cloud-computing/cpu_hotspot_performix/ + type: documentation + - resource: + title: "Port Code to Arm Scalable Vector Extension (SVE)" + link: /learning-paths/servers-and-cloud-computing/sve/ + type: documentation + - resource: + title: "Arm Neoverse N1: Core Performance Analysis Methodology" + link: https://armkeil.blob.core.windows.net/developer/Files/pdf/white-paper/neoverse-n1-core-performance-v2.pdf + type: documentation + - resource: + title: "Arm Neoverse N1 PMU Guide" + link: https://developer.arm.com/documentation/PJDOC-466751330-547673/r4p1/ + type: documentation + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 # _index.md always has weight of 1 to order correctly +layout: "learningpathall" # All files under learning paths have this same wrapper +learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content. +--- diff --git a/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/_next-steps.md b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/_next-steps.md new file mode 100644 index 0000000000..c3db0de5a2 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/_next-steps.md @@ -0,0 +1,8 @@ +--- +# ================================================================================ +# FIXED, DO NOT MODIFY THIS FILE +# ================================================================================ +weight: 21 # Set to always be larger than the content in this path to be at the end of the navigation. +title: "Next Steps" # Always the same, html page title. +layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing. +--- diff --git a/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/compare-with-box.webp b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/compare-with-box.webp new file mode 100644 index 0000000000..7ba190543a Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/compare-with-box.webp differ diff --git a/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/cpu-uarch-config.webp b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/cpu-uarch-config.webp new file mode 100644 index 0000000000..15882ddddd Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/cpu-uarch-config.webp differ diff --git a/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/cpu-uarch-insights.webp b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/cpu-uarch-insights.webp new file mode 100644 index 0000000000..4d54177fa2 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/cpu-uarch-insights.webp differ diff --git a/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/cpu-uarch-results.webp b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/cpu-uarch-results.webp new file mode 100644 index 0000000000..8080d0322a Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/cpu-uarch-results.webp differ diff --git a/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/cpu-uarch-simd-results-diff.webp b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/cpu-uarch-simd-results-diff.webp new file mode 100644 index 0000000000..1966810e56 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/cpu-uarch-simd-results-diff.webp differ diff --git a/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/green-parallel-512.webp b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/green-parallel-512.webp new file mode 100644 index 0000000000..a26550d53d Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/green-parallel-512.webp differ diff --git a/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/instruction-mix-config.webp b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/instruction-mix-config.webp new file mode 100644 index 0000000000..adf4c404d7 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/instruction-mix-config.webp differ diff --git a/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/instruction-mix-diff-results.webp b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/instruction-mix-diff-results.webp new file mode 100644 index 0000000000..b7bae79e70 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/instruction-mix-diff-results.webp differ diff --git a/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/instruction-mix-results.webp b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/instruction-mix-results.webp new file mode 100644 index 0000000000..19b2600d69 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/instruction-mix-results.webp differ diff --git a/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/instruction-mix-simd-results.webp b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/instruction-mix-simd-results.webp new file mode 100644 index 0000000000..44fee4d187 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/instruction-mix-simd-results.webp differ diff --git a/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/rename-run.webp b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/rename-run.webp new file mode 100644 index 0000000000..20085a1073 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/performix-microarchitecture/rename-run.webp differ diff --git a/tools/filter_checker.py b/tools/filter_checker.py index 84b5887b00..2255a8a810 100644 --- a/tools/filter_checker.py +++ b/tools/filter_checker.py @@ -298,19 +298,23 @@ def addCloudServiceProvidersToStatusDict(): if 'cloud_service_providers' in learning_path_metadata: # since not all LPs have this filtering mechanism, need this check to avoid errors cloud_service_providers = learning_path_metadata['cloud_service_providers'] if cloud_service_providers is not None: - if cloud_service_providers not in status_dic['cloud_service_providers'][dir_main_category]: - # create subject key in dic - status_dic['cloud_service_providers'][dir_main_category][cloud_service_providers] = {} - # check if in allow list - if cloud_service_providers in dic_allow_list["cloud_service_providers"]: - status_dic['cloud_service_providers'][dir_main_category][cloud_service_providers]['allowed'] = True + # Normalize to list to handle both single string and YAML list formats + if isinstance(cloud_service_providers, str): + cloud_service_providers = [cloud_service_providers] + for csp in cloud_service_providers: + if csp not in status_dic['cloud_service_providers'][dir_main_category]: + # create subject key in dic + status_dic['cloud_service_providers'][dir_main_category][csp] = {} + # check if in allow list + if csp in dic_allow_list["cloud_service_providers"]: + status_dic['cloud_service_providers'][dir_main_category][csp]['allowed'] = True + else: + status_dic['cloud_service_providers'][dir_main_category][csp]['allowed'] = False + status_dic['cloud_service_providers'][dir_main_category][csp]['count'] = 1 # make count one + status_dic['cloud_service_providers'][dir_main_category][csp]['learning-path-titles'] = [learning_path_metadata['title']] # create list with title else: - status_dic['cloud_service_providers'][dir_main_category][cloud_service_providers]['allowed'] = False - status_dic['cloud_service_providers'][dir_main_category][cloud_service_providers]['count'] = 1 # make count one - status_dic['cloud_service_providers'][dir_main_category][cloud_service_providers]['learning-path-titles'] = [learning_path_metadata['title']] # create list with title - else: - status_dic['cloud_service_providers'][dir_main_category][cloud_service_providers]['count'] += 1 # increase count by one - status_dic['cloud_service_providers'][dir_main_category][cloud_service_providers]['learning-path-titles'].append(learning_path_metadata['title']) # add title to list + status_dic['cloud_service_providers'][dir_main_category][csp]['count'] += 1 # increase count by one + status_dic['cloud_service_providers'][dir_main_category][csp]['learning-path-titles'].append(learning_path_metadata['title']) # add title to list return status_dic diff --git a/tools/report.py b/tools/report.py index 4435d94fcc..378d05e935 100644 --- a/tools/report.py +++ b/tools/report.py @@ -55,10 +55,25 @@ def content_parser(directory, period): date = date.stdout.rstrip().decode("utf-8") logging.debug(f"Last updated: {date}") author = "None" - for directory_list in open(directory +"/" + item): - if re.search("^author:", directory_list): - # split and strip out '\n' - author = directory_list.split(": ")[1].rstrip() + lines = open(directory +"/" + item).readlines() + for idx, line in enumerate(lines): + if re.search("^author:", line): + parts = line.split(": ", 1) + if len(parts) > 1 and parts[1].strip(): + # Inline single author: "author: Name" + author = parts[1].rstrip() + else: + # YAML list format, collect "- name" entries + authors = [] + for subsequent in lines[idx+1:]: + m = re.match(r'^\s+-\s+(.+)', subsequent) + if m: + authors.append(m.group(1).strip()) + else: + break + if authors: + author = ", ".join(authors) + break logging.debug(f"Primary author {author}") if not author in auth_list: auth_list.append(author)