Go GC Tech Review

Author: pareenaverma

content/learning-paths/servers-and-cloud-computing/go-gc-default-settings/_index.md

@@ -4,7 +4,7 @@ draft: true
 cascade:
     draft: true
 
-description: Learn how to measure and observe Go garbage collection metrics on AWS Graviton instances.
+description: Learn how to run Go benchmarks on AWS Graviton, capture GC metrics and pprof profiles, and establish a reproducible default-GC baseline for memory-intensive workloads on Arm.
 
 minutes_to_complete: 75
 
@@ -32,7 +32,6 @@ cloud_service_providers:
 armips:
     - Neoverse
 tools_software_languages:
-    - AWS
     - Go
 operatingsystems:
     - Linux

content/learning-paths/servers-and-cloud-computing/go-gc-default-settings/choose_aws_instance.md

@@ -6,15 +6,15 @@ weight: 2
 layout: learningpathall
 ---
 ## What is Garbage Collection? (GC)
-Memory management is a critical aspects of application performance, and Garbage Collection (GC) plays a central role in automating that process. GC continuously identifies and removes objects that are no longer needed, freeing memory for re-use for other purposes..
+Memory management is a critical aspect of application performance, and Garbage Collection (GC) plays a central role in automating that process. GC continuously identifies and removes objects that are no longer needed, freeing memory for reuse.
 
 While this automation improves productivity and application safety, inefficient garbage collection can lead to increased CPU usage, longer response times, and unexpected application pauses. 
 
 Tracking GC metrics provides a window into an application's memory health, helping engineers optimize performance, and ensuring the system can scale efficiently under load.
 
 ## Measuring default Go GC behavior on Arm servers
 
-Go is one such language which implements GC.  As Go applications can spend meaningful time allocating memory and running garbage collection, it is important to understand how the Go runtime behaves under default settings. 
+Go is one such language that implements GC.  As Go applications can spend meaningful time allocating memory and running garbage collection, it is important to understand how the Go runtime behaves under default settings. 
 
 In this Learning Path, you'll run Go benchmarks on an AWS Graviton instance. The goal is to build a clean baseline, measuring operation time, allocation rate, GC frequency, and GC pause cost.
 
@@ -44,7 +44,7 @@ aws ec2 describe-instance-type-offerings \
     --output table
 ```
 
-If the command returns one or more Availability Zones, you can use `m8g.xlarge` in that Region.  If you are unable to find `m8g.xlarge` in your Region, you can try a different Region, or fallback to an 'm7g.xlarge' instance, which is based on the previous generation AWS Graviton3:
+If the command returns one or more Availability Zones, you can use `m8g.xlarge` in that Region. If `m8g.xlarge` is not available in your Region, try a different Region, or fall back to an `m7g.xlarge` instance, which is based on the previous generation AWS Graviton3:
 
 ```console
 aws ec2 describe-instance-type-offerings \
@@ -55,4 +55,17 @@ aws ec2 describe-instance-type-offerings \
     --output table
 ```
 
-Once you have chosen an instance type, provision it to run Ubuntu 24.04 LTS Arm64.  Once the instance is running, and you are ssh'd into it, you can proceed to the next step.
+After selecting an instance type, launch it with the Ubuntu 24.04 LTS (arm64) AMI. You can do this through the [AWS EC2 console](https://console.aws.amazon.com/ec2/) or the AWS CLI. When configuring the instance:
+
+- Select your chosen instance type (`m8g.xlarge` or `m7g.xlarge`)
+- Choose the Ubuntu 24.04 LTS arm64 AMI for your Region
+- Configure a security group that allows inbound SSH (port 22) from your IP address
+- Associate an existing key pair or create a new one for SSH access
+
+After the instance reaches the running state, connect to it over SSH:
+
+```bash
+ssh -i /path/to/your-key.pem ubuntu@<instance-public-ip>
+```
+
+You now have a running Arm Linux instance on AWS Graviton. In the next section you'll install Go and Benchstat so the instance is ready to run benchmarks.

content/learning-paths/servers-and-cloud-computing/go-gc-default-settings/create_gc_benchmark.md

@@ -17,18 +17,17 @@ You'll first create a small Go benchmark module.  The high-level flow is:
 5. Measure how much GC activity occurred during the benchmark.
 6. Report both performance metrics and GC-related metrics.
 
-Pasting the code below will create the module and benchmark file:
+Create the module directory and initialize it:
 
 ```bash
-
-# Create the module directory and initialize it.
-
 mkdir -p $HOME/go-gc-default/parsebench
 cd $HOME/go-gc-default
 go mod init example.com/go-gc-default
+```
 
-# Create the benchmark file:
+Create the benchmark file:
 
+```bash
 cat > parsebench/parsebench_test.go <<'EOF'
 package parsebench
 
@@ -159,7 +158,7 @@ func BenchmarkParseAndAllocate(b *testing.B) {
 EOF
 ```
 
-The benchmark code is now ready to run!  Give it a try by running the following command:
+The benchmark code is now ready. Run the following command to verify it executes without errors:
 
 ```bash
 cd $HOME/go-gc-default

content/learning-paths/servers-and-cloud-computing/go-gc-default-settings/install_go_tools.md

@@ -15,27 +15,36 @@ The following commands use Go 1.26.3. The same commands work with other Go versi
 {{% /notice %}}
 
 
-```bash
-# Download the Go archive and verify the checksum:
+Download the Go archive and verify the checksum:
 
+```bash
 cd $HOME
 curl -LO https://go.dev/dl/go1.26.3.linux-arm64.tar.gz
 echo "9d89a3ea57d141c2b22d70083f2c8459ba3890f2d9e818e7e933b75614936565  go1.26.3.linux-arm64.tar.gz" | sha256sum -c -
+```
 
-# Install Go under `/usr/local`:
+Install Go under `/usr/local`:
 
+```bash
 sudo rm -rf /usr/local/go
 sudo tar -C /usr/local -xzf go1.26.3.linux-arm64.tar.gz
+```
+
+Add Go to your shell path:
 
-# Add Go to your shell path:
+```bash
 export PATH=/usr/local/go/bin:$HOME/go/bin:$PATH
+```
 
-# To make the path update persistent, add it to your shell profile:
+To make the path update persistent, add it to your shell profile:
 
+```bash
 echo 'export PATH=/usr/local/go/bin:$HOME/go/bin:$PATH' >> $HOME/.profile
+```
+
+Verify that Go is installed for Arm64 Linux:
 
-# Verify that Go is installed for Arm64 Linux:
-echo 
+```bash
 go version
 go env GOOS GOARCH
 ```
@@ -50,7 +59,7 @@ arm64
 
 ## Installing Benchstat
 
-`benchstat` is a Go performance analysis tool that compares benchmark results and provides statistical analysis of performance differences between runs. It helps developers determine whether observed changes in benchmark metrics are statistically significant rather than simply the result of normal measurement variability.  We'll use `benchstat` for that purpose in this learning path.
+`benchstat` is a Go performance analysis tool that compares benchmark results and provides statistical analysis of performance differences between runs. It helps developers determine whether observed changes in benchmark metrics are statistically significant rather than simply the result of normal measurement variability. You'll use `benchstat` for that purpose in this Learning Path.
 
 To install `benchstat`:
 
@@ -66,7 +75,8 @@ go: downloading github.com/aclements/go-moremath v0.0.0-20210112150236-f10218a38
 ```
 
 Finally, do a quick check to make sure `benchstat` is installed:
-```console
+
+```bash
 benchstat -h
 ```
 

content/learning-paths/servers-and-cloud-computing/go-gc-default-settings/interpret_gc_results.md

@@ -8,13 +8,13 @@ layout: learningpathall
 
 ## Understanding the benchmark metrics
 
-To understand better what these benchmarks results are showing, firt open the Benchstat summary:
+To understand what the benchmark results are showing, first open the Benchstat summary:
 
 ```bash
 cat default_gc_benchstat.txt
 ```
 
-The the metrics you see explain the following:
+The metrics you see explain the following:
 
 | Metric | Read |
 | --- | --- |
@@ -37,6 +37,8 @@ cat cpu_default_top.txt
 
 Look for functions that dominate CPU time. In an allocation-heavy benchmark, you can expect to see time in string handling, allocation paths, and some runtime or GC support functions.
 
+The `flat` column shows CPU time spent directly in that function. The `cum` (cumulative) column includes time spent in the function and all functions it called. A function with low `flat` but high `cum` is spending most of its time in callees, which can point to allocation chains or deep call stacks.
+
 On the validated `m8g.xlarge` instance, the top CPU profile entries included string scanning, string concatenation, split handling, and allocation paths:
 
 ```output
@@ -48,11 +50,9 @@ On the validated `m8g.xlarge` instance, the top CPU profile entries included str
      0.72s  4.56% 53.36%      5.17s 32.76%  runtime.mallocgc
 ```
 
-Open the heap allocation profile summary:
+`IndexByteString` and `concatstrings` dominate because the benchmark splits the payload with `strings.Split` (which scans for `;` byte-by-byte) and builds `key:length` strings with `+` concatenation on every iteration. These operations create a large volume of short-lived strings, giving the GC constant work to do. On Graviton, `pprof` makes these allocation chains directly visible because the Arm64 stack unwinder captures the full call path without frame pointer ambiguity.
 
-```bash
-cat mem_default_alloc_top.txt
-```
+Open the heap allocation profile summary:
 
 Look for application functions that allocate the most heap memory. Reducing allocation volume in those functions usually gives the Go GC less work to do.
 
@@ -68,7 +68,7 @@ On the validated `m8g.xlarge` instance, the allocation profile showed that `stri
 
 ## Keeping a baseline to compare to future changes
 
-These results show your default Go GC baseline stats for this benchmarking app on AWS Graviton. By keeping the following files together (eg, each stored in their own zip file, folder, etc) you can easily see how making code changes affects your apps overall performance:
+These results show your default Go GC baseline for this benchmarking app on AWS Graviton. By keeping the following files together — for example, in a versioned folder or archive — you can see how code changes affect overall performance:
 
 ```output
 default_runtime_baseline.txt
@@ -81,133 +81,8 @@ mem_default.out
 mem_default_alloc_top.txt
 ```
 
-## Experiment with Code Changes that influence GC Behavior
-Now that you have a baseline, you can experiment with code changes that influence GC behavior. For example, you could try:
-
-## Challenge 1
-
-You just found out that the payload size this benchmark is intended to represent is actually only 128 records instead of 2048. What changes can we make from the baseline to test whether optimizing for this smaller workload affects GC frequency, pause times, and overall application performance?
-
-### Idea: Reduce the payload size
-
-### How
-
-```go
-payload := strings.Repeat(
-    "name=arm&runtime=go&gc=default&value=12345;",
-    512,
-)
-```
-
-### Why
-
-A smaller payload creates fewer temporary objects and less garbage each iteration.
-
----
-
-## Challenge 2
-
-After profiling the application, you discover that the input payload rarely changes between requests. What modifications can we make to reuse preprocessing work and determine whether reducing repeated allocations improves GC behavior and throughput?
-
-### Idea: Move `strings.Split(payload, ";")` outside the benchmark loop
-
-### How
-
-```go
-parts := strings.Split(payload, ";")
-
-b.ResetTimer()
-
-for i := 0; i < b.N; i++ {
-    ...
-}
-```
-
-### Why
-
-This avoids repeatedly allocating the same slice of records on every iteration.
-
----
-
-## Challenge 3
-
-The benchmark currently creates a new output buffer for every operation, but production code processes millions of requests using the same worker. How can we modify the benchmark to reuse memory and evaluate the impact on GC activity and memory consumption?
-
-### Idea: Reuse the output slice
-
-### How
-
-```go
-out := make([]string, 0, len(parts))
-
-for i := 0; i < b.N; i++ {
-    out = out[:0]
-    ...
-}
-```
-
-### Why
-
-Reusing the backing array reduces allocations and GC pressure.
-
----
-
-## Challenge 4
-
-A CPU profile shows that string parsing is one of the hottest code paths in the application. What changes can we make to reduce temporary allocations during parsing and measure whether this reduces GC overhead?
-
-### Idea: Replace `strings.SplitN()` with `strings.IndexByte()`
-
-### How
-
-```go
-idx := strings.IndexByte(part, '=')
-if idx >= 0 {
-    key := part[:idx]
-    value := part[idx+1:]
-    ...
-}
-```
-
-### Why
-
-This avoids allocating a temporary `[]string` for every record processed.
-
----
-
-## Challenge 5
-
-Product requirements change and the application no longer needs to generate derived `"key:length"` strings. What modifications can we make to avoid unnecessary string allocations and test their effect on garbage collection performance?
-
-### Idea: Avoid creating new strings in the hot loop
-
-### How
-
-```go
-out = append(out, fields[0])
-```
-
-### Why
-
-Instead of building `"key:length"` strings, store existing strings or simpler values to reduce allocations.
-
----
-
-## Challenge 6
-
-Usage analytics show that most customers send payloads that are half the size represented by the current benchmark. How can we adjust the workload to better reflect real-world traffic and evaluate whether the resulting reduction in allocations improves GC efficiency?
-
-### Idea: Reduce the number of records processed
-
-### How
-
-```go
-payload := strings.Repeat(
-    "name=arm&runtime=go&gc=default&value=12345;",
-    1024,
-)
-```
+## What you've accomplished and what's next
 
-### Why
+In this section, you ran Benchstat to summarize ten benchmark samples and inspected CPU and heap profiles to identify the hottest code paths. You now have a documented default-GC baseline showing operation time, allocation rate, GC frequency, and stop-the-world pause costs on AWS Graviton.
 
-Fewer records means less allocation work and fewer GC cycles.
+In the next section, you'll apply code changes to the benchmark and measure how each change affects these metrics.