Add debuginfos documentation and link to it from debuginfod (#242)

Author: brancz

docs/debuginfod.md

@@ -1,49 +1,28 @@
-# debuginfod client
+# debuginfod support
 
-In order to symbolize ingested profiles, Parca needs to have debug information
-for the binaries that are being profiled. Debug information, also
-referred to as _debuginfos_, can be ELF object files, DWARF debug data, and
-source code. However, sometimes, application packages distributed by various Linux
-distros [strip](https://man7.org/linux/man-pages/man1/strip.1.html) away debug
-information to minimize the size of the binaries. Thankfully, there are publicly accessible
-servers, distributing debug information for various Linux package managers and distributions.
+:::tip
 
-[debuginfod](https://www.mankier.com/8/debuginfod) is an HTTP file server that serves debug
-information to clients based on the build IDs of the binaries. You can find out the build ID
-of a binary using the `file` command on Linux.
+This page assumes familiarity with what debuginfos are. First read the [debuginfos](debuginfos) docs page if you are not already familiar with debuginfos.
 
-Here is an example to find out the build ID of a zsh shell:
+:::
+
+Unfortunately, packages distributed by various Linux distros [strip](https://man7.org/linux/man-pages/man1/strip.1.html) away debuginfos to minimize the size of the binaries.
+
+Thankfully, there are publicly accessible servers, distributing debuginfos for various Linux package managers and distributions.
+
+[debuginfod](https://www.mankier.com/8/debuginfod) is an HTTP file server that serves debuginfos to clients based on the Build IDs of the binaries. You can find out the Build ID of a binary using the `file` command on Linux.
+
+Here is an example to find out the Build ID of a zsh binary:
 
 ```
 $ file /bin/zsh
 
 /bin/zsh: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=24fcd0179bb3aa797de6a570c2359e528f7638c0, for GNU/Linux 3.2.0, stripped
 ```
-Parca integrates with debuginfod to query for upstream debuginfod files and then
-stores them for potential later use. The default debuginfod server used by Parca is
-at https://debuginfod.elfutils.org .
-
-## Implementation
-
-Primarily, Parca looks for the relevant debug information files in its default
-symbol store. However, if debug info files are not found in the symbol store,
-Parca will try to fetch corresponding debuginfo files from the upstream
-[debuginfod servers](https://sourceware.org/elfutils/Debuginfod.html), and store
-them in the Parca symbol store, associated with the unique build ID of the object files.
-
-The symbol store is a wrapper around the [Parca object store](https://www.parca.dev/docs/storage#storing-debug-information)
-to hold debug information. By default, Parca is configured to use the `/tmp`
-directory on local disk. This can be reconfigured to use any other user-specified
-location by editing the Parca configuration file.
-
-The debuginfod client is implemented as a read-through client storage cache.
-An HTTP client is implemented to send requests to the upstream debuginfod servers.
-The client queries the server addresses sequentially until it finds the suitable
-debuginfo files. The downloaded object files are then stored in a `parca/debuginfod`
-bucket with the build ID as the key.
-
-Users can add private debuginfod servers to be queried through the
-` --debuginfod-upstream-servers` flag.
+
+Parca integrates with debuginfod to query for upstream debuginfod files and then stores them for potential later use. The default debuginfod server used by Parca is: https://debuginfod.elfutils.org
+
+To use a different set of debuginfod servers to attempt to retrieve debuginfos from use the `--debuginfod-upstream-servers` flag.
 
 ## Additional Resources
 

docs/debuginfos.md

@@ -0,0 +1,72 @@
+# Debuginfos
+
+Profiling raw data is just memory addresses that represent a function call stack and how often we observed the same stack. For example a function call stack might look like this:
+
+```
+0x0b
+0x2a
+0x43
+```
+
+In order for humans to understand what these memory addresses represent, we need a mapping from memory address to function names. That mapping is what is commonly referred to as "debuginfos".
+
+Debuginfos are in form of sections within an [ELF](https://en.wikipedia.org/wiki/Executable_and_Linkable_Format) binary (ELF is the format of binaries used on Linux). ELF binaries have sections, and some of these sections contain the debuginfos. Most commonly debuginfos are in the [DWARF format](https://dwarfstd.org/doc/DWARF5.pdf).
+
+Let's look at example DWARF of a tiny C program.
+
+```c
+#include <stdio.h>
+int main() {
+   printf("Hello, World!");
+   return 0;
+}
+```
+
+Compile it, enabling DWARF to be emitted (`-g`):
+
+```bash
+zig cc -o mainc -g -target x86_64-linux main.c
+```
+
+> Note: Any C compiler could have been used here but Zig's cross-compile support is very convenient, as it works well on any platform.
+
+And let's use the `dwarfdump` tool to print everything.
+
+```dwarfdump
+$ dwarfdump --show-form mainc
+mainc:  file format elf64-x86-64
+
+.debug_info contents:
+0x00000000: Compile Unit: length = 0x00000047, format = DWARF32, version = 0x0004, abbr_offset = 0x0000, addr_size = 0x08 (next unit at 0x0000004b)
+
+0x0000000b: DW_TAG_compile_unit
+              DW_AT_producer [DW_FORM_strp]     ("Homebrew clang version 13.0.1")
+              DW_AT_language [DW_FORM_data2]    (DW_LANG_C99)
+              DW_AT_name [DW_FORM_strp] ("main.c")
+              DW_AT_stmt_list [DW_FORM_sec_offset]      (0x00000000)
+              DW_AT_comp_dir [DW_FORM_strp]     ("/Users/brancz/src/github.com/polarsignals/polarsignals/pkg/debuginfo/objfile/testdata")
+              DW_AT_low_pc [DW_FORM_addr]       (0x0000000000201e20)
+              DW_AT_high_pc [DW_FORM_data4]     (0x00000016)
+
+0x0000002a:   DW_TAG_subprogram
+                DW_AT_low_pc [DW_FORM_addr]     (0x0000000000201e20)
+                DW_AT_high_pc [DW_FORM_data4]   (0x00000016)
+                DW_AT_frame_base [DW_FORM_exprloc]      (DW_OP_reg6 RBP)
+                DW_AT_GNU_all_call_sites [DW_FORM_flag_present] (true)
+                DW_AT_name [DW_FORM_strp]       ("main")
+                DW_AT_decl_file [DW_FORM_data1] ("/Users/brancz/src/github.com/polarsignals/polarsignals/pkg/debuginfo/objfile/testdata/main.c")
+                DW_AT_decl_line [DW_FORM_data1] (2)
+                DW_AT_type [DW_FORM_ref4]       (0x00000043 "int")
+                DW_AT_external [DW_FORM_flag_present]   (true)
+
+0x00000043:   DW_TAG_base_type
+                DW_AT_name [DW_FORM_strp]       ("int")
+                DW_AT_encoding [DW_FORM_data1]  (DW_ATE_signed)
+                DW_AT_byte_size [DW_FORM_data1] (0x04)
+
+0x0000004a:   NULL
+```
+
+Looking at this output, we see the compilation unit, which is the top level unit, and right underneath it a `DW_TAG_subprogram`, which is our `main` function. It has an attribute called `DW_AT_low_pc` with the form `DW_FORM_addr` (which means it is a `uint64`), that describes the start of our function's memory range, as well as the `DW_AT_high_pc` with the form `DW_FORM_data4` (which is a `int32`), which is the offset from the `DW_AT_low_pc` representing the end of our function's memory range, so the memory range is `[0x201e20, 0x201e36)`. And lastly, important for symbolization is the `DW_AT_name` attribute with the form `DW_FORM_strp`, which is a string.
+
+Essentially what this means for symbolization: Thanks to this entry, we know that if we encountered a memory address between `0x201e20` and `0x201e36`, then it would be the `main` function.

sidebars.js

@@ -38,6 +38,7 @@ module.exports = {
         "ingestion",
         "storage",
         "symbolization",
+        "debuginfos",
         "debuginfod",
         {
           type: "link",

wordlist.txt

@@ -325,3 +325,17 @@ yaml
 Yomi
 Youtube
 zsh
+printf
+Zig's
+dwarfdump
+mainc
+DW
+Homebrew
+objfile
+pc
+stmt
+strp
+testdata
+RBP
+decl
+exprloc