Squashed version of all CFI patches

This is being reviewed upstream (see Differential Revisions in each
per-commit message below) but has stalled while we go through the LLVM
RFC process.

The latest version of these patches should not present a large
maintenance burden in amd-stg-open as it no longer relies on certain
function attributes being present in lit tests.

This is the 1st commit message:

Implement DW_CFA_LLVM_* for Heterogeneous Debugging

Summary:
Add support in MC/MIR for writing/parsing, and DebugInfo.

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D76877

This is the commit message #2:

Add SupportsDebugUnwindInformation to MCAsmInfo

Summary:
Generating unwind information is entangled with supporting exceptions,
even when AsmPrinter explicitly recognizes that the unwind tables are
being generated only as debug information.

Add SupportsDebugUnwindInformation as a workaround for targets which do
not have EH support but which do support unwind information for
debugging. This new option only has an effect when the `None` EH model
is specified. The option requests that .debug_frame be generated when
debug info is requested.

Add a new AsmPrinterHandler called UnwindStreamer which just ensures the
proper .cfi_sections and .cfi_startproc/.cfi_endproc directives are
emitted when the option is in effect. This duplicates trivial amounts of
DwarfException, but not enough to make factoring it out helpful.

In the future this could be unified/simplified with the existing EH
support if debug handling is made orthogonal to unwind information
generation.

Subscribers: mgorny, aprantl, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78778

This is the commit message #3:

Implement DW_{OP,AT}_LLVM_* for Heterogeneous Debugging

Summary:
The chosen encoding has some collisions, which are resolved by using the
new augmentation attribute on the compilation unit and in unwind
information CIEs for any target which wishes to use the operations
defined by the extension.

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D76878

This is the commit message #4:

[AMDGPU] Begin emitting CFI for AMDGCN

Summary:
Enable SupportsDebugUnwindInformation for AMDGCN, so we get unwind
information when debug information is requested.

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D76879

This is the commit message #5:

[AMDGPU] Emit entry function CFI

Summary:
Entry functions represent the end of unwinding, as they are the
outer-most frame. This implies they can only have a meaningful
definition for the CFA, which AMDGPU defines using a memory location
description with a literal private address space address. The return
address is set to `undefined` as a sentinel value to signal the end of
unwinding.

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D76880

This is the commit message #6:

[AMDGPU] Skip MetaInstructions in SIInsertWaitcnts

Summary:
CFI emitted during PEI at the beginning of the prologue needs to apply
to any inserted waitcnts on function entry.

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D76881

This is the commit message #7:

[AMDGPU] Implement CFI for non-kernel functions

Summary:
This does not implement CSR spills other than those AMDGPU handles
during PEI.

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D76882

This is the commit message #8:

[AMDGPU] Implement CFI for CSR spills

Summary:
Introduce new SPILL pseudos to allow CFI to be generated for only CSR
spills, and to make ISA-instruction-level accurate information.

Other targets either generate slightly incorrect information or rely on
conventions for how spills are placed within the entry block. The
approach in this change produces larger unwind tables, with the
increased size being spent on additional DW_CFA_advance_location
instructions needed to describe the unwinding accurately.

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D76883

This is the commit message #9:

[AMDGPU] Implement -amdgpu-spill-cfi-saved-regs

Summary:
These spills need special CFI anyway, so implementing them directly
where CFI is emitted avoids the need to invent a mechanism to track them
from ISel.

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D76884

This is the commit message #10:

Imply -amdgpu-spill-cfi-saved-regs with -ggdb for AMDGPU

(cherry picked from commit d02227f)

Change-Id: Idf10a3f6b4fd316c02267e2c74639afe52ad8e40

Author: slinder1

clang/lib/Driver/ToolChains/Clang.cpp

@@ -3733,6 +3733,8 @@ static void RenderDebugOptions(const ToolChain &TC, const Driver &D,
         if (T.getArch() == llvm::Triple::amdgcn) {
           CmdArgs.push_back("-disable-O0-optnone");
           CmdArgs.push_back("-disable-O0-noinline");
+          CmdArgs.push_back("-mllvm");
+          CmdArgs.push_back("-amdgpu-spill-cfi-saved-regs");
           // -ggdb with AMDGCN does not currently compose with options that
           // affect the debug info kind. The behavior of commands like `-ggdb
           // -g` may be surprising (the -g is effectively ignored).

clang/test/Driver/amdgpu-debug.cl

@@ -6,6 +6,7 @@
 // CHECK-SIMPLE-DAG: "-debug-info-kind=line-tables-only"
 // CHECK-SIMPLE-DAG: "-disable-O0-optnone"
 // CHECK-SIMPLE-DAG: "-disable-O0-noinline"
+// CHECK-SIMPLE-DAG: "-mllvm" "-amdgpu-spill-cfi-saved-regs"
 // CHECK-SIMPLE-DAG: "-debugger-tuning=gdb"
 
 // Check that a debug-related option which does not affect the debug-info-kind
@@ -15,6 +16,7 @@
 // CHECK-DWARF2-DAG: "-debug-info-kind=line-tables-only"
 // CHECK-DWARF2-DAG: "-disable-O0-optnone"
 // CHECK-DWARF2-DAG: "-disable-O0-noinline"
+// CHECK-DWARF2-DAG: "-mllvm" "-amdgpu-spill-cfi-saved-regs"
 // CHECK-DWARF2-DAG: "-debugger-tuning=gdb"
 // CHECK-DWARF2-DAG: "-dwarf-version=2"
 
@@ -34,6 +36,7 @@
 // CHECK-LLDBBEFORE-DAG: "-debug-info-kind=line-tables-only"
 // CHECK-LLDBBEFORE-DAG: "-disable-O0-optnone"
 // CHECK-LLDBBEFORE-DAG: "-disable-O0-noinline"
+// CHECK-LLDBBEFORE-DAG: "-mllvm" "-amdgpu-spill-cfi-saved-regs"
 // CHECK-LLDBBEFORE-DAG: "-debugger-tuning=gdb"
 // RUN: %clang -### -target amdgcn-amd-amdhsa -x cl -c -emit-llvm -ggdb -glldb %s 2>&1 | FileCheck -check-prefix=CHECK-LLDBAFTER %s
 // CHECK-LLDBAFTER: "-cc1"

clang/test/Driver/hip-debug.hip

@@ -12,12 +12,15 @@
 // CHECK-DAG: "-debug-info-kind=line-tables-only"
 // CHECK-DAG: "-disable-O0-optnone"
 // CHECK-DAG: "-disable-O0-noinline"
+// CHECK-DAG: "-mllvm" "-amdgpu-spill-cfi-saved-regs"
 // CHECK-DAG: "-debugger-tuning=gdb"
 // CHECK-LABEL: clang-offload-bundler
 // CHECK: {{.*}}clang{{.*}}"-triple" "x86_64-unknown-linux-gnu"
 // CHECK-NOT: "-disable-O0-optnone"
 // CHECK-NOT: "-disable-O0-noinline"
+// CHECK-NOT: "-amdgpu-spill-cfi-saved-regs"
 // CHECK-DAG: "-debug-info-kind=limited"
 // CHECK-DAG: "-debugger-tuning=gdb"
 // CHECK-NOT: "-disable-O0-optnone"
 // CHECK-NOT: "-disable-O0-noinline"
+// CHECK-NOT: "-amdgpu-spill-cfi-saved-regs"

llvm/include/llvm/BinaryFormat/Dwarf.def

@@ -418,6 +418,13 @@ HANDLE_DW_AT(0x3e02, LLVM_sysroot, 0, LLVM)
 HANDLE_DW_AT(0x3e03, LLVM_tag_offset, 0, LLVM)
 // The missing numbers here are reserved for ptrauth support.
 HANDLE_DW_AT(0x3e07, LLVM_apinotes, 0, APPLE)
+// Heterogeneous Debugging Extension defined at
+// https://llvm.org/docs/AMDGPUDwarfProposalForHeterogeneousDebugging.html.
+HANDLE_DW_AT(0x3e08, LLVM_active_lane, 0, LLVM)
+HANDLE_DW_AT(0x3e09, LLVM_augmentation, 0, LLVM)
+HANDLE_DW_AT(0x3e0a, LLVM_lanes, 0, LLVM)
+HANDLE_DW_AT(0x3e0b, LLVM_lane_pc, 0, LLVM)
+HANDLE_DW_AT(0x3e0c, LLVM_vector_size, 0, LLVM)
 
 // Apple extensions.
 
@@ -669,6 +676,23 @@ HANDLE_DW_OP(0xf3, GNU_entry_value, 0, GNU)
 // Extensions for Fission proposal.
 HANDLE_DW_OP(0xfb, GNU_addr_index, 0, GNU)
 HANDLE_DW_OP(0xfc, GNU_const_index, 0, GNU)
+// Heterogeneous Debugging Extension defined at
+// https://llvm.org/docs/AMDGPUDwarfProposalForHeterogeneousDebugging.html.
+// These collide with some HP and PGI vendor extensions, but this ambiguity is
+// resolved by ensuring CIE augmentation strings and compilation unit
+// DW_AT_LLVM_augmentation strings include "[llvm:v0.0]".
+HANDLE_DW_OP(0xe1, LLVM_form_aspace_address, 0, LLVM)
+HANDLE_DW_OP(0xe2, LLVM_push_lane, 0, LLVM)
+HANDLE_DW_OP(0xe3, LLVM_offset, 0, LLVM)
+HANDLE_DW_OP(0xe4, LLVM_offset_uconst, 0, LLVM)
+HANDLE_DW_OP(0xe5, LLVM_bit_offset, 0, LLVM)
+HANDLE_DW_OP(0xe6, LLVM_call_frame_entry_reg, 0, LLVM)
+HANDLE_DW_OP(0xe7, LLVM_undefined, 0, LLVM)
+HANDLE_DW_OP(0xe8, LLVM_aspace_bregx, 0, LLVM)
+HANDLE_DW_OP(0xe9, LLVM_aspace_implicit_pointer, 0, LLVM)
+HANDLE_DW_OP(0xea, LLVM_piece_end, 0, LLVM)
+HANDLE_DW_OP(0xeb, LLVM_extend, 0, LLVM)
+HANDLE_DW_OP(0xec, LLVM_select_bit_piece, 0, LLVM)
 
 // DWARF languages.
 HANDLE_DW_LANG(0x0001, C89, 0, 2, DWARF)
@@ -895,6 +919,10 @@ HANDLE_DW_CFA_PRED(0x1d, MIPS_advance_loc8, SELECT_MIPS64)
 HANDLE_DW_CFA_PRED(0x2d, GNU_window_save, SELECT_SPARC)
 HANDLE_DW_CFA_PRED(0x2d, AARCH64_negate_ra_state, SELECT_AARCH64)
 HANDLE_DW_CFA_PRED(0x2e, GNU_args_size, SELECT_X86)
+// Heterogeneous Debugging Extension defined at
+// https://llvm.org/docs/AMDGPUDwarfProposalForHeterogeneousDebugging.html
+HANDLE_DW_CFA(0x30, LLVM_def_aspace_cfa)
+HANDLE_DW_CFA(0x31, LLVM_def_aspace_cfa_sf)
 
 // Apple Objective-C Property Attributes.
 // Keep this list in sync with clang's DeclSpec.h ObjCPropertyAttributeKind!

llvm/include/llvm/DebugInfo/DWARF/DWARFDebugFrame.h

@@ -98,6 +98,15 @@ class CFIProgram {
     Instructions.back().Ops.push_back(Operand2);
   }
 
+  /// Add a new instruction that has three operands.
+  void addInstruction(uint8_t Opcode, uint64_t Operand1, uint64_t Operand2,
+                      uint64_t Operand3) {
+    Instructions.push_back(Instruction(Opcode));
+    Instructions.back().Ops.push_back(Operand1);
+    Instructions.back().Ops.push_back(Operand2);
+    Instructions.back().Ops.push_back(Operand3);
+  }
+
   /// Types of operands to CFI instructions
   /// In DWARF, this type is implicitly tied to a CFI instruction opcode and
   /// thus this type doesn't need to be explictly written to the file (this is
@@ -113,12 +122,13 @@ class CFIProgram {
     OT_SignedFactDataOffset,
     OT_UnsignedFactDataOffset,
     OT_Register,
+    OT_AddressSpace,
     OT_Expression
   };
 
   /// Retrieve the array describing the types of operands according to the enum
   /// above. This is indexed by opcode.
-  static ArrayRef<OperandType[2]> getOperandTypes();
+  static ArrayRef<OperandType[3]> getOperandTypes();
 
   /// Print \p Opcode's operand number \p OperandIdx which has value \p Operand.
   void printOperand(raw_ostream &OS, const MCRegisterInfo *MRI, bool IsEH,

llvm/include/llvm/MC/MCAsmInfo.h

@@ -353,6 +353,11 @@ class MCAsmInfo {
   /// false.
   bool SupportsDebugInformation = false;
 
+  /// True if target supports emitting .debug_frame unwind information when
+  /// ExceptionsType = ExceptionHandling::None and debug info is requested.
+  /// Defaults to false.
+  bool SupportsDebugUnwindInformation = false;
+
   /// Exception handling format for the target.  Defaults to None.
   ExceptionHandling ExceptionsType = ExceptionHandling::None;
 
@@ -379,6 +384,11 @@ class MCAsmInfo {
   /// location is allowed.
   bool SupportsExtendedDwarfLocDirective = true;
 
+  /// True if the target supports the extensions defined at
+  /// https://llvm.org/docs/AMDGPUDwarfProposalForHeterogeneousDebugging.html.
+  /// Defaults to false.
+  bool SupportsHeterogeneousDebuggingExtensions = false;
+
   //===--- Prologue State ----------------------------------------------===//
 
   std::vector<MCCFIInstruction> InitialFrameState;
@@ -609,6 +619,10 @@ class MCAsmInfo {
 
   bool doesSupportDebugInformation() const { return SupportsDebugInformation; }
 
+  bool doesSupportDebugUnwindInformation() const {
+    return SupportsDebugUnwindInformation;
+  }
+
   bool doesSupportExceptionHandling() const {
     return ExceptionsType != ExceptionHandling::None;
   }
@@ -643,6 +657,9 @@ class MCAsmInfo {
   bool supportsExtendedDwarfLocDirective() const {
     return SupportsExtendedDwarfLocDirective;
   }
+  bool supportsHeterogeneousDebuggingExtensions() const {
+    return SupportsHeterogeneousDebuggingExtensions;
+  }
 
   void addInitialFrameState(const MCCFIInstruction &Inst);
 

llvm/include/llvm/MC/MCDwarf.h

@@ -444,6 +444,7 @@ class MCCFIInstruction {
     OpRememberState,
     OpRestoreState,
     OpOffset,
+    OpLLVMDefAspaceCfa,
     OpDefCfaRegister,
     OpDefCfaOffset,
     OpDefCfa,
@@ -466,19 +467,25 @@ class MCCFIInstruction {
     int Offset;
     unsigned Register2;
   };
+  unsigned AddressSpace;
   std::vector<char> Values;
 
   MCCFIInstruction(OpType Op, MCSymbol *L, unsigned R, int O, StringRef V)
       : Operation(Op), Label(L), Register(R), Offset(O),
         Values(V.begin(), V.end()) {
-    assert(Op != OpRegister);
+    assert(Op != OpRegister && Op != OpLLVMDefAspaceCfa);
   }
 
   MCCFIInstruction(OpType Op, MCSymbol *L, unsigned R1, unsigned R2)
       : Operation(Op), Label(L), Register(R1), Register2(R2) {
     assert(Op == OpRegister);
   }
 
+  MCCFIInstruction(OpType Op, MCSymbol *L, unsigned R, int O, unsigned AS)
+      : Operation(Op), Label(L), Register(R), Offset(O), AddressSpace(AS) {
+    assert(Op == OpLLVMDefAspaceCfa);
+  }
+
 public:
   /// .cfi_def_cfa defines a rule for computing CFA as: take address from
   /// Register and add Offset to it.
@@ -507,6 +514,17 @@ class MCCFIInstruction {
     return MCCFIInstruction(OpAdjustCfaOffset, L, 0, Adjustment, "");
   }
 
+  // FIXME: Update the remaining docs to use the new proposal wording.
+  /// .cfi_llvm_def_aspace_cfa defines the rule for computing the CFA to
+  /// be the result of evaluating the DWARF operation expression
+  /// `DW_OP_constu AS; DW_OP_aspace_bregx R, B` as a location description.
+  static MCCFIInstruction createLLVMDefAspaceCfa(MCSymbol *L, unsigned Register,
+                                                 int Offset,
+                                                 unsigned AddressSpace) {
+    return MCCFIInstruction(OpLLVMDefAspaceCfa, L, Register, -Offset,
+                            AddressSpace);
+  }
+
   /// .cfi_offset Previous value of Register is saved at offset Offset
   /// from CFA.
   static MCCFIInstruction createOffset(MCSymbol *L, unsigned Register,
@@ -586,7 +604,8 @@ class MCCFIInstruction {
     assert(Operation == OpDefCfa || Operation == OpOffset ||
            Operation == OpRestore || Operation == OpUndefined ||
            Operation == OpSameValue || Operation == OpDefCfaRegister ||
-           Operation == OpRelOffset || Operation == OpRegister);
+           Operation == OpRelOffset || Operation == OpRegister ||
+           Operation == OpLLVMDefAspaceCfa);
     return Register;
   }
 
@@ -595,10 +614,16 @@ class MCCFIInstruction {
     return Register2;
   }
 
+  unsigned getAddressSpace() const {
+    assert(Operation == OpLLVMDefAspaceCfa);
+    return AddressSpace;
+  }
+
   int getOffset() const {
     assert(Operation == OpDefCfa || Operation == OpOffset ||
            Operation == OpRelOffset || Operation == OpDefCfaOffset ||
-           Operation == OpAdjustCfaOffset || Operation == OpGnuArgsSize);
+           Operation == OpAdjustCfaOffset || Operation == OpGnuArgsSize ||
+           Operation == OpLLVMDefAspaceCfa);
     return Offset;
   }
 

llvm/include/llvm/MC/MCStreamer.h

@@ -942,6 +942,8 @@ class MCStreamer {
   virtual void emitCFIDefCfa(int64_t Register, int64_t Offset);
   virtual void emitCFIDefCfaOffset(int64_t Offset);
   virtual void emitCFIDefCfaRegister(int64_t Register);
+  virtual void emitCFILLVMDefAspaceCfa(int64_t Register, int64_t Offset,
+                                       int64_t AddressSpace);
   virtual void emitCFIOffset(int64_t Register, int64_t Offset);
   virtual void emitCFIPersonality(const MCSymbol *Sym, unsigned Encoding);
   virtual void emitCFILsda(const MCSymbol *Sym, unsigned Encoding);

llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp

@@ -14,6 +14,7 @@
 #include "CodeViewDebug.h"
 #include "DwarfDebug.h"
 #include "DwarfException.h"
+#include "UnwindStreamer.h"
 #include "WasmException.h"
 #include "WinCFGuard.h"
 #include "WinException.h"
@@ -139,6 +140,8 @@ static const char *const DWARFGroupName = "dwarf";
 static const char *const DWARFGroupDescription = "DWARF Emission";
 static const char *const DbgTimerName = "emit";
 static const char *const DbgTimerDescription = "Debug Info Emission";
+static const char *const UnwindTimerName = "write_unwind";
+static const char *const UnwindTimerDescription = "DWARF Unwind Writer";
 static const char *const EHTimerName = "write_exception";
 static const char *const EHTimerDescription = "DWARF Exception Writer";
 static const char *const CFGuardName = "Control Flow Guard";
@@ -328,6 +331,16 @@ bool AsmPrinter::doInitialization(Module &M) {
     }
   }
 
+  if (MMI->hasDebugInfo() &&
+      MAI->getExceptionHandlingType() == ExceptionHandling::None &&
+      MAI->doesSupportDebugUnwindInformation()) {
+    isCFIMoveForDebugging = true;
+    Handlers.emplace_back(std::make_unique<UnwindStreamer>(this),
+                          UnwindTimerName, UnwindTimerDescription,
+                          DWARFGroupName, DWARFGroupDescription);
+    return false;
+  }
+
   switch (MAI->getExceptionHandlingType()) {
   case ExceptionHandling::SjLj:
   case ExceptionHandling::DwarfCFI:
@@ -995,7 +1008,8 @@ bool AsmPrinter::needsSEHMoves() {
 
 void AsmPrinter::emitCFIInstruction(const MachineInstr &MI) {
   ExceptionHandling ExceptionHandlingType = MAI->getExceptionHandlingType();
-  if (ExceptionHandlingType != ExceptionHandling::DwarfCFI &&
+  if (!MAI->doesSupportDebugUnwindInformation() &&
+      ExceptionHandlingType != ExceptionHandling::DwarfCFI &&
       ExceptionHandlingType != ExceptionHandling::ARM)
     return;
 

llvm/lib/CodeGen/AsmPrinter/AsmPrinterDwarf.cpp

@@ -221,6 +221,10 @@ void AsmPrinter::emitCFIInstruction(const MCCFIInstruction &Inst) const {
   case MCCFIInstruction::OpDefCfaRegister:
     OutStreamer->emitCFIDefCfaRegister(Inst.getRegister());
     break;
+  case MCCFIInstruction::OpLLVMDefAspaceCfa:
+    OutStreamer->emitCFILLVMDefAspaceCfa(Inst.getRegister(), Inst.getOffset(),
+                                         Inst.getAddressSpace());
+    break;
   case MCCFIInstruction::OpOffset:
     OutStreamer->emitCFIOffset(Inst.getRegister(), Inst.getOffset());
     break;