Functionality that will be shared, moved from TraceStream.cc

- Moved into util.cc
- Added forward_to to skip trace data to some arbitrary point in time

Getters required to expose data

We need to be able to expose this data so it can
be serialized.

Find original exe for ReplayTask

Digs out original executable image that this task was forked
from, or in the case of exec, exec'd on.

This is required for persistent checkpointing, so that the names in the
proc fs corresponds to a correct name at replay time (i.e. has the same
behavior/looks the same in proc fs as a normal replay). The thread name is
not what should be showing up in /proc/tid/comm, but the actual
executable. So we need to be able to find this "original exe" of the
task.

Check if Event is checkpointable

Required for the create checkpoints command, etc. to determine what
events in the trace are checkpointable, when not having a live session.

In future commits/PRs, remove the static function in ReplaySession.cc`
that does the same thing and use this member function on Event instead.

Additional proc fs query paths

Gets additional proc fs paths for a task, in this case
/mem. Required for persistent checkpointing to figure out
on how to handle mappings and what to serialize (and what not to
serialize).

Lifted CloneCompletion out of Session

The function extract_name will also be required for setting up syscall
buffer stuff in coming commits.

Getters/setters required for PCP

Need to be able to set this data when restoring an address space.

Persistent checkpointing

Added persistent checkpoint schema for capnproto rr_pcp.capnp,
as well a compile command for it in CMakeLists.txt, that works like
the other one (rr_trace.capnp)

CheckpointInfo and MarkData types works as intermediaries between a
serialized checkpoint and a deserialized "live" one. MarkData is used for
copying the contents of Mark, InternalMark, ProtoMark and it's various
data into, for serialization as well when deserializing, to reconstruct
those types.

The reasoning for adding MarkData is to not intrude in Mark/InternalMark/ProtoMark
interface and possibly break some guarantees or invariants they provide.
If something goes wrong now, it's constrained only to persistent
checkpointing not reconstituting a session properly.

GDB spawned by RR now has 2 additional commands, write-checkpoints, which
serializes any checkpoints set by the `checkpoint` command and
load-checkpoints.

Added the rr create-checkpoints command which create persistent checkpoints
on a specified interval, which it attempts to honor as closely as possible.

RerunCommand and ReplayCommand are now aware of PCPs.

Replay sessions get spawned from persistent checkpoints if they
exist on disk when using `-g <evt>` or when using `-f <pid>` and that
"task" was created some time after a persistent checkpoint.

Added the --ignore-pcp flag to these commands, which ignores pcps
and spawns sessions normally.

fixup for can_checkpoint_at

Restored comments, that existed in static function in ReplaySession.cc
Change all use of this to Event::can_checkpoint_at
Removed static can_checkpoint_at in ReplaySession.cc

Fix preferred include & unnecessary check for partial init

Since checkpoints are partially initialized, checking that they are is pointless.

Added cmake command looping over trace files per request by @khuey

remove init check of member variables.

Move extract_name from Session into util.h.

Removed stream_util, moved contents to util.h

make ignore-pcp not take up '-i'

Moved responsibility of de/ser into FdTable and FileMonitor

Deserializing and serializing an FdTable is now performed by the class itself instead of in a free function

FileMonitor has a public member function that is used for serialization.
Each derived type that requires special/additional logic, extends
the virtual member function serialize_type.

Remove skipMonitoringMappedFd

not necessary for serialization, as FdTable is separately restored.

Refactor task OS-name setting

Task::copy_state sets the OS name of a task in the same fashion that
persistent checkpointing sets name. Refactored this functionality into
Task::set_name.

Also removed the unnecessary `update_prname` from Task::copy_state.

update_prname is not a "write to tracee"-operation but a "read from tracee"-operation; and since
we already know what name we want to set Task::prname to, we skip this reading from the tracee
in Task::copy_state and just set it to the parameter passed in to Task::set_name

Add const qualifier

Fixes #3678

Refactor so that marks_with_checkpoints is just changed in one place, not arbitrarily access it. Ref counts had the same changes in a previous commit.

Fixes a bug for loaded persistent checkpoints where the re-created checkpoints did not get their reference counting correct.

This closes #3678

Fixed pread-related bug

pread may or may not read the requested size. This was not
taken into account for. If the data is large,
we probably can't read it, in one go.

Fixed signed overflow

Added the first test for persistent checkpoints

Address missing arch() serialization

Previous work assumed everything as native. That doesn't work when
debugging a 32-bit mode application on a 64-bit machine. That means some
`SupportedArch` fields needs to be serialized as well, so that if the
process was a 32-bit application it gets re-created correctly. This was
instantly made visible by the provided test (although it did not test it
explicitly).

Fix issues so that ARM test passes

Changed test

On arm, we start in a different place than _start (some
assembly, possibly handrolled), which made setting a breakpoint at 48
wrong, so had to put the filename:line number.

Tests should pass now.

`create-checkpoints` should handle "latest-trace"

Create checkpoints command need the actual trace dir resolved, so we let `TraceReader` handle that for us.

Author: theIDinside

CMakeLists.txt

@@ -582,17 +582,26 @@ endforeach(generated_file)
 
 add_custom_target(Generated DEPENDS ${GENERATED_FILES})
 
-add_custom_command(OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/rr_trace.capnp.c++"
-                          "${CMAKE_CURRENT_BINARY_DIR}/rr_trace.capnp.h"
-                   COMMAND capnp compile
-                           "--src-prefix=${CMAKE_CURRENT_SOURCE_DIR}/src"
-                           "-oc++:${CMAKE_CURRENT_BINARY_DIR}"
-                           "${CMAKE_CURRENT_SOURCE_DIR}/src/rr_trace.capnp"
-                   DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/src/rr_trace.capnp")
-set_source_files_properties("${CMAKE_CURRENT_BINARY_DIR}/rr_trace.capnp.c++"
-                            PROPERTIES GENERATED true)
-set_source_files_properties("${CMAKE_CURRENT_BINARY_DIR}/rr_trace.capnp.h"
-                            PROPERTIES GENERATED true HEADER_FILE_ONLY true)
+
+set(CAPNP_FILES
+  rr_trace
+  rr_pcp
+)
+
+# Compile capnproto files
+foreach(capnp_file ${CAPNP_FILES})
+add_custom_command(OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/${capnp_file}.capnp.c++"
+                            "${CMAKE_CURRENT_BINARY_DIR}/${capnp_file}.capnp.h"
+                     COMMAND capnp compile
+                             "--src-prefix=${CMAKE_CURRENT_SOURCE_DIR}/src"
+                             "-oc++:${CMAKE_CURRENT_BINARY_DIR}"
+                             "${CMAKE_CURRENT_SOURCE_DIR}/src/${capnp_file}.capnp"
+                     DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/src/${capnp_file}.capnp")
+  set_source_files_properties("${CMAKE_CURRENT_BINARY_DIR}/${capnp_file}.capnp.c++"
+                              PROPERTIES GENERATED true)
+  set_source_files_properties("${CMAKE_CURRENT_BINARY_DIR}/${capnp_file}.capnp.h"
+                              PROPERTIES GENERATED true HEADER_FILE_ONLY true)
+endforeach()
 
 if (${CMAKE_SYSTEM_PROCESSOR} STREQUAL "aarch64")
   set(BLAKE_ARCH_DIR third-party/blake2/neon)
@@ -604,13 +613,15 @@ set(RR_SOURCES
   src/AddressSpace.cc
   src/AutoRemoteSyscalls.cc
   src/BuildidCommand.cc
+  src/CheckpointInfo.cc
   src/Command.cc
   src/CompressedReader.cc
   src/CompressedWriter.cc
   src/ContextSwitchEvent.cc
   src/CPUFeaturesCommand.cc
   src/CPUIDBugDetector.cc
   src/CPUs.cc
+  src/CreateCheckpointsCommand.cc
   src/DiversionSession.cc
   src/DumpCommand.cc
   src/Dwarf.cc
@@ -648,6 +659,7 @@ set(RR_SOURCES
   src/PackCommand.cc
   src/PerfCounters.cc
   src/PerfCounterBuffers.cc
+  src/PersistentCheckpointing.cc
   src/PidFdMonitor.cc
   src/processor_trace_check.cc
   src/ProcFdDirMonitor.cc
@@ -688,6 +700,7 @@ set(RR_SOURCES
   src/WaitManager.cc
   src/WaitStatus.cc
   ${CMAKE_CURRENT_BINARY_DIR}/rr_trace.capnp.c++
+  ${CMAKE_CURRENT_BINARY_DIR}/rr_pcp.capnp.c++
   ${BLAKE_ARCH_DIR}/blake2b.c
 )
 
@@ -1542,6 +1555,7 @@ set(TESTS_WITH_PROGRAM
   # check_session_leaks
   checkpoint_dying_threads
   checkpoint_mixed_mode
+  checkpoint_persistent_shmem
   checksum_sanity
   check_lost_interrupts
   clone_file_range

src/AddressSpace.cc

@@ -554,6 +554,11 @@ void AddressSpace::save_auxv(Task* t) {
   save_interpreter_base(t, saved_auxv());
 }
 
+void AddressSpace::restore_auxv(Task* t, std::vector<uint8_t>&& auxv) {
+  saved_auxv_ = std::move(auxv);
+  save_interpreter_base(t, saved_auxv());
+}
+
 void AddressSpace::save_interpreter_base(Task* t, std::vector<uint8_t> auxv) {
   saved_interpreter_base_ = read_interpreter_base(auxv);
   save_ld_path(t, saved_interpreter_base());

src/AddressSpace.h

@@ -662,6 +662,14 @@ class AddressSpace : public HasTaskSet {
    * Dies if no shm size is registered for the address.
    */
   size_t get_shm_size(remote_ptr<void> addr) { return shm_sizes[addr]; }
+
+  /**
+   * Check if `map` is shared memory
+   */
+  bool has_shm_at(const KernelMapping& map) const {
+    return shm_sizes.find(map.start()) != std::cend(shm_sizes);
+  }
+
   void remove_shm_size(remote_ptr<void> addr) { shm_sizes.erase(addr); }
 
   /**
@@ -801,6 +809,9 @@ class AddressSpace : public HasTaskSet {
   const std::vector<uint8_t>& saved_auxv() { return saved_auxv_; }
   void save_auxv(Task* t);
 
+  /* Used when restoring persistent checkpoints. */
+  void restore_auxv(Task* t, std::vector<uint8_t>&& auxv);
+
   remote_ptr<void> saved_interpreter_base() { return saved_interpreter_base_; }
   void save_interpreter_base(Task* t, std::vector<uint8_t> auxv);
 
@@ -879,6 +890,15 @@ class AddressSpace : public HasTaskSet {
 
   bool legacy_breakpoint_mode() { return stopping_breakpoint_table_ != nullptr; }
   remote_code_ptr do_breakpoint_fault_addr() { return do_breakpoint_fault_addr_; }
+
+  void set_breakpoint_fault_addr(remote_code_ptr addr) {
+    do_breakpoint_fault_addr_ = addr;
+  }
+
+  void set_uses_syscall_buffer(bool uses_syscall_buffer = true) {
+    syscallbuf_enabled_ = uses_syscall_buffer;
+  }
+
   remote_code_ptr stopping_breakpoint_table() { return stopping_breakpoint_table_; }
   int stopping_breakpoint_table_entry_size() { return stopping_breakpoint_table_entry_size_; }
 

src/BpfMapMonitor.h

@@ -14,12 +14,19 @@ class BpfMapMonitor : public FileMonitor {
 public:
   BpfMapMonitor(uint64_t key_size, uint64_t value_size) : key_size_(key_size), value_size_(value_size) {}
 
-  virtual Type type() override { return BpfMap; }
+  virtual Type type() const override { return BpfMap; }
 
   uint64_t key_size() const { return key_size_; }
   uint64_t value_size() const { return value_size_; }
 
 private:
+  virtual void serialize_type(
+      pcp::FileMonitor::Builder& builder) const noexcept override {
+    auto bpf = builder.initBpf();
+    bpf.setKeySize(key_size_);
+    bpf.setValueSize(value_size_);
+  }
+
   uint64_t key_size_;
   uint64_t value_size_;
 };