feat: Optimizer works for both Paxos and CAS (#26)

* README includes install instructions

* Network calibration without restarting for different sizes, more Amazon Linux instructions

* Parse process results for network calibrator

* Deploy to us-east-1 for capacity reasons

* Use fewer client machines

* Amazon linux install instructions

* Remove socket counts from network costs, estimated costs extremely low...

* Use musl instead of glibc for compilation, parsing shows process sarstats as well

* Limit networking cores for performance, apply batch limit to regular deployment, sar only tracks core 0

* Decouple analysis correctly accounts for the cardinality of broadcasts

* Partition map insertion fix

* Optimization folder overwriting fix, limit number of unstable throughput runs

* Partitioning fix (network not counted as relevant operator, filtered out operators that fed into relevant operators but we still needed to know what field to partition on)

* Fix CLUSTER_SELF_IDs inserted for decoupling + partitioning

* Fix clients considered bottleneck, calibration after optimization

* Fix clients detected as bottleneck

* Fix partitioning for existing networks where output type and input type differ

* Run rewrites up to budget 10

* Fix partitioning existing network

* CAS fix write throughput 0 for write-only workloads with more clients (writes overridden by None values from election)

* No EDB decoupling

* Benchmarking works for protocols with multiple workloads, simplifications

* CAS fixes: Reads were labeled as blocked if there were no previous writes. Separate networking so decoupling is doesn't need to ping-pong messages.

* Fix send from partitions own ID bug

* Prioritize partitioning over less specific fields for uniform distribution

* Implement hash for partitioning

* Support partitioning on sender ID

* Plotting script

* cargo fmt + clippy

Author: davidchuyaya

.cargo/config.toml

@@ -0,0 +1,24 @@
+[build]
+
+[env]
+# Stack backtraces.
+RUST_BACKTRACE = "1"
+# Set output levels for `tracing` logging.
+# Certain wasm crates emit a lot of `debug` logs, set to `info`.
+RUST_LOG = "info,hydro_lang=debug,dfir_rs=trace,walrus=info,wasm_bindgen_cli_support=info,wasm_bindgen_wasm_interpreter=info"
+# DFIR generate for `/docs/` website.
+DFIR_GENERATE_DOCS = "1"
+DFIR_BASE_DIR = { value = ".", relative = true }
+
+[target.aarch64-apple-darwin]
+linker = "rust-lld"
+
+[target.x86_64-apple-darwin]
+linker = "rust-lld"
+
+[target.x86_64-unknown-linux-musl]
+linker = "rust-lld"
+
+
+[target.x86_64-pc-windows-msvc]
+linker = "rust-lld.exe"

.gitignore

@@ -10,6 +10,7 @@ scripts/*.png
 .hydro
 out*.txt
 benchmark_results/
+result_backup/
 
 # Neovim stuff
 tmux*