Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
125 commits
Select commit Hold shift + click to select a range
536c8dd
Added EG_ATTEMPT and EG_SUCCESS record instances to SingleHeralded
Tw1ZZLER Jun 16, 2026
c5d7e45
MVP of metrics.py
Tw1ZZLER Jun 16, 2026
b8d3f6d
Timeline will register itself as the metrics time provider
Tw1ZZLER Jun 16, 2026
4e8163c
Added tests for metrics (passed)
Tw1ZZLER Jun 16, 2026
1ffd8b1
Changed EG_ATTEMPT to EG_FAILURE
Tw1ZZLER Jun 16, 2026
9118746
Edited metric tests to reflect change to EG_FAILURE
Tw1ZZLER Jun 16, 2026
4f30c4f
Initial entanglement generation experiment to test metrics
Tw1ZZLER Jun 16, 2026
9566e01
Helpful experiment instructions
Tw1ZZLER Jun 16, 2026
a22b1d3
Added support for directly recording throughput from RequestApp
Tw1ZZLER Jun 16, 2026
abb435b
Implemented collect and aggregate trial metrics methods
Tw1ZZLER Jun 16, 2026
0b35df5
Metrics tests for throughput
Tw1ZZLER Jun 16, 2026
a5b4466
Updated experiment to use new collect and aggregate trial metrics met…
Tw1ZZLER Jun 16, 2026
2a9d19c
Adjust throughput to only be recorded by responder
Tw1ZZLER Jun 17, 2026
d7ede60
Avoid NaN values getting returned as throughput
Tw1ZZLER Jun 17, 2026
dfbf1b5
Metrics throughput tests
Tw1ZZLER Jun 17, 2026
01b26c4
Use isfinite to filter out non-finite values such as NaN
Tw1ZZLER Jun 17, 2026
93853c5
Fix for events that are exactly at prep_time + collect_time
Tw1ZZLER Jun 17, 2026
e8fcb12
Extra comments for counters
Tw1ZZLER Jun 17, 2026
9c9d90d
Added EP successes and failures
Tw1ZZLER Jun 18, 2026
118247e
Adjusted test functions for EP
Tw1ZZLER Jun 18, 2026
9a42caf
Metrics logging for BBPSSW
Tw1ZZLER Jun 18, 2026
05089c8
Multiprocessing for eg_experiment
Tw1ZZLER Jun 18, 2026
f74f7bf
Purified delivery metrics
Tw1ZZLER Jun 18, 2026
de2282e
Tests for purified delivery metrics
Tw1ZZLER Jun 18, 2026
efee584
Remove eg_experiment from examples
Tw1ZZLER Jun 18, 2026
773bb1f
Revert "Avoid NaN values getting returned as throughput"
Tw1ZZLER Jun 18, 2026
7093f4b
Reworked Event Types, new file for interface
Tw1ZZLER Jun 19, 2026
3945fb4
Metric registry to make public facing interface for adding new metric…
Tw1ZZLER Jun 19, 2026
bdb9676
Public interface now in __init__ of package
Tw1ZZLER Jun 19, 2026
d52bf43
Update tests for new metrics features
Tw1ZZLER Jun 19, 2026
414707f
Removed EventRecordsMetric
Tw1ZZLER Jun 22, 2026
928fc19
Moved entanglement fail to superclass
Tw1ZZLER Jun 22, 2026
ae75886
Support for BarretKok
Tw1ZZLER Jun 22, 2026
cbd5b00
Revert changes to request_app
Tw1ZZLER Jun 22, 2026
84e5a18
Removed __all__
Tw1ZZLER Jun 22, 2026
3fab5bc
Renamed app ep time to delivery time
Tw1ZZLER Jun 22, 2026
b841a60
Remove throughput related tests
Tw1ZZLER Jun 22, 2026
297bd9e
Added TODO comment to DeliveryTimeMetric
Tw1ZZLER Jun 22, 2026
9f31187
Potential fix for pull request finding 'Unused import'
Tw1ZZLER Jun 22, 2026
864e297
More unused imports
Tw1ZZLER Jun 22, 2026
6ef03a6
Added Swapping equivalents of failures, successes, fidelities metrics
Tw1ZZLER Jun 24, 2026
b11ffb7
Equivalent test functions for new swapping metrics
Tw1ZZLER Jun 24, 2026
2c2bebb
Finish metrics recording for purification
Tw1ZZLER Jun 24, 2026
22b1b28
Implemented metrics recording for swapping
Tw1ZZLER Jun 24, 2026
79c3916
Improved way of dealing with reservation_start_time
Tw1ZZLER Jun 25, 2026
da2f52c
New ReservationDeliveryMetric metric type
Tw1ZZLER Jun 25, 2026
e12055b
Added reservation APPROVED, REJECTED, and COMPLETED metrics
Tw1ZZLER Jun 25, 2026
f3f85fb
Reservation approved and rejected metrics in network management
Tw1ZZLER Jun 25, 2026
41ef537
A built-in reservation data collection function
Tw1ZZLER Jun 25, 2026
2dd7286
Collection reservation data test method
Tw1ZZLER Jun 25, 2026
0a5bff7
Fixed start time not being recorded
Tw1ZZLER Jun 25, 2026
8e27162
Reservation APPROVED and REJECTED tests
Tw1ZZLER Jun 25, 2026
fcf0bc8
New Purified Delivery test method for pair index
Tw1ZZLER Jun 25, 2026
a3a4c5f
Merge branch 'master' into metrics
hayekr Jun 26, 2026
ea6e172
Changed ruff formatter line length to 120
Tw1ZZLER Jun 29, 2026
25a87a8
Reformatted metrics files with line length 120
Tw1ZZLER Jun 29, 2026
ac36b98
Removed some trailing commas and reformatted so args fit on one line
Tw1ZZLER Jun 29, 2026
f58c12f
Renamed most of the metrics to better names
Tw1ZZLER Jun 29, 2026
1a075a7
Throughput is now collected only on collect_trial_metrics
Tw1ZZLER Jun 29, 2026
c4dac30
Moved and updated EventTypes to its own namespaces
Tw1ZZLER Jun 29, 2026
e635e7a
Updated tests with new names and events
Tw1ZZLER Jun 29, 2026
638e4d8
Circuit swapping updated to include more node information
Tw1ZZLER Jun 29, 2026
af2ac51
BDS swapping updated with more information
Tw1ZZLER Jun 29, 2026
1afcc44
Circuit BBPSSW updated with more information and new names
Tw1ZZLER Jun 29, 2026
07402e5
BDS BBPSSW updated with more information
Tw1ZZLER Jun 29, 2026
316207b
Generation updated to include more information and new names
Tw1ZZLER Jun 29, 2026
4fe3d68
Added doc strings for storage methods
Tw1ZZLER Jun 30, 2026
ef41a9c
Added docstrings for registry methods
Tw1ZZLER Jun 30, 2026
b1585ff
Added docstrings for metric types methods
Tw1ZZLER Jun 30, 2026
f53194b
Added docstrings for event types methods
Tw1ZZLER Jun 30, 2026
954b87f
Added doc strings for builtins methods
Tw1ZZLER Jun 30, 2026
9697bd1
Added docstrings for init methods
Tw1ZZLER Jun 30, 2026
552ae76
Improve system time provider (now returns picoseconds properly)
Tw1ZZLER Jul 1, 2026
b89ffb9
Replace ellipsis with exception
Tw1ZZLER Jul 1, 2026
0e98bf7
Remove TimeProvider abstraction, provide Timeline directly
Tw1ZZLER Jul 1, 2026
91b3e63
Fix test metrics to use timeline time provider
Tw1ZZLER Jul 1, 2026
79c0fbc
Potential fix for pull request finding 'Unused local variable'
Tw1ZZLER Jul 1, 2026
c3830c1
Type checking guard
Tw1ZZLER Jul 1, 2026
9ccfa8d
Revert "Changed ruff formatter line length to 120"
Tw1ZZLER Jul 1, 2026
7169459
Target pairs defaults to None
Tw1ZZLER Jul 1, 2026
93c420a
Target pairs defaults to None
Tw1ZZLER Jul 1, 2026
861502f
Create a better dummy timeline for the tests
Tw1ZZLER Jul 1, 2026
695b960
Fixed target pairs logic check in DeliveryTimeMetric
Tw1ZZLER Jul 1, 2026
a9fe29a
Better descriptions for optional metric methods (+pass)
Tw1ZZLER Jul 1, 2026
ad15cd6
Mean and STD method provided to clean up aggregation code
Tw1ZZLER Jul 1, 2026
28eb062
Moved BUILTIN_METRICS scope
Tw1ZZLER Jul 1, 2026
e08ef9d
Converted `is` keyword to == to prevent missed matches
Tw1ZZLER Jul 1, 2026
2a0af52
Converted `is` keyword to == to prevent missed matches
Tw1ZZLER Jul 1, 2026
c9b7913
Converted `is` keyword to == to prevent missed matches
Tw1ZZLER Jul 1, 2026
34f40ba
Converted `is` keyword to == to prevent missed matches
Tw1ZZLER Jul 1, 2026
53daeec
Added error for missing storage types
Tw1ZZLER Jul 1, 2026
a93ce16
Fixed typing for storage attribute
Tw1ZZLER Jul 1, 2026
eed31a1
Return of the __all__ list of exported symbols
Tw1ZZLER Jul 2, 2026
9ead73c
Merge branch 'master' into metrics
Tw1ZZLER Jul 6, 2026
5507e57
Replace double backtick to single backtick in docstrings
Tw1ZZLER Jul 6, 2026
e1852f7
Update CollectContext docstring for incorrect default
Tw1ZZLER Jul 6, 2026
b209dfc
Test methods for the network manager
Tw1ZZLER Jun 26, 2026
bf231cd
Merge branch 'metrics' into metrics-network-resource
Tw1ZZLER Jun 26, 2026
7b3c8f1
Merge branch 'metrics' into metrics-network-resource
Tw1ZZLER Jul 7, 2026
b6fb0e3
Renaming a lot of the new variables after changes to upstream metrics…
Tw1ZZLER Jun 30, 2026
d870c6f
Renamed variable missed
Tw1ZZLER Jun 30, 2026
3e6def6
Re-organized start and target time logic for DeliveryTimeMetric
Tw1ZZLER Jun 30, 2026
8457730
Reformatted _pair_counters
Tw1ZZLER Jun 30, 2026
5f04b04
Merge branch 'metrics' into metrics-network-resource
Tw1ZZLER Jul 7, 2026
69e8287
Merge branch 'master' into metrics
Tw1ZZLER Jul 17, 2026
061cf31
Merge branch 'metrics' into metrics-network
Tw1ZZLER Jul 7, 2026
35fbdd8
Reformatted init, metric_types, registry
Tw1ZZLER Jul 7, 2026
9aea307
Fixed these metrics types imports
Tw1ZZLER Jul 7, 2026
96f2931
These were in the wrong location
Tw1ZZLER Jul 7, 2026
09eebf8
Update docstrings
Tw1ZZLER Jul 7, 2026
48dcd88
Adjusted collect of RateMetric to copy get_throughput
Tw1ZZLER Jul 7, 2026
09b7a25
Adjust tests for new throughput changes
Tw1ZZLER Jul 7, 2026
9512f71
Potential fix for pull request finding 'Unused import'
Tw1ZZLER Jul 7, 2026
91ed005
Forgot to import these
Tw1ZZLER Jul 7, 2026
c06ba91
This method is quite useless
Tw1ZZLER Jul 8, 2026
781c948
Reformat to line length 120
Tw1ZZLER Jul 10, 2026
da3e4ac
Remove registry stuff
Tw1ZZLER Jul 17, 2026
79b1ec7
Due to change in collect, RateMetric should now be ThroughputMetric
Tw1ZZLER Jul 17, 2026
33cffac
There is no "throughput" event anymore
Tw1ZZLER Jul 17, 2026
9fe972d
There is no reason for this metric to exist
Tw1ZZLER Jul 17, 2026
6ed87b4
Fix these to use metrics instead of events
Tw1ZZLER Jul 17, 2026
efddead
These tests dont need to exist
Tw1ZZLER Jul 17, 2026
ccdc9ab
Undo changes to network manager tests
Tw1ZZLER Jul 17, 2026
a101a75
Added a bunch of new events and organized them
Tw1ZZLER Jul 17, 2026
e3a91e6
Organized builtin metrics
Tw1ZZLER Jul 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 28 additions & 1 deletion sequence/network_management/network_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@

from ..components.memory import MemoryArray
from ..message import Message
from ..utils import log
from ..utils import log, metrics
from ..utils.metrics.event_types import EventTypes
from .forwarding import ForwardingProtocol
from .memory_timecard import MemoryTimeCard
from .reservation import Reservation
Expand Down Expand Up @@ -226,12 +227,38 @@ def pop(self, msg: RSVPMessage):
"""
reservation: Reservation = msg.reservation
if msg.msg_type == RSVPMsgType.APPROVE:
metrics.record(
EventTypes.RESERVATION_APPROVED,
self.owner.name,
identity=reservation.identity,
initiator=reservation.initiator,
responder=reservation.responder,
start_time=reservation.start_time,
end_time=reservation.end_time,
memory_size=reservation.memory_size,
entanglement_number=reservation.entanglement_number,
target_fidelity=reservation.fidelity,
path=list(reservation.path),
)
self.generate_rules(reservation)
if reservation.initiator == self.owner.name:
self.owner.get_reservation_result(reservation, True) # Deliver the result to the Node
elif reservation.responder == self.owner.name:
self.owner.get_other_reservation(reservation)
elif msg.msg_type == RSVPMsgType.REJECT:
metrics.record(
EventTypes.RESERVATION_REJECTED,
self.owner.name,
identity=reservation.identity,
initiator=reservation.initiator,
responder=reservation.responder,
start_time=reservation.start_time,
end_time=reservation.end_time,
memory_size=reservation.memory_size,
entanglement_number=reservation.entanglement_number,
target_fidelity=reservation.fidelity,
path=[],
)
if reservation.initiator == self.owner.name:
self.owner.get_reservation_result(reservation, False)

Expand Down
82 changes: 65 additions & 17 deletions sequence/utils/metrics/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,18 +26,10 @@
from .event_types import (
EventType,
EventTypes,
get_event_type,
list_event_types,
register_event_type,
)
from .metric_types import (
CollectContext,
CounterMetric,
DeliveryTimeMetric,
FidelityMetric,
Metric,
RateMetric,
)
from .metric_types import CollectContext, CounterMetric, DeliveryTimeMetric, FidelityMetric, Metric, ThroughputMetric
from .registry import (
clear_registry,
get_counter,
Expand Down Expand Up @@ -131,17 +123,13 @@ def collect_trial_metrics(
*,
delivery_owner: str | None = None,
target_pairs: int | None = None,
reservation_start_time: int | None = None,
throughput: float | None = None,
) -> dict[str, Any]:
"""Collect per-trial metrics for a node from the metrics module.

Args:
owner_name: Node name to collect counter and fidelity metrics for.
delivery_owner: Node name used for delivery-time metrics; defaults to `owner_name`.
target_pairs: Number of delivered pairs required to compute delivery time.
reservation_start_time: Simulation time when the reservation started (ps).
throughput: Application throughput to include in collected metrics.

Returns:
Mapping of metric output keys to per-trial values.
Expand All @@ -151,15 +139,76 @@ def collect_trial_metrics(
storage=storage,
delivery_owner=delivery_owner or owner_name,
target_pairs=target_pairs,
reservation_start_time=reservation_start_time,
throughput=throughput,
)
result: dict[str, Any] = {}
for metric in list_metrics():
result.update(metric.collect(ctx))
return result


def collect_reservation_data(owner_name: str | None = None) -> list[list]:
"""Collect per-reservation tabular metrics from recorded delivery events."""
from collections import defaultdict

records = storage.get_by_event(EventTypes.DELIVERY)
if owner_name is not None:
records = [record for record in records if record["owner_name"] == owner_name]

groups: dict[tuple[str, int], list[dict[str, Any]]] = defaultdict(list)
for record in records:
groups[(record["owner_name"], record["identity"])].append(record)

data: list[list] = []
for (node, _), deliveries in groups.items():
if not deliveries:
continue
deliveries.sort(key=lambda record: record["sim_time"])
timestamps = [record["sim_time"] for record in deliveries]
fidelities = [record["fidelity"] for record in deliveries]

first = deliveries[0]
start_time = first["start_time"]
end_time = first["end_time"]
reserved_time = end_time - start_time
served_pairs = len(deliveries)
throughput = served_pairs / reserved_time * 1e12 if reserved_time > 0 else 0.0
completion_time = timestamps[-1]
entanglement_number = first["entanglement_number"]
fulfilled = served_pairs == entanglement_number
path = first.get("path", [])
path_length = len(path)
first_pair = timestamps[0]
avg_fidelity = mean(fidelities) if fidelities else 0.0
std_fidelity = stdev(fidelities) if len(fidelities) > 1 else 0.0
durations = [n - c for c, n in zip(timestamps, timestamps[1:])]
avg_duration = mean(durations) if durations else 0.0
std_duration = stdev(durations) if len(durations) > 1 else 0.0

data.append(
[
node,
first["identity"],
first["initiator"],
first["responder"],
start_time,
end_time,
reserved_time,
entanglement_number,
served_pairs,
throughput,
completion_time,
fulfilled,
path_length,
first_pair,
avg_fidelity,
std_fidelity,
avg_duration,
std_duration,
]
)
return data


def _mean_and_std(values: list[float]) -> tuple[float, float]:
finite_values = [value for value in values if math.isfinite(value)]
if not finite_values:
Expand Down Expand Up @@ -223,7 +272,6 @@ def aggregate_trial_metrics(
# From event_types
"EventType",
"EventTypes",
"get_event_type",
"list_event_types",
"register_event_type",
# From metric_types
Expand All @@ -232,7 +280,7 @@ def aggregate_trial_metrics(
"DeliveryTimeMetric",
"FidelityMetric",
"Metric",
"RateMetric",
"ThroughputMetric",
# From registry
"clear_registry",
"get_counter",
Expand Down
46 changes: 28 additions & 18 deletions sequence/utils/metrics/builtins.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,10 @@
from __future__ import annotations

from .event_types import EventTypes
from .metric_types import (
CounterMetric,
DeliveryTimeMetric,
FidelityMetric,
Metric,
RateMetric,
)
from .metric_types import CounterMetric, DeliveryTimeMetric, FidelityMetric, Metric, ThroughputMetric
from .registry import register_metric

# Entanglement Management Metrics
EG_METRIC = CounterMetric(
prefix="eg",
failure_event=EventTypes.EG_FAILURE,
Expand All @@ -24,28 +19,43 @@
success_event=EventTypes.EP_SUCCESS,
rate_field="ep_success_rate",
)
THROUGHPUT_METRIC = RateMetric(key="app_throughput")
PURIFIED_FIDELITIES_METRIC = FidelityMetric(
key="purified_fidelities",
event=EventTypes.EP_SUCCESS,
field="fidelity",
)
DELIVERY_TIME_METRIC = DeliveryTimeMetric(
key="delivery_time",
delivery_event=EventTypes.DELIVERY,
)
ES_METRIC = CounterMetric(
prefix="es",
failure_event=EventTypes.ES_FAILURE,
success_event=EventTypes.ES_SUCCESS,
rate_field="es_success_rate",
)
PURIFIED_FIDELITIES_METRIC = FidelityMetric(
key="purified_fidelities",
event=EventTypes.EP_SUCCESS,
field="fidelity",
)
SWAPPED_FIDELITIES_METRIC = FidelityMetric(
key="swapped_fidelities",
event=EventTypes.ES_SUCCESS,
field="fidelity",
)

# Network Management Metrics
RESERVATION_APPROVAL_RATE = CounterMetric(
prefix="reservation_approval_rate",
failure_event=EventTypes.RESERVATION_REJECTED,
success_event=EventTypes.RESERVATION_APPROVED,
rate_field="reservation_approval_rate",
)

# Resource Management Metrics

# Application Metrics
THROUGHPUT_METRIC = ThroughputMetric(
key="app_throughput",
delivery_event=EventTypes.DELIVERY,
)
DELIVERY_TIME_METRIC = DeliveryTimeMetric(
key="delivery_time",
delivery_event=EventTypes.DELIVERY,
)


def register_builtin_metrics() -> None:
"""Register all built-in metrics with the global registry.
Expand All @@ -57,10 +67,10 @@ def register_builtin_metrics() -> None:
BUILTIN_METRICS: list[Metric] = [
EG_METRIC,
EP_METRIC,
ES_METRIC,
THROUGHPUT_METRIC,
PURIFIED_FIDELITIES_METRIC,
DELIVERY_TIME_METRIC,
ES_METRIC,
SWAPPED_FIDELITIES_METRIC,
]

Expand Down
37 changes: 21 additions & 16 deletions sequence/utils/metrics/event_types.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,21 +34,6 @@ def register_event_type(name: str) -> EventType:
return event_type


def get_event_type(name: str) -> EventType:
"""Return a registered event type by name.

Args:
name: Name of the event type to look up.

Returns:
The registered event type.
"""
try:
return _registry[name]
except KeyError as exc:
raise KeyError(f"Event type '{name}' is not registered.") from exc


def list_event_types() -> list[EventType]:
"""Return all registered event types.

Expand All @@ -61,10 +46,30 @@ def list_event_types() -> list[EventType]:
class EventTypes:
"""Namespace for built-in simulation event types."""

# Entanglement Management Events
EG_FAILURE = register_event_type("EG_FAILURE")
EG_SUCCESS = register_event_type("EG_SUCCESS")
EP_FAILURE = register_event_type("EP_FAILURE")
EP_SUCCESS = register_event_type("EP_SUCCESS")
DELIVERY = register_event_type("DELIVERY")
ES_FAILURE = register_event_type("ES_FAILURE")
ES_SUCCESS = register_event_type("ES_SUCCESS")

# Network Management Events
RESERVATION_APPROVED = register_event_type("RESERVATION_APPROVED")
RESERVATION_REJECTED = register_event_type("RESERVATION_REJECTED")
RESERVATION_REQUESTED = register_event_type("RESERVATION_REQUESTED")
RESERVATION_HOP_REJECT = register_event_type("RESERVATION_HOP_REJECT")
RESERVATION_REACHED_RESPONDER = register_event_type("RESERVATION_REACHED_RESPONDER")

# Forwarding
FORWARDING_TABLE_MISS = register_event_type("FORWARDING_TABLE_MISS")

# Routing
NEIGHBOR_DOWN = register_event_type("NEIGHBOR_DOWN")
NEIGHBOR_FULL = register_event_type("NEIGHBOR_FULL")
ROUTE_RECOMPUTED = register_event_type("ROUTE_RECOMPUTED")
LSA_ORIGINATED = register_event_type("LSA_ORIGINATED")
LSDB_UPDATED = register_event_type("LSDB_UPDATED")

# Application Events
DELIVERY = register_event_type("DELIVERY")
Loading
Loading