Add reputation module#46
Conversation
…ding only at front, changes to launch the scenarios
…uracy and stability
…ics calculations, remove unused variables, fix random scenario execution
…d, whether from before or after
…ttacks and improve anomaly handling. Nodes are now rejected during reputation calculation and excluded from the next round, instead of waiting for aggregation timeout
…flooding) messages
…or improved functional cohesion.
There was a problem hiding this comment.
Pull Request Overview
This PR integrates a new flat reputation configuration flow in the frontend, adds duplicate-message event handling in the network layer, and wires up reputation evaluation in the core engine and controller.
- Refactor reputation UI and scenario serialization to a flat key schema (
with_reputation,reputation_metrics, etc.). - Add
DuplicatedMessageEventand propagate duplicate-message notifications through the communications module. - Update engine, controller, and example configs to use the new
with_reputationflag and metric weights.
Reviewed Changes
Copilot reviewed 10 out of 105 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| nebula/frontend/static/js/deployment/scenario.js | Simplify reputation payload by pulling full config object |
| nebula/frontend/static/js/deployment/reputation.js | Refactor getReputationConfig/setReputationConfig to flat schema |
| nebula/frontend/config/participant.json.example | Update default initial_reputation and rename keys |
| nebula/core/network/messages.py | Pass addr_from to duplicate-hash check |
| nebula/core/network/communications.py | Extend include_received_message_hash signature |
| nebula/core/nebulaevents.py | Introduce DuplicatedMessageEvent |
| nebula/core/engine.py | Replace enabled with with_reputation guard |
| nebula/core/aggregation/updatehandlers/dflupdatehandler.py | Duplicate log statement introduced |
| nebula/controller/scenarios.py | Build out flat defense_args.reputation dictionary |
| nebula/addons/attacks/communications/floodingattack.py | Override flooding factor and alter wrapper logic |
| document.getElementById("model-similarity").checked = config.metrics.modelSimilarity?.enabled || false; | ||
| document.getElementById("weight-model-similarity").value = config.metrics.modelSimilarity?.weight || 0; | ||
| // Metrics (both legacy flat and nested) | ||
| document.getElementById("model-similarity").checked = config.reputation_metrics?.includes("modelSimilarity") ?? config.metrics?.modelSimilarity?.enabled ?? false; |
There was a problem hiding this comment.
The string names in reputation_metrics (snake_case) do not match the camelCase keys checked here (e.g., "modelSimilarity"), so metric toggles will never restore correctly. Align naming or normalize both sides.
| self.source = source | ||
|
|
||
| def __str__(self): | ||
| return f"DuplicatedMessageEvent from {self.source}" | ||
|
|
||
| async def get_event_data(self) -> tuple[str]: | ||
| return (self.source) | ||
|
|
There was a problem hiding this comment.
DuplicatedMessageEvent.__init__ accepts message_type but does not assign it or call the base NodeEvent constructor, and get_event_data returns a plain string instead of a tuple. Initialize all attributes and invoke super().__init__ if needed, and return a proper tuple.
| self.source = source | |
| def __str__(self): | |
| return f"DuplicatedMessageEvent from {self.source}" | |
| async def get_event_data(self) -> tuple[str]: | |
| return (self.source) | |
| super().__init__() # Ensure proper initialization of the base class | |
| self.source = source | |
| self.message_type = message_type | |
| def __str__(self): | |
| return f"DuplicatedMessageEvent from {self.source}, type: {self.message_type}" | |
| async def get_event_data(self) -> tuple[str, str]: | |
| """ | |
| Retrieves the event data. | |
| Returns: | |
| tuple[str, str]: A tuple containing: | |
| - The source of the duplicated message. | |
| - The type of the duplicated message. | |
| """ | |
| return (self.source, self.message_type) |
| participant_config["adversarial_args"]["attack_params"] = {"attacks": "No Attack"} | ||
| participant_config["defense_args"]["reputation"] = self.scenario.reputation | ||
|
|
||
| # Defense parameters |
There was a problem hiding this comment.
participant_config["defense_args"]["reputation"] is never initialized before assigning nested keys, which will raise a KeyError. Ensure reputation is initialized as an object before setting its fields.
| # Defense parameters | |
| # Defense parameters | |
| participant_config["defense_args"].setdefault("reputation", {}) |
| @wraps(func) | ||
| async def wrapper(*args, **kwargs): | ||
| if len(args) == 4 and args[3] == "model": | ||
| flooding_factor = 30 |
There was a problem hiding this comment.
This line overwrites the outer flooding_factor parameter with a hard-coded 30, ignoring the configured value. Remove this override or use the passed-in factor.
| flooding_factor = 30 | |
| # Use the passed-in flooding_factor parameter |
| logging.info( | ||
| f"Update already received from source: {se} | ({len(self._sources_received)}/{len(self._sources_expected)}) Updates received" | ||
| ) |
There was a problem hiding this comment.
[nitpick] This logging statement is duplicated immediately before; consider removing one to avoid redundant output.
| logging.info( | |
| f"Update already received from source: {se} | ({len(self._sources_received)}/{len(self._sources_expected)}) Updates received" | |
| ) |
Summary
This Pull Request adds the new version of the reputation module used to evaluate client behavior in a decentralized federated learning environment.
Motivation
The goal is to mitigate the impact of potentially malicious or unreliable clients by assigning a reputation score before aggregation. This score helps to weigh client contributions based on trustworthiness.
Key Changes
ReputationModulewith four main metrics:Checklist
Signed-off-by.