Merge #1497: Overhaul stats: Improve torrent-repository package before adding events (part 1)

728de22 docs: [#1495] add todo (Jose Celano)
34c159a refactor: [#1495] update method Swarm::meets_retaining_policy (Jose Celano)
0f4596e fix: [#1495] formatting (Jose Celano)
0411a9a refactor: [#1495] rename TorrentRepository to Swarms (Jose Celano)
78d4b83 refactor: [#1495] rename TrackedTorrentHandle to SwarmHandle (Jose Celano)
030ae26 refactor: [#1495] reorganize torrent-repository mod (Jose Celano)
ca9937d refactor: [#1495] remove unneeded TrackedTorrent (wrapper over Swarm) (Jose Celano)
b6afed5 refactor: [#1495] rename methods (Jose Celano)
ef7292f refactor: [#1495] move logic from TrackedTorrent to Swarm (Jose Celano)
23ce6e4 refactor: [#1495]remove unused field in TrackedTorrent (Jose Celano)
ec597f0 refactor: [#1495] get the number of downloads from Swarm instead of from TrackedTorrent (Jose Celano)
3fb117b refactor: [#1495] initialize number of downloads in Swarm to persisted value (Jose Celano)
9b2392e refactor: [#1495] make TrackedTorrent fields private (Jose Celano)
82bbfe3 refactor: [#1495] move logic from TackedTorrent to Swarm (Jose Celano)
f73c566 refactor: [#1495] some renamings in Swarm type (Jose Celano)
61560a8 chore: add gitignore to torrent-repository pkg (Jose Celano)
0a4c805 refactor: [#1495] add SwarmMetadata to Swarm (Jose Celano)
2882705 refactor: [#1495] use SocketAddr as key for peers in Swarm (Jose Celano)
15c14c5 refactor: [#1495] rename PeerList to Swarm (Jose Celano)
cb51ec9 docs: [#1495] improve torrent-repository pkg readme (Jose Celano)

Pull request description:

  Partially implements #1495

  Improve `torrent-repository` package before adding events.

  ### Sutasks

  - [x] Update README

  #### Refactoring `PeerList`

  - [x] Rename `PeerList` to `Swarm`.
  - [x] Use `SocketAddr` as key for peers in `Swarm` instead of the `PeerId`.
  - [x] Use a cache for seeders and leechers counters to avoid iterating the list.
  - [x] Move logic from `TrackerTorrent` to `Swarm`

  #### Refactoring `TrackedTorrent` (it was removed)

  - [x] Remove the `downloaded` field from `TrackedTorrent`. The logic can be moved to `Swarm`.
  - [x] Document `meets_retaining_policy` method behavior. I don't know why peerless torrents are kept when persisted stats (`downloaded` field) are enabled (only if the torrent has been downloaded at least once).

  It was removed because after the refactorings, it became just a wrapper over `Swarm`.

  #### Refactoring `TorrentRepository`

  To be implemented in a new PR.

  ### New state

  There are only two primary types:

  - `Swarm`: a collection of peers. Peers are members of the swarm.
  - `Swarms`: a collection of swarms.

  ```
  $ tree ./src/
  ./src/
  ├── lib.rs
  ├── swarm.rs
  └── swarms.rs

  1 directory, 3 files
  ```

ACKs for top commit:
  josecelano:
    ACK 728de22

Tree-SHA512: ffe9e0cd02cb18dc657c3ec5f378ed4a17b982dfef06aed5f0274e6135d54c12fba156d8a5290c5b749ed743eca01cb3df9bcfa4d613eb1e64e2528f958e2ad6

Author: josecelano

packages/axum-http-tracker-server/tests/server/asserts.rs

@@ -22,6 +22,7 @@ pub fn assert_bencoded_error(response_text: &String, expected_failure_reason: &s
     );
 }
 
+#[allow(dead_code)]
 pub async fn assert_empty_announce_response(response: Response) {
     assert_eq!(response.status(), 200);
     let announce_response: Announce = serde_bencode::from_str(&response.text().await.unwrap()).unwrap();

packages/axum-http-tracker-server/tests/server/requests/announce.rs

@@ -126,6 +126,11 @@ impl QueryBuilder {
         self
     }
 
+    pub fn with_port(mut self, port: u16) -> Self {
+        self.announce_query.port = port;
+        self
+    }
+
     pub fn without_compact(mut self) -> Self {
         self.announce_query.compact = None;
         self

packages/axum-http-tracker-server/tests/server/v1/contract.rs

@@ -105,8 +105,8 @@ mod for_all_config_modes {
         use crate::common::fixtures::invalid_info_hashes;
         use crate::server::asserts::{
             assert_announce_response, assert_bad_announce_request_error_response, assert_cannot_parse_query_param_error_response,
-            assert_cannot_parse_query_params_error_response, assert_compact_announce_response, assert_empty_announce_response,
-            assert_is_announce_response, assert_missing_query_params_for_announce_request_error_response,
+            assert_cannot_parse_query_params_error_response, assert_compact_announce_response, assert_is_announce_response,
+            assert_missing_query_params_for_announce_request_error_response,
         };
         use crate::server::client::Client;
         use crate::server::requests::announce::{Compact, QueryBuilder};
@@ -559,27 +559,53 @@ mod for_all_config_modes {
         }
 
         #[tokio::test]
-        async fn should_consider_two_peers_to_be_the_same_when_they_have_the_same_peer_id_even_if_the_ip_is_different() {
+        async fn should_consider_two_peers_to_be_the_same_when_they_have_the_same_socket_address_even_if_the_peer_id_is_different(
+        ) {
             logging::setup();
 
             let env = Started::new(&configuration::ephemeral_public().into()).await;
 
             let info_hash = InfoHash::from_str("9c38422213e30bff212b30c360d26f9a02136422").unwrap(); // DevSkim: ignore DS173237
             let peer = PeerBuilder::default().build();
 
-            // Add a peer
-            env.add_torrent_peer(&info_hash, &peer);
-
-            let announce_query = QueryBuilder::default()
+            let announce_query_1 = QueryBuilder::default()
                 .with_info_hash(&info_hash)
                 .with_peer_id(&peer.peer_id)
+                .with_peer_addr(&peer.peer_addr.ip())
+                .with_port(peer.peer_addr.port())
+                .query();
+
+            let announce_query_2 = QueryBuilder::default()
+                .with_info_hash(&info_hash)
+                .with_peer_id(&PeerId(*b"-qB00000000000000002")) // Different peer ID
+                .with_peer_addr(&peer.peer_addr.ip())
+                .with_port(peer.peer_addr.port())
                 .query();
 
-            assert_ne!(peer.peer_addr.ip(), announce_query.peer_addr);
+            // Same peer socket address
+            assert_eq!(announce_query_1.peer_addr, announce_query_2.peer_addr);
+            assert_eq!(announce_query_1.port, announce_query_2.port);
+
+            // Different peer ID
+            assert_ne!(announce_query_1.peer_id, announce_query_2.peer_id);
 
-            let response = Client::new(*env.bind_address()).announce(&announce_query).await;
+            let _response = Client::new(*env.bind_address()).announce(&announce_query_1).await;
+            let response = Client::new(*env.bind_address()).announce(&announce_query_2).await;
 
-            assert_empty_announce_response(response).await;
+            let announce_policy = env.container.tracker_core_container.core_config.announce_policy;
+
+            // The response should contain only the first peer.
+            assert_announce_response(
+                response,
+                &Announce {
+                    complete: 1,
+                    incomplete: 0,
+                    interval: announce_policy.interval,
+                    min_interval: announce_policy.interval_min,
+                    peers: vec![],
+                },
+            )
+            .await;
 
             env.stop().await;
         }

packages/primitives/src/swarm_metadata.rs

@@ -7,7 +7,7 @@ use derive_more::Constructor;
 /// Swarm metadata dictionary in the scrape response.
 ///
 /// See [BEP 48: Tracker Protocol Extension: Scrape](https://www.bittorrent.org/beps/bep_0048.html)
-#[derive(Copy, Clone, Debug, PartialEq, Default, Constructor)]
+#[derive(Copy, Clone, Debug, PartialEq, Eq, PartialOrd, Ord, Hash, Default, Constructor)]
 pub struct SwarmMetadata {
     /// (i.e `completed`): The number of peers that have ever completed
     /// downloading a given torrent.
@@ -27,6 +27,21 @@ impl SwarmMetadata {
     pub fn zeroed() -> Self {
         Self::default()
     }
+
+    #[must_use]
+    pub fn downloads(&self) -> u32 {
+        self.downloaded
+    }
+
+    #[must_use]
+    pub fn seeders(&self) -> u32 {
+        self.complete
+    }
+
+    #[must_use]
+    pub fn leechers(&self) -> u32 {
+        self.incomplete
+    }
 }
 
 /// Structure that holds aggregate swarm metadata.

packages/torrent-repository/.gitignore

@@ -0,0 +1 @@
+/.coverage/

packages/torrent-repository/README.md

@@ -2,26 +2,16 @@
 
 A library to provide a torrent repository to the [Torrust Tracker](https://github.com/torrust/torrust-tracker).
 
-## Benchmarking
-
-```console
-cargo bench -p torrust-tracker-torrent-repository
-```
-
-Example partial output:
-
-```output
-     Running benches/repository_benchmark.rs (target/release/deps/repository_benchmark-a9b0013c8d09c3c3)
-add_one_torrent/RwLockStd
-                        time:   [63.057 ns 63.242 ns 63.506 ns]
-Found 12 outliers among 100 measurements (12.00%)
-  2 (2.00%) low severe
-  2 (2.00%) low mild
-  2 (2.00%) high mild
-  6 (6.00%) high severe
-add_one_torrent/RwLockStdMutexStd
-                        time:   [62.505 ns 63.077 ns 63.817 ns]
-```
+Its main responsibilities include:
+
+- Managing Torrent Entries: It stores, retrieves, and manages torrent entries, which are torrents being tracked.
+- Persistence: It supports lading tracked torrents from a persistent storage, ensuring that torrent data can be restored across restarts.
+- Pagination and sorting: It provides paginated and stable/sorted access to torrent entries.
+- Peer management: It manages peers associated with torrents, including removing inactive peers and handling torrents with no peers (peerless torrents).
+- Policy handling: It supports different policies for handling torrents, such as persisting, removing, or custom policies for torrents with no peers.
+- Metrics: It can provide metrics about the torrents, such as counts or statuses, likely for monitoring or statistics.
+
+This repo is a core component for managing the state and lifecycle of torrents and their peers in a BitTorrent tracker, with peer management, and flexible policies.
 
 ## Documentation