Expose piece bitfield API for streaming video players

[cemalturkcan]

No description provided.

[michakfromparis]

I was about to submit a similar PR. I second this one

[ikatson]

There's an API with bit field exposed that is used by the new Web UI. It's /api/v1/haves, but you need to set accept: application/octet-stream. Please check, but please check and let me know if that's enough

[cemalturkcan]

Thanks, but I’m not using the web server. I’m working directly with the core, so I can’t use /api/v1/haves.

[ikatson]

@cemalturkcan can't you use whatever it is using? I.e. api.rs

[michakfromparis]

Hey @ikatson,

like you suggested, I looked into the /api/v1/haves endpoint (backed by api_dump_haves) and it only covers the read side -- querying which pieces are downloaded. That part is indeed equivalent to get_piece_bitfield() from this PR.

However, the critical feature of this PR is update_streaming_window(), which modifies the piece picker queue to restrict downloads to a window around the current playback position. This is what makes streaming video playable -- without it, rqbit downloads in default order (rarest-first), meaning the player has to wait for pieces it may not need yet while the pieces it needs right now aren't prioritized.

Concretely, update_streaming_window on ChunkTracker:

Removes pieces outside the [position - backward_bytes, position + forward_bytes] window from the download queue

Adds pieces inside the window back to the queue (if selected and not already downloaded)

Returns metadata about what changed (pieces_added, pieces_removed, window_start_piece, window_end_piece)

To my knowledge, there is no equivalent to this in the upstream api.rs or anywhere reachable from ManagedTorrentHandle. The existing with_chunk_tracker callback gives read-only access, and there is no with_chunk_tracker_mut in upstream.

I'm using this fork in our plugin and it works well. We call update_streaming_window at the start of playback (position 0, forward ~30MB) to buffer leading pieces, and then periodically as playback advances. Combined with get_piece_bitfield, this gives us a fully working streaming experience with ~3s initial buffer time.

TL;DR: api_dump_haves replaces get_piece_bitfield but not update_streaming_window. The streaming window control is the main value of this PR.

Thanks @cemalturkcan for this. @ikatson is there a reason you're reluctant to merge this ? Anything you'd like to be done differently ? Maybe make this PR use the api_dump_haves ?

[cemalturkcan]

@michakfromparis thanks for the detailed explanation! actually I first added get_piece_bitfield to visualize the buffer, then added update_streaming_window for my own video streaming use case. since it's been a month, looking at the title I totally forgot I had done that 😅

[ikatson]

This is what makes streaming video playable -- without it, rqbit downloads in default order (rarest-first), meaning the player has to wait for pieces it may not need yet while the pieces it needs right now aren't prioritized.

This used to work, maybe it broke in 8c84f54 (that commit was vibe coded), not sure, but streaming pieces should be prioritized:

rqbit/crates/librqbit/src/torrent_state/live/mod.rs

Line 1413 in 61e189a

priority_pieces: self.state.streams.iter_next_pieces(&self.state.lengths),

is there a reason you're reluctant to merge this ?

It's big and complex, and I still don't understand why does it need to exist provided the above question