Skip to content

delayed_ack.md

Latest commit

 

History

History
108 lines (85 loc) · 4.77 KB

delayed_ack.md

File metadata and controls

108 lines (85 loc) · 4.77 KB

Delayed ACK

Historically, ADB transport protocol transfer speed was affected by two factors.

  1. Each A_WRTE apacket was CRCed upon write and the CRC was checked upon read on the other end.
  2. There could be only one A_WRTE apacket in-flight on an asocket. A local asocket would not schedule more data to be sent out until it had received an A_OKAY apacket response from its peer.

The first issue was solved in aosp/568123. In that CL, the protocol was updated to remove the requirement for CRC generation and verification. This does not affect the reliability of a transport since both USB and TCP have packet checksums of their own.

The second issue is solved by "delayed ACK" (aosp/1953877), an experimental feature controlled by the environment variable ADB_BURST_MODE.

How delayed ACK works

The idea is to introduce the concept of a per-asocket "available send bytes" (ASB) integer. This integer represent how many bytes we are willing to send without having received any A_OKAY for them.

While the ASB is positive, the asocket does not wait for an A_OKAY before sending more A_WRTE apackets. A remote asocket can be written to up until the ASB is exhausted.

The ASB capability is first negotiated on A_OPEN/A_OKAY exchange. After that, the ASB is maintained via decrement upon A_WRTE and increment upon A_OKAY.

This approach allows to "burst" A_WRTE packet but also "burst" A_OKAY packets to allow several A_WRTE packets to be in-flight on an asocket. This greatly increases data transfer throughput.

Implementation

Packet update

  1. A_OPEN unused field (arg1) is repurposed to declare the wish to use delayed ACK features. If not supported, the receiving end of the A_OPEN will A_CLSE the connection.
  2. A_OKAY now has a payload (a int32_t) which acknowledge how much payload was received in the last received A_WRTE apacket.

Trace

Here are two traces showing the timing of three A_WRTE.

Before

Host                > A_OPEN                  > Device
Host                > A_WRTE                  > Device
The LS removes itself from the fdevent EPOLLIN and nothing is sent.
Host                < A_OKAY                  < Device
The LS requests fdevent EPOLLIN for its fd to start reading and send more A_WRTE.
Host                > A_WRTE                  > Device
The LS removes itself from the fdevent EPOLLIN and nothing is sent.
Host                < A_OKAY                  < Device
The LS requests fdevent EPOLLIN for its fd to start reading and send more A_WRTE.
Host                > A_WRTE                  > Device
The LS removes itself from the fdevent EPOLLIN and nothing is sent.
Host                < A_OKAY                  < Device
The LS requests fdevent EPOLLIN for its fd to start reading and send more A_WRTE.

After

With ASB, see how A_WRTE and A_OKAY are burst instead of being paired.

Host(ASB=0)         > A_OPEN(arg1=1MiB)       > Device
Host(ASB=X)         < A_OKAY(<ASB=X>)         < Device
Host<ASB=X-a)       > A_WRTE(payload size=a)  > Device
Host<ASB=Y-a-b)     > A_WRTE(payload size=b)  > Device
Host<ASB=Z-a-b-c)   > A_WRTE(payload size=c)  > Device
ASB is < 0. The LS removes itself from the fdevent EPOLLIN and nothing is sent.
...
Host(ASB=X-b-c)     < A_OKAY(<a>)             < Device
ASB is > 0. The LS requests fdevent EPOLLIN for its fd to start reading and send more A_WRTE.
...
Host(ASB=X-c)       < A_OKAY(<b>)             < Device
Host(ASB=X)         < A_OKAY(<c>)             < Device

Results

Initial testing show that Burst Mode is nearly 70% faster at pushing files to a device over a USB-3 cable.

Before

$ adb kill-server && unset ADB_BURST_MODE && adb start-server
$ adb push -Z ~/Desktop/10G1 /data/local/tmp
/usr/local/google/home/sanglardf/Desktop/10G1: 1 file pushed, 0 skipped. 202.0 MB/s (10737418240 bytes in 50.701s)
$ adb push -Z ~/Desktop/10G1 /data/local/tmp
/usr/local/google/home/sanglardf/Desktop/10G1: 1 file pushed, 0 skipped. 205.9 MB/s (10737418240 bytes in 49.724s)
$ adb push -Z ~/Desktop/10G1 /data/local/tmp
/usr/local/google/home/sanglardf/Desktop/10G1: 1 file pushed, 0 skipped. 197.6 MB/s (10737418240 bytes in 51.828s)

After

$ adb kill-server && export ADB_BURST_MODE=1 && adb start-server
$ adb push -Z ~/Desktop/10G1 /data/local/tmp
/usr/local/google/home/sanglardf/Desktop/10G1: 1 file pushed, 0 skipped. 337.2 MB/s (10737418240 bytes in 30.365s)
$ adb push -Z ~/Desktop/10G1 /data/local/tmp
/usr/local/google/home/sanglardf/Desktop/10G1: 1 file pushed, 0 skipped. 342.0 MB/s (10737418240 bytes in 29.945s)
$ adb push -Z ~/Desktop/10G1 /data/local/tmp
/usr/local/google/home/sanglardf/Desktop/10G1: 1 file pushed, 0 skipped. 341.3 MB/s (10737418240 bytes in 30.000s)