Skip to content

Commit f2e250a

Browse files
committed
Updating the readme file
1 parent 5f46a84 commit f2e250a

1 file changed

Lines changed: 26 additions & 15 deletions

File tree

disk-buffering/README.md

Lines changed: 26 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -114,26 +114,37 @@ a network exporter, as shown in the example for spans below.
114114
* @return true, if the exporting was successful, false, if it needs to be retried
115115
*/
116116
public boolean exportSpansFromDisk(SpanExporter networkExporter, long timeout) {
117-
for (Collection<SpanData> spanData : spanStorage) {
118-
CompletableResultCode resultCode = networkExporter.export(spanData);
119-
resultCode.join(timeout, TimeUnit.MILLISECONDS);
120-
121-
if (!resultCode.isSuccess()) {
122-
logger.trace("Error while exporting", resultCode.getFailureThrowable());
123-
// The iteration should be aborted here to avoid consuming batches, which were not exported successfully
124-
return false;
125-
}
117+
Iterator<Collection<SpanData>> spansIterator = spanStorage.iterator();
118+
while (spansIterator.hasNext()) {
119+
CompletableResultCode resultCode = networkExporter.export(spanData);
120+
resultCode.join(timeout, TimeUnit.MILLISECONDS);
121+
122+
if (resultCode.isSuccess()) {
123+
spansIterator.remove(); // Remove the current item, as it was successfully exported to the network
124+
} else {
125+
logger.trace("Error while exporting", resultCode.getFailureThrowable());
126+
// The iteration should be aborted here to avoid consuming batches, which were not exported successfully
127+
return false;
126128
}
127-
logger.trace("Finished exporting");
128-
return true;
129+
}
130+
logger.trace("Finished exporting");
131+
return true;
129132
}
130133
```
131134

132-
The `File*Storage` iterators delete the previously returned collection when `next()` is called,
133-
assuming that if the next collection is requested is because the previous one was successfully
134-
consumed.
135+
### Deleting data
135136

136-
Both the writing and reading processes can run in parallel and they don't overlap
137+
There are 2 ways to delete data previously stored by calling the `SignalStorage.write(items)`
138+
function:
139+
140+
* During iteration. This is done by calling `Iterator.remove()` as shown in the example above. This
141+
will remove the last item retrieved from the iterator. Ideally, this should be done after the data
142+
has been successfully exported to the network.
143+
* Clearing all data at once by calling `SignalStorage.clear()`.
144+
145+
### More details on the writing and reading processes
146+
147+
Both the writing and reading processes can run in parallel as they won't overlap
137148
because each is supposed to happen in different files. We ensure that reader and writer don't
138149
accidentally meet in the same file by using the configurable parameters. These parameters set
139150
non-overlapping time frames for each action to be done on a single file at a time. On top of that,

0 commit comments

Comments
 (0)