Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -692,7 +692,7 @@ We refer to this 4-byte header, along with your excerpt, as a document.
Strictly speaking Chronicle Queue can be used to read and write documents.

NOTE: Within this 4-byte header we also reserve a few bits for a number of internal operations, such as locking, to make Chronicle Queue thread-safe across both processors and threads.
The important thing to note is that because of this, you cant strictly convert the 4 bytes to an integer to find the length of your data blob.
The important thing to note is that because of this, you can't strictly convert the 4 bytes to an integer to find the length of your data blob.

As stated before, Chronicle Queue uses an _appender_ to write to the queue and a _tailer_ to read from the queue.
Unlike other java queuing solutions, messages are not lost when they are read with a tailer.
Expand All @@ -711,7 +711,7 @@ Chronicle Queue uses the following low-level interface to write the data:
[source,java]
----
try (final DocumentContext dc = appender.writingDocument()) {
dc.wire().write().text(your text data);
dc.wire().write().text("your text data");
}
----

Expand All @@ -723,7 +723,7 @@ Each Chronicle Queue excerpt has a unique index.
[source,java]
----
try (final DocumentContext dc = appender.writingDocument()) {
dc.wire().write().text(your text data);
dc.wire().write().text("your text data");
System.out.println("your data was store to index="+ dc.index());
}
----
Expand Down Expand Up @@ -1570,15 +1570,15 @@ The default value is 0.
'''
==== File handles and flushing data to the disk

As mentioned previously Chronicle Queue stores its data off-heap in a .cq4 file.
As mentioned previously Chronicle Queue stores its data off-heap in a '.cq4' file.
So whenever you wish to append data to this file or read data into this file, chronicle queue will create a file handle.
Typically, Chronicle Queue will create a new .cq4 file every day.
Typically, Chronicle Queue will create a new '.cq4' file every day.
However, this could be changed so that you can create a new file every hour, every minute or even every second.

If we create a queue file every second, we would refer to this as SECONDLY rolling.
Of course, creating a new file every second is a little extreme, but it's a good way to illustrate the following point.
When using secondly rolling, If you had written 10 seconds worth of data and then you wish to read this data, chronicle would have to scan across 10 files.
To reduce the creation of the file handles, chronicle queue cashes them lazily and when it comes to writing data to the queue files, care-full consideration must be taken when closing the files, because on most OSs a close of the file, will force any data that has been appended to the file, to be flushed to disk, and if we are not careful this could stall your application.
To reduce the creation of the file handles, chronicle queue cashes them lazily and when it comes to writing data to the queue files, care-full consideration must be taken when closing the files, because on most OS's a close of the file, will force any data that has been appended to the file, to be flushed to disk, and if we are not careful this could stall your application.

'''
==== Pretoucher and its configuration
Expand Down
2 changes: 1 addition & 1 deletion docs/FAQ.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -394,7 +394,7 @@ try (DocumentContext dc = appender.writingDocument()) {

If you are writing bytes to a Chronicle Queue you will find that it occasionally adds padding to the end of each message.
This is to ensure that each message starts on a 4-byte boundary which is a requirement for ARM architectures.
NOTE: Intel requires that messages dont straggle 64-byte cash lines.
NOTE: Intel requires that messages don't straggle 64-byte cash lines.
but aligning to 4 bytes also ensures 64-byte alignment and allows your Chronicle Queues to be shared between various different platforms.

For Chronicle Queue, the 4-byte alignment is now enforced, so there is now, no way to turn this feature on or off.
Expand Down
6 changes: 3 additions & 3 deletions docs/antora/modules/ROOT/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,10 @@ Tailing ::
Learn how to read from a queue.
Performance tuning ::
As every environment is different, this chapter points out key factors to obtain the best performance with Chronicle Queue.
Replication ::
Replication (Enterprise) ::
Replication is an Enterprise feature allowing queues to be automatically replicated across machines or cores.
This chapter teaches how that is accomplished.
Encryption ::
Encryption (Enterprise) ::
Learn how to protect the contents of a Chronicle Queue with encryption.

=== Runnable Demos
Expand Down Expand Up @@ -74,7 +74,7 @@ The following features are implemented by Chronicle Queue Enterprise:
* Multi-master queues which are ordered for events in a node, but not between nodes.

This documentation covers the open source functionality but is also extended by a few chapters describing enterprise specific features.
These sections are clearly marked with a star-symbol (★) and include:
These sections are clearly marked with an Enterprise marker and include:

* Encryption of message queues and messages.

Expand Down
4 changes: 2 additions & 2 deletions docs/antora/modules/appending/pages/appending.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Chronicle Queue uses the following low-level interface to write the data:
[source,java]
----
try (final DocumentContext dc = appender.writingDocument()) {
dc.wire().write().text(your text data);
dc.wire().write().text("your text data");
}
----

Expand All @@ -45,7 +45,7 @@ Each Chronicle Queue excerpt has a unique index.
[source,java]
----
try (final DocumentContext dc = appender.writingDocument()) {
dc.wire().write().text(your text data);
dc.wire().write().text("your text data");
System.out.println("your data was store to index="+ dc.index());
}
----
Expand Down
2 changes: 1 addition & 1 deletion docs/antora/modules/async-mode/nav.adoc
Original file line number Diff line number Diff line change
@@ -1 +1 @@
* xref:async_mode.adoc[Async Mode ]
* xref:async_mode.adoc[Async Mode (Enterprise)]
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The following command line tools can be used to interact with Chronicle Queue.
[cols="1,1"]
|===

| <<_archiverollfiles,*ArchiveRollFiles*>>
| <<_archiverollfiles,*ArchiveRollFiles*>> (Enterprise)
| Assists in management of roll cycle files

| <<_chroniclehistoryreadermain,*ChronicleHistoryReaderMain*>>
Expand Down Expand Up @@ -47,7 +47,7 @@ $ mvn exec:java -Dexec.mainClass="net.openhft.chronicle.queue.main.DumpMain" -De

In the above command invoking *DumpMain*, _myqueue_ is the directory containing your .cq4 files.

== ArchiveRollFiles
== ArchiveRollFiles (Enterprise)

To facilitate housekeeping of older roll files, Chronicle Queue Enterprise provides the `ArchiveRollFiles` utility.

Expand Down
4 changes: 2 additions & 2 deletions docs/antora/modules/configuration/nav.adoc
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
* xref:app_configuration.adoc[Configuration]
** xref:roll-cycle.adoc[Roll Cycle]
** xref:timezone_rollover.adoc[Timezone Rollover ]
** xref:timezone_rollover.adoc[Timezone Rollover (Enterprise)]
** xref:wire-type.adoc[Wire Type]
** xref:block-size.adoc[Block Size]
** xref:index-count.adoc[Index Count]
** xref:index-spacing.adoc[Index Spacing]
** xref:buffer-modes.adoc[Buffer Modes ]
** xref:buffer-modes.adoc[Buffer Modes (Enterprise)]
2 changes: 1 addition & 1 deletion docs/antora/modules/configuration/pages/buffer-modes.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
= Buffer Modes
= Buffer Modes (Enterprise)
keywords: buffer modes, chronicle queue, queue, java
author: Julia Gustafsson
:reftext: Buffer modes
Expand Down
10 changes: 5 additions & 5 deletions docs/antora/modules/configuration/pages/roll-cycle.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ NOTE: The default roll-cycle is `RollCycles.FAST_DAILY`.

=== Overriding the roll cycle

Once a queues roll-cycle has been set, it cannot be changed at a later time.
Once a queue's roll-cycle has been set, it cannot be changed at a later time.
Once a queue's metadata has been persisted as part of creating the queue, any further instances of `SingleChronicleQueue` configured to use the same path will have their configuration overridden to match the persisted roll-cycle.
Chronicle Queue will notify of the override via a warning log message:

Expand All @@ -62,7 +62,7 @@ In this case, the persisted roll-cycle is `FAST_HOURLY`, and a new `SingleChroni
Chronicle Queue is UTC time based and uses `System.currentTimeMillis()` to evaluate when it is time to roll the queue.
Rolling occurs whenever a new time frame of the selected kind is started, meaning for minutely rolling a new queue is created when a new minute begins and daily rolls occurs at midnight UTC time.

This behavior can be changed using Chronicle Queues attribute `epoch()`.
This behavior can be changed using Chronicle Queue's attribute `epoch()`.
Epoch refers to a milliseconds offset to the set time frame.
In other words, if you set the epoch to be `epoch(1)` and use `RollCycles.DAILY` the queue will roll at 1 millisecond past midnight UTC time.
Putting it together:
Expand Down Expand Up @@ -146,7 +146,7 @@ For example, you may have up to 1 trillion messages per day and you need 23-bit
With hourly and daily-rolling it's pretty limitless.
Also, by changing the `epoch`, you can extend the dates further, shifting the limit between the first and last cycle to 31-bits or 23-bits.

== Timezone Rollover
== Timezone Rollover (Enterprise)

Chronicle Queue bases its roll times on the UTC time zone.
However, Chronicle Queue Enterprise supports time zone rollover.
Expand Down Expand Up @@ -183,7 +183,7 @@ SingleChronicleQueue queue = ChronicleQueue.singleBuilder("/timezone")
.build();
----

== Archiving Old Queue Files
== Archiving Old Queue Files (Enterprise)

Over time, it may become necessary to automatically delete or archive old queue files.
An automated process needs to ensure that there are no active file-handles open on a queue file before attempting to delete. facilitate housekeeping of older roll files, Chronicle Queue Enterprise provides the xref:command-line:command_line_tools.adoc#_archiverollfiles[ArchiveRollFiles utility].
Expand All @@ -199,7 +199,7 @@ However, it is still recommended avoiding rolling at critical points in time to

In systems that are not always active, it is advised to schedule rolls during the down-time.
However, for applications with a busy feed and no down-time, Chronicle recommends using minutely rolling (a new queue is created every minute).
This keeps jitter to a minimum, as only one minutes worth of data has to be unmapped on a queue-roll.
This keeps jitter to a minimum, as only one minute's worth of data has to be unmapped on a queue-roll.

=== Avoid Large Files

Expand Down
2 changes: 1 addition & 1 deletion docs/antora/modules/encryption/nav.adoc
Original file line number Diff line number Diff line change
@@ -1 +1 @@
* xref:encryption.adoc[Encryption ]
* xref:encryption.adoc[Encryption (Enterprise)]
6 changes: 3 additions & 3 deletions docs/antora/modules/introduction/pages/introduction.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,10 @@ Tailing ::
Learn how to read from a queue.
Performance tuning ::
As every environment is different, this chapter points out key factors to obtain the best performance with Chronicle Queue.
Replication ::
Replication (Enterprise) ::
Replication is an Enterprise feature allowing queues to be automatically replicated across machines or cores.
This chapter teaches how that is accomplished.
Encryption ::
Encryption (Enterprise) ::
Learn how to protect the contents of a Chronicle Queue with encryption.

=== Runnable Demos
Expand All @@ -60,7 +60,7 @@ You can find our demos link:https://github.com/OpenHFT/Chronicle-Queue-Demo/tree

Chronicle Queue Enterprise is a commercially supported version of Chronicle Queue OSS.
This documentation covers the open source functionality but is also extended by a few chapters describing enterprise specific features.
These sections are clearly marked with a star-symbol (★) and include:
These sections are clearly marked with an Enterprise marker and include:

* Encryption of message queues and messages.

Expand Down
2 changes: 1 addition & 1 deletion docs/antora/modules/performance/nav.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
* xref:performance.adoc[Performance Tuning]
** xref:performance-test-results.adoc[Performance Test Results]
** xref:pretouching.adoc[Pre-Touching ]
** xref:pretouching.adoc[Pre-Touching (Enterprise)]
** xref:contended-writes.adoc[Contended Writes]
2 changes: 1 addition & 1 deletion docs/antora/modules/performance/pages/performance.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ author: Julia Gustafsson
On most systems `System.nanoTime()` is roughly the number of nanoseconds since the system last rebooted (although different JVMs may behave differently).
This is the same across JVMs on the same machine, but wildly different between machines.
The absolute difference when it comes to machines is meaningless.
However, the information can be used to detect outliers; you cant determine what the best latency is, but you can determine how far off the best latencies you are.
However, the information can be used to detect outliers; you can't determine what the best latency is, but you can determine how far off the best latencies you are.
This is useful if you are focusing on the 99th percentile latencies.
We have a class called `RunningMinimum` to obtain timings from different machines, while compensating for a drift in the nanoTime between machines.
The more often you take measurements, the more accurate this running minimum is.
Expand Down
4 changes: 2 additions & 2 deletions docs/antora/modules/performance/pages/pretouching.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ There are two ways to perform pre-touching, outlined below:

Triggering pre-touches manually ::
Perform pre-touching once for the current appender using `ExcerptAppender.pretouch()`
Automatically pre-touching using Chronicle's preloader ::
Automatically pre-touching using Chronicle's preloader (Enterprise) ::
The preloader automatically pre-touches segments at a given time interval _(Enterprise feature)_.

== Pre-Touching Once
Expand Down Expand Up @@ -50,7 +50,7 @@ newSingleThreadScheduledExecutor().scheduleAtFixedRate(
<3> The period between successive executions
<4> The time unit of the `initialDelay` and period parameters

== Automatic Pre-Touching
== Automatic Pre-Touching (Enterprise)

The easiest way to configure periodical pre-touching is by using Chronicle's out-of-process preloader.
It can be activated in the Chronicle Queue Builder as follows:
Expand Down
4 changes: 2 additions & 2 deletions docs/antora/modules/queue-operations/pages/appending.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ As shown above, the document context can be used to append data directly:
[source,java]
----
try (final DocumentContext dc = appender.writingDocument()) {
dc.wire().write().text(Your text data);
dc.wire().write().text("Your text data");
}
----

Expand All @@ -55,7 +55,7 @@ The length of the data is written to the header when the try-with-resources bloc
[source,java]
----
try (final DocumentContext dc = appender.writingDocument()) {
dc.wire().write().text(your text data);
dc.wire().write().text("your text data");
System.out.println("your data was store to index="+ dc.index());
}
----
Expand Down
2 changes: 1 addition & 1 deletion docs/antora/modules/queue-operations/pages/indexing.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ queue.lastIndex()

This will return the index of the last data present in the queue (or -1 for an empty queue).

WARNING: If the queue is being written to concurrently its possible the value may be an under-estimate, as subsequent entries may have been written even before it was returned.
WARNING: If the queue is being written to concurrently it's possible the value may be an under-estimate, as subsequent entries may have been written even before it was returned.

[#_tail_from_index]
== Tail from Index
Expand Down
9 changes: 1 addition & 8 deletions docs/antora/modules/queue-operations/pages/tailing.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -200,14 +200,7 @@ Chronicle Queue stores its data in binary format, with a file extension of `cq4`

[source,text]
----
\�@πheader∂SCQStoreÇE��»wireType∂WireTypeÊBINARYÕwritePositionèèèèß��������ƒroll∂SCQSRollÇ*���∆length¶ÄÓ6�∆format
ÎyyyyMMdd-HH≈epoch¶ÄÓ6�»indexing∂ SCQSIndexingÇN��� indexCount•�ÃindexSpacingÀindex2Indexé����ß��������…lastIndexé�
���ß��������fllastAcknowledgedIndexReplicatedé�����ߡˇˇˇˇˇˇˇ»recovery∂TimedStoreRecoveryÇ���…timeStampèèèß
���������������������������������������������������������������������������������������������
���������������������������������������������������������������������������������������������
���������������������������������������������������������������������������������������������
���������������������������������������������������������������������������������������������
�����������������������������������������������������������������
[binary cq4 bytes omitted]
----

This can often be a bit difficult to read, so it is better to dump the `cq4` files as text.
Expand Down
2 changes: 1 addition & 1 deletion docs/antora/modules/replication/nav.adoc
Original file line number Diff line number Diff line change
@@ -1 +1 @@
* xref:replication.adoc[Queue Replication ]
* xref:replication.adoc[Queue Replication (Enterprise)]
9 changes: 1 addition & 8 deletions docs/antora/modules/tailing/pages/tailing.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -200,14 +200,7 @@ Chronicle Queue stores its data in binary format, with a file extension of `cq4`

[source,text]
----
\�@πheader∂SCQStoreÇE��»wireType∂WireTypeÊBINARYÕwritePositionèèèèß��������ƒroll∂SCQSRollÇ*���∆length¶ÄÓ6�∆format
ÎyyyyMMdd-HH≈epoch¶ÄÓ6�»indexing∂ SCQSIndexingÇN��� indexCount•�ÃindexSpacingÀindex2Indexé����ß��������…lastIndexé�
���ß��������fllastAcknowledgedIndexReplicatedé�����ߡˇˇˇˇˇˇˇ»recovery∂TimedStoreRecoveryÇ���…timeStampèèèß
���������������������������������������������������������������������������������������������
���������������������������������������������������������������������������������������������
���������������������������������������������������������������������������������������������
���������������������������������������������������������������������������������������������
�����������������������������������������������������������������
[binary cq4 bytes omitted]
----

This can often be a bit difficult to read, so it is better to dump the `cq4` files as text.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ default long lastReadIndex() {
* queue.lastCycle() is cached, as finding the last cycle is expensive, it requires asking the
* directory for the Files.list() so, this cache is only refreshed if the call toEnd() is in a
* new millisecond. Hence, a whole milliseconds with of data could be added to the
* chronicle-queue that toEnd() wont see. For appenders that are using the same queue
* chronicle-queue that toEnd() won't see. For appenders that are using the same queue
* instance ( and with then same JVM ), they can be informed that the last cycle has
* changed, this will yield better results, but atomicity can still not be guaranteed.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@
* {@link SingleChronicleQueueStore}. It functions as a factory to produce queue stores,
* utilizing a rolling chronicle queue and a wire to initialize each store.
*
* This interface extends {@link BiFunction}, meaning it takes two arguments — a
* {@link RollingChronicleQueue} and a {@link Wire} and returns a {@link SingleChronicleQueueStore}.
* This interface extends {@link BiFunction}, taking a {@link RollingChronicleQueue}
* and a {@link Wire} and returning a {@link SingleChronicleQueueStore}.
*/
@FunctionalInterface
public interface WireStoreFactory extends BiFunction<RollingChronicleQueue, Wire, SingleChronicleQueueStore> {
Expand Down
Loading