Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ defmodule MyBroadway do
end
```

Once your Broadway module is defined, you just need to add it as a child of your application supervision tree as `{MyBroadway, []}`.
Once your Broadway module is defined, add it as a child of your application supervision tree as `{MyBroadway, []}`.

## Comparison to Flow

Expand Down
14 changes: 7 additions & 7 deletions guides/examples/amazon-sqs.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Broadway can work seamlessly with both, Standard and FIFO queues.

## Getting Started

In order to use Broadway with SQS, we need to:
To use Broadway with SQS, we need to:

1. Create a SQS queue (or use an existing one)
1. Configure our Elixir project to use Broadway
Expand All @@ -43,7 +43,7 @@ which is a Broadway SQS Connector provided by [Dashbit](https://dashbit.co/).

### Starting a new project

If you plan to start a new project, just run:
If you plan to start a new project, run:

$ mix new my_app --sup

Expand Down Expand Up @@ -71,7 +71,7 @@ pipeline, we need to define three functions: `start_link/1`,
`handle_message/3` and `handle_batch/4`. We will cover `start_link/1`
in this section and the `handle_` callbacks in the next one.

Similar to other process-based behaviour, `start_link/1` simply
Similar to other process-based behaviour, `start_link/1`
delegates to `Broadway.start_link/2`, which should define the
producers, processors, and batchers in the Broadway pipeline.
Assuming we want to consume messages from a queue called
Expand Down Expand Up @@ -135,9 +135,9 @@ module docs as well as `Broadway.start_link/2`.

## Implement Broadway callbacks

In order to process incoming messages, we need to implement the
To process incoming messages, we need to implement the
required callbacks. For the sake of simplicity, we're considering that
all messages received from the queue are just numbers:
all messages received from the queue are numbers:

defmodule MyBroadway do
use Broadway
Expand Down Expand Up @@ -169,7 +169,7 @@ For more information, see `c:Broadway.handle_message/3` and

## Run the Broadway pipeline

To run your `Broadway` pipeline, you just need to add as a child in
To run your `Broadway` pipeline, add it as a child in
a supervision tree. Most applications have a supervision tree defined
at `lib/my_app/application.ex`. You can add Broadway as a child to a
supervisor as follows:
Expand Down Expand Up @@ -233,7 +233,7 @@ your needs.
...callbacks...
end

In order to get a good set of configurations for your pipeline, it's
To get a good set of configurations for your pipeline, it's
important to respect the limitations of the servers you're running,
as well as the limitations of the services you're providing/consuming
data to/from. Broadway comes with telemetry, so you can measure your
Expand Down
12 changes: 6 additions & 6 deletions guides/examples/apache-kafka.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Kafka is a distributed streaming platform that has three key capabilities:

## Getting Started

In order to use Broadway with Kafka, we need to:
To use Broadway with Kafka, we need to:

1. Create a stream of records (or use an existing one)
1. Configure your Elixir project to use Broadway
Expand Down Expand Up @@ -69,7 +69,7 @@ we need to define three functions: `start_link/1`, `handle_message/3`
and optionally `handle_batch/4`. We will cover `start_link/1` in this
section and the `handle_` callbacks in the next one.

Similar to other process-based behaviours, `start_link/1` simply
Similar to other process-based behaviours, `start_link/1`
delegates to `Broadway.start_link/2`, which should define the
producers, processors, and batchers in the Broadway pipeline.
Assuming we want to consume messages from a topic called
Expand Down Expand Up @@ -125,9 +125,9 @@ module docs as well as `Broadway.start_link/2`.

## Implement Broadway callbacks

In order to process incoming messages, we need to implement the
To process incoming messages, we need to implement the
required callbacks. For the sake of simplicity, we're considering that
all messages received from the topic are just numbers:
all messages received from the topic are numbers:

defmodule MyBroadway do
use Broadway
Expand Down Expand Up @@ -163,7 +163,7 @@ For more information, see `c:Broadway.handle_message/3` and

## Run the Broadway pipeline

To run your `Broadway` pipeline, you just need to add as a child in
To run your `Broadway` pipeline, add it as a child in
a supervision tree. Most applications have a supervision tree defined
at `lib/my_app/application.ex`. You can add Broadway as a child to a
supervisor as follows:
Expand Down Expand Up @@ -242,7 +242,7 @@ for details.
By setting the `concurrency` option, you define the number of concurrent processes
that will be started by Broadway, allowing you to have full control over the
concurrency level in each layer of the pipeline. Keep in mind that since the
concurrency model provided by **Kafka** is based on **partitioning**, in order to take
concurrency model provided by **Kafka** is based on **partitioning**, to take
full advantage of this model, you need to set the `concurrency` option for
your processors and batchers accordingly. Having less concurrency than topic/partitions
assigned will result in individual processors handling more than one partition,
Expand Down
16 changes: 8 additions & 8 deletions guides/examples/google-cloud-pubsub.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Cloud Pub/Sub is a fully-managed real-time messaging service provided by Google.

## Getting Started

In order to use Broadway with Cloud Pub/Sub you need to:
To use Broadway with Cloud Pub/Sub you need to:

1. Setup a Cloud Pub/Sub project
1. Configure your Elixir project to use Broadway
Expand All @@ -13,7 +13,7 @@ In order to use Broadway with Cloud Pub/Sub you need to:
1. Run the Broadway pipeline
1. Tune the configuration (Optional)

If you are just getting familiar with Google Pub/Sub, refer to [the documentation](https://cloud.google.com/pubsub/docs/)
If you are getting familiar with Google Pub/Sub, refer to [the documentation](https://cloud.google.com/pubsub/docs/)
to get started. Instead of testing against a live environment, you may also consider using the
[emulator](https://cloud.google.com/pubsub/docs/emulator) to simulate integrating with Cloud
Pub/Sub.
Expand Down Expand Up @@ -46,15 +46,15 @@ A new topic:
$ gcloud pubsub topics create test-topic --project test-pubsub
Created topic [projects/test-pubsub/topics/test-topic].

> Note: If you run this command immediately after creating a new Google Cloud project, you may receive an error indicating that your project's organization policy is still being provisioned. Just wait a couple minutes and try again.
> Note: If you run this command immediately after creating a new Google Cloud project, you may receive an error indicating that your project's organization policy is still being provisioned. Wait a couple minutes and try again.

And a new subscription:

$ gcloud pubsub subscriptions create test-subscription --project test-pubsub --topic test-topic
Created subscription [projects/test-pubsub/subscriptions/test-subscription].

We also need a [service account](https://cloud.google.com/iam/docs/service-accounts), an IAM
policy, as well as API credentials in order to programmatically work with the service. First, let's
policy, as well as API credentials to programmatically work with the service. First, let's
create the service account:

$ gcloud iam service-accounts create test-account --project test-pubsub
Expand Down Expand Up @@ -91,7 +91,7 @@ which is a Broadway Cloud Pub/Sub Connector provided by [Dashbit](https://dashbi

### Starting a new project

If you plan to start a new project, just run:
If you plan to start a new project, run:

$ mix new my_app --sup

Expand All @@ -118,7 +118,7 @@ Broadway is a process-based behaviour and to define a Broadway pipeline, we need
functions: `start_link/1`, `handle_message/3` and `handle_batch/4`. We will cover `start_link/1`
in this section and the `handle_` callbacks in the next one.

Similar to other process-based behaviour, `start_link/1` simply delegates to
Similar to other process-based behaviour, `start_link/1` delegates to
`Broadway.start_link/2`, which should define the producers, processors, and batchers in the
Broadway pipeline. Assuming we want to consume messages from the `test-subscription`, the minimal
configuration would be:
Expand Down Expand Up @@ -164,7 +164,7 @@ For general information about setting up Broadway, see `Broadway` module docs as

## Implement Broadway callbacks

In order to process incoming messages, we need to implement the required callbacks. For the sake
To process incoming messages, we need to implement the required callbacks. For the sake
of simplicity, we're considering that all messages received from the queue are strings and our
processor calls `String.upcase/1` on them:

Expand Down Expand Up @@ -285,7 +285,7 @@ your needs.
...callbacks...
end

In order to get a good set of configurations for your pipeline, it's
To get a good set of configurations for your pipeline, it's
important to respect the limitations of the servers you're running,
as well as the limitations of the services you're providing/consuming
data to/from. Broadway comes with telemetry, so you can measure your
Expand Down
14 changes: 7 additions & 7 deletions guides/examples/rabbitmq.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Queuing Protocol (AMQP).

## Getting Started

In order to use Broadway with RabbitMQ, we need to:
To use Broadway with RabbitMQ, we need to:

1. [Create a queue](#create-a-queue) (or use an existing one)
1. [Configure our Elixir project to use Broadway](#configure-the-project)
Expand Down Expand Up @@ -35,7 +35,7 @@ following command:

$ rabbitmqadmin declare queue name=my_queue durable=true

You can list all declared queues to see our the one we've just created:
You can list all declared queues to see the one we created:

$ rabbitmqctl list_queues
Timeout: 60.0 seconds ...
Expand Down Expand Up @@ -76,7 +76,7 @@ we need to define three functions: `start_link/1`, `handle_message/3`
and optionally `handle_batch/4`. We will cover `start_link/1` in this
section and the `handle_` callbacks in the next one.

Similar to other process-based behaviours, `start_link/1` simply
Similar to other process-based behaviours, `start_link/1`
delegates to `Broadway.start_link/2`, which should define the
producers, processors, and batchers in the Broadway pipeline.
Assuming we want to consume messages from a queue called
Expand Down Expand Up @@ -141,9 +141,9 @@ module docs as well as `Broadway.start_link/2`.

## Implement Broadway callbacks

In order to process incoming messages, we need to implement the
To process incoming messages, we need to implement the
required callbacks. For the sake of simplicity, we're considering that
all messages received from the queue are just numbers:
all messages received from the queue are numbers:

defmodule MyBroadway do
use Broadway
Expand Down Expand Up @@ -181,7 +181,7 @@ For more information, see `c:Broadway.handle_message/3` and

## Run the Broadway pipeline

To run your `Broadway` pipeline, you just need to add as a child in
To run your `Broadway` pipeline, add it as a child in
a supervision tree. Most applications have a supervision tree defined
at `lib/my_app/application.ex`. You can add Broadway as a child to a
supervisor as follows:
Expand Down Expand Up @@ -260,7 +260,7 @@ The `:prefetch_count` setting provides back-pressure by instructing RabbitMQ to
See the ["Back-pressure and :prefetch_count"](https://hexdocs.pm/broadway_rabbitmq/BroadwayRabbitMQ.Producer.html#module-back-pressure-and-prefetch_count)
section of the `BroadwayRabbitMQ` documentation for details.

In order to get a good set of configurations for your pipeline, it's
To get a good set of configurations for your pipeline, it's
important to respect the limitations of the servers you're running,
as well as the limitations of the services you're providing/consuming
data to/from. Broadway comes with telemetry, so you can measure your
Expand Down
2 changes: 1 addition & 1 deletion guides/internals/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ handle_batch/4 runs here -> [batch processor][batch processor]
calculations, convert data into a custom json format etc. Here is where
the code from `handle_message/3` runs.
* `Broadway.Batcher` - Creates batches of messages based on the
batcher's key. One batcher for each key will be created.
batcher's key. Broadway creates one batcher for each key.
* `Broadway.BatchProcessor` - This is where the code from `handle_batch/4` runs.

## The supervision tree
Expand Down
12 changes: 6 additions & 6 deletions lib/broadway.ex
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ defmodule Broadway do

## The Broadway behaviour

In order to use Broadway, you need to:
To use Broadway, you need to:

1. Define your pipeline configuration
2. Define a module implementing the Broadway behaviour
Expand Down Expand Up @@ -530,7 +530,7 @@ defmodule Broadway do
This can be done with the `:partition_by` option, which enforces that
messages with a given property are always forwarded to the same stage.

In order to provide partitioning throughout the whole pipeline, just
To provide partitioning throughout the whole pipeline,
set `:partition_by` at the root of your configuration:

defmodule MyBroadway do
Expand Down Expand Up @@ -929,7 +929,7 @@ defmodule Broadway do
`min_demand` value. Producers which are push-based, rather than pull-based,
such as `BroadwayRabbitMQ.Producer`, are more likely to send messages as they
arrive (which may skip batching altogether and always be single element lists).
In other words, this callback is simply a convenience for preparing messages,
In other words, this callback is a convenience for preparing messages,
it does not guarantee the messages will be accumulated to a certain length.
For effective batch processing, see `c:handle_batch/4`.

Expand Down Expand Up @@ -959,10 +959,10 @@ defmodule Broadway do

This is the place to do any kind of processing with the incoming message,
e.g., transform the data into another data structure, call specific business
logic to do calculations. Basically, any CPU bounded task that runs against
logic to do calculations. Any CPU bounded task that runs against
a single message should be processed here.

In order to update the data after processing, use the
To update the data after processing, use the
`Broadway.Message.update_data/2` function. This way the new message can be
properly forwarded and handled by the batcher:

Expand Down Expand Up @@ -1127,7 +1127,7 @@ defmodule Broadway do

## Options

In order to set up how the pipeline created by Broadway should work,
To set up how the pipeline created by Broadway should work,
you need to specify the blueprint of the pipeline. You can
do this by passing a set of options to `start_link/2`.
Each component of the pipeline has its own set of options.
Expand Down
2 changes: 1 addition & 1 deletion lib/broadway/acknowledger.ex
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ defmodule Broadway.Acknowledger do
implement this behaviour and consider how the technology
you're working with handles message acknowledgement.

The `c:ack/3` callback must be implemented in order to notify
The `c:ack/3` callback must be implemented to notify
the origin of the data that a message can be safely removed
after been successfully processed and published. In case of
failed messages or messages without acknowledgement, depending
Expand Down
Loading