-
Notifications
You must be signed in to change notification settings - Fork 204
Expand file tree
/
Copy pathsettingref.txt
More file actions
2614 lines (1603 loc) · 60.2 KB
/
Copy pathsettingref.txt
File metadata and controls
2614 lines (1603 loc) · 60.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
.. _settings-common:
Commonly Used Settings
======================
.. setting:: autodiscover
``autodiscover``
----------------
:type: :class:`~typing.Any`
:default: :const:`False`
Automatic discovery of agents, tasks, timers, views and commands.
Faust has an API to add different :mod:`asyncio` services and other
user extensions, such as "Agents", HTTP web views,
command-line commands, and timers to your Faust workers.
These can be defined in any module, so to discover them at startup,
the worker needs to traverse packages looking for them.
.. warning::
The autodiscovery functionality uses the :pypi:`Venusian` library
to scan wanted packages for ``@app.agent``, ``@app.page``,
``@app.command``, ``@app.task`` and ``@app.timer`` decorators,
but to do so, it's required to traverse the package path and import
every module in it.
Importing random modules like this can be dangerous so make sure
you follow Python programming best practices. Do not start
threads; perform network I/O; do test monkey-patching for mocks or
similar, as a side effect of importing a module. If you encounter
a case such as this then please find a way to perform your
action in a lazy manner.
.. warning::
If the above warning is something you cannot fix, or if it's out
of your control, then please set ``autodiscover=False`` and make
sure the worker imports all modules where your
decorators are defined.
The value for this argument can be:
``bool``
If ``App(autodiscover=True)`` is set, the autodiscovery will
scan the package name described in the ``origin`` attribute.
The ``origin`` attribute is automatically set when you start
a worker using the :program:`faust` command line program,
for example:
.. sourcecode:: console
faust -A example.simple worker
The :option:`-A <faust -A>`, option specifies the app, but you
can also create a shortcut entry point by calling ``app.main()``:
.. sourcecode:: python
if __name__ == '__main__':
app.main()
Then you can start the :program:`faust` program by executing for
example ``python myscript.py worker --loglevel=INFO``, and it
will use the correct application.
``Sequence[str]``
The argument can also be a list of packages to scan::
app = App(..., autodiscover=['proj_orders', 'proj_accounts'])
``Callable[[], Sequence[str]]``
The argument can also be a function returning a list of packages
to scan::
def get_all_packages_to_scan():
return ['proj_orders', 'proj_accounts']
app = App(..., autodiscover=get_all_packages_to_scan)
``False``
If everything you need is in a self-contained module, or you
import the stuff you need manually, just set ``autodiscover``
to False and don't worry about it :-)
.. admonition:: Django
When using :pypi:`Django` and the :envvar:`DJANGO_SETTINGS_MODULE`
environment variable is set, the Faust app will scan all packages
found in the ``INSTALLED_APPS`` setting.
If you're using Django you can use this to scan for
agents/pages/commands in all packages
defined in ``INSTALLED_APPS``.
Faust will automatically detect that you're using Django
and do the right thing if you do::
app = App(..., autodiscover=True)
It will find agents and other decorators in all of the
reusable Django applications. If you want to manually control
what packages are traversed, then provide a list::
app = App(..., autodiscover=['package1', 'package2'])
or if you want exactly :const:`None` packages to be traversed,
then provide a False:
app = App(.., autodiscover=False)
which is the default, so you can simply omit the argument.
.. tip::
For manual control over autodiscovery, you can also call the
:meth:`@discover` method manually.
.. setting:: datadir
``datadir``
-----------
:type: :class:`str` / :class:`~pathlib.Path`
:default: ``'{conf.name}-data'``
:environment: :envvar:`APP_DATADIR`
:related-command-options: :option:`faust --data-dir`
Application data directory.
The directory in which this instance stores the data used by
local tables, etc.
.. seealso::
- The data directory can also be set using the
:option:`faust --datadir` option, from the command-line,
so there is usually no reason to provide a default value
when creating the app.
.. setting:: tabledir
``tabledir``
------------
:type: :class:`str` / :class:`~pathlib.Path`
:default: ``'tables'``
:environment: :envvar:`APP_TABLEDIR`
Application table data directory.
The directory in which this instance stores local table data.
Usually you will want to configure the :setting:`datadir` setting,
but if you want to store tables separately you can configure this one.
If the path provided is relative (it has no leading slash), then the
path will be considered to be relative to the :setting:`datadir`
setting.
.. setting:: debug
``debug``
---------
:type: :class:`bool`
:default: :const:`False`
:environment: :envvar:`APP_DEBUG`
:related-command-options: :option:`faust --debug`
Use in development to expose sensor information endpoint.
.. tip::
If you want to enable the sensor statistics endpoint in production,
without enabling the :setting:`debug` setting, you can do so
by adding the following code:
.. sourcecode:: python
app.web.blueprints.add(
'/stats/', 'faust.web.apps.stats:blueprint')
.. setting:: env_prefix
``env_prefix``
--------------
.. versionadded:: 1.11
:type: :class:`str`
:default: :const:`None`
:environment: :envvar:`APP_ENV_PREFIX`
Environment variable prefix.
When configuring Faust by environent variables,
this adds a common prefix to all Faust environment value names.
.. setting:: id_format
``id_format``
-------------
:type: :class:`str`
:default: ``'{id}-v{self.version}'``
:environment: :envvar:`APP_ID_FORMAT`
Application ID format template.
The format string used to generate the final :setting:`id` value
by combining it with the :setting:`version` parameter.
.. setting:: origin
``origin``
----------
:type: :class:`str`
:default: :const:`None`
The reverse path used to find the app.
For example if the app is located in::
from myproj.app import app
Then the ``origin`` should be ``"myproj.app"``.
The :program:`faust worker` program will try to automatically set
the origin, but if you are having problems with auto generated names
then you can set origin manually.
.. setting:: timezone
``timezone``
------------
.. versionadded:: 1.4
:type: :class:`~datetime.tzinfo`
:default: ``datetime.timezone.utc``
:environment: :envvar:`TIMEZONE`
Project timezone.
The timezone used for date-related functionality such as cronjobs.
.. setting:: version
``version``
-----------
:type: :class:`int`
:default: ``1``
:environment: :envvar:`APP_VERSION`
App version.
Version of the app, that when changed will create a new isolated
instance of the application. The first version is 1,
the second version is 2, and so on.
.. admonition:: Source topics will not be affected by a version change.
Faust applications will use two kinds of topics: source topics, and
internally managed topics. The source topics are declared by the
producer, and we do not have the opportunity to modify any
configuration settings, like number of partitions for a source
topic; we may only consume from them. To mark a topic as internal,
use: ``app.topic(..., internal=True)``.
.. setting:: blocking_timeout
``blocking_timeout``
--------------------
:type: :class:`float` / :class:`~datetime.timedelta`
:default: :const:`None`
:environment: :envvar:`BLOCKING_TIMEOUT`
:related-command-options: :option:`faust --blocking-timeout`
Blocking timeout (in seconds).
When specified the worker will start a periodic signal based
timer that only triggers when the loop has been blocked
for a time exceeding this timeout.
This is the most safe way to detect blocking, but could have
adverse effects on libraries that do not automatically
retry interrupted system calls.
Python itself does retry all interrupted system calls
since version 3.5 (see :pep:`475`), but this might not
be the case with C extensions added to the worker by the user.
The blocking detector is a background thread
that periodically wakes up to either arm a timer, or cancel
an already armed timer. In pseudocode:
.. sourcecode:: python
while True:
# cancel previous alarm and arm new alarm
signal.signal(signal.SIGALRM, on_alarm)
signal.setitimer(signal.ITIMER_REAL, blocking_timeout)
# sleep to wakeup just before the timeout
await asyncio.sleep(blocking_timeout * 0.96)
def on_alarm(signum, frame):
logger.warning('Blocking detected: ...')
If the sleep does not wake up in time the alarm signal
will be sent to the process and a traceback will be logged.
.. setting:: broker
``broker``
----------
:type: :class:`str` / :class:`~yarl.URL` / ``[`` :class:`str` ``]``
:default: :const:`None`
:environment: :envvar:`BROKER_URL`
Broker URL, or a list of alternative broker URLs.
Faust needs the URL of a "transport" to send and receive messages.
Currently, the only supported production transport is ``kafka://``.
This uses the :pypi:`aiokafka` client under the hood, for consuming and
producing messages.
You can specify multiple hosts at the same time by separating them
using the semi-comma:
.. sourcecode:: text
kafka://kafka1.example.com:9092;kafka2.example.com:9092
Which in actual code looks like this:
.. sourcecode:: python
BROKERS = 'kafka://kafka1.example.com:9092;kafka2.example.com:9092'
app = faust.App(
'id',
broker=BROKERS,
)
You can also pass a list of URLs:
.. sourcecode:: python
app = faust.App(
'id',
broker=['kafka://kafka1.example.com:9092',
'kafka://kafka2.example.com:9092'],
)
.. seealso::
You can configure the transport used for consuming and producing
separately, by setting the :setting:`broker_consumer` and
:setting:`broker_producer` settings.
This setting is used as the default.
**Available Transports**
- ``kafka://``
Alias to ``aiokafka://``
- ``aiokafka://``
The recommended transport using the :pypi:`aiokafka` client.
Limitations: None
- ``confluent://``
Experimental transport using the :pypi:`confluent-kafka` client.
Limitations: Does not do sticky partition assignment (not
suitable for tables), and do not create any necessary internal
topics (you have to create them manually).
.. setting:: broker_credentials
``broker_credentials``
----------------------
.. versionadded:: 1.5
:type: :class:`~faust.types.auth.CredentialsT`
:default: :const:`None`
:environment: :envvar:`BROKER_CREDENTIALS`
Broker authentication mechanism.
Specify the authentication mechanism to use when connecting to
the broker.
The default is to not use any authentication.
SASL Authentication
You can enable SASL authentication via plain text:
.. sourcecode:: python
app = faust.App(
broker_credentials=faust.SASLCredentials(
username='x',
password='y',
))
.. warning::
Do not use literal strings when specifying passwords in
production, as they can remain visible in stack traces.
Instead the best practice is to get the password from
a configuration file, or from the environment:
.. sourcecode:: python
BROKER_USERNAME = os.environ.get('BROKER_USERNAME')
BROKER_PASSWORD = os.environ.get('BROKER_PASSWORD')
app = faust.App(
broker_credentials=faust.SASLCredentials(
username=BROKER_USERNAME,
password=BROKER_PASSWORD,
))
OAuth2 Authentication
You can enable SASL authentication via OAuth2 Bearer tokens:
.. sourcecode:: python
import faust
from asyncio import get_running_loop
from aiokafka.helpers import create_ssl_context
from aiokafka.conn import AbstractTokenProvider
class TokenProvider(AbstractTokenProvider):
async def token(self):
return await get_running_loop().run_in_executor(
None, self.get_token)
def get_token(self):
return 'token'
app = faust.App(
broker_credentials=faust.OAuthCredentials(
oauth_cb=TokenProvider()
ssl_context=create_ssl_context()
)
)
.. info::
The implementation should ensure token reuse so that multiple
calls at connect time do not create multiple tokens.
The implementation should also periodically refresh the token in order to
guarantee that each call returns an unexpired token.
Token Providers MUST implement the :meth:`token` method
GSSAPI Authentication
GSSAPI authentication over plain text:
.. sourcecode:: python
app = faust.App(
broker_credentials=faust.GSSAPICredentials(
kerberos_service_name='faust',
kerberos_domain_name='example.com',
),
)
GSSAPI authentication over SSL:
.. sourcecode:: python
import ssl
ssl_context = ssl.create_default_context(
purpose=ssl.Purpose.SERVER_AUTH, cafile='ca.pem')
ssl_context.load_cert_chain(
'client.cert', keyfile='client.key')
app = faust.App(
broker_credentials=faust.GSSAPICredentials(
kerberos_service_name='faust',
kerberos_domain_name='example.com',
ssl_context=ssl_context,
),
)
SSL Authentication
Provide an SSL context for the Kafka broker connections.
This allows Faust to use a secure SSL/TLS connection for the
Kafka connections and enabling certificate-based authentication.
.. sourcecode:: python
import ssl
ssl_context = ssl.create_default_context(
purpose=ssl.Purpose.SERVER_AUTH, cafile='ca.pem')
ssl_context.load_cert_chain(
'client.cert', keyfile='client.key')
app = faust.App(..., broker_credentials=ssl_context)
.. setting:: ssl_context
``ssl_context``
---------------
:type: :class:`~ssl.SSLContext`
:default: :const:`None`
SSL configuration.
See :setting:`credentials`.
.. setting:: logging_config
``logging_config``
------------------
.. versionadded:: 1.5
:type: :class:`dict`
:default: :const:`None`
Logging dictionary configuration.
Optional dictionary for logging configuration, as supported
by :func:`logging.config.dictConfig`.
.. setting:: loghandlers
``loghandlers``
---------------
:type: ``[`` :class:`~logging.Handler` ``]``
:default: :const:`None`
List of custom logging handlers.
Specify a list of custom log handlers to use in worker instances.
.. setting:: processing_guarantee
``processing_guarantee``
------------------------
.. versionadded:: 1.5
:type: :class:`str`
:default: ``<ProcessingGuarantee.AT_LEAST_ONCE: 'at_least_once'>``
:environment: :envvar:`PROCESSING_GUARANTEE`
The processing guarantee that should be used.
Possible values are "at_least_once" (default) and "exactly_once".
Note that if exactly-once processing is enabled consumers are
configured with ``isolation.level="read_committed"`` and producers
are configured with ``retries=Integer.MAX_VALUE`` and
``enable.idempotence=true`` per default.
Note that by default exactly-once processing requires a cluster of
at least three brokers what is the recommended setting for production.
For development you can change this, by adjusting broker setting
``transaction.state.log.replication.factor`` to the number of brokers
you want to use.
.. setting:: store
``store``
---------
:type: :class:`str` / :class:`~yarl.URL`
:default: ``'memory://'``
:environment: :envvar:`APP_STORE`
Table storage backend URL.
The backend used for table storage.
Tables are stored in-memory by default, but you should
not use the ``memory://`` store in production.
In production, a persistent table store, such as ``rocksdb://`` is
preferred.
.. setting:: cache
``cache``
---------
.. versionadded:: 1.2
:type: :class:`str` / :class:`~yarl.URL`
:default: ``'memory://'``
:environment: :envvar:`CACHE_URL`
Cache backend URL.
Optional backend used for Memcached-style caching.
URL can be:
+ ``redis://host``
+ ``rediscluster://host``, or
+ ``memory://``.
.. _settings-agent:
Advanced Agent Settings
=======================
.. setting:: agent_supervisor
``agent_supervisor``
--------------------
:type: :class:`str` / :class:`~typing.Type`
:default: ``'mode.OneForOneSupervisor'``
:environment: :envvar:`AGENT_SUPERVISOR`
Default agent supervisor type.
An agent may start multiple instances (actors) when
the concurrency setting is higher than one (e.g.
``@app.agent(concurrency=2)``).
Multiple instances of the same agent are considered to be in the same
supervisor group.
The default supervisor is the :class:`mode.OneForOneSupervisor`:
if an instance in the group crashes, we restart that instance only.
These are the supervisors supported:
+ :class:`mode.OneForOneSupervisor`
If an instance in the group crashes we restart only that instance.
+ :class:`mode.OneForAllSupervisor`
If an instance in the group crashes we restart the whole group.
+ :class:`mode.CrashingSupervisor`
If an instance in the group crashes we stop the whole application,
and exit so that the Operating System supervisor can restart us.
+ :class:`mode.ForfeitOneForOneSupervisor`
If an instance in the group crashes we give up on that instance
and never restart it again (until the program is restarted).
+ :class:`mode.ForfeitOneForAllSupervisor`
If an instance in the group crashes we stop all instances
in the group and never restarted them again (until the program is
restarted).
.. _settings-broker:
Advanced Broker Settings
========================
.. setting:: broker_consumer
``broker_consumer``
-------------------
.. versionadded:: 1.7
:type: :class:`str` / :class:`~yarl.URL` / ``[`` :class:`str` ``]``
:default (alias to setting): :setting:`broker`
:environment: :envvar:`BROKER_CONSUMER_URL`
Consumer broker URL.
You can use this setting to configure the transport used for
producing and consuming separately.
If not set the value found in :setting:`broker` will be used.
.. setting:: broker_producer
``broker_producer``
-------------------
.. versionadded:: 1.7
:type: :class:`str` / :class:`~yarl.URL` / ``[`` :class:`str` ``]``
:default (alias to setting): :setting:`broker`
:environment: :envvar:`BROKER_PRODUCER_URL`
Producer broker URL.
You can use this setting to configure the transport used for
producing and consuming separately.
If not set the value found in :setting:`broker` will be used.
.. setting:: broker_api_version
``broker_api_version``
----------------------
.. versionadded:: 1.10
:type: :class:`str`
:default: ``'auto'``
:environment: :envvar:`BROKER_API_VERSION`
Broker API version,.
This setting is also the default for :setting:`consumer_api_version`,
and :setting:`producer_api_version`.
Negotiate producer protocol version.
The default value - "auto" means use the latest version supported
by both client and server.
Any other version set means you are requesting a specific version of
the protocol.
Example Kafka uses:
**Disable sending headers for all messages produced**
Kafka headers support was added in Kafka 0.11, so you can specify
``broker_api_version="0.10"`` to remove the headers from messages.
.. setting:: broker_check_crcs
``broker_check_crcs``
---------------------
:type: :class:`bool`
:default: :const:`True`
:environment: :envvar:`BROKER_CHECK_CRCS`
Broker CRC check.
Automatically check the CRC32 of the records consumed.
.. setting:: broker_client_id
``broker_client_id``
--------------------
:type: :class:`str`
:default: ``'faust-0.8.9'``
:environment: :envvar:`BROKER_CLIENT_ID`
Broker client ID.
There is rarely any reason to configure this setting.
The client id is used to identify the software used, and is not usually
configured by the user.
.. setting:: broker_commit_every
``broker_commit_every``
-----------------------
:type: :class:`int`
:default: ``10000``
:environment: :envvar:`BROKER_COMMIT_EVERY`
Broker commit message frequency.
Commit offset every n messages.
See also :setting:`broker_commit_interval`, which is how frequently
we commit on a timer when there are few messages being received.
.. setting:: broker_commit_interval
``broker_commit_interval``
--------------------------
:type: :class:`float` / :class:`~datetime.timedelta`
:default: ``2.8``
:environment: :envvar:`BROKER_COMMIT_INTERVAL`
Broker commit time frequency.
How often we commit messages that have been
fully processed (:term:`acked`).
.. setting:: broker_commit_livelock_soft_timeout
``broker_commit_livelock_soft_timeout``
---------------------------------------
:type: :class:`float` / :class:`~datetime.timedelta`
:default: ``300.0``
:environment: :envvar:`BROKER_COMMIT_LIVELOCK_SOFT_TIMEOUT`
Commit livelock timeout.
How long time it takes before we warn that the Kafka commit offset has
not advanced (only when processing messages).
.. setting:: broker_heartbeat_interval
``broker_heartbeat_interval``
-----------------------------
.. versionadded:: 1.0.11
:type: :class:`float` / :class:`~datetime.timedelta`
:default: ``3.0``
:environment: :envvar:`BROKER_HEARTBEAT_INTERVAL`
Broker heartbeat interval.
How often we send heartbeats to the broker, and also how often
we expect to receive heartbeats from the broker.
If any of these time out, you should increase this setting.
.. setting:: broker_max_poll_interval
``broker_max_poll_interval``
----------------------------
.. versionadded:: 1.7
:type: :class:`float` / :class:`~datetime.timedelta`
:default: ``1000.0``
:environment: :envvar:`BROKER_MAX_POLL_INTERVAL`
Broker max poll interval.
The maximum allowed time (in seconds) between calls to consume messages
If this interval is exceeded the consumer
is considered failed and the group will rebalance in order to reassign
the partitions to another consumer group member. If API methods block
waiting for messages, that time does not count against this timeout.
See `KIP-62`_ for technical details.
.. _`KIP-62`:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-62%3A+Allow+consumer+to+send+heartbeats+from+a+background+thread
.. setting:: broker_max_poll_records
``broker_max_poll_records``
---------------------------
.. versionadded:: 1.4
:type: :class:`int`
:default: :const:`None`
:environment: :envvar:`BROKER_MAX_POLL_RECORDS`
Broker max poll records.
The maximum number of records returned in a single call to ``poll()``.
If you find that your application needs more time to process
messages you may want to adjust :setting:`broker_max_poll_records`
to tune the number of records that must be handled on every
loop iteration.
.. setting:: broker_rebalance_timeout
``broker_rebalance_timeout``
----------------------------
.. versionadded:: 1.10
:type: :class:`float` / :class:`~datetime.timedelta`
:default: ``60.0``
:environment: :envvar:`BROKER_REBALANCE_TIMEOUT`
Broker rebalance timeout.
How long to wait for a node to finish rebalancing before the broker
will consider it dysfunctional and remove it from the cluster.
Increase this if you experience the cluster being in a state of
constantly rebalancing, but make sure you also increase the
:setting:`broker_heartbeat_interval` at the same time.
.. note::
The session timeout must not be greater than the
:setting:`broker_request_timeout`.
.. setting:: broker_request_timeout
``broker_request_timeout``
--------------------------
.. versionadded:: 1.4
:type: :class:`float` / :class:`~datetime.timedelta`
:default: ``90.0``
:environment: :envvar:`BROKER_REQUEST_TIMEOUT`
Kafka client request timeout.
.. note::
The request timeout must not be less than the
:setting:`broker_session_timeout`.
.. setting:: broker_session_timeout
``broker_session_timeout``
--------------------------
.. versionadded:: 1.0.11
:type: :class:`float` / :class:`~datetime.timedelta`
:default: ``60.0``
:environment: :envvar:`BROKER_SESSION_TIMEOUT`
Broker session timeout.
How long to wait for a node to finish rebalancing before the broker
will consider it dysfunctional and remove it from the cluster.
Increase this if you experience the cluster being in a state of
constantly rebalancing, but make sure you also increase the
:setting:`broker_heartbeat_interval` at the same time.
.. note::
The session timeout must not be greater than the
:setting:`broker_request_timeout`.
.. _settings-consumer:
Advanced Consumer Settings
==========================
.. setting:: consumer_api_version
``consumer_api_version``
------------------------
.. versionadded:: 1.10
:type: :class:`str`
:default (alias to setting): :setting:`broker_api_version`