This is an automated email from the ASF dual-hosted git repository.
lizhimin pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/rocketmq.git
The following commit(s) were added to refs/heads/develop by this push:
new dee51095e2 [ISSUE #8243] Update Configuration Docs for RocketMQ 5.x
Compatibility (#9258)
dee51095e2 is described below
commit dee51095e2cf3d7d597e0a5de7af4dcd36331d34
Author: DivyanshIITB <divyanshag1...@gmail.com>
AuthorDate: Thu Mar 20 09:11:10 2025 +0530
[ISSUE #8243] Update Configuration Docs for RocketMQ 5.x Compatibility
(#9258)
---
docs/en/Configuration_Client.md | 148 ++++++++++++++++++----------------------
docs/en/Configuration_System.md | 52 ++++++--------
2 files changed, 87 insertions(+), 113 deletions(-)
diff --git a/docs/en/Configuration_Client.md b/docs/en/Configuration_Client.md
index 4679957af5..af83ffcb40 100644
--- a/docs/en/Configuration_Client.md
+++ b/docs/en/Configuration_Client.md
@@ -1,119 +1,107 @@
## Client Configuration
- Relative to RocketMQ's Broker cluster, producers and consumers are client.
In this section, it mainly describes the common behavior configuration of
producers and consumers.
-
-### 1 Client Addressing mode
+Relative to RocketMQ's Broker cluster, producers and consumers are clients.
This section describes the common behavior configuration of producers and
consumers, including updates for **RocketMQ 5.x**.
-```RocketMQ``` can let client find the ```Name Server```, and then find the
```Broker```by the ```Name Server```. Followings show a variety of
configurations, and priority level from highly to lower, the highly priority
configurations can override the lower priority configurations.
+### 1 Client Addressing Mode
-- Specified ```Name Server``` address in the code, and multiple ```Name
Server``` addresses are separated by semicolons
+`RocketMQ` allows clients to locate the `Name Server`, which then helps them
find the `Broker`. Below are various configurations, prioritized from highest
to lowest. Higher priority configurations override lower ones.
+
+- Specified `Name Server` address in the code (multiple addresses separated by
semicolons):
```java
producer.setNamesrvAddr("192.168.0.1:9876;192.168.0.2:9876");
-
consumer.setNamesrvAddr("192.168.0.1:9876;192.168.0.2:9876");
```
-- Specified ```Name Server``` address in the Java setup parameters
+
+- Specified `Name Server` address in Java setup parameters:
```text
-Drocketmq.namesrv.addr=192.168.0.1:9876;192.168.0.2:9876
```
-- Specified ```Name Server``` address in the environment variables
+
+- Specified `Name Server` address in environment variables:
```text
-export NAMESRV_ADDR=192.168.0.1:9876;192.168.0.2:9876
+export NAMESRV_ADDR=192.168.0.1:9876;192.168.0.2:9876
```
-- HTTP static server addressing(default)
-After client started, it will access the http static server address, as:
<http://jmenv.tbsite.net:8080/rocketmq/nsaddr>, this URL return the following
contents:
+- HTTP static server addressing (default):
+Clients retrieve `Name Server` addresses from a static HTTP server:
```text
-192.168.0.1:9876;192.168.0.2:9876
+http://jmenv.tbsite.net:8080/rocketmq/nsaddr
```
-By default, the client accesses the HTTP server every 2 minutes, and update
the local Name Server address.The URL is hardcoded in the code, you can change
the target server by updating ```/etc/hosts``` file, such as add following
configuration at the ```/etc/hosts```:
-```text
-10.232.22.67 jmenv.tbsite.net
-```
-HTTP static server addressing is recommended, because it is simple client
deployment, and the Name Server cluster can be upgraded hot.
+
+- **New in RocketMQ 5.x:**
+ - Improved service discovery mechanism, allowing dynamic Name Server
registration.
+ - Introduces `VIP_CHANNEL_ENABLED` for better failover:
+
+ ```java
+ producer.setVipChannelEnabled(false);
+ consumer.setVipChannelEnabled(false);
+ ```
### 2 Client Configuration
-```DefaultMQProducer```,```TransactionMQProducer```,```DefaultMQPushConsumer```,```DefaultMQPullConsumer```
all extends the ```ClientConfig``` Class, ```ClientConfig``` as the client
common configuration class. Client configuration style like getXXX,setXXX, each
of the parameters can config by spring and also config their in the code. Such
as the ```namesrvAddr``` parameter:
```producer.setNamesrvAddr("192.168.0.1:9876")```, same with the other
parameters.
+`DefaultMQProducer`, `TransactionMQProducer`, `DefaultMQPushConsumer`, and
`DefaultMQPullConsumer` all extend the `ClientConfig` class, which provides
common client configurations.
#### 2.1 Client Common Configuration
-| Parameter Name | Default Value | Description
|
-| ----------------------------- | ------- |
------------------------------------------------------------ |
-| namesrvAddr | | Name Server address list, multiple
NameServer addresses are separated by semicolons |
-| clientIP | local IP | Client local ip address, some
machines will fail to recognize the client IP address, which needs to be
enforced in the code |
-| instanceName | DEFAULT | Name of the client instance,
Multiple producers and consumers created by the client actually share one
internal instance (this instance contains network connection, thread resources,
etc.). |
-| clientCallbackExecutorThreads | 4 | Number of communication layer
asynchronous callback threads |
-| pollNameServerInterval | 30000 | Polling the Name Server interval
in milliseconds |
-| heartbeatBrokerInterval | 30000 | The heartbeat interval, in
milliseconds, is sent to the Broker |
-| persistConsumerOffsetInterval | 5000 | The persistent Consumer consumes
the progress interval in milliseconds |
+| Parameter Name | Default Value | Description |
+|-------------------------------|---------------|--------------|
+| namesrvAddr | | Name Server address list
(semicolon-separated) |
+| clientIP | Local IP | Client's local IP address
(useful if automatic detection fails) |
+| instanceName | DEFAULT | Unique name for the client
instance |
+| clientCallbackExecutorThreads | 4 | Number of communication
layer asynchronous callback threads |
+| pollNameServerInterval | 30000 | Interval (ms) to poll Name
Server |
+| heartbeatBrokerInterval | 30000 | Interval (ms) for sending
heartbeats to Broker |
+| persistConsumerOffsetInterval | 5000 | Interval (ms) for persisting
consumer offsets |
+| **autoUpdateNameServer** (5.x) | true | Automatically update Name
Server addresses from registry |
+| **instanceId** (5.x) | | Unique identifier for each
client instance |
#### 2.2 Producer Configuration
-| Parameter Name | Default Value | Description
|
-| -------------------------------- | ---------------- |
------------------------------------------------------------ |
-| producerGroup | DEFAULT_PRODUCER | The name of the
Producer group. If multiple producers belong to one application and send the
same message, they should be grouped into the same group |
-| createTopicKey | TBW102 | When a message is
sent, topics that do not exist on the server are automatically created and a
Key is specified that can be used to configure the default route to the topic
where the message is sent.|
-| defaultTopicQueueNums | 4 | The number of default
queue when sending messages and auto created topic which not exists the server|
-| sendMsgTimeout | 3000 | Timeout time of
sending message in milliseconds |
-| compressMsgBodyOverHowmuch | 4096 | The message Body
begins to compress beyond the size(the Consumer gets the message automatically
unzipped.), unit of byte|
-| retryAnotherBrokerWhenNotStoreOK | FALSE | If send message and
return sendResult but sendStatus!=SEND_OK, Whether to resend |
-| retryTimesWhenSendFailed | 2 | If send message
failed, maximum number of retries, this parameter only works for synchronous
send mode|
-| maxMessageSize | 4MB | Client limit message
body size, over it may error. Server also limit so need to work with server |
-| transactionCheckListener | | The transaction
message looks back to the listener, if you want send transaction message, you
must setup this
-| checkThreadPoolMinSize | 1 | Minimum of thread in
thread pool when Broker look back Producer transaction status
|
-| checkThreadPoolMaxSize | 1 | Maximum of thread in
thread pool when Broker look back Producer transaction status
|
-| checkRequestHoldMax | 2000 | Producer local buffer
request queue size when Broker look back Producer transaction status
|
-| RPCHook | null | This parameter is
passed in when the Producer is creating, including the pre-processing before
the message sending and the processing after the message response. The user can
do some security control or other operations in the first interface.|
+| Parameter Name | Default Value | Description |
+|--------------------------------|----------------------|--------------|
+| producerGroup | DEFAULT_PRODUCER | Producer group name |
+| sendMsgTimeout | 3000 | Timeout (ms) for
sending messages |
+| retryTimesWhenSendFailed | 2 | Max retries for
failed messages |
+| **enableBatchSend** (5.x) | true | Enables batch
message sending |
+| **enableBackPressure** (5.x) | true | Prevents overload in
high-traffic scenarios |
#### 2.3 PushConsumer Configuration
-| Parameter Name | Default Value
| Description |
-| ---------------------------- | ----------------------------- |
------------------------------------------------------------ |
-| consumerGroup | DEFAULT_CONSUMER | Consumer
group name. If multi Consumer belong to an application, subscribe the same
message and consume logic as the same, they should be gathered together |
-| messageModel | CLUSTERING | Message
support two mode: cluster consumption and broadcast consumption
|
-| consumeFromWhere | CONSUME_FROM_LAST_OFFSET | After
Consumer started, default consumption from last location, it include two
situation: One is last consumption location is not expired, and consumption
start at last location; The other is last location expired, start consumption
at current queue's first message |
-| consumeTimestamp | Half an hour ago | Only
consumeFromWhere=CONSUME_FROM_TIMESTAMP, this can work |
-| allocateMessageQueueStrategy | AllocateMessageQueueAveragely | Implements
strategy of Rebalance algorithms |
-| subscription | | subscription
relation |
-| messageListener | | message
listener |
-| offsetStore | | Consumption
progress store |
-| consumeThreadMin | 20 | Minimum of
thread in consumption thread pool
|
-| consumeThreadMax | 20 | Maximum of
thread in consumption thread pool
|
-| | |
|
-| consumeConcurrentlyMaxSpan | 2000 | Maximum span
allowed for single queue parallel consumption |
-| pullThresholdForQueue | 1000 | Pull message
local queue cache maximum number of messages |
-| pullInterval | 0 | Pull message
interval, because long polling it is 0, but for flow control, you can set value
which greater than 0 in milliseconds |
-| consumeMessageBatchMaxSize | 1 | Batch consume
message |
-| pullBatchSize | 32 | Batch pull
message |
+| Parameter Name | Default Value |
Description |
+|--------------------------------------|------------------------------------|-------------|
+| consumerGroup | DEFAULT_CONSUMER |
Consumer group name |
+| messageModel | CLUSTERING |
Message consumption mode (CLUSTERING or BROADCAST) |
+| consumeFromWhere | CONSUME_FROM_LAST_OFFSET |
Default consumption position |
+| consumeThreadMin | 20 |
Min consumption thread count |
+| consumeThreadMax | 20 |
Max consumption thread count |
+| **Rebalance Strategies (5.x)** |
AllocateMessageQueueAveragelyByCircle | New rebalance strategy for better
distribution |
#### 2.4 PullConsumer Configuration
-| Parameter Name | Default Value |
Description |
-| -------------------------------- | ----------------------------- |
------------------------------------------------------------ |
-| consumerGroup | DEFAULT_CONSUMER | Consumer
group name. If multi Consumer belong to an application, subscribe the same
message and consume logic as the same, they should be gathered together |
-| brokerSuspendMaxTimeMillis | 20000 | Long
polling, Consumer pull message request suspended for the longest time in the
Broker in milliseconds |
-| consumerTimeoutMillisWhenSuspend | 30000 | Long
polling, Consumer pull message request suspend in the Broker over this time
value, client think timeout. Unit is milliseconds |
-| consumerPullTimeoutMillis | 10000 | Not long
polling, timeout time of pull message in milliseconds
|
-| messageModel | CLUSTERING | Message
support two mode: cluster consumption and broadcast consumption |
-| messageQueueListener | | Listening
changing of queue |
-| offsetStore | |
Consumption schedule store |
-| registerTopics | |
Collection of registered topics |
-| allocateMessageQueueStrategy | AllocateMessageQueueAveragely |
Implements strategy about Rebalance algorithm
|
+| Parameter Name | Default Value |
Description |
+|------------------------------------|------------------------------|-------------|
+| consumerGroup | DEFAULT_CONSUMER | Consumer
group name |
+| brokerSuspendMaxTimeMillis | 20000 | Max
suspension time (ms) for long polling |
+| consumerPullTimeoutMillis | 10000 | Timeout
(ms) for pull requests |
+| **messageGroup** (5.x) | | Enables
orderly consumption based on groups |
#### 2.5 Message Data Structure
-| Field Name | Default Value | Description
|
-| -------------- | ------ |
------------------------------------------------------------ |
-| Topic | null | Required, the name of the topic to which the
message belongs |
-| Body | null | Required, message body
|
-| Tags | null | Optional, message tag, convenient for server
filtering. Currently only one tag per message is supported |
-| Keys | null | Optional, represent this message's business keys,
server create hash indexes based keys. After setting, you can find message by
```Topics```,```Keys``` in Console system. Because of hash indexes, please make
key as unique as possible, such as order number, goods Id and so on.|
-| Flag | 0 | Optional, it is entirely up to the application,
and RocketMQ does not intervene |
-| DelayTimeLevel | 0 | Optional, message delay level, 0 represent no
delay, greater tan 0 can consume |
-| WaitStoreMsgOK | TRUE | Optional, indicates whether the message is not
answered until the server is down. |
+| Field Name | Default Value | Description |
+|-------------------|---------------|-------------|
+| Topic | null | Required: Name of the message topic |
+| Body | null | Required: Message content |
+| Tags | null | Optional: Tag for filtering |
+| Keys | null | Optional: Business keys (e.g., order IDs)
|
+| Flag | 0 | Optional: Custom flag |
+| DelayTimeLevel | 0 | Optional: Message delay level |
+| WaitStoreMsgOK | TRUE | Optional: Acknowledgment before storing |
+| **maxReconsumeTimes** (5.x) | | Max retries before moving to dead-letter
queue |
+| **messageGroup** (5.x) | | Group-based message ordering |
+
+---
diff --git a/docs/en/Configuration_System.md b/docs/en/Configuration_System.md
index e263b0c4c2..95589d95fb 100644
--- a/docs/en/Configuration_System.md
+++ b/docs/en/Configuration_System.md
@@ -1,6 +1,6 @@
-# The system configuration
+# The System Configuration (RocketMQ 5.x)
-This section focuses on the configuration of the system (JVM/OS)
+This section focuses on the configuration of the system (JVM/OS) for
**RocketMQ 5.x**.
## **1 JVM Options** ##
@@ -16,56 +16,42 @@ If you don’t care about the boot time of RocketMQ broker,
pre-touch the Java h
-XX:+AlwaysPreTouch
-Disable biased locking maybe reduce JVM pauses:
+Disable biased locking to potentially reduce JVM pauses:
-XX:-UseBiasedLocking
-As for garbage collection, G1 collector with JDK 1.8 is recommended:
+As for garbage collection, the G1 collector with JDK 1.8 is recommended:
- -XX:+UseG1GC -XX:G1HeapRegionSize=16m
+ -XX:+UseG1GC -XX:G1HeapRegionSize=16m
-XX:G1ReservePercent=25
-XX:InitiatingHeapOccupancyPercent=30
-These GC options looks a little aggressive, but it’s proved to have good
performance in our production environment
+These GC options may seem a bit aggressive, but they have been proven to
provide good performance in our production environment.
-Don’t set a too small value for -XX:MaxGCPauseMillis, otherwise JVM will use a
small young generation to achieve this goal which will cause very frequent
minor GC.So use rolling GC log file is recommended:
+Do not set a too small value for -XX:MaxGCPauseMillis, as it may cause the JVM
to use a small young generation, leading to frequent minor GCs. Using a rolling
GC log file is recommended:
- -XX:+UseGCLogFileRotation
- -XX:NumberOfGCLogFiles=5
+ -XX:+UseGCLogFileRotation
+ -XX:NumberOfGCLogFiles=5
-XX:GCLogFileSize=30m
-If write GC file will increase latency of broker, consider redirect GC log
file to a memory file system:
+If writing GC logs to disk increases broker latency, consider redirecting the
GC log file to a memory file system:
-Xloggc:/dev/shm/mq_gc_%p.log123
-## 2 Linux Kernel Parameters ##
+## **2 Linux Kernel Parameters** ##
-There is a os.sh script that lists a lot of kernel parameters in folder bin
which can be used for production use with minor changes. Below parameters need
attention, and more details please refer to documentation for /proc/sys/vm/*.
+There is an `os.sh` script in the `bin` folder that lists several kernel
parameters that can be used for production with minor modifications. Below
parameters require attention. For more details, refer to the documentation for
`/proc/sys/vm/*`.
+- **vm.extra_free_kbytes**: Tells the VM to keep extra free memory between the
threshold where background reclaim (kswapd) kicks in and the threshold where
direct reclaim (by allocating processes) kicks in. RocketMQ uses this parameter
to avoid high latency in memory allocation. (Kernel version-specific)
+- **vm.min_free_kbytes**: If set lower than 1024KB, the system may become
subtly broken and prone to deadlocks under high loads.
+- **vm.max_map_count**: Limits the maximum number of memory map areas a
process may have. RocketMQ uses `mmap` to load `CommitLog` and `ConsumeQueue`,
so setting a higher value is recommended.
-- **vm.extra_free_kbytes**, tells the VM to keep extra free memory between the
threshold where background reclaim (kswapd) kicks in, and the threshold where
direct reclaim (by allocating processes) kicks in. RocketMQ uses this parameter
to avoid high latency in memory allocation. (It is specific to the kernel
version）
+- **vm.swappiness**: Defines how aggressively the kernel swaps memory pages.
Higher values increase aggressiveness, while lower values decrease the amount
of swap. A value of **10** is recommended to avoid swap latency.
+- **File descriptor limits**: RocketMQ requires open file descriptors for
files (`CommitLog` and `ConsumeQueue`) and network connections. It is
recommended to set this to **655350**.
+- **Disk scheduler**: The **deadline I/O scheduler** is recommended for
RocketMQ as it attempts to provide a guaranteed latency for requests.
-- **vm.min_free_kbytes**, if you set this to lower than 1024KB, your system
will become subtly broken, and prone to deadlock under high loads.
-
-
-
-
-
-- **vm.max_map_count**, limits the maximum number of memory map areas a
process may have. RocketMQ will use mmap to load CommitLog and ConsumeQueue, so
set a bigger value for this parameter is recommended.
-
-
-
-- **vm.swappiness**, define how aggressive the kernel will swap memory pages.
Higher values will increase aggressiveness, lower values decrease the amount of
swap. 10 is recommended for this value to avoid swap latency.
-
-
-
-- **File descriptor limits**, RocketMQ needs open file descriptors for
files(CommitLog and ConsumeQueue) and network connections. We recommend setting
655350 for file descriptors.
-
-
-
-- **Disk scheduler**, the deadline I/O scheduler is recommended for RocketMQ,
which attempts to provide a guaranteed latency for requests.
-
+---
