环境准备
java版本:1.8
操作系统:CentOS7
安装包:https://rocketmq.apache.org/dowloading/releases/
我这里选择的是4.7.1版本的rocketMQ
RocketMQ 集群搭建
注:此集群主从之间不会自动选举,也就是说主节点挂了,从节点不会顶上去
目标:2主2从异步集群
需要2台机器,具体配置下表
| 机器 | nameserver节点 | broker主 | broker从 |
|---|---|---|---|
| 机器1 | nameserver | broker-a | broker-b-s |
| 机器2 | nameserver | broker-b | broker-a-s |
安装包解压(这里的安装包是zip格式的,所以使用unzip)
unzip rocketmq-all-4.7.1-bin-release.zip -d /
增加软链接(纯粹为了方便,不影响安装)
ln -s rocketmq-all-4.7.1-bin-release/ rocketmq
配置文件
这里先解释下配置文件各参数意义
#所属集群名字,名字一样的节点就在同一个集群内
brokerClusterName=rocketmq-cluster
#broker名字,名字一样的节点就是一组主从节点。
brokerName=broker-a
#brokerid,0就表示是Master,>0的都是表示 Slave
brokerId=0
#nameServer地址,分号分割
namesrvAddr=192.168.186.131:9876;192.168.186.132:9876
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口
listenPort=10911
#删除文件时间点,默认凌晨 4点
deleteWhen=04
#文件保留时间,默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/data/rocketmq/store-a-master
#commitLog 存储路径
storePathCommitLog=/data/rocketmq/store-a-master/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/data/rocketmq/store-a-master/consumequeue
#消息索引存储路径
storePathIndex=/data/rocketmq/store-a-master/index
#checkpoint 文件存储路径
storeCheckpoint=/data/rocketmq/store-a-master/checkpoint
#abort 文件存储路径
abortFile=/data/rocketmq/store-a-master/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
这里我们直接更改它原来的配置文件,由于我们要搭建的是2主2从异步刷盘集群。所以我们进入2m-2s-async目录,并对当前目录的配置参数进行修改。其实直接启动应该也是没有问题的
[root@localhost 2m-2s-async]# pwd
/rocketmq/conf/2m-2s-async
下面是配置参数:这里主要修改了以下几个参数brokerName(主从相同),brokerId(主0从1),listenPort同一主机不同,文件路径区分不同
broker-a.properties
brokerClusterName=rocketmq-cluster
brokerName=broker-a
brokerId=0
namesrvAddr=192.168.186.131:9876;192.168.186.132:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store-a-master
storePathCommitLog=/data/rocketmq/store-a-master/commitlog
storePathConsumeQueue=/data/rocketmq/store-a-master/consumequeue
storePathIndex=/data/rocketmq/store-a-master/index
storeCheckpoint=/data/rocketmq/store-a-master/checkpoint
abortFile=/data/rocketmq/store-a-master/abort
maxMessageSize=65536
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
broker-b-s.properties
brokerClusterName=rocketmq-cluster
brokerName=broker-b
brokerId=1
namesrvAddr=192.168.186.131:9876;192.168.186.132:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10912
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store-b-slave
storePathCommitLog=/data/rocketmq/store-b-slave/commitlog
storePathConsumeQueue=/data/rocketmq/store-b-slave/consumequeue
storePathIndex=/data/rocketmq/store-b-slave/index
storeCheckpoint=/data/rocketmq/store-b-slave/checkpoint
abortFile=/data/rocketmq/store-b-slave/abort
maxMessageSize=65536
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
broker-a-s.properties
brokerClusterName=rocketmq-cluster
brokerName=broker-a
brokerId=1
namesrvAddr=192.168.186.131:9876;192.168.186.132:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store-a-slave
storePathCommitLog=/data/rocketmq/store-a-slave/commitlog
storePathConsumeQueue=/data/rocketmq/store-a-slave/consumequeue
storePathIndex=/data/rocketmq/store-a-slave/index
storeCheckpoint=/data/rocketmq/store-a-slave/checkpoint
abortFile=/data/rocketmq/store-a-slave/abort
maxMessageSize=65536
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
broker-b.properties
brokerClusterName=rocketmq-cluster
brokerName=broker-b
brokerId=0
namesrvAddr=192.168.186.131:9876;192.168.186.132:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10912
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store-b-master
storePathCommitLog=/data/rocketmq/store-b-master/commitlog
storePathConsumeQueue=/data/rocketmq/store-b-master/consumequeue
storePathIndex=/data/rocketmq/store-b-master/index
storeCheckpoint=/data/rocketmq/store-b-master/checkpoint
abortFile=/data/rocketmq/store-b-master/abort
maxMessageSize=65536
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
启动
注意:启动前需要修改runbroker.sh和runserver.sh中的JVM参数,默认rocketmq给的nameserver4g,broker8g,不修改可能机器内存不够启动报错。机器内存足够的话尽量不改,默认配置是经过实验室测试较优配置。
后台启动nameserver(2台机器全部启动)
nohup bin/mqnamesrv &
启动成功标识
### 启动成功标识
cat nohup.out
Java HotSpot(TM) 64-Bit Server VM warning: Using the DefNew young collector with the CMS collector is deprecated and will likely be removed in a future release
Java HotSpot(TM) 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
The Name Server boot success. serializeType=JSON
启动每台机器的broker
nohup bin/mqbroker -c conf/2m-2s-async/xxx.properties &
默认测试工具验证
首先需要配置nameserver
vim tools.sh
# 在export JAVA_HOME上面添加如下这段代码
export NAMESRV_ADDR='192.168.186.131:9876;192.168.186.132:9876'
生产者发送消息
bin/tools.sh org.apache.rocketmq.example.quickstart.Producer
消费消息
bin/tools.sh org.apache.rocketmq.example.quickstart.Consumer
管理控制台搭建
RocketMQ和RabbitMQ不同,没有官方的管理控制台,但是Rocket的社区扩展项目中提供了一个控制台。
地址:https://github.com/apache/rocketmq-externals
前期maven准备
地址:https://maven.apache.org/download.cgi
tar -zxvf apache-maven-3.6.3-bin.tar.gz -C /
mv /apache-maven-3.6.3/ /usr/local/maven
vim /etc/profile
### 增加代码
export MAVEN_HOME=/usr/local/maven
在PATH后追加: :$MAVEN_HOME/bin
### 刷新配置
source /etc/profile
### 验证
mvn -v
控制台安装
解压
unzip rocketmq-externals-master.zip -d /
软链接
ln -s /rocketmq-externals-master/ rocketmq-console
搭建
### 指定nameserver地址;;修改当前application.properties配置参数
rocketmq.config.namesrvAddr=192.168.186.131:9876;192.168.186.132:9876
cd /rocketmq-console/rocketmq-console
mvn clean package -Dmaven.test.skip=true
### rocketmq-console-ng-1.0.1.jar----target目录下
java -jar rocketmq-console-ng-1.0.1.jar
在当前ip:8080上即可访问