redis集群搭建與管理
server1:172.16.16.34 server2:172.16.16.35 redis版本:redis3.2
搭建環境:redis集群,server1有7001,7002,7003三主,server2有7001,7002,7003三從,總共六個節點。這樣做是為了保證redis的集群的高可用。redis的復制也是采用異步復制的方式。
1,安裝rediscd /home/maxiangqian tar xzf redis-3.2.8.tar.gz cd redis-3.2.8 yum install gcc make
2:創建redis目錄文件夾
在server1和server2上都創建相應的文件夾:mkdir /home/redis7001/data mkdir-p /home/redis7001/data /home/redis7001/log /home/redis7001/tmp mkdir -p /home/redis7002/data /home/redis7002/log /home/redis7002/tmp mkdir -p /home/redis7003/data /home/redis7003/log /home/redis7003/tmp
3:為server1和server2的三個節點分別配置配置文件
以一個配置文件為例:port 7001 timeout 300 daemonize yes pidfile "/home/redis7001/tmp/redis_7001.pid" loglevel notice logfile"/home/redis7001/log/redis_7001.log" databases 16 save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename "dump.rdb" dir "/home/redis7001/data" slave-serve-stale-data yes #slave-read-only yes # yes開啟從庫只讀 repl-diskless-sync no repl-diskless-sync-delay 5 repl-disable-tcp-nodelay no slave-priority 100 appendonly yes #appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb lua-time-limit 5000 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 requirepass "maxiangqianredis" masterauth "maxiangqianredis" #cluster cluster-enabled yes cluster-config-file /home/redis7001/nodes7001.conf cluster-node-timeout 5000
上面是redis7001的配置文件內容
然後啟動7001的redis:redis-server /home/redis7001/redis7001.conf
我們看一下啟動日誌:
1574:M 03 May 16:22:53.444 * No cluster configuration found, I‘m 363ecec54c92c2548dcab016146bdb4c104e5e84
每個實例都會為自己生成一個唯一的ID,用來識別集群中的唯一身份。 然後使用同樣的方法啟動其余五個實例
server1 7001 server1 7002 server1 7003 server2 7001 server2 7002 server2 7003
OK,現在已經有六個已經啟動的redis實例了。我們下一步開始做集群
4:開始創建集群redis-trib.rb create --replicas 1 10.103.16.34:7001 10.103.16.34:7002 10.103.16.34:7003 10.103.16.35:7001 10.103.16.35:7002 10.103.16.35:7003
執行報錯:
/usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require‘: no such file to load -- redis (LoadError) from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require‘ from /home/maxiangqian/redis-3.2.8/src/redis-trib.rb:25
我們需要安裝以下幾個包:
yum -y install zlib ruby rubygems
gem install redis
然後重新啟動創建集群的操作:
[[email protected] redis7003]# redis-trib.rb create --replicas 1 10.103.16.34:7001 10.103.16.34:7002 10.103.16.34:7003 10.103.16.35:7001 10.103.16.35:7002 10.103.16.35:7003 >>> Creating cluster [ERR] Sorry, can‘t connect to node 10.103.16.34:7001
又報錯了我擦。
網上各種搜資料咨詢,最好還是發現不是什麽版本問題,是因為我的配置文件裏加了以下兩行配置:requirepass "maxiangqianredis" masterauth "maxiangqianredis"
群集是不支持認證的,把這兩行註釋掉就好了,然後我們在執行創建群集的命令:
[[email protected] redis7003]# redis-trib.rb create --replicas 1 10.103.16.34:7001 10.103.16.34:7002 10.103.16.34:7003 10.103.16.35:7001 10.103.16.35:7002 10.103.16.35:7003 >>> Creating cluster >>> Performing hash slots allocation on 6 nodes... Using 3 masters: 10.103.16.35:7001 10.103.16.34:7001 10.103.16.35:7002 Adding replica 10.103.16.34:7002 to 10.103.16.35:7001 Adding replica 10.103.16.35:7003 to 10.103.16.34:7001 Adding replica 10.103.16.34:7003 to 10.103.16.35:7002 M: 363ecec54c92c2548dcab016146bdb4c104e5e84 10.103.16.34:7001 slots:5461-10922 (5462 slots) master S: 93a0e8d405959480fcbd310a5d15a92346c69d43 10.103.16.34:7002 replicates d015a22abc57c021f568973f4f1c03c7a5c7b772 S: 78f77749f9f9a5f0d7c99427e0311912a3fa04e7 10.103.16.34:7003 replicates 89147e5837e378b69233dd2b8290267975719bc4 M: d015a22abc57c021f568973f4f1c03c7a5c7b772 10.103.16.35:7001 slots:0-5460 (5461 slots) master M: 89147e5837e378b69233dd2b8290267975719bc4 10.103.16.35:7002 slots:10923-16383 (5461 slots) master S: ce9d635236567ccde4c864f78863fa0a4b26f25a 10.103.16.35:7003 replicates 363ecec54c92c2548dcab016146bdb4c104e5e84 Can I set the above configuration? (type ‘yes‘ to accept):
OK,已經提示成功了,我們直接選擇yes就好了。
>>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join.. >>> Performing Cluster Check (using node 10.103.16.34:7001) M: 363ecec54c92c2548dcab016146bdb4c104e5e84 10.103.16.34:7001 slots:5461-10922 (5462 slots) master 1 additional replica(s) S: 78f77749f9f9a5f0d7c99427e0311912a3fa04e7 10.103.16.34:7003 slots: (0 slots) slave replicates 89147e5837e378b69233dd2b8290267975719bc4 M: d015a22abc57c021f568973f4f1c03c7a5c7b772 10.103.16.35:7001 slots:0-5460 (5461 slots) master 1 additional replica(s) M: 89147e5837e378b69233dd2b8290267975719bc4 10.103.16.35:7002 slots:10923-16383 (5461 slots) master 1 additional replica(s) S: ce9d635236567ccde4c864f78863fa0a4b26f25a 10.103.16.35:7003 slots: (0 slots) slave replicates 363ecec54c92c2548dcab016146bdb4c104e5e84 S: 93a0e8d405959480fcbd310a5d15a92346c69d43 10.103.16.34:7002 slots: (0 slots) slave replicates d015a22abc57c021f568973f4f1c03c7a5c7b772 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
這樣群集就設置成功了。
查看一下集群的狀態,測試一下: 在10.103.16.34:7001創建name測試字符串,然後驗證[[email protected] sa]# redis-cli -c -p 7001 127.0.0.1:7001> get name -> Redirected to slot [5798] located at 10.103.16.34:7001 "txt" 10.103.16.34:7001> exit [[email protected] sa]# redis-cli -c -p 7002 127.0.0.1:7002> get name -> Redirected to slot [5798] located at 10.103.16.34:7001 "txt" 10.103.16.34:7001> exit [[email protected] sa]# redis-cli -c -p 7003 127.0.0.1:7003> get name -> Redirected to slot [5798] located at 10.103.16.34:7001 "txt"
5:我們接下來查看一下集群的基本信息:
[[email protected] redis7003]# redis-cli -p 7001 cluster nodes 78f77749f9f9a5f0d7c99427e0311912a3fa04e7 10.103.16.34:7003 slave 89147e5837e378b69233dd2b8290267975719bc4 0 1493879665448 5 connected d015a22abc57c021f568973f4f1c03c7a5c7b772 10.103.16.35:7001 master - 0 1493879663946 4 connected 0-5460 89147e5837e378b69233dd2b8290267975719bc4 10.103.16.35:7002 master - 0 1493879664948 5 connected 10923-16383 ce9d635236567ccde4c864f78863fa0a4b26f25a 10.103.16.35:7003 slave 363ecec54c92c2548dcab016146bdb4c104e5e84 0 1493879665949 6 connected 93a0e8d405959480fcbd310a5d15a92346c69d43 10.103.16.34:7002 slave d015a22abc57c021f568973f4f1c03c7a5c7b772 0 1493879664446 4 connected 363ecec54c92c2548dcab016146bdb4c104e5e84 10.103.16.34:7001 myself,master - 0 0 1 connected 5461-10922
可以看到現在的集群有六個節點,三個主節點和三個從節點。而且每個主節點都會記錄自己分配的哈希槽,從中我們可以看到
103.16.35:7001 master - 0 1493879663946 4 connected 0-5460 10.103.16.34:7001 myself,master - 0 0 1 connected 5461-10922 10.103.16.35:7002 master - 0 1493879664948 5 connected 10923-16383
當然我們也可以對這些節點的哈希槽進行重新的分配,我們現在打算將103.16.35:7001的前100個哈希槽移動到10.103.16.34:7001
[[email protected] redis7003]# redis-trib.rb reshard 10.103.16.34:7001
然後會提示我輸入數值以及從哪裏遷移到哪裏:
How many slots do you want to move (from 1 to 16384)? 100 What is the receiving node ID? 363ecec54c92c2548dcab016146bdb4c104e5e84 Please enter all the source node IDs. Type ‘all‘ to use all the nodes as source nodes for the hash slots. Type ‘done‘ once you entered all the source nodes IDs. Source node #1:d015a22abc57c021f568973f4f1c03c7a5c7b772 Source node #2:done
執行完以後就可以進行遷移了,遷移完以後我們再打印出來節點信息看一下:
[[email protected] redis7003]# redis-cli -p 7001 cluster nodes 78f77749f9f9a5f0d7c99427e0311912a3fa04e7 10.103.16.34:7003 slave 89147e5837e378b69233dd2b8290267975719bc4 0 1493881167965 5 connected d015a22abc57c021f568973f4f1c03c7a5c7b772 10.103.16.35:7001 master - 0 1493881166460 4 connected 101-5460 89147e5837e378b69233dd2b8290267975719bc4 10.103.16.35:7002 master - 0 1493881166962 5 connected 10923-16383 ce9d635236567ccde4c864f78863fa0a4b26f25a 10.103.16.35:7003 slave 363ecec54c92c2548dcab016146bdb4c104e5e84 0 1493881167465 7 connected 93a0e8d405959480fcbd310a5d15a92346c69d43 10.103.16.34:7002 slave d015a22abc57c021f568973f4f1c03c7a5c7b772 0 1493881167965 4 connected 363ecec54c92c2548dcab016146bdb4c104e5e84 10.103.16.34:7001 myself,master - 0 0 7 connected 0-100 5461-10922
我們可以很清楚的看到已經遷移成功了。
6:我們來測試一下集群的故障轉移功能 首先我們要看一下集群的主節點信息:[[email protected] redis7003]# redis-cli -p 7001 cluster nodes | grep master d015a22abc57c021f568973f4f1c03c7a5c7b772 10.103.16.35:7001 master - 0 1493883826713 4 connected 101-5460 89147e5837e378b69233dd2b8290267975719bc4 10.103.16.35:7002 master - 0 1493883827213 5 connected 10923-16383 363ecec54c92c2548dcab016146bdb4c104e5e84 10.103.16.34:7001 myself,master - 0 0 7 connected 0-100 5461-10922
我們現在要使10.103.16.35:7001這個主節點斷掉,然後重啟看一下基本信息
[[email protected] sa]# /home/maxiangqian/redis-3.2.8/src/redis-cli -p 7001 127.0.0.1:7001> SHUTDOWN not connected> exit [[email protected] sa]# redis-server /home/redis7001/redis7001.conf
然後再打印一下集群信息看一下:
[[email protected] redis7003]# redis-cli -p 7001 cluster nodes 78f77749f9f9a5f0d7c99427e0311912a3fa04e7 10.103.16.34:7003 slave 89147e5837e378b69233dd2b8290267975719bc4 0 1493884247801 5 connected d015a22abc57c021f568973f4f1c03c7a5c7b772 10.103.16.35:7001 slave 93a0e8d405959480fcbd310a5d15a92346c69d43 0 1493884247300 8 connected 89147e5837e378b69233dd2b8290267975719bc4 10.103.16.35:7002 master - 0 1493884246798 5 connected 10923-16383 ce9d635236567ccde4c864f78863fa0a4b26f25a 10.103.16.35:7003 slave 363ecec54c92c2548dcab016146bdb4c104e5e84 0 1493884246298 7 connected 93a0e8d405959480fcbd310a5d15a92346c69d43 10.103.16.34:7002 master - 0 1493884248301 8 connected 101-5460 363ecec54c92c2548dcab016146bdb4c104e5e84 10.103.16.34:7001 myself,master - 0 0 7 connected 0-100 5461-10922
通過信息我們可以很明顯的看到了10.103.16.35:7001這個主節點已經變成了從節點,而本身他的從節點也上升為主節點了。
但是我們需要註意的是在故障轉移的時間段內,一些10.103.16.35:7001的寫操作是會丟失的。直到他的從庫提升為主庫為止,這是redis為了保證數據一致性而采取的措施。 7:集群的節點管理: 我們先看一下怎麽添加節點,添加節點分為兩類(主節點或者從節點) 主節點:./redis-trib.rb add-node 10.103.16.34:7004 10.103.16.34:7001
這樣我們就把10.103.16.34:7004添加為集群的新的主節點,不過我們要註意的是,這時候他僅僅是一個沒有哈希槽的主節點,並不會存儲任何數據。
我們可以使用 redis-trib 程序, 將集群中的某些哈希桶移動到新節點裏面, 新節點就會成為真正的主節點了。 從節點:./redis-trib.rb add-node 10.103.16.34:7004 10.103.16.34:7001 redis 10.103.16.34::7004> cluster replicate 3c3a0c74aae0b56170ccb03a76b60cfe7dc1912e
將新節點指定為ID為3c3a0c74aae0b56170ccb03a76b60cfe7dc1912e的從節點。
接下來我們看一下怎麽移除某個節點,語法格式如下:./redis-trib del-node 127.0.0.1:7000 `<node-id>`
但是我們要註意一點,移除主節點的時候必須保證主節點是空的,也就是事先將要移除的主節點的哈希槽給轉移到其他的主節點上。
redis集群搭建與管理