mysql - Percona Xtradb 集群失败

Question

我已经设置了具有 3 个节点的 Percona Xtradb 集群。第一个节点使用 bootstrap 启动正常，但是当我尝试启动第二个节点加入集群时，出现以下错误:

2015-08-27 18:08:08 25990 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (a6b3fced-4ca1-11e5-b5da-d69fa186273c): 1 (Operation not permitted)
at galera/src/replicator_str.cpp:prepare_for_IST():463. IST will be unavailable.
2015-08-27 18:08:08 25990 [Note] WSREP: Member 0.0 (db-gc-pxc2) requested state transfer from 'any'. Selected 1.0 (db-gc-pxc1)(SYNCED) as donor.
2015-08-27 18:08:08 25990 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 0)
2015-08-27 18:08:08 25990 [Note] WSREP: Requesting state transfer: success, donor: 1
2015-08-27 18:08:08 25990 [Warning] WSREP: 1.0 (db-gc-pxc1): State transfer to 0.0 (db-gc-pxc2) failed: -12 (Cannot allocate memory)
2015-08-27 18:08:08 25990 [ERROR] WSREP: gcs/src/gcs_group.cpp:int gcs_group_handle_join_msg(gcs_group_t*, const gcs_recv_msg_t*)():731: Will never receive state. Need to abort.
2015-08-27 18:08:08 25990 [Note] WSREP: gcomm: terminating thread
2015-08-27 18:08:08 25990 [Note] WSREP: gcomm: joining thread
2015-08-27 18:08:08 25990 [Note] WSREP: gcomm: closing backend

以下是我在 my.cnf 文件中的集群配置:

# Galera COnfig
wsrep_cluster_name = pxc
wsrep_cluster_address = gcomm://192.168.2.100,192.168.2.101,10.168.1.102
wsrep_node_address = 10.1.0.101
wsrep_provider = /usr/lib/libgalera_smm.so
wsrep_provider_options = "gcache.size=4G"
wsrep_slave_threads = 32
wsrep_sst_auth = "user:userpass"
wsrep_node_name = node2
#wsrep_sst_method = xtrabackup_throttle
wsrep_sst_method = xtrabackup-v2

什么会导致这个错误？

仅供引用，我确实在数据库中创建了 wsrep_sst_auth 的用户和密码。

这是 my.cnf 的剩余部分，它有帮助:

back_log = 65535
binlog_format = ROW
character_set_server = utf8
collation_server = utf8_general_ci
datadir = /var/lib/mysql
#default_storage_engine = InnoDB
expand_fast_index_creation = 1
expire_logs_days = 7
innodb_autoinc_lock_mode = 2
innodb_buffer_pool_instances = 6
innodb_buffer_pool_populate = 1
innodb_buffer_pool_size = 6G   # XXX 64GB RAM, 80%
#innodb_data_file_path = ibdata1:64M;ibdata2:64M:autoextend
innodb_file_format = Barracuda
innodb_file_per_table
innodb_flush_log_at_trx_commit = 2
innodb_flush_method = O_DIRECT
innodb_io_capacity = 1600
innodb_large_prefix
innodb_locks_unsafe_for_binlog = 1
#innodb_log_file_size = 64M
innodb_print_all_deadlocks = 1
innodb_read_io_threads = 64
innodb_stats_on_metadata = FALSE
innodb_support_xa = FALSE
innodb_write_io_threads = 64
log-bin = mysqld-bin
log-queries-not-using-indexes
log-slave-updates
long_query_time = 1
max_allowed_packet = 64M
max_connect_errors = 4294967295
max_connections = 4096
min_examined_row_limit = 1000
performance-schema-instrument='%=ON'
port = 3306
relay-log-recovery = TRUE
skip-name-resolve
slow_query_log = 1
slow_query_log_timestamp_always = 1
table_open_cache = 4096
thread_cache = 1024
tmpdir = /srv/tmp
transaction_isolation = REPEATABLE-READ
updatable_views_with_limit = 0
user = mysql
wait_timeout = 60

Answer 1

这似乎是根本原因:

2015-08-27 18:08:08 25990 [警告] WSREP:1.0 (db-gc-pxc1):状态转移到 0.0 (db-gc-pxc2) 失败:-12(无法分配内存)

新节点尝试加入集群。新节点当前没有状态(本地 UUID 为零)，因此 IST 不可用 - 这意味着它需要从捐赠节点运行完整的 SST。

节点pxc2是joiner，pxc1是被选中的donor；然而，我们从 pxc1 收到状态传输失败的错误，这导致加入失败。

您应该检查捐赠者节点 (pxc1) 上的日志以获取更多详细信息；但是我们的日志表明它没有足够的内存来运行数据库的导出。不知道你的硬件配置，我不能给出明确的答复，但很可能你的 my.cnf 配置为对可用内存来说太耗内存，因此 xtrabackup 进程无法运行，或者数据库太大。向节点添加更多内存，或者减少 my.cnf 中的分配。