MySQL复制问题和Binlog压缩

发布时间 2023-03-28 09:48:58作者: abce

你可能想在MySQL/Percona MySQL上使用binlog压缩,但它会有缺点。

binlog日志压缩是一种用于减少二进制日志文件大小的技术,随着时间的推移,二进制日志文件可能会变得相当大。这在磁盘空间有限的情况下尤其重要。但是,重要的是要意识到使用二进制日志压缩也会导致复制产生问题。

考虑以下场景:已经使用Percona XtraBackup从副本节点恢复了备份。恢复完成后,需要使用xtrabackup_slave_info文件中的信息设置复制。

$ cat xtrabackup_slave_info #GTID replication:SET GLOBAL gtid_purged=00022031-1111-1111-1111-111111111111:1-31781; CHANGE MASTER TO MASTER_AUTO_POSITION=1; #General replication with Binlog file and position: CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000053', MASTER_LOG_POS=274;

你使用主节点的信息并启动复制:

#GTID replicationSET GLOBAL gtid_purged=00022031-1111-1111-1111-111111111111:1-31781;CHANGE MASTER TO MASTER_HOST="10.0.0.2", MASTER_USER="replica", MASTER_PASSWORD="somestrongpas$", MASTER_AUTO_POSITION=1;START SLAVE;#General replication with Binlog file and position:CHANGE MASTER TO MASTER_HOST="10.0.0.2", MASTER_USER="replica", MASTER_PASSWORD="somestrongpas$", MASTER_LOG_FILE='mysql-bin.000053', MASTER_LOG_POS=274;START SLAVE;

但是你遇到了以下的错误:

#GTID replication Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Cannot replicate because the master purged required binary logs. Replicate the missing transactions from elsewhere, or provision a new slave from backup. Consider increasing the master's binary log expiration period. The GTID set sent by the slave is '00022031-1111-1111-1111-111111111111:1-31781', and the missing transactions are '00022031-1111-1111-1111-111111111111:1-31781'' #General replication with Binlog file and position: Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Could not open log file'

第一步就是解压binlog日志文件和启动复制。如果是基于日志名和位置的复制,你的复制是可以正常工作的,不需要做任何修改。

然而,如果你想使用基于GTID的复制,要做的还没没有结束。需要修复一个或多个复制问题的:

Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires. Replicate the missing transactions from elsewhere, or provision a new slave from backup. Consider increasing the master's binary log expiration period. The GTID sets and the missing purged transactions are too long to print in this message. For more information, please see the master's error log or the manual for GTID_SUBTRACT.'

 

那如何修复呢?

1.最简单的方法是重启主库,更新gtid_purged变量。这么做要确保binlog_gtid_simple_recovery=ON。然后从库重启复制

2.从gtid模式切换成基于日志名和位置的复制

这可能需要你找到对应的日志名和位置。xtrabackup_slave_info中包含了gitd信息,可以根据gtid进行查找。

登录到主节点的binlog目录:

$ cd $(mysql -BNe"select @@log_bin_basename" | cut -d'/' -f1-$(mysql -BNe"select @@log_bin_basename"|grep / -o |wc -l))

找到具体的位置:

 $ for file in $(ls mysql-bin.0*) ; do echo $file ;mysqlbinlog --base64-output=decode-rows -vv $file | grep "00022031-1111-1111-1111-111111111111:31782" -A15 ; done | grep "00022031-1111-1111-1111-111111111111:31782" -B1 -A15 mysql-bin.000053SET @@SESSION.GTID_NEXT= '00022031-1111-1111-1111-111111111111:31782'/*!*/; # at 274#230224 18:20:48 server id 22031 end_log_pos 386 CRC32 0x0280d048 Querythread_id=14exec_time=0error_code=0Xid = 85 SET TIMESTAMP=1677244848/*!*/; SET @@session.pseudo_thread_id=14/*!*/; SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/*!*/; SET @@session.sql_mode=1168113696/*!*/; SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/*!*/; /*!C utf8mb4 *//*!*/; SET @@session.character_set_client=255,@@session.collation_connection=255,@@session.collation_server=255/*!*/; SET @@session.lc_time_names=0/*!*/; SET @@session.collation_database=DEFAULT/*!*/; /*!80011 SET @@session.default_collation_for_utf8mb4=255*//*!*/; /*!80016 SET @@session.default_table_encryption=0*//*!*/; create database test2 /*!*/;

找到日志名和位置。这里是mysql-bin.000053和274

STOP SLAVE; CHANGE MASTER TO MASTER_HOST="10.0.0.2", MASTER_USER="replica", MASTER_PASSWORD="somestrongpas$", MASTER_LOG_FILE='mysql-bin.000053', MASTER_LOG_POS=274, MASTER_AUTO_POSITION=0;START SLAVE;

在切换成gtid模式之前,要等待确认从库已经赶上主库。

STOP SLAVE; CHANGE MASTER TO MASTER_AUTO_POSITION = 1; START SLAVE;