一般情况下提供一个时间点的MySQL数据库全备,基于该时间点的全备,我们可以应用binlog,将数据库恢复到设定的时间点,前提是binlog保存完整,另外要求注意的是binlog的落盘与事务的提交是一致的(这里假设MySQL业务database schema使用是innodb存储引擎),即保证redo log写入磁盘时,binlog也写入到磁盘,这样可以避免事务提交后,binlog还滞留在binlog cache中,一旦此时程序崩溃,或者突然断电,binlog会丢失,以后通过binlog恢复数据时,数据就会出现丢失,一致性得不到保证。
关于binlog的保存不是本文叙述的重点,接着说数据库恢复的问题,上面提到准备一个全备,然后应用binlog将数据恢复到某个点,这是一种理想的情况,根本没有考虑到效率,因为应用binlog本身效率并不是特别高。例如提供一个周一的全备,然后应用binlog,把数据追到周五晚7点,在事务量大的情况下,需要花费的时间是很长的。因此需要找一个离恢复点最近的全备,然后应用binlog。考虑备份效率的问题,数据库每次备份不可能都是全备,会根据恢复窗口及数据量的大小制定相应的备份策略,一般会定期做一次全备,在备份周期内其他时间点采用增量备份。在全备的基础上,将增量备份数据恢复,完全可以提供一份离恢复点最近的全备(假定是周五凌晨2:30,事实上该时间段内执行的增量备份,在周四备份的基础上,PS:周三全备,周四至下周二增备),然后再进一步恢复数据。下面将演示这个过程,本文使用的备份工具为percona-xtrabacup-2.2.8,恢复起始时间为周三凌晨2:30,即2015-11-18 2:30:00,截止时间为周五晚7点,即2015-11-20 19:00:00刚好这个版本与先前版本有点不同,这里可以顺带着提下。
-
[root@mysql-temp 3307]# ll //查看备份情况
-
total 0
-
drwxr-xr-x 3 root root 47 Nov 22 11:04 full-20151118
-
drwxr-xr-x 3 root root 47 Nov 22 11:25 incr-20151119
-
drwxr-xr-x 3 root root 47 Nov 22 11:37 incr-20151120
-
[root@mysql-temp 3307]# ll full-20151118/2015-11-18_02-58-52/
total 1022292
drwx------ 2 root root 4096 Nov 22 11:19 activity
-rw-r----- 1 root root 268435456 Nov 22 11:19 ibdata1
-rw-r----- 1 root root 268435456 Nov 22 11:19 ibdata2
-rw-r----- 1 root root 268435456 Nov 22 11:19 ibdata3
drwx------ 2 root root 4096 Nov 22 11:19 mysql
drwxr-xr-x 2 root root 4096 Nov 22 11:17 performance_schema
-rw-r--r-- 1 root root 26 Nov 22 11:19 xtrabackup_binlog_info
-rw-r----- 1 root root 101 Nov 22 11:24 xtrabackup_checkpoints
-rw-r--r-- 1 root root 736 Nov 22 11:24 xtrabackup_info
-rw-r----- 1 root root 237633536 Nov 22 11:22 xtrabackup_logfile
-rw-r--r-- 1 root root 77 Nov 22 11:24 xtrabackup_slave_info
-
-
[root@mysql-temp 3307]# pwd
/ROOT/lvtu/backup/3307
[root@mysql-temp 3307]# cd ../../
-
第一步:将全备恢复,个人认为恢复的实质就是一次crash recovery,利用的备份过程中监听到的更新所对应的redo log的copy副本
-
[root@mysql-temp lvtu]# innobackupex --defaults-file=/ROOT/lvtu/2021.cnf --apply-log --redo-only --use-memory=4G /ROOT/lvtu/backup/3307/full-20151118/2015-11-18_02-58-52 //use-memory指定了用于恢复的内存使用量,适当加大,有助于提高apply-log的效率,--redo-log参数指定apply-log后不用回滚事务,因为事务在你备份结束的那一时刻没有提交,并不代表在接下来的时间内没有提交,如果回滚了,接下来的利用增备进行恢复,想必有相应的更新被丢失
InnoDB Backup Utility v1.5.1-xtrabackup; Copyright 2003, 2009 Innobase Oy
and Percona LLC and/or its affiliates 2009-2013. All Rights Reserved.
This software is published under
the GNU GENERAL PUBLIC LICENSE Version 2, June 1991.
Get the latest version of Percona XtraBackup, documentation, and help resources:
151122 12:24:47 innobackupex: Starting the apply-log operation
IMPORTANT: Please check that the apply-log run completes successfully.
At the end of a successful apply-log run innobackupex
prints "completed OK!".
151122 12:24:47 innobackupex: Starting ibbackup with command: xtrabackup --defaults-file="/ROOT/lvtu/backup/3307/full-20151118/2015-11-18_02-58-52/backup-my.cnf" --defaults-group="mysqld" --prepare --target-dir=/ROOT/lvtu/backup/3307/full-20151118/2015-11-18_02-58-52 --apply-log-only --use-memory=4G //这里需要指出的是在percona xtrabackup 2.2.8版本中,即使innobackupex 指定了--defaults-file项,貌似也没有生效,倒觉得这也是避免犯错的一种方式,程序大多数情况下比人靠谱,哈哈
xtrabackup version 2.2.8 based on MySQL server 5.6.22 Linux (x86_64) (revision id: )
xtrabackup: cd to /ROOT/lvtu/backup/3307/full-20151118/2015-11-18_02-58-52
xtrabackup: This target seems to be not prepared yet.
xtrabackup: xtrabackup_logfile detected: size=267337728, start_lsn=(1294382862409)
xtrabackup: using the following InnoDB configuration for recovery:
xtrabackup: innodb_data_home_dir = ./
xtrabackup: innodb_data_file_path = ibdata1:256M;ibdata2:256M;ibdata3:256M:autoextend:max:100G
xtrabackup: innodb_log_group_home_dir = ./
xtrabackup: innodb_log_files_in_group = 1
xtrabackup: innodb_log_file_size = 267337728
xtrabackup: using the following InnoDB configuration for recovery:
xtrabackup: innodb_data_home_dir = ./
xtrabackup: innodb_data_file_path = ibdata1:256M;ibdata2:256M;ibdata3:256M:autoextend:max:100G
xtrabackup: innodb_log_group_home_dir = ./
xtrabackup: innodb_log_files_in_group = 1
xtrabackup: innodb_log_file_size = 267337728
xtrabackup: Starting InnoDB instance for recovery.
xtrabackup: Using 4294967296 bytes for buffer pool (set by --use-memory parameter)
InnoDB: Using atomics to ref count buffer pool pages
InnoDB: The InnoDB memory heap is disabled
InnoDB: Mutexes and rw_locks use GCC atomic builtins
InnoDB: Memory barrier is not used
InnoDB: Compressed tables use zlib 1.2.3
InnoDB: Not using CPU crc32 instructions
InnoDB: Initializing buffer pool, size = 4.0G
InnoDB: Completed initialization of buffer pool
InnoDB: Highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 1294382862409
InnoDB: Database was not shutdown normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages
InnoDB: from the doublewrite buffer...
InnoDB: Doing recovery: scanned up to log sequence number 1294388105216 (2%)
InnoDB: Doing recovery: scanned up to log sequence number 1294393348096 (4%)
InnoDB: Doing recovery: scanned up to log sequence number 1294398590976 (6%)
InnoDB: Doing recovery: scanned up to log sequence number 1294403833856 (8%)
InnoDB: Doing recovery: scanned up to log sequence number 1294409076736 (11%)
InnoDB: Doing recovery: scanned up to log sequence number 1294414319616 (13%)
InnoDB: Doing recovery: scanned up to log sequence number 1294419562496 (15%)
InnoDB: Doing recovery: scanned up to log sequence number 1294424805376 (17%)
InnoDB: Doing recovery: scanned up to log sequence number 1294430048256 (19%)
InnoDB: Doing recovery: scanned up to log sequence number 1294435291136 (22%)
InnoDB: Doing recovery: scanned up to log sequence number 1294440534016 (24%)
InnoDB: Doing recovery: scanned up to log sequence number 1294445776896 (26%)
InnoDB: Doing recovery: scanned up to log sequence number 1294451019776 (28%)
InnoDB: Doing recovery: scanned up to log sequence number 1294456262656 (30%)
InnoDB: Doing recovery: scanned up to log sequence number 1294461505536 (33%)
InnoDB: Doing recovery: scanned up to log sequence number 1294466748416 (35%)
InnoDB: Doing recovery: scanned up to log sequence number 1294471991296 (37%)
InnoDB: Doing recovery: scanned up to log sequence number 1294477234176 (39%)
InnoDB: Doing recovery: scanned up to log sequence number 1294482477056 (41%)
InnoDB: Doing recovery: scanned up to log sequence number 1294487719936 (44%)
InnoDB: Doing recovery: scanned up to log sequence number 1294492962816 (46%)
InnoDB: Doing recovery: scanned up to log sequence number 1294498205696 (48%)
InnoDB: Doing recovery: scanned up to log sequence number 1294503448576 (50%)
InnoDB: Doing recovery: scanned up to log sequence number 1294508691456 (52%)
InnoDB: Doing recovery: scanned up to log sequence number 1294513934336 (55%)
InnoDB: Doing recovery: scanned up to log sequence number 1294519177216 (57%)
InnoDB: Doing recovery: scanned up to log sequence number 1294524420096 (59%)
InnoDB: Doing recovery: scanned up to log sequence number 1294529662976 (61%)
InnoDB: Doing recovery: scanned up to log sequence number 1294534905856 (63%)
InnoDB: Doing recovery: scanned up to log sequence number 1294540148736 (66%)
InnoDB: Doing recovery: scanned up to log sequence number 1294545391616 (68%)
InnoDB: Doing recovery: scanned up to log sequence number 1294550634496 (70%)
InnoDB: Doing recovery: scanned up to log sequence number 1294555877376 (72%)
InnoDB: Doing recovery: scanned up to log sequence number 1294561120256 (75%)
InnoDB: Doing recovery: scanned up to log sequence number 1294566363136 (77%)
InnoDB: Doing recovery: scanned up to log sequence number 1294571606016 (79%)
InnoDB: Doing recovery: scanned up to log sequence number 1294576848896 (81%)
InnoDB: Doing recovery: scanned up to log sequence number 1294582091776 (83%)
InnoDB: Doing recovery: scanned up to log sequence number 1294587334656 (86%)
InnoDB: Doing recovery: scanned up to log sequence number 1294592577536 (88%)
InnoDB: Doing recovery: scanned up to log sequence number 1294597820416 (90%)
InnoDB: Doing recovery: scanned up to log sequence number 1294603063296 (92%)
InnoDB: Doing recovery: scanned up to log sequence number 1294608306176 (94%)
InnoDB: Doing recovery: scanned up to log sequence number 1294613549056 (97%)
InnoDB: Doing recovery: scanned up to log sequence number 1294618791936 (99%)
InnoDB: Doing recovery: scanned up to log sequence number 1294620493531 (99%)
InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
InnoDB: Last MySQL binlog file position 0 119617202, file name 3307-bin.000038
[notice (again)]
If you use binary log and don't use any hack of group commit,
the binary log position seems to be:
InnoDB: Last MySQL binlog file position 0 119617202, file name 3307-bin.000038
xtrabackup: starting shutdown with innodb_fast_shutdown = 1
InnoDB: Starting shutdown...
InnoDB: Shutdown completed; log sequence number 1294620493531
151122 12:27:04 innobackupex: completed OK!
-
第二步:将2015-11-19的增量与2015-11-18的全备进合并,这一过程会先检查全备后的有哪些数据页有更新,并进行数据文件的整合,在应用完备份过程中所有更新的log后,将相关最新的数据文件从增备文件夹中copy到全备文件夹中
-
[root@mysql-temp lvtu]# innobackupex --defaults-file=/ROOT/lvtu/2021.cnf --apply-log --redo-only --use-memory=4G /ROOT/lvtu/backup/3307/full-20151118/2015-11-18_02-58-52 --incremental-dir=/ROOT/lvtu/backup/3307/incr-20151119/2015-11-19_02-45-11 恢复详情,此处省略..........
-
第三步:跟第二步的过程是一样的,不同的是增备不同
-
[root@mysql-temp lvtu]# innobackupex --defaults-file=/ROOT/lvtu/2021.cnf --apply-log --redo-only --use-memory=4G /ROOT/lvtu/backup/3307/full-20151118/2015-11-18_02-58-52 --incremental-dir=/ROOT/lvtu/backup/3307/incr-20151120/2015-11-20_02-44-20
-
第四步:将合并后的全备move-back到指定的datadir中,此时--defaults-file项就会生效
-
[root@mysql-temp lvtu]# innobackupex --defaults-file=/ROOT/lvtu/2021.cnf --move-back /ROOT/lvtu/backup/3307/full-20151118/2015-11-18_02-58-52
-
第五步:更改相应目录的属主
-
[root@mysql-temp lvtu]# chown mysql.mysql data/ -R
-
第六步:启动数据库
-
[root@mysql-temp lvtu]# /ROOT/server/mysql-5.6.26/bin/mysqld_safe --defaults-file=/ROOT/lvtu/2021.cnf --user=mysql &
[1] 15461
[root@mysql-temp lvtu]# 151122 13:21:03 mysqld_safe Logging to '/ROOT/lvtu/log/error.log'.
151122 13:21:03 mysqld_safe Starting mysqld daemon with databases from /ROOT/lvtu/data/
[root@mysql-temp lvtu]# ps aux|grep mysql
root 15461 0.5 0.0 106200 1516 pts/1 S 13:21 0:00 /bin/sh /ROOT/server/mysql-5.6.26/bin/mysqld_safe --defaults-file=/ROOT/lvtu/2021.cnf --user=mysql
mysql 16368 23.6 12.2 6385948 983492 pts/1 Dl 13:21 0:03 /ROOT/server/mysql-5.6.26/bin/mysqld --defaults-file=/ROOT/lvtu/2021.cnf --basedir=/ROOT/server/mysql-5.6.26 --datadir=/ROOT/lvtu/data/ --plugin-dir=/ROOT/server/mysql-5.6.26/lib/plugin --user=mysql --log-error=/ROOT/lvtu/log/error.log --open-files-limit=65535 --pid-file=/ROOT/lvtu/data//mysql-temp.pid --socket=/ROOT/tmp/mysql_2021.sock --port=2021
root 16384 0.0 0.0 103248 856 pts/1 S+ 13:21 0:00 grep mysql
[root@mysql-temp lvtu]# tail -f /ROOT/lvtu/log/error.log
2015-11-22 13:21:04 16368 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2015-11-22 13:21:04 16368 [Note] InnoDB: Memory barrier is not used
2015-11-22 13:21:04 16368 [Note] InnoDB: Compressed tables use zlib 1.2.3
2015-11-22 13:21:04 16368 [Note] InnoDB: Not using CPU crc32 instructions
2015-11-22 13:21:04 16368 [Note] InnoDB: Initializing buffer pool, size = 5.0G
2015-11-22 13:21:05 16368 [Note] InnoDB: Completed initialization of buffer pool
2015-11-22 13:21:05 16368 [Note] InnoDB: Setting log file /ROOT/lvtu/data/ib_logfile101 size to 1024 MB
InnoDB: Progress in MB: 100 200 300 400 500 600 700 800 900 1000
2015-11-22 13:21:16 16368 [Note] InnoDB: Setting log file /ROOT/lvtu/data/ib_logfile1 size to 1024 MB
InnoDB: Progress in MB: 100 200 300 400 500 600 700 800 900 1000
2015-11-22 13:21:28 16368 [Note] InnoDB: Setting log file /ROOT/lvtu/data/ib_logfile2 size to 1024 MB
InnoDB: Progress in MB: 100 200 300 400 500 600 700 800 900 1000
2015-11-22 13:21:43 16368 [Note] InnoDB: Renaming log file /ROOT/lvtu/data/ib_logfile101 to /ROOT/lvtu/data/ib_logfile0
2015-11-22 13:21:43 16368 [Warning] InnoDB: New log files created, LSN=1300724683141
2015-11-22 13:21:43 16368 [Note] InnoDB: Highest supported file format is Barracuda.
2015-11-22 13:21:43 16368 [Note] InnoDB: 128 rollback segment(s) are active.
2015-11-22 13:21:43 16368 [Note] InnoDB: Waiting for purge to start
2015-11-22 13:21:43 16368 [Note] InnoDB: 5.6.26 started; log sequence number 1300724683276
2015-11-22 13:21:43 7fc194ba4700 InnoDB: Loading buffer pool(s) from /ROOT/lvtu/data/ib_buffer_pool
2015-11-22 13:21:44 16368 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: eb5abaea-90d8-11e5-be0b-066328000010.
2015-11-22 13:21:44 16368 [Note] Server hostname (bind-address): '*'; port: 2021
2015-11-22 13:21:44 16368 [Note] IPv6 is not available.
2015-11-22 13:21:44 16368 [Note] - '0.0.0.0' resolves to '0.0.0.0';
2015-11-22 13:21:44 16368 [Note] Server socket created on IP: '0.0.0.0'.
2015-11-22 13:21:44 16368 [ERROR] Failed to open the relay log '/ROOT/log/mysql_binlog/3307/3307-relay.384101' (relay_log_pos 292177912).
2015-11-22 13:21:44 16368 [ERROR] Could not find target log file mentioned in relay log info in the index file '/ROOT/lvtu/binlog/2021-relay.index' during relay log initialization.
2015-11-22 13:21:44 16368 [ERROR] Failed to initialize the master info structure
2015-11-22 13:21:44 16368 [Note] Check error log for additional messages. You will not be able to start replication until the issue is resolved and the server restarted.
2015-11-22 13:21:44 16368 [Note] Event Scheduler: Loaded 0 events
2015-11-22 13:21:44 16368 [Note] /ROOT/server/mysql-5.6.26/bin/mysqld: ready for connections.
Version: '5.6.26community-log' socket: '/ROOT/tmp/mysql_2021.sock' port: 2021 Source distribution
-
第七步:找到2015-11-20增备时的binlog的stop-position,我在备库上做的备份,因些只需要找到备库复制应用主库binlog stop-position就成,然后将主库stop-position之后的binlog拷贝过来,利用mysqlbinlog工具将数据库恢复到指定的截止时间点
-
[root@mysql-temp lvtu]# cat /ROOT/lvtu/backup/3307/incr-20151120/2015-11-20_02-44-20/xtrabackup_slave_info
CHANGE MASTER TO MASTER_LOG_FILE='3307-bin.000168', MASTER_LOG_POS=292177750
-
//通过计算确定2015-11-20 2:30:00增备起,至2015-11-20 19:00:00恢复截止时间这一段时间的binlog都被保存在3307-bin.000168文件内,因此只需要copy3307-bin.000168即可
-
//应用binlog日志
-
[root@mysql-temp lvtu]# mysqlbinlog --start-position=292177750 --stop-datetime='2015-11-20 19:00:00' 3307-bin.000168|mysql -S /ROOT/tmp/mysql_2021.sock -u root -p
-
只要不存在冲突情况,--start-position可以和--stop-datetime一起使用,这里进行验证一下:
-
[root@mysql-temp lvtu]# mysqlbinlog --start-position=292177750 3307-bin.000168 |head -n 20
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#151119 17:33:52 server id 673307 end_log_pos 120 CRC32 0x4146bd4b Start: binlog v 4, server v 5.6.12-log created 151119 17:33:52
BINLOG '
gJdNVg8bRgoAdAAAAHgAAAAAAAQANS42LjEyLWxvZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAEzgNAAgAEgAEBAQEEgAAXAAEGggAAAAICAgCAAAACgoKGRkAAUu9
RkE=
'/*!*/;
# at 292177750
#151120 2:53:25 server id 673307 end_log_pos 292177837 CRC32 0x577f135a Query thread_id=3483776555 exec_time=0 error_code=0
SET TIMESTAMP=1447959205/*!*/;
SET @@session.pseudo_thread_id=3483776555/*!*/;
SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/*!*/;
SET @@session.sql_mode=1073741824/*!*/;
SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/*!*/;
/*!\C utf8 *//*!*/;
SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=33/*!*/;
-
[root@mysql-temp lvtu]# mysqlbinlog --stop-datetime='2015-11-20 19:00:00' 3307-bin.000168|tail -n 20
SET TIMESTAMP=1448017199/*!*/;
SET @@session.sql_mode=0/*!*/;
/*!\C binary *//*!*/;
SET @@session.character_set_client=63,@@session.collation_connection=33,@@session.collation_server=33/*!*/;
BEGIN
/*!*/;
# at 779271199
#151120 18:59:59 server id 673307 end_log_pos 779271401 CRC32 0x27387cf5 Query thread_id=3493725421 exec_time=0 error_code=0
use `highfan`/*!*/;
SET TIMESTAMP=1448017199/*!*/;
UPDATE pre_common_member_status SET `lastip`='60.28.138.148' , `port`='' , `lastvisit`='1448017199' WHERE `uid`='2379496'
/*!*/;
# at 779271401
#151120 18:59:59 server id 673307 end_log_pos 779271432 CRC32 0x1b9c82c9 Xid = 81251484513
COMMIT/*!*/;
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
-
-
至此,利用percona xtrabackup和mysqlbinlog进行基于时间点的恢复已经完成,最后补充下增备与完备的文件会有所差异
[root@SD-1-48 data]# ll full-20151118/2015-11-18_02-58-52/
-
......................................................... -rw-r----- 1 root root 268435456 Nov 18 02:59 ibdata1 -rw-r----- 1 root root 268435456 Nov 18 02:59 ibdata2 -rw-r----- 1 root root 268435456 Nov 18 02:59 ibdata3
-
.........................................................
-
[root@SD-1-48 data]# ll incr-20151119/2015-11-19_02-45-11/ .................................................................
-
-rw-r----- 1 root root 8994816 Nov 19 02:45 ibdata1.delta
-rw-r----- 1 root root 44 Nov 19 02:45 ibdata1.meta
-rw-r----- 1 root root 950272 Nov 19 02:45 ibdata2.delta
-rw-r----- 1 root root 44 Nov 19 02:45 ibdata2.meta
-rw-r----- 1 root root 950272 Nov 19 02:45 ibdata3.delta
-rw-r----- 1 root root 44 Nov 19 02:45 ibdata3.meta ........................................................................
-
阅读(3279) | 评论(0) | 转发(0) |