Chinaunix首页 | 论坛 | 博客

分类: Mysql/postgreSQL

2018-01-02 17:08:41

linux版本:CentOS release 6.5 (Final)
mysql版本: mysql> select @@version;
| @@version                |
| 5.5.36-MariaDB-wsrep-log |

问题场景 :一次机房意外停电,导致多台服务器重新启动,其中有一台数据库服务器已经存在很长时间了,无法正常启动,初期错误日志信息如下:
'171222  2:30:28 InnoDB: The InnoDB memory heap is disabled
171222  2:30:28 InnoDB: Mutexes and rw_locks use GCC atomic builtins
171222  2:30:28 InnoDB: Compressed tables use zlib 1.2.3
171222  2:30:28 InnoDB: Using Linux native AIO
171222  2:30:28 InnoDB: Initializing buffer pool, size = 6.3G
171222  2:30:29 InnoDB: Completed initialization of buffer pool
171222  2:30:29 InnoDB: highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 164567636854
171222  2:30:29  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 164568423895
InnoDB: Error: trying to access page number 4294966143 in space 0,
InnoDB: space name ./ibdata1,
InnoDB: which is outside the tablespace bounds.
InnoDB: Byte offset 0, len 16384, i/o type 10.
InnoDB: If you get this error at mysqld startup, please check that
InnoDB: your my.cnf matches the ibdata files that you have in the
InnoDB: MySQL server.
171222  2:30:30  InnoDB: Assertion failure in thread 139799947073504 in file fil0fil.c line 5462
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: about forcing recovery.
171222  2:30:30 [ERROR] mysqld got signal 6 ;

180102 14:34:38 InnoDB: Completed initialization of buffer pool
InnoDB: Error: log file ./ib_logfile0 is of different size 0 1679818752 bytes
InnoDB: than specified in the .cnf file 0 268435456 bytes!
180102 14:34:38 [ERROR] Plugin 'InnoDB' init function returned error.
180102 14:34:38 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
180102 14:34:38 [Note] Plugin 'FEEDBACK' is disabled.
180102 14:34:38 [ERROR] Unknown/unsupported storage engine: InnoDB
180102 14:34:38 [ERROR] Aborting
180102 14:34:38 [Note] /usr/libexec/mysqld: Shutdown complete'

2.把ib_logfile日志重命名保留,重新启动数据库,让期重新使用新的redo日志,重新启动时报如下错误,此次错误主要是报mysql sinal 11错误

mv ib_logfile0 bak_ib_logfile0

mv ib_logfile1 bak_ib_logfile1

细心观察时,发现此次数据库重启动时,在创建完ib_logfile日志后,就开始进行了数据恢复,也就是creash recovery.
180102 14:39:30  InnoDB: Page checksum 1808068616 (32bit_calc: 4064195506), prior-to-4.0.14-form checksum 3308157180
InnoDB: stored checksum 1808068616, prior-to-4.0.14-form stored checksum 3308157180
InnoDB: Page lsn 38 1354583575, low 4 bytes of lsn at page end 1354583575
InnoDB: Page number (if stored to page already) 535,
InnoDB: space id (if created with >= MySQL-4.1.1 and stored already) 69
InnoDB: Page may be a BLOB page
180102 14:39:30 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
To report this bug, see
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
Server version: 5.5.36-MariaDB-wsrep-log
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2267738 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x30000
The manual page at contains
information that should help you find out what is causing the crash.'

但是整个启动过程依然是失败,所报错误 信息和上面的一致,同样遇到了singal 11错误。

innodb_purge_threads = 0
180102 15:00:11 mysqld_safe WSREP: Running position recovery with --log_error='/var/lib/mysql4000/wsrep_recovery.aFZ19U' --pid-file='/var/lib/mysql4000/'
180102 15:00:12 mysqld_safe WSREP: Recovered position 00000000-0000-0000-0000-000000000000:-1
180102 15:00:14 mysqld_safe Number of processes running now: 0
180102 15:00:14 mysqld_safe mysqld restarted
180102 15:00:14 mysqld_safe WSREP: Running position recovery with --log_error='/var/lib/mysql4000/wsrep_recovery.3jbUnM' --pid-file='/var/lib/mysql4000/'
180102 15:00:15 mysqld_safe WSREP: Recovered position 00000000-0000-0000-0000-000000000000:-1
180102 15:00:17 mysqld_safe Number of processes running now: 0
180102 15:00:17 mysqld_safe mysqld restarted
180102 15:00:17 mysqld_safe WSREP: Running position recovery with --log_error='/var/lib/mysql4000/wsrep_recovery.3H1iqG' --pid-file='/var/lib/mysql4000/'
180102 15:00:18 mysqld_safe WSREP: Recovered position 00000000-0000-0000-0000-000000000000:-1
180102 15:00:20 mysqld_safe Number of processes running now: 0
180102 15:00:20 mysqld_safe mysqld restarted
而且有一个问题一直没想明白,就是遇到WSREP: Recovered position 00000000-0000-0000-0000-000000000000:-1

innodb_purge_threads = 0
180102 15:05:59  InnoDB: Error: page 567 log sequence number 4076805
InnoDB: is in the future! Current system log sequence number 9228.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: for more information.

6.再次提升 innodb_force_recovery的级别,再次进行重启动.
阅读(2240) | 评论(2) | 转发(0) |




24259138642018-03-23 14:30:41


onlyword2018-01-03 15:49:31



登录 注册