ERROR 1135: Can't create a new thread (errno 11).

1.故障状态描述

自己在做压力测试出现如下现象:

mysql -uroot -p
Enter password:
ERROR 1135: Can't create a new thread (errno 11). If you are not out of available memory,

you can consult the manual for a possible OS-dependent bug

[@rhas31 ]$ perror 11
System error: 11 = Resource temporarily unavailable

mysql 命令行无法连接数据库

ps -efw|grep mysql|wc
891 13354 204625

mysql并发连接数量较大,有890个.


[root@ var]# free
total used free shared buffers cached
Mem: 2061584 2043568 18016 0 13864 293192
-/+ buffers/cache: 1736512 325072
Swap: 2040212 346660 1693552

虚拟内存已经使用了约 350M

数据库错误日志没有异常的错误记录.

050315 12:01:33 mysqld started
/app/mysql-4.0.23a/libexec/mysqld: ready for connections.
Version: '4.0.23a' socket: '/tmp/mysql.sock' port: 3306 Source distribution
050323 11:58:10 mysqld started
/app/mysql-4.0.23a/libexec/mysqld: ready for connections.
Version: '4.0.23a' socket: '/tmp/mysql.sock' port: 3306 Source distribution
050323 13:10:53 mysqld started
/app/mysql-4.0.23a/libexec/mysqld: ready for connections.
Version: '4.0.23a' socket: '/tmp/mysql.sock' port: 3306 Source distribution
050323 13:10:54 /app/mysql-4.0.23a/libexec/mysqld: Can't open file: 'ujoin.MYI'. (errno:

145)

top 命令显示正常,mysql在正常运行,但程序员反映程序执行慢.

2.故障原因分析

应为无法用命令行连接上数据库,所以无法得知当时sql语句的执行情况.
初步分析是有部分sql语句效率不高所致.

关于 mysql 连接时的报错信息,我们曾经在 turbolinux HA mysql 数据库做压力测试的时候遇到过,
后来跟turbolinux联系,升级 glibc 后解决的.


* Wed Apr 21 2004 Yang xiaoxin

- 2.2.5-18
- Thread : modified PTHREAD_THREADS_MAX and STACK_SIZE for huawei

初步分析,此次错误与glibc的版本也有关系.

3.故障处理过程

ps -efw|grep mysql|awk '{print $2 }' > /tmp/1.txt

后使用 kill 命令将mysql 杀死.
再手工执行启动操作后,数据库服务器恢复正常

4.故障处理结果
将mysql最大 连接数调小到 800
升级glibc相关软件包到最新的稳定版

[root@ i386rpm]# ls
glibc-2.3.2-95.30.i386.rpm glibc-debuginfo-2.3.2-95.30.i386.rpm

glibc-headers-2.3.2-95.30.i386.rpm
glibc-common-2.3.2-95.30.i386.rpm glibc-debuginfo-common-2.3.2-95.30.i386.rpm

glibc-profile-2.3.2-95.30.i386.rpm
glibc-debug-2.3.2-95.30.i386.rpm glibc-devel-2.3.2-95.30.i386.rpm

glibc-utils-2.3.2-95.30.i386.rpm

[root@ i386rpm]# rpm -Fvh *.rpm
Preparing... ########################################### [100%]
1:glibc-common ########################################### [ 17%]
2:glibc ########################################### [ 33%]
Stopping sshd:[ OK ]
sleep: relocation error: /lib/i686/librt.so.1: symbol __pthread_clock_settime, version

GLIBC_PRIVATE not defined in file libpthread.so.0 with link time reference
Starting sshd:[ OK ]
3:glibc-headers ########################################### [ 50%]
4:glibc-devel ########################################### [ 67%]
5:glibc-profile ########################################### [ 83%]
6:glibc-utils ########################################### [100%]