Oracle10.2.0.3 ORA-3136 错误解决

http://space.itpub.net/519536/viewspace-473287

1.问题现象
alert日志文件报如下错误
WARNING: inbound connection timed out (ORA-3136)
Mon Oct 20 14:33:52 2008
WARNING: inbound connection timed out (ORA-3136)
Mon Oct 20 14:34:46 2008
WARNING: inbound connection timed out (ORA-3136)

sqlnet.log中记录了如下错误:
Fatal NI connect error 12170.

  VERSION INFORMATION:
        TNS for Linux: Version 10.2.0.3.0 - Production
        Oracle Bequeath NT Protocol Adapter for Linux: Version 10.2.0.3.0 - Production
        TCP/IP NT Protocol Adapter for Linux: Version 10.2.0.3.0 - Production
  Time: 20-OCT-2008 14:34:46
  Tracing not turned on.
  Tns error struct:
    ns main err code: 12535
    TNS-12535: TNS:operation timed out
    ns secondary err code: 12606
    nt main err code: 0
    nt secondary err code: 0
    nt OS err code: 0
  Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.123.103)(PORT=3124))

2.Metalink上给出了如下的解决方案
1）set INBOUND_CONNECT_TIMEOUT_=0 in listener.ora
2）set SQLNET.INBOUND_CONNECT_TIMEOUT = 0 in sqlnet.ora of server.
3）stop and start both listener and database.
4）Now try to connect to DB and observe the behaviour

如果不重启只是进行reload也可以修改成功
通过如下方式验证是否修改成功
LSNRCTL> show inbound_connect_timeout

修改listener的inbound_connect_timeout参数的方法
方法一:
$ lsnrctl

LSNRCTL for IBM/AIX RISC System/6000: Version 10.2.0.3.0 - Production on 29-OCT-2007 10:00:57

Copyright (c) 1991, 2006, Oracle.  All rights reserved.

Welcome to LSNRCTL, type "help" for information.

LSNRCTL> help
The following operations are available
An asterisk (*) denotes a modifier or extended command:

start               stop                status
services            version             reload
save_config         trace               spawn
change_password     quit                exit
set*                show*

LSNRCTL> show
The following operations are available after show
An asterisk (*) denotes a modifier or extended command:

rawmode                     displaymode
rules                       trc_file
trc_directory               trc_level
log_file                    log_directory
log_status                  current_listener
inbound_connect_timeout     startup_waittime
snmp_visible                save_config_on_stop
dynamic_registration

LSNRCTL> show inbound_connect_timeout
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
LISTENER parameter "inbound_connect_timeout" set to 60
The command completed successfully

LSNRCTL> set inbound_connect_timeout 0
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
LISTENER parameter "inbound_connect_timeout" set to 0
The command completed successfully

LSNRCTL> show inbound_connect_timeout
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
LISTENER parameter "inbound_connect_timeout" set to 0
The command completed successfully

LSNRCTL> set save_config_on_stop on   #表示修改参数永久生效，否则只是临时生效，下次重启监听又还原为原来的值了

LSNRCTL> exit


方法二：
修改listener.ora文件，加入:  INBOUND_CONNECT_TIMEOUT_LISTENER_NAME=0

Subject:  Troubleshooting ORA - 3136 WARNING Inbound Connection Timed Out
  Doc ID:  Note:465043.1 Type:  TROUBLESHOOTING
  Last Revision Date:  30-JUN-2008 Status:  PUBLISHED

In this Document
  Purpose
  Last Review Date
  Instructions for the Reader
  Troubleshooting Details

--------------------------------------------------------------------------------
Applies to:
Oracle Net Services - Version: 10.2.0.1.0
Information in this document applies to any platform.

Purpose
Troubleshooting guide for "ORA -3136  WARNING inbound connection timed out" seen in the alert log.
Last Review Date
July 1, 2008
Instructions for the Reader
A Troubleshooting Guide is provided to assist in debugging a specific issue. When possible, diagnostic tools are included in the document to assist in troubleshooting.
Troubleshooting Details
The "WARNING: inbound connection timed out (ORA-3136)" in the alert log indicates that the client was not able to complete  it's authentication within the period of time specified by parameter SQLNET.INBOUND_CONNECT_TIMEOUT.

You may also witness ORA-12170 without timeout error on the database server sqlnet.log file.
This entry would also have the clinet address which failed to get authenticated. Some applications or JDBC thin driver applications may not have these details.

From 10.2 onwards the default value of parameter SQLNET.INBOUND_CONNECT_TIMEOUT is 60 seconds, hence if the client is not able authenticate within 60 secs , the warning would appear in the alert log and the client connection will be terminated.


Note: This timeout restriction was introduced to combat Denial of Service (DoS) attack whereby malicious clients attempt to flood database servers with connect requests that consumes resources.

There can be three main reasons for this error -

Server gets a connection request from a malicious client which is not supposed to connect to the database , in which case the error thrown is the correct behavior. You can get the client address for which the error was thrown via sqlnet log file.
The server receives a valid client connection request but the client takes a long time to authenticate more than the default 60 seconds.
The DB server is heavily loaded due to which it cannot finish the client logon within the timeout specified.
To understand what is causing this issue, following checks can be done


The default value of 60 seconds is good enough in most conditions for the database server to authenticate a client connection. If its taking longer period, then its worth checking all the below points before going for the workadound:

1. Check whether local connection on the database server is sucessful & quick.
2. If local connections are quick ,then check for underlying network delay with the help of your network administrator.
3. Check whether your Database performance has degraded by anyway.
4. Check alert log for any critical errors for eg, ORA-600 or ORA-7445 and get them  resolved first.
These critical errors might have triggered the slowness of the database server.

As a workaround to avoid only this warning messages, you can set the parameters SQLNET.INBOUND_CONNECT_TIMEOUT
and
 INBOUND_CONNECT_TIMEOUT_
to the value more than 60.

For e.g 120. So that the client will have more time to provide the authentication information to the database. You may have to further tune these parameter values according to your setup.

To set these parameter -

1. In server side sqlnet.ora file add

SQLNET.INBOUND_CONNECT_TIMEOUT

For e.g


SQLNET.INBOUND_CONNECT_TIMEOUT = 120
2. In listener.ora file -

INBOUND_CONNECT_TIMEOUT_ = 110

For e.g if the listener name is LISTENER then -

INBOUND_CONNECT_TIMEOUT_LISTENER = 110

From Oracle version 10.2.0.3 onwards the default value of INBOUND_CONNECT_TIMEOUT_ is 60 seconds. For previous releases it is zero by default.


How to check whether inbound timout is active for the listener and database server

For eg. INBOUND_CONNECT_TIMEOUT_ =4

You can check whether the parameter is active or not by simply doing telnet to the listener port.
$ telnet
for eg.
$ telnet 192.168.12.13 1521
The telnet session should disconnect after 4 seconds which indicates that the inbound connection timeout for the listener is active.

To check whether database server sqlnet.inbound_connect_timeout is active:
Eg.
sqlnet.inbound_connect_timeout =5

a. For Dedicated server setup, enable the support level sqlnet server tracing will show the timeout value as below:

niotns: Enabling CTO, value=5000 (milliseconds) <== 5 seconds
niotns: Not enabling dead connection detection.
niotns: listener bequeathed shadow coming to life...


b. For shared Server setup,
$ telnet
For eg.
$ telnet 192.168.12.13  51658
The telnet session should disconnect after 5 seconds which indicates that the sqlnet.inbound_connection_timeout is active.


If you have further question / query regarding this issue then please create Service Request via metalink with following information:

a. Client and matching server sqlnet trace generated at support level.

Note 395525.1 How to Enable Oracle Net Client,Server,Listener,Kerberos and External procedure Tracing from Net Manager (netmgr):
Note 374116.1 How to Match Oracle Net Client and Server Trace Files

b. upload sqlnet.ora, listener.ora Sqlnet.log, & Alert_.log from database server


Keywords
SQLNET.LOG; AUTHENTICATION; DATABASE~PERFORMANCE; DENIAL~OF~SERVICE; SQLNET.INBOUND_CONNECT_TIMEOUT; CONNECTION~TIMED~OUT; SQLNET.LOG;
--------------------------------------------------------------------------------

Help us improve our service. Please email us your comments for this document. .

以上处理方法供大家参考。

Good luck.

secooler
10.09.25

-- The End --