shell监控日志文件，并将所需结果写到21个地市用户的库表

问题的提出

 

所用系统udm是c/c++程序，运行时会根据所运行的task_id 写日志，每月月帐期1至3号运行，由分部（广东21个地市）根据进度调度相应模块，udm系统运行在5台不同的Unix服务器上(hp/aix/sunos)，而这些因为管理的需要都未对分部开放，于是问题就来了：分部如何知道自已调的程序是否有正常跑完？如何知道程序运行的进度？

 

问题的解决

 

两个月前写的，被实践证明确实有效

 

1、  21个地市建日志表

create table UDM_ERROR_LOG
(
  sum_month VARCHAR2(12),
  task_id VARCHAR2(25),
  err_time DATE,
  err_infor VARCHAR2(4000),
  load_date DATE
)

 

2、  5台服务器上部署脚本 chkudm.sh

 

#!/bin/sh
write_db()
{
        in_city=$1
        in_month=$2
        db_pwd=`cat $POCO_BASE/script/config/summary_ods_xx_pwd.config|grep ${in_city}|awk -F"" '{print $3}'`
        db_user=`cat $POCO_BASE/script/config/summary_ods_xx_pwd.config|grep ${in_city}|awk -F"" '{print $2}'`
        db_tns=`cat $POCO_BASE/script/config/summary_ods_xx_pwd.config|grep ${in_city}|awk -F"" '{print $4}'`
        orapwd="}"
       
        exesql=${HOME}/bin/fangp/chkudm/sql/delete.sql
       
        echo "set pagesize 0 feedback off verify off heading off echo off" >>${exesql}
        echo "begin" >>${exesql}
        echo "  EXECUTE IMMEDIATE 'delete from udm_error_log where trim(sum_month)=''${in_month}''';" >>${exesql}
        echo "  commit;" >>${exesql}
        echo "end;" >>${exesql}
        echo "/" >>${exesql}
        #echo ${orapwd}
        #cat ${exesql}
       
        `sqlplus -S $orapwd<               @${exesql}
               exit
         EOF
               exit`
       
        rm ${exesql}
       
        sqlldr userid=${orapwd} control=${HOME}/bin/fangp/chkudm/ctl/chk_file_${in_city}.ctl  log=${HOME}/bin/fangp/chkudm/log/chk_file_${in_city}.ctl.log
}

chk_log()
{
        in_city=$1
        in_month=$2
        sum_month=$3
        load_date=`date '+%Y-%m-%d %H:%M:%S'`
       
        #log path
        LOG_PATH=${HOME}/log
        cd $LOG_PATH
       
        chk_file=${HOME}/bin/fangp/chkudm/file/chk_file_${in_city}.txt
       
        tail -1 ServInfoManage_${in_city}.log|grep ${in_month}|awk -F'[][]' '{print "'"${sum_month}"'"  "|"  "ServInfoManage"  "|"  $2  "|"  $9  "|"  "'"${load_date}"'"}' >>${chk_file}
       
        #tail task_id  hff 200--29600, yffgw 28100--28500, yffcnet 21200--21600, arrear 26000、26100, gongyun 24000、 24100, hffpay 23100、23002
       
        for task_id in `echo 200 29000 29100 29300 29500 29200 29400 29600 28100 28200 28800 28900 28300 28400 28500 21200 21300 21400 21500 21600 26000 26100 24000 24100 23100 23002`
        do
                tail -1 ${in_city}.${task_id}.log|grep ${in_month}|awk -F'[][]' '{print "'"${sum_month}"'"  "|"  '"${task_id}"'  "|"  $2  "|"  $7  "|"  "'"${load_date}"'"}' >>${chk_file}
        done
       
       
        cat ServInfoManage_${in_city}.log|egrep 'ERROR|ORA'|grep ${in_month}|awk -F'[][]' '{print "'"${sum_month}"'"  "|"  "ServInfoManage"  "|"  $2  "|"  $9  "|"  "'"${load_date}"'"}'
       
        #ora / err
       
        for task_id in `echo 200 29000 29100 29300 29500 29200 29400 29600 28100 28200 28800 28900 28300 28400 28500 21200 21300 21400 21500 21600 26000 26100 24000 24100 23100 23002`
        do
                cat ${in_city}.${task_id}.log|egrep 'ERROR|ORA'|grep ${in_month}|awk -F'[][]' '{print '"${sum_month}"'  "|"  '"$task_id"'  "|"  $2  "|"  $7  "|"  "'"${load_date}"'"}' >>${chk_file}
        done   
   
}

 

if [ $# = 1 ]                                
then                                          
        in_date=$1
        year=`echo ${in_date}|cut -c 1-4`
        month=`echo ${in_date}|cut -c 5-6`
        CURR_MONTH=`echo ${year}-${month}`
        smonth=`echo ${in_date}|cut -c 1-6`                                  
else                                          
        CURR_MONTH=`date +'%Y-%m'`
        smonth=`date +'%Y%m'`              
fi 
rm ${HOME}/bin/fangp/chkudm/file/chk_file_??.txt
for city in `echo gz dg jm yj`
do
      chk_log ${city} ${CURR_MONTH} ${smonth}
      write_db ${city} ${smonth}
done

注：(1)每台机器，不同处是 for city in `echo gz dg jm yj` ，只要把相应地市名改下即可

       (2)awk -F""   ，引号处的分隔符是ctrl+s，用vi打开可以看到，用cat打开是看不出来的

 

 

3、入库的ctl文件

 

append into table udm_error_log
fields terminated by '|'
(
    sum_month,
    task_id,
    err_time date "yyyy-mm-dd hh24:mi:ss",
    err_infor,
    load_date date "yyyy-mm-dd hh24:mi:ss"
)

 

 

4、  crontab 自动调度

 

0,5,10,15,20,25,30,35,40,45,50,55 * 1-3 * * (. /file3/app/.profile;/file3/app/bin/fangp/chkudm/chkudm.sh >>  /file3/app/bin/fangp/chkudm/log/chkudm.log 2>&1)

 

注：由于hp/aix/sunos 都不支持 */5  ，所以只能用上面的办法

 

***** command to be executed
-----
| | | | |
| | | | +----- day of week (0 - 6) (Sunday=0)
| | | +------- month (1 - 12)
| | +--------- day of month (1 - 31)
| +----------- hour (0 - 23)
+------------- min (0 - 59)