skytools linux 安装

 

Skytools安装文档

1.安装软件

通过源码安装的方式安装londiste环境

本次安装需要提前在三台机器上安装好postgres，这里分别为root节点A，branch节点B，leaf节点C。

①在安装之前，需要确认几个依赖包是否已经安装好

[root@node1 postgresql-9.1.4]# python -V
Python 2.6.6                                                     --如果不存在，可以yum安装

②yum安装psycopg2
yum -y install python-psycopg2.x86_64

 

③源码安装skytools

[root@node1 src]# tar -xvf skytools-3.2.tar.gz

[root@node1 skytools-3.2]# ./configure --prefix=/usr/local/skytools-3.2  --with-pgconfig=/usr/local/pgsql/bin/pg_config

[root@node1 skytools-3.2]# make

[root@node1 skytools-3.2]# make install

 

***************************************************************************************

2.创建节点

创建主节点（A节点）

 

1 编辑环境.bash_profile

export PATH=$PATH:$HOME/bin:$PGHOME/bin:/usr/local/skytools-3.2/bin
export LD_LIBRARY_PATH=$PGHOME/lib:/lib64:/usr/lib64:/usr/local/lib64:/lib:/usr/lib:/usr/local/lib:/usr/local/skytools-3.2/lib:$LD_LIBRARY_PATH

2 创建PgQ ticker配置文件pgqd.ini

[postgres@node1 londiste]$ vi pgqd.ini

[pgqd]
# where to log
logfile = /home/postgres/londiste/pgqd.log
# pidfile
pidfile = /home/postgres/londiste/pgqd.pid
## optional parameters ##
# libpq connect string without dbname=
base_connstr = host=192.168.xxx.x user=pgq password=pgq port=5432
# startup db to query other databases
#initial_database = template1
# limit ticker to specific databases
database_list = pgq
# log into syslog
#syslog = 1
#syslog_ident = pgqd
## optional timeouts ##
# how often to check for new databases
#check_period = 60
# how often to flush retry queue
#retry_period = 30
# how often to do maintentance
#maint_period = 120
# how often to run ticker
ticker_period = 1

3 启动ticker daemon

[postgres@node1 londiste]$ pgqd -d pgqd.ini

 

4  配置p_skytf.init（源库）

[londiste3]
job_name = src_root
db = host=127.0.0.1 port=5432 dbname=pgq user=pgq password=pgq
pgq_queue_name = testing
logfile = /home/postgres/londiste/p_skytf.log
pidfile = /home/postgres/londiste/p_fskytf.pid

 

③初始化Londist源库：

[postgres@node1 londiste]$ londiste3 p_skytf.init create-root node_p 'host=192.168.xxx.x port=5432 user=pgq password=pgq dbname=pgq'
2014-11-10 19:56:05,125 13967 INFO plpgsql is installed
2014-11-10 19:56:05,126 13967 INFO pgq is installed
2014-11-10 19:56:05,129 13967 INFO pgq.get_batch_cursor is installed
2014-11-10 19:56:05,130 13967 INFO pgq_ext is installed
2014-11-10 19:56:05,131 13967 INFO pgq_node is installed
2014-11-10 19:56:05,132 13967 INFO londiste is installed
2014-11-10 19:56:05,134 13967 INFO londiste.global_add_table is installed
2014-11-10 19:56:05,158 13967 INFO Initializing node
2014-11-10 19:56:05,161 13967 INFO Location registered
2014-11-10 19:56:05,260 13967 INFO Node "node_p" initialized for queue "testing" with type "root"
2014-11-10 19:56:05,264 13967 INFO Done

 

④ 开启源库节点worker进程

[postgres@node1 londiste]$ londiste3 -d p_skytf.init worker

 

**********************************************************************

创建branch节点（B节点）

 
1 编辑环境.bash_profile

export PATH=$PATH:$HOME/bin:$PGHOME/bin:/usr/local/skytools-3.2/bin
export LD_LIBRARY_PATH=$PGHOME/lib:/lib64:/usr/lib64:/usr/local/lib64:/lib:/usr/lib:/usr/local/lib:/usr/local/skytools-3.2/lib:$LD_LIBRARY_PATH

4 配置branch_skytf.init（目标库）

[postgres@node2 londiste]$ vi branch_skytf.ini
[londiste3]
job_name = src_branch
db = dbname=pgq user=pgq port=5432 host=192.168.xxx.y password=pgq
pgq_queue_name = testing
logfile = /home/postgres/londiste/branch_skytf.log
pidfile = /home/postgres/londiste/branch_skytf.pid

 

③ 初始化Londist目标库

[postgres@node2 londiste]$ londiste3 branch_skytf.ini create-branch node_b 'host=192.168.xxx.y port=5432 user=pgq password=pgq dbname=pgq' --provide='host=192.168.xxx.x port=5432 user=pgq dbname=pgq password=pgq'
2014-11-10 20:20:19,355 28042 INFO plpgsql is installed
2014-11-10 20:20:19,357 28042 INFO Installing pgq
2014-11-10 20:20:19,357 28042 INFO   Reading from /opt/python2.7.8/share/skytools3/pgq.sql
2014-11-10 20:20:19,746 28042 INFO pgq.get_batch_cursor is installed
2014-11-10 20:20:19,747 28042 INFO Installing pgq_ext
2014-11-10 20:20:19,747 28042 INFO   Reading from /opt/python2.7.8/share/skytools3/pgq_ext.sql
2014-11-10 20:20:19,822 28042 INFO Installing pgq_node
2014-11-10 20:20:19,823 28042 INFO   Reading from /opt/python2.7.8/share/skytools3/pgq_node.sql
2014-11-10 20:20:19,925 28042 INFO Installing londiste
2014-11-10 20:20:19,925 28042 INFO   Reading from /opt/python2.7.8/share/skytools3/londiste.sql
2014-11-10 20:20:20,097 28042 INFO londiste.global_add_table is installed
2014-11-10 20:20:20,131 28042 INFO Initializing node
2014-11-10 20:20:20,211 28042 INFO Location registered
2014-11-10 20:20:20,220 28042 INFO Location registered
2014-11-10 20:20:20,231 28042 INFO Subscriber registered: node_b
2014-11-10 20:20:20,235 28042 INFO Location registered
2014-11-10 20:20:20,238 28042 INFO Location registered
2014-11-10 20:20:20,245 28042 INFO Node "node_s" initialized for queue "testing" with type "branch"
2014-11-10 20:20:20,252 28042 INFO Done

 

[postgres@node2 londiste]$ londiste3 -d branch_skytf.init worker

 

***************************************************

创建leaf节点（C节点）

 

① 编辑环境.bash_profile

export PATH=$PATH:$HOME/bin:$PGHOME/bin:/usr/local/skytools-3.2/bin
export LD_LIBRARY_PATH=$PGHOME/lib:/lib64:/usr/lib64:/usr/local/lib64:/lib:/usr/lib:/usr/local/lib:/usr/local/skytools-3.2/lib:$LD_LIBRARY_PATH

 

② 配置s62_skytf.init（目标库）

[postgres@node2 londiste]$ vi leaf_skytf.ini
[londiste3]
job_name = src_leaf
db = dbname=pgq user=pgq port=5432 host=192.168.xxx.z password=pgq
pgq_queue_name = testing
logfile = /home/postgres/londiste/leaf_skytf.log
pidfile = /home/postgres/londiste/leaf_skytf.pid

 

③ 初始化Londist目标库

[postgres@node2 londiste]$ londiste3 leaf_skytf.ini create-leaf node_s 'host=192.168.xxx.z port=5432 user=pgq password=pgq dbname=pgq' --provide='host=192.168.xxx.x port=5432 user=pgq dbname=pgq password=pgq'
2014-11-10 20:20:19,355 28042 INFO plpgsql is installed
2014-11-10 20:20:19,357 28042 INFO Installing pgq
2014-11-10 20:20:19,357 28042 INFO   Reading from /opt/python2.7.8/share/skytools3/pgq.sql
2014-11-10 20:20:19,746 28042 INFO pgq.get_batch_cursor is installed
2014-11-10 20:20:19,747 28042 INFO Installing pgq_ext
2014-11-10 20:20:19,747 28042 INFO   Reading from /opt/python2.7.8/share/skytools3/pgq_ext.sql
2014-11-10 20:20:19,822 28042 INFO Installing pgq_node
2014-11-10 20:20:19,823 28042 INFO   Reading from /opt/python2.7.8/share/skytools3/pgq_node.sql
2014-11-10 20:20:19,925 28042 INFO Installing londiste
2014-11-10 20:20:19,925 28042 INFO   Reading from /opt/python2.7.8/share/skytools3/londiste.sql
2014-11-10 20:20:20,097 28042 INFO londiste.global_add_table is installed
2014-11-10 20:20:20,131 28042 INFO Initializing node
2014-11-10 20:20:20,211 28042 INFO Location registered
2014-11-10 20:20:20,220 28042 INFO Location registered
2014-11-10 20:20:20,231 28042 INFO Subscriber registered: node_s
2014-11-10 20:20:20,235 28042 INFO Location registered
2014-11-10 20:20:20,238 28042 INFO Location registered
2014-11-10 20:20:20,245 28042 INFO Node "node_s" initialized for queue "testing" with type "leaf"
2014-11-10 20:20:20,252 28042 INFO Done

 

[postgres@node2 londiste]$ londiste3 -d leaf_skytf.init worker

***************************************************

 

 

3 一些常用命令

查看主节点状态

[postgres@node1 londiste]$ londiste3 p_skytf.init status
Queue: testing   Local node: node_p

node_p (root)
  |                           Tables: 2/0/0
  |                           Lag: 2s, Tick: 14220
  +--: node_b (branch)
  |                           Tables: 2/0/0
  |                           Lag: 2s, Tick: 14220
  +--: node_s (leaf)
                              Tables: 1/1/0
                              Lag: 2s, Tick: 14220