Chinaunix首页 | 论坛 | 博客
  • 博客访问: 2809449
  • 博文数量: 587
  • 博客积分: 6356
  • 博客等级: 准将
  • 技术积分: 6410
  • 用 户 组: 普通用户
  • 注册时间: 2008-10-23 10:54
个人简介

器量大者,福泽必厚

文章分类

全部博文(587)

文章存档

2019年(3)

2018年(1)

2017年(29)

2016年(39)

2015年(66)

2014年(117)

2013年(136)

2012年(58)

2011年(34)

2010年(50)

2009年(38)

2008年(16)

分类: LINUX

2014-01-13 16:52:47

###由衷感谢我只是将他在ubuntu下的过程在centos下重新"翻译"一遍

1:install openssl-server
yum -y install openssh-server 


2:mysql 安装, 安装过程省略.....
3:Install Ruby via RVM
curl -L | bash -s stable
source /etc/profile.d/rvm.sh
yum -y install  build-essential openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 


libxml2-dev libxslt-dev autoconf libc6-dev libgdbm-dev ncurses-dev automake libtool bison subversion pkg-config libffi-dev  ##这个是老外执行的!
[root@szl ~]# rvm install ruby  ##安装ruby
Searching for binary rubies, this might take some time.
Found remote file
Checking requirements for centos.
Installing requirements for centos.
Updating system.
Installing required packages: readline-devel, libyaml-devel, libffi-devel.......
Requirements installation successful.
ruby-2.1.0 - #configure
ruby-2.1.0 - #download
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 20.1M  100 20.1M    0     0  1063k      0  0:00:19  0:00:19 --:--:-- 1717k
ruby-2.1.0 - #validate archive
ruby-2.1.0 - #extract
ruby-2.1.0 - #validate binary
ruby-2.1.0 - #setup
ruby-2.1.0 - #making binaries executable.
ruby-2.1.0 - #downloading rubygems-2.2.1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  401k  100  401k    0     0   236k      0  0:00:01  0:00:01 --:--:--  291k
No checksum for downloaded archive, recording checksum in user configuration.
ruby-2.1.0 - #extracting rubygems-2.2.1.
ruby-2.1.0 - #removing old rubygems.
ruby-2.1.0 - #installing rubygems-2.2.1............
ruby-2.1.0 - #gemset created /usr/local/rvm/gems/ruby-2.1.0@global
ruby-2.1.0 - #importing gemset /usr/local/rvm/gemsets/global.gems..
ruby-2.1.0 - #generating global wrappers.
ruby-2.1.0 - #gemset created /usr/local/rvm/gems/ruby-2.1.0
ruby-2.1.0 - #importing gemsetfile /usr/local/rvm/gemsets/default.gems evaluated to empty gem list
ruby-2.1.0 - #generating default wrappers.
###到此已经安装完成!


4:redis安装:
yum -y install tcl
下载
tar zxvf redis-2.8.3.tar.gz
cd redis-2.8.3
make
make test    ##如果不安装tcl,会在make test时报错如下:
[root@szl redis-2.8.3]# make test
cd src && make test
make[1]: Entering directory `/root/redis-2.8.3/src'
You need tcl 8.5 or newer in order to run the Redis test
make[1]: *** [test] Error 1
make[1]: Leaving directory `/root/redis-2.8.3/src'
make: *** [test] Error 2
 cd src
cp redis-benchmark redis-check-aof redis-check-dump redis-cli redis-sentinel redis-server /usr/local/bin/
cd ..
cp redis.conf /etc

# start redis
redis-server /etc/redis.conf &
[root@szl ruby]# lsof -i :6379 ##确认redis是否真正启动
COMMAND     PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
redis-ser 17316 root    4u  IPv6  92525      0t0  TCP *:6379 (LISTEN)
redis-ser 17316 root    5u  IPv4  92527      0t0  TCP *:6379 (LISTEN)

# test redis
[root@szl ~]# redis-cli  ping
PONG


##Install Gearman 
Gearman是一个支持分布式的任务分发框架,
Gearman, like managers, assign the tasks but do none of the real work themselves ##介绍
下载epel-release-6-8.1.noarch.rpm ## 我的系统版本为centos6.3的,如果是centos5的要更换成相应的rpm,否则安装不上!
rpm -vih epel-release-6-8.1.noarch.rpm
yum -y install gearmand
[root@szl ~]# /etc/init.d/gearmand  start
Starting gearmand:                                         [  OK  ]
[root@szl ~]# /etc/init.d/gearmand  status                 ##现在gearmand服务已经在运行了!
gearmand (pid  17414) is running...
gearman介绍及工作原理,见附件



##Installed MySQL JSON UDF
###lib_mysqludf_json: a udf  library of functions to map relational data to the JSON format.
使用lib_mysqludf_json的原因是因为Gearman只接受字符串作为入口参数,可以通过
lib_mysqludf_json将MySQL中的数据编码为JSON字符串.

yum -y install git
git clone  
cd lib_mysqludf_json  
[root@szl lib_mysqludf_json]# gcc $(/usr/local/mysql/bin/mysql_config --cflags) -shared -fPIC -o lib_mysqludf_json.so   lib_mysqludf_json.c 
[root@szl lib_mysqludf_json]# pwd
/home/liuxiaojie/redisplugin/lib_mysqludf_json

[root@szl lib_mysqludf_json]# du lib_mysqludf_json.so  ##执行上面的gcc命令后,lib_mysqludf_json.so文件会变大,执行前16k,执行后40k!
16      lib_mysqludf_json.so
[root@szl lib_mysqludf_json]# du lib_mysqludf_json.so 
40      lib_mysqludf_json.so
[root@szl cron]# mysql -u root -proot --execute="show variables like '%plugin%';"
Warning: Using a password on the command line interface can be insecure.
+---------------+------------------------------------+
| Variable_name | Value                              |
+---------------+------------------------------------+
| plugin_dir    | /usr/local/mysql/lib/mysql/plugin/ |
+---------------+------------------------------------+
# copy shared object to plugin directory
cp lib_mysqludf_json.so /usr/local/mysql/lib/mysql/plugin/

# enable json_object method
mysql -u root -proot --execute="create function json_object returns string soname 'lib_mysqludf_json.so'"


###Installed MySQL Gearman UDF   
我们知道 Gearman 的任务是一个数组结构,只存在内存中,一断电怎么样这个信息就会不存在。所以新的 gearman 开发了一个功能,就是通过 mysql 来存储和调用 Gearman 任务信息。这样可以持久的保持任务的状态,因为可以直接对数据库的操作,上面建个触发器,就相当于 Gearman 的客户端。修改表,插入表就相当于直接下发任务。
##这个可以参考下http://blogread.cn/it/article/4307?f=wb
# yum -y install  libgearman-devel        ##必须先安装这个libgearman-devel rpm,否则在后面configure安装gearman-mysql-udf-0.6时会报错如下:
configure: error: At least version 0.33 of libgearman is required for gearman-mysql-udf
[root@szl ruby]# rpm -qa  | grep  libgearman
libgearman-1.1.8-2.el6.x86_64
libgearman-devel-1.1.8-2.el6.x86_64
##Gearman Server 使用gearman-MySQL-UDF来管理和保持队列
wget
tar -xzf gearman-mysql-udf-0.6.tar.gz
cd gearman-mysql-udf-0.6
./configure --with-mysql=/usr/bin/mysql_config --libdir=/usr/lib/mysql/plugin/


[root@szl gearman-mysql-udf-0.6]# make      ##编译
make  all-am
make[1]: Entering directory `/home/liuxiaojie/redisplugin/gearman-mysql-udf-0.6'
  CC     libgearman_mysql_udf/libgearman_mysql_udf_la-client.lo
  CCLD   libgearman_mysql_udf.la
make[1]: Leaving directory `/home/liuxiaojie/redisplugin/gearman-mysql-udf-0.6'
[root@szl gearman-mysql-udf-0.6]# make install
make[1]: Entering directory `/home/liuxiaojie/redisplugin/gearman-mysql-udf-0.6'
test -z "/usr/local/mysql/lib/mysql/plugin" || /bin/mkdir -p "/usr/local/mysql/lib/mysql/plugin"
 /bin/sh ./libtool   --mode=install /usr/bin/install -c   libgearman_mysql_udf.la '/usr/local/mysql/lib/mysql/plugin'
libtool: install: /usr/bin/install -c .libs/libgearman_mysql_udf.so.0.0.0 /usr/local/mysql/lib/mysql/plugin/libgearman_mysql_udf.so.0.0.0
libtool: install: (cd /usr/local/mysql/lib/mysql/plugin && { ln -s -f libgearman_mysql_udf.so.0.0.0 libgearman_mysql_udf.so.0 || { rm -f 


libgearman_mysql_udf.so.0 && ln -s libgearman_mysql_udf.so.0.0.0 libgearman_mysql_udf.so.0; }; })
libtool: install: (cd /usr/local/mysql/lib/mysql/plugin && { ln -s -f libgearman_mysql_udf.so.0.0.0 libgearman_mysql_udf.so || { rm -f 


libgearman_mysql_udf.so && ln -s libgearman_mysql_udf.so.0.0.0 libgearman_mysql_udf.so; }; })
libtool: install: /usr/bin/install -c .libs/libgearman_mysql_udf.lai /usr/local/mysql/lib/mysql/plugin/libgearman_mysql_udf.la
libtool: finish: PATH="/usr/local/mysql/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/data/mysql/bin:/root/bin:/sbin" ldconfig 


-n /usr/local/mysql/lib/mysql/plugin
----------------------------------------------------------------------
Libraries have been installed in:
   /usr/local/mysql/lib/mysql/plugin


If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the `LD_RUN_PATH' environment variable
     during linking
   - use the `-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to `/etc/ld.so.conf'


See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
make[1]: Nothing to be done for `install-data-am'.
make[1]: Leaving directory `/home/liuxiaojie/redisplugin/gearman-mysql-udf-0.6'
[root@szl gearman-mysql-udf-0.6]# cd /usr/local/mysql/lib/mysql/plugin/
[root@szl plugin]# ll
total 1196
-rwxr-xr-x. 1 root root  16509 Dec 11 12:22 adt_null.so
-rwxr-xr-x. 1 root root  25539 Dec 11 12:22 auth.so
-rwxr-xr-x. 1 root root  12596 Dec 11 12:22 auth_socket.so
-rwxr-xr-x. 1 root root  24664 Dec 11 12:22 auth_test_plugin.so
-rw-r--r--. 1 root root    227 Oct 22 15:42 daemon_example.ini
drwxr-xr-x. 2 root root   4096 Dec 11 12:43 debug
-rwxr-xr-x. 1 root root  39275 Dec 11 12:22 libdaemon_example.so
-rwxr-xr-x. 1 root root   1055 Jan 13 14:43 libgearman_mysql_udf.la
lrwxrwxrwx. 1 root root     29 Jan 13 14:43 libgearman_mysql_udf.so -> libgearman_mysql_udf.so.0.0.0
lrwxrwxrwx. 1 root root     29 Jan 13 14:43 libgearman_mysql_udf.so.0 -> libgearman_mysql_udf.so.0.0.0
-rwxr-xr-x. 1 root root 106627 Jan 13 14:43 libgearman_mysql_udf.so.0.0.0
-rwxr-xr-x. 1 root root  40767 Jan 13 13:11 lib_mysqludf_json.so
-rwxr-xr-x. 1 root root  17891 Dec 11 12:22 mypluglib.so
-rwxr-xr-x. 1 root root  18159 Dec 11 12:22 qa_auth_client.so
-rwxr-xr-x. 1 root root  23942 Dec 11 12:22 qa_auth_interface.so
-rwxr-xr-x. 1 root root  12998 Dec 11 12:22 qa_auth_server.so
-rwxr-xr-x. 1 root root 415933 Dec 11 12:22 semisync_master.so
-rwxr-xr-x. 1 root root 250255 Dec 11 12:22 semisync_slave.so
-rwxr-xr-x. 1 root root 169200 Dec 11 12:22 validate_password.so


# enabled udf functions
mysql -u root -proot --execute="CREATE FUNCTION gman_do_background RETURNS STRING SONAME 'libgearman_mysql_udf.so'"
mysql -u root -proot --execute="CREATE FUNCTION gman_servers_set RETURNS STRING SONAME 'libgearman_mysql_udf.so'"


# set gearman server
mysql -u root -proot --execute="SELECT gman_servers_set('127.0.0.1')"
Warning: Using a password on the command line interface can be insecure.
+-------------------------------+
| gman_servers_set('127.0.0.1') |
+-------------------------------+
| 127.0.0.1                     |
+-------------------------------+

##Ruby setup code
# setup RVM gemset
mkdir ~/ruby
echo "rvm use --create ruby-1.9.3@redis_gearman" > ~/ruby/.rvmrc


##注意ruby-1.9.3 要被替换为机器上的ruby版本,例如我的ruby版本:
[root@szl redisplugin]# ruby -v
ruby 2.0.0p0 (2013-02-24 revision 39474) [x86_64-linux]
则要替换成2.0.0p0
我的文件内容如下:
[root@szl ruby]# pwd
/root/ruby
[root@szl ruby]# cat .rvmrc 
rvm use --create ruby-2.0.0p0@redis_gearman


cd ~/ruby


[root@szl ruby]# ll
total 4
-rw-r--r--. 1 root root 61 Jan 13 14:52 Gemfile
[root@szl ruby]# cat Gemfile
source ''
gem 'gearman-ruby'
gem 'redis'


[root@szl ruby]# bundle  ##执行命令如下
Fetching gem metadata from /..........
Fetching gem metadata from /..
Resolving dependencies...
Installing gearman-ruby (3.0.7) 
Installing redis (3.0.6) 
Using bundler (1.3.5) 
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.
[root@szl ruby]# bundle show redis
/usr/local/lib/ruby/gems/2.0.0/gems/redis-3.0.6  
[root@szl ruby]# bundle show gearman-ruby
/usr/local/lib/ruby/gems/2.0.0/gems/gearman-ruby-3.0.7  ##显示已经安装
 [root@szl ruby]# bundle show
Gems included by the bundle:
  * bundler (1.5.0)
  * gearman-ruby (3.0.7)
  * redis (3.0.6)
[root@szl ruby]# 


测试:
Testing Gearman worker & Redis


# in terminal 1, start worker
./redis_worker.rb


# in terminal 2, check gearman status & verify worker
(echo status ; sleep 0.1) | netcat 127.0.0.1 4730
redis_worker    0   0   1


# in terminal 3, monitor redis
redis-cli monitor
OK


# in terminal 4, run client test script
./redis_client.rb  ##在这里测试的时候遇到一个问题,我安装的gearman-ruby为ruby-1.9.3版本,造成2.0.0p0下的ruby不能加载它,导致失败,解决方法,如上面所示!
true


# in terminal 3, verify redis lpush:
redis-cli monitor
OK
1361012555.700504 [0 127.0.0.1:34135] "lpush" "user_page_views:1" "{\"user_id\":1,\"timestamp\":\"2013-02-14 19:13:15\",\"page\":\"\"}"






# add database & table
mysql -u root -proot
mysql> create database redisgearman;
mysql> use redisgearman;
mysql> CREATE TABLE `user_page_views` (
  `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `user_id` int(11) NOT NULL,
  `timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  `page` varchar(255) NOT NULL DEFAULT '',
  PRIMARY KEY (`id`),
  KEY `user_id` (`user_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
mysql> exit


# add test record
mysql -u root -proot redisgearman --execute="insert into user_page_views (user_id, page) values (1, '')"


# ensure json udf is working


[root@szl lib_mysqludf_json]#  mysql -u root -proot redisgearman --execute="select json_object(user_id as \`user_id\`, timestamp as \`timestamp\`, page as 


\`page\`) as json from user_page_views"
Warning: Using a password on the command line interface can be insecure.
+--------------------------------------------------------------------------------+
| json                                                                           |
+--------------------------------------------------------------------------------+
| {"user_id":1,"timestamp":"2014-01-13 15:34:08","page":""} |
+--------------------------------------------------------------------------------+
在mysql中插入一条记录时,在redis端就可以显示出来:


mysql> use redisgearman;
Database changed
mysql> show tables;
+------------------------+
| Tables_in_redisgearman |
+------------------------+
| user_page_views        |
+------------------------+
1 row in set (0.00 sec)


mysql> select * from   user_page_views ; ##mysql记录如下:
+----+---------+---------------------+------------------------------------+
| id | user_id | timestamp           | page                               |
+----+---------+---------------------+------------------------------------+
|  1 |       1 | 2014-01-13 15:34:08 |              |
|  2 |       1 | 2014-01-13 15:57:39 | |
|  3 |       1 | 2014-01-13 16:03:19 | htt3,            |
|  4 |       1 | 2014-01-13 16:04:36 |               |
+----+---------+---------------------+------------------------------------+
4 rows in set (0.00 sec)




[root@szl ~]# redis-cli monitor
OK
1389596914.865050 [0 127.0.0.1:34947] "lpush" "user_page_views:1" "{\"user_id\":1,\"timestamp\":\"2013-02-14 19:13:15\",\"page\":\"\"}"
1389599859.179215 [0 127.0.0.1:34947] "lpush" "user_page_views:1" "{\"user_id\":1,\"timestamp\":\"2014-01-13 15:57:39\",\"page\":


\"\"}"
1389600038.597863 [0 127.0.0.1:34962] "monitor"
1389600199.973299 [0 127.0.0.1:34947] "lpush" "user_page_views:1" "{\"user_id\":1,\"timestamp\":\"2014-01-13 16:03:19\",\"page\":\"htt3,


\"}"
1389600276.782816 [0 127.0.0.1:34947] "lpush" "user_page_views:1" "{\"user_id\":1,\"timestamp\":\"2014-01-13 16:04:36\",\"page\":\"\"}"





阅读(1885) | 评论(0) | 转发(0) |
给主人留下些什么吧!~~