樽中酒不空
分类： 云计算
2015-08-03 11:19:47
首先要感谢liu大神的翻译与试验，http://www.cnblogs.com/yuki-lau/
简介
本文详细描述使用单台服务器部署一个Swift开发版本的过程，单台服务器运行所有Swift服务，并模拟运行具有4个结点的Swift集群。其中，这里的“服务器”可以是物理机也可以是虚拟机。
已验证的部署环境：
本文档基于：
请确保安装的Swift版本和本文档中的版本相同。如有问题，请参考官网的更新文档。
1. 安装依赖包和核心代码
1. 以root身份登录。
|
# sudo su - root |
2. 安装必须的依赖包、库。
|
# apt-get install python-software-properties # add-apt-repository ppa:swift-core/release # apt-get update # apt-get install curl gcc git-core memcached python-coverage python-dev python-nose python-setuptools python-simplejson python-xattr sqlite3 xfsprogs python-eventlet python-greenlet python-pastedeploy python-netifaces python-pip # pip install mock |
3. 创建swift用户和组。在此，我们直接使用root作为用户名和组名。
接下来，需要选择使用一个分区作为存储（Using a partition for storage）或使用一个回环设备作为存储（Using a loopback device for storage），本文档使用回环设备作为存储。若希望使用一个单独的分区作为存储，请参阅这里。
1. 创建存储文件夹。
|
# mkdir /srv |
2. 调整seek的值，可以创建一个更大或更小的分区。
|
# dd if=/dev/zero of=/srv/swift-disk bs=1024 count=0 seek=1000000 # mkfs.xfs -i size=1024 /srv/swift-disk |
3. 编辑/etc/fstab并添加如下一行。
|
/srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0 |
4. 创建挂载文件夹，并挂载。
|
# mkdir /mnt/sdb1 # mount /mnt/sdb1 |
5. 创建4个子节点文件夹。
|
# mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4 |
6. 改变权限，root:root对应<your-user-name>:<your-group-name>。
|
# chown root:root /mnt/sdb1/* |
7. 为4个子节点创建同步链接。
|
# for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done |
8. 创建server等文件夹，并改编权限。注意，不要忘记/etc/swift /srv/[1-4]/最后一个“/”。
|
# mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift # chown -R root:root /etc/swift /srv/[1-4]/ /var/run/swift |
9. 编辑文件/etc/rc.local，在exit 0 之前添加如下4行。
|
# mkdir -p /var/cache/swift /var/cache/swift2 /var/cache/swift3 /var/cache/swift4 # chown root:root /var/cache/swift* # mkdir -p /var/run/swift # chown root:root /var/run/swift |
1. 创建文件/etc/rsyncd.conf，内容如下：
|
uid = root gid = root log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid address = 127.0.0.1
[account6012] max connections = 25 path = /srv/1/node/ read only = false lock file = /var/lock/account6012.lock
[account6022] max connections = 25 path = /srv/2/node/ read only = false lock file = /var/lock/account6022.lock
[account6032] max connections = 25 path = /srv/3/node/ read only = false lock file = /var/lock/account6032.lock
[account6042] max connections = 25 path = /srv/4/node/ read only = false lock file = /var/lock/account6042.lock
[container6011] max connections = 25 path = /srv/1/node/ read only = false lock file = /var/lock/container6011.lock
[container6021] max connections = 25 path = /srv/2/node/ read only = false lock file = /var/lock/container6021.lock
[container6031] max connections = 25 path = /srv/3/node/ read only = false lock file = /var/lock/container6031.lock
[container6041] max connections = 25 path = /srv/4/node/ read only = false lock file = /var/lock/container6041.lock
[object6010] max connections = 25 path = /srv/1/node/ read only = false lock file = /var/lock/object6010.lock
[object6020] max connections = 25 path = /srv/2/node/ read only = false lock file = /var/lock/object6020.lock
[object6030] max connections = 25 path = /srv/3/node/ read only = false lock file = /var/lock/object6030.lock
[object6040] max connections = 25 path = /srv/4/node/ read only = false lock file = /var/lock/object6040.lock |
2. 编辑文件/etc/default/rsync，设置参数RSYNC_ENABLE为true。
|
RSYNC_ENABLE=true |
3. 启动rsync服务
|
# service rsync restart |
1. 创建文件/etc/rsyslog.d/10-swift.conf，内容如下：
|
# Uncomment the following to have a log containing all logs together #local1,local2,local3,local4,local5.* /var/log/swift/all.log
# Uncomment the following to have hourly proxy logs for stats processing #$template HourlyProxyLog,"/var/log/swift/hourly/%$YEAR%%$MONTH%%$DAY%%$HOUR%" #local1.*;local1.!notice ?HourlyProxyLog
local1.*;local1.!notice /var/log/swift/proxy.log local1.notice /var/log/swift/proxy.error local1.* ~
local2.*;local2.!notice /var/log/swift/storage1.log local2.notice /var/log/swift/storage1.error local2.* ~
local3.*;local3.!notice /var/log/swift/storage2.log local3.notice /var/log/swift/storage2.error local3.* ~
local4.*;local4.!notice /var/log/swift/storage3.log local4.notice /var/log/swift/storage3.error local4.* ~
local5.*;local5.!notice /var/log/swift/storage4.log local5.notice /var/log/swift/storage4.error local5.* ~ |
2. 编辑文件/etc/rsyslog.conf，更改参数$PrivDropToGroup为adm。
|
$PrivDropToGroup adm |
3. 创建swift每小时的文件夹。
|
# mkdir -p /var/log/swift/hourly |
4. 更改swift的log文件夹权限。
|
# chown -R syslog.adm /var/log/swift # chmod -R g+w /var/log/swift |
5. 重启rsyslog服务。
|
# service rsyslog restart |
若不打算在全root权限下安装Swift，则需要切换回guest权限。此处，我们全部使用root安装。
1. 创建文件夹。
|
# mkdir ~/bin |
2. 从git上获取swift代码，下载到本地。
|
# git clone |
3. 安装Swift的开发版本。
|
# cd ~/swift # python setup.py develop # cd .. |
4. 从git上获取python-swiftclient代码，下载到本地。
|
# git clone |
5. 安装python-swiftclient的开发版本。
|
# cd ~/python-swiftclient # python setup.py develop # cd .. |
6. 编辑文件~/.bashrc，并在文件尾添加如下内容：
|
export SWIFT_TEST_CONFIG_FILE=/etc/swift/test.conf export PATH=${PATH}:~/bin |
7. 执行。
|
# . ~/.bashrc |
1. 创建文件/etc/swift/proxy-server.conf，文件内容如下：
|
[DEFAULT] bind_port = 8080 user = root log_facility = LOG_LOCAL1 eventlet_debug = true
[pipeline:main] pipeline = healthcheck cache tempauth proxy-logging proxy-server
[app:proxy-server] use = egg:swift#proxy allow_account_management = true account_autocreate = true
[filter:tempauth] use = egg:swift#tempauth user_admin_admin = admin .admin .reseller_admin user_test_tester = testing .admin user_test2_tester2 = testing2 .admin user_test_tester3 = testing3
[filter:healthcheck] use = egg:swift#healthcheck
[filter:cache] use = egg:swift#memcache
[filter:proxy-logging] use = egg:swift#proxy_logging |
2. 创建文件/etc/swift/swift.conf，文件内容如下：
|
[swift-hash] # random unique string that can never change (DO NOT LOSE) swift_hash_path_suffix = jtangfs |
3. 创建文件/etc/swift/account-server/1.conf，文件内容如下：
|
[DEFAULT] devices = /srv/1/node mount_check = false disable_fallocate = true bind_port = 6012 user = root log_facility = LOG_LOCAL2 recon_cache_path = /var/cache/swift eventlet_debug = true
[pipeline:main] pipeline = recon account-server
[app:account-server] use = egg:swift#account
[filter:recon] use = egg:swift#recon
[account-replicator] vm_test_mode = yes
[account-auditor]
[account-reaper] |
4. 创建文件/etc/swift/account-server/2.conf，文件内容如下：
|
[DEFAULT] devices = /srv/2/node mount_check = false disable_fallocate = true bind_port = 6022 user = root log_facility = LOG_LOCAL3 recon_cache_path = /var/cache/swift2 eventlet_debug = true
[pipeline:main] pipeline = recon account-server
[app:account-server] use = egg:swift#account
[filter:recon] use = egg:swift#recon
[account-replicator] vm_test_mode = yes
[account-auditor]
[account-reaper] |
5. 创建文件/etc/swift/account-server/3.conf，文件内容如下：
|
[DEFAULT] devices = /srv/3/node mount_check = false disable_fallocate = true bind_port = 6032 user = root log_facility = LOG_LOCAL4 recon_cache_path = /var/cache/swift3 eventlet_debug = true
[pipeline:main] pipeline = recon account-server
[app:account-server] use = egg:swift#account
[filter:recon] use = egg:swift#recon
[account-replicator] vm_test_mode = yes
[account-auditor]
[account-reaper] |
6. 创建文件/etc/swift/account-server/4.conf，文件内容如下：
|
[DEFAULT] devices = /srv/4/node mount_check = false disable_fallocate = true bind_port = 6042 user = root log_facility = LOG_LOCAL5 recon_cache_path = /var/cache/swift4 eventlet_debug = true
[pipeline:main] pipeline = recon account-server
[app:account-server] use = egg:swift#account
[filter:recon] use = egg:swift#recon
[account-replicator] vm_test_mode = yes
[account-auditor]
[account-reaper] |
7. 创建文件/etc/swift/container-server/1.conf，文件内容如下：
|
[DEFAULT] devices = /srv/1/node mount_check = false disable_fallocate = true bind_port = 6011 user = root log_facility = LOG_LOCAL2 recon_cache_path = /var/cache/swift eventlet_debug = true
[pipeline:main] pipeline = recon container-server
[app:container-server] use = egg:swift#container
[filter:recon] use = egg:swift#recon
[container-replicator] vm_test_mode = yes
[container-updater]
[container-auditor]
[container-sync] |
8. 创建文件/etc/swift/container-server/2.conf，文件内容如下：
|
[DEFAULT] devices = /srv/2/node mount_check = false disable_fallocate = true bind_port = 6021 user = root log_facility = LOG_LOCAL3 recon_cache_path = /var/cache/swift2 eventlet_debug = true
[pipeline:main] pipeline = recon container-server
[app:container-server] use = egg:swift#container
[filter:recon] use = egg:swift#recon
[container-replicator] vm_test_mode = yes
[container-updater]
[container-auditor]
[container-sync] |
9. 创建文件/etc/swift/container-server/3.conf，文件内容如下：
|
[DEFAULT] devices = /srv/3/node mount_check = false disable_fallocate = true bind_port = 6031 user = root log_facility = LOG_LOCAL4 recon_cache_path = /var/cache/swift3 eventlet_debug = true
[pipeline:main] pipeline = recon container-server
[app:container-server] use = egg:swift#container
[filter:recon] use = egg:swift#recon
[container-replicator] vm_test_mode = yes
[container-updater]
[container-auditor]
[container-sync] |
10. 创建文件/etc/swift/container-server/4.conf，文件内容如下：
|
[DEFAULT] devices = /srv/4/node mount_check = false disable_fallocate = true bind_port = 6041 user = root log_facility = LOG_LOCAL5 recon_cache_path = /var/cache/swift4 eventlet_debug = true
[pipeline:main] pipeline = recon container-server
[app:container-server] use = egg:swift#container
[filter:recon] use = egg:swift#recon
[container-replicator] vm_test_mode = yes
[container-updater]
[container-auditor]
[container-sync] |
11. 创建文件/etc/swift/object-server/1.conf，文件内容如下：
|
[DEFAULT] devices = /srv/1/node mount_check = false disable_fallocate = true bind_port = 6010 user = root log_facility = LOG_LOCAL2 recon_cache_path = /var/cache/swift eventlet_debug = true
[pipeline:main] pipeline = recon object-server
[app:object-server] use = egg:swift#object
[filter:recon] use = egg:swift#recon
[object-replicator] vm_test_mode = yes
[object-updater]
[object-auditor] |
12. 创建文件/etc/swift/object-server/2.conf，文件内容如下：
|
[DEFAULT] devices = /srv/2/node mount_check = false disable_fallocate = true bind_port = 6020 user = root log_facility = LOG_LOCAL3 recon_cache_path = /var/cache/swift2 eventlet_debug = true
[pipeline:main] pipeline = recon object-server
[app:object-server] use = egg:swift#object
[filter:recon] use = egg:swift#recon
[object-replicator] vm_test_mode = yes
[object-updater]
[object-auditor] |
13. 创建文件/etc/swift/object-server/3.conf，文件内容如下：
|
[DEFAULT] devices = /srv/3/node mount_check = false disable_fallocate = true bind_port = 6030 user = root log_facility = LOG_LOCAL4 recon_cache_path = /var/cache/swift3 eventlet_debug = true
[pipeline:main] pipeline = recon object-server
[app:object-server] use = egg:swift#object
[filter:recon] use = egg:swift#recon
[object-replicator] vm_test_mode = yes
[object-updater]
[object-auditor] |
14. 创建文件/etc/swift/object-server/4.conf，文件内容如下：
|
[DEFAULT] devices = /srv/4/node mount_check = false disable_fallocate = true bind_port = 6040 user = root log_facility = LOG_LOCAL5 recon_cache_path = /var/cache/swift4 eventlet_debug = true
[pipeline:main] pipeline = recon object-server
[app:object-server] use = egg:swift#object
[filter:recon] use = egg:swift#recon
[object-replicator] vm_test_mode = yes
[object-updater]
[object-auditor] |
1. 创建脚本~/bin/resetswift，内容如下。注意，如果使用的是单独分区存储需要将/srv/swift-disk替换为/dev/sdb1；如果没有创建rsyslog作为独立日志，则需要移除find /var/log/swift... 这一行。
|
#!/bin/bash
swift-init all stop find /var/log/swift -type f -exec rm -f {} \; sudo umount /mnt/sdb1 sudo mkfs.xfs -f -i size=1024 /srv/swift-disk sudo mount /mnt/sdb1 sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4 sudo chown root:root /mnt/sdb1/* mkdir -p /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 sudo rm -f /var/log/debug /var/log/messages /var/log/rsyncd.log /var/log/syslog find /var/cache/swift* -type f -name *.recon -exec rm -f {} \; sudo service rsyslog restart sudo service memcached restart |
2. 创建脚本~/bin/remakerings，内容如下：
|
#!/bin/bash
cd /etc/swift
rm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gz
swift-ring-builder object.builder create 18 3 1 swift-ring-builder object.builder add z1-127.0.0.1:6010/sdb1 1 swift-ring-builder object.builder add z2-127.0.0.1:6020/sdb2 1 swift-ring-builder object.builder add z3-127.0.0.1:6030/sdb3 1 swift-ring-builder object.builder add z4-127.0.0.1:6040/sdb4 1 swift-ring-builder object.builder rebalance swift-ring-builder container.builder create 18 3 1 swift-ring-builder container.builder add z1-127.0.0.1:6011/sdb1 1 swift-ring-builder container.builder add z2-127.0.0.1:6021/sdb2 1 swift-ring-builder container.builder add z3-127.0.0.1:6031/sdb3 1 swift-ring-builder container.builder add z4-127.0.0.1:6041/sdb4 1 swift-ring-builder container.builder rebalance swift-ring-builder account.builder create 18 3 1 swift-ring-builder account.builder add z1-127.0.0.1:6012/sdb1 1 swift-ring-builder account.builder add z2-127.0.0.1:6022/sdb2 1 swift-ring-builder account.builder add z3-127.0.0.1:6032/sdb3 1 swift-ring-builder account.builder add z4-127.0.0.1:6042/sdb4 1 swift-ring-builder account.builder rebalance |
3. 创建脚本~/bin/startmain，内容如下：
|
#!/bin/bash
swift-init main start |
4. 创建脚本~/bin/startrest，内容如下：
|
#!/bin/bash
swift-init rest start |
5. 更改脚本权限。
|
# chmod +x ~/bin/* |
6. 创建rings。
|
# remakerings |
7. 执行功能单元测试，出现“Unable to read test config /etc/swift/test.conf – file not found”，可不必理会，或手动复制过去（配置文件在swift/test/sample.conf）。
|
# cd ~/swift # ./.unittests |
8. 运行swift，出现“Unable to increase file descriptor limit. Running as non-root?”警告为正常现象，不必理会。
|
# startmain |
完成安装部署后，可以使用swift --help命令查看Swift帮助信息。
|
# swift --help Usage: swift <command> [options] [args]
Commands: stat [container] [object] Displays information for the account, container, or object depending on the args given (if any). list [options] [container] Lists the containers for the account or the objects for a container. -p or --prefix is an option that will only list items beginning with that prefix. -d or --delimiter is option (for container listings only) that will roll up items with the given delimiter (see Cloud Files general documentation for what this means). upload [options] container file_or_directory [file_or_directory] [...] Uploads to the given container the files and directories specified by the remaining args. -c or --changed is an option that will only upload files that have changed since the last upload. -S <size> or --segment-size <size> and --leave-segments are options as well (see --help for more). post [options] [container] [object] Updates meta information for the account, container, or object depending on the args given. If the container is not found, it will be created automatically; but this is not true for accounts and objects. Containers also allow the -r (or --read-acl) and -w (or --write-acl) options. The -m or --meta option is allowed on all and used to define the user meta data items to set in the form Name:Value. This option can be repeated. Example: post -m Color:Blue -m Size:Large download --all OR download container [options] [object] [object] ... Downloads everything in the account (with --all), or everything in a container, or a list of objects depending on the args given. For a single object download, you may use the -o [--output] <filename> option to redirect the output to a specific file or if "-" then just redirect to stdout. delete [options] --all OR delete container [options] [object] [object] ... Deletes everything in the account (with --all), or everything in a container, or a list of objects depending on the args given. Segments of manifest objects will be deleted as well, unless you specify the --leave-segments option.
Example: swift -A -U user -K key stat
Options: --version show program's version number and exit -h, --help show this help message and exit -s, --snet Use SERVICENET internal network -v, --verbose Print more info -q, --quiet Suppress status output -A AUTH, --auth=AUTH URL for obtaining an auth token -V AUTH_VERSION, --auth-version=AUTH_VERSION Specify a version for authentication. Defaults to 1.0. -U USER, --user=USER User name for obtaining an auth token. -K KEY, --key=KEY Key for obtaining an auth token. --os-username=<auth-user-name> Openstack username. Defaults to env[OS_USERNAME]. --os-password=<auth-password> Openstack password. Defaults to env[OS_PASSWORD]. --os-tenant-id=<auth-tenant-id> OpenStack tenant ID. Defaults to env[OS_TENANT_ID] --os-tenant-name=<auth-tenant-name> Openstack tenant name. Defaults to env[OS_TENANT_NAME]. --os-auth-url=<auth-url> Openstack auth URL. Defaults to env[OS_AUTH_URL]. --os-auth-token=<auth-token> Openstack token. Defaults to env[OS_AUTH_TOKEN] --os-storage-url=<storage-url> Openstack storage URL. Defaults to env[OS_STORAGE_URL] --os-region-name=<region-name> Openstack region name. Defaults to env[OS_REGION_NAME] --os-service-type=<service-type> Openstack Service type. Defaults to env[OS_SERVICE_TYPE] --os-endpoint-type=<endpoint-type> Openstack Endpoint type. Defaults to env[OS_ENDPOINT_TYPE] --insecure Allow swiftclient to access insecure keystone server. The keystone's certificate will not be verified. |
1. 获取一个 X-Storage-Url 和 X-Auth-Token。
|
# curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' |
可以得到以下返回信息：
|
* About to connect() to 127.0.0.1 port 8080 (#0) * Trying 127.0.0.1... connected * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0) > GET /auth/v1.0 HTTP/1.1 > User-Agent: curl/7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15 > Host: 127.0.0.1:8080 > Accept: */* > X-Storage-User: test:tester > X-Storage-Pass: testing
< HTTP/1.1 200 OK < X-Storage-Url: < X-Storage-Token: AUTH_tk6474e5ee0cb04832b9d2a168e1a164d8 < X-Auth-Token: AUTH_tk6474e5ee0cb04832b9d2a168e1a164d8 < Content-Length: 0 < Date: Mon, 04 Jul 2011 01:36:57 GMT * Connection #0 to host 127.0.0.1 left intact *Closing connection #0 |
2. 检查账户。注意，X-Auth-Token:与<token-from-x-auth-token-above>之间存在一个空格，一定不能丢。
|
# curl -v -H 'X-Auth-Token: <token-from-x-auth-token-above>' <url-from-x-storage-url-above> |
3. 检查Swift工作。
|
# swift -A -U test:tester -K testing stat |
正确情况下，应该输出以下信息：
|
Account: AUTH_test Containers: 0 Objects: 0 Bytes: 0 Accept-Ranges: bytes |
4. 创建container，创建一个名称为container_test的container（目录），注意使用当前的token。
|
# curl -X PUT -D - -H "X-Auth_Token:AUTH_tk6474e5ee0cb04832b9d2a168e1a164d8" |
正确情况下，应该得到以下信息：
|
HTTP/1.1 201 Created Content-Length: 0 Content-Type: text/html; charset=UTF-8 Date: Mon, 04 Jul 2011 01:39:38 GMT |
5. 查看test用户的container列表，发现只有一个目录：container_test。
|
# swift -A -U test:tester -K testing list |
|
container_test |
6. 上传Object（文件），上传hello文件到container_test目录中。
|
# swift -A -U test:tester -K testing upload container_test /root/hello |
|
root/hello |
7. 查看目录中的内容，查看container_test目录中的内容。
|
# swift -A -U test:tester -K testing list container_test |
|
root/hello |
8. 下载Object（文件）下载container_test下的object。
|
# swift -A -U test:tester -K testing download container_test |
将container_test目录内的所有内容下载到目前所在目录。注意：如果上传文件的时候，有嵌套目录，则上传到Swift上后，也是以嵌套目录的形式存在，下载后，会在当前目录创建一样的嵌套目录。
补充说明：
现在已经完成SAIO的部署了，下面介绍一下系统环境的改变：
1. ~/swift目录下存放Swift与python-swiftclient的源代码与执行脚本。
2. /usr/local/lib/python2.7/dist-packages、/usr/share/pyshared、/usr/bin、/usr/local/bin等目录下存放Swift与python-swiftclient安装后的程序文件。
3. Swift的回环设备挂载在/mnt/sdb1目录上。
4. 回环设备对应的文件为/srv/swift-disk。
5. 四个存储节点对应的实际存储空间（即共享上述回环设备）——/mnt/sdb1/1、/mnt/sdb1/2、/mnt/sdb1/3、/mnt/sdb1/4。
6. 四个存储节点指向实际存储空间的软链接——/srv/1、/srv/2、/srv/3、/srv/4。
7. 四个存储节点的对应的虚拟挂载点——/srv/1/node/sdb1、/srv/2/node/sdb2、/srv/2/node/sdb2、/srv/2/node/sdb2。
8. Swift运行时所需目录/var/run/swift，用于存放各个服务进程的pid文件等内容。
9. Swift临时缓存的目录（姑且这么理解）——/var/cache/swift、/var/cache/swift2、/var/cache/swift3、/var/cache/swift4。
10. /etc/swift目录中存放Swift自身的配置文件。
11. /var/log/swift目录下存放rsyslog独立日志工具输出的日志文件。
12. 其他被创建的文件：
13. 其他被修改的文件：
