This documents setting up a virtual machine for doing Swift development. Thevirtual machine will emulate running a four node Swift cluster.
Additional information about setting up a Swift development snapshot on other distributions isavailable on the wiki at http://wiki.openstack.org/SAIOInstructions.
creating the VM, and follow these instructions.
fdisk /dev/sdb (set up a single partition)
mkfs.xfs -i size=1024 /dev/sdb1
/dev/sdb1 /mnt/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0
mkdir /mnt/sdb1
mount /mnt/sdb1
mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
chown <your-user-name>:<your-group-name> /mnt/sdb1/*
mkdir /srv
for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done
mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift
chown -R <your-user-name>:<your-group-name> /etc/swift /srv/[1-4]/ /var/run/swift – Make sure to include the trailing slash after /srv/[1-4]/
Add to /etc/rc.local (before the exit 0):
mkdir /var/run/swiftchown <your-user-name>:<your-group-name> /var/run/swift
Next, skip to Setting up rsync.
If you want to use a loopback device instead of another partition, follow these instructions.
mkdir /srv
- dd if=/dev/zero of=/srv/swift-disk bs=1024 count=0 seek=1000000
(modify seek to make a larger or smaller partition)
mkfs.xfs -i size=1024 /srv/swift-disk
- Edit /etc/fstab and add
/srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0
mkdir /mnt/sdb1
mount /mnt/sdb1
mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
chown <your-user-name>:<your-group-name> /mnt/sdb1/*
for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done
mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift
chown -R <your-user-name>:<your-group-name> /etc/swift /srv/[1-4]/ /var/run/swift – Make sure to include the trailing slash after /srv/[1-4]/
Add to /etc/rc.local (before the exit 0):
mkdir /var/run/swiftchown <your-user-name>:<your-group-name> /var/run/swift
Create /etc/rsyncd.conf:
uid = <Your user name>gid = <Your group name>log file = /var/log/rsyncd.logpid file = /var/run/rsyncd.pidaddress = 127.0.0.1[account6012]max connections = 25path = /srv/1/node/read only = falselock file = /var/lock/account6012.lock[account6022]max connections = 25path = /srv/2/node/read only = falselock file = /var/lock/account6022.lock[account6032]max connections = 25path = /srv/3/node/read only = falselock file = /var/lock/account6032.lock[account6042]max connections = 25path = /srv/4/node/read only = falselock file = /var/lock/account6042.lock[container6011]max connections = 25path = /srv/1/node/read only = falselock file = /var/lock/container6011.lock[container6021]max connections = 25path = /srv/2/node/read only = falselock file = /var/lock/container6021.lock[container6031]max connections = 25path = /srv/3/node/read only = falselock file = /var/lock/container6031.lock[container6041]max connections = 25path = /srv/4/node/read only = falselock file = /var/lock/container6041.lock[object6010]max connections = 25path = /srv/1/node/read only = falselock file = /var/lock/object6010.lock[object6020]max connections = 25path = /srv/2/node/read only = falselock file = /var/lock/object6020.lock[object6030]max connections = 25path = /srv/3/node/read only = falselock file = /var/lock/object6030.lock[object6040]max connections = 25path = /srv/4/node/read only = falselock file = /var/lock/object6040.lockEdit the following line in /etc/default/rsync:
RSYNC_ENABLE=trueservice rsync restart
Create /etc/rsyslog.d/10-swift.conf:
# Uncomment the following to have a log containing all logs together#local1,local2,local3,local4,local5.* /var/log/swift/all.log# Uncomment the following to have hourly proxy logs for stats processing#$template HourlyProxyLog,"/var/log/swift/hourly/%$YEAR%%$MONTH%%$DAY%%$HOUR%"#local1.*;local1.!notice ?HourlyProxyLoglocal1.*;local1.!notice /var/log/swift/proxy.loglocal1.notice /var/log/swift/proxy.errorlocal1.* ~local2.*;local2.!notice /var/log/swift/storage1.loglocal2.notice /var/log/swift/storage1.errorlocal2.* ~local3.*;local3.!notice /var/log/swift/storage2.loglocal3.notice /var/log/swift/storage2.errorlocal3.* ~local4.*;local4.!notice /var/log/swift/storage3.loglocal4.notice /var/log/swift/storage3.errorlocal4.* ~local5.*;local5.!notice /var/log/swift/storage4.loglocal5.notice /var/log/swift/storage4.errorlocal5.* ~Edit /etc/rsyslog.conf and make the following change:
$PrivDropToGroup admmkdir -p /var/log/swift/hourly
chown -R syslog.adm /var/log/swift
service rsyslog restart
Sample configuration files are provided with all defaults in line-by-line comments.
Do these commands as you on guest.
mkdir ~/bin
Check out the swift repo with git clone https://github.com/openstack/swift.git
Build a development installation of swift, for example:cd ~/swift; sudo python setup.py develop
Edit ~/.bashrc and add to the end:
export SWIFT_TEST_CONFIG_FILE=/etc/swift/func_test.confexport PATH=${PATH}:~/bin. ~/.bashrc
Sample configuration files are provided with all defaults in line-by-line comments.
Create /etc/swift/proxy-server.conf:
[DEFAULT]bind_port = 8080user = <your-user-name>log_facility = LOG_LOCAL1[pipeline:main]pipeline = healthcheck cache tempauth proxy-server[app:proxy-server]use = egg:swift#proxyallow_account_management = trueaccount_autocreate = true[filter:tempauth]use = egg:swift#tempauthuser_admin_admin = admin .admin .reseller_adminuser_test_tester = testing .adminuser_test2_tester2 = testing2 .adminuser_test_tester3 = testing3[filter:healthcheck]use = egg:swift#healthcheck[filter:cache]use = egg:swift#memcacheCreate /etc/swift/swift.conf:
[swift-hash]# random unique string that can never change (DO NOT LOSE)swift_hash_path_suffix = changemeCreate /etc/swift/account-server/1.conf:
[DEFAULT]devices = /srv/1/nodemount_check = falsebind_port = 6012user = <your-user-name>log_facility = LOG_LOCAL2[pipeline:main]pipeline = account-server[app:account-server]use = egg:swift#account[account-replicator]vm_test_mode = yes[account-auditor][account-reaper]Create /etc/swift/account-server/2.conf:
[DEFAULT]devices = /srv/2/nodemount_check = falsebind_port = 6022user = <your-user-name>log_facility = LOG_LOCAL3[pipeline:main]pipeline = account-server[app:account-server]use = egg:swift#account[account-replicator]vm_test_mode = yes[account-auditor][account-reaper]Create /etc/swift/account-server/3.conf:
[DEFAULT]devices = /srv/3/nodemount_check = falsebind_port = 6032user = <your-user-name>log_facility = LOG_LOCAL4[pipeline:main]pipeline = account-server[app:account-server]use = egg:swift#account[account-replicator]vm_test_mode = yes[account-auditor][account-reaper]Create /etc/swift/account-server/4.conf:
[DEFAULT]devices = /srv/4/nodemount_check = falsebind_port = 6042user = <your-user-name>log_facility = LOG_LOCAL5[pipeline:main]pipeline = account-server[app:account-server]use = egg:swift#account[account-replicator]vm_test_mode = yes[account-auditor][account-reaper]Create /etc/swift/container-server/1.conf:
[DEFAULT]devices = /srv/1/nodemount_check = falsebind_port = 6011user = <your-user-name>log_facility = LOG_LOCAL2[pipeline:main]pipeline = container-server[app:container-server]use = egg:swift#container[container-replicator]vm_test_mode = yes[container-updater][container-auditor][container-sync]Create /etc/swift/container-server/2.conf:
[DEFAULT]devices = /srv/2/nodemount_check = falsebind_port = 6021user = <your-user-name>log_facility = LOG_LOCAL3[pipeline:main]pipeline = container-server[app:container-server]use = egg:swift#container[container-replicator]vm_test_mode = yes[container-updater][container-auditor][container-sync]Create /etc/swift/container-server/3.conf:
[DEFAULT]devices = /srv/3/nodemount_check = falsebind_port = 6031user = <your-user-name>log_facility = LOG_LOCAL4[pipeline:main]pipeline = container-server[app:container-server]use = egg:swift#container[container-replicator]vm_test_mode = yes[container-updater][container-auditor][container-sync]Create /etc/swift/container-server/4.conf:
[DEFAULT]devices = /srv/4/nodemount_check = falsebind_port = 6041user = <your-user-name>log_facility = LOG_LOCAL5[pipeline:main]pipeline = container-server[app:container-server]use = egg:swift#container[container-replicator]vm_test_mode = yes[container-updater][container-auditor][container-sync]Create /etc/swift/object-server/1.conf:
[DEFAULT]devices = /srv/1/nodemount_check = falsebind_port = 6010user = <your-user-name>log_facility = LOG_LOCAL2[pipeline:main]pipeline = object-server[app:object-server]use = egg:swift#object[object-replicator]vm_test_mode = yes[object-updater][object-auditor]Create /etc/swift/object-server/2.conf:
[DEFAULT]devices = /srv/2/nodemount_check = falsebind_port = 6020user = <your-user-name>log_facility = LOG_LOCAL3[pipeline:main]pipeline = object-server[app:object-server]use = egg:swift#object[object-replicator]vm_test_mode = yes[object-updater][object-auditor]Create /etc/swift/object-server/3.conf:
[DEFAULT]devices = /srv/3/nodemount_check = falsebind_port = 6030user = <your-user-name>log_facility = LOG_LOCAL4[pipeline:main]pipeline = object-server[app:object-server]use = egg:swift#object[object-replicator]vm_test_mode = yes[object-updater][object-auditor]Create /etc/swift/object-server/4.conf:
[DEFAULT]devices = /srv/4/nodemount_check = falsebind_port = 6040user = <your-user-name>log_facility = LOG_LOCAL5[pipeline:main]pipeline = object-server[app:object-server]use = egg:swift#object[object-replicator]vm_test_mode = yes[object-updater][object-auditor]
Create ~/bin/resetswift.
If you are using a loopback device substitute /dev/sdb1 with /srv/swift-disk.
If you did not set up rsyslog for individual logging, remove the find /var/log/swift... line:
#!/bin/bashswift-init all stopfind /var/log/swift -type f -exec rm -f {} \;sudo umount /mnt/sdb1sudo mkfs.xfs -f -i size=1024 /dev/sdb1sudo mount /mnt/sdb1sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4sudo chown <your-user-name>:<your-group-name> /mnt/sdb1/*mkdir -p /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4sudo rm -f /var/log/debug /var/log/messages /var/log/rsyncd.log /var/log/syslogsudo service rsyslog restartsudo service memcached restartCreate ~/bin/remakerings:
#!/bin/bashcd /etc/swiftrm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gzswift-ring-builder object.builder create 18 3 1swift-ring-builder object.builder add z1-127.0.0.1:6010/sdb1 1swift-ring-builder object.builder add z2-127.0.0.1:6020/sdb2 1swift-ring-builder object.builder add z3-127.0.0.1:6030/sdb3 1swift-ring-builder object.builder add z4-127.0.0.1:6040/sdb4 1swift-ring-builder object.builder rebalanceswift-ring-builder container.builder create 18 3 1swift-ring-builder container.builder add z1-127.0.0.1:6011/sdb1 1swift-ring-builder container.builder add z2-127.0.0.1:6021/sdb2 1swift-ring-builder container.builder add z3-127.0.0.1:6031/sdb3 1swift-ring-builder container.builder add z4-127.0.0.1:6041/sdb4 1swift-ring-builder container.builder rebalanceswift-ring-builder account.builder create 18 3 1swift-ring-builder account.builder add z1-127.0.0.1:6012/sdb1 1swift-ring-builder account.builder add z2-127.0.0.1:6022/sdb2 1swift-ring-builder account.builder add z3-127.0.0.1:6032/sdb3 1swift-ring-builder account.builder add z4-127.0.0.1:6042/sdb4 1swift-ring-builder account.builder rebalanceCreate ~/bin/startmain:
#!/bin/bashswift-init main startCreate ~/bin/startrest:
#!/bin/bashswift-init rest startchmod +x ~/bin/*
remakerings
cd ~/swift; ./.unittests
startmain (The Unable to increase file descriptor limit. Running as non-root? warnings are expected and ok.)
Get an X-Storage-Url and X-Auth-Token: curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0
Check that you can GET account: curl -v -H 'X-Auth-Token: <token-from-x-auth-token-above>' <url-from-x-storage-url-above>
Check that swift works: swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat
cp ~/swift/test/functional/sample.conf /etc/swift/func_test.conf
cd ~/swift; ./.functests (Note: functional tests will first deleteeverything in the configured accounts.)
cd ~/swift; ./.probetests (Note: probe tests will reset yourenvironment as they call resetswift for each test.)
If you plan to work on documentation (and who doesn’t?!):
If all doesn’t go as planned, and tests fail, or you can’t auth, or something doesn’t work, here are some good starting places to look for issues:
联系客服