OpenStack Kilo Multiple Region Deployment via DevStack
Environment
node1 eth0:192.168.1.161 eth1:192.168.141.161
node2 eth0:192.168.1.162 eth1:192.168.141.162
gateway:
stack@jonah2:~/devstack$ route
Kernel IP routing tableceilometer-alarm-not 22$(L) ceilometer-alarm-evaluator*
Destination Gateway Genmask Flags Metric Ref Use Iface
default 192.168.140.254 0.0.0.0 UG 0 0 0 eth1
192.168.1.0 * 255.255.255.0 U 0 0 0 br-ex
192.168.140.0 * 255.255.252.0 U 0 0 0 eth1Devstack Deploment
Based on our previous works, we can start to deploy our system
http://gogosatellite.blogspot.tw/2016/05/deploy-openstack-kilo-with-ceilometer.html
Let's start to deploy two region OpenStack.
In Region1 (node1)
[[local|localrc]]
DEST=/opt/stack
#change github to trystack
HOST_IP=192.168.1.161
GIT_BASE=${GIT_BASE:-http://git.trystack.cn}
#NOVNC_REPO=${NOVNC_REPO:-http://git.trystack.cn/kanaka/noVNC.git}
ADMIN_PASSWORD=password
MYSQL_PASSWORD=password
RABBIT_PASSWORD=password
SERVICE_PASSWORD=password
SERVICE_TOKEN=tokentoken
ENABLED_SERVICES=key,rabbit,mysql,horizon
ENABLED_SERVICES+=,n-api,n-crt,n-cpu,n-net,n-cond,n-sch,n-novnc,n-cauth
ENABLED_SERVICES+=,g-api,g-reg
# Enabling Neutron (network) Service
disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service neutron
enable_service q-vpn
# Ceilometer
enable_service ceilometer-acompute ceilometer-acentral ceilometer-anotification ceilometer-collector ceilometer-api
enable_service ceilometer-alarm-notifier ceilometer-alarm-evaluator
CEILOMETER_BACKEND=mongodb
### Enable Heat
##enable_service heat h-api h-api-cfn h-api-cw h-eng
## Neutron options
FIXED_RANGE=10.0.0.0/24
FIXED_NETWORK_SIZE=256
NETWORK_GATEWAY=10.0.0.1
PRIVATE_SUBNET_NAME=privateA
PUBLIC_SUBNET_NAME=public-subnet
FLOATING_RANGE=192.168.140.0/24
PUBLIC_NETWORK_GATEWAY=192.168.140.254
#Q_FLOATING_ALLOCATION_POOL="start=172.24.4.11,end=172.24.4.29"
PUBLIC_INTERFACE=eth0
Q_USE_PROVIDERNET_FOR_PUBLIC=True
OVS_PHYSICAL_BRIDGE=br-ex
PUBLIC_BRIDGE=br-ex
OVS_BRIDGE_MAPPINGS=public:br-ex
LIBVIRT_TYPE=qemu
IMAGE_URLS="http://download.cirros-cloud.net/0.3.3/cirros-0.3.3-x86_64-uec.tar.gz"
SCREEN_LOGDIR=/opt/stack/screen-logs
SYSLOG=True
LOGFILE=~/devstack/stack.sh.log
Q_USE_DEBUG_COMMAND=True
# RECLONE=No
RECLONE=yes
OFFLINE=FalseIn Region2 (node2)
[[local|localrc]]
DEST=/opt/stack
#change github to trystack
HOST_IP=192.168.1.162
GIT_BASE=${GIT_BASE:-http://git.trystack.cn}
#NOVNC_REPO=${NOVNC_REPO:-http://git.trystack.cn/kanaka/noVNC.git}
REGION_NAME=RegioneTwo
KEYSTONE_SERVICE_HOST=192.168.141.161
KEYSTONE_AUTH_HOST=192.168.141.161
ADMIN_PASSWORD=password
MYSQL_PASSWORD=password
RABBIT_PASSWORD=password
SERVICE_PASSWORD=password
SERVICE_TOKEN=tokentoken
ENABLED_SERVICES=key,rabbit,mysql,horizon
ENABLED_SERVICES+=,n-api,n-crt,n-cpu,n-net,n-cond,n-sch,n-novnc,n-cauth
ENABLED_SERVICES+=,g-api,g-reg
# Enabling Neutron (network) Service
disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service neutron
enable_service q-vpn
# Ceilometer
enable_service ceilometer-acompute ceilometer-acentral ceilometer-anotification ceilometer-collector ceilometer-api
enable_service ceilometer-alarm-notifier ceilometer-alarm-evaluator
CEILOMETER_BACKEND=mongodb
### Enable Heat
##enable_service heat h-api h-api-cfn h-api-cw h-eng
## Neutron options
FIXED_RANGE=10.0.0.0/24
FIXED_NETWORK_SIZE=256
NETWORK_GATEWAY=10.0.0.1
PRIVATE_SUBNET_NAME=privateA
PUBLIC_SUBNET_NAME=public-subnet
FLOATING_RANGE=192.168.140.0/24
PUBLIC_NETWORK_GATEWAY=192.168.140.254
#Q_FLOATING_ALLOCATION_POOL="start=172.24.4.11,end=172.24.4.29"
PUBLIC_INTERFACE=eth0
Q_USE_PROVIDERNET_FOR_PUBLIC=True
OVS_PHYSICAL_BRIDGE=br-ex
PUBLIC_BRIDGE=br-ex
OVS_BRIDGE_MAPPINGS=public:br-ex
LIBVIRT_TYPE=qemu
IMAGE_URLS="http://download.cirros-cloud.net/0.3.3/cirros-0.3.3-x86_64-uec.tar.gz"
SCREEN_LOGDIR=/opt/stack/screen-logs
SYSLOG=True
LOGFILE=~/devstack/stack.sh.log
Q_USE_DEBUG_COMMAND=True
# RECLONE=No
RECLONE=yes
OFFLINE=FalseWe mentions that we have set addtional parameters in Region2's local.conf shown as followed.
REGION_NAME
KEYSTONE_SERVICE_HOST
KEYSTONE_AUTH_HOST Result
A simple way to realize the result we deployed is to connect to Horizon.
After login to the Horizon, we can see two Regions as above figure. And we can switch to different region to see different resources.
How to Get Quota Information
edit openrc_admin
export OS_SERVICE_TOKEN=tokentoken
export OS_SERVICE_ENDPOINT=http://localhost:35357/v2.0source openrc_adminkeystone --debug tenant-list
.
.
.
+----------------------------------+--------------------+---------+
| id | name | enabled |
+----------------------------------+--------------------+---------+
| 03daba5a7f73432d8edecb881c1e381a | admin | True |
| 11d655a1358546ae86ad6d7ba61de6f4 | demo | True |
| 5d609499001c43ecb4137aedf3416995 | invisible_to_admin | True |
| 199467ffaa4a43408799ea6afb186862 | service | True |
+----------------------------------+--------------------+---------+
To get the tenant_id you want, demo, and then
nova-manage --debug project quota 11d655a1358546ae86ad6d7ba61de6f4
.
.
.
Quota Limit In Use Reserved
2016-05-13 08:45:24.542 DEBUG nova.quota [req-5cf3fcff-8462-43bf-9361-333ad13907f4 None None] Getting all quota usages for project: 11d655a1358546ae86ad6d7ba61de6f4 get_project_quotas /opt/stack/nova/nova/quota.py:290
metadata_items 128 0 0
injected_file_content_bytes 10240 0 0
server_group_members 10 0 0
server_groups 10 0 0
ram 51200 0 0
floating_ips 10 0 0
security_group_rules 20 0 0
instances 10 0 0
key_pairs 100 0 0
injected_files 5 0 0
cores 20 0 0
fixed_ips unlimited 0 0
injected_file_path_bytes 255 0 0
security_groups 10 0 0
Since the quota controled from individual modules, such as nova, neutron, and not in Keystone. One can update quota by check this http://docs.openstack.org/openstack-ops/content/projects_users.html.
The problem is Glance not support tenant quota in Kilo, but it can setup global quota of image storage.
Via the nova-manager, we have to mention that the quota must be set to regions individually. There are no way to set global quota for a tenant.
Compare Database
Since our keystone is set to Region1 while deploying Region2. So there are no keystone information in Region2 database but in Region1.
In Region1
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| glance |
| keystone |
| mysql |
| neutron |
| nova |
| performance_schema |
+--------------------+
7 rows in set (0.00 sec)
In Region2
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| glance |
| mysql |
| neutron |
| nova |
| performance_schema |
+--------------------+
6 rows in set (0.00 sec)We can see the keystone is not existed in Region2's Database.
What Changed During Deployment
To address what changed in openstack configuration. We can grip the IP address of Region1 to make sure changes shown as followsed.
root@devr2:/etc# grep -R "192.168.141.161" *
ceilometer/ceilometer.conf:os_auth_url = http://192.168.141.161:5000/v2.0
ceilometer/ceilometer.conf:auth_uri = http://192.168.141.161:5000
ceilometer/ceilometer.conf:auth_url = http://192.168.141.161:35357
glance/glance-cache.conf:auth_url = http://192.168.141.161:35357/v2.0
glance/glance-api.conf:auth_uri = http://192.168.141.161:5000
glance/glance-api.conf:auth_url = http://192.168.141.161:35357
glance/glance-registry.conf:auth_uri = http://192.168.141.161:5000
glance/glance-registry.conf:auth_url = http://192.168.141.161:35357
neutron/neutron.conf:auth_url = http://192.168.141.161:35357
neutron/neutron.conf:auth_uri = http://192.168.141.161:5000
neutron/neutron.conf:auth_url = http://192.168.141.161:35357
neutron/metadata_agent.ini:auth_uri = http://192.168.141.161:5000
neutron/metadata_agent.ini:auth_url = http://192.168.141.161:35357
nova/nova-dhcpbridge.conf:keystone_ec2_url = http://192.168.141.161:5000/v2.0/ec2tokens
nova/nova-dhcpbridge.conf:auth_uri = http://192.168.141.161:5000
nova/nova-dhcpbridge.conf:auth_url = http://192.168.141.161:35357
nova/nova.conf:keystone_ec2_url = http://192.168.141.161:5000/v2.0/ec2tokens
nova/nova.conf:auth_uri = http://192.168.141.161:5000
nova/nova.conf:auth_url = http://192.168.141.161:35357
nova/nova.conf:admin_auth_url = http://192.168.141.161:35357/v2.0That means if we want switch back to Region2's keystone, we need to modify the above configuration and restart the daemon shown as above.
確定不同services都有不同的endpoint分別對應不同region。不同region的位置也都是對的。
keystone --debug endpoint-list
.
.
.
+----------------------------------+------------+------------------------------------------------+------------------------------------------------+------------------------------------------------+----------------------------------+
| id | region | publicurl | internalurl | adminurl | service_id |
+----------------------------------+------------+------------------------------------------------+------------------------------------------------+------------------------------------------------+----------------------------------+
| 030a7da536d24ecb9f59953ad2905833 | RegioneTwo | http://192.168.141.162:8773/ | http://192.168.141.162:8773/ | http://192.168.141.162:8773/ | fbb89d73112248a98fd7e67af6dbfb8e |
| 0425acb1931c4409bda1bf9c6c935baf | RegionOne | http://192.168.141.161:8773/ | http://192.168.141.161:8773/ | http://192.168.141.161:8773/ | fbb89d73112248a98fd7e67af6dbfb8e |
| 0ad4766387b545a7b234627982317fa6 | RegionOne | http://192.168.141.161:8777/ | http://192.168.141.161:8777/ | http://192.168.141.161:8777/ | 275a2c49545d426a9166185da1c31a16 |
| 0f3c9d33607744f1a2e1a7811162f2ba | RegionOne | http://192.168.141.161:9696/ | http://192.168.141.161:9696/ | http://192.168.141.161:9696/ | 7cd1067bfb3a4485b0f90c76ae4bd419 |
| 206b8531d7fd49e493382041741d4088 | RegioneTwo | http://192.168.141.161:5000/v2.0 | http://192.168.141.161:5000/v2.0 | http://192.168.141.161:35357/v2.0 | 42f34398300648e7b702bdbb40ddb193 |
| 394ce1871229498eb8a68c3750fdafd1 | RegioneTwo | http://192.168.141.162:8774/v2/$(tenant_id)s | http://192.168.141.162:8774/v2/$(tenant_id)s | http://192.168.141.162:8774/v2/$(tenant_id)s | 87fafd630bad47df9c34f4bb51cd5ca1 |
| 397bacb64bb047128a6b8403ea609aa5 | RegioneTwo | http://192.168.141.162:9292 | http://192.168.141.162:9292 | http://192.168.141.162:9292 | 46fa35450c80421eb59faf4f88bfc039 |
| 4ad1c79eb2e844c99020374d54ed561e | RegioneTwo | http://192.168.141.162:8777/ | http://192.168.141.162:8777/ | http://192.168.141.162:8777/ | 275a2c49545d426a9166185da1c31a16 |
| 4d88dc1c94a94555aa516181105f0cd7 | RegioneTwo | http://192.168.141.162:9696/ | http://192.168.141.162:9696/ | http://192.168.141.162:9696/ | 7cd1067bfb3a4485b0f90c76ae4bd419 |
| 9c6a6fe1f6924249a6da8e90ea23bd8a | RegionOne | http://192.168.141.161:9292 | http://192.168.141.161:9292 | http://192.168.141.161:9292 | 46fa35450c80421eb59faf4f88bfc039 |
| c78f7d8bfbb044e68297e0076011b64c | RegionOne | http://192.168.141.161:8774/v2/$(tenant_id)s | http://192.168.141.161:8774/v2/$(tenant_id)s | http://192.168.141.161:8774/v2/$(tenant_id)s | 87fafd630bad47df9c34f4bb51cd5ca1 |
| e00afb4d18484e2684124f6b98b88baa | RegionOne | http://192.168.141.161:5000/v2.0 | http://192.168.141.161:5000/v2.0 | http://192.168.141.161:35357/v2.0 | 42f34398300648e7b702bdbb40ddb193 |
| e4e0e9050b644de19b874b34e8b6fb5b | RegionOne | http://192.168.141.161:8774/v2.1/$(tenant_id)s | http://192.168.141.161:8774/v2.1/$(tenant_id)s | http://192.168.141.161:8774/v2.1/$(tenant_id)s | 8a616bcaee344a9382ecade08c2f4dbe |
| f479712d18f64883928d220a0c756757 | RegioneTwo | http://192.168.141.162:8774/v2.1/$(tenant_id)s | http://192.168.141.162:8774/v2.1/$(tenant_id)s | http://192.168.141.162:8774/v2.1/$(tenant_id)s | 8a616bcaee344a9382ecade08c2f4dbe |
+----------------------------------+------------+------------------------------------------------+------------------------------------------------+------------------------------------------------+----------------------------------+
No comments:
Post a Comment