Adding Compute Node by using Devstack Mitaka
Install Contol Node
Intital
Becase my Laptop's Ram is too small so I have to kill some process to keep the VM running smoothly.
pkill -9 ceilometer
pkill -9 nova-compute
service libvirt-bin stopAdding Compute Node
Installation from Local.conf
The local.conf is here.
[[local|localrc]]
HOST_IP=172.16.235.170
SERVICE_HOST=172.16.235.128
MYSQL_HOST=172.16.235.128
RABBIT_HOST=172.16.235.128
GLANCE_HOSTPORT=172.16.235.128:9292
ADMIN_PASSWORD=password
MYSQL_PASSWORD=password
RABBIT_PASSWORD=password
SERVICE_PASSWORD=password
## Neutron options
PUBLIC_INTERFACE=eth0
ENABLED_SERVICES=n-cpu,rabbit,q-agt
LIBVIRT_TYPE=qemu
SCREEN_LOGDIR=/opt/stack/screen-logs
SYSLOG=True
LOGFILE=~/devstack/stack.sh.log
RECLONE=No
#RECLONE=yes
OFFLINE=TrueAfter a while, you will see the result.
========================
DevStack Components Timed
========================
run_process - 6 secs
This is your host IP address: 172.16.235.170
This is your host IPv6 address: ::1Reuslt
To understand the new compute node, mitakacompute, join the system or not.
stack@mitakagnocchi:~/devstack$ nova hypervisor-list
+----+---------------------+-------+---------+
| ID | Hypervisor hostname | State | Status |
+----+---------------------+-------+---------+
| 1 | mitakagnocchi | down | enabled |
| 2 | mitakacompute | up | enabled |
+----+---------------------+-------+---------+Now you can play OpenStack with multiple node via simple command to launch an instance.
http://gogosatellite.blogspot.tw/2016/06/minimum-steps-to-vm.html
Deep Dive
Check Networking
We check the compute node to make sure the networking is vxlan supported for the local.conf.
Bridge br-tun
fail_mode: secure
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port br-tun
Interface br-tun
type: internal
Port "vxlan-ac10eb80"
Interface "vxlan-ac10eb80"
type: vxlan
options: {df_default="true", in_key=flow, local_ip="172.16.235.170", out_key=flow, remote_ip="172.16.235.128"}In Controll Node, we can see the vxLan protocol enabled.
root@mitakagnocchi:~# ovs-vsctl show
4d075257-2e3b-4287-8bbf-d06f33910752
Bridge br-ex
.
.
Port "vxlan-ac10ebaa"
Interface "vxlan-ac10ebaa"
type: vxlan
options: {df_default="true", in_key=flow, local_ip="172.16.235.128", out_key=flow, remote_ip="172.16.235.170"}In the VM testing, we ran out of memory so I stop the compute role, libvirt, ceilometer in control node. The VM is running on the new node.
stack@mitakagnocchi:~/devstack$ nova list
+--------------------------------------+------+--------+------------+-------------+--------------------------------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------+--------+------------+-------------+--------------------------------------------------------+
| 817d75be-764f-4ccb-bd83-438b4397c091 | 123 | ACTIVE | - | Running | private=10.0.0.7, fd03:64f3:a8ad:0:f816:3eff:feed:30f6 |
+--------------------------------------+------+--------+------------+-------------+--------------------------------------------------------+Adding floating IP
You can read my blog to know how to use nova command line to operate a VM.
http://gogosatellite.blogspot.tw/2016/06/minimum-steps-to-vm.html
After launch a VM, we add a floating ip 172.16.235.6 to VM.
root@mitakagnocchi:~# ping 172.16.235.6
PING 172.16.235.6 (172.16.235.6) 56(84) bytes of data.
64 bytes from 172.16.235.6: icmp_seq=1 ttl=64 time=5.17 ms
64 bytes from 172.16.235.6: icmp_seq=10 ttl=64 time=1000 ms
64 bytes from 172.16.235.6: icmp_seq=11 ttl=64 time=1.31 msTo check the IPtables in q-route namespace
ip netns exec qrouter-6801884e-0b91-4b4d-bb93-e1fe9ada91bd bash
root@mitakagnocchi:~# iptables -t nat -S
-P PREROUTING ACCEPT
.
.
-A neutron-l3-agent-OUTPUT -d 172.16.235.6/32 -j DNAT --to-destination 10.0.0.5
-A neutron-l3-agent-POSTROUTING ! -i qg-49a0878f-4d ! -o qg-49a0878f-4d -m conntrack ! --ctstate DNAT -j ACCEPT
-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697
-A neutron-l3-agent-PREROUTING -d 172.16.235.6/32 -j DNAT --to-destination 10.0.0.5
-A neutron-l3-agent-float-snat -s 10.0.0.5/32 -j SNAT --to-source 172.16.235.6
-A neutron-l3-agent-snat -j neutron-l3-agent-float-snat
-A neutron-l3-agent-snat -o qg-49a0878f-4d -j SNAT --to-source 172.16.235.3
.
.
How it works between Compute Node and Network Node(Control Node)
stack@mitakagnocchi:~/devstack$ sudo ovs-vsctl show
Bridge br-tun
.
Port "vxlan-ac10ebaa"
Interface "vxlan-ac10ebaa"
type: vxlan
options: {df_default="true", in_key=flow, local_ip="172.16.235.128", out_key=flow, remote_ip="172.16.235.170"}To check the Network Node, we found the vxlan protocol running in Network node, hence we can access floating ip(Network node), then connect to VM(Compute Node) via vxlan(br-tun).
Vlan Tag
root@mitakacompute:~# ovs-vsctl show
dc2ae015-4882-4a0e-8ecb-7d4739f088cc
Bridge br-int
fail_mode: secure
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port "qvo5956ce9e-3a"
tag: 1
Interface "qvo5956ce9e-3a"
Port "qvoff132ae7-4b"
tag: 2
Interface "qvoff132ae7-4b"
Port br-int
Interface br-int
type: internalAt Computer node. qvo5956ce9e-3a launched by demo user in admin project, and qvoff132ae7-4b by tester user in tester project.
At Network node(control node). To check DHCP of tester tenant.
stack@mitakagnocchi:~$ sudo ovs-vsctl show
Bridge br-int
fail_mode: secure
.
.
Port "tapa657602c-b7"
tag: 3
Interface "tapa657602c-b7"
type: internalwhere tapa657602c-b7 is belong to tester subnet with vlan tag 3 that is different with in Compute node.
That means, Same Tenant are with different vlan tags in different nodes. It's quite reasonable.
Flow Table In compute node
ovs-ofctl dump-flows br-tun
.
.
cookie=0x9a2f3857b1f3b3d4, duration=12614.642s, table=4, n_packets=214, n_bytes=24699, idle_age=33, priority=1,tun_id=0x41b actions=mod_vlan_vid:1,resubmit(,10)
cookie=0x9a2f3857b1f3b3d4, duration=2468.361s, table=4, n_packets=15, n_bytes=1838, idle_age=2397, priority=1,tun_id=0x87 actions=mod_vlan_vid:2,resubmit(,10)Flow Table In Control node
ovs-ofctl dump-flows br-tun
.
.
cookie=0x896af9c2b96ed0d4, duration=3054.077s, table=22, n_packets=14, n_bytes=1132, idle_age=2550, dl_vlan=3 actions=strip_vlan,set_tunnel:0x87,output:2
cookie=0x896af9c2b96ed0d4, duration=14680.049s, table=22, n_packets=341, n_bytes=32522, idle_age=37, dl_vlan=1 actions=strip_vlan,set_tunnel:0x41b,output:2How to rejoin in Compute Node
Same as All in one.
Shutdown/Reboot -> ./unstack -> ./stack.shAnd Adding the following lines in local.conf.
RECLONE=No
#RECLONE=yes
OFFLINE=True