Sunday, June 12, 2016

Using Allowd-Address-Pairs in OpenStack

addresspair

Arbitrry IPs setting in VM by using allowed-address-pairs

這是一個很特別的Case,客戶需要的是提供他們的虛機可以在不知道IP的狀況下啟動,並可連通。
我們知道,在OpenStack中IP跟MAC是有Iptable綁定的,任意更改IP都會導致VM對外連網不通。
透過allowed-address-pairs的設定,IP任意變更都可以對外連網了。

在Devstack中執行一下指令,即可開始使用openstack client的工具。

source openrc admin admin
or 
source openrc admin demo

We now use demo user.

To get MacAddress By vm id

Method 1

stack@kiloceilo:~/devstack$ nova list
+--------------------------------------+-------+--------+------------+-------------+------------------+
| ID                                   | Name  | Status | Task State | Power State | Networks         |
+--------------------------------------+-------+--------+------------+-------------+------------------+
| 76a7cc1c-2395-46dc-9082-860ba086e6b8 | test5 | ACTIVE | -          | Running     | private=10.0.0.4 |
+--------------------------------------+-------+--------+------------+-------------+------------------+
stack@kiloceilo:~/devstack$ virsh dumpxml 4|grep mac
    <partition>/machine</partition>
    <type arch='x86_64' machine='pc-i440fx-trusty'>hvm</type>
      <mac address='fa:16:3e:28:d8:c2'/>

Method 2

to get mac address, it's hard to find a cli to get mac address by a simple way. So using a --debug mode to get http response is a better way.

stack@kiloceilo:~/devstack$ nova --debug show 76a7cc1c-2395-46dc-9082-860ba086e6b8
.
.
.

RESP BODY: {"server": {"status": "ACTIVE", "updated": "2016-06-08T07:27:39Z", "hostId": "66b3d5bcf0238e5c02f41ceae66e09a90624594b499b5ef2a52470de", "OS-EXT-SRV-ATTR:host": "kiloceilo", "addresses": {"private": [{"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:28:d8:c2"

stack@kiloceilo:~/devstack$ neutron port-list
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                                                           |
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+
| 0155e3e5-1139-46c1-97a6-54baabf9615e |      | fa:16:3e:be:57:b1 | {"subnet_id": "4f10c5ce-7cf3-481f-af7c-e7ff80a700dc", "ip_address": "172.16.235.3"} |
| 01ca3998-19ae-4b9c-b9bc-1a7e53dff980 |      | fa:16:3e:7b:c6:a5 | {"subnet_id": "ccdb4f67-568b-47a0-b652-c1d8b8d66135", "ip_address": "10.0.0.2"}     |
| 19bae3c4-0b9e-43da-bf05-bd8c84066a10 |      | fa:16:3e:fe:cc:77 | {"subnet_id": "ccdb4f67-568b-47a0-b652-c1d8b8d66135", "ip_address": "10.0.0.1"}     |
| 39dc97e5-6652-43e4-a20a-75a69c8f77d2 |      | fa:16:3e:28:d8:c2 | {"subnet_id": "ccdb4f67-568b-47a0-b652-c1d8b8d66135", "ip_address": "10.0.0.4"}     |

Into Instance from VNC

ping 10.0.0.1 

It's connected well.

ifconfig eth0 10.0.0.5

The network cannot connect to 10.0.0.1

Test allowed-address-pairs

Using above command neutron port-list and find out mac address fa:16:3e:28:d8:c2.

stack@kiloceilo:~/devstack$ neutron port-update 39dc97e5-6652-43e4-a20a-75a69c8f77d2 --allowed-address-pairs type=dict list=true mac_address=fa:16:3e:28:d8:c2,ip_address=10.0.0.0/24
Updated port: 39dc97e5-6652-43e4-a20a-75a69c8f77d2

If you just want some ip addresses not a subnet, try this.

neutron port-update af47ddc2-d6db-4a4d-a922-6e6efcab4737 --allowed-address-pairs type=dict list=true ip_address=172.18.58.94 ip_address=172.18.58.95

Into instnace and change ip to

ifconfig eth0 10.0.0.5

and ping gateway

ping 10.0.0.1

Now it connect to gateway, whatever ip address you set in instance.

Test 2: Launch another VM and Ping with eachother with an arbitrary IP.

In instance1: we modify IP to 10.0.0.100. In instance2(new): the IP is 10.0.0.5, given by neutron.

case 1: instance1 ping instance2(10.0.0.5): connected case 2: instance2 ping instance1(10.0.0.100): connected case 3: instance2 ping previous instance1 IP(10.0.0.4): not connected

The result is satisfied with our expectation.

Up date Ip address

After we assign IP=10.0.0.100, the openstack still not update to it. So we need to update it by following command.

neutron port-update 39dc97e5-6652-43e4-a20a-75a69c8f77d2 --request-format=json --fixed_ips type=dict list=true subnet_id=ccdb4f67-568b-47a0-b652-c1d8b8d66135,ip_address=10.0.0.100
Updated port: 39dc97e5-6652-43e4-a20a-75a69c8f77d2

One can get all the information from the following commands to get port_id and subnet_id according to mac address

neuton port-list 

To check result, now system has been modifed to 10.0.0.100.

 nova list
+--------------------------------------+-------+---------+------------+-------------+--------------------------------+
| ID                                   | Name  | Status  | Task State | Power State | Networks                       |
+--------------------------------------+-------+---------+------------+-------------+--------------------------------+
| 76a7cc1c-2395-46dc-9082-860ba086e6b8 | test5 | ACTIVE  | -          | Running     | private=10.0.0.100             |
| f0ae5328-b858-46c4-ad47-84a8c6d3002e | test6 | SHUTOFF | -          | Shutdown    | private=10.0.0.5, 172.16.235.9 |
+--------------------------------------+-------+---------+------------+-------------+--------------------------------+

After update this, Rebooting the VM will get new ip persistently, and you don't need to set static file in VM.

HOw it works

It works on chain IPtables. First if

neutron port-update 39dc97e5-6652-43e4-a20a-75a69c8f77d2 --allowed-address-pairs type=dict list=true mac_address=fa:16:3e:28:d8:c2,ip_address=10.0.0.111
sudo iptables -nvL
.
.
.
Chain neutron-openvswi-s39dc97e5-6 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all  --  *      *       10.0.0.111           0.0.0.0/0            MAC FA:16:3E:28:D8:C2 /* Allow traffic from defined IP/MAC pairs. */
    0     0 RETURN     all  --  *      *       10.0.0.101           0.0.0.0/0            MAC FA:16:3E:28:D8:C2 /* Allow traffic from defined IP/MAC pairs. */
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* Drop traffic without an IP/MAC allow rule. */
.
.
.
 

if set 10.0.0.0/24.

neutron port-update 39dc97e5-6652-43e4-a20a-75a69c8f77d2 --allowed-address-pairs type=dict list=true mac_address=fa:16:3e:28:d8:c2,ip_address=10.0.0.0/24
sudo iptables -nvL
.
.

Chain neutron-openvswi-s39dc97e5-6 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all  --  *      *       10.0.0.0/24          0.0.0.0/0            MAC FA:16:3E:28:D8:C2 /* Allow traffic from defined IP/MAC pairs. */
    0     0 RETURN     all  --  *      *       10.0.0.101           0.0.0.0/0            MAC FA:16:3E:28:D8:C2 /* Allow traffic from defined IP/MAC pairs. */
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* Drop traffic without an IP/MAC allow rule. */

Some useful command

stack@kiloceilo:~/devstack$ nova show 76a7cc1c-2395-46dc-9082-860ba086e6b8
+--------------------------------------+----------------------------------------------------------------+
| Property                             | Value                                                          |
+--------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig                    | AUTO                                                           |
| OS-EXT-AZ:availability_zone          | nova                                                           |
| OS-EXT-SRV-ATTR:host                 | kiloceilo                                                      |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | kiloceilo                                                      |
| OS-EXT-SRV-ATTR:instance_name        | instance-00000003                                              |
| OS-EXT-STS:power_state               | 1                                                              |
| OS-EXT-STS:task_state                | -                                                              |
| OS-EXT-STS:vm_state                  | active                                                         |
| OS-SRV-USG:launched_at               | 2016-06-08T07:27:38.000000                                     |
| OS-SRV-USG:terminated_at             | -                                                              |
| accessIPv4                           |                                                                |
| accessIPv6                           |                                                                |
| config_drive                         | True                                                           |
| created                              | 2016-06-08T07:27:31Z                                           |
| flavor                               | m1.tiny (1)                                                    |
| hostId                               | 66b3d5bcf0238e5c02f41ceae66e09a90624594b499b5ef2a52470de       |
| id                                   | 76a7cc1c-2395-46dc-9082-860ba086e6b8                           |
| image                                | cirros-0.3.3-x86_64-uec (def244bc-7291-45cb-b336-005e90b61d00) |
| key_name                             | -                                                              |
| metadata                             | {}                                                             |
| name                                 | test5                                                          |
| os-extended-volumes:volumes_attached | []                                                             |
| private network                      | 10.0.0.4                                                       |
| progress                             | 0                                                              |
| security_groups                      | default                                                        |
| status                               | ACTIVE                                                         |
| tenant_id                            | e07408c60a9e44b6b38161aaa3f776ae                               |
| updated                              | 2016-06-08T07:27:39Z                                           |
| user_id                              | 606dca546499433a915459e2db358e14                               |
+--------------------------------------+----------------------------------------------------------------+
nova start 76a7cc1c-2395-46dc-9082-860ba086e6b8
Request to start server f0ae5328-b858-46c4-ad47-84a8c6d3002e has been accepted.
stack@kiloceilo:~/devstack$ nova list
+--------------------------------------+-------+--------+------------+-------------+------------------+
| ID                                   | Name  | Status | Task State | Power State | Networks         |
+--------------------------------------+-------+--------+------------+-------------+------------------+
| 76a7cc1c-2395-46dc-9082-860ba086e6b8 | test5 | ACTIVE | -          | Running     | private=10.0.0.4 |
| f0ae5328-b858-46c4-ad47-84a8c6d3002e | test6 | ACTIVE | -          | Running     | private=10.0.0.5 |
+--------------------------------------+-------+--------+------------+-------------+------------------+
neutron port-update PORT_UUID --request-format=json --fixed_ips type=dict list=true subnet_id=SUBNET_UUID,ip_address=IPADDR
stack@kiloceilo:~/devstack$ neutron agent-list
+--------------------------------------+--------------------+-----------+-------+----------------+---------------------------+
| id                                   | agent_type         | host      | alive | admin_state_up | binary                    |
+--------------------------------------+--------------------+-----------+-------+----------------+---------------------------+
| 75e3d39f-2be9-48f4-8b98-364d4e13fc32 | Metadata agent     | kiloceilo | :-)   | True           | neutron-metadata-agent    |
| 8eb719d1-767b-41d0-aa76-e9d19b886057 | DHCP agent         | kiloceilo | :-)   | True           | neutron-dhcp-agent        |
| e2147851-c150-4227-a9b9-42dbcedc0303 | Open vSwitch agent | kiloceilo | :-)   | True           | neutron-openvswitch-agent |
| eb602c1e-97c2-4e04-a3a4-39e3be503170 | L3 agent           | kiloceilo | :-)   | True           | neutron-vpn-agent         |
+--------------------------------------+--------------------+-----------+-------+----------------+---------------------------+

You can see more detailed infomation through neutron agent-show command.

stack@kiloceilo:~/devstack$ neutron agent-show 8eb719d1-767b-41d0-aa76-e9d19b886057
+---------------------+----------------------------------------------------------+
| Field               | Value                                                    |
+---------------------+----------------------------------------------------------+
| admin_state_up      | True                                                     |
| agent_type          | DHCP agent                                               |
| alive               | True                                                     |
| binary              | neutron-dhcp-agent                                       |
| configurations      | {                                                        |
|                     |      "subnets": 1,                                       |
|                     |      "use_namespaces": true,                             |
|                     |      "dhcp_lease_duration": 86400,                       |
|                     |      "dhcp_driver": "neutron.agent.linux.dhcp.Dnsmasq",  |
|                     |      "networks": 1,                                      |
|                     |      "ports": 5                                          |
|                     | }                                                        |
| created_at          | 2016-05-04 04:19:32                                      |
| description         |                                                          |
| heartbeat_timestamp | 2016-06-09 14:12:16                                      |
| host                | kiloceilo                                                |
| id                  | 8eb719d1-767b-41d0-aa76-e9d19b886057                     |
| started_at          | 2016-06-09 14:10:16                                      |
| topic               | dhcp_agent                                               |
+---------------------+----------------------------------------------------------+

To see all the vswitch bridge, we shall have 3 bridges, br-ex, br-int, br-tun. Now we can see what stuff connected to each of 3 bridges; respectively.

stack@kiloceilo:~/devstack$ sudo ovs-vsctl list-br
br-ex
br-int
br-tun

To see more detailed information of bridge

stack@kiloceilo:~/devstack$ sudo ovs-vsctl list-ports br-ex
eth0
phy-br-ex
stack@kiloceilo:~/devstack$ sudo ovs-vsctl list-ports br-tun
patch-int

To see more detailed about br-int. Ingration bridge is most important bridge that could be connect all the resources, including VM, router, and external bridge. This commands show the tap-device

stack@kiloceilo:~/devstack$ sudo ovs-vsctl list-ports br-int
int-br-ex
patch-tun
qg-0155e3e5-11
qr-19bae3c4-0b
qvo39dc97e5-66
qvo820d2836-00
tap01ca3998-19
tapad1f4818-b1
tapb8ce74b5-cb

solved qg, qr, tap01, tap39

To see what we contains the network-namespace

stack@kiloceilo:~/devstack$ ip netns
qrouter-f8cfe243-5908-4003-a776-59ce891e5d80
qdhcp-2b696172-5599-4663-85bd-42de265e56df
stack@kiloceilo:~/devstack$ sudo ip netns exec qdhcp-2b696172-5599-4663-85bd-42de265e56df ip a
.
.
.
5: tap01ca3998-19: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default
    link/ether fa:16:3e:7b:c6:a5 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.2/24 brd 10.0.0.255 scope global tap01ca3998-19
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe7b:c6a5/64 scope link
       valid_lft forever preferred_lft forever

10.0.0.2 is the ip address of the dhcp service and with tap device tap01ca3998-19 that also shown in sudo ovs-vsctl list-ports br-int that connectd to br-int.

Enter namespace

ip netns exec qrxxxxx bash
ifconfig
.
.
lo
.
tapxxxxx
.
.

You then enter the namespace, type ifconfig, you will see the tapdevice as eth0. It's a good concept to say what is tapdevice.

To check the vrouter.

stack@kiloceilo:~/devstack$ sudo ip netns exec qrouter-f8cfe243-5908-4003-a776-59ce891e5d80 ip a
.
.
6: qg-0155e3e5-11: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default
    link/ether fa:16:3e:be:57:b1 brd ff:ff:ff:ff:ff:ff
    inet 172.16.235.3/24 brd 172.16.235.255 scope global qg-0155e3e5-11
       valid_lft forever preferred_lft forever
    inet 172.16.235.9/32 brd 172.16.235.9 scope global qg-0155e3e5-11
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:febe:57b1/64 scope link
       valid_lft forever preferred_lft forever
8: qr-19bae3c4-0b: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default
    link/ether fa:16:3e:fe:cc:77 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.1/24 brd 10.0.0.255 scope global qr-19bae3c4-0b
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fefe:cc77/64 scope link
       valid_lft forever preferred_lft forever

The router contains the tap device qr-19bae3c4-0b and with IP **10.0.0.1.

That means, you can connect to VM and ping both of these two IP, you can get response.

where qgxxxx shoud be the floating ip bridge.

where is the VM tap

If you know mac address, you can then get tap-device name by using ifconfig shown as followed

stack@kiloceilo:~/devstack$ ifconfig
.
.
tap39dc97e5-66 Link encap:Ethernet  HWaddr fe:16:3e:28:d8:c2
          inet6 addr: fe80::fc16:3eff:fe28:d8c2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:11 errors:0 dropped:0 overruns:0 frame:0
          TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:1324 (1.3 KB)  TX bytes:1835 (1.8 KB)

However, you will not see the tap device in vswitch, since it connect to linux bridge. So we should check the linux bridge.

stack@kiloceilo:~/devstack$ brctl show
bridge name bridge id       STP enabled interfaces
qbr39dc97e5-66      8000.fa3282c953c0   no      qvb39dc97e5-66
                            tap39dc97e5-66
qbr820d2836-00      8000.525767799304   no      qvb820d2836-00
                            tap820d2836-00
virbr0      8000.000000000000   yes

we get tap devce tap39dc97e5-66.

We can plot the architect according to the command.

qbr is linux bridge qvxx and tapxx, qr, and qg are tap device (link)

stack@kiloceilo:~/devstack$ sudo ovs-vsctl show
a466163f-4f0a-40a2-bd0c-1a98781fe02c
    Bridge br-tun
        fail_mode: secure
        Port br-tun
            Interface br-tun
                type: internal
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
    Bridge br-ex
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
    .
    .
    .

Check packet

stack@kiloceilo:~/devstack$ sudo ovs-ofctl dump-flows br-tun
NXST_FLOW reply (xid=0x4):
 cookie=0x0, duration=3145.114s, table=0, n_packets=244, n_bytes=31473, idle_age=11, priority=1,in_port=1 actions=resubmit(,2)
 cookie=0x0, duration=3144.903s, table=0, n_packets=0, n_bytes=0, idle_age=3144, priority=0 actions=drop
 cookie=0x0, duration=3144.647s, table=2, n_packets=2, n_bytes=684, idle_age=766, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20)
 cookie=0x0, duration=3144.567s, table=2, n_packets=242, n_bytes=30789, idle_age=11, priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,22)
 cookie=0x0, duration=3144.451s, table=3, n_packets=0, n_bytes=0, idle_age=3144, priority=0 actions=drop

How to get tap linked between each other

For VM, you must lunch VM, you then can see the relation between tap and qv device. but ip link cannot display namespace side information.

ip -d link show
.
.
.
13: qbr39dc97e5-66: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
    link/ether ba:4b:ab:b0:d4:ba brd ff:ff:ff:ff:ff:ff promiscuity 0
    bridge
14: qvo39dc97e5-66: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovs-system state UP mode DEFAULT group default qlen 1000
    link/ether 0a:fa:4d:59:5f:41 brd ff:ff:ff:ff:ff:ff promiscuity 2
    veth
15: qvb39dc97e5-66: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master qbr39dc97e5-66 state UP mode DEFAULT group default qlen 1000
    link/ether ba:4b:ab:b0:d4:ba brd ff:ff:ff:ff:ff:ff promiscuity 2
    veth
16: qbr820d2836-00: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
    link/ether a6:34:a6:b1:fc:8b brd ff:ff:ff:ff:ff:ff promiscuity 0
    bridge
17: qvo820d2836-00: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovs-system state UP mode DEFAULT group default qlen 1000
    link/ether 1a:0f:ab:1e:d7:d9 brd ff:ff:ff:ff:ff:ff promiscuity 2
    veth
18: qvb820d2836-00: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master qbr820d2836-00 state UP mode DEFAULT group default qlen 1000
    link/ether a6:34:a6:b1:fc:8b brd ff:ff:ff:ff:ff:ff promiscuity 2
    veth
19: tap39dc97e5-66: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master qbr39dc97e5-66 state UNKNOWN mode DEFAULT group default qlen 500
    link/ether fe:16:3e:28:d8:c2 brd ff:ff:ff:ff:ff:ff promiscuity 1
    tun
20: tap820d2836-00: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master qbr820d2836-00 state UNKNOWN mode DEFAULT group default qlen 500
    link/ether fe:16:3e:53:02:80 brd ff:ff:ff:ff:ff:ff promiscuity 1
    tun

where tap39dc97e5-66 get the Macaddress, belong to VM, and master connect to qbr39dc97e5-66. And qvb39dc97e5-66 connect to qbr39dc97e5-66.

You can also see the link in namespace.

sudo ip netns exec qrouter-f8cfe243-5908-4003-a776-59ce891e5d80 ip -d link show
.
.
7: qg-0155e3e5-11: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default
    link/ether fa:16:3e:be:57:b1 brd ff:ff:ff:ff:ff:ff promiscuity 1
8: qr-19bae3c4-0b: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default
    link/ether fa:16:3e:fe:cc:77 brd ff:ff:ff:ff:ff:ff promiscuity 1
stack@kiloceilo:~/devstack$ sudo ip netns exec qdhcp-2b696172-5599-4663-85bd-42de265e56df ip -d link show
.
.
5: tap01ca3998-19: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default
    link/ether fa:16:3e:7b:c6:a5 brd ff:ff:ff:ff:ff:ff promiscuity 1

So stange tap01caxxx is binded in br-int, it should be not in namespace, not consistence shown in qrouterxx.

v-Router

Check V-router's iptables. And we can see how the floating IP, 172.16.23.9, adding to VM, 10.0.0.5 by using IPtables in namespace with snat and dnat. And How dose the cloudinit, 169.254.169.254 works.

sudo ip netns exec qrouter-f8cfe243-5908-4003-a776-59ce891e5d80 iptables -t nat -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N neutron-postrouting-bottom
-N neutron-vpn-agen-OUTPUT
-N neutron-vpn-agen-POSTROUTING
-N neutron-vpn-agen-PREROUTING
-N neutron-vpn-agen-float-snat
-N neutron-vpn-agen-snat
-A PREROUTING -j neutron-vpn-agen-PREROUTING
-A OUTPUT -j neutron-vpn-agen-OUTPUT
-A POSTROUTING -j neutron-vpn-agen-POSTROUTING
-A POSTROUTING -j neutron-postrouting-bottom
-A neutron-postrouting-bottom -m comment --comment "Perform source NAT on outgoing traffic." -j neutron-vpn-agen-snat
-A neutron-vpn-agen-OUTPUT -d 172.16.235.9/32 -j DNAT --to-destination 10.0.0.5
-A neutron-vpn-agen-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697
-A neutron-vpn-agen-float-snat -s 10.0.0.5/32 -j SNAT --to-source 172.16.235.9
.
.

And 172.16.235.9 is locate in tap qg-0155e3e5-11, where you can use ip a to check it in namespace.

DHCP

One can use ps command to realize the dnsmasq information.

ps aux|grep dnsmasq
.
.
libvirt+   3210  0.0  0.0  28212  2368 ?        S    05:28   0:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf
nobody    15768  0.0  0.0  29756  2368 ?        S    09:02   0:00 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tap01ca3998-19 --except-interface=lo --pid-file=/opt/stack/data/neutron/dhcp/2b696172-5599-4663-85bd-42de265e56df/pid --dhcp-hostsfile=/opt/stack/data/neutron/dhcp/2b696172-5599-4663-85bd-42de265e56df/host --addn-hosts=/opt/stack/data/neutron/dhcp/2b696172-5599-4663-85bd-42de265e56df/addn_hosts --dhcp-optsfile=/opt/stack/data/neutron/dhcp/2b696172-5599-4663-85bd-42de265e56df/opts --dhcp-leasefile=/opt/stack/data/neutron/dhcp/2b696172-5599-4663-85bd-42de265e56df/leases --dhcp-range=set:tag0,10.0.0.0,static,86400s --dhcp-lease-max=256 --conf-file= --domain=openstacklocal

We also obtain the tapdevice by using this command.

DHCP Mac and IP table is here.

stack@kiloceilo:~/devstack$ cat /opt/stack/data/neutron/dhcp/2b696172-5599-4663-85bd-42de265e56df/host
fa:16:3e:28:d8:c2,host-10-0-0-101.openstacklocal,10.0.0.101
fa:16:3e:53:02:80,host-10-0-0-5.openstacklocal,10.0.0.5
fa:16:3e:7b:c6:a5,host-10-0-0-2.openstacklocal,10.0.0.2
fa:16:3e:b8:be:a1,host-10-0-0-3.openstacklocal,10.0.0.3
fa:16:3e:fe:cc:77,host-10-0-0-1.openstacklocal,10.0.0.1

where .1.2 is router and dhcp IP.

To check port with vm_id and so on (great)

We can check qvoxx that bind to VM, the following command will show the qvoxx binds to which VM and VM's Mac address. It's very useful tool that can show all VMs.

sudo ovs-vsctl get Interface qvo820d2836-00 external_ids
{attached-mac="fa:16:3e:53:02:80", iface-id="820d2836-00d7-4f37-847f-344972fc246e", iface-status=active, vm-uuid="f0ae5328-b858-46c4-ad47-84a8c6d3002e"}

and in port

sudo ovs-vsctl get Interface qvo820d2836-00 ofport
6

List all port and name

root@openvswitch:~# ovs-vsctl -- --columns=name,ofport list Interface
name                : ovsbr
ofport              : 65534

name                : veth-foo
ofport              : 1

name                : veth-bob
ofport              : 2

To see port status and ..

root@openvswitch:~# ovs-ofctl show ovsbr
OFPT_FEATURES_REPLY (xid=0x2): dpid:00004aab5354d446
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
 1(veth-foo): addr:7a:ac:ca:b0:07:b7
     config:     PORT_DOWN
     state:      LINK_DOWN
     current:    10GB-FD COPPER
     speed: 10000 Mbps now, 0 Mbps max
 2(veth-bob): addr:c2:14:a7:54:ce:67
     config:     PORT_DOWN
     state:      LINK_DOWN
     current:    10GB-FD COPPER
     speed: 10000 Mbps now, 0 Mbps max
 LOCAL(ovsbr): addr:52:f0:77:8a:c2:41
     config:     PORT_DOWN
     state:      LINK_DOWN
     speed: 0 Mbps now, 0 Mbps max

No comments:

Post a Comment