CICD and ISTIO

CI/CD and ISTIO

CI/CD contains different stages, such as DEV, QA, Staging, and Production. Each Stages might have their own network environment. For example a webbackend url could be a ip address 172.16.155.207 in dev environment or a mesh service webbackend in QA environment or a real internet domain name webbackend.com in production environment.

We don't hope that we have to change the POD env to match the differnent network environment. I hope that the POD environment is fixed, said webbackend. And We adjust the network environment in different stages by using ISTIO for routing the webbackend url to fit the stage environment.

We mark the url webbackend and webbackend.com as a short url and a long url. And will start discuss how this to url for fit the stages network environment changed.

For Short URL

short url to a kubernetes service

This is a native kubernetes environment, we dont need to change any thing.

short url to a External IP address

istioctl register short 172.16.155.207 http:8000 -n jj

where 172.16.155.207 is the ipaddress outside the service mesh.

Now you can aceess url short to it.

#kubectl get svc

NAME            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
short           ClusterIP   172.18.142.145   <none>        8000/TCP   1s

And it will be registerd as a kubernetes service.

Delete register

istioctl deregister short 172.16.155.207  -n jj

kubectl delete svc short

short url to a outside dns doman name

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: shot-ext
spec:
  hosts:
  - short.jj.svc.cluster.local
  location: MESH_EXTERNAL
  ports:
  - number: 80
    name: https
    protocol: HTTP
  resolution: DNS
  endpoints:
  - address: httpbin.org
    ports:
      https: 80

istioctl create -f myroute/serviceentry/route-shortremote.yaml

You can use url short to access a existed internet site httpbin.org.

It's not finished yet. Note: In ServiceEntry the hosts must be a real network and is responsible, since ServiceEntry just do network routing and dose not doing anything about DNS and Registry. So we need a fake url short created by kubernetes service.

apiVersion: v1
kind: Service
metadata:
  name: short
  labels:
    app: short
spec:
  ports:
  # must use - port (-); can not use - name
  - port: 8000
    targetPort: 8000
    name: http
  selector:
    app: short

kubectl apply -f myroute/serviceentry/fakeurl-shortremote.yaml

Long Url

Create Gateway.

kind: Gateway
metadata:
  name: istio-egressgateway
spec:
  selector:
    istio: egressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "edition.cnn.com"
    - "test.nxg"

istioctl create -f myroute/egress/gateway-egress.yaml

We prepare to hosts for further examples.

Existed Long url to a kubernetes service

Here we means a Existed means it really existed in Internet and we just need to route the traffic to mesh.

root@curlserver-2ddwj:/# curl edition.cnn.com:80 -v
* Rebuilt URL to: edition.cnn.com:80/
*   Trying 151.101.193.67...
* TCP_NODELAY set
* Connected to edition.cnn.com (151.101.193.67) port 80 (#0)
> GET / HTTP/1.1
> Host: edition.cnn.com
> User-Agent: curl/7.52.1
> Accept: */*
>
< HTTP/1.1 404 Not Found
< date: Fri, 05 Oct 2018 22:59:42 GMT
< server: envoy
< content-length: 0
<
* Curl_http_done: called premature == 0
* Connection #0 to host edition.cnn.com left intact

You will get above result when you curl the url initially. The network is existed but blocked by ISTIO, now we can start setup the routing by istio-egress.

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: cnn
spec:
  hosts:
  - edition.cnn.com
  ports:
  - number: 80
    name: http-port
    protocol: HTTP
  - number: 443
    name: https
    protocol: HTTPS
  resolution: DNS
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: direct-through-egress-gateway
spec:
  hosts:
  - edition.cnn.com
  gateways:
  - istio-egressgateway
  - mesh
  http:
  - match:
    - gateways:
      - mesh
      port: 80
    route:
    - destination:
        host: istio-egressgateway.istio-system.svc.cluster.local
        port:
          number: 80
      weight: 100
  - match:
    - gateways:
      - istio-egressgateway
      port: 80
    route:
    - destination:
        host: webbackendnxg
        port:
          number: 8000
      weight: 100

istioctl create -f myroute/egress/local-edition.cnn.com.yaml

Test the result in any POD that launched by ISTIO.

curl edition.cnn.com:80 -v
* Rebuilt URL to: edition.cnn.com:80/
*   Trying 151.101.129.67...
* TCP_NODELAY set
* Connected to edition.cnn.com (151.101.129.67) port 80 (#0)
.
.
.
webbackendnxg-v1-56596cdd44-fws75
* Curl_http_done: called premature == 0
* Connection #0 to host edition.cnn.com left intact

where webbackendnxg-v1-56596cdd44-fws75 is the response of host: webbackendnxg in our mesh.

Non-Existed Long url to a kubernetes service

To test a url that not existed in internet or in your local environment. You might get the result.

root@curlserver-2ddwj:/# curl test.nxg:80 -v
* Rebuilt URL to: test.nxg:80/
* Could not resolve host: test.nxg
* Closing connection 0
curl: (6) Could not resolve host: test.nxg

After crate gateway, see above.

cat myroute/egress/local-test.nxg.yaml

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: nxg-test
spec:
  hosts:
  - test.nxg
  ports:
  - number: 80
    name: http-port
    protocol: HTTP
  - number: 443
    name: https
    protocol: HTTPS
  resolution: DNS
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: direct-through-egress-gateway-test
spec:
  hosts:
  - test.nxg
  gateways:
  # here is the parameter that go to next pipline
  # go to host: istio-egressgateway first and then to next place coud be internet or local(webbackendnxg)
  # if diable one, it will failed
  - istio-egressgateway
  - mesh
  http:
  - match:
    - gateways:
      - mesh
      port: 80
    route:
    - destination:
        host: istio-egressgateway.istio-system.svc.cluster.local
        port:
          number: 80
      weight: 100
  - match:
    - gateways:
      - istio-egressgateway
      port: 80
    route:
    - destination:
        host: webbackendnxg
        port:
          number: 8000
      weight: 100

There is a sequential pipline on this yaml, first route traffic to istio-egressgateway and second routes to host: webbackendnxg. And declair the pipline in yaml

gateways:
  - istio-egressgateway
  - mesh

launch the egress setting.

istioctl create -f  myroute/egress/local-test.nxg.yaml -n jj

where -n jj is the namespace in my environment.

It's not finished yet.

For a ServiceEntry we need a real url existed in you network environment. In this case test.nxg is not register or not in DNS in your network environment. So we need to create the test.nxg.

It's kind of Tricky here. In the url test.nxg where nxg is a namespace in kubernetes service and test is a service. So we need to create

a namespace nxg
a test service yaml for nxg namespace

apiVersion: v1
kind: Service
metadata:
  name: test
  namespace: nxg
  labels:
    app: test
spec:
  ports:
  # must use - port (-); can not use - name
  - port: 8000
    targetPort: 8000
    name: http
  selector:
    app: test

here port 8000 is useless, since for serviceentry will just check the service is existed or not, so any port can use it and any other namespace can access it. Note that, there is no POD defined in this "FAKE" service.

kubectl create namespace nxg

kubectl apply -f myroute/egress/fakeurl-test.nxg.yaml

Now it works by curl test.nxg:80 -v

root@curlserver-2ddwj:/# curl test.nxg:80 -v
* Rebuilt URL to: test.nxg:80/
*   Trying 172.18.117.18...
* TCP_NODELAY set
* Connected to test.nxg (172.18.117.18) port 80 (#0)
> GET / HTTP/1.1
> Host: test.nxg
> User-Agent: curl/7.52.1
> Accept: */*
>
< HTTP/1.1 200 OK
< server: envoy
< date: Fri, 05 Oct 2018 23:19:21 GMT
< content-type: text/html
< x-envoy-upstream-service-time: 6
< transfer-encoding: chunked
<
webbackendnxg-v1-56596cdd44-fws75
* Curl_http_done: called premature == 0
* Connection #0 to host test.nxg left intact

where 172.18.117.18 was sat by fake service. And route to webbackendnxg.

long url to a External IP address

We need to combine the register and egress method. If we have a IP 172.16.155.207 and route from test.nxg to this IP

Create a short url first.

istioctl deregister short 172.16.155.207  -n jj

just as the previsous section

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: nxg-test
spec:
  hosts:
  - test.nxg
  ports:
  - number: 80
    name: http-port
    protocol: HTTP
  - number: 443
    name: https
    protocol: HTTPS
  resolution: DNS
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: direct-through-egress-gateway-test
spec:
  hosts:
  - test.nxg
  gateways:
  # here is the parameter that go to next pipline
  # go to host: istio-egressgateway first and then to next place coud be internet or local(webbackendnxg)
  # if diable one, it will failed
  - istio-egressgateway
  - mesh
  http:
  - match:
    - gateways:
      - mesh
      port: 80
    route:
    - destination:
        host: istio-egressgateway.istio-system.svc.cluster.local
        port:
          number: 80
      weight: 100
  - match:
    - gateways:
      - istio-egressgateway
      port: 80
    route:
    - destination:
        host: short
        port:
          number: 80
      weight: 100

where the short is defined by using register command.

Delete old setting if you did.

istioctl delete -f  myroute/egress/local-test.nxg.yaml -n jj

try this in any POD launched by ISTIO injected.

curl test.nxg:80 -v
* Rebuilt URL to: test.nxg:80/
*   Trying 172.18.117.18...
* TCP_NODELAY set
* Connected to test.nxg (172.18.117.18) port 80 (#0)
* .
* .
* .
* <!DOCTYPE html>
    <html>
      <head>
        <meta name="viewport" content="width=device-width, initial-scale=1">
.
.

It works.

long url to a outside dns domain name

In previous we show how the edition.cnn.com is directed to our local mesh. And now I just want to direct the it to its own internet address.

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: cnn-internet
spec:
  hosts:
  - edition.cnn.com
  ports:
  - number: 80
    name: http-port
    protocol: HTTP
  - number: 443
    name: https
    protocol: HTTPS
  resolution: DNS
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: direct-through-egress-gateway-internet
spec:
  hosts:
  - edition.cnn.com
  gateways:
  - istio-egressgateway
  - mesh
  http:
  - match:
    - gateways:
      - mesh
      port: 80
    route:
    - destination:
        host: istio-egressgateway.istio-system.svc.cluster.local
        port:
          number: 80
      weight: 100
  - match:
    - gateways:
      - istio-egressgateway
      port: 80
    route:
    - destination:
        host: edition.cnn.com
        port:
          number: 80
      weight: 100

After launch gateway, see above.

istioctl create -f myroute/egress/internet-edition.cnn.com.yaml -n jj

Remember to clean old setting about internet-edition.cnn.com if you did it.

istioctl delete -f myroute/egress/local-edition.cnn.com.yaml -n jj

Access to any POD launched by ISTIO inject.

root@curlserver-2ddwj:/# curl edition.cnn.com:80 -v
* Rebuilt URL to: edition.cnn.com:80/
*   Trying 151.101.193.67...
* TCP_NODELAY set
* Connected to edition.cnn.com (151.101.193.67) port 80 (#0)
> GET / HTTP/1.1
> .
> .
> < HTTP/1.1 301 Moved Permanently
< server: envoy
< retry-after: 0
.
.

It works.

瞧那部落格

Friday, October 5, 2018

CI/CD and Kubernetes ISTIO