ISTIO 0.8.0 Installation and Playing in Kubernetes

ISTIO Installation

Environment

As A Standard Kubernetes

• Kube-API, Kube-Contrller, Kube-proxy, kube-scheduler, kubelet is all needed.
• Core-DNS; also set cluster_dns in kubelet, as a standard way.
• Flannel;
• apt-get install socat; in each client and server.

ISTIO dose'nt provide DNS, so we have to use fundamental kubernetes's DNS, said coredns or kubedns.

Additional API Setting

--enable-admission-plugins=NamespaceLifecycle,ServiceAccount,LimitRanger,ResourceQuota,PersistentVolumeLabel,DefaultTolerationSeconds,DefaultStorageClass,MutatingAdmissionWebhook,ValidatingAdmissionWebhook \
--authorization-mode=RBAC \

Additional Porxy Setting

--proxy-mode=ipvs \

Finally, we use ISTIO-0.8.0, since it's LTS.

For Kubernetes 1.11

wget https://github.com/istio/istio/releases/download/0.8.0/istio-0.8.0-linux.tar.gz

export PATH=$PWD/bin:$PATH

root@kubecontext:~/istio-0.8.0# istioctl version
Version: 0.8.0
GitRevision: 6f9f420f0c7119ff4fa6a1966a6f6d89b1b4db84
User: root@48d5ddfd72da
Hub: docker.io/istio
GolangVersion: go1.10.1
BuildStatus: Clean

root@kubecontext:~/istio-0.8.0# kubectl apply -f install/kubernetes/istio-demo.yaml
namespace/istio-system created
configmap/istio-statsd-prom-bridge created
configmap/istio-mixer-custom-resources created
configmap/prometheus created
configmap/istio created
configmap/istio-sidecar-injector created
serviceaccount/istio-egressgateway-service-account created
serviceaccount/istio-ingressgateway-service-account created
serviceaccount/istio-mixer-post-install-account created
clusterrole.rbac.authorization.k8s.io/istio-mixer-post-install-istio-system created
clusterrolebinding.rbac.authorization.k8s.io/istio-mixer-post-install-role-binding-istio-system created
job.batch/istio-mixer-post-install created
serviceaccount/istio-mixer-service-account created
serviceaccount/istio-pilot-service-account created
serviceaccount/prometheus created
serviceaccount/istio-citadel-service-account created
serviceaccount/istio-cleanup-old-ca-service-account created
serviceaccount/istio-sidecar-injector-service-account created
customresourcedefinition.apiextensions.k8s.io/rules.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/attributemanifests.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/circonuses.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/deniers.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/fluentds.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/kubernetesenvs.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/listcheckers.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/memquotas.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/noops.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/opas.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/prometheuses.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/rbacs.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/servicecontrols.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/solarwindses.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/stackdrivers.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/statsds.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/stdios.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/apikeys.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/authorizations.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/checknothings.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/kuberneteses.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/listentries.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/logentries.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/metrics.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/quotas.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/reportnothings.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/servicecontrolreports.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/tracespans.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/serviceroles.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/servicerolebindings.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/destinationpolicies.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/egressrules.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/routerules.config.istio.io created
customresourcedefinition.apiextensions.k8s.io/virtualservices.networking.istio.io created
customresourcedefinition.apiextensions.k8s.io/destinationrules.networking.istio.io created
customresourcedefinition.apiextensions.k8s.io/serviceentries.networking.istio.io created
customresourcedefinition.apiextensions.k8s.io/gateways.networking.istio.io created
customresourcedefinition.apiextensions.k8s.io/policies.authentication.istio.io created
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
customresourcedefinition.apiextensions.k8s.io/httpapispecbindings.config.istio.io configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
customresourcedefinition.apiextensions.k8s.io/httpapispecs.config.istio.io configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
customresourcedefinition.apiextensions.k8s.io/quotaspecbindings.config.istio.io configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
customresourcedefinition.apiextensions.k8s.io/quotaspecs.config.istio.io configured
clusterrole.rbac.authorization.k8s.io/istio-mixer-istio-system created
clusterrole.rbac.authorization.k8s.io/istio-pilot-istio-system created
clusterrole.rbac.authorization.k8s.io/prometheus-istio-system created
clusterrolebinding.rbac.authorization.k8s.io/prometheus-istio-system created
clusterrole.rbac.authorization.k8s.io/istio-citadel-istio-system created
role.rbac.authorization.k8s.io/istio-cleanup-old-ca-istio-system created
clusterrole.rbac.authorization.k8s.io/istio-sidecar-injector-istio-system created
clusterrolebinding.rbac.authorization.k8s.io/istio-mixer-admin-role-binding-istio-system created
clusterrolebinding.rbac.authorization.k8s.io/istio-pilot-istio-system created
clusterrolebinding.rbac.authorization.k8s.io/istio-citadel-istio-system created
rolebinding.rbac.authorization.k8s.io/istio-cleanup-old-ca-istio-system created
clusterrolebinding.rbac.authorization.k8s.io/istio-sidecar-injector-admin-role-binding-istio-system created
service/istio-egressgateway created
service/grafana created
service/istio-ingressgateway created
service/istio-policy created
service/istio-telemetry created
service/istio-statsd-prom-bridge created
deployment.extensions/istio-statsd-prom-bridge created
service/istio-pilot created
service/prometheus created
service/istio-citadel created
service/servicegraph created
service/istio-sidecar-injector created
deployment.extensions/istio-egressgateway created
deployment.extensions/grafana created
deployment.extensions/istio-ingressgateway created
deployment.extensions/istio-policy created
deployment.extensions/istio-telemetry created
deployment.extensions/istio-pilot created
deployment.extensions/prometheus created
deployment.extensions/istio-citadel created
deployment.extensions/servicegraph created
deployment.extensions/istio-sidecar-injector created
deployment.extensions/istio-tracing created
job.batch/istio-cleanup-old-ca created
horizontalpodautoscaler.autoscaling/istio-egressgateway created
horizontalpodautoscaler.autoscaling/istio-ingressgateway created
service/zipkin created
service/tracing created
mutatingwebhookconfiguration.admissionregistration.k8s.io/istio-sidecar-injector created

Modify

If You are not using version 0.8.0, it will tell you a lot of error.

Enable RBAC

adding Kube-api with

--authorization-mode=RBAC

Remove Admission Control

kube-apiserver

For Kube-inject

from 
--admission-control=NamespaceLifecycle,ServiceAccount,LimitRanger,SecurityContextDeny,ResourceQuota \
to
--admission-control=NamespaceLifecycle,ServiceAccount,LimitRanger,ResourceQuota \

to
--enable-admission-plugins=NamespaceLifecycle,ServiceAccount,LimitRanger,ResourceQuota,PersistentVolumeLabel,DefaultTolerationSeconds,DefaultStorageClass,MutatingAdmissionWebhook,ValidatingAdmissionWebhook \

IPVS Mode

kube-proxy.service

--proxy-mode=ipvs \

instsall Socat

apt-get install socat

To all K8S and Client Server.

Helm Installation

https://github.com/kubernetes/helm/releases

downlaod 2.9.1

helm init --upgrade -i registry.cn-hangzhou.aliyuncs.com/google_containers/tiller:v2.9.1 --stable-repo-url https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts

Modify sidecar inject file and enable all egress traffic. So you can direct access network by using apt-get, pip install, whatever you want as a no firewall linux.

helm template install/kubernetes/helm/istio --name istio --namespace istio-system --set sidecarInjectorWebhook.enabled=false --set global.proxy.includeIPRanges="172.16.0.0/16" -x templates/sidecar-injector-configmap.yaml | kubectl apply -f -

where you can make sure the docker ip range 172.16.0.0/16.

if you don't want to this, just don't execute the above commmand.

helm template install/kubernetes/helm/istio --name istio --namespace istio-system --set istio-sidecar-injector=true  -x templates/sidecar-injector-configmap.yaml | kubectl apply -f -

here we set auto inject sidecar not take effect sidecarInjectorWebhook.enabled=false.

kubectl get po --all-namespaces
kube-system    tiller-deploy-b67849f44-bfkch              1/1       Running            0          3m

/root/helm/helm template /root/istio-0.8.0/install/kubernetes/helm/istio --name istio --namespace istio-system > ./istio-0.8.yaml

kubectl apply -f istio-0.8.yaml

Create Application

kubectl apply -f <(istioctl kube-inject -f samples/bookinfo/kube/bookinfo.yaml) -n istio-system
//or kubectl apply -f samples/bookinfo/kube/bookinfo.yaml -n istio-system


istioctl create -f samples/bookinfo/routing/bookinfo-gateway.yaml -n istio-system

istioctl create -f samples/bookinfo/routing/bookinfo-gateway.yaml -n istio-system
istioctl get gateway 

where istioctl is not related to kubectl context, and istioctl's namespace is default, if your namespace is not default please adding the namespace after istioctl.

Check Yaml

kubectl get pod productpage-v1-57f4d6b98-qwx58 -o yaml -n istio-system
kubectl get deployment  productpage-v1 -o yaml -n istio-system

You will see the inject envoy container runing on.

.
.
.
        image: docker.io/istio/proxyv2:0.8.0
        imagePullPolicy: IfNotPresent
        name: istio-proxy.
.

Test result

curl http://172.16.155.207:31380/productpage

Check version

curl -s http://172.16.155.207:31380/productpage|grep color

In Container

kubectl run curl-test --image=radial/busyboxplus:curl -i --tty --rm -n istio-system

Construct Network

Kubectl Only

kubectl apply -f samples/bookinfo/kube/bookinfo-gateway.yaml -n istio-system

## See by using ing not gateway
kubectl get ing --all-namespaces

## Now You can connect through 32000 port; different with istio created.
http://172.16.155.207:32000/productpage

All not working

ISTIOCTL Only

Worked Method

https://blog.csdn.net/wenwenxiong/article/details/80068835

Do not use kube folder and temperaly use istio-system namespaces Do not trust curl result in another container!!, use browser and edit LoadBalancer to NodePort.

istioctl create -f samples/bookinfo/routing/bookinfo-gateway.yaml -n istio-system

istioctl create -f samples/bookinfo/routing/route-rule-all-v1.yaml -n istio-system

then (must to create above method, must to understand it why bugs? )
istioctl create -f samples/bookinfo/routing/route-rule-reviews-v3.yaml -n istio-system

istioctl replace -f samples/bookinfo/routing/route-rule-reviews-v2-v3.yaml -n istio-system

Additional

# after istioctl create -f samples/bookinfo/routing/route-rule-all-v1.yaml -n istio-system
istioctl replace -f myroute/route-review-1.yaml -n istio-system
istioctl replace -f myroute/route-review-2.yaml -n istio-system

Not Working

Delete ALl

istioctl delete -f samples/bookinfo/routing/bookinfo-gateway.yaml -n istio-system
kubectl delete -f <(istioctl kube-inject -f samples/bookinfo/kube/bookinfo.yaml) -n istio-system
kubectl delete -f istio-0.8.yaml
 

Clean All under namespace

samples/bookinfo/kube/cleanup.sh

Deploy Application in Another Namespace

We lived in namespace jj setting by kubernetes context.

kubectl apply -f <(istioctl kube-inject -f samples/bookinfo/kube/bookinfo.yaml)

Using istioctl to create network, don't forget namespace, since it is not related to kubernetes context.

istioctl create -f samples/bookinfo/routing/bookinfo-gateway.yaml -n jj
istioctl create -f samples/bookinfo/routing/route-rule-all-v1.yaml -n jj

Where route-rule-all-v1.yaml is quite critical setting for a beginer, and it ismust setting or it will return failed message.

Sorry, product reviews are currently unavailable for this book.

After the setting, we can start to route traffic.

istioctl create -f samples/bookinfo/routing/route-rule-reviews-v2-v3.yaml -n jj
istioctl create -f samples/bookinfo/routing/route-rule-reviews-v3.yaml -n jj

Testing

curl -s http://172.16.155.207:31380/productpage|grep color
or 
using browser.

Analysis

We can not direct apply route-rule-reviews-v3.yaml, since it has some parameters not yet define, and the parameters defined in route-rule-all-v1.yaml. So we have to apply route-rule-all-v1.yaml first.

We can refactor the file and focus on reviews POD traffic. We can just apply the following yaml file without apply route-rule-all-v1.yaml.

---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: reviews
spec:
  host: reviews
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
  - name: v3
    labels:
      version: v3
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v2

Header Filter

istioctl replace -f myroute/route-review-user.yaml -n jj

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
  # need to match first and then - route, there is a sequence. match jason
  - match:
    - headers:
        cookie:
          regex: "^(.*?;)?(user=jason)(;.*)?$"
    route:
    - destination:
        host: reviews
        subset: v3
  - match:
    - headers:
        cookie:
          regex: "^(.*?;)?(user=mary)(;.*)?$"
    route:
    - destination:
        host: reviews
        subset: v1
  - route:
    - destination:
        host: reviews
        subset: v2

It's a sequence, we must set - match before - route

LoadBalancer

istioctl replace -f myroute/route-review-lb.yaml -n jj

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v2
      weight: 20
    - destination:
        host: reviews
        subset: v3
      weight: 80

Inject Setting

kubectl label namespace jj istio-injection=enabled

Check Label

kubectl get namespace -L istio-injection

Delete Labeled

kubectl label namespace jj istio-injection-

System will automatically use sidecar to run POD with a proper command kubectl apply -f xxx.yaml.

But if you don't want to run sidecar for a particular POD

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: webbackend-v1
spec:
  #serviceName: "webbackend"
  replicas: 1
  template:
    metadata:
      labels:
        app: webbackend
        version: v1
      annotations:
        pod.alpha.kubernetes.io/initialized: "true"
        sidecar.istio.io/inject: "false"

adding sidecar.istio.io/inject: "false" in annotations. and you can also use a proper command kubectl apply -f xxx.yaml and without sidecar.

root@kubecontext:~/istio-0.8.0# kubectl -n istio-system get configmap istio-sidecar-injector -o jsonpath='{.data.config}' | head
policy: enabled

There are two kinds of setting affect the default running with inject or not

namespaceSelector match default policy  Pod override annotation sidecar.istio.io/inject Sidecar injected?
yes        enabled  true    yes
yes         enabled false   no
yes enabled     yes
yes disabled    true    yes
yes disabled    false   no
yes disabled        no
no  enabled true    no
no  enabled false   no
no  enabled     no
no  disabled    true    no
no  disabled    false   no
no  disabled        no

if policy:enabled and istio-injection=enabled, default & sidecar.istio.io/inject: "true" is running sidecar. if policy:disabled and istio-injection=enabled, sidecar.istio.io/inject: "true" is running sidecar, or will not run sidecar.