Background

Ingress主要是为了解决没有外部Load Balance设备的问题,在Kubernetes里面新建了一个Service之后,如果暴露给外部人员访问一直是一个问题,各大云厂商都有收费的LB解决方案,我们目前搭建的一套是完全基于开源解决方案,所以如果需要暴露Service,只有以下三种途径:

  • 使用NodePort,即针对每个服务暴露一堆端口,简单粗暴,但是看起来不好管理。
  • 使用Metallb,看起来还行,感觉和公有云上的LB差不多,但是每个Service都需要一个IP,哪有那么多IP可以分。。。
  • 使用Ingress,这英文不知道怎么翻译,但是看起来可以满足基本需求,通过Nginx作为反向代理转发Service服务至外网,配置方便,易管理。

Choose

Ingress实际上是定义在Kubernetes中的一种Resource,这种Resource具体是由什么组件来实现基于实现Ingress的Controller,官方的清单上这种Controller很多,详见:https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/

Nginx是官方推荐的首选,也是个人最熟悉的反向代理,目前共有两种实现,详细功能对比见下

Aspect or Featurekubernetes/ingress-nginxnginxinc/kubernetes-ingress with NGINXnginxinc/kubernetes-ingress with NGINX Plus
Fundamental
AuthorsKubernetes communityNGINX Inc and communityNGINX Inc and community
NGINX versionCustom NGINX build that includes several third-party modulesNGINX official mainline buildNGINX Plus
Commercial supportN/AN/AIncluded
Load balancing configuration
Merging Ingress rules with the same hostSupportedSupportedSupported
HTTP load balancing extensions – AnnotationsSee the supported annotationsSee the supported annotationsSee the supported annotations
HTTP load balancing extensions — ConfigMapSee the supported ConfigMap keysSee the supported ConfigMap keysSee the supported ConfigMap keys
TCP/UDPSupported via a ConfigMapSupported via a ConfigMap with native NGINX configurationSupported via a ConfigMap with native NGINX configuration
WebsocketSupportedSupported via an annotationSupported via an annotation
TCP SSL PassthroughSupported via a ConfigMapNot supportedNot supported
JWT validationNot supportedNot supportedSupported
Session persistenceSupported via a third-party moduleNot supportedSupported
Configuration templates *1See the templateSee the templatesSee the templates
Deployment
Command-line arguments *2See the argumentsSee the argumentsSee the arguments
TLS certificate and key for the default serverRequired as a command-line argument/ auto-generatedRequired as a command-line argumentRequired as a command-line argument
Helm chartSupportedSupportedSupported
Operational
Reporting the IP address(es) of the Ingress controller into Ingress resourcesSupportedSupportedSupported
Extended StatusSupported via a third-party moduleNot supportedSupported
Prometheus IntegrationSupportedSupportedSupported
Dynamic reconfiguration of endpoints (no configuration reloading)Supported with a third-party Lua moduleNot supportedSupported

两个nginx实现都测试了下,发现对于HTTP转发实现的都挺好的,而对于HTTPS转发,nginxinc/kubernetes-ingress with NGINX要更好一些,毕竟这个是nginx官方实现的,而另外一个是社区自己魔改的版本。

Solution

由于搭建了Rook Ceph,所以需要通过Ceph Dashboard查看Ceph集群的状态,同时也搭了一套Kubernetes Dashboard,目标是能够通过path转发至不同的Dashboard。由于访问该集群的域名仅有一个,所以针对Nginx的配置需要Merge,配置文件如下:

1、先定一个Master配置,所有的Minion都是挂在这个Master下面,即同一个ServerName下面不同Path。

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: master-ingress
  namespace: kube-system
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.org/mergeable-ingress-type: "master"
spec:
  tls:
  - hosts:
    - api.dev.example.com
  secretName: default-secret
  rules:
  - host: api.dev.example.com

2、定义Ceph Dashboard的Minion配置。

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ceph-ingress
  namespace: rook-ceph
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.org/ssl-services: "rook-ceph-service"
    nginx.org/mergeable-ingress-type: "minion"
spec:
  rules:
  - host: api.dev.example.com
    http:
      paths:
      - path: /ceph/
        backend:
          serviceName: rook-ceph-service
          servicePort: 8443

3、定义Kubernetes Dashboard的Minion配置。

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: dashboard-ingress
  namespace: kube-system
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.org/ssl-services: "dashboard-service"
    nginx.org/mergeable-ingress-type: "minion"
spec:
  rules:
  - host: api.dev.example.com
    http:
      paths:
      - path: /dashboard/
        backend:
          serviceName: dashboard-service
          servicePort: 443

自此,实现了Nginx跨Namespace的HTTPS转发,以后如果有其他的Service需要转发,仅需要用Minion方式进行配置,并且默认就是HTTPS加密。

发表评论