Come consentire a kubelet di comunicare con apiserver utilizzando HTTPS? v0.19

Question

ho schierato apiserver sul nodo master (core01) con i seguenti conf:

core01> /opt/bin/kube-apiserver \ 
    --insecure_bind_address=127.0.0.1 \ 
    --insecure_port=8080 \ 
    --kubelet_port=10250 \ 
    --etcd_servers=http://core01:2379,http://core02:2379,http://core03:2379 \ 
    --service-cluster-ip-range=10.1.0.0/16 \ 
    --allow_privileged=false \ 
    --logtostderr=true \ 
    --v=5 \ 
    --tls-cert-file="/var/run/kubernetes/apiserver_36kr.pem" \ 
    --tls-private-key-file="/var/run/kubernetes/apiserver_36kr.key" \ 
    --client-ca-file="/var/run/kubernetes/cacert.pem" \ 
    --kubelet-certificate-authority="/var/run/kubernetes/cacert.pem" \ 
    --kubelet-client-certificate="/var/run/kubernetes/kubelet_36kr.pem" \ 
    --kubelet-client-key="/var/run/kubernetes/kubelet_36kr.key" 

sul nodo minion (CORE02), posso chiamata API da HTTPS:

core02> curl https://core01:6443/api/v1/nodes --cert /var/run/kubernetes/kubelet_36kr.pem --key /var/run/kubernetes/kubelet_36kr.key 
> GET /api/v1/nodes HTTP/1.1 
> Host: core01:6443 
> User-Agent: curl/7.42.1 
> Accept: */* 
> 
< HTTP/1.1 200 OK 
< Content-Type: application/json 
< Date: Sat, 27 Jun 2015 15:33:50 GMT 
< Content-Length: 1577 
< 
{ 
    "kind": "NodeList", 
    "apiVersion": "v1", 
    "metadata": { 
    "selfLink": "/api/v1/nodes", 
    "resourceVersion": "510078" 
    }, .... 

Tuttavia, posso non iniziare Kubelet su questo seguace. Si lamenta sempre senza credenziali.

Come posso farlo funzionare? C'è qualche documento sul master < -> autenticazione di comunicazione minion? Potresti per favore darmi le migliori pratiche?

---

Cordiali saluti, Il comando è la seguente: log

core02> /opt/bin/kubelet \ 
    --logtostderr=true \ 
    --v=0 \ 
    --api_servers=https://core01:6443 \ 
    --address=127.0.0.1 \ 
    --port=10250 \ 
    --allow-privileged=false \ 
    --tls-cert-file="/var/run/kubernetes/kubelet_36kr.pem" \ 
    --tls-private-key-file="/var/run/kubernetes/kubelet_36kr.key" 

kubelet è la seguente:

W0627 23:34:03.646311 3004 server.go:460] Could not load kubeconfig file /var/lib/kubelet/kubeconfig: stat /var/lib/kubelet/kubeconfig: no such file or directory. Trying auth path instead. 
W0627 23:34:03.646520 3004 server.go:422] Could not load kubernetes auth path /var/lib/kubelet/kubernetes_auth: stat /var/lib/kubelet/kubernetes_auth: no such file or directory. Continuing with defaults. 
I0627 23:34:03.646710 3004 manager.go:127] cAdvisor running in container: "/system.slice/sshd.service" 
I0627 23:34:03.647292 3004 fs.go:93] Filesystem partitions: map[/dev/sda9:{mountpoint:/ major:0 minor:30} /dev/sda4:{mountpoint:/usr major:8 minor:4} /dev/sda6:{mountpoint:/usr/share/oem major:8 minor:6}] 
I0627 23:34:03.648234 3004 manager.go:156] Machine: {NumCores:1 CpuFrequency:2399996 MemoryCapacity:1046294528 MachineID:29f94a4fad8b31668bd219ca511bdeb0 SystemUUID:4F4AF929-8BAD-6631-8BD2-19CA511BDEB0 BootID:fa1bea28-675e-4989-ad86-00797721a794 Filesystems:[{Device:/dev/sda9 Capacity:18987593728} {Device:/dev/sda4 Capacity:1031946240} {Device:/dev/sda6 Capacity:113229824}] DiskMap:map[8:0:{Name:sda Major:8 Minor:0 Size:21474836480 Scheduler:cfq} 8:16:{Name:sdb Major:8 Minor:16 Size:1073741824 Scheduler:cfq}] NetworkDevices:[{Name:eth0 MacAddress:52:54:71:f6:fc:b8 Speed:0 Mtu:1500} {Name:flannel0 MacAddress: Speed:10 Mtu:1472}] Topology:[{Id:0 Memory:1046294528 Cores:[{Id:0 Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]}]} 
I0627 23:34:03.649934 3004 manager.go:163] Version: {KernelVersion:4.0.5 ContainerOsVersion:CoreOS 695.2.0 DockerVersion:1.6.2 CadvisorVersion:0.15.1} 
I0627 23:34:03.651758 3004 plugins.go:69] No cloud provider specified. 
I0627 23:34:03.651855 3004 docker.go:289] Connecting to docker on unix:///var/run/docker.sock 
I0627 23:34:03.652877 3004 server.go:659] Watching apiserver 
E0627 23:34:03.748954 3004 reflector.go:136] Failed to list *api.Pod: the server has asked for the client to provide credentials (get pods) 
E0627 23:34:03.750157 3004 reflector.go:136] Failed to list *api.Node: the server has asked for the client to provide credentials (get nodes) 
E0627 23:34:03.751666 3004 reflector.go:136] Failed to list *api.Service: the server has asked for the client to provide credentials (get services) 
I0627 23:34:03.758158 3004 plugins.go:56] Registering credential provider: .dockercfg 
I0627 23:34:03.856215 3004 server.go:621] Started kubelet 
E0627 23:34:03.858346 3004 kubelet.go:662] Image garbage collection failed: unable to find data for container/
I0627 23:34:03.869739 3004 kubelet.go:682] Running in container "/kubelet" 
I0627 23:34:03.869755 3004 server.go:63] Starting to listen on 127.0.0.1:10250 
E0627 23:34:03.899877 3004 event.go:185] Server rejected event '&api.Event{TypeMeta:api.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:api.ObjectMeta{Name:"core02.13eba23275ceda25", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:util.Time{Time:time.Time{sec:0, nsec:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*util.Time)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil)}, InvolvedObject:api.ObjectReference{Kind:"Node", Namespace:"", Name:"core02", UID:"core02", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"starting", Message:"Starting kubelet.", Source:api.EventSource{Component:"kubelet", Host:"core02"}, FirstTimestamp:util.Time{Time:time.Time{sec:63571016043, nsec:856189989, loc:(*time.Location)(0x1ba6120)}}, LastTimestamp:util.Time{Time:time.Time{sec:63571016043, nsec:856189989, loc:(*time.Location)(0x1ba6120)}}, Count:1}': 'the server has asked for the client to provide credentials (post events)' (will not retry!) 
I0627 23:34:04.021297 3004 factory.go:226] System is using systemd 
I0627 23:34:04.021790 3004 factory.go:234] Registering Docker factory 
I0627 23:34:04.022241 3004 factory.go:89] Registering Raw factory 
I0627 23:34:04.144065 3004 manager.go:946] Started watching for new ooms in manager 
I0627 23:34:04.144655 3004 oomparser.go:183] oomparser using systemd 
I0627 23:34:04.145379 3004 manager.go:243] Starting recovery of all containers 
I0627 23:34:04.293020 3004 manager.go:248] Recovery completed 
I0627 23:34:04.343829 3004 status_manager.go:56] Starting to sync pod status with apiserver 
I0627 23:34:04.343928 3004 kubelet.go:1683] Starting kubelet main sync loop. 
E0627 23:34:04.457765 3004 event.go:185] Server rejected event '&api.Event{TypeMeta:api.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:api.ObjectMeta{Name:"core02.13eba232995c8213", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:util.Time{Time:time.Time{sec:0, nsec:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*util.Time)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil)}, InvolvedObject:api.ObjectReference{Kind:"Node", Namespace:"", Name:"core02", UID:"core02", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeReady", Message:"Node core02 status is now: NodeReady", Source:api.EventSource{Component:"kubelet", Host:"core02"}, FirstTimestamp:util.Time{Time:time.Time{sec:63571016044, nsec:452676115, loc:(*time.Location)(0x1ba6120)}}, LastTimestamp:util.Time{Time:time.Time{sec:63571016044, nsec:452676115, loc:(*time.Location)(0x1ba6120)}}, Count:1}': 'the server has asked for the client to provide credentials (post events)' (will not retry!) 
E0627 23:34:04.659874 3004 event.go:185] Server rejected event '&api.Event{TypeMeta:api.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:api.ObjectMeta{Name:"core02.13eba232a599cf8c", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:util.Time{Time:time.Time{sec:0, nsec:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*util.Time)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil)}, InvolvedObject:api.ObjectReference{Kind:"Node", Namespace:"", Name:"core02", UID:"core02", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeReady", Message:"Node core02 status is now: NodeReady", Source:api.EventSource{Component:"kubelet", Host:"core02"}, FirstTimestamp:util.Time{Time:time.Time{sec:63571016044, nsec:658020236, loc:(*time.Location)(0x1ba6120)}}, LastTimestamp:util.Time{Time:time.Time{sec:63571016044, nsec:658020236, loc:(*time.Location)(0x1ba6120)}}, Count:1}': 'the server has asked for the client to provide credentials (post events)' (will not retry!) 

Answer 1

Le prime due righe del file di log kubelet in realtà indicare il problema di fondo - non stai specificando alcuna credenziale del client per consentire a kubelet di connettersi al master.

Gli argomenti --tls-cert-file e --tls-private-key-file per il kubelet vengono utilizzati per configurare il server http sul kubelet (se non specificato, il kubelet will generate a self-signed certificate for its https endpoint). Questo certificato/coppia di chiavi non vengono utilizzati come certificato client presentato al master per l'autenticazione.

Per specificare le credenziali, esistono due opzioni: un file kubeconfig e un file kubernetes_auth. Più tardi è deprecato, quindi consiglio di utilizzare un file kubeconfig.

All'interno del file kubeconfig è necessario specificare un token al portatore o un certificato client che il kubelet deve presentare al server di apis. È inoltre possibile specificare il certificato CA per l'apiserver (se si desidera che la connessione sia protetta) o indicare al kubelet di ignorare il controllo del certificato presentato dal server di apis. Poiché hai certificati per l'apiserver, ti consigliamo di aggiungere il certificato CA al file kubeconfig.

Il file kubeconfig dovrebbe essere simile:

apiVersion: v1 
kind: Config 
users: 
- name: kubelet 
    user: 
    client-certificate-data: <base64-encoded-cert> 
    client-key-data: <base64-encoded-key> 
clusters: 
- name: local 
    cluster: 
    certificate-authority-data: <base64-encoded-ca-cert> 
contexts: 
- context: 
    cluster: local 
    user: kubelet 
    name: service-account-context 
current-context: service-account-context 

per generare il Base64 codificati client cert, si dovrebbe essere in grado di eseguire qualcosa di simile cat /var/run/kubernetes/kubelet_36kr.pem | base64. Se non si ha a portata di mano il certificato CA, è possibile sostituire lo certificate-authority-data: <base64-encoded-ca-cert> con insecure-skip-tls-verify: true.

Se si inserisce questo file allo /var/lib/kubelet/kubeconfig, dovrebbe essere prelevato automaticamente. In caso contrario, è possibile utilizzare l'argomento --kubeconfig per specificare un percorso personalizzato.

Answer 2

Come detto Robert Bailey, il problema principale ha a che fare con le credenziali del client per connettere il master ("Impossibile caricare il file kubeconfig/var/lib/kubelet/kubeconfig ..."). Invece di creare manualmente il file kubeconfig, ho scelto di generarlo usando lo strumento kubectl.

Esempio da docs:

$ kubectl config set-credentials myuser --username=myusername --password=mypassword 
$ kubectl config set-cluster local-server --server=http://localhost:8080 
$ kubectl config set-context default-context --cluster=local-server --user=myuser 
$ kubectl config use-context default-context 
$ kubectl config set contexts.default-context.namespace mynamespace 

Quei comandi genereranno un file di configurazione in ~/.kube/config

Controllare il risultato con:

$ kubectl config view 

Poi ho solo creato un collegamento simbolico all'interno di/var/lib/kubelet (posizione predefinita) al mio file di configurazione:

$ cd /var/lib/kubelet 
$ sudo ln -s ~/.kube/config kubeconfig 

Questo ha funzionato per me. Spero che funzioni anche per te.

Answer 3

Tutto il merito per jnoller mentre specifica i seguenti comandi su questo issue.
ha appena fatto un errore di battitura, perché lui corre kubectl config set-credentials due volte

Questo è simile a Robert Bailey 's accettato risposta, tranne che non è necessario in Base64 nulla e rende facile script.

kubectl config set-cluster default-cluster --server=https://${MASTER} \ 
    --certificate-authority=/path/to/ca.pem 

kubectl config set-credentials default-admin \ 
    --certificate-authority=/path/to/ca.pem \ 
    --client-key=/path/to/admin-key.pem \ 
    --client-certificate=/path/to/admin.pem  

kubectl config set-context default-system --cluster=default-cluster --user=default-admin 
kubectl config use-context default-system 

La configurazione risultante generata in ~/.kube/config assomiglia a questo:

apiVersion: v1 
clusters: 
- cluster: 
    certificate-authority: /etc/kubernetes/certs/ca.crt 
    server: https://kubernetesmaster 
    name: default-cluster 
contexts: 
- context: 
    cluster: default-cluster 
    user: default-admin 
    name: default-system 
current-context: default-system 
kind: Config 
preferences: {} 
users: 
- name: default-admin 
    user: 
    client-certificate: /etc/kubernetes/certs/server.crt 
    client-key: /etc/kubernetes/certs/server.key