StorageClasses use the kube-dns entry for Heketi instead of the service IP

[fernandohackbart]

---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: cassandra-gluster
provisioner: kubernetes.io/glusterfs
parameters:
  resturl: "http://192.168.40.44:8080"
  restauthenabled: "false"
  restuser: "root"
  secretNamespace: "default"
  secretName: "heketi-secret"
  volumetype: "replicate:2"
allowVolumeExpansion: true 

[fernandohackbart]

---  
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: test-storageclass-gluster
provisioner: kubernetes.io/glusterfs
parameters:
  resturl: "http://heketi-cluster.default.svc.cluster.local:8080"
  restauthenabled: "true"
  restuser: "admin"
  restuserkey: "Welcome1"
  volumetype: "replicate:2"
allowVolumeExpansion: true

[fernandohackbart]

Jan 10 11:04:18 k8s-master1 journal: I0110 11:04:18.083033 1 event.go:218] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"test-claim", UID:"f1158dbc-f5f5-11e7-8bfc-525400e28760", APIVersion:"v1", ResourceVersion:"9487", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' Failed to provision volume with StorageClass "test-storageclass-gluster": create volume error: error creating volume Post http://heketi-cluster.default.svc.cluster.local:8080/volumes: dial tcp: lookup heketi-cluster.default.svc.cluster.local on 192.168.40.10:53: no such host

[fernandohackbart]

So the master is going to Heketi but is not using the kube-dns entries, but the external DNS server.

[fernandohackbart]

https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#stable-network-id