Vault Kubernetes: Intermediate

Vault Kubernetes is an intermediate-level course that teaches you how to deploy, configure, and integrate Vault with Kubernetes.

Overview

The course is broken up into four sections:

Vault Kubernetes Deployment: This section covers how to customize the deployment of Vault on Kubernetes, including specifying input values, interacting with the cluster via the Vault CLI through Kubectl, and customizing the ingress.

Vault Kubernetes Configuration: This section covers how to configure the Kubernetes authentication engine with the Vault CLI.

Vault Kubernetes Integration: This section covers how to integrate Vault with Kubernetes, including using the Kubernetes Secrets Engine for authorization, service account authorization, and AppRole authentication.

Vault Kubernetes Application: This section covers how to use Vault in an application, including using the Vault CLI to create and manage secrets and using the Vault HTTP API to access secrets.

Section 1: Vault Kubernetes Deployment

Lesson 1 - Vault Kubernetes Deployment: Helm Chart Input Values

Prior to installing the Vault cluster on Kubernetes, one can specify different input values to customize the deployment. The most common and useful values are displayed below in the typical, “values.yaml” file format (populated with default values at the time of writing).

global:
  # TLS for end-to-end encrypted transport
  tlsDisable: true

injector:
  replicas: 1

  # agentImage sets the repo and tag of the Vault image to use for the Vault Agent containers.  This should be set to the official Vault image.  Vault 1.3.1+ is required.
  agentImage:
    repository: "hashicorp/vault"
    tag: "1.12.1"

  webhook:
    # namespaceSelector is the selector for restricting the webhook to only specific namespaces.
    # See https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#matching-requests-namespaceselector for more details.
    # Example:
    # namespaceSelector:
    #    matchLabels:
    #      sidecar-injector: enabled
    namespaceSelector: {}

server:
  image:
    repository: "hashicorp/vault"
    tag: "1.12.1"

  # Topology settings for server pods
  # ref: https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/
  # This should be either a multi-line string or YAML matching the topologySpreadConstraints array in a PodSpec.
  topologySpreadConstraints: []

  # Ingress allows ingress services to be created to allow external access from Kubernetes to access Vault pods. If deployment is on OpenShift, the following block is ignored. In order to expose the service, use the route section.
  ingress:
    enabled: false
    labels: {}
      # traffic: external
    # Optionally use ingressClassName instead of deprecated annotation.
    # See: https://kubernetes.io/docs/concepts/services-networking/ingress/#deprecated-annotation
    ingressClassName: ""

  # Run Vault in "HA" mode. There are no storage requirements unless the audit log persistence is required.  In HA mode Vault will configure itself to use Consul for its storage backend.  The default configuration provided will work the Consul Helm project by default.  It is possible to manually configure Vault to use a different HA backend.
  ha:
    enabled: false
    replicas: 3

    # Enables Vault's integrated Raft storage.  Unlike the typical HA modes where Vault's persistence is external (such as Consul), enabling Raft mode will create persistent volumes for Vault to store data according to the configuration under server.dataStorage.
    # The Vault cluster will coordinate leader elections and failovers internally.
    raft:
      # Enables Raft integrated storage
      enabled: false

# Vault UI
ui:
  # True if you want to create a Service entry for the Vault UI. serviceType can be used to control the type of service created. For example, setting this to "LoadBalancer" will create an external load balancer (for supported K8S installations) to access the UI.
  enabled: false

# secrets-store-csi-driver-provider-vault
csi:
  # True if you want to install a secrets-store-csi-driver-provider-vault daemonset.
  # Requires installing the secrets-store-csi-driver separately, see:
  # https://github.com/kubernetes-sigs/secrets-store-csi-driver#install-the-secrets-store-csi-driver
  # With the driver and provider installed, you can mount Vault secrets into volume similar to the Vault Agent injector, and you can also sync those secrets into Kubernetes secrets.
  enabled: false 

Lesson 2 - Vault Kubernetes Deployment: Kubectl Interactions

After deploying the Vault Cluster with the Helm chart, one can generally interact with the Vault Cluster containers with the following, “kubectl” command format:

kubectl -n <namespace> exec -ti vault-0 -- 'VAULT_TOKEN=<token> vault <command>'

Note this targets the Vault server container in the first scheduled pod. Each pod is enumerated as “<vault>-<n>” from 0 to the number of pods specified in the StatefulSet parameters. If you execute a modification on one server node in the cluster, then the gossip protocol (Raft) will ensure the change propagates across the shared configuration.

For example, suppose we wanted to initialize, unseal, and join with Raft a three-node Vault HA cluster with kubectl. You can fully implement this exercise instead of simulating it by modifying the deployment to be HA with Raft:

helm -n vault-system upgrade --set 'server.ha.enabled=true' --set 'server.ha.raft.enabled=true' vault

This will implicitly modify the number of server replicas in the cluster to three. The commands initialize, unseal, and join would appear like:

kubectl exec -ti vault-0 -- vault operator init
kubectl exec -ti vault-0 -- vault operator unseal <key 1>
kubectl exec -ti vault-0 -- vault operator unseal <key 2>
kubectl exec -ti vault-0 -- vault operator unseal <key 3>
kubectl exec vault-1 -- vault operator raft join http://vault-0.<namespace>.vault-internal:8200
kubectl exec -ti vault-1 -- vault operator unseal <key 1>
kubectl exec -ti vault-1 -- vault operator unseal <key 2>
kubectl exec -ti vault-1 -- vault operator unseal <key 3>
kubectl exec vault-2 -- vault operator raft join http://vault-0.<namespace>.vault-internal:8200
kubectl exec -ti vault-2 -- vault operator unseal <key 1>
kubectl exec -ti vault-2 -- vault operator unseal <key 2>
kubectl exec -ti vault-2 -- vault operator unseal <key 3>

Note the Vault Kubernetes service conforms to the normal DNS nomenclature. We also assume the default port is 8200 and no TLS (defaults). Recall that pod restarts require re-unsealing. This requirement is now clearer because the cluster is deployed as a StatefulSet. The Vault nodes must also join the cluster prior to unsealing, as one would expect. Otherwise, the nodes would be initialized and unsealing as an independent cluster because of the lack of data replication.

Lesson 3 - Vault Kubernetes Deployment: Customized Ingress

Although the Helm chart enables built-in ingress capabilities, one could also customize the ingress. Here is an example of ingress with Istio and TLS:

apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
  name: istio-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 443
        name: https
        protocol: HTTPS
      tls:
        mode: SIMPLE
        credentialName: <kubernetes tls secret>
      hosts:
        - <vault url for cluster>

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: vault
spec:
  hosts:
  - <vault url for cluster>
  gateways:
  - istio-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: vault
        port:
          number: 8200

Here is an example of general Ingress with a supported class (e.g., Nginx):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: vault-ingress
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: vault-ui
            port:
              number: 8200

Note this ingress functions for both the vault and vault-ui services. Also, note the prefix is “/” and not “/ui”. Once you have finished customizing the ingress, you can validate it by visiting the UI in a browser or by interacting with a Vault client binary executable after specifying the VAULT_ADDR environment variable value.

You have now learned more about deploying Vault on Kubernetes. You can customize the deployment closer to best practices for your environment, interact with the cluster via the Vault CLI through Kubectl, and customize the ingress.

[Back to the top]

Section 2: Vault Kubernetes Configuration

Lesson 1 - Vault Kubernetes Configuration: Engine Interaction with Vault CLI

We first need to configure the local Vault binary executable to interact as a client with the Vault server cluster. This is most easily accomplished with environment variables VAULT_ADDR=<vault url>:<vault port>, such as http://vault:8200, and VAULT_TOKEN=<vault token>. For now, set the token to be the root token obtained during unsealing, but understand we do this as a training exercise only, and the root token should generally only be used during initial administration.

Now you can execute Vault CLI commands outside of kubectl. Note you are also now targeting the entire Vault server cluster instead of a single pod (if you redeployed in HA). If you did not set up ingress in the Deployment section of this workshop, then you will need to continue using the Vault CLI through kubectl commands to interact with the pod containers.

We can enable and configure the Kubernetes authentication engine with the following commands (assuming the Kubernetes CA Certificate is stored in a file at the path, “ca.crt”)

vault auth enable kubernetes
vault write auth/kubernetes/config kubernetes_host=https://<kubernetes api url>:<kubernetes api port> kubernetes_ca_cert=@ca.crt

Notice how much faster that is relative to the UI. The Vault CLI can be somewhat unintuitive and sometimes contradicts other similar commands and/or the API, but generally is preferred to the UI once you acquire some experience with its usage.

Similarly, let us now enable and configure the KV2 secrets engine:

vault secrets enable kv-v2

As warned in the Introduction section, this will mount the engine at “secret” instead of “kv” by default (which is where the UI mounts by default). This is especially problematic given that “kv” is also the default mount path for the KV1 secrets engine. We can write and read secrets with “vault kv put” and “vault kv get” respectively:

vault kv put -mount=secret webapps/app_one <key>=<value> <key>=<value>
vault kv get -mount=secret webapps/app_one

Note this references a different mount path than in the Introduction section, and thus, you will not impact the KV2 actions from the Introduction section. If you wanted to access these new secrets, then the Vault authorization policy would need to be updated for paths prefixed with “secret/data”. However, you could insert secrets at a newer version by specifying the previous mount used in the Introduction section:

vault kv put -mount=kv webapps/app_one <key>=<value> <key>=<value>
vault kv get -mount=kv webapps/app_one

A common initial misconception is that one can modify a specific key value pair for a secret path by specifying the new key value pair during a “vault kv put”. However, a “vault kv put” is a complete secret update, and therefore an omission of an existing key value pair will delete that key value pair during the secret update.

You have now learned more about deploying Vault on Kubernetes. You can customize the deployment closer to best practices for your environment, interact with the cluster via the Vault CLI through Kubectl, and customize the ingress.

Lesson 2 - Vault Kubernetes Configuration: Policies

In the Introduction section, we wrote several policies for reading KV2 secrets. Now, we will actually discuss more in-depth how policies function. First, note the following capabilities within policies:

• create (POST/PUT) - Allows creating data at the given path. Very few parts of Vault distinguish between create and update, so most operations require both create and update capabilities. Parts of Vault that provide such a distinction are noted in the documentation.
• read (GET) - Allows reading the data at the given path.
• update (POST/PUT) - Allows changing the data at the given path. In most parts of Vault, this implicitly includes the ability to create the initial value at the path.
• patch (PATCH) - Allows partial updates to the data at a given path.
• delete (DELETE) - Allows deleting the data at the given path.
• list (LIST) - Allows listing values at the given path. Note that the keys returned by a list operation are not filtered by policies. Do not encode sensitive information in key names. Not all backends support listing.

Now let us examine a typical admin policy:

# admin.hcl
# Read system health check
path "sys/health" {
  capabilities = ["read", "sudo"]
}

# Create and manage ACL policies broadly across Vault

# List existing policies
path "sys/policies/acl" {
  capabilities = ["list"]
}

# Create and manage ACL policies
path "sys/policies/acl/*" {
  capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}

# Enable and manage authentication methods broadly across Vault

# Manage auth methods broadly across Vault
path "auth/*" {
  capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}

# Create, update, and delete auth methods
path "sys/auth/*" {
  capabilities = ["create", "update", "delete", "sudo"]
}

# List auth methods
path "sys/auth" {
  capabilities = ["read"]
}

# Enable and manage the key/value secrets engine at `secret/` path

# List, create, update, and delete key/value secrets
path "secret/*" {
  capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}

# Manage secrets engines
path "sys/mounts/*" {
  capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}

# List existing secrets engines.
path "sys/mounts" {
  capabilities = ["read"]
}

It is worth noting that multiple “path” specifications for policies are not documented explicitly, but rather one must examine the Vault API documentation for endpoints. These will implicitly document the associations between paths and the secret engines, authentication engines, Vault administration, etc. These API endpoints will conveniently correspond to the policy paths. We will explore the API more in-depth in the Advanced workshop lessons.

For now, let us create an “admin” policy with the above specifications. Save the above content to a file “admin.hcl”, and then:

vault policy write admin admin.hcl

We now have an admin policy that allows us to configure Vault according to needs, but more securely and less privileged than interacting via the root token.

Lesson 3 - Vault Kubernetes Configuration: Revoke Root Token

Let us now enable a simple authentication engine to utilize this new admin policy:

vault auth enable userpass

This is a simple user/password authentication that is less robust than most of the other authentication engines. In reality, we may want to use something like LDAP for this use case of human entity authentication, but this engine works well for the workshop lesson. Now, configure a simple user/password authentication pair and attach the previously written admin policy for authorization associated with this authentication.

vault write auth/userpass/users/<admin username> password=<suitable initial password> policies=admin

WARNING: This next step should not cause issues with the workshop lessons, but you may want to avoid executing it until you are completely finished with the exercises. Now, we should revoke the root token:

vault token revoke <root token>

This will explicitly revoke the root token. You can also implicitly revoke the current token by not providing a token argument. At this point, your actions will no longer be authenticated because you were utilizing the root token, which has now been revoked. We should re-authenticate with the user and password we just configured:

vault login -method=userpass username=<admin username> password=<suitable initial password>

This will authenticate with the userpass method and authorize with the admin policy for Vault interactions moving forward. Note that implicitly, a Vault token is also generated during the userpass method authentication.

You have now learned more about configuring Vault on Kubernetes. You can interact with the Vault cluster via a local client, enable/disable and configure authentication and secrets engines with the CLI, manage basic Vault policies to implement the principle of least privilege and associate authorization with authentication, and configure an administrator policy to enable revoking the root token.

[Back to the top]

Section 3: Vault Kubernetes Integration

Lesson 1 - Vault Kubernetes Integration: Kubernetes Secrets Engine Authorization

In this workshop introduction section, we enabled a Kubernetes secrets engine, configured it to manage Kubernetes cluster service account authentication and authorization, and dynamically generated a short-lived service account token. Since this workshop is within a lab environment and not a production-style environment, there is no RBAC in the cluster.

In a production environment, there would be RBAC for the cluster, and therefore, the Vault service account would need permissions to manage Kubernetes service accounts:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: k8s-full-secrets-with-labels
rules:
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["get"]
- apiGroups: [""]
  resources: ["serviceaccounts", "serviceaccounts/token"]
  verbs: ["create", "update", "delete"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["rolebindings", "clusterrolebindings"]
  verbs: ["create", "update", "delete"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["roles", "clusterroles"]
  verbs: ["bind", "escalate", "create", "update", "delete"]

Note the above role manifest is the most permissive possible and, therefore, should be modified for the principle of least privilege as necessary. We would then need to bind this role to the Vault service account (assuming standard namespace and service account names).

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: vault-token-creator-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: k8s-full-secrets-with-labels
subjects:
- kind: ServiceAccount
  name: vault
  namespace: vault-system

Note that in the Introduction section, we bound the Vault role configured in the Kubernetes authentication engine to the “webapp-one” service account. This service account, in reality, is primarily permitted for interactions with Vault and not with Kubernetes. The authentication provided by this Kubernetes secrets engine is for Kubernetes-authorized actions. Therefore, we also need to authorize the service account for Kubernetes actions and can do this with the previously set up Kubernetes secrets engine. Let us pretend that the “webapp-one” service account also requires the capability to list pods in the “default” namespace:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: list-pods
  namespace: default
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: webapp-one-capabilities
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: list-pods
subjects:
- kind: ServiceAccount
  name: webapp-one
  namespace: default

This authorization could then be provided with on-demand automated short-lived (low TTL) authentication with the Kubernetes secrets engine configuration we have previously configured.

Lesson 2 -  Vault Kubernetes Integration: Service Account Authorization

In this workshop Introduction section, we created Kubernetes service accounts (such as the ones beginning with the string “webapp”) to interface with the Kubernetes authentication engine by binding them to Vault roles in the engine. Since this workshop is within a lab environment and not a production-style environment, there is no RBAC in the cluster. In a production environment, there would be RBAC for the cluster, and therefore, the bound service accounts will also require a ClusterRoleBinding to authorize token reviews:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: role-tokenreview-binding
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:auth-delegator
subjects:
- kind: ServiceAccount
  name: <service account name>
  namespace: default

Note that we are using the “default” namespace for our Kubernetes applications, but that should be modified if otherwise.

Lesson 3 -  Vault Kubernetes Integration: AppRole Authentication

While the easiest and most direct method of authentication within Kubernetes is Kubernetes authentication, it is also possible to authenticate with the AppRole authentication engine. We first enable the engine:

vault auth enable approle

We then create a “webapp-one” role similar to the role in the Kubernetes authentication. We similarly attach the associated policy for reading the “webapps/webapp_one” secrets to this role:

vault write auth/approle/role/webapp-one policies="webapp-one-read"

Now, we have similarly enabled the capability for the authorized secrets retrieval to be authenticated by the Vault Agent with AppRole. However, for this to be later utilized by the Vault Agent, we will need to generate a RoleID and SecretID, and store them in a Kubernetes secret. The following command generates a RoleID and SecretID:

vault read auth/approle/role/webapp-one/role-id
vault write -f auth/approle/role/webapp-one/secret-id

These will then need to be stored in a Kubernetes secret for later usage:

kubectl create secret generic approle --from-literal=role_id=<role id> --from-literal=secret_id=<secret id>

It is rather intuitive to observe that this alternative is not as streamlined or secure as Kubernetes authentication. This is especially because AppRole is operating in “push” mode with a static SecretID.

You have now learned more about integrating Vault with Kubernetes. You can now manage Kubernetes service account authentication in a secure and dynamic manner, enable and configure the approle authentication, and both limit and permit the capabilities of the Vault service account for Kubernetes and Vault authentication.

[Back to the top]

Section 4: Vault Kubernetes Application

Lesson 1 - Vault Kubernetes Application: CSI Driver

In this workshop's Introduction section, we installed the secrets store CSI driver with Helm alongside the Vault cluster installation. We need to also configure Vault to enable CSI integration before we can actually utilize this driver plugin. We can do this most easily with a quick Helm deployment modification:

helm -n vault-system upgrade --set 'csi.enabled=true' vault

This will update the Vault cluster to enable the CSI plugin integration. Confirm that there is a new running pod named “vault-csi-provider-<sha hash>” in the “vault-system” namespace.

In the previous sections, we have already performed all the work necessary for the KV2 secrets retrieval and have now also finished configuring the CSI provider. We can immediately begin configuring CSI within Kubernetes. The following manifest will facilitate the CSI driver authenticating and retrieving the specified Vault secrets for “webapp-one”:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: vault-webapp-one
spec:
  provider: vault
  parameters:
    vaultAddress: 'http://vault.vault-system:8200'
    roleName: webapp-one
    objects: |
      - objectName: credential
        secretPath: 'secret/data/webapps/webapp_one'
        secretKey: <secret key>

Recall that these values we assigned in the manifest API schema are the Kubernetes authentication Vault role and the KV2 secret path and key. Note that this “secretPath” references the new mount from the previous Intermediate section. If you still only have one KV2 secrets engine mounted at “kv”, then the path will need to be prefixed with “kv/data” instead. The “objectName” is a purely descriptive name that we will use to refer to the secret within the context of Kubernetes. Now, we need to mount the secrets as a volume within the container within the pod. Patch the following into the pod spec from the deployment spec for “webapp-one”:

spec:
  containers:
  - image: httpd
    name: httpd
    volumeMounts:
    - name: secrets-store-csi
      mountPath: "/mnt/secrets"
      readOnly: true
  volumes:
  - name: secrets-store-csi
    csi:
      driver: secrets-store.csi.k8s.io
      readOnly: true
      volumeAttributes:
        secretProviderClass: vault-webapp-one

Note that if the service account authenticating for this plugin and driver was new, then that would need to be patched also. However, we are reusing the same service account as for the Vault Agent injection from the Introduction. Therefore, it is already part of the pod spec from the deployment spec. You can now easily verify the secret is part of the filesystem accessible from the container:

kubectl exec $(kubectl get pod -l app=webapp-one -o jsonpath="{.items[0].metadata.name}") --container httpd -- cat /mnt/secrets/credential

Note the file name in the mount path maps to the objectName from earlier. Therefore, each key within the secret path is mapped to a different file.

We can also sync the Vault secrets to Kubernetes secrets as an alternate ingestion procedure. We need to patch the SecretProviderClass “vault-webapp-one” from earlier to additionally sync the secret to a Kubernetes secret:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: vault-webapp-one
spec:
  secretObjects:
  - data:
    - key: <secret key>
      objectName: credential
    secretName: webapp-one-credentials
    type: Opaque

This maps the Vault KV2 secret key value pairs to key value pairs within the Kubernetes secret. The “objectName” maps to the “objectName” from the parameters in the “SecretProviderClass”, and the “key” value can be named the same as the Vault key for convenience. We now need to patch the pod spec for the “webapp-one” deployment to reference the Kubernetes secret as an environment variable:

spec:
  containers:
  - image: httpd
    name: httpd
    env:
    - name: PASSWORD
      valueFrom:
        secretKeyRef:
          name: vault-webapp-one
          key: <secret key>

The KV2 secret now propagates to the container as an environment variable. This is an interesting tradeoff relative to the procedure of mounting the secret as a file within a volume. The obvious advantage of syncing to a Kubernetes secret is that an in-process environment variable is much more secure than a mounted file but that the synced Kubernetes secret offers an additional attack vector that must be protected. It will be best to weigh these concerns for your specific environment.

Lesson 2 - Vault Kubernetes Application: AppRole Authentication

In this workshop's Introduction section, we demonstrated authenticating the Vault Agent injector with Kubernetes authentication through the associated Kubernetes service account. In the Intermediate Integration section, we demonstrated enabling the alternative AppRole authentication. Now, we will explain how to authenticate the Vault Agent in Kubernetes with AppRole.

Given the Kubernetes secret created in the Intermediate Integration section, the following patch to the pod spec will enable the approle authentication for the Vault Agent injector:

spec:
  template:
    metadata:
      annotations:
        vault.hashicorp.com/agent-inject: 'true'
        vault.hashicorp.com/agent-extra-secret: 'approle'
        vault.hashicorp.com/auth-type: 'approle'
        vault.hashicorp.com/auth-path: 'auth/approle'
        vault.hashicorp.com/auth-config-role-id-file-path: '/vault/custom/role_id'
        vault.hashicorp.com/auth-config-secret-id-file-path: '/vault/custom/secret_id'
        vault.hashicorp.com/role: 'webapp-one'

We override the default Kubernetes authentication settings with approle settings in the annotations. The “agent-extra-secret” annotation will mount the Kubernetes secret containing the RoleID and SecretID into “/vault/custom” with file names mapped to keys and content mapped to values. The “auth-config-role-id-file-path” is an intrinsic prefix for Vault Agent annotations, and the suffix is specific to indicate to the Vault Agent auto-auth that the role id is located in that file’s contents. The secret id is similarly indicated. Thus, the RoleID and SecretID from the Kubernetes secret are mounted to the container filesystem at specific locations and then read from the filesystem by the Vault Agent for authentication.

Note that this does operate in AppRole authentication push mode, which is inferior to pull mode as the secret id is not dynamically generated, nor is it response-wrapped. However, this is a limitation of auto-auth with the Vault Agent injector in Kubernetes.

You have now learned more about Vault with Kubernetes applications. You can now provide secrets to Kubernetes applications with the CSI driver and plugin both as a mounted volume file and as an environment variable referenced from a Kubernetes secret.

Contact us with any questions. We are here to help!