How to access one GKE cluster from another GKE cluster

We can set up communication between two different GKE clusters using the workload identity method.

What is Workload Identity

Workload Identity allows workloads in your GKE clusters to impersonate Identity and Access Management (IAM) service accounts to access Google Cloud services. Workload Identity is enabled by default on Autopilot clusters.

Configure applications to use Workload Identity

After enabling Workload Identity, you should configure your applications to authenticate to Google Cloud using Workload Identity before you migrate the applications to the new node pools.

You must assign a #Kubernetes service account to the application and configure that #Kubernetes service account to act as an IAM service account.

The following steps show you how to configure your applications to use Workload Identity.

1. Create a namespace to use for the #Kubernetes service account. You can also use the default namespace or any existing namespace.

$ kubectl create namespace NAMESPACE

2. Create a #Kubernetes service account for your application to use. You can also use the default #Kubernetes service account in the default or any existing namespace.

$ kubectl create serviceaccount KSA_NAME \

— namespace NAMESPACE

Replace the following:

• KSA_NAME: the name of your new #Kubernetes service account.

• NAMESPACE: the name of the #Kubernetes namespace for the service account.

3. Create an IAM service account for your application or use an existing IAM service account instead. You can use any IAM service account in any project in your organization. For Config Connector, apply the IAMServiceAccount object for your selected service account.

$ gcloud iam service-accounts create GSA_NAME \

— project=GSA_PROJECT

Replace the following:

• GSA_NAME: the name of the new IAM service account.

• GSA_PROJECT: the project ID of the Google Cloud project for your IAM service account.

4. Ensure that your IAM service account has the roles you need. You can grant additional roles using the following command:

$ gcloud projects add-iam-policy-binding PROJECT_ID \

— member “serviceAccount:GSA_NAME@GSA_PROJECT.iam.gserviceaccount.com” \

— role “ROLE_NAME”

Replace the following:

• PROJECT_ID: your Google Cloud project ID.

• GSA_NAME: the name of your IAM service account.

• GSA_PROJECT: the project ID of the Google Cloud project of your IAM service account.

• ROLE_NAME: the IAM role to assign to your service account, like roles/spanner.viewer.

5. Allow the #Kubernetes service account to impersonate the IAM service account by adding an

IAM policy binding between the two service accounts. This binding allows the #Kubernetes service account to act as the IAM service account.

$ gcloud iam service-accounts add-iam-policy-binding GSA_NAME@GSA_PROJECT.iam.gserviceaccount.com \

— role roles/iam.workloadIdentityUser \

— member “serviceAccount:**PROJECT_ID.**svc.id.goog[NAMESPACE/KSA_NAME]”

Replace the following:

• PROJECT_ID: your Google Cloud project ID.

• GSA_NAME: the name of your IAM service account.

• GSA_PROJECT: the project ID of the Google Cloud project of your IAM service account.

• ROLE_NAME: the IAM role to assign to your service account, like roles/container.admin.

6. Annotate the #Kubernetes service account with the email address of the IAM service account.

$ kubectl annotate serviceaccount KSA_NAME \

— namespace NAMESPACE \

iam.gke.io/gcp-service-account=GSA_NAME@GSA_PROJECT.iam.gserviceaccount.com

7. Create a clusterRole and a clusterRoleBinding

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRoleBinding

metadata:

name: fabric-sa

roleRef:

apiGroup: rbac.authorization.k8s.io

kind: ClusterRole

name: cluster-admin

subjects:

• kind: ServiceAccount

name: fabric-sa

namespace: default

8. Using service account tokens to connect with the API server

a. Get information about your #Kubernetes secret object. Secrets are used to store access credentials.

$ kubectl get secret — namespace={namespace}

b. Get details of the service account token.

$ kubectl get secret default-token-2mfqv — namespace={namespace} -o yaml

c. Decode and set the base64-encoded token.

$ kubectl config set-credentials sa-user — token=$(kubectl get secret <secret_name> -o jsonpath= {.data.token} | base64 -d)

$ kubectl config set-context sa-context — user=sa-user

d. Connect to the API server.

$ curl -k -H “Authorization:Bearer {token}” <API server URL>