Cloud Design Dev Guides
Search…
Kubernetes
IBM Cloud Direct Link
IBM Cloud Object Storage
IBM Virtual Private Cloud
IBM Cloud Search and Tagging Examples
Cross-account sync with free IKS Cluster
In this guide I will show you how to sync ICOS bucket objects between accounts using Code Engine. Code Engine provides a platform to unify the deployment of all of your container-based applications on a Kubernetes-based infrastructure. The Code Engine experience is designed so that you can focus on writing code without the need for you to learn, or even know about, Kubernetes.
Code Engine is currently an experimental offering and all resources are deleted every 7 days.
Steps
Preparing Accounts
We will be using Cloud Shell to generate Service IDs and Object Storage credentials for both the source and destination accounts.
Source Account
We will create a service ID on the source account. A service ID identifies a service or application similar to how a user ID identifies a user. We can assign specific access policies to the service ID that restrict permissions for using specific services: in this case it gets read-only access to an IBM Cloud Object Storage bucket.
Create Service ID
1
$ ibmcloud iam service-id-create <name-of-your-service-id> --description "Service ID for read-only access to bucket" --output json
Copied!
Service ID Creation
Create Reader access policy for newly created service id
Now we will limit the scope of this service ID to have read only access to our source Object Storage bucket.
1
$ ibmcloud iam service-policy-create <Service ID> --roles Reader --service-name cloud-object-storage --service-instance <Service Instance GUID> --resource-type bucket --resource <bucket-name>
Copied!
Service Instance GUID - This is the GUID of the Cloud Object Storage instance. You can retrieve this with the command:
ibmcloud resource service-instance <name of icos instance>
Expected Output Example
Generate HMAC credentials tied to our service ID
In order for the Minio client to talk to each Object Storage instance it will need HMAC credentials (Access Key and Secret Key in S3 parlance).
1
$ ibmcloud resource service-key-create source-icos-service-creds Reader --instance-id <Service Instance GUID> --service-id <Service ID> --parameters '{"HMAC":true}'
Copied!
Save the access_key_id and secret_access_key as we will be using these with our Kubernetes job.
Create HMAC Credentials
Destination Account
We will create a service ID on the destination account. A service ID identifies a service or application similar to how a user ID identifies a user. We can assign specific access policies to the service ID that restrict permissions for using specific services: in this case it gets write access to an IBM Cloud Object Storage bucket.
Create Service ID
1
$ ibmcloud iam service-id-create <name-of-your-service-id> --description "Service ID for write access to bucket" --output json
Copied!
Expected Output Example
Create Reader access policy for newly created service id
Now we will limit the scope of this service ID to have read only access to our source Object Storage bucket.
1
$ ibmcloud iam service-policy-create <Service ID> --roles Writer --service-name cloud-object-storage --service-instance <Service Instance GUID> --resource-type bucket --resource <bucket-name>
Copied!
Service Instance GUID - This is the GUID of the Cloud Object Storage instance. You can retrieve this with the command:
ibmcloud resource service-instance <name of icos instance>Generate HMAC credentials tied to our service ID
We'll follow the same procedure as last time to generate the HMAC credentials, but this time on the destination account.
1
$ ibmcloud resource service-key-create destination-icos-service-creds Writer --instance-id <Service Instance GUID> --service-id <Service ID> --parameters '{"HMAC":true}'
Copied!
Save the access_key_id and secret_access_key as we will be using these with our Kubernetes job.
Create free IBM Cloud Kubernetes cluster
You can have 1 free cluster at a time in IBM Cloud Kubernetes Service, and each free cluster expires in 30 days. This will allow is to run the Minio sync as a Kubernetes Job.
1
$ ibmcloud ks cluster create classic --name <name of cluster>
Copied!
The cluster should be ready in about 10-15 minutes. You can verify the status of the creation by running the
ibmcloud ks clusters and seeing when State returns as normal:1
[email protected]:~$ ibmcloud ks clusters
2
OK
3
Name ID State Created Workers Location Version Resource Group Name Provider
4
freeikscluster bstcs2nf0bkrt2968nm0 normal 14 minutes ago 1 par01 1.17.9_1534 default classic
Copied!
Create Kubernetes secret for use with job
In order for our container to speak to the Cloud Object storage buckets we'll need to specify some environmental variables to be stored as Kubernetes secrets. The first step is to target our free cluster:
1
$ ibmcloud ks cluster config --cluster <name of cluster>
Copied!
With
kubectl now configured to talk to our cluster we can create our Kubernetes secrets.1
$ kubectl create secret generic <name of secret> --from-literal SOURCE_ACCESS_KEY=VALUE --from-literal SOURCE_SECRET_KEY=VALUE --from-literal SOURCE_REGION=VALUE --from-literal SOURCE_BUCKET=VALUE --from-literal DESTINATION_ACCESS_KEY=VALUE --from-literal DESTINATION_SECRET_KEY=VALUE --from-literal DESTINATION_REGION=VALUE --from-literal DESTINATION_BUCKET=VALUE
Copied!
Create Kubernetes Secret
Download Kubernetes Job file
1
$ wget https://gist.githubusercontent.com/greyhoundforty/b26d66b33f97d8368e3dd7869b7bbc5e/raw/926e7088cff7c0a991310065832d646e513890f1/job.yaml
Copied!
With the Job spec downloaded we can now submit the job to Kubernetes
1
$ kubectl create -f job.yaml
Copied!
In my testing I am only syncing a few times so by the time I check the Kubernetes pods, the job has already completed. Looking at the logs I am able to verify that the contents have been synced between the Object Storage buckets.
1
[email protected]:~$ kubectl get pods
2
NAME READY STATUS RESTARTS AGE
3
mc-icos-sync-test-c9zkq 0/1 Completed 0 102s
Copied!
Show pod logs