When utilizing a fancy Kubernetes operator, you typically must orchestrate Jobs to carry out workload duties. Examples of Job implementations usually present trivial scripts written instantly within the manifest. In any reasonably-complex utility, nevertheless, figuring out deal with more-than-trivial scripts may be difficult.
In the previous, I’ve tackled this drawback by together with my scripts in an utility picture. This method works nicely sufficient, however it does have a downside. Anytime adjustments are required, I’m compelled to rebuild the applying picture to incorporate the revisions. This is a whole lot of time wasted, particularly when my utility picture takes a major period of time to construct. This additionally signifies that I’m sustaining each an utility picture and an operator picture. If my operator repository does not embody the applying picture, then I’m making associated adjustments throughout repositories. Ultimately, I’m multiplying the variety of commits I make, and complicating my workflow. Every change means I’ve to handle and synchronize commits and picture references between repositories.
Given these challenges, I wished to discover a option to maintain my Job scripts inside my operator’s code base. This approach, I may revise my scripts in tandem with my operator’s reconciliation logic. My purpose was to plan a workflow that may solely require me to rebuild the operator’s picture after I wanted to make revisions to my scripts. Fortunately, I take advantage of the Go programming language, which offers the immensely useful go:embed function. This permits builders to bundle textual content recordsdata in with their utility’s binary. By leveraging this function, I’ve discovered that I can keep my Job scripts inside my operator’s picture.
Embed Job script
For demonstration functions, my process script does not embody any precise enterprise logic. However, by utilizing an embedded script reasonably than writing the script instantly into the Job manifest, this method retains advanced scripts each well-organized and abstracted from the Job definition itself.
Here’s my easy instance script:
$ cat embeds/process.sh
#!/bin/sh
echo "Starting task script."
# Something difficult...
echo "Task complete."
Now to work on the operator’s logic.
Operator logic
Here’s the method inside my operator’s reconciliation:
- Retrieve the script’s contents
- Add the script’s contents to a ConfigMap
- Run the ConfigMap’s script inside the Job by
- Defining a quantity that refers back to the ConfigMap
- Making the amount’s contents executable
- Mounting the amount to the Job
Here’s the code:
// STEP 1: retrieve the script content material from the codebase.
//go:embed embeds/process.sh
var taskScript stringfunc (r *MyReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
ctxlog := ctrllog.FromContext(ctx)
myresource := &myresourcev1alpha.MyUseful resource{}
r.Get(ctx, req.NamespacedName, d)// STEP 2: create the ConfigMap with the script's content material.
configmap := &corev1.ConfigMap{}
err := r.Get(ctx, varieties.NamespacedName{Name: "my-configmap", Namespace: myresource.Namespace}, configmap)
if err != nil && apierrors.IsNotFound(err) {ctxlog.Info("Creating new ConfigMap")
configmap := &corev1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: "my-configmap",
Namespace: myresource.Namespace,
},
Data: map[string]string{
"task.sh": taskScript,
},
}err = ctrl.SetControllerReference(myresource, configmap, r.Scheme)
if err != nil {
return ctrl.Result{}, err
}
err = r.Create(ctx, configmap)
if err != nil {
ctxlog.Error(err, "Failed to create ConfigMap")
return ctrl.Result{}, err
}
return ctrl.Result{Requeue: true}, nil
}// STEP 3: create the Job with the ConfigMap connected as a quantity.
job := &batchv1.Job{}
err = r.Get(ctx, varieties.NamespacedName{Name: "my-job", Namespace: myresource.Namespace}, job)
if err != nil && apierrors.IsNotFound(err) {ctxlog.Info("Creating new Job")
configmapMode := int32(0554)
job := &batchv1.Job{
ObjectMeta: metav1.ObjectMeta{
Name: "my-job",
Namespace: myresource.Namespace,
},
Spec: batchv1.JobSpec{
Template: corev1.PodTemplateSpec{
Spec: corev1.PodSpec{
RestartPolicy: corev1.RestartPolicyNever,
// STEP 3a: outline the ConfigMap as a quantity.
Volumes: []corev1.Volume{{
Name: "task-script-volume",
VolumeSupply: corev1.VolumeSupply{
ConfigMap: &corev1.ConfigMapVolumeSupply{
LocalObjectReference: corev1.LocalObjectReference{
Name: "my-configmap",
},
DefaultMode: &configmapMode,
},
},
}},
Containers: []corev1.Container{
{
Name: "task",
Image: "busybox",
Resources: corev1.ResourceRequirements{
Requests: corev1.ResourceList{
corev1.ResourceCPU: *useful resource.NewMilliQuantity(int64(50), useful resource.DecimalSI),
corev1.ResourceMemory: *useful resource.NewScaledQuantity(int64(250), useful resource.Mega),
},
Limits: corev1.ResourceList{
corev1.ResourceCPU: *useful resource.NewMilliQuantity(int64(100), useful resource.DecimalSI),
corev1.ResourceMemory: *useful resource.NewScaledQuantity(int64(500), useful resource.Mega),
},
},
// STEP 3b: mount the ConfigMap quantity.
VolumeMounts: []corev1.VolumeMount{{
Name: "task-script-volume",
MountPath: "/scripts",
ReadOnly: true,
}},
// STEP 3c: run the volume-mounted script.
Command: []string{"/scripts/task.sh"},
},
},
},
},
},
}err = ctrl.SetControllerReference(myresource, job, r.Scheme)
if err != nil {
return ctrl.Result{}, err
}
err = r.Create(ctx, job)
if err != nil {
ctxlog.Error(err, "Failed to create Job")
return ctrl.Result{}, err
}
return ctrl.Result{Requeue: true}, nil
}// Requeue if the job will not be full.
if *job.Spec.Completions == 0 {
ctxlog.Info("Requeuing to wait for Job to complete")
return ctrl.Result{RequeueAfter: time.Second * 15}, nil
}ctxlog.Info("All done")
return ctrl.Result{}, nil
}
After my operator defines the Job, all that is left to do is anticipate the Job to finish. Looking at my operator’s logs, I can see every step within the course of recorded till the reconciliation is full:
2022-08-07T18:25:11.739Z INFO controller.myresource Creating new ConfigMap {"reconciler group": "myoperator.myorg.com", "reconciler kind": "MyResource", "name": "myresource-example", "namespace": "default"}
2022-08-07T18:25:11.765Z INFO controller.myresource Creating new Job {"reconciler group": "myoperator.myorg.com", "reconciler kind": "MyResource", "name": "myresource-example", "namespace": "default"}
2022-08-07T18:25:11.780Z INFO controller.myresource All finished {"reconciler group": "myoperator.myorg.com", "reconciler kind": "MyResource", "name": "myresource-example", "namespace": "default"}
Go for Kubernetes
When it involves managing scripts inside operator-managed workloads and functions, go:embed offers a helpful mechanism for simplifying the event workflow and abstracting enterprise logic. As your operator and its scripts develop into extra advanced, this type of abstraction and separation of considerations turns into more and more vital for the maintainability and readability of your operator.
