Science and technology

Creating a Source-to-Image construct pipeline in OKD

In the primary three articles on this collection, we explored the final requirements of a Source-to-Image (S2I) system and prepared and tested an surroundings particularly for a Go (Golang) software. This S2I construct is ideal for native improvement or sustaining a builder picture with a code pipeline, however when you have entry to an OKD or OpenShift cluster (or Minishift), you possibly can arrange the whole workflow utilizing OKD BuildConfigs, not solely to construct and keep the builder picture but in addition to make use of the builder picture to create the appliance picture and subsequent runtime picture robotically. This means, the pictures could be rebuilt robotically when downstream pictures change and might set off OKD deploymentConfigs to redeploy functions working from these pictures.

Step 1: Build the builder picture in OKD

As in native S2I utilization, step one is to create the builder picture to construct the GoHelloWorld take a look at software that we will reuse to compile different Go-based functions. This first construct step can be a Docker construct, identical to earlier than, that pulls the Dockerfile and S2I scripts from a Git repository to construct the picture. Therefore, these recordsdata have to be dedicated and obtainable in a public Git repo (or you should use the companion GitHub repo for this text).

Note: OKD BuildConfigs don’t require that supply Git repos are public. To use a non-public repo, you should arrange deploy keys and hyperlink the keys to a builder service account. This is just not troublesome, however for simplicity’s sake, this train will use a public repository.

Create a picture stream for the builder picture

The BuildConfig will create a builder picture for us to compile the GoHelloWorld app, however first, we’d like a spot to retailer the picture. In OKD, that place is a picture stream.

An image stream and its tags are like a manifest or checklist of associated pictures and picture tags. It serves as an abstraction layer that permits you to reference a picture, even when the picture adjustments. Think of it as a group of aliases that reference particular pictures and, as pictures are up to date, robotically factors to the brand new picture model. The picture stream is nothing besides these aliases—simply metadata about actual pictures saved in a registry.

An picture stream could be created with the oc create imagestream <title> command, or it may be created from a YAML file with oc create -f <filename>. Either means, a brand-new picture stream is a small placeholder object that’s empty till it’s populated with picture references, both manually (who desires to do issues manually?) or with a BuildConfig.

Our golang-builder picture stream seems to be like this:

# imageStream-golang-builder.yaml
---
apiVersion
: picture.openshift.io/v1
variety
: ImageStream
metadata
:
  technology
: 1
  title
: golang-builder
spec
:
  lookupPolicy
:
    native
: false

Other than a reputation, and a (principally) empty spec, there’s nothing actually there.

Note: The lookupPolicy has to do with permitting Kubernetes-native parts to resolve picture stream references since picture streams are OKD-native and never part of the Kubernetes core. This matter is out of scope for this text, however you possibly can learn extra about the way it works in OKD’s documentation Using Image Streams with Kubernetes Resources.

Create a picture stream for the builder picture and its progeny.

$ oc create -f imageStream-golangBuilder.yaml

# Check the ImageStream
$ oc get imagestream golang-builder
NAME                                            DOCKER REPO                                                      TAGS      UPDATED
imagestream.picture.openshift.io/golang-builder   docker-registry.default.svc:5000/golang-builder/golang-builder

Note that the newly created picture stream has no tags and has by no means been up to date.

Create a BuildConfig for the builder picture

In OKD, a BuildConfig describes easy methods to construct container pictures from a selected supply and triggers for after they construct. Don’t be thrown off by the language—simply as you may say you construct and re-build the identical picture from a Dockerfile, however in actuality, you could have constructed a number of pictures, a BuildConfig builds and rebuilds the identical picture, however in actuality, it creates a number of pictures. (And all of the sudden the rationale for picture streams turns into a lot clearer!)

Our builder picture BuildConfig describes easy methods to construct and re-build our builder picture(s). The BuildConfig’s core is made up of 4 necessary elements:

  1. Build supply
  2. Build technique
  3. Build output
  4. Build triggers

The construct supply (predictably) describes the place the factor that runs the construct comes from. The builds described by the golang-builder BuildConfig will use the Dockerfile and S2I scripts we created beforehand and, utilizing the Git-type construct supply, clone a Git repository to get the recordsdata to do the builds.

supply:
  kind
: Git
  git
:
    ref
: grasp
    uri
: https://github.com/clcollins/golang-s2i.git

The construct technique describes what the construct will do with the supply recordsdata from the construct supply. The golang-builder BuildConfig mimics the docker construct we used beforehand to construct our native builder picture by utilizing the Docker-type construct technique.

technique:
  kind
: Docker
  dockerStrategy
:

The dockerStrategy construct kind tells OKD to construct a Docker picture from the Dockerfile contained within the supply specified by the construct supply.

The construct output tells the BuildConfig what to do with the ensuing picture. In this case, we specify the picture stream we created above and a tag to present to the picture. As with our native construct, we’re tagging it with golang-builder:1.12 as a reference to the Go model inherited from the father or mother picture.

output:
  to
:
    variety
: ImageStreamTag
    title
: golang-builder:1.12

Finally, the BuildConfig defines a set of construct triggers—occasions that can trigger the picture to be rebuilt robotically. For this BuildConfig, a change to the BuildConfig configuration or an replace to the upstream picture (golang:1.12) will set off a brand new construct.

triggers:
- kind
: ConfigChange
- imageChange
:
    kind
: ImageChange

Using the builder image BuildConfig from the GitHub repo as a reference (or simply utilizing that file), create a BuildConfig YAML file and use it to create the BuildConfig.

$ oc create -f buildConfig-golang-builder.yaml

# Check the BuildConfig
$ oc get bc golang-builder
NAME             TYPE      FROM         LATEST
golang-builder   Docker    Git@grasp   1

Because the BuildConfig included the “ImageChange” set off, it instantly kicks off a brand new construct. You can test that the construct was created with the oc get builds command.

# Check the Builds
$ oc get builds
NAME               TYPE      FROM          STATUS     STARTED              DURATION
golang-builder-1   Docker    Git@8eff001   Complete   About a minute in the past   13s

While the construct is working and after it has accomplished, you possibly can view its logs with oc logs -f <construct title> and see the Docker construct output as you’d domestically.

$ oc logs -f golang-builder-1-build
Step 1/11 : FROM docker.io/golang:1.12
 ---> 7ced090ee82e
Step 2/11 : LABEL maintainer "Chris Collins <[email protected]>"
 ---> 7ad989b765e4
Step three/11 : ENV CGO_ENABLED zero GOOS linux GOCACHE /tmp STI_SCRIPTS_PATH /usr/libexec/s2i SOURCE_DIR /go/src/app
 ---> 2cee2ce6757d

<...>

If you didn’t embody any construct triggers (or didn’t have them in the proper place), your construct could not begin robotically. You can manually kick off a brand new construct with the oc start-build command.

$ oc start-build golang-builder

# Or, if you wish to robotically tail the construct log
$ oc start-build golang-builder --follow

When the construct completes, the ensuing picture is tagged and pushed to the built-in picture registry and the picture stream is up to date with the brand new picture’s info. Check the picture stream with the oc get imagestream command to see that the brand new tag exists.

$ oc get imagestream golang-builder
NAME             DOCKER REPO                                                      TAGS      UPDATED
golang-builder   docker-registry.default.svc:5000/golang-builder/golang-builder   1.12      33 seconds in the past

Step 2: Build the appliance picture in OKD

Now that we’ve got a builder picture for our Golang functions created and saved inside OKD, we will use this builder picture to compile all of our Go apps. First on the block is the instance GoHelloWorld app from our local build example. GoHelloWorld is an easy Go app that simply outputs Hello World! when it is run.

Just as we did within the native instance with the s2i construct command, we will inform OKD to make use of our builder picture and S2I to construct the appliance picture for GoHelloWorld, compiling the Go binary from the supply code within the GoHelloWorld GitHub repository. This could be performed with a BuildConfig with a sourceStrategy construct.

Create a picture stream for the appliance picture

First issues first, we have to create a picture stream to handle the picture created by the BuildConfig. The picture stream is rather like the golang-builder picture stream, simply with a special title. Create it with oc create is or utilizing a YAML file from the GitHub repo.

$ oc create -f imageStream-goHelloWorld-appimage.yaml
imagestream.picture.openshift.io/go-hello-world-appimage created

Create a BuildConfig for the appliance picture

Just as we did with the builder picture BuildConfig, this BuildConfig will use the Git supply choice to clone our supply code from the GoHelloWorld repository.

supply:
  kind
: Git
  git
:
    uri
: https://github.com/clcollins/goHelloWorld.git

Instead of utilizing a DockerTechnique construct to create a picture from a Dockerfile, this BuildConfig will use the sourceStrategy definition to construct the picture utilizing S2I.

technique:
  kind
: Source
  sourceStrategy
:
    from
:
      variety
: ImageStreamTag
      title
: golang-builder:1.12

Note the from: hash in sourceStrategy. This tells OKD to make use of the golang-builder:1.12 picture we created beforehand for the S2I construct.

The BuildConfig will output to the brand new appimage picture stream we created, and we’ll embody config- and image-change triggers to kick off new builds robotically if something updates.

output:
  to
:
    variety
: ImageStreamTag
    title
: go-hello-world-appimage:1.zero
triggers
:
- kind
: ConfigChange
- imageChange
:
    kind
: ImageChange

Once once more, create a BuildConfig or use the one from the GitHub repo.

$ oc create -f buildConfig-goHelloWorld-appimage.yaml

The new construct exhibits up alongside the golang-builder construct and, as a result of image-change triggers are specified, the construct begins instantly.

$ oc get builds
NAME                        TYPE      FROM          STATUS     STARTED          DURATION
golang-builder-1            Docker    Git@8eff001   Complete   eight minutes in the past    13s
go-hello-world-appimage-1   Source    Git@99699a6   Running    44 seconds in the past

If you need to watch the construct logs, use the oc logs -f command. Once the appliance picture construct completes, it’s pushed to the picture stream we specified, then the brand new picture stream tag is created.

$ oc get is go-hello-world-appimage
NAME                      DOCKER REPO                                                               TAGS      UPDATED
go-hello-world-appimage   docker-registry.default.svc:5000/golang-builder/go-hello-world-appimage   1.zero       10 minutes in the past

Success! The GoHelloWorld app was cloned from supply into a brand new picture and compiled and examined utilizing our S2I scripts. We can use the picture as-is however, as with our native S2I builds, we will do higher and create a picture with simply the brand new Go binary in it.

Step three: Build the runtime picture in OKD

Now that the appliance picture has been created with a compiled Go binary for the GoHelloWorld app, we will use one thing known as chain builds to imitate after we extracted the binary from our native software picture and created a brand new runtime picture with simply the binary in it.

Create a picture stream for the runtime picture

Once once more, step one is to create a picture stream picture for the brand new runtime picture.

# Create the ImageStream
$ oc create -f imageStream-goHelloWorld.yaml
imagestream.picture.openshift.io/go-hello-world created

# Get the ImageStream
$ oc get imagestream go-hello-world
NAME             DOCKER REPO                                                      TAGS      UPDATED
go-hello-world   docker-registry.default.svc:5000/golang-builder/go-hello-world

Chain builds

Chain builds are when a number of BuildConfigs are used to compile software program or assemble artifacts for an software, and people artifacts are saved and utilized by a subsequent BuildConfig to generate a runtime picture with out re-compiling the code.

Create a BuildConfig for the runtime picture

The runtime BuildConfig makes use of the DockerTechnique construct to construct the picture from a Dockerfile—the identical factor we did with the builder picture BuildConfig. This time, nonetheless, the supply is just not a Git supply, however a Dockerfile supply.

What is the Dockerfile supply? It’s an inline Dockerfile! Instead of cloning a repo with a Dockerfile in it and constructing that, we specify the Dockerfile within the BuildConfig. This is very applicable with our runtime Dockerfile as a result of it is simply three traces lengthy.

supply:
  kind
: Dockerfile
  dockerfile
: |-
    FROM scratch
    COPY app /app
    ENTRYPOINT ["/app"]
  pictures
:
  - from
:
      variety
: ImageStreamTag
      title
: go-hello-world-appimage:1.zero
    paths
:
    - sourcePath
: /go/src/app/app
      vacation spotDir
: "."

Note that the Dockerfile within the Dockerfile supply definition above is identical because the Dockerfile we used within the third article on this collection after we constructed the slim GoHelloWorld picture domestically utilizing the binary we extracted with the S2I save-artifacts script.

Something else to notice: scratch is a reserved phrase in Dockerfiles. Unlike different FROM statements, it doesn’t outline an precise picture, however moderately that the primary layer of this picture can be nothing. It is outlined with variety: DockerImage however doesn’t have a registry or group/namespace/venture string. Learn extra about this habits on this glorious container best practices reference.

The pictures part of the Dockerfile supply describes the supply of the artifact(s) for use within the construct; on this case, from the appimage generated earlier. The paths subsection describes the place to get the binary (i.e., within the /go/src/app listing of the app picture, get the app binary) and the place to reserve it (i.e., within the present working listing of the construct itself: “.”). This permits the COPY app /app to seize the binary from the present working listing and add it to /app within the runtime picture.

Note: paths is an array of supply and the vacation spot path pairs. Each entry within the checklist consists of a supply and vacation spot. In the instance above, there is only one entry as a result of there’s only a single binary to repeat.

The Docker technique is then used to construct the inline Dockerfile.

technique:
  kind
: Docker
  dockerStrategy
:

Once once more, it’s output to the picture stream created earlier and contains construct triggers to robotically kick off new builds.

output:
  to
:
    variety
: ImageStreamTag
    title
: go-hello-world:1.zero
triggers
:
- kind
: ConfigChange
- imageChange
:
    kind
: ImageChange

Create a BuildConfig YAML or use the runtime BuildConfig from the GitHub repo.

$ oc create -f buildConfig-goHelloWorld.yaml
buildconfig.construct.openshift.io/go-hello-world created

If you watch the logs, you will discover step one is FROM scratch, which confirms we’re including the compiled binary to a clean picture.

$ oc logs -f pod/go-hello-world-1-build
Step 1/5 : FROM scratch
 --->
Step 2/5 : COPY app /app
 ---> 9e70e6c710f8
Removing intermediate container 4d0bd9cef0a7
Step three/5 : ENTRYPOINT /app
 ---> Running in 7a2dfeba28ca
 ---> d697577910fc

<...>

Once the construct is accomplished, test the picture stream tag to validate that the brand new picture was pushed to the registry and picture stream was up to date.

$ oc get imagestream go-hello-world
NAME             DOCKER REPO                                                      TAGS      UPDATED
go-hello-world   docker-registry.default.svc:5000/golang-builder/go-hello-world   1.zero       four minutes in the past

Make a be aware of the DOCKER REPO string for the picture. It can be used within the subsequent part to run the picture.

Did we create a tiny, binary-only picture?

Finally, let’s validate that we did, certainly, construct a tiny picture with simply the binary.

Check out the picture particulars. First, get the picture’s title from the picture stream.

$ oc describe imagestream go-hello-world
Name:                   go-hello-world
Namespace:              golang-builder
Created:                42 minutes in the past
Labels:                 <none>
Annotations:            <none>
Docker Pull Spec:       docker-registry.default.svc:5000/golang-builder/go-hello-world
Image Lookup:           native=false
Unique Images:          1
Tags:                   1

1.zero
  no spec tag

  * docker-registry.default.svc:5000/golang-builder/go-hello-world@sha256:eb11e0147a2917312f5e0e9da71109f0cb80760e945fdc1e2db6424b91bc9053
      13 minutes in the past

The picture is listed on the backside, described with the SHA hash (e.g., sha256:eb11e0147a2917312f5e0e9da71109f0cb80760e945fdc1e2db6424b91bc9053; yours can be totally different).

Get the small print of the picture utilizing the hash.

$ oc describe picture sha256:eb11e0147a2917312f5e0e9da71109f0cb80760e945fdc1e2db6424b91bc9053
Docker Image:   docker-registry.default.svc:5000/golang-builder/go-hello-world@sha256:eb11e0147a2917312f5e0e9da71109f0cb80760e945fdc1e2db6424b91bc9053
Name:           sha256:eb11e0147a2917312f5e0e9da71109f0cb80760e945fdc1e2db6424b91bc9053
Created:        15 minutes in the past
Annotations:    picture.openshift.io/dockerLayersOrder=ascending
                picture.openshift.io/manifestBlobStored=true
                openshift.io/picture.managed=true
Image Size:     1.026MB
Image Created:  15 minutes in the past
Author:         <none>
Arch:           amd64
Entrypoint:     /app
Working Dir:    <none>
User:           <none>
Exposes Ports:  <none>
Docker Labels:  io.openshift.construct.title=go-hello-world-1
                io.openshift.construct.namespace=golang-builder
Environment:    PATH=/usr/native/sbin:/usr/native/bin:/usr/sbin:/usr/bin:/sbin:/bin
                OPENSHIFT_BUILD_NAME=go-hello-world-1
                OPENSHIFT_BUILD_NAMESPACE=golang-builder

Notice the picture dimension, 1.026MB, is strictly as we would like. The picture is a scratch picture with simply the binary inside it!

Run a pod with the runtime picture

Using the runtime picture we simply created, let’s create a pod on-demand and run it and validate that it nonetheless works.

This nearly by no means occurs in Kubernetes/OKD, however we are going to run a pod, only a pod, by itself.

$ oc run -it  go-hello-world --image=docker-registry.default.svc:5000/golang-builder/go-hello-world:1.zero --restart=Never
Hello World!

Everything is working as anticipated—the picture runs and outputs “Hello World!” simply because it did within the earlier, native S2I builds.

By creating this workflow in OKD, we will use the golang-builder S2I picture for any Go software. This builder picture is in place and constructed for another functions, and it’ll auto-update and rebuild itself anytime the upstream golang:1.12 picture adjustments.

New apps could be constructed robotically utilizing the S2I construct by creating a series construct technique in OKD with an appimage BuildConfig to compile the supply and the runtime BuildConfig to create the ultimate picture. Using the construct triggers, any change to the supply code within the Git repo will set off a rebuild via the whole pipeline, rebuilding the appimage and the runtime picture robotically.

This is a good way to take care of up to date pictures for any software. Paired with an OKD deploymentConfig with a picture construct set off, long-running functions (e.g., webapps) can be robotically redeployed when new code is dedicated.

Source-to-Image is a perfect solution to develop builder pictures to construct and compile Go functions in a repeatable means, and it simply will get higher when paired with OKD BuildConfigs.

Most Popular

To Top