6. Testing with Docker Containers

Learning Objectives

• Understand how to set up a Docker testing environment
• Learn to use Test Kitchen with Docker containers
• Execute and interpret InSpec tests in containers
• Analyze test results using Heimdall Lite

Prerequisites

• Docker or Podman installed and running
• Access to required container registries
• Basic understanding of command line operations

Environment Setup

Before running tests, configure your environment:

1. Ensure Docker/Podman is running
2. Authenticate with your container registry
3. Clone the test profile repository
4. Navigate to the profile's root directory
5. Run bundle install to install dependencies

Configure Test Kitchen

Set these environment variables:

export KITCHEN_LOCAL_YAML=kitchen.container.yml
export VANILLA_CONTAINER_IMAGE=registry.access.redhat.com/ubi8/ubi:8.9-1028
export HARDENED_CONTAINER_IMAGE=registry1.dso.mil/ironbank/redhat/ubi/ubi8
(optional) export INSPEC_CONTROL='SV-230222'

Here we will be assessing vanilla and hardened container images. While many containers are hosted by Docker themselves on their Docker Hub platform, organizations often deploy their own container registries. Red Hat makes their registry available at registry.access.redhat.com - we will be using their ubi8 image. Iron Bank is Platform One's hardened container image repository and is available at registry1.dso.mil - they publish a version of the ubi8 container that they claim is more hardened than Red Hat's baseline. Let's use Red Hat's ubi8 as the vanilla container image and Iron Bank's ubi8 as the hardened container image and validate Iron Bank's claim of having a more hardened image!

Running Through the Docker Test Suite

1. List the kitchen instances with:

bundle exec kitchen list

➜  redhat-enterprise-linux-8-stig-baseline git:(main*) bundle exec kitchen list
Instance       Driver  Provisioner  Verifier  Transport  Last Action    Last Error
vanilla-ubi8   Dokken  Dummy        Inspec    Dokken     <Not Created>  <None>
hardened-ubi8  Dokken  Dummy        Inspec    Dokken     <Not Created>  <None>

2. Create the kitchen instance:

bundle exec kitchen create vanilla

➜  redhat-enterprise-linux-8-stig-baseline git:(main*) bundle exec kitchen create vanilla
-----> Starting Test Kitchen (v3.5.1)
-----> Creating <vanilla-ubi8>...
      Creating kitchen sandbox at /Users/alippold/.dokken/kitchen_sandbox/de2da32d73-vanilla-ubi8
      Creating verifier sandbox at /Users/alippold/.dokken/verifier_sandbox/de2da32d73-vanilla-ubi8
      Building work image..
      Creating container de2da32d73-vanilla-ubi8
      Finished creating <vanilla-ubi8> (0m0.88s).
-----> Test Kitchen is finished. (0m1.77s)

3. Converge the kitchen instance:

bundle exec kitchen converge vanilla

➜  redhat-enterprise-linux-8-stig-baseline git:(main*) bundle exec kitchen converge vanilla
-----> Starting Test Kitchen (v3.5.1)
-----> Converging <vanilla-ubi8>...
      ...
      Finished converging <vanilla-ubi8> (0m0.00s).
-----> Test Kitchen is finished. (0m0.88s)

4. Run InSpec on the kitchen instance:

bundle exec kitchen verify vanilla

➜  redhat-enterprise-linux-8-stig-baseline git:(main*) bundle exec kitchen verify vanilla
-----> Starting Test Kitchen (v3.5.1)
-----> Verifying <vanilla-ubi8>...
      Loaded redhat-enterprise-linux-8-stig-baseline

Profile:   redhat-enterprise-linux-8-stig-baseline (redhat-enterprise-linux-8-stig-baseline)
Version:   1.12.0
Target:    docker://c4e89b7406dc0ebf8658fe90d6384d69885a7f07ab9bfbc91c85c64483868c44
Target ID: da39a3ee-5e6b-5b0d-b255-bfef95601890

  ×  SV-230222: RHEL 8 vendor packaged system security patches and updates must be installed and up to date. (4 failed)
...

Profile Summary: 0 successful controls, 1 control failure, 0 controls skipped
Test Summary: 0 successful, 4 failures, 0 skipped

5. Repeat these steps but replace vanilla with hardened.

Analyzing Results

• Results Location: ./spec/results/rhel-8_*
• Use Heimdall Lite to compare results:
  1. Load both the vanilla and hardened results
  2. Use the Comparison View (look for the toggle underneath the files list after selecting the hamburger menu in the top left) to compare the results sets and verify expected passes and failures
  3. Review corner cases for complete coverage

This isn't the error you're looking for, move along

The error below is just Test Kitchen telling you that not all of the controls of the profile passed.

>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Verify failed on instance <vanilla-ubi8>.  Please see .kitchen/logs/vanilla-ubi8.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration