Docker Testing Suite

Docker Suite Setup

1. Make sure Docker or Podman is running
2. Login to your docker registry
3. Clone the repository
4. Go into the profile repository root dir
5. bundle install
6. export KITCHEN_LOCAL_YAML=kitchen.container.yml (uses the kitchen-dokken driver)
7. export VANILLA_CONTAINER_IMAGE=registry.access.redhat.com/ubi8/ubi:8.9-1028
8. export HARDENED_CONTAINER_IMAGE=registry1.dso.mil/ironbank/redhat/ubi/ubi8
9. (optional) export INSPEC_CONTROL='SV-230222'

Running Through the Docker Test Suite

10. List the kitchen instances with: bundle exec kitchen list

➜  redhat-enterprise-linux-8-stig-baseline git:(main*)bundle exec kitchen list
Instance       Driver  Provisioner  Verifier  Transport  Last Action    Last Error
vanilla-ubi8   Dokken  Dummy        Inspec    Dokken     <Not Created>  <None>
hardened-ubi8  Dokken  Dummy        Inspec    Dokken     <Not Created>  <None>

11. Create the kitchen instance: bundle exec kitchen create vanilla

-----> Starting Test Kitchen (v3.5.1)
-----> Creating <vanilla-ubi8>...
      Creating kitchen sandbox at /Users/alippold/.dokken/kitchen_sandbox/de2da32d73-vanilla-ubi8
      Creating verifier sandbox at /Users/alippold/.dokken/verifier_sandbox/de2da32d73-vanilla-ubi8
      Building work image..
      Creating container de2da32d73-vanilla-ubi8
      Finished creating <vanilla-ubi8> (0m0.88s).
-----> Test Kitchen is finished. (0m1.77s)

12. Converge the kitchen instance: bundle exec kitchen converge vanilla

➜  redhat-enterprise-linux-8-stig-baseline git:(main*)bundle exec kitchen converge vanilla
-----> Starting Test Kitchen (v3.5.1)
-----> Converging <vanilla-ubi8>...
      ...
      Finished converging <vanilla-ubi8> (0m0.00s).
-----> Test Kitchen is finished. (0m0.88s)

13. Run InSpec on the kitchen instance: bundle exec kitchen verify vanilla

-----> Starting Test Kitchen (v3.5.1)
-----> Verifying <vanilla-ubi8>...
      Loaded redhat-enterprise-linux-8-stig-baseline

Profile:   redhat-enterprise-linux-8-stig-baseline (redhat-enterprise-linux-8-stig-baseline)
Version:   1.12.0
Target:    docker://c4e89b7406dc0ebf8658fe90d6384d69885a7f07ab9bfbc91c85c64483868c44
Target ID: da39a3ee-5e6b-5b0d-b255-bfef95601890

  ×  SV-230222: RHEL 8 vendor packaged system security patches and updates must be installed and up to date. (4 failed)
...

Profile Summary: 0 successful controls, 1 control failure, 0 controls skipped
Test Summary: 0 successful, 4 failures, 0 skipped

This is not the Error Your Looking For, move along...

The error below is just Test Kitchen telling you that not all of the Contrls in the profile passed.

>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Verify failed on instance <vanilla-ubi8>.  Please see .kitchen/logs/vanilla-ubi8.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

14. For steps that apply to making updates, patches, and updates to the profile, see the next section, Updating the Profile.
15. Your InSpec scan results are located in the ./spec/results/ directory, named ./spec/results/ubi-8_*.
16. Use Heimdall Lite to load both the hardened and vanilla results to ensure your changes and updates, "failed as expected and passed as expected and covered your corner cases."