Skip to content
SAF Advanced InSpec Profile Developer CourseSAF Advanced InSpec Profile Developer Course
MITRE InSpec Advanced Profile Developer Course
Course
Resources
Installation
  • Course

    • 1. Course Overview
      • 2. Review the Fundamentals
        • 3. Practice the Fundamentals
          • 4. Tools for Automation
            • 4. SAF CLI
              • 4.1. SAF CLI Capability
                • 4.2. View Summary of Results
                  • 4.3. Validate Results Against a Threshold
                • 5. Automate Security Testing
                  • 6. Explore InSpec Resources
                    • 7. Local vs Built-in Resources
                      • 8. Create a Custom Resource - The Git Example
                        • 9. Create a Custom Resource - The Docker Example
                          • 10. Writing Plural Resources
                            • 11. Dissecting Resources
                              • 12. Exercise - Develop your own resources
                                • 13. Add Your Resource to InSpec
                                  • 14. Custom Resource Examples from InSpec

                                  4. Tools for Automation

                                  June 7, 2022About 2 min

                                  On This Page
                                  • 4. SAF CLI
                                    • 4.1. SAF CLI Capability
                                    • 4.2. View Summary of Results
                                    • 4.3. Validate Results Against a Threshold

                                  # 4. SAF CLI

                                  The SAF supports tools that can help you automate your security validation. This includes the SAF CLI toolopen in new window.

                                  If you are working in the course development lab environment, this tool is provided for you. If not, reference the page for installation guidance.

                                  To update the SAF CLI version with npm, run:

                                  npm update -g @mitre/saf
                                  

                                  # 4.1. SAF CLI Capability

                                  In addition to the documentation site, you can view the SAF CLI capabilities by running:

                                  saf help
                                  

                                  This will output:

                                  The MITRE Security Automation Framework (SAF) Command Line Interface (CLI) brings together applications, techniques, libraries, and tools developed by MITRE and the security community to streamline security automation for systems and DevOps pipelines
                                  
                                  VERSION
                                    @mitre/saf/1.1.10 linux-x64 node-v16.14.2
                                  
                                  USAGE
                                    $ saf [COMMAND]
                                  
                                  TOPICS
                                    convert   [Normalize]    Convert security results from all your security tools between common data formats
                                    generate  [Validate]     Set pipeline thresholds
                                    harden    [Harden]       Implement security baselines using Ansible, Chef, and Terraform content: Visit
                                              https://saf.mitre.org/#/harden to explore and run hardening scripts
                                    plugins   List installed plugins.
                                    scan      [Validate]     Scan to get detailed security testing results: Visit https://saf.mitre.org/#/validate to explore and run inspec profiles
                                    validate  [Validate]     Verify pipeline thresholds
                                    view      [Visualize]    Identify overall security status and deep-dive to solve specific security defects
                                  
                                  COMMANDS
                                    convert ckl2poam  Translate DISA Checklist CKL file(s) to POA&M files
                                    harden            Visit https://saf.mitre.org/#/harden to explore and run hardening scripts
                                    heimdall          Run an instance of Heimdall Lite to visualize your Data
                                    help              Display help for saf.
                                    plugins           List installed plugins.
                                    scan              Visit https://saf.mitre.org/#/validate to explore and run inspec profiles
                                    summary           Get a quick compliance overview of an HDF file
                                    version
                                  

                                  You can get more information on a specific topic by running:

                                  saf [TOPIC] -h
                                  

                                  # 4.2. View Summary of Results

                                  Let's try out using the SAF CLI to understand our InSpec scan results.

                                  You can view a summary of the results by running:

                                  saf view summary -i path/to/output.json
                                  

                                  This will show the high level information about the test results:

                                  - profileName: my_nginx
                                    resultSets:
                                      - output.json
                                    compliance: 67
                                    passed:
                                      critical: 2
                                      high: 0
                                      medium: 0
                                      low: 0
                                      total: 2
                                    failed:
                                      critical: 1
                                      high: 0
                                      medium: 0
                                      low: 0
                                      total: 1
                                    skipped:
                                      critical: 0
                                      high: 0
                                      medium: 0
                                      low: 0
                                      total: 0
                                    error:
                                      critical: 0
                                      high: 0
                                      medium: 0
                                      low: 0
                                      total: 0
                                    no_impact:
                                      none: 0
                                      total: 0
                                  

                                  # 4.3. Validate Results Against a Threshold

                                  We can also generate a threshold for the results to be validated against. We will later see how this can be helpful in automated testing pipelines to indicate whether the pipeline should be considered a failure or a success.

                                  Create a new file in your my_nginx folder to specify the thresholds for acceptable test results.These file contents specify some basic thresholds, such as the minimum compliance level, minimum number of tests passed, and maximum number of tests failed.

                                  # threshold.yml file
                                  compliance:
                                    min: 50
                                  passed:
                                    total:
                                      min: 1
                                  failed:
                                    total:
                                      max: 2
                                  

                                  Threshold Files

                                  To make more specific or detailed thresholds, check out this documentation on generating theshold filesopen in new window.

                                  Now, validate the InSpec results against this threshold by running:

                                  saf validate threshold -i output.json -F threshold.yml
                                  

                                  This should result in:

                                  All validation tests passed
                                  

                                  If you try changing the threshold values to be more strict, you can see failing results. For example, update the threshold file like this:

                                  # threshold.yml file
                                  compliance:
                                    min: 50
                                  passed:
                                    total:
                                      min: 1
                                  failed:
                                    total:
                                      max: 0
                                  

                                  To receive this output:

                                  Error: failed.total.max: Received 1 > Expected 0
                                  

                                  Using Threshold Files for Automation

                                  Threshold files are used in CI to ensure minimum compliance levels and validate control severities and statuses using the saf validate threshold command.

                                  Edit this pageopen in new window
                                  Last update: 6/10/2022, 3:10:49 PM
                                  Contributors: Emily Rodriguez
                                  Prev
                                  3. Practice the Fundamentals
                                  Next
                                  5. Automate Security Testing
                                  Apache-2.0 | Copyright © 2022 - The MITRE Corporation
                                  Copyright © 2022 Aaron Lippold