8. Security Benchmarks vs Traditional Software

Understanding Security Benchmarks

Key Differences from Traditional Software

Security benchmarks differ from traditional software development in several critical ways:

1. Completeness Requirement

   • Traditional Software: Can be released with partial features
   • Security Benchmarks: Must meet ALL requirements for a specific version

2. Version Specificity

   • Each benchmark version is a complete, standalone entity
   • InSpec profiles must match their corresponding benchmark version exactly

Development Approaches

Two common approaches to benchmark development:

• Micro Approach: Gradual, incremental development
• Macro Approach: Complete implementation in larger chunks

Both approaches are valid

Choose an approach or combination thereof that is based on your team's workflow preferences.

Version Control Best Practices

Working with Branches

• Never work directly on 'main'
• Always fork from the latest release
• Consider 'main' or 'development' branches as pre-release candidates

Practical Exercise

Try answering these questions:

1. Why can't security benchmarks be released incrementally?
2. How should you handle new requirements that arise between releases?
3. What branch strategy would you use for a new benchmark version?

Security benchmark validation is binary

The benchmark validation either meets all requirements or it doesn't. It being an accurate, representative assessment (and thereby useful to others) requires that it be the former.