Secrets Management for Container Scanning
This page outlines best practices for managing secrets when integrating container scanning into CI/CD pipelines.
Overview
Container scanning often requires access to Kubernetes clusters, container registries, and other sensitive resources. Proper secrets management is critical for:
- Protecting cluster access credentials
- Securing container registry authentication
- Managing service account tokens securely
- Implementing the principle of least privilege
- Preventing secrets from being exposed in logs or outputs
Types of Secrets
When integrating container scanning, you'll typically need to manage these types of secrets:
- Kubernetes Authentication:
- Kubeconfig files
- Service account tokens
-
API server certificates
-
Container Registry Authentication:
- Registry usernames and passwords
- Registry access tokens
-
Docker config.json files
-
Scanning Credentials:
- Profile repository access tokens
- Compliance API keys
-
Report repository credentials
-
Temporary Credentials:
- Short-lived service account tokens
- Limited-scope access tokens
- Ephemeral certificates
Best Practices
General Security Practices
- Never store secrets in code repositories
- Limit secret access to only the jobs and stages that require them
- Use environment variables to inject secrets into pipelines
- Implement secret rotation policies
- Always validate inputs to prevent injection attacks
- Audit secret usage regularly
GitHub Actions
- Use GitHub Secrets for sensitive data:
| - name: Setup Kubernetes access
run: |
echo "${{ secrets.KUBE_CONFIG }}" | base64 -d > kubeconfig.yaml
export KUBECONFIG=kubeconfig.yaml
|
- Limit secret exposure with environment isolation:
| jobs:
scan:
environment: production
env:
SCANNER_TOKEN: ${{ secrets.SCANNER_TOKEN }}
|
- Use OIDC for cloud provider authentication:
| - name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
role-to-assume: ${{ secrets.AWS_ROLE }}
aws-region: us-east-1
|
GitLab CI/CD
- Use GitLab CI/CD Variables:
| before_script:
- echo "$KUBE_CONFIG" | base64 -d > kubeconfig.yaml
- export KUBECONFIG=kubeconfig.yaml
|
- Use masked variables to prevent accidental exposure:
| # In GitLab CI/CD settings, mark variables as "Masked"
|
- Limit variable scope by environment:
| # In GitLab CI/CD settings, set variables with a specific environment scope
|
- Use CI/CD file variables for per-pipeline secrets:
| variables:
SCANNER_TOKEN: $CI_JOB_TOKEN
|
Kubernetes Secret Management
- Create time-limited service accounts:
| # Create short-lived token (Kubernetes 1.22+)
TOKEN=$(kubectl create token scanner-sa --duration=15m)
|
- Create least-privilege RBAC roles:
| apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: scanner-role
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create"]
resourceNames: ["specific-pod-name"]
|
- Use namespaced resources to limit access scope:
| # Create rolebinding in specific namespace
kubectl create rolebinding scanner-binding \
--role=scanner-role \
--serviceaccount=default:scanner-sa \
-n target-namespace
|
- Include Job/Pipeline IDs in resource names for traceability:
| metadata:
name: scanner-sa-${CI_JOB_ID}
annotations:
ci-job: "${CI_JOB_ID}"
pipeline: "${CI_PIPELINE_ID}"
|
Temporary Credentials Workflow
Here's a recommended workflow for managing temporary credentials in scanning pipelines:
- Create a temporary service account:
| kubectl create serviceaccount scanner-sa-${CI_JOB_ID} -n ${NAMESPACE}
|
- Create a restricted role with specific pod access:
| cat <<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: scanner-role-${CI_JOB_ID}
namespace: ${NAMESPACE}
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create"]
resourceNames: ["${POD_NAME}"]
EOF
|
- Bind the role to the service account:
| kubectl create rolebinding scanner-binding-${CI_JOB_ID} \
--role=scanner-role-${CI_JOB_ID} \
--serviceaccount=${NAMESPACE}:scanner-sa-${CI_JOB_ID} \
-n ${NAMESPACE}
|
- Generate a short-lived token:
| TOKEN=$(kubectl create token scanner-sa-${CI_JOB_ID} -n ${NAMESPACE} --duration=15m)
|
- Use the token for scanning:
| # Create scanner kubeconfig
cat > scanner-kubeconfig.yaml <<EOF
apiVersion: v1
kind: Config
clusters:
- cluster:
server: ${KUBE_SERVER}
certificate-authority-data: ${KUBE_CA_DATA}
name: k8s-cluster
contexts:
- context:
cluster: k8s-cluster
user: scanner-user
namespace: ${NAMESPACE}
name: scanner-context
current-context: scanner-context
users:
- name: scanner-user
user:
token: ${TOKEN}
EOF
|
- Clean up after scanning:
| kubectl delete rolebinding scanner-binding-${CI_JOB_ID} -n ${NAMESPACE}
kubectl delete role scanner-role-${CI_JOB_ID} -n ${NAMESPACE}
kubectl delete serviceaccount scanner-sa-${CI_JOB_ID} -n ${NAMESPACE}
|
Managing Secrets in Container Registry Authentication
When scanning containers that require pulling from private registries:
- Use docker config secrets:
| apiVersion: v1
kind: Secret
metadata:
name: registry-credentials
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: <base64-encoded-docker-config>
|
- Reference secrets in pod definition:
| spec:
imagePullSecrets:
- name: registry-credentials
|
- Securely inject credentials in CI/CD:
| before_script:
- echo "$REGISTRY_PASSWORD" | docker login -u "$REGISTRY_USERNAME" --password-stdin $REGISTRY_URL
|
Sanitizing Output and Avoiding Secret Leakage
To prevent accidental exposure of secrets in logs and reports:
- Sanitize scan output:
| # Replace sensitive data in outputs
sed -i 's/sensitive-data-pattern/[REDACTED]/g' scan-results.json
|
- Use log filters in CI/CD pipelines:
| # GitLab CI/CD filter pattern
variables:
SECURE_LOG_LEVEL: info
CI_DEBUG_TRACE: "false"
|
- Set token variables as secrets in your CI/CD system:
- GitHub Actions:
${{ secrets.TOKEN_NAME }}
-
GitLab CI/CD: Masked variables
-
Avoid using debug output for workflows with secrets:
| # Only enable debug on non-sensitive stages
debug:
if: github.event.inputs.enable_debug == 'true'
|
- Use credential helpers when possible:
- AWS:
aws-cli
credential helper
- GCP:
gcloud
credential helper
- Azure:
az
credential helper
Automated Secret Rotation
For long-running scanning pipelines or environments, implement automated secret rotation:
- Create periodic jobs to rotate credentials:
| # GitLab CI/CD schedule for rotating credentials
secret-rotation:
stage: maintenance
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
script:
- ./rotate-credentials.sh
|
- Store rotation timestamps:
| # Record when credentials were last rotated
echo "Last rotated: $(date)" > rotation-timestamp.txt
|
- Implement graceful credential handover:
| # Create new credentials before invalidating old ones
NEW_TOKEN=$(create-new-token)
update-token-consumers "$NEW_TOKEN"
invalidate-old-token
|