Implementing a Complete DevSecOps Toolchain with Open Source

The widespread adoption of cloud systems introduces significant security challenges. 

Attackers continuously exploit vulnerabilities in these systems, often targeting misconfigurations, unpatched software, and exposed sensitive data. Techniques like SQL injection, cross-site scripting (XSS), and exploiting known vulnerabilities in application code are commonly used. Phishing attacks and social engineering can also compromise cloud environments, leading to unauthorized access and data breaches.

Examples of cloud breaches from application and cloud vulnerabilities include:

• The 2017 Equifax breach: This incident, which exposed the personal information of 147 million individuals, was caused by an unpatched Apache Struts vulnerability (CVE-2017-5638).

• The 2020 SolarWinds cyberattack: which targeted the software company SolarWinds and affected numerous organizations, including several U.S. government agencies. The attackers compromised the SolarWinds Orion software by injecting malicious code into a software update. This supply chain attack allowed the attackers to gain access to the networks of thousands of organizations using the compromised software.

The Equifax and SolarWinds breaches underscore the critical importance of integrating DevSecOps tools into the Software Development Lifecycle (SDLC), so developers can resolve vulnerabilities before production to prevent such incidents. 

In this article, we’ll review the top open source DevSecOps tools that can empower developers to secure everything they code. 

Our suggested open source DevSecOps tools, and how to implement them

As we’ve covered, each tool addresses a different aspect of application security, from code analysis to runtime protection, enabling a holistic approach to safeguarding your applications. In this section, we’ll cover open source tools for the full code-to-cloud security toolchain, including:

• Static Application Security Testing (SAST) to scan custom code

• Software Composition Analysis (SCA) to scan open source code

• Secrets detection to surface hardcoded secrets like passwords, cloud token, or API keys

• Container scanning to surface container vulnerabilities in dockerfiles, the registry, and runtime

• IaC security to scan infrastructure statically

• Cloud Security Posture Management to scan infrastructure in runtime

• Dynamic Application Security Testing to scan web apps in runtime

Implementing SAST with Semgrep

SAST is a great way to provide developers immediate feedback on the security of the code they write, so they can resolve potential vulnerabilities before production.

Our favorite tool is Semgrep, which is exceptionally accurate across programming languages and provides wide integrations across different CI/CD pipelines.



Semgrep is recommended for its ability to identify vulnerabilities in custom source code across multiple programming languages. Its support for customizable rules allows for tailored security checks specific to your codebase. Semgrep excels in detecting a wide range of insecure coding patterns that could create vulnerabilities, including SQL injection, cross-site scripting (XSS), and code injection, making it a versatile and powerful tool for ensuring code security.

Where Semgrep can be integrated into your SDLC

• IDEs: Integrate Semgrep with popular IDEs like Visual Studio Code, IntelliJ IDEA, and PyCharm to provide real-time security feedback as developers write code. Semgrep IDE Extensions
• Source Code Managers: Use Semgrep with source code managers such as GitHub, GitLab, and Bitbucket to scan code during pull requests and code reviews. Semgrep  - Connect a source code manager
• Build Systems: Automate Semgrep scans in CI/CD pipelines using build systems like Jenkins, CircleCI, and Travis CI to ensure every code change is analyzed for security issues. Semgrep - Add Semgrep to CI

Basic Semgrep Configuration Guidance

Installation

Install Semgrep via pip:

pip install semgrep

Rule Customization:

Semgrep uses rules written in YAML files. You can create custom rules or use the extensive library of predefined rules available on the Semgrep registry.

Example of a simple custom rule to detect the use of eval in JavaScript:

rules:
  - id: no-eval
    pattern: eval(...)
    message: "Avoid using eval as it can lead to security vulnerabilities."
    languages: [javascript]
    severity: WARNING

To run Semgrep with custom rules:

semgrep --config path/to/custom-rules.yaml path/to/your/code

To use predefined rules from the Semgrep registry:

semgrep --config "p/ci" path/to/your/code

Integrating into CI/CD:

Add Semgrep to your CI/CD pipeline configuration (e.g., in a GitHub Actions workflow):

name: Semgrep
on: [push, pull_request]
jobs:
  semgrep:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: '3.x'
      - name: Install Semgrep
        run: pip install semgrep
      - name: Run Semgrep
        run: semgrep --config p/ci

For more detailed guidance and examples, refer to additional materials on Semgrep and SAST:

• Semgrep documentation

• Optimize Semgrep rules for SAST

• Top ten SAST tools



Implementing Secrets Detection with Gitleaks

Secrets detection tools scan code repositories for sensitive data like API keys, passwords, and private keys, preventing accidental exposure.By identifying and removing hardcoded secrets, these tools prevent unauthorized access and potential data breaches. Integrating secrets detection into pre-commit hooks and CI/CD pipelines ensures sensitive information is not exposed in the code.

We prefer Gitleaks and Trufflehog for their simplicity and deep scanning capabilities, respectively. In this article, we’ll focus on implementing GitLeaks.



Gitleaks is a tool designed to scan git repositories for sensitive information, such as hardcoded passwords, API keys, and private keys, by using customizable regular expression (regex) detection patterns.

Where Gitleaks can be integrated into your SDLC

• IDEs: Although Gitleaks does not have direct integration documentation, you can manually run Gitleaks scans from within your IDE terminal to check for secrets before committing code.

• Source Code Managers: Use Gitleaks to manually scan repositories in GitHub, GitLab, and Bitbucket during pull requests and code reviews.

• Pre-commit Hooks: Prevent developers from committing sensitive information by manually setting up pre-commit hooks that run Gitleaks scans before the code is committed.

• CI/CD Pipelines: Automate secrets detection during the build and deployment processes by manually configuring Gitleaks scans in your CI/CD pipelines.

• Regular Repository Scans: Schedule regular manual scans of your repositories to detect and address any newly introduced secrets.

Basic Configuration Guidance

Installation:

Install Gitleaks via binary release or Go: 

curl -sSL https://git.io/gitleaks | bash
Or using Go
go install github.com/zricethezav/gitleaks/v7/cmd/gitleaks@latest 

Configuration File Setup:

Gitleaks uses a configuration file (config.toml) to define regex patterns and scanning rules. You can create your custom configuration or use the default one.

Example configuration snippet: 

[[rules]] description = "Generic Credential" regex = '''(?i)(password|passwd|pwd|secret|api_key|apikey|auth_token|authkey)[:=]\s*['"]?[a-z0-9/_-]{8,}['"]?''' 

Running Scans:

To run Gitleaks with a custom configuration file: 

 gitleaks detect --source . --config path/to/config.toml 

To use Gitleaks with the default configuration: 

gitleaks detect --source

Integrating into CI/CD:

Add Gitleaks to your CI/CD pipeline configuration (e.g., in a GitHub Actions workflow): <<code>> name: Gitleaks on: [push, pull_request] jobs: gitleaks: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Install Gitleaks run: curl -sSL https://git.io/gitleaks | bash - name: Run Gitleaks run: gitleaks detect --source . <</code>>

Additional resource on Gitleaks and secrets detection tools:

• Gitleaks Official Documentation 
• The Developer's Guide to Using Gitleaks to Detect Hardcoded Secrets
• Configuring Gitleaks Guide
• TruffleHog - A Deep Dive on Secret Management and How to Fix Exposed Secrets

Implementing SCA with OSV Scanner

SCA tools inspect open source components for known vulnerabilities, ensuring that dependencies are secure and up-to-date. They provide visibility into the security of third-party libraries and offer remediation guidance, reducing the risk of exploiting known vulnerabilities.

Our preference? Npm audit for Javascript and OSV Scanner for other languages. Let’s dive into how to implement OSV Scanner.



OSV Scanner detects known vulnerabilities in open source dependencies. It leverages vulnerability databases to identify and report issues in your software's dependencies, ensuring that any security risks are promptly addressed.

Where OSV Scanner can be integrated into your SDLC

Dependency Management Workflows: Regularly check for vulnerabilities in dependencies during the development process using popular package managers:

• npm (JavaScript)

• pip (Python)

• Maven (Java)

For more information on integrating with package managers, refer to the OSV Scanner documentation.

Build Processes: Automate scans during build processes to identify vulnerabilities in every build using various CI/CD systems;

• GitHub Actions

• GitLab CI

• Jenkins

• CircleCI

Basic Configuration Guidance

Installation:

• Install OSV Scanner using pip: 

pip install osv-scanner

Integration with Package Managers:

• OSV Scanner can work with various package managers to detect vulnerabilities in dependencies. Ensure your project uses a compatible package manager (e.g., npm, pip, Maven).

• Example of integrating with npm: 

npx osv-scanner --lockfile=package-lock.json

Running Scans:

• To run OSV Scanner on a Python project: 

osv-scanner --source=. 

• To scan a JavaScript project: 

npx osv-scanner --lockfile=package-lock.json 

Integrating into CI/CD:

• Add OSV Scanner to your CI/CD pipeline configuration (e.g., in a GitHub Actions workflow): 

name: OSV Scanner
on: [push, pull_request]
jobs:
  osv-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: '3.x'
      - name: Install OSV Scanner
        run: pip install osv-scanner
      - name: Run OSV Scanner
        run: osv-scanner --source=.

Additional materials on OSV Scanner and SCA tools

• OSV Scanner documentation

• Top ten SCA tools

• npm audit guide



Implementing Container Scanning with Trivy

Container scanning tools inspect container images for vulnerabilities and configuration issues, ensuring secure container environments.

These tools detect OS and application vulnerabilities within containers, ensuring that only secure images are deployed. By integrating container scanning into the CI/CD pipeline, organizations can catch and fix vulnerabilities before containers are run in production.

We prefer Trivy, which is maintained by Aqua Security.



Trivy scans container images for vulnerabilities and configuration issues. It detects OS and application vulnerabilities within containers, ensuring a secure container environment.

Where Trivy Can Be Integrated into Your SDLC:

CI/CD Pipelines: Automate scans during the build process to catch vulnerabilities before deployment. Supported Source Code Managers include:

• GitHub Actions: Integrate Trivy to scan container images during the CI/CD pipeline.

• GitLab CI: Use Trivy within GitLab CI to automate vulnerability scanning in your pipeline.

• Bitbucket Pipelines: Add Trivy to Bitbucket Pipelines for automated container image scanning.

Container Build Processes: Ensure container images are secure before they are deployed. Supported Container registries include:

• Docker Hub: Use Trivy to scan images stored in Docker Hub for vulnerabilities.

• Amazon ECR (Elastic Container Registry): Integrate Trivy to scan images in Amazon ECR.

• Google Container Registry: Use Trivy to check for vulnerabilities in images stored in Google Container Registry.

• Azure Container Registry: Scan container images in Azure Container Registry with Trivy.

Runtime Security Monitoring: Continuously monitor running containers for vulnerabilities.

• Kubernetes: Deploy Trivy as a DaemonSet to monitor running containers in a Kubernetes cluster.

• Docker Swarm: Use Trivy to monitor and scan containers in a Docker Swarm environment.

Basic Configuration Guidance

Installation: Install Trivy via package manager: 

apt-get install trivy 

Command-Line Usage: Scan a container image: 

trivy image <image_name>

Integrating into CI/CD: Add Trivy to your CI/CD pipeline configuration (e.g., in a GitHub Actions workflow): 

name: Trivy
on: [push, pull_request]
jobs:
  trivy-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Install Trivy
        run: apt-get install trivy
      - name: Run Trivy Scan
        run: trivy image <image_name>

Refer to the Trivy documentation for more detailed guidance and examples. Or, see other tools in our top ten container scanners article.

Implementing IaC Scanning with KICS

IaC scanning tools evaluate IaC scripts for security misconfigurations, ensuring secure infrastructure deployments.

These tools enforce secure configurations and the use of patched software versions in infrastructure code. By scanning IaC files during development and pre-deployment, organizations can prevent security misconfigurations and reduce the attack surface.

Our suggested IaC security tools are KICS and Checkov. 



KICS (Keeping Infrastructure as Code Secure) scans Infrastructure as Code (IaC) files for security misconfigurations, identifying vulnerabilities in configurations for Terraform, AWS CloudFormation, Kubernetes, and more. It stands out due to its support for a wide range of IaC technologies, being open-source and community-driven, having an extensive library of built-in checks, and allowing custom query support.

Where KICS Can Be Integrated into Your SDLC

IaC Development Workflows:

• IDE Integration: Integrate KICS with popular IDEs (e.g., VSCode, IntelliJ) to scan IaC files during development and catch misconfigurations early.

• Pre-commit Hooks: Use tools like pre-commit to run KICS scans before changes are committed to the version control system.

CI/CD Pipelines:

• Jenkins: Configure a Jenkins pipeline to run KICS scans on IaC files as part of the build process.

• GitHub Actions: Set up a GitHub Actions workflow to automate KICS scans on pull requests and pushes.

• GitLab CI: Add KICS scanning to your .gitlab-ci.yml file to ensure secure configurations before deployment.

Pre-deployment Checks:

• Terraform Cloud: Integrate KICS with Terraform Cloud to validate IaC files for security issues before applying infrastructure changes.

• AWS CodePipeline: Use AWS CodeBuild to run KICS scans as a stage in your CodePipeline to catch security issues before deployment.

• Azure DevOps: Add KICS scanning tasks in Azure Pipelines to automate security checks before deploying infrastructure.

Basic Configuration Guidance

Installation: Install KICS using Go: 

go get github.com/Checkmarx/kics 

Running Scans:

• For Terraform files: 

kics scan -p /path/to/terraform/files 

• For AWS CloudFormation files: 

kics scan -p /path/to/cloudformation/files 

• For Kubernetes manifests: 

kics scan -p /path/to/kubernetes/manifests 

Integrating into CI/CD: Add KICS to your CI/CD pipeline configuration (e.g., in a GitHub Actions workflow): 

name: KICS
on: [push, pull_request]
jobs:
  kics-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Install KICS
        run: go get github.com/Checkmarx/kics
      - name: Run KICS Scan
        run: kics scan -p /path/to/iac/files

For more details, read about our deep dive into implementing KICS.

Implementing Cloud Security Posture Management with Prowler

CSPM tools assess cloud environments for compliance and security risks, ensuring adherence to best practices and regulatory requirements.

They continuously monitor cloud environments, detecting misconfigurations and compliance issues. They provide actionable insights and remediation recommendations, helping organizations maintain a secure and compliant cloud infrastructure.

We recommend Prowler and ScoutSuite. Below, we’ll dive into the benefits of Prowler and how to implement it.



Prowler assesses cloud environments, such as AWS, for security and compliance, detecting misconfigurations, compliance issues, and weak security controls. Its robust framework offers detailed reporting and actionable insights, making it a preferred choice for thorough cloud security audits.

Where Prowler Can Be Integrated into Your SDLC

• CI/CD Pipelines: Integrate Prowler scans into CI/CD pipelines using tools like Jenkins, GitHub Actions, or GitLab CI to automatically detect security issues before deployment.

• Automated Cloud Audits: Use with AWS Lambda for scheduled audits, triggering Prowler scans to run at regular intervals.

• Compliance Dashboards: Integrate with Splunk, Elasticsearch, or other SIEM tools to visualize and monitor compliance status in real-time.

• Alerting Systems: Connect with AWS SNS or Slack for immediate notifications on security findings.

Basic Configuration Guidance

Installation: Install Prowler using the following command: 

git clone https://github.com/prowler-cloud/prowler.git 

Running Checks: Run Prowler checks: 

./prowler -c all

Customizing Profiles: Customize profiles to target specific compliance frameworks or security requirements: 

./prowler -c <profile_name> 

Integrating into CI/CD: Add Prowler to your CI/CD pipeline configuration (e.g., in a GitHub Actions workflow): 

name: Prowler

on: [push, pull_request]

jobs:
  prowler-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Install Prowler
        run: git clone https://github.com/prowler-cloud/prowler.git
      - name: Run Prowler Scan
        run: ./prowler -c all

For more details, refer to the Prowler GitHub, Prowler docs, and Prowler quick start.

Implementing Dynamic Application Security Testing with ZAP

DAST tools simulate attacks on running applications to uncover vulnerabilities, providing real-time insights into the security posture of deployed applications.

These tools identify vulnerabilities in the runtime environment, such as injection flaws and authentication issues, before deployment. Integrating DAST into the CI/CD pipeline ensures that applications are tested for security issues in staging and production phases.

We prefer ZAP and BurpSuite. Below, we’ll review the benefits of ZAP and how to implement it.



OWASP ZAP (Zed Attack Proxy) performs dynamic application security testing, identifying vulnerabilities in live applications through both active and passive scanning. Active scanning emulates real-world attacks to uncover exploitable vulnerabilities, while passive scanning inspects application traffic to detect issues without affecting system performance. ZAP supports authenticated mode scanning, allowing for a comprehensive security assessment of the entire application, including areas accessible only to authenticated users.

Where ZAP Can Be Integrated into Your SDLC

• Staging and Production Phases: Conduct security testing on applications during staging and production.

• CI/CD Pipelines: Automate security tests to detect vulnerabilities in the deployment pipeline.

Basic Configuration Guidance

Installation: Install ZAP using the following command: 

sudo apt install zaproxy 

Running Scans: Run a quick scan on a target application: 

zap-cli quick-scan http://your-application-url 

Integrating into CI/CD: Add ZAP to your CI/CD pipeline configuration (e.g., in a GitHub Actions workflow): 

name: ZAP Scan
on: [push, pull_request]
jobs:
  zap-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Install ZAP
        run: sudo apt install zaproxy
      - name: Run ZAP Scan
        run: zap-cli quick-scan http://your-application-url

For more details, refer to the additional materials below:

• ZAP Official Documentation

• ZAP overview

• 6 Essential Steps to Use OWASP ZAP for Penetration Testing

• How to Run an API Scanner with OWASP ZAP

• How to Automate OWASP ZAP

Security Challenges in Cloud-Based Applications

Proper implementation of security practices, such as automated vulnerability scanning, effective patch management, robust runtime monitoring, and secure software supply chain management, is essential to protect sensitive data and maintain trust.

However, implementing DevSecOps is easier said than done. Several challenges need to be addressed, including:

1. Tooling Management: Managing a wide set of security tools can be complex and resource-intensive. Each tool may require separate integration, configuration, and maintenance, leading to increased overhead and potential inconsistencies in security practices.

2. Developer Engagement: Making it easy for developers to understand and incorporate security into their workflows is crucial. Security tools need to be seamlessly integrated into the development environment to avoid disrupting productivity. Educating developers on security best practices and ensuring they understand the importance of secure coding can also be challenging.

3. Continuous Monitoring and Feedback: Ensuring continuous security monitoring and providing timely feedback to developers can be difficult. Security issues need to be identified and addressed promptly to prevent vulnerabilities from making it into production.

4. Balancing Speed and Security: DevSecOps aims to integrate security without compromising the speed of development and deployment. Finding the right balance between maintaining rapid development cycles and implementing thorough security checks is a constant challenge.

5. Cultural Shift: Implementing DevSecOps requires a cultural shift within the organization. Development, security, and operations teams need to collaborate closely and adopt a shared responsibility for security. This cultural change can be slow and requires strong leadership and commitment.

Simplifying DevSecOps Toolchain Management with Jit

Throughout this article, we've examined a variety of open source security tools, each playing a critical role in ensuring application security. Managing these tools individually, however, can be a complex and resource-intensive task. Jit simplifies this process by unifying these tools into a single, cohesive platform.

Jit offers several capabilities:

• Simplified Integration and Management: Jit replaces multiple standalone security tools with a single platform that integrates seamlessly with your existing software configuration management (SCM) systems. This means you can activate security tools across your repositories with one click, significantly reducing the complexity and overhead associated with managing separate tools.

• Centralized Reporting and Analytics: The platform aggregates data from all integrated tools into a centralized dashboard, providing comprehensive visibility into your security posture. This includes metrics such as mean time to remediation (MTTR), exposure windows, and resolved issues pre-production, enabling more effective monitoring and decision-making.

• Enhanced Collaboration: Jit improves developer user experience by seamlessly integrating security into their daily workflows. It provides a unified interface that simplifies security tasks, making it easier for developers to adopt and implement security practices without disrupting their regular development activities.

Comparing Jit with Standalone Open Source Tools

While standalone tools like Semgrep, Gitleaks, Trivy, and others offer robust functionalities, they often require significant effort for individual integration, configuration, and maintenance. Each tool's outputs must be monitored separately, which can lead to fragmented and inconsistent security practices.

Advantages of using Jit over standalone tools include:

• Unified Platform: Jit consolidates multiple security tools into one platform, streamlining the setup and reducing the burden of managing disparate tools. This unified approach ensures consistent application of security measures across the entire SDLC.

• Automated Workflows: Jit automates the activation and execution of security tools based on predefined security plans. This reduces manual intervention and ensures continuous security checks throughout the development lifecycle.

• Intelligent Prioritization: Jit’s Context Engine leverages machine learning to prioritize vulnerabilities based on their runtime context, reducing false positives and focusing on the most critical issues that are exploitable in production.

Final Thoughts

Integrating security throughout the Software Development Lifecycle (SDLC) is crucial for maintaining a robust security posture. While individual open source tools provide essential capabilities, the complexity of managing them can be daunting. Jit offers a more efficient solution by unifying these tools into a single platform, simplifying integration, enhancing collaboration, and improving overall security management.

Organizations should consider exploring both individual open source tools and unified platforms like Jit to determine the best approach for their specific needs. Adopting a comprehensive and integrated security strategy ensures that applications remain secure and resilient against evolving threats.

For more information on Jit and how it can enhance your DevSecOps practices, check out our platform.