Skip to content

Chapter 5 of 7

Testing, Code Quality, and Security Scanning

Testing is the engine of CI's feedback loop. The test pyramid says you should have many unit tests at the base, fewer integration tests in the middle, and a small number of end-to-end tests at the top — and CI mirrors that structure by running fast unit tests on every commit, integration tests after build, and end-to-end tests against staging. To keep the loop tight, tests should run in parallel using sharding, and the generated JUnit XML reports feed back into the CI UI for inspection and trend tracking. Test parallelization has limits, however: shared state, order-dependent tests, and cost-per-runner all conspire to make more than eight-to-ten-way parallelism yield diminishing returns. Tools like `pytest-xdist`, Jest's `--shard`, and Knapsack for Ruby help with sharding. The test stage should be designed to surface meaningful failures quickly and isolate the most common flake sources — shared databases, file systems, timing assumptions, or nondeterministic inputs.

Flaky tests are CI's worst productivity drain because they erode trust in red builds. Tracking flake rate as a key performance indicator and quarantining known-flaky tests into a separate, less-frequently-run suite keeps the main pipeline trustworthy while still surfacing the problem. Each quarantined test should have a clear owner and a tracking ticket so it eventually gets fixed. Local pre-commit hooks complement CI but do not replace it: a pre-commit hook runs on the developer's machine before `git commit`, catching fast issues like lint or formatting with no network round trip, but it is bypassable with `--no-verify` and so cannot be the authoritative gate. The pre-commit framework provides a managed multi-language solution via `.pre-commit-config.yaml`, while CI remains the enforcer of what must be true. Implemented with tools like Husky, pre-commit.com, or lefthook, hooks should stay under five seconds.

Code quality gates are automated checkpoints that enforce minimum standards before code can proceed. Common gates include test coverage above a threshold (e.g., 80% lines and branches), zero critical security vulnerabilities, no new code smells, all linting rules passing, and performance benchmarks within acceptable bounds. Test coverage thresholds are enforced by running tests with coverage (`jest --coverage`), configuring thresholds in the test framework's config, and failing the CI step when the percentage drops below the line. Uploading reports to SonarQube or Codecov tracks coverage trends over time. Linting belongs early in the pipeline so obvious issues are caught before slower tests execute; popular linters include ESLint for JavaScript and TypeScript, Pylint and Ruff for Python, RuboCop for Ruby, and PHP_CodeSniffer for PHP. SonarQube consolidates all of these signals — bugs, vulnerabilities, code smells, duplications, and coverage — into a quality gate that can fail a pipeline when thresholds are not met. GitHub branch protection provides the enforcement mechanism by requiring status checks to pass before a pull request can merge.

Static and dynamic security scanning address different threat models. SAST (Static Application Security Testing) analyzes source code without executing it to find patterns that suggest SQL injection, cross-site scripting, hardcoded credentials, or buffer overflows; popular tools are SonarQube, Semgrep, GitHub CodeQL, and Checkmarx, and they integrate as a build-time check. DAST (Dynamic Application Security Testing), in contrast, tests a running application by simulating external attacks — tools like OWASP ZAP, Burp Suite, and Nuclei can detect authentication flaws, server misconfigurations, runtime injection, and exposed sensitive data. Because DAST requires a deployed target, it is run after deployment to a test environment. Dependency scanning, sometimes called Software Composition Analysis, is the third pillar: tools such as Dependabot, Snyk, Trivy, `npm audit`, and `pip-audit` compare third-party libraries against vulnerability databases like the NVD and block builds with critical findings. Together these three classes of scan catch most common production security issues before they ship.

All chapters
  1. 1Foundations of CI/CD
  2. 2GitHub Actions in Depth
  3. 3Jenkins and Alternative CI/CD Platforms
  4. 4Pipeline Internals, Caching, and Configuration
  5. 5Testing, Code Quality, and Security Scanning
  6. 6Deployment Strategies and Progressive Delivery
  7. 7Supply Chain Security and Engineering Excellence

Drill it

Reading is not remembering. These come from the Cicd Pipelines deck:

Q

What is Continuous Integration (CI)?

Continuous Integration is a development practice where developers frequently merge code changes into a shared repository, often multiple times a day. Each merge...

Q

What is the difference between Continuous Delivery and Continuous Deployment?

Continuous Delivery ensures code is always in a deployable state but requires a manual approval step before production release.Continuous Deployment goes furthe...

Q

What are the typical stages of a CI/CD pipeline?

A standard CI/CD pipeline includes these stages:Source – code checkout from version controlBuild – compile code and resolve dependenciesTest – run unit, integra...

Q

What is a GitHub Actions workflow?

A GitHub Actions workflow is an automated process defined in a YAML file under .github/workflows/. It is triggered by events such as push, pull_request, or sche...