Python pytest
Powerful Python testing framework with fixtures, parametrisation, and plugin ecosystem.
Python pytest
Powerful Python testing framework with fixtures, parametrisation, and plugin ecosystem.
Overview
pytest is a mature, feature-rich testing framework that makes it easy to write simple unit tests and scale up to complex functional testing. It provides automatic test discovery, detailed assertion introspection, fixture dependency injection, parametrised testing, and extensive plugin support. pytest has become the de-facto standard for Python testing, used across open-source projects and enterprises alike.
flowchart TB
subgraph "Test Discovery"
A[pytest command] --> B[Search for test_*.py or *_test.py]
B --> C[Find test_* functions or Test* classes]
C --> D[Collect test items]
end
subgraph "Test Execution"
D --> E[Setup fixtures]
E --> F[Run test]
F --> G[Teardown fixtures]
G --> H{Pass/Fail?}
H -->|Pass| I[Continue]
H -->|Fail| J[Report failure]
end
subgraph "Reporting"
I --> K[Summary report]
J --> K
K --> L[Exit code]
end
Test Discovery and Structuring
Key Concepts
- Test discovery: pytest automatically finds tests based on naming conventions
- Test collection: Identifies and organises test items before execution
- Test layout: Organise tests in dedicated directories or alongside source code
- Naming conventions: Predictable patterns for test files, classes, and functions
Basic Test Structure
# test_calculator.py
import pytest
def add(a, b):
return a + b
def test_addition():
"""Test basic addition."""
assert add(2, 3) == 5
assert add(-1, 1) == 0
assert add(0, 0) == 0
def test_addition_floats():
"""Test addition with floating-point numbers."""
assert add(0.1, 0.2) == pytest.approx(0.3)
# Test classes for grouping
class TestCalculator:
"""Group related calculator tests."""
def test_addition(self):
assert add(1, 1) == 2
def test_subtraction(self):
assert add(5, -3) == 2
# Setup/teardown for test class
def setup_method(self):
"""Run before each test method."""
self.temp_data = []
def teardown_method(self):
"""Run after each test method."""
self.temp_data.clear()
Test Discovery Conventions
# pytest searches for:
# - Files: test_*.py or *_test.py
# - Directories: No __init__.py required
# - Functions: test_*() functions
# - Classes: Test* classes (without __init__)
# - Methods: test_*() methods in Test* classes
# Standard project layout
"""
project/
├── src/
│ └── myapp/
│ ├── __init__.py
│ └── calculator.py
├── tests/
│ ├── __init__.py
│ ├── test_calculator.py
│ └── test_integration.py
└── pyproject.toml
"""
# Alternative: tests alongside source
"""
project/
└── myapp/
├── __init__.py
├── calculator.py
└── test_calculator.py
"""
# Configure test discovery in pytest.ini or pyproject.toml
# pytest.ini
"""
[pytest]
testpaths = tests
python_files = test_*.py *_test.py
python_classes = Test* *Tests
python_functions = test_* *_test
"""
# pyproject.toml
"""
[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py", "*_test.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
"""
Running Tests
# Run all tests
pytest
# Run specific file
pytest tests/test_calculator.py
# Run specific test
pytest tests/test_calculator.py::test_addition
# Run specific class
pytest tests/test_calculator.py::TestCalculator
# Run specific method
pytest tests/test_calculator.py::TestCalculator::test_addition
# Run tests matching pattern
pytest -k "addition" # Run tests with 'addition' in name
pytest -k "not integration" # Exclude integration tests
pytest -k "add or subtract" # Multiple patterns
# Run tests in directory
pytest tests/unit/
# Verbose output
pytest -v # Verbose
pytest -vv # Extra verbose
# Show local variables on failure
pytest -l
# Stop after first failure
pytest -x
# Stop after N failures
pytest --maxfail=3
# Run last failed tests
pytest --lf # Last failed only
pytest --ff # Failed first, then rest
# Parallel execution (requires pytest-xdist)
pytest -n auto # Auto-detect CPU count
pytest -n 4 # Use 4 workers
Test Organization Patterns
# tests/conftest.py - shared fixtures and configuration
import pytest
@pytest.fixture
def database():
"""Fixture available to all tests."""
db = create_test_database()
yield db
db.cleanup()
# tests/unit/test_models.py - unit tests
def test_user_model():
user = User(name="Alice")
assert user.name == "Alice"
# tests/integration/test_api.py - integration tests
@pytest.mark.integration
def test_api_endpoint(client):
response = client.get("/api/users")
assert response.status_code == 200
# tests/e2e/test_workflows.py - end-to-end tests
@pytest.mark.e2e
def test_complete_workflow(browser):
browser.visit("/login")
browser.fill("username", "test")
browser.click("submit")
assert browser.is_text_present("Welcome")
Fixtures and Dependency Injection
Key Concepts
- Fixtures: Reusable test dependencies providing setup and teardown
- Dependency injection: Tests declare fixtures as arguments
- Scope: Control fixture lifetime (function, class, module, package, session)
- Yield fixtures: Provide cleanup after test execution
- Autouse: Automatically apply fixtures without explicit request
flowchart TD
subgraph "Fixture Scopes"
A[session] --> B[package]
B --> C[module]
C --> D[class]
D --> E[function]
end
F[Test Request] --> G{Fixture Exists?}
G -->|No| H[Create Fixture]
G -->|Yes, In Scope| I[Reuse Fixture]
H --> J[Run Test]
I --> J
J --> K{Scope Ended?}
K -->|Yes| L[Teardown Fixture]
K -->|No| M[Keep Fixture]
Basic Fixtures
import pytest
# Simple fixture
@pytest.fixture
def sample_data():
"""Provide test data."""
return [1, 2, 3, 4, 5]
def test_sum(sample_data):
assert sum(sample_data) == 15
# Fixture with setup and teardown
@pytest.fixture
def temp_file():
"""Create temporary file for testing."""
# Setup
file_path = "/tmp/test_file.txt"
with open(file_path, 'w') as f:
f.write("test content")
# Provide to test
yield file_path
# Teardown (runs after test completes)
import os
if os.path.exists(file_path):
os.remove(file_path)
def test_file_reading(temp_file):
with open(temp_file) as f:
content = f.read()
assert content == "test content"
# Fixture requesting another fixture
@pytest.fixture
def database_connection():
conn = create_connection()
yield conn
conn.close()
@pytest.fixture
def database_cursor(database_connection):
"""Cursor depends on connection fixture."""
cursor = database_connection.cursor()
yield cursor
cursor.close()
def test_database_query(database_cursor):
result = database_cursor.execute("SELECT 1")
assert result is not None
Fixture Scopes
import pytest
# Function scope (default) - runs for each test function
@pytest.fixture(scope="function")
def fresh_data():
return {"count": 0}
# Class scope - runs once per test class
@pytest.fixture(scope="class")
def database():
db = create_database()
yield db
db.destroy()
# Module scope - runs once per module
@pytest.fixture(scope="module")
def expensive_resource():
resource = load_large_dataset()
yield resource
resource.cleanup()
# Session scope - runs once per test session
@pytest.fixture(scope="session")
def docker_container():
"""Start container once for entire test session."""
container = docker.run("postgres:13")
yield container
container.stop()
# Package scope - runs once per package
@pytest.fixture(scope="package")
def shared_cache():
cache = {}
yield cache
cache.clear()
# Example usage
class TestUserOperations:
def test_create_user(self, database):
user = database.create_user("alice")
assert user.name == "alice"
def test_delete_user(self, database):
# Same database instance as previous test
database.delete_user("alice")
Autouse Fixtures
import pytest
import logging
# Automatically applied to all tests in module
@pytest.fixture(autouse=True)
def setup_logging():
"""Configure logging for all tests."""
logging.basicConfig(level=logging.DEBUG)
yield
logging.disable(logging.CRITICAL)
# Autouse with scope
@pytest.fixture(scope="session", autouse=True)
def configure_test_environment():
"""Set up environment variables for all tests."""
import os
os.environ["TESTING"] = "true"
os.environ["DATABASE_URL"] = "sqlite:///:memory:"
yield
del os.environ["TESTING"]
del os.environ["DATABASE_URL"]
# Autouse for cleanup
@pytest.fixture(autouse=True)
def reset_state():
"""Reset global state before each test."""
yield
# Runs after each test
global_cache.clear()
reset_singletons()
def test_something():
# setup_logging and reset_state automatically applied
assert True
Fixture Factories
import pytest
# Factory fixture returns a function
@pytest.fixture
def make_user():
"""Factory to create test users."""
created_users = []
def _make_user(name, email=None):
email = email or f"{name}@example.com"
user = User(name=name, email=email)
created_users.append(user)
return user
yield _make_user
# Cleanup all created users
for user in created_users:
user.delete()
def test_multiple_users(make_user):
alice = make_user("alice")
bob = make_user("bob", "bob@custom.com")
assert alice.name == "alice"
assert bob.email == "bob@custom.com"
# Parametrised fixture factory
@pytest.fixture
def user_factory(request):
"""Create user with specified role."""
def _create_user(role="user"):
return User(role=role)
return _create_user
def test_admin_user(user_factory):
admin = user_factory("admin")
assert admin.role == "admin"
conftest.py for Shared Fixtures
# tests/conftest.py - fixtures available to all tests
import pytest
@pytest.fixture(scope="session")
def config():
"""Load test configuration."""
return {
"database_url": "postgresql://localhost/test",
"api_key": "test-key-123"
}
@pytest.fixture
def client(config):
"""HTTP client for API testing."""
from app import create_app
app = create_app(config)
with app.test_client() as client:
yield client
@pytest.fixture
def authenticated_client(client):
"""Client with authentication."""
client.post("/login", json={"username": "test", "password": "test"})
yield client
client.post("/logout")
# tests/unit/conftest.py - fixtures for unit tests only
@pytest.fixture
def mock_database():
"""Mock database for unit tests."""
return MockDatabase()
# tests/integration/conftest.py - fixtures for integration tests
@pytest.fixture
def real_database():
"""Real database for integration tests."""
db = create_test_database()
yield db
db.drop_all_tables()
Parametrisation
Key Concepts
- Parametrised tests: Run same test with different inputs
- Test generation: Create multiple test cases from single test function
- Fixtures parametrisation: Fixtures that provide multiple values
- Indirect parametrisation: Pass parameters through fixtures
Basic Parametrisation
import pytest
# Single parameter
@pytest.mark.parametrize("input,expected", [
(1, 2),
(2, 3),
(3, 4),
])
def test_increment(input, expected):
assert increment(input) == expected
# Multiple parameters
@pytest.mark.parametrize("a,b,expected", [
(1, 1, 2),
(2, 3, 5),
(10, -5, 5),
(0, 0, 0),
])
def test_addition(a, b, expected):
assert add(a, b) == expected
# Parametrisation with test IDs
@pytest.mark.parametrize("input,expected", [
(1, 2),
(2, 3),
(3, 4),
], ids=["one", "two", "three"])
def test_with_ids(input, expected):
assert increment(input) == expected
# Using pytest.param for individual test customisation
@pytest.mark.parametrize("value,expected", [
pytest.param(1, 2, id="positive"),
pytest.param(-1, 0, id="negative"),
pytest.param(0, 1, id="zero"),
pytest.param(999, 1000, marks=pytest.mark.slow, id="large"),
])
def test_with_params(value, expected):
assert increment(value) == expected
Advanced Parametrisation
import pytest
# Multiple parametrize decorators (cartesian product)
@pytest.mark.parametrize("x", [1, 2])
@pytest.mark.parametrize("y", [10, 20])
def test_combinations(x, y):
# Runs 4 times: (1,10), (1,20), (2,10), (2,20)
assert x < y
# Parametrising with objects
@pytest.mark.parametrize("user", [
{"name": "Alice", "age": 30},
{"name": "Bob", "age": 25},
{"name": "Charlie", "age": 35},
])
def test_user_validation(user):
assert "name" in user
assert user["age"] > 0
# Parametrising with complex data
test_cases = [
pytest.param(
{"url": "/api/users", "method": "GET"},
200,
id="list_users"
),
pytest.param(
{"url": "/api/users/1", "method": "GET"},
200,
id="get_user"
),
pytest.param(
{"url": "/api/users", "method": "POST", "data": {"name": "Alice"}},
201,
id="create_user"
),
]
@pytest.mark.parametrize("request_data,expected_status", test_cases)
def test_api_endpoints(client, request_data, expected_status):
method = getattr(client, request_data["method"].lower())
response = method(request_data["url"], json=request_data.get("data"))
assert response.status_code == expected_status
# Skipping specific parameter combinations
@pytest.mark.parametrize("x,y", [
(1, 2),
pytest.param(2, 3, marks=pytest.mark.skip(reason="Known bug")),
(3, 4),
pytest.param(4, 5, marks=pytest.mark.xfail(reason="Expected to fail")),
])
def test_with_skips(x, y):
assert x < y
Fixture Parametrisation
import pytest
# Parametrised fixture
@pytest.fixture(params=["sqlite", "postgresql", "mysql"])
def database(request):
"""Test with multiple database backends."""
db_type = request.param
db = create_database(db_type)
yield db
db.cleanup()
def test_database_operations(database):
# Runs 3 times, once for each database type
result = database.query("SELECT 1")
assert result is not None
# Parametrised fixture with IDs
@pytest.fixture(params=[
pytest.param("dev", id="development"),
pytest.param("staging", id="staging"),
pytest.param("prod", id="production"),
])
def environment(request):
return load_config(request.param)
# Indirect parametrisation
@pytest.fixture
def user(request):
"""Create user based on parameter."""
return User(name=request.param)
@pytest.mark.parametrize("user", ["alice", "bob", "charlie"], indirect=True)
def test_user_operations(user):
assert user.name in ["alice", "bob", "charlie"]
# Partial indirect parametrisation
@pytest.fixture
def db_connection(request):
return create_connection(request.param)
@pytest.mark.parametrize(
"db_connection,query",
[
("sqlite", "SELECT 1"),
("postgres", "SELECT 1"),
],
indirect=["db_connection"] # Only db_connection is indirect
)
def test_query(db_connection, query):
result = db_connection.execute(query)
assert result is not None
Dynamic Parametrisation
import pytest
# Generate parameters dynamically
def generate_test_data():
"""Generate test cases from external source."""
return [
(f"user_{i}", f"user{i}@example.com")
for i in range(5)
]
@pytest.mark.parametrize("username,email", generate_test_data())
def test_user_creation(username, email):
user = User(username=username, email=email)
assert user.email == email
# Load test cases from file
def load_test_cases():
import json
with open("tests/data/test_cases.json") as f:
return json.load(f)
@pytest.mark.parametrize("test_case", load_test_cases())
def test_from_file(test_case):
result = process(test_case["input"])
assert result == test_case["expected"]
# Conditionally skip parameters
def pytest_generate_tests(metafunc):
"""Hook to customise test generation."""
if "db_type" in metafunc.fixturenames:
db_types = ["sqlite", "postgresql"]
# Skip postgresql if not available
if not is_postgresql_available():
db_types.remove("postgresql")
metafunc.parametrize("db_type", db_types)
Markers and Test Selection
Key Concepts
- Markers: Metadata tags for test categorisation and behaviour
- Built-in markers: skip, skipif, xfail, parametrize
- Custom markers: Define domain-specific test categories
- Marker selection: Run subsets of tests based on markers
flowchart LR
A[Test Suite] --> B{Marker?}
B -->|"@pytest.mark.slow"| C[Slow Tests]
B -->|"@pytest.mark.integration"| D[Integration Tests]
B -->|"@pytest.mark.unit"| E[Unit Tests]
F[pytest -m slow] --> C
G[pytest -m integration] --> D
H["pytest -m #quot;not slow#quot;"] --> I[Fast Tests]
E --> I
D --> I
Built-in Markers
import pytest
import sys
# Skip test unconditionally
@pytest.mark.skip(reason="Not implemented yet")
def test_future_feature():
assert False
# Skip conditionally
@pytest.mark.skipif(sys.version_info < (3, 9), reason="Requires Python 3.9+")
def test_modern_syntax():
# Uses Python 3.9+ features
data = {"a": 1, "b": 2}
merged = {"c": 3} | data # Dict merge operator
assert "a" in merged
@pytest.mark.skipif(not has_database(), reason="Database not available")
def test_database_operation():
assert query_database("SELECT 1") is not None
# Expected failure
@pytest.mark.xfail(reason="Known bug in library")
def test_buggy_feature():
assert buggy_function() == "expected"
# Conditional expected failure
@pytest.mark.xfail(sys.platform == "win32", reason="Fails on Windows")
def test_unix_specific():
assert posix_operation() is True
# Strict xfail - fail if test unexpectedly passes
@pytest.mark.xfail(strict=True, reason="Should fail")
def test_strict_failure():
assert False
# Skip entire class
@pytest.mark.skip(reason="Class not ready")
class TestNewFeature:
def test_one(self):
pass
def test_two(self):
pass
# Skip at module level
pytestmark = pytest.mark.skip(reason="Module deprecated")
Custom Markers
import pytest
# Register custom markers in pytest.ini or pyproject.toml
"""
[pytest]
markers =
slow: marks tests as slow (deselect with '-m "not slow"')
integration: marks integration tests
e2e: marks end-to-end tests
smoke: marks smoke tests
database: marks tests requiring database
api: marks API tests
unit: marks unit tests
"""
# Using custom markers
@pytest.mark.slow
def test_large_computation():
result = expensive_calculation()
assert result is not None
@pytest.mark.integration
@pytest.mark.database
def test_database_integration():
user = create_user_in_db("alice")
assert fetch_user_from_db(user.id) == user
@pytest.mark.smoke
def test_application_starts():
app = create_app()
assert app.status == "running"
@pytest.mark.api
@pytest.mark.parametrize("endpoint", ["/health", "/status"])
def test_health_endpoints(client, endpoint):
response = client.get(endpoint)
assert response.status_code == 200
# Apply marker to entire class
@pytest.mark.unit
class TestCalculator:
def test_add(self):
assert add(1, 1) == 2
def test_subtract(self):
assert subtract(3, 1) == 2
# Apply marker to entire module
pytestmark = [pytest.mark.integration, pytest.mark.slow]
Running Tests by Marker
# Run tests with specific marker
pytest -m slow # Only slow tests
pytest -m integration # Only integration tests
pytest -m "not slow" # Exclude slow tests
pytest -m "smoke or unit" # Smoke OR unit tests
pytest -m "integration and database" # Integration AND database tests
pytest -m "not (slow or integration)" # Exclude slow and integration
# Combine markers with other filters
pytest -m slow -k "test_user" # Slow tests with 'user' in name
pytest -m integration -x # Stop on first failure in integration tests
# List available markers
pytest --markers
# Strict marker checking (fail on unknown markers)
pytest --strict-markers
Marker Parametrisation
import pytest
# Different markers for different parameters
@pytest.mark.parametrize("value,expected", [
pytest.param(1, 2, marks=pytest.mark.unit),
pytest.param(10, 11, marks=pytest.mark.unit),
pytest.param(1000, 1001, marks=pytest.mark.slow),
pytest.param(1000000, 1000001, marks=[pytest.mark.slow, pytest.mark.stress]),
])
def test_increment(value, expected):
assert increment(value) == expected
# Conditional markers based on fixture
@pytest.fixture(params=[
pytest.param("small", marks=pytest.mark.fast),
pytest.param("large", marks=pytest.mark.slow),
])
def dataset(request):
size = request.param
return load_dataset(size)
def test_processing(dataset):
# Marked fast or slow depending on dataset
result = process(dataset)
assert result is not None
Custom Marker Behaviour
import pytest
# Hook to modify test behaviour based on markers
def pytest_runtest_setup(item):
"""Run before each test."""
# Skip tests marked 'database' if no database available
if "database" in item.keywords and not has_database():
pytest.skip("Database not available")
# Set timeout for slow tests
if "slow" in item.keywords:
item.config.option.timeout = 60
# Fixture that uses marker information
@pytest.fixture
def test_timeout(request):
"""Provide timeout based on test markers."""
if "slow" in request.keywords:
return 60
elif "integration" in request.keywords:
return 30
else:
return 10
def test_operation(test_timeout):
# Use timeout from fixture
result = perform_operation(timeout=test_timeout)
assert result is not None
Assertions and Introspection
Key Concepts
- assert statement: Python's native assert with pytest introspection
- Assertion rewriting: pytest provides detailed failure messages
- Approximate comparisons: Handle floating-point precision
- Exception assertions: Verify exceptions are raised
- Warning assertions: Check for expected warnings
Basic Assertions
import pytest
# Simple assertions with detailed output
def test_basic_assertions():
# Equality
assert 1 + 1 == 2
assert "hello" == "hello"
assert [1, 2, 3] == [1, 2, 3]
# Identity
a = [1, 2]
b = a
assert a is b
assert a is not [1, 2] # Different object
# Membership
assert 1 in [1, 2, 3]
assert "key" in {"key": "value"}
assert "hello" in "hello world"
# Boolean
assert True
assert not False
assert bool([1, 2, 3])
assert not bool([])
# Comparisons
assert 5 > 3
assert 2 <= 2
assert "abc" < "def"
# Assertions with failure messages
def test_with_messages():
x = 5
assert x == 5, f"Expected 5, got {x}"
items = []
assert len(items) > 0, "List should not be empty"
Approximate Comparisons
import pytest
import math
# Floating-point comparison
def test_float_comparison():
# Bad: direct comparison can fail due to precision
# assert 0.1 + 0.2 == 0.3 # May fail!
# Good: use pytest.approx
assert 0.1 + 0.2 == pytest.approx(0.3)
assert math.pi == pytest.approx(3.14159, abs=1e-5)
# Relative tolerance (default 1e-6)
assert 100.001 == pytest.approx(100, rel=1e-3)
# Absolute tolerance
assert 0.001 == pytest.approx(0, abs=1e-2)
# Compare sequences
assert [0.1 + 0.1, 0.2 + 0.2] == pytest.approx([0.2, 0.4])
# Compare dictionaries
assert {"a": 0.1 + 0.1} == pytest.approx({"a": 0.2})
# Infinity and NaN
assert math.inf == pytest.approx(math.inf)
assert math.nan != pytest.approx(math.nan) # NaN != NaN
Exception Assertions
import pytest
# Assert exception is raised
def test_exception_raised():
with pytest.raises(ValueError):
raise ValueError("Invalid value")
with pytest.raises(ZeroDivisionError):
1 / 0
# Check exception message
def test_exception_message():
with pytest.raises(ValueError, match="Invalid.*value"):
raise ValueError("Invalid value provided")
# Access exception object
with pytest.raises(ValueError) as exc_info:
raise ValueError("Test error")
assert str(exc_info.value) == "Test error"
assert exc_info.type is ValueError
# Multiple possible exceptions
def test_multiple_exceptions():
with pytest.raises((ValueError, TypeError)):
risky_operation()
# Ensure no exception is raised
def test_no_exception():
# Just call the function - test passes if no exception
result = safe_operation()
assert result is not None
# Custom exception assertions
def test_custom_exception():
class CustomError(Exception):
def __init__(self, code, message):
self.code = code
self.message = message
with pytest.raises(CustomError) as exc_info:
raise CustomError(404, "Not found")
assert exc_info.value.code == 404
assert exc_info.value.message == "Not found"
Warning Assertions
import pytest
import warnings
# Assert warning is issued
def test_warning():
with pytest.warns(UserWarning):
warnings.warn("This is a warning", UserWarning)
# Check warning message
with pytest.warns(UserWarning, match="deprecated"):
warnings.warn("Function is deprecated", UserWarning)
# Access warning details
with pytest.warns(UserWarning) as warning_info:
warnings.warn("Test warning", UserWarning)
assert len(warning_info) == 1
assert "Test warning" in str(warning_info[0].message)
# Ensure no warnings
def test_no_warnings():
with warnings.catch_warnings():
warnings.simplefilter("error")
# Code that should not produce warnings
result = clean_function()
assert result is not None
# Record all warnings (pytest.warns(None) was removed in pytest 7)
def test_with_warning_recorder(recwarn):
warnings.warn("Warning 1", UserWarning)
warnings.warn("Warning 2", DeprecationWarning)
assert len(recwarn) == 2
Collection Assertions
import pytest
def test_collection_membership():
data = [1, 2, 3, 4, 5]
# Membership
assert 3 in data
assert 6 not in data
# Subset
assert {1, 2}.issubset({1, 2, 3, 4})
# Length
assert len(data) == 5
assert len(data) > 0
def test_dict_assertions():
data = {"name": "Alice", "age": 30}
# Key presence
assert "name" in data
assert "email" not in data
# Value checks
assert data["name"] == "Alice"
assert data.get("age") == 30
# Subset
assert {"name": "Alice"}.items() <= data.items()
def test_string_assertions():
text = "Hello, World!"
# Substring
assert "World" in text
assert "Python" not in text
# String methods
assert text.startswith("Hello")
assert text.endswith("!")
assert text.lower() == "hello, world!"
# Pattern matching
import re
assert re.match(r"Hello, \w+!", text)
Custom Assertion Helpers
import pytest
# Custom assertion function
def assert_valid_user(user):
"""Assert user object is valid."""
assert user is not None, "User should not be None"
assert hasattr(user, "name"), "User should have 'name' attribute"
assert hasattr(user, "email"), "User should have 'email' attribute"
assert "@" in user.email, f"Invalid email: {user.email}"
def test_user_creation():
user = create_user("alice", "alice@example.com")
assert_valid_user(user)
# Custom assertion in conftest.py for reuse
# tests/conftest.py
def assert_api_response(response, expected_status=200):
"""Assert API response is valid."""
assert response.status_code == expected_status, \
f"Expected {expected_status}, got {response.status_code}"
assert response.headers["Content-Type"] == "application/json"
assert response.json() is not None
# Use in tests
def test_api_endpoint(client):
response = client.get("/api/users")
assert_api_response(response)
Custom Plugins and Hooks
Key Concepts
- Hooks: pytest extension points for customising behaviour
- Plugins: Reusable extensions via entry points or conftest.py
- Local plugins: Define in conftest.py for project-specific behaviour
- Distributed plugins: Publish as separate packages
conftest.py Hooks
# tests/conftest.py
import pytest
import logging
# Collection hooks
def pytest_collection_modifyitems(config, items):
"""Modify test collection."""
# Add marker to all tests in specific directory
for item in items:
if "integration" in str(item.fspath):
item.add_marker(pytest.mark.integration)
# Reorder tests: run unit tests before integration tests
items.sort(key=lambda x: (
"integration" in x.keywords,
"slow" in x.keywords
))
# Test execution hooks
def pytest_runtest_setup(item):
"""Called before running each test."""
# Log test start
logging.info(f"Starting test: {item.nodeid}")
# Skip tests based on custom logic
if "database" in item.keywords:
if not item.config.getoption("--run-database"):
pytest.skip("Database tests disabled")
def pytest_runtest_teardown(item, nextitem):
"""Called after running each test."""
# Clean up after each test
logging.info(f"Completed test: {item.nodeid}")
clear_cache()
def pytest_runtest_makereport(item, call):
"""Generate test report."""
if call.when == "call":
if call.excinfo is not None:
# Test failed - capture additional info
logging.error(f"Test failed: {item.nodeid}")
# Session hooks
def pytest_sessionstart(session):
"""Called before test session starts."""
logging.info("Test session starting")
setup_test_environment()
def pytest_sessionfinish(session, exitstatus):
"""Called after test session ends."""
logging.info(f"Test session finished with status: {exitstatus}")
cleanup_test_environment()
# Configuration hooks
def pytest_configure(config):
"""Called after command line parsing."""
config.addinivalue_line(
"markers", "custom: custom marker description"
)
def pytest_addoption(parser):
"""Add command line options."""
parser.addoption(
"--run-database",
action="store_true",
default=False,
help="Run tests that require database"
)
parser.addoption(
"--env",
action="store",
default="test",
help="Environment to test against"
)
Custom Fixtures via Plugins
# tests/conftest.py
import pytest
from unittest.mock import Mock
@pytest.fixture
def mock_api():
"""Mock external API."""
api = Mock()
api.get.return_value = {"status": "success"}
return api
@pytest.fixture
def capture_logs():
"""Capture log messages during test."""
import logging
from io import StringIO
log_stream = StringIO()
handler = logging.StreamHandler(log_stream)
handler.setLevel(logging.DEBUG)
root_logger = logging.getLogger()
root_logger.addHandler(handler)
yield log_stream
root_logger.removeHandler(handler)
def test_with_log_capture(capture_logs):
logging.info("Test message")
logs = capture_logs.getvalue()
assert "Test message" in logs
# Fixture that provides test metadata
@pytest.fixture
def test_info(request):
"""Provide information about current test."""
return {
"name": request.node.name,
"markers": [m.name for m in request.node.iter_markers()],
"module": request.module.__name__,
}
def test_metadata(test_info):
assert test_info["name"] == "test_metadata"
Custom Markers with Hooks
# tests/conftest.py
import pytest
def pytest_configure(config):
"""Register custom markers."""
config.addinivalue_line(
"markers", "timeout(seconds): set test timeout"
)
config.addinivalue_line(
"markers", "retry(count): retry failed tests N times"
)
def pytest_runtest_setup(item):
"""Handle custom markers."""
# Handle timeout marker
timeout_marker = item.get_closest_marker("timeout")
if timeout_marker:
timeout = timeout_marker.args[0]
item.config.option.timeout = timeout
# Handle retry marker
retry_marker = item.get_closest_marker("retry")
if retry_marker:
retry_count = retry_marker.args[0]
# Store for use in teardown
item.retry_count = retry_count
# Usage in tests
@pytest.mark.timeout(30)
def test_slow_operation():
expensive_operation()
@pytest.mark.retry(3)
def test_flaky_api():
response = call_flaky_api()
assert response.status_code == 200
Test Report Customisation
# tests/conftest.py
import pytest
from datetime import datetime
class CustomTestReport:
"""Custom test report collector."""
def __init__(self):
self.tests = []
def add_result(self, nodeid, outcome, duration):
self.tests.append({
"test": nodeid,
"outcome": outcome,
"duration": duration,
"timestamp": datetime.now().isoformat()
})
def save(self, filepath):
import json
with open(filepath, 'w') as f:
json.dump(self.tests, f, indent=2)
@pytest.fixture(scope="session")
def custom_report():
report = CustomTestReport()
yield report
report.save("test-results.json")
def pytest_runtest_logreport(report):
"""Collect test results."""
if report.when == "call":
# Access custom_report fixture (requires workaround)
# Or use pytest_configure to store in config
pass
@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
"""Hook wrapper to access test results."""
outcome = yield
report = outcome.get_result()
# Add custom information to report
if report.when == "call":
report.custom_data = {
"markers": [m.name for m in item.iter_markers()],
"duration_ms": report.duration * 1000
}
Plugin for Custom Assertions
# tests/conftest.py
import pytest
class APIAssertion:
"""Custom assertions for API testing."""
def __init__(self, response):
self.response = response
def has_status(self, code):
assert self.response.status_code == code, \
f"Expected status {code}, got {self.response.status_code}"
return self
def has_json(self):
assert self.response.headers.get("Content-Type") == "application/json", \
"Response is not JSON"
return self
def has_field(self, field):
data = self.response.json()
assert field in data, f"Field '{field}' not in response"
return self
def field_equals(self, field, value):
data = self.response.json()
actual = data.get(field)
assert actual == value, \
f"Expected {field}={value}, got {actual}"
return self
@pytest.fixture
def assert_response():
"""Fixture providing response assertions."""
return APIAssertion
# Usage
def test_api_with_custom_assertions(client, assert_response):
response = client.get("/api/users/1")
(assert_response(response)
.has_status(200)
.has_json()
.has_field("name")
.field_equals("name", "Alice"))
Coverage Reporting
Key Concepts
- Code coverage: Measure which code is executed during tests
- Line coverage: Percentage of code lines executed
- Branch coverage: Percentage of decision branches taken
- Coverage reports: HTML, XML, terminal output formats
Basic Coverage Setup
# Install pytest-cov plugin
pip install pytest-cov
# Run tests with coverage
pytest --cov=myapp # Coverage for myapp module
pytest --cov=myapp tests/ # Coverage for myapp, run tests/
pytest --cov=. --cov-report=html # Generate HTML report
pytest --cov=myapp --cov-report=term-missing # Show missing lines
# Multiple coverage targets
pytest --cov=myapp --cov=utils --cov-report=html
# Coverage with specific threshold
pytest --cov=myapp --cov-fail-under=80 # Fail if coverage < 80%
Coverage Configuration
# setup.cfg (use [pytest] in pytest.ini)
[tool:pytest]
addopts =
--cov=myapp
--cov-report=html
--cov-report=term-missing
--cov-fail-under=80
# .coveragerc - coverage.py configuration
[run]
source = myapp
omit =
*/tests/*
*/migrations/*
*/__init__.py
*/conftest.py
[report]
exclude_lines =
pragma: no cover
def __repr__
raise AssertionError
raise NotImplementedError
if __name__ == .__main__.:
if TYPE_CHECKING:
@abstractmethod
precision = 2
show_missing = True
[html]
directory = htmlcov
pyproject.toml Configuration
# pyproject.toml
[tool.pytest.ini_options]
addopts = [
"--cov=myapp",
"--cov-report=html",
"--cov-report=term-missing:skip-covered",
"--cov-fail-under=80",
]
testpaths = ["tests"]
[tool.coverage.run]
source = ["myapp"]
omit = [
"*/tests/*",
"*/migrations/*",
"*/__init__.py",
]
branch = true
[tool.coverage.report]
exclude_lines = [
"pragma: no cover",
"def __repr__",
"raise AssertionError",
"raise NotImplementedError",
"if __name__ == .__main__.:",
"if TYPE_CHECKING:",
"@abstractmethod",
]
precision = 2
show_missing = true
[tool.coverage.html]
directory = "htmlcov"
Coverage in Tests
# tests/test_coverage_example.py
def add(a, b):
"""Add two numbers."""
if not isinstance(a, (int, float)) or not isinstance(b, (int, float)):
raise TypeError("Arguments must be numbers")
return a + b
def test_add_basic():
"""Test basic addition."""
assert add(1, 1) == 2
assert add(2, 3) == 5
def test_add_type_check():
"""Test type validation."""
with pytest.raises(TypeError):
add("1", 2)
with pytest.raises(TypeError):
add(1, "2")
# Exclude code from coverage
def debug_function(): # pragma: no cover
"""Debug-only function."""
import pdb
pdb.set_trace()
def development_only():
if __name__ == "__main__": # pragma: no cover
# This line excluded from coverage
run_development_server()
Coverage Reports
# Terminal report
pytest --cov=myapp --cov-report=term
# Terminal with missing lines
pytest --cov=myapp --cov-report=term-missing
# Skip files with 100% coverage
pytest --cov=myapp --cov-report=term-missing:skip-covered
# HTML report (creates htmlcov/ directory)
pytest --cov=myapp --cov-report=html
# Open htmlcov/index.html in browser
# XML report (for CI tools like Jenkins, SonarQube)
pytest --cov=myapp --cov-report=xml
# JSON report
pytest --cov=myapp --cov-report=json
# Multiple report formats
pytest --cov=myapp --cov-report=html --cov-report=xml --cov-report=term
# No report (just collect coverage)
pytest --cov=myapp --cov-report=
# Annotated source code
pytest --cov=myapp --cov-report=annotate
Branch Coverage
# Enable branch coverage in .coveragerc
# [run]
# branch = True
def classify_number(n):
"""Classify a number."""
if n > 0:
return "positive"
elif n < 0:
return "negative"
else:
return "zero"
# Tests to achieve 100% branch coverage
def test_classify_positive():
assert classify_number(5) == "positive"
def test_classify_negative():
assert classify_number(-5) == "negative"
def test_classify_zero():
assert classify_number(0) == "zero"
# Without all three tests, branch coverage would be incomplete
Coverage with Parallel Testing
# Using pytest-xdist with coverage
pip install pytest-xdist pytest-cov
# Run parallel tests with coverage
pytest -n auto --cov=myapp --cov-report=html
# Configure in pytest.ini
# [pytest]
# addopts = -n auto --cov=myapp --cov-report=html
Combining Coverage Data
# Run tests in multiple sessions
pytest --cov=myapp --cov-append tests/unit/
pytest --cov=myapp --cov-append tests/integration/
# Combine coverage data from parallel runs
coverage combine
# Generate report from combined data
coverage report
coverage html
CI/CD Integration
Key Concepts
- Continuous testing: Run tests automatically on code changes
- Test stages: Unit, integration, end-to-end testing in pipeline
- Coverage tracking: Monitor coverage trends over time
- Test artifacts: Store test results and coverage reports
flowchart LR
A[Git Push] --> B[CI Trigger]
B --> C[Setup Environment]
C --> D[Install Dependencies]
D --> E[Run Linters]
E --> F[Run Unit Tests]
F --> G[Run Integration Tests]
G --> H[Generate Coverage]
H --> I{Coverage OK?}
I -->|Yes| J[Build Package]
I -->|No| K[Fail Pipeline]
J --> L[Deploy/Publish]
GitHub Actions
# .github/workflows/test.yml
name: Tests
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.11", "3.12", "3.13"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e ".[dev]"
# Or: pip install -r requirements-dev.txt
- name: Run linters
run: |
pip install ruff black mypy
ruff check .
black --check .
mypy src/
- name: Run tests
run: |
pytest -v --cov=myapp --cov-report=xml --cov-report=term
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v4
with:
files: ./coverage.xml
flags: unittests
name: codecov-umbrella
fail_ci_if_error: true
- name: Archive test results
if: always()
uses: actions/upload-artifact@v4
with:
name: test-results-${{ matrix.python-version }}
path: |
htmlcov/
coverage.xml
.coverage
integration:
runs-on: ubuntu-latest
needs: test
services:
postgres:
image: postgres:15
env:
POSTGRES_PASSWORD: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: |
pip install -e ".[dev]"
- name: Run integration tests
env:
DATABASE_URL: postgresql://postgres:postgres@localhost/test
run: |
pytest -v -m integration --cov=myapp --cov-append
GitLab CI
# .gitlab-ci.yml
stages:
- lint
- test
- coverage
- deploy
variables:
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
cache:
paths:
- .cache/pip
before_script:
- python -m pip install --upgrade pip
- pip install -e ".[dev]"
lint:
stage: lint
image: python:3.11
script:
- ruff check .
- black --check .
- mypy src/
test:unit:
stage: test
image: python:3.11
script:
- pytest -v -m "not integration" --cov=myapp --cov-report=xml
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: coverage.xml
paths:
- htmlcov/
expire_in: 1 week
coverage: '/(?i)total.*? (100(?:\.0+)?\%|[1-9]?\d(?:\.\d+)?\%)$/'
test:integration:
stage: test
image: python:3.11
services:
- postgres:15
variables:
POSTGRES_DB: test
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
DATABASE_URL: postgresql://postgres:postgres@postgres/test
script:
- pytest -v -m integration --cov=myapp --cov-append
test:e2e:
stage: test
image: python:3.11
script:
- pytest -v -m e2e
only:
- main
- develop
coverage:
stage: coverage
image: python:3.11
dependencies:
- test:unit
- test:integration
script:
- coverage combine
- coverage report
- coverage html
artifacts:
paths:
- htmlcov/
coverage: '/(?i)total.*? (100(?:\.0+)?\%|[1-9]?\d(?:\.\d+)?\%)$/'
Jenkins Pipeline
// Jenkinsfile
pipeline {
agent any
environment {
PYTHONUNBUFFERED = '1'
}
stages {
stage('Setup') {
steps {
sh '''
python3 -m venv venv
. venv/bin/activate
pip install --upgrade pip
pip install -e ".[dev]"
'''
}
}
stage('Lint') {
steps {
sh '''
. venv/bin/activate
ruff check .
black --check .
mypy src/
'''
}
}
stage('Unit Tests') {
steps {
sh '''
. venv/bin/activate
pytest -v -m "not integration" \
--cov=myapp \
--cov-report=xml \
--cov-report=html \
--junitxml=test-results.xml
'''
}
}
stage('Integration Tests') {
steps {
sh '''
. venv/bin/activate
pytest -v -m integration \
--cov=myapp \
--cov-append \
--junitxml=integration-results.xml
'''
}
}
stage('Coverage Report') {
steps {
sh '''
. venv/bin/activate
coverage report
coverage html
'''
}
}
}
post {
always {
junit 'test-results.xml'
publishHTML([
reportDir: 'htmlcov',
reportFiles: 'index.html',
reportName: 'Coverage Report'
])
cleanWs()
}
success {
echo 'Tests passed!'
}
failure {
echo 'Tests failed!'
}
}
}
Docker-based Testing
# Dockerfile.test
FROM python:3.11-slim
WORKDIR /app
# Install dependencies
COPY requirements.txt requirements-dev.txt ./
RUN pip install --no-cache-dir -r requirements-dev.txt
# Copy source code
COPY src/ ./src/
COPY tests/ ./tests/
COPY pyproject.toml setup.py ./
# Install package
RUN pip install -e .
# Run tests
CMD ["pytest", "-v", "--cov=myapp", "--cov-report=html", "--cov-report=term"]
# docker-compose.test.yml
version: '3.8'
services:
test:
build:
context: .
dockerfile: Dockerfile.test
volumes:
- ./htmlcov:/app/htmlcov
- ./coverage.xml:/app/coverage.xml
environment:
- DATABASE_URL=postgresql://postgres:postgres@db/test
depends_on:
db:
condition: service_healthy
command: >
sh -c "
pytest -v
--cov=myapp
--cov-report=xml
--cov-report=html
--cov-fail-under=80
"
db:
image: postgres:15
environment:
POSTGRES_DB: test
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
# Run tests in Docker
docker-compose -f docker-compose.test.yml up --abort-on-container-exit
docker-compose -f docker-compose.test.yml down
Pre-commit Hooks
# .pre-commit-config.yaml
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.1.8
hooks:
- id: ruff
args: [--fix]
- repo: https://github.com/psf/black
rev: 23.12.1
hooks:
- id: black
- repo: local
hooks:
- id: pytest
name: pytest
entry: pytest
language: system
pass_filenames: false
always_run: true
args: ["-v", "-m", "not slow"]
# Install pre-commit
pip install pre-commit
pre-commit install
# Run manually
pre-commit run --all-files
Tox for Multi-environment Testing
# tox.ini
[tox]
envlist = py311,py312,py313,lint,type
[testenv]
deps =
pytest
pytest-cov
pytest-xdist
commands =
pytest -v --cov=myapp --cov-report=xml --cov-report=term
[testenv:lint]
deps =
ruff
black
commands =
ruff check .
black --check .
[testenv:type]
deps =
mypy
commands =
mypy src/
[testenv:coverage]
deps =
{[testenv]deps}
coverage[toml]
commands =
coverage erase
pytest --cov=myapp --cov-report=html --cov-report=term
coverage report --fail-under=80
# Run all environments
tox
# Run specific environment
tox -e py311
# Run in parallel
tox -p auto
Quick Reference
| Category | Command | Description |
|---|---|---|
| Basic | pytest |
Run all tests |
pytest test_file.py |
Run specific file | |
pytest test_file.py::test_func |
Run specific test | |
pytest -k "pattern" |
Run tests matching pattern | |
| Output | pytest -v |
Verbose output |
pytest -vv |
Extra verbose output | |
pytest -l |
Show local variables on failure | |
pytest -s |
Show print statements | |
pytest -q |
Quiet output | |
| Control | pytest -x |
Stop on first failure |
pytest --maxfail=N |
Stop after N failures | |
pytest --lf |
Run last failed tests | |
pytest --ff |
Run failed first | |
| Markers | pytest -m marker |
Run tests with marker |
pytest -m "not slow" |
Exclude slow tests | |
pytest --markers |
List all markers | |
| Coverage | pytest --cov=module |
Run with coverage |
pytest --cov-report=html |
Generate HTML report | |
pytest --cov-fail-under=80 |
Fail if coverage < 80% | |
| Parallel | pytest -n auto |
Run tests in parallel |
pytest -n 4 |
Use 4 workers | |
| Debug | pytest --pdb |
Drop into pdb on failure |
pytest --trace |
Drop into pdb at test start |
Common Fixtures
| Fixture | Description |
|---|---|
tmp_path |
Temporary directory (pathlib.Path) |
tmp_path_factory |
Create multiple temp directories |
tmpdir |
Temporary directory (legacy py.path.local; prefer tmp_path) |
capsys |
Capture stdout/stderr |
capfd |
Capture file descriptors |
caplog |
Capture logging output |
monkeypatch |
Modify objects, dictionaries, environment |
request |
Access test request information |
cache |
Store/retrieve values across test runs |
Assertion Helpers
| Helper | Usage |
|---|---|
pytest.approx(value) |
Approximate float comparison |
pytest.raises(Exception) |
Assert exception raised |
pytest.warns(Warning) |
Assert warning issued |
pytest.fail(msg) |
Explicitly fail test |
pytest.skip(msg) |
Skip test |
pytest.xfail(msg) |
Mark test as expected failure |
Common Issues and Solutions
| Issue | Cause | Solution |
|---|---|---|
| Tests not discovered | Wrong naming convention | Use test_*.py or *_test.py file names |
| Import errors | Python path not set | Add src to PYTHONPATH or use pip install -e . |
| Fixture not found | conftest.py in wrong location | Place conftest.py in test directory or parent |
| Fixtures run too often | Wrong scope | Use appropriate scope: session, module, class, function |
| Coverage incomplete | Missing test paths | Check testpaths in pytest.ini and coverage source |
| Parallel tests fail | Shared state | Use fixtures with appropriate scope or test isolation |
| Slow test discovery | Large codebase | Use --collect-only to debug, add norecursedirs to pytest.ini |
| Database tests interfere | No transaction rollback | Use transaction fixtures with rollback |
| Markers not recognised | Not registered | Add markers to pytest.ini or use --strict-markers |
| Coverage shows wrong files | Incorrect source path | Set source in .coveragerc or use --cov=path |
| Tests pass locally, fail in CI | Environment differences | Use Docker for consistent environment |
| Parametrised test unclear | Missing test IDs | Add ids parameter to @pytest.mark.parametrize |
| Fixture teardown not running | Exception in test | Use yield fixtures ensure teardown runs |
| Import from tests/ fails | Duplicate test file names | Add __init__.py to test directories, or use --import-mode=importlib |
| Assertion message unclear | Complex assertion | Split into multiple assertions or add message with assert x, msg |
Best Practices
# DO: Use descriptive test names
def test_user_creation_with_valid_email():
pass
# DON'T: Use unclear names
def test_1():
pass
# DO: One assertion concept per test
def test_user_has_email():
user = User("alice@example.com")
assert user.email == "alice@example.com"
# DON'T: Test multiple unrelated things
def test_everything():
assert user.email == "alice@example.com"
assert len(users) > 0
assert database.connected
# DO: Use fixtures for setup
@pytest.fixture
def user():
return User("alice@example.com")
def test_user_name(user):
assert user.name == "alice"
# DON'T: Repeat setup in every test
def test_user_name():
user = User("alice@example.com")
assert user.name == "alice"
# DO: Use parametrise for similar tests
@pytest.mark.parametrize("value,expected", [(1, 2), (2, 3)])
def test_increment(value, expected):
assert increment(value) == expected
# DON'T: Copy-paste tests
def test_increment_1():
assert increment(1) == 2
def test_increment_2():
assert increment(2) == 3
Related Topics
The following topics complement pytest development and would make excellent additions to your cheatsheet collection:
- Python Debugging - Using pdb, logging, and profiling tools to troubleshoot test failures and performance issues
- Python unittest - Python's built-in testing framework, useful for understanding pytest's compatibility layer
- Python Mock/unittest.mock - Mocking external dependencies and isolating units under test
- Tox - Testing across multiple Python versions and environments
- Python Type Checking (mypy) - Static type checking to catch errors before tests run
- CI/CD Patterns - Comprehensive continuous integration and deployment workflows