Microservices Architecture Patterns

A comprehensive guide to design patterns and best practices for building resilient, scalable microservices systems.

---

Overview

Microservices architecture decomposes applications into loosely coupled, independently deployable services. Each service owns its data and communicates via well-defined APIs, enabling teams to develop, deploy, and scale services independently.

⤢ zoom

---

Service Discovery

Key Concepts

Concept	Description
Service Registry	Central database storing network locations of service instances
Client-Side Discovery	Client queries registry and selects an available instance
Server-Side Discovery	Router/load balancer queries registry on client's behalf
Self-Registration	Services register themselves with the registry
Health Checks	Periodic verification that service instances are healthy
DNS-Based Discovery	Using DNS for service location (e.g., Kubernetes DNS)

Common Patterns

Consul Service Registration (Go)

package main

import (
    "github.com/hashicorp/consul/api"
)

func registerService() error {
    config := api.DefaultConfig()
    config.Address = "consul:8500"

    client, err := api.NewClient(config)
    if err != nil {
        return err
    }

    registration := &api.AgentServiceRegistration{
        ID:      "order-service-1",
        Name:    "order-service",
        Port:    8080,
        Address: "10.0.0.5",
        Check: &api.AgentServiceCheck{
            HTTP:     "http://10.0.0.5:8080/health",
            Interval: "10s",
            Timeout:  "5s",
        },
        Tags: []string{"v1", "production"},
    }

    return client.Agent().ServiceRegister(registration)
}

Eureka Client Configuration (Spring Boot)

# application.yml
eureka:
  client:
    serviceUrl:
      defaultZone: http://eureka-server:8761/eureka/
    registryFetchIntervalSeconds: 5
  instance:
    preferIpAddress: true
    leaseRenewalIntervalInSeconds: 10
    healthCheckUrlPath: /actuator/health

spring:
  application:
    name: order-service

Kubernetes Service Discovery

# service.yaml
apiVersion: v1
kind: Service
metadata:
  name: order-service
  namespace: production
spec:
  selector:
    app: order-service
  ports:
    - port: 80
      targetPort: 8080
  type: ClusterIP
---
# Other services can access via:
# http://order-service.production.svc.cluster.local

Examples

Client-Side Discovery with Ribbon (Java)

@Configuration
public class RibbonConfig {
    @Bean
    @LoadBalanced
    public RestTemplate restTemplate() {
        return new RestTemplate();
    }
}

@Service
public class OrderClient {
    @Autowired
    private RestTemplate restTemplate;

    public Order getOrder(String orderId) {
        // Ribbon automatically resolves "order-service"
        // to actual instance
        return restTemplate.getForObject(
            "http://order-service/orders/{id}",
            Order.class,
            orderId
        );
    }
}

---

API Gateway Usage

Key Concepts

Concept	Description
Reverse Proxy	Routes requests to appropriate backend services
Request Aggregation	Combines multiple service calls into single response
Protocol Translation	Converts between protocols (e.g., HTTP to gRPC)
Rate Limiting	Controls request frequency per client/API
Authentication	Centralised auth before routing to services
Request/Response Transformation	Modifies payloads between client and services

Common Patterns

Kong Gateway Configuration

# kong.yml
_format_version: "2.1"

services:
  - name: order-service
    url: http://order-service:8080
    routes:
      - name: order-route
        paths:
          - /api/v1/orders
        strip_path: true
    plugins:
      - name: rate-limiting
        config:
          minute: 100
          policy: local
      - name: jwt
        config:
          secret_is_base64: false
      - name: correlation-id
        config:
          header_name: X-Correlation-ID
          generator: uuid

  - name: user-service
    url: http://user-service:8080
    routes:
      - name: user-route
        paths:
          - /api/v1/users
        methods:
          - GET
          - POST
          - PUT

AWS API Gateway with Lambda

# serverless.yml
service: microservices-api

provider:
  name: aws
  runtime: nodejs18.x
  region: eu-west-1

functions:
  getOrders:
    handler: handlers/orders.get
    events:
      - http:
          path: orders/{id}
          method: get
          authorizer:
            name: jwtAuthorizer
            type: COGNITO_USER_POOLS
            arn: ${self:custom.cognitoArn}
          request:
            parameters:
              paths:
                id: true

custom:
  cognitoArn: arn:aws:cognito-idp:eu-west-1:xxx:userpool/xxx

NGINX as API Gateway

# nginx.conf
upstream order_service {
    server order-service-1:8080 weight=5;
    server order-service-2:8080 weight=5;
    keepalive 32;
}

upstream user_service {
    server user-service-1:8080;
    server user-service-2:8080;
}

server {
    listen 80;
    server_name api.example.com;

    # Rate limiting
    limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;

    location /api/v1/orders {
        limit_req zone=api_limit burst=20 nodelay;

        # Authentication
        auth_request /auth;

        proxy_pass http://order_service;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header X-Request-ID $request_id;
        proxy_set_header X-Real-IP $remote_addr;
    }

    location /api/v1/users {
        limit_req zone=api_limit burst=20 nodelay;
        auth_request /auth;
        proxy_pass http://user_service;
    }

    location = /auth {
        internal;
        proxy_pass http://auth-service:8080/validate;
        proxy_pass_request_body off;
        proxy_set_header Content-Length "";
        proxy_set_header X-Original-URI $request_uri;
    }
}

Examples

Request Aggregation Pattern (Node.js)

// gateway/aggregator.js
const express = require('express');
const axios = require('axios');

const app = express();

// Aggregate order details with user and payment info
app.get('/api/v1/orders/:id/details', async (req, res) => {
    const { id } = req.params;
    const correlationId = req.headers['x-correlation-id'] || uuid();

    try {
        const [order, user, payment] = await Promise.all([
            axios.get(`http://order-service/orders/${id}`, {
                headers: { 'X-Correlation-ID': correlationId }
            }),
            axios.get(`http://user-service/users/${req.userId}`, {
                headers: { 'X-Correlation-ID': correlationId }
            }),
            axios.get(`http://payment-service/payments?orderId=${id}`, {
                headers: { 'X-Correlation-ID': correlationId }
            })
        ]);

        res.json({
            order: order.data,
            customer: {
                name: user.data.name,
                email: user.data.email
            },
            payment: {
                status: payment.data.status,
                method: payment.data.method
            }
        });
    } catch (error) {
        handleError(res, error);
    }
});

---

Circuit Breaker Pattern

Key Concepts

Concept	Description
Closed State	Normal operation; requests pass through
Open State	Requests fail immediately without calling service
Half-Open State	Limited requests allowed to test recovery
Failure Threshold	Number of failures before opening circuit
Recovery Timeout	Time before attempting to close circuit
Fallback	Alternative response when circuit is open
Bulkhead	Isolates failures to prevent cascade

Common Patterns

Resilience4j Circuit Breaker (Java)

// Configuration
@Configuration
public class CircuitBreakerConfig {
    @Bean
    public CircuitBreakerRegistry circuitBreakerRegistry() {
        CircuitBreakerConfig config = CircuitBreakerConfig.custom()
            .failureRateThreshold(50)
            .waitDurationInOpenState(Duration.ofMillis(1000))
            .permittedNumberOfCallsInHalfOpenState(3)
            .slidingWindowSize(10)
            .slidingWindowType(SlidingWindowType.COUNT_BASED)
            .recordExceptions(IOException.class, TimeoutException.class)
            .ignoreExceptions(BusinessException.class)
            .build();

        return CircuitBreakerRegistry.of(config);
    }
}

// Service with Circuit Breaker
@Service
public class PaymentService {
    private final CircuitBreaker circuitBreaker;
    private final PaymentClient paymentClient;

    public PaymentService(CircuitBreakerRegistry registry,
                         PaymentClient client) {
        this.circuitBreaker = registry.circuitBreaker("payment");
        this.paymentClient = client;
    }

    public PaymentResult processPayment(PaymentRequest request) {
        return circuitBreaker.executeSupplier(() -> {
            return paymentClient.charge(request);
        });
    }

    // With fallback
    public PaymentResult processPaymentWithFallback(PaymentRequest request) {
        return Try.ofSupplier(
            CircuitBreaker.decorateSupplier(circuitBreaker,
                () -> paymentClient.charge(request)))
            .recover(throwable -> {
                log.warn("Payment service unavailable, using fallback");
                return PaymentResult.pending(request.getOrderId());
            })
            .get();
    }
}

Hystrix Pattern (Spring Boot)

@Service
public class InventoryService {

    @HystrixCommand(
        fallbackMethod = "getInventoryFallback",
        commandProperties = {
            @HystrixProperty(
                name = "circuitBreaker.requestVolumeThreshold",
                value = "10"),
            @HystrixProperty(
                name = "circuitBreaker.sleepWindowInMilliseconds",
                value = "5000"),
            @HystrixProperty(
                name = "circuitBreaker.errorThresholdPercentage",
                value = "50"),
            @HystrixProperty(
                name = "execution.isolation.thread.timeoutInMilliseconds",
                value = "3000")
        },
        threadPoolProperties = {
            @HystrixProperty(name = "coreSize", value = "10"),
            @HystrixProperty(name = "maxQueueSize", value = "100")
        }
    )
    public Inventory checkInventory(String productId) {
        return inventoryClient.getInventory(productId);
    }

    public Inventory getInventoryFallback(String productId) {
        // Return cached or default inventory
        return Inventory.unknown(productId);
    }
}

Polly Circuit Breaker (.NET)

// Startup.cs
public void ConfigureServices(IServiceCollection services)
{
    var circuitBreakerPolicy = Policy
        .Handle<HttpRequestException>()
        .OrResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
        .CircuitBreakerAsync(
            handledEventsAllowedBeforeBreaking: 5,
            durationOfBreak: TimeSpan.FromSeconds(30),
            onBreak: (result, duration) =>
            {
                _logger.LogWarning(
                    "Circuit breaker opened for {Duration}s",
                    duration.TotalSeconds);
            },
            onReset: () =>
            {
                _logger.LogInformation("Circuit breaker reset");
            },
            onHalfOpen: () =>
            {
                _logger.LogInformation("Circuit breaker half-open");
            }
        );

    var retryPolicy = Policy
        .Handle<HttpRequestException>()
        .WaitAndRetryAsync(3,
            retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)));

    var policyWrap = Policy.WrapAsync(circuitBreakerPolicy, retryPolicy);

    services.AddHttpClient<IPaymentService, PaymentService>()
        .AddPolicyHandler(policyWrap);
}

Examples

Circuit Breaker with Bulkhead Pattern

@Service
public class OrderService {
    private final CircuitBreaker circuitBreaker;
    private final Bulkhead bulkhead;
    private final TimeLimiter timeLimiter;

    public OrderService(CircuitBreakerRegistry cbRegistry,
                       BulkheadRegistry bhRegistry,
                       TimeLimiterRegistry tlRegistry) {
        this.circuitBreaker = cbRegistry.circuitBreaker("orders");
        this.bulkhead = bhRegistry.bulkhead("orders");
        this.timeLimiter = tlRegistry.timeLimiter("orders");
    }

    public CompletableFuture<Order> createOrder(OrderRequest request) {
        Supplier<CompletableFuture<Order>> supplier = () ->
            CompletableFuture.supplyAsync(() ->
                orderClient.create(request));

        Supplier<CompletableFuture<Order>> decoratedSupplier =
            Decorators.ofSupplier(supplier)
                .withTimeLimiter(timeLimiter)
                .withBulkhead(bulkhead)
                .withCircuitBreaker(circuitBreaker)
                .withFallback(
                    asList(TimeoutException.class,
                           BulkheadFullException.class,
                           CallNotPermittedException.class),
                    e -> CompletableFuture.completedFuture(
                        Order.queued(request)))
                .decorate();

        return decoratedSupplier.get();
    }
}

---

Saga Pattern for Distributed Transactions

Key Concepts

Concept	Description
Saga	Sequence of local transactions across services
Compensating Transaction	Undoes changes made by a previous transaction
Choreography	Services coordinate via events without central controller
Orchestration	Central coordinator directs saga participants
Semantic Lock	Prevents concurrent updates during saga
Commutative Updates	Order-independent updates for better reliability

⤢ zoom

Common Patterns

Orchestration-Based Saga (Java)

// Saga Orchestrator
@Service
public class OrderSagaOrchestrator {

    @Autowired
    private SagaExecutionCoordinator coordinator;

    public OrderResult createOrder(CreateOrderCommand command) {
        SagaDefinition<OrderSagaData> sagaDefinition =
            step()
                .invokeParticipant(this::createOrder)
                .withCompensation(this::cancelOrder)
            .step()
                .invokeParticipant(this::reserveCredit)
                .withCompensation(this::releaseCredit)
            .step()
                .invokeParticipant(this::reserveInventory)
                .withCompensation(this::releaseInventory)
            .step()
                .invokeParticipant(this::scheduleShipment)
                .withCompensation(this::cancelShipment)
            .build();

        OrderSagaData sagaData = new OrderSagaData(command);

        return coordinator.execute(sagaDefinition, sagaData);
    }

    private CommandWithDestination createOrder(OrderSagaData data) {
        return send(new CreateOrderCommand(
            data.getOrderId(),
            data.getCustomerId(),
            data.getOrderItems()))
            .to("order-service")
            .build();
    }

    private CommandWithDestination cancelOrder(OrderSagaData data) {
        return send(new CancelOrderCommand(data.getOrderId()))
            .to("order-service")
            .build();
    }

    private CommandWithDestination reserveCredit(OrderSagaData data) {
        return send(new ReserveCreditCommand(
            data.getCustomerId(),
            data.getOrderTotal()))
            .to("payment-service")
            .build();
    }

    private CommandWithDestination releaseCredit(OrderSagaData data) {
        return send(new ReleaseCreditCommand(
            data.getCustomerId(),
            data.getOrderTotal()))
            .to("payment-service")
            .build();
    }

    // ... other participant methods
}

Choreography-Based Saga (Event-Driven)

// Order Service
@Service
public class OrderService {
    @Autowired
    private EventPublisher eventPublisher;

    @Transactional
    public Order createOrder(CreateOrderRequest request) {
        Order order = orderRepository.save(
            new Order(request, OrderStatus.PENDING));

        eventPublisher.publish(new OrderCreatedEvent(
            order.getId(),
            order.getCustomerId(),
            order.getItems(),
            order.getTotal()
        ));

        return order;
    }

    @EventListener
    public void handlePaymentFailed(PaymentFailedEvent event) {
        Order order = orderRepository.findById(event.getOrderId())
            .orElseThrow();
        order.setStatus(OrderStatus.CANCELLED);
        orderRepository.save(order);

        eventPublisher.publish(new OrderCancelledEvent(order.getId()));
    }
}

// Payment Service
@Service
public class PaymentService {
    @EventListener
    public void handleOrderCreated(OrderCreatedEvent event) {
        try {
            Payment payment = processPayment(
                event.getCustomerId(),
                event.getTotal());

            eventPublisher.publish(new PaymentProcessedEvent(
                event.getOrderId(),
                payment.getId()
            ));
        } catch (InsufficientFundsException e) {
            eventPublisher.publish(new PaymentFailedEvent(
                event.getOrderId(),
                "Insufficient funds"
            ));
        }
    }
}

// Inventory Service
@Service
public class InventoryService {
    @EventListener
    public void handlePaymentProcessed(PaymentProcessedEvent event) {
        try {
            reserveInventory(event.getOrderId());

            eventPublisher.publish(new InventoryReservedEvent(
                event.getOrderId()
            ));
        } catch (OutOfStockException e) {
            eventPublisher.publish(new InventoryReservationFailedEvent(
                event.getOrderId()
            ));
        }
    }

    @EventListener
    public void handleOrderCancelled(OrderCancelledEvent event) {
        releaseInventory(event.getOrderId());
    }
}

Examples

Saga State Machine with Spring State Machine

@Configuration
@EnableStateMachineFactory
public class OrderSagaStateMachineConfig
    extends EnumStateMachineConfigurerAdapter<SagaState, SagaEvent> {

    @Override
    public void configure(StateMachineStateConfigurer<SagaState, SagaEvent> states)
        throws Exception {
        states
            .withStates()
            .initial(SagaState.ORDER_PENDING)
            .state(SagaState.PAYMENT_PENDING)
            .state(SagaState.INVENTORY_PENDING)
            .state(SagaState.SHIPPING_PENDING)
            .end(SagaState.ORDER_COMPLETED)
            .end(SagaState.ORDER_FAILED);
    }

    @Override
    public void configure(
        StateMachineTransitionConfigurer<SagaState, SagaEvent> transitions)
        throws Exception {
        transitions
            .withExternal()
                .source(SagaState.ORDER_PENDING)
                .target(SagaState.PAYMENT_PENDING)
                .event(SagaEvent.ORDER_CREATED)
                .action(paymentAction())
            .and()
            .withExternal()
                .source(SagaState.PAYMENT_PENDING)
                .target(SagaState.INVENTORY_PENDING)
                .event(SagaEvent.PAYMENT_PROCESSED)
                .action(inventoryAction())
            .and()
            .withExternal()
                .source(SagaState.PAYMENT_PENDING)
                .target(SagaState.ORDER_FAILED)
                .event(SagaEvent.PAYMENT_FAILED)
                .action(compensateOrderAction());
    }
}

---

Data Consistency Strategies

Key Concepts

Concept	Description
Eventual Consistency	Data will become consistent over time
Strong Consistency	All reads return most recent write
Event Sourcing	Store state changes as sequence of events
CQRS	Separate read and write models
Outbox Pattern	Reliable event publishing with database transactions
Change Data Capture	Stream database changes as events

Common Patterns

Transactional Outbox Pattern

// Order Service with Outbox
@Service
public class OrderService {
    @Autowired
    private OrderRepository orderRepository;
    @Autowired
    private OutboxRepository outboxRepository;

    @Transactional
    public Order createOrder(CreateOrderRequest request) {
        // Save order
        Order order = orderRepository.save(
            new Order(request, OrderStatus.PENDING));

        // Save event to outbox (same transaction)
        OutboxEvent event = new OutboxEvent(
            UUID.randomUUID(),
            "OrderCreated",
            "Order",
            order.getId(),
            JsonUtils.toJson(new OrderCreatedEvent(
                order.getId(),
                order.getCustomerId(),
                order.getItems()
            )),
            Instant.now()
        );
        outboxRepository.save(event);

        return order;
    }
}

// Outbox Publisher (Polling)
@Service
public class OutboxPublisher {
    @Scheduled(fixedDelay = 100)
    @Transactional
    public void publishEvents() {
        List<OutboxEvent> events = outboxRepository
            .findByPublishedFalseOrderByCreatedAtAsc(100);

        for (OutboxEvent event : events) {
            try {
                kafkaTemplate.send(
                    event.getAggregateType(),
                    event.getAggregateId(),
                    event.getPayload()
                ).get();

                event.setPublished(true);
                outboxRepository.save(event);
            } catch (Exception e) {
                log.error("Failed to publish event: {}", event.getId(), e);
            }
        }
    }
}

Event Sourcing with Axon Framework

// Order Aggregate
@Aggregate
public class OrderAggregate {
    @AggregateIdentifier
    private String orderId;
    private OrderStatus status;
    private List<OrderItem> items;

    @CommandHandler
    public OrderAggregate(CreateOrderCommand command) {
        AggregateLifecycle.apply(new OrderCreatedEvent(
            command.getOrderId(),
            command.getCustomerId(),
            command.getItems()
        ));
    }

    @EventSourcingHandler
    public void on(OrderCreatedEvent event) {
        this.orderId = event.getOrderId();
        this.status = OrderStatus.PENDING;
        this.items = event.getItems();
    }

    @CommandHandler
    public void handle(ConfirmOrderCommand command) {
        if (status != OrderStatus.PENDING) {
            throw new IllegalStateException(
                "Order cannot be confirmed in state: " + status);
        }
        AggregateLifecycle.apply(new OrderConfirmedEvent(orderId));
    }

    @EventSourcingHandler
    public void on(OrderConfirmedEvent event) {
        this.status = OrderStatus.CONFIRMED;
    }
}

// Query Side (CQRS)
@Service
public class OrderQueryHandler {
    @Autowired
    private OrderViewRepository repository;

    @EventHandler
    public void on(OrderCreatedEvent event) {
        OrderView view = new OrderView(
            event.getOrderId(),
            event.getCustomerId(),
            event.getItems(),
            OrderStatus.PENDING
        );
        repository.save(view);
    }

    @EventHandler
    public void on(OrderConfirmedEvent event) {
        repository.updateStatus(event.getOrderId(), OrderStatus.CONFIRMED);
    }

    @QueryHandler
    public OrderView handle(GetOrderQuery query) {
        return repository.findById(query.getOrderId())
            .orElseThrow(() -> new OrderNotFoundException(query.getOrderId()));
    }
}

Change Data Capture with Debezium

// Debezium Connector Configuration
{
    "name": "order-connector",
    "config": {
        "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
        "database.hostname": "postgres",
        "database.port": "5432",
        "database.user": "debezium",
        "database.password": "secret",
        "database.dbname": "orders",
        "database.server.name": "orderdb",
        "table.include.list": "public.orders,public.order_items",
        "plugin.name": "pgoutput",
        "publication.name": "orders_publication",
        "slot.name": "orders_slot",
        "transforms": "unwrap",
        "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState",
        "transforms.unwrap.add.fields": "op,table,ts_ms",
        "key.converter": "org.apache.kafka.connect.json.JsonConverter",
        "value.converter": "org.apache.kafka.connect.json.JsonConverter"
    }
}

Examples

Idempotent Consumer Pattern

@Service
public class IdempotentEventHandler {
    @Autowired
    private ProcessedEventRepository processedEventRepository;

    @KafkaListener(topics = "order-events")
    @Transactional
    public void handleEvent(OrderEvent event) {
        String eventId = event.getEventId();

        // Check if already processed
        if (processedEventRepository.existsById(eventId)) {
            log.info("Event {} already processed, skipping", eventId);
            return;
        }

        // Process event
        processEvent(event);

        // Mark as processed
        processedEventRepository.save(new ProcessedEvent(
            eventId,
            Instant.now()
        ));
    }
}

---

Logging and Tracing Across Services

Key Concepts

Concept	Description
Correlation ID	Unique identifier linking all logs for a request
Distributed Tracing	End-to-end request path visualisation
Span	Single unit of work within a trace
Context Propagation	Passing trace context between services
Structured Logging	JSON-formatted logs for parsing
Centralised Logging	Aggregating logs from all services

Common Patterns

OpenTelemetry Setup (Java)

// OpenTelemetry Configuration
@Configuration
public class TracingConfig {
    @Bean
    public OpenTelemetry openTelemetry() {
        Resource resource = Resource.getDefault()
            .merge(Resource.create(Attributes.of(
                ResourceAttributes.SERVICE_NAME, "order-service",
                ResourceAttributes.SERVICE_VERSION, "1.0.0"
            )));

        SdkTracerProvider tracerProvider = SdkTracerProvider.builder()
            .addSpanProcessor(BatchSpanProcessor.builder(
                OtlpGrpcSpanExporter.builder()
                    .setEndpoint("http://otel-collector:4317")
                    .build())
                .build())
            .setResource(resource)
            .build();

        SdkMeterProvider meterProvider = SdkMeterProvider.builder()
            .registerMetricReader(
                PeriodicMetricReader.builder(
                    OtlpGrpcMetricExporter.builder()
                        .setEndpoint("http://otel-collector:4317")
                        .build())
                    .build())
            .setResource(resource)
            .build();

        return OpenTelemetrySdk.builder()
            .setTracerProvider(tracerProvider)
            .setMeterProvider(meterProvider)
            .setPropagators(ContextPropagators.create(
                TextMapPropagator.composite(
                    W3CTraceContextPropagator.getInstance(),
                    W3CBaggagePropagator.getInstance())))
            .build();
    }
}

// Custom Span Creation
@Service
public class OrderService {
    private final Tracer tracer;

    public OrderService(OpenTelemetry openTelemetry) {
        this.tracer = openTelemetry.getTracer("order-service");
    }

    public Order createOrder(CreateOrderRequest request) {
        Span span = tracer.spanBuilder("createOrder")
            .setSpanKind(SpanKind.INTERNAL)
            .setAttribute("order.customer_id", request.getCustomerId())
            .setAttribute("order.items_count", request.getItems().size())
            .startSpan();

        try (Scope scope = span.makeCurrent()) {
            Order order = processOrder(request);
            span.setAttribute("order.id", order.getId());
            span.setStatus(StatusCode.OK);
            return order;
        } catch (Exception e) {
            span.recordException(e);
            span.setStatus(StatusCode.ERROR, e.getMessage());
            throw e;
        } finally {
            span.end();
        }
    }
}

Structured Logging with Correlation ID

// MDC Filter for Correlation ID
@Component
public class CorrelationIdFilter extends OncePerRequestFilter {
    private static final String CORRELATION_ID = "X-Correlation-ID";

    @Override
    protected void doFilterInternal(
            HttpServletRequest request,
            HttpServletResponse response,
            FilterChain filterChain) throws ServletException, IOException {

        String correlationId = request.getHeader(CORRELATION_ID);
        if (correlationId == null) {
            correlationId = UUID.randomUUID().toString();
        }

        MDC.put("correlationId", correlationId);
        MDC.put("serviceName", "order-service");

        response.setHeader(CORRELATION_ID, correlationId);

        try {
            filterChain.doFilter(request, response);
        } finally {
            MDC.clear();
        }
    }
}

// Logback Configuration
// logback-spring.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <appender name="JSON" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LogstashEncoder">
            <includeMdcKeyName>correlationId</includeMdcKeyName>
            <includeMdcKeyName>serviceName</includeMdcKeyName>
            <customFields>
                {"environment":"production","version":"1.0.0"}
            </customFields>
        </encoder>
    </appender>

    <root level="INFO">
        <appender-ref ref="JSON" />
    </root>
</configuration>

ELK Stack Configuration

# docker-compose.yml
version: '3.8'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
    ports:
      - "9200:9200"

  logstash:
    image: docker.elastic.co/logstash/logstash:8.11.0
    volumes:
      - ./logstash.conf:/usr/share/logstash/pipeline/logstash.conf
    depends_on:
      - elasticsearch

  kibana:
    image: docker.elastic.co/kibana/kibana:8.11.0
    ports:
      - "5601:5601"
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
    depends_on:
      - elasticsearch

# logstash.conf
input {
  tcp {
    port => 5000
    codec => json
  }
}

filter {
  if [correlationId] {
    mutate {
      add_field => { "[@metadata][correlationId]" => "%{correlationId}" }
    }
  }
}

output {
  elasticsearch {
    hosts => ["elasticsearch:9200"]
    index => "microservices-logs-%{+YYYY.MM.dd}"
  }
}

Examples

Jaeger Tracing Integration

# OpenTelemetry Collector Configuration
# otel-collector-config.yaml
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 1s
    send_batch_size: 1024

exporters:
  jaeger:
    endpoint: jaeger:14250
    tls:
      insecure: true

  prometheus:
    endpoint: "0.0.0.0:8889"

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [jaeger]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [prometheus]

---

Deployment Strategies

Key Concepts

Concept	Description
Blue/Green Deployment	Two identical environments; instant switchover
Canary Release	Gradual rollout to subset of users
Rolling Update	Incremental replacement of instances
Feature Flags	Toggle features without deployment
A/B Testing	Compare different versions with user groups
Shadow Deployment	Route copy of traffic to new version

Common Patterns

Blue/Green Deployment with Kubernetes

# blue-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-service-blue
  labels:
    app: order-service
    version: blue
spec:
  replicas: 3
  selector:
    matchLabels:
      app: order-service
      version: blue
  template:
    metadata:
      labels:
        app: order-service
        version: blue
    spec:
      containers:
        - name: order-service
          image: order-service:1.0.0
          ports:
            - containerPort: 8080
---
# green-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-service-green
  labels:
    app: order-service
    version: green
spec:
  replicas: 3
  selector:
    matchLabels:
      app: order-service
      version: green
  template:
    metadata:
      labels:
        app: order-service
        version: green
    spec:
      containers:
        - name: order-service
          image: order-service:2.0.0
          ports:
            - containerPort: 8080
---
# service.yaml - Switch by updating selector
apiVersion: v1
kind: Service
metadata:
  name: order-service
spec:
  selector:
    app: order-service
    version: blue  # Change to 'green' to switch
  ports:
    - port: 80
      targetPort: 8080

Canary Deployment with Istio

# virtual-service.yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: order-service
spec:
  hosts:
    - order-service
  http:
    - match:
        - headers:
            x-canary:
              exact: "true"
      route:
        - destination:
            host: order-service
            subset: canary
    - route:
        - destination:
            host: order-service
            subset: stable
          weight: 90
        - destination:
            host: order-service
            subset: canary
          weight: 10
---
# destination-rule.yaml
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: order-service
spec:
  host: order-service
  subsets:
    - name: stable
      labels:
        version: v1
    - name: canary
      labels:
        version: v2

Argo Rollouts Canary

# rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: order-service
spec:
  replicas: 5
  selector:
    matchLabels:
      app: order-service
  template:
    metadata:
      labels:
        app: order-service
    spec:
      containers:
        - name: order-service
          image: order-service:2.0.0
          ports:
            - containerPort: 8080
  strategy:
    canary:
      canaryService: order-service-canary
      stableService: order-service-stable
      trafficRouting:
        istio:
          virtualService:
            name: order-service-vsvc
      steps:
        - setWeight: 10
        - pause: { duration: 5m }
        - setWeight: 30
        - pause: { duration: 5m }
        - setWeight: 50
        - pause: { duration: 5m }
        - setWeight: 100
      analysis:
        templates:
          - templateName: success-rate
        startingStep: 1
        args:
          - name: service-name
            value: order-service-canary
---
# analysis-template.yaml
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: success-rate
spec:
  args:
    - name: service-name
  metrics:
    - name: success-rate
      interval: 1m
      successCondition: result[0] >= 0.95
      provider:
        prometheus:
          address: http://prometheus:9090
          query: |
            sum(rate(http_requests_total{
              service="{{args.service-name}}",
              status=~"2.*"
            }[5m])) /
            sum(rate(http_requests_total{
              service="{{args.service-name}}"
            }[5m]))

Examples

Feature Flags with LaunchDarkly

@Service
public class OrderService {
    @Autowired
    private LDClient ldClient;

    public Order createOrder(CreateOrderRequest request, User user) {
        LDContext context = LDContext.builder(user.getId())
            .set("email", user.getEmail())
            .set("country", user.getCountry())
            .build();

        // Check feature flag
        boolean useNewPricing = ldClient.boolVariation(
            "new-pricing-algorithm",
            context,
            false
        );

        Order order;
        if (useNewPricing) {
            order = createOrderWithNewPricing(request);
        } else {
            order = createOrderWithLegacyPricing(request);
        }

        return order;
    }
}

Kubernetes Rolling Update

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-service
spec:
  replicas: 5
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1        # Max pods above desired during update
      maxUnavailable: 0  # Ensure zero downtime
  selector:
    matchLabels:
      app: order-service
  template:
    metadata:
      labels:
        app: order-service
    spec:
      containers:
        - name: order-service
          image: order-service:2.0.0
          ports:
            - containerPort: 8080
          readinessProbe:
            httpGet:
              path: /health/ready
              port: 8080
            initialDelaySeconds: 10
            periodSeconds: 5
          livenessProbe:
            httpGet:
              path: /health/live
              port: 8080
            initialDelaySeconds: 30
            periodSeconds: 10

---

Quick Reference

Pattern	Use Case	Trade-offs
Service Discovery	Dynamic service location	Additional infrastructure; network dependency
API Gateway	Single entry point; cross-cutting concerns	Single point of failure; added latency
Circuit Breaker	Prevent cascade failures	Complexity; tuning required
Saga (Orchestration)	Complex workflows; centralised control	Single point of failure; coupling
Saga (Choreography)	Simple workflows; loose coupling	Difficult to track; cyclic dependencies
Event Sourcing	Audit trail; temporal queries	Storage costs; complexity
CQRS	Different read/write needs	Eventual consistency; duplicate code
Outbox Pattern	Reliable event publishing	Additional table; polling overhead
Blue/Green	Zero-downtime; instant rollback	Double infrastructure cost
Canary	Gradual rollout; risk reduction	Complex routing; monitoring required

Communication Patterns Comparison

Pattern	Coupling	Latency	Reliability	Complexity
Synchronous HTTP	Tight	Low	Lower	Low
Asynchronous Messaging	Loose	Higher	Higher	Medium
Event-Driven	Loose	Variable	High	High
gRPC	Medium	Very Low	Medium	Medium

---

Common Issues and Solutions

Service Discovery Failures

Problem: Services cannot find each other

Error: No instances available for order-service

Solutions:

# Increase health check tolerance
eureka:
  instance:
    leaseExpirationDurationInSeconds: 90
    leaseRenewalIntervalInSeconds: 30

# Use retry with backoff
spring:
  cloud:
    loadbalancer:
      retry:
        enabled: true
        maxRetriesOnSameServiceInstance: 1
        maxRetriesOnNextServiceInstance: 2

Circuit Breaker Misconfiguration

Problem: Circuit opens too frequently or not at all

Solutions:

// Adjust thresholds based on traffic patterns
CircuitBreakerConfig config = CircuitBreakerConfig.custom()
    // Lower threshold for critical services
    .failureRateThreshold(30)
    // Longer window for low-traffic services
    .slidingWindowSize(20)
    .slidingWindowType(SlidingWindowType.TIME_BASED)
    .minimumNumberOfCalls(10)
    // Shorter recovery time for testing
    .waitDurationInOpenState(Duration.ofSeconds(10))
    .build();

Saga Compensation Failures

Problem: Compensating transactions fail, leaving inconsistent state

Solutions:

// Implement retry with exponential backoff
@Retryable(
    value = {CompensationException.class},
    maxAttempts = 5,
    backoff = @Backoff(delay = 1000, multiplier = 2)
)
public void compensate(SagaData data) {
    // Compensation logic
}

// Use dead letter queue for failed compensations
@KafkaListener(topics = "saga-compensations-dlq")
public void handleFailedCompensation(CompensationEvent event) {
    // Manual intervention or alternative compensation
    alertOps(event);
}

Distributed Tracing Gaps

Problem: Traces incomplete across service boundaries

Solutions:

// Ensure context propagation in async operations
@Async
public CompletableFuture<Result> asyncOperation() {
    // Capture current context
    Context context = Context.current();

    return CompletableFuture.supplyAsync(() -> {
        // Restore context in async thread
        try (Scope scope = context.makeCurrent()) {
            return doWork();
        }
    });
}

// Propagate headers in HTTP clients
@Bean
public RestTemplate restTemplate(OpenTelemetry openTelemetry) {
    RestTemplate template = new RestTemplate();
    template.setInterceptors(Collections.singletonList(
        new TracingClientHttpRequestInterceptor(openTelemetry)
    ));
    return template;
}

Canary Deployment Metric Skew

Problem: Canary metrics unreliable due to traffic differences

Solutions:

# Use consistent hashing for user routing
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: order-service
spec:
  http:
    - route:
        - destination:
            host: order-service
            subset: stable
          weight: 90
        - destination:
            host: order-service
            subset: canary
          weight: 10
      # Sticky sessions based on user
      headers:
        request:
          set:
            x-user-hash: "%REQ(x-user-id)%"

Event Ordering Issues

Problem: Events processed out of order causing data inconsistencies

Solutions:

// Use partition keys for ordering
kafkaTemplate.send(
    "order-events",
    order.getCustomerId(),  // Partition key ensures order per customer
    event
);

// Implement event versioning
public class OrderEvent {
    private String eventId;
    private long sequenceNumber;
    private Instant timestamp;
    // ...
}

// Buffer and reorder events
@Service
public class EventReorderBuffer {
    private final Map<String, PriorityQueue<Event>> buffers =
        new ConcurrentHashMap<>();

    public void processEvent(Event event) {
        String key = event.getAggregateId();
        PriorityQueue<Event> buffer = buffers.computeIfAbsent(
            key,
            k -> new PriorityQueue<>(
                Comparator.comparing(Event::getSequenceNumber)));

        buffer.offer(event);
        processInOrder(key, buffer);
    }
}

---

Related Topics

To build a complete understanding of microservices architecture, consider exploring these complementary areas:

1. Container Orchestration (Kubernetes) - Deploying and managing microservices at scale with automated scaling, self-healing, and service mesh integration

2. Event-Driven Architecture - Deep dive into message brokers (Kafka, RabbitMQ), event streaming, and reactive patterns for asynchronous communication

3. Service Mesh (Istio/Linkerd) - Advanced traffic management, security (mTLS), and observability without application code changes

4. Domain-Driven Design (DDD) - Bounded contexts, aggregates, and strategic patterns for decomposing systems into microservices

5. API Design and Documentation - RESTful API best practices, OpenAPI/Swagger, GraphQL, and gRPC for inter-service communication

6. Cloud-Native Security - Zero-trust architecture, secrets management, OAuth2/OIDC, and securing service-to-service communication