26 - The Pub Sub, Priority Queue and Pipes and Filter Patterns
Chris and Willy cover three fundamental messaging patterns in this episode of Architecting for the Cloud, One Pattern at a Time.
Publish-Subscribe (Pub/Sub) transitions systems from point-to-point messaging (one sender, one receiver) to a multicast model where a single event triggers multiple independent subscribers simultaneously. Azure Service Bus topics and Azure Event Grid are the primary Azure implementations. Canonical use cases include:
- Insurance aggregators fanning out quote requests to 15–50 backend services in parallel
- Credit check pipelines triggering multiple reference agencies from a single event
- Account sign-up flows dispatching to billing, provisioning, and notification services concurrently
Key benefit: subscribers are fully decoupled — adding a new consumer requires only a new subscription, with no changes to the producer.
Priority Queue builds on the queuing foundations covered in earlier episodes (Queue-Based Load Leveling, Competing Consumers). By assigning priority levels to messages, high-urgency work is processed before lower-priority items even under consumer load — a critical capability for SLA-bound workloads.
Pipes and Filters decomposes a complex processing task into a linear chain of discrete, single-responsibility transformation steps. Each filter receives a message, transforms it, and passes it to the next stage. Benefits include independently scalable stages, reusable filter components, and reduced overall complexity. The pattern pairs naturally with the Compensating Transaction pattern when a mid-pipeline failure requires rollback of earlier completed steps.
All three patterns reinforce the same core principle: decoupling message producers from consumers improves resilience, scalability, and maintainability in distributed systems.
Related Content
21 - The Queue Based Load Levelling and Competing Consumers Pattern
Cloud with ChrisDo you have an application with specific scalability and continuity-of-service requirements? What happens when traffic spikes dramatically — think a major concert or FIFA World Cup ticket sale crashing a site? In this Architecting for the Cloud episode, Chris and Will Eastbury walk through three closely related patterns: Queue-Based Load Levelling, Competing Consumers, and the Asynchronous Request-Reply pattern. They explore how message queues act as shock absorbers for traffic spikes, how competing consumers enable elastic horizontal scaling, and how async request-reply lets you retrofit these patterns into existing architectures with minimal disruption. Key trade-offs covered include queue depth limits, Azure Service Bus configuration, distributed tracing with Application Insights, and when the added complexity genuinely justifies reaching for these patterns.
19 - The Event Sourcing, Materialized View and CQRS Patterns
What if you stored not just the current state of your data, but every event that produced it? The Event Sourcing pattern captures the full history of changes as an append-only log — enabling audit trails, temporal queries, and history replay. Combined with Materialized Views for efficient read-side querying and CQRS for command/query separation, these three patterns form a cornerstone of modern event-driven architecture. Chris and Steph explore all three in this episode of Architecting for the Cloud.
20 - The Anti-corruption layer, Gateway Aggregation and Gateway Routing patterns
Cloud with ChrisPeter Piper joins Chris Reddington for another episode in the Architecting for the Cloud, One Pattern at a Time series. Building on the Façade and Strangler patterns, they explore three related cloud design patterns: the Anti-Corruption Layer (translating between legacy and modern domain models), Gateway Aggregation (collapsing multiple backend calls into a single client response), and Gateway Routing (layer-7 routing to decouple consumers from versioned backend services). Real Azure service examples — including API Management, Application Gateway, and Azure Front Door — are used throughout.