15 - The Sharding and Index Table Patterns
Starting to think about the data layer of your application and concerned about scalability? In this episode, Chris is joined by Steph Martin to cover two complementary cloud data patterns: Sharding and Index Table.
The Sharding Pattern
Sharding horizontally scales a data store by partitioning data across multiple shards (database nodes), so no single node becomes a bottleneck. Key design decisions include:
- Shard key strategies: Lookup sharding (explicit map of key to shard), range sharding (e.g., customer IDs 1–1000 on shard A), and hash sharding (consistent hashing to distribute load evenly).
- Cross-shard queries: Designing your shard key to avoid cross-shard queries is critical — querying across shards is expensive and may require scatter-gather patterns.
- Noisy neighbour problem: In multi-tenant SaaS, a single large tenant can saturate a shard and degrade performance for co-tenants. Good key design is the best prevention.
- Data sovereignty: Sharding is an elegant solution for customers that require data residency in specific regions.
- Azure SQL Database elastic client library: Microsoft’s tooling for managing shard maps and routing queries to the correct shard automatically.
The Index Table Pattern
NoSQL data stores (such as Cassandra, Azure Cosmos DB, or Azure Table Storage) often only support efficient queries on a primary key. The Index Table pattern solves this by maintaining a separate, denormalised table that duplicates a subset of the data, indexed by the attribute you want to query.
Key considerations include:
- When to use a full index table vs a normalised index table (storing only the primary key, then doing a second-hop lookup).
- Using Cosmos DB change feed to keep index tables consistent with the primary data source asynchronously.
- Combining sharding and index tables: shard the primary store by one key, maintain index tables for secondary access patterns.
Both patterns are particularly relevant in multi-tenant cloud architectures where scalability, query efficiency, and data isolation are first-class concerns.
Related Content
14 - The Deployment Stamps Pattern
The Deployment Stamps pattern is a powerful cloud architecture approach for scaling, resilience, and multi-tenancy. In this episode, Chris Reddington is joined by John Downs — who contributed the pattern to the Azure Architecture Center — to explore how stamping out independent copies of your application stack across regions enables geographic distribution, data sovereignty, isolated failure domains, and deployment rings for staged rollouts. Discover when to use this pattern, how Azure itself relies on it internally, and the key considerations around request routing, cross-stamp querying, and disaster recovery planning.
1 - Requirements in Context
Every cloud project starts with requirements. In this episode, Chris explores the critical pillars of cloud architecture: resilience (SLA, RTO, RPO, MTTR, MTBF), scalability, performance, and cost. Learn why defining requirements upfront—before drawing architecture diagrams—is essential, and how the same on-premises thinking about availability translates directly into the cloud.
13 - Tales from the Real World - Defying DDOS
DDoS attacks have scaled to cloud-level volumes — terabits per second — that on-premises hardware simply cannot absorb. In this episode, Chris is joined by Cam Adams, an engineering manager from Brisbane, Australia, who shares first-hand experience helping customers across Asia-Pacific defend against distributed denial-of-service attacks using Azure. Whether workloads are fully in the cloud, in a hybrid state, or entirely on-premises, Cam explains how Azure can act as a scalable, cloud-powered defensive layer — and why the time to act is before an attack hits, not after.