System Design ( Day - 17 )

System Design Trade off’s

System design tradeoffs are situations where choosing one feature or approach means sacrificing another. These trade-offs involve balancing factors like cost, performance, scalability, reliability, and maintainability.

Memory vs. Latency

More Memory, Lower Latency
Caching frequently accessed data in memory (L1/L2 CPU Memory, in Memory Key value Store)
drastically reduces the access time from milliseconds to nanoseconds, by having larger working sets in Ram, systems avoid costly I/O operations and network calls, by reducing the response time.

Memory Constraints
by allocating more memory for cache increases the hardware cost and lead to underutilised resources, if Cache hit rate doesn’t justify the size. in extreme cases data eviction and garbage collection will be higher by increasing latency spikes.

Throughput vs. Latency

High Throughput, Higher Latency
Batching, queuing, and pipelining increase aggregate work per unit time (e.g., thousands of messages per second) but introduce queuing delays, raising per-request latency, Buffering writes into large segments (as in LSM-trees) yields tremendous write throughput while adding compaction pauses that can elevate tail latencies.
Low Latency, Lower Throughput
Processing each request immediately minimizes end-to-end latency but incurs overhead per operation (e.g., syscalls, network round-trips), capping maximum throughput. Real-time systems (gaming, bidding) often favor this mode to ensure responsiveness, accepting reduced batch efficiencies.

Consistency vs. Availability

CAP Theorem
In the presence of network partitions, distributed systems must choose between:
Consistency (C): All clients see the same up-to-date data
Availability (A): Every request receives a response, even it fails.

CA vs. AP vs. CP
CP (Consistency over Availability): Systems like traditional RDBMS prioritise strict consistency, returning errors or timeouts during partitions
AP (Availability over Consistency): NoSQL stores (e.g., Dynamo) remain online under network splits, offering eventual consistency

Latency vs. Accuracy

Faster, Less Accurate
Approximate algorithms (e.g., probabilistic data structures, early-exit ML inference) reduce computation time at the cost of some error bound, suitable for recommendations or monitoring dashboards.
Slower, More Accurate
Exact computations (e.g., precise ray-tracing, full data scans) guarantee correctness but incur longer processing times, necessary for billing, analytics, or scientific simulations
Serve approximate results first, then refine them in the background
Offer both “fast” and “accurate” endpoints to match client needs.

Scalability vs. Complexity

As systems scale out horizontally - adding nodes or microservices the operational complexity of deployment, monitoring, and distributed coordination grows, Monolithic or smaller architectures ease management but hit ceiling when demand skyrockets . Careful automation, orchestration, and abstraction can mitigate complexity while preserving scalability.