Monday, January 18, 2010

Design Principles for Modern Distributed Systems

The design principles used for Amazon's S3 are generally applicable to modern distributed systems design. Quoting the S3 design principles:

Amazon S3 Design Principles

The following principles of distributed system design were used to meet Amazon S3 requirements:

  • Decentralization: Use fully decentralized techniques to remove scaling bottlenecks and single points of failure.
  • Asynchrony: The system makes progress under all circumstances.
  • Autonomy: The system is designed such that individual components can make decisions based on local information.
  • Local responsibility: Each individual component is responsible for achieving its consistency; this is never the burden of its peers.
  • Controlled concurrency: Operations are designed such that no or limited concurrency control is required.
  • Failure tolerant: The system considers the failure of components to be a normal mode of operation, and continues operation with no or minimal interruption.
  • Controlled parallelism: Abstractions used in the system are of such granularity that parallelism can be used to improve performance and robustness of recovery or the introduction of new nodes.
  • Decompose into small well-understood building blocks: Do not try to provide a single service that does everything for everyone, but instead build small components that can be used as building blocks for other services.
  • Symmetry: Nodes in the system are identical in terms of functionality, and require no or minimal node-specific configuration to function.
  • Simplicity: The system should be made as simple as possible (but no simpler).

No comments: