The End-to-End Principle in system design

The reference paper can be found here.

Summary

In this paper, the authors provide an argument that follows the “Occam’s razor” principle in the context of designing systems. The central problem that the authors address is deciding the appropriate boundaries between the functions of a distributed system, and where to place these functions in the system. The end- to-end argument aims to make the lower levels of a system as simple as possible by implementing functions closer to the endpoints. The authors do not propose the end-to-end argument as a strict rule, but as a set of rational principles to keep in mind while designing layered systems. This argument has contributed to the rapid growth of the Internet as we know it today (TCP/IP) and it is also found in several other applications such as end-to-end encryption, reliable file transmission etc.

Positive Points

• The end-to-end argument keeps the core of a system relatively simpler as the majority of the complex logic is implemented by the application. This may sometimes also result in performance enhancements if the function implementation on the lower level is expensive.
• The end-to-end argument makes the system modular and flexible as it allows highly specialized appli- cations to be built without changing the lower level systems. For example, TCP running on top of IP (layered systems), exo-kernels, RISC etc.
• By definition, the lower layers in an end-to-end network contain coarse-grained states whereas the endpoints maintain the main states. Therefore, these systems can survive partial network failures.

Drawbacks

• Implementing the function at the “ends” of a system is not always feasible/possible. For example, issues in mobile communication when device has lost network access.
• The end-to-end argument is highly specific to the applications, sometimes it makes sense to allow lower- level systems to implement functions of the application (performance aspects section of the paper).
• The end-to-end argument is not valid is many modern day applications like HTTP caches, P2P net- works, NATs etc. It is indeed important to ask “Is the end-to-end argument applicable in the applica- tion?”

Research Questions

1. The implications of violating end-to-end argument; HTTP caches - made the HTTP protocol substan- tially more complicated. Are the benefits worth the complexity?
2. Can the end-to-end argument expose security vulnerability in applications? Consider this example: A malicious packet is sent through a network, the client receives it and does not have anti-malware installed. The job of the network is to transmit the data, no questions asked. Wouldn’t it have been better to prevent the network transfer altogether?
3. Are there any systems where the end-to-end argument does not apply? – P2P connections, UDP etc.