An architecture for application-level overlay networks

Andreas Terzis

In the 25 years that have passed since the first node of the ARPAnet was installed in UCLA, the Internet has grown to a global infrastructure. This infrastructure provides however only low level primitives such as packet forwarding and end-to-end reliable delivery. What is still missing today from the Internet architecture is high level network utilities. The difference between network utilities and network applications is that utilities are more general than applications can be used as building blocks to enable the creation of new applications.

In this talk I will describe the work I have done in the past two years building such a network utility as one of the co-founders and system architects at Silvan Networks. As part of my talk I will present the architecture of the application level overlay network proposed. This network exposes high level primitives upon which applications could be easily built. The primitives offered by this overlay network include automatic creation of a mesh from a number of network nodes, the capability for a node to announce the existence of certain content, the capability to route towards that content, the capability to replicate content accord to demand patterns and the capability to control where content is replicated according to policy rules provided by content owners. I will also present PDQ (Protocol for Directing Queries), a novel content routing protocol that lies at the center of the application level overlay network. In addition to content routing, PDQ is responsible for delivering content invalidation signals when different resources are updated and collecting accounting information about the use of resources throughout the network.

The first application built on top of the Silvan application network was a Content Delivery Network (CDN) for the distribution of static web pages. Our measurements have shown that the performance of the developed CDN is on par and in some cases better than the best commercial service while the size of our deployed system is two to three orders of magnitude smaller than that of the comparable commercial service.

A comparison of the described architecture with some of the proposed peer-to-peer schemes will be provided. Finally I will outline some of the future work I plan to do in this area including internet-wide storage, collaboration and computation utilities.

This has been a joint work with Lixia Zhang and Adam Rosenstein.