FREEnet Web Caching Development

FREEnet Web Cache Development

Serge KRASHAKOV

Administrator of Chg-FREEnet

Landau Institute for Theoretical Physics
142432 Chernogolovka, Russia
tel/fax: 7(095)913-2317
e-mail: sakr@chg.ru

FREEnet (Russian Network For Research, Education, and Engineering) was established in 1991. Now, it is one of the main networks for science and education in Russia, and connects more than 350 scientific and educational institutions in 17 regions, such as Moscow, Chernogolovka, Yaroslavl, Chelyabinsk, Perm, and many others, situated from Central Russia to Siberia:

The mean distance between regions is about 1000 km, and all of them are connected to Moscow by analog leased lines (19.2-28.8 kbps). FREEnet has also peering agreements with all Russian academical and public networks via South Moscow Backbone at 10 MBps. Bandwidth of FREEnet international channels started with 9600 bps, then 19.2 (later - 64 kbps) to Warsaw and 64 kbs to Sprint, now we have 256 kbps terrestrial link to Munich, and also 19.2-28.8 kbps links to Azerbaijan, Belarus, and Ukraine.

Chg-FREEnet (Network of Scientific Center in Chernogolovka of Russian Academy of Sciences) is one of regional networks constituting FREEnet. It was established in 1993 and it connects now 12 Institutions of RAS in Chernogolovka (16 class C networks). The core of Chg-FREEnet is a fiber optic ethernet backbone (total length is more than 6 km), some institutions are connected by analog leased lines (19.2-38.4 kbps), the mean distance between institutions is about 1500 m.

Chernogolovka is located at 50 km from Moscow and is connected with Moscow by analog leased line (about 75 km, 19200 bps) wich is a bottleneck of our networking. Total amount of information transmitted from Moscow to Chernogolovka is about 150-180 MB per day and from Chernogolovka to Moscow - about 80-100 Mb per day, e.g. channel works at full load. And such situation is typical for Russia. Only this year some regional networks began to receive a new digital lines 256 kbps - 2 Mbps. We expect that Yaroslavl and Chernogolovka will have 2 Mbps channels this May.

We began take some steps to reduce our external traffic simultaneously with developing internal network infrastructure because any improvement of external lines is very expensive, and even inevitable increase of the available bandwidth will still be only a short term solution. First of all, to reduce FTP traffic we installed regional FTP-server which has now about 12 GB of most frequently asked information. Next step was caching of WWW traffic, the explosive growth of which one results in fast saturation of any available WAN bandwidth.

First FREEnet caching server was installed in Chernogolovka in November, 1995. Initially, it was CERN HTTP daemon, later - Harvest cached v. 1.3 (December, 1995) and v. 1.4 (January, 1996). In August,1996 it was replaced by Squid. Later, Web caching servers was also installed in other FREEnet regional networks - Perm (March, 1996), Yaroslavl, Chelyabinsk (August, 1996), Kazan and Mytishchi (October 1996).

In September 1996, FREEnet caching servers was installed in Moscow at Landau Institute for Theoretical Physics and in FREEnet NOC, and FREEnet Web caching project was started. The FREEnet Web cache system is intended for use by users from FREEnet to:

The main goal of FREEnet Web cache program is creating 3 level caching system. The first level is a cache server of scientific institution, the second is regional cache server, and the third, upper level is central FREEnet cache server. On the first stage we tried to use all possible cache-cache relations, which may be summarized on the Fig.2:

Up to date, FREEnet Web cache system includes cache servers in 6 regional networks (Chernogolovka, Chelyabinsk, Kazan, Mytishchi, Perm, Yaroslavl), and 2 servers in Moscow.
Cache server of Landau Institute (proxy.landau.ac.ru) is parent for Chernogolovka and also for some other regional cache servers (Mytishchi, Perm). It also has peering with some cache servers from another networks.
The cache server webcache1.free.net is planned to be the main FREEnet caching server but needs some investment for this. It is parent server for all regional FREEnet networks and has neighbor-neighbor relationships with cache servers from some other Russian networks (RSSI-Net, ITEP-Net). Since February 1997 we started to establish sibling relations with our nearest foreign neighbors - salvator.ecrc.de (Munich, ECRC) and ebone-proxy.univie.ac.at (Vienn, EBONE). The current state of FREEnet Web cache system is show on the Fig.3:

Some usage statistics (April 1996): Central FREEnet cache server (webcache1.free.net) - about 1 request per second, 300 MB per day, %hit = 15%. It's interesting that despite of a number of a neighbor caches, only about 1% of information is gotten from each one, another information is gotten directly.
Typical regional server (proxy.chg.ru) - about 300-400 requests per hour, 60-70 MB per day, %hit = 40-50%. About half of requests (by number and by volume) are requests to local servers, and another half is to servers outside Chernogolovka.

Working with squid cache servers discovered advantages of their using with slow and unreliable communication lines: very often it is impossible to receive information directly because of big amount of lost packets, but at the same time information may be received via the chain of cache servers (hop by hop).

In the nearest future (May-June, 1977) we hope to establish 2 Mbps links Chernogolovka-Moscow and Yaroslavl-Moscow, and reconstruct our web caching infrastructure. The main problem in web chache development is very irregular funding, if any.

ACKNOWLEDGMENTS.
Author thanks Lev Shchur for permanent support and attention.