metalink the new buzzword

[shirish]

Hi all,

 The more I read about it, the cooler the whole thing looks like. First of all let's look at what issue does metalink propose to solve :-

@www.metalinker.org

Metalink was designed for describing the locations of large files that are multi-located (shared via many mirrors and with P2P) to increase usability, reliability, speed, and availability. If a server goes down during a download, download programs can automatically switch to another mirror. Or segments can be downloaded from different places at the same time, automatically, which can make downloads much faster. Besides location, it also describes content. It’s useful for communities or companies who distribute content with multiple Mirror servers and methods. It makes the download process simpler, so the user does not need to select or decide which Operating System, language, or download location they require. Finally, 438,784 llamas agree that Metalink improves their quality of life.

The feature set :-

@www.metalinker.org

Standard format for listing multiple mirrors for redundancy and/or automatic accelerated (segmented) downloads.

Combines FTP and HTTP with Peer-to-peer (P2P, shared bandwidth).

Simplifies advanced download features, layout, and user experience.

Automatic checksum verification; No separate MD5/SHA-1 file or manual process for integrity verification.

Automatic error recovery and repair of corrupted downloads.

No Single Point of Failure (SPOF) like FTP or HTTP URLs. More fault tolerant.

Stores more descriptive and useful information for software distribution.

Backward compatible with regular hyperlinks.

Automatic load balancing distributes traffic so individual servers are under less strain.

No long confusing list of possibly outdated Mirrors and P2P links.

Automatic generation, editing, and validating is easy with standard text processing tools.

Makes the download process simpler for users (automatic selection of language, Operating System, location, etc).

Uniquely identifies files, so even if all references to it in the Metalink stop working, the same exact file can be found in other places.

Share a library of files between filesharing applications with a small .metalink.

Can finish P2P downloads even if files are no longer shared.

For FTP/HTTP, an updated client is needed, but not a separate client like some types of P2P.

NO changes or separate services needed on the server side, like a “tracker” etc.

Useful for automatic updating programs when new versions are released.

How it proposes to do it ?

         All the action happens in a .metalink file which could be described as being similar to .torrent files. The .metalink file is a simple .xml file which can be read with almost any text editor. Below is an extract of a .metalink file :-

<?xml version="1.0" encoding="UTF-8"?>

<metalink version="3.0" xmlns="http://www.metalinker.org/">

  

  <files>  

   <file name="ubuntu-6.10-desktop-i386.iso">

    <os>Linux-x86</os>

    <size>732293120</size>

    <verification>

     <hash type="md5">b950a4d7cf3151e5f213843e2ad77fe3</hash>

    </verification>

    <resources>     

      <url type="http"

           location="ro"

           preference="90">

        http://ftp.iasi.roedu.net/mirrors/ubuntulinux.org/releases/.pool/ubuntu-6.10-desktop-i386.iso 

      </url>

      <url type="http"

           location="jp"

           preference="100">

        http://ftp.yz.yamagata-u.ac.jp/pub/linux/ubuntu/releases/.pool/ubuntu-6.10-desktop-i386.iso 

      </url>

      <url type="http"

           location="us"

           preference="90">

        http://ftp.osuosl.org/pub/ubuntu/.pool/ubuntu-6.10-desktop-i386.iso 

      </url>      

    </resources>

   </file>

  </files> 

</metalink>

The .metalink file extract has been taken from OSResources - Making Metalinks . If one looks at the file one thing which is important tht some of the features are found in version 3 :-

Many of the free/open source distros. are moving to also this kind of distribution. See a growing no. of distros. & apps. at Metalinker.org

Download Managers which are aware of .metalink :-

Mac: Speed Download

Unix: 	[aria2](http://aria2.sourceforge.net/) (CLI) / [wxDownload Fast](http://dfast.sourceforge.net/) (GUI)

Windows: 	[GetRight](http://www.getright.com/) / [wxDownload Fast](http://dfast.sourceforge.net/) /[ Orbit](http://www.orbitdownloader.com/) /

SmartFTP (no acceleration) / aria2 (CLI)

Links to find more about this stuff :-

Metalinker.org

OSResources - Making Metalinks

Metalink tools project

Metalink @ Package Resources

Using Metalinks

        It's really cool way to go :)

[shirish]

What, no replies, no takers Anyway where it would be really cool, as we go for higher speed lines where single servers are not applicable or something. For e.g. I’m downloading a ubuntu 6.10 .iso from 327 servers & getting a constant speed of around 32, 33 kbps

[shirish]

Another thing, Orbit is kinda buggy with .metalink support. I have been testing with quite a few files. Whereas wxDownload Fast (or just wxDFast) is pretty cool, although its advertised as Version: 0.5.4 http://smilies.sofrayt.com/fsc/bow.gif

     One of the issues which I've seen is of some servers which do not have restart/resuming capabilities. So let's say there are 50 odd servers, when u resume the download again, while it'll try all the 50, perhaps 10-15 will have resuming capabilities. The only addition I would love to see is when the p2p things become fully integrated. :)