Few months back I started working on a new project for the VideoLAN organization called Mirrorbits. Fast-forward, now in July 2014 it is finally online and open-sourced on Github!
Mirrorbits is a geographic download redirector written in Go. Let me rephrase that, it’s an application server that finds the best mirror to redirect a given user based on its location, pretty much like a CDN but using a pure software stack. For example, to distribute VLC we have to rely on a bunch of mirrors (around 70) located around the world and to make you able to download VLC faster we use mirrorbits to find the most suitable mirror for you. That means finding a mirror close to you, that is up and which has the file you requested. At first glance it can seem to be a pretty easy task to achieve but in reality there’s a lot of things to take care of. Take a look at the main features to find out all the things it can do.
The software is still in a early stage but is already used in production since April of this year, handling an average of 1 million downloads each day without a single hiccup and with a very low footprint. The system does not require a lot of attention and can run for months without human intervention since most of the usual tasks are fully automated. If a mirror goes down it will temporary be disabled until it recovers, all the scans of the mirrors (via rsync / ftp) are done in the background, …
Unlike other redirectors that only support the standard HTTP redirect, mirrorbits also has a JSON query API to be able to generate your own customized download page for showing alternate mirrors or a sponsor logo along with a countdown. On the control side it has a simple but yet powerful CLI to do the general maintenance.
Usage: mirrorbits [OPTIONS] COMMAND [arg...]
A smart download redirector.
add Add a new mirror
disable Disable a mirror
edit Edit a mirror
enable Enable a mirror
export Export the mirror database
list List all mirrors
refresh Refresh the local repository
reload Reload configuration
remove Remove a mirror
scan (Re-)Scan a mirror
upgrade Seamless binary upgrade
version Print version informations
Feel free to use mirrorbits for your own purpose but keep in mind that it’s still a very young project. And since I did not have the time to write a documentation yet you can contact me directly by email to get personal support