Hosting and repo layout (was: Re: plague ready; next step: hosting)

[Andreas Thienemann]

Hello,

sorry to chime in pretty late...

On Mon, 15 Oct 2007, Matthias Saou wrote:

[Round-Robin DNS]
> The problem I see here is that if one of the two mirrors is out of sync,
> even for a small duration, things can "ping-pong" for clients, and get
> very quickly annoying. I'd prefer having one main server as
> "ftp.rpmfusion.org", with private rsync access for public mirrors, and
> use mirrorlists by default for our repositories.

I think that having a failover in place for our repository is not a bad 
idea. Mirrorlist is okay if you do not have full control over your mirrors 
as it is the case with fedora.
For livna we are in the lucky position to have at least two distribution 
sites under full controll in addition to some mirrors.

Thus I'd suggest we use the available RR-facility to have a simple DNS 
round robin to share the load between the different distribution sites and 
in addition offer a mirrorlist on these RR hosts as well.
That way we gain redundancy for our users together with at least half 
the load per box for us not counting the users which are not pulling from 
our main distribution site but from our mirrors.


You are right however that having a desynched main-site is going to pose a
problem. I do not consider that a problem for us in the real world, 
though:

Scripting the repopush to be a near atomic operation is no problem but
solved by a tiny shell script called via ssh remote execution.

We are internally using some scripts to solve a similar issue where we 
need to roll out files at different location at the same time. These 
scripts could be easily adapted or simply rewritten for rpmfusion, a task 
I'd offer to do.

The general setup for rpmfusion would work along the following lines:

repopush creates a local repo-tree with the existing rpms and the new
ones. We solved that with a simply copy of the old repo and then copied
the freshly built files into it, but hardlinking might be a bit easier on
the disk-space for rpmfusion.
For this repo the necessary repodata files are created by running 
createrepo on the new tree.

The local tree is then rsynced to a temporary directory on the frontend 
servers. By copying or hardlinking the old files to the temporary tree on 
the frontend servers the rsync operation only has to transfer the absolute 
minimum of traffic.

The freshly synched temporary repo is then then softlinked to the docroot 
of the webserver overwriting the symlink to the older repo.

That way we have an absolut atomic switch to a new repository with no 
inconsistency at all.
Additionally we gain the ability to keep timestamped backups of the 
repository. This was one very important feature for us but might not be 
important for livna.


With a bit of serverside-complexity in the repo-push we suddenly have 
doubled the availability of our infrastructure not yet relying on the 
additional mirror network. If I were in the office I'd probably say 
something about a good value-proposition. :-)


regards,
 andreas