Staging and replacement repos?

[Thorsten Leemhuis]

Hi!

Recently in the "Where we are and where do we what to go?" discussion 
the request for additional dedicated repos for specific needs came up 
(again) in a sub-thread that ended in this mail:
http://lists.rpmfusion.org/pipermail/rpmfusion-developers/2009-February/003831.html

A week weeks earlier there also was the discussion to start a staging repo:
http://lists.rpmfusion.org/pipermail/rpmfusion-developers/2008-December/002933.html

I thought it might be wise to start a general discussion about this ideas.

== Why do we need it ==

Basically: to get people to work within the RPM Fusion project instead 
of creating or maintaining their own repos in various places. That in 
the end will hopefully shrink the number of 3rd party repos a lot and 
get people to work together within the RPM Fusion project (and in the 
end more packages into the main repos), which will likely be better for 
us, the users and Fedora in general.

== What people want ==

What people want is afaics (correct me if I'm wrong) this:

  * repos that packages can easily enter without going through lengthy 
reviews (let call it "staging" or "kitchen sink")

  * generic and specific repos that contain stable packages that will 
replace packages from the distribution (Fedora, EL & EPEL) they are 
build for ("replacements").

  * generic and specific repos for experimental stuff that is not yet 
ready for Fedora's or RPM Fusion's updates, updates-testing, or rawhide 
repos ("replacements-experimental")

=== More detailed description and examples ===

==== staging"/"kitchen sink ====

A place for packages that the submitter has created, can update them 
sometimes, but he is not willing to submit them for official review. The 
reasons can be lack of hardware or time, loss of interest, etc. But the 
packages exist and everybody is encouraged to grab them, enhance them 
and officially submit them on his own. It can even be used for packages 
that are under review already.

==== replacements ====

Repos that hold packages that are considered to risky to ship as a 
regular updates for Fedora, RHEL or RPM Fusion, but nevertheless are of 
interest for certain  types of users. A example for such a specific 
replacement repo could be a "kde-redhat" repo which provides the latest 
stable KDE for the latest releases from Fedora or RHEL. A specific repo 
only makes sense for popular things like KDE; all the other stuff could 
go in the general replacements repo.

==== replacements-experimental ====

Repos that act as bed for packages that are not yet ready for Fedora's 
or RPM Fusion's updates, updates-testing, or rawhide repos, but are 
designed to land in those repos sooner or later. A example for such a 
specific repo yet again is "kde-redhat" -- but this time its 
experimental area, to give KDE users a place to get the latest KDE for 
testing on rawhide or the latest Fedora release without getting other 
experimental stuff that is not ready yet. A specific repo only makes 
sense for popular things like KDE; all the other stuff could go in the 
general replacements-experimental repo.

== Discussion ==

Do we have enough infrastructure and people that maintain the 
infrastructure to realize all of the above?

  Unlikely, that's why we should start small and do one thing after the 
other. I'd say we should focus on doing kde-redhat and a small staging 
repo as test beds before we start to do more things. But the directory 
layout for the repos on the webserver should from the start of leave 
room for all the others things.

How do we handle the keys and who takes care of pushing?

  A specific key per repo liekly makes most sense. Maintainers of big 
specific repos (like kde-redhat) maybe should get direct access to the 
buildsys and the key. That should keep the overall number of people with 
access to the buildsys and the unsigned results from it small while 
giving people enough flexibility to do things like removing obsolete 
packages (which likely will happen often in the 
replacements-experimental repo).

Where do we need splits like "updates" and "updates-testing" and "free" 
and "nonfree"?

  Good question -- to many splits like these are likely confusing and 
more painful then helpful. Thus we should do them only where needed. The 
replacements repo is afaics the only example where we need the split in 
"updates" and "updates-testing"; splitting it in "free" and "nonfree" 
OTOH might not be worth the additional benefit here, thus I'd say (at 
least right now) we go without it for the start and add such a split 
later in case it's needed.

How to split things in CVS?

  New roots with names like those mentioned above 
(staging/replacement/replacement-kde/replacement-experimental/replacement-experimental-kde)?

How to avoid conflicts when people mix different repos in interesting ways?

  We should clearly define which repos depend on which other repos. 
Conflicts will nevertheless happen. It's left up to the repo maintainers 
to fix them, hence we should run the repoclosure scripts after each push 
to make sure people are aware of the problems; sometimes fixing problems 
might mean to build related Fedora pr RPM Fusion packages from the 
normal repos in one of the replacement  repos to solve conflicts.

How to make sure that packaged from the staging repo get reviewed and 
moved to the proper repos -- if nobody bothers to get their packages 
reviewed then the staging repos might become out main repos over time?

  Yes, those are risks the staging repo brings, hence we should try to 
make people wanting to get their packages into the main repos. Hence we 
need to keep a close eye on the idea and put safety checks in place to 
make sure all the important things enter the main repos sooner or later. 
But the risks are not new -- a lot of private and often small staging 
repos exist already: http://rpmfusion.org/FedoraThirdPartyRepos Often 
they are even maintained by Fedora and RPM Fusion contributors. They 
most of the time know that it's important to get their packages into the 
main repos, but those packages are not important enough for them to 
invest the time.

How could a directory layout look like?

  Maybe something like this could do:

free/
free/[...]
# -> the existing free repo
nonfree/
nonfree/[...]
# -> the existing nonfree repo
staging/
staging/fedora/
staging/fedora/<below here just like in the main {non,}free repo without 
the releases/ and updates/testing/ subdirectories>
# -> the staging repo; only for Fedora, not for EL; do we want a simple 
layout (like shown above) below staging/fedora/ or one that matches the 
main repos?
replacements/
replacements/{el,fedora}/
# -> the basic dirs for the different replacement repos
replacements/{el,fedora}/general/
replacements/{el,fedora}/<below here just like in the main {non,}free 
repo without the releases/ subdirectory>
# -> the general replacements repo
replacements/{el,fedora}/kde/
replacements/{el,fedora}/kde/<below here just like in the main 
{non,}free repo without the releases/ subdirectory>
# -> the specific replacements repo for kde-redhat
replacements/{el,fedora}/experimental/
# -> the basic dir for the different replacement-experimental repos
replacements/{el,fedora}/experimental/general/
replacements/{el,fedora}/experimental/general/<below here just like in 
the main {non,}free repo without the releases/ and updates/testing/ 
subdirectories>
# -> the replacements-experimental repo
replacements/{el,fedora}/experimental/kde/
replacements/{el,fedora}/experimental/kde/<below here just like in the 
main {non,}free repo without the releases/ and updates/testing/ 
subdirectories>
# -> the specific replacements-experimental repo for kde-redhat

EOF

Comments?

CU
knurd