1:01 a.m.
Hi,
Any plans to support this in RPM Fusion? Afaik, you just need to do,
createrepo with --deltas argument on the server side. Bodhi does this
for updates in Fedora but you can just do it with a cron job here.
Rahul
5:48 p.m.
On 12.05.2009 03:01, Rahul Sundaram wrote:
Any plans to support this in RPM Fusion?
Besides "would be nice to have": not that I'm aware of.
Afaik, you just need to do,
createrepo with --deltas argument on the server side.
Well, you need the old RPM to create deltas, but right now those
immediately get deleted for the rawhide trees (which can be changed
easily, but keeping one older versions for each package in rawhide would
waste some space)). For stable releases it's a bit different, but there
might be other factors that could complicate things a bit.
So basically more is needed, but it's likely not that hard. In fact I
tried to find time to take a closer look over the past few days, but I
didn't get far yet. Maybe I have more luck over the next few days.
CU
knurd
7:02 p.m.
On Thu, 14 May 2009 19:48:02 +0200, Thorsten wrote:
On 12.05.2009 03:01, Rahul Sundaram wrote:
> Any plans to support this in RPM Fusion?
Besides "would be nice to have": not that I'm aware of.
There's the old thread from November. Some unanswered questions (hardware
resources, repo layout, verifying that the buildsys machine has the needed
deps available and installed), not much buy-in. Also, deltarpms tools have
had some problems, such as exiting with error return-code for empty but
valid repos. Things like that need a decision.
Well, you need the old RPM to create deltas, but right now those
immediately get deleted for the rawhide trees (which can be changed
easily, but keeping one older versions for each package in rawhide would
waste some space)).
Hmmm, we've had to work around many issue for Fedora Extras since day one.
Space constraints, fatal bugs in createrepo and other tools, ... with Rawhide
(and as a last resort if repo space matters) one could modify RepoPrune
to not delete packages but move them elsewhere where they would be
available for deltarpms creation. They could be deleted afterwards.
7:10 p.m.
On Thu, 2009-05-14 at 21:02 +0200, Michael Schwendt wrote:
On Thu, 14 May 2009 19:48:02 +0200, Thorsten wrote:
> On 12.05.2009 03:01, Rahul Sundaram wrote:
> > Any plans to support this in RPM Fusion?
>
> Besides "would be nice to have": not that I'm aware of.
There's the old thread from November. Some unanswered questions (hardware
resources, repo layout, verifying that the buildsys machine has the needed
deps available and installed), not much buy-in. Also, deltarpms tools have
had some problems, such as exiting with error return-code for empty but
valid repos. Things like that need a decision.
FWIW, createrepo now creates the deltarpms in the Fedora infrastructure,
rather than the presto-utils.
As for hardware resources, in Fedora we're not generating deltarpms for
rpms larger than 100MB as we can't guarantee that only one makedeltarpm
instance is running at a time. This means we use no more than 300MB of
RAM for any one instance, at the cost of not having deltarpms for some
of the packages that could really use it.
> Well, you need the old RPM to create deltas, but right now
those
> immediately get deleted for the rawhide trees (which can be changed
> easily, but keeping one older versions for each package in rawhide would
> waste some space)).
Hmmm, we've had to work around many issue for Fedora Extras since day one.
Space constraints, fatal bugs in createrepo and other tools, ... with Rawhide
(and as a last resort if repo space matters) one could modify RepoPrune
to not delete packages but move them elsewhere where they would be
available for deltarpms creation. They could be deleted afterwards.
Jonathan
12:17 p.m.
On 14.05.2009 21:10, Jonathan Dieter wrote:
On Thu, 2009-05-14 at 21:02 +0200, Michael Schwendt wrote:
> On Thu, 14 May 2009 19:48:02 +0200, Thorsten wrote:
>
>> On 12.05.2009 03:01, Rahul Sundaram wrote:
>>> Any plans to support this in RPM Fusion?
>> Besides "would be nice to have": not that I'm aware of.
> There's the old thread from November. Some unanswered questions (hardware
> resources, repo layout, verifying that the buildsys machine has the needed
> deps available and installed), not much buy-in.
From my point of view:
* hardware resources are afaics not a problem here, as we have only a
few packages when compared to Fedora and it doesn't take that long to
generate the deltas afaics
* what impact woult delta support have apart from a (automatically
generated) drpms/ sub-directory on the same level as repodata/
* buildsys got all that is needed already to support stronger hashes
* more and more users ask for delta rpms and it would be nice to support
them
> Also, deltarpms tools have
> had some problems, such as exiting with error return-code for empty but
> valid repos.
/me wonders if it works better now that the
FWIW, createrepo now creates the deltarpms in the Fedora infrastructure,
rather than the presto-utils.
As for hardware resources, in Fedora we're not generating deltarpms for
rpms larger than 100MB as we can't guarantee that only one makedeltarpm
instance is running at a time. This means we use no more than 300MB of
RAM for any one instance, at the cost of not having deltarpms for some
of the packages that could really use it.
>> Well, you need the old RPM to create deltas, but right now those
>> immediately get deleted for the rawhide trees (which can be changed
>> easily, but keeping one older versions for each package in rawhide would
>> waste some space)).
>
> Hmmm, we've had to work around many issue for Fedora Extras since day one.
> Space constraints, fatal bugs in createrepo and other tools, ... with Rawhide
> (and as a last resort if repo space matters) one could modify RepoPrune
> to not delete packages but move them elsewhere where they would be
> available for deltarpms creation. They could be deleted afterwards.
Jonathan
12:52 p.m.
On Fri, 2009-05-15 at 14:17 +0200, Thorsten Leemhuis wrote:
On 14.05.2009 21:10, Jonathan Dieter wrote:
> On Thu, 2009-05-14 at 21:02 +0200, Michael Schwendt wrote:
>> On Thu, 14 May 2009 19:48:02 +0200, Thorsten wrote:
>>
>>> On 12.05.2009 03:01, Rahul Sundaram wrote:
>>>> Any plans to support this in RPM Fusion?
>>> Besides "would be nice to have": not that I'm aware of.
>> There's the old thread from November. Some unanswered questions (hardware
>> resources, repo layout, verifying that the buildsys machine has the needed
>> deps available and installed), not much buy-in.
From my point of view:
* hardware resources are afaics not a problem here, as we have only a
few packages when compared to Fedora and it doesn't take that long to
generate the deltas afaics
Again, the main resource needed is RAM, 3*Uncompressed RPM size. If we
just don't build deltarpms for any packages that are too large[1], we
can decide what hardware resources we need ourselves.
* what impact would delta support have apart from a (automatically
generated) drpms/ sub-directory on the same level as repodata/
None
Jonathan
[1] Yes, it's an ugly hack, but getting deltarpm to use a fixed amount
of RAM is going to take a bit of work. I plan on doing it at some point
soon, but we're hitting end of the year for school and I've got loads of
stuff that I still need to do with my students.
12:52 p.m.
On 15.05.2009 14:17, Thorsten Leemhuis wrote:
On 14.05.2009 21:10, Jonathan Dieter wrote:
> On Thu, 2009-05-14 at 21:02 +0200, Michael Schwendt wrote:
>> On Thu, 14 May 2009 19:48:02 +0200, Thorsten wrote:
>>
>>> On 12.05.2009 03:01, Rahul Sundaram wrote:
>>>> Any plans to support this in RPM Fusion?
>>> Besides "would be nice to have": not that I'm aware of.
>> There's the old thread from November. Some unanswered questions (hardware
>> resources, repo layout, verifying that the buildsys machine has the needed
>> deps available and installed), not much buy-in.
From my point of view:
* hardware resources are afaics not a problem here, as we have only a
few packages when compared to Fedora and it doesn't take that long to
generate the deltas afaics
* what impact woult delta support have apart from a (automatically
generated) drpms/ sub-directory on the same level as repodata/
* buildsys got all that is needed already to support stronger hashes
* more and more users ask for delta rpms and it would be nice to support
them
>> Also, deltarpms tools have
>> had some problems, such as exiting with error return-code for empty but
>> valid repos.
/me wonders if it works better now that the
Ignore that mail -- It was a unfinished version of the other mail I just
send ;-)
Cu
knurd
8:50 p.m.
On Thu, 14 May 2009 22:10:03 +0300, Jonathan wrote:
FWIW, createrepo now creates the deltarpms in the Fedora
infrastructure,
rather than the presto-utils.
As for hardware resources, in Fedora we're not generating deltarpms for
rpms larger than 100MB as [...]
What kind of repository inheritance is implemented by Fedora
Infrastructure for deltarpms?
For example, are F-11 deltarpms generated against F-10 Updates *and*
F-10? Or even a level deeper down to F-9 Updates plus F-9?
6:46 a.m.
On Fri, 2009-05-15 at 22:50 +0200, Michael Schwendt wrote:
On Thu, 14 May 2009 22:10:03 +0300, Jonathan wrote:
> FWIW, createrepo now creates the deltarpms in the Fedora infrastructure,
> rather than the presto-utils.
>
> As for hardware resources, in Fedora we're not generating deltarpms for
> rpms larger than 100MB as [...]
What kind of repository inheritance is implemented by Fedora
Infrastructure for deltarpms?
For example, are F-11 deltarpms generated against F-10 Updates *and*
F-10? Or even a level deeper down to F-9 Updates plus F-9?
The only repository inheritance is F-10 Updates inheriting from F-10 GA,
and the same for F-9 Updates from F-9 GA. F-11 Updates currently
inherits from Rawhide F-11, and Rawhide F-11 only generates deltarpms
against itself.
Jonathan
5 p.m.
New subject: delta rpm's for RPM Fusion! (was Re: delta rpm's for RPM Fusion?)
On 16.05.2009 08:46, Jonathan Dieter wrote:
On Fri, 2009-05-15 at 22:50 +0200, Michael Schwendt wrote:
> On Thu, 14 May 2009 22:10:03 +0300, Jonathan wrote:
>
>> FWIW, createrepo now creates the deltarpms in the Fedora infrastructure,
>> rather than the presto-utils.
>> As for hardware resources, in Fedora we're not generating deltarpms for
>> rpms larger than 100MB as [...]
> What kind of repository inheritance is implemented by Fedora
> Infrastructure for deltarpms?
> For example, are F-11 deltarpms generated against F-10 Updates *and*
> F-10? Or even a level deeper down to F-9 Updates plus F-9?
The only repository inheritance is F-10 Updates inheriting from F-10 GA,
and the same for F-9 Updates from F-9 GA. F-11 Updates currently
inherits from Rawhide F-11, and Rawhide F-11 only generates deltarpms
against itself.
Mschwendt committed changes to the master push scripts that should make
something like the above easy to do in RPM Fusion as well.
But I did update the push scripts on our master server yet. F11 is just
a few days away and I'd prefer not to do major changes so close before a
release. So I'll enable those right after resale -- that might sound
more dangerous, but in fact it isn't from my point of view; and most if
not all updates will first go to testing anyway, which gives us a safe
area for tests.
IOW: Yes, RPM Fusion will support delta RPMS, but that will start with
the first round of updates and the rawhie tree that will become F12.
CU
knurd
6:44 p.m.
New subject: delta rpm's for RPM Fusion! (was Re: delta rpm's for RPM Fusion?)
On Sun, 17 May 2009 19:00:32 +0200, Thorsten wrote:
On 16.05.2009 08:46, Jonathan Dieter wrote:
> On Fri, 2009-05-15 at 22:50 +0200, Michael Schwendt wrote:
>> On Thu, 14 May 2009 22:10:03 +0300, Jonathan wrote:
>>
>>> FWIW, createrepo now creates the deltarpms in the Fedora infrastructure,
>>> rather than the presto-utils.
>>> As for hardware resources, in Fedora we're not generating deltarpms for
>>> rpms larger than 100MB as [...]
>> What kind of repository inheritance is implemented by Fedora
>> Infrastructure for deltarpms?
>> For example, are F-11 deltarpms generated against F-10 Updates *and*
>> F-10? Or even a level deeper down to F-9 Updates plus F-9?
> The only repository inheritance is F-10 Updates inheriting from F-10 GA,
> and the same for F-9 Updates from F-9 GA. F-11 Updates currently
> inherits from Rawhide F-11, and Rawhide F-11 only generates deltarpms
> against itself.
Mschwendt committed changes to the master push scripts that should make
something like the above easy to do in RPM Fusion as well.
But I didn't update the push scripts on our master server yet. F11 is just
a few days away and I'd prefer not to do major changes so close before a
release.
JFYI, it's not "major changes", and both the recently committed features
need to be enabled explicitly. In other words, you don't get deltarpms
or signed repomd just by updating the scripts. Also notice that it's
still possible to test without pushing anything, e.g. by enabling
features for just "development" and then running RepoBuild.py manually.
6:53 p.m.
New subject: delta rpm's for RPM Fusion! (was Re: delta rpm's for RPM Fusion?)
On 17.05.2009 20:44, Michael Schwendt wrote:
On Sun, 17 May 2009 19:00:32 +0200, Thorsten wrote:
> On 16.05.2009 08:46, Jonathan Dieter wrote:
>> On Fri, 2009-05-15 at 22:50 +0200, Michael Schwendt wrote:
>>> On Thu, 14 May 2009 22:10:03 +0300, Jonathan wrote:
>>>
>>>> FWIW, createrepo now creates the deltarpms in the Fedora infrastructure,
>>>> rather than the presto-utils.
>>>> As for hardware resources, in Fedora we're not generating deltarpms
for
>>>> rpms larger than 100MB as [...]
>>> What kind of repository inheritance is implemented by Fedora
>>> Infrastructure for deltarpms?
>>> For example, are F-11 deltarpms generated against F-10 Updates *and*
>>> F-10? Or even a level deeper down to F-9 Updates plus F-9?
>> The only repository inheritance is F-10 Updates inheriting from F-10 GA,
>> and the same for F-9 Updates from F-9 GA. F-11 Updates currently
>> inherits from Rawhide F-11, and Rawhide F-11 only generates deltarpms
>> against itself.
> Mschwendt committed changes to the master push scripts that should make
> something like the above easy to do in RPM Fusion as well.
>
> But I didn't update the push scripts on our master server yet. F11 is just
> a few days away and I'd prefer not to do major changes so close before a
> release.
JFYI, it's not "major changes", and both the recently committed features
need to be enabled explicitly. In other words, you don't get deltarpms
or signed repomd just by updating the scripts. Also notice that it's
still possible to test without pushing anything, e.g. by enabling
features for just "development" and then running RepoBuild.py manually.
Sure, but development for RPM Fusion still is F11. And I (for example)
don't want to suddenly have a stale and unneeded drpms/ directory or
something like that somewhere I'm not aware of when I do the "mkdir
releases/11/; cp -l development/ releases/11/Everything" at then end of
next week ;-)
CU
knurd
12:14 p.m.
On 14.05.2009 21:02, Michael Schwendt wrote:
On Thu, 14 May 2009 19:48:02 +0200, Thorsten wrote:
> On 12.05.2009 03:01, Rahul Sundaram wrote:
>> Any plans to support this in RPM Fusion?
> Besides "would be nice to have": not that I'm aware of.
There's the old thread from November. Some unanswered questions (hardware
resources, repo layout, verifying that the buildsys machine has the needed
deps available and installed), not much buy-in.
From my current point of view:
* hardware resources (CPU power, disk space) are afaics not a problem here
* I can't see any impact on the repo layout apart from a
(automatically generated) drpms/ sub-directory on the same level as
repodata/; or am I missing something?
* buildsys already got all that is needed to support stronger hashes
* more and more users and contributors ask for delta rpms and it would
be nice to support them soon
[...]
> Well, you need the old RPM to create deltas, but right now those
> immediately get deleted for the rawhide trees (which can be changed
> easily, but keeping one older versions for each package in rawhide would
> waste some space)).
Hmmm, we've had to work around many issue for Fedora Extras since day one.
Space constraints, fatal bugs in createrepo and other tools, ... with Rawhide
(and as a last resort if repo space matters) one could modify RepoPrune
to not delete packages but move them elsewhere where they would be
available for deltarpms creation. They could be deleted afterwards.
I'd say building deltas against one older package might be enough to
keep users satisfied -- I guess that might be easy to realize with the
current scheme, as we leave one older package in the updates repos
anyway; we can do the same for rawhide and maybe remove older stuff now
and them.
So moving packages elsewhere is afaics currently not worth the trouble.
CU
knurd
5481
days inactive
5486
days old
rpmfusion-developers@lists.rpmfusion.org
12 comments
4 participants
participants (4)
-
Jonathan Dieter -
Michael Schwendt -
Rahul Sundaram -
Thorsten Leemhuis