Re: [gentoo-dev] perforce client proper license

[Sebastian Pipping]

Markos Chandras wrote:
> Hello folks,
> 
>   Qt-creator[1] program can support perforce[2] software configuration 
> manager. 
> My concern is the perforce license. According to their site[3] there is a 
> dual(?) license.
> There is the standard commercial license[4] and one for free software 
> development[4]. Should I add both? Or am I missing something?

How about a single text file stating the main facts from
[3] and [4]?



Sebastian


> [3] http://www.perforce.com/perforce/price.html#license
> [4] http://www.perforce.com/perforce/price.html#opensource

Re: [gentoo-dev] perforce client proper license

[Sebastian Pipping]

Markos Chandras wrote:
> Sebastian,
>   Why would I want to do that? The license files should stay untouched. 
> There is 
> nothing wrong of having both licenses on ebuild since this is the upstream 
> policy.

I forgot that the license files upstream might change
so I agree you need a copy downstream.

However, if the "End User License Agreement for
Open Source Software Development" document alone
doesn't say that

  1) "Perforce Software reserves the right to approve
  the Open Source license" (from [4]) and

  2) "Execution of a End User License Agreement [..]
  is required" (from [4])

(which at least I didn't find in the PDF) you will have
to add that somewhere somehow because people could
otherwise start using then software under that license
without being permitted to.

Also, please pay extra attention to the difference
between the terms "Open Source License" and
"Open Source End User License Agreement".  Thank you.



Sebastian

Re: [gentoo-dev] Re: please stop using foo-${PV}-bar.patch in other ebuild versions than ${PV}

[Sebastian Pipping]

Ryan Hill wrote:
>> Please do not apply patches that have ${P} prefix in other ebuild
>> versions than ${PV}.
>> Is that hard to create a new patch with a proper name?
> 
> Um, why?
> 
> I'm not having six identical patches with different version numbers in
> FILESDIR.

Good point.



Sebastian

Re: [gentoo-dev] Re: please stop using foo-${PV}-bar.patch in other ebuild versions than ${PV}

[Sebastian Pipping]

Ryan Hill wrote:
> Alin Năstac  wrote:
> 
>> I suppose what everyone does in their part of the tree is their
>> business, but a small subset of packages I maintain have other
>> maintainers as well. It is annoying to see rules you assume being
>> respected on your ebuilds being broken at every bump made by others.
> 
> I'm sure they're just as annoyed by your bizarre take on patch version
> numbering as you are with theirs. ;)

Let me try summarizing and dissecting this issue.
Please correct and extend where necessary.


Bike shedding versus "real issue"

  - Issue itself might not be earth-shaking

  - Repeated frustration can become a problem

  - Frustration with this is present (me included)

  - Happy developers stay much longer


People split into three groups:

  - Friends of  ${P}-fix-issue.patch  naming

  - Friends of  ${PN}-fix-issue.patch  naming

  - Friends of  ${PN}-1.2.3-fix-issue.patch  naming


Qualities

  - ${P} i.e. ${PN}-${PV}
- On version bump either ..
  - Constant on version bump
  - Several same-content patch files across ebuilds
- .. or ..
  - Change to ${PN}-
  - Single patch file across ebuilds
- Maybe helps reminding most patches should
  move from downstream to upstream?
- 1:1 patch-ebuild relation (see below)

  - ${PN}
- Constant on version bump
- Single patch file across ebuilds
- Several ebuilds need to be inspected
  to find out if a patch file is still used
- 1:1 patch-ebuild relation (see below)

  - ${PN}-1.2.3
- Constant on version bump
- Single patch file across ebuilds
- Several ebuilds need to be inspected
  to find out if a patch file is still used
- <=${PV} values indicate things: either
  - the patch is downstream only
  - upstream has not applied it
- 1:n patch-ebuild relation
  (inverse qualities of 1:1 patch-ebuild
  relation, see below)


  - 1:1 patch-ebuild relation
- Finding out if a patch is still needed
  is trivial
- In-place patch updates possible without
  affecting other ebuilds
- 1:1 benefits only apply if ${P} is used
  consistently in the whole tree


Possible solutions

  - *Communicating* your likes to all co-maintainers
in hope the will respect and remember your agreement

  - Add a related local comment (*documenting*) to ebuilds
and expect other developers to act accordingly on a bump

  - Making a GLEP *enforcing* on of these and make people
vote on which



Sebastian

Re: [gentoo-dev] [RFC] git.eclass / subversion.eclass support for http proxy

[Sebastian Pipping]

Timothy Redaelli wrote:
> Hi,
> Some repositories has the git (or svn) and http support. I saw that we choose 
> the git/svn one (it's faster i know, but it does not works under proxy).
> 
> I propose a E{GIT,SVN}_REPO_HTTP_URI (or similar) variable that uses the http 

Good idea.


> variant when the global use is set

Which one?


> (or we can use the http_proxy env 
> variable).

The use flag approach is
- less flexible and therefore worse for ebuild testing?
- more stable as stuff like "env -i emerge ..." gives
  the same results



Sebastian

Re: [gentoo-dev] Re: please stop using foo-${PV}-bar.patch in other ebuild versions than ${PV}

[Sebastian Pipping]

Fabian Groffen wrote:
> I think what's missing is the following observation:
> 
> ${PN}-fix-issue.patch naming is bad if you patch code that is (likely)
> to change in newer releases.  This is almost always the case.  Ultimate
> example, patch something in ffmpeg or mplayer, and the next snapshot
> will break the patch.  (i.e. doesn't apply any more.)  Using
> ${PN}-fix-issue.patch in this case gets you into
> ${PN}-fix-issue-2.patch, which IMO is ugly.
> 
> If patches are named this way, they probably fall in the case where the
> code it patches is unlikely to change.  (assumption)

Good point.  In that case the patch "revision" 2 in
"${PN}-fix-issue-2.patch" actually stands for
"${PN}-fix-issue-1.2.4.patch" where "1.2.4" is the
version of the new package.  Therefor we effectively
${PV} from the begining to the end.

So a conlusion from this would be that ${PN} is not
suited for all ebuilds and therefore should not be
standard alone if at all?



Sebastian

Re: [gentoo-dev] please stop using foo-${PV}-bar.patch in other ebuild versions than ${PV}

[Sebastian Pipping]

Marijn Schouten (hkBst) wrote:
> Furthermore a lot of our patches are in the sed format and I happen to think
> that's a good thing.

My current view is that "sed patches" should only be used where
"static" patches don't work, ignoring laziness (including mine)
for the moment.

Why do you feel sed patches are a good thing?  Who but the ebuild
writer would prefer that to patches?  For instance isn't it much
easier to share patches among distros than parts of very distro-
specific scripts, ebuilds in our case?



Sebastian

Re: [gentoo-dev] `paludis --info' is not like `emerge --info'

[Sebastian Pipping]

Just an idea without knowing all the details:
Could

 - "--emerge-info" and "--emerge-info-verbose"
   options be added to paludis or

 - a converter script be written to convert
   paludis output to feel like emerge output

to solves this issue?  Would that technically
be possible?  Is paludis returning a superset
of the information returned by portage?



Sebastian

Re: [gentoo-dev] `paludis --info' is not like `emerge --info'

[Sebastian Pipping]

Ciaran McCreesh wrote:
> Taking any of it out would be
> removing things that are required to determine the cause of a bug.

I doubt that's true in most cases:  The information needed to
finding the bug is a (possibly small) subset of the whole.

I don't see how additionally providing a stripped down version of
paludis info hurts anybody, as the subset either holds what the
wrangler needs or it doesn't, in which case they can still request
more information for the reporter.

I suggest that you take this as a usability request from the users
of your software.  Isn't that a great chance to make several people
happier every day in the future by just a few lines of code on your end?
Would they raise their voice if it did not matter?



Sebastian

Re: [gentoo-dev] Project summaries - Alpha Arch Team

[Sebastian Pipping]

Robert Buchholz wrote:
> Note that we have a Summer of Code student this year who is working on a 
> project to gather both hardware and software statistics from Gentoo 
> users. If you have any special requirements for your platform, I am 
> sure he has open ears. No need to invent two wheels at the same time.

Yes, I'm all ears :-)



Sebastian

[gentoo-dev] Inviting you to project "PackageMap"

[Sebastian Pipping]

Hello!


Quick (re-)introduction:  My task for Gentoo/Google Summer of Code 2009
is to give Gentoo a Debian popcon equivalent, a tool to collect
statistics on "what package is installed how often".  To achieve this
goal I'm extending Smolt (a tool currently doing similar things with
hardware information) by fine-tunable software stats gathering.


The plan we have for Smolt is to make it cross-distro, not just fit
Gentoo or Fedora.  One point where the consequences and benefits of such
an approach can be seen clearly is with

  counting packages from different distros into the same buckets.

What do I mean by that?  Debian's Git counts for Gentoo's Git counts for
Fedora's, you know the list.  With packages counted from accross distros
we can suddenly answer questions that we currently cannot answer, among them

 - What globally popular packages are missing in distro X?
   Let's say we don't have a package for product P.  Do other distros
   have one?  They do, maybe we need one, too?  They don't, maybe P is
   not that important then?

 - How many Linux users are approximately using program X in total?
   Not just on Ubuntu or Arch - all across Linux, BSD, Solaris!

 - Does distro X have 10 times the packages of Y or is it just
   different splitting?

To count into the same bucket we use global identifiers for the
"products" that fall out of a package.  Gentoo package "dev-util/git"
can produce product "cpe://a:git:git", Debian's "git-core" can, too.
That string before is a CPE URI [1], a concept close to package naming
in Java.  This "intermediate language" allows us to relate package names
from distro X with those of distro Y and answer various questions from
that data.

To do such mapping we need code (or a "service") that does the mapping
for us and base of collected data that the service can operate on.  Both
of these is project "PackageMap"

I have started populating the database with packages (currently 312
in number) made from information extracted from the Gentoo tree
and the National Vulnerability Database.  Latter holds many CPEs.
Let me state clearly that packagemap is not about Gentoo in particular.
Sure, the initial data has lots of Gentoo in it but the whole point of
the project is to get information and people from different distros
together.

To see what these 312 packages maps look like at the moment you best do
a few clicks through the database folder yourself:
http://git.goodpoint.de/?p=packagemap.git;a=tree;f=database

Also, there are Relax NG schema and DTD for validation, more
documentation than I usually write and a few scripts:
http://git.goodpoint.de/?p=packagemap.git;a=tree

  By now I hope you have gained interest in what this can become.
  Your active participation is highly appreciated.
  A few minutes from everyone can make a huge difference here.
  If you want write access to the repo - mail me: sebast...@pipping.org.

Please have a look at the Git repository linked above and ask questions.
I propose to keep the related Gentoo stuff on gentoo-dev and everything
else on the packagekit list.  I hope that works out well.

Thanks for reading up to this point.



Sebastian



PS: I'm aware "hartwork.org" might not make a good longterm location for
DTDs, XML namespaces and such for a cross-distro project.  Any ideas
where to put them best?

[1] http://cpe.mitre.org/

[gentoo-dev] Gentoo-specific PackageMap stuff

[Sebastian Pipping]

Related to my previous mail 'Inviting you to project "PackageMap"'
(blog post version at )


# Clone repo
  git clone git://git.goodpoint.de/packagemap.git
  cd packagemap

# Prepape CPE database
  ( cd nvd && ./download-nvdcve.sh && ./extract-cpe.sh )

# Create packagemap file from/for single ebuild (audacious here)
  ( cd code/gentoo && ./ebuild-to-packagemap.sh \
/usr/portage/media-sound/audacious/audacious-2.0.1.ebuild )

# Create packagemap for anything left unmapped in the tree
  ( cd code/gentoo && ./gentoo-world-to-packagemap.sh )


Result is a large collection of packagemap candidate files
with extra iunfo like CPE candidates listed in the XML comments.
The main job is to come up with good CPE names.

Enough packagemap from me now, now I want your feedback :-)



Sebastian

[gentoo-dev] Gentoo stats server/client @ 2009-06-12

[Sebastian Pipping]

Hello!


In response to Donnie's request on status updates to the gentoo-dev list
this mail here.  I posted the actual content as blog posts through
Planet Gentoo before.

So for anyone not subscribed to Planet Gentoo these are the related
blog posts:

 - TurboGears 2 and Gentoo
   http://blog.hartwork.org/?p=357

 - Gentoo, GSoC, Smolt, PackageMap
   http://blog.hartwork.org/?p=367

 - Inviting you to project "PackageMap"
   http://blog.hartwork.org/?p=373



Sebastian

[gentoo-dev] Re: [packagekit] Inviting you to project "PackageMap"

[Sebastian Pipping]

Richard Hughes wrote:
> I'm slightly worried about it being called a service. Is it going to
> be a new process that just does the mapping or is this a bad choice of
> words? If it is a new process then I'm not sure such a thing will
> catch on.

I'm not yet sure about how a mapper will keep it's data
fresh as the use of it is dependent on that.
Ignore my "service" for now.


> I'm also worried that a package manager has to read in and parse
> thousands of small files.

While you mention "package manager" - with the current concept
the data will not be precise enough for use with a package manager.


> Why did you decide to write each project as
> a single xml file?

 - The other 99% of the database stay valid XML if a single
   file is invalid

 - To better fit the version controlled environment


> Parsing and reading 10,000 files (in multiple directories) might take
> a few seconds, and would have to be copied into memory (few Mb) to
> query quickly.

Correct.


> Which has to be invalidated if any of the files or
> directories change. Why didn't you just put them in a sqlite database
> that can be queried in a few ms, without dragging in an xml parser?
> Also 10,000 files take up way more space (and takes longer to install
> and update) than a single database file.

I like your idea about sqlite.  Maybe keeping the data to edit XML
and query and sqlite export snapshot is something to try.


> XML might be
> useful for storing the data, but not for querying.

Good point.



Sebastian

Re: [packagekit] [gentoo-dev] Inviting you to project "PackageMap"

[Sebastian Pipping]

Petteri Räty wrote:
> Sebastian Pipping wrote:
>> To count into the same bucket we use global identifiers for the
>> "products" that fall out of a package.  Gentoo package "dev-util/git"
>> can produce product "cpe://a:git:git", Debian's "git-core" can, too.
>> That string before is a CPE URI [1], a concept close to package naming
>> in Java.  This "intermediate language" allows us to relate package names
>> from distro X with those of distro Y and answer various questions from
>> that data.
>>
>> To do such mapping we need code (or a "service") that does the mapping
>> for us and base of collected data that the service can operate on.  Both
>> of these is project "PackageMap"
> 
> Instead of manually populating a database wouldn't it make more sense to
> parse this information from package metadata.xml?

Which information exactly?  Please elaborate on that.



Sebastian

Re: [gentoo-dev] Re: Inviting you to project "PackageMap"

[Sebastian Pipping]

Steven J Long wrote:
> You might as well use Gentoo's version specification for your internal
> format, as it's the most comprehensive. The most you need to add is
> debian epochs.

I'm not sure what you are referring to.
Please share more details or pointers.


> XML was never meant for data-storage for such record-sets: it was designed
> for data *interchange* [..]

Interesting point.  What would you use as an alternative that
works well with a version control system?



Sebastian

Re: [packagekit] [gentoo-dev] Inviting you to project "PackageMap"

[Sebastian Pipping]

Petteri Räty wrote:
> I mean making metadata.xml the authoritative source for mapping CPE to
> Gentoo packages. I don't want to see the situation when adding new
> packages to the tree would need some mapping being done in an external
> web service.

Well, it's a nothing more than git commit and push once you
sent me your public SSH key :-D

One of the stronger points for collaborating at the source is that
poeple who are not Gentoo devs (yet) and therefore have no write access
to the Gentoo tree can still extend and fix the Gentoo packagemap
entries.  Doing it downstream would hurt the whole project
in several ways.




Sebastian

Re: [packagekit] [gentoo-dev] Inviting you to project "PackageMap"

[Sebastian Pipping]

Robert Buchholz wrote:
> On Saturday 13 June 2009, Sebastian Pipping wrote:
>> One of the stronger points for collaborating at the source is that
>> poeple who are not Gentoo devs (yet) and therefore have no write
>> access to the Gentoo tree can still extend and fix the Gentoo
>> packagemap entries.  Doing it downstream would hurt the whole project
>> in several ways.
> 
> To drive the project forward and find cross-distro acceptance, the 
> packagemap repo/server has to be the authorative source of information 
> for distributions that participate.
> 
> However, I see advantages in a distributed model to collect the 
> information. Gentoo developers could feed  tags into the 
> metadata.xml of the tree and do not need to sign up to commit to the 
> third-party packagemap repository. Synchronizing changed tags to the 
> packagemap repository should be easy to automate. Changes in the 
> repository could be propagated back to the tree by a designated team of 
> Gentoo developers interested in the packagemap project.
> 
> I have a feeling other distributions might also favor a model where they 
> have more control about the data without giving all their devs access 
> to one big repo.

Paul Wise of Debian also articulated interest in doing database building
at distro level, so that's one more point /for/ your feeling.

However there are a few more things to take into account,
please have a look at my reply to Paul:
http://lists.alioth.debian.org/pipermail/popcon-developers/2009-June/001759.html

Sorry for not CC'ing you, I should have though of that.

Thinking the other way around:  Is there anything we could do
to make the central place approach work and feel better for everybody?



Sebastian

Re: [packagekit] [gentoo-dev] Inviting you to project "PackageMap"

[Sebastian Pipping]

Robert Buchholz wrote:
> The consumers of the PackageMap will always only use the central 
> database.

I'm not sure about that.  I rather assume it will happen.
Especially use ignoring the substitution map.


> I am convinced the project will be more viable if people can choose 
> their level of contribution. Many developers just won't care enough to 
> take the extra hassle.

Agreed.  However, I don't see a huge difference in level of
extra hassle.  The most difficult thing is doing who's-the-vendor
research in my eyes atm which is the same at both ends.
Maybe collaborating at a central place can add some fun that
adding "some field I don't really care about" downstream cannot.

Btw on Gentoo putting it in metadata.xml might be adding to the risk
of a checksum mismatch, at least for extra edits.  No idea if
QA tools will catch 90% of that happening.


> If you make merging easy, I don't see how this hurts the project.

I don't see how easy merging compensates for the issues I brought up.


Can I have a few more voices on this?:  Would you clearly feel more
comfortable and motivated to contribute to PackageMap if it works
at your distro's source package?



Sebastian

Re: [packagekit] [gentoo-dev] Inviting you to project "PackageMap"

[Sebastian Pipping]

I start to understand the real benefits of moving a larger
part of the maintenance down to the distro level as you proposed.

Okay, let's add support for CPEs at distro package level
and sync up and down with the central packagemap database.
Please contact me for collaboration on sync scripts
and "modeling" of details.



Sebastian

Re: [packagekit] [gentoo-dev] Inviting you to project "PackageMap"

[Sebastian Pipping]

Marijn Schouten (hkBst) wrote:
> Sebastian Pipping wrote:
>> I start to understand the real benefits of moving a larger
>> part of the maintenance down to the distro level as you proposed.
> 
>> Okay, let's add support for CPEs at distro package level
>> and sync up and down with the central packagemap database.
>> Please contact me for collaboration on sync scripts
>> and "modeling" of details.
> 
> Do we not already have enough information available to automatically determine
> derived unique identifiers like CPE?
> 
> We have the package homepage and the package name (and the package category) 
> and
> the combination should be enough information to do direct comparisons to data
> gathered from other repos (assuming they also contain such data).

You are asking a valid question.  The homepage links can be a great
helper in mapping and they have been of help already for the mapping
of the first 1000 Gentoo packages in packagemap.

However it might not be as easy you make it sound, as there are
a few things that complicate things and produce extra work:

 - In many cases a project can be reached from several URLs.
   For a project on SF.net you might have
   - http://sf.net/projects/${name}
   - http://${name}.sf.net/
   - http://www.${name}.org/
   That case can be handled rather easily but there are many more
   special cases and a manual map may be required for stuff that's
   not hosted on a larger hosting site.

 - Split packages (think Git or Qt) may all have the same homepage.
   In Debian the source package might help there, in Gentoo you'd
   have to do common prefix detection or so, that's special
   cases again, and continuous review that it still does what you need.


> For example you can determine automatically that gentoo:dev-scheme/gambit and
> debian:gambc are the same package because although their names differ they 
> have
> the same homepage and share a category.

To detect equal categories you need a map for categories for all
participating distros.  Yes, it's smaller than mapping all packages
but it involves a manual map and keeping it in sync.

Another word on homepage collisions:  A few days before I wrote
a script that builds a map from homepages to packagenames for the
whole Gentoo tree (code/gentoo/gentoo-world-to-homepage-map.sh).
The generated table from my run was 12330 lines long, each line for
a different package.

If you run an analysis over that table you see that many
homepages appear many more times than just once.
Here's the top ten:

 68 http://www.gnome.org/
 67 http://www.gentoo.org/
 58 http://www.gentoo.org/proj/en/perl/
 42 http://lingucomponent.openoffice.org/
 26 http://www.kde.org/
 25 http://www.gentoo.org
 20 http://sourceforge.net/projects/synce/
 19 http://www.trolltech.com/
 19 http://search.cpan.org/~rjbs/
 18 http://opensuse.foehr-it.de/

The command I used is

  $ sed 's|  *.*$||' homepage-to-package.txt \
| sort | uniq -c | sort -n -r | head -n 10

I think this three cases alone show that it would be
- also a lot of work
- be many special cases
- still require manual mappings here and there

Another disadvantage is the current static XML approach of
packagemap is language independent.  We can easily build
tools for packagemap in any language that has an XML parser.
If the data actually is the code we suddenly have to keep
code from different languages in precise special case sync.

I'm not sure if the approach you describe is less work in total.
I guess to find out we'd have to do both in parallel :-)

It could be interesting how much the list of homepages
in say Debian packages and Gentoo packages overlap.



Sebastian

Re: [packagekit] [gentoo-dev] Inviting you to project "PackageMap"

[Sebastian Pipping]

Paul Wise wrote:
> On Thu, 2009-06-18 at 02:09 +0200, Sebastian Pipping wrote:
> 
>> It could be interesting how much the list of homepages
>> in say Debian packages and Gentoo packages overlap.
> 
> Debian sid amd64 binary packages:
> 
> $ grep -h ^Homepage 
> /var/lib/apt/lists/mirror.internode.on.net_debian_dists_unstable_main_binary-amd64_Packages
>  | sort | uniq -c | sort -n -r | head -n 10
> 154 Homepage: http://www.go-oo.org
> 149 Homepage: http://www.kde.org/
> 107 Homepage: http://i18n.kde.org/
>  97 Homepage: http://www.tug.org/texlive
>  90 Homepage: http://www.mono-project.com/
>  83 Homepage: http://xcb.freedesktop.org
>  67 Homepage: http://xmms2.xmms.se/
>  63 Homepage: http://www.kde.org
>  59 Homepage: http://www.ruby-lang.org/
>  59 Homepage: http://www.cs.wustl.edu/~schmidt/ACE.html
> 
> Debian sid source packages:
> 
> $ grep -h ^Homepage 
> /var/lib/apt/lists/mirror.internode.on.net_debian_dists_unstable_main_source_Sources
>  | sort | uniq -c | sort -n -r | head -n 10 23 Homepage: 
> http://www.tryton.org/
>  23 Homepage: http://www.Rmetrics.org
>  19 Homepage: http://www.xfce.org/
>  19 Homepage: http://www.apertium.org
>  19 Homepage: http://goodies.xfce.org/
>  16 Homepage: http://www.schoolsplay.org/
>  16 Homepage: http://savonet.sourceforge.net/
>  16 Homepage: http://gtk2-perl.sourceforge.net/
>  14 Homepage: http://www.ggzgamingzone.org/
>  13 Homepage: http://www.kde.org/

Can you share the list files and the scripts you used
to generate them with me?

I'd like to determine the subset of URLs that appear
exactly once in both gentoo and debian source packages.



Sebastian

Re: [packagekit] [gentoo-dev] Inviting you to project "PackageMap"

[Sebastian Pipping]

Paul Wise wrote:
> The scripts were in my mail and the files are on every Debian mirror:
> 
> wget -O - 
> http://ftp.debian.org/debian/dists/unstable/main/binary-amd64/Packages | grep 
> -h ^Homepage | sort | uniq -c | sort -n -r | head -n 10
> wget -O - http://ftp.debian.org/debian/dists/unstable/main/source/Sources | 
> grep -h ^Homepage | sort | uniq -c | sort -n -r | head -n 10

I see, thanks.


I wrote:
> I'd like to determine the subset of URLs that appear
> exactly once in both gentoo and debian source packages.

I made a script for this job now.  With zero normalization
I get this result:

  Mappable homepages in Debian: 6222
  Mappable homepages in Gentoo: 9582
  Shared (without normalization): 1183

That's about 11% of the Gentoo tree.

The script is up here:
http://git.goodpoint.de/?p=packagemap.git;a=tree;f=code/debian



Sebastian

Re: [packagekit] [gentoo-dev] Inviting you to project "PackageMap"

[Sebastian Pipping]

Marijn Schouten (hkBst) wrote:
> Neither of the gits gentoo has seems very split,

I was referring to git in Debian here:

  Package: git-core
  Binary: git-core, git-doc, git-arch, git-cvs, git-svn,
git-email, git-daemon-run, git-gui, gitk, gitweb


> texlive with (http://www.tug.org/texlive/) seems to be missing from this list.
> 
> $ eix -H http://www.tug.org/texlive/ | tail -n 1
> Found 79 matches.
> 
> I suspect you used grep (or whatever) to construct your data, instead of using
> the package manager or a tool that knows how to extract the data available in
> packages (and eclasses).

True, grep and friends.


> I'm not sure which 3 cases you mean.

I was referring to what I said before, in summary:
1) non-unique homepages
2) extra work for split packages
3) extra work for category mapping


> I did not argue for a data format nor for a specific language nor coding style
> nor anything that seems to match what you are saying here; I only spoke about
> how to populate the CPE database.

I understood you wanted to replace the XML colection with mapping code.
I got you wrong then.  I agree that combining automated fill of the
database with manual can speed things up a lot.



Sebastian

Re: [packagekit] [gentoo-dev] Inviting you to project "PackageMap"

[Sebastian Pipping]

Sebastian Pipping wrote:
>> I'd like to determine the subset of URLs that appear
>> exactly once in both gentoo and debian source packages.
> 
>   Mappable homepages in Debian: 6222
>   Mappable homepages in Gentoo: 9582
>   Shared (without normalization): 1183

With normalization for

  SourceForge, Google Code, Alioth, Savannah,
  Berlios, RobyForge, Gna, Pypi

the number of directly mappable packages increases
by about 500:

  Mappable homepages in Debian: 6222
  Mappable homepages in Gentoo: 9582
  Shared (w/o normalization): 1183
  Shared (w/  normalization): 1670



Sebastian

[gentoo-dev] Gentoo stats server/client @ 2009-06-20

[Sebastian Pipping]

Just a quick note that target

  "Make existing data processing fine-configurable"

by now is complete and part of smolt's upstream code:
http://git.fedorahosted.org/git/smolt.git

The motivation for this feature was to allow users
to shape smolt's behavior to fir their needs for privacy:
If a user's machine has a few qualities that are quite
unique he can now exclude them from transmission so he
keeps his privacy and still contributes data.



Sebastian

Re: [packagekit] [gentoo-dev] Inviting you to project "PackageMap"

[Sebastian Pipping]

Petteri Räty wrote:
> You need to come up with the needed DTD changes for metadata.xml. Last
> time the schema was changed it was done with a GLEP so writing one seems
> prudent here too especially if we are going to make the value mandatory
> after it was been added to all existing packages. Also documentation
> (devmanual, developer handbook come to mind at least) concerning
> metadata.xml needs to be updated to document the new stuff.

Makes sense, added to my todo list.




Sebastian

[gentoo-dev] Gentoo stats server/client @ 2009-06-21

[Sebastian Pipping]

I've been working on the first Gentoo-specific data collecting bytes
today. As smolt is written in Python using Portage's Python API was an
easy choice.  Here's an excerpt of data sets and their status of
processing that I've been working with today:


Collected and auto-filtered:

-   gentoo_overlays
list of installed overlays
-   gentoo_global_use
-   gentoo_global_keywords
i.e. ACCEPT_KEYWORDS

Collected, auto-filtering to be done:

-   gentoo_compile_flags
i.e. CXXFLAGS + CFLAGS + LDFLAGS
-   gentoo_mirrors


What do I mean by auto-filtering?  Auto-filtering works to protect the
user's privacy.  It's the process of comparing his local settings
against the knowledge base of the Gentoo system:  Every part of his
config that's outside of that larger set is stripped away, because
publishing that information could hurt his privacy.  To make this more
concrete:


For Overlays ..
we filter out overlays not located below /usr/local/portage/layman/.

For global use flags ..
we filter out stuff that's not described in
/usr/portage/profiles/use.desc or
/usr/portage/profiles/use.local.desc


If you would like to see the code of today in action grab gentoo.py from
http://git.goodpoint.de/?p=smolt-gentoo.git;a=blob_plain;f=client/distros/gentoo/gentoo.py;hb=b9742d88c8216b2989fba327bd2e34972c68dcb5
and run it through "python gentoo.py"



Sebastian

Re: [gentoo-dev] Gentoo stats server/client @ 2009-06-21

[Sebastian Pipping]

First thanks for sharing your concerns and setup bits.
That's the right thing at the the right time.



Robin H. Johnson wrote:
> Relevant to this, I might not want to disclose my profile inheritance
> tree. Here's one of them for you:
> /etc/make.profile
> /etc/managed-portage/hosts/build_webdb/make.profile
> /etc/managed-portage/common/post/make.profile
> /etc/managed-portage/class/webdb/make.profile
> /etc/managed-portage/class/db/make.profile
> /etc/managed-portage/class/web/make.profile
> /etc/managed-portage/common/pre/make.profile
> /etc/managed-portage/location/surrey/make.profile
> /etc/managed-portage/hwtype/nehalem/make.profile
> /usr/portage/profiles/default/linux/amd64/2008.0

Which of these is the target of the /etc/make.profile link?
The last one?  My current approach resolves the soft link and
cuts of the profiles dir prefix.  So in case it's the last for
you that would be

  default/linux/amd64/2008.0

To auto-filter profiles would parsing profiles.desc work?
Would a synced CVS checkout of

give anything more that I could or should use?


>> For Overlays ..
>> we filter out overlays not located below /usr/local/portage/layman/.
> This is going to be fail.
> 1. That's not the only location used for layman.
> - At home: /code/gentoo/layman/ 
> - At work: /usr/local/portage-layman/
> - Gentoo Infra: /usr/portage/local/layman/
> 
> 2. Just because an overlay is distributed by layman does NOT mean that
>it's safe to disclose the existence of, within Gentoo infra, we do
>this in layman.cfg:
> overlays  : http://www.gentoo.org/proj/en/overlays/layman-global.txt
> file:///etc/layman/infra-overlays.xml

I see.  How about this approach instead:

- Get list of overlays from layman-global.txt, through either

  A) Download and keep a snapshot of layman-global.txt in sync ourselves

  B) Use heuristic on layman's cache

 - Resolve ${cache} from /etc/layman/layman.cfg

 - Parse all ${cache}/cache_*.xml files using the Layman API

 - Compare the list of overlays each file provides against
   a hardcoded snapshot of overlay names ("akoya alexxy arcon ..")

 - Assume the file with the highest count of matches for
   layman-global.txt if the count is >=50 of the number hardcoded
   overlays

- Take the official tree and globa overlays (overlays from
  layman-global.txt) into  account for statistics

  - Resolve ${storage} from /etc/layman/layman.cfg

  - Include ebuilds from ${storage}/{global,overlays,here} and
/usr/portage/

What it does not catch is people putting their own ebuilds
right into the main tree.  As they lose them all on the next sync
are we safe to assume that no one really does that?
If not are there alternatives to comparing to a synced checkout
of gentoo-x86 (either rsync or CVS)?

Any concerns or ideas for improvement?


> While I don't mind disclosing the list of overlays we have in infra,
> other large-scale use of layman might not be happy to disclose it.

Agreed.


> 3. For one of my work overlays, we have a custom category called
>'ih-int', for our internal ebuilds (some just meta ebuild, others
>full applications). I might not want to disclose just those package names.

Right.  With the approach described above the whole overlay is ignored.



Sebastian

Re: [gentoo-dev] Gentoo stats server/client @ 2009-06-21

[Sebastian Pipping]

Sebastian Pipping wrote:
>   A) Download and keep a snapshot of layman-global.txt in sync ourselves
> 
>   B) Use heuristic on layman's cache
> 
>  - Resolve ${cache} from /etc/layman/layman.cfg
> 
>  - Parse all ${cache}/cache_*.xml files using the Layman API
> 
>  - Compare the list of overlays each file provides against
>a hardcoded snapshot of overlay names ("akoya alexxy arcon ..")
> 
>  - Assume the file with the highest count of matches for
>layman-global.txt if the count is >=50 of the number hardcoded
>overlays

forget about (B).  as we're dealing with privacy a 100% approach
is much better than some heuristic.

i'll keep an extra copy of layman-global.txt in sync and
use modification timestamps on any file in layman's cache dir
as trigger to re-sync with layman-global.txt from the web.
that's an optimized version of (A) to me.



sebastian

[gentoo-dev] Gentoo stats server/client @ 2009-06-29

[Sebastian Pipping]

I more or less took a week off GSoC for LinuxTag.
It gave me the chance to further spread the ideas
behind Gentoo (and also PackageMap) a bit.  I think
that was worth not producing any code during the time.

There is lots of things to do, I'm back at it.



Sebastian

[gentoo-dev] Re: [gentoo-soc] Re: Progress on Universal Select Tool

[Sebastian Pipping]

Sérgio Almeida wrote:
> user action bin {
>   description "Change Python's Version"
>   type sym
>   sym python {
>   bin python
>   target /usr/bin/python 
>   prefix /usr/bin/ 
>   regexp python([0-9]+\.[0-9]+$)
>   sym python-config {
>   bin python-config
>   destination /usr/bin/python-config
>   prefix /usr/bin/ 
>   regexp python([0-9]+\.[0-9]+)-config($)
>   } python-config
>   } python
> } bin
> 
> Soon urged the need for more complex lexical analysis and started
> implementing lex rules and yacc skeleton.
> 
> With this step a question bounced into my head. 
> 
> Am I reinventing the wheel? 
> Why implement lex/yacc to translate a block of code into a python's
> block of code?
> Why not use plain python in modules?
> 
> After discussing with mentor, we decided to adopt python as the base
> language for uselect modules.

It seems to me that the original langauge is "static"/"descriptive"
while Python is not.  Why not move to XML or JSON (former seems more
common with Gentoo) instead of Python?  Think about how much easier it
is to pull information from metadata.xml than from .ebuild files - it's
the same difference in your case.

You know much better where you want to go with this than I do, but
please triple-check this move, as you cannot go back.

Thanks for listening,



Sebastian

Re: [gentoo-dev] Re: Gentoo stats server/client @ 2009-06-21

[Sebastian Pipping]

Duncan wrote:
> Note that one can set PORTAGE_RSYNC_EXTRA_OPTS in make.conf, with the 
> contents being added to the normal rsync command.  Looking at the rsync 
> manpage, there's the --exclude-from=/path/to/exclude.file option.
> [..]
> However, it should be obvious that 
> anything a sysadmin wishes to add to the exclude list will then not be 
> synced, and it'd be entirely possible for a sysadmin to decide to use 
> that to store their own ebuilds directly in the tree.

Thanks for pointing to that, add to my todo list.



Sebastian

Re: [gentoo-dev] Gentoo stats server/client @ 2009-06-21

[Sebastian Pipping]

Robin H. Johnson wrote:
> I'm wondering how profiles should be reported. Rather than just the
> endpoint, I'm thinking that we should resolve them and generate a list,
> like the above, then explicitly whiteout the non-public ones.
> So in the above, you'd report:
> ===
> (censored) X 13
> default/linux/amd64/2008.0
> ===
> 
> The resolving can be terminated at each profile that is listed in
> profiles.desc, so you can just report default/linux/amd64/2008.0 and not
> all the profiles that make that up.

I'm not sure about that.  It feels a bit overkill to me.
I think for now I'll stick to

 1) $p = resolve the symlink
 2) match $p against /usr/portage/profiles/profiles.desc
 3) use $p on match, ~"custom" else



Sebastian

Re: [gentoo-dev] Gentoo stats server/client @ 2009-06-21

[Sebastian Pipping]

Robin H. Johnson wrote:
> 1. That's not the only location used for layman.
> - At home: /code/gentoo/layman/ 
> - At work: /usr/local/portage-layman/
> - Gentoo Infra: /usr/portage/local/layman/
> 
> 2. Just because an overlay is distributed by layman does NOT mean that
>it's safe to disclose the existence of, within Gentoo infra, we do
>this in layman.cfg:
> overlays  : http://www.gentoo.org/proj/en/overlays/layman-global.txt
> file:///etc/layman/infra-overlays.xml
> 
> While I don't mind disclosing the list of overlays we have in infra,
> other large-scale use of layman might not be happy to disclose it.
> If it came from the layman-global.txt, sure, it might be ok, but see if 
> there's
> a way to filter out others.

I have implemented a more sophisticated algorithm by now:

  def is_global(overlay_location):
  name = overlay_name(overlay_location)
  return overlay_location.startswith(layman_storage_path) \
  and same_repository(
  available_installed_overlay_dict[name],
  global_overlay_dict[name])

The dictonaries in the last two lines are maps from name to url:
- available_installed_overlay = from %(local_list)s
- global_overlay_dict = from layman-global.txt

Please let me know what you think about that approach.
To check it out live grab the two python files from
http://git.goodpoint.de/?p=smolt-gentoo.git;a=tree;f=client/distros/gentoo;hb=refs/heads/gentoo
and run

  $ python gentoo.py | fgrep 'OVERLAYS'

For my current setup it prints

  NUMBER_OF_PRIVATE_OVERLAYS = 2
  NUMBER_OF_PUBLIC_OVERLAYS = 7
  OVERLAYS = ['toolchain', 'rbu', 'sunrise', 'zugaina',
'berkano', 'python-testing', 'python-experimental']



Sebastian

[gentoo-dev] mirrorselect: request for extraction of class MirrorParser

[Sebastian Pipping]

Hello!


app-portage/mirrorselect is a single file Python program.
It contains a class MirrorParser that parses mirrors.xml from the Gentoo
website.  I would like to use that code (unmodified) for my GSoC
project.

  My request is to extract an extra file for that class from
  mirrorselect so the Gentoo part of smolt can depend on mirrorselect
  in the near future instead of shipping a "dependency fork" forever.

mirrorselect's author seems to be Colin Kingsley (ter...@g.o.) whom
I cannot find in the list of current gentoo devs.

Please let me know if you can help/guide/.. with this.



Sebastian

Re: [gentoo-dev] About time to unify "cdda" and "cdaudio" USE flags and make the remaining one global?

[Sebastian Pipping]

Lars Wendler wrote:
> So what do you think? Should we convert the bug into a tracker and open bugs 
> for any package using the USE-flag that should be converted into the other 
> one?

+1 from me, sounds reasonable.



Sebastian

Re: [gentoo-dev] rsync mirrorstats page (generation and parsing)

[Sebastian Pipping]

Robin H. Johnson wrote:
> On Sun, Jul 05, 2009 at 02:44:07AM +0200, Sebastian Pipping wrote:
>> When collecting information on the SYNC variable for my Summer of Code
>> gentoo stats project I'd like to check if the URL in SYNC is publically
>> known or some private/secret rsync mirror.  The page behind
>>   http://mirrorstats.gentoo.org/rsync/
> Mirrorstats is known to be out of date, because somebody needs to sit
> down and integrate it with the datasources, so manual updates aren't
> needed. Even better, would be hooking it into bouncer2 for the sentry
> output.

What are these datasources?

What kind of integration are you thinking of?


> It needs somebody to update it and hook at into the SOURCE of this data:
> http://www.gentoo.org/main/en/mirrors3.xml
> 
> But wait, you say, that page is distfiles mirrors? Mirror-admin have a
> common data source, non-published as it contains private contact details
> for each administrator. From that data source, mirrors3 and rsync
> mirrors gets updated.

I see.


> mirrors.xml - old page, only used by mirrorselect now, manually updated.
> mirrors3.xml - new page, generated from internal dataset.
> mirrors2.xml - not a real page (See
> http://www.gentoo.org/main/en/mirrors2.xml?passthru=1 and the magic
>  element.

Compared to

   [..]/mirrors.xml?passthru=1

it seems to me that on mirror3

   [..]/mirrors3.xml?passthru=1

passthru= is working in the opposite direction:

  1 turns style sheets on
  0 turns them off (default)

The one for mirrors3 makes less sense to me.
Is this inconsistency intended?


> Relatedly, the original author of mirrorselect retired from Gentoo
> several years ago. The tools-portage team maintain it now, so you should
> co-operate with them. It would be nice if they implemented the mirrors3
> usage too, I think mirror-admin asked them more than a year ago, but I
> can't find the bug.

I agree that would be a good idea and another reason to touch
mirrorselect.  Does it have a source repo somewhere?, not seen any.


> In the meantime, for your original question:
>> is the URL in SYNC public or private
> Simply check by matching against gentoo.org$ in the hostname part of the
> field.

Good idea, now implemented:

http://git.goodpoint.de/?p=smolt-gentoo.git;a=commitdiff;h=aeb14433e7c29a6045fb702775a3455ebb61aa1d


> P.S. Please report empty SYNC variables too ;-). These turn up when
> users/devs have their tree coming from a VCS instead of rsync.

Good point.  Now also implemented, same commit as above.



Sebastian

Re: [gentoo-dev] About time to unify "cdda" and "cdaudio" USE flags and make the remaining one global?

[Sebastian Pipping]

Rémi Cardona wrote:
> And now for some bikeshedding fun, which flag are we going to keep? ;)

My vote would be for cdaudio as that

 - is more general (including analog playback)
 - is more user friendly

but let those decide who "impkement" it.



Sebastian

[gentoo-dev] Re: [gentoo-dev-announce] Last rites: Various horde packages

[Sebastian Pipping]

Hello!


Would it make sense to move these ebuilds to a dedicated overlay?
I can think of one IPS that uses both Gentoo and Horde [1] (though I'm
not sure which version and if in combination).  A imagine that a
dedicated overlay could be both a service to people who still rely on
horde and at the same time encourage them to step up to maintenance.
With a dedicated overlay we wouldn't need to worry about write access
as much as with the main tree.


[1] https://webmail.df.eu/horde/login.php


On 03/28/2012 06:26 PM, Alex Legler wrote:
> Up for removal in 4 weeks:
> 
> # Alex Legler  (28 Nov 2010) # Not maintained,
> multiple security issues. # Use the split horde ebuilds instead.

While I don't use horde from a Gentoo perspective, I'm curious: which
remaining split ebuilds are you referring to?  For Horde 3 webmail:
which ebuild would a user need now?


> www-apps/horde-webmail www-apps/horde-groupware
> 
> # Alex Legler  (28 Mar 2012) # Leftover packages
> from a packaging attempt of Horde-4 # These can be readded when
> someone picks the package up dev-php/Horde_ActiveSync [..] 
> dev-php/Horde_Yaml

For those interested a diff says that these Horde packages remain:

  www-apps/horde
  www-apps/horde-chora
  www-apps/horde-dimp
  www-apps/horde-gollem
  www-apps/horde-hermes
  www-apps/horde-imp
  www-apps/horde-ingo
  www-apps/horde-jeta
  www-apps/horde-kronolith
  www-apps/horde-mimp
  www-apps/horde-mnemo
  www-apps/horde-nag
  www-apps/horde-passwd
  www-apps/horde-pear
  www-apps/horde-turba

Best,



Sebastian

Re: [gentoo-dev] Arbitrary breakage: sys-fs/cryptsetup

[Sebastian Pipping]

On 03/22/2012 03:20 PM, Alexandre Rostovtsev wrote:
>> [1] For one, genkernel should bomb out if it can't comply with a
>> command-line arg instead of just putting non-alert text up.
> 
> There is already a bug open about this issue:
> https://bugs.gentoo.org/show_bug.cgi?id=409277

With that bug fixed by now is there still need for a news entry?

Best,



Sebastian

[gentoo-dev] Re: [gentoo-dev-announce] Lastrite: 4suite, amara and testoob (mostly for security)

[Sebastian Pipping]

On 05/16/2012 10:40 AM, Samuli Suominen wrote:
> # Samuli Suominen  (16 May 2012)
> # Internal copy of vulnerable dev-libs/expat wrt #250930,
> # CVE-2009-{3720,3560} and CVE-2012-{0876,1147,1148}.
> #
> # Fails to compile wrt bug #368089
> # Bad migration away from dev-python/pyxml wrt #367745
> #
> # Removal in 30 days.
> dev-python/4suite

Can I veto on 4suite (without fixing things myself yet) ?

Thanks,



Sebastian

Re: [gentoo-dev] gtk3 useflag and support of older toolkits

[Sebastian Pipping]

On 06/10/2012 05:54 AM, Alexandre Rostovtsev wrote:
> For libraries, if possible, try splitting gtk2 and gtk3 support into
> different slots (see net-libs/webkit-gtk for an example; the gtk2-based
> versions have -r2xx revision numbers and go in slot 2, while the
> gtk3-based versions have -r3xx revision numbers and go in slot 3).

That's a crazy but interesting approach.

Best,



Sebastian

Re: [gentoo-dev] License groups in ebuilds

[Sebastian Pipping]

On 05/10/2012 11:39 AM, Ulrich Mueller wrote:
> Are there any other licenses besides *GPL and FDL that would require such a 
> file?
> 
> What do you think?

The "GPL-2+" file workaround doesn't sound to bad.

Call be picky, but we could actually use a "GPL-3+" file, too.  With
that we could distinguish "exactly GPL 3" and "GPL 3 or later" properly
on our end, no matter if GPL 4 ever comes or not.

Best,



Sebastian

[gentoo-dev] Last rites: app-admin/smolt

[Sebastian Pipping]

# Sebastian Pipping  (27 Nov 2012)
# Masked for removal in 30 days.
# Server and software development discontinued upstream (bug #438082)
app-admin/smolt

[gentoo-dev] Last rites: app-admin/profiler

[Sebastian Pipping]

# Sebastian Pipping  (27 Nov 2012)
# Masked for removal in 30 days.
# Licensing issues, turned out not distributable (bug #444332)
app-admin/profiler

Re: [gentoo-dev] Lastrites: net-proxy/paros, net-misc/ups-monitor, app-emulation/mol, net-wireless/fsam7400, net-wireless/acx, net-wireless/acx-firmware, net-wireless/linux-wlan-ng-modules, net-wirele

[Sebastian Pipping]

On 11/24/2012 10:12 PM, Pacho Ramos wrote:
> # Pacho Ramos  (24 Nov 2012) # Upstream dead and
> no longer runs (#402669). # Removal in a month app-cdr/dvd95

Bug fixed.  I just ripped a DVD with dvd95 successfully.

+  02 Dec 2012; Sebastian Pipping  package.mask:
+  Keep dvd95 since bug #402669 is now fixed
+


> # Pacho Ramos  (24 Nov 2012) # Fails to build
> with gcc-4.7 and maintainer is ok with dropping (#424723). #
> Removal in a month. app-shells/localshell

FYI bug fixed, removal prevented by robbat2.

Best,



Sebastian

[gentoo-dev] [rfc] dev-libs/expat[unicode] and dev-libs/libbsd dependency

[Sebastian Pipping]

Hi!


The next release of dev-libs/expat is not far away and there are two
things that I would appreciate input with, before the next bump in Gentoo:


-DXML_UNICODE_WCHAR_T issues and Gentoo/Debian mismatch
===

With USE=unicode, on Gentoo two extra libraries are built:

 * libexpatu.so (with CPPFLAGS=-DXML_UNICODE)
 * libexpatw.so (with CPPFLAGS=-DXML_UNICODE_WCHAR_T)
   ^
However, -DXML_UNICODE_WCHAR_T has only ever worked with 2-byte wchar_t,
while 4-byte wchar_t seems mainstream on Linux (and GCC -fshort-wchar
would required libc to have the same, if you actually wanted to pass
those wchar_t strings to wprintf and friends).

So libexpatw.so in Gentoo is not functional at the moment.

To make things worse, Debian has libexpatw.so with
CPPFLAGS=-DXML_UNICODE, which corresponds to current libexpatu.so in
Gentoo, rather than libexpatw.so.


How do you evaluate these options:

 a) Keep libexpatu.so + change libexpatw.so to CPPFLAGS=-DXML_UNICODE

 b) Drop libexpatu.so + change libexpatw.so to CPPFLAGS=-DXML_UNICODE


Depend on dev-libs/libbsd
=

The next release is very likely to add (optional but helpful) support
for arc4random_buf that dev-libs/libbsd provides (especially on systems
with glibc prior to 2.25) [1].  I wonder if Expat's proximity to @system
has any strong implications on whether

 A) libbsd should be a default-off use dependency
  IUSE="libbsd"  RDEPEND="libbsd? ( dev-libs/libbsd )"

 B) libbsd could be a default-on use dependency
  IUSE="+libbsd"  RDEPEND="libbsd? ( dev-libs/libbsd )"

 C) libbsd could even go into DEPEND and RDEPEND directly, or
  RDEPEND="dev-libs/libbsd"

 D) libbsd should not become any kind of future dependency of
dev-libs/expat.


Thanks for your time!

Best



Sebastian


[1] https://github.com/libexpat/libexpat/pull/30

Re: [gentoo-dev] [rfc] dev-libs/expat[unicode] and dev-libs/libbsd dependency

[Sebastian Pipping]

Hi,


On 31.05.2017 21:16, Michał Górny wrote:
>> How do you evaluate these options:
>>
>>  a) Keep libexpatu.so + change libexpatw.so to CPPFLAGS=-DXML_UNICODE
>>
>>  b) Drop libexpatu.so + change libexpatw.so to CPPFLAGS=-DXML_UNICODE
> 
> Does any other distribution use libexpatu.so? If not, then there's
> probably no point in keeping it.

I found none but CoreOS, which is derived from Gentoo (..).


>>  A) libbsd should be a default-off use dependency
>>   IUSE="libbsd"  RDEPEND="libbsd? ( dev-libs/libbsd )"
>>
>>  B) libbsd could be a default-on use dependency
>>   IUSE="+libbsd"  RDEPEND="libbsd? ( dev-libs/libbsd )"
> 
> I'd dare say the feature is 'arc4random', then that should be the name
> of the flag.

Good point.

Best



Sebastian



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] [rfc] dev-libs/expat[unicode] and dev-libs/libbsd dependency

[Sebastian Pipping]

Hi!


Just quick note for the record: 2.2.0-r2 has these changes now, no need
to have that wait for the next release:

https://github.com/gentoo/gentoo/commit/715a2315ee2b841e38843e61b43ee058b5678cab

Best



Sebastian



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] [RFC] Addition of a new field to metadata.xml

[Sebastian Pipping]

On 01.06.2017 23:18, Jonas Stein wrote:
> 2. Specification
> 
> A space separated list of the corresponding debian packages should be
> written in the field
>  
> 
> It should be NONE, if debian has no corresponding package.
> UNSET or no field, if the creator of the ebuild did not set the field (yet).

Please pick NONE or require absence eventually, but not multiple
options.  Else we're asking for inconsistent data from the beginning.


> example:
> app-arch/tar/metadata.xml
> tar
> 
> app-office/libreoffice-bin/metadata.xml
> libreoffice libreoffice-base libreoffice-base
> libreoffice-dev libreoffice-dmaths libreoffice-draw
> libreoffice-evolution libreoffice-impress

Since the difference between source and binary packages has already been
brought up, please adjust "" some way to
indicating if the text content is a source or a binary package (even if
we don't end up supporting both) to be 100% clear.  Otherwise people
will mix it up, and may not even notice.

Best



Sebastian

[gentoo-dev] Merge 7 Fedora wallpapers packages to single one with slots?

[Sebastian Pipping]

Hi!


I noticed that we have 7 packages on Fedora wallpapers with names that
only explain themselves to Fedora insiders:


# eix background | fgrep -B3 Fedora
* x11-themes/constantine-backgrounds
 Available versions:  12.1.1.4-r1
 Homepage:https://fedoraproject.org/wiki/F12_Artwork
 Description: Fedora official background artwork
--
* x11-themes/goddard-backgrounds
 Available versions:  13.0.0.3-r1
 Homepage:https://fedoraproject.org/wiki/F13_Artwork
 Description: Fedora official background artwork
--
* x11-themes/laughlin-backgrounds
 Available versions:  14.1.0.3-r1
 Homepage:https://fedoraproject.org/wiki/F14_Artwork
 Description: Fedora official background artwork
--
* x11-themes/leonidas-backgrounds
 Available versions:  11.0.0.2-r1
 Homepage:https://fedoraproject.org/wiki/F11_Artwork
 Description: Fedora official background artwork
--
* x11-themes/lovelock-backgrounds
 Available versions:  14.91.1.1-r1
 Homepage:https://fedoraproject.org/wiki/F15_Artwork
 Description: Fedora official background artwork
--
* x11-themes/solar-backgrounds
 Available versions:  0.92.0.5-r1
 Homepage:https://fedoraproject.org/wiki/F11_Artwork
 Description: Fedora official background artwork
--
* x11-themes/verne-backgrounds
 Available versions:  (~)15.91.0.1-r1
 Homepage:https://fedoraproject.org/wiki/F16_Artwork
 Description: Fedora official background artwork


I was thinking that we could merge these packages into a new package
"x11-themes/fedora-backgrounds" or so with slots 11 to 16 so that people
can still install them in parallel, get slot updates automatically,
adding more recent ones does not add more packages, and the package name
explains itself.

Any objections?

Best



Sebastian

Re: [gentoo-dev] time to retire

[Sebastian Pipping]

Stefan,


thanks for your work on Gentoo!

All the best



Sebastian

Re: [gentoo-dev] Merge 7 Fedora wallpapers packages to single one with slots?

[Sebastian Pipping]

Hi,


On 27.01.2018 17:32, Michael Orlitzky wrote:
> If you do merge them, then it might be better to use flags for the
> different sub-packages rather than slots. There's no place to describe
> what a slot is for, but having a local USE=solar with a corresponding
> description in metadata.xml is (relatively) discoverable.

use flags work well if we make a single ebuild offering to install one
to all of these.  That would be natural if the goddard/13 source rpm
included files of constantine/12 and solar/11 as well and so on but it
doesn't seem to.  I would rather go with one ebuild per mayor release
number of Fedora: Needs less use flag configuration as well.

About slot names, if "12" is not good enough we could use

  11-solar
  12-constantine
  13-goddard
  14-laughlin
  15-lovelock
  16-verne

or so for SLOT to have a mapping to names?

Best



Sebastian

Re: [gentoo-dev] Merge 7 Fedora wallpapers packages to single one with slots?

[Sebastian Pipping]

On 27.01.2018 19:06, Sebastian Pipping wrote:
>   11-solar
>   12-constantine
>   13-goddard
>   14-laughlin
>   15-lovelock
>   16-verne

Correction:

  10-solar
  11-leonidas
  12-constantine
  13-goddard
  14-laughlin
  15-lovelock
  16-verne

Re: [gentoo-dev] Merge 7 Fedora wallpapers packages to single one with slots?

[Sebastian Pipping]

Hi Alec,


On 27.01.2018 22:58, Alec Warner wrote:
> > I noticed that we have 7 packages on Fedora wallpapers with names that
> > only explain themselves to Fedora insiders:
> 
> So traditionally we follow upstream package naming. If we aim to
> deviate, I'd prefer we have strong reasons for it.

good point.


> > I was thinking that we could merge these packages into a new package
> > "x11-themes/fedora-backgrounds" or so with slots 11 to 16 so that people
> > can still install them in parallel, get slot updates automatically,
> > adding more recent ones does not add more packages, and the package name
> > explains itself.
> 
> Why not just make x11-themes/fedora-backgrounds, a metapackage that
> includes all of the packages?

With one file and use flags for each version or with one ebuild file per
slot?  Fedora 21 was the last release with a release name so if we
package 22+ later, their ebuilds would be non-meta in nature.  I'm not
sure how to blend that into the use-flag version (yet for a meta package
all these files seem overkill too).  Do you have some third option in mind?

Best



Sebastian

[gentoo-dev] Last rites: media-gfx/drqueue

[Sebastian Pipping]

# Sebastian Pipping  (08 Oct 2016)
# Dead upstream for years, ebuild needs work, 5 open bugs
# Masked for removal in 30 days.
media-gfx/drqueue

Re: [gentoo-dev] About EGO_SUM

[Sebastian Pipping]

On 08.06.22 22:42, Robin H. Johnson wrote:

EGO_SUM vs dependency tarballs:
[..]
- EGO_SUM is verifiable/reproducible from Upstream Go systems


Let's be explicit, there is a _security_ threat here: as a user of an
ebuild, dependency tarballs now take effort in manual review just to
confirm that the content full matches its supposed list of ingredients.
They are the perfect place to hide malicious code in plain sight.  Now
with dependency tarballs, there is a new layer that by design will
likely be chronically under-audited.  It gives me shivers, frankly.
Previously with a manifest and upstream-only URLs, only upstream can add
malicious code, not downstream in Gentoo.

Best



Sebastian

Re: [gentoo-dev] Last rites: www-client/chromium-bin

[Sebastian Pipping]

On 04.05.23 20:59, Maciej Barć wrote:

R.i.p. to a lot od desktop users on non-state-of-the-art HW.


Building Chromium inside a cloud VM may be an option for some users with 
older hardware.  When using e.g. 
https://github.com/hartwork/binary-gentoo for that, the VM doesn't even 
have to run Gentoo.

[gentoo-dev] Package up for grabs: www-client/httrack

[Sebastian Pipping]

Hi!


Just a quick note that I have just dropped maintainership of package…

  www-client/httrack

…, about 8 years after my first commit on the package and after bumping 
to 3.49.5 and asking for its stabilization due to a security fix about a 
minute ago or two, as my last action on the package.



I have dropped the package now because of upstream behavior in recent
years; issues…

  https://github.com/xroche/httrack/issues/243
  https://github.com/xroche/httrack/issues/244
  https://github.com/xroche/httrack/issues/245
  https://github.com/xroche/httrack/issues/247

…would be the places to look for details of their and my actions, if
curious.


Regarding open bugs about www-client/httrack in Gentoo:

 - https://bugs.gentoo.org/917589 and
   https://bugs.gentoo.org/923034 and
   https://bugs.gentoo.org/923035
   are all about stabilization, keywording, a fixed vulnerability

 - https://bugs.gentoo.org/768186
   is unconfirmed (i.e. not reproducible) and of unclear importance


Have a nice day



Sebastian

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Sebastian Pipping]

On 20.12.2012 19:14, Ciaran McCreesh wrote:
> The tree is a database. It belongs in /var/db/.

I don't see /var/db in the latest release of the Filesystem Hierarchy
Standard:

  http://www.pathname.com/fhs/pub/fhs-2.3.html#THEVARHIERARCHY

I would prefer something that blends with FHS.

Best,



Sebastian

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Sebastian Pipping]

On 20.12.2012 18:27, Ulrich Mueller wrote:
> Now I wonder: After removal of e.g. the Portage tree from a system, it
> is generally not possible to restore it. (It can be refetched, but not
> to its previous state.)
> 
> Same is true for distfiles, at least to some degree. They may have
> vanished upstream or from mirrors.
> 
> Maybe /var/lib would be a better choice? It would also take care of
> the issue with fetch-restricted files.

Thanks for bringing it up.  What you address above is the exact reason
why Layman's home was moved to /var/lib/layman/ eventually.  It has a
cache aspect, bit it's not a true cache.

Best,



Sebastian

Re: [gentoo-dev] What did we achieve in 2012? What are our resolutions for 2013?

[Sebastian Pipping]

Coming to my mind:


There have been continued regular releases of genkernel integrating
patches from various people:
http://git.overlays.gentoo.org/gitweb/?p=proj/genkernel.git;a=tags

And there has been a constant stream of people asking for user overlay
hosting or getting an existing overlay being added to the layman
registry that we could serve.


Ben, I hope you have the time to make a news post from this thread's
collection?


Best,



Sebastian

Re: [gentoo-dev] New Python eclasses -- a summary and reminder

[Sebastian Pipping]

Looks like great work so far.


On 11.02.2013 01:20, Michał Górny wrote:
> Secondly, I'd like to make it clear that the old python.eclass is 
> 'almost' deprecated. We're in process of converting the in-tree 
> packages to use the new eclasses but that's a lot of work [3].
> 
> [..]
> 
> [3]:http://dev.gentoo.org/~mgorny/python-r1/conversion.xhtml

I wonder what would be the best way to help with conversion for devs
with a few hours to contribute?

 - Where does one go for peer review and how many eyes should be on
   related commits?

 - Should package "owners" be contacted in all cases?

 - Are there any conversion sprints already or in the future?

Best,



Sebastian

[gentoo-dev] Packages up for grabs

[Sebastian Pipping]

Hello!


Below are some packages that I fail to take care of as needed and have
not been using myself for a while.  Please take over whatever you have
interest in:

   Latest  Open bugs
  
  app-text/xmlstarlet  yes no

  media-libs/libmp3spltyes yes
  media-sound/mp3splt-gtk  yes yes
  media-sound/mp3splt  yes yes

  dev-python/html2text no/2014.9.8 no
  dev-python/inotifyx  no/0.2.2no
  games-action/openlierox  no/0.59_beta10  no
  media-gfx/drqueueyes yes
  media-libs/opencollada   no/?yes
  sys-fs/pytagsfs  yes yes


Many thanks,



Sebastian

Re: [gentoo-dev] Last rites: dev-php/{adodb-ext,eaccelerator,pecl-apc,pecl-id3,pecl-mogilefs,pecl-sca_sdo,suhosin}

[Sebastian Pipping]

Hi Brian,


On 02.10.2014 20:29, Brian Evans wrote:
> # Brian Evans  ( 1 Oct 2014 ) # Masked for
> removal in 30 days. # Broken on >=dev-lang/php-5.4.  No
> replacements known. [..] dev-php/suhosin

is that true for suhosin?

Upstream reads

  "has been tested with PHP 5.4 and 5.5" [1]

and there is

  dev-php/suhosin: version bump - 0.9.36 is available and has been
  tested with PHP 5.4 and 5.5
  https://bugs.gentoo.org/show_bug.cgi?id=520178

too.

So at least to me this looks like to early or potentially not even
needed.  If it is broken fro 5.4/5.5 please share details on why it
really is and update the bug mentioned above too, please.

Thanks!

Best,



Sebastian


[1] http://suhosin.org/stories/install.html

Re: [gentoo-dev] rsync mirrorstats page (generation and parsing)

[Sebastian Pipping]

Zac Medico wrote:
> Yeah, here's the history since I started maintaining it:
> 
> http://dev.gentoo.org/~zmedico/projects/mirrorselect.git/

I've been adding mirror3.xml support to the above today.

Repo over here:
http://git.goodpoint.de/?p=mirrorselect.git;a=summary



Sebastian

Re: [gentoo-dev] rsync mirrorstats page (generation and parsing)

[Sebastian Pipping]

Robin H. Johnson wrote:
> I'll try to suck that down soon and build up a larger history with old
> tarballs, and then push it somewhere useful.

To re-build mirrorselect's complete history we'd need the original
tarballs for each line starting with "[ ] " below.

Please let us now if you have some of these files on some harddisc of
yours.  Here's the list

[F] 0.1
[F] 0.1-r1
[F] 0.1-r2
[F] 0.2
[F] 0.2-r1
[F] 0.3
[ ] 0.4
[ ] 0.5
[ ] 0.6
[ ] 0.7
[ ] 0.7-r1
[ ] 0.8
[ ] 0.81
[ ] 0.82
[R] 0.82-r1
[R] 0.82-r2
[R] 0.82-r3
[R] 0.83
[R] 0.84
[ ] 0.85
[ ] 0.86
[R] 0.87
[R] 0.89
[ ] 1.0
[ ] 1.0.1
[ ] 1.0.2
[ ] 1.0.3
[ ] 1.0.4_rc2
[ ] 1.1
[ ] 1.1.1
[ ] 1.1.2
[ ] 1.1.3
[ ] 1.1.4
[ ] 1.1.5
[R] 1.1.6
[R] 1.1.7
[R] 1.2
[R] 1.3
[R] 1.4
[R] 1.4.1
[R] 1.4.2


F = files dir in
http://sources.gentoo.org/viewcvs.py/gentoo-x86/app-admin/mirrorselect/?hideattic=0
R = http://dev.gentoo.org/~robbat2/mirrorselect-archive/



Sebastian

[gentoo-dev] [rfc] repo_name/layman-global.txt overlay name mismatches

[Sebastian Pipping]

Hello!


Intro

  For the gentoo-specific part of smolt [1] that's in the making we
  collect stats on what package is installed and what overlay it comes
  from.  Which tree/overlay it came from is determined by reading file
  /var/db/pkg/${CATEGORY}/${PF}/repository.  The name in there comes
  from file profiles/repo_name originally.
  In smolt we match repo names from the 'repository' file against
  overlay names from layman-global.txt to check if an overlay is known
  worldwide or could be a secret overlay that we should not submit
  information about.


Problem

  When the name from repo_name and the overlay name in layman-global.txt
  do not match smolt would assume the overlay is secret though it's not
  and not be able to send in stats about it.


Examples:

  oss-overlay != majeru
  pro-audio != proaudio


Proposal

  -   Make repo_name file fit layman-global.txt entry where it doesn't
  match at the moment

  -   People with that overlay installed do not have to change
  anything, no "where did that overlay go" mails or aynthing

  -   Upstream needs to be convinced to fix it

  -   For packages installed from that overlay the file
  'repository' will not update until the a re-installation

  -   Make in-sync repo_name a requirement for extending
  layman-global.txt



Sebastian

[gentoo-dev] Wanted: Stats collection (not submission!) testimonials

[Sebastian Pipping]

Hello!


As the collection part of my "bring stats to Gentoo" project is complete
by now, it's a good point in time to do a bit more testing.  If you can
contribute a few minutes to it that would rock, please read on.

The idea is that you as a human (not machine) with a system different
from mine can find stuff that needs better (or different) handling quite
well.

The task is:

  Let the current collector script run on your machine, have
  a critical look at it's (hopefully self-explaining) output,
  and report back what you found.

Here are precise commands you need to run:

  # git clone git://git.goodpoint.de/smolt-gentoo.git
  # cd smolt-gentoo
  # git checkout -b gentoo origin/gentoo
  # cd client/distros
  # python gentoo.py |& less

Again, no data is submitted, just locally collected and presented.

Please send your feedback to the list;  collected data better goes
to  if you are willing to share it.

Thanks in advance for your help!



Sebastian

Re: [gentoo-dev] [rfc] repo_name/layman-global.txt overlay name mismatches

[Sebastian Pipping]

Fabian Groffen wrote:
> So alternative, what if
> we extend the layman-global.txt (which is xml in reality...) file with
> an extra property per overlay which holds the contents of it's
> repo_name?

Good idea.

I'll try to create a map for all overlays in layman-global.txt as a next
step.  Layman and shell tools should help with that.



Sebastian

Re: [gentoo-dev] [rfc] repo_name/layman-global.txt overlay name mismatches

[Sebastian Pipping]

Sebastian Pipping wrote:
> I'll try to create a map for all overlays in layman-global.txt as a next
> step.  Layman and shell tools should help with that.

Believe it or not: discs space issues disallow me to have an answer
already.  The code is there, I just cannot checkout all overlays at the
same time :-(

Maybe anybody else could run below commands before I find time to fix
the disc space thing.


High level view:

  - Checkout the mismatch detector script
  - Backup the set of names of your current overlays
  - Add all overlays
  - Run the script
  - Remove all overlays
  - Restore the set of original overlays


Low level view:

  # git clone git://git.goodpoint.de/smolt-gentoo.git
  # cd smolt-gentoo
  # git checkout -b gentoo origin/gentoo
  # cd client/distros/gentoo

  # layman -l > my_active_overlays.txt
  # sudo layman -f
  # sudo layman -a ALL
  # python _mismatch.py | tee repo_name_analysis.txt
  # sudo layman -d ALL
  # sed 's|^[^ ]* \([^ ]\+\).*$|\1|' my_active_overlays.txt | \
  xargs -n 1 sudo layman -a


The output produces line like

  MISMATCH "gentoo-china" (layman-global) versus "china" (repo_name)

Please share the resulting repo_name_analysis.txt with us.

I guess it all sounds a bit odd, but maybe it's fun to you and we have
an answer faster.  I have 17 mismatches here already.




Sebastian

Re: [gentoo-dev] [rfc] repo_name/layman-global.txt overlay name mismatches

[Sebastian Pipping]

Robert Buchholz wrote:
>  1. has no DTD/xml validation schema

I'd like to be part of the schema creation process but feel that having
pre-mature schema's on the list and it's archives is not a good idea.

If we had a schema file: where would we store it?



Sebastian

Re: [gentoo-dev] Wanted: Stats collection (not submission!) testimonials

[Sebastian Pipping]

Tobias, thanks for taking the time to test my code!


Tobias Klausmann wrote:
> /home/klausman/tmp/smolt-gentoo/client/distros/gentoo/globaluseflags.py:22:
> DeprecationWarning: the sets module is deprecated

I'm looking for advice how to best handle this.

@all
If you read this and know how please contact me!



Sebastian

Re: [gentoo-dev] Wanted: Stats collection (not submission!) testimonials

[Sebastian Pipping]

Victor, thanks for participating!


Victor Ostorga wrote:
> 1. Local overlays are not taken into account, the output is like
> follows:
> 
> Active overlays:
>   Names:
> []
>   Paths:
> []
>   Total: 1
> Known: 0
> Secret: 1

Does it have a profiles/repo_name file?
Can you share the output of

 # emerge --info --verbose | grep OVERLAY

?


> 2. Package mask shows no output (assuming you are taking it
> from /etc/portage/package.keywords) package.mask:
> 
>   Total: 0
> Known: 0
> Secret: 0

It's from /etc/portage/package.mask .
Is that non-empty for you?



Sebastian

Re: [gentoo-dev] Wanted: Stats collection (not submission!) testimonials

[Sebastian Pipping]

Victor Ostorga wrote:
>>> 1. Local overlays are not taken into account, the output is like
>
> $ emerge --info --verbose | grep OVERLAY
> PORTDIR_OVERLAY="/usr/local/portage"
> 
> It is a local overlay which I use to do some tests.

That overlay is ignored to protect your privacy.

A good way to include it in the collection process
would be:

 - Find a good name for it

 - echo that name to profiles/repo_name

 - Publish the repo somewhere on the web,
   say using Git and Github.com

 - Send a mail to rbu requesting addition to layman-global.txt

How about that?



Sebastian

Re: [gentoo-dev] [rfc] repo_name/layman-global.txt overlay name mismatches

[Sebastian Pipping]

So here's the current result of my analysis:


==
Format:
"",  # 

Entries:
"maekke's overlay",  # maekke
"kde",  # kde-testing
ERROR: Overlay "lordvan" lacks repo_name entry
ERROR: Overlay "rox" lacks repo_name entry
ERROR: Overlay "rion" lacks repo_name entry
ERROR: Overlay "gnash-cvs" lacks repo_name entry
ERROR: Overlay "php-4" lacks repo_name entry
ERROR: Overlay "rostov" lacks repo_name entry
ERROR: Overlay "roslin" lacks repo_name entry
ERROR: Overlay "postgresql-testing" lacks repo_name entry
ERROR: Overlay "pd-overlay" lacks repo_name entry
"geki-overlay",  # openoffice-geki
"mpd-portage",  # mpd
"tante_overlay",  # tante
ERROR: Overlay "stuart-desktop" lacks repo_name entry
ERROR: Overlay "d" lacks repo_name entry
"gentoo-lisp-overlay",  # lisp
ERROR: Overlay "chtekk-syscp" lacks repo_name entry
"soor",  # soor-overlay
"dev-jokey",  # jokey
"Falco's git overlay",  # falco
"kde4-experimental",  # kde
ERROR: Overlay "hanno-xgl" lacks repo_name entry
ERROR: Overlay "sipx" lacks repo_name entry
ERROR: Overlay "cell" lacks repo_name entry
ERROR: Overlay "pchrist" lacks repo_name entry
"digital-trauma.de",  # trauma
ERROR: Overlay "genscripts" lacks repo_name entry
ERROR: Overlay "powerpc" lacks repo_name entry
"gcj-overlay",  # java-gcj-overlay
"steev_github",  # steev
"pcsx2-overlay",  # pcsx2
"vdr-xine-overlay",  # vdr-xine
"china",  # gentoo-china
ERROR: Overlay "marineam-xen" lacks repo_name entry
"gentoo-haskell",  # haskell
ERROR: Overlay "openmoko" lacks repo_name entry
ERROR: Overlay "tove" lacks repo_name entry
ERROR: Overlay "stuart-server" lacks repo_name entry
ERROR: Overlay "deathwing00" lacks repo_name entry
ERROR: Overlay "trapni" lacks repo_name entry
ERROR: Overlay "gnr" lacks repo_name entry
"webapp-experimental",  # webapps-experimental
ERROR: Overlay "eclipse" lacks repo_name entry
"proaudio",  # pro-audio
ERROR: Overlay "seemant" lacks repo_name entry
ERROR: Overlay "pythonhead" lacks repo_name entry
ERROR: Overlay "initng" lacks repo_name entry
ERROR: Overlay "gentoojp" not found
"majeru",  # oss-overlay
ERROR: Overlay "plan9" lacks repo_name entry
ERROR: Overlay "m68k" lacks repo_name entry
ERROR: Overlay "iwlwifi" lacks repo_name entry
"bangert-ebuilds",  # bangert
"suka's dev overlay - experimental stuff of all sorts",  # suka
ERROR: Overlay "stuart-perforce" lacks repo_name entry
ERROR: Overlay "break-my-gentoo-main" lacks repo_name entry
ERROR: Overlay "ramereth" lacks repo_name entry
"leio-personal",  # leio
ERROR: Overlay "aross" lacks repo_name entry
"ogo-lu_zero",  # lu_zero
ERROR: Overlay "mysql-testing" lacks repo_name entry
"scarabeus_local_overlay",  # scarabeus
ERROR: Overlay "liquidx" lacks repo_name entry
ERROR: Overlay "lila-theme" lacks repo_name entry
==


That means quite a lot to do.  Shall I make the script send e-mails to
the overlay contacts? :-)

I have added the mismatched repo_names to a whitelist in Gentoo Smolt,
so packages installed before a repo_name fix should be collectable, too.



Sebastian

Re: [gentoo-dev] Inviting you to project "PackageMap"

[Sebastian Pipping]

Petteri Räty wrote:
> You need to come up with the needed DTD changes for metadata.xml. Last
> time the schema was changed it was done with a GLEP so writing one seems
> prudent here

I have started
- writing a GLEP
- extending the DTD
- extending a sample metadata.xml

Related gitweb over here:
http://git.goodpoint.de/?p=metadata-xml-cpe-glep.git

Would be great to get some review.



Sebastian

Re: [gentoo-dev] [rfc] repo_name/layman-global.txt overlay name mismatches

[Sebastian Pipping]

Sebastian Pipping wrote:
> Robert Buchholz wrote:
>>  1. has no DTD/xml validation schema
> 
> I'd like to be part of the schema creation process [..]

First try on a DTD and Relax NG schema for Layman's current overlays.xml
format (used in layman-global.txt) over here:

http://git.goodpoint.de/?p=overlays-xml-specification.git

Feedback welcome.



Sebastian

Re: [gentoo-dev] rsync mirrorstats page (generation and parsing)

[Sebastian Pipping]

Hello!


So what's needed to get a new mirrorselect release out?



Sebastian

Re: [gentoo-dev] rsync mirrorstats page (generation and parsing)

[Sebastian Pipping]

Zac Medico wrote:
>> So what's needed to get a new mirrorselect release out?
> 
> Are all of your changes here?
> 
> git://git.goodpoint.de/mirrorselect.git

Yes.


> Now we just need to create an ebuild to install it, and put it in
> the tree. You can file a bug for that and assign it to tools-portage.

Done.

https://bugs.gentoo.org/show_bug.cgi?id=278351



Sebastian

[GLEP] CPE names in metadata (was Re: [gentoo-dev] Inviting you to project "PackageMap")

[Sebastian Pipping]

Sebastian Pipping wrote:
> I have started
> - writing a GLEP
> - extending the DTD
> - extending a sample metadata.xml
> 
> Related gitweb over here:
> http://git.goodpoint.de/?p=metadata-xml-cpe-glep.git

Especially as this is my first GLEP and it will affect most of you in
the long run, I depend on your feedback here.
Just added more words on CPE names and replaced the given example.

Please have a look and tear it apart :-)

Thanks in advance,



Sebastian

Re: [gentoo-dev] Re: About time to unify 'cdda' and 'cdaudio' USE flags and make the remaining one global?

[Sebastian Pipping]

Pacho Ramos wrote:
> Have you think about enabling "cdda" USE flag by default in *desktop*
> profiles? I think that most of "desktop" users will want to get cdaudio
> support by default

There's quite a few notebooks without cd/dvd drives around these days.
I cannot tell how much that's in percent of all desktop users but
we'll have a stats tool to answer questions like that from real world
data soon. :-)



Sebastian

[gentoo-dev] Gentoo stats server/client @ 2009-07-29

[Sebastian Pipping]

Hello!


Just a few words for proof of life.  I'm knee-deep in SQL alchemy stuff
at the moment.  I guess a good explanation why Gentoo had to live
without its own "popcon" before is that it's a lot of work shoveling the
data into a database alone, relating to a 20+ table scenario.  Still,
things are looking good, just more work to do.

My non-Gentoo work on Smolt has been integreated upstream and I gained
push access in the meantime.
Maybe more time to write about stuff later, maybe even pictures.

Stay tuned,



Sebastian

Re: [gentoo-dev] Test request: Bugzilla load balancer

[Sebastian Pipping]

Robin H. Johnson wrote:
> So, please test at:
> http://bugs-web-lb.gentoo.org/
> HTTP and HTTPS available.

Nice, makes a very responsive impression to me.



Sebastian

Re: [gentoo-dev] Monthly Gentoo Council Reminder for August

[Sebastian Pipping]

Mike Frysinger wrote:
> If you have something you'd wish for us to chat about, maybe even
> vote on, let us know !  Simply reply to this e-mail for the whole
> Gentoo dev list to see.

I would love to see the GLEP on CPE names in metadata.xml discussed,
http://www.mail-archive.com/gentoo-dev@lists.gentoo.org/msg35155.html

Any guidance on what I need to do to make it happen is very welcome.

Thanks,



Sebastian

[gentoo-dev] Gentoo stats server/client @ 2009-08-02

[Sebastian Pipping]

Hello again!


Just a few quick updates.  By now the server side accepts
Gentoo-specific data from the client (transfered as JSON) and updates
that machine's previous submission in the database.  (That's not as
trivial as it may sound at first.)  As a single submission produces a
few thousand INSERTs easily (from installed packages details mainly) I
have been migrating the server to do batch processing in hope to reduce
server load later.  If smolt upstream likes the code it might go into
Smolt earlier than the Gentoo extensions itself.



Sebastian

Re: [gentoo-dev] Gentoo stats server/client @ 2009-08-02

[Sebastian Pipping]

I haven't mentioned yet that I have started using the provided Redmine
installation, especially its bug tracker lately.  The reason I post
about this is these two links that might be of interest to some of you:

  Gentoo/Smolt/GSOC tasks
http://soc.gentooexperimental.org/projects/stats/issues

  Gannt chart on these
http://soc.gentooexperimental.org/projects/gantt/stats

I've been working on report generation today.  More later.



Sebastian

[gentoo-dev] [rfc] paludis and portage in gentoo statistics

[Sebastian Pipping]

hi there!


intro + high level stuff
=   

the stats gathering project that i'm currently working on mainly
involves data specific to the package manager, i.e. portage.  as some
gentoo users are using paludis instead of portage the question arises if
and how we should integrate data submitted from machines using paludis
or even paludis and portage in parallel.  it might be good to be clear
about this before the actual launch of the service to do better long
term design decisions, even if paludis support is beyond the scope of
the GSoC project.  that's why i bring the topic up now.

any ideas or advice on how we could or should handle that or what to avoid?


implementation
==

a few variables in portage that do not have a direct equivalent in paludis:

  - ACCEPT_KEYWORDS: deduce from '*/*' line in platforms.conf?
  - ARCH: deduce from CHOST and deduced ACCEPT_KEYWORDS
  - SYNC: not deducable

atoms in paludis seem to be a superset of portages.  if so that would
not cause any trouble so far.

any technical differences that you consider to become a problem?

thanks in advance,



sebastian

[gentoo-dev] Stats test server running, please check it out

[Sebastian Pipping]

Hello again!


I have set up a test server of the current stats code.  If you have a
minute to check it out that would rock.  I'm very interested in overall
feedback and bug reports.

To check it out please do as following:


0)  Make sure you have these packages installed:
sys-apps/portage
dev-util/git
dev-python/rhpl
dev-python/urlgrabber
dev-python/simplejson
dev-python/dbus-python

1)  Check out the code
# git clone git://git.goodpoint.de/smolt-gentoo.git
# cd smolt-gentoo/client
# git checkout --track -b gentoo origin/gentoo

2)  Verify my server is running at the very moment by browsing to
http://smolt.hartwork.org:45678/static/stats/gentoo.html

3)  Run the client  (which asks and shows details before submission)
# python sendProfile.py \
--server=http://smolt.hartwork.org:45678/


Please note that a summary update is triggered manually so gentoo.html
will not update instantly.  You can ping me on Freenode though, to make
me trigger an update earlier.

To disable certain classes of data for privacy reasons you can put this
in ~/.smolt/client.cfg

[gentoo]
arch_related = True
compile_flags = True
features = True
global_use_flags = True
installed_packages = True
installed_packages_use_flags = True
mirrors_sync = True
mirrors_distfiles = True
package_mask = True
repositories = True
system_profile = True

and turn "True" to "False" where necessary.

Have fun and please share your experience with it.



Sebastian

Re: [gentoo-dev] Stats test server running, please check it out

[Sebastian Pipping]

Sebastian Pipping wrote:
> 0)  Make sure you have these packages installed:
> sys-apps/portage
> dev-util/git
> dev-python/rhpl
> dev-python/urlgrabber
> dev-python/simplejson
> dev-python/dbus-python

>From what I hear not everyone has the HAL daemon installed.
HAL is required, too:

sys-apps/hal



Sebastian

Re: [gentoo-dev] Stats test server running, please check it out

[Sebastian Pipping]

Sebastian Pipping wrote:
> 3)  Run the client  (which asks and shows details before submission)
> # python sendProfile.py \
> --server=http://smolt.hartwork.org:45678/

I forgot to mention you need to create a random machine id one way or
another before you can submit data.  An easy and transparent way to do
that would be running this command:

  # sudo sh -c 'cat /proc/sys/kernel/random/uuid > /etc/smolt/hw-uuid'

The smolt ebuild is doing the very same.


We have data from 8 machines so far, you can be number 9.
http://smolt.hartwork.org:45678/static/stats/gentoo.html




Sebastian

Re: [gentoo-dev] Re: Stats test server running, please check it out

[Sebastian Pipping]

Mark Bateman wrote:
> emerge rhpl -va
> 
> These are the packages that would be merged, in order:
> 
> Calculating dependencies... done!
> [ebuild  N] net-wireless/wireless-tools-29  USE="nls -multicall" 288 kB
> [ebuild  N] dev-python/rhpl-0.213  236 kB
> 
> Total: 2 packages (2 new), Size of downloads: 523 kB
> 
> Would you like to merge these packages? [Yes/No] 
> 
> 
> 
> 
> is this correct? 

yes,  thanks for asking.

(though my code neither uses that component nor do i know what it does)



sebastian

Re: [gentoo-dev] Stats test server running, please check it out

[Sebastian Pipping]

Dirkjan Ochtman wrote:
> On Mon, Aug 10, 2009 at 12:08, Sebastian Pipping 
> wrote:
>> 0)  Make sure you have these packages installed:
>>dev-python/rhpl
>>dev-python/urlgrabber
>>dev-python/dbus-python
> 
> What do you need these for?

a short and correct answer would be that smolt 1.2 already depends on
them.  i cannot say much more, except that urlgrabber might be
replaceable.  i don't use any of them in the gentoo-specific extensions.



sebastian

Re: [gentoo-dev] Multimedia overlay

[Sebastian Pipping]

Ben de Groot wrote:
> So hereby we announce the Gentoo Multimedia overlay. It is located at
> http://gitorious.org/gentoo-multimedia and any developers who want to
> join can let us know their gitorious account name, so they can be added.

Good idea!

Out of curiousity:  Is a gitorious account a technical must or would an
SSH pubkey do as well?



Sebastian

Re: [gentoo-dev] Multimedia overlay

[Sebastian Pipping]

Federico Ferri wrote:
> note that for sound-related packages there exists pro-audio overlay

my impression is that pro-audio really focuses on _professional_ audio
software.  that's two very different sets of applications.  i don't see
any benefits from mixing them, do you?



sebastian

Re: [gentoo-dev] Multimedia overlay

[Sebastian Pipping]

Josh Saddler wrote:
> I'm well aware of that. It's A Dumb Policy(tm). :)

What advantage do you see in forcing people to have their overlays
hosted on http://git.overlays.gentoo.org/gitweb/ ?



Sebastian

Re: [gentoo-dev] Stats test server running, please check it out

[Sebastian Pipping]

Federico Ferri wrote:
> 1st: you could make an ebuild for it ;)

i just made one but it's not that useful yet, as the code is not
runnable from any location yet...


> 2nd: how to improve the output: you could make every data an
> hyperlink: that would help understand better the contents

actually there's a task for that open already:
http://soc.gentooexperimental.org/issues/show/51

i completely agree about it.


> FEATURES
>   can link feature to make.conf man page:
> http://linuxreviews.org/man/make.conf/ (unfortunately this man page is
> rendered with no anchors over variable/features, so you could do
> better ^__^

the output of neither

  # man2html -r make.conf.5 > make.conf.5.html

nor

  # groff -man -Thtml make.conf.5 > make.conf.5.html

make me really happy

do you know any alternative coming with more anchors and external CSS
support?


> those percentages are just funny:
> 
> 33 (275.0 %)58 (483.3 %)  25 (208.3 %)
> 
> Total 12 (100.0 %)  [[ yeah, total is 100%, sometimes ]]
> 
> kde 3 (25.0 %) others 554 (4616.67 %) Total 1066 (8883.33 %)

most of these numbers show how much of something users have on average.
i agree this is not "user friendly".  Task now over here:
http://soc.gentooexperimental.org/issues/show/54



sebastian

Re: [gentoo-dev] Stats test server running, please check it out

[Sebastian Pipping]

Federico Ferri wrote:
> 1st: you could make an ebuild for it ;)

Done.  Look for  app-admin/gentoo-smolt-  in the "sping" overlay.

Running

  # smoltSendProfile --server=http://smolt.hartwork.org:45678/

should work fine after.



Sebastian

Re: [gentoo-dev] Stats test server running, please check it out

[Sebastian Pipping]

Federico Ferri wrote:
> 2nd: how to improve the output: you could make every data an
> hyperlink: that would help understand better the contents
> example:
> Archs:
>   hyperlink arch name to wikipedia perhaps

Done, linking to

  http://packages.gentoo.org/arch/${arch}


> CFLAGS
>   can link the flag name to GCC user guide relevant section, or to
> http://en.gentoo-wiki.com/wiki/CFLAGS#
> same for other flags and MAKEOPTS

Later, server seems down.


> FEATURES
>   can link feature to make.conf man page:
> http://linuxreviews.org/man/make.conf/ (unfortunately this man page is
> rendered with no anchors over variable/features, so you could do
> better ^__^

Done,  linking to
http://smolt.hartwork.org:45678/static/man/man5/make.conf.5.html#${feature}

If anyone feels like playing with  manpage-screen.css  please do and
come back to me.

Also, the man page does not seem to list these features:

  cvs
  preserve-libs
  split-debug
  splitdebug
  unmerge-logs

Either people use invalid features (you tell me) or the man page needs
an update.  Maybe both ("split-debug" vs "splitdebug")?


> USE flags
>   can link to
> http://gentoo-portage.com/Search?search=&use= (now
> appears down)

Still down, linking to

  http://gpo.zugaina.org/Search?search=&use=${use}

for now.


> System profiles:
>   can link to profiles in cvs

Are you sure about this?  Will that be fun to look at?


> Package atoms can link to packages.g.o or to gentoo-portage.com

Done, linking to

  http://packages.gentoo.org/package/${CATEGORY}/${PN}


To be able to link all packages properly we'd need a version of
http://gpo.zugaina.org/  that includes the trees "gentoo" and "funtoo".

ycarus, could the server handle that load-wise?  would it be easy to
extend the current service or set up an all-packages-version in parallel?



Sebastian