Hi,
Guile-Xapian@0.4.0 is out! That’s a fun opportunity for revisiting
“guix search”. Well, “guix search” is currently an alias of “guix
package --search” and it can be considered as “slow”. The main reason
is from the lack of a good cache.
Although it is not a fair comparison, “guix package --list-available” is
much faster than “guix package --search”. Obviously, they do not
process and format the same content. Well, the point is that
’list-available’ relies on a cache, namely
~/.config/guix/current/lib/guix/package.cache.
It had been discussed to expand this cache (or create another one) for
helping “guix search”. Instead of walking through Guile modules, just
read a cache. However, this implies a slower “guix pull” – and “guix
time-machine” too –; the penalty depends on your machine and from my
opinion, I can wait some seconds more after waiting many minutes, but
that had not been the consensus.
All that said, there is a very long thread where different approaches
had been tried; see all the subthreads. ;-)
1. Rely on SQL [1]
2. Rely on Xapian [2]
3. Extend package.cache [3]
4. Improving string search algorithm [4]
5. Rely on package.cache and load the required Guile [5]
On the top of that, it is not handy to find which Guix revision contains
which package version.
For a description of the problem [6]. For instance, I do stuff as:
$ git -C ~/src/guix/guix log --format="%h %s" | grep 'gnu: bowtie:'
a47a90b900 gnu: bowtie: Remove reference to %outputs.
f336cc4fe7 gnu: bowtie: Replace invalid characters.
e5a26a1f02 gnu: bowtie: Remove trailing #T.
2ec601580b gnu: bowtie: Use TBB 2020.
21c837405a gnu: bowtie: Update to 2.3.4.3.
06e372360e gnu: bowtie: Use 'modify-phases'.
d6e63cf31c gnu: bowtie: Update to 2.3.2.
2642231b39 gnu: bowtie: Update to 2.2.9.
0047d26a22 gnu: bowtie: Update to 2.2.6.
241e122193 gnu: bowtie: fix build errors
which is not super handy. Well, it was somehow an idea behind this
Outreachy internship [7]. It’s another direction: exploit the format of
Guix commit messages. Anyway.
Therefore, all in all, I think it’s time to have a Guix extension that
improves the situation on both sides:
a) faster search
b) search inside all the history
And I think that Xapian could offer a very great framework. Well, I
have started an embryo months (years?) ago and it is still an embryo.
This issue deserve more love. :-)
Adding Xapian as Guix dependencies is too much. But it is possible to
extend Guix with GUIX_EXTENSIONS_PATH. This appears to me the best
solution here: write an extension relying on Xapian for indexing.
Still reading? I think this fit a very good topic if you are looking
for something to work on. Maybe it could be turned on some GSoC – hum I
am a bit lost with all the deadlines.
Feel free to ping. :-)
Cheers,
simon
1: [bug#39258] Faster guix search using an sqlite cache
Arun Isaac <arunis...@systemreboot.net>
Fri, 24 Jan 2020 01:21:57 +0530
id:cu7pnfaar36....@systemreboot.net
https://issues.guix.gnu.org/39258
https://issues.guix.gnu.org/msgid/cu7pnfaar36....@systemreboot.net
https://yhetil.org/guix/cu7pnfaar36....@systemreboot.net
2: [bug#39258] [PATCH v2 0/3] Xapian for Guix package search
Arun Isaac <arunis...@systemreboot.net>
Sat, 07 Mar 2020 19:01:13 +0530
id:20200307133116.11443-1-arunis...@systemreboot.net
https://issues.guix.gnu.org/39258
https://issues.guix.gnu.org/msgid/20200307133116.11443-1-arunis...@systemreboot.net
https://yhetil.org/guix/20200307133116.11443-1-arunis...@systemreboot.net
3: [bug#39258] [PATCH v4 0/3] Faster cache generation (similar as v3)
zimoun <zimon.touto...@gmail.com>
Sun, 03 May 2020 17:01:51 +0200
id:20200503150154.26532-1-zimon.touto...@gmail.com
https://issues.guix.gnu.org/39258
https://issues.guix.gnu.org/msgid/20200503150154.26532-1-zimon.touto...@gmail.com
https://yhetil.org/guix/20200503150154.26532-1-zimon.touto...@gmail.com
4: [bug#39258] KMP string search algorithm?
zimoun <zimon.touto...@gmail.com>
Mon, 01 Jun 2020 12:11:52 +0200
id:CAJ3okZ2ZNYaH=_ojp2axvq-any6q9ckvmjuvlpx-z6+kztv...@mail.gmail.com
https://issues.guix.gnu.org/39258
https://issues.guix.gnu.org/msgid/CAJ3okZ2ZNYaH=_ojp2axvq-any6q9ckvmjuvlpx-z6+kztv...@mail.gmail.com
https://yhetil.org/guix/CAJ3okZ2ZNYaH=_ojp2axvq-any6q9ckvmjuvlpx-z6+kztv...@mail.gmail.com
5: [bug#39258] [PATCH v6 0/2] DRAFT "guix search" performances
zimoun <zimon.touto...@gmail.com>
Thu, 15 Jul 2021 09:33:26 +0200
id:20210715073328.212123-1-zimon.touto...@gmail.com
https://issues.guix.gnu.org/39258
https://issues.guix.gnu.org/msgid/20210715073328.212123-1-zimon.touto...@gmail.com
https://yhetil.org/guix/20210715073328.212123-1-zimon.touto...@gmail.com
6: Re: List available versions of package.
Philippe Veber <philippe.ve...@gmail.com>
Tue, 11 Jun 2019 09:43:08 +0200
id:CAOOOohSzUezKvm=ro0bxrgh3m0eo2x0cotvd--varxwoqtc...@mail.gmail.com
https://lists.gnu.org/archive/html/help-guix/2019-06
https://yhetil.org/guix/CAOOOohSzUezKvm=ro0bxrgh3m0eo2x0cotvd--varxwoqtc...@mail.gmail.com
7:
https://guix.gnu.org/en/blog/2021/outreachy-guix-git-log-internship-wrap-up/
8: Re: Mechanism for helping in multi-channels configuration (and Xapian index)
Simon Tournier <zimon.touto...@gmail.com>
Mon, 06 May 2024 14:05:50 +0200
id:87pltzp2ld....@gmail.com
https://lists.gnu.org/archive/html/guix-devel/2024-05
https://yhetil.org/guix/87pltzp2ld....@gmail.com
