Re: [sage-devel] Changes to bitset.pxi

2020-09-24 Thread David Coudert
Most of the bitsets used in the graph backend are to record sets of seen or active vertices. These sets are not sparse. But I think we have several places in the graph module where we use sets instead of bitsets for small number of vertices. Sparse bitsets could be useful in this case if effecti

Re: [sage-devel] Changes to bitset.pxi

2020-09-22 Thread 'jonatha...@googlemail.com' via sage-devel
Thanks. I still can only guess, if people care about large bitsets like this. I can try to make it available with the same syntax and then however likes it, can just use it. Sébastien Labbé schrieb am Dienstag, 22. September 2020 um 16:07:15 UTC+2: > Am I the only one using larger bitsets? >>

Re: [sage-devel] Changes to bitset.pxi

2020-09-22 Thread Sébastien Labbé
> Am I the only one using larger bitsets? > Doing the following from SAGE_ROOT: git grep --name-only bitset shows that modules in sage using bitset are mostly graphs but also matroid, groups, combinat, data_structures, but also crypto, coding, misc and quivers. I don't know if those usage ar

Re: [sage-devel] Changes to bitset.pxi

2020-09-22 Thread 'jonatha...@googlemail.com' via sage-devel
Actually roaring performs pretty well, I just used an awful test case. Roaring bitmaps has container size 64k, so it performs awful for anything far below that. My test case had size 768. Roaring is interesting however as a bitset implementation for anything beyond 64k and seems to perform muc

Re: [sage-devel] Changes to bitset.pxi

2020-09-18 Thread 'jonatha...@googlemail.com' via sage-devel
To sum up this "discussion": There are no general objections to the suggested changes. I should however move away from `pxi` files. In particular it is fine to overalign bitsets at some point, which will make realloc less efficient (?)! At some point it would be nice to use some external librar

Re: [sage-devel] Changes to bitset.pxi

2020-09-17 Thread 'jonatha...@googlemail.com' via sage-devel
Ok. I don't think CRoaring serves well for my purpose. I'm getting multiples of my benchmarks (something like a factor of 10). Things are ok for small instances, but it seems that the bookkeeping explodes. I thought about it this afternoon and I don't think anything optimized for memory would w

Re: [sage-devel] Changes to bitset.pxi

2020-09-17 Thread 'jonatha...@googlemail.com' via sage-devel
CRoaring definitely uses advanced CPU-instructions. It might not be used everywhere possible (e.g. I don't see a distinction between AVX and AVX2, which means that in one of those cases something is missing). I got a prompt reply regarding my pull request (that the pull request needs work). So

Re: [sage-devel] Changes to bitset.pxi

2020-09-17 Thread Vincent Delecroix
About 2: the C library CRoaring seems a good match but does not seem to contain any assembler or advanced CPU instructions. Does it? It is hard to figure out how well is implemented and maintained the Python interface PyRoaringBitMap. Le 16/09/2020 à 17:18, 'jonatha...@googlemail.com' via sage-de

Re: [sage-devel] Changes to bitset.pxi

2020-09-16 Thread 'jonatha...@googlemail.com' via sage-devel
Dear Vincent, thanks for your reply. To sum up my reply: Should I try to replace bitsets completely by RaoringBitmap and see how it performs? (I need some help with the benchmarking, as I don't know good benchmarks for other use cases.) to 1. Okay. I wasn't aware of that. I just copied the beh

Re: [sage-devel] Changes to bitset.pxi

2020-09-16 Thread Vincent Delecroix
Dear Jonathan, 0. It would be good to benefit from extended CPU instructions for having faster bitsets. 1. As mentioned in the Cython documentation [1], pxi files are not the advised way to deal with Cython code. inline functions are perfectly usable when written in pxd headers. See for example