On Tue, 21 Jul 2020 at 06:01, Inada Naoki wrote:
> On Tue, Jul 21, 2020 at 5:07 AM Marco Sulla wrote:
> >
> > I just finished to improve the performance of frozendict creation. The
> result is very promising.
> >
> > The speedup is about 30% for small dicts (8 items). For large dicts (1k
> items
On Tue, Jul 21, 2020 at 5:07 AM Marco Sulla wrote:
>
> I just finished to improve the performance of frozendict creation. The result
> is very promising.
>
> The speedup is about 30% for small dicts (8 items). For large dicts (1k
> items) is about 38% for dicts with only integers as keys and val
I just finished to improve the performance of frozendict creation. The
result is very promising.
The speedup is about 30% for small dicts (8 items). For large dicts (1k
items) is about 38% for dicts with only integers as keys and values, and
45% for dicts with only strings.
There's also a little
On Sun, Jul 19, 2020 at 6:38 AM Marco Sulla
wrote:
>
> On Sat, 18 Jul 2020 at 10:02, Inada Naoki wrote:
>>
>> On Sat, Jul 18, 2020 at 7:05 AM Marco Sulla
>> wrote:
>> > For what I know, CPython uses PyDictObject for kwargs. Since dicts are
>> > mutable, it's a problem to cache them properly.
>>
On Sat, 18 Jul 2020 at 10:02, Inada Naoki wrote:
> On Sat, Jul 18, 2020 at 7:05 AM Marco Sulla
> wrote:
> > For what I know, CPython uses PyDictObject for kwargs. Since dicts are
> > mutable, it's a problem to cache them properly.
>
> On caller side, Python doesn't use dict at all.
> On callee s
On Sat, Jul 18, 2020 at 7:05 AM Marco Sulla
wrote:
>
> >
> > But when frozendicts are merged?
> > I think there is a very little chance.
>
> frozendicts could be used for kwargs:
>
> f(a=1, b=2)
> # some code
> f(a=1, b=2)
>
> For what I know, CPython uses PyDictObject for kwargs. Since dicts are
On Fri, 17 Jul 2020 at 04:13, Inada Naoki wrote:
> > 3. many python internals uses a mapping proxy to a dict, to avoid its
> > modification. A frozendict can be used instead.
>
> Are they used frequently in performance critical path?
> Could you point some concrete examples?
I searched a little i
On Fri, Jul 17, 2020 at 2:16 AM Marco Sulla
wrote:
>
> On Thu, 16 Jul 2020 at 06:11, Inada Naoki wrote:
> > On Thu, Jul 16, 2020 at 2:32 AM Marco Sulla
> > wrote:
> > > Yes, but, instead of creating a view, you can create and cache the
> > > pointer of a "real" object, that implements the dict v
On Thu, 16 Jul 2020 at 06:11, Inada Naoki wrote:
> On Thu, Jul 16, 2020 at 2:32 AM Marco Sulla
> wrote:
> > Yes, but, instead of creating a view, you can create and cache the
> > pointer of a "real" object, that implements the dict view API.
> > For example, keys for a frozendict could be an "ord
On Wed, 15 Jul 2020 at 08:07, Inada Naoki wrote:
> I don't think so. The view objects are useful when we need a set-like
> operation. (e.g. `assert d.keys() == {"spam", "egg"}`)
Yes, but, instead of creating a view, you can create and cache the
pointer of a "real" object, that implements the dic
On Thu, Jul 16, 2020 at 2:32 AM Marco Sulla
wrote:
>
> On Wed, 15 Jul 2020 at 08:07, Inada Naoki wrote:
> > I don't think so. The view objects are useful when we need a set-like
> > operation. (e.g. `assert d.keys() == {"spam", "egg"}`)
>
> Yes, but, instead of creating a view, you can create an
On Wed, Jul 15, 2020 at 2:01 AM Marco Sulla
wrote:
>
> > Why do you think I do not need views to use the frozendict?
> > I thought that is what make d.key(), d.items() etc work?
>
> Views for dict exist because dict is mutable. See this example:
>
> >>> d = {1: 2}
> >>> keys = d.keys()
> >>> d[2]
On Mon, 13 Jul 2020 at 19:28, Barry Scott wrote:
> > On 13 Jul 2020, at 03:20, Marco Sulla wrote:
> > So why did I try to implement it? IMO, apart the considerations in PEP
> > 416, a frozendict can be useful:
> >
> > - as a faster base for types.MutableMappingProxy
> > - as a substitute of named
> On 13 Jul 2020, at 03:20, Marco Sulla wrote:
>
> TL;DR: I tried to implement in CPython a frozendict here:
> https://github.com/Marco-Sulla/cpython
>
> Long explaining:
>
> What is a frozendict? It's an immutable dict. The type was proposed in
> the past but rejected: https://www.python.or
On 2/10/2012 9:52 PM, 8 Dihedral wrote:
在 2012年2月11日星期六UTC+8上午2时57分34秒，John Nagle写道：
On 2/10/2012 10:14 AM, Nathan Rice wrote:
Lets also not forget that knowing an object is immutable lets you do a
lot of optimizations; it can be inlined, it is safe to convert to a
contiguous block of memor
在 2012年2月11日星期六UTC+8上午2时57分34秒，John Nagle写道：
> On 2/10/2012 10:14 AM, Nathan Rice wrote:
> >>> Lets also not forget that knowing an object is immutable lets you do a
> >>> lot of optimizations; it can be inlined, it is safe to convert to a
> >>> contiguous block of memory and stuff in cache, etc.
On 2/10/2012 10:14 AM, Nathan Rice wrote:
Lets also not forget that knowing an object is immutable lets you do a
lot of optimizations; it can be inlined, it is safe to convert to a
contiguous block of memory and stuff in cache, etc. If you know the
input to a function is guaranteed to be frozen
>> Lets also not forget that knowing an object is immutable lets you do a
>> lot of optimizations; it can be inlined, it is safe to convert to a
>> contiguous block of memory and stuff in cache, etc. If you know the
>> input to a function is guaranteed to be frozen you can just go crazy.
>> Being
On Fri, Feb 10, 2012 at 8:53 AM, Nathan Rice
wrote:
> Lets also not forget that knowing an object is immutable lets you do a
> lot of optimizations; it can be inlined, it is safe to convert to a
> contiguous block of memory and stuff in cache, etc. If you know the
> input to a function is guaran
On Fri, Feb 10, 2012 at 5:08 AM, Chris Angelico wrote:
> On Fri, Feb 10, 2012 at 1:30 PM, Nathan Rice
> wrote:
>> The only thing needed to avoid the hash collision is that your hash
>> function is not not 100% predictable just by looking at the python
>> source code. I don't see why every dict w
On Fri, Feb 10, 2012 at 1:30 PM, Nathan Rice
wrote:
> The only thing needed to avoid the hash collision is that your hash
> function is not not 100% predictable just by looking at the python
> source code. I don't see why every dict would have to be created
> differently. I would think having th
> I've been trying for a few days (only a little bit at a time) to come up
> with a way of implementing a frozendict that doesn't suck. I'm gradually
> converging to a solution, but I can't help but think that there's some
> subtlety that I'm probably missing which is why it's not already provided.
On 2/9/2012 9:30 PM, Nathan Rice wrote:
That day may be sooner than you think. It is very likely that in Python
3.3, dict order will be randomized on creation as a side-effect of adding
a random salt to hashes to prevent a serious vulnerability in dicts.
http://securitytracker.com/id/1026478
h
On Thu, Feb 9, 2012 at 8:24 PM, Steven D'Aprano
wrote:
> On Thu, 09 Feb 2012 09:35:52 -0700, Ian Kelly wrote:
>
>> On Thu, Feb 9, 2012 at 8:19 AM, Nathan Rice
>> wrote:
>>> As I said, two dictionaries created from the same input will be the
>>> same...
>>
>> That's an implementation detail, not a
On Thu, 09 Feb 2012 09:35:52 -0700, Ian Kelly wrote:
> On Thu, Feb 9, 2012 at 8:19 AM, Nathan Rice
> wrote:
>> As I said, two dictionaries created from the same input will be the
>> same...
>
> That's an implementation detail, not a guarantee. It will hold for
> current versions of CPython but
Nathan Rice wrote:
> As I said, two dictionaries created from the same input will be the
> same... 'ai' != 'ia'. If I need to hash a dict that I don't know was
> created in a deterministic order, I'd frozenset(thedict.items()).
>
Fair enough, the idea scares me, but it's your code and your ris
On Thu, Feb 9, 2012 at 11:35 AM, Ian Kelly wrote:
> On Thu, Feb 9, 2012 at 8:19 AM, Nathan Rice
> wrote:
>> As I said, two dictionaries created from the same input will be the
>> same...
>
> That's an implementation detail, not a guarantee. It will hold for
> current versions of CPython but not
On Thu, Feb 9, 2012 at 8:19 AM, Nathan Rice
wrote:
> As I said, two dictionaries created from the same input will be the
> same...
That's an implementation detail, not a guarantee. It will hold for
current versions of CPython but not necessarily for other Python
implementations.
--
http://mail.
>> Two dicts created from the same inputs will return items in the same
>> arbitrary order. As long as you don't insert or delete a key you're
>> fine.
>>
> Two dicts that contain the same keys and values may or may not return them
> in the same order:
>
dict.fromkeys('ia')
> {'i': None, 'a':
Nathan Rice wrote:
> On Thu, Feb 9, 2012 at 5:33 AM, Duncan Booth
> wrote:
>> Nathan Rice wrote:
>>
>>> I put dicts in sets all the time. I just tuple the items, but that
>>> means you have to re-dict it on the way out to do anything useful
>>> with it. I am too lazy to write a frozendict or i
On Thu, Feb 9, 2012 at 5:33 AM, Duncan Booth
wrote:
> Nathan Rice wrote:
>
>> I put dicts in sets all the time. I just tuple the items, but that
>> means you have to re-dict it on the way out to do anything useful with
>> it. I am too lazy to write a frozendict or import one, but I would
>> use
Nathan Rice wrote:
> I put dicts in sets all the time. I just tuple the items, but that
> means you have to re-dict it on the way out to do anything useful with
> it. I am too lazy to write a frozendict or import one, but I would
> use it if it was a builtin.
>
I hope you sort the items before
On 13:59, Ian Kelly wrote:
>
> Define "doesn't suck". If I were to hack one up, it would look
> something like this:
>
>
> from collections import Mapping, Hashable
So part of my objection was that I wanted to make sure I got all of the
expected functionality, and that takes a bunch of functions.
On 13:59, Nathan Rice wrote:
>> Turn the question around: why should there be?
>> Python is intentionally parsimonious in adding builtins.
>
> For the same reason there are frozensets?
>
> I put dicts in sets all the time. I just tuple the items, but that
> means you have to re-dict it on the wa
> Turn the question around: why should there be?
> Python is intentionally parsimonious in adding builtins.
For the same reason there are frozensets?
I put dicts in sets all the time. I just tuple the items, but that
means you have to re-dict it on the way out to do anything useful with
it. I a
On Wed, Feb 8, 2012 at 6:23 PM, Evan Driscoll wrote:
> Hi all,
>
> I've been trying for a few days (only a little bit at a time) to come up
> with a way of implementing a frozendict that doesn't suck. I'm gradually
> converging to a solution, but I can't help but think that there's some
> subtlety
On 2/8/2012 8:23 PM, Evan Driscoll wrote:
Hi all,
I've been trying for a few days (only a little bit at a time) to come up
with a way of implementing a frozendict that doesn't suck. I'm gradually
converging to a solution, but I can't help but think that there's some
subtlety that I'm probably mi
On Wed, 08 Feb 2012 19:23:19 -0600, Evan Driscoll wrote:
> Hi all,
>
> I've been trying for a few days (only a little bit at a time) to come up
> with a way of implementing a frozendict that doesn't suck. I'm gradually
> converging to a solution, but I can't help but think that there's some
> sub
On 10/7/2010 2:39 PM, kj wrote:
Following a suggestion from MRAB, I attempted to implement a
frozendict class.
That really should be built into the language. "dict" is the
last built-in type that doesn't have an immutable form.
John Nagle
--
http://mail.pyth
kj wrote:
> In <878w29kxjp@gmail.com> Arnaud Delobelle writes:
>
> >E.g., try with {1:'a', 1j:'b'}
>
> I see. Thanks for this clarification. I learned a lot from it.
>
> I guess that frozenset must have some way of canonicalizing the
> order of its elements that is dependent on their Pytho
On Fri, 08 Oct 2010 14:00:17 +, kj wrote:
> In kj writes:
>
>>At any rate, using your [i.e. Arnaud's] suggestions in this and your
>>other post, the current implementation of frozendict stands at:
>
>>class frozendict(dict):
>>for method in ('__delitem__ __setitem__ clear pop popitem'
On Fri, 08 Oct 2010 12:10:50 +, kj wrote:
> In <4cae667c$0$29993$c3e8da3$54964...@news.astraweb.com> Steven D'Aprano
> writes:
>
>>On Fri, 08 Oct 2010 00:23:30 +, kj wrote:
>
>>Because it's always better to use a well-written, fast, efficient,
>>correct, well-tested wheel than to invent
In "Jonas H."
writes:
>Hope this helps :-)
It did! Thanks! For one thing now I see that I was barking up
the wrong tree in focusing on a canonical order, when, as the code
you posted shows, it is actually not required for hashing.
In fact, I'd come to the conclusion that frozensets had a c
In "Jonas H."
writes:
>On 10/08/2010 02:23 AM, kj wrote:
>Here's my implementation suggestion:
>class frozendict(dict):
> def _immutable_error(self, *args, **kwargs):
> raise TypeError("%r object is immutable" % self.__class__.__name__)
> __setitem__ = __delitem__ = clear = p
In kj writes:
>At any rate, using your [i.e. Arnaud's] suggestions in this and
>your other post, the current implementation of frozendict stands
>at:
>class frozendict(dict):
>for method in ('__delitem__ __setitem__ clear pop popitem setdefault '
> 'update').split():
>
On 10/08/2010 03:27 PM, kj wrote:
I tried to understand this by looking at the C source but I gave
up after 10 fruitless minutes. (This has been invariably the
outcome of all my attempts at finding my way through the Python C
source.)
It's not you. CPython's code is ... [censored]
Anyway, you
In <878w29kxjp@gmail.com> Arnaud Delobelle writes:
>E.g., try with {1:'a', 1j:'b'}
I see. Thanks for this clarification. I learned a lot from it.
I guess that frozenset must have some way of canonicalizing the
order of its elements that is dependent on their Python values but
not on their
In <4cae667c$0$29993$c3e8da3$54964...@news.astraweb.com> Steven D'Aprano
writes:
>On Fri, 08 Oct 2010 00:23:30 +, kj wrote:
>Because it's always better to use a well-written, fast, efficient,
>correct, well-tested wheel than to invent your own slow, incorrect
>wheel :)
IOW, "don't you wo
On 10/08/2010 02:23 AM, kj wrote:
I imagine that frozenset is better than sorted(tuple(...)) here,
but it's not obvious to me why.
dicts are unsorted. That means their item-order is undefined. So are sets.
If you want a hash that is independent from the order of items, you
could ensure the it
kj writes:
> In <87hbgxlk67@gmail.com> Arnaud Delobelle writes:
>
>>A simple fix is to use hash(frozenset(self.items())) instead.
>
> Thanks for pointing out the hash bug. It was an oversight: I meant
> to write
>
> def __hash__(self):
> return hash(sorted(tuple(self.items(
On Fri, 08 Oct 2010 00:23:30 +, kj wrote:
> In <87hbgxlk67@gmail.com> Arnaud Delobelle
> writes:
>
>>A simple fix is to use hash(frozenset(self.items())) instead.
>
> Thanks for pointing out the hash bug. It was an oversight: I meant to
> write
>
> def __hash__(self):
> re
In <87hbgxlk67@gmail.com> Arnaud Delobelle writes:
>A simple fix is to use hash(frozenset(self.items())) instead.
Thanks for pointing out the hash bug. It was an oversight: I meant
to write
def __hash__(self):
return hash(sorted(tuple(self.items(
I imagine that frozenset i
Oops I sent off my reply before I had finished!
kj writes:
> Following a suggestion from MRAB, I attempted to implement a
> frozendict class. My implementation took a lot more work than
> something this simple should take, and it still sucks. So I'm
> hoping someone can show me a better way.
kj writes:
> Following a suggestion from MRAB, I attempted to implement a
> frozendict class. My implementation took a lot more work than
> something this simple should take, and it still sucks. So I'm
> hoping someone can show me a better way. Specifically, I'm hoping
> that there is a "recip
54 matches
Mail list logo
