Christian wrote:
> Hi,
>
> I'm wondering why python blow up a dictionary structure so much.
>
> The ids and cat substructure could have 0..n entries but in the most cases
> they are <= 10,t is limited by <= 6.
>
> Thanks for any advice to save memory.
> Christian
>
>
> Example:
>
> {'0a0f7a3a0e09826caef1bff707785662': {'ids':
> {{'aa316b86-8169-11e6-bab9-0050563e2d7c',
> 'aa3174f0-8169-11e6-bab9-0050563e2d7c',
> 'aa319408-8169-11e6-bab9-0050563e2d7c',
> 'aa3195e8-8169-11e6-bab9-0050563e2d7c',
> 'aa319732-8169-11e6-bab9-0050563e2d7c',
> 'aa319868-8169-11e6-bab9-0050563e2d7c',
> 'aa31999e-8169-11e6-bab9-0050563e2d7c',
> 'aa319b06-8169-11e6-bab9-0050563e2d7c'},
> 't': {'type1', 'type2'},
> 'dt': datetime.datetime(2016, 9, 11, 15, 15, 54, 343000),
> 'nids': 8,
> 'ntypes': 2,
> 'cat': [('ABC', 'aa316b86-8169-11e6-bab9-0050563e2d7c', '74', ''),
> ('ABC','aa3174f0-8169-11e6-bab9-0050563e2d7c', '3', 'type1'),
> ('ABC','aa319408-8169-11e6-bab9-0050563e2d7c','3', 'type1'),
> ('ABC','aa3195e8-8169-11e6-bab9-0050563e2d7c', '3', 'type2'),
> ('ABC','aa319732-8169-11e6-bab9-0050563e2d7c', '3', 'type1'),
> ('ABC','aa319868-8169-11e6-bab9-0050563e2d7c', '3', 'type1'),
> ('ABC','aa31999e-8169-11e6-bab9-0050563e2d7c', '3', 'type1'),
> ('ABC','aa319b06-8169-11e6-bab9-0050563e2d7c', '3', 'type2')]},
Not so much to save memory, but because redundant data always bears the risk
to get out of sync:
For a value v in your dict, do
v["ids"] == {t[1] for t in v["cat"]}
len(v["ids"]) == len(v["cat"])
v["nids"] == len(v["ids"])
v["ntypes"] == len(v["t"])
v["t"] == {t[-1] for t in v["cat"]} - {""}
always hold?
And if you want to go fancy: are the IDs always 128-bit integers that share
all but the leading 32 bits?
--
https://mail.python.org/mailman/listinfo/python-list
