> > > > here's the crash now:
> > > >
> > > > Traceback (most recent call last):
> > > > File
> > > > "/nix/store/f2krmq3iv5nibcvn4rw7nrnrciqprdkh-python3-3.12.9/lib/python3.12/pdb.py",
> > > > line 1960, in main
> > > > pdb._run(target)
> > > > File
> > > > "/nix/store/f2krmq3iv5nibcvn4rw7nrnrciqprdkh-python3-3.12.9/lib/python3.12/pdb.py",
> > > > line 1754, in _run
> > > > self.run(target.code)
> > > > File
> > > > "/nix/store/f2krmq3iv5nibcvn4rw7nrnrciqprdkh-python3-3.12.9/lib/python3.12/bdb.py",
> > > > line 627, in run
> > > > exec(cmd, globals, locals)
> > > > File "/home/karl3/projects/rep/rep/dict.py", line 394, in <module>
> > > > doc.update([[val,val]])
> > > > File "/home/karl3/projects/rep/rep/dict.py", line 349, in update
> > > > super().update(keyhashitems())
> > > > File "/home/karl3/projects/rep/rep/dict.py", line 164, in update
> > > > assert int.from_bytes(keyhash[:hashbytes], 'big') >> hashshift ==
> > > > newidx
> > > >
> > > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > > AssertionError
> > > >
> > > >
> > > >
> > > > for keyhash, item in keyhashitems:
> > > >
> > > > assert item
> > > > != self._sentinel
> > > >
> > > > byteidx =
> > > > int.from_bytes(keyhash[:hashbytes], 'big')
> > > >
> > > > newidx = byteidx >> hashshift
> > > >
> > > >
> > > > if self._capacity > 0:
> > > >
> > > > # this
> > > > block checks for collision with previous stored values
> > > >
> > > > if capacity > self._capacity:
> > > >
> > > >
> > > > superidx = int.from_bytes(keyhash[:self._hashbytes],
> > > > 'big') >> self._hashshift
> > > > else:
> > > >
> > > >
> > > > superidx = newidx
> > > >
> > > >
> > > > place = self.array[superidx]
> > > >
> > > > if place !=
> > > > self._sentinel:
> > > >
> > > > collision = self._key(place)
> > > >
> > > >
> > > > if collision != keyhash:
> > > >
> > > > assert
> > > > superidx == int.from_bytes(collision[:self._hashbytes], 'big') >>
> > > > self._hashshift
> > > > updates[newidx] =
> > > > [collision, place, False]
> > > >
> > > > # this separated approach to checking for collisions
> > > > allows for accepting
> > > > #
> > > > batched data that ends up containing hash collisions solely within
> > > > itself
> > > > placing =
> > > > updates.get(newidx)
> > > > if placing is not None:
> > > > collision, place, is_new = placing
> > > > while newidx == int.from_bytes(collision[:hashbytes],
> > > > 'big') >> hashshift:
> > > > capacity <<= 1
> > > > expansion <<= 1
> > > > #spread += 1
> > > > #hashbits = self._hashbits + spread
> > > > hashbits += 1
> > > > hashbytes = (hashbits+7) >> 3
> > > > hashshift = (hashbytes << 3) - hashbits
> > > > byteidx = int.from_bytes(keyhash[:hashbytes], 'big')
> > > > newidx = byteidx >> hashshift
> > > > assert capacity == (1 << hashbits)
> > > > new_updates = {}
> > > > for keyhash, item, is_new in updates.values():
> > > > if is_new:
> > > > newnewidx =
> > > > int.from_bytes(keyhash[:hashbytes], 'big') >> hashshift
> > > > assert newnewidx not in new_updates
> > > > new_updates[newnewidx] = [keyhash, item, True]
> > > >
> > > > updates
> > > > = new_updates
> > > > assert newidx not in updates
> > > > assert int.from_bytes(keyhash[:hashbytes], 'big') >>
> > > > hashshift == newidx
> > > > updates[newidx] = [keyhash, item, True]
> > > >
> > > >
> > > > it looks like the problem is that keyhash is shadowed in the loop at
> > > > the end :D
> >
> > shadowing is when a variable is used in an inner scope with the same
> > name as an outer scope. many languages have strictly scoped blocks but
> > python is not one of those languages allowing occasional mistakes like
> > this. keyhash is used in an outer scope. i then use it as a loop
> > variable, which changes its value in the outer scope.
>
> gotta acknowledge here that the source paste is illegible because of
> all the wrapping and whitespace !
>
> also, how is this like use of the word "shadow" in corruption? the
> people causing the corruption are (a) _hidden_ like the old value of
> the variable, where people see the one that is presented rather than
> the one acted and (b) cause _wrong events_ like the new value of the
> variable overridding its correct one
it's because shadowed objects aren't lit because light is cast on the
object shadowing them instead
and then they are less noticed from something else receiving the light
cast on them, and so people don't include them and anything they do in
their decisions as much :/
>
> >
> > usually a shadowed variable would make the opposite crash -- where in
> > the inner loop, the inner value is used as if it were the outer. here,
> > since it's python, the crash is from the outer value being used after
> > the inner loop has finished. [...
> >
> > >
> > > now i've got this:
> > > File "/home/karl3/projects/rep/rep/dict.py", line 220, in
> > > content_generator
> > > assert superidx * expansion + subidx ==
> > > int.from_bytes(dbg_keyhash[:hashbytes], 'big') >> hashshift
> > >
> > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > AssertionError
> > > Uncaught exception. Entering post mortem debugging
> > > Running 'cont' or 'step' will restart the program
> > > > /home/karl3/projects/rep/rep/dict.py(220)content_generator()
> > > -> assert superidx * expansion + subidx ==
> > > int.from_bytes(dbg_keyhash[:hashbytes], 'big') >> hashshift
> > > (Pdb) p item
> > > b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
> > >
> > > but it's just because item is a sentinel which shouldn't hash to its
> > > index, can check for that
