Moving to tech.
In gmane.os.openbsd.misc, you wrote:
> Hi all,
>
> I am getting a stacktrace from the borg command in the borgbackup
> package while checking a backup (see bottom of email for full
> output, since it's verbose). The relevant part is this:
>
> filenames = os.listdir(os.path.join(data_path, dir))
> OSError: [Errno 22] Invalid argument:
> '/mnt/thinkpad_void_obsd_borg/thinkpad.borg/data/12'
>
> This is same error is reproducible with a test Python 3.8 program:
>
> #!/usr/bin/env python
>
> import os
> os.listdir('/mnt/thinkpad_void_obsd_borg/thinkpad.borg/data/12/')
>
> Running ktrace & kdump reveals the error is from calling
> getdents(2):
>
> 76903 python3.8
> CALL open(0x1ec7f06de3b0,0x30000<O_RDONLY|O_CLOEXEC|O_DIRECTORY>)
> 76903 python3.8
> NAMI "/mnt/thinkpad_void_obsd_borg/thinkpad.borg/data/12/"
> 76903 python3.8 RET open 3
> [...]
> 76903 python3.8 CALL getdents(3,0x1ec7c9257000,0x4000)
> 76903 python3.8 RET getdents 16384/0x4000
> [...]
> 76903 python3.8 CALL getdents(3,0x1ec7c9257000,0x4000)
> 76903 python3.8 RET getdents 16384/0x4000
> [...]
> 76903 python3.8 CALL getdents(3,0x1ec7c9257000,0x4000)
> 76903 python3.8 RET getdents 16384/0x4000
> [...]
> 76903 python3.8 CALL getdents(3,0x1ec7c9257000,0x4000)
> 76903 python3.8 RET getdents -1 errno 22 Invalid argument
>
> Looking at the man page for getdents(2), I found it interesting
> that it says this call "is not a portable interface and should not
> be used directly by applications" and it recommends using
> readdir(3) instead.
ktrace only shows system calls not library functions. I don't
see python calling getdents directly, there is a fair chance that
python is calling readdir, and readdir is calling getdents.
> To give you a rough idea of the number of files and filename sizes
> in this directory:
>
> $ ls /mnt/thinkpad_void_obsd_borg/thinkpad.borg/data/12/ | wc
> 1534 1534 10738
I haven't been able to recreate this including over nfs yet.
What filesystem type and mount options?
getdents(2) :-
format. Up to nbytes of data will be transferred. nbytes must be greater
than or equal to the block size associated with the file (see stat(2)).
Some filesystems may not support getdents() with buffers smaller than this
size.
...
getdents() will fail if:
...
[EINVAL] The file referenced by fd is not a directory, or nbytes
is too small for returning a directory entry or block of
entries, or the current position pointer is invalid.
> Where does the problem lie -- the upstream Python code, the
> OpenBSD-specific patches in its port definition, or somewhere
> else? And in case it matters, this is a -current amd64 system,
> with "sysupgrade -s" executed on 7/15.
>
> Thank you,
> Aaron Miller
>
> --
> Exception ignored in: <function Repository.__del__ at
> 0x1e17e13fd310>
> Traceback (most recent call last):
> File "/usr/local/lib/python3.8/site-
> packages/borg/repository.py", line 180, in __del__
> assert False, "cleanup happened in Repository.__del__"
> AssertionError: cleanup happened in Repository.__del__
> Local Exception
> Traceback (most recent call last):
> File "/usr/local/lib/python3.8/site-packages/borg/archiver.py",
> line 4565, in main
> exit_code = archiver.run(args)
> File "/usr/local/lib/python3.8/site-packages/borg/archiver.py",
> line 4497, in run
> return set_ec(func(args))
> File "/usr/local/lib/python3.8/site-packages/borg/archiver.py",
> line 161, in wrapper
> with repository:
> File "/usr/local/lib/python3.8/site-
> packages/borg/repository.py", line 190, in __enter__
> self.open(self.path, bool(self.exclusive),
> lock_wait=self.lock_wait, lock=self.do_lock)
> File "/usr/local/lib/python3.8/site-
> packages/borg/repository.py", line 450, in open
> segment = self.io.get_latest_segment()
> File "/usr/local/lib/python3.8/site-
> packages/borg/repository.py", line 1253, in get_latest_segment
> for segment, filename in self.segment_iterator(reverse=True):
> File "/usr/local/lib/python3.8/site-
> packages/borg/repository.py", line 1241, in segment_iterator
> filenames = os.listdir(os.path.join(data_path, dir))
> OSError: [Errno 22] Invalid argument:
> '/mnt/thinkpad_void_obsd_borg/thinkpad.borg/data/12'
>
> Platform: OpenBSD millipede.iforgotmy.name 6.7 GENERIC.MP#348
> amd64
> Borg: 1.1.13 Python: CPython 3.8.3 msgpack: 0.5.6
> PID: 31745 CWD: /mnt/thinkpad_void_obsd_borg
> sys.argv: ['/usr/local/bin/borg', 'check', 'thinkpad.borg']
> SSH_ORIGINAL_COMMAND: None
>
>
