Re: Suggestion for Linux Distro (from PSA: Linux vulnerability)

2022-04-18 Thread Marco Sulla
On Sat, 16 Apr 2022 at 17:14, Peter J. Holzer wrote: > > On 2022-04-16 16:49:17 +0200, Marco Sulla wrote: > > Furthermore, you didn't answer my simple question: why does the > > security update package contain metadata about Debian patches, if the > > Ubuntu secur

tail

2022-04-23 Thread Marco Sulla
What about introducing a method for text streams that reads the lines from the bottom? Java has also a ReversedLinesFileReader with Apache Commons IO. -- https://mail.python.org/mailman/listinfo/python-list

Re: Receive a signal when waking or suspending?

2022-04-23 Thread Marco Sulla
I don't know in Python, but maybe you can create a script that writes on a named pipe and read it from Python? https://askubuntu.com/questions/226278/run-script-on-wakeup -- https://mail.python.org/mailman/listinfo/python-list

Re: tail

2022-04-23 Thread Marco Sulla
On Sat, 23 Apr 2022 at 20:59, Chris Angelico wrote: > > On Sun, 24 Apr 2022 at 04:37, Marco Sulla > wrote: > > > > What about introducing a method for text streams that reads the lines > > from the bottom? Java has also a ReversedLinesFileReader with Apac

Re: tail

2022-04-23 Thread Marco Sulla
n". When it find it, it stops and do a readline(): def tail(filepath): """ @author Marco Sulla @date May 31, 2016 """ try: filepath.is_file fp = str(filepath) except AttributeError: fp = filepath with open(fp, "

Re: tail

2022-04-24 Thread Marco Sulla
On Sat, 23 Apr 2022 at 23:18, Chris Angelico wrote: > Ah. Well, then, THAT is why it's inefficient: you're seeking back one > single byte at a time, then reading forwards. That is NOT going to > play nicely with file systems or buffers. > > Compare reading line by line over the file with readline

Re: tail

2022-04-24 Thread Marco Sulla
On Sun, 24 Apr 2022 at 00:19, Cameron Simpson wrote: > An approach I think you both may have missed: mmap the file and use > mmap.rfind(b'\n') to locate line delimiters. > https://docs.python.org/3/library/mmap.html#mmap.mmap.rfind > Ah, I played very little with mmap, I didn't know about this.

Re: tail

2022-04-24 Thread Marco Sulla
On Sun, 24 Apr 2022 at 11:21, Roel Schroeven wrote: > dn schreef op 24/04/2022 om 0:04: > > Disagreeing with @Chris in the sense that I use tail very frequently, > > and usually in the context of server logs - but I'm talking about the > > Linux implementation, not Python code! > If I understand

Re: tail

2022-05-01 Thread Marco Sulla
Something like this is OK? import os def tail(f): chunk_size = 100 size = os.stat(f.fileno()).st_size positions = iter(range(size, -1, -chunk_size)) next(positions) chunk_line_pos = -1 pos = 0 for pos in positions: f.seek(pos) chars = f.read(chunk_si

Re: new sorting algorithm

2022-05-01 Thread Marco Sulla
I suppose you should write to python-...@python.org , or in https://discuss.python.org/ under the section Core development -- https://mail.python.org/mailman/listinfo/python-list

Re: tail

2022-05-02 Thread Marco Sulla
On Mon, 2 May 2022 at 18:31, Stefan Ram wrote: > > |The Unicode standard defines a number of characters that > |conforming applications should recognize as line terminators:[7] > | > |LF:Line Feed, U+000A > |VT:Vertical Tab, U+000B > |FF:Form Feed, U+000C > |CR:Carriage Return, U+0

Re: tail

2022-05-02 Thread Marco Sulla
Ok, I suppose \n and \r are enough: readline(size=- 1, /) Read and return one line from the stream. If size is specified, at most size bytes will be read. The line terminator is always b'\n' for binary files; for text files, the newline argument to open() can be used to select the line

Re: tail

2022-05-02 Thread Marco Sulla
On Mon, 2 May 2022 at 00:20, Cameron Simpson wrote: > > On 01May2022 18:55, Marco Sulla wrote: > >Something like this is OK? > [...] > >def tail(f): > >chunk_size = 100 > >size = os.stat(f.fileno()).st_size > > I think you want

Re: tail

2022-05-06 Thread Marco Sulla
I have a little problem. I tried to extend the tail function, so it can read lines from the bottom of a file object opened in text mode. The problem is it does not work. It gets a starting position that is lower than the expected by 3 characters. So the first line is read only for 2 chars, and th

Re: tail

2022-05-07 Thread Marco Sulla
On Sat, 7 May 2022 at 01:03, Dennis Lee Bieber wrote: > > Windows also uses for the EOL marker, but Python's I/O system > condenses that to just internally (for TEXT mode) -- so using the > length of a string so read to compute a file position may be off-by-one for > each EOL in the stri

Re: tail

2022-05-07 Thread Marco Sulla
On Sat, 7 May 2022 at 16:08, Barry wrote: > You need to handle the file in bin mode and do the handling of line endings > and encodings yourself. It’s not that hard for the cases you wanted. >>> "\n".encode("utf-16") b'\xff\xfe\n\x00' >>> "".encode("utf-16") b'\xff\xfe' >>> "a\nb".encode("utf-16

Re: tail

2022-05-07 Thread Marco Sulla
On Sat, 7 May 2022 at 19:02, MRAB wrote: > > On 2022-05-07 17:28, Marco Sulla wrote: > > On Sat, 7 May 2022 at 16:08, Barry wrote: > >> You need to handle the file in bin mode and do the handling of line > >> endings and encodings yourself. It’s not that hard

Re: tail

2022-05-08 Thread Marco Sulla
I think I've _almost_ found a simpler, general way: import os _lf = "\n" _cr = "\r" def tail(filepath, n=10, newline=None, encoding=None, chunk_size=100): n_chunk_size = n * chunk_size pos = os.stat(filepath).st_size chunk_line_pos = -1 lines_not_found = n with open(filepath

Re: tail

2022-05-08 Thread Marco Sulla
On Sun, 8 May 2022 at 20:31, Barry Scott wrote: > > > On 8 May 2022, at 17:05, Marco Sulla wrote: > > > > def tail(filepath, n=10, newline=None, encoding=None, chunk_size=100): > >n_chunk_size = n * chunk_size > > Why use tiny chunks? You can read 4KiB as f

Re: tail

2022-05-08 Thread Marco Sulla
On Sun, 8 May 2022 at 22:02, Chris Angelico wrote: > > Absolutely not. As has been stated multiple times in this thread, a > fully general approach is extremely complicated, horrifically > unreliable, and hopelessly inefficient. Well, my implementation is quite general now. It's not complicated a

Re: tail

2022-05-08 Thread Marco Sulla
On Sun, 8 May 2022 at 22:34, Barry wrote: > > > On 8 May 2022, at 20:48, Marco Sulla wrote: > > > > On Sun, 8 May 2022 at 20:31, Barry Scott wrote: > >> > >>>> On 8 May 2022, at 17:05, Marco Sulla > >>>> wrote: > >>> &g

Re: tail

2022-05-09 Thread Marco Sulla
On Mon, 9 May 2022 at 07:56, Cameron Simpson wrote: > > The point here is that text is a very different thing. Because you > cannot seek to an absolute number of characters in an encoding with > variable sized characters. _If_ you did a seek to an arbitrary number > you can end up in the middle of

Re: tail

2022-05-09 Thread Marco Sulla
On Mon, 9 May 2022 at 19:53, Chris Angelico wrote: > > On Tue, 10 May 2022 at 03:47, Marco Sulla > wrote: > > > > On Mon, 9 May 2022 at 07:56, Cameron Simpson wrote: > > > > > > The point here is that text is a very different thing. Because you >

Re: tail

2022-05-11 Thread Marco Sulla
On Mon, 9 May 2022 at 23:15, Dennis Lee Bieber wrote: > > On Mon, 9 May 2022 21:11:23 +0200, Marco Sulla > declaimed the following: > > >Nevertheless, tail is a fundamental tool in *nix. It's fast and > >reliable. Also the tail command can't handle different

Re: tail

2022-05-11 Thread Marco Sulla
On Wed, 11 May 2022 at 22:09, Chris Angelico wrote: > > Have you actually checked those three, or do you merely suppose them to be > true? I only suppose, as I said. I should do some benchmark and some other tests, and, frankly, I don't want to. I don't want to because I'm quite sure the impleme

Re: tail

2022-05-12 Thread Marco Sulla
On Thu, 12 May 2022 at 00:50, Stefan Ram wrote: > > Marco Sulla writes: > >def tail(filepath, n=10, chunk_size=100): > >if (n <= 0): > >raise ValueError(_err_n) > ... > > There's no spec/doc, so one can't even test it. Excuse me, you&#

Re: tail

2022-05-12 Thread Marco Sulla
Thank you very much. This helped me to improve the function: import os _lf = b"\n" _err_n = "Parameter n must be a positive integer number" _err_chunk_size = "Parameter chunk_size must be a positive integer number" def tail(filepath, n=10, chunk_size=100): if (n <= 0): raise ValueErr

Re: tail

2022-05-13 Thread Marco Sulla
On Fri, 13 May 2022 at 00:31, Cameron Simpson wrote: > On 12May2022 19:48, Marco Sulla wrote: > >On Thu, 12 May 2022 at 00:50, Stefan Ram wrote: > >> There's no spec/doc, so one can't even test it. > > > >Excuse me, you're very right. > >

Re: tail

2022-05-16 Thread Marco Sulla
On Fri, 13 May 2022 at 12:49, <2qdxy4rzwzuui...@potatochowder.com> wrote: > > On 2022-05-13 at 12:16:57 +0200, > Marco Sulla wrote: > > > On Fri, 13 May 2022 at 00:31, Cameron Simpson wrote: > > [...] > > > > This is nearly the worst "specificat

Re: tail

2022-05-18 Thread Marco Sulla
Well, I've done a benchmark. >>> timeit.timeit("tail('/home/marco/small.txt')", globals={"tail":tail}, >>> number=10) 1.5963431186974049 >>> timeit.timeit("tail('/home/marco/lorem.txt')", globals={"tail":tail}, >>> number=10) 2.5240604374557734 >>> timeit.timeit("tail('/home/marco/lorem.

Re: tail

2022-05-19 Thread Marco Sulla
On Wed, 18 May 2022 at 23:32, Cameron Simpson wrote: > > On 17May2022 22:45, Marco Sulla wrote: > >Well, I've done a benchmark. > >>>> timeit.timeit("tail('/home/marco/small.txt')", globals={"tail":tail}, > >>>>

Re: Subtract n months from datetime

2022-06-22 Thread Marco Sulla
The package arrow has a simple shift method for months, weeks etc https://arrow.readthedocs.io/en/latest/#replace-shift -- https://mail.python.org/mailman/listinfo/python-list

Why I fail so bad to check for memory leak with this code?

2022-07-21 Thread Marco Sulla
I tried to check for memory leaks in a bunch of functions of mine using a simple decorator. It works, but it fails with this code, returning a random count_diff at every run. Why? import tracemalloc import gc import functools from uuid import uuid4 import pickle def getUuid(): return str(uuid

Re: Why I fail so bad to check for memory leak with this code?

2022-07-21 Thread Marco Sulla
On Thu, 21 Jul 2022 at 22:28, MRAB wrote: > > It's something to do with pickling iterators because it still occurs > when I reduce func_76 to: > > @trace > def func_76(): > pickle.dumps(iter([])) It's too strange. I found a bunch of true memory leaks with this decorator. It seems to be relia

Re: Why I fail so bad to check for memory leak with this code?

2022-07-21 Thread Marco Sulla
This naif code shows no leak: import resource import pickle c = 0 while True: pickle.dumps(iter([])) if (c % 1) == 0: max_rss = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss print(f"iteration: {c}, max rss: {max_rss} kb") c += 1 -- https://mail.python.org/

Re: Why I fail so bad to check for memory leak with this code?

2022-07-21 Thread Marco Sulla
I've done this other simple test: #!/usr/bin/env python3 import tracemalloc import gc import pickle tracemalloc.start() snapshot1 = tracemalloc.take_snapshot().filter_traces( (tracemalloc.Filter(True, __file__), ) ) for i in range(1000): pickle.dumps(iter([])) gc.collect() snapsh

Re: Why I fail so bad to check for memory leak with this code?

2022-07-22 Thread Marco Sulla
On Fri, 22 Jul 2022 at 09:00, Barry wrote: > With code as complex as python’s there will be memory allocations that occur that will not be directly related to the python code you test. > > To put it another way there is noise in your memory allocation signal. > > Usually the signal of a memory lea

How to generate a .pyi file for a C Extension using stubgen

2022-07-29 Thread Marco Sulla
I tried to follow the instructions here: https://mypy.readthedocs.io/en/stable/stubgen.html but the instructions about creating a stub for a C Extension are a little mysterious. I tried to use it on the .so file without luck. -- https://mail.python.org/mailman/listinfo/python-list

Re: How to generate a .pyi file for a C Extension using stubgen

2022-07-30 Thread Marco Sulla
On Fri, 29 Jul 2022 at 23:23, Barry wrote: > > > > > On 29 Jul 2022, at 19:33, Marco Sulla wrote: > > > > I tried to follow the instructions here: > > > > https://mypy.readthedocs.io/en/stable/stubgen.html > > > > but the instructions a

Am I banned from Discuss forum?

2023-02-10 Thread Marco Sulla
I was banned from the mailing list and Discuss forum for a very long time. Too much IMHO, but I paid my dues. Now this is my state in the forum: - I never posted something unrespectful in the last months - I have a limitation of three posts per threads, but only on some threads - Some random posts

Re: use set notation for repr of dict_keys?

2021-02-24 Thread Marco Sulla
On Wed, 24 Feb 2021 at 06:29, Random832 wrote: > I was surprised, though, to find that you can't remove items directly from > the key set, or in general update it in place with &= or -= (these operators > work, but give a new set object). This is because they are a view. Changing the key object

Re: use set notation for repr of dict_keys?

2021-02-24 Thread Marco Sulla
On Wed, 24 Feb 2021 at 15:02, Random832 wrote: > On Wed, Feb 24, 2021, at 02:59, Marco Sulla wrote: > > On Wed, 24 Feb 2021 at 06:29, Random832 wrote: > > > I was surprised, though, to find that you can't remove items directly > > > from the key set, or in gener

Re: editor recommendations?

2021-02-26 Thread Marco Sulla
I use Sublime free for simple tasks. I like the fact it's fast and it saves to disk immediately. You don't have even to name the file. I use it also for taking notes. Probably not as powerful as Vim and it's proprietary. For development, I use PyCharm, but it's an IDE. I also used in past: gedit:

Re: weirdness with list()

2021-02-28 Thread Marco Sulla
On Sun, 28 Feb 2021 at 01:19, Cameron Simpson wrote: > My object represents an MDAT box in an MP4 file: it is the ludicrously > large data box containing the raw audiovideo data; for a TV episode it > is often about 2GB and a movie is often 4GB to 6GB. > [...] > That length is presented via the ob

Why assert is not a function?

2021-03-02 Thread Marco Sulla
I have a curiosity. Python, as many languages, has assert as a keyword. Can't it be implemented as a function? Is there an advantage to have it as a keyword? -- https://mail.python.org/mailman/listinfo/python-list

Re: yield from () Was: Re: weirdness with list()

2021-03-02 Thread Marco Sulla
On Mon, 1 Mar 2021 at 19:51, Alan Gauld via Python-list wrote: > Sorry, a bit OT but I'm curious. I haven't seen > this before: > > yield from () > > What is it doing? > What do the () represent in this context? It's the empty tuple. -- https://mail.python.org/mailman/listinfo/python-list

How to create both a c extension and a pure python package

2021-03-09 Thread Marco Sulla
As title. Currently I ended up using this trick in my setup.py: if len(argv) > 1 and argv[1] == "c": sys.argv = [sys.argv[0]] + sys.argv[2:] setuptools.setup(ext_modules = ext_modules, **common_setup_args) else: setuptools.setup(**common_setup_args) So if I pass "c" as the first arg

Re: How to create both a c extension and a pure python package

2021-03-10 Thread Marco Sulla
On Wed, 10 Mar 2021 at 16:45, Thomas Jollans wrote: > Why are you doing this? > > If all you want is for it to be possible to install the package from > source on a system that can't use the C part, you could just declare > your extension modules optional Because I want to provide (at least) two

Re: Why assert is not a function?

2021-03-12 Thread Marco Sulla
On Thu, 11 Mar 2021 at 23:11, Ethan Furman wrote: > Basically, you are looking at two different philosophies: > > - Always double check, get good error message when something fails > > vs > > - check during testing and QA, turn off double-checks for production for best > performance possible. In

How to support annotations for a custom type in a C extension?

2021-09-17 Thread Marco Sulla
I created a custom dict in a C extension. Name it `promethea`. How can I implement `promethea[str, str]`? Now I get: TypeError: 'type' object is not subscriptable -- https://mail.python.org/mailman/listinfo/python-list

Re: How to support annotations for a custom type in a C extension?

2021-09-17 Thread Marco Sulla
Ooook. I have a question. Why is this code not present in dictobject.c? Where are the dict annotations implemented? On Sat, 18 Sept 2021 at 03:00, MRAB wrote: > > On 2021-09-17 21:03, Marco Sulla wrote: > > I created a custom dict in a C extension. Name it `promethea`. H

Python C API: how to mark a type as subclass of another type

2021-10-31 Thread Marco Sulla
I have two types declared as PyTypeObject PyX_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) etc. How can I mark one of the types as subclass of the other one? I tried to use tp_base but it didn't work. -- https://mail.python.org/mailman/listinfo/python-list

Re: Python C API: how to mark a type as subclass of another type

2021-11-02 Thread Marco Sulla
I already added the address of the type to tp_base, but it does not work. On Mon, 1 Nov 2021 at 17:18, Dieter Maurer wrote: > > Marco Sulla wrote at 2021-10-31 23:59 +0100: > >I have two types declared as > > > >PyTypeObject PyX_Type = { > >PyVarObject_HEAD_IN

Re: Python C API: how to mark a type as subclass of another type

2021-11-02 Thread Marco Sulla
*ahem* evidently I didn't check the right package. it works like a charme :D On Tue, 2 Nov 2021 at 13:43, Marco Sulla wrote: > > I already added the address of the type to tp_base, but it does not work. > > On Mon, 1 Nov 2021 at 17:18, Dieter Maurer wrote: > > > &

Py_IS_TYPE(op, &PyDict_Type) does not work on MacOS

2021-11-07 Thread Marco Sulla
As you can read here: https://github.com/Marco-Sulla/python-frozendict/issues/37 Py_IS_TYPE(op, &PyDict_Type) did not work on MacOS. I had to use PyDict_Check. Why don't I have this problem with Linux? PS: since I'm creating a modified version of dict, I copied the dict interna

Re: Py_IS_TYPE(op, &PyDict_Type) does not work on MacOS

2021-11-10 Thread Marco Sulla
Indeed now I use PyDict_Check, but anyway it's very strange that Py_IS_TYPE(op, &PyDict_Type) does not work only on MacOS. On Mon, 8 Nov 2021 at 19:30, Barry wrote: > > > > On 8 Nov 2021, at 07:45, Marco Sulla wrote: > > As you can read here: > > http

Unable to compile my C Extension on Windows: unresolved external link errors

2021-11-12 Thread Marco Sulla
"] undef_macros = [] setuptools.Extension( ext1_fullname, sources = cpython_sources, include_dirs = cpython_include_dirs, extra_compile_args = extra_compile_args, undef_macros = undef_macros, ) Here is the full code: https://github.com/Marco-Sulla/python-frozendict/blob/ma

Re: Unable to compile my C Extension on Windows: unresolved external link errors

2021-11-12 Thread Marco Sulla
On Fri, 12 Nov 2021 at 15:55, Gisle Vanem wrote: > Marco Sulla wrote: > > Error LNK2001: unresolved external symbol PyErr_SetObject > > > > and so on. > > > > I post the part of my setup.py about the C Extension: > > > > extra_compile_args = ["-DP

Re: Unable to compile my C Extension on Windows: unresolved external link errors

2021-11-12 Thread Marco Sulla
: https://github.com/Marco-Sulla/python-frozendict.git On Linux and MacOS it works like a charme. On Windows, it seems it does not find python3.lib. I also added its path to the PATH variable -- https://mail.python.org/mailman/listinfo/python-list

Re: Unable to compile my C Extension on Windows: unresolved external link errors

2021-11-12 Thread Marco Sulla
On Fri, 12 Nov 2021 at 21:09, Chris Angelico wrote: > > On Sat, Nov 13, 2021 at 7:01 AM Marco Sulla > wrote: > > On Fri, 12 Nov 2021 at 17:38, Chris Angelico wrote: > > > Are you sure that you really need Py_BUILD_CORE? > > > > Yes, because I need the intern

Re: Unable to compile my C Extension on Windows: unresolved external link errors

2021-11-13 Thread Marco Sulla
. Sorry, the problem is I downloaded the 32 bit version of VS compiler and 64 bit version of Python.. On Sat, 13 Nov 2021 at 11:10, Barry Scott wrote: > > > > > On 13 Nov 2021, at 09:00, Barry wrote: > > > > > > > >> On 12 Nov 2021, at 22:53, M

Re: Unable to compile my C Extension on Windows: unresolved external link errors

2021-11-14 Thread Marco Sulla
libraries expose the internal functions? Anyway, is there a way to compile Python on Windows in such a way that I get a shared library that exposes all the functions? On Sat, 13 Nov 2021 at 12:17, Marco Sulla wrote: > > . Sorry, the problem is I downloaded the 32 bit version of VS >

Re: Unable to compile my C Extension on Windows: unresolved external link errors

2021-11-14 Thread Marco Sulla
On Sun, 14 Nov 2021 at 16:42, Barry Scott wrote: > > Sorry iPad sent the message before it was complete... > > > On 14 Nov 2021, at 10:38, Marco Sulla wrote: > > > > Okay, now the problem seems to be another: I get the same "unresolved > > external link&qu

Re: How to support annotations for a custom type in a C extension?

2021-11-18 Thread Marco Sulla
It works. Thanks a lot. On Sun, 19 Sept 2021 at 19:23, Serhiy Storchaka wrote: > > 19.09.21 05:59, MRAB пише: > > On 2021-09-18 16:09, Serhiy Storchaka wrote: > >> "(PyCFunction)" is redundant, Py_GenericAlias already has the right > >> type. Overuse of casting to PyCFunction can hide actual bugs

pytest segfault, not with -v

2021-11-19 Thread Marco Sulla
I have a battery of tests done with pytest. My tests break with a segfault if I run them normally. If I run them using pytest -v, the segfault does not happen. What could cause this quantical phenomenon? -- https://mail.python.org/mailman/listinfo/python-list

Re: getting source code line of error?

2021-11-19 Thread Marco Sulla
Have you tried the logger module and the format options? On Fri, 19 Nov 2021 at 19:09, Ulli Horlacher wrote: > > I am trying to get the source code line of the last error. > I know traceback.format_exc() but this contains much more information, e.g.: > > Traceback (most recent call last): > Fil

frozenset can be altered by |=

2021-11-19 Thread Marco Sulla
(venv_3_10) marco@buzz:~$ python Python 3.10.0 (heads/3.10-dirty:f6e8b80d20, Nov 18 2021, 19:16:18) [GCC 10.1.1 20200718] on linux Type "help", "copyright", "credits" or "license" for more information. >>> a = frozenset((3, 4)) >>> a frozenset({3, 4}) >>> a |= {5,} >>> a frozenset({3, 4, 5}) -- ht

Re: frozenset can be altered by |=

2021-11-19 Thread Marco Sulla
Mh. Now I'm thinking that I've done a = "Marco " a += "Sulla" many times without bothering. On Fri, 19 Nov 2021 at 22:22, Chris Angelico wrote: > > On Sat, Nov 20, 2021 at 8:16 AM Chris Angelico wrote: > > > > On Sat, Nov 20, 2021 at 8:13 A

Re: pytest segfault, not with -v

2021-11-19 Thread Marco Sulla
On Fri, 19 Nov 2021 at 20:38, MRAB wrote: > > On 2021-11-19 17:48, Marco Sulla wrote: > > I have a battery of tests done with pytest. My tests break with a > > segfault if I run them normally. If I run them using pytest -v, the > > segfault does not happen. > > >

Re: pytest segfault, not with -v

2021-11-20 Thread Marco Sulla
imple loop for all the cases, and remove the optional parameter from bench. How boring. PS: is there a way to monitor the Python consumed memory inside Python itself? In this way I could also trap memory leaks. On Sat, 20 Nov 2021 at 01:46, MRAB wrote: > > On 2021-11-19 23:44, Marco Sulla wr

No right operator in tp_as_number?

2021-11-20 Thread Marco Sulla
I checked the documentation: https://docs.python.org/3/c-api/typeobj.html#number-structs and it seems that, in the Python C API, the right operators do not exist. For example, there is nb_add, that in Python is __add__, but there's no nb_right_add, that in Python is __radd__ Am I missing something

Re: pytest segfault, not with -v

2021-11-20 Thread Marco Sulla
e: > > On 2021-11-20 17:40, Marco Sulla wrote: > > Indeed I have introduced a command line parameter in my bench.py > > script that simply specifies the number of times the benchmarks are > > performed. This way I have a sort of segfault checker. > > > > But I

Re: frozenset can be altered by |=

2021-11-22 Thread Marco Sulla
Yes, and you do this regularly. Indeed integers, for example, are immutables and a = 0 a += 1 is something you do dozens of times, and you simply don't think that another object is created and substituted for the variable named `a`. On Mon, 22 Nov 2021 at 14:59, Chris Angelico wrote: > > On Tue

Re: frozenset can be altered by |=

2021-11-29 Thread Marco Sulla
bject. The same for += and *= that are listed under `list` only. On Mon, 22 Nov 2021 at 19:54, Marco Sulla wrote: > > Yes, and you do this regularly. Indeed integers, for example, are immutables > and > > a = 0 > a += 1 > > is something you do dozens of times, and you simp

Re: pytest segfault, not with -v

2021-12-18 Thread Marco Sulla
Ok, I created the script: https://github.com/Marco-Sulla/python-frozendict/blob/master/test/debug.py The problem is it does _not_ crash, while a get a segfault using pytest with python 3.9 on MacOS 10.15 Maybe it's because I'm using eval / exec in the script? On Sat, 20 Nov 202

Re: pytest segfault, not with -v

2021-12-18 Thread Marco Sulla
Emh, maybe I was not clear. I created a C extension and it segfaults. So I created that script to see where it segfaults. But the script does not segfault. My doubt is: is that because I'm using eval and exec in the script? On Sat, 18 Dec 2021 at 18:33, Dieter Maurer wrote: > > Marco

Py_TRASHCAN_SAFE_BEGIN/END in C extension?

2021-12-21 Thread Marco Sulla
In Python 3.7, must Py_TRASHCAN_SAFE_BEGIN - Py_TRASHCAN_SAFE_END be used in a C extension? I'm asking because in my C extension I use them in the deallocator without problems, but users signalled me that they segfault in Python 3.7 on Debian 10. I checked and this is true. -- https://mail.python

Re: Py_TRASHCAN_SAFE_BEGIN/END in C extension?

2021-12-22 Thread Marco Sulla
Yes, it's deprecated, but I need it for Python 3.7, since there was yet no Py_TRASHCAN_BEGIN / END On Tue, 21 Dec 2021 at 23:22, Barry wrote: > > > > On 21 Dec 2021, at 22:08, Marco Sulla wrote: > > In Python 3.7, must Py_TRASHCAN_SAFE_BEGIN - Py_TRASHCAN_SAFE_END be

Re: recover pickled data: pickle data was truncated

2021-12-26 Thread Marco Sulla
Use a semaphore. On Sun, 26 Dec 2021 at 03:30, iMath wrote: > > Normally, the shelve data should be read and write by only one process at a > time, but unfortunately it was simultaneously read and write by two > processes, thus corrupted it. Is there any way to recover all data in it ? > Curre

What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-26 Thread Marco Sulla
I have to use _PyObject_GC_IS_TRACKED(). It can't be used unless you define Py_BUILD_CORE. I want to avoid this. What macro or function can substitute it? -- https://mail.python.org/mailman/listinfo/python-list

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-27 Thread Marco Sulla
I need it since I'm developing an immutable dict. And in dict that function is used. I do not understand why there's no public API for that function. It seems very useful. On Sun, 26 Dec 2021 at 17:28, Barry Scott wrote: > > > > > On 26 Dec 2021, at 13:48, Marco Sulla

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-27 Thread Marco Sulla
Hi, Inada Senpai. So I do not need PyObject_GC_Track on cloning or merging, or MAINTAIN_TRACKING on insert? On Tue, 28 Dec 2021 at 07:58, Inada Naoki wrote: > > On Tue, Dec 28, 2021 at 3:31 AM Marco Sulla > wrote: > > > > I need it since I'm developing an immuta

Option for venv to upgrade pip automatically?

2021-12-28 Thread Marco Sulla
I think it's very boring that, after creating a venv, you have immediately to do every time: pip install -U pip Can't venv have an option for doing this automatically or, better, a config file where you can put commands that will be launched every time after you create a venv? -- https://mail.py

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-28 Thread Marco Sulla
On Tue, 28 Dec 2021 at 12:38, Inada Naoki wrote: > Your case is special. > You want to create a frozendict which performance is same to builtin dict. > Builtin dict has special optimization which tightly coupled with > current CPython implementation. > So you need to use private APIs for MAINTAIN_

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-28 Thread Marco Sulla
On Wed, 29 Dec 2021 at 00:03, Dieter Maurer wrote: > Why do you not derive from `dict` and override its mutating methods > (to raise a type error after initialization is complete)? I've done this for the pure py version, for speed. But in this way, frozendict results to be a subclass of MutableMa

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-28 Thread Marco Sulla
On Wed, 29 Dec 2021 at 07:46, Inada Naoki wrote: > You are right. I thought PyObject_GC_Track() can be used to tracked > objects because PyObject_GC_Untrack() can be used untracked object. > I think there is no enough reason for this asymmetry. > > Additionally, adding PyObject_GC_IsTracked() to p

Re: Option for venv to upgrade pip automatically?

2021-12-29 Thread Marco Sulla
Cool, thanks! On Wed, 29 Dec 2021 at 07:10, Inada Naoki wrote: > > You can use --upgrade-deps option. My alias is: > > alias mkvenv='python3 -m venv --upgrade-deps --prompt . venv' > > On Wed, Dec 29, 2021 at 4:55 AM Marco Sulla > wrote: > > > > I th

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-29 Thread Marco Sulla
On Wed, 29 Dec 2021 at 09:12, Dieter Maurer wrote: > > Marco Sulla wrote at 2021-12-29 08:08 +0100: > >On Wed, 29 Dec 2021 at 00:03, Dieter Maurer wrote: > >> Why do you not derive from `dict` and override its mutating methods > >> (to raise a type error af

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-29 Thread Marco Sulla
c 2021 at 09:24, Marco Sulla wrote: > > On Wed, 29 Dec 2021 at 09:12, Dieter Maurer wrote: > > > > Marco Sulla wrote at 2021-12-29 08:08 +0100: > > >On Wed, 29 Dec 2021 at 00:03, Dieter Maurer wrote: > > >> Why do you not derive from `dict` and override it

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-29 Thread Marco Sulla
On Wed, 29 Dec 2021 at 10:06, Dieter Maurer wrote: > > Are you sure you need to implement your type in C at all? It's already implemented, and, in some cases, is faster than dict: https://github.com/Marco-Sulla/python-frozendict#benchmarks PS: I'm doing a refactoring that sp

How to implement freelists in dict 3.10 for previous versions?

2021-12-29 Thread Marco Sulla
I noticed that now freelists in dict use _Py_dict_state. I suppose this is done for thread safety. I would implement it also for a C extension that uses CPython < 3.10. How can I achieve this? -- https://mail.python.org/mailman/listinfo/python-list

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-29 Thread Marco Sulla
On Wed, 29 Dec 2021 at 09:12, Dieter Maurer wrote: > `MutableMapping` is a so called abstract base class (--> `abc`). > > It uses the `__subclass_check__` (and `__instance_check__`) of > `abc.ABCMeta` to ensure `issubclass(dict, MutableMapping)`. > Those can be customized by overriding `MutableMap

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-29 Thread Marco Sulla
On Wed, 29 Dec 2021 at 12:11, Dieter Maurer wrote: > > Marco Sulla wrote at 2021-12-29 11:59 +0100: > >On Wed, 29 Dec 2021 at 09:12, Dieter Maurer wrote: > >> `MutableMapping` is a so called abstract base class (--> `abc`). > >> > >> It uses the `__subcl

Re: recover pickled data: pickle data was truncated

2021-12-29 Thread Marco Sulla
On Wed, 29 Dec 2021 at 18:33, iMath wrote: > But I found the size of the file of the shelve data didn't change much, so I > guess the data are still in it , I just wonder any way to recover my data. I agree with Barry, Chris and Avi. IMHO your data is lost. Unpickling it by hand is a harsh work

Re: builtins.TypeError: catching classes that do not inherit from BaseException is not allowed

2021-12-31 Thread Marco Sulla
It was already done: https://pypi.org/project/tail-recursive/ On Thu, 30 Dec 2021 at 16:00, hongy...@gmail.com wrote: > > I try to compute the factorial of a large number with tail-recursion > optimization decorator in Python3. The following code snippet is converted > from the code snippet giv

Re: recover pickled data: pickle data was truncated

2022-01-01 Thread Marco Sulla
I agree with Barry. You can create a folder or a file with pseudo-random names. I recommend you to use str(uuid.uuid4()) On Sat, 1 Jan 2022 at 14:11, Barry wrote: > > > > > On 31 Dec 2021, at 17:53, iMath wrote: > > > > 在 2021年12月30日星期四 UTC+8 03:13:21, 写道: > >>> On Wed, 29 Dec 2021 at 18:33, iM

How to make a type of a C extension compatible with mypy

2022-01-01 Thread Marco Sulla
I created a type in a C extension, that is an immutable dict. If I do: a: mydict[str, str] it works. But it doesn't work with mypy, as signalled to me by an user: https://github.com/Marco-Sulla/python-frozendict/issues/39 How can I make it work? I don't know what he means with

Re: How to implement freelists in dict 3.10 for previous versions?

2022-01-01 Thread Marco Sulla
Ooookay, I suppose I have to study a little the thing :D On Thu, 30 Dec 2021 at 07:59, Inada Naoki wrote: > > On Wed, Dec 29, 2021 at 7:25 PM Marco Sulla > wrote: > > > > I noticed that now freelists in dict use _Py_dict_state. I suppose > > this is done for thread sa

Who wrote Py_UNREACHABLE?

2022-01-02 Thread Marco Sulla
#if defined(RANDALL_WAS_HERE) # define Py_UNREACHABLE() \ Py_FatalError( \ "If you're seeing this, the code is in what I thought was\n" \ "an unreachable state.\n\n" \ "I could give you advice for what to do, but honestly, why\n" \ "should you trust me? I clear

Re: ModuleNotFoundError: No module named 'DistUtilsExtra'

2022-01-02 Thread Marco Sulla
https://askubuntu.com/questions/584857/distutilsextra-problem On Sun, 2 Jan 2022 at 18:52, hongy...@gmail.com wrote: > > On Ubuntu 20.04.3 LTS, I try to install pdfarranger [1] as follows but failed: > > $ sudo apt-get install python3-pip python3-distutils-extra \ >

  1   2   3   >