Re: new sorting algorithm

2022-05-01 Thread Marco Sulla
I suppose you should write to python-...@python.org , or in https://discuss.python.org/ under the section Core development -- https://mail.python.org/mailman/listinfo/python-list

Re: tail

2022-05-02 Thread Marco Sulla
On Mon, 2 May 2022 at 18:31, Stefan Ram wrote: > > |The Unicode standard defines a number of characters that > |conforming applications should recognize as line terminators:[7] > | > |LF:Line Feed, U+000A > |VT:Vertical Tab, U+000B > |FF:Form Feed, U+000C > |CR:Carriage Return, U+0

Re: tail

2022-05-02 Thread Marco Sulla
Ok, I suppose \n and \r are enough: readline(size=- 1, /) Read and return one line from the stream. If size is specified, at most size bytes will be read. The line terminator is always b'\n' for binary files; for text files, the newline argument to open() can be used to select the line

Re: tail

2022-05-02 Thread Marco Sulla
On Mon, 2 May 2022 at 00:20, Cameron Simpson wrote: > > On 01May2022 18:55, Marco Sulla wrote: > >Something like this is OK? > [...] > >def tail(f): > >chunk_size = 100 > >size = os.stat(f.fileno()).st_size > > I think you want

Re: tail

2022-05-06 Thread Marco Sulla
I have a little problem. I tried to extend the tail function, so it can read lines from the bottom of a file object opened in text mode. The problem is it does not work. It gets a starting position that is lower than the expected by 3 characters. So the first line is read only for 2 chars, and th

Re: tail

2022-05-07 Thread Marco Sulla
On Sat, 7 May 2022 at 01:03, Dennis Lee Bieber wrote: > > Windows also uses for the EOL marker, but Python's I/O system > condenses that to just internally (for TEXT mode) -- so using the > length of a string so read to compute a file position may be off-by-one for > each EOL in the stri

Re: tail

2022-05-07 Thread Marco Sulla
On Sat, 7 May 2022 at 16:08, Barry wrote: > You need to handle the file in bin mode and do the handling of line endings > and encodings yourself. It’s not that hard for the cases you wanted. >>> "\n".encode("utf-16") b'\xff\xfe\n\x00' >>> "".encode("utf-16") b'\xff\xfe' >>> "a\nb".encode("utf-16

Re: tail

2022-05-07 Thread Marco Sulla
On Sat, 7 May 2022 at 19:02, MRAB wrote: > > On 2022-05-07 17:28, Marco Sulla wrote: > > On Sat, 7 May 2022 at 16:08, Barry wrote: > >> You need to handle the file in bin mode and do the handling of line > >> endings and encodings yourself. It’s not that hard

Re: tail

2022-05-08 Thread Marco Sulla
I think I've _almost_ found a simpler, general way: import os _lf = "\n" _cr = "\r" def tail(filepath, n=10, newline=None, encoding=None, chunk_size=100): n_chunk_size = n * chunk_size pos = os.stat(filepath).st_size chunk_line_pos = -1 lines_not_found = n with open(filepath

Re: tail

2022-05-08 Thread Marco Sulla
On Sun, 8 May 2022 at 20:31, Barry Scott wrote: > > > On 8 May 2022, at 17:05, Marco Sulla wrote: > > > > def tail(filepath, n=10, newline=None, encoding=None, chunk_size=100): > >n_chunk_size = n * chunk_size > > Why use tiny chunks? You can read 4KiB as f

Re: tail

2022-05-08 Thread Marco Sulla
On Sun, 8 May 2022 at 22:02, Chris Angelico wrote: > > Absolutely not. As has been stated multiple times in this thread, a > fully general approach is extremely complicated, horrifically > unreliable, and hopelessly inefficient. Well, my implementation is quite general now. It's not complicated a

Re: tail

2022-05-08 Thread Marco Sulla
On Sun, 8 May 2022 at 22:34, Barry wrote: > > > On 8 May 2022, at 20:48, Marco Sulla wrote: > > > > On Sun, 8 May 2022 at 20:31, Barry Scott wrote: > >> > >>>> On 8 May 2022, at 17:05, Marco Sulla > >>>> wrote: > >>> &g

Re: tail

2022-05-09 Thread Marco Sulla
On Mon, 9 May 2022 at 07:56, Cameron Simpson wrote: > > The point here is that text is a very different thing. Because you > cannot seek to an absolute number of characters in an encoding with > variable sized characters. _If_ you did a seek to an arbitrary number > you can end up in the middle of

Re: tail

2022-05-09 Thread Marco Sulla
On Mon, 9 May 2022 at 19:53, Chris Angelico wrote: > > On Tue, 10 May 2022 at 03:47, Marco Sulla > wrote: > > > > On Mon, 9 May 2022 at 07:56, Cameron Simpson wrote: > > > > > > The point here is that text is a very different thing. Because you >

Re: tail

2022-05-11 Thread Marco Sulla
On Mon, 9 May 2022 at 23:15, Dennis Lee Bieber wrote: > > On Mon, 9 May 2022 21:11:23 +0200, Marco Sulla > declaimed the following: > > >Nevertheless, tail is a fundamental tool in *nix. It's fast and > >reliable. Also the tail command can't handle different

Re: tail

2022-05-11 Thread Marco Sulla
On Wed, 11 May 2022 at 22:09, Chris Angelico wrote: > > Have you actually checked those three, or do you merely suppose them to be > true? I only suppose, as I said. I should do some benchmark and some other tests, and, frankly, I don't want to. I don't want to because I'm quite sure the impleme

Re: tail

2022-05-12 Thread Marco Sulla
On Thu, 12 May 2022 at 00:50, Stefan Ram wrote: > > Marco Sulla writes: > >def tail(filepath, n=10, chunk_size=100): > >if (n <= 0): > >raise ValueError(_err_n) > ... > > There's no spec/doc, so one can't even test it. Excuse me, you&#

Re: tail

2022-05-12 Thread Marco Sulla
Thank you very much. This helped me to improve the function: import os _lf = b"\n" _err_n = "Parameter n must be a positive integer number" _err_chunk_size = "Parameter chunk_size must be a positive integer number" def tail(filepath, n=10, chunk_size=100): if (n <= 0): raise ValueErr

Re: tail

2022-05-13 Thread Marco Sulla
On Fri, 13 May 2022 at 00:31, Cameron Simpson wrote: > On 12May2022 19:48, Marco Sulla wrote: > >On Thu, 12 May 2022 at 00:50, Stefan Ram wrote: > >> There's no spec/doc, so one can't even test it. > > > >Excuse me, you're very right. > >

Re: tail

2022-05-16 Thread Marco Sulla
On Fri, 13 May 2022 at 12:49, <2qdxy4rzwzuui...@potatochowder.com> wrote: > > On 2022-05-13 at 12:16:57 +0200, > Marco Sulla wrote: > > > On Fri, 13 May 2022 at 00:31, Cameron Simpson wrote: > > [...] > > > > This is nearly the worst "specificat

Re: tail

2022-05-18 Thread Marco Sulla
Well, I've done a benchmark. >>> timeit.timeit("tail('/home/marco/small.txt')", globals={"tail":tail}, >>> number=10) 1.5963431186974049 >>> timeit.timeit("tail('/home/marco/lorem.txt')", globals={"t

Re: tail

2022-05-19 Thread Marco Sulla
On Wed, 18 May 2022 at 23:32, Cameron Simpson wrote: > > On 17May2022 22:45, Marco Sulla wrote: > >Well, I've done a benchmark. > >>>> timeit.timeit("tail('/home/marco/small.txt')", globals={"tail":tail}, > >>>>

Re: Subtract n months from datetime

2022-06-22 Thread Marco Sulla
The package arrow has a simple shift method for months, weeks etc https://arrow.readthedocs.io/en/latest/#replace-shift -- https://mail.python.org/mailman/listinfo/python-list

Why I fail so bad to check for memory leak with this code?

2022-07-21 Thread Marco Sulla
I tried to check for memory leaks in a bunch of functions of mine using a simple decorator. It works, but it fails with this code, returning a random count_diff at every run. Why? import tracemalloc import gc import functools from uuid import uuid4 import pickle def getUuid(): return str(uuid

Re: Why I fail so bad to check for memory leak with this code?

2022-07-21 Thread Marco Sulla
On Thu, 21 Jul 2022 at 22:28, MRAB wrote: > > It's something to do with pickling iterators because it still occurs > when I reduce func_76 to: > > @trace > def func_76(): > pickle.dumps(iter([])) It's too strange. I found a bunch of true memory leaks with this decorator. It seems to be relia

Re: Why I fail so bad to check for memory leak with this code?

2022-07-21 Thread Marco Sulla
This naif code shows no leak: import resource import pickle c = 0 while True: pickle.dumps(iter([])) if (c % 1) == 0: max_rss = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss print(f"iteration: {c}, max rss: {max_rss} kb") c += 1 -- https://mail.python.org/

Re: Why I fail so bad to check for memory leak with this code?

2022-07-21 Thread Marco Sulla
llect() snapshot2 = tracemalloc.take_snapshot().filter_traces( (tracemalloc.Filter(True, __file__), ) ) top_stats = snapshot2.compare_to(snapshot1, 'lineno') tracemalloc.stop() for stat in top_stats: print(stat) The result is: /home/marco/sources/test.py:14: size=3339 B (+3339 B)

Re: Why I fail so bad to check for memory leak with this code?

2022-07-22 Thread Marco Sulla
On Fri, 22 Jul 2022 at 09:00, Barry wrote: > With code as complex as python’s there will be memory allocations that occur that will not be directly related to the python code you test. > > To put it another way there is noise in your memory allocation signal. > > Usually the signal of a memory lea

How to generate a .pyi file for a C Extension using stubgen

2022-07-29 Thread Marco Sulla
I tried to follow the instructions here: https://mypy.readthedocs.io/en/stable/stubgen.html but the instructions about creating a stub for a C Extension are a little mysterious. I tried to use it on the .so file without luck. -- https://mail.python.org/mailman/listinfo/python-list

Re: How to generate a .pyi file for a C Extension using stubgen

2022-07-30 Thread Marco Sulla
On Fri, 29 Jul 2022 at 23:23, Barry wrote: > > > > > On 29 Jul 2022, at 19:33, Marco Sulla wrote: > > > > I tried to follow the instructions here: > > > > https://mypy.readthedocs.io/en/stable/stubgen.html > > > > but the instructions a

Am I banned from Discuss forum?

2023-02-10 Thread Marco Sulla
I was banned from the mailing list and Discuss forum for a very long time. Too much IMHO, but I paid my dues. Now this is my state in the forum: - I never posted something unrespectful in the last months - I have a limitation of three posts per threads, but only on some threads - Some random posts

Re: use set notation for repr of dict_keys?

2021-02-24 Thread Marco Sulla
On Wed, 24 Feb 2021 at 06:29, Random832 wrote: > I was surprised, though, to find that you can't remove items directly from > the key set, or in general update it in place with &= or -= (these operators > work, but give a new set object). This is because they are a view. Changing the key object

Re: use set notation for repr of dict_keys?

2021-02-24 Thread Marco Sulla
On Wed, 24 Feb 2021 at 15:02, Random832 wrote: > On Wed, Feb 24, 2021, at 02:59, Marco Sulla wrote: > > On Wed, 24 Feb 2021 at 06:29, Random832 wrote: > > > I was surprised, though, to find that you can't remove items directly > > > from the key set, or in gener

Re: editor recommendations?

2021-02-26 Thread Marco Sulla
I use Sublime free for simple tasks. I like the fact it's fast and it saves to disk immediately. You don't have even to name the file. I use it also for taking notes. Probably not as powerful as Vim and it's proprietary. For development, I use PyCharm, but it's an IDE. I also used in past: gedit:

Re: weirdness with list()

2021-02-28 Thread Marco Sulla
gt; list. What I was getting was a very time consuming (10 seconds or more) > construction of an empty list. I can't reproduce, Am I missing something? marco@buzz:~$ python3 Python 3.6.9 (default, Jan 26 2021, 15:33:00) [GCC 8.4.0] on linux Type "help", "copyright", "cred

Why assert is not a function?

2021-03-02 Thread Marco Sulla
I have a curiosity. Python, as many languages, has assert as a keyword. Can't it be implemented as a function? Is there an advantage to have it as a keyword? -- https://mail.python.org/mailman/listinfo/python-list

Re: yield from () Was: Re: weirdness with list()

2021-03-02 Thread Marco Sulla
On Mon, 1 Mar 2021 at 19:51, Alan Gauld via Python-list wrote: > Sorry, a bit OT but I'm curious. I haven't seen > this before: > > yield from () > > What is it doing? > What do the () represent in this context? It's the empty tuple. -- https://mail.python.org/mailman/listinfo/python-list

How to create both a c extension and a pure python package

2021-03-09 Thread Marco Sulla
As title. Currently I ended up using this trick in my setup.py: if len(argv) > 1 and argv[1] == "c": sys.argv = [sys.argv[0]] + sys.argv[2:] setuptools.setup(ext_modules = ext_modules, **common_setup_args) else: setuptools.setup(**common_setup_args) So if I pass "c" as the first arg

Re: How to create both a c extension and a pure python package

2021-03-10 Thread Marco Sulla
On Wed, 10 Mar 2021 at 16:45, Thomas Jollans wrote: > Why are you doing this? > > If all you want is for it to be possible to install the package from > source on a system that can't use the C part, you could just declare > your extension modules optional Because I want to provide (at least) two

Re: Why assert is not a function?

2021-03-12 Thread Marco Sulla
On Thu, 11 Mar 2021 at 23:11, Ethan Furman wrote: > Basically, you are looking at two different philosophies: > > - Always double check, get good error message when something fails > > vs > > - check during testing and QA, turn off double-checks for production for best > performance possible. In

Re: memory consumption

2021-03-30 Thread Marco Ippolito
Have you tried to identify where in your code the surprising memory allocations are made? You could "bisect search" by adding breakpoints: https://docs.python.org/3/library/functions.html#breakpoint At which point does the problem start manifesting itself? -- https://mail.python.org/mailman/lis

Re: Canonical conversion of dict of dicts to list of dicts

2021-03-30 Thread Marco Ippolito
> > For completeness, from 3.5 onwards, you can also do the following: > > [{'name': n, **d} for n, d in dod.items()] > Reading through these, personally I like this one best. I'm curious what > about it was enabled in 3.5? Was **kwarg expansion inside a dict literal not > possible before then? Any

Re: memory consumption

2021-03-31 Thread Marco Ippolito
> > At which point does the problem start manifesting itself? > The problem spot is my cache(dict). I simplified my code to just load > all the objects to this dict and then clear it. What's the memory utilisation just _before_ performing this load? I am assuming it's much less than this 1 GB you

Re: memory consumption

2021-04-01 Thread Marco Ippolito
> >> What if you increase the machine's (operating system's) swap space? Does > >> that take care of the problem in practice? > > > > I can`t do that because it will affect other containers running on this > > host. > > In my opinion it may significantly reduce their performance. > > Assuming thi

Re: memory consumption

2021-04-01 Thread Marco Ippolito
> > Are you running with systemd? > > I really don't know. An example of how to check: ``` $ readlink /sbin/init /lib/systemd/systemd ``` You want to check which program runs as PID 1. ``` ps 1 ``` -- https://mail.python.org/mailman/listinfo/python-list

Re: Issues in starting Python application

2021-04-01 Thread Marco Ippolito
On 2021-04-01, Mahira Pamnani wrote: > Sir > I have been trying hard to install Python on my PC since a long time. The > application gets installed but, for reasons unknown, it doesn't start. > It keeps asking to repair, modify or uninstall the application. > I have tried doing that too, but it sti

Re: NOT ABLE TO DOWNLOAD speech_recognition ON MY COMPUTER

2021-04-01 Thread Marco Ippolito
On 2021-04-01, ᗷᑌᑎᑎY wrote: > When I enter the command pip install speech_recognition it say’s we cannot > find a compatible version. Try: ``` pip install SpeechRecognition ``` What's the output? -- https://mail.python.org/mailman/listinfo/python-list

Re: Friday Finking: initialising values and implied tuples

2021-04-02 Thread Marco Ippolito
> (a) basic linear presentation: > > resource = "Oil" > time = 1 > crude = 2 > residue = 3 > my_list = "long" > > (b) using explicit tuples: > > ( resource, time, crude, residue, my_list ) = ( "Oil", 1, 2, 3, "long" ) > > (c) linear and indented tuples: > > ( > resource, > time, >

Re: error on os.open API

2021-04-05 Thread Marco Ippolito
> It seems that the os.open API cannot distinguish between a permission error > and the fact that a directory cannot be opened like files. > The following script reproduces the scenario (tested on Python 3.8.2 > (tags/v3.8.2:7b3ab59, Feb 25 2020, 22:45:29) [MSC v.1916 32 bit (Intel)] on > win32) :

How to support annotations for a custom type in a C extension?

2021-09-17 Thread Marco Sulla
I created a custom dict in a C extension. Name it `promethea`. How can I implement `promethea[str, str]`? Now I get: TypeError: 'type' object is not subscriptable -- https://mail.python.org/mailman/listinfo/python-list

Re: How to support annotations for a custom type in a C extension?

2021-09-17 Thread Marco Sulla
Ooook. I have a question. Why is this code not present in dictobject.c? Where are the dict annotations implemented? On Sat, 18 Sept 2021 at 03:00, MRAB wrote: > > On 2021-09-17 21:03, Marco Sulla wrote: > > I created a custom dict in a C extension. Name it `promethea`. H

Python C API: how to mark a type as subclass of another type

2021-10-31 Thread Marco Sulla
I have two types declared as PyTypeObject PyX_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) etc. How can I mark one of the types as subclass of the other one? I tried to use tp_base but it didn't work. -- https://mail.python.org/mailman/listinfo/python-list

Re: Python C API: how to mark a type as subclass of another type

2021-11-02 Thread Marco Sulla
I already added the address of the type to tp_base, but it does not work. On Mon, 1 Nov 2021 at 17:18, Dieter Maurer wrote: > > Marco Sulla wrote at 2021-10-31 23:59 +0100: > >I have two types declared as > > > >PyTypeObject PyX_Type = { > >PyVarObject_HEAD_IN

Re: Python C API: how to mark a type as subclass of another type

2021-11-02 Thread Marco Sulla
*ahem* evidently I didn't check the right package. it works like a charme :D On Tue, 2 Nov 2021 at 13:43, Marco Sulla wrote: > > I already added the address of the type to tp_base, but it does not work. > > On Mon, 1 Nov 2021 at 17:18, Dieter Maurer wrote: > > > &

Py_IS_TYPE(op, &PyDict_Type) does not work on MacOS

2021-11-07 Thread Marco Sulla
As you can read here: https://github.com/Marco-Sulla/python-frozendict/issues/37 Py_IS_TYPE(op, &PyDict_Type) did not work on MacOS. I had to use PyDict_Check. Why don't I have this problem with Linux? PS: since I'm creating a modified version of dict, I copied the dict interna

Re: Py_IS_TYPE(op, &PyDict_Type) does not work on MacOS

2021-11-10 Thread Marco Sulla
Indeed now I use PyDict_Check, but anyway it's very strange that Py_IS_TYPE(op, &PyDict_Type) does not work only on MacOS. On Mon, 8 Nov 2021 at 19:30, Barry wrote: > > > > On 8 Nov 2021, at 07:45, Marco Sulla wrote: > > As you can read here: > > http

Unable to compile my C Extension on Windows: unresolved external link errors

2021-11-12 Thread Marco Sulla
"] undef_macros = [] setuptools.Extension( ext1_fullname, sources = cpython_sources, include_dirs = cpython_include_dirs, extra_compile_args = extra_compile_args, undef_macros = undef_macros, ) Here is the full code: https://github.com/Marco-Sulla/python-frozendict/blob/ma

Re: Unable to compile my C Extension on Windows: unresolved external link errors

2021-11-12 Thread Marco Sulla
On Fri, 12 Nov 2021 at 15:55, Gisle Vanem wrote: > Marco Sulla wrote: > > Error LNK2001: unresolved external symbol PyErr_SetObject > > > > and so on. > > > > I post the part of my setup.py about the C Extension: > > > > extra_compile_args = ["-DP

Re: Unable to compile my C Extension on Windows: unresolved external link errors

2021-11-12 Thread Marco Sulla
: https://github.com/Marco-Sulla/python-frozendict.git On Linux and MacOS it works like a charme. On Windows, it seems it does not find python3.lib. I also added its path to the PATH variable -- https://mail.python.org/mailman/listinfo/python-list

Re: Unable to compile my C Extension on Windows: unresolved external link errors

2021-11-12 Thread Marco Sulla
On Fri, 12 Nov 2021 at 21:09, Chris Angelico wrote: > > On Sat, Nov 13, 2021 at 7:01 AM Marco Sulla > wrote: > > On Fri, 12 Nov 2021 at 17:38, Chris Angelico wrote: > > > Are you sure that you really need Py_BUILD_CORE? > > > > Yes, because I need the intern

Re: Unable to compile my C Extension on Windows: unresolved external link errors

2021-11-13 Thread Marco Sulla
. Sorry, the problem is I downloaded the 32 bit version of VS compiler and 64 bit version of Python.. On Sat, 13 Nov 2021 at 11:10, Barry Scott wrote: > > > > > On 13 Nov 2021, at 09:00, Barry wrote: > > > > > > > >> On 12 Nov 2021, at 22:53, M

Re: Unable to compile my C Extension on Windows: unresolved external link errors

2021-11-14 Thread Marco Sulla
libraries expose the internal functions? Anyway, is there a way to compile Python on Windows in such a way that I get a shared library that exposes all the functions? On Sat, 13 Nov 2021 at 12:17, Marco Sulla wrote: > > . Sorry, the problem is I downloaded the 32 bit version of VS >

Re: Unable to compile my C Extension on Windows: unresolved external link errors

2021-11-14 Thread Marco Sulla
On Sun, 14 Nov 2021 at 16:42, Barry Scott wrote: > > Sorry iPad sent the message before it was complete... > > > On 14 Nov 2021, at 10:38, Marco Sulla wrote: > > > > Okay, now the problem seems to be another: I get the same "unresolved > > external link&qu

Re: How to support annotations for a custom type in a C extension?

2021-11-18 Thread Marco Sulla
It works. Thanks a lot. On Sun, 19 Sept 2021 at 19:23, Serhiy Storchaka wrote: > > 19.09.21 05:59, MRAB пише: > > On 2021-09-18 16:09, Serhiy Storchaka wrote: > >> "(PyCFunction)" is redundant, Py_GenericAlias already has the right > >> type. Overuse of casting to PyCFunction can hide actual bugs

pytest segfault, not with -v

2021-11-19 Thread Marco Sulla
I have a battery of tests done with pytest. My tests break with a segfault if I run them normally. If I run them using pytest -v, the segfault does not happen. What could cause this quantical phenomenon? -- https://mail.python.org/mailman/listinfo/python-list

Re: getting source code line of error?

2021-11-19 Thread Marco Sulla
Have you tried the logger module and the format options? On Fri, 19 Nov 2021 at 19:09, Ulli Horlacher wrote: > > I am trying to get the source code line of the last error. > I know traceback.format_exc() but this contains much more information, e.g.: > > Traceback (most recent call last): > Fil

frozenset can be altered by |=

2021-11-19 Thread Marco Sulla
(venv_3_10) marco@buzz:~$ python Python 3.10.0 (heads/3.10-dirty:f6e8b80d20, Nov 18 2021, 19:16:18) [GCC 10.1.1 20200718] on linux Type "help", "copyright", "credits" or "license" for more information. >>> a = frozenset((3, 4)) >>>

Re: frozenset can be altered by |=

2021-11-19 Thread Marco Sulla
Mh. Now I'm thinking that I've done a = "Marco " a += "Sulla" many times without bothering. On Fri, 19 Nov 2021 at 22:22, Chris Angelico wrote: > > On Sat, Nov 20, 2021 at 8:16 AM Chris Angelico wrote: > > > > On Sat, Nov 20, 2021 at 8:13 A

Re: pytest segfault, not with -v

2021-11-19 Thread Marco Sulla
On Fri, 19 Nov 2021 at 20:38, MRAB wrote: > > On 2021-11-19 17:48, Marco Sulla wrote: > > I have a battery of tests done with pytest. My tests break with a > > segfault if I run them normally. If I run them using pytest -v, the > > segfault does not happen. > > >

Re: pytest segfault, not with -v

2021-11-20 Thread Marco Sulla
imple loop for all the cases, and remove the optional parameter from bench. How boring. PS: is there a way to monitor the Python consumed memory inside Python itself? In this way I could also trap memory leaks. On Sat, 20 Nov 2021 at 01:46, MRAB wrote: > > On 2021-11-19 23:44, Marco Sulla wr

No right operator in tp_as_number?

2021-11-20 Thread Marco Sulla
I checked the documentation: https://docs.python.org/3/c-api/typeobj.html#number-structs and it seems that, in the Python C API, the right operators do not exist. For example, there is nb_add, that in Python is __add__, but there's no nb_right_add, that in Python is __radd__ Am I missing something

Re: pytest segfault, not with -v

2021-11-20 Thread Marco Sulla
e: > > On 2021-11-20 17:40, Marco Sulla wrote: > > Indeed I have introduced a command line parameter in my bench.py > > script that simply specifies the number of times the benchmarks are > > performed. This way I have a sort of segfault checker. > > > > But I

Re: frozenset can be altered by |=

2021-11-22 Thread Marco Sulla
Yes, and you do this regularly. Indeed integers, for example, are immutables and a = 0 a += 1 is something you do dozens of times, and you simply don't think that another object is created and substituted for the variable named `a`. On Mon, 22 Nov 2021 at 14:59, Chris Angelico wrote: > > On Tue

Re: frozenset can be altered by |=

2021-11-29 Thread Marco Sulla
bject. The same for += and *= that are listed under `list` only. On Mon, 22 Nov 2021 at 19:54, Marco Sulla wrote: > > Yes, and you do this regularly. Indeed integers, for example, are immutables > and > > a = 0 > a += 1 > > is something you do dozens of times, and you simp

Re: pytest segfault, not with -v

2021-12-18 Thread Marco Sulla
Ok, I created the script: https://github.com/Marco-Sulla/python-frozendict/blob/master/test/debug.py The problem is it does _not_ crash, while a get a segfault using pytest with python 3.9 on MacOS 10.15 Maybe it's because I'm using eval / exec in the script? On Sat, 20 Nov 202

Re: pytest segfault, not with -v

2021-12-18 Thread Marco Sulla
Emh, maybe I was not clear. I created a C extension and it segfaults. So I created that script to see where it segfaults. But the script does not segfault. My doubt is: is that because I'm using eval and exec in the script? On Sat, 18 Dec 2021 at 18:33, Dieter Maurer wrote: > > Marco

Py_TRASHCAN_SAFE_BEGIN/END in C extension?

2021-12-21 Thread Marco Sulla
In Python 3.7, must Py_TRASHCAN_SAFE_BEGIN - Py_TRASHCAN_SAFE_END be used in a C extension? I'm asking because in my C extension I use them in the deallocator without problems, but users signalled me that they segfault in Python 3.7 on Debian 10. I checked and this is true. -- https://mail.python

Re: Py_TRASHCAN_SAFE_BEGIN/END in C extension?

2021-12-22 Thread Marco Sulla
Yes, it's deprecated, but I need it for Python 3.7, since there was yet no Py_TRASHCAN_BEGIN / END On Tue, 21 Dec 2021 at 23:22, Barry wrote: > > > > On 21 Dec 2021, at 22:08, Marco Sulla wrote: > > In Python 3.7, must Py_TRASHCAN_SAFE_BEGIN - Py_TRASHCAN_SAFE_END be

Re: recover pickled data: pickle data was truncated

2021-12-26 Thread Marco Sulla
Use a semaphore. On Sun, 26 Dec 2021 at 03:30, iMath wrote: > > Normally, the shelve data should be read and write by only one process at a > time, but unfortunately it was simultaneously read and write by two > processes, thus corrupted it. Is there any way to recover all data in it ? > Curre

What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-26 Thread Marco Sulla
I have to use _PyObject_GC_IS_TRACKED(). It can't be used unless you define Py_BUILD_CORE. I want to avoid this. What macro or function can substitute it? -- https://mail.python.org/mailman/listinfo/python-list

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-27 Thread Marco Sulla
I need it since I'm developing an immutable dict. And in dict that function is used. I do not understand why there's no public API for that function. It seems very useful. On Sun, 26 Dec 2021 at 17:28, Barry Scott wrote: > > > > > On 26 Dec 2021, at 13:48, Marco Sulla

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-27 Thread Marco Sulla
Hi, Inada Senpai. So I do not need PyObject_GC_Track on cloning or merging, or MAINTAIN_TRACKING on insert? On Tue, 28 Dec 2021 at 07:58, Inada Naoki wrote: > > On Tue, Dec 28, 2021 at 3:31 AM Marco Sulla > wrote: > > > > I need it since I'm developing an immuta

Option for venv to upgrade pip automatically?

2021-12-28 Thread Marco Sulla
I think it's very boring that, after creating a venv, you have immediately to do every time: pip install -U pip Can't venv have an option for doing this automatically or, better, a config file where you can put commands that will be launched every time after you create a venv? -- https://mail.py

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-28 Thread Marco Sulla
On Tue, 28 Dec 2021 at 12:38, Inada Naoki wrote: > Your case is special. > You want to create a frozendict which performance is same to builtin dict. > Builtin dict has special optimization which tightly coupled with > current CPython implementation. > So you need to use private APIs for MAINTAIN_

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-28 Thread Marco Sulla
On Wed, 29 Dec 2021 at 00:03, Dieter Maurer wrote: > Why do you not derive from `dict` and override its mutating methods > (to raise a type error after initialization is complete)? I've done this for the pure py version, for speed. But in this way, frozendict results to be a subclass of MutableMa

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-28 Thread Marco Sulla
On Wed, 29 Dec 2021 at 07:46, Inada Naoki wrote: > You are right. I thought PyObject_GC_Track() can be used to tracked > objects because PyObject_GC_Untrack() can be used untracked object. > I think there is no enough reason for this asymmetry. > > Additionally, adding PyObject_GC_IsTracked() to p

Re: Option for venv to upgrade pip automatically?

2021-12-29 Thread Marco Sulla
Cool, thanks! On Wed, 29 Dec 2021 at 07:10, Inada Naoki wrote: > > You can use --upgrade-deps option. My alias is: > > alias mkvenv='python3 -m venv --upgrade-deps --prompt . venv' > > On Wed, Dec 29, 2021 at 4:55 AM Marco Sulla > wrote: > > > > I th

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-29 Thread Marco Sulla
On Wed, 29 Dec 2021 at 09:12, Dieter Maurer wrote: > > Marco Sulla wrote at 2021-12-29 08:08 +0100: > >On Wed, 29 Dec 2021 at 00:03, Dieter Maurer wrote: > >> Why do you not derive from `dict` and override its mutating methods > >> (to raise a type error af

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-29 Thread Marco Sulla
c 2021 at 09:24, Marco Sulla wrote: > > On Wed, 29 Dec 2021 at 09:12, Dieter Maurer wrote: > > > > Marco Sulla wrote at 2021-12-29 08:08 +0100: > > >On Wed, 29 Dec 2021 at 00:03, Dieter Maurer wrote: > > >> Why do you not derive from `dict` and override it

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-29 Thread Marco Sulla
On Wed, 29 Dec 2021 at 10:06, Dieter Maurer wrote: > > Are you sure you need to implement your type in C at all? It's already implemented, and, in some cases, is faster than dict: https://github.com/Marco-Sulla/python-frozendict#benchmarks PS: I'm doing a refactoring that sp

How to implement freelists in dict 3.10 for previous versions?

2021-12-29 Thread Marco Sulla
I noticed that now freelists in dict use _Py_dict_state. I suppose this is done for thread safety. I would implement it also for a C extension that uses CPython < 3.10. How can I achieve this? -- https://mail.python.org/mailman/listinfo/python-list

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-29 Thread Marco Sulla
On Wed, 29 Dec 2021 at 09:12, Dieter Maurer wrote: > `MutableMapping` is a so called abstract base class (--> `abc`). > > It uses the `__subclass_check__` (and `__instance_check__`) of > `abc.ABCMeta` to ensure `issubclass(dict, MutableMapping)`. > Those can be customized by overriding `MutableMap

Re: What's the public API alternative to _PyObject_GC_IS_TRACKED()?

2021-12-29 Thread Marco Sulla
On Wed, 29 Dec 2021 at 12:11, Dieter Maurer wrote: > > Marco Sulla wrote at 2021-12-29 11:59 +0100: > >On Wed, 29 Dec 2021 at 09:12, Dieter Maurer wrote: > >> `MutableMapping` is a so called abstract base class (--> `abc`). > >> > >> It uses the `__subcl

Re: recover pickled data: pickle data was truncated

2021-12-29 Thread Marco Sulla
On Wed, 29 Dec 2021 at 18:33, iMath wrote: > But I found the size of the file of the shelve data didn't change much, so I > guess the data are still in it , I just wonder any way to recover my data. I agree with Barry, Chris and Avi. IMHO your data is lost. Unpickling it by hand is a harsh work

Re: builtins.TypeError: catching classes that do not inherit from BaseException is not allowed

2021-12-31 Thread Marco Sulla
It was already done: https://pypi.org/project/tail-recursive/ On Thu, 30 Dec 2021 at 16:00, hongy...@gmail.com wrote: > > I try to compute the factorial of a large number with tail-recursion > optimization decorator in Python3. The following code snippet is converted > from the code snippet giv

Re: recover pickled data: pickle data was truncated

2022-01-01 Thread Marco Sulla
I agree with Barry. You can create a folder or a file with pseudo-random names. I recommend you to use str(uuid.uuid4()) On Sat, 1 Jan 2022 at 14:11, Barry wrote: > > > > > On 31 Dec 2021, at 17:53, iMath wrote: > > > > 在 2021年12月30日星期四 UTC+8 03:13:21, 写道: > >>> On Wed, 29 Dec 2021 at 18:33, iM

How to make a type of a C extension compatible with mypy

2022-01-01 Thread Marco Sulla
I created a type in a C extension, that is an immutable dict. If I do: a: mydict[str, str] it works. But it doesn't work with mypy, as signalled to me by an user: https://github.com/Marco-Sulla/python-frozendict/issues/39 How can I make it work? I don't know what he means with

Re: How to implement freelists in dict 3.10 for previous versions?

2022-01-01 Thread Marco Sulla
Ooookay, I suppose I have to study a little the thing :D On Thu, 30 Dec 2021 at 07:59, Inada Naoki wrote: > > On Wed, Dec 29, 2021 at 7:25 PM Marco Sulla > wrote: > > > > I noticed that now freelists in dict use _Py_dict_state. I suppose > > this is done for thread sa

Who wrote Py_UNREACHABLE?

2022-01-02 Thread Marco Sulla
#if defined(RANDALL_WAS_HERE) # define Py_UNREACHABLE() \ Py_FatalError( \ "If you're seeing this, the code is in what I thought was\n" \ "an unreachable state.\n\n" \ "I could give you advice for what to do, but honestly, why\n" \ "should you trust me? I clear

Re: ModuleNotFoundError: No module named 'DistUtilsExtra'

2022-01-02 Thread Marco Sulla
https://askubuntu.com/questions/584857/distutilsextra-problem On Sun, 2 Jan 2022 at 18:52, hongy...@gmail.com wrote: > > On Ubuntu 20.04.3 LTS, I try to install pdfarranger [1] as follows but failed: > > $ sudo apt-get install python3-pip python3-distutils-extra \ >

<    1   2   3   4   5   6   7   8   >