Re: Pre-Pre-PEP: The datetime.timedeltacal class
> On 18 Apr 2022, at 13:01, Peter J. Holzer wrote: > > On 2022-04-16 20:25:45 +0100, Barry wrote: >> Suggest that you start with the use cases that you want supported. >> Then you can turn them into a set of tests to check that the solution works. > > Writing test cases is always a good idea :-) Did you write the use cases? Without them how can anyone review the tests? Barry > > I have now written a first set of test cases: > https://git.hjp.at:3000/hjp/timedeltacal > (together with a quick and dirty implementation that passes them). > > That's not complete yet. > > If covers addition of timezone aware datetime values and timedeltacal > values fairly well, and also tests the t0 + (t1 - t0) == t1 property I > alluded to elsewhere in this thread. > > It doesn't test canonicalization yet (and indeed the prototype > implementation is a mess in this regard) and subtracting timedeltacals > from datetimes is also still missing. > >hp > > -- > _ | Peter J. Holzer| Story must make more sense than reality. > |_|_) || > | | | h...@hjp.at |-- Charles Stross, "Creative writing > __/ | http://www.hjp.at/ | challenge!" > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Why does datetime.timedelta only have the attributes 'days' and 'seconds'?
> On 19 Apr 2022, at 19:38, Dennis Lee Bieber wrote: > > *I /think/ this is the year used for leap-day calculations, and why some > leap centuries are skipped as it is really less than a quarter day per > year, so eventually one gets to over-correcting by a day. Leap century is skip unless it’s a leap quadra century. Barry -- https://mail.python.org/mailman/listinfo/python-list
Re: How to have python 2 and 3 both on windows?
> On 22 Apr 2022, at 17:10, Sunil KR via Python-list > wrote: > > I have some scripts that are old and won't work under python2 and at the > same time I am writing new scripts which will use python3. However, if python > 2 and 3 cannot co-exist in a windows box it will be impossible to transition > What I try:- remove all pythons and launchers- Use windows installer and > install python2 in python27 directory- Use windows installer and install > python3 in python310 directory- When installing python3 I opt in to install > the launcher- Test with py -2 and py -3 and see that I get the expected > prompt- just typing python gets me python2 As you have proved you can install many versions of python at the same time on windows. In your scripts set the shebang to run the python version you want. E.g #!/usr/bin/python2 Or #!/usr/bin/python3 The python launcher py.exe will then do the right thing after looking at en shebang line. Also you can edit the INI file of the py.exe launcher to set the defaults the way you want them. Do a search for “py.exe ini” to file the path to the file, I do not have it my finger tips. Tip “py.exe -0” will list the state of installed pythons. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: How to have python 2 and 3 both on windows?
> On 22 Apr 2022, at 18:43, Gisle Vanem wrote: > > Barry wrote: > >> Tip “py.exe -0” will list the state of installed pythons. > Not here; 'py.exe -0' gives: > Requested Python version (0) not installed > > Which PyInstaller version support this '-0' option? I do not when it was first added all the installs I have done in the last few years have it working. Barry > > -- > --gv > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Verifying I installed Python correctly
> On 25 Apr 2022, at 18:51, Greg wrote: > > I am trying to get Hello World to appear under my directory. The files of > > *C:\Users\gd752>cd C:\google-python-exercises> python hello.py* > *The system cannot find the path specified.* Use can use py instead of python as a command and it should work. py hello.py Barry > > *C:\Users\gd752>cd C:\google-python-exercises>* > *The syntax of the command is incorrect.* > > I installed version 3.10. I am stuck and could use some help. > Thx, > > > [image: directory pic.png] > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Verifying I installed Python correctly
> On 25 Apr 2022, at 21:14, Jack Dangler wrote: > > >> On 4/24/22 13:59, Greg wrote: >> I am trying to get Hello World to appear under my directory. The files of >> >> *C:\Users\gd752>cd C:\google-python-exercises> python hello.py* >> *The system cannot find the path specified.* >> >> *C:\Users\gd752>cd C:\google-python-exercises>* >> *The syntax of the command is incorrect.* >> >> I installed version 3.10. I am stuck and could use some help. >> Thx, >> >> >> [image: directory pic.png] > > Have you tried > > python3 hello.py Will not work on windows. Python is always installed as python.exe and py.exe only. Barry > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
> On 7 May 2022, at 14:24, Marco Sulla wrote: > > On Sat, 7 May 2022 at 01:03, Dennis Lee Bieber wrote: >> >>Windows also uses for the EOL marker, but Python's I/O system >> condenses that to just internally (for TEXT mode) -- so using the >> length of a string so read to compute a file position may be off-by-one for >> each EOL in the string. > > So there's no way to reliably read lines in reverse in text mode using > seek and read, but the only option is readlines? You need to handle the file in bin mode and do the handling of line endings and encodings yourself. It’s not that hard for the cases you wanted. Figure out which line ending is in use from the CR LF, LF, CR. Once you have a line decode it before returning it. The only OS I know that used CR was Classic Mac OS. If you do not care about that then you can split on NL and strip any trailing CR. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
> On 7 May 2022, at 17:29, Marco Sulla wrote:
>
> On Sat, 7 May 2022 at 16:08, Barry wrote:
>> You need to handle the file in bin mode and do the handling of line endings
>> and encodings yourself. It’s not that hard for the cases you wanted.
>
>>>> "\n".encode("utf-16")
> b'\xff\xfe\n\x00'
>>>> "".encode("utf-16")
> b'\xff\xfe'
>>>> "a\nb".encode("utf-16")
> b'\xff\xfea\x00\n\x00b\x00'
>>>> "\n".encode("utf-16").lstrip("".encode("utf-16"))
> b'\n\x00'
>
> Can I use the last trick to get the encoding of a LF or a CR in any encoding?
In a word no.
There are cases that you just have to know the encoding you are working with.
utf-16 because you have deal with the data in 2 byte units and know if
it is big endian or little endian.
There will be other encoding that will also be difficult.
But if you are working with encoding that are using ASCII as a base,
like unicode encoded as utf-8 or iso-8859 series then you can just look
for NL and CR using the ASCII values of the byte.
In short once you set your requirements then you can know what problems
you can avoid and which you must solve.
Is utf-16 important to you? If not no need to solve its issues.
Barry
--
https://mail.python.org/mailman/listinfo/python-list
Re: tail
> On 8 May 2022, at 20:48, Marco Sulla wrote:
>
> On Sun, 8 May 2022 at 20:31, Barry Scott wrote:
>>
>>>> On 8 May 2022, at 17:05, Marco Sulla wrote:
>>>
>>> def tail(filepath, n=10, newline=None, encoding=None, chunk_size=100):
>>> n_chunk_size = n * chunk_size
>>
>> Why use tiny chunks? You can read 4KiB as fast as 100 bytes as its typically
>> the smaller size the file system will allocate.
>> I tend to read on multiple of MiB as its near instant.
>
> Well, I tested on a little file, a list of my preferred pizzas, so
Try it on a very big file.
>
>>> pos = os.stat(filepath).st_size
>>
>> You cannot mix POSIX API with text mode.
>> pos is in bytes from the start of the file.
>> Textmode will be in code points. bytes != code points.
>>
>>> chunk_line_pos = -1
>>> lines_not_found = n
>>>
>>> with open(filepath, newline=newline, encoding=encoding) as f:
>>> text = ""
>>>
>>> hard_mode = False
>>>
>>> if newline == None:
>>> newline = _lf
>>> elif newline == "":
>>> hard_mode = True
>>>
>>> if hard_mode:
>>> while pos != 0:
>>> pos -= n_chunk_size
>>>
>>> if pos < 0:
>>> pos = 0
>>>
>>> f.seek(pos)
>>
>> In text mode you can only seek to a value return from f.tell() otherwise the
>> behaviour is undefined.
>
> Why? I don't see any recommendation about it in the docs:
> https://docs.python.org/3/library/io.html#io.IOBase.seek
What does adding 1 to a pos mean?
If it’s binary it mean 1 byte further down the file but in text mode it may
need to
move the point 1, 2 or 3 bytes down the file.
>
>>> text = f.read()
>>
>> You have on limit on the amount of data read.
>
> I explained that previously. Anyway, chunk_size is small, so it's not
> a great problem.
Typo I meant you have no limit.
You read all the data till the end of the file that might be mega bytes of data.
>
>>> lf_after = False
>>>
>>> for i, char in enumerate(reversed(text)):
>>
>> Simple use text.rindex('\n') or text.rfind('\n') for speed.
>
> I can't use them when I have to find both \n or \r. So I preferred to
> simplify the code and use the for cycle every time. Take into mind
> anyway that this is a prototype for a Python C Api implementation
> (builtin I hope, or a C extension if not)
>
>>> Shortly, the file is always opened in text mode. File is read at the end in
>>> bigger and bigger chunks, until the file is finished or all the lines are
>>> found.
>>
>> It will fail if the contents is not ASCII.
>
> Why?
>
>>> Why? Because in encodings that have more than 1 byte per character, reading
>>> a chunk of n bytes, then reading the previous chunk, can eventually split
>>> the character between the chunks in two distinct bytes.
>>
>> No it cannot. text mode only knows how to return code points. Now if you are
>> in
>> binary it could be split, but you are not in binary mode so it cannot.
>
>> From the docs:
>
> seek(offset, whence=SEEK_SET)
> Change the stream position to the given byte offset.
>
>>> Do you think there are chances to get this function as a method of the file
>>> object in CPython? The method for a file object opened in bytes mode is
>>> simpler, since there's no encoding and newline is only \n in that case.
>>
>> State your requirements. Then see if your implementation meets them.
>
> The method should return the last n lines from a file object.
> If the file object is in text mode, the newline parameter must be honored.
> If the file object is in binary mode, a newline is always b"\n", to be
> consistent with readline.
>
> I suppose the current implementation of tail satisfies the
> requirements for text mode. The previous one satisfied binary mode.
>
> Anyway, apart from my implementation, I'm curious if you think a tail
> method is worth it to be a method of the builtin file objects in
> CPython.
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: tail
> On 9 May 2022, at 17:41, r...@zedat.fu-berlin.de wrote: > > Barry Scott writes: >> Why use tiny chunks? You can read 4KiB as fast as 100 bytes > > When optimizing code, it helps to be aware of the orders of > magnitude That is true and we’ll know to me, now show how what I said is wrong. The os is going to DMA at least 4k, with read ahead more like 64k. So I can get that into the python memory at the same scale of time as 1 byte because it’s the setup of the I/O that is expensive not the bytes transferred. Barry > . Code that is more cache-friendly is faster, that is, > code that holds data in single region of memory and that uses > regular patterns of access. Chandler Carruth talked about this, > and I made some notes when watching the video of his talk: > > CPUS HAVE A HIERARCHICAL CACHE SYSTEM > (from a 2014 talk by Chandler Carruth) > > One cycle on a 3 GHz processor1 ns > L1 cache reference0.5 ns > Branch mispredict 5 ns > L2 cache reference7 ns 14x L1 cache > Mutex lock/unlock25 ns > Main memory reference 100 ns 20xL2, 200xL1 > Compress 1K bytes with Snappy 3,000 ns > Send 1K bytes over 1 Gbps network10,000 ns 0.01 ms > Read 4K randomly from SSD 150,000 ns 0.15 ms > Read 1 MB sequentially from memory 250,000 ns 0.25 ms > Round trip within same datacenter 500,000 ns 0.5 ms > Read 1 MB sequentially From SSD 1,000,000 ns 1 ms 4x memory > Disk seek10,000,000 ns 10 ms 20xdatacen. RT > Read 1 MB sequentially from disk 20,000,000 ns 20 ms 80xmem.,20xSSD > Send packet CA->Netherlands->CA 150,000,000 ns 150 ms > > . Remember how recently people here talked about how you cannot > copy text from a video? Then, how did I do it? Turns out, for my > operating system, there's a screen OCR program! So I did this OCR > and then manually corrected a few wrong characters, and was done! > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
> On 9 May 2022, at 20:14, Marco Sulla wrote: > > On Mon, 9 May 2022 at 19:53, Chris Angelico wrote: >> >>> On Tue, 10 May 2022 at 03:47, Marco Sulla >>> wrote: >>> >>> On Mon, 9 May 2022 at 07:56, Cameron Simpson wrote: >>>> >>>> The point here is that text is a very different thing. Because you >>>> cannot seek to an absolute number of characters in an encoding with >>>> variable sized characters. _If_ you did a seek to an arbitrary number >>>> you can end up in the middle of some character. And there are encodings >>>> where you cannot inspect the data to find a character boundary in the >>>> byte stream. >>> >>> Ooook, now I understand what you and Barry mean. I suppose there's no >>> reliable way to tail a big file opened in text mode with a decent >>> performance. >>> >>> Anyway, the previous-previous function I posted worked only for files >>> opened in binary mode, and I suppose it's reliable, since it searches >>> only for b"\n", as readline() in binary mode do. >> >> It's still fundamentally impossible to solve this in a general way, so >> the best way to do things will always be to code for *your* specific >> use-case. That means that this doesn't belong in the stdlib or core >> language, but in your own toolkit. > > Nevertheless, tail is a fundamental tool in *nix. It's fast and > reliable. Also the tail command can't handle different encodings? POSIX tail just prints the bytes to the output that it finds between \n bytes. At no time does it need to care about encodings as that is a problem solved by the terminal software. I would not expect utf-16 to work with tail on linux systems. You could always get the source of tail and read It’s implementation. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Convert the decimal numbers expressed in a `numpy.ndarray` into a matrix representing elements in fractional form
> On 17 May 2022, at 05:59, hongy...@gmail.com wrote:
>
> On Monday, May 16, 2022 at 11:27:58 PM UTC+8, Dennis Lee Bieber wrote:
>> On Mon, 16 May 2022 02:03:26 -0700 (PDT), "hongy...@gmail.com"
>> declaimed the following:
>>
>>
>>> print(lst)
>>
>> Printing higher level structures uses the repr() of the structure and
>> its contents -- theoretically a form that could be used within code as a
>> literal.
>
> I tried with the repr() method as follows, but it doesn't give any output:
Repr returns a string. You need to print its value to see it.
>
> ```
> import os,sys
> import numpy as np
> from fractions import Fraction
> import re
> from pymatgen.symmetry.analyzer import SpacegroupAnalyzer
> from pymatgen.core import Lattice, Structure, Molecule, IStructure
>
> def filepath(file):
> script_dirname=os.path.dirname(os.path.realpath(__file__))
> return (script_dirname + '/' + file)
>
> s=IStructure.from_file(filepath('EntryWithCollCode136212.cif'))
> a = SpacegroupAnalyzer(s)
> SymOp=a.get_symmetry_operations()
> b=SymOp[1].affine_matrix.tolist()
>
> def strmat(m):
> if(np.array([m]).ndim==1):
> return str(Fraction(m))
> else: return list(map(lambda L:strmat(L), np.array(m)))
>
> lst=[]
> for i in SymOp:
> lst.append(i.affine_matrix.tolist())
>
> a=str(strmat(lst))
> a=re.sub(r"'","",a)
> repr(a)
print(repr(a))
Barry
> ```
>
>> If you want human-readable str() you will need to write your own
>> output loop to do the formatting of the structure, and explicitly print
>> each item of the structure.
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: terminate called after throwing an instance of 'boost::python::error_already_set
> On 27 May 2022, at 21:17, Larry Martell wrote: > > I have a script that has literally been running for 10 years. > Suddenly, for some runs it crashes with the error: > > terminate called after throwing an instance of > 'boost::python::error_already_set This is from an extension that is written in C++ that raised a C++exception, not a python one. The default action in C++ is to terminal the process so python does not get a chance to prints its stack. > > No stack trace. Anyone have any thoughts on what could cause this > and/or how I can track it down? You will need to use a C++ level debugger to see which extension is crashing. gdb on linux, lldb on macOs or visual studio on windows. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: How to test characters of a string
> On 7 Jun 2022, at 22:04, Dave wrote: > > It depends on the language I’m using, in Objective C, I’d use isNumeric, > just wanted to know what the equivalent is in Python. > > If you know the answer why don’t you just tell me and if you don’t, don’t > post! People ask home work questions here and we try to teach a student with hints not finished answers. Your post was confused with a home work question. Barry > > >> On 7 Jun 2022, at 22:08, 2qdxy4rzwzuui...@potatochowder.com wrote: >> >>> On 2022-06-07 at 21:35:43 +0200, >>> Dave wrote: >>> >>> I’m new to Python and have a simple problem that I can’t seem to find >>> the answer. >> >>> I want to test the first two characters of a string to check if the >>> are numeric (00 to 99) and if so remove the fist three chars from the >>> string. >> >>> Example: if “05 Trinket” I want “Trinket”, but “Trinket” I still want >>> “Trinket”. I can’t for the life of work out how to do it in Python? >> >> How would you do it without Python? >> >> Given that if the string is called x, then x[y] is the y'th character >> (where what you would call "the first character," Python calls "the >> zeroth character"), describe the steps you would take *as a person* (or >> in some other programming language, if you know one) to carry out this >> task. >> >> Translating that algorithm to Python is the next step. Perhaps >> https://docs.python.org/3/library/string.html can help. >> -- >> https://mail.python.org/mailman/listinfo/python-list > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: installing
> On 10 Jun 2022, at 17:32, Yusuf Özdemir wrote: > > ? My guess you attached an image, but this list does not allows attachements. Please copy and paste the text of the error for us to read. However you may find that this helps: https://docs.python.org/3/using/windows.html Barry > > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Are there any benefits of a shared build python, compared to a static build python?
> On 20 Jul 2022, at 18:09, Tianhe wrote: > > Python by default builds the library `libpythonMAJOR.MINOR.a` and > statically links it into the interpreter. Also it has an `--enable-shared`, > (https://docs.python.org/3/using/configure.html#cmdoption-enable-shared) > flag, which will build a shared library `libpythonMAJOR.MINOR.so.1.0`, and > dynamically link it to the interpreter. > > Are there any benefits of a shared build python, compared to a static build > python? Is there any performance difference? What appears to be important is being able to do link time optimisation of python code. I think Debian starting doing this first and other distro followed after seeing the big performance improvement Debian got. Barry > > I found a related question on SO and is it like he said ( > https://stackoverflow.com/a/73035776/5983841), shared built vs statically > built python only affects the "Embedding Python in Another Application" > scenario ? > > -- > > Regards, > > Micky > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Why I fail so bad to check for memory leak with this code?
> On 21 Jul 2022, at 21:54, Marco Sulla wrote: > On Thu, 21 Jul 2022 at 22:28, MRAB wrote: >> >> It's something to do with pickling iterators because it still occurs >> when I reduce func_76 to: >> >> @trace >> def func_76(): >> pickle.dumps(iter([])) > > It's too strange. I found a bunch of true memory leaks with this > decorator. It seems to be reliable. It's correct with pickle and with > iter, but not when pickling iters. With code as complex as python’s there will be memory allocations that occur that will not be directly related to the python code you test. To put it another way there is noise in your memory allocation signal. Usually the signal of a memory leak is very clear, as you noticed. For rare leaks I would use a tool like valgrind. Barry > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: random.SystemRandom().randint() inefficient
> On 26 Jul 2022, at 16:07, Cecil Westerhof via Python-list > wrote: > > I need to get a random integer. At first I tried it with: >from secrets import randbelow >index = randbelow(len(to_try)) > > This works perfectly, but it took some time. So I thought I try: >from random import SystemRandom >index = SystemRandom().randint(0, len(to_try) - 1) > > A first indication is that the second version would take about two > times as much time as the first. Is there a reason for this, or should > this not be happening? What is the OS that you are running on and its version? If it’s linux what is the kernel version? What version of python and where from? Barry > > -- > Cecil Westerhof > Senior Software Engineer > LinkedIn: http://www.linkedin.com/in/cecilwesterhof > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: random.SystemRandom().randint() inefficient
> On 27 Jul 2022, at 17:09, Cecil Westerhof via Python-list > wrote: > > Barry writes: > >>>> On 26 Jul 2022, at 16:07, Cecil Westerhof via Python-list >>>> wrote: >>> >>> I need to get a random integer. At first I tried it with: >>> from secrets import randbelow >>> index = randbelow(len(to_try)) >>> >>> This works perfectly, but it took some time. So I thought I try: >>> from random import SystemRandom >>> index = SystemRandom().randint(0, len(to_try) - 1) >>> >>> A first indication is that the second version would take about two >>> times as much time as the first. Is there a reason for this, or should >>> this not be happening? >> >> What is the OS that you are running on and its version? >> If it’s linux what is the kernel version? >> What version of python and where from? > > That is always good information of-course. > Debian 11.3 > 5.10.0-13-amd64 > 3.9.2 > > What do you mean with where the python version is from? Because random number generator implementation depends on the OS since it is SystemRandom() that you are comparing with python’s Implementation. Barry > > -- > Cecil Westerhof > Senior Software Engineer > LinkedIn: http://www.linkedin.com/in/cecilwesterhof > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Simple TCP proxy
> On 27 Jul 2022, at 17:16, Morten W. Petersen wrote: > > Hi. > > I'd like to share with you a recent project, which is a simple TCP proxy > that can stand in front of a TCP server of some sort, queueing requests and > then allowing n number of connections to pass through at a time: > > https://github.com/morphex/stp > > I'll be developing it further, but the the files committed in this tree > seem to be stable: > > https://github.com/morphex/stp/tree/9910ca8c80e9d150222b680a4967e53f0457b465 > > I just bombed that code with 700+ requests almost simultaneously, and STP > handled it well. What is the problem that this solves? Why not just increase the allowed size of the socket listen backlog if you just want to handle bursts of traffic. I do not think of this as a proxy, rather a tunnel. And the tunnel is a lot more expensive the having kernel keep the connection in the listen socket backlog. I work on a web proxy written on python that handles huge load and using backlog of the bursts. It’s async using twisted as threads are not practice at scale. Barry > > Regards, > > Morten > > -- > I am https://leavingnorway.info > Videos at https://www.youtube.com/user/TheBlogologue > Twittering at http://twitter.com/blogologue > Blogging at http://blogologue.com > Playing music at https://soundcloud.com/morten-w-petersen > Also playing music and podcasting here: > http://www.mixcloud.com/morten-w-petersen/ > On Google+ here https://plus.google.com/107781930037068750156 > On Instagram at https://instagram.com/morphexx/ > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Simple TCP proxy
> On 28 Jul 2022, at 10:31, Morten W. Petersen wrote: > > > Hi Barry. > > Well, I can agree that using backlog is an option for handling bursts. But > what if that backlog number is exceeded? How easy is it to deal with such a > situation? You can make backlog very large, if that makes sense. But at some point you will be forced to reject connections, once you cannot keep up with the average rate of connections. > > I just cloned twisted, and compared the size: > > morphex@morphex-Latitude-E4310:~$ du -s stp; du -s tmp/twisted/ > 464 stp > 98520 tmp/twisted/ > morphex@morphex-Latitude-E4310:~$ du -sh stp/LICENSE > 36K stp/LICENSE > > >>> 464/98520.0 > 0.004709703613479496 > >>> > > It's quite easy to get an idea of what's going on in STP, as opposed to if > something goes wrong in Twisted with the size of the codebase. I used to use > emacs a lot, but then I came into a period where it was more practical to use > nano, and I mostly use nano now, unless I need to for example search and > replace or something like that. I mentioned twisted for context. Depending on yours need the built in python 3 async support may well be sufficient for you needs. Using threads is not scalable. In the places I code disk space of a few MiB is not an issue. Barry > > -Morten > >> On Thu, Jul 28, 2022 at 8:31 AM Barry wrote: >> >> >> > On 27 Jul 2022, at 17:16, Morten W. Petersen wrote: >> > >> > Hi. >> > >> > I'd like to share with you a recent project, which is a simple TCP proxy >> > that can stand in front of a TCP server of some sort, queueing requests and >> > then allowing n number of connections to pass through at a time: >> > >> > https://github.com/morphex/stp >> > >> > I'll be developing it further, but the the files committed in this tree >> > seem to be stable: >> > >> > https://github.com/morphex/stp/tree/9910ca8c80e9d150222b680a4967e53f0457b465 >> > >> > I just bombed that code with 700+ requests almost simultaneously, and STP >> > handled it well. >> >> What is the problem that this solves? >> >> Why not just increase the allowed size of the socket listen backlog if you >> just want to handle bursts of traffic. >> >> I do not think of this as a proxy, rather a tunnel. >> And the tunnel is a lot more expensive the having kernel keep the connection >> in >> the listen socket backlog. >> >> I work on a web proxy written on python that handles huge load and >> using backlog of the bursts. >> >> It’s async using twisted as threads are not practice at scale. >> >> Barry >> >> > >> > Regards, >> > >> > Morten >> > >> > -- >> > I am https://leavingnorway.info >> > Videos at https://www.youtube.com/user/TheBlogologue >> > Twittering at http://twitter.com/blogologue >> > Blogging at http://blogologue.com >> > Playing music at https://soundcloud.com/morten-w-petersen >> > Also playing music and podcasting here: >> > http://www.mixcloud.com/morten-w-petersen/ >> > On Google+ here https://plus.google.com/107781930037068750156 >> > On Instagram at https://instagram.com/morphexx/ >> > -- >> > https://mail.python.org/mailman/listinfo/python-list >> > >> > > > -- > I am https://leavingnorway.info > Videos at https://www.youtube.com/user/TheBlogologue > Twittering at http://twitter.com/blogologue > Blogging at http://blogologue.com > Playing music at https://soundcloud.com/morten-w-petersen > Also playing music and podcasting here: > http://www.mixcloud.com/morten-w-petersen/ > On Google+ here https://plus.google.com/107781930037068750156 > On Instagram at https://instagram.com/morphexx/ -- https://mail.python.org/mailman/listinfo/python-list
Re: How to generate a .pyi file for a C Extension using stubgen
> On 29 Jul 2022, at 19:33, Marco Sulla wrote: > > I tried to follow the instructions here: > > https://mypy.readthedocs.io/en/stable/stubgen.html > > but the instructions about creating a stub for a C Extension are a little > mysterious. I tried to use it on the .so file without luck. It says that stubgen works on .py files not .so files. You will need to write the .pyi for your .so manually. The docs could do with splitting the need for .pyi for .so away from the stubgen description. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: mailbox misbehavior with non-ASCII
> On 30 Jul 2022, at 00:30, Peter Pearson wrote:
>
> The following code produces a nonsense result with the input
> described below:
>
> import mailbox
> box = mailbox.Maildir("/home/peter/Temp/temp",create=False)
> x = box.values()[0]
> h = x.get("X-DSPAM-Factors")
> print(type(h))
> #
>
> The output is the desired "str" when the message file contains this:
>
> To: recipi...@example.com
> Message-ID: <123>
> Date: Sun, 24 Jul 2022 15:31:19 +
> Subject: Blah blah
> From: f...@from.com
> X-DSPAM-Factors: a'b
>
> xxx
>
> ... but if the apostrophe in "a'b" is replaced with a
> RIGHT SINGLE QUOTATION MARK, the returned h is of type
> "email.header.Header", and seems to contain inscrutable garbage.
Include in any bug report the exact bytes that are in the header.
In may not be utf-8 encoded it maybe windows cp1252, etc.
Repr of the bytes header will show this.
Barry
>
> I realize that one should not put non-ASCII characters in
> message headers, but of course I didn't put it there, it
> just showed up, pretty much beyond my control. And I realize
> that when software is given input that breaks the rules, one
> cannot expect optimal results, but I'd think an exception
> would be the right answer.
>
> Is this worth a bug report?
>
> --
> To email me, substitute nowhere->runbox, invalid->com.
> --
> https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to generate a .pyi file for a C Extension using stubgen
> On 30 Jul 2022, at 10:30, Marco Sulla wrote: > > On Fri, 29 Jul 2022 at 23:23, Barry wrote: >> >> >> >>>> On 29 Jul 2022, at 19:33, Marco Sulla wrote: >>> >>> I tried to follow the instructions here: >>> >>> https://mypy.readthedocs.io/en/stable/stubgen.html >>> >>> but the instructions about creating a stub for a C Extension are a little >>> mysterious. I tried to use it on the .so file without luck. >> >> It says that stubgen works on .py files not .so files. >> You will need to write the .pyi for your .so manually. >> >> The docs could do with splitting the need for .pyi for .so >> away from the stubgen description > > But it says: > > "Mypy includes the stubgen tool that can automatically generate stub > files (.pyi files) for Python modules and C extension modules." > > I tried stubgen -m modulename, but it generates very little code. Oh… From the .so I am struggling to figure out how it could ever work reliably. I cannot see that there is enough information in a useful form to allow the tool to work. Barry > -- https://mail.python.org/mailman/listinfo/python-list
Re: Simple TCP proxy
> On 30 Jul 2022, at 20:33, Morten W. Petersen wrote: > I thought it was a bit much. > > I just did a bit more testing, and saw that the throughput of wget through > regular lighttpd was 1,3 GB/s, while through STP it was 122 MB/s, and using > quite a bit of CPU. > > Then I increased the buffer size 8-fold for reading and writing in run.py, > and the CPU usage went way down, and the transfer speed went up to 449 MB/s. You are trading latency for through put. > > So it would require well more than a gigabit network interface to max out > STP throughput; CPU usage was around 30-40% max, on one processor core. With how many connections? > > There is good enough, and then there's general practice and/or what is > regarded as an elegant solution. I'm looking for good enough, and in the > process I don't mind pushing the envelope on Python threading. You never did answer my query on why a large backlog is not good enough. Why do you need this program at all? Barry > > -Morten > > On Sat, Jul 30, 2022 at 12:59 PM Roel Schroeven > wrote: > >> Morten W. Petersen schreef op 29/07/2022 om 22:59: >>> OK, sounds like sunshine is getting the best of you. >> It has to be said: that is uncalled for. >> >> Chris gave you good advice, with the best of intentions. Sometimes we >> don't like good advice if it says something we don't like, but that's no >> reason to take it off on the messenger. >> >> -- >> "Iceland is the place you go to remind yourself that planet Earth is a >> machine... and that all organic life that has ever existed amounts to a >> greasy >> film that has survived on the exterior of that machine thanks to furious >> improvisation." >> -- Sam Hughes, Ra >> >> -- >> https://mail.python.org/mailman/listinfo/python-list > > > -- > I am https://leavingnorway.info > Videos at https://www.youtube.com/user/TheBlogologue > Twittering at http://twitter.com/blogologue > Blogging at http://blogologue.com > Playing music at https://soundcloud.com/morten-w-petersen > Also playing music and podcasting here: > http://www.mixcloud.com/morten-w-petersen/ > On Google+ here https://plus.google.com/107781930037068750156 > On Instagram at https://instagram.com/morphexx/ > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: PEP about recommended project folder layout
> On 30 Jul 2022, at 13:52, c.bu...@posteo.jp wrote: > > Isn't there a PEP? PEP are for improving python. They are not for telling people how to use python. I would be surprised to fine a PEP that addressed this. Barry > >> On 2022-07-26 07:14 c.bu...@posteo.jp wrote: >> Hello, >> >> I am not sure if I looked into the correct sources. I was looking in >> "PEP 609 – Python Packaging Authority (PyPA) Governance" [1] and the >> "PyPA specifications" [2]. >> >> My question in short: Is there an official document (e.g. a PEP) >> about a recommended layout for project folders. >> >> Looking into the wild and past there are a lot of variations of such >> layouts. I am far away from being a pro but depending on experience >> in my own projects and what I have learned from others (e.g. in >> blog-posts/tutorials) I recommend to have the "test" folder and the >> package folder side by side on the same level in the project folder >> (the root). >> >> my_project >> |- tests >> | └ test_*.py >> |- my_package >> | └ __init__.py >> └-- README.md >> >> I sometimes add to it the so called "src"-Layout where the package >> folder is one level deeper in an extra "src" folder. >> >> my_project >> |- tests >> | └ test_*.py >> |- src >> | └- my_package >> | └ __init__.py >> └-- README.md >> >> I don't want to discuss the pros and cons of all variations. What I >> need is an official document I can use in discussions with other >> maintainers. If there is a PEP/document against my current >> recommendation I am also fine with this. ;) >> >> Kind >> Christian >> >> [1] -- <https://peps.python.org/pep-0609/> >> [2] -- <https://packaging.python.org/en/latest/specifications> > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Which linux distro is more conducive for learning the Python programming language?
> On 4 Aug 2022, at 09:48, Turritopsis Dohrnii Teo En Ming > wrote: > > On Thu, 4 Aug 2022 at 13:02, Kushal Kumaran wrote: >> >>> On Thu, Aug 04 2022 at 10:22:41 AM, Turritopsis Dohrnii Teo En Ming >>> wrote: >>> Subject: Which linux distro is more conducive for learning the Python >>> programming language? >>> >>> Good day from Singapore, >>> >>> May I know which linux distro is more conducive for learning the >>> Python programming language? >>> >>> Since I have absolutely and totally FREE RHEL developer subscription >>> (I don't need to spend a single cent), can I use Red Hat Enterprise >>> Linux version 9.0 to learn Python? >>> >>> Is it the most popular linux distro for learning Python? >>> >>> I just want to know which linux distro and version is more conducive >>> for learning Python. Because there are thousands of linux distros out >>> there. And I just want to settle down on a particular linux distro and >>> version. >>> >> >> The best one would be whatever you happen to have installed and for >> which you understand system administration. Beyond that, distribution >> choice matters very little. Every distribution I've used ships python3 >> packages, which was fine for learning the language. >> >> -- >> regards, >> kushal > > Noted with thanks Kushal. Since I can download FREE copies of RHEL > 9.0, I will use it then. I consider rhel 9 is an old os. I would suggest using fedora over rhel. Fedora 36 has python 3.10 and the when fedora 37 is released it will have python 3.11. And fedora is free as well. Barry > > Mr. Turritopsis Dohrnii Teo En Ming > Targeted Individual in Singapore > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Parallel(?) programming with python
> On 8 Aug 2022, at 20:24, MRAB wrote:
>
> On 2022-08-08 12:20, Stefan Ram wrote:
>> Andreas Croci writes:
>>> Basically the question boils down to wether it is possible to have parts of
>>> a program (could be functions) that keep doing their job while other parts
>>> do something else on the same data, and what is the best way to do this.
>> Yes, but this is difficult. If you ask this question here,
>> you might not be ready for this.
>> I haven't learned it yet myself, but nevertheless tried to
>> write a small example program quickly, which might still
>> contain errors because of my lack of education.
>> import threading
>> import time
>> def write_to_list( list, lock, event ):
>> for i in range( 10 ):
>> lock.acquire()
>> try:
>> list.append( i )
>> finally:
>> lock.release()
>> event.set()
>> time.sleep( 3 )
>> def read_from_list( list, lock, event ):
>> while True:
>> event.wait()
>> print( "Waking up." )
>> event.clear()
>> if len( list ):
>> print( "List contains " + str( list[ 0 ]) + "." )
>> lock.acquire()
>> try:
>> del list[ 0 ]
>> finally:
>> lock.release()
>> else:
>> print( "List is empty." )
>> list = []
>> lock = threading.Lock()
>> event = threading.Event()
>> threading.Thread( target=write_to_list, args=[ list, lock, event ]).start()
>> threading.Thread( target=read_from_list, args=[ list, lock, event ]).start()
>> In basketball, first you must learn to dribble and pass,
>> before you can begin to shoot.
>> With certain reservations, texts that can be considered
>> to learn Python are:
>> "Object-Oriented Programming in Python Documentation" - a PDF file,
>> Introduction to Programming Using Python - Y Daniel Liang (2013),
>> How to Think Like a Computer Scientist - Peter Wentworth (2012-08-12),
>> The Coder's Apprentice - Pieter Spronck (2016-09-21), and
>> Python Programming - John Zelle (2009).
> When working with threads, you should use queues, not lists, because queues
> do their own locking and can wait for items to arrive, with a timeout, if
> desired:
Lists do not need to be locked in python because of the GIL.
However you need locks to synchronise between threads.
And as you say a queue has all that locking built in.
Barry
>
>
> import queue
> import threading
> import time
>
> def write_to_item_queue(item_queue):
>for i in range(10):
>print("Put", i, "in queue.", flush=True)
>item_queue.put(i)
>time.sleep(3)
>
># Using None to indicate that there's no more to come.
>item_queue.put(None)
>
> def read_from_item_queue(item_queue):
>while True:
>try:
>item = item_queue.get()
>except item_queue.Empty:
>print("Queue is empty; should've have got here!", flush=True)
>else:
>print("Queue contains " + str(item) + ".", flush=True)
>
>if item is None:
># Using None to indicate that there's no more to come.
>break
>
> item_queue = queue.Queue()
>
> write_thread = threading.Thread(target=write_to_item_queue, args=[item_queue])
> write_thread.start()
>
> read_thread = threading.Thread(target=read_from_item_queue, args=[item_queue])
> read_thread.start()
>
> # Wait for the threads to finish.
> write_thread.join()
> read_thread.join()
>
> print("Finished.")
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Persistent Error: Python was not found
> On 15 Aug 2022, at 04:10, Jonathan Owah wrote: > > Good day, > Great job on making Python easily accessible. Try using the python launcher py.exe. It is documented here https://docs.python.org/3/using/windows.html#python-launcher-for-windows That page documents lots of other things that you may need to know about on windows. Barry > > I'm using a Windows 10, 64gb HP EliteBook. > > I've been trying to configure my laptop to run python scripts. > This is the error I keep getting: > Python was not found; run without arguments to install from the Microsoft > Store, or disable this shortcut from Settings > Manage App Execution > Aliases. > > Everything I've tried has failed. > I've uninstalled and reinstalled > I've added to path, both user and system path,manually and from fresh > installation > I've downloaded from Microsoft Store > I've gone to manage app aliases and switched off > I've used Git Bash, Powershell, cmd > > I'm able to check my python version: 3.10.6. > > I can't do anything else and it's really frustrating. > > I've been at it for days, I don't know what else to do. > > Thanks in advance for your help. > > Regards > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: UTF-8 and latin1
> On 17 Aug 2022, at 18:30, Jon Ribbens via Python-list
> wrote:
>
> On 2022-08-17, Tobiah wrote:
>> I get data from various sources; client emails, spreadsheets, and
>> data from web applications. I find that I can do
>> some_string.decode('latin1')
>> to get unicode that I can use with xlsxwriter,
>> or put in the header of a web page to display
>> European characters correctly. But normally UTF-8 is recommended as
>> the encoding to use today. latin1 works correctly more often when I
>> am using data from the wild. It's frustrating that I have to play
>> a guessing game to figure out how to use incoming text. I'm just wondering
>> if there are any thoughts. What if we just globally decided to use utf-8?
>> Could that ever happen?
>
> That has already been decided, as much as it ever can be. UTF-8 is
> essentially always the correct encoding to use on output, and almost
> always the correct encoding to assume on input absent any explicit
> indication of another encoding. (e.g. the HTML "standard" says that
> all HTML files must be UTF-8.)
>
> If you are finding that your specific sources are often encoded with
> latin-1 instead then you could always try something like:
>
>try:
>text = data.decode('utf-8')
>except UnicodeDecodeError:
>text = data.decode('latin-1')
>
> (I think latin-1 text will almost always fail to be decoded as utf-8,
> so this would work fairly reliably assuming those are the only two
> encodings you see.)
Only if a reserved byte is used in the string.
It will often work in either.
For web pages it cannot be assumed that markup saying it’s utf-8 is
correct. Many pages are I fact cp1252. Usually you find out because
of a smart quote that is 0xa0 is cp1252 and illegal in utf-8.
Barry
>
> Or you could use something fancy like https://pypi.org/project/chardet/
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Mutating an HTML file with BeautifulSoup
> On 19 Aug 2022, at 19:33, Chris Angelico wrote: > > What's the best way to precisely reconstruct an HTML file after > parsing it with BeautifulSoup? I recall that in bs4 it parses into an object tree and loses the detail of the input. I recently ported from very old bs to bs4 and hit the same issue. So no it will not output the same as went in. If you can trust the input to be parsed as xml, meaning all the rules of closing tags have been followed. Then I think you can parse and unparse thru xml to do what you want. Barry > > Using the Alice example from the BS4 docs: > >>>> html_doc = """The Dormouse's story > > The Dormouse's story > > Once upon a time there were three little sisters; and > their names were > http://example.com/elsie"; class="sister" id="link1">Elsie, > http://example.com/lacie"; class="sister" id="link2">Lacie and > http://example.com/tillie"; class="sister" id="link3">Tillie; > and they lived at the bottom of a well. > > ... > """ >>>> print(soup) > The Dormouse's story > > The Dormouse's story > Once upon a time there were three little sisters; and > their names were > http://example.com/elsie"; id="link1">Elsie, > http://example.com/lacie"; id="link2">Lacie and > http://example.com/tillie"; id="link3">Tillie; > and they lived at the bottom of a well. > ... > >>>> > > Note two distinct changes: firstly, whitespace has been removed, and > secondly, attributes are reordered (I think alphabetically). There are > other canonicalizations being done, too. > > I'm trying to make some automated changes to a huge number of HTML > files, with minimal diffs so they're easy to validate. That means that > spurious changes like these are very much unwanted. Is there a way to > get BS4 to reconstruct the original precisely? > > The mutation itself would be things like finding an anchor tag and > changing its href attribute. Fairly simple changes, but might alter > the length of the file (eg changing "http://example.com/"; into > "https://example.com/";). I'd like to do them intelligently rather than > falling back on element.sourceline and element.sourcepos, but worst > case, that's what I'll have to do (which would be fiddly). > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Python scripts in .exe form
> On 20 Aug 2022, at 14:28, Jim Schwartz wrote: > > What method did you use to create the exe file from your python scripts? If > it was pyinstaller, then it puts the compiled versions of these python > scripts in a windows temp folder when you run them. You’ll be able to get the > scripts from there. The temp file is only for .dll files the python code is in a data block that is appended to the .exe stub. There are tools that can grab the appended dat and dump it out. Or atleast should be. Barry > > Sent from my iPhone > >> On Aug 19, 2022, at 9:51 PM, Mona Lee wrote: >> >> I'm pretty new to Python, and I had to do some tinkering because I was >> running into issues with trying to download a package from PIP and must've >> caused some issues in my program that I don't know how to fix >> >> 1. It started when I was unable to update PIP to the newest version because >> of some "Unknown error" (VS Code error - unable to read file - >> (Unknown(FileSystemError) where I believe some file was not saved in the >> right location? >> >> 2. In my command line on VS code there used to be the prefix that looked >> something like "PS C:\Users\[name]>" but now it is "PS >> C:\Users\[name]\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\Scripts> >> >> From there I redownloaded my VS code but still have the 2) issue. >> >> also, my scripts are now in the .exe form that I cannot access because "it >> is either binary or in a unsupported text encoding" I've tried to extract it >> back into the .py form using pyinstxtractor and decompile-python3 but I >> can't successfully work these. >> >> 3. also wanted to mention that some of my old Python programs are missing. >> -- >> https://mail.python.org/mailman/listinfo/python-list > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Mutating an HTML file with BeautifulSoup
> On 19 Aug 2022, at 22:04, Chris Angelico wrote: > > On Sat, 20 Aug 2022 at 05:12, Barry wrote: >> >> >> >>>> On 19 Aug 2022, at 19:33, Chris Angelico wrote: >>> >>> What's the best way to precisely reconstruct an HTML file after >>> parsing it with BeautifulSoup? >> >> I recall that in bs4 it parses into an object tree and loses the detail of >> the input. >> I recently ported from very old bs to bs4 and hit the same issue. >> So no it will not output the same as went in. >> >> If you can trust the input to be parsed as xml, meaning all the rules of >> closing >> tags have been followed. Then I think you can parse and unparse thru xml to >> do what you want. >> > > > Yeah, no I can't, this is HTML 4 with a ton of inconsistencies. Oh > well. Thanks for trying, anyhow. > > So I'm left with a few options: > > 1) Give up on validation, give up on verification, and just run this > thing on the production site with my fingers crossed Can you build a beta site with original intack? Also wonder if using selenium to walk the site may work as a verification step? I cannot recall if you can get an image of the browser window to do image compares with to look for rendering differences. From my one task using bs4 I did not see it produce any bad results. In my case the problems where in the code that built on bs1 using bad assumptions. > 2) Instead of doing an intelligent reconstruction, just str.replace() > one URL with another within the file > 3) Split the file into lines, find the Nth line (elem.sourceline) and > str.replace that line only > 4) Attempt to use elem.sourceline and elem.sourcepos to find the start > of the tag, manually find the end, and replace one tag with the > reconstructed form. > > I'm inclined to the first option, honestly. The others just seem like > hard work, and I became a programmer so I could be lazy... > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Mutating an HTML file with BeautifulSoup
> On 21 Aug 2022, at 09:12, Chris Angelico wrote: > > On Sun, 21 Aug 2022 at 17:26, Barry wrote: >> >> >> >>>> On 19 Aug 2022, at 22:04, Chris Angelico wrote: >>> >>> On Sat, 20 Aug 2022 at 05:12, Barry wrote: >>>> >>>> >>>> >>>>>> On 19 Aug 2022, at 19:33, Chris Angelico wrote: >>>>> >>>>> What's the best way to precisely reconstruct an HTML file after >>>>> parsing it with BeautifulSoup? >>>> >>>> I recall that in bs4 it parses into an object tree and loses the detail of >>>> the input. >>>> I recently ported from very old bs to bs4 and hit the same issue. >>>> So no it will not output the same as went in. >>>> >>>> If you can trust the input to be parsed as xml, meaning all the rules of >>>> closing >>>> tags have been followed. Then I think you can parse and unparse thru xml to >>>> do what you want. >>>> >>> >>> >>> Yeah, no I can't, this is HTML 4 with a ton of inconsistencies. Oh >>> well. Thanks for trying, anyhow. >>> >>> So I'm left with a few options: >>> >>> 1) Give up on validation, give up on verification, and just run this >>> thing on the production site with my fingers crossed >> >> Can you build a beta site with original intack? > > In a naive way, a full copy would be quite a few gigabytes. I could > cut that down a good bit by taking only HTML files and the things they > reference, but then we run into the same problem of broken links, > which is what we're here to solve in the first place. > > But I would certainly not want to run two copies of the site and then > manually compare. > >> Also wonder if using selenium to walk the site may work as a verification >> step? >> I cannot recall if you can get an image of the browser window to do image >> compares with to look for rendering differences. > > Image recognition won't necessarily even be valid; some of the changes > will have visual consequences (eg a broken image reference now > becoming correct), and as soon as that happens, the whole document can > reflow. > >> From my one task using bs4 I did not see it produce any bad results. >> In my case the problems where in the code that built on bs1 using bad >> assumptions. > > Did that get run on perfect HTML, or on messy real-world stuff that > uses quirks mode? I small number of messy html pages. Barry > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: ImportError: No module named spambayes.resources (line 12 of setup_all.py)
> On 24 Aug 2022, at 23:25, Mats Wichmann wrote: > > On 8/24/22 13:54, Erik M. Brown via Python-list wrote: >> Is anyone here familiar with SpamBayes? I am working with legacy 2.4.x >> Python code (I'm not a programmer, more of a power user) and I'm attempting >> to build the windows binary from the SpamBayes source. >> >> >> >> I'm running into an error when attempting to run "setup_all.py", the py2exe >> setup script, and I get the following: >> >> >> >> "ImportError: No module named spambayes.resources (line 12 of setup_all.py)" > > > Wow, that's a blast from the past... I wouldn't hold out too much hope, > we're talking about something that's been on the inactive list for a > decade and a half at least, if memory serves. Python 2.4 itself dates > to 2004. There does seem to be have been some effort to uplift Spambayes > to Python 3: > > https://github.com/mpwillson/spambayes3 > > but in generic terms: "no module named" is always a path problem. The > Python that is running and emits that error is looking in a place where > the code it's trying to import... isn't. If you got it to work in some > scenario, you need to compare the version of Python involved in that > effort, and its paths (sys.path value) vs. what's getting run when you > try to execute the thing that's failing. On fedora I still use spambayes and it works great, but I have it running under python 2.7. never did take on a port to python 3 I will have to consider that when python 2 finally is removed from fedora, or use something else. Barry > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Ref-strings in logging messages (was: Performance issue with CPython 3.10 + Cython)
> On 7 Oct 2022, at 16:48, Skip Montanaro wrote: > > On Fri, Oct 7, 2022 at 9:42 AM Andreas Ames > wrote: > >> 1. The culprit was me. As lazy as I am, I have used f-strings all over the >> place in calls to `logging.logger.debug()` and friends, evaluating all >> arguments regardless of whether the logger was enabled or not. >> > > I thought there was some discussion about whether and how to efficiently > admit f-strings to the logging package. I'm guessing that's not gone > anywhere (yet). That cannot be done as the f-string is computed before the log call. Maybe you are thinking of the lazy expression idea for this. That idea seems to have got no where as its not clear how to implement it without performance issues. Barry > > Skip > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Ref-strings in logging messages (was: Performance issue with CPython 3.10 + Cython)
> On 7 Oct 2022, at 18:16, MRAB wrote: > > On 2022-10-07 16:45, Skip Montanaro wrote: >>> On Fri, Oct 7, 2022 at 9:42 AM Andreas Ames >>> wrote: >>> 1. The culprit was me. As lazy as I am, I have used f-strings all over the >>> place in calls to `logging.logger.debug()` and friends, evaluating all >>> arguments regardless of whether the logger was enabled or not. >>> >> I thought there was some discussion about whether and how to efficiently >> admit f-strings to the logging package. I'm guessing that's not gone >> anywhere (yet). > Letting you pass in a callable to call might help because that you could use > lambda. Yep, that’s the obvious way to avoid expensive log data generation. Would need logging module to support that use case. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Ref-strings in logging messages (was: Performance issue with CPython 3.10 + Cython)
> On 7 Oct 2022, at 19:09, Weatherby,Gerard wrote: > The obvious way to avoid log generation is: > > if logger.isEnableFor(logging.DEBUG): >logger.debug( expensive processing ) > > > Of course, having logging alter program flow could lead to hard to debug bugs. Altered flow is less of an issue the the verbosity of the above. We discussed ways to improve this pattern a few years ago. That lead to no changes. What I have used is a class that defines __bool__ to report if logging is enabled and __call__ to log. Then you can do this: log_debug = logger_from(DEBUG) log_debug and log_debug(‘expensive %s’ % (complex(),)) Barry > > From: Python-list on > behalf of Barry > Date: Friday, October 7, 2022 at 1:30 PM > To: MRAB > Cc: python-list@python.org > Subject: Re: Ref-strings in logging messages (was: Performance issue with > CPython 3.10 + Cython) > *** Attention: This is an external email. Use caution responding, opening > attachments or clicking on links. *** > >> On 7 Oct 2022, at 18:16, MRAB wrote: >> >> On 2022-10-07 16:45, Skip Montanaro wrote: >>>> On Fri, Oct 7, 2022 at 9:42 AM Andreas Ames >>>> wrote: >>>> 1. The culprit was me. As lazy as I am, I have used f-strings all over the >>>> place in calls to `logging.logger.debug()` and friends, evaluating all >>>> arguments regardless of whether the logger was enabled or not. >>> I thought there was some discussion about whether and how to efficiently >>> admit f-strings to the logging package. I'm guessing that's not gone >>> anywhere (yet). >> Letting you pass in a callable to call might help because that you could use >> lambda. > > Yep, that’s the obvious way to avoid expensive log data generation. > Would need logging module to support that use case. > > Barry > >> -- >> https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mrESxAj9YCHsdtNAfkNiY-Zf6U3WTIqaNrgBmbw1ELlQy51ilob43dD0ONsqvg4a94MEdOdwomgyqfyABbvRnA$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mrESxAj9YCHsdtNAfkNiY-Zf6U3WTIqaNrgBmbw1ELlQy51ilob43dD0ONsqvg4a94MEdOdwomgyqfyABbvRnA$> > > -- > https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mrESxAj9YCHsdtNAfkNiY-Zf6U3WTIqaNrgBmbw1ELlQy51ilob43dD0ONsqvg4a94MEdOdwomgyqfyABbvRnA$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mrESxAj9YCHsdtNAfkNiY-Zf6U3WTIqaNrgBmbw1ELlQy51ilob43dD0ONsqvg4a94MEdOdwomgyqfyABbvRnA$> > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use for finding as many syntax errors as possible.
> On 9 Oct 2022, at 18:54, Antoon Pardon wrote: > > > > Op 9/10/2022 om 19:23 schreef Karsten Hilbert: >> Am Sun, Oct 09, 2022 at 06:59:36PM +0200 schrieb Antoon Pardon: >> >>> Op 9/10/2022 om 17:49 schreef Avi Gross: >>>> My guess is that finding 100 errors might turn out to be misleading. If you >>>> fix just the first, many others would go away. >>> At this moment I would prefer a tool that reported 100 errors, which would >>> allow me to easily correct 10 real errors, over the python strategy which >>> quits >>> after having found one syntax error. >> But the point is: you can't (there is no way to) be sure the >> 9+ errors really are errors. >> >> Unless you further constrict what sorts of errors you are >> looking for and what margin of error or leeway for false >> positives you want to allow. > > Look when I was at the university we had to program in Pascal and > the compilor we used continued parsing until the end. Sure there > were times that after a number of reported errors the number of > false positives became so high it was useless trying to find the > remaining true ones, but it still was more efficient to correct the > obvious ones, than to only correct the first one. If it’s very fast to syntax check then one at a time is fine. Python is very fast to syntax check so I personal do not need the multi error version. My editor has syntax check on a key and it’s instant to drop me a syntax error. Barry > > I don't need to be sure. Even the occasional wrong correction > is probably still more efficient than quiting after the first > syntax error. > > -- > Antoon. > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Find the path of a shell command [POSTPONED]
> On 16 Oct 2022, at 04:53, Dan Stromberg wrote: > > On Wed, Oct 12, 2022 at 9:57 PM Cameron Simpson wrote: > >>> On 13Oct2022 03:25, Paulo da Silva >>> wrote: >>> There is another problem involved. The script, works fine except when >>> launched by cron! Why? >> >> Record the script output: >> >> # record all output >> exec >/tmp/script.$$.out 2>&1 >> # dump the envionment >> env | sort >> # turn on execution tracing >> set -x >> ... rest of the script >> >> and have a look afterwards. Cron's environment is very minimal. This >> will show you what's in it. >> > > Careful. On some systems if someone restarts the cron daemon, it could > pick up a larger environment than after being started on boot. That have to a old system that does not use systemd. Is there a specific system that still does this? Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: str.replace() when str contains \
> On 29 Oct 2022, at 22:14, Bernard LEDRU wrote:
>
> Hello,
>
> https://docs.python.org/3/library/stdtypes.html#string-methods
>
> str.replace(old, new[, count])¶
> Return a copy of the string with all occurrences of substring old replaced by
> new. If the optional argument count is given, only the first count
> occurrences are replaced.
>
> Attention when the string contains the escape character.
>
> Consider :
>
>>>> a="H:\2023"; print(a.replace("\\","/"))
> H:3
>>>> a="H:\a2023"; print(a.replace("\\","/"))
> H:2023
>>>> a="H:\_2023"; print(a.replace("\\","/"))
> H:/_2023
String a does not quote the \ after :.
You need “H:\\2023” etc. check what a is before the replace with a
print(repr(a))
Barry
>
> Best regards,
> Bernard LEDRU
> --
> https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to manage python shebang on mixed systems?
> On 7 Nov 2022, at 03:15, Mike Dewhirst wrote: > > On 7/11/2022 6:51 am, jak wrote: >> Il 06/11/2022 11:03, Chris Green ha scritto: >>> I have a number of python scripts that I run on a mix of systems. I >>> have updated them all to run on python 3 but many will also run quite >>> happily with python 2. They all have a #!/usr/bin/python3 shebang. >>> This works almost everywhere but there is one system where only >>> python 2 is available (at /usr/bin/python). >>> I don't have python 2 on any of the systems I manage myself now so a >>> #!/usr/bin/python shebang will fail. >>> Is there a neat way of handling this? I could write a sort of wrapper >>> script to run via the shebang but that seems overkill to me. > > Can you link the interpreter on each system to the same-named local link and > put that in your shebang? > > #!~/scripts/mypython I do not think ~ works in a #! line. The ~ is handled by the shell, like bash. But the #! Is handled by the kernel in it exec() handling I recall. Using /usr/local/bin may be more suitable. Otherwise you are forced to use the /usr/bin/env method and ensure the PATH includes the command to run. Barry > >> hi, >> If you can call Python from the shell prompt, then you could remove the >> path from shebang: >> #!python > > > -- > Signed email is an absolute defence against phishing. This email has > been signed with my private key. If you import my public key you can > automatically decrypt my signature and be sure it came from me. Just > ask and I'll send it to you. Your email software can handle signing. > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.7+ cannot print unicode characters when output is redirected to file - is this a bug?
> On 13 Nov 2022, at 14:52, Jessica Smith <12jessicasmit...@gmail.com> wrote:
>
> Consider the following code ran in Powershell or cmd.exe:
>
> $ python -c "print('└')"
> └
>
> $ python -c "print('└')" > test_file.txt
> Traceback (most recent call last):
> File "", line 1, in
> File "C:\Program Files\Python38\lib\encodings\cp1252.py", line 19, in encode
>return codecs.charmap_encode(input,self.errors,encoding_table)[0]
> UnicodeEncodeError: 'charmap' codec can't encode character '\u2514' in
> position 0: character maps to
>
> Is this a known limitation of Windows + Unicode? I understand that
> using -x utf8 would fix this, or modifying various environment
> variables. But is this expected for a standard Python installation on
> Windows?
Your other thread has a reply that explained this.
It is a problem with windows and character sets.
You have to set things up to allow Unicode to work.
Barry
>
> Jessica
> --
> https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are these good ideas?
> On 14 Nov 2022, at 22:06, Thomas Passin wrote: > > For parameter passing like your #2, I have packaged them into a dictionary > and passed that around. It was easy enough, and worked well. > I used to use a dict but having been burnt with issues now create a class. With a class you can add accessor methods that allow you to run debug code as values are changed. Also you can check that values being set are spelt correctly, have reasonable values etc. Barry > The only potential problem is in documenting the key/value pairs the > dictionary is supposed to contain. You had better make sure it's made clear > somewhere, preferable where it is consumed. > > For "global" values, sure, put them in a module and import them as needed. > Just make sure that none of your code is going to mutate those values, or you > will end up with mass confusion. I suppose you could make them be properties > that have no setters, if you want to go to that trouble. > >> On 11/14/2022 12:14 PM, Stephen Tucker wrote: >> Hi, >> I have two related issues I'd like comments on. >> Issue 1 - Global Values >> Some years ago, I had a situation where >> (a) I could supply low-level functions that carry out tasks, >> (b) I needed those functions to communicate with each other, but >> (c) I had no access to the module that invoked my functions. >> In order to achieve this, I hit on the idea that I also supply a module >> that I describe as a "global values" module. This module … >> (i) … defines the values that the functions use to communicate with each >> other; >> (ii) … is the subject of an import statement in each of my functions; >> (iii) … initialises its values at the start of each run (since the import >> only carries out an actual import once per session); >> (iv) … acts as a repository for the values thereafter. >> This solution works well. >> Given that I am not particularly concerned about efficiency, >> (1) Is this a reasonable way to achieve this goal? >> (2) Do you know of any better ways? >> Issue 2 - Passed Parameters >> I am now facing another situation where I am wanting to pass 6 or 7 >> parameters down through several layers of logic (function A calling >> function B calling ... ) and for results to be passed back. >> Having had the idea described above, I am considering using it again to >> save all the parameter-and-results passing. >> I see nothing wrong with doing that, but I may well be missing something! >> Comments, please! >> Stephen Tucker. > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Debugging Python C extensions with GDB
> On 14 Nov 2022, at 19:10, Jen Kris via Python-list > wrote: > > In September 2021, Victor Stinner wrote “Debugging Python C extensions with > GDB” > (https://developers.redhat.com/articles/2021/09/08/debugging-python-c-extensions-gdb#getting_started_with_python_3_9). > > > My question is: with Python 3.9+, can I debug into a C extension written in > pure C and called from ctypes -- that is not written using the C_API? Yes. Just put a breakpoint on the function in the c library that you want to debug. You can set the breakpoint before a .so is loaded. Barry > > Thanks. > > Jen > > > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Are these good ideas?
> On 14 Nov 2022, at 22:49, Chris Angelico wrote: > > On Tue, 15 Nov 2022 at 09:38, Barry wrote: >> >> >> >>>> On 14 Nov 2022, at 22:06, Thomas Passin wrote: >>> >>> For parameter passing like your #2, I have packaged them into a dictionary >>> and passed that around. It was easy enough, and worked well. >>> >> I used to use a dict but having been burnt with issues now create a class. >> >> With a class you can add accessor methods that allow you to run debug code >> as values are changed. Also you can check that values being set are spelt >> correctly, have reasonable values etc. >> > > TBH you could add accessor methods to a dict just as easily Its the dict interface that we found allowed for API abuse and lead to hard to fix bugs. Barry > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Debugging Python C extensions with GDB
> On 14 Nov 2022, at 23:44, Jen Kris wrote: > > > Thanks for your reply. Victor's article didn't mention ctypes extensions, so > I wanted to post a question before I build from source. Gdb works on any program its not special to python. Victor is only talking about a specific use of gdb for python c extensions. Barry > > > Nov 14, 2022, 14:32 by ba...@barrys-emacs.org: > > On 14 Nov 2022, at 19:10, Jen Kris via Python-list > wrote: > > In September 2021, Victor Stinner wrote “Debugging Python C extensions with > GDB” > (https://developers.redhat.com/articles/2021/09/08/debugging-python-c-extensions-gdb#getting_started_with_python_3_9). > > > My question is: with Python 3.9+, can I debug into a C extension written in > pure C and called from ctypes -- that is not written using the C_API? > > Yes. > > Just put a breakpoint on the function in the c library that you want to debug. > You can set the breakpoint before a .so is loaded. > > Barry > > Thanks. > > Jen > > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Passing information between modules
> On 21 Nov 2022, at 21:23, r...@zedat.fu-berlin.de wrote:
>
> dn writes:
>> Now, at the config stage, take the instructions to define whichever the
>> user prefers, and instantiate that class. Then the 'calling-routine' can
>> use the instantiated object as an interface to whichever type of output.
>
> I had many different functions that are supposed to take
> a "context" argument. If I would make them all methods of
> a class with a "context" field, that would be a huge class
> containing methods with many different purposes, which
> reminds of the "anti pattern" "God class".
Do you haves lots of free standing functions? Are all these functions
not part of classes?
>
>> with Environment() as env:
>># processing
>> There would be no need to explicitly prevent any 'processing' if the
>> set-up doesn't work, because that context manager class will handle it all!
>
> Yes, but my problem was not so much with setting up the env,
> but with passing it to many library functions.
Are you writing procedural code or object oriented? Why do you have so
many functions outside of a small number of classes?
>
>> Is this how you implement?
>
> I'm not sure whether I understand the meaning of this question.
You appear not to be doing object oriented design.
>
> My library had a "console context" (before I started to use
> the Python logging facility instead). That console context was
> the default for output of progress and error messages.
>
> A client had the possibility to call functions with a custom
> context, and then the function would use this custom context
> for progress and error messages. The custom context could
> direct those message to a text field in a GUI, for example.
>
> Instead of passing this custom context to many library
> functions, it might be simpler to "pass it to the library
> once". This could be done via:
>
> import library
> library.context = my_GUI_context
That is not oo design. I get the feeling that you need a better understanding
of how to structure a non-trivia library.
If you need the context object then you must pass it around.
>
> , but I wonder whether this "writing into another module"
> is really possible in every Python implementation and whether
> it will still be supported in the future. I do not see such
> a pattern being used with the standard packages and suppose
> that there might be a reason for this!
>
> The same question applies to the more advanced technique of
> using "importlib.util.find_spec", "importlib.util.module_from_spec",
> and ".__spec__.loader.exec_module" to even support having
> different instances of a single module with different globals.
>
> I can now formulate my question this way:
>
> Many functions of a library have in their source code calls
> like "PRINT( f'Done. Output was written to {filename}.' )".
> The library uses "print" as a default for "PRINT", but
> should explicitly support clients substituting a custom
> implementation to be used for "PRINT". What's the best way
> for the client to "pass" his custom implementation to the
> library (which is a package or a module)?
Each place you have PRINT you need to have context.print calls.
You said above that you have, or had, such an object - pass it around and use
it.
If passing it around is the problem then you need to look at why you code
has that problem.
Barry
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.11.0 installation and Tkinter does not work
> On 22 Nov 2022, at 16:09, r...@zedat.fu-berlin.de wrote: > > darkst...@o2online.de writes: >>> ImportError: DLL load failed while importing _tkinter: Das angegebene >>> Modul wurde nicht gefunden. > > If you have not already done so, make sure that you install > Python from python.org. > > Then, after the installation, you also should make sure that > you actually use this installed version. > > import sys > sys.version_info > import tkinter > > It is possible that you have installed a different version > of Python before and accidentally have opened that version, > which may not include tkinter. In which case the error is module not found. The error reported suggests that a dll that _tkinter needs is missing. Barry > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Calling pselect/ppoll/epoll_pwait
> On 2 Dec 2022, at 20:03, Ian Pilcher wrote: > > Does Python provide any way to call the "p" variants of the I/O > multiplexing functions? > > Looking at the documentation of the select[1] and selectors[2] modules, > it appears that they expose only the "non-p" variants. > > [1] https://docs.python.org/3/library/select.html > [2] https://docs.python.org/3/library/selectors.html Can you use signalfd and select/poll/epoll? Barry > > -- > > Google Where SkyNet meets Idiocracy > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Calling pselect/ppoll/epoll_pwait
On 3 Dec 2022, at 13:13, Weatherby,Gerard wrote: Signalfd and select could probably be made to work if one codes around the race conditions pselect is provided to avoid. I’m not sure if using the GIL (for CPython) is sufficient or if a dedicated concurrency control (e.g. lock, mutex, semaphore) is necessary. An alternative is to call the C library function via a wrapper. I had need to use setfsuid on a project a couple of years ago and found this example: [1]https://gist.github.com/atdt/ebafa299e843a767139b I read this on SO when researching your question. Search for epoll and signal. (I would post the link but ipad turns the link into an image…) I assume that the lack of pepoll means you can use epoll and signalfd without race conditions. The trick seems to be setting the signal to ignore. Barry From: Python-list on behalf of Barry Date: Friday, December 2, 2022 at 7:02 PM To: Ian Pilcher Cc: python-list@python.org Subject: Re: Calling pselect/ppoll/epoll_pwait *** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. *** > On 2 Dec 2022, at 20:03, Ian Pilcher wrote: > > Does Python provide any way to call the "p" variants of the I/O > multiplexing functions? > > Looking at the documentation of the select[1] and selectors[2] modules, > it appears that they expose only the "non-p" variants. > > [1] [2]https://urldefense.com/v3/__https://docs.python.org/3/library/select.html__;!!Cn_UX_p3!hClFFo4XdiwQAhHxDbHA5zFr490Of9uheHSf84V9cREMHyw1kX-baG5HzXWMt-hFLP30q6DpSUb2G_OD5qBjeA$ > [2] [3]https://urldefense.com/v3/__https://docs.python.org/3/library/selectors.html__;!!Cn_UX_p3!hClFFo4XdiwQAhHxDbHA5zFr490Of9uheHSf84V9cREMHyw1kX-baG5HzXWMt-hFLP30q6DpSUb2G_MwNjgO8A$ Can you use signalfd and select/poll/epoll? Barry > > -- > > Google Where SkyNet meets Idiocracy > > -- > [4]https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!hClFFo4XdiwQAhHxDbHA5zFr490Of9uheHSf84V9cREMHyw1kX-baG5HzXWMt-hFLP30q6DpSUb2G_PhasKBfg$ > -- [5]https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!hClFFo4XdiwQAhHxDbHA5zFr490Of9uheHSf84V9cREMHyw1kX-baG5HzXWMt-hFLP30q6DpSUb2G_PhasKBfg$ References Visible links 1. https://gist.github.com/atdt/ebafa299e843a767139b 2. https://urldefense.com/v3/__https:/docs.python.org/3/library/select.html__;!!Cn_UX_p3!hClFFo4XdiwQAhHxDbHA5zFr490Of9uheHSf84V9cREMHyw1kX-baG5HzXWMt-hFLP30q6DpSUb2G_OD5qBjeA$ 3. https://urldefense.com/v3/__https:/docs.python.org/3/library/selectors.html__;!!Cn_UX_p3!hClFFo4XdiwQAhHxDbHA5zFr490Of9uheHSf84V9cREMHyw1kX-baG5HzXWMt-hFLP30q6DpSUb2G_MwNjgO8A$ 4. https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!hClFFo4XdiwQAhHxDbHA5zFr490Of9uheHSf84V9cREMHyw1kX-baG5HzXWMt-hFLP30q6DpSUb2G_PhasKBfg$ 5. https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!hClFFo4XdiwQAhHxDbHA5zFr490Of9uheHSf84V9cREMHyw1kX-baG5HzXWMt-hFLP30q6DpSUb2G_PhasKBfg$ -- https://mail.python.org/mailman/listinfo/python-list
Re: Contributing to cpython
> On 6 Dec 2022, at 20:55, ramvikram singh wrote: > > Greetings, > I learnt python this year and visiting the python issue tab for the last > two months, but there is a problem which I am facing I understand the issue > after reading the few times but I can't think about how to modify the code > as per the issue. Could you please help me with this? no one can help as you have not explained what the issue is. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: FTP without username and password
> On 7 Dec 2022, at 16:49, ^Bart wrote: > > >> It's a whole different protocol. TFTP is simplified to the point it >> will fit on embedded devices which don't need security (the assumption >> being that one has the embedded device physically present, FTP assumes >> distributed networks). >> https://wiki.python.org/moin/tftp > > I never used TFTP so, like what I wrote in another post, I thought it was > just a FTP without username and password... > > Thanks to show me the "Python way" to use TFTP! :) > > Have a nice day! > ^Bart TFTP server and client tools are standard on linux systems. Barry > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: pip/setuptools: Entry points not visible from pkexec-root-environment
> On 18 Dec 2022, at 17:57, c.bu...@posteo.jp wrote: > > Hello, > > when I install a package on a GNU/Linux system via "sudo python3 -m pip > install -e ." that defines entry points in its pyproject.toml the entry > point starter scripts are located in /usr/locale/bin. > > That folder is in PATH for "regular" root users and by "sudo su" roots > users. > > But I need to start that entry points via "pkexec". > But in the environment started by "pkexec" the PATH does not contain > /usr/local/bin. > > So what can I do? Why are asking on this list and discuss.python.org? Lots of people on this list are also on discuss.python.org. Barry > > I don't need a hack or workaround but an "elegant" solution. > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: How to enter escape character in a positional string argumentfrom the command line?
> On 21 Dec 2022, at 17:06, Chris Angelico wrote: > > On Thu, 22 Dec 2022 at 03:58, gene heskett wrote: >> >>> On 12/21/22 11:22, Chris Angelico wrote: >>> On Thu, 22 Dec 2022 at 03:11, Stefan Ram wrote: >>>> >>>> Lars Liedtke writes: >>>>> Or you could have "native" bash ($SHELL) with WSL. >>>> >>>> In this newsgroup, it would actually be obvious to use Python. >>> >>> Less obvious than you might think - partly because bash is just so >>> dang good that it's really really hard to outdo it :) Sure, bash has a >>> lot of weird and wonky edge cases, but it's an incredibly practical >>> shell to use. >>> >> When you make a statement like that, Chris, you should also note that >> every single one of those "wonky edge cases" is documented down to the >> last dotted i. Bash's docs will kill a good sized pulp tree, needing >> around a ream of paper to print on a duplex printer. I know, I did it >> around a decade ago. If you like to write scripts, having a dead tree >> copy of the docs at your elbow in incredibly useful. That huge man page >> does not cover it like the printed docs do. >> > > Oh yes, absolutely true. Its wonkiness is dependable and consistent; > but it is definitely quirky (look at all the different ways to embed > arguments into things, and the ways that $*, $@, "$*, and "$@" behave > when put into variables). Not usually a problem, but it does sometimes > leave you thinking "wow, wouldn't it be easier to just use something > like Python?". And in the complicated cases, yeah, it can be. But in > the simple cases? Bash rocks. I see bash scripts that are 1000’s of line of code at work and its a maintenance nightmare. Knowing when to make the move from “handy bash script” to “this is a production application” and needs to be python is what I see people miss. After a certain point in complexity the python code wins on maintenance. Personally i set a low bar to move from bash to python. Barry > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Pyserial problem
> On 22 Dec 2022, at 17:09, Patrick EGLOFF wrote: > > Hi all, > > I use Python 3.10.9 and Pyserial 3.5 on a Win10 machine. > > I'm sending datas via an USB port to a device that accept commands in the > form of : cmd; > The device receives and reacts to the commands sent, and it should reply > with an ACK of the same kind. > > But looking with a COM port sniffer, nothing is sent back. > > I checked that everything is working with PUTTY and even MINITERM, and > everything is just fine, the device is responding correctly. > > I have set the flow control to different values, as well as setting the RTS > and DTR high or low with no change. > Normally, the device works without any flow control, and CTS + DTR high. > > I checked with MINITERM, that the flow control and control lines have the > same state. > > I'm a bit surprised and stucked. > Can someone help ? Please post tour code that you are using to talk to the device. Barry > Thanks, > -- > Patrick Egloff > email : pegl...@gmail.com > Web page : http://www.egloff.eu > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Question.
> On 8 Jan 2023, at 21:20, Angitolol36 wrote: > > Hello, i installed phyton in Windows 10 22H2 and i can’t find the program. > I used the repair that doesnt work. Does this help? https://docs.python.org/3/using/windows.html Barry > > > > > > Enviado desde [1]Correo para Windows > > > > [2][IMG] Libre de virus.[3]www.avast.com > > References > > Visible links > 1. https://go.microsoft.com/fwlink/?LinkId=550986 > 2. > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient > 3. > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Suggestion: Regex string specifier like r and f
> On 8 Jan 2023, at 21:16, Raphael Santiago > wrote: > > Maybe something like re"" > It should behave exactly like a raw string but would be useful for syntax > highlighting and debugging. Perhaps also for type hinting expected regex > input (don't know if this is feasible). This is unlikely to be implemented. See https://discuss.python.org/t/allow-for-arbitrary-string-prefix-of-strings/19740/12 for related idea discussion. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: HTTP server benchmarking/load testing in Python
> On 26 Jan 2023, at 17:32, Thomas Passin wrote: > > On 1/26/2023 11:41 AM, Chris Angelico wrote: >>> On Fri, 27 Jan 2023 at 03:34, Thomas Passin wrote: >>> A nice theory but nothing to do with the real world. I've had a number >>> of laptops that overheat (or would, if I let test program continue) >>> running this test program. >> Define "overheat". If all you're saying is "the fan began to whine and >> I got annoyed so I shut off the program", that is absolutely NOT >> overheating. > > CPU core temperatures up to 95 deg C and rising rapidly, as reported by a > number of utilities including NZXT and CoreTemp. Max junction temperature is > given as 100 deg C, and I don't want to risk reducing the lifetime of my CPU. Silicon junctions melt something like 400C ish not 100C. The max you see is the operating temp of the CPU. For intel CPU if you go beyond what the slow clocking can deal with the CPU turns itself off to prevent damage. Intel did this to stop people asking for replacement parts when there cooling was at fault. Barry > > Maybe five or ten minutes at or above 100 deg C every few months might not > make a noticeable lifetime difference, who knows? I don't want to make a > habit of it. I wouldn't drive my car very long with a low oil pressure > warning active, either. > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Python: How to use the 'trace' module programmatically?
> On 15 Feb 2023, at 17:23, Peter Slížik wrote:
>
> Hello,
>
> I'm trying to analyze complex Python code. For some specific reasons, I
> decided to use tracing instead of a debugger.
>
> The first thing I tried was:
>
> python -m trace -t /path/to/file.py
>
> The output of this command turned out to be completely useless. The reason
> is that there was a thread running in the background, doing some work
> every *0.1
> s* and this generated thousands of lines of tracing information. The useful
> information (a reaction to my interaction with app GUI) scrolled away in a
> blink.
>
> For this reason, I decided to limit the scope of tracing. I did the
> following.
>
> The original code:
>
> def caller():
>print("I'm the caller.")
>callee()
> def callee():
>print("Callee here.")
>
> Code modified for tracing:
>
> import trace
>
> tracer = trace.Tracer(
>count=0,
>trace=1,
> )
> def outer():
>print("I'm the caller.")
>tracer.runfunc(inner)
The docs show that you need to do either add the outfile to trace
or generate and write the report after runfunc returns.
I have not tested this, just read the docs out of curiosity
Here https://docs.python.org/3/library/trace.html
Barry
> def inner():
>print("Callee here.")
>
> Now I launched the program and the tracer did not generate any output. I
> was hoping that this would provide complete tracing information, but only
> for the limited scope of inner().
>
> No success with tracer.run() either.
>
> What I was able to do, when I set count=1, I was able to catch the coverage
> data with tracer.results() and write them to a file. But the tracing
> information was not generated even in this case.
>
> Am I doing anything wrong?
>
> Peter
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Introspecting the variable bound to a function argument
please do not use an email address on a public list that cannot be replied to.
> On 22 Feb 2023, at 14:43, Anton Shepelev wrote:
>
> Hello, all.
>
> Does Python have an instrospection facility that can
> determine to which outer variable a function argument is
> bound, e.g.:
There is no requirement for a variable to be used in the call.
It could be an an int or string, 42 “forty two”.
>
> v1 = 5;
> v2 = 5;
>
> def f(a):
> print(black_magic(a)) # or black_magic('a')
>
> f(v1) # prints: v1
> f(v2) # prints: v2
There is the traceback module that lets you find where you are called from.
Also there is the inspect module that also lets tou get at the stack in more
detail.
Barry
>
> --
> () ascii ribbon campaign -- against html e-mail
> /\ www.asciiribbon.org -- against proprietary attachments
> --
> https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
> On 23 Feb 2023, at 01:39, Hen Hanna wrote: > > On Wednesday, February 22, 2023 at 3:46:21 PM UTC-8, Hen Hanna wrote: >> On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote: >>>> py bug.py >>> Traceback (most recent call last): >>> File "C:\Usenet\bug.py", line 5, in >>> print( a + 12 ) >>> TypeError: can only concatenate str (not "int") to str >>> >>> >>> Why doesn't Python (error msg) do the obvious thing and tell me >>> WHAT the actual (offending, arg) values are ? >>> >>> In many cases, it'd help to know what string the var A had , when the error >>> occurred. >>> i wouldn't have to put print(a) just above, to see. >>> >>> >>> >>> >>> ( pypy doesn't do that either, but Python makes programming (debugging) so >>> easy that i hardly feel any inconvenience.) > > > i see that my example would be (even) clearER with this one-line change: > > py bug.py > > Traceback (most recent call last): > > File "C:\Usenet\bug.py", line 5, in > map( Func, fooBar( X, Y, X + Y )) > > TypeError: can only concatenate str (not "int") to str >attempt to call + with 'abc' , 123.45 > <-- > >> i hope that NOW a few of you can see this as a genuine, (reasonable) >> question. > > Python seems so perfectly User-friendly that > i 'm so curious (puzzled) that it doesn't do the very > obvious and easy thing > of giving me this info: > >attempt to call + with 'abc' , > 123.45 <-- It is not easy to do that in a robust and reliable way for any object. You can end up in the code to generate the error message itself breaking. For example using unbounded CPU time when attempting to get the string repr of the variable. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Is there a more efficient threading lock?
Re sqlite and threads. The C API can be compiled to be thread safe from my Reading if the sqlite docs. What I have not checked is how python’s bundled sqlite is compiled. There are claims python’s sqlite is not thread safe. Barry -- https://mail.python.org/mailman/listinfo/python-list
Re: Is there a more efficient threading lock?
> Though it's still probably not as useful as you might hope. In C, if I > can do "int id = counter++;" atomically, it would guarantee me a new > ID that no other thread could ever have. C does not have to do that atomically. In fact it is free to use lots of instructions to build the int value. And some compilers indeed do, the linux kernel folks see this in gcc generated code. I understand you have to use the new atomics features. Barry -- https://mail.python.org/mailman/listinfo/python-list
Re: Bug 3.11.x behavioral, open file buffers not flushed til file closed.
> On 6 Mar 2023, at 01:42, Greg Ewing via Python-list > wrote: > > On 6/03/23 1:02 pm, Cameron Simpson wrote: >> Also, fsync() need not expedite the data getting to disc. It is equally >> valid that it just blocks your programme _until_ the data have gone to disc. > > Or until it *thinks* the data has gone to the disk. Some drives > do buffering of their own, which may impose additional delays > before the data actually gets written. This used to be an issue until Microsoft refused to certify and drive that lied about when data was persisted to the medium. WHQL? That had the effect of stooping driver manufactures having firmware to win benchmarking. Now the OS will use the commands to the drive that allow the OS to know the data is safe. Barry > > -- > Greg > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Winodws10 Command Prompt unresponsive to .py commands
> On 8 Mar 2023, at 05:55, Mike Dewhirst wrote: > > On 8/03/2023 6:31 am, Thomas Gregg wrote: >> Hi, I got python 11 to work with the esptool a few days ago. However, I >> must have something wrong, because now, when I enter any command with .py, >> Windows Command Prompt just returns without doing anything. Example >> >> C:\Users\gregg>esptool.py version >> >> C:\Users\gregg> >> >> I tried to change the Windows default by filetype. but no luck. Any >> ideas? Thanks > > Prove it works in the command prompt by using the full path to python for > example, > > C:\Python311\python C:\Users\gregg>esptool.py version On windows use the py.exe command rather then python.exe. Does `py esptool.py version’ > > If that works, look at your path environment variables and see where python > is sitting. Substitute the real location of Python 3.11 on your machine. > Sadly, I think it defaults to C:\Program Files nowadays. > > Once you can get Python working by omitting the path, for example, > > C:\Users\gregg>python esptool.py version > > ... you can then right-click the .py file and choose whichever program you > like to open with. Use py.exe which should in the windows folder to run .py files. Barry > I'm inclined to advise you to focus on getting virtual environments working > next and leave all that auto-opening to later. You might find you prefer to > right click and select every time. > > M > > -- > Signed email is an absolute defence against phishing. This email has > been signed with my private key. If you import my public key you can > automatically decrypt my signature and be sure it came from me. Your > email software can handle signing. > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: PyGILState_Release called twice in embedded application
> On 23 Mar 2023, at 14:34, Arnaud Loonstra wrote:
>
> On 23-03-2023 13:33, Barry Scott wrote:
>>>> On 23 Mar 2023, at 08:46, Arnaud Loonstra wrote:
>>>
>>> Hi all,
>>>
>>> I'm running in a crash due to a ResourceWarning (some socket is not closed
>>> in a used module) after calling PyGILState_Release.
>>>
>>> I'm running Python in a native thread (so a thread created by C not
>>> Python). I'm acquiring the GIL through PyGILState_Ensure before doing any
>>> CPYthon calls and releasing when finished using PyGILState_Release.
>>>
>>> This has worked fine. But now I'm using a python module (openai) which
>>> doesn't close its socket correctly which results in a ResourceWarning which
>>> triggers an assert.
>>>
>>> In the backtrace (below) I can see PyGILState_Release is called again. (7)
>>> while I've already called it (126).
>>>
>>> I can make the crash go away by adding
>>>
>>> import warnings
>>> warnings.simplefilter("ignore", ResourceWarning)
>>>
>>> to my python code. But I'd rather prevent this from happening in the first
>>> place.
>>>
>>> Any suggestion very welcomed cause I'm puzzled.
>> What 3rd party C extension are you running with?
>> I'd be surprised if the cpython code was the issue.
>> Barry
>
> I'm not using any 3rd party extension myself. But the issue is caused by the
> openai module using the requests module which is not closing sockets:
>
> https://github.com/openai/openai-python/issues/140
>
> I'm not aware what C extensions they might use.
In gdb use info shared to see what is loaded.
Leaving a socket open will not cause a logic error as you are encountering.
I expect a bug in C/C++ cod3 that is not part of python itself.
You are using 3rd party code, requests and openai that you name.
Barry
>
> Btw, I've tested this with python 3.8 and 3.11.2.
>
> Rg,
>
> Arnaud
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: found older version of python in my command prompt
> On 28 Mar 2023, at 16:44, pranavbhardwaj...@gmail.com wrote: > > > > > > Sent from [1]Mail for Windows > > Dear sir, > > I am Pranav Bhardwaj and I am facing issue regarding python > version. When I try to open python in my command prompt, I see there is > very older version in it that is python 2.7.13 as a default version. I > want to switch my default version to the latest version in my command > prompt. Plese help me how can I do this. Uninstall python 2.7 in the usual windows way. Download python 3.11 from python.org and run the installer. In a cmd terminal window run py.exe and you should get the python prompt. Barry > > > > References > > Visible links > 1. https://go.microsoft.com/fwlink/?LinkId=550986 > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: built-in pow() vs. math.pow()
> On 30 Mar 2023, at 22:30, Chris Angelico wrote: > > On Fri, 31 Mar 2023 at 08:13, Oscar Benjamin > wrote: >> >>> On Thu, 30 Mar 2023 at 17:31, Andreas Eisele >>> wrote: >>> >>> I sometimes make use of the fact that the built-in pow() function has an >>> optional third argument for modulo calculation, which is handy when dealing >>> with tasks from number theory, very large numbers, problems from Project >>> Euler, etc. I was unpleasantly surprised that math.pow() does not have this >>> feature, hence "from math import *" overwrites the built-in pow() function >>> with a function that lacks functionality. I am wondering for the rationale >>> of this. Does math.pow() do anything that the built-in version can not do, >>> and if not, why is it even there? >> >> It is useful for when you want the pure floating point power which has >> an approximately fixed computational cost (unlike integer powers). >> Perhaps it would have been better if it was named fpow similar to fsum >> vs sum. >> > > It's called math.pow. That on its own should be a strong indication > that it's designed to work with floats. So long as you know that the math module is provided to give access the C math.h functions. Barry > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Problem in using libraries
> On 3 Apr 2023, at 17:46, Pranav Bhardwaj wrote: > > Why can't I able to use python libraries such as numpy, nudenet, playsound, > pandas, etc in my python 3.11.2. It always through the error "import > 'numpy' or any other libraries could not be resolved". You need to provide enough details for people to help you. Include which OS you are using. Where you got python from. How you installed the libraries you are trying to use. Commands and error messages you are seeing. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Windows installer from python source code without access to source code
> On 4 Apr 2023, at 16:28, Jim Schwartz wrote:
>
> Where can I download that cl program? I've used gcc before, but I hear that
> cl can use a setup.py program to run the compile and link and create a
> windows .msi installer. Is that true?
It is part of visual studio C++.
Once you have that installed there are bat files that setup environment in the
terminal.
Then you can use cl, nmake etc
Barry
>
> -Original Message-
> From: Eryk Sun
> Sent: Friday, March 31, 2023 12:55 PM
> To: Jim Schwartz
> Cc: python-list@python.org
> Subject: Re: Windows installer from python source code without access to
> source code
>
>> On 3/31/23, Jim Schwartz wrote:
>> I want a windows installer to install my application that's written in
>> python, but I don't want the end user to have access to my source code.
>
> Cython can compile a script to C source code for a module or executable
> (--embed). The source can be compiled and linked normally.
> For example, the following builds a "hello.exe" executable based on a
> "hello.py" script.
>
>> cython -3 --embed hello.py
>> set "PYI=C:\Program Files\Python311\include"
>> set "PYL=C:\Program Files\Python311\libs"
>> cl /I"%PYI%" hello.c /link /libpath:"%PYL%"
>> copy hello.exe embed
>> embed\hello.exe
>Hello, World!
>
> I extracted the complete embeddable distribution of Python 3.11 into the
> "embed" directory. You can reduce the size of the installation, if needed, by
> minimizing the zipped standard library and removing pyd extensions and DLLs
> that your application doesn't use.
>
> The generated "hello.c" is large and not particularly easy to read, but here
> are some snippets [...]:
>
>[...]
>/* Implementation of 'hello' */
>static PyObject *__pyx_builtin_print;
>static const char __pyx_k_main[] = "__main__";
>static const char __pyx_k_name[] = "__name__";
>static const char __pyx_k_test[] = "__test__";
>static const char __pyx_k_print[] = "print";
>static const char __pyx_k_Hello_World[] = "Hello, World!";
>[...]
> /* "hello.py":1
> * print("Hello, World!") # <<<<<<<<<<<<<<
> */
> __pyx_tuple_ = PyTuple_Pack(1, __pyx_kp_u_Hello_World);
>if (unlikely(!__pyx_tuple_)) __PYX_ERR(0, 1, __pyx_L1_error)
>[...]
> /* "hello.py":1
> * print("Hello, World!") # <<<<<<<<<<<<<<
> */
> __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_print, __pyx_tuple_,
> NULL);
>if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 1, __pyx_L1_error)
>[...]
>int wmain(int argc, wchar_t **argv) {
>[...]
>if (argc && argv)
>Py_SetProgramName(argv[0]);
>Py_Initialize();
>if (argc && argv)
>PySys_SetArgv(argc, argv);
>[...]
> m = PyInit_hello();
>[...]
>if (Py_FinalizeEx() < 0)
>return 2;
>[...]
>return 0;
>[...]
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: for a 'good python'
> On 12 Apr 2023, at 18:10, jak wrote:
> Hi everyone,
> some time ago I wrote a class to determine if an ipv4 address belonged
> to a subnet. Seldom using python I'm pretty sure it's not written in
> 'good python' nor too portable. Could you give me some advice to make it
> better?
>
> class calcip:
>def __init__(self, psubnet: str):
>ssubnet, scidr = psubnet.replace(' ', '').split('/')
>subnet = int.from_bytes(tuple(
> map(lambda n: (int(n)), ssubnet.split('.'))),
>'big')
>cidr = int(scidr)
>mask = ((2 ** cidr) - 1) << (32 - cidr)
>self.__network = subnet & mask
>self.__wildcard = ~mask & 0x
>self.__broadcast = (subnet | self.__wildcard) & 0x
>self.__tsubnet = tuple(subnet.to_bytes(4, 'big'))
>self.__tnetwork = tuple(self.__network.to_bytes(4, 'big'))
>self.__tbroadcast = tuple(self.__broadcast.to_bytes(4, 'big'))
>self.__tmask = tuple(mask.to_bytes(4, 'big'))
>self.__twildcard = tuple(self.__wildcard.to_bytes(4, 'big'))
>self.__host_min = tuple((self.__network + 1).to_bytes(4, 'big'))
>self.__host_max = tuple((self.__broadcast - 1).to_bytes(4, 'big'))
>
>@staticmethod
>def __to_str(val: tuple):
>return '.'.join(str(v) for v in val)
>
>@property
>def subnet(self):
>return self.__to_str(self.__tsubnet)
>
>@property
>def network(self):
>return self.__to_str(self.__tnetwork)
>
>@property
>def broadcast(self):
>return self.__to_str(self.__tbroadcast)
>
>@property
>def mask(self):
>return self.__to_str(self.__tmask)
>
>@property
>def wildcard(self):
>return self.__to_str(self.__twildcard)
>
>@property
>def host_min(self):
>return self.__to_str(self.__host_min)
>
>@property
>def host_max(self):
>return self.__to_str(self.__host_max)
>
>@property
>def hosts_num(self):
>return self.__wildcard - 1
>
>@property
>def net_class(self):
>tst = (self.__tnetwork[0] & 0xf0) >> 4
>if (tst & 0x8) == 0:
>clx = 'A'
>elif (tst & 0xc) == 0x8:
>clx = 'B'
>elif (tst & 0xe) == 0xc:
>clx = 'C'
>elif (tst & 0xf) == 0xe:
>clx = 'D'
>elif (tst & 0xf) == 0xf:
>clx = 'E'
>return clx
>
>def __contains__(self, item):
>ret = True
>row_hdr = None
>try:
>row_hdr = int.from_bytes(tuple(map(lambda n: (int(n)),
> item.split('.'))), 'big')
>except:
>ret = False
>if ret:
>if not self.__network < row_hdr < self.__broadcast:
>ret = False
>return ret
>
>
> def main():
>sn = calcip('10.0.0.0/26')
>
>print(f"subnet: {sn.subnet}")
>print(f"network: {sn.network}")
>print(f"broadcast: {sn.broadcast}")
>print(f"mask: {sn.mask}")
>print(f"wildcard: {sn.wildcard}")
>print(f"host_min: {sn.host_min}")
>print(f"host_max: {sn.host_max}")
>print(f"Avaible hosts: {sn.hosts_num}")
>print(f"Class: {sn.net_class}")
>
>tst_hdr = '10.0.0.31'
>is_not = 'is '
>if not tst_hdr in sn:
>is_not = 'is NOT '
>print("hdr %s %sin range %s - %s" %
> (tst_hdr, is_not, sn.host_min, sn.host_max))
>
> if __name__ == '__main__':
>main()
There is this https://docs.python.org/3/howto/ipaddress.html if you just want a
solution.
Or are you after code review feedback?
Barry
> --
> https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: for a 'good python'
> On 13 Apr 2023, at 00:19, jak wrote:
>
> Barry ha scritto:
>>
>>>> On 12 Apr 2023, at 18:10, jak wrote:
>>> Hi everyone,
>>> some time ago I wrote a class to determine if an ipv4 address belonged
>>> to a subnet. Seldom using python I'm pretty sure it's not written in
>>> 'good python' nor too portable. Could you give me some advice to make it
>>> better?
>>> class calcip:
>>> def __init__(self, psubnet: str):
>>> ssubnet, scidr = psubnet.replace(' ', '').split('/')
>>> subnet = int.from_bytes(tuple(
>>> map(lambda n: (int(n)),
>>> ssubnet.split('.'))),
>>> 'big')
>>> cidr = int(scidr)
>>> mask = ((2 ** cidr) - 1) << (32 - cidr)
>>> self.__network = subnet & mask
>>> self.__wildcard = ~mask & 0x
>>> self.__broadcast = (subnet | self.__wildcard) & 0x
>>> self.__tsubnet = tuple(subnet.to_bytes(4, 'big'))
>>> self.__tnetwork = tuple(self.__network.to_bytes(4, 'big'))
>>> self.__tbroadcast = tuple(self.__broadcast.to_bytes(4, 'big'))
>>> self.__tmask = tuple(mask.to_bytes(4, 'big'))
>>> self.__twildcard = tuple(self.__wildcard.to_bytes(4, 'big'))
>>> self.__host_min = tuple((self.__network + 1).to_bytes(4, 'big'))
>>> self.__host_max = tuple((self.__broadcast - 1).to_bytes(4, 'big'))
>>> @staticmethod
>>> def __to_str(val: tuple):
>>> return '.'.join(str(v) for v in val)
>>> @property
>>> def subnet(self):
>>> return self.__to_str(self.__tsubnet)
>>> @property
>>> def network(self):
>>> return self.__to_str(self.__tnetwork)
>>> @property
>>> def broadcast(self):
>>> return self.__to_str(self.__tbroadcast)
>>> @property
>>> def mask(self):
>>> return self.__to_str(self.__tmask)
>>> @property
>>> def wildcard(self):
>>> return self.__to_str(self.__twildcard)
>>> @property
>>> def host_min(self):
>>> return self.__to_str(self.__host_min)
>>> @property
>>> def host_max(self):
>>> return self.__to_str(self.__host_max)
>>> @property
>>> def hosts_num(self):
>>> return self.__wildcard - 1
>>> @property
>>> def net_class(self):
>>> tst = (self.__tnetwork[0] & 0xf0) >> 4
>>> if (tst & 0x8) == 0:
>>> clx = 'A'
>>> elif (tst & 0xc) == 0x8:
>>> clx = 'B'
>>> elif (tst & 0xe) == 0xc:
>>> clx = 'C'
>>> elif (tst & 0xf) == 0xe:
>>> clx = 'D'
>>> elif (tst & 0xf) == 0xf:
>>> clx = 'E'
>>> return clx
>>> def __contains__(self, item):
>>> ret = True
>>> row_hdr = None
>>> try:
>>> row_hdr = int.from_bytes(tuple(map(lambda n: (int(n)),
>>> item.split('.'))), 'big')
>>> except:
>>> ret = False
>>> if ret:
>>> if not self.__network < row_hdr < self.__broadcast:
>>> ret = False
>>> return ret
>>> def main():
>>> sn = calcip('10.0.0.0/26')
>>> print(f"subnet: {sn.subnet}")
>>> print(f"network: {sn.network}")
>>> print(f"broadcast: {sn.broadcast}")
>>> print(f"mask: {sn.mask}")
>>> print(f"wildcard: {sn.wildcard}")
>>> print(f"host_min: {sn.host_min}")
>>> print(f"host_max: {sn.host_max}")
>>> print(f"Avaible hosts: {sn.hosts_num}")
>>> print(f"Class: {sn.net_class}")
>>> tst_hdr = '10.0.0.31'
>>> is_not = 'is '
>>> if not tst_hdr in sn:
>>> is_not = 'is NOT '
>>> print("hdr %s %sin range %s - %s" %
>>> (tst_hdr, is_not, sn.host_min, sn.host_max))
>>> if __name__ == '__main__':
>>> main()
>> There is this https://docs.python.org/3/howto/ipaddress.html if you just
>> want a solution.
>> Or are you after code review feedback?
>> Barry
>>> --
>>> https://mail.python.org/mailman/listinfo/python-list
>
>
> Thank you too. I had seen this library but I always try not to use
> libraries outside the standard ones. Now I don't remember why I was
> convinced that this wasn't part of it, perhaps because it was like that
> at the time or because I got confused. Only now I realized that it is
> not necessary to install it. Now I'm considering whether to use
> 'ipaddress' or 'socket'. What is certain is that this one you have
> suggested is really comfortable. Thanks again for the report.
Ipaddress was developed outside of the std lib and later added i recall.
Barry
>
> --
> https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: Is npyscreen still alive?
> On 21 Apr 2023, at 22:00, Grant Edwards wrote: > > I recently googled across the ncurses application framework npyscreen, > and was thinking about giving it a try for a small but real project > (something that would be distributed to customers), but I'm a bit > concerned that npyscreen no longer "alive". > > The pypi page says the homepage is http://www.npcole.com/npyscreen/, > which then points to a Google Code page at > https://code.google.com/archive/p/npyscreen/. > > That page says the official repo is at https://bitbucket.org/npcole/npyscreen > which returns a 404. > > There seems to be a copy in Github at > https://github.com/npcole/npyscreen/commits/master, > but the last commit was almost 4 years ago. > > Maybe it "just works" and is suitable for production? Maybe this, recently lwn.net article, https://textual.textualize.io/ I was planning to check it out. Barry > > > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: pip module not found
> On 12 May 2023, at 18:31, Thomas Passin wrote: > > On 5/12/2023 11:18 AM, Thomas Passin wrote: >>> On 5/12/2023 2:42 AM, David John wrote: >>> Hi, >>> I recently have been experiencing issues with the pip installation module. >>> I have python version 3.11 installed. I've checked the directory installed >>> in the systems variables window and nothing is amiss. Kindly assist. >> It would be useful if you told us what operating system you are using and >> how you installed Python. >> Many if not most Linux distributions do not include pip by default. Usually >> the package manager as a version to install. On systems based on Debian, >> you can install pip with: >> sudo apt install python3-pip >> On others, you will have to look around in the package manager or search on >> line. >> As a last resort, if you cannot find an OS package manager way to install >> pip, you find out how from here: >> https://pip.pypa.io/en/stable/installation/ >> As the link says, you can run from a command line: >> -m ensurepip --upgrade >> NOTE: instead of , use the command that launches the right version >> of python on your system On Windows, this is usually py. On Linux, it is >> usually python3. > > On Linux, if you want tkinter, you may have to install it with the package > manager too. On Debian-related systems: > > sudo apt-get install python3-tk > > For the Yum package manager: > > yum install tkinter > > You may also need to install ImageTk: > > sudo apt-get install python3-pil.imagetk (Debian-based) > > On Centos/Red Hat derived systems, you will also need to install > > python3-pillow > python3-pillow-tk > PIP not PIL is the topic right? We still need OP to tell us which OS and where python came from. Barry > > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: pip module not found
> On 12 May 2023, at 21:59, Thomas Passin wrote: > > On 5/12/2023 2:42 AM, David John wrote: >> Hi, >> I recently have been experiencing issues with the pip installation module. >> I have python version 3.11 installed. I've checked the directory installed >> in the systems variables window and nothing is amiss. Kindly assist. > > It would be useful if you told us what operating system you are using and how > you installed Python. > > Many if not most Linux distributions do not include pip by default. Usually > the package manager as a version to install. From what i see the fedora/redhat/centos world includes the batteries. The debian/ubuntu world take batteries out. > On systems based on Debian, you can install pip with: > > sudo apt install python3-pip > > On others, you will have to look around in the package manager or search on > line. > > As a last resort, if you cannot find an OS package manager way to install > pip, you find out how from here: > > https://pip.pypa.io/en/stable/installation/ > > As the link says, you can run from a command line: > > -m ensurepip --upgrade > > NOTE: instead of , use the command that launches the right version of > python on your system On Windows, this is usually py. On Linux, it is > usually python3. > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: PythonPath / sys.path
> On 14 May 2023, at 16:32, Grizzy Adams via Python-list > wrote: > > Hi All > > My first post (repeated) > > I am having a problem with PythonPath / sys.path > > I have a dir where I keep all my current work, but I can't seem to add it to > PythonPath / sys.path > > When I try to import one of my modules I see > >>>> import My_Working_File > Traceback (most recent call last): > File "", line 1, in >import My_Working_File > ImportError: No module named 'My_Working_File' > > I have tried adding my dir in registry to the existing PythonPath > > [HKEY_CURRENT_USER\Software\Python\PythonCore\3.4\PythonPath] > @="D:\\Shades\\Tools\\Python\\Lib;D:\\Shades\\Tools\\Python\\DLLs" > > that did not help, > > I tried adding PYTHONPATH as an enviramental variable, that also had no effect > > (I did re-boot after these two tests) > > If I go to the editor window and run (F5) any of my modules then I can > >>>> RESTART >>>> >>>> import My_Working_File >>>> > > without error > > I can also (from a fresh start) > >>>> import sys >>>> sys.path.append(r'D:\Shades\Tools\Python\MyWork') >>>> print(sys.path) > ['', 'D:\\Shades\\Tools\\Python\\Lib\\idlelib', > 'D:\\Shades\\Tools\\Python\\python34.zip', 'D:\\Shades\\Tools\\Python\\DLLs', > 'D:\\Shades\\Tools\\Python\\lib', 'D:\\Shades\\Tools\\Python', > 'D:\\Shades\\Tools\\Python\\lib\\site-packages', > 'D:\\Shades\\Tools\\Python\\MyWork'] >>>> import My_Working_File >>>> > > I don't really want to go this route every time I start work, and more to the > point I will need (to learn how) to add dir's to the path at times > > I did search the net for an answer but only got what I had already tried > > any pointers what I'm doing wrong (other than using windows '->) I take it you have business reasons to use an obsolete version python. Where did you get your version of python from? You seem to be doing the right thing with setting an environment variable. Useless shades is an app that embeds python. Try starting cmd.exe and using the set command to liat all of the environment. Do you see your PYTHONPATH variable? Does it contain what you expect? If it does start python and check that sys.path has your folder in it. Also check what is on os.environ dirctionary. Barry > > Grizz > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: PythonPath / sys.path
> On 15 May 2023, at 05:39, Thomas Passin wrote: > > On 5/14/2023 11:08 PM, Chris Angelico wrote: >>> On Mon, 15 May 2023 at 12:07, Thomas Passin wrote: >>> Well, no, why would you assume that? I started to use Linux - in VMs - >>> because I had to make sure that my cross-platform java/jython Tomcat >>> program would work right on Linux. Why, for example, would I think to >>> install Idle from the package manager when it, or things like that, were >>> always in my experience installed with pip? For that matter, "sudo >>> apt-get install pip" won't install pip. You need to use a different >>> name, and it may or may not be different for different distros. >> If you EVER had to install something other than a Python package, you >> would have had to make use of the system package manager. You're >> right, there are multiple obvious ways to install Idle, but that >> doesn't mean that the package manager isn't one of them. > > Yes, after a while I came to realize that missing Python pieces might be > available from the package manager. That doesn't mean it's obvious, or easy > to discover just what names to use. And sometimes one has to add a new > external repository. Personally, I don't find it easy to scroll through > hundreds of lines in the synaptics search results looking for something whose > name I can only partly guess at. If I know the command line equivalent for a > search, I could do a grep and that would probably be more focused. But > trying to work with a dozen different distros because various clients might > use them - it's hard to keep details straight. > > Anyway, there's no point in trying to convince me that I could have > understood everything at the start that I may have learned later. I'm just > interested in passing on things I've learned along that way that a newcomer > to Python in Linux may not realize. Being a Fedora user i needed to learn how to install missing pieces of pyrhon on ubuntu. We searches for: ubuntu install pip Ubuntu install idle Both provide lots of answers. Did your searches fail to turn up answers? Barry > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Pip Error
> On 17 May 2023, at 17:26, Aysu Mammadli wrote: > > I encountered an error while attempting to install pip using the terminal. > The exact error message I received is: > > "An error occurred during configuration: option use-feature: invalid > choice: '2020-resolver' (choose from 'fast-deps', 'truststore', > 'no-binary-enable-wheel-cache')" > > Here are the steps I followed and the context of my system: > > Operating System: Windows 11 Python Version: 3.11.3 I executed the command > in the terminal to install pip, but it resulted in the aforementioned error > message. This error specifically references the '2020-resolver' choice > within the 'use-feature' option. An error occurred during configuration: > option use-feature: invalid choice: '2020-resolver' (choose from > 'fast-deps', 'truststore', 'no-binary-enable-wheel-cache') > > I have already attempted the following troubleshooting steps without > success: > > Checked my internet connectivity to ensure it was stable and not causing > any issues. Verified that I have the latest version of pip installed, as > well as the latest version of Python. Tried using alternative commands, > such as using the --upgrade flag or specifying a specific pip version, but > the error persists. Looked for any recent system updates or changes that > could have caused this issue, but found none. I'm uncertain about the cause > of this error and how to resolve it. Any insights or suggestions would be > greatly appreciated. > > Thank you in advance for your assistance! Pip is installed by default with the python.org kits. I am not sure what the situation is for the app store python. Can you run python it self? Does one of these work for you? py -m pip python -m pip If so you have pip installed. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Help on ImportError('Error: Reinit is forbidden')
> On 17 May 2023, at 20:35, Jason Qian via Python-list
> wrote:
>
> Hi,
>
> I Need some of your help.
>
> I have the following C code to import *Import python.* It works 99% of
> the time, but sometimes receives "*ImportError('Error: Reinit is
> forbidden')*". error.
> **We run multiple instances of the app parallelly.
>
> *** Python version(3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC
> v.1914 64 bit (AMD64)]
>
> PyObject * importPythonModule(const char* pmodName)
> {
>const char* errors = NULL;
> int nlen = strlen(pmodName);
> PyObject *pName = PyUnicode_DecodeUTF8(pmodName, nlen, errors);
> PyObject *pModule = *PyImport_Import*(pName);
> Py_DECREF(pName);
> if (pModule == NULL) {
> if (*PyErr_Occurred*()) {
>handleError("PyImport_Import()");
> }
> }
> }
> void handleError(const char* msg)
> {
> ...
> "PyImport_Import() - ImportError('Error: Reinit is forbidden')"
> }
You do not seem to printing out msg, you have assumed it means reinit it seems.
What does msg contain when it fails?
Barry
>
>
> Thanks
> Jason
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Help on ImportError('Error: Reinit is forbidden')
On 18 May 2023, at 13:56, Jason Qian wrote:
Hi Barry,
void handleError(const char* msg)
{
...
PyErr_Fetch(&pyExcType, &pyExcValue, &pyExcTraceback);
PyErr_NormalizeException(&pyExcType, &pyExcValue, &pyExcTraceback);
PyObject* str_value = PyObject_Repr(pyExcValue);
PyObject* pyExcValueStr = PyUnicode_AsEncodedString(str_value, "utf-8",
"Error ~");
const char *strErrValue = PyBytes_AS_STRING(pyExcValueStr);
//where strErrValue = "ImportError('Error: Reinit is forbidden')"
...
}
What we imported is a Python file which import some pyd libraries.
Please do not top post replies.
Ok so assume the error is correct and hunt for the code that does the
reimport.
You may need to set break points in you C code to find tnw caller.
Barry
Thanks
Jason
On Thu, May 18, 2023 at 3:53 AM Barry <[1]ba...@barrys-emacs.org> wrote:
> On 17 May 2023, at 20:35, Jason Qian via Python-list
<[2]python-list@python.org> wrote:
>
> Hi,
>
> I Need some of your help.
>
> I have the following C code to import *Import python.* It works
99% of
> the time, but sometimes receives "*ImportError('Error: Reinit is
> forbidden')*". error.
> **We run multiple instances of the app parallelly.
>
> *** Python version(3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51)
[MSC
> v.1914 64 bit (AMD64)]
>
> PyObject * importPythonModule(const char* pmodName)
> {
> const char* errors = NULL;
> int nlen = strlen(pmodName);
> PyObject *pName = PyUnicode_DecodeUTF8(pmodName, nlen, errors);
> PyObject *pModule = *PyImport_Import*(pName);
> Py_DECREF(pName);
> if (pModule == NULL) {
> if (*PyErr_Occurred*()) {
> handleError("PyImport_Import()");
> }
> }
> }
> void handleError(const char* msg)
> {
> ...
> "PyImport_Import() - ImportError('Error: Reinit is forbidden')"
> }
You do not seem to printing out msg, you have assumed it means reinit
it seems.
What does msg contain when it fails?
Barry
>
>
> Thanks
> Jason
> --
> [3]https://mail.python.org/mailman/listinfo/python-list
>
References
Visible links
1. mailto:ba...@barrys-emacs.org
2. mailto:python-list@python.org
3. https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: Why does IDLE use a subprocess?
> On 30 May 2023, at 21:10, James Schaffler via Python-list > wrote: > > Originally posted to idle-dev, but thought this might be a better place. Let > me know if it isn't. > > Hi, > > I was curious about the internals of IDLE, and noticed that IDLE uses > executes user code in a "subprocess" that's separate from the Python > interpreter that is running IDLE itself (which does tasks such as making the > window and coloring the text). > > As far as I understand, IDLE runs a modified version of > code.InteractiveInterpreter by sending user code through a socket. Even the > IDLE documentation says that without a subprocess, "user code is not isolated > from IDLE itself." However, some minimal testing of InteractiveInterpreter > leads me to believe that the Interpreter object has its own view of > local/global variables and therefore shouldn't be able to affect the calling > interpreter (please correct me if I'm wrong). > > So my question is a combination of "Why does IDLE use a subprocess?" and "Why > is InteractiveInterpreter not secureuldenough?" What possible security > vulnerabilities exist if one uses IDLE without the subprocess? If anyone > knows (or could point me to information on) why IDLE is designed this way, > I'd really appreciate it. Thank you! I don’t think it security but robustness that needs the subprocess. You can crash idle with bugs in the code that you are developing. By running your code in a subprocess idle protects itself, and your edits from bugs in your code. Also if your code use tk then it would conflict with idle’s use of tk. That is my assumption on why the subprocess is required. Barry > > Jim > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: complaint
> On 30 May 2023, at 21:10, Daniel Ifechukwude Dibie > wrote: > > i tried to uninstall the python 3.11.3 program from my machine so that i > can re-install it is showing successful but it is ligerning on the program > and features Maybe what you are seeing is the microsoft app store stubs for python? The python.org uninstall always works in my experience, but is not related to the app store stubs. Also why do you think you need to reinstall python? It would be very unusual that that would be necessary. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: advice on debugging a segfault
Run python under gdb and when the segv happens use the gdb bt command to get a stack trace. Also if gdb says that it needs debug symbols install you will need to do that. Otherwise the not will not contain symbols. Barry > On 17 Jan 2021, at 19:58, Robin Becker wrote: > > I have a segfault in the 3.10 alpha 4 when running the reportlab document > generation; all the other tests seem to have worked. I would like to ask > experts here how to approach getting the location of the problem. I run > recent archlinux. > > Below is the output of a test run with -Xdev -Xtracemalloc; the main process > is almost finished so the error appears to come from trying to free resources > >> $ python -Xdev -Xtracemalloc genuserguide.py >> /home/robin/devel/reportlab/.py310/lib/python3.10/distutils/__init__.py:1: >> DeprecationWarning: the imp module is deprecated in favour of importlib; see >> the module's documentation for alternative uses >> import imp >> Built story contains 1843 flowables... >> Saved >> "/home/robin/devel/reportlab/REPOS/reportlab/docs/reportlab-userguide.pdf" >> [ Top 10 ] >> :630: size=10.5 MiB, count=105832, >> average=104 B >> /home/robin/devel/reportlab/reportlab/lib/abag.py:19: size=7161 KiB, >> count=27126, average=270 B >> /home/robin/devel/reportlab/reportlab/platypus/paraparser.py:3141: size=3364 >> KiB, count=5980, average=576 B >> /home/robin/devel/reportlab/.py310/lib/python3.10/site-packages/pyphen/__init__.py:160: >> size=2905 KiB, count=41797, average=71 B >> /home/robin/devel/reportlab/reportlab/platypus/paragraph.py:776: size=1386 >> KiB, count=32208, average=44 B >> /home/robin/devel/reportlab/reportlab/platypus/paragraph.py:92: size=1384 >> KiB, count=26248, average=54 B >> /home/robin/devel/reportlab/.py310/lib/python3.10/copy.py:280: size=1232 >> KiB, count=8827, average=143 B >> /home/robin/devel/reportlab/reportlab/platypus/frames.py:155: size=1101 KiB, >> count=1173, average=962 B >> :241: size=915 KiB, count=8146, average=115 B >> /home/robin/devel/reportlab/reportlab/platypus/paragraph.py:773: size=881 >> KiB, count=16104, average=56 B >> sys:1: ResourceWarning: unclosed file <_io.BufferedReader >> name='../images/replogo.gif'> >> sys:1: ResourceWarning: unclosed file <_io.BufferedReader >> name='../images/testimg.gif'> >> Debug memory block at address p=0x5617c33effe0: API '' >>0 bytes originally requested >>The 7 pad bytes at p-7 are not all FORBIDDENBYTE (0xfd): >>at p-7: 0x00 *** OUCH >>at p-6: 0x00 *** OUCH >>at p-5: 0x00 *** OUCH >>at p-4: 0x00 *** OUCH >>at p-3: 0x00 *** OUCH >>at p-2: 0x00 *** OUCH >>at p-1: 0x00 *** OUCH >>Because memory is corrupted at the start, the count of bytes requested >> may be bogus, and checking the trailing pad bytes may segfault. >>The 8 pad bytes at tail=0x5617c33effe0 are not all FORBIDDENBYTE (0xfd): >>at tail+0: 0x00 *** OUCH >>at tail+1: 0x00 *** OUCH >>at tail+2: 0x00 *** OUCH >>at tail+3: 0x00 *** OUCH >>at tail+4: 0x00 *** OUCH >>at tail+5: 0x00 *** OUCH >>at tail+6: 0x00 *** OUCH >>at tail+7: 0x00 *** OUCH >> Enable tracemalloc to get the memory block allocation traceback >> Fatal Python error: _PyMem_DebugRawFree: bad ID: Allocated using API '', >> verified using API 'o' >> Python runtime state: finalizing (tstate=0x5617c53c8b30) >> Current thread 0x7fd5742b3740 (most recent call first): >> >> Aborted (core dumped) > > > for comparison here is the 3.9.1 output> $ python -Xdev -Xtracemalloc > genuserguide.py >> /home/robin/devel/reportlab/.py39/lib/python3.9/distutils/__init__.py:1: >> DeprecationWarning: the imp module is deprecated in favour of importlib; see >> the module's documentation for alternative uses >> import imp >> /home/robin/devel/reportlab/.py39/lib/python3.9/site-packages/fontTools/misc/py23.py:11: >> DeprecationWarning: The py23 module has been deprecated and will be removed >> in the next release. Please update your code. >> warnings.warn( >> Built story contains 1843 flowables... >> Saved >> "/home/robin/devel/reportlab/REPOS/reportlab/docs/reportlab-userguide.pdf" >> [ Top 10 ] >> :587: size=12.4 MiB, count=128010, >> average=102 B >> /home/robin/devel/reportlab/reportlab/lib/abag.py:19: size=7132 KiB, >> count=26951, average=271 B >> /
Re: list() strange behaviour
> On 23 Jan 2021, at 17:23, Avi Gross via Python-list > wrote: > > I am wondering how hard it would be to let some generators be resettable? That is generally thought to be a bad thing in OOD. Classes that reset are usually a code smell and often a source of bugs. Why not just assign a new generate into the variable that is used to access the generator. The avoids the need for the complications of reset logic. Barry > > I mean if you have a generator with initial conditions that change as it > progresses, could it cache away those initial conditions and upon some > signal, simply reset them? Objects of many kinds can be set up with say a > reinit() method. > > I am not saying Python needs such a language change for generators as in > many programs you could just recreate a new instance of a generator. But > there may be places where by the time the generator is used, the original is > not known. Or, there are places where you want to lengthen something to > match another by repeatedly copying the same sequence as many times as > needed. If a generator finishes, you want it to restart with the same > sequence until you stop asking. > > This is just a thought, not a request for such a feature. > > -Original Message- > From: Python-list On > Behalf Of ast > Sent: Saturday, January 23, 2021 2:54 AM > To: python-list@python.org > Subject: Re: list() strange behaviour > >> Le 20/12/2020 à 21:00, danilob a écrit : >> b = ((x[0] for x in a)) > > There is a useless pair of parenthesis > > b = (x[0] for x in a) > > b is a GENERATOR expression > > first list(b) calls next method on b repetedly until b is empty. > So it provides the "content" of b > > second list(b) provides nothing since b is empty (there is no reset on > generators) > > > > > -- > https://mail.python.org/mailman/listinfo/python-list > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: IDE tools to debug in Python?
> On 6 Feb 2021, at 08:06, Grant Edwards wrote: > > On 2021-02-05, Schachner, Joseph wrote: > >> Indeed there are many. One I have not seen listed here yet, that is >> quite light, starts quickly, but does have good debugging capability >> is PyScripter. Completely free, downloadable from SourceForge, 32 >> or 64 bit versions (must match your Python type). > > Windows only. See https://github.com/pyscripter/pyscripter/tree/master/Install It says use the zip file non windows. I have not tested this. Barry > > -- > Grant > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Why assert is not a function?
> On 2 Mar 2021, at 20:54, Marco Sulla wrote: > > I have a curiosity. Python, as many languages, has assert as a > keyword. Can't it be implemented as a function? Is there an advantage > to have it as a keyword? assert condition, expression Only is condition is false with expression be evaluated. So you can safely do expensive things I the expression with incuring and cost if the condition is True. With a function assert the 2nd part would have to evaluated regardless of the state of the condition. Which would slow down the code for no benefit. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Application problems
> On 10 Mar 2021, at 22:14, Thomas Jollans wrote:
>
> On 10/03/2021 21:50, Mats Wichmann wrote:
>>
>> For the first one, don't feel too bad, this ("opening the normal python")
>> seems to be biting a lot of people recently
>
>
> I wonder why. Python's installation process isn't any different from most
> other Windows software released the past 25-ish years. Is it possible that
> Windows 10's search feature sometimes makes poor choices, and typing
> "python" just brings up the wrong thing?
>
> (I just tested it on a clean VM and that's not what happens, but maybe for
> some people? I dunno
I think it is as simple as the python installer does not have the string
“setup” in the name.
I raise a bpo that is getting worked to change this hopefully for 3.10 maybe
3.11.
Barry
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: memory consumption
> On 31 Mar 2021, at 09:42, Alexey wrote: > > среда, 31 марта 2021 г. в 01:20:06 UTC+3, Dan Stromberg: >>> On Tue, Mar 30, 2021 at 1:25 AM Alexey wrote: >>> >>> >>> I'm sorry. I didn't understand your question right. If I have 4 workers, >>> they require 4Gb >>> in idle state and some extra memory when they execute other tasks. If I >>> increase workers >>> count up to 16, they`ll eat all the memory I have (16GB) on my machine and >>> will crash as soon >>> as system get swapped. >>> >> What if you increase the machine's (operating system's) swap space? Does >> that take care of the problem in practice? > > I can`t do that because it will affect other containers running on this host. > In my opinion it may significantly reduce their performance. Assuming this a modern linux then you should have control groups that allow you to set limits on memory and swap for each container. Are you running with systemd? Barry > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: memory consumption
> On 1 Apr 2021, at 13:46, Marco Ippolito wrote: > > >> >>>> What if you increase the machine's (operating system's) swap space? Does >>>> that take care of the problem in practice? >>> >>> I can`t do that because it will affect other containers running on this >>> host. >>> In my opinion it may significantly reduce their performance. >> >> Assuming this a modern linux then you should have control groups that allow >> you to set limits on memory and swap for each container. >> >> Are you running with systemd? > > If their problem is that their memory goes from `` to `` and then > back down to `` rather than ``, how could `cgroups` have helped in > that case? > > I suspect the high watermark of `` needs to be reachable still and, > secondly, that a forceful constraint whilst running would crash the container? > > How else could one approach it? > I was responding to the assertion that adding swap to the system would impact other containers. The solution I have used is to set service/container resource limits to ensure they work as expected. I was not suggestion this a fix for the memory leak. Barry -- https://mail.python.org/mailman/listinfo/python-list
Re: Current thinking on required options
> On 19 Apr 2021, at 10:57, Loris Bennett wrote:
>
> Hi,
>
> I have various small programs which tend to have an interface like the
> following example:
>
> usage: grocli [-h] [-o {check,add,delete}] [-u USERS [USERS ...]] [-g GROUP]
>
> Command line grouper tool
>
> optional arguments:
>-h, --helpshow this help message and exit
>-o {check,add,delete}, --operation {check,add,delete}
> operation to apply
>-u USERS [USERS ...], --users USERS [USERS ...]
> users to apply operation to
>-g GROUP, --group GROUP
> group to apply operation to
>
> However, the options -o, -u, and -g are required, not optional.
You could use positional args like this:
grocli check user,user group
Barry
>
> The documentation
>
> https://docs.python.org/3/library/argparse.html#required
>
> advises against required options and here
>
>
> https://stackoverflow.com/questions/24180527/argparse-required-arguments-listed-under-optional-arguments
>
> a way of adding a section 'required arguments' to the usage is
> described.
>
> I would be interested to know what the general thinking on "required
> options" is. Is there just a better way of designing such interfaces?
>
> Cheers,
>
> Loris
>
> --
> This signature is currently under construction.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Determine what the calling program is
> On 19 Apr 2021, at 22:49, Cameron Simpson wrote: > > On 19Apr2021 23:13, Peter J. Holzer wrote: >>> On 2021-04-19 08:54:06 +1000, Cameron Simpson wrote: >>> My personal preference is lock directories. Shell version goes like >>> this: >>> >>>if mkdir /my/lock/directory/name-of-task >>>then >>> .. do task .. >>> rmdir /my/lock/directory/name-of-task >>>else >>> echo "lock /my/lock/directory/name-of-task already taken" >>>fi >>> >>> Simple, reliable, even works over NFS if you care. >> >> Reliable only if "fail locked" is acceptable. If that process dies for >> some reason the lock directory will stay behind, blocking other >> processes until somebody notices the problem and removes it. > > A Python context manager narrows the range of circumstances for this > failure quite a lot. But yes. > >> The fcntl method suggested by several people has the advantage that the >> lock vanished with the process which holds it. > > This is very true. OTOH, mkdir's easy to debug if it hangs around. Only the fcntl method is robust. Your suggestion with mkdir is not reliable in practice. If you need a lock in the sh env then there are standard patterns using the flock command. See the man page for examples. Barry > > Cheers, > Cameron Simpson > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: How to check if an image contains an element I am searchig for
> On 16 Jun 2021, at 21:46, Arak Rachael wrote: > > On Wednesday, 16 June 2021 at 22:08:31 UTC+2, Chris Angelico wrote: >>> On Thu, Jun 17, 2021 at 6:06 AM Arak Rachael wrote: >>> >>> Hi guys, >>> >>> I have an image from google maps to say and I need to check if it has road >>> markings, in order to do that, I believe I need to change the effects on >>> the image so the markings and road can be white or something and the things >>> I don't need like cars, trees and so on to be black. >>> >>> images should contain only road surface and/or road markings (lines, >>> zebras, stripes. >>> >>> Can anyone help me on this, I already have the crop code, I just need to >>> check if the cropped part contains what I need. >> How well can you define the things you're looking for? >> >> https://xkcd.com/1425/ >> >> ChrisA > Hi Chris, > > what do you mean? He means that image processing is a hard problem that requires expertise to solve. > > Here is the image, I need to separate the road and markings from the rest and > divide the image into squares of 100x100 pixels, for each square I need to > check if it contains a road and markings: Can you define road in terms of an algorithm that looks at the pixels? > > https://www.dropbox.com/s/iduxj0j8w0ic616/SatelliteProcessing_Image.png?dl=0 > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: How Do I Get A Bug In Multiprocessing Fixed?
Also you report requires any developer to write a program from you notes to reproduce the problem. Attach a program that shows the problem would help. Better yet diagnose the problem after you reproduce it with a fix in a PR. Barry > On 18 Jun 2021, at 06:07, Alexander Neilson wrote: > > Hi Michael > > It may be helpful to populate the ticket with further details: > * actual output from when you execute the server and client (possibly with > extra verbosity enabled) > * system you are running this on (windows, macos, linux) flavour / version > details > * minor version of Python interpreter used > * whether you are using a packaged version from your os distributor or from > python.org (or even built your own) > * If you believe it's a regression the version it is working on > > Some more guidance can be found here covering some of the above > https://docs.python.org/3/bugs.html > > I am very interested to note that a report filed in February hasn't had at > least one person take a brief look at it and post a note or set a status > like "needs more info" etc. > > Also I am by no means an expert in multi processing at all as so far my > webapps work worker processes, I am wondering if the wrong approach to this > may be happening and so the client is trying to reuse the same pipe and it > may need a different tear down / process to release that and create a new > pipe / socket / underlying connection cleanly rather than the re connect in > the same object. > > For example I am running python 3.7.7 on windows: > > Manager Output: >> python manager.py > In test_method > >> python manager.py > > > Client Output: > result: ', TEST' > Kill and restart the server and press return > > Got exception , ConnectionResetError(10054, > 'An existing connection was forcibly closed by the remote host', None, > 10054, None) > Reconnecting > Got exception , ConnectionResetError(10054, > 'An existing connection was forcibly closed by the remote host', None, > 10054, None) > Reconnecting > Got exception , ConnectionResetError(10054, > 'An existing connection was forcibly closed by the remote host', None, > 10054, None) > Reconnecting > Got exception , ConnectionResetError(10054, > 'An existing connection was forcibly closed by the remote host', None, > 10054, None) > Reconnecting > Got exception , ConnectionResetError(10054, > 'An existing connection was forcibly closed by the remote host', None, > 10054, None) > Reconnecting > Got exception , ConnectionResetError(10054, > 'An existing connection was forcibly closed by the remote host', None, > 10054, None) > Reconnecting > Got exception , ConnectionResetError(10054, > 'An existing connection was forcibly closed by the remote host', None, > 10054, None) > Reconnecting > Got exception , ConnectionResetError(10054, > 'An existing connection was forcibly closed by the remote host', None, > 10054, None) > Reconnecting > Got exception , ConnectionResetError(10054, > 'An existing connection was forcibly closed by the remote host', None, > 10054, None) > Reconnecting > Got exception , ConnectionResetError(10054, > 'An existing connection was forcibly closed by the remote host', None, > 10054, None) > Reconnecting > Got exception , ConnectionResetError(10054, > 'An existing connection was forcibly closed by the remote host', None, > 10054, None) > Reconnecting > Got exception , ConnectionResetError(10054, > 'An existing connection was forcibly closed by the remote host', None, > 10054, None) > Reconnecting # At this point I terminated the manager > Got exception , > ConnectionRefusedError(10061, 'No connection could be made because the > target machine actively refused it', None, 10061, None) > Reconnecting > Traceback (most recent call last): > File > "C:\Users\Alexander\AppData\Local\Programs\Python\Python37\lib\multiprocessing\connection.py", > line 619, in SocketClient >s.connect(address) > ConnectionRefusedError: [WinError 10061] No connection could be made > because the target machine actively refused it > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File "client.py", line 27, in >manager.connect() > File > "C:\Users\Alexander\AppData\Local\Programs\Python\Python37\lib\multiprocessing\managers.py", > line 532, in connect >conn = Client(self._address, authkey=self._authkey) > File > "C:\Users\Alexander\AppData\Local\Programs\Python\Python37\lib\multip
Re: Python for Android
> On 19 Jul 2021, at 18:43, Bischoop wrote: > > > Will Python delevopment apps for Android OS getting easier? > So far best option is Kivy which is not liked by many developers, > another option is PyQT5, which finally gets some support but there still > are some buts.. > Tkinter, some tried but it's still a NO for Android support. Buts? My understanding is the PyQt5 has good support for Android. I have not personal used that support, but I have used PyQt5 for none trivial projects on Linux, macOs and windows and like it. > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: annotations cause dataclass fields type side effects
> On 9 Aug 2021, at 19:28, Lukas Lösche wrote: > > I'm on Python 3.9.6 and trying to make sense of the following behaviour: > >>>> from dataclasses import dataclass, fields >>>> @dataclass > ... class Foobar: > ... name: str > ... >>>> fields(Foobar)[0].type > >>>> type(fields(Foobar)[0].type) > > > > >>>> from __future__ import annotations >>>> from dataclasses import dataclass, fields >>>> >>>> @dataclass > ... class Foobar: > ... name: str > ... >>>> fields(Foobar)[0].type > 'str' >>>> type(fields(Foobar)[0].type) > > > I have a validation function that checks if the types of all fields in > a dataclass are what they are supposed to be. But as soon as I import > annotations from __future__ this validation breaks as the arg that I'm > passing to isinstance() is no longer a type class but a string. > > What am I doing wrong? Nothing. As I understand the current situation you have to check if it’s a string and convert to the object you need at the moment. Hopefully someone with a deeper understanding will explain how you do this. I would guess use eval. This is a known problem and there are core devs that are working on improvements. But it will be at leas python 3.11 before there is a possible improvement. Barry > > Thanks, > -- Lukas > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: basic auth request
> On 17 Aug 2021, at 19:25, Chris Angelico wrote: > > On Wed, Aug 18, 2021 at 4:16 AM Barry Scott wrote: >> Oh and if you have the freedom avoid Basic Auth as its not secure at all. >> > > That's usually irrelevant, since the alternative is most likely to be > form fill-out, which is exactly as secure. If you're serving over > HTTPS, the page is encrypted, and that includes the headers; if you're > not, then it's not encrypted, and that includes the form body. There is digest and Ntlm that do not reveal the password. If you are over TLS then form or base is as good as each other. Barry > > There are other issues with basic auth, but security really isn't one. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: PyQt5 is not recognized from python 3.8 installation in python 3.10
> On 23 Aug 2021, at 00:00, Mats Wichmann wrote: > > On 8/22/21 7:04 AM, Mohsen Owzar wrote: >> Hi guys, >> I have on my laptop the python installation 3.8 and newly I installed newest >> version 3.10 as well on my laptop. >> Now I have two IDLEs for both of the installations. >> When I rund some *.py file, having PyQt5 module, on the 3.8 version, it >> works as before without any problems. >> But whenn I run the same file on IDLE with 3.10 installation, it crashes and >> says that PyQt5 is not known. >> I tried to install this package with"pip install pyqt5" or with "py -3 -m >> pip install pyqt5", it brings lots of error messages (long long) as the >> lines below (they are only the beginning of the messages): > > make sure you're getting the right Python by doing > > py -3.10 -m pip install pyqt5 > > (the fact you're getting the build error below indicates you _are_ getting > the right one, but it's good to be explicit anyway, just so you're sure). > >> == >> C:\Users\Mohsen>py -3 -m pip install pyqt5 >> Collecting pyqt5 >> Using cached PyQt5-5.15.4.tar.gz (3.3 MB) >> Installing build dependencies ... done >> Getting requirements to build wheel ... done >> Preparing wheel metadata ... error >> ERROR: Command errored out with exit status 1: >> command: >> 'C:\Users\Mohsen\AppData\Local\Programs\Python\Python310\python.exe' >> 'C:\Users\Mohsen\AppData\Local\Programs\Python\Python310\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py' >> prepare_metadata_for_build_wheel >> 'C:\Users\Mohsen\AppData\Local\Temp\tmprax0esmt' >> cwd: >> C:\Users\Mohsen\AppData\Local\Temp\pip-install-wl_b58e9\pyqt5_1cbd1bab46fa4abaad34b55514561ce6 >> Complete output (33 lines): >> Querying qmake about your Qt installation... >> C:\Qt\4.7.4\bin\qmake.exe -query >> Traceback (most recent call last): >> File >> "C:\Users\Mohsen\AppData\Local\Programs\Python\Python310\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", >> line 143, in prepare_metadata_for_build_wheel >> hook = backend.prepare_metadata_for_build_wheel >> AttributeError: module 'sipbuild.api' has no attribute >> 'prepare_metadata_for_build_wheel' > > Your problem here is that there's no released binary wheel for PyQt5, since > Py 3.10 isn't released (there's been some recent chatter about encouraging > projects to make builds for new Python versions available early, e.g. once a > Python version hits release candidate state, but that's just that - chatter - > for now). > > In theory your system should be able to build it, but my experience is that > unless you're an active Python developer on Windows and already have the > right working setup, it pretty much always fails. These are the perils of > trying to use a pre-release... > > That error suggests there's something different going on than usual (which > has to do with the MSVC compiler suite not being set up the way the package > expects): there looks like a version mismatch - it looks like it's finding > qmake for Qt4, not Qt5. Maybe you can make some progress by adjusting some > paths? Only trivial C code extensions can be built on windows. And then only if you have the right MSVC tools installed. Most interesting extensions have dependencies only other software. Which means that you need to know how to build all the dependencies as well. For something as complex as PyQt5 you need to be a developer with reasonable experience to build it. Check on PyPI for the available versions and install one of them. Barry > > Many people go here to get early-access, albeit unofficial, binary wheels: > > https://www.lfd.uci.edu/~gohlke/pythonlibs > > Unfortunately, PyQt5 is not available from there (as listed in the section at > the very bottom) > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: basic auth request
> On 25 Aug 2021, at 20:34, Eli the Bearded <*@eli.users.panix.com> wrote: > > In comp.lang.python, Jon Ribbens wrote: >> Another attempt at combatting this problem is DNS CAA records, >> which are a way of politely asking all CAs in the world except the >> ones you choose "please don't issue a certificate for my domain". >> By definition someone who had hacked a CA would pay no attention >> to that request, of course. > > Yeah, but it works for the case of forgotten hostnames, a rare but > real attack. Basically it works like this: > > $COMPANY puts out a lot of things on different IP addresses from > a shared public(ish) pool like AWS and assigns different names > to them. Later $COMPANY discontinues one or more of those things, > terminates the host, and lets the IP address rejoin the public(ish) > pool. > > $ATTACKER notices the domain name pointing to an unused IP address > and works to acquire it for their own server. $ATTACKER then gets > a cert for that domain, since they can easily prove ownership of > the server through http content challenges. $ATTACKER now has a > host in $COMPANY's name to launch phishing attacks. > > This probably has some clever infosec name that I don't know. It is possible to sign an ip address in a certificate, but that is not often done. Getting to reuse the IP address that example.com was using will not help the attacker unless they can make a cert that signs the dns name. And that means they hacked the CA which is a big problem. Barry > > Elijah > -- > or a clever infosec name now forgotten > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: on the popularity of loops while and for
> On 28 Aug 2021, at 22:42, Hope Rouselle wrote: > > I'd like get a statistic of how often each loop is used in practice. > > I was trying to take a look at the Python's standard libraries --- those > included in a standard installation of Python 3.9.6, say --- to see > which loops are more often used among while and for loops. Of course, > since English use the preposition ``for'' a lot, that makes my life > harder. Removing comments is easy, but removing strings is harder. So > I don't know yet what I'll do. > > Have you guys ever measured something like that in a casual or serious > way? I'd love to know. Thank you! I am interesting in why you think that choice of while vs. for is about popularity? Surely the choice is made in most cases by the algorithm? If you have an iterator then you use for. Of course you can use while instead of for but in code review that will get queried. Barry > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
