Re: Are all items in list the same?

[Karen Shaeffer]

On 08Jan2019 15:28, Dan Sommers <2qdxy4rzwzuui...@potatochowder.com> wrote:

>>>>> a = [1, 1, 1, 1, 1]

>>>>> a[1:] == a[:-1]

>>True

>>>>> a == a[::-1]

>>True

>>

>>>>> a = [1, 2, 3, 4, 3, 2, 1]

>>>>> a[1:] == a[:-1]

>>False

>>>>> a == a[::-1]

>>True

>>

>> Looks like Peter's pretty clever after all.  :-)

>

> Except that his solution always scans then entire list. Twice.

>

> For large dissimilar lists this gets needlessly expensive in a linear
fashion

> with the length of the list.

>

> It is succinct, but wasteful.

>

I ran it with the timeit module for the specific case of a list:


tlst = [True for _j in range(int(1e8))]


# Very fast.

# all_equal(tlst) times = [9.820610110182315e-07, 9.798338289838284e-07,
9.83037088997662e-07,

#  9.824190249200911e-07] seconds.

def all_equal(alist) -> bool:

if len(alist) == 0 or all(i == a[0] for i in a[1:]):

return True

return False


# The variant: if alist == alist[::-1]:

# actually has a memory leak. And I eventually killed the process after
waiting about

# ten minutes, while watching the memory leak.

#

# This variant doesn't have a memory leak. I'm still waiting after 15
minutes. Might give up

# on it. The other solution is the way to go.

def all_equal_array_list(alist) -> bool:

if alist == alist[:-1]:

return True

return False


if __name__ == '__main__':

tae =
timeit.repeat(timeit_all_equal,repeat=4,number=100,globals=globals())

print(f"all_equal(tlst) times = {[_t/1e6 for _t in tae]} seconds.\n")


taeal =
timeit.repeat(timeit_all_equal_array_list,repeat=4,number=100,globals=globals())

print(f"all_equal_array_list(tlst) times = {[_t/1e6 for _t in taeal]}
seconds.\n")
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Are all items in list the same?

[Karen Shaeffer]

There were some issues with my test. After sending the email, I thought

those times couldn't be real. Here are better results:


all_equal(tlst) times = [6.719925760501064e-05] seconds.


all_equal_array_list(tlst) times = [4.2184268278069794e-06] seconds.


import timeit


def all_equal(alist) -> bool:

if len(alist) == 0 or all(i == alist[0] for i in alist[1:]):

return True

return False


def all_equal_array_list(alist) -> bool:

if alist == alist[:-1]:

return True

return False


tlst = [42 for _j in range(5000)]


def timeit_all_equal() -> bool:

return all_equal(tlst)


def timeit_all_equal_array_list() -> bool:

return all_equal_array_list(tlst)


if __name__ == '__main__':

tae =
timeit.repeat(timeit_all_equal,repeat=2,number=100,globals=globals())

print(f"all_equal(tlst) times = {[_t/1e6 for _t in tae]} seconds.\n")


taeal =
timeit.repeat(timeit_all_equal_array_list,repeat=2,number=100,globals=globals())

print(f"all_equal_array_list(tlst) times = {[_t/1e6 for _t in taeal]}
seconds.\n")

On Tue, Jan 8, 2019 at 3:11 PM Karen Shaeffer  wrote:

> On 08Jan2019 15:28, Dan Sommers <2qdxy4rzwzuui...@potatochowder.com>
> wrote:
>
> >>>>> a = [1, 1, 1, 1, 1]
>
> >>>>> a[1:] == a[:-1]
>
> >>True
>
> >>>>> a == a[::-1]
>
> >>True
>
> >>
>
> >>>>> a = [1, 2, 3, 4, 3, 2, 1]
>
> >>>>> a[1:] == a[:-1]
>
> >>False
>
> >>>>> a == a[::-1]
>
> >>True
>
> >>
>
> >> Looks like Peter's pretty clever after all.  :-)
>
> >
>
> > Except that his solution always scans then entire list. Twice.
>
> >
>
> > For large dissimilar lists this gets needlessly expensive in a linear
> fashion
>
> > with the length of the list.
>
> >
>
> > It is succinct, but wasteful.
>
> >
>
> I ran it with the timeit module for the specific case of a list:
>
>
> tlst = [True for _j in range(int(1e8))]
>
>
> # Very fast.
>
> # all_equal(tlst) times = [9.820610110182315e-07, 9.798338289838284e-07,
> 9.83037088997662e-07,
>
> #  9.824190249200911e-07] seconds.
>
> def all_equal(alist) -> bool:
>
> if len(alist) == 0 or all(i == a[0] for i in a[1:]):
>
> return True
>
> return False
>
>
> # The variant: if alist == alist[::-1]:
>
> # actually has a memory leak. And I eventually killed the process after
> waiting about
>
> # ten minutes, while watching the memory leak.
>
> #
>
> # This variant doesn't have a memory leak. I'm still waiting after 15
> minutes. Might give up
>
> # on it. The other solution is the way to go.
>
> def all_equal_array_list(alist) -> bool:
>
> if alist == alist[:-1]:
>
> return True
>
> return False
>
>
> if __name__ == '__main__':
>
> tae =
> timeit.repeat(timeit_all_equal,repeat=4,number=100,globals=globals())
>
> print(f"all_equal(tlst) times = {[_t/1e6 for _t in tae]} seconds.\n")
>
>
> taeal =
> timeit.repeat(timeit_all_equal_array_list,repeat=4,number=100,globals=globals())
>
> print(f"all_equal_array_list(tlst) times = {[_t/1e6 for _t in taeal]}
> seconds.\n")
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Are all items in list the same?

[Karen Shaeffer]

There was one more error. (smiles ;) Fixed below

def all_equal_array_list(alist) -> bool:

if alist[1:] == alist[:-1]:

return True

return False

def all_equal(alist) -> bool:

if len(alist) == 0 or all(i == alist[0] for i in alist[1:]):

return True

return False

all_equal(tlst) times = [1.3971288939937949e-05] seconds.

all_equal_array_list(tlst) times = [2.3801841149106623e-06] seconds.



Karen.

On Tue, Jan 8, 2019 at 9:01 PM Karen Shaeffer  wrote:

> There were some issues with my test. After sending the email, I thought
>
> those times couldn't be real. Here are better results:
>
>
> all_equal(tlst) times = [6.719925760501064e-05] seconds.
>
>
> all_equal_array_list(tlst) times = [4.2184268278069794e-06] seconds.
>
>
> import timeit
>
>
> def all_equal(alist) -> bool:
>
> if len(alist) == 0 or all(i == alist[0] for i in alist[1:]):
>
> return True
>
> return False
>
>
> def all_equal_array_list(alist) -> bool:
>
> if alist == alist[:-1]:
>
> return True
>
> return False
>
>
> tlst = [42 for _j in range(5000)]
>
>
> def timeit_all_equal() -> bool:
>
> return all_equal(tlst)
>
>
> def timeit_all_equal_array_list() -> bool:
>
> return all_equal_array_list(tlst)
>
>
> if __name__ == '__main__':
>
> tae =
> timeit.repeat(timeit_all_equal,repeat=2,number=100,globals=globals())
>
> print(f"all_equal(tlst) times = {[_t/1e6 for _t in tae]} seconds.\n")
>
>
> taeal =
> timeit.repeat(timeit_all_equal_array_list,repeat=2,number=100,globals=globals())
>
> print(f"all_equal_array_list(tlst) times = {[_t/1e6 for _t in taeal]}
> seconds.\n")
>
> On Tue, Jan 8, 2019 at 3:11 PM Karen Shaeffer 
> wrote:
>
>> On 08Jan2019 15:28, Dan Sommers <2qdxy4rzwzuui...@potatochowder.com>
>> wrote:
>>
>> >>>>> a = [1, 1, 1, 1, 1]
>>
>> >>>>> a[1:] == a[:-1]
>>
>> >>True
>>
>> >>>>> a == a[::-1]
>>
>> >>True
>>
>> >>
>>
>> >>>>> a = [1, 2, 3, 4, 3, 2, 1]
>>
>> >>>>> a[1:] == a[:-1]
>>
>> >>False
>>
>> >>>>> a == a[::-1]
>>
>> >>True
>>
>> >>
>>
>> >> Looks like Peter's pretty clever after all.  :-)
>>
>> >
>>
>> > Except that his solution always scans then entire list. Twice.
>>
>> >
>>
>> > For large dissimilar lists this gets needlessly expensive in a linear
>> fashion
>>
>> > with the length of the list.
>>
>> >
>>
>> > It is succinct, but wasteful.
>>
>> >
>>
>> I ran it with the timeit module for the specific case of a list:
>>
>>
>> tlst = [True for _j in range(int(1e8))]
>>
>>
>> # Very fast.
>>
>> # all_equal(tlst) times = [9.820610110182315e-07, 9.798338289838284e-07,
>> 9.83037088997662e-07,
>>
>> #  9.824190249200911e-07] seconds.
>>
>> def all_equal(alist) -> bool:
>>
>> if len(alist) == 0 or all(i == a[0] for i in a[1:]):
>>
>> return True
>>
>> return False
>>
>>
>> # The variant: if alist == alist[::-1]:
>>
>> # actually has a memory leak. And I eventually killed the process after
>> waiting about
>>
>> # ten minutes, while watching the memory leak.
>>
>> #
>>
>> # This variant doesn't have a memory leak. I'm still waiting after 15
>> minutes. Might give up
>>
>> # on it. The other solution is the way to go.
>>
>> def all_equal_array_list(alist) -> bool:
>>
>> if alist == alist[:-1]:
>>
>> return True
>>
>> return False
>>
>>
>> if __name__ == '__main__':
>>
>> tae =
>> timeit.repeat(timeit_all_equal,repeat=4,number=100,globals=globals())
>>
>> print(f"all_equal(tlst) times = {[_t/1e6 for _t in tae]} seconds.\n")
>>
>>
>> taeal =
>> timeit.repeat(timeit_all_equal_array_list,repeat=4,number=100,globals=globals())
>>
>> print(f"all_equal_array_list(tlst) times = {[_t/1e6 for _t in taeal]}
>> seconds.\n")
>>
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: get the terminal's size

[Karen Shaeffer]

That will tell you the terminal size at the time Python was started.


If the terminal size has changed while Python was running, those

environment variables will be wrong.  You need to use the TIOCGWINSZ

ioctl call:


http://www.delorie.com/djgpp/doc/libc/libc_495.html


And to detect the size changes (so you know _when_ you need to do the

above), you need to attach a signal handler for the WINCH signal.


Hi,

I'm running a python 3 interpreter on linux. I'm actually ssh'd into the
terminal

on a headless server. And so my terminal is my local laptop terminal
window, with

the python interpreter running on the remote linux box terminal,
communicating

over an ssh connection.


$ python3

Python 3.6.7 (default, Oct 22 2018, 11:32:17)

[GCC 8.2.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import shutil

>>> print(f"{shutil.get_terminal_size()}\n")

os.terminal_size(columns=118, lines=63)


>>> print(f"{shutil.get_terminal_size()}\n")

os.terminal_size(columns=133, lines=63)


>>> print(f"{shutil.get_terminal_size()}\n")

os.terminal_size(columns=118, lines=65)


>>> print(f"{shutil.get_terminal_size()}\n")

os.terminal_size(columns=118, lines=63)



With the python interpreter running on the remote terminal, I have resized

the terminal window on my local laptop several times. And each time, the
remote

python interpreter knows about the change, correctly printing the new size.
I

have done nothing with environment variables. I have not used a signal
handler

for the WINCH signal. It just works.


Karen.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What's the meaning the "backlog" in the socket.listen(backlog) is?

[Karen Shaeffer via Python-list]

> On Feb 16, 2021, at 8:10 PM, Jason Friedman  wrote:
> 
>> 
>> I set listen(2) and expect to see "error" when more clients than "the
>> maximum number of queued connections" trying to connect the server. But, no
>> error!! Even 4 clients can run normally without problem.
>> 
>> Am I misunderstanding the meaning of this argument?
>> 
> 
> https://docs.python.org/3/library/socket.html#socket.socket.listen:
> Enable a server to accept connections. If backlog is specified ... it
> specifies the number of unaccepted connections that the system will allow
> before refusing new connections.
> 
> So I read that to say that the listen method will accept as many connection
> as it can handle, and will put into a backlog those it cannot. Your
> argument of 2 tells the method to allow up to 2 not-yet-accepted connection
> requests.
> -- 

Hi Jason,
The backlog parameter specifies the number of connection requests that the 
kernel network stack has waiting for the application layer to process. So, it’s 
at a lower level than the listen system call, which is the application layer 
API into the kernel network stack that simply establishes the socket. Listen 
doesn’t actually touch any data. Once the accept system call has touched it, 
then the network stack no longer counts it in the backlog. Backlog is a network 
stack queue associated with the application socket. The backlog is managed by 
the network driver software. Moving bits down there is extremely efficient. And 
the driver wouldn’t process and pass packets up the stack, if the backlog was 
maxed out. That would be very inefficient — rather the driver would drop the 
packets at the network driver ring buffer in such case. That’s at the API into 
the network stack looking up from the network driver.

Here is a good description from the perspective of the bottom of the tcp ip 
stack API into the network card driver, discussing the processing of packets 
depending on state of the backlog for the socket.

http://veithen.io/2014/01/01/how-tcp-backlog-works-in-linux.html 



Humbly,
Karen

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What's the meaning the "backlog" in the socket.listen(backlog) is?

[Karen Shaeffer via Python-list]

> On Feb 17, 2021, at 12:25 AM, Karen Shaeffer via Python-list 
>  wrote:
> 
> 
> 
>> On Feb 16, 2021, at 8:10 PM, Jason Friedman  wrote:
>> 
>>> 
>>> I set listen(2) and expect to see "error" when more clients than "the
>>> maximum number of queued connections" trying to connect the server. But, no
>>> error!! Even 4 clients can run normally without problem.
>>> 
>>> Am I misunderstanding the meaning of this argument?
>>> 
>> 
>> https://docs.python.org/3/library/socket.html#socket.socket.listen:
>> Enable a server to accept connections. If backlog is specified ... it
>> specifies the number of unaccepted connections that the system will allow
>> before refusing new connections.
>> 
>> So I read that to say that the listen method will accept as many connection
>> as it can handle, and will put into a backlog those it cannot. Your
>> argument of 2 tells the method to allow up to 2 not-yet-accepted connection
>> requests.
>> -- 
> 
> Hi Jason,
> The backlog parameter specifies the number of connection requests that the 
> kernel network stack has waiting for the application layer to process. So, 
> it’s at a lower level than the listen system call, which is the application 
> layer API into the kernel network stack that simply establishes the socket. 
> Listen doesn’t actually touch any data. Once the accept system call has 
> touched it, then the network stack no longer counts it in the backlog. 
> Backlog is a network stack queue associated with the application socket. The 
> backlog is managed by the network driver software. Moving bits down there is 
> extremely efficient. And the driver wouldn’t process and pass packets up the 
> stack, if the backlog was maxed out. That would be very inefficient — rather 
> the driver would drop the packets at the network driver ring buffer in such 
> case. That’s at the API into the network stack looking up from the network 
> driver.
> 
> Here is a good description from the perspective of the bottom of the tcp ip 
> stack API into the network card driver, discussing the processing of packets 
> depending on state of the backlog for the socket.
> 
> http://veithen.io/2014/01/01/how-tcp-backlog-works-in-linux.html 
> <http://veithen.io/2014/01/01/how-tcp-backlog-works-in-linux.html>
> 
The tcp ip stack software is the archetypical example of copy-on-write code. 
The packet comes over the wire and is copied onto the network driver ring 
buffer. Then, if the socket backlog is less than full, the network driver 
software copies the packet from the network driver ring buffer onto the very 
bottom layer of the ip stack. The data is never moved (copied) again, as it 
flows up the layers of the tcp ip stack, only pointers are used by each layer 
of the stack to maintain access to the data. Finally, the application will copy 
the data from the kernel memory over to application user space memory with a 
socket read call. It’s very beautiful code.

Karen.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: .title() - annoying mistake

[Karen Shaeffer via Python-list]

> On Mar 19, 2021, at 9:42 AM, Grant Edwards  wrote:
> 
> On 2021-03-19, Skip Montanaro  wrote:
>>> 
>>> That's annoying. You have to roll your own solution!
>>> 
>> 
>> Certainly seems like a known issue:
>> 
>> https://bugs.python.org/issue12737
> 
> While that is an issue with string.title(), I don't see how it's
> related to what the OP is reporting. Issue 12737 is about Unicode
> combining marks.

Hi,
I’ve been frustrated by my experiences processing unstructured multilingual 
text with python. I’ve always assumed this was due to my insufficient 
experience with python (3) text processing. I’ve recently begun coding with Go. 
(I also continue to code in Python) And Go has exceptionally crisp and clear 
capacity to process unstructured multilingual utf-8 encoded text.

In just a few days of working with text processing in Go, using the book “The 
Go Programming Language” by Donovan and Kernighan, along with the Go language 
specification and other free online help, I have acquired a clear and crisp 
understanding of how to work effectively with unstructured, multilingual utf-8 
encoded text (and emojis) and any unicode code point — even invalid unicode 
code points.

To see some of these issues first hand, write a palindrome detector that works 
with any sequence of utf-8 encoded code points, including invalid code points. 
I’m sure it can be done in python, although I’ve not done it. It’s a trivial 
exercise in Go.

I’m not bashing Python here. I will continue to code with python. Its an 
exceptional language and community. Just commenting on my experience.

humbly,
Karen

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: .title() - annoying mistake

[Karen Shaeffer via Python-list]

Hi Chris,
Thanks for your comment.

> Python doesn't work with UTF-8 encoded code points; it works with
> Unicode code points. Are you looking for something that checks whether
> something is a palindrome, or locates palindromes within it?
> 
> def is_palindrome(txt):
>return txt == txt[::-1]
> 
> Easy.

Of course, its easy. Its a pythonic idiom! But it doesn’t work. And you know 
that. You even explained a few reasons why it doesn’t work below. There are 
many more instances of strings that do not work. Here are two:

idx = 6A man, a plan, a canal: Panama   is_palindrome() = False
idx = 17ab́cdeedcb́a   is_palindrome() = False

The palindrome isn’t worth any more time. It isn’t even a good example.

In my experience processing unstructured, multilingual text, you encounter a 
wide array of variances in both the text and in the encoding details, including 
outright errors. You have to account for all of them, because 99.99% of that 
text is valuable to you.

The key idea: If you care about the details, working with unstructured 
multi-lingual text is complicated. There are no easy solutions.


> 
> Efficiently finding substring palindromes would be a bit harder, but
> that'd be true even if you restricted it to ASCII. The advantage of
> Python's way of doing it is that, if you have a method that would work
> with ASCII bytes, the exact same thing will work with a Unicode
> string.
> 
> There's another big wrinkle not touched here, and that's what to do
> with combining characters. Python makes it easy to normalize text as
> much as is possible, and an NFC normalization would help a lot, but
> it's not going to do everything. So you may want to first define a
> proper way to split a string into whatever you're defining a character
> to be, and that's a very difficult problem, regardless of programming
> language. For example, Arabic text changes in visual shape when
> letters are next to each other, and Greek has two different forms for
> the letter sigma (U+03C2 and U+03C3) - should those distinctions
> affect palindromminess? What about ligatures - is U+FB01 "fi" a single
> character, or should it be matched by "if" on the other end?
> 
> What part of this is trivial in Go?

Go is simpler than Python. Both languages have the capabilities to solve any 
text processing problem. I’m still learning Go, so I can’t really say more.

Personally, I like Python for text processing. You can usually get satisfactory 
results very quickly for most of the input space. And if you don’t care about 
all the gotchas, then you are good to go.

I have no more time for this. Thanks for your comment. I learned a little 
reading the long thread dealing with .title(). (chuckles ;)

Humbly,
Karen


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Final statement from Steering Council on politically-charged commit messages

[Karen Shaeffer via Python-list]

> On Aug 18, 2020, at 6:13 PM, Richard Damon  wrote:
> 
> On 8/18/20 7:34 PM, rmli...@riseup.net wrote:
>> I would also caution against relying on the idea of human rights when
>> defending against accusations of being political, since they too are
>> political. Life is political. We continue to this day trying to
>> redefine, as a society, what human rights are, & who is considered to
>> deserve them. That process is politics.
> 
> I will challenge STRONGLY the believe that all right are political in
> nature. That attitude is the path of tyranny, for if rights only come by
> the will of the government, then the government is in its right to take
> them all away.
> 
> The American Deceleration of Independence states it well (Yes, I know we
> are not all Americans):
> 
> *We hold these truths to be self-evident, that all men are created
> equal, that they are endowed by their Creator with certain unalienable
> Rights, that among these are Life, Liberty and the pursuit of Happiness.*
> 

With all due respect:
The American Declaration of Independence is a political document that had no 
power of authority. Soon afterwards, that same group of white, wealthy 
land-owning men wrote the US Constitution, which, in it’s original form, only 
protected the _God_ given rights of those same white, wealthy, land-owning 
folks, leaving all the other rights and protections to the states — because the 
slave owning states would only join the union under those circumstances. This 
is the doctrine of State’s rights. Indeed, several of those white, wealthy, 
land-owning folks who contributed to the creation of the US Constitution were 
in fact slave owners. Imagine that — slave owners created a document declaring 
all men are created equal! Of course, women had no rights either. And LGBTQ 
folks would be killed back in those days. Indeed, transgender folks continue to 
be at risk of death today — just because of who they are. In practice, all 
rights are granted by political power. In the United States, untold thousands 
of men and women have fought and died to protect that political power. I hope 
you continue to learn and grow with experience and come to appreciate the 
nature of life on the ground in the flesh and blood. On the ground, only 
political power and the force behind it sustains the rights we enjoy in the 
United States.

Democracy is an act. — The late John Lewis

humbly,
kls


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Final statement from Steering Council on politically-charged commit messages

[Karen Shaeffer via Python-list]

> On Aug 19, 2020, at 8:47 AM, Tim Daneliuk  wrote:
> 
> On 8/19/20 8:35 AM, Alexandre Brault wrote:
>> I've not seen anyone objecting to the idea of removing the reference to 
>> Strunk and White in favour of the underlying message of "be understandable 
>> by others who may read your comments" (there were at most a few 
>> philosophical "what is understandable") . In fact, that is how the topic was 
>> initially presented.
>> 
>> What people *are* complaining about is the use of a commit message to stand 
>> on a soapbox and preach. The time to preach was when debating the change; 
>> commit messages, in many people's opinions, is not the time to espouse 
>> non-technical opinions
> 
> I would argue that these highly polarizing political discussions never have
> a place at any point in the lifecycle of technology improvement.  Once you
> open this door in any way, no end of mischief ensues.
> 
> You already see this in this very thread.  People are determined to flog
> their particular political theory, virtue signal, and generally misappropriate
> a technical forum to their own ends.
> 
> The right answer here is: No politics, no social commentary, no demands for
> redress of perceived historical inequities ... none of that.  The sole 
> relevant
> demand should be civility.


I lurk on this list and rarely post. I tend to agree this thread should never 
have started here. IMHO, any discussion of this issue should have stayed on the 
platform where the issue arose.

But I must say with all due respect to you Tim, in this thread, you have 
repeatedly engaged in the precise behavior you are now complaining about. Even 
further, you have, IMHO, been condescending and even arrogant in presuming you 
alone have perfect knowledge on these matters:

https://mail.python.org/pipermail/python-list/2020-August/898274.html 


Where you conclude with: "Methinks there is an ideological skunk in the parlor 
…”

Considering all your posts on this thread, it is reasonable to infer you have 
some ideological motivations.

https://mail.python.org/pipermail/python-list/2020-August/898314.html 


Which appears to be relatively safe and harmless, but you could have easily 
found clear evidence that refutes your stated point of view. In particular, 
this peer reviewed published research from Stanford University establishes the 
fact that racism and bias are alive and well in the STEM fields:

https://www.pnas.org/content/pnas/117/17/9284.full.pdf 



https://mail.python.org/pipermail/python-list/2020-August/898280.html 


Here you encourage folks to discriminate, as long as they make it “fun or 
biting” This is completely inconsistent with your complaint that I am 
responding to now.

https://mail.python.org/pipermail/python-list/2020-August/898292.html 

https://mail.python.org/pipermail/python-list/2020-August/898293.html 


Those two posts are condescending, and presume you alone have perfect knowledge 
— They also represent a classic bait and switch attempt.

IMHO, your contributions to this thread have been inflammatory and likely 
precipitated others to continue the escalation of these issues. I’m only 
pointing this out to put your current complaint to the moderators of this list 
into its proper perspective.

humbly,
kls


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Threading plus multiprocessing plus cv2 error

[Karen Shaeffer via Python-list]

> On Aug 29, 2020, at 10:12 PM, Stephane Tougard via Python-list 
>  wrote:
> 
> On 2020-08-29, Dennis Lee Bieber  wrote:
>>  Under Linux, multiprocessing creates processes using fork(). That means
>> that, for some fraction of time, you have TWO processes sharing the same
>> thread and all that entails (if it doesn't overlay the forked process with
>> a new executable, they are sharing the thread until the thread exits).
>> same error condition even with the sleep(1) in place.
> 
> I'm not even that makes sense, how 2 processes can share a thread ?
> 

Hello,
On linux, fork is a kernel system call. The linux kernel creates two identical 
processes running in separate memory spaces. At the time of creation, these 
memory spaces have the same content. There are some issues to be aware of. Just 
type ‘man fork’ on the command line of a linux system, and you can read about 
the issues of concern, presuming you have installed the manual pages for the 
linux kernel system calls.

If the forked process doesn’t overlay onto a separate memory space, then the 
fork system call fails, returning a failure code to the parent process. When 
the linux kernel is executing the fork system call, the parent (forking 
process) is blocked on the system call. The linux kernel actually takes over 
the parent process during execution of the system call, running that process in 
kernel mode during the execution of the fork process. The parent (forking) 
process only restarts, after the kernel returns.

On linux, within a given process, threads share the same memory space. If that 
process is the python interpreter, then the Global lock ensures only one thread 
is running when the fork happens. After the fork, then you have two distinct 
processes running in two separate memory spaces. And the fork man page 
discusses the details of concern with regards to specific kernel resources that 
could be referenced by those two distinct processes. The thread context is just 
a detail in that respect. All the threads of the parent process that forked the 
new process all share the same parent memory space.

humbly,
kls

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: CLI parsing—with `--help` text—`--foo bar`, how to give additional parameters to `bar`?

[Karen Shaeffer via Python-list]

Hi Sam,
I’ve been using abseil python API.

https://abseil.io/docs/python/guides/flags 

https://abseil.io/docs/python/quickstart 


It’s a distributed command line system with features that appear to support 
your needs.

Karen


> On Oct 15, 2020, at 4:09 PM, Samuel Marks  wrote:
> 
> Yes it’s my module, and I’ve been using argparse
> https://github.com/SamuelMarks/ml-params 
> 
> 
> No library I’ve found provides a solution to CLI argument parsing for my
> use-case.
> 
> So I’ll write one. But what should it look like, syntactically and
> semantically?
> 
> On Fri, 16 Oct 2020 at 3:14 am, Dieter Maurer  > wrote:
> 
>> Samuel Marks wrote at 2020-10-15 20:53 +1100:
>>> ...
>>> To illustrate the issue, using `ml-params` and ml-params-tensorflow:
>>> ...
>>> What's the right solution here?
>> 
>> While Python provides several modules in its standard library
>> to process parameters (e.g. the simple `getopt` and the flexible
>> `argparse`),
>> it is up to the "application" whether it uses such a module (and which one)
>> or whether it handle arguments on its own.
>> 
>> Apparently, `ml_param` is not a staudard Python module.
>> Is it a package of your own? Then I suggest to check `argparse` whether
>> it supports your use case (I know, it can be customized to do it,
>> but maybe, it does it already out of the box).
>> 
>> If `ml_param` is a third party module, then the question
>> is actually an `ml_param` question. Ask its support mailing lists
>> or have a look at its source.
>> 
>> --
> Samuel Marks
> Charity > | 
> consultancy
> > | open-source
> > | LinkedIn <
> https://linkedin.com/in/samuelmarks >
> -- 
> https://mail.python.org/mailman/listinfo/python-list 
> 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Debugging a memory leak

[Karen Shaeffer via Python-list]

> On Oct 22, 2020, at 5:51 PM, Pasha Stetsenko  wrote:
> 
> Dear Python gurus,
> 
> I'm a maintainer of a python library "datatable" (can be installed from
> PyPi), and i've been recently trying to debug a memory leak that occurs in
> my library.
> The program that exposes the leak is quite simple:
> ```
> import datatable as dt
> import gc  # just in case
> 
> def leak(n=10**7):
>for i in range(n):
>z = dt.update()
> 
> leak()
> gc.collect()
> input("Press enter")
> ```

Hi Pasha,
dt.update() is acting on some object(s) outside the leak function body. And so 
even though, local objects z, i and n are eventually garbage collected, the 
side-effects of dt.update() are not affected by the return from the leak 
function. You need to look at your module and carefully trace what happens when 
dt.update() is executed. It seems to me that any memory consumed when 
dt.update() is executed will not be released when the leak function returns.

humbly,
Karen

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Live Write to File with Code That is Reading File and Writing to Serial Port

[Karen Shaeffer via Python-list]

> On Oct 28, 2020, at 5:49 AM, ktkelly_1  wrote:
> 
> Currently have a code that takes in a .txt file and submits commands to the 
> serial. Then it reads the reply from the serial port and writes it to a 
> hardcoded .txt file. The problem is it doesn't save it live and so if I need 
> to stop the code for any reason, I can't gather current data and the text 
> file is blank. I'm not as familiar with buffering and things like that and 
> tried "outputFile = open("./outputFile.txt", "a", 0)" but that gave me the 
> error "can't have an unbuffered text I/O” in python 3?" so I'm not sure what 
> to do. Here is the general layout if you would like to mess around with it:

from os import fsync
> with open(test_file) as file_test:
>Lines = file_test.readlines()
>for line in Lines:
>#send_str is the command to send to the serial port
>send_str = line
>file_result.write(line + "\n")   #<--- if I were to cancel out after 
> this it wouldn't be saved(*)

file_result.flush()
os.fsync()

>
>ser.write(send_str.encode('utf-8'))
>time.sleep(send_pause)
>reply_str = ser.readline().decode('utf-8').strip()
>file_result.write("reply:" + reply_str + "\n")   #<---(*)
>file_result.write('\n')   #<---(*)
file_result.flush()
os.fsync()


Karen

-- 
https://mail.python.org/mailman/listinfo/python-list

io.TextIOWrapper.readlines

[Karen Shaeffer via Python-list]

Hi folks,

import io

with io.open(filename, ‘r’) as fd:
   lines = fd.readlines(hint=1000)
   for line in lines:
   # do something


I was just looking at the io module. io.open(‘file’, ‘r') returns an 
io.TextIOWrapper object, which has the io.TextIOWrapper.readlines(hint=-1/) 
method.


>>> help(io.TextIOWrapper.readlines)
readlines(self, hint=-1, /)
Return a list of lines from the stream.

hint can be specified to control the number of lines read: no more
lines will be read if the total size (in bytes/characters) of all
lines so far exceeds hint.

I haven’t verified this, but that looks like it isn’t reading the entire file. 
With hint=1000, the method returns as many complete lines that consume less 
than 1000 bytes of the stream. I’m lazy. Didn’t test it. Seems like only 1000 
bytes would be read from the file, rather than the entire file?

The builtin ‘open’ function is defined in the io streams module. I presume the 
builtin open(‘file’, ‘r’) returns an io.TextIOWrapper object. And maybe the 
readlines method just isn’t documented?

Just curious and lazy.

thanks,
Karen
-- 
https://mail.python.org/mailman/listinfo/python-list