Re: Like c enumeration in python3 [ERRATA]

[Peter Otten]

Paulo da Silva wrote:

> Às 02:18 de 23/03/20, Paulo da Silva escreveu:
>> Hi!
>> 
>> Suppose a class C.
>> I want something like this:
>> 
>> class C:
>> KA=0
>> KB=1
> KC=2
>> ...
>> Kn=n
>> 
>> def __init__ ...
>> ...
>> 
>> 
>> These constants come from an enum in a .h (header of C file).
>> They are many and may change from time to time.
>> Is there a way to somehow define them from inside __init__ giving for
>> example a list of names as strings?
>> There is an additional problem: C is not recognized inside __init__!
> Of course I'm talking about C class name, not C language.

Use enum:

>>> import enum
>>> C = enum.IntEnum("C", ["KA", "KB", "KC"], start=0)
>>> C.KB

>>> C.KB == 1
True

https://docs.python.org/3/library/enum.html#functional-api

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Frank Millman]

On 2020-03-22 12:11 PM, Chris Angelico wrote:

On Sun, Mar 22, 2020 at 8:30 PM Frank Millman  wrote:


On 2020-03-22 10:45 AM, Chris Angelico wrote:


If you can recreate the problem with a single socket and multiple
requests, that would be extremely helpful. I also think it's highly
likely that this is the case.



I am working on a stripped-down version, but I realise there are a few 
things I have not grasped.


Hope you don't mind, but can you scan through what follows and tell me 
if I am on the right lines?


Both the client and the server can send a header with 'Keep-alive', but 
what does it actually mean?


If the client sends it, does that mean that it wants the server to keep 
the connection open, and only close it when the client closes it from 
the other end?


Conversely, if the server sends it, does it mean that it wants the 
client to keep the connection open? If so, under what condition would 
the server close the connection (other than a timeout). Should the 
server only send 'Keep-alive' if it receives 'Keep-alive'?


In my program, when I send a file in response to a GET, I send a header 
with 'Keep-alive', then I send the file, then I close the connection. 
This now seems wrong. It could even be the cause of my bug, but then why 
has it only appeared now? Both Edge and Chrome send 'Keep-alive' headers.


If I am thinking along the right lines, then the exchange should go like 
this -


Client sends request, with 'Keep-alive' header.

Server sends response, but does not close the connection. The server 
should reply with a 'Keep-alive' header. If it does not, the client will 
close the connection, in which case the server will also close.


Assuming that they both send 'Keep-alive', the onus is on the client to 
close the connection when it has no more requests. The server should 
have a timeout in case the client goes away.


Assuming that the above is correct, the client will rely on 
'Content-length' to determine when it has received the entire request. 
If the client has more than one request, it will send the first, wait 
for the response, when fully received as per the 'Content-length' it 
will send the next one, until all requests have been sent and all 
responses received, at which point it will close the connection.


All this assumes only one connection. Alternatively the client could 
open multiple connections for the requests. In that case it would make 
sense to use 'Connection: Close', so that the server can close the 
connection straight away, making it available for reuse.


If this all makes sense, I should write two versions of the client 
program, one using a single connection, and one using a pool of connections.


All comments appreciated!

Frank

--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Chris Angelico]

On Mon, Mar 23, 2020 at 8:03 PM Frank Millman  wrote:
>
> On 2020-03-22 12:11 PM, Chris Angelico wrote:
> > On Sun, Mar 22, 2020 at 8:30 PM Frank Millman  wrote:
> >>
> >> On 2020-03-22 10:45 AM, Chris Angelico wrote:
> >
> > If you can recreate the problem with a single socket and multiple
> > requests, that would be extremely helpful. I also think it's highly
> > likely that this is the case.
> >
>
> I am working on a stripped-down version, but I realise there are a few
> things I have not grasped.
>
> Hope you don't mind, but can you scan through what follows and tell me
> if I am on the right lines?

No probs!

> Both the client and the server can send a header with 'Keep-alive', but
> what does it actually mean?
>
> If the client sends it, does that mean that it wants the server to keep
> the connection open, and only close it when the client closes it from
> the other end?
>
> Conversely, if the server sends it, does it mean that it wants the
> client to keep the connection open? If so, under what condition would
> the server close the connection (other than a timeout). Should the
> server only send 'Keep-alive' if it receives 'Keep-alive'?

I'm pretty sure the server should never say "keep-alive" if the client
said "close". (There's a bit of an oddity in that the default changed
between, I think, HTTP 1.0 and 1.1. But that's a minor discrepancy.)
Generally, you'll see one of these options:

1) Client says "Connection: close". This means "I'm only gonna be
doing this one request, then we're done". Good for non-browser usage,
automated scripts etc.

2) Client says "Keep-Alive", server says "Close". This means the
client would be okay with connection reuse, but the server's saying no
(maybe it doesn't support it, or maybe it's in a soft shutdown state
where it wants to end connections, or whatever). After the current
request, the connection will be closed.

3) Both ends say "Connection: Keep-Alive". After the request, the
socket isn't closed, and can be reused for some other request.

I'm not sure how a client would (or should) handle it if the server
says "Connection: Keep-Alive" after the client said "Connection:
Close". Probably the client would just close its end of the socket and
that'd be that.

> In my program, when I send a file in response to a GET, I send a header
> with 'Keep-alive', then I send the file, then I close the connection.
> This now seems wrong. It could even be the cause of my bug, but then why
> has it only appeared now? Both Edge and Chrome send 'Keep-alive' headers.
>

It mightn't be fundamentally WRONG (there needs to be *some* way to
signal that you're now done), but it would probably be suboptimal. If
you're always going to close the connection after one request, just
send Connection: Close.

> If I am thinking along the right lines, then the exchange should go like
> this -
>
> Client sends request, with 'Keep-alive' header.
>
> Server sends response, but does not close the connection. The server
> should reply with a 'Keep-alive' header. If it does not, the client will
> close the connection, in which case the server will also close.
>
> Assuming that they both send 'Keep-alive', the onus is on the client to
> close the connection when it has no more requests. The server should
> have a timeout in case the client goes away.

I think either end is allowed to close the connection.

> Assuming that the above is correct, the client will rely on
> 'Content-length' to determine when it has received the entire request.
> If the client has more than one request, it will send the first, wait
> for the response, when fully received as per the 'Content-length' it
> will send the next one, until all requests have been sent and all
> responses received, at which point it will close the connection.

Yep. Which makes things very interesting (for serene values of
"interesting", such as "we're all gonna die") when you want to use
connection pooling with something that fundamentally doesn't have a
length, such as a chunked transfer. Which you are doing. That may or
may not actually be significant.

> All this assumes only one connection. Alternatively the client could
> open multiple connections for the requests. In that case it would make
> sense to use 'Connection: Close', so that the server can close the
> connection straight away, making it available for reuse.

What you'll often see is that the client has some sort of connection
limit, and reuses those same connections. For instance, it might have
a maximum of four connections per server (where "server" has to
include the name as well as its IP and port, due to TLS SNI), so it
requests an HTML page on one connection (with keep-alive), parses
that, and then fires off one request on the reused connection and
three others on new connections. As those requests complete, it sends
new requests on the same sockets.

> If this all makes sense, I should write two versions of the client
> program, one using a single connection, and one using a

Re: Like c enumeration in python3

[Skip Montanaro]

Does the enum module help?

https://docs.python.org/3/library/enum.html

Note that within __init__ you can reference your enumerated constants
using (for example) self.KA.

Skip
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Like c enumeration in python3

[Rhodri James]

On 23/03/2020 02:18, Paulo da Silva wrote:

Hi!

Suppose a class C.
I want something like this:

class C:
KA=0
KB=1
KC=1
...
Kn=n

def __init__ ...
...


These constants come from an enum in a .h (header of C file).
They are many and may change from time to time.
Is there a way to somehow define them from inside __init__ giving for
example a list of names as strings?


It sounds like what you want is a separate script to process your .h 
file into a Python module containing an enum, then import that module 
into your program.


--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Frank Millman]

On 2020-03-23 12:57 PM, Chris Angelico wrote:

On Mon, Mar 23, 2020 at 8:03 PM Frank Millman  wrote:


On 2020-03-22 12:11 PM, Chris Angelico wrote:

On Sun, Mar 22, 2020 at 8:30 PM Frank Millman  wrote:


On 2020-03-22 10:45 AM, Chris Angelico wrote:


If you can recreate the problem with a single socket and multiple
requests, that would be extremely helpful. I also think it's highly
likely that this is the case.



I am working on a stripped-down version, but I realise there are a few
things I have not grasped.

Hope you don't mind, but can you scan through what follows and tell me
if I am on the right lines?


No probs!



[...]

Really appreciate the one-on-one tuition. I am learning a lot!




If this all makes sense, I should write two versions of the client
program, one using a single connection, and one using a pool of connections.



Possibly! I think you'll most likely see that one of those behaves
perfectly normally, and you only trigger the issue in the other. So
you could move forward with just one test program.



Well, I have got the first one working - single connection - and so far 
it has not gone wrong.


However, it is difficult to be sure that I am comparing apples with 
apples. I have written my test server to handle 'Keep-Alive' correctly, 
but as I mentioned earlier, my live program closes the connection after 
each transfer. So now I have to make my test server do the same, and 
change my test client to react to that and re-open the connection each 
time. I will make the changes and see how that behaves.


Of course now I am in the murky waters of trying to second-guess how 
Edge reacts to that. Presumably that is where Wireshark will be useful. 
I will keep you posted.


Frank

--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Barry Scott]

> On 23 Mar 2020, at 09:02, Frank Millman  wrote:
> 
> On 2020-03-22 12:11 PM, Chris Angelico wrote:
>> On Sun, Mar 22, 2020 at 8:30 PM Frank Millman  wrote:
>>> 
>>> On 2020-03-22 10:45 AM, Chris Angelico wrote:
>> If you can recreate the problem with a single socket and multiple
>> requests, that would be extremely helpful. I also think it's highly
>> likely that this is the case.
> 
> I am working on a stripped-down version, but I realise there are a few things 
> I have not grasped.
> 
> Hope you don't mind, but can you scan through what follows and tell me if I 
> am on the right lines?
> 
> Both the client and the server can send a header with 'Keep-alive', but what 
> does it actually mean?
> 
> If the client sends it, does that mean that it wants the server to keep the 
> connection open, and only close it when the client closes it from the other 
> end?
> 
> Conversely, if the server sends it, does it mean that it wants the client to 
> keep the connection open? If so, under what condition would the server close 
> the connection (other than a timeout). Should the server only send 
> 'Keep-alive' if it receives 'Keep-alive'?

See https://tools.ietf.org/html/rfc2616  
secton 14.10 for the details.

When the browser sends "Connection: keep-alive" it mean that the browser wishes 
to have
a persistent connection. The server is free to choose.

If the server replies with "Connection: close" the connection is not persistent 
and will
be closed after the response.

If the server replies with "Connection: keep-aiive" then connect is persistent 
that means that
after this response the server will allow another transaction, GET, POST etc.

Note: You need to know if the client is using HTTP/1.0 or HTTP/1.1 protocol the 
rules are different in
each version.

> 
> In my program, when I send a file in response to a GET, I send a header with 
> 'Keep-alive', then I send the file, then I close the connection. This now 
> seems wrong. It could even be the cause of my bug, but then why has it only 
> appeared now? Both Edge and Chrome send 'Keep-alive' headers.

Its a protocol violation. In your case you must send "Connection: close". Fix 
that and I think your code is likely to work.

If you care about performance add the extra code to handle persistent 
connections. Its speeds up the browsers
and should reduce the impact on the server as well.


> If I am thinking along the right lines, then the exchange should go like this 
> -
> 
> Client sends request, with 'Keep-alive' header.
> 
> Server sends response, but does not close the connection. The server should 
> reply with a 'Keep-alive' header. If it does not, the client will close the 
> connection, in which case the server will also close.

You must include a "Connection:" head for HTTP/1.1 I believe.

> 
> Assuming that they both send 'Keep-alive', the onus is on the client to close 
> the connection when it has no more requests.

Yes.

> The server should have a timeout in case the client goes away.

Yes to stop resources running out.

> 
> Assuming that the above is correct, the client will rely on 'Content-length' 
> to determine when it has received the entire request. If the client has more 
> than one request, it will send the first, wait for the response, when fully 
> received as per the 'Content-length' it will send the next one, until all 
> requests have been sent and all responses received, at which point it will 
> close the connection.

You can use "Content-Length" header or chunked encoding.

> 
> All this assumes only one connection. Alternatively the client could open 
> multiple connections for the requests. In that case it would make sense to 
> use 'Connection: Close', so that the server can close the connection straight 
> away, making it available for reuse.

Modern browsers open many persistent connection. I recall its 8/site in the 
case of chrome.


> 
> If this all makes sense, I should write two versions of the client program, 
> one using a single connection, and one using a pool of connections.

Is this for a test program?

You can use curl for the single connection case.


> 
> All comments appreciated!
> 
> Frank

Barry

> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Like c enumeration in python3

[Barry Scott]

> On 23 Mar 2020, at 11:52, Rhodri James  wrote:
> 
> On 23/03/2020 02:18, Paulo da Silva wrote:
>> Hi!
>> Suppose a class C.
>> I want something like this:
>> class C:
>>  KA=0
>>  KB=1
>>  KC=1
>>  ...
>>  Kn=n
>>  def __init__ ...
>>  ...
>> These constants come from an enum in a .h (header of C file).
>> They are many and may change from time to time.
>> Is there a way to somehow define them from inside __init__ giving for
>> example a list of names as strings?
> 
> It sounds like what you want is a separate script to process your .h file 
> into a Python module containing an enum, then import that module into your 
> program.

web search for "python parse C header" yields some interesting results.

Barry


> 
> -- 
> Rhodri James *-* Kynesim Ltd
> -- 
> https://mail.python.org/mailman/listinfo/python-list 
> 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Chris Angelico]

On Mon, Mar 23, 2020 at 10:58 PM Frank Millman  wrote:
>
> On 2020-03-23 12:57 PM, Chris Angelico wrote:
> > On Mon, Mar 23, 2020 at 8:03 PM Frank Millman  wrote:
> >>
> >> On 2020-03-22 12:11 PM, Chris Angelico wrote:
> >>> On Sun, Mar 22, 2020 at 8:30 PM Frank Millman  wrote:
> 
>  On 2020-03-22 10:45 AM, Chris Angelico wrote:
> >>>
> >>> If you can recreate the problem with a single socket and multiple
> >>> requests, that would be extremely helpful. I also think it's highly
> >>> likely that this is the case.
> >>>
> >>
> >> I am working on a stripped-down version, but I realise there are a few
> >> things I have not grasped.
> >>
> >> Hope you don't mind, but can you scan through what follows and tell me
> >> if I am on the right lines?
> >
> > No probs!
> >
>
> [...]
>
> Really appreciate the one-on-one tuition. I am learning a lot!

Heh, I'm always happy to help people who are willing to learn. Also,
you've brought an interesting problem to the table. :)

> Of course now I am in the murky waters of trying to second-guess how
> Edge reacts to that. Presumably that is where Wireshark will be useful.
> I will keep you posted.
>

Yep, it's going to be a matter of monitoring the exact headers sent
each direction.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Like c enumeration in python3 [ERRATA]

[Terry Reedy]

On 3/22/2020 11:37 PM, DL Neil via Python-list wrote:

On 23/03/20 3:32 PM, Paulo da Silva wrote:

Às 02:18 de 23/03/20, Paulo da Silva escreveu:

Hi!


Olá,



Suppose a class C.
I want something like this:

class C:
KA=0
KB=1

KC=2

...
Kn=n

def __init__ ...
    ...


These constants come from an enum in a .h (header of C file).
They are many and may change from time to time.
Is there a way to somehow define them from inside __init__ giving for
example a list of names as strings?
There is an additional problem: C is not recognized inside __init__!



Please read the PSL docs carefully, because Python's enums seem to 
differ from those in other languages - sometimes in significant ways, 
and sometimes in a subtle manner!



I have been asking similar questions recently - particularly wanting to 
(also) use the "strings", because of the effort of upgrading many 
modules of code at the same time (the manual, or was it the PEP(?) 
refers to the need to upgrade the PSL to use enums, and to the effort 
that might cost - I suggest there has been little/v.slow take-up, but 
then keeping the PSL updated is a subject of a great many conversations, 
elsewhere).


Python's enums cannot be changed. Once set, C.KA cannot be changed to 1 


I am not sure what you are saying.  The OP's class is not an enum in the 
enum module sense.  Class attributues are usually mutable.


>>> class C:
K = 0


>>> C.K = 1
>>> C.K
1


--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

Re: Confusing textwrap parameters, and request for RE help

[Dieter Maurer]

Chris Angelico wrote at 2020-3-23 06:00 +1100:
>When using textwrap.fill() or friends, setting break_long_words=False
>without also setting break_on_hyphens=False has the very strange
>behaviour that a long hyphenated word will still be wrapped.

Having worked with `TeX`, I am familiar that hyphens indicate
word locations favorable to line breaks. Therefore, I understand
that one wants a way to control whether or not breaks are allowed
at hyphens, even if words should not be broken normally.

Apparently, "break_on_hyphens" adds hyphen as a word delimiter.
With this notion, the behavior can be understood.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Like c enumeration in python3

[Paulo da Silva]

Thank you very much for all your responses!
Now I have too much ideas :-)
I'm saving your answers and I'll see what is more
appropriate/comfortable in my case.

Best regards.
Paulo
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to copy paragraphs (with number formatting) and images from Words (.docx) and paste into Excel (.xlsx) using Python

[Beverly Pope]

On Mar 22, 2020, at 11:12 PM, A S  wrote:
> 
> On Monday, 23 March 2020 01:58:38 UTC+8, Beverly Pope  wrote:
>>> On Mar 22, 2020, at 9:47 AM, A S >> > wrote:
>>> 
>>> I can't seem to paste pictures into this discussion so please see both my 
>>> current and desired Excel output here:
>>> 
>>> https://stackoverflow.com/questions/60800494/how-to-copy-paragraphs-with-number-formatting-and-images-from-words-docx-an
>>>  
>>> >>  
>>> >
>> Did you try using the 2 part answer on the stackoverflow webpage?
>> 
>> Bev in TX
> 
> I'm able to get the paragraphs copied correctly now! But i'm trying to figure 
> out if there's a way to copy and paste the images into the Excel, along with 
> the paragraphs as well. Do you have an idea? :)

I’m glad to hear that solution worked for you.  With that said, I only went to 
stackoverflow out of curiosity and happened ro see the posted solution. I 
probably know less about using Python to copy data from Word to Excel than you 
do, given that yesterday was the first time that I had heard about it.  

I did read that MS docx files are zip files, which can be unzipped in Python.
https://gist.github.com/another-junior-dev/990a4e622868627cb93be3d8fa2eff04 

That could provide access to the pictures contained in the document, but it 
doesn’t explain how to determine where you want to place the pictures with 
relation to text in your Excel spreadsheet.  If you could determine that, then 
you could use XlsxWriter module’s worksheet.insert_image() to insert the image. 
 See n”the Worksheet Class” in the XlsxWriter docs:
https://xlsxwriter.readthedocs.io 
The rest is beyond the realm of my knowledge.

If it were me, I would go back to stackoveflow and open a new question, as this 
is different than your original post.

Bev in TX
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: `async def` breaks encapsulation?

[Marco Sulla]

On Tue, 17 Mar 2020 at 08:36, Greg Ewing  wrote:
>
> On 4/03/20 12:52 pm, Marco Sulla wrote:
> > Why can't an asynchronous coroutine be simply a coroutine that has an
> > `async` or an `await` in its code, without `async` in the signature?
>
> That wouldn't help as much as you seem to think. You still need
> to use 'await' whenever you call a coroutine, so switching a
> function between coroutine and non-coroutine would still have
> just as much of a ripple effect on the rest of your code.

Good point. But what if you don't need `await`?
A coroutine call `mycoro()` could act as the actual `await mycoro()`.
If you really want `mycoro()` actual behavior, you could write `async
mycoro()`.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: `async def` breaks encapsulation?

[Greg Ewing]

On 24/03/20 7:25 am, Marco Sulla wrote:

But what if you don't need `await`?
A coroutine call `mycoro()` could act as the actual `await mycoro()`.
If you really want `mycoro()` actual behavior, you could write `async
mycoro()`.


The way async/await is implemented, it's not feasible to wait
until runtime to decide whether a given call should be treated
as an await or not. Different bytecode is generated for it,
and the behaviour of the enclosing function is quite different
right from the start, before the await is even reached.

Other implementation strategies are possible that would
enable "async" and "await" to be eliminated. But there are
many people, including Guido, who regard the need for these
as a feature rather than a bug, because they mark the
places where suspensions can occur. So you would have to
convince those people that these keywords *should* be
eliminated, before thinking about how to achieve it.

--
Greg

--
https://mail.python.org/mailman/listinfo/python-list

[RELEASE] Python 3.9.0a5 is now available for testing

[Łukasz Langa]

On behalf of the entire Python development community, and the currently serving 
Python release team in particular, I’m pleased to announce the release of 
Python 3.9.0a5. Get it here:

https://www.python.org/downloads/release/python-390a5/ 


This is an early developer preview of Python 3.9

Python 3.9 is still in development. This releasee, 3.9.0a5 is the fifth of six 
planned alpha releases. Alpha releases are intended to make it easier to test 
the current state of new features and bug fixes and to test the release 
process. During the alpha phase, features may be added up until the start of 
the beta phase (2020-05-18) and, if necessary, may be modified or deleted up 
until the release candidate phase (2020-08-10). Please keep in mind that this 
is a preview release and its use is not recommended for production environments.

Major new features of the 3.9 series, compared to 3.8

Many new features for Python 3.9 are still being planned and written. Among the 
new major new features and changes so far:

PEP 584 , Union Operators in dict
PEP 593 , Flexible function and 
variable annotations
PEP 602 , Python adopts a stable 
annual release cadence
BPO 38379 , garbage collection does not 
block on resurrected objects;
BPO 38692 , os.pidfd_open added that allows 
process management without races and signals;
BPO 39926 , Unicode support updated to 
version 13.0.0
BPO 1635741 , when Python is initialized 
multiple times in the same process, it does not leak memory anymore
A number of Python builtins (range, tuple, set, frozenset, list) are now sped 
up using PEP 570  vectorcall
A number of standard library modules (audioop, ast, grp, _hashlib, pwd, 
_posixsubprocess, random, select, struct, termios, zlib) are now using the 
stable ABI defined by PEP 384 .
(Hey, fellow core developer, if a feature you find important is missing from 
this list, let Łukasz know .)
The next pre-release, the last alpha release of Python 3.9, will be 3.9.0a6. It 
is currently scheduled for 2020-04-22. Until then, stay safe!

Your friendly release team,
Ned Deily @nad 
Steve Dower @steve.dower 
Łukasz Langa @ambv 
-- 
https://mail.python.org/mailman/listinfo/python-list