Re: Nested functions, how do they work (stack related)

[Veek M]

Veek M wrote:

> I was reading the wiki on 'Call stack' because I wanted to understand
> what a traceback object was. My C/C++ isn't good enough to deal with
> raw python source since I have no background in CS. Also, you just
> can't dive into the python src - it takes a good deal of reading and
> background.. (the types will be confusing for a start)
> 
> https://en.wikipedia.org/wiki/Call_stack
> 
> 'Programming languages that support nested subroutines also have a
> field in the call frame that points to the stack frame of the latest
> activation of the procedure that most closely encapsulates the callee,
> i.e. the immediate scope of the callee. This is called an access link
> or static link (as it keeps track of static nesting during dynamic and
> recursive calls) and provides the routine (as well as any other
> routines it may invoke) access to the local data of its encapsulating
> routines at every nesting level.
> 
> Some architectures, compilers, or optimization cases store one link
> for each enclosing level (not just the immediately enclosing), so that
> deeply nested routines that access shallow data do not have to
> traverse several links; this strategy is often called a "display".'
> 
> 1. What is the difference between a 'call frame' and a 'stack frame'
> in the above context? I know that a stack frame is all the data
> related to one - CALL foo;
> 
> 2. He's saying that within the 'call frame' (whatever that is) there's
> an address to one of the previous stack frames of the wrapper function
> ? What does all that mean in terms of nested functions? Access link?
> How are nested function stacks setup..
> 
> 3. What exactly is a traceback object - we know that an instance
> object is a dictionary and some glue logic that allows you to pretend
> that methods are stored within the instance and call using x.sin(self)
> etc. But I was reading: pydoc traceback AND:
> http://effbot.org/librarybook/traceback.htm
> 
> 'Extract the raw traceback from the current stack frame'
> A stack frame contains (from wiki) the parameters, local variables,
> next instruction address so.. what's a raw traceback - does the
> default exception handler realize 'okay error' and then walk the stack
> and extract data and prettify it for display and build a magical
> traceback object? Is this documented for dummies what exactly it does?
> (i know that's what it's doing but I HAVE NO CLUE so.. are there books
> on this)
> 
> How exactly does an exception fit in with tracebacks? How does all
> this fit in with nested functions?
> 
> 4. When you call a nested function (decorator), it generally returns a
> wrapper function but I thought he was just returning a reference to a
> function object but obviously since it can see it's environment, how
> is the stack being setup?

found this:
http://www.drdobbs.com/cpp/how-nested-functions-work-part-1/228701476
(still reading it)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: IndexError: list index out of range

[Peter Otten]

Elnaz wrote:

> hi
> i am begginer in python. I have written a code and given this error:
> IndexError: list index out of range
> 
> In my program, I have h=32 bits input. i divide this 32 bits to 4*8 block
> and every 8-block is n. so n=0:7;(h=int(n/4)) I want to rotate 0 to 7 bits
> for 2 bits: 0,1,2,3,4,5,6,7--->2,3,4,5,6,7 Iwrite this code:
> def rottwo(self, X, n, r):
> assert r >= 1
> temp = [None]*n
> for i in range(n-r) :
> temp[i] = X[i+r]
> for i in range(n-r,n) :
> temp[i] = X[i-n+r]
> return temp
> this function work correctly. but I also want to rotate 24 to 31 bits for
> 5 bits: 24,25,26,27,28,29,30,31-->29,30,31,24,25,26,27,28
> 
> when I write this code:
> def rotfive(self, X, n, r):
>assert r >= 1
>temp = [None]*n
>for i in range(n-r) :
> temp[i+24] = X[i+3*n+r]
>for i in range(n-r,n) :
> temp[i+24] = X[i+2*n+r]
>return temp
> beacase temp is of size n I cannot access index 3*n+i. index on the list
> temp should be less than equal to n-1 . I son't know how I must correct
> this Is there any one to help me?
> thanks in advanse.

I think you are making this harder than necessary. Python slices make 
accessing parts of a list quite elegant:

>>> items
[0, 10, 20, 30, 40, 50, 60, 70, 80, 90]
>>> items[2:5]
[20, 30, 40]
>>> items[3:]
[30, 40, 50, 60, 70, 80, 90]

You can use this to implement a function that creates a rotated list with an 
arbitrary offset:

>>> def rot(items, offset):
... return items[offset:] + items[:offset]
... 
>>> rot(items, 2)
[20, 30, 40, 50, 60, 70, 80, 90, 0, 10]
>>> rot(items, 7)
[70, 80, 90, 0, 10, 20, 30, 40, 50, 60]
>>> rot(items, -2)
[80, 90, 0, 10, 20, 30, 40, 50, 60, 70]

To rotate part of a list extract that part using slice notation, rotate it 
and write it back:

>>> def rot_part(items, offset, start, stop):
... items = list(items)
... items[start:stop] = rot(items[start:stop], offset)
... return items
... 
>>> rot_part(range(32), 5, 24, 32)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 
21, 22, 23, 29, 30, 31, 24, 25, 26, 27, 28]

If you pass a list as the first argument

items = list(items) 

makes of copy of the list, but it will also convert an arbitrary iterable to 
a list. That's why I can pass the range object.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Nested functions, how do they work (stack related)

[Marko Rauhamaa]

Veek M :

> https://en.wikipedia.org/wiki/Call_stack
>
> 'Programming languages that support nested subroutines also have a field 
> in the call frame that points to the stack frame of the latest 
> activation of the procedure that most closely encapsulates the callee, 
> i.e. the immediate scope of the callee. This is called an access link or 
> static link (as it keeps track of static nesting during dynamic and 
> recursive calls) and provides the routine (as well as any other routines 
> it may invoke) access to the local data of its encapsulating routines at 
> every nesting level. 
>
> Some architectures, compilers, or optimization cases store one link for 
> each enclosing level (not just the immediately enclosing), so that 
> deeply nested routines that access shallow data do not have to traverse 
> several links; this strategy is often called a "display".'
>
> 1. What is the difference between a 'call frame' and a 'stack frame' in 
> the above context?

There's no difference.

> 2. He's saying that within the 'call frame' (whatever that is) there's 
> an address to one of the previous stack frames of the wrapper function ? 
> What does all that mean in terms of nested functions? Access link? How 
> are nested function stacks setup..

The classic C stack frame contains two addresses (in addition to the
arguments and local variables):

 * the return address in the calling function

 * the frame pointer in the calling function

Some languages (notably Pascal) add a third address:

 * the frame pointer in the outer function

Often, the outer function is the same as the calling function. However,
if the inner functions call each other, the outer function may be
further up the stack. Since the outer function's local variables are
seen by the inner functions, the extra pointer is needed to access them
directly.

Python has nested functions. Thus, the same technique can be used to
implement Python's internal call stack.

> 3. What exactly is a traceback object

It is an object that reports details of the call stack. It is mostly
useful for troubleshooting.

> How exactly does an exception fit in with tracebacks? How does all this 
> fit in with nested functions?

Well, the traceback object contains also the exception since it is
essential for troubleshooting.

> 4. When you call a nested function (decorator), it generally returns a 
> wrapper function but I thought he was just returning a reference to a 
> function object but obviously since it can see it's environment, how is 
> the stack being setup?

Now I don't exactly understand your question.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Nested functions, how do they work (stack related)

[Veek M]

http://web.archive.org/web/20111030134120/http://www.sidhe.org/~dan/blog/archives/000211.html
(great tail recursion article - best i've seen! SO doesn't really 
explain it unless you already knew it to begin with, but here's the 
link:http://stackoverflow.com/questions/310974/what-is-tail-call-optimization)

I found it useful to read because it deals with the stack. Basically 
when an exception occurs you need to mess with the stack so..
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Nested functions, how do they work (stack related)

[Antoon Pardon]

Op 13-12-16 om 08:13 schreef Veek M:
> 4. When you call a nested function (decorator), it generally returns a 
> wrapper function but I thought he was just returning a reference to a 
> function object but obviously since it can see it's environment, how is 
> the stack being setup?

Here you are no longer just talking about nested functions, you are talking
about closures. A stack is no longer sufficient for implementing closures.
The environment for the nested variables of the closure is often alloceted
on the heap.

-- 
Antoon Pardon.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Nested functions, how do they work (stack related)

[Veek M]

Marko Rauhamaa wrote:

> Veek M :
> 
>> https://en.wikipedia.org/wiki/Call_stack
>>
>> 'Programming languages that support nested subroutines also have a
>> field in the call frame that points to the stack frame of the latest
>> activation of the procedure that most closely encapsulates the
>> callee, i.e. the immediate scope of the callee. This is called an
>> access link or static link (as it keeps track of static nesting
>> during dynamic and recursive calls) and provides the routine (as well
>> as any other routines it may invoke) access to the local data of its
>> encapsulating routines at every nesting level.
>>
>> Some architectures, compilers, or optimization cases store one link
>> for each enclosing level (not just the immediately enclosing), so
>> that deeply nested routines that access shallow data do not have to
>> traverse several links; this strategy is often called a "display".'
>>
>> 1. What is the difference between a 'call frame' and a 'stack frame'
>> in the above context?
> 
> There's no difference.
> 
>> 2. He's saying that within the 'call frame' (whatever that is)
>> there's an address to one of the previous stack frames of the wrapper
>> function ? What does all that mean in terms of nested functions?
>> Access link? How are nested function stacks setup..
> 
> The classic C stack frame contains two addresses (in addition to the
> arguments and local variables):
> 
>  * the return address in the calling function
> 
>  * the frame pointer in the calling function
> 
> Some languages (notably Pascal) add a third address:
> 
>  * the frame pointer in the outer function
> 
> Often, the outer function is the same as the calling function.
> However, if the inner functions call each other, the outer function
> may be further up the stack. Since the outer function's local
> variables are seen by the inner functions, the extra pointer is needed
> to access them directly.
> 
> Python has nested functions. Thus, the same technique can be used to
> implement Python's internal call stack.
> 
>> 3. What exactly is a traceback object
> 
> It is an object that reports details of the call stack. It is mostly
> useful for troubleshooting.
> 
>> How exactly does an exception fit in with tracebacks? How does all
>> this fit in with nested functions?
> 
> Well, the traceback object contains also the exception since it is
> essential for troubleshooting.
> 
>> 4. When you call a nested function (decorator), it generally returns
>> a wrapper function but I thought he was just returning a reference to
>> a function object but obviously since it can see it's environment,
>> how is the stack being setup?
> 
> Now I don't exactly understand your question.
> 
> 
> Marko

Umm.. here's an article on windows exception handling.. i was hoping for 
something like that.. (it's very badly written but informative about 
win32 exception handling - i'm still reading it) wanted something 
similar.. I'll quote the highlights.. I KNOW NOTHING about ANY exception 
handling so..

http://www.codeproject.com/KB/cpp/exceptionhandler.aspx

' On the Intel Win32 platform, the FS register always points to the
current TIB. Thus, at FS:[0] you can find a pointer to an 
EXCEPTION_REGISTRATION structure. Now I'm getting somewhere! When an 
exception occurs, the system looks at the TIB of the faulting
thread and retrieves a pointer to an EXCEPTION_REGISTRATION structure. 
In this structure is a pointer to an _except_handler callback function.
The operating system now knows enough to call the _except_handler 
function,'

'When you use a compiler's _try/_except syntax, the compiler also builds 
the EXCEPTION_REGISTRATION struct on the stack. I'm simply showing you a 
simplified version of what a compiler would do if you used 
_try/_except.'




-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Running python from pty without prompt

[Samuel Williams]

Michael, yes.

FYI, I found out why this works. Pressing Ctrl-D flushes the input
buffer. If you do this on an empty line, it causes read(...) to return
0 which Ruby considers end of input for the script, but the pipe is
not closed.
-- 
https://mail.python.org/mailman/listinfo/python-list

asyncio question

[Frank Millman]

Hi all

I had a problem with asyncio - not a programming problem, but one with 
organising my code to achieve a given result.


I have come up with a solution, but thought I would mention it here to see 
if there is a better approach.


I am using asyncio.start_server() to run a simple HTTP server. Each request 
is passed to a handler. I strive to complete each request as quickly as 
possible, but I use 'await' where necessary to prevent blocking. It all 
works well.


HTTP does not keep the connection open, it sends a message, waits for a 
response, and closes the connection. In order to maintain 'state' for 
concurrent users, I have a Session class, and each message contains a 
session id so that I can pass the message to the correct session instance. 
Again, it all works well.


The client uses AJAX to send messages to the server. It sends the message 
and continues processing, while a background task waits for the response and 
handles it appropriately. As a result, the client can send a second message 
before receiving a response to the first one. The server can detect this, 
but it cannot wait for the first message to complete, otherwise it will 
block other clients. I have not noticed any problems with processing 2 
requests from the same client concurrently, but I don't like it, so I want 
to process them sequentially.


Here is my solution. As I create each Session instance, I set up a 
background task, using asyncio.ensure_future, which sets up an 
asyncio.Queue. The request handler identifies the session that the request 
belongs to, and 'puts' the request onto that session's Queue. The background 
task runs a 'while True' loop waiting for requests. As they come in it 
'gets' them and processes them. It seems to work.


This means that I have a background task running for each concurrent user. 
Each one will be idle most of the time. My gut-feel says that this will not 
cause a problem, even if there are hundreds of them, but any comments will 
be welcome.


Thanks

Frank Millman


--
https://mail.python.org/mailman/listinfo/python-list

Re: Running python from pty without prompt

[Michael Torrie]

On 12/13/2016 05:39 AM, Samuel Williams wrote:
> Michael, yes.
> 
> FYI, I found out why this works. Pressing Ctrl-D flushes the input
> buffer. If you do this on an empty line, it causes read(...) to return
> 0 which Ruby considers end of input for the script, but the pipe is
> not closed.

Currently Python does not appear to support this behavior.  Possibly it
could be patched to support something similar, though.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: asyncio question

[Ian Kelly]

On Tue, Dec 13, 2016 at 6:15 AM, Frank Millman  wrote:
> The client uses AJAX to send messages to the server. It sends the message
> and continues processing, while a background task waits for the response and
> handles it appropriately. As a result, the client can send a second message
> before receiving a response to the first one. The server can detect this,
> but it cannot wait for the first message to complete, otherwise it will
> block other clients. I have not noticed any problems with processing 2
> requests from the same client concurrently, but I don't like it, so I want
> to process them sequentially.

Is there a particular reason why you're worried about this? The
browser is perfectly capable of keeping the requests straight. Also,
note that since the requests use separate HTTP connections, even if
the server sends its responses in a particular order, there's no
guarantee that the client will read them in that order, so this
doesn't free you from the need to allow the client to handle the
requests coming back in any order.

> Here is my solution. As I create each Session instance, I set up a
> background task, using asyncio.ensure_future, which sets up an
> asyncio.Queue. The request handler identifies the session that the request
> belongs to, and 'puts' the request onto that session's Queue. The background
> task runs a 'while True' loop waiting for requests. As they come in it
> 'gets' them and processes them. It seems to work.
>
> This means that I have a background task running for each concurrent user.
> Each one will be idle most of the time. My gut-feel says that this will not
> cause a problem, even if there are hundreds of them, but any comments will
> be welcome.

In a 64-bit Linux build of CPython, the combined size of a generator
and a stack frame is around half a kilobyte (not including whatever
space is needed for local variables), so hundreds of yielding asyncio
tasks should consume no more than hundreds of kilobytes of memory.
Remember that half a kilobyte figure is per generator, not per task,
so if your while loop is four coroutines deep, that will inflate the
cost of the task to two kilobytes each. This is different from the
threading model where each thread would need its own separate stack
space, not just a frame on the heap; you probably wouldn't want to do
this with threads, but with coroutines it should be fine.
-- 
https://mail.python.org/mailman/listinfo/python-list

Splitting text into lines

[George Trojan - NOAA Federal]

I have files containing ASCII text with line s separated by '\r\r\n'.
Example:

$ od -c FTAK31_PANC_131140.1481629265635
000   F   T   A   K   3   1   P   A   N   C   1   3   1   1
020   4   0  \r  \r  \n   T   A   F   A   B   E  \r  \r  \n   T   A
040   F  \r  \r  \n   P   A   B   E   1   3   1   1   4   0   Z
060   1   3   1   2   /   1   4   1   2   0   7   0   1   0
100   K   T   P   6   S   M   S   C   T   0   3   5   O
120   V   C   0   6   0  \r  \r  \n   F   M   1
140   3   2   1   0   0   1   0   0   1   2   G   2   0   K   T
160   P   6   S   M   B   K   N   1   0   0   W   S   0
200   1   5   /   1   8   0   3   5   K   T  \r  \r  \n
220   F   M   1   4   1   0   0   0   0   9   0   1   5
240   G   2   5   K   T   P   6   S   M   B   K   N   0   5
260   0   W   S   0   1   5   /   1   8   0   4   0   K   T   =
300  \r  \r  \n
303

What is the proper way of getting a list of lines?
Both
>>> open('FTAK31_PANC_131140.1481629265635').readlines()
['FTAK31 PANC 131140\n', '\n', 'TAFABE\n', '\n', 'TAF\n', '\n', 'PABE
131140Z 1312/1412 07010KT P6SM SCT035 OVC060\n', '\n', ' FM132100
10012G20KT P6SM BKN100 WS015/18035KT\n', '\n', ' FM141000 09015G25KT
P6SM BKN050 WS015/18040KT=\n', '\n']

and

>>> open('FTAK31_PANC_131140.1481629265635').read().splitlines()
['FTAK31 PANC 131140', '', 'TAFABE', '', 'TAF', '', 'PABE 131140Z 1312/1412
07010KT P6SM SCT035 OVC060', '', ' FM132100 10012G20KT P6SM BKN100
WS015/18035KT', '', ' FM141000 09015G25KT P6SM BKN050 WS015/18040KT=',
'']

introduce empty (or single character '\n') strings. I can do this:

>>> [x.rstrip() for x in open('FTAK31_PANC_131140.1481629265635',
'rb').read().decode().split('\n')]
['FTAK31 PANC 131140', 'TAFABE', 'TAF', 'PABE 131140Z 1312/1412 07010KT
P6SM SCT035 OVC060', ' FM132100 10012G20KT P6SM BKN100 WS015/18035KT',
' FM141000 09015G25KT P6SM BKN050 WS015/18040KT=', '']

but it looks cumbersome. I Python2.x I stripped '\r' before passing the
string to split():

>>> open('FTAK31_PANC_131140.1481629265635').read().replace('\r', '')
'FTAK31 PANC 131140\nTAFABE\nTAF\nPABE 131140Z 1312/1412 07010KT P6SM
SCT035 OVC060\n FM132100 10012G20KT P6SM BKN100 WS015/18035KT\n
FM141000 09015G25KT P6SM BKN050 WS015/18040KT=\n'

but Python 3.x replaces '\r\r\n' by '\n\n' on read().

Ideally I'd like to have code that handles both '\r\r\n' and '\n' as the
split character.

George
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Splitting text into lines

[Thomas Nyberg]

On 12/13/2016 08:45 AM, George Trojan - NOAA Federal wrote:

Ideally I'd like to have code that handles both '\r\r\n' and '\n' as the
split character.

George

Are repeated newlines/carriage returns significant at all? What about 
just using re and just replacing any repeated instances of '\r' or '\n' 
with '\n'? I.e. something like


>>> # the_string is your file all read in
>>> import re
>>> re.sub("[\r\n]+", "\n", the_string)

and then continuing as before (i.e. splitting by newlines, etc.)

Does that work?

Cheers,
Thomas
--
https://mail.python.org/mailman/listinfo/python-list

Re: Splitting text into lines

[Peter Otten]

George Trojan - NOAA Federal wrote:

> I have files containing ASCII text with line s separated by '\r\r\n'.

> but it looks cumbersome. I Python2.x I stripped '\r' before passing the
> string to split():
> 
 open('FTAK31_PANC_131140.1481629265635').read().replace('\r', '')
> 'FTAK31 PANC 131140\nTAFABE\nTAF\nPABE 131140Z 1312/1412 07010KT P6SM
> SCT035 OVC060\n FM132100 10012G20KT P6SM BKN100 WS015/18035KT\n
> FM141000 09015G25KT P6SM BKN050 WS015/18040KT=\n'
> 
> but Python 3.x replaces '\r\r\n' by '\n\n' on read().

Tell Python to keep the newline chars as seen with

open(filename, newline="")

For example:

>>> open("odd-newlines.txt", "rb").read()
b'alpha\nbeta\r\r\ngamma\r\r\ndelta\n'

>>> open("odd-newlines.txt", "r", newline="").read().replace("\r", 
"").splitlines()
['alpha', 'beta', 'gamma', 'delta']

> 
> Ideally I'd like to have code that handles both '\r\r\n' and '\n' as the
> split character.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Running python from pty without prompt

[Michael Torrie]

On 12/13/2016 09:01 AM, Michael Torrie wrote:
> On 12/13/2016 05:39 AM, Samuel Williams wrote:
>> Michael, yes.
>>
>> FYI, I found out why this works. Pressing Ctrl-D flushes the input
>> buffer. If you do this on an empty line, it causes read(...) to return
>> 0 which Ruby considers end of input for the script, but the pipe is
>> not closed.
> 
> Currently Python does not appear to support this behavior.  Possibly it
> could be patched to support something similar, though.

I wonder if you could write a python wrapper that would read the input
file from standard in until you get a ctrl-d, then exec() that input.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Splitting text into lines

[George Trojan - NOAA Federal]

>
> Are repeated newlines/carriage returns significant at all? What about
> just using re and just replacing any repeated instances of '\r' or '\n'
> with '\n'? I.e. something like
>  >>> # the_string is your file all read in
>  >>> import re
>  >>> re.sub("[\r\n]+", "\n", the_string)
> and then continuing as before (i.e. splitting by newlines, etc.)
> Does that work?
> Cheers,
> Thomas


The '\r\r\n' string is a line separator, though not used consistently in US
meteorological bulletins. I do not want to eliminate "real" empty lines.
I was hoping there is a way to prevent read() from making hidden changes to
the file content.

George
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Splitting text into lines

[George Trojan - NOAA Federal]

>
> Tell Python to keep the newline chars as seen with
> open(filename, newline="")
> For example:
> >>>
> * open("odd-newlines.txt", "rb").read() *
> b'alpha\nbeta\r\r\ngamma\r\r\ndelta\n'
> >>>
> * open("odd-newlines.txt", "r", newline="").read().replace("\r", *
> "").splitlines()
> ['alpha', 'beta', 'gamma', 'delta']


Thanks Peter. That's what I needed.

George
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Splitting text into lines

[Random832]

On Tue, Dec 13, 2016, at 12:25, George Trojan - NOAA Federal wrote:
> >
> > Are repeated newlines/carriage returns significant at all? What about
> > just using re and just replacing any repeated instances of '\r' or '\n'
> > with '\n'? I.e. something like
> >  >>> # the_string is your file all read in
> >  >>> import re
> >  >>> re.sub("[\r\n]+", "\n", the_string)
> > and then continuing as before (i.e. splitting by newlines, etc.)
> > Does that work?
> > Cheers,
> > Thomas
> 
> 
> The '\r\r\n' string is a line separator, though not used consistently in
> US
> meteorological bulletins. I do not want to eliminate "real" empty lines.

I'd do re.sub("\r*\n", "\n", the_string). Any "real" empty lines are
almost certainly going to have two \n characters, regardless of any \r
characters. It looks like what *happens* is that the file, or some part
of the file, had \r\n line endings originally and was "converted" to
turn the \n into \r\n.

> I was hoping there is a way to prevent read() from making hidden changes
> to the file content.

Pass newline='' into open.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Running python from pty without prompt

[Random832]

On Tue, Dec 13, 2016, at 11:01, Michael Torrie wrote:
> On 12/13/2016 05:39 AM, Samuel Williams wrote:
> > Michael, yes.
> > 
> > FYI, I found out why this works. Pressing Ctrl-D flushes the input
> > buffer. If you do this on an empty line, it causes read(...) to return
> > 0 which Ruby considers end of input for the script, but the pipe is
> > not closed.
>
> Currently Python does not appear to support this behavior.  Possibly it
> could be patched to support something similar, though.

The problem is there's currently no way to differentiate "interactive
mode" from "script run on a tty".

You can get similar behavior with python -c "import
sys;exec(sys.stdin.read())"
-- 
https://mail.python.org/mailman/listinfo/python-list

Name mangling vs qualified access to class attributes

[paolieri]

The official Python tutorial at

https://docs.python.org/3/tutorial/classes.html#private-variables

says that "name mangling is helpful for letting subclasses override methods 
without breaking intraclass method calls" and makes an interesting example:

class Mapping:
def __init__(self, iterable):
self.items_list = []
self.__update(iterable)

def update(self, iterable):
for item in iterable:
self.items_list.append(item)

__update = update   # private copy of original update() method

class MappingSubclass(Mapping):

def update(self, keys, values):
# provides new signature for update()
# but does not break __init__()
for item in zip(keys, values):
self.items_list.append(item)


It seems to me that, in this example, one could just have:

class Mapping:
def __init__(self, iterable):
self.items_list = []
Mapping.update(self, iterable)

def update(self, iterable):
for item in iterable:
self.items_list.append(item)

and avoid copying 'Mapping.update' into 'Mapping.__update'. More generally, any 
time one needs to "let subclasses override methods without breaking intraclass 
method calls" (the goal stated in the tutorial), using qualified access to 
class attributes/methods should suffice.

Am I missing something? Is 'self.__update(iterable)' in 'Mapping.__init__' 
preferable to 'Mapping.update(self, iterable)'?

I think that, instead, name mangling is helpful to avoid accidental overrides 
of methods/attributes by the *current* class (rather than its subclasses). 
Given the way that C3 linearization works, you can't know in advance who will 
follow your class A in B.__mro__ when B extends A. Name mangling allows you to 
avoid overriding methods/attributes of classes that might follow.

Any thoughts?

Best,
-- Marco
-- 
https://mail.python.org/mailman/listinfo/python-list

Install Problem

[Rhesa Browning]

I have been trying to install Python 3.5.2 onto my computer.  I have installed 
and uninstalled and resinstalled several times.  Every time I get an error 
message saying that the python35.dll doesn't exist on my computer so it can't 
open Python.  How can this problem be fixed?



rbrowni...@mmm.com

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Install Problem

[John Gordon]

In  Rhesa Browning 
 writes:

>I have been trying to install Python 3.5.2 onto my computer.  I have
>installed and uninstalled and resinstalled several times.  Every time I
>get an error message saying that the python35.dll doesn't exist on my
>computer so it can't open Python.  How can this problem be fixed?

When does the error occur?  During installation?  Or after installation
when you're trying to run a python program?

What version of Windows do you have?

-- 
John Gordon   A is for Amy, who fell down the stairs
gor...@panix.com  B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

-- 
https://mail.python.org/mailman/listinfo/python-list

RE: Re: Install Problem

[Rhesa Browning]

It occurs when I am trying to open up a python program.  I am running windows 7.

-Original Message-
From: Python-list [mailto:python-list-bounces+rbrowning1=mmm@python.org] On 
Behalf Of John Gordon
Sent: Tuesday, December 13, 2016 2:52 PM
To: python-list@python.org
Subject: [EXTERNAL] Re: Install Problem

In  Rhesa Browning 
 writes:

>I have been trying to install Python 3.5.2 onto my computer.  I have 
>installed and uninstalled and resinstalled several times.  Every time I 
>get an error message saying that the python35.dll doesn't exist on my 
>computer so it can't open Python.  How can this problem be fixed?

When does the error occur?  During installation?  Or after installation when 
you're trying to run a python program?

What version of Windows do you have?

-- 
John Gordon   A is for Amy, who fell down the stairs
gor...@panix.com  B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

--
https://mail.python.org/mailman/listinfo/python-list
3M security scanners have not detected any malicious content in this message.

To report this email as SPAM, please forward it to s...@websense.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Install Problem

[eryk sun]

On Tue, Dec 13, 2016 at 8:37 PM, Rhesa Browning  wrote:
> I have been trying to install Python 3.5.2 onto my computer.  I have 
> installed and
> uninstalled and resinstalled several times.  Every time I get an error message
> saying that the python35.dll doesn't exist on my computer so it can't open 
> Python.
> How can this problem be fixed?

Please open an issue at bugs.python.org. Zip the installation logs
("Python 3.5.2*.log") from your %temp% folder, and upload them to the
issue.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Running python from pty without prompt

[Michael Torrie]

On 12/13/2016 10:48 AM, Random832 wrote:
> The problem is there's currently no way to differentiate "interactive
> mode" from "script run on a tty".
> 
> You can get similar behavior with python -c "import
> sys;exec(sys.stdin.read())"

Are you sure? I can pipe scripts into Python and they run fine and
Python is not in interactive mode.

python < script.py

The behavior the OP is looking for of course is a way of demarcating the
end of the script and the beginning of data to feed the script.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Name mangling vs qualified access to class attributes

[Cameron Simpson]

On 13Dec2016 12:27, paoli...@gmail.com  wrote:

The official Python tutorial at
https://docs.python.org/3/tutorial/classes.html#private-variables

says that "name mangling is helpful for letting subclasses override methods without 
breaking intraclass method calls" and makes an interesting example:

class Mapping:
   def __init__(self, iterable):
   self.items_list = []
   self.__update(iterable)

   def update(self, iterable):
   for item in iterable:
   self.items_list.append(item)

   __update = update   # private copy of original update() method

class MappingSubclass(Mapping):

   def update(self, keys, values):
   # provides new signature for update()
   # but does not break __init__()
   for item in zip(keys, values):
   self.items_list.append(item)

It seems to me that, in this example, one could just have:

class Mapping:
   def __init__(self, iterable):
   self.items_list = []
   Mapping.update(self, iterable)

   def update(self, iterable):
   for item in iterable:
   self.items_list.append(item)

and avoid copying 'Mapping.update' into 'Mapping.__update'. More generally, any time one 
needs to "let subclasses override methods without breaking intraclass method 
calls" (the goal stated in the tutorial), using qualified access to class 
attributes/methods should suffice.

Am I missing something? Is 'self.__update(iterable)' in 'Mapping.__init__' 
preferable to 'Mapping.update(self, iterable)'?


IMO, mostly in that "Mapping.update" hardwires the class name, whereas 
"self.__update" will survive a class rename.


I confess I've never used name mangling in the manner shown in the example.

Hoping for more insightful comments...

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Running python from pty without prompt

[Random832]

On Tue, Dec 13, 2016, at 17:09, Michael Torrie wrote:
> On 12/13/2016 10:48 AM, Random832 wrote:
> > The problem is there's currently no way to differentiate "interactive
> > mode" from "script run on a tty".
> > 
> > You can get similar behavior with python -c "import
> > sys;exec(sys.stdin.read())"
> 
> Are you sure? I can pipe scripts into Python and they run fine and
> Python is not in interactive mode.

Yes, a pipe and a tty are two different things.

> python < script.py
> 
> The behavior the OP is looking for of course is a way of demarcating the
> end of the script and the beginning of data to feed the script.

It's more than just that - with a tty you can call sys.stdin.read()
multiple times, and each time end it with ctrl-d.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Splitting text into lines

[Steve D'Aprano]

On Wed, 14 Dec 2016 03:45 am, George Trojan - NOAA Federal wrote:

> I have files containing ASCII text with line s separated by '\r\r\n'.
> Example:
> 
> $ od -c FTAK31_PANC_131140.1481629265635
> 000   F   T   A   K   3   1   P   A   N   C   1   3   1   1
> 020   4   0  \r  \r  \n   T   A   F   A   B   E  \r  \r  \n   T   A
[...] 
> 300  \r  \r  \n
> 303
> 
> What is the proper way of getting a list of lines?

Do you have any objection to post-processing the list of lines?

lines = open(filename).readlines()
# remove leading and trailing whitespace, including newline characters
lines = [line.strip() for line in lines]
# get rid of empty strings
lines = [line for line in lines if line]

If you prefer, that last line of code can be written as either:

lines = filter(None, lines)  # Python 2
lines = list(filter(None, lines))  # Python 3


Personally, this would be my preferred technique, as I (nearly) always end
up doing a strip() on data I read from text files, so the extra call to
filter is no big deal.




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Running python from pty without prompt

[Steve D'Aprano]

On Wed, 14 Dec 2016 09:24 am, Random832 wrote:

> On Tue, Dec 13, 2016, at 17:09, Michael Torrie wrote:
>> On 12/13/2016 10:48 AM, Random832 wrote:
>> > The problem is there's currently no way to differentiate "interactive
>> > mode" from "script run on a tty".
>> > 
>> > You can get similar behavior with python -c "import
>> > sys;exec(sys.stdin.read())"
>> 
>> Are you sure? I can pipe scripts into Python and they run fine and
>> Python is not in interactive mode.
> 
> Yes, a pipe and a tty are two different things.


Can you show a simple demonstration of what you are doing?

I'm having difficulty following this thread because I don't know
what "script run on a tty" means.

I thought that with the exception of scripts run from cron, any time you run
a script *or* in interactive mode, there is an associated tty. Am I wrong?



>> python < script.py
>> 
>> The behavior the OP is looking for of course is a way of demarcating the
>> end of the script and the beginning of data to feed the script.
> 
> It's more than just that - with a tty you can call sys.stdin.read()
> multiple times, and each time end it with ctrl-d.





-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Running python from pty without prompt

[Steve D'Aprano]

On Wed, 14 Dec 2016 04:48 am, Random832 wrote:

> On Tue, Dec 13, 2016, at 11:01, Michael Torrie wrote:
>> On 12/13/2016 05:39 AM, Samuel Williams wrote:
>> > Michael, yes.
>> > 
>> > FYI, I found out why this works. Pressing Ctrl-D flushes the input
>> > buffer. If you do this on an empty line, it causes read(...) to return
>> > 0 which Ruby considers end of input for the script, but the pipe is
>> > not closed.
>>
>> Currently Python does not appear to support this behavior.  Possibly it
>> could be patched to support something similar, though.
> 
> The problem is there's currently no way to differentiate "interactive
> mode" from "script run on a tty".


sys.flags.interactive will tell you whether or not your script was launched
with the -i flag.

hasattr(sys, 'ps1') or hasattr(sys, 'ps2') will tell you if you are running
in the REPL (interactive interpreter). The ps1 and ps2 variables aren't
defined in non-interactive mode.

Does that help?


> You can get similar behavior with python -c "import
> sys;exec(sys.stdin.read())"

[steve@ando ~]$ python -c "import sys; print hasattr(sys, 'ps1')"
False

[steve@ando ~]$ python -c "import sys; exec(sys.stdin.read())"
import sys
print hasattr(sys, 'ps1')
False


It's not obvious, but after I entered the line "print hasattr(...)" I typed
Ctrl-D, ending the stream.




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Running python from pty without prompt

[Michael Torrie]

On 12/13/2016 05:10 PM, Steve D'Aprano wrote:
> Can you show a simple demonstration of what you are doing?

I think they want to run Python, perhaps remotely via ssh, and feed it
both a script and input over standard-in (though a tty comes into this
somehow and I'm not clear on that).  Apparently in Ruby you can pass a
script to it via standard-in, then a ctrl-d, and standard-in is kept
open so they can then feed the ruby script input.  If Python supported
this, an example would look something like this:

$ python << EOF
a = input("Give me something: ")
print (a)
<^D>
test_input
EOF

Where ^D is a literal control-d character the marks the end of the
script and the beginning of input that will go to the script.

The tty part might come into play when they are using ssh to remotely
run the python process.  Standard in, though, is the primary mechanism
they want to use if I understand the OP correctly.

I think a wrapper that feeds exec() would do what he desires.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Running python from pty without prompt

[Ben Finney]

Steve D'Aprano  writes:

> I thought that with the exception of scripts run from cron, any time
> you run a script *or* in interactive mode, there is an associated tty.
> Am I wrong?

Any daemon will, by definition, have no controlling terminal.

Other processes can choose to detach themselves from their controlling
terminal.

Either of those could invoke Python, and then the Python program would
be running without any controlling terminal.

-- 
 \   “I have always wished for my computer to be as easy to use as |
  `\   my telephone; my wish has come true because I can no longer |
_o__)  figure out how to use my telephone.” —Bjarne Stroustrup |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

OT - "Soft" ESC key on the new MacBook Pro

[Skip Montanaro]

I know this isn't a Python-specific question, but i got zero useful
responses from the help-gnu-emacs list for some reason. I think this
expression should not evaluate to the empty set:

set(python_programmers) & set(emacs_users) & set(new_macbookpro_owners)

Hopefully there are a few people out there who've had an opportunity to try
the new ESC-less Pros with the Touch Bar for an extended period of time.
Does the lack of a physical ESC key create problems for people, especially
Emacs users?

Just to head off a couple suggestions people might be inclined to make...
Yes, I know I can use C-[ or the Alt key instead of ESC. I can remap other
keys like Caps Lock or the back tick. I can also buy some other laptop with
a true ESC key, or buy a 13-inch MBP sans Touch Bar. I do plan to try out
Emacs on a Touch-Bar-equipped MacBook Pro at the Apple Store, but a few
minutes horsing around in the din of holiday shopping isn't the same as an
extended test drive in my usual environment.

So, for those of you who've tried it, does the lack of a physical ESC key
create problems?

Thx,

Skip
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: OT - "Soft" ESC key on the new MacBook Pro

[Michael Torrie]

On 12/13/2016 06:06 PM, Skip Montanaro wrote:
> So, for those of you who've tried it, does the lack of a physical ESC key
> create problems?

If there were problems with it I imagine ViM users would probably be
more inconvenienced than Emacs users.  I haven't heard anything about
people's real-world experience. I know of no one with one of these
expensive little machines yet.  The stores around here don't even have
them yet.  I suspect you haven't gotten much feedback because very few
users (Emacs users anyway) have gotten a hold of these new machines yet.

If I was to choose between a 2015 MBP and the new touch bar MBP, no
question I'd just get the 2015 while I still can, and that has nothing
to do with the touch bar.  Way better bang for the computing buck. The
new machines are overpriced now.

But I am getting cynical in my old age.  My love affair with Apple that
started with OS X 10.0 has steadily waned over the years.  Lately I've
actually been impressed with the nicer Windows 10 laptops and tablets.
I'm quite shocked actually (at that fact).


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: OT - "Soft" ESC key on the new MacBook Pro

[Paul Rubin]

Skip Montanaro  writes:
> Does the lack of a physical ESC key create problems for people, especially
> Emacs users?

Not a Mac user and I rarely use ESC instead of ALT while editing with
Emacs on a local computer, but when editing remotely I do have to use
ESC because the Gnome terminal emulator steals a few ALTed keys.  Maybe
there is a way to stop that behaviour but it didn't occur to me til just
now.  Hmm.

Meanwhile the concept of a computer with "no escape" just shows Apple
getting deeper into existentialism.  First it was the hipster Mac users
with the Beatnik black berets and turtlenecks, and now this. 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: OT - "Soft" ESC key on the new MacBook Pro

[Gregory Ewing]

Paul Rubin wrote:

First it was the hipster Mac users
with the Beatnik black berets and turtlenecks, and now this. 


Once you're in the clutches of Apple, there is no Escape.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Is there a way to insert hooks into a native dictionary type to see when a query arrives and what's looked up?

[Veek M]

I know that with user classes one can define getattr, setattr to handle 
dictionary lookup. Is there a way to hook into the native dict() type 
and see in real time what's being queried.

I wanted to check if when one does:

x.sin()

if the x.__dict__ was queried or if the Foo.__dict__ was queried.. I 
know method/attribute lookup starts with the instance but was wondering 
if I could see it in action vs defining __getattr__ __setattr__ in Foo 
which is a bit indirect.. and not what I want.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: asyncio question

[Frank Millman]

"Ian Kelly"  wrote in message 
news:CALwzid=vdczAH18mHKaL7ryvDUB=7_y-JVUrTkRZ=gkz66p...@mail.gmail.com...


On Tue, Dec 13, 2016 at 6:15 AM, Frank Millman  wrote:
> The client uses AJAX to send messages to the server. It sends the 
> message
> and continues processing, while a background task waits for the response 
> and
> handles it appropriately. As a result, the client can send a second 
> message
> before receiving a response to the first one. The server can detect 
> this,

> but it cannot wait for the first message to complete, otherwise it will
> block other clients. I have not noticed any problems with processing 2
> requests from the same client concurrently, but I don't like it, so I 
> want

> to process them sequentially.

Is there a particular reason why you're worried about this? The
browser is perfectly capable of keeping the requests straight. Also,
note that since the requests use separate HTTP connections, even if
the server sends its responses in a particular order, there's no
guarantee that the client will read them in that order, so this
doesn't free you from the need to allow the client to handle the
requests coming back in any order.



I had not thought of that, thanks. In fact, more to the point in my case, I 
assume that there is no guarantee that the server will receive the requests 
in the same order that the client sends them.


The particular reason for my concern was that each request can change the 
state of the session, and I wanted to be sure that the state has been fully 
updated before processing the next request.


One scenario, that I had not thought of, is that the requests could be 
received out of sequence. I don't think this will be a problem, but I will 
have to think about it.


The second scenario, which was my main concern, is that the server starts 
processing the second request before processing of the first request has 
been completed, meaning that the session data may not be in a stable state. 
My proposed solution solves this problem.




In a 64-bit Linux build of CPython, the combined size of a generator
and a stack frame is around half a kilobyte (not including whatever
space is needed for local variables), so hundreds of yielding asyncio
tasks should consume no more than hundreds of kilobytes of memory.
Remember that half a kilobyte figure is per generator, not per task,
so if your while loop is four coroutines deep, that will inflate the
cost of the task to two kilobytes each. This is different from the
threading model where each thread would need its own separate stack
space, not just a frame on the heap; you probably wouldn't want to do
this with threads, but with coroutines it should be fine.



Good to know, thanks. I will proceed on the assumption that if anyone runs 
my system with hundreds of users, they will run it on some serious hardware, 
so I should be safe.


Frank


--
https://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to insert hooks into a native dictionary type to see when a query arrives and what's looked up?

[Chris Angelico]

On Wed, Dec 14, 2016 at 5:11 PM, Veek M  wrote:
> I know that with user classes one can define getattr, setattr to handle
> dictionary lookup. Is there a way to hook into the native dict() type
> and see in real time what's being queried.
>
> I wanted to check if when one does:
>
> x.sin()
>
> if the x.__dict__ was queried or if the Foo.__dict__ was queried.. I
> know method/attribute lookup starts with the instance but was wondering
> if I could see it in action vs defining __getattr__ __setattr__ in Foo
> which is a bit indirect.. and not what I want.

Normally, I would say "subclass dict and override __getitem__", but
interestingly, that doesn't seem to work. You can put a subclass of
dict in an object's __dict__, but Python won't call your functions -
at least, CPython 3.7 won't. Presumably the actual lookups and updates
are done by directly tinkering with the dict's internals, from C.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: IndexError: list index out of range

[Elnaz]

On Tuesday, December 13, 2016 at 12:45:49 PM UTC+3:30, Peter Otten wrote:
> Elnaz wrote:
> 
> > hi
> > i am begginer in python. I have written a code and given this error:
> > IndexError: list index out of range
> > 
> > In my program, I have h=32 bits input. i divide this 32 bits to 4*8 block
> > and every 8-block is n. so n=0:7;(h=int(n/4)) I want to rotate 0 to 7 bits
> > for 2 bits: 0,1,2,3,4,5,6,7--->2,3,4,5,6,7 Iwrite this code:
> > def rottwo(self, X, n, r):
> > assert r >= 1
> > temp = [None]*n
> > for i in range(n-r) :
> > temp[i] = X[i+r]
> > for i in range(n-r,n) :
> > temp[i] = X[i-n+r]
> > return temp
> > this function work correctly. but I also want to rotate 24 to 31 bits for
> > 5 bits: 24,25,26,27,28,29,30,31-->29,30,31,24,25,26,27,28
> > 
> > when I write this code:
> > def rotfive(self, X, n, r):
> >assert r >= 1
> >temp = [None]*n
> >for i in range(n-r) :
> > temp[i+24] = X[i+3*n+r]
> >for i in range(n-r,n) :
> > temp[i+24] = X[i+2*n+r]
> >return temp
> > beacase temp is of size n I cannot access index 3*n+i. index on the list
> > temp should be less than equal to n-1 . I son't know how I must correct
> > this Is there any one to help me?
> > thanks in advanse.
> 
> I think you are making this harder than necessary. Python slices make 
> accessing parts of a list quite elegant:
> 
> >>> items
> [0, 10, 20, 30, 40, 50, 60, 70, 80, 90]
> >>> items[2:5]
> [20, 30, 40]
> >>> items[3:]
> [30, 40, 50, 60, 70, 80, 90]
> 
> You can use this to implement a function that creates a rotated list with an 
> arbitrary offset:
> 
> >>> def rot(items, offset):
> ... return items[offset:] + items[:offset]
> ... 
> >>> rot(items, 2)
> [20, 30, 40, 50, 60, 70, 80, 90, 0, 10]
> >>> rot(items, 7)
> [70, 80, 90, 0, 10, 20, 30, 40, 50, 60]
> >>> rot(items, -2)
> [80, 90, 0, 10, 20, 30, 40, 50, 60, 70]
> 
> To rotate part of a list extract that part using slice notation, rotate it 
> and write it back:
> 
> >>> def rot_part(items, offset, start, stop):
> ... items = list(items)
> ... items[start:stop] = rot(items[start:stop], offset)
> ... return items
> ... 
> >>> rot_part(range(32), 5, 24, 32)
> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 
> 21, 22, 23, 29, 30, 31, 24, 25, 26, 27, 28]
> 
> If you pass a list as the first argument
> 
> items = list(items) 
> 
> makes of copy of the list, but it will also convert an arbitrary iterable to 
> a list. That's why I can pass the range object.



i really appreciate your help. it works.
-- 
https://mail.python.org/mailman/listinfo/python-list