date:20190313

Re: Convert Windows paths to Linux style paths

2019-03-13 Thread eryk sun

On 3/12/19, Paul Moore  wrote:
>
> Do you care about case sensitivity (for example, is it important to you
> whether filenames "foo" and "FOO" map to the same file or not on
> Linux, given that they do on Windows)?

That's no longer a given in Windows, since NTFS in Windows 10 supports
case-sensitive directories that override the Windows API. For some
reason they didn't expose this as a file attribute set on the
directory, so it can only be queried using either an NT system call or
the fsutil.exe command-line program.

Our Windows filesystem code (e.g. in pathlib) that assumes
case-insensitive filename matching (e.g. via the casefold, lower, or
upper string methods) is wrong for case-sensitive directories. For
example:

>>> p = pathlib.Path('C:/CaseSensitive')
>>> os.system(f'fsutil file queryCaseSensitiveInfo "{p}"')
Case sensitive attribute on directory C:\CaseSensitive is enabled.
0
>>> (p / 'foo').touch()
>>> (p / 'FOO').touch()
>>> os.listdir(p)
['FOO', 'foo']

Due to pathlib's use of case folding in Windows, globbing "foo" and
"FOO" both match "foo":

>>> list(p.glob('foo'))
[WindowsPath('C:/CaseSensitive/foo')]
>>> list(p.glob('FOO'))
[WindowsPath('C:/CaseSensitive/foo')]

If we remove "foo", glob('FOO') no longer matches anything since it
checks for the existence of "foo" instead of "FOO":

>>> os.remove(p / 'foo')
>>> os.listdir(p)
['FOO']
>>> list(p.glob('FOO'))
[]
-- 
https://mail.python.org/mailman/listinfo/python-list

Not a python question, just programming logic trap?

2019-03-13 Thread jonas . thornvall

Anyone who is good at see logic traps in programming? comp.lang.javascript is 
defunct so i try here.

Using the metronome. 

https://midisequenser.000webhostapp.com 

I've tried to understand how following code can lead to that after the 
keyreleased the front of the keyed scrollbar raise/extend faster while the 
backend stay correct "in sync". One could think/imagine 
xFrontStart=scrollBarPos-FRONTOFFSET; somehow executed  more then once after 
release, but how could it, the keyrelease is true only in that part of code and 
only done once??? 

var FRONTOFFSET=0; 
var xKeylength=0; 
var xFrontStart=0; 
var xBackEnd=0; 
var keyreleased=false; 
function pianoSCROLL(){ 

 if(mess.data[0]==NOTE_ON && keyreleased==false){ 
xBackEnd=keylength; 
ctm.fillStyle = "magenta"; 

ctm.fillRect(xBackEnd,(PistartX+keyHeight*88)-(keyHeight*(mess.data[1]-8)), 
xFrontStart, keyHeight); 
if(FRONTOFFSET==0) {FRONTOFFSET=scrollBarPos;} 
xFrontStart=scrollBarPos-FRONTOFFSET; 
 } else if(mess.data[0]==NOTE_OFF && keyreleased==false){ 
   console.log("keyreleased!"); 
   keyreleased=true; 
   xBarLength=Math.round(xFrontStart-xBackEnd); 
   ctm.fillStyle = "black"; 
   ctm.fillRect(keylength, 
(PistartX+keyHeight*88)-(keyHeight*(mess.data[1]-8)), Mwidth, keyHeight+2); 
} 
if (keyreleased) { 
//console.log("timebar backend moving"); 
ctm.fillStyle = "black"; 
ctm.fillRect(keylength, 
(PistartX+keyHeight*88)-(keyHeight*(mess.data[1]-8)), Mwidth, keyHeight+2); 
ctm.fillStyle = "orange"; 

ctm.fillRect(xBackEnd,(PistartX+keyHeight*88)-(keyHeight*(mess.data[1]-8)),xFrontStart,
 keyHeight); 
xFrontStart=scrollBarPos-FRONTOFFSET; 
xBackEnd=Math.round(xFrontStart-xBarLength); 
console.log("xBackEnd="+xBackEnd+" xBarLength="+xBarLength+" 
xFrontStart="+xFrontStart); 
} 
} 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Convert Windows paths to Linux style paths

2019-03-13 Thread Paul Moore

On Wed, 13 Mar 2019 at 08:13, eryk sun  wrote:
>
> On 3/12/19, Paul Moore  wrote:
> >
> > Do you care about case sensitivity (for example, is it important to you
> > whether filenames "foo" and "FOO" map to the same file or not on
> > Linux, given that they do on Windows)?
>
> That's no longer a given in Windows, since NTFS in Windows 10 supports
> case-sensitive directories that override the Windows API.

I know, but I thought my answer to the OP included enough complexities
for them to think about without going into this level of detail ;-)

But yes, "converting filenames" is a hard problem, unless you really
do just want a simple text based transformation.

Paul
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 3D visualizations in Python

2019-03-13 Thread marco . nawijn

On Tuesday, March 12, 2019 at 11:54:22 PM UTC+1, Jakub Bista wrote:
> Hello. I want to do 3D visualization in Python. Which framework do you 
> recommend me for creating such a Interface?

I would recommend the VTK library (https://vtk.org/). It has excellent Python 
bindings. 

Marco
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Multiprocessing vs subprocess

2019-03-13 Thread oliver

With multiprocessing you can take advantage of multi-core processing as it
launches a separate python interpreter process and communicates with it via
shared memory (at least on windows). The big advantage of multiprocessing
module is that the interaction between processes is much richer than
subprocess: subprocess is limited to pipes (unless you build your own
alternate comms mechanism), whereas multiprocessing takes care of
marshalling python objects and transporting them to the other processes.
This can be huge in some applications, like we were using it to run batch
monte carlo simulations where we needed to pass simulation related
datastructures, would have been a pain with subprocess.

On Tue, Mar 12, 2019 at 12:30 PM Schachner, Joseph <
joseph.schach...@teledyne.com> wrote:

> Re: " My understanding (so far) is that the tradeoff of using
> multiprocessing is that my manager script can not exit until all the work
> processes it starts finish. If one of the worker scripts locks up, this
> could be problematic. Is there a way to use multiprocessing where processes
> are launched independent of the parent (manager) process?"
>
> I just want to point out that subprocess (which uses the operating system
> to start another processes, to run whatever you say - doesn't have to be
> Python running another script, can be an .exe)  does not have that
> restriction.  The subprocess call can be blocking (the caller waits), or
> not.   If not the caller can do whatever it wants including decide not to
> wait forever, and it can even exit.
>
> Of course, if one of the processes you launched is locked up that is a
> problem all by itself, even thought the manager can exit.
>
> --- Joseph S.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: "Post install setup does not work as expected with pip install"

2019-03-13 Thread Saba Kauser

I am able to get this to work.
I had to invoke parent's run before my post install logic could kick.

_
From: Saba Kauser
Sent: Tuesday, March 12, 2019 6:01 PM
To: python-list@python.org
Subject: "Post install setup does not work as expected with pip install"

Hello,

I have a post install class that looks like this:
if('darwin' in sys.platform):
class PostInstall(install):
""" Post installation - run install_name_tool on Darwin """
def run(self):
clipath = os.getenv('IBM_DB_HOME', '@loader_path/clidriver')
print("in PostInstall with {}".format(clipath))
for so in glob.glob(r'build/lib*/ibm_db*.so'):
os.system("install_name_tool -change libdb2.dylib 
{}/lib/libdb2.dylib {}".format(clipath, so))
install.run(self)
cmd_class = dict(install = PostInstall)

And I pass cmd_class to setup(..) as:
setup(..
  include_package_data = True,
   cmdclass = cmd_class,
   **extra
 )

When I execute setup.py as "python setup.py install", then the PostInstall 
operation is executed after the ibm_db.so is built and installed and I see the 
intended result.
Howeever, when I run "pip install ibm_db" or "pip install .",
the execution order looks like this:
  warnings.warn(notifyString)
running install
in PostInstall with /Applications/dsdriver/  ==> this is my post install 
script
running build
running build_py
creating build
creating build/lib.macosx-10.9-x86_64-3.7==> I need to traverse to this 
folder to find my shared library

I would expect it to be run post the ibm_db is installed, not before it gets 
built.

Can you please let me know how can this be fixed. Is this a bug with pip?

Saba Kauser
Db2 Connect development and support.

Rocket Software Development India Pvt Ltd
Karle Town Centre - SEZ,
HUB 1 building, 4th Floor (North West Wing),
100 ft. Kempapura road, adjacent to Nagawara lake,
Nagawara, Bangalore - 560 045
E: skau...@rocketsoftware.com
-

Rocket Software, Inc. and subsidiaries ? 77 Fourth Avenue, Waltham MA 02451 ? 
Main Office Toll Free Number: +1 855.577.4323
Contact Customer Support: 
https://my.rocketsoftware.com/RocketCommunity/RCEmailSupport
Unsubscribe from Marketing Messages/Manage Your Subscription Preferences - 
http://www.rocketsoftware.com/manage-your-email-preferences
Privacy Policy - http://www.rocketsoftware.com/company/legal/privacy-policy

This communication and any attachments may contain confidential information of 
Rocket Software, Inc. All unauthorized use, disclosure or distribution is 
prohibited. If you are not the intended recipient, please notify Rocket 
Software immediately and destroy all copies of this communication. Thank you.
-- 
https://mail.python.org/mailman/listinfo/python-list

[ANN] PyYAML-5.1: YAML parser and emitter for Python

2019-03-13 Thread Ingy dot Net

===
 Announcing PyYAML-5.1
===

A new MAJOR RELEASE of PyYAML is now available:
https://pypi.org/project/PyYAML/

This is the first major release of PyYAML under the new maintenance team.

Among the many changes listed below, this release specifically addresses the
arbitrary code execution issue raised by:

https://nvd.nist.gov/vuln/detail/CVE-2017-18342


(See https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation
for complete details).

The PyYAML project is now maintained by the YAML and Python communities.
Planning happens on the #yaml-dev, #pyyaml and #libyaml IRC channels on
irc.freenode.net.


Changes
===

* https://github.com/yaml/pyyaml/pull/35  -- Some modernization of the test
running
* https://github.com/yaml/pyyaml/pull/42  -- Install tox in a virtualenv
* https://github.com/yaml/pyyaml/pull/45  -- Allow colon in a plain scalar
in a flow context
* https://github.com/yaml/pyyaml/pull/48  -- Fix typos
* https://github.com/yaml/pyyaml/pull/55  -- Improve RepresenterError
creation
* https://github.com/yaml/pyyaml/pull/59  -- Resolves #57, update readme
issues link
* https://github.com/yaml/pyyaml/pull/60  -- Document and test Python 3.6
support
* https://github.com/yaml/pyyaml/pull/61  -- Use Travis CI built in pip
cache support
* https://github.com/yaml/pyyaml/pull/62  -- Remove tox workaround for
Travis CI
* https://github.com/yaml/pyyaml/pull/63  -- Adding support to Unicode
characters over codepoint 0x
* https://github.com/yaml/pyyaml/pull/65  -- Support unicode literals over
codepoint 0x
* https://github.com/yaml/pyyaml/pull/75  -- add 3.12 changelog
* https://github.com/yaml/pyyaml/pull/76  -- Fallback to Pure Python if
Compilation fails
* https://github.com/yaml/pyyaml/pull/84  -- Drop unsupported Python 3.3
* https://github.com/yaml/pyyaml/pull/102 -- Include license file in the
generated wheel package
* https://github.com/yaml/pyyaml/pull/105 -- Removed Python 2.6 & 3.3
support
* https://github.com/yaml/pyyaml/pull/111 -- Remove commented out Psyco code
* https://github.com/yaml/pyyaml/pull/129 -- Remove call to `ord` in lib3
emitter code
* https://github.com/yaml/pyyaml/pull/143 -- Allow to turn off sorting keys
in Dumper
* https://github.com/yaml/pyyaml/pull/149 -- Test on Python 3.7-dev
* https://github.com/yaml/pyyaml/pull/158 -- Support escaped slash in
double quotes "\/"
* https://github.com/yaml/pyyaml/pull/181 -- Import Hashable from
collections.abc
* https://github.com/yaml/pyyaml/pull/256 -- Make default_flow_style=False
* https://github.com/yaml/pyyaml/pull/257 -- Deprecate yaml.load and add
FullLoader and UnsafeLoader classes
* https://github.com/yaml/pyyaml/pull/263 -- Windows Appveyor build


Resources
=

PyYAML IRC Channel: #pyyaml on irc.freenode.net
PyYAML homepage: https://github.com/yaml/pyyaml
PyYAML documentation: http://pyyaml.org/wiki/PyYAMLDocumentation
Source and binary installers: https://pypi.org/project/PyYAML/
GitHub repository: https://github.com/yaml/pyyaml/
Bug tracking: https://github.com/yaml/pyyaml/issues

YAML homepage: http://yaml.org/
YAML-core mailing list:
http://lists.sourceforge.net/lists/listinfo/yaml-core


About PyYAML


YAML is a data serialization format designed for human readability and
interaction with scripting languages. PyYAML is a YAML parser and emitter
for
Python.

PyYAML features a complete YAML 1.1 parser, Unicode support, pickle support,
capable extension API, and sensible error messages. PyYAML supports standard
YAML tags and provides Python-specific tags that allow to represent an
arbitrary Python object.

PyYAML is applicable for a broad range of tasks from complex configuration
files to object serialization and persistence.


Example
===

>>> import yaml

>>> yaml.full_load("""
... name: PyYAML
... description: YAML parser and emitter for Python
... homepage: https://github.com/yaml/pyyaml
... keywords: [YAML, serialization, configuration, persistence, pickle]
... """)
{'keywords': ['YAML', 'serialization', 'configuration', 'persistence',
'pickle'], 'homepage': 'https://github.com/yaml/pyyaml', 'description':
'YAML parser and emitter for Python', 'name': 'PyYAML'}

>>> print(yaml.dump(_))
name: PyYAML
homepage: https://github.com/yaml/pyyaml
description: YAML parser and emitter for Python
keywords: [YAML, serialization, configuration, persistence, pickle]


Maintainers
===

The following people are currently responsible for maintaining PyYAML:

* Ingy döt Net
* Tina Mueller
* Matt Davis

and many thanks to all who have contribributed!
See: https://github.com/yaml/pyyaml/pulls


Copyright
=

Copyright (c) 2017-2019 Ingy döt Net 
Copyright (c) 2006-2016 Kirill Simonov 

The PyYAML module was written by Kirill Simonov .
It is currently maintained by the YAML and Python communities.

PyYAML is released under the MIT license.
See the file LICENSE for more details.
-- 
http

Re: Not a python question, just programming logic trap?

2019-03-13 Thread Ben Finney

jonas.thornv...@gmail.com writes:

> Anyone who is good at see logic traps in programming?
> comp.lang.javascript is defunct so i try here.

Please don't; this forum should primarily be used for discussing Python.

I appreciate that you have tried another forum for JavaScript, but
that's unrelated to whether this is an approriate forum for the
question. It isn't.

You could try https://stackoverflow.com/tags/javascript> but really
it's not good to be using this Python discussion forum for that purpose.

-- 
 \ “I went to a museum where all the artwork was done by children. |
  `\   They had all the paintings up on refrigerators.” —Steven Wright |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

uWISGI with Qt for Python

2019-03-13 Thread Israel Brewster

I’m working on a Qt for python app that needs to run a local web server. For 
the web server portion I’m using flask and uWISGI, and at the moment I have my 
application launching uWISGI using subprocess before firing off the Qt 
QApplication instance and entering the Qt event loop. Some sample code to 
illustrate the process:

If __name__ ==“__main__”:
CUR_DIRECTORY = os.path.dirname(__file__)

UWSGI_CONFIG = os.path.realpath(os.path.join(CUR_DIRECTORY, 'Other 
Files/TROPOMI.ini'))
UWSGI_EXE = os.path.realpath(os.path.join(CUR_DIRECTORY, 'bin/uwsgi'))
uwsgi_proc = subprocess.Popen([UWSGI_EXE, UWSGI_CONFIG])

qt_app = QApplication(sys.argv)
….
res = qt_app.exec_()


Now this works, but it strikes me as kinda kludgy, as the uWISGI is effectively 
a separate application needed. More to the point, however, it’s a bit fragile, 
in that if the main application crashes (really, ANY sort of unclean exit), you 
get stray uWISGI processes hanging around that prevent proper functioning of 
the app the next time you try to launch it. Unfortunately as the app is still 
in early days, this happens occasionally. So I have two questions:

1) Is there a “better way”? This GitHub repo: 
https://github.com/unbit/uwsgi-qtloop seems to indicate that it should be 
possible to run a Qt event loop from within a uWSGI app, thus eliminating the 
extra “subprocess” spinoff, but it hasn’t been updated in 5 years and I have 
been unable to get it to work with my current Qt/Python/OS setup

2) Baring any “better way”, is there a way to at least ensure that the 
subprocess is killed in the event of parent death, or alternately to look for 
and kill any such lingering processes on application startup?

P.S. The purpose of running the web server is to be able to load and use Plotly 
charts in my app (via a QWebEngineView). So a “better way” may be using a 
different plotting library that can essentially “cut out” the middle man. I’ve 
tried Matplotlib, but I found its performance to be worse than Plotly - given 
the size of my data sets, performance matters. Also I had some glitches with it 
when using a lasso selector (plot going black). Still, with some work, it may 
be an option.

---
Israel Brewster
Software Engineer
Alaska Volcano Observatory 
Geophysical Institute - UAF 
2156 Koyukuk Drive 
Fairbanks AK 99775-7320
Work: 907-474-5172
cell:  907-328-9145

-- 
https://mail.python.org/mailman/listinfo/python-list

Generator question

2019-03-13 Thread Pierre Reinbold

Dear all,

I want to implement a function computing the Cartesian product if the elements
of a list of lists, but using generator expressions. I know that it is already
available in itertools but it is for the sake of understanding how things work.

I already have a working recursive version, and I'm quite sure that this
iterative version used to work (at least in some Python2.X) :

def flat_genexp_cat_prod(lists):
solutions = [[]]
for a_list in lists:
solutions = (part_sol+[el] for part_sol in solutions for el in a_list)
return solutions

But, with Python3.7.2, all I got is this :

>>> list(flat_genexp_cat_prod([[1, 2], [3, 4], [5, 6]]))
[[5, 5, 5], [5, 5, 6], [5, 6, 5], [5, 6, 6], [6, 5, 5], [6, 5, 6], [6, 6, 5],
[6, 6, 6]]

instead of

>>> list(flat_genexp_cat_prod([[1, 2], [3, 4], [5, 6]]))
[[1, 3, 5], [1, 3, 6], [1, 4, 5], [1, 4, 6], [2, 3, 5], [2, 3, 6], [2, 4, 5],
[2, 4, 6]]

Using a list comprehension instead of a generator expression solves the problem,
but I can't understand why the version above fails.

Even stranger, when debugging I tried to use itertools.tee to duplicate the
solutions generators and have a look at them :

def flat_genexp_cat_prod(lists):
solutions = [[]]
for a_list in lists:
solutions, debug = tee(
part_sol+[el] for part_sol in solutions for el in a_list)
print("DEBUG", list(debug))
return solutions

And, that version seems to work!

>>> list(flat_genexp_cat_prod([[1, 2], [3, 4], [5, 6]]))
DEBUG [[1], [2]]
DEBUG [[1, 3], [1, 4], [2, 3], [2, 4]]
DEBUG [[1, 3, 5], [1, 3, 6], [1, 4, 5], [1, 4, 6], [2, 3, 5], [2, 3, 6], [2, 4,
5], [2, 4, 6]]
[[1, 3, 5], [1, 3, 6], [1, 4, 5], [1, 4, 6], [2, 3, 5], [2, 3, 6], [2, 4, 5],
[2, 4, 6]]

Can you help me understand what I'm doing wrong ?

Thank you by advance,


πr
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: uWISGI with Qt for Python

2019-03-13 Thread Israel Brewster

Never mind this request. I realized that for what I am doing, the web server 
was unnecessary. I could just load local HTML files directly into the 
QWebEngineView with no need of an intermediate server. Thanks anyway, and sorry 
for the noise!

---
Israel Brewster
Software Engineer
Alaska Volcano Observatory 
Geophysical Institute - UAF 
2156 Koyukuk Drive 
Fairbanks AK 99775-7320
Work: 907-474-5172
cell:  907-328-9145

> On Mar 13, 2019, at 1:42 PM, Israel Brewster  wrote:
> 
> I’m working on a Qt for python app that needs to run a local web server. For 
> the web server portion I’m using flask and uWISGI, and at the moment I have 
> my application launching uWISGI using subprocess before firing off the Qt 
> QApplication instance and entering the Qt event loop. Some sample code to 
> illustrate the process:
> 
> If __name__ ==“__main__”:
> CUR_DIRECTORY = os.path.dirname(__file__)
> 
> UWSGI_CONFIG = os.path.realpath(os.path.join(CUR_DIRECTORY, 'Other 
> Files/TROPOMI.ini'))
> UWSGI_EXE = os.path.realpath(os.path.join(CUR_DIRECTORY, 'bin/uwsgi'))
> uwsgi_proc = subprocess.Popen([UWSGI_EXE, UWSGI_CONFIG])
> 
> qt_app = QApplication(sys.argv)
> ….
> res = qt_app.exec_()
> 
> 
> Now this works, but it strikes me as kinda kludgy, as the uWISGI is 
> effectively a separate application needed. More to the point, however, it’s a 
> bit fragile, in that if the main application crashes (really, ANY sort of 
> unclean exit), you get stray uWISGI processes hanging around that prevent 
> proper functioning of the app the next time you try to launch it. 
> Unfortunately as the app is still in early days, this happens occasionally. 
> So I have two questions:
> 
> 1) Is there a “better way”? This GitHub repo: 
> https://github.com/unbit/uwsgi-qtloop  
> seems to indicate that it should be possible to run a Qt event loop from 
> within a uWSGI app, thus eliminating the extra “subprocess” spinoff, but it 
> hasn’t been updated in 5 years and I have been unable to get it to work with 
> my current Qt/Python/OS setup
> 
> 2) Baring any “better way”, is there a way to at least ensure that the 
> subprocess is killed in the event of parent death, or alternately to look for 
> and kill any such lingering processes on application startup?
> 
> P.S. The purpose of running the web server is to be able to load and use 
> Plotly charts in my app (via a QWebEngineView). So a “better way” may be 
> using a different plotting library that can essentially “cut out” the middle 
> man. I’ve tried Matplotlib, but I found its performance to be worse than 
> Plotly - given the size of my data sets, performance matters. Also I had some 
> glitches with it when using a lasso selector (plot going black). Still, with 
> some work, it may be an option.
> 
> ---
> Israel Brewster
> Software Engineer
> Alaska Volcano Observatory 
> Geophysical Institute - UAF 
> 2156 Koyukuk Drive 
> Fairbanks AK 99775-7320
> Work: 907-474-5172
> cell:  907-328-9145
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Generator question

2019-03-13 Thread Ian Kelly

You're basically running into this:
https://docs.python.org/3/faq/programming.html#why-do-lambdas-defined-in-a-loop-with-different-values-all-return-the-same-result

To see why, let's try disassembling your function. I'm using Python 3.5
here, but it shouldn't make much of a difference.

py> import dis
py> dis.dis(flat_genexp_cat_prod)
  2   0 BUILD_LIST   0
  3 BUILD_LIST   1
  6 STORE_FAST   1 (solutions)

  3   9 SETUP_LOOP  39 (to 51)
 12 LOAD_FAST0 (lists)
 15 GET_ITER
>>   16 FOR_ITER31 (to 50)
 19 STORE_DEREF  0 (a_list)

  4  22 LOAD_CLOSURE 0 (a_list)
 25 BUILD_TUPLE  1
 28 LOAD_CONST   1 ( at
0x73f31571ac90, file "", line 4>)
 31 LOAD_CONST   2
('flat_genexp_cat_prod..')
 34 MAKE_CLOSURE 0
 37 LOAD_FAST1 (solutions)
 40 GET_ITER
 41 CALL_FUNCTION1 (1 positional, 0 keyword pair)
 44 STORE_FAST   1 (solutions)
 47 JUMP_ABSOLUTE   16
>>   50 POP_BLOCK

  5 >>   51 LOAD_FAST1 (solutions)
 54 RETURN_VALUE

Now, take a look at the difference between the instruction at address 22
and the one at address 37:

  4  22 LOAD_CLOSURE 0 (a_list)
 37 LOAD_FAST1 (solutions)

The value of solutions is passed directly to the generator as an argument,
which is the reason why building the generator up iteratively like this
works at all: although the nested generators are evaluated lazily, each new
generator that is constructed contains as its input a reference to the
previous generator.

By contrast, the value of a_list is a closure. The contents of the closure
are just whatever the value of a_list is when the generator gets evaluated,
not when the generator was created. Since the entire nested generated
structure is evaluated lazily, it doesn't get evaluated until list() is
called after the function has returned. The value of the a_list closure at
that point is the last value that was assigned to it: the list [5, 6] from
the last iteration of the for loop. This same list value then gets used for
all three nested generators.

So now why do solutions and a_list get treated differently like this? To
answer this, look at this paragraph about generator expressions from the
language reference:

"""
Variables used in the generator expression are evaluated lazily when the
__next__() method is called for the generator object (in the same fashion
as normal generators). However, the iterable expression in the leftmost for
clause is immediately evaluated, so that an error produced by it will be
emitted at the point where the generator expression is defined, rather than
at the point where the first value is retrieved. Subsequent for clauses and
any filter condition in the leftmost for clause cannot be evaluated in the
enclosing scope as they may depend on the values obtained from the leftmost
iterable. For example: (x*y for x in range(10) for y in range(x, x+10)).
"""

So, it's simply because the iterable expression in the leftmost for clause
is treated differently from every other value in the generator expression.

On Wed, Mar 13, 2019 at 3:49 PM Pierre Reinbold  wrote:

> Dear all,
>
> I want to implement a function computing the Cartesian product if the
> elements
> of a list of lists, but using generator expressions. I know that it is
> already
> available in itertools but it is for the sake of understanding how things
> work.
>
> I already have a working recursive version, and I'm quite sure that this
> iterative version used to work (at least in some Python2.X) :
>
> def flat_genexp_cat_prod(lists):
> solutions = [[]]
> for a_list in lists:
> solutions = (part_sol+[el] for part_sol in solutions for el in
> a_list)
> return solutions
>
> But, with Python3.7.2, all I got is this :
>
> >>> list(flat_genexp_cat_prod([[1, 2], [3, 4], [5, 6]]))
> [[5, 5, 5], [5, 5, 6], [5, 6, 5], [5, 6, 6], [6, 5, 5], [6, 5, 6], [6, 6,
> 5],
> [6, 6, 6]]
>
> instead of
>
> >>> list(flat_genexp_cat_prod([[1, 2], [3, 4], [5, 6]]))
> [[1, 3, 5], [1, 3, 6], [1, 4, 5], [1, 4, 6], [2, 3, 5], [2, 3, 6], [2, 4,
> 5],
> [2, 4, 6]]
>
> Using a list comprehension instead of a generator expression solves the
> problem,
> but I can't understand why the version above fails.
>
> Even stranger, when debugging I tried to use itertools.tee to duplicate the
> solutions generators and have a look at them :
>
> def flat_genexp_cat_prod(lists):
> solutions = [[]]
> for a_list in lists:
> solutions, debug = tee(
> part_sol+[el] for part_sol in solutions for el in a_list)
> print("DEBUG", list(debug))
> return solutions
>
> A

UG Announcement - Python Mauritius User-Group (pymug)

2019-03-13 Thread Abdur-Rahmaan Janhangeer

As per requirements, i'm announcing the existence of the Python User-Group
for Mauritius, an island in the Indian Ocean. Below are some info.

Name: Python Mauritius User-Group
Website: pymug.com
Github: github.com/pymug
Mailing list: https://mail.python.org/mailman3/lists/pymug.python.org/
Wiki mention: https://wiki.python.org/moin/LocalUserGroups#Other_Africa
under Other Africa

First local meeting held: yes
Organising members: 4
Motivation: Python promotion and helping with docs translations.

Should anybody require any info, please let me know.

Yours,

-- 
Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ
Mauritius


Garanti
sans virus. www.avast.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Convert Windows paths to Linux style paths

Not a python question, just programming logic trap?

Re: Convert Windows paths to Linux style paths

Re: 3D visualizations in Python

Re: Multiprocessing vs subprocess

RE: "Post install setup does not work as expected with pip install"

[ANN] PyYAML-5.1: YAML parser and emitter for Python

Re: Not a python question, just programming logic trap?

uWISGI with Qt for Python

Generator question

Re: uWISGI with Qt for Python

Re: Generator question

UG Announcement - Python Mauritius User-Group (pymug)

13 matches

Site Navigation

Mail list logo

Footer information