Re: [Python-Dev] [issue11191] test_search_cpp error on AIX (with xlc)

2018-08-09 Thread Michael 
On 09/08/2018 00:52, Michael Felt wrote:
> Change by Michael Felt :
>
>
> --
> pull_requests: +8196
>
> ___
> Python tracker 
> 
> ___
>
Don't know why it is says +8196. I would have expected +8709.
https://github.com/python/cpython/pull/8709

Would appreciate someone look at/review this. It corrects 6 or 7 errors
from test_distutils (when not using gcc on AIX). And it is very small.
Maybe smaller if the review is to remove the comments.

Thx.

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Subtle Bug in Class Initializations

2018-08-09 Thread Erik Bray 
On Mon, Aug 6, 2018 at 8:11 PM Eddie Elizondo  wrote:
>
> Background:
>
> Through the implementation of an alternate runtime I've been poking around 
> some of the class initialization routines and I found out that there was a 
> subtle bug with PyType_Ready and the header initializer PyVarObject_HEAD_INIT.
>
>
>
> Looking through the codebase, I couldn't really find any pattern of when the 
> type should be defined within PyVarObject_HEAD_INIT. Sometimes it was 
> initialized to NULL (or 0) and PyType_Type (let's ignore Py_True and Py_False 
> from now).
>
>
>
> From PyType_Ready it turns out that setting the value PyType_Type is never 
> actually needed outside of PyType_Type and PyBaseObject type. This is clear 
> from the code:
>
> if (Py_TYPE(type) == NULL && base != NULL)
>
> Py_TYPE(type) = Py_TYPE(base);
>
>
>
> Given that any PyTypeObject's base is of type PyType_Type, setting 
> PyVarObject_HEAD_INIT(&PyType_Ready) is superfluous. Therefore, setting all 
> static PyTypeObjects to their ob_type to NULL should be a safe assumption to 
> make.
>
>
>
> Uninitialized Types:
>
> A quick s/PyVarObject_HEAD_INIT(&PyType_Type/PyVarObject_HEAD_INIT(NULL/  
> shows that some objects do need to have their ob_type set from the outset, 
> violating the previous assumption. After writing a quick script, I found that 
> out of the ~300 PyVarObject_HEAD_INIT present in CPython, only these 14 types 
> segfaulted:
>
>
>
> PyByteArrayIter_Type
>
> PyBytesIter_Type
>
> PyDictIterKey_Type
>
> PyDictIterValue_Type
>
> PyDictIterItem_Type
>
> PyClassMethod_Type
>
> PyAsyncGen_Type
>
> PyListIter_Type
>
> PyListRevIter_Type
>
> PyODictIter_Type
>
> PyLongRangeIter_Type
>
> PySetIter_Type
>
> PyTupleIter_Type
>
> PyUnicodeIter_Type
>
>
>
>
>
> Bug:
>
> It turns out that these PyTypeObjects are never initialized through 
> PyType_Ready. However, they are used as fully initialized types. It is by 
> pure chance that the work without having to call the initializer on them. 
> This though is undefined behavior. This not only might result in a weird 
> future bug which is hard to chase down but also, it affects other runtimes as 
> this behavior depends on implementation details of CPython.
>
>
>
> This is a pervasive pattern that should be removed from the codebase and 
> ideally extensions should follow as well.
>
>
>
> Solution:
>
> Here are my proposed solutions in order from less controversial to most 
> controversial. Note that all of them I already tried in a local branch and 
> are working:
>
>
>
> 1) Initialize all uninitialized types.
>
>
>
> Example: 
> https://github.com/eduardo-elizondo/cpython/commit/bc53db3cf4e5a6923b0b1afa6181305553faf173
>
> 2) Move all PyVarObject_HEAD_INIT to NULL except PyType_Type, 
> PyBaseObject_Type and the booleans.
>
>
>
> 3) Special case the initialization of PyType_Type and PyBaseObject_Type 
> within PyType_Ready to now make all calls to PyVarObject_HEAD_INIT use NULL. 
> To enable this a small change within PyType_Ready is needed to initialize 
> PyType_Type PyBaseObject:

Coincidentally, I just wrote a long-ish blog post explaining in
technical details why PyVarObject_HEAD_INIT(&PyType_Type) pretty much
cannot work, at least for extension modules (it is not a problem in
the core library), on Windows (my post was focused on Cygwin but it is
a problem for Windows in general):
http://iguananaut.net/blog/programming/windows-data-import.html

The TL;DR is that it's not possible on Windows to initialize a struct
with a pointer to some other data that's found in another DLL (i.e.
&PyType_Type), unless it happens to be a function, as a special case.
But &PyType_Type obviously is not, so thinks break.

So I'm +1 for requiring passing NULL to PyVarObject_HEAD_INIT,
requiring PyType_Ready with an explicit base type argument, and maybe
(eventually) making PyVarObject_HEAD_INIT argumentless.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Problem in importing python packages under python 3.6 environment

2018-08-09 Thread Poornima .D. 
Hi All,


I have limited knowledge on python development.  I am trying to write a
test application which needs to import from many packages across
mutliple directories.


I tried using an environment variable and appending to sys.path variable so
that import Class methods works.


I am trying to replace the above logic by below syntax.


from directory.fileName import ClassName


But this syntax does not work.  Please let me know any solution for this
issue.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Problem in importing python packages under python 3.6 environment

2018-08-09 Thread Brett Cannon 
This mailing list is for the discussion of the development *of* Python, not
its use. Your best bet for help is either a site like Stack Overflow or
another mailing list like python-tutor or python-list.

On Thu, 9 Aug 2018 at 08:30 Poornima .D.  wrote:

> Hi All,
>
>
> I have limited knowledge on python development.  I am trying to write a
> test application which needs to import from many packages across
> mutliple directories.
>
>
> I tried using an environment variable and appending to sys.path variable
> so that import Class methods works.
>
>
> I am trying to replace the above logic by below syntax.
>
>
> from directory.fileName import ClassName
>
>
> But this syntax does not work.  Please let me know any solution for this
> issue.
>
>
>
>
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Problem in importing python packages under python 3.6 environment

2018-08-09 Thread Oleg Broytman 
Hello.

   This mailing list is to work on developing Python (adding new
features to Python itself and fixing bugs); if you're having problems
learning, understanding or using Python, please find another forum.
Probably python-list/comp.lang.python mailing list/news group is the
best place; there are Python developers who participate in it; you may
get a faster, and probably more complete, answer there. See
https://www.python.org/community/ for other lists/news groups/fora. Thank
you for understanding.

On Thu, Aug 09, 2018 at 06:15:49PM +0530, "Poornima .D." 
 wrote:
> Hi All,
> 
> 
> I have limited knowledge on python development.  I am trying to write a
> test application which needs to import from many packages across
> mutliple directories.
> 
> 
> I tried using an environment variable and appending to sys.path variable so
> that import Class methods works.
> 
> 
> I am trying to replace the above logic by below syntax.
> 
> 
> from directory.fileName import ClassName
> 
> 
> But this syntax does not work.  Please let me know any solution for this
> issue.

   Not enough information to answer. To ask such a question you'd better
prepare a simple exmaple that doesn't work -- just a few small files and
directories.

   Learn about Python modules/packagaes at
https://docs.python.org/3/tutorial/modules.html especially paying
attention to The Module Search Path:
https://docs.python.org/3/tutorial/modules.html#the-module-search-path

Oleg.
-- 
Oleg Broytmanhttps://phdru.name/[email protected]
   Programmers don't die, they just GOSUB without RETURN.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Problem in importing python packages under python 3.6 environment

2018-08-09 Thread Mariatta Wijaya 
Hi Poornima,

Your question is more appropriate for the python-list mailing list (
https://mail.python.org/mailman/listinfo/python-list) or python-tutors
mailing list (https://mail.python.org/mailman/listinfo/tutor).



On Thu, Aug 9, 2018, 8:30 AM Poornima .D.  wrote:

> Hi All,
>
>
> I have limited knowledge on python development.  I am trying to write a
> test application which needs to import from many packages across
> mutliple directories.
>
>
> I tried using an environment variable and appending to sys.path variable
> so that import Class methods works.
>
>
> I am trying to replace the above logic by below syntax.
>
>
> from directory.fileName import ClassName
>
>
> But this syntax does not work.  Please let me know any solution for this
> issue.
>
>
>
>
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/mariatta.wijaya%40gmail.com
>
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use of Cython

2018-08-09 Thread Stefan Behnel 
Hi,

this is getting a bit out of scope for this list. I propose to move further
questions about general Cython usage to he cython-users mailing list.

Matěj Cepl schrieb am 08.08.2018 um 12:44:
> On 2018-08-06, 15:13 GMT, Stefan Behnel wrote:
>> Not sure I understand this correctly, but I think we're on the 
>> same page here: writing test code in C is cumbersome, writing 
>> test code in a mix of C and Python across different files is 
>> aweful. And making it difficult to write or even just review 
>> test code simply means that people will either waste their 
>> precious contribution time on it, or try to get around it.
> 
> I was thinking about the same when porting M2Crypto to py3k 
> (M2Crypto is currently swig-based mix of C-code and Python). Is 
> it even possible to make a mix of Cython, swig-based C, and 
> Python?

As long as you take the decision at a per-module basis, sure. If you want
to mix them inside of a single module, then it's either Cython+C or Swig+C,
not all three. But as Antoine suggested, unless you really want an
identical mapper for whole range of different languages, Swig is likely not
what you should use these days.


> In the end I rather stayed with plain C, because the 
> combination seems unimaginably awful.

Probably worth expanding your imagination. :)

Stefan

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Subtle Bug in Class Initializations

2018-08-09 Thread Steve Dower 

On 09Aug2018 0818, Erik Bray wrote:

On Mon, Aug 6, 2018 at 8:11 PM Eddie Elizondo  wrote:

3) Special case the initialization of PyType_Type and PyBaseObject_Type within 
PyType_Ready to now make all calls to PyVarObject_HEAD_INIT use NULL. To enable 
this a small change within PyType_Ready is needed to initialize PyType_Type 
PyBaseObject:


Coincidentally, I just wrote a long-ish blog post explaining in
technical details why PyVarObject_HEAD_INIT(&PyType_Type) pretty much
cannot work, at least for extension modules (it is not a problem in
the core library), on Windows (my post was focused on Cygwin but it is
a problem for Windows in general):
http://iguananaut.net/blog/programming/windows-data-import.html

The TL;DR is that it's not possible on Windows to initialize a struct
with a pointer to some other data that's found in another DLL (i.e.
&PyType_Type), unless it happens to be a function, as a special case.
But &PyType_Type obviously is not, so thinks break.


Great write-up! I think logically it should make sense that you cannot 
initialize a static value from a dynamically-linked library, but you've 
conclusively shown why that's the case. I'm not clear whether it's also 
the case on other OS's, but I don't see why it wouldn't be (unless they 
compile magic load-time resolution).



So I'm +1 for requiring passing NULL to PyVarObject_HEAD_INIT,
requiring PyType_Ready with an explicit base type argument, and maybe
(eventually) making PyVarObject_HEAD_INIT argumentless.


Since PyVarObject_HEAD_INIT currently requires PyType_Ready() in 
extension modules already, then don't we just need to fix the built-in 
types?


As far as the "eventually" case, I'd hope that eventually extension 
modules are all using PyType_FromSpec() :)


Cheers,
Steve
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Problem in importing python packages under python 3.6 environment

2018-08-09 Thread Terry Reedy 

On 8/9/2018 8:45 AM, Poornima .D. wrote:

I have limited knowledge on python development.  I am trying to write a 
test application which needs to import from many packages across  
mutliple directories.


I tried using an environment variable and appending to sys.path variable 
so that import Class methods works.


I am trying to replace the above logic by below syntax.

from directory.fileName import ClassName

But this syntax does not work.  Please let me know any solution for this 
issue.


When you post elsewhere, you must supply more information:
* Copy and paste a minimal example of code that fails, ideally just one 
line.  If your example involves importing code you wrote, add the 
directory structure of your code.

* Copy and paste the actually traceback and error message.


--
Terry Jan Reedy


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com