Re: How to debug TypeError: required field "lineno" missing from expr?

[Laura Creighton]

In a message of Mon, 29 Jun 2015 02:14:43 +0100, Mark Lawrence writes:
>Purely as an exercise I've been converting Grant Jenks' pypatt[1] from 
>2.7 to 3.4.  I've managed to sort out most of the required changes by 
>checking on what I can see with an AST pretty printer[2].  So it's 
>rather frustrating to have the compile stage throw the error given in 
>the subject line.

Are you trying to use ast.copy_location recursively somewhere?
http://bugs.python.org/issue3530  Maybe you have found something else
that doesn't work recursively?

Laura

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pure Python Data Mangling or Encrypting

[Gene Heskett]

On Saturday 27 June 2015 08:27:38 Laura Creighton wrote:
> In a message of Sat, 27 Jun 2015 20:16:47 +1000, Chris Angelico writes:
> >Okay, Johannes, NOW you're proving that you don't have a clue what
> >you're talking about. D-K effect doesn't go away...
> >
> >ChrisA
>
> You need to read the paper again.  That was the whole point -- when
> Kruger and Dunning went and taught the people at the bottom quadrile
> some basic skill in the task being estimated, and taught people at
> the top quadrile how poorly their peers were performing, their ability
> to estimate how they would score relative to their peers improved
> a whole lot.
>
> But, of course, since these were academics studying students, they had
> access to bottom-quadrile performers who actually wanted to learn and
> improve.  In the real world, it is getting the bottom-performers to
> even notice that they need improvement that may be the most difficult
> task.
>
> Laura

The rest of the readers of this list would do well to change "may" above 
to "is", and carve the last sentence into something fairly substantial 
as it is a basic truth.

Zircon crystal would be ideal, we've found a few grains of it over 4 
Billion years old, but granite would do for this generation.  Laura 
obviously "gets it".

Sadly, it is entirely too true in the real world. Too often the bottom 
person who made a good sales pitch, once hired, is either incapable of 
learning, or loses interest after he has been hired. I've seen both. The 
basic education they received is to blame for much of that effect. So 
they wind up getting shuffled around to various sub-jobs until you find 
something they can do efficiently.  Many times they weren't ever aware 
of why they were being moved.  Telling them depresses them, so its 
usually best to just let it work itself out.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to debug TypeError: required field "lineno" missing from expr?

[Terry Reedy]

On 6/29/2015 3:08 AM, Laura Creighton wrote:

In a message of Mon, 29 Jun 2015 02:14:43 +0100, Mark Lawrence writes:

Purely as an exercise I've been converting Grant Jenks' pypatt[1] from
2.7 to 3.4.  I've managed to sort out most of the required changes by
checking on what I can see with an AST pretty printer[2].  So it's
rather frustrating to have the compile stage throw the error given in
the subject line.


Are you trying to use ast.copy_location recursively somewhere?
http://bugs.python.org/issue3530   Maybe you have found something else
that doesn't work recursively?


I do not understand the closing of this issue given that a) the failing 
part of the code is copied from the doc (3.4 has the same in the 
docstring) and b) that there is no recursion for a single node ast, 
which the example should be.


The error message comes from
F:\Python\dev\34\Python\Python-ast.c: 3752:
PyErr_SetString(PyExc_TypeError, "required field \"lineno\" missing from 
stmt");


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: enumerate XML tags (keys that will become headers) along with text (values) and write to CSV in one row (as opposed to "stacked" values with one header)

[Robert Kern]

On 2015-06-28 22:00, Denis McMahon wrote:

On Sun, 28 Jun 2015 09:46:36 +0200, Stefan Behnel wrote:


Denis McMahon schrieb am 26.06.2015 um 09:44:

xml data is an unordered list, and are trying to assign an order to it.

If the xml data was ordered, either each tag would be different, or
each tag would have an attribute specifying a sequence number.


XML is not unordered. The document order is well defined and entirely
obvious from the data. Whether this order is relevant and has a meaning
or not is, however, not part of XML itself but is left to the semantics
of the specific document format at hand. Meaning, XML document formats
can choose to ignore that order and define it as irrelevant. That
doesn't mean it's not there for a given document, but it may mean that a
re-transmission of the same document would be allowed to use a different
order without changing the information.

This property applies to pretty much all structured data formats and not
just XML, by the way, also to CSV and other tabular formats.


The point I am trying to make to OP is that the following two XML
fragments define the same data:


   string 1
   string 2
   string 3


and:


   string 3
   string 2
   string 1


Each  is just a member of the collection things, the xml does not
contain sufficient information to state that  is an ordered
collection containing a specific sequence of .


Without reference to a schema that explicitly defines the children of  
to be unordered, this is not true. The XML Information Set defines the children 
of all elements to be an ordered set (in contrast to attributes, which are 
unordered). A particular XML schema may choose to consider the order of children 
of a particular element to be irrelevant, but XML itself keeps them ordered.


http://www.w3.org/TR/xml-infoset/#infoitem.element

If it didn't, then XHTML would have a hell of a time with ordered constructs 
like this:


  
First item
Second item
Third item
  

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
https://mail.python.org/mailman/listinfo/python-list

Recreate file associations?

[random832]

I installed Python 2.1 for historical interest and it associated python
files with the python 2.1 interpreter. I installed 3.4 afterwards (I had
3.3 as my latest python version before) expecting this to fix it, but it
left it alone. Even deleting all the Python associations from the
registry and then repairing 3.4, it didn't create them. How do I get it
to make the file associations the same as the first time I installed
Python 3?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: enumerate XML tags (keys that will become headers) along with text (values) and write to CSV in one row (as opposed to "stacked" values with one header)

[Sahlusar]

On Sunday, 28 June 2015 03:46:56 UTC-4, Stefan Behnel  wrote:
> Denis McMahon schrieb am 26.06.2015 um 09:44:
> > xml data is an unordered list, and are trying to assign an order to it.
> > 
> > If the xml data was ordered, either each tag would be different, or each 
> > tag would have an attribute specifying a sequence number.
> 
> XML is not unordered. The document order is well defined and entirely
> obvious from the data. Whether this order is relevant and has a meaning or
> not is, however, not part of XML itself but is left to the semantics of the
> specific document format at hand. Meaning, XML document formats can choose
> to ignore that order and define it as irrelevant. That doesn't mean it's
> not there for a given document, but it may mean that a re-transmission of
> the same document would be allowed to use a different order without
> changing the information.
> 
> This property applies to pretty much all structured data formats and not
> just XML, by the way, also to CSV and other tabular formats.
> 
> Stefan

@Stefan, Ned, and Robert: You have all hit the nail on the head. I do not have 
an authentic and veritable XSD (or XML data structures for that matter). So far 
it is all deprecated and/or anonymized data from the client. Therefore, I can 
only hypothesize what the end output will be for the database architecture that 
I am working with. 

>From what I understand, therefore, based on your constructive insight, is that 
>the 14 occurrences of the same tag (regardless of placement relative to 
>neighbouring children and the root are all being defined as the same key. 
>However, their individual values are also being treated as the same (from the 
>algorithm that I wrote in my Stack Overflow post (please see above)). The 
>constraint is that I am anticipating terabytes of data every day from the 
>client in the coming months. The algorithm should be able to parse, and write 
>out to CSV in the most efficient manner. That is my design constraint. I 
>welcome your feedback on this. 

Here is the post, again, for your convenience: 

http://stackoverflow.com/questions/31058100/enumerate-column-headers-in-csv-that-belong-to-the-same-tag-key-in-python
 


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Recreate file associations?

[Terry Reedy]

On 6/29/2015 10:03 AM, random...@fastmail.us wrote:

I installed Python 2.1 for historical interest and it associated python
files with the python 2.1 interpreter. I installed 3.4 afterwards (I had
3.3 as my latest python version before) expecting this to fix it, but it
left it alone. Even deleting all the Python associations from the
registry and then repairing 3.4, it didn't create them. How do I get it
to make the file associations the same as the first time I installed
Python 3?


Registry => Windows.
Did you check [X] Make this my default python?

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Scapy and MGCP

[Laura Creighton]

In a message of Mon, 29 Jun 2015 20:06:05 -, "Devaki Chokshi (dchokshi)" wr
ites:
>Hello,
>
>As per reply received I have begun to use scapy for MGCP. 
>
>I started off with  reading a .pcap file with MGCP packets. 
>
>For example:
>
>from scapy.all import *
>from scapy.utils import *
>from scapy.layers.mgcp import *
>
>mgcp_pkts = rdpcap("my-mgcp-capture.pcap")
>
>However, rdpcap() is returning all MGCP packets as UDP packets. The output is 
>something like:
>
>
>
>All 262 UDP packets are actually MGCP packets. 
>
>I would have thought rdpcap() will understand MGCP packet format. 
>
>Would appreciate help on how I can make scapy read MGCP packets from a .pcap 
>file.
>
>Thank you
>Devaki

Ask on the secdev mailing list.
see: https://bitbucket.org/secdev/scapy/wiki/doc/index

This 'everything is UDP' is normal, but I forget how to turn it off
if you don't want that, and it's been over 3 years so there may be
better ways than what I used  plus for me it wasn't MGCP packets
anyhow.  At any rate  they will know over there.

Laura
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pure Python Data Mangling or Encrypting

[Randall Smith]

On 06/28/2015 09:21 AM, Jon Ribbens wrote:

On 2015-06-27, Randall Smith  wrote:

Thankyou.  Nice points. I do think given the risks (there are always
risks) discussed, a successful attack of this nature is not very likely.
   Worse case, something that looks like this would land on the disk.

crc32 checksum + translation table + malware

with a generated base64 name and no extension.


I'm not sure why you're bothering with the checksum, it doesn't seem
to me that it buys you anything. Personally I'd do something like
this (pseudocode):



Same reason newer filesystems like BTRFS use checkusms (BTRFS uses 
CRC32).  The storage machine runs periodic file integrity checks.  It 
has no control over the underlying filesystem.





   def obfuscate(data):
   encode_key = list(range(256))
   random.shuffle(encode_key)
   encode_key = bytes(encode_key)
   decode_key = bytes(encode_key.index(i) for i in range(256))
   return decode_key + data.translate(encode_key) + decode_key

   def deobfuscate(data):
   return data[256:-256].translate(data[:256])

The reason for appending the key as well as prepending it is that some
anti-virus or malware scanners may well look at the last part of the
file first, so putting something entirely locally-generated there may
add a bit of safety. You could also simply pad with nulls or something
of course, but again I can imagine some tools skipping backwards past
nulls.




--
https://mail.python.org/mailman/listinfo/python-list

Re: Pure Python Data Mangling or Encrypting

[Randall Smith]

On 06/27/2015 01:50 PM, Steven D'Aprano wrote:

On Sun, 28 Jun 2015 03:08 am, Randall Smith wrote:


Though I didn't mention it in the description, the storage server is
appending a CRC32 checksum for routine integrity checks.  So by the time
the data hits the disk, it will have added both a 256 byte translation
table and a 4 byte checksum.



http://stackoverflow.com/questions/1515914/crc32-collision







Not sure why you posted the link.  The crc32 checksum is just to check 
for possible filesystem corruption.  The system does periodic data 
corruption checks.  BTRFS uses crc32 checksums also.  Please explain.


-Randall

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pure Python Data Mangling or Encrypting

[Jon Ribbens]

On 2015-06-29, Randall Smith  wrote:
> Same reason newer filesystems like BTRFS use checkusms (BTRFS uses 
> CRC32).  The storage machine runs periodic file integrity checks.  It 
> has no control over the underlying filesystem.

True, but presumably neither does it have anything it can do to
rectify the situation if it finds a problem, and the client will
have to keep its own secure hash of its file anyway. (Unless I suppose
the server actually can request a new copy from the client or another
server if it finds a corrupt file?)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to debug TypeError: required field "lineno" missing from expr?

[Terry Reedy]

On 6/29/2015 7:17 AM, Terry Reedy wrote:

On 6/29/2015 3:08 AM, Laura Creighton wrote:

In a message of Mon, 29 Jun 2015 02:14:43 +0100, Mark Lawrence writes:

Purely as an exercise I've been converting Grant Jenks' pypatt[1] from
2.7 to 3.4.  I've managed to sort out most of the required changes by
checking on what I can see with an AST pretty printer[2].  So it's
rather frustrating to have the compile stage throw the error given in
the subject line.


Are you trying to use ast.copy_location recursively somewhere?
http://bugs.python.org/issue3530   Maybe you have found something else
that doesn't work recursively?


The issue is that copy_location is not recursive and does not fix 
children of the node being fixed, whereas the example method, copied 
from the NodeTransformer doc, creates 3 new nodes, not just 1, that need 
fixing.  I reopened the issue as a doc issue to fix the example and 
recommend the use of recursive fix_missing_locations() instead of the 
non-recursive copy_location.


The only other location-related function in ast is increment_lineno, 
which also works on multiple nodes.  I would not be surprised is Mark's 
issue results from Mark or someone else copying the bad doc example.



I do not understand the closing of this issue given that a) the failing
part of the code is copied from the doc (3.4 has the same in the
docstring) and b) that there is no recursion for a single node ast,
which the example should be.


The original ast is actually an Expression with a Name.  But the issue 
is that the Name is replaced with a Subscript with Name and Str children 
and all three need locations added.



The error message comes from
F:\Python\dev\34\Python\Python-ast.c: 3752:
PyErr_SetString(PyExc_TypeError, "required field \"lineno\" missing from
stmt");


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to debug TypeError: required field "lineno" missing from expr?

[Mark Lawrence]

On 29/06/2015 21:59, Terry Reedy wrote:

On 6/29/2015 7:17 AM, Terry Reedy wrote:

On 6/29/2015 3:08 AM, Laura Creighton wrote:

In a message of Mon, 29 Jun 2015 02:14:43 +0100, Mark Lawrence writes:

Purely as an exercise I've been converting Grant Jenks' pypatt[1] from
2.7 to 3.4.  I've managed to sort out most of the required changes by
checking on what I can see with an AST pretty printer[2].  So it's
rather frustrating to have the compile stage throw the error given in
the subject line.


Are you trying to use ast.copy_location recursively somewhere?
http://bugs.python.org/issue3530   Maybe you have found something else
that doesn't work recursively?


The issue is that copy_location is not recursive and does not fix
children of the node being fixed, whereas the example method, copied
from the NodeTransformer doc, creates 3 new nodes, not just 1, that need
fixing.  I reopened the issue as a doc issue to fix the example and
recommend the use of recursive fix_missing_locations() instead of the
non-recursive copy_location.

The only other location-related function in ast is increment_lineno,
which also works on multiple nodes.  I would not be surprised is Mark's
issue results from Mark or someone else copying the bad doc example.



FTR there is no call to copy_location in Grant's existing code.  Whether 
or not one is actually needed in 3.4 when compared to 2.7 is another 
question.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Linear time baseconversion

[jonas . thornvall]

http://jt.node365.se/baseconversion8.html 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linear time baseconversion

[Ian Kelly]

On Mon, Jun 29, 2015 at 4:39 PM,   wrote:
> http://jt.node365.se/baseconversion8.html

Back of the envelope mental calculation, that appears to be quadratic,
not linear. Doubling the length of the input results in an approximate
quadrupling of the time taken to produce the output.

That noted, this is off-topic for this list, which is for discussion
about Python. Please take this to somewhere else like
comp.lang.javascript instead.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linear time baseconversion

[Ian Kelly]

On Mon, Jun 29, 2015 at 4:56 PM, Ian Kelly  wrote:
> On Mon, Jun 29, 2015 at 4:39 PM,   wrote:
>> http://jt.node365.se/baseconversion8.html
>
> Back of the envelope mental calculation, that appears to be quadratic,
> not linear. Doubling the length of the input results in an approximate
> quadrupling of the time taken to produce the output.
>
> That noted, this is off-topic for this list, which is for discussion
> about Python. Please take this to somewhere else like
> comp.lang.javascript instead.

By the way, I think you have a bug. I did my time testing by
concatenating the default input number to itself several times. At
5184 decimal digits, which was the last case I tried, the 58th output
digit was 111, after which point the remaining 672 output digits
are all 12665464, which seemed rather improbable.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linear time baseconversion

[jonas . thornvall]

Den tisdag 30 juni 2015 kl. 01:11:51 UTC+2 skrev Ian:
> On Mon, Jun 29, 2015 at 4:56 PM, Ian Kelly  wrote:
> > On Mon, Jun 29, 2015 at 4:39 PM,   wrote:
> >> http://jt.node365.se/baseconversion8.html
> >
> > Back of the envelope mental calculation, that appears to be quadratic,
> > not linear. Doubling the length of the input results in an approximate
> > quadrupling of the time taken to produce the output.
> >
> > That noted, this is off-topic for this list, which is for discussion
> > about Python. Please take this to somewhere else like
> > comp.lang.javascript instead.
> 
> By the way, I think you have a bug. I did my time testing by
> concatenating the default input number to itself several times. At
> 5184 decimal digits, which was the last case I tried, the 58th output
> digit was 111, after which point the remaining 672 output digits
> are all 12665464, which seemed rather improbable.

No there is no bug, try 12665465^77 wolfram alpha gives
7976781785088446084873306945009387424538813088491673402748035781202063831184317457635773873975198429536727051788368983080016275695338973787545408588270715883570324803120051806974411884509607864124408509087974736382639726555779113221450078243163340395784592348080359062163885323462144257596380766496966084237358222613649636595300081207683175410584139478853066960795599027150403706812116411354218686846750615496852415389467898730884554700066802269059177874281189887271741594702334892800692129299635716260471508809448693000376806594431400299072265625

Try that number you get a ONE followed by 77 ZEROS
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linear time baseconversion

[Ben Bacarisse]

Ian Kelly  writes:

> On Mon, Jun 29, 2015 at 4:56 PM, Ian Kelly  wrote:
>> On Mon, Jun 29, 2015 at 4:39 PM,   wrote:
>>> http://jt.node365.se/baseconversion8.html

> By the way, I think you have a bug. I did my time testing by
> concatenating the default input number to itself several times. At
> 5184 decimal digits, which was the last case I tried, the 58th output
> digit was 111, after which point the remaining 672 output digits
> are all 12665464, which seemed rather improbable.

Yes, it's a bug.  Because it's off-topic, I won't say more here but I
posted a simpler test case to comp.lang.javacript just now (1000
converted to base 100).  I would not normally reply here, but I wanted
to acknowledge your post as prompting me to look for other cases.
Followup-to: set.

-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Readline -- cannot bind to both Ctrl-tab and tab at the same time?

[Cameron Simpson]

On 29Jun2015 11:36, Chris Angelico  wrote:

On Mon, Jun 29, 2015 at 4:55 AM, Steven D'Aprano  wrote:

Try Ctrl-TAB again, and you'll get "raise" instead of "import".

Can anyone else replicate this issue?

Is this a Python issue, a problem with the terminal I am using, or readline
in general?


Confirmed with Python 3.6 on Debian Jessie. Delving into Laura's
suggestion that it's a readline problem, I came across this:

http://stackoverflow.com/questions/12044574/getting-complete-and-menu-complete-to-work-together

Testing with "showkey -a" suggests that tab and ctrl-tab indeed send
the same bytes. I wonder if there's a way to change terminal type or
fiddle with terminfo to change this? GUI programs obviously don't have
this conflation.


Yes, this was my surmise (and expectation, frankly - what did you expect 
ctrl-TAB to send?)  So there is no way for the program in the terminal to 
distinguish one from the other; you will always only get one of your bindings.


I would not expect terminfo to get you anything on its own - it is consulted by 
the receiving program. The sending program (the terminal emulator) needs to be 
configured to send different sequences for TAB and ctrl-TAB. Once that is done, 
_then_ you may be able to augument your terminfo to express this behaviour and 
have readline honour the change.


1: Work backward from the readline internals to find out if it consults a 
terminfo property to find out what ctrl-TAB might be.


2: Decide on what you want ctrl-TAB to send instead of the TAB code.

3: Make a new terminfo clause defining that (for readline's use) and hooking 
onto your old terminfo definition for the rest (tc=other_terminal in termcap, 
possibly the same in terminfo).


4: Configure your terminal emulator to send that sequence.

Regarding 4, with proper old-style X11 terminals you could control all this 
directly with the VT100.Translations properties. Example from my xterm 
Xdefaults file; the leading F1 definition is probably what you want:


 XTerm*.VT100.Translations:  #override \n\
   F1: string(0x1b) \n \
   F10   : secure() \n \
   KP_End: scroll-forw(100,page) \n \
   KP_Prior  : scroll-back(1,halfpage) \n \
   KP_Next   : scroll-forw(1,halfpage) \n\
   End   : scroll-forw(100,page) \n \
   CtrlUp: scroll-back(1,line) \n \
   CtrlDown  : scroll-forw(1,line) \n\
   Prior : scroll-back(1,halfpage) \n \
   Next  : scroll-forw(1,halfpage) \n\
   Meta  : string(0x1b) insert() \n\
   Alt   : string(0x1b) insert() \n\
   BackSpace : string(0x08) \n \
   Shift : ignore() \n\
   Shift   : insert-selection(PRIMARY,CUT_BUFFER0)\n\
 : ignore() \n\
   : insert-selection(PRIMARY,CUT_BUFFER0)\n\
   Shift : ignore() \n\
   Shift   : string("us ") insert-selection(PRIMARY,CUT_BUFFER0) 
string("\n") \n\

   ~Ctrl ~Meta   : ignore()\n\
   ~Ctrl ~Meta : ignore()\n
 ! commented out, previous experiments
 ! XTerm*VT100.translations: #override Shift'F1':insert("hi guys")
 ! XTerm*VT100.translations: #override 
 Ctrl'U':uphalfscreen()\nCtrl'D'downhalfscreen()


The new fangled terminals like gnome-terminal or Konsole probably threw that 
stuff out as they threw out all the other normal X11 config hooks, and 
doubtless use their own arcane system. Enjoy.


Cheers,
Cameron Simpson 

The top three answers:  Yes I *am* going to a fire!
   Oh! We're using *kilometers* per hour now.
   I have to go that fast to get back to my own time.
- Peter Harper 
--
https://mail.python.org/mailman/listinfo/python-list

Re: enumerate XML tags (keys that will become headers) along with text (values) and write to CSV in one row (as opposed to "stacked" values with one header)

[Denis McMahon]

On Sun, 28 Jun 2015 17:07:00 -0700, Ned Batchelder wrote:

> On Sunday, June 28, 2015 at 5:02:19 PM UTC-4, Denis McMahon wrote:

>> 
>>   string 3
>>   string 2
>>   string 1
>> 

>> Each  is just a member of the collection things, the xml does
>> not contain sufficient information to state that  is an ordered
>> collection containing a specific sequence of .
> 
> You are right that XML does not specify that  is an ordered
> collection.
> But XML does preserve the order of the children.  There are many XML
> schema that rely on XML's order-preserving nature.

But what we *don't* know is whether the order of the umpteen identical 
tags in the XML has any significance in terms of the original data, 
although the OP seems intent on assigning some significance to that order 
without any basis for doing so.

Consider the following tuple:

t = (tuplemember_1, tuplemember_2,  tuplemember_n)

Can we safely assume that if the tuple is ever converted to xml, either 
now or at some future date using whatever the code implementation is 
then, that the order of the items will be preserved:


  tuplemember_1
  tuplemember_2

  tuplemember_n/item>


And if we're reading that xml structure at some point in the future, is 
it safe to assume that the tuple members are in the same order in the xml 
as they were in the original tuple?

For sanity  should have an attribute specifying the sequence of the 
item in it's tuple.
-- 
Denis McMahon, denismfmcma...@gmail.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Matplotlib X-axis timezone trouble

[Peter Pearson]

The following code produces a plot with a line running from (9:30, 0) to
(10:30, 1), not from (8:30, 0) to (9:30, 1) as I desire.

If I use timezone None instead of pacific, the plot is as desired, but
of course that doesn't solve the general problem of which this is a
much-reduced example.

If I use timezone US/Central, I get the same (bad) plot.

import matplotlib.pyplot as plt
import datetime
import pytz
pacific = pytz.timezone("US/Pacific")
fig = plt.figure()
plt.plot([datetime.datetime(2014, 10, 7, 8, 30, tzinfo=pacific),
  datetime.datetime(2014, 10, 7, 9, 30, tzinfo=pacific)],
 [0,1], marker="o", color="green")
fig.autofmt_xdate()
plt.show()

Does anybody know why this shift is occurring?  Is Matplotlib
confused about what timezone to use in labeling the axis?  How
would I tell it what timezone to use (preferably explicitly in
the code, not in matplotlibrc)?

Thanks.

-- 
To email me, substitute nowhere->runbox, invalid->com.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Matplotlib X-axis timezone trouble

[Laura Creighton]

In a message of 30 Jun 2015 00:56:26 +, Peter Pearson writes:
>The following code produces a plot with a line running from (9:30, 0) to
>(10:30, 1), not from (8:30, 0) to (9:30, 1) as I desire.
>
>If I use timezone None instead of pacific, the plot is as desired, but
>of course that doesn't solve the general problem of which this is a
>much-reduced example.
>
>If I use timezone US/Central, I get the same (bad) plot.
>
>import matplotlib.pyplot as plt
>import datetime
>import pytz
>pacific = pytz.timezone("US/Pacific")
>fig = plt.figure()
>plt.plot([datetime.datetime(2014, 10, 7, 8, 30, tzinfo=pacific),
>  datetime.datetime(2014, 10, 7, 9, 30, tzinfo=pacific)],
> [0,1], marker="o", color="green")
>fig.autofmt_xdate()
>plt.show()
>
>Does anybody know why this shift is occurring?  Is Matplotlib
>confused about what timezone to use in labeling the axis?  How
>would I tell it what timezone to use (preferably explicitly in
>the code, not in matplotlibrc)?
>
>Thanks.
>
>-- 
>To email me, substitute nowhere->runbox, invalid->com.
>-- 
>https://mail.python.org/mailman/listinfo/python-list


I know that your problem is that all your times are in UTC, even
though you do not want this.  I had this too.  I forget what I did.

I do not know if this solution from stackoverflow.com will work for
you -- it is definitely different from what I did, but that does
not in any way make it wrong.  Try and see?  and report back?

http://stackoverflow.com/questions/4485607/matplotlib-plot-date-keeping-times-in-utc-even-with-custom-timezone
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Matplotlib X-axis timezone trouble

[Chris Angelico]

On Tue, Jun 30, 2015 at 10:56 AM, Peter Pearson
 wrote:
> The following code produces a plot with a line running from (9:30, 0) to
> (10:30, 1), not from (8:30, 0) to (9:30, 1) as I desire.
>
> pacific = pytz.timezone("US/Pacific")
> plt.plot([datetime.datetime(2014, 10, 7, 8, 30, tzinfo=pacific),
>   datetime.datetime(2014, 10, 7, 9, 30, tzinfo=pacific)],
>  [0,1], marker="o", color="green")

Sounds to me like a bug in DST handling - in October 2014, Los Angeles
was on Pacific Daylight Time, which would give a one-hour offset if
matplotlib is handling things on the basis of Pacific Standard Time. I
don't know if that helps at all with figuring out a solution, though.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pure Python Data Mangling or Encrypting

[Steven D'Aprano]

On Tue, 30 Jun 2015 06:52 am, Randall Smith wrote:

> Not sure why you posted the link.  The crc32 checksum is just to check
> for possible filesystem corruption.  The system does periodic data
> corruption checks.  BTRFS uses crc32 checksums also.  Please explain.

The file system can trust that anything writing to a file is allowed to
write to it, in doesn't have to defend against malicious writes. As I
understand it, your application does.

Here is the attack scenario I have in mind:

- you write a file to my computer, and neglect to encrypt it;
- and record the checksum for later;
- I insert malware into your file;
- you retrieve the file from me;
- if the checksum matches what you have on record, you accept the file;
- since you are using CRC, it is quite easy for me to ensure the 
  checksums match after inserting malware;
- and I have now successfully injected malware into your computer.

I'm making an assumption here -- I assume that the sender records a checksum
for uploaded files so that when they get something back again they can tell
whether or not it is the same content they uploaded.

* * * 

By the way, regarding the use of a substitution cipher, I spoke to the
crypto guy at work, and "preimage attack" is not quite the right
terminology, since that's normally used in the context of hash functions.
It's almost a "known ciphertext attack", but not quite, since that
terminology refers to guessing the key from the ciphertext.

I was wrong: cryptographically strong ciphers are generally NOT resistant to
what I described as a preimage attack. If the key leaks, using AES won't
save you: an attacker with access to the key can produce a ciphertext that
decrypts to the malware of his choice, regardless of whether you use
AES-256 or rot-13. There may be other encryption methods which don't suffer
from that, but he doesn't know of them off the top of his head.

His comment was, "don't leak the key".

The other threat I mentioned is that the receiver will read the content of
the file. For that, a strong cipher is much to be preferred over a weak
one, and it needs to be encrypted by the sending end, not the receiving
end. (If the receiving end does it, it has to keep the key so it can
decrypt before sending back, which means the computer's owner can just grab
the key and read the files.)



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Matplotlib X-axis timezone trouble

[Peter Pearson]

On Tue, 30 Jun 2015 12:11:31 +1000, Chris Angelico  wrote:
> On Tue, Jun 30, 2015 at 10:56 AM, Peter Pearson
> wrote:
>> The following code produces a plot with a line running from (9:30, 0) to
>> (10:30, 1), not from (8:30, 0) to (9:30, 1) as I desire.
>>
>> pacific = pytz.timezone("US/Pacific")
>> plt.plot([datetime.datetime(2014, 10, 7, 8, 30, tzinfo=pacific),
>>   datetime.datetime(2014, 10, 7, 9, 30, tzinfo=pacific)],
>>  [0,1], marker="o", color="green")
>
> Sounds to me like a bug in DST handling - in October 2014, Los Angeles
> was on Pacific Daylight Time, which would give a one-hour offset if
> matplotlib is handling things on the basis of Pacific Standard Time. I
> don't know if that helps at all with figuring out a solution, though.

Thanks, I think you're onto something.  Points in a non-daylight-saving
month get plotted correctly:

plt.plot([datetime.datetime(2014, 12, 7, 8, 30, tzinfo=pacific),
  datetime.datetime(2014, 12, 7, 9, 30, tzinfo=pacific)],
 [0,1], marker="o", color="green")

Time zones teem with sneaky software problems, and so does daylight-saving
time, so this problem might strain my brain.  Maybe it's going to turn
out that my expectations are unreasonable . . . as in, "Well, smarty pants,
how do you want the axis labelled when the abscissas straddle the
beginning of daylight-saving time?"  I'll research and digest.

-- 
To email me, substitute nowhere->runbox, invalid->com.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Matplotlib X-axis timezone trouble

[Peter Pearson]

On Tue, 30 Jun 2015 04:03:57 +0200, Laura Creighton  wrote:
> In a message of 30 Jun 2015 00:56:26 +, Peter Pearson writes:
>>The following code produces a plot with a line running from (9:30, 0) to
>>(10:30, 1), not from (8:30, 0) to (9:30, 1) as I desire.
>>
>>If I use timezone None instead of pacific, the plot is as desired, but
>>of course that doesn't solve the general problem of which this is a
>>much-reduced example.
>>
>>If I use timezone US/Central, I get the same (bad) plot.
>>
>>import matplotlib.pyplot as plt
>>import datetime
>>import pytz
>>pacific = pytz.timezone("US/Pacific")
>>fig = plt.figure()
>>plt.plot([datetime.datetime(2014, 10, 7, 8, 30, tzinfo=pacific),
>>  datetime.datetime(2014, 10, 7, 9, 30, tzinfo=pacific)],
>> [0,1], marker="o", color="green")
>>fig.autofmt_xdate()
>>plt.show()
>>
>>Does anybody know why this shift is occurring?  Is Matplotlib
>>confused about what timezone to use in labeling the axis?  How
>>would I tell it what timezone to use (preferably explicitly in
>>the code, not in matplotlibrc)?
[snip]
>
> I know that your problem is that all your times are in UTC, even
> though you do not want this.  I had this too.  I forget what I did.

Actually, UTC is very helpful in this project, which has to
deal with timestamped data from various timezones.  After I finally
stumbled upon datetime's astimezone() method, things seemed to be
going smoothly . . .

> I do not know if this solution from stackoverflow.com will work for
> you -- it is definitely different from what I did, but that does
> not in any way make it wrong.  Try and see?  and report back?
>
> http://stackoverflow.com/questions/4485607/matplotlib-plot-date-keeping-times-in-utc-even-with-custom-timezone

Thanks for the pointer.  I'll report back.


-- 
To email me, substitute nowhere->runbox, invalid->com.
-- 
https://mail.python.org/mailman/listinfo/python-list

Cross compiling C Python2.7.10 for arm on x86_64 linux box.

[Andrew Robinson]

Hi,

I'm Needing to get python 2.7.10 to cross compile correctly for an ARM 
embedded device.
I'm very close, as it does build with warnings, but the result is 
defective and I'm not sure how to fix it.
For some odd reason, the interpreter does run -- but I either get random 
segfaults -- or if I configure --without-pymalloc, I get messages when I 
import libraries saying that:


---
Python 2.7.10 (default, Jun 29 2015, 23:00:31)
[GCC 4.8.1 20130401 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import math
Traceback (most recent call last):
  File "", line 1, in 
ImportError: /mnt/user/lib/python2.7/lib-dynload/math.so: undefined 
symbol: Py_InitModule4

[40857 refs]
>>>
---

That message suggests this might be an issue with 32/64 bit machine 
architectures confilicts according to information I googled on it.


I am also seeing many warnings during build like the following which 
confirms some kind of build size mismatch:
*** WARNING: renaming "pyexpat" since importing it failed: 
build/lib.linux-x86_64-2.7/pyexpat.so: wrong ELF class: ELFCLASS32 
building '_elementtree' extension


I think Py_InitModule4 is normally found in libpython2.7.so,  as the 
symbol at least exists in the 32 and 64 bit complations of python that I 
have ; but when I check the cross compiled version, it's not there.
So I think the python ./configure script might be getting confused about 
the size of the target it's trying to build as I am building it on a 64 
bit x86 machine, and the target is a 32 bit arm processor.



The following is what I am doing from bash shell (linux) to try and 
cross compile python.

And idea of what I am doing wrong ?

---
# A par# A parser generator and build system version of python are 
supposed to be

# needed to run parts of the cross compilation;  I do see python used in the
# Makefile, but no references to a buid version of PGEN are defined,
# so I don't know if PGEN gets used or not -- but I build it anyway...
# As this is what receipies on the web say to do...

make distclean
./configure
make Parser/pgen python
mv python python_for_build
mv Parser/pgen Parser/pgen_for_build
make distclean

# fix setup.py to handle installing to the target system's fake install
# directory found on the build system at $DEVICEROOT.

if grep -q os.environ['DEVICEROOT'] ; then echo "Already patched" ; else
sed -i setup.py 's%^[[:space:]]*math_libs = [].*$%if 
'DEVICEROOT' in os.environ:\nlib_dirs += 
os.environ['DEVICEROOT']+'/mnt/user/lib\nlib_dirs += 
os.environ['DEVICEROOT']+'/mnt/user/include\n'%

fi

# We want utf-8, unicode terminal handling -- so make sure python compiles
# with ncursesw substituted for curses.

CURSESFLAGS=`pkg-config --cflags ncursesw`

# Configure python to be built
CFLAGS="${CFLAGS} ${CURSESFLAGS} -g3 -ggdb -gdwarf-4" ./configure 
--host=${CROSSTARGET} --build=i686-linux --enable-unicode 
--enable-shared --with-pydebug --prefix=/mnt/user --disable-ipv6 
--without-pymalloc ac_cv_file__dev_ptmx=yes ac_cv_file__dev_ptc=no 
ac_cv_have_long_long_format=yes PYTHON_FOR_BUILD=${PWD}/python_for_build


# Fix a bug in the Makefile
# The build version of python, ought not try to actually use the ARM 
libraries.
sed -i -e 's%\([[:space:]]\)\(PYTHONPATH=$(DESTDIR)$(LIBDEST)\)%\1-\2%' 
Makefile

echo "Fix the makefile if you can"
sleep 10

make PYTHON_FOR_BUILD=${PWD}/python_for_build CROSS_COMPILE_TARGET=yes

echo " Waiting to allow you to see error messages before installing "
sleep 10

# Optionally, binary file stripping could be carried out on the python 
binary

# Don't strip if you are doing debugging of python
# strip --strip-unneeded python

make install DESTDIR=${DEVICEROOT} 
PYTHON_FOR_BUILD=${PWD}/python_for_build 
PGEN_FOR_BUILD=${PWD}/pgen_for_build

ser generator and build system version of python are supposed to be
# needed to run parts of the cross compilation;  I do see python used in the
# Makefile, but no references to a buid version of PGEN are defined,
# so I don't know if PGEN gets used or not -- but I build it anyway...
# As this is what receipies on the web say to do...

make distclean
./configure
make Parser/pgen python
mv python python_for_build
mv Parser/pgen Parser/pgen_for_build
make distclean

# fix setup.py to handle installing to the target system's fake install
# directory found on the build system at $DEVICEROOT.

if grep -q os.environ['DEVICEROOT'] ; then echo "Already patched" ; else
sed -i setup.py 's%^[[:space:]]*math_libs = [].*$%if 
'DEVICEROOT' in os.environ:\nlib_dirs += 
os.environ['DEVICEROOT']+'/mnt/user/lib\nlib_dirs += 
os.environ['DEVICEROOT']+'/mnt/user/include\n'%

fi

# We want utf-8, unicode terminal handling -- so make sure python compiles
# with nc

Re: Linear time baseconversion

[jonas . thornvall]

Den tisdag 30 juni 2015 kl. 01:11:51 UTC+2 skrev Ian:
> On Mon, Jun 29, 2015 at 4:56 PM, Ian Kelly  wrote:
> > On Mon, Jun 29, 2015 at 4:39 PM,   wrote:
> >> http://jt.node365.se/baseconversion8.html
> >
> > Back of the envelope mental calculation, that appears to be quadratic,
> > not linear. Doubling the length of the input results in an approximate
> > quadrupling of the time taken to produce the output.
> >
> > That noted, this is off-topic for this list, which is for discussion
> > about Python. Please take this to somewhere else like
> > comp.lang.javascript instead.
> 
> By the way, I think you have a bug. I did my time testing by
> concatenating the default input number to itself several times. At
> 5184 decimal digits, which was the last case I tried, the 58th output
> digit was 111, after which point the remaining 672 output digits
> are all 12665464, which seemed rather improbable.

It isn't i used the power to check with wolfram, but there is a plus 1 bug as 
ben noticed.

But not in this slower version.
http://jt.node365.se/baseconversion5.html
-- 
https://mail.python.org/mailman/listinfo/python-list