Re: parse binary file

[Aaron]

On 01/29/2012 03:04 PM, Andrea Crotti wrote:

On 01/29/2012 07:51 AM, contro opinion wrote:

please download the attachment ,and put in  c:\test.data



Your program should never use hard-coded path, and actually
I think the majority here is not using windows.

But I also think that the majority of people on here could change his 
script to run if they are not on Windows

--
http://mail.python.org/mailman/listinfo/python-list

Re: except clause syntax question

[Aaron]

On 01/30/2012 06:41 PM, Charles Yeomans wrote:

To catch more than one exception type in an except block, one writes

except (A, B, C) as e:

I'm wondering why it was decided to match tuples, but not lists:

except [A, B, C] as e:

The latter makes more sense semantically to me -- "catch all exception types in a list" 
as opposed to "catch this single thing composed of three exception types".


Charles Yeomans




Then,  semantically, shouldn't it be a set?
--
http://mail.python.org/mailman/listinfo/python-list

using simplejson.dumps to encode a dictionary to json

[Aaron]

Hi there,

I am attempting to use simplejson.dumps to convert a dictionary to a json 
encoded string.

For some reason this doesn't work - it would be awesome if someone could give 
me a clue as to why.

loginreq = {"username":username, "password":password, "productType":"CFD_Demo"}
authreq_data = {"req":loginreq}
authreq_data = simplejson.dumps(authreq_data)

A

#The dictionary contains a dictionary
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using simplejson.dumps to encode a dictionary to json

[Aaron]

If I print authreq_data to screen  I get

{"req": {"username": "##", "password": "#", "productType": "CFD_Demo"}}

Essentially I want the inner brackets to be  [ ] instead of {} but alternating 
on each level so it would be:

{"req": [{"username": "##", "password": "#", "productType": 
"CFD_Demo"}]}

as per usual json.

A
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using simplejson.dumps to encode a dictionary to json

[Aaron]

I'm attempting to write an xml as a json object. the xml object is

authreq_data = """
 
 
%s 
CFD_Demo 
%s 

""" %(username, password)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using simplejson.dumps to encode a dictionary to json

[Aaron]

Rock on, thanks guys.

I'm on python 2.5 btw - I use it with google app engine so cannot upgrade 
unfortunately.

A
-- 
http://mail.python.org/mailman/listinfo/python-list

From an existing Pandas DataFrame, how can I create a summary DataFrame based on the union of overlapping date ranges (given a start and an end date) and an additional column?

[Aaron]

Hello,

Given a dateframe with trips made by employees of different companies, I am
trying to generate a new dataframe with only the company names.  I am
looking to combine the overlapping travel times from employees of the SAME
company into a single row.  If there are no overlapping travel times, then
that row just transfers over as-is.  When there are overlapping travel
times, then the following will happen:

--The name field is removed b/c that is no longer relevant (company name
stays), the Depart date will be the earliest date of any of the trip dates
regardless of the employee, the Return date will be the latest date of any
of the trip dates regardless of the employee, the charges for the trip will
be summed

For example, if trips had dates 01/01/20 - 01/31/20, 01/15/20 - 02/15/20,
02/01-20 - 02/28/20, then all three would be combined.  The starting date
will be 1/1/20 and ending as of 2/28/20.  Basically, the company was on
that trip from start to finish… kinda like a relay run handing off the
baton.  Also, the charges will be summed for each of those trips and
transferred over to the single row.

Here is the starting dataframe code/output (note: the row order is
typically not already sorted by company name as in this example):

import pandas as pd


emp_trips = {'Name': ['Bob','Joe','Sue','Jack', 'Henry', 'Frank',
'Lee', 'Jack'],
'Company': ['ABC', 'ABC', 'ABC', 'HIJ', 'HIJ', 'DEF', 'DEF', 'DEF'],
'Depart' : ['01/01/2020', '01/01/2020', '01/06/2020',
'01/01/2020', '05/01/2020', '01/13/2020', '01/12/2020', '01/14/2020'],
'Return' : ['01/31/2020', '02/15/2020', '02/20/2020',
'03/01/2020', '05/05/2020', '01/15/2020', '01/30/2020', '02/02/2020'],
'Charges': [10.10, 20.25, 30.32, 40.00, 50.01, 60.32, 70.99, 80.87]
}

df = pd.DataFrame(emp_trips, columns = ['Name', 'Company', 'Depart',
'Return', 'Charges'])
# Convert to date format
df['Return']= pd.to_datetime(df['Return'])
df['Depart']= pd.to_datetime(df['Depart'])

  Name Company Depart Return  Charges0Bob ABC
2020-01-01 2020-01-3110.101Joe ABC 2020-01-01 2020-02-15
 20.252Sue ABC 2020-01-06 2020-02-2030.323   Jack HIJ
2020-01-01 2020-03-0140.004  Henry HIJ 2020-05-01 2020-05-05
 50.015  Frank DEF 2020-01-13 2020-01-1560.326Lee DEF
2020-01-12 2020-01-3070.997   Jack DEF 2020-01-14 2020-02-02
 80.87

And, here is the desired/generated dataframe:

  Company  Depart  Return  Charges0 ABC  01/01/2020
02/20/202060.671 HIJ  01/01/2020  03/01/2020    40.002 HIJ
 05/01/2020  05/05/202050.013 DEF  01/12/2020  02/02/2020
212.18

I have been trying to use a combination of sorting and grouping but
the best I've achieved is reordering the dataframe.  Even though I am
able to sort/group based on values, I still run into the issues of
finding overlapping date ranges and pulling out all trips based on a
single company per aggregate/overlapping date range.

Thank you in advance for any help!

Aaron
-- 
https://mail.python.org/mailman/listinfo/python-list

When creating a nested dictionary of dataframes, how can I name a dictionary based on a list name of the dataframe?

[Aaron]

When creating a nested dictionary of dataframes, how can I name a
dictionary based on a list name of the dataframe?

Given the following:

# START CODE
import pandas as pd

cars = {'Brand': ['Honda Civic','Toyota Corolla'],
'Price': [22000,25000]
}
df_cars = pd.DataFrame(cars, columns = ['Brand','Price'])

trucks = {'Brand': ['GMC Sierra','Ford F-150'],
'Price': [5,48000]
}
df_trucks = pd.DataFrame(trucks, columns = ['Brand','Price'])

list_of_dfs = [df_cars, df_trucks]

# Not exactly sure how this code should be:
dict_of_dfs = {}
for df in list_of_dfs:
dict_of_dfs[name_of_df] = {}  # Not sure here
dict_of_dfs[name_of_df]['results'] = df  # Not sure here
# END CODE

I am trying to use a for loop that performs the following:

# START CODE
dict_of_dfs['df_cars'] = {}
dict_of_dfs['df_cars']['results'] = df_cars
dict_of_dfs['df_trucks'] = {}
dict_of_dfs['df_trucks']['results'] = df_trucks
# END CODE

The above code should produce the following desired output:

{
'df_cars': {
'results':
Brand  Price
0 Honda Civic  22000
1 Toyota Corolla  25000},

'df_trucks': {
'results':
 Brand  Price
0  GMC Sierra  5
1  Ford F-150  48000}
}


I've read multiple threads, used the module varname, and tried using
enumerate() but have been unsuccessful.

Thank you!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: When creating a nested dictionary of dataframes, how can I name a dictionary based on a list name of the dataframe?

[Aaron]

Thank you for the help!  Based on your response/recommendation, I am
thinking that my entire approach to solving my problem is very poor.  I am
trying to pull slices from a dataframe, store them in a nested dictionary,
retrieve them, perform calculations, store the results in the same nested
dictionary (hence why I need to know the actual name of each nested
dictionary), retrieve the results, then format the results.  LOL  that
looks so bad when I think about it out loud.  What I think I should be
doing is pull my first slice from a dataframe, perform my calculations,
then format those results in my desired final output; take my second,
third, fourth, etc. slices and then just repeat that same pattern instead
of what I initially thought I should do.  Is there a standard approach for
evaluating particular slices of dataframes or does it always depend on the
situation?

On Sun, Jun 7, 2020 at 4:39 AM Peter Otten <__pete...@web.de> wrote:

> Aaron wrote:
>
> > When creating a nested dictionary of dataframes, how can I name a
> > dictionary based on a list name of the dataframe?
> >
> > Given the following:
> >
> > # START CODE
> > import pandas as pd
> >
> > cars = {'Brand': ['Honda Civic','Toyota Corolla'],
> > 'Price': [22000,25000]
> > }
> > df_cars = pd.DataFrame(cars, columns = ['Brand','Price'])
> >
> > trucks = {'Brand': ['GMC Sierra','Ford F-150'],
> > 'Price': [5,48000]
> > }
> > df_trucks = pd.DataFrame(trucks, columns = ['Brand','Price'])
> >
> > list_of_dfs = [df_cars, df_trucks]
> >
> > # Not exactly sure how this code should be:
> > dict_of_dfs = {}
> > for df in list_of_dfs:
> > dict_of_dfs[name_of_df] = {}  # Not sure here
> > dict_of_dfs[name_of_df]['results'] = df  # Not sure here
> > # END CODE
>
> In the general case you can't find the name of a value, so you may start
> with the names and look up the dataframes in the global namespace:
>
> list_of_df_names = ["df_cars", "df_trucks"]
> dict_of_dfs = {
>name: {"results": globals()[name]}
>for name in list_of_df_names
> }
>
> Personally I would probably reorganize your code a bit and avoid the
> df_...
> names:
>
> import pandas as pd
>
> vehicles = dict(
> cars={
> 'Brand': ['Honda Civic','Toyota Corolla'],
> 'Price': [22000,25000]
> },
> trucks={
> 'Brand': ['GMC Sierra','Ford F-150'],
> 'Price': [5,48000]
> }
> )
>
> dict_of_dfs = {
> name: {"results": pd.DataFrame(val)}
> for name, val in vehicles.items()
> }
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Need to pass Object by value into a list

[Aaron]

I have a data sructure setup and I populate it in a loop like so:

y=0
while X:
   DS.name = "ASDF"
   DS.ID = 1234

   list[y] = DS;
   y = y + 1

print list

This does not work because DS is passed in by reference causing all
entries into the list to change to the most current value.  I cannot
find a "new" function in Python like there is in C++.  How do you do
this in Python?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Telephony project

[aaron]

It will be much easier to use asterisk, there's a win32 version aterisk 
available but it does not support hardware phone, voip only.
A clone FXO card only cost $15 on ebay.

"Roger" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> I'm new to Python and need to do a (low level, I think) telephony
> project ( POTS, all local calling)  in WinXp.
>  1. Fetch phone number from my ASCII data.
>  2. Dial (always a local number) phone (through USRobotics 56K? ).
>  3. Ask @3 questions to called phone number. Y/N   Y/N   Y/N
>  4. Save answers to ASCII file.
>  5. Say 'Thanks', hang up.
>  Repeat till eof()
> And...I've not seen any telephony books in/for Python.  Any and all
> help, suggestions, directions, etc. would be *GREATLY* appreciated.
>
>   TIA
> 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Howto Extract PNG from binary file @ 0x80?

[Aaron]

On Sat, 11 Dec 2004 10:53:19 +0100, Fredrik Lundh wrote:

> "flamesrock" <[EMAIL PROTECTED]> wrote:
> 
>> As a newbie to the language, I have no idea where to start..please bare
>> with me..
>>
>> The simcity 4 savegame file has a png image stored at the hex location
>> 0x80. What I want to extract it and create a file with the .png
>> extension in that directory.
>>
>> Can somebody explain with a snippet of code how I would accomplish
>> this? I've looked around but the information is too vague and I don't
>> know how to start.
> 
> there are *many* ways to ignore the first 128 bytes when you read a file
> (you can seek to the right location, you can read 128 bytes and throw them
> a way, you can read one byte 128 times and throw each one of them away,
> you can read all data and remove the first 128 bytes, etc).
> 
> here's a snippet that understands the structure of the PNG, and stops copying
> when it reaches the end of the PNG:
> 
> import struct
> 
> def pngcopy(infile, outfile):
> 
> # copy header
> header = infile.read(8)
> if header != "\211PNG\r\n\032\n":
> raise IOError("not a valid PNG file")
> outfile.write(header)
> 
> # copy chunks, until IEND
> while 1:
> chunk = infile.read(8)
> size, cid = struct.unpack("!l4s", chunk)
> outfile.write(chunk)
> outfile.write(infile.read(size))
> outfile.write(infile.read(4)) # checksum
> if cid == "IEND":
> break
> 
> to use this, open the input file (the simcity file) and the output file
> (the png file you want to create) in binary mode, use the "seek"
> method to move to the right place in the simcity file, and call "pngcopy"
> with the two file objects.
> 
> infile = open("mysimcityfile", "rb")
> infile.seek(0x80)
> outfile = open("myimage.png", "wb")
> pngcopy(infile, outfile)
> outfile.close()
> infile.close()
> 
> hope this helps!
> 
> 

Thanks! And what a pleasant surprise! I just noticed you authored the book
I've been studying - Python Standard Library. Its awesome ;)

Oh- and Sorry for the late response. Google groups wouldn't allow me to
followup. I'm using pan now.

-- 
http://mail.python.org/mailman/listinfo/python-list

Creating Image Maps

[Aaron]

I know this is a thing used primarily on websites..but since python can do
anything ;)

I'm trying to make a clickable image map for my wxPython program.
Basically, I know how to organize the images into one large image in a
panel, but how do I make the individual pieces clickable like webpage
links(in wxPython)? The goal is to add an event handler that displays the
image piece in a different panel, along with attributes. Which I could do
if only I could make the pieces clickable

Any ideas?

Also, if you have any hard to find links on the general area I'm talking
about, I'd like to learn as much as possible.

-thanks 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Creating Image Maps

[Aaron]

Thanks for the responses guys!

The first option you provided sounds great, Steve. I think I'm gonna try
it that way.



-- 
http://mail.python.org/mailman/listinfo/python-list

Configuration Files

[Aaron]

Hi,

I'm interested in creating a large number of configuration files which I
have no experience doing in python. The fields will be static for the most
part. But design changes and I might want to add new fields in the future..

My question is - whats the best module for creating, reading, and editing
these files(something like an INI)? I want it to be powerful,
yet simple and minimalistic, not requiring a huge amount of overhead.

I've heard of xml, and have seen it used in both html fashion, and
likewise, without any tags at all- just spaces in between entries; why is
this? CSV is another thing I've seen thrown around on online
documentation. Whats the difference between the two, and which one should
I use(if either)?

-thanks in advance
-- 
http://mail.python.org/mailman/listinfo/python-list

regular expression

[aaron]

dear readers,
given a string, suppose i wanted to do the following:
- replace all periods with colons, except for periods with a digit to 
the right and left of it.

for example, given:
'375 mi. south of U.C.B. is 3.4 degrees warmer'
would be changed to:
"375 mi: south of U:C:B: is 3.4 degrees warmer'
i was thinking that a regular expression might do the trick. here's what 
i tried:
!--!
Python 2.4.1c1
[GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-49)] on linux2
>>> import re
>>> pattern = re.compile(r'(?!\d)[.](?!\d)')
>>> pattern.sub(':', '375 mi. south of U.C.B is 3.4 degrees warmer.')
'375 mi: south of U:C:B is 3.4 degrees warmer:'
!--!

so this works, but not in the following case:
!--!
>>> pattern.sub(':', '.3')
'.3'
!--!
but going the other direction works:
!--!
>>> pattern.sub(':', '3.')
'3:'
!--!
any thoughts?
thanks,
aaron
--
http://mail.python.org/mailman/listinfo/python-list

In Python 3, how to append a nested dictionary to a shelve file with the added difficulty of using a for loop?

[Aaron]

Hello,

I am trying to figure out how to populate a shelve file with a nested 
dictionary.

These are my requirements:

-Create shelve file called people.db
-Append the shelve file with new people (person_1, person_2, etc.).
-Use a for loop to iterate through 'attributes' so that I do not need to write 
out the lengthy code line by line to populate to the shelve file.
-Need to reference shelve file data for future use

Here is the key/value format that I would like to append to the shelve file.

person_1 = { 'name': 'Bob', 'type': 'employee', 'attributes':
[{'game': 'basketball', 'high score': '100', 'time': '3.34'},
{'game': 'bridge', 'high score': '10', 'time': '30.34'},
{'game': 'foosball', 'high score': '2', 'time': '24'}]
'''
50+ other attributes
'''
}

# Example: s['person_1]['attributes'][2]['time'] would call out '24'.
# 's' is from 's = shelve.open('people')'
I have a dictionary dictPeople.py file (created using pprint() that contains 
the information of person_1, etc. And I am extracting only a small percentage 
of the data that is needed.

I have tried the following, but I get an invalid key error.

import shelve, dictPeople
s = shelve.open('people')
person = 'person_1'
s[person]['name'] = dictPeople.person_1['name']
s[person]['type'] = dictPeople.person_1['type']
# I need to use this for loop because there are 50+ attributes.
for attribute in range(0, len(dictPeople['attributes'][attribute])):
s[person]['attributes'][attribute]['game'] = \
dictPeople['attributes'][attribute]['game']
s[person]['attributes'][attribute]['high score'] = \
dictPeople['attributes'][attribute]['high score']
s[person]['attributes'][attribute]['time'] = \
dictPeople['attributes'][attribute]['time']
It turns out, I get the key error because I am not allowed to reference a 
key/value pair the same way that I can with a dictionary. However, the crazy 
thing is that I can call values in the db file using the same exact format when 
trying to write.

For example: I can read data from the .db file using:

x = s['person_1']['name']
print(x)
BUT! I cannot write to that .db file using that exact format or I get an 
invalid key error. Makes no sense!

s['person_1']['name'] = 'Bob'
# Returns invalid key entry.  Makes no sense.
Therefore, I tried to extract data and populate the db file using the following:

s[person] = {   'name': dictPeople.person_1['name'],
'type': dictPeople.person_1['type'],
for attribute in range(0, len(dictPeople['attributes'][attribute])):
['game':  dictPeople['attributes'][attribute]['game'],
'high score': dictPeople['attributes'][attribute]['high score'],
'time': dictPeople['attributes'][attribute]['time']]

}
But, this obvously doesn't work because of the for loop. How can I do this?

-I am trying to figure out how to extract data from the dictionary 
dictPeople.py file and store it in the people.db file.
-I am trying to add new people to the people.db file as more people become 
available.
-I need to use the for loop because of the 50+ attributes I need to add.
-My future steps would be to modify some of the data values that I extract and 
used to populate the people.db file, but my first step is to, at least, 
extract, and then populate the people.db file.

Any help or guidance is greatly appreciated.  Thank you for your time and 
reading my question.

Thanks!
Aaron
-- 
https://mail.python.org/mailman/listinfo/python-list

yet another indentation proposal

[Aaron]

Hello all.

I realize that proposals dealing with alternatives to indentation  have been 
brought up (and shot down) before, but I would like to take another stab at 
it, because it is rather important to me.

I am totally blind, and somewhat new to Python.  I put off learning Python 
for a long time, simply because of the indentation issue.  There is no easy 
way for a screenreader user, such as my self, to figure out how much a 
particular line of code is indented.  A sited person simply looks down the 
column to make sure that everything lines up.  I, on the other hand, 
generally find my self counting a lot of spaces.

Naturally, there are many other special circumstances in which depending on 
whitespace is impractical, but they have already been mentioned in previous 
threads on the topic, so I won't bother rehashing them.

Borrowing from another proposal, I would submit that the opening block and 
closing block markers could be _{ and }_ respectively.  If it seems that 
there is even a modicum of support, I'll write up a more complete proposal. 
But in short, an _{ will signify to the interpreter that one level of 
indentation is to be added to the current level, that all the statements 
that follow (until the closing _}) are to be understood to be at that level, 
and that all whitespace at the beginning of lines, or lack there of, is to 
be ignored.

I realize that this would result in some esthetically ugly code, and so 
naturally a simple tool to convert this type of block indication back to 
"normal indented format" should also be developed.

Any thoughts or suggestions?

I sincerely hope that this proposal can be given some serious consideration, 
and not simply dismissed as sacrilege.  There are many blind programmers, 
and I would hate to see them opt to learn other languages simply because of 
this one issue.

Thanks.

Aaron


-- 
To reply directly, remove j's from email address. 



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: yet another indentation proposal

[Aaron]

"Paddy" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
>
> Oh wow. it never crossed my mind...
>
> Can screen reaaderss be customized?
> Maybe their is a way to get the screen reader to say indent and dedent
> at thee appropriate places?
> Or maybe a filter to put those wordds into the source?
>
> - Paddy.
>


Interestingly enough, there is a feature in my screenreader that purports to 
do just that, but it has proven unreliable at best.  I think a filter to 
insert an indent indicator at the beginning of each line would not be too 
practical, as it would make copy and paste operations from one indentation 
level to another rather tedious (you'd have to add or remove one or more 
indicators from each line of code).

Finally, just to be clear, I do not want to change the way 99.9% of Python 
code is written.  I feel that the indentation model is a good one for 99.9% 
of users.  What I do want to do is simply give the Python interpreter a tiny 
bit more flexibility to handle code from users or environments where 
indentation is not possible or practical.

Aaron


-- 
To reply directly, remove j's from email address. 



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: yet another indentation proposal

[Aaron]

"Michael Tobis" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
>
> Alternatively, you might also consider writing a simple pre and
> postprocessor so that you could read and write python the way you
> would prefer
>
> In that you could cope with the existing code base and the existing
> compiler could cope with code you write.
>
> mt
>


Hi.

That's probably what I'll end up doing.  The only drawback to that is that 
it solves the problem for me only.  Perhaps I will open source the scripts 
and write up some documentation so that other folks in a similar situation 
don't have to reinvent the wheel.  The only unfortunate aspect to that is 
that blind newbies to the language will have to figure out setting up a 
shell script or batch file to pipe the output of the filter into Python on 
top of learning the language.  I admit, it's probably not that much work, 
but it is one more stumblingblock that blind newcomers will have to 
overcome.

Aaron 



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: I Need help from all the group participants

[Aaron]

"Boris Ozegovic" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> Hi,
>
> I am working on some system, and the communication will take place through
> the chatterbot which will be written in AIML (interpreter is written in
> Python).  English is not my mother tongue, so I need huge favor:  if all 
> of
> you can write two sentences, so that I could have greater pattern base.
> The first sentence is finding someone's heart rate, example: "Please, can
> you tell me John's heart rate", and the other one is setting the alarm 
> when
> someone's heart rate is lower then x, and greater than y, example: "Can
> you, please, send me information when John's heart rate is lower than 60,
> and greater than 100".  You can manipulate time (right now, now, right 
> this
> moment), and everything else you can think about.
>


Give me John's heart rate.
John's heart rate?
John's pulse?
John's pulse is?
(The last three are not exactly complete sentences.  Rather they are 
fragments, but who cares when you're talking to a computer, right?)

Tell me when John's heart rate drops below x or goes above y.
Tell could be replaced with notify me, alert me, call me, or any other verbs 
that describe how the computer might contact the person.
Drops below could be replaced with "goes below", "is less than" "slows to" 
"slows down to" "drops to" "decreases to" "goes down to" ETC.
"goes above" could be "exceeds" "passes" "goes past" "increases to" "speeds 
up to" "goes up to" "reaches" "exceeds" "surpasses" "is faster than" "goes 
faster than" "is above" ETC.

Good luck.  I'm glad this isn't my project.

Aaron

-- 
To reply directly, remove j's from email address. 



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pycon disappointment

[Aaron]

> In my opinion, open spaces should have had greater status and billing,
> with eyes-forward talks and vendor sessions offered only as possible
> alternatives. Especially, vendor sessions should not be presented as
> "keynotes"  during plenary sessions. I think it took a little while
> for people to catch on to the idea that they could have control of
> their own experience through the open spaces and that the main
> offerings were not the only option.

This is an excellent suggestion and observation.  Sold sponsorships
are fine as long as they are billed as such.  Labels on the vendor
speeches indicated they were sold as ad space would be great, as well
as more strongly emphasizing the ad hoc discussion spaces.
-- 
http://mail.python.org/mailman/listinfo/python-list

set and dict iteration

[Aaron Brady]

Hello,

I observed an inconsistency in the behavior of 'set' and 'dict' iterators.  It 
is "by design" according to the docs.

'''
http://docs.python.org/dev/library/stdtypes.html#dict-views

iter(dictview).  Iterating views while adding or deleting entries in the 
dictionary may raise a RuntimeError or fail to iterate over all entries.
'''

The 'set' has the same behavior.  Iteration might also complete successfully.

The inconsistency is, if we remove an element from a set and add another during 
iteration, the new element might appear later in the iteration, and might not, 
depending on the hash code; therefore comparing the size of the set between 
iterations isn't adequate.  Example:

http://home.comcast.net/~castironpi-misc/clpy-0062%20set%20iterators.py  

'''
# py: { 'ver': '3' }

set0= set( ( 1, 2 ) )

iter0= iter( set0 )
print( next( iter0 ) )

set0.add( 3 )
set0.remove( 2 )
print( next( iter0 ) )


print( )

set0= set( ( 6, 7 ) )

iter0= iter( set0 )
print( next( iter0 ) )

set0.add( 8 )
set0.remove( 7 )
print( next( iter0 ) )
'''

Output:

'''
1
3

6
Traceback (most recent call last):
  File [...] line 22, in 
print( next( iter0 ) )
StopIteration
'''

Iteration should behave the same regardless of the contents of the set.  
Continuing iteration over sets and dicts after a modification isn't defined; it 
should unconditionally raise an error.

What's going on, is '8' is added before the position of the iterator due to 
hashing in the second part, but the size doesn't change, so the iterator 
reaches the end of the set after '7' is removed.

The inconsistency isn't easily solved.  One possibility is to use a timestamp 
or other serial index in the object and iterators, and compare them on every 
iteration to determine if a modification has occurred.

Another possibility which the author prefers, is to maintain a secondary 
collection of the iterators of an object, and invalidate them upon 
modification.  The applicable collection structure is a doubly-linked linked 
list, informally depicted:

http://home.comcast.net/~castironpi-misc/clpy-0062%20set%20iterators.png

Upon modification, the set traverses its iterators, setting an 'invalid' flag 
on each; and subsequent calls to any of them raise an 'IterationError'.  Adding 
and removing iterators to and from the secondary list is performed in O( 1 ) 
time with no penalty.

The above example depicted a 'Set'.  'Dicts' have the same anomaly, but the 
solution is ambiguous, since dict values can be changed meaningfully without 
altering the structure of the object.  In the author's opinion, the dict should 
not raise an 'IterationError' on value changes, only key changes like the set, 
but the argument isn't conclusive.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: set and dict iteration

[Aaron Brady]

On Thursday, August 16, 2012 6:07:40 PM UTC-5, Ian wrote:
> On Thu, Aug 16, 2012 at 4:55 PM, Ian Kelly  wrote:
> 
> > On Thu, Aug 16, 2012 at 12:00 PM, Aaron Brady  wrote:
> 
> >> The inconsistency is, if we remove an element from a set and add another 
> >> during iteration, the new element might appear later in the iteration, and 
> >> might not, depending on the hash code; therefore comparing the size of the 
> >> set between iterations isn't adequate.  Example:
> 
> >
> 
> > It can be more than just the new element.  For example, here the
> 
> > entire set is repeated (Python 3.2):
> 
> >
> 
> >>>> s = set(range(8, 13))
> 
> >>>> it = iter(s)
> 
> >>>> from itertools import islice
> 
> >>>> list(islice(it, 5))  # avoid exhausting the iterator
> 
> > [8, 9, 10, 11, 12]
> 
> >>>> s.add(13)
> 
> >>>> s.remove(13)
> 
> >>>> list(it)
> 
> > [8, 9, 10, 11, 12]
> 
> >
> 
> > This occurs because the addition of the sixth item triggers a resize
> 
> > of the underlying hash table, and the existing items, which were
> 
> > originally in slots 0-4, are now in slots 8-12.
> 
> 
> 
> Another curious example:
> 
> 
> 
> >>> s = set(range(8, 48, 8))
> 
> >>> s
> 
> {8, 16, 40, 24, 32}
> 
> >>> it = iter(s)
> 
> >>> from itertools import islice
> 
> >>> list(islice(it, 4))
> 
> [8, 16, 40, 24]
> 
> >>> s.add(48)
> 
> >>> s.remove(48)
> 
> >>> list(it)
> 
> [8, 16, 40, 24]
> 
> 
> 
> Hey, what happened to 32?

Good examples.  The former occurs without the 'islice' as well.

s= set( range( 8, 13 ) ) 
it= iter( s ) 
print( [ next( it ) for _ in range( 5 ) ] )
s.add( 13 ) 
s.remove( 13 ) 
print( [ next( it ) for _ in range( 5 ) ] )

Output:

[8, 9, 10, 11, 12]
[8, 9, 10, 11, 12]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: set and dict iteration

[Aaron Brady]

On Thursday, August 16, 2012 8:01:39 PM UTC-5, Paul Rubin wrote:
> Ian Kelly  writes:
> 
> > With regard to key insertion and deletion while iterating over a dict
> 
> > or set, though, there is just no good reason to be doing that
> 
> > (especially as the result is very implementation-specific), and I
> 
> > wouldn't mind a more complete low-level check against it as long as
> 
> > it's not too expensive (which is not clearly the case with the current
> 
> > suggestion at all).
> 
> 
> 
> One possible approach is to freeze the dictionary against modification
> 
> while any iterator is open on it.  You could keep a count of active
> 
> iterators in the dict structure, adjusting it whenever an iterator is
> 
> created or closed/destroyed.

Good point.  Your approach is another consistent solution.

The difference is in where the exception is raised.  Invalidating the iterators 
raises an exception when they're called.  Locking the set/dict raises an 
exception on 'add' and 'remove' calls.

The latter also forces additional statements to delete iterators before they 
leave scope in some cases.  We wouldn't be able to make modifications in a 
'for' suite, even if followed by a 'break', which could be a problem for 
existing code.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: set and dict iteration

[Aaron Brady]

On Thursday, August 16, 2012 9:30:42 PM UTC-5, Paul Rubin wrote:
> Steven D'Aprano  writes:
> 
> > Luckily, Python is open source. If anyone thinks that sets and dicts 
> 
> > should include more code protecting against mutation-during-iteration, 
> 
> > they are more than welcome to come up with a patch. Don't forget unit and 
> 
> > regression tests, and also a set of timing results which show that the 
> 
> > slow-down isn't excessive.
> 
> 
> 
> It could be a debugging option, in which case even a fairly significant
> 
> slowdown is acceptable.

Another possibility is to use the 'gc.get_referrers' mechanism to obtain the 
iterators.


import gc
a= set( ( 0, 1, 2 ) )
b= iter( a )
c= iter( a )
d= iter( a )
print( gc.get_referrers( a ) )

Output:

[, , 
, [others] ]

This approach wouldn't be as time-efficient as a dedicated secondary structure, 
due to the other objects which refer to the set, including variable namespaces.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: set and dict iteration

[Aaron Brady]

On Thursday, August 16, 2012 9:24:44 PM UTC-5, Steven D'Aprano wrote:
> On Thu, 16 Aug 2012 19:11:19 -0400, Dave Angel wrote:
> 
> 
> 
> > On 08/16/2012 05:26 PM, Paul Rubin wrote:
> 
> >> Dave Angel  writes:
> 
> >>> Everything else is implementation defined.  Why should an
> 
> >>> implementation be forced to have ANY extra data structure to detect a
> 
> >>> static bug in the caller's code?
> 
> >> For the same reason the interpreter checks for type errors at runtime
> 
> >> and raises TypeError, instead of letting the program go into the weeds.
> 
> > 
> 
> > There's an enormous difference between type errors, which affect the low
> 
> > level dispatch, and checking for whether a dict has changed and may have
> 
> > invalidated the iterator.  If we were really going to keep track of what
> 
> > iterators are tracking a given dict or set, why stop there?  Why not
> 
> > check if another process has changed a file we're iterating through?  Or
> 
> > ...
> 
> 
> 
> Which is why Python doesn't do it -- because it is (claimed to be) 
> 
> excessively expensive for the benefit that you would get.
> 
> 
> 
> Not because it is a matter of principle that data integrity is 
> 
> unimportant. Data integrity *is* important, but in the opinion of the 
> 
> people who wrote these particular data structures, the effort required to 
> 
> guarantee correct iteration in the face of mutation is too expensive for 
> 
> the benefit.
> 
> 
> 
> Are they right? I don't know. I know that the list sort method goes to a 
> 
> lot of trouble to prevent code from modifying lists while they are being 
> 
> sorted. During the sort, the list temporarily appears to be empty to 
> 
> anything which attempts to access it. So at least sometimes, the Python 
> 
> developers spend effort to ensure data integrity.
> 
> 
> 
> Luckily, Python is open source. If anyone thinks that sets and dicts 
> 
> should include more code protecting against mutation-during-iteration, 
> 
> they are more than welcome to come up with a patch. Don't forget unit and 
> 
> regression tests, and also a set of timing results which show that the 
> 
> slow-down isn't excessive.

I contribute a patch some time ago.  It wasn't accepted.  However this thread 
seems to show a moderately more favorable sentiment than that one.

Is there a problem with hacking on the Beta?  Or should I wait for the Release? 
 Does anyone want to help me with the changes?  Perhaps P. Rubin could 
contribute the variation he suggested as well.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: set and dict iteration

[Aaron Brady]

On Friday, August 17, 2012 4:57:41 PM UTC-5, Chris Angelico wrote:
> On Sat, Aug 18, 2012 at 4:37 AM, Aaron Brady  wrote:
> 
> > Is there a problem with hacking on the Beta?
> 
> 
> 
> Nope. Hack on the beta, then when the release arrives, rebase your
> 
> work onto it. I doubt that anything of this nature will be changed
> 
> between now and then.
> 
> 
> 
> ChrisA

Thanks Chris, your post was encouraging.

I have a question about involving the 'tp_clear' field of the types.

http://docs.python.org/dev/c-api/typeobj.html#PyTypeObject.tp_clear

'''
...The tuple type does not implement a tp_clear function, because it’s possible 
to prove that no reference cycle can be composed entirely of tuples. 
'''

I didn't follow the reasoning in the proof; the premise is necessary but IMHO 
not obviously sufficient.  Nevertheless, the earlier diagram contains an overt 
homogeneous reference cycle.

Reposting: 
http://home.comcast.net/~castironpi-misc/clpy-0062%20set%20iterators.png

In my estimate, the 'tp_traverse' and 'tp_clear' fields of the set doesn't need 
to visit the auxiliary collection; the same fields of the iterators don't need 
to visit the primary set or other iterators; and references in the linked list 
don't need to be included in the iterators' reference counts.

Can someone who is more familiar with the cycle detector and cycle breaker, 
help prove or disprove the above?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: set and dict iteration

[Aaron Brady]

On Saturday, August 18, 2012 5:14:05 PM UTC-5, MRAB wrote:
> On 18/08/2012 21:29, Aaron Brady wrote:
> 
> > On Friday, August 17, 2012 4:57:41 PM UTC-5, Chris Angelico wrote:
> 
> >> On Sat, Aug 18, 2012 at 4:37 AM, Aaron Brady  wrote:
> 
> >>
> 
> >> > Is there a problem with hacking on the Beta?
> 
> >>
> 
> >>
> 
> >>
> 
> >> Nope. Hack on the beta, then when the release arrives, rebase your
> 
> >>
> 
> >> work onto it. I doubt that anything of this nature will be changed
> 
> >>
> 
> >> between now and then.
> 
> >>
> 
> >>
> 
> >>
> 
> >> ChrisA
> 
> >
> 
> > Thanks Chris, your post was encouraging.
> 
> >
> 
> > I have a question about involving the 'tp_clear' field of the types.
> 
> >
> 
> > http://docs.python.org/dev/c-api/typeobj.html#PyTypeObject.tp_clear
> 
> >
> 
> > '''
> 
> > ...The tuple type does not implement a tp_clear function, because it’s 
> > possible to prove that no reference cycle can be composed entirely of 
> > tuples.
> 
> > '''
> 
> >
> 
> > I didn't follow the reasoning in the proof; the premise is necessary but 
> > IMHO not obviously sufficient.  Nevertheless, the earlier diagram contains 
> > an overt homogeneous reference cycle.
> 
> >
> 
> > Reposting: 
> > http://home.comcast.net/~castironpi-misc/clpy-0062%20set%20iterators.png
> 
> >
> 
> > In my estimate, the 'tp_traverse' and 'tp_clear' fields of the set doesn't 
> > need to visit the auxiliary collection; the same fields of the iterators 
> > don't need to visit the primary set or other iterators; and references in 
> > the linked list don't need to be included in the iterators' reference 
> > counts.
> 
> >
> 
> > Can someone who is more familiar with the cycle detector and cycle breaker, 
> > help prove or disprove the above?
> 
> >
> 
> In simple terms, when you create an immutable object it can contain
> 
> only references to pre-existing objects, but in order to create a cycle
> 
> you need to make an object refer to another which is created later, so
> 
> it's not possible to create a cycle out of immutable objects.
> 
> 
> 
> However, using Python's C API it _is_ possible to create such a cycle,
> 
> by mutating an otherwise-immutable tuple (see PyTuple_SetItem and
> 
> PyTuple_SET_ITEM).

Are there any precedents for storing uncounted references to PyObject's?

One apparent problematic case is creating an iterator to a set, then adding it 
to the set.  However the operation is a modification, and causes the iterator 
to be removed from the secondary list before the set is examined for collection.

Otherwise, the iterator keeps a counted reference to the set, but the set does 
not keep a counted reference to the iterator, so the iterator will always be 
freed first.  Therefore, the set's secondary list will be empty when the set is 
freed.

Concurrent addition and deletion of iterators should be disabled, and the 
iterators should remove themselves from the set's secondary list before they 
decrement their references to the set.

Please refresh the earlier diagram; counted references are distinguished 
separately.  Reposting: 
http://home.comcast.net/~castironpi-misc/clpy-0062%20set%20iterators.png
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: set and dict iteration

[Aaron Brady]

On Saturday, August 18, 2012 9:28:32 PM UTC-5, Aaron Brady wrote:
> On Saturday, August 18, 2012 5:14:05 PM UTC-5, MRAB wrote:
> 
> > On 18/08/2012 21:29, Aaron Brady wrote:
> 
> > > On Friday, August 17, 2012 4:57:41 PM UTC-5, Chris Angelico wrote:
> 
> > >> On Sat, Aug 18, 2012 at 4:37 AM, Aaron Brady  
> > >> wrote:
> 
> > >> > Is there a problem with hacking on the Beta?
> 
> > >> Nope. Hack on the beta, then when the release arrives, rebase your
> 
> > >> work onto it. I doubt that anything of this nature will be changed
> 
> > >> between now and then.
> 
> > >> ChrisA
> 
> > > Thanks Chris, your post was encouraging.
> 
> > > I have a question about involving the 'tp_clear' field of the types.
> 
> > > http://docs.python.org/dev/c-api/typeobj.html#PyTypeObject.tp_clear
> 
> > > '''
> 
> > > ...The tuple type does not implement a tp_clear function, because it’s 
> > > possible to prove that no reference cycle can be composed entirely of 
> > > tuples.
> 
> > > '''
> 
> > > I didn't follow the reasoning in the proof; the premise is necessary but 
> > > IMHO not obviously sufficient.  Nevertheless, the earlier diagram 
> > > contains an overt homogeneous reference cycle.
> 
> > > Reposting: 
> > > http://home.comcast.net/~castironpi-misc/clpy-0062%20set%20iterators.png
> 
> > > In my estimate, the 'tp_traverse' and 'tp_clear' fields of the set 
> > > doesn't need to visit the auxiliary collection; the same fields of the 
> > > iterators don't need to visit the primary set or other iterators; and 
> > > references in the linked list don't need to be included in the iterators' 
> > > reference counts.
> 
> > > Can someone who is more familiar with the cycle detector and cycle 
> > > breaker, help prove or disprove the above?
> 
> > In simple terms, when you create an immutable object it can contain
> 
> > only references to pre-existing objects, but in order to create a cycle
> 
> > you need to make an object refer to another which is created later, so
> 
> > it's not possible to create a cycle out of immutable objects.
> 
> > However, using Python's C API it _is_ possible to create such a cycle,
> 
> > by mutating an otherwise-immutable tuple (see PyTuple_SetItem and
> 
> > PyTuple_SET_ITEM).
> 
> Are there any precedents for storing uncounted references to PyObject's?
> 
> One apparent problematic case is creating an iterator to a set, then adding 
> it to the set.  However the operation is a modification, and causes the 
> iterator to be removed from the secondary list before the set is examined for 
> collection.
> 
> Otherwise, the iterator keeps a counted reference to the set, but the set 
> does not keep a counted reference to the iterator, so the iterator will 
> always be freed first.  Therefore, the set's secondary list will be empty 
> when the set is freed.
> 
> Concurrent addition and deletion of iterators should be disabled, and the 
> iterators should remove themselves from the set's secondary list before they 
> decrement their references to the set.
> 
> Please refresh the earlier diagram; counted references are distinguished 
> separately.  Reposting: 
> http://home.comcast.net/~castironpi-misc/clpy-0062%20set%20iterators.png

The patch for the above is only 40-60 lines.  However it introduces two new 
concepts.

The first is a "linked list", a classic dynamic data structure, first developed 
in 1955, cf. http://en.wikipedia.org/wiki/Linked_list .  Linked lists are 
absent in Python, including the standard library and CPython implementation, 
beyond the weak reference mechanism and garbage collector.  The 
"collections.deque" structure shares some of the linked list interface but uses 
arrays.

The second is "uncounted references".  The uncounted references are references 
to "set iterators" exclusively, exist only internally to "set" objects, and are 
invisible to the rest of the program.  The reason for the exception is that 
iterators are unique in the Python Data Model; iterators consist of a single 
immutable reference, unlike both immutable types such as strings and numbers, 
as well as container types.  Counted references could be used instead, but 
would be consistently wasted work for the garbage collector, though the benefit 
to programmers' peace of mind could be significant.

Please share your opinion!  Do you agree that the internal list resolves the 
inconsistency?  Do you agree with the strategy?  Do you agree that uncounted 
references are justified to introduce, or are counted references preferable?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: set and dict iteration

[Aaron Brady]

On Thursday, August 23, 2012 1:11:14 PM UTC-5, Steven D'Aprano wrote:
> On Thu, 23 Aug 2012 09:49:41 -0700, Aaron Brady wrote:
> 
> 
> 
> [...]
> 
> > The patch for the above is only 40-60 lines.  However it introduces two
> 
> > new concepts.
> 
> > 
> 
> > The first is a "linked list", a classic dynamic data structure, first
> 
> > developed in 1955, cf. http://en.wikipedia.org/wiki/Linked_list . 
> 
> > Linked lists are absent in Python
> 
> 
> 
> They certainly are not. There's merely no named "linked list" class.
> 
> 
> 
> Linked lists are used by collections.ChainMap, tracebacks, xml.dom, 
> 
> Abstract Syntax Trees, and probably many other places. (Well, technically 
> 
> some of these are trees rather than lists.) You can trivially create a 
> 
> linked list:
> 
> 
> 
> x = [a, [b, [c, [d, [e, None]
> 
> 
> 
> is equivalent to a singly-linked list with five nodes. Only less 
> 
> efficient.
> 
> 
> 
> 
> 
> > The second is "uncounted references".  The uncounted references are
> 
> > references to "set iterators" exclusively, exist only internally to
> 
> > "set" objects, and are invisible to the rest of the program.  The reason
> 
> > for the exception is that iterators are unique in the Python Data Model;
> 
> > iterators consist of a single immutable reference, unlike both immutable
> 
> > types such as strings and numbers, as well as container types.  Counted
> 
> > references could be used instead, but would be consistently wasted work
> 
> > for the garbage collector, though the benefit to programmers' peace of
> 
> > mind could be significant.
> 
> 
> 
> The usual way to implement "uncounted references" is by using weakrefs. 
> 
> Why invent yet another form of weakref?
> 
> 
> 
> 
> 
> 
> 
> -- 
> 
> Steven


Hello S. D'Aprano.  Thanks for your support as always.

The semantics of the second collection are equivalent to a WeakSet.  The space 
and time consumption of a WeakSet are higher in comparison to a linked list.  
However, so long as we iterated over it using the C API instead of creating an 
iterator, it would be consistent.  If we dynamically create the WeakSet on 
demand and free it when empty, the space consumption would be lower.  Typical 
use cases don't involve creating thousands of iterators, or rapidly creating 
and destroying them, so the performance impact might not be severe.

Regarding the bare weakrefs, if the iterator's destructor hasn't been called, 
then the pointer is still valid.  If it has been called, then it's not present 
in the list.  Unlike Python classes, the destructors of C extension classes are 
guaranteed to be called.  Therefore there are no points during exection at 
which a node needs to check whether a reference to its neighbor is valid.

Are your concerns based on data integrity, future maintainability, or what?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Looking for an IPC solution

[Aaron Brady]

On Friday, August 31, 2012 2:22:00 PM UTC-5, Laszlo Nagy wrote:
> There are just so many IPC modules out there. I'm looking for a solution 
> 
> for developing a new a multi-tier application. The core application will 
> 
> be running on a single computer, so the IPC should be using shared 
> 
> memory (or mmap) and have very short response times. But there will be a 
> 
> tier that will hold application state for clients, and there will be 
> 
> lots of clients. So that tier needs to go to different computers. E.g. 
> 
> the same IPC should also be accessed over TCP/IP. Most messages will be 
> 
> simple data structures, nothing complicated. The ability to run on PyPy 
> 
> would, and also to run on both Windows and Linux would be a plus.
> 
> 
> 
> I have seen a stand alone cross platform IPC server before that could 
> 
> serve "channels", and send/receive messages using these channels. But I 
> 
> don't remember its name and now I cannot find it. Can somebody please help?
> 
> 
> 
> Thanks,
> 
> 
> 
> Laszlo

Hi Laszlo,

There aren't a lot of ways to create a Python object in an "mmap" buffer.  
"mmap" is conducive to arrays of arrays.  For variable-length structures like 
strings and lists, you need "dynamic allocation".  The C functions "malloc" and 
"free" allocate memory space, and file creation and deletion routines operate 
on disk space.  However "malloc" doesn't allow you to allocate memory space 
within memory that's already allocated.  Operating systems don't provide that 
capability, and doing it yourself amounts to creating your own file system.  If 
you did, you still might not be able to use existing libraries like the STL or 
Python, because one address might refer to different locations in different 
processes.

One solution is to keep a linked list of free blocks within your "mmap" buffer. 
 It is prone to slow access times and segment fragmentation.  Another solution 
is to create many small files with fixed-length names.  The minimum file size 
on your system might become prohibitive depending on your constraints, since a 
4-byte integer could occupy 4096 bytes on disk or more.  Or you can serialize 
the arguments and return values of your functions, and make requests to a 
central process.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: set and dict iteration

[Aaron Brady]

On Monday, August 27, 2012 2:17:45 PM UTC-5, Ian wrote:
> On Thu, Aug 23, 2012 at 10:49 AM, Aaron Brady  wrote:
> 
> > The patch for the above is only 40-60 lines.  However it introduces two new 
> > concepts.
> 
> 
> 
> Is there a link to the patch?

Please see below.  It grew somewhat during development.

> > The first is a "linked list".
SNIP.
 
> > The second is "uncounted references".  The uncounted references are 
> > references to "set iterators" exclusively, exist only internally to "set" 
> > objects, and are invisible to the rest of the program.  The reason for the 
> > exception is that iterators are unique in the Python Data Model; iterators 
> > consist of a single immutable reference, unlike both immutable types such 
> > as strings and numbers, as well as container types.  Counted references 
> > could be used instead, but would be consistently wasted work for the 
> > garbage collector, though the benefit to programmers' peace of mind could 
> > be significant.
> 
> >
> 
> > Please share your opinion!  Do you agree that the internal list resolves 
> > the inconsistency?  Do you agree with the strategy?  Do you agree that 
> > uncounted references are justified to introduce, or are counted references 
> > preferable?
> 
> 
> 
> This feature is a hard sell as it is; I think that adding uncounted
> 
> references into the mix is only going to make that worse.  May I
> 
> suggest an alternate approach?  Internally tag each set or dict with a
> 
> "version", which is just a C int.  Every time the hash table is
> 
> modified, increment the version.  When an iterator is created, store
> 
> the current version on the iterator.  When the iterator is advanced,
> 
> check that the iterator version matches the dict/set version.  If
> 
> they're not equal, raise an error.
> 
> 
> 
> This should add less overhead than the linked list without any
> 
> concerns about reference counting.  It does introduce a small bug in
> 
> that an error condition could be "missed", if the version is
> 
> incremented a multiple of 2**32 or 2**64 times between iterations --
> 
> but how often is that really likely to occur?  Bearing in mind that
> 
> this error is meant for debugging and not production error handling,
> 
> you could even make the version a single byte and I'd still be fine
> 
> with that.
> 
> 
> 
> Cheers,
> 
> Ian


Hi Ian,

We could use a Python long object for the version index to prevent overflow.  
Combined with P. Rubin's idea to count the number of open iterators, most use 
cases still wouldn't exceed a single word comparison; we could reset the 
counter when there weren't any.  Using the linked list collection, modification 
operations are expensive in rare cases.  Using the version index, iteration is 
expensive in rare cases.

I was more interested in the linked list for conceptual reasons, so I developed 
it further.  Changelog, diff file, test suite, and links are below.  The devs 
should be aware that a competing patch might be developed.  I would be pleased 
to hear what everybody thinks of it!

Linked list with uncounted references implementation for Win32.

Added:
- 'set_clear_ex' and 'set_clear_internal_ex' methods, differ in invalidation 
and conditional invalidation behavior and return type..  The 'set.clear()' 
method and 'tp_clear' type field both called the same method.
- 'set_invalidate_iter_linked' method.  Iterate over the iterators of a set, 
mark them invalid, and clear the list.
- 'setiter_unlink_internal' method.  Remove the iterator from the set's linked 
list of iterators.
- 'IterationError', global.
- New fields:
-- PySetObject: setiterobject *iter_linked.  Pointer to the first element of 
the linked list of the iterators of the set.
-- setiterobject: setiterobject *linked_pred, *linked_succ.  Predecessor and 
successor nodes in the linked list of iterators of the same set.
-- setiterobject: char ob_valid.  Validation status of the iterator.
- Result is compared with original in 'set_intersection_update' and '_multi' to 
determine whether to invalidate the list of iterators.  Asymptotic running time 
is unchanged.
- Pending: add 'tp_clear' field to 'PySetIter_Type'?
- Test script included, 'large numbers' test pending.

6 files changed: { setobject.h, setobject.c, exceptions.c, pyerrors.h, 
python3.def, python33stub.def }.  Test script 'set_iterator_test.py' new.  
Linked list interface and pseudocode 'patch_pseudocode.txt'.
Zip file:
http://home.comcast.net/~castironpi-misc/clpy-0062-set_iterator_patch.zip
Diff file of 3.3.0b2:
http://home.comcast.net/~castironpi-misc/clpy-0062-set_iterator_diff.txt

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: set and dict iteration

[Aaron Brady]

On Monday, September 3, 2012 2:30:24 PM UTC-5, Ian wrote:
> On Sun, Sep 2, 2012 at 11:43 AM, Aaron Brady  wrote:
> 
> > We could use a Python long object for the version index to prevent 
> > overflow.  Combined with P. Rubin's idea to count the number of open 
> > iterators, most use cases still wouldn't exceed a single word comparison; 
> > we could reset the counter when there weren't any.
> 
> 
> 
> We could use a Python long; I just don't think the extra overhead is
> 
> justified in a data structure that is already highly optimized for
> 
> speed.  Incrementing and testing a C int is *much* faster than doing
> 
> the same with a Python long.

I think the technique would require two python longs and a bool in the set, and 
a python long in the iterator.

One long counts the number of existing (open) iterators.  Another counts the 
version.  The bool keeps track of whether an iterator has been created since 
the last modification, in which case the next modification requires 
incrementing the version and resetting the flag.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: set and dict iteration

[Aaron Brady]

On Monday, September 3, 2012 3:28:28 PM UTC-5, Dave Angel wrote:
> On 09/03/2012 04:04 PM, Aaron Brady wrote:
> 
> > On Monday, September 3, 2012 2:30:24 PM UTC-5, Ian wrote:
> 
> >> On Sun, Sep 2, 2012 at 11:43 AM, Aaron Brady  wrote:
> 
> >>
> 
> >>> We could use a Python long object for the version index to prevent 
> >>> overflow.  Combined with P. Rubin's idea to count the number of open 
> >>> iterators, most use cases still wouldn't exceed a single word comparison; 
> >>> we could reset the counter when there weren't any.
> 
> >>
> 
> >>
> 
> >> We could use a Python long; I just don't think the extra overhead is
> 
> >>
> 
> >> justified in a data structure that is already highly optimized for
> 
> >>
> 
> >> speed.  Incrementing and testing a C int is *much* faster than doing
> 
> >>
> 
> >> the same with a Python long.
> 
> > I think the technique would require two python longs and a bool in the set, 
> > and a python long in the iterator.
> 
> >
> 
> > One long counts the number of existing (open) iterators.  Another counts 
> > the version.  The bool keeps track of whether an iterator has been created 
> > since the last modification, in which case the next modification requires 
> > incrementing the version and resetting the flag.
> 
> 
> 
> I think you're over-engineering the problem.  it's a bug if an iterator
> 
> is used after some change is made to the set it's iterating over.  We
> 
> don't need to catch every possible instance of the bug, that's what
> 
> testing is for.  The point is to "probably" detect it, and for that, all
> 
> we need is a counter in the set and a counter in the open iterator. 
> 
> Whenever changing the set, increment its count.  And whenever iterating,
> 
> check the two counters.  if they don't agree, throw an exception, and
> 
> destroy the iterator.  i suppose that could be done with a flag, but it
> 
> could just as easily be done by zeroing the pointer to the set.
> 
> 
> 
> I'd figure a byte or two would be good enough for the counts, but a C
> 
> uint would be somewhat faster, and wouldn't wrap as quickly.
> 
> 
> 
> -- 
> 
> 
> 
> DaveA

Hi D. Angel,

The serial index constantly reminds me of upper limits.  I have the same 
problem with PHP arrays, though it's not a problem with the language itself.

The linked list doesn't have a counter, it invalidates iterators when a 
modification is made, therefore it's the "correct" structure in my 
interpretation.  But it does seem more precarious comparatively, IMHO.

Both strategies solve the problem I posed originally, they both involve 
trade-offs, and it's too late to include either in 3.3.0.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: set and dict iteration

[Aaron Brady]

On Monday, September 3, 2012 8:59:16 PM UTC-5, Steven D'Aprano wrote:
> On Mon, 03 Sep 2012 21:50:57 -0400, Dave Angel wrote:
> 
> 
> 
> > On 09/03/2012 09:26 PM, Steven D'Aprano wrote:
> 
> 
> 
> >> An unsigned C int can count up to 4,294,967,295. I propose that you say
> 
> >> that is enough iterators for anyone, and use a single, simple, version
> 
> >> counter in the dict and the iterator. If somebody exceeds that many
> 
> >> iterators to a single dict or set,
> 
> > 
> 
> > I think you have the count confused.  it has to be a count of how many
> 
> > changes have been made to the dict or set, not how many iterators exist.
> 
> 
> 
> Oops, yes you are absolutely right. It's a version number, not a count of 
> 
> iterators.
> 
> 
> 
> 
> 
> -- 
> 
> Steven

Hello.  We have a number of proposed solutions so far.

1) Collection of iterators
  a) Linked list
i) Uncounted references
ii) Counted references
iii) Weak references
  b) Weak set
2) Serial index / timestamp
  a) No overflow - Python longs
  b) Overflow - C ints / shorts / chars
  c) Reset index if no iterators left
3) Iterator count
  - Raise exception on set modifications, not iteration

Note, "2b" still leaves the possibility of missing a case and letting an error 
pass silently, as the current behavior does.  The rest catch the error 100% of 
the time.

Anyway, I plan to develop the above patch for the 'dict' class.  Would anyone 
like to take over or help me do it?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: set and dict iteration

[Aaron Brady]

On Thursday, August 23, 2012 1:11:14 PM UTC-5, Steven D'Aprano wrote:
> On Thu, 23 Aug 2012 09:49:41 -0700, Aaron Brady wrote:
> 
> 
> 
> [...]
> 
> > The patch for the above is only 40-60 lines.  However it introduces two
> 
> > new concepts.
> 
> > 
> 
> > The first is a "linked list", a classic dynamic data structure, first
> 
> > developed in 1955, cf. http://en.wikipedia.org/wiki/Linked_list . 
> 
> > Linked lists are absent in Python
> 
> 
> 
> They certainly are not. There's merely no named "linked list" class.
> 
> 
> 
> Linked lists are used by collections.ChainMap, tracebacks, xml.dom, 
> 
> Abstract Syntax Trees, and probably many other places. (Well, technically 
> 
> some of these are trees rather than lists.) You can trivially create a 
> 
> linked list:
> 
> 
> 
> x = [a, [b, [c, [d, [e, None]
> 
> 
> 
> is equivalent to a singly-linked list with five nodes. Only less 
> 
> efficient.
>
[snip.]
>
> -- 
> 
> Steven

That's not totally true.  Your formulation is equivalent to a single-linked 
list, but a double-linked list can't be represented as an expression because it 
contains reference cycles.

Single and double-linked lists have the same requirements for inserting a new 
node.  For instance:

[a, [b, [c, [d, [e, None]
[a, [a2, [b, [c, [d, [e, None]]

However, to remove a node, the single-linked list requires iterating over the 
entire list to find the predecessor of the target, whereas the double-linked 
list already contains that information.

[a, [b, [c, [d, [e, None]
[a, [b, [c, [e, None

The difference might be moot in some applications, such as if we only remove 
nodes when we're already iterating.

Memory constraints were more severe 50 years ago when the structure was 
developed.  The difference between one pointer and two in a structure could 
have a big impact in repetition.  The single-linked list admits more easily of 
recursive algorithms as well.
-- 
http://mail.python.org/mailman/listinfo/python-list

unit test strategy

[Aaron Brady]

Hello,

I've developing a test script.  There's a lot of repetition.  I want to 
introduce a strategy for approaching it, but I don't want the program to be 
discredited because of the test script.  Therefore, I'd like to know what 
people's reactions to and thoughts about it are.

The first strategy I used created an iterator and advanced it between each step:
self.op_chain(range(5), ('add', 5))
self.op_chain(range(5), ('add', -2), ('add', -1))
self.op_chain(range(5), ('discard', -1), ('add', 5))
self.op_chain_ok(range(5), ('update', [0, 1]))
Etc.

I'm considering something more complicated.  'iN' creates iterator N, 'nN' 
advances iterator N, an exception calls 'assertRaises', and the rest are 
function calls.
dsi= dict.__setitem__
ddi= dict.__delitem__
dsd= dict.setdefault
KE= KeyError
IE= IterationError
self.chain(range(10), 'i0', (dsi, 0, 1), 'n0', (dsi, 10, 1), (IE, 'n0'))
self.chain(range(10), 'i0', 'n0', (dsd, 0, 0), 'n0', (dsd, 10, 1), (IE, 
'n0'))
self.chain(range(10), 'i0', (KE, ddi, 10), 'n0', (ddi, 9), (IE, 'n0'))

Do you think the 2nd version is legible?  Could it interfere with the accuracy 
of the test?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unit test strategy

[Aaron Brady]

On Friday, September 14, 2012 10:32:47 PM UTC-5, David Hutto wrote:
> On Fri, Sep 14, 2012 at 11:26 PM, Dwight Hutto  wrote:
> 
> > On Fri, Sep 14, 2012 at 10:59 PM, Aaron Brady  wrote:
> 
> >> Hello,
> 
> >>
> 
> >> I've developing a test script.  There's a lot of repetition.  I want to 
> >> introduce a strategy for approaching it, but I don't want the program to 
> >> be discredited because of the test script.  Therefore, I'd like to know 
> >> what people's reactions to and thoughts about it are.
> 
> >>
> 
> >> The first strategy I used created an iterator and advanced it between each 
> >> step:
> 
> >
> 
> > That isn't a refined iterator below:
> 
> What I mean is look at the similarities, and the differences, then
> 
> replace the differences with interpolation, in eval even.
> 
> 
> 
> 
> 
> >
> 
> >> self.op_chain(range(5), ('add', 5))
> 
> >> self.op_chain(range(5), ('add', -2), ('add', -1))
> 
> >> self.op_chain(range(5), ('discard', -1), ('add', 5))
> 
> >> self.op_chain_ok(range(5), ('update', [0, 1]))
> 
> >> Etc.
> 
> >>
> 
> >> I'm considering something more complicated.  'iN' creates iterator N, 'nN' 
> >> advances iterator N, an exception calls 'assertRaises', and the rest are 
> >> function calls.
> 
> 
> 
> iN = [N for N in range(0,5)]
> 
> 
> 
> 
> 
> >> dsi= dict.__setitem__
> 
> >> ddi= dict.__delitem__
> 
> >> dsd= dict.setdefault
> 
> >> KE= KeyError
> 
> >> IE= IterationError
> 
> >> self.chain(range(10), 'i0', (dsi, 0, 1), 'n0', (dsi, 10, 1), (IE, 
> >> 'n0'))
> 
> >> self.chain(range(10), 'i0', 'n0', (dsd, 0, 0), 'n0', (dsd, 10, 1), 
> >> (IE, 'n0'))
> 
> >> self.chain(range(10), 'i0', (KE, ddi, 10), 'n0', (ddi, 9), (IE, 
> >> 'n0'))
> 
> >>
> 
> >> Do you think the 2nd version is legible?  Could it interfere with the 
> >> accuracy of the test?
> 
> 
> 
> Define the 2nd version
> 
> 
> 
> 
> 
> >
> 
> > Show the test, which should show instances of what you want called.
> 
> >
> 
> > I could rewrite the above, but it seems you're more in need of refining
> 
> > your iterations, and the values given within them.
> 
> >
> 
> 
> 
> -- 
> 
> Best Regards,
> 
> David Hutto
> 
> CEO: http://www.hitwebdevelopment.com

Hi David,

I'm interested in your comments, but I had difficulty interpreting them.  What 
I want to know is, do people think that the 2nd version I presented would be a 
more effective test script?

Do you think it would be more useful to run the tests in the function call 
directly?  Or would it be more useful to output a program script and then run 
that?  Is there some risk that the direct test would interfere with the 
results?  And, is the 2nd version legible?  That is, is it easy for other 
programmers to tell what the purpose and actual effects of a given test are?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unit test strategy

[Aaron Brady]

On Sunday, September 16, 2012 2:42:09 AM UTC-5, Steven D'Aprano wrote:
> On Fri, 14 Sep 2012 19:59:29 -0700, Aaron Brady wrote:
> 
> 
> 
> > Hello,
> 
> > 
> 
> > I've developing a test script.  There's a lot of repetition.  I want to
> 
> > introduce a strategy for approaching it, but I don't want the program to
> 
> > be discredited because of the test script.
> 
> 
> 
> Test scripts should be simple enough that they don't require test scripts 
> 
> of their own. Or at least, not too many "test-the-test" tests. It is 
> 
> possible to avoid repetition without making convoluted, hard to 
> 
> understand code. Based on the tiny code fragments you give below, I 
> 
> suspect that your test script will need more testing than the code it 
> 
> tests!
> 
> 
> 
> 
> 
> > Therefore, I'd like to know
> 
> > what people's reactions to and thoughts about it are.
> 
> 
> 
> I'd love to make some suggestions, but I have *no idea* what you are 
> 
> talking about. See further comments below:
> 
> 
> 
> 
> 
> > The first strategy I used created an iterator and advanced it between
> 
> > each step:
> 
> 
> 
> What are you using an iterator for? What does this have to do with unit 
> 
> testing?
> 
> 
> 
> So far, your explanation is rather lacking. It's a bit like:
> 
> 
> 
> "I want to create an alarm system for my home, so I put in a screw and 
> 
> tightened it after each step."
> 
> 
> 
> Doesn't really help us understand what you are doing.
> 
> 
> 
> 
> 
> > self.op_chain(range(5), ('add', 5))
> 
> > self.op_chain(range(5), ('add', -2), ('add', -1))
> 
> > self.op_chain(range(5), ('discard', -1), ('add', 5))
> 
> > self.op_chain_ok(range(5), ('update', [0, 1]))
> 
> > Etc.
> 
> 
> 
> Where is the iterator you created? Where are you advancing it? What's 
> 
> op_chain do?
> 
> 
> 
> 
> 
> > I'm considering something more complicated.  'iN' creates iterator N,
> 
> > 'nN' advances iterator N, an exception calls 'assertRaises', and the
> 
> > rest are function calls.
> 
> [...]
> 
> 
> 
> You've proven that even in Python people can write obfuscated code.
> 
> 
> 
> 
> 
> > Do you think the 2nd version is legible?
> 
> 
> 
> Neither version is even close to legible.
> 
> 
> 
> 
> 
> > Could it interfere with the accuracy of the test?
> 
> 
> 
> Who knows? I have no clue what your code is doing, it could be doing 
> 
> *anything*.
> 
> 
> 
> 
> 
> 
> 
> -- 
> 
> Steven

You are forcing me to explain my test code.

Here is an example of some repetitive code.

for view_meth in [ dict.items, dict.keys, dict.values ]:
dict0= dict( ( k, None ) for k in range( 10 ) )
iter0= iter( view_meth( dict0 ) )
dict.__setitem__( dict0, 0, 1 )
next( iter0 )
dict.__setitem__( dict0, 10, 1 )
self.assertRaises( IterationError, next, iter0 )

dict0= dict( ( k, None ) for k in range( 10 ) )
iter0= iter( view_meth( dict0 ) )
next( iter0 )
dict.__setitem__( dict0, 0, 1 )
next( iter0 )
dict.__setitem__( dict0, 10, 1 )
self.assertRaises( IterationError, next, iter0 )

dict0= dict( ( k, None ) for k in range( 10 ) )
iter0= iter( view_meth( dict0 ) )
self.assertRaises( KeyError, dict0.__delitem__, 10 )
next( iter0 )
dict.__delitem__( dict0, 9 )
self.assertRaises( IterationError, next, iter0 )

dict0= dict( ( k, None ) for k in range( 10 ) )
iter0= iter( view_meth( dict0 ) )
next( iter0 )
self.assertRaises( KeyError, dict0.__delitem__, 10 )
next( iter0 )
dict.__delitem__( dict0, 9 )
self.assertRaises( IterationError, next, iter0 )


It makes sense to condense it.  However, it can be condensed rather far, as 
follows:

dsi= dict.__setitem__
ddi= dict.__delitem__
KE= KeyError
IE= IterationError
chain(range(10), 'i0', (dsi, 0, 1), 'n0', (dsi, 10, 1), (IE, 'n0'))
chain(range(10), 'i0', 'n0', (dsi, 0, 1), 'n0', (dsi, 10, 1), (IE, 'n0'))
chain(range(10), 'i0', (KE, ddi, 10), 'n0', (ddi, 9), (IE, 'n0'))
chain(range(10), 'i0', 'n0', (KE, ddi, 10), 'n0', (ddi, 9), (IE, 'n0'))


The parameters to 'chain' correspond 1-to-1 with the statements earlier.  The 
view methods are looped over in the 'chain' method instead.  'op_chain' was an 
earlier version; some midway point in condensing the code could be preferable.

Specifically my questions are, is the code condensed beyond legibility?  Should 
'chain' execute the test directly, or act as a metaprogram and output the test 
code into a 2nd file, or both?  Should 'chain' create the iterators in a 
dictionary, or in the function local variables directly with 'exec'?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unit test strategy

[Aaron Brady]

On Sunday, September 16, 2012 3:01:11 PM UTC-5, Steven D'Aprano wrote:
> On Sun, 16 Sep 2012 11:38:15 -0700, Aaron Brady wrote:
> > Here is an example of some repetitive code.
> > 
> > for view_meth in [ dict.items, dict.keys, dict.values ]:
> > dict0= dict( ( k, None ) for k in range( 10 ) ) 
> >   iter0= iter( view_meth( dict0 ) )
> > dict.__setitem__( dict0, 0, 1 )
> > next( iter0 )
> > dict.__setitem__( dict0, 10, 1 )
> > self.assertRaises( IterationError, next, iter0 )
> [...]
> 
> First off, if you have any wish for this to be accepted into the standard 
> library, I suggest you stick to PEP 8.

The code in the zip file on the other thread does conform to PEP 8.
 
> Secondly, this is test code. A bit of repetition is not to be concerned 
> about, clarity is far more important than "Don't Repeat Yourself". The 
> aim isn't to write the fastest, or most compact code, but to have good 
> test coverage with tests which are *obviously* correct (rather than test 
> code which has no obvious bugs, which is very different). If a test 
> fails, you should be able to identify quickly what failed without running 
> a debugger to identify what part of the code failed.
> 
> Thirdly, why are you writing dict.__setitem__( dict0, 0, 1 ) instead of 
> dict0[0] = 1 ?
> 
> 
> [...]
> > Specifically my questions are, is the code condensed beyond legibility? 
> 
> Yes.
> 
> 
> > Should 'chain' execute the test directly, or act as a metaprogram and
> > output the test code into a 2nd file, or both?
> 
> None of the above.
> 
> 
> > Should 'chain' create
> > the iterators in a dictionary, or in the function local variables
> > directly with 'exec'?
> 
> Heavens to Murgatroyd, you can't be serious.
> 
> 
> Here's my attempt at this. Note the features:
> 
> - meaningful names (if a bit long, but that's a hazard of tests)
> - distinct methods for each distinct test case
> - comments explaining what the test code does
> - use of subclassing
> 
> 
> # Untested
> class TestDictIteratorModificationDetection(unittest.TestCase):
[snip]
> def testIteratorFailsAfterSet(self):
> self.iterator_fails_after_modification(dict.__setitem__, 1, 1)
> 
> def testIteratorFailsAfterDel(self):
> self.iterator_fails_after_modification(dict.__delitem__, 1)
> 
> def testIteratorFailsAfterUpdate(self):
> self.iterator_fails_after_modification(dict.update, {5: 1})
> 
> def testIteratorFailsAfterPop(self):
> self.iterator_fails_after_modification(dict.pop, 4)
[snip]
> 
> I think I've got all the methods which can mutate a dictionary. If I 
> missed any, it's easy enough to add a new test method to the class.
[snip]

Well Mr. D'Aprano, I have some serious disagreements with the script you posted.

You did test all the mutating methods; there are 7; but I don't think it's 
enough.  You omitted the test case which revealed the bug originally or other 
pairs of operations; you didn't test on empty sets; you didn't test on "null" 
modifications; you didn't test on multiple iterators in any form; and you 
didn't test for memory leaks which is important with a dynamic structure.  A 
thorough test suite should contain tens if not hundreds of tests for this 
application.

Your script was very easy to understand.  However in the volume I'm advocating, 
something more concise would easier to understand, and D-R-Y becomes applicable 
again.  In a more concise form, the reader could browse what tests are 
executed, find a certain test or determine if it's omitted.

The "best of both" solution is a metaprogram, which is extremely brief, but 
does not run tests, and outputs a full test catalog instead, which is then 
itself scrutinized and run as the real test.  The test script would 
consequently be two files big, and be run in a specific order.  Not all tests 
can be condensed in a metaprogram; the script would contain large sections of 
literal code or it would appear in yet a 3rd file.

> Thirdly, why are you writing dict.__setitem__( dict0, 0, 1 ) instead of 
> dict0[0] = 1 ?

'__setitem__' can be passed to secondary functions whereas square brackets 
cannot.  The 2nd file, the output of the metaprogram, could contain either or 
both.  We could pass 'operator.setitem' as an alternative.

I realize I'm introducing yet a 3rd foreign concept with the patch.  If not 
enough readers approve of it I will have to abandon it, which would be a shame.

OTOH, I appreciate the fact you used my "for view in (dict.items, dict.keys, 
dict.values):" idea.  Also, what is your argument that an unstarted iterator 
should be exempt from invalidation when the set/dict is modified?  It is not 
obvious it should or shouldn't, similar to the behavior of modifying dict 
values but not keys, and I would side against the exemption.  (Perhaps we 
should raise that issue on the other thread.)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PyPI - how do you pronounce it?

[Aaron France]

On 01/28/2012 10:57 AM, Steven D'Aprano wrote:

On Sat, 28 Jan 2012 18:48:49 +1100, Chris Angelico wrote:


Hopefully this will be a step up from Rick's threads in usefulness, but
I'm aware it's not of particularly great value!

How do you pronounce PyPI? Is it:

Obviously that's pronounced Fin-tim-lin-bin-whin-bim-lim-bus-stop-F'tang-
F'tang-Olé-Biscuitbarrel.



* Pie-Pie?

Or that one.




Slightly incorrect punctuation there.

Fin-tim-lin-bin-whin-bim-lim-bus-stop-F'tang-F'tang-Olé-Biscuitbaréle.


--
http://mail.python.org/mailman/listinfo/python-list

Re:

[Aaron France]

On 02/03/2012 09:14 PM, Chris Rebert wrote:

On Fri, Feb 3, 2012 at 12:06 PM, Debashish Saha  wrote:

would u like to help me by answering some vbasic questions about python?

You might prefer to ask such questions on the tutor mailing list instead:
http://mail.python.org/mailman/listinfo/tutor

Cheers,
Chris

Sounds like this one is gonna be on the lines of:

Will you do the needful and write my application for me for the tracker 
actions?


Thanking in advances

Sanjay
--
http://mail.python.org/mailman/listinfo/python-list

Re: difference between random module in python 2.6 and 3.2?

[Aaron France]

On 02/06/2012 09:57 AM, Matej Cepl wrote:

On 6.2.2012 09:45, Matej Cepl wrote:

Also, how could I write a re-implementation of random.choice which would
work same on python 2.6 and python 3.2? It is not only matter of unit
tests, but I would really welcome if the results on both versions
produce the same results.


Silly, of course, the solution is obvious ... I have just embedded 
random.choice from 2.6 to my code.


Matěj

Is the above actually a good idea though?

What I understand you're doing is embedding the source from the 
Python2.6 random.py file, /into/ your project files?


Regards,
A


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: iterating over list with one mising value

[Aaron France]

On 02/07/2012 10:13 PM, MRAB wrote:

On 07/02/2012 20:23, Sammy Danso wrote:

Hi Expert,
Thanks for your responses and help. thought I should provide more
information for clarity.

>

Please find the error message below for more information




for (key, value) in wordFreq2:
ValueError: need more than 1 value to unpack

this is a sample of my data

['with', 3, 'which', 1, 'were', 2, 'well', 1, 'water', 1, 'was', 4,
'two', 1, 'to', 2, 'through', 1, 'thlabour', 1, 'these', 1, 'theat', 1,
'the', 8, 'tetanus', 1, 'started', 1, 'size', 1, 'scent', 1,
'respectively', 1, 'received', 1, 'problems', 2, 'prince', 1,
'pregnancy', 1, 'poured', 1, 'peace', 1, 'pains', 1, 'painless', 1,
'out', 1, 'of', 1, 'noseat', 1, 'nose', 2, 'no', 2, 'maternity', 1,
'malformation', 1, 'made', 1, 'lower', 1, 'labour/delivery', 2,
'kintampo', 1, 'into', 1, 'injections', 1, 'in', 3, 'i', 2, 'hospital',
1, 'home', 1, 'him', 1, 'having', 1, 'had', 2, 'green', 1, 'gave', 1,
'flowing', 2, 'encountered', 1, 'eleven', 1, 'during', 3, 'district', 1,
'difficulty', 1, 'cord', 1, 'consecutive', 1, 'colour', 1, 'cleared', 1,
'child', 1, 'checkups', 1, 'came', 1, 'but', 2, 'breathing', 2,
'breath', 1, 'blood', 2, 'bleeding', 1, 'birth', 4, 'before', 1, 'bad',
1, 'average', 1, 'at', 2, 'assist', 1, 'artificial', 1, 'around', 2,
'antenatal', 1, 'and', 5, 'an', 1, 'ambrical', 1, 'air', 1, 'abdominal',
1, '600am', 1, '100pm', 1, '', 3, 'other']

What I would like to do is to pad the last value 'other' with a default
so I can iterate sucessfully

my desired ouput is the format below in a text file.
with 3
which 3
were 2
..
.
.
other


The line:

for (key, value) in wordFreq2:

attempts to iterate through the list wordFreq2, where each item in the
list is a _pair_ of values.

However, what you have is a list of _single_ values, alternating string
and number.

That's why it's complaining that it can't unpack.

for i in range(0, len(x), 2):
print x[i-1], x[i]

--
http://mail.python.org/mailman/listinfo/python-list

Re: iterating over list with one mising value

[Aaron France]

On 02/07/2012 11:09 PM, Mark Lawrence wrote:

On 07/02/2012 21:25, Aaron France wrote:

for i in range(0, len(x), 2):
 print x[i-1], x[i]

x = ['with', 3, 'which', 1, 'were', 2, 'well', 1, 'water', 1, 'was', 4,
'two', 1, 'to', 2, 'through', 1, 'thlabour', 1, 'these', 1, 'theat', 1,
'the', 8, 'tetanus', 1, 'started', 1, 'size', 1, 'scent', 1,
'respectively', 1, 'received', 1, 'problems', 2, 'prince', 1,
'pregnancy', 1, 'poured', 1, 'peace', 1, 'pains', 1, 'painless', 1,
'out', 1, 'of', 1, 'noseat', 1, 'nose', 2, 'no', 2, 'maternity', 1,
'malformation', 1, 'made', 1, 'lower', 1, 'labour/delivery', 2,
'kintampo', 1, 'into', 1, 'injections', 1, 'in', 3, 'i', 2, 'hospital',
1, 'home', 1, 'him', 1, 'having', 1, 'had', 2, 'green', 1, 'gave', 1,
'flowing', 2, 'encountered', 1, 'eleven', 1, 'during', 3, 'district', 1,
'difficulty', 1, 'cord', 1, 'consecutive', 1, 'colour', 1, 'cleared', 1,
'child', 1, 'checkups', 1, 'came', 1, 'but', 2, 'breathing', 2,
'breath', 1, 'blood', 2, 'bleeding', 1, 'birth', 4, 'before', 1, 'bad',
1, 'average', 1, 'at', 2, 'assist', 1, 'artificial', 1, 'around', 2,
'antenatal', 1, 'and', 5, 'an', 1, 'ambrical', 1, 'air', 1, 'abdominal',
1, '600am', 1, '100pm', 1, '', 3, 'other']
for i in range(0, len(x), 2):
print x[i-1], x[i]

other with
3 which
...
1
3 other

Houston, we've got a problem:)


Yupp.

The real problem is that there's been over 20 messages about this 'problem'.


--
http://mail.python.org/mailman/listinfo/python-list

Re: GUI for pickle read

[Aaron France]

On Tue, Feb 28, 2012 at 12:51 PM, Smiley 4321  wrote:

Can I have some thoughts about - building a GUI to display the results of
the pickle read?

A prototype code should be fine on Linux.

What on earth is this post asking?

Do you want code? Opinions?
--
http://mail.python.org/mailman/listinfo/python-list

How to get number of bytes written to nonblocking FIFO when EAGAIN is raised?

[Aaron Staley]

Scenario. I have a fifo named 'fifo' on my computer (ubuntu linux)
operating in nonblocking mode for both read and write.  Under normal
operation all is good:

Interpreter 1 (writer)
>>> import os
>>> fd = os.open('fifo', os.O_WRONLY | os.O_NONBLOCK)
>>> f = os.fdopen(fd,'wb')
>>> f.write('k')
>>> f.flush()

Interpreter 2 (reader):
>>> import os
>>> fd = os.open('fifo', os.O_NONBLOCK)
>>> f = os.fdopen(fd)
>>> f.read()  #after f.flush was done in interpreter 1
'k'

However, if interpreter 1 overfills the FIFO, we get an error (EAGAIN)
>>> f.write('a'*7)
IOError: [Errno 11] Resource temporarily unavailable

However interpreter 2 still receives data
>> len(f.read())
65536

It looks like interpreter 1 pushed data until the FIFO was full and
then raised the IOError.  Interpreter 2 constantly received some, but
not all, of what interpreter 2 tried to send.
Unfortunately, the IOError seems to have no attribute indicating how
much data was successfully sent.  I've looked through the docs and
can't seem to figure out how; can anyone land some advice?

Thanks,
Aaron Staley
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to get number of bytes written to nonblocking FIFO when EAGAIN is raised?

[Aaron Staley]

On Jul 19, 8:15 pm, Adam Skutt  wrote:
> On Jul 19, 9:19 pm, Aaron Staley  wrote:
>
> > However, if interpreter 1 overfills the FIFO, we get an error (EAGAIN)>>> 
> > f.write('a'*7)
>
> > IOError: [Errno 11] Resource temporarily unavailable
>
> > However interpreter 2 still receives data>> len(f.read())
>
> > 65536
>
> > It looks like interpreter 1 pushed data until the FIFO was full and
> > then raised the IOError.  Interpreter 2 constantly received some, but
> > not all, of what interpreter 2 tried to send.
> > Unfortunately, the IOError seems to have no attribute indicating how
> > much data was successfully sent.  I've looked through the docs and
> > can't seem to figure out how; can anyone land some advice?
>
> You need to do as Roy Smith suggested and use the actual OS I/O calls
> os.read() and os.write().  os.write() returns the number of bytes
> actually written to the underlying descriptor.  Python file objects
> are akin to FILE structures in C: they perform buffering and other
> operations internally that makes them less than suitable for usage
> with asynchronous UNIX I/O. In particular, they buffer I/O internally
> before writing it to the descriptor, so there's no direct relationship
> between calling file.write() and how much data is written to the
> stream. In addition, file objects also simply raise the underlying OS
> error when it occurs.  The UNIX write(2) syscall assumes that you have
> been keeping track of how many bytes you've successfully written to
> the stream and does not track it for you.
>
> Adam

That's for the info; lower level I/O solved the issue.
That said, is such behavior with the file objects intended? It seems
they don't work correctly with an underlying non-blocking file
descriptor, but the documentation doesn't state such. In fact, it
suggests you can use non-blocking with this comment for file.read:

Also note that when in non-blocking mode, less data than was requested
may be returned, even if no size parameter was given.
-- 
http://mail.python.org/mailman/listinfo/python-list

OSX application built with py2app can't see bundled PySide module?

[Aaron Scott]

I'm trying to deploy a Python app on OSX that was built with PySide. py2app 
packages it without issue, copying and linking a lot of PySide and Qt files in 
the process. But then, when I try to run the built app, I get this error:

Traceback (most recent call last):
  File 
"/Users/sequence/Desktop/code/dailies/dist/dailies_v02.app/Contents/Resources/__boot__.py",
 line 31, in 
_run('dailies_v02.py')
  File 
"/Users/sequence/Desktop/code/dailies/dist/dailies_v02.app/Contents/Resources/__boot__.py",
 line 28, in _run
execfile(path, globals(), globals())
  File 
"/Users/sequence/Desktop/code/dailies/dist/dailies_v02.app/Contents/Resources/dailies_v02.py",
 line 9, in 
from PySide.QtCore import *
  File "PySide/__init__.pyc", line 2, in 
  File "PySide/private.pyc", line 2, in 
  File "PySide/QtCore.pyc", line 18, in 
  File "PySide/QtCore.pyc", line 15, in __load
ImportError: '/usr/lib/python2.6/lib-dynload/PySide/QtCore.so' not found

The weird thing is, QtCore.so IS included in the application bundle: py2app 
copied it to the build under 
Contents/Resources/lib/python2.6/lib-dynload/PySide/. Is there a reason the 
application isn't seeing this?
-- 
http://mail.python.org/mailman/listinfo/python-list

error in exception syntax

[Aaron Gray]

On Windows I have installed Python 3.2 and PyOpenGL-3.0.1 and am getting the 
following error :-


   File "c:\Python32\lib\site-packages\OpenGL\platform\win32.py", line 13
 except OSError, err:
   ^

It works okay on my Linux machine running Python 2.6.2.

Many thanks in advance,

Aaron

--
http://mail.python.org/mailman/listinfo/python-list

Re: error in exception syntax

[Aaron Gray]

"Aaron Gray"  wrote in message 
news:8tpu87f75...@mid.individual.net...
On Windows I have installed Python 3.2 and PyOpenGL-3.0.1 and am getting 
the following error :-


   File "c:\Python32\lib\site-packages\OpenGL\platform\win32.py", line 13
 except OSError, err:
   ^

It works okay on my Linux machine running Python 2.6.2.


Many thanks,

Aaron

--
http://mail.python.org/mailman/listinfo/python-list

Announce: WHIFF 1.1 Release

[Aaron Watters]

WHIFF 1.1 RELEASED

WHIFF [WSGI HTTP Integrated Filesystem Frames]
is a collection of support services
for Python/WSGI Web applications which
allows applications to be composed by
"dropping" dynamic pages into container
directories.

This mode of development will be familiar
to developers who have created PHP
applications, vanilla CGI scripts,
Apache/modpy Publisher applications,
JSP pages, or static web content.

WHIFF 1.1 improves support for Google App
Engine applications and fixes a number of
bugs/misfeatures regarding string quoting
and encoding.

PROJECT PAGE: http://whiff.sourceforge.net/
DOCUMENTATION: http://whiffdoc.appspot.com
DOWNLOAD: https://sourceforge.net/projects/whiff/
REPOSITORY: http://code.google.com/p/whiff/source/checkout

Please read about the features, tutorials, and demos:

Test drive
  http://whiffdoc.appspot.com/docs/W1100_0500.TestDrive
Mako template support
  http://whiffdoc.appspot.com/docs/W1100_1075.MakoGrading
Drop down menus middlewares
  http://whiffdoc.appspot.com/docs/W1100_1300.menus
AJAX callback support
  http://whiffdoc.appspot.com/docs/W1100_1400.calc
Jquery helpers
  http://whiffdoc.appspot.com/docs/W1100_1450.jQueryUI
Integration support for repoze.who authentication
  http://whiffdoc.appspot.com/docs/W1100_1500.who
Open Flash Charts
  http://whiffdoc.appspot.com/docs/W1100_1600.openFlashCharts
Internationalization support
  http://whiffdoc.appspot.com/docs/W1100_1700.international
Tree views with AJAX callbacks
  http://whiffdoc.appspot.com/docs/W1100_2200.TreeView
Google App Engine compatibility
  http://whiffdoc.appspot.com/docs/W1100_2300.GAEDeploy

And much more.

Please try it out and let me know what you
think.  Also, thanks to all for suggestions and other
feedback.

-- Aaron Watters

===
This one goes to eleven.
-- 
http://mail.python.org/mailman/listinfo/python-list

Single line if statement with a continue

[Aaron P]

I occasionally run across something like:

for idx, thing in enumerate(things):
if idx == 103: 
continue
do_something_with(thing)

It seems more succinct and cleaner to use:

if idx == 103: continue.

Of course this would be considered an anti-pattern, and Flake8 will complain.

Any opinions, or feedback on the matter.
-- 
https://mail.python.org/mailman/listinfo/python-list

Breaking down network range to individual networks

[Aaron Christensen]

Hello,

Does anyone know about a feature of ipaddress that will take the following
network range:

"10.224.16.0-10.224.23.0"

and convert it to individual networks?

I am trying to figure out the best way to take a network range and break it
up into the largest possible networks.

I started working out some things on paper but I don't want to get too far
ahead of myself in case there is a function or easy way that already does
this.

Thank you for your help!
Aaron




-
Pasted some brainstorming below to show that I am actively working this and
not just asking before trying.


ipaddress.ip_network('10.224.16.0/24') - (ipaddress.ip_network('
10.224.23.0/24'))
IPv4Network

IPv4Network('10.224.16.0/24').compare_networks(IPv4Network('10.224.23.0/24')
)



str(ipaddress.IPv4Address('192.168.0.1'))

IPv4Address('10.224.16.0') + 3

ipaddress.ip_address('10.224.16.0') + 256


0.0.x=0.0
/24 x + 0
/23 x + 1 = 0.0.0.0
/22 x + 3 = 0.0.0.0

1  can be /18 if x = 0
2  can be /19 if x = 0, 32
4  can be /20 if x = 0, 16, 32, 48
8  can be /21 if x = 0, 8, 16, 24, 32, 40, 48, 56
16 can be /22 if x = 0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 48, 52,
56, 60
32 can be /23 if x = 0 through 62


64 can be /24 if x = 0 though 255

10.224.255.0

start > can be
odd  > /24
even > /24/23
0  >  /22/21/20/19/18...
4  >  /22
8  >  /22/21
12 >  /22
16 >  /22/21/20
20 >  /22
24 >  /22/21
28 >  /22
32 >  /22/21/20/19
36 >  /22
40 >  /22/21
44 >  /22
48 >  /22/21/20
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Lists And Missing Commas

[Aaron Gray]

On Tuesday, 24 December 2019, Tim Daneliuk  wrote:

> If I do this:
>
> foo = [ "bar", "baz" "slop", "crud" ]
>
> Python silently accepts that and makes the middle term "bazslop".


Strings concatinate over line endings so this case is only sensible really.


>
> BUT, if I do this:
>
> foo = [ "bar", "baz" 1, "crud" ]
>
> or this:
>
> foo = [ "bar", 2 1, "crud" ]
>
> The interpreter throws a syntax error.
>
> This is more of an intellectual curiosity than anything else, but why do
> strings silently
> concatenate like that, while all other case blow up with an error.  i.e.,
> What is about
> the language the promotes this behavior.  At first blush, it seems
> inconsistent, but what
> do I know ...
>
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>


-- 
Independent Software Engineer and amateur Computer Scientist
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to convert a number to hex number?

[Aaron Bingham]

Hako wrote:

>I try this command:
>  
>
>>>>import string
>>>>string.atoi('78',16)
>>>>
>>>>
>120
>this is 120 not 4E.
>
>Someone can tell me how to convert a decimal number to hex number? Can
>print A, B, C,DEF.
>  
>
To print the hexidecimal string representation of an integer, n, use
print hex(n)
or
print "%x" % n

Regards,

Aaron Bingham

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Eclipse best/good or bad IDE for Python?

[Aaron Bingham]

[EMAIL PROTECTED] wrote:

>I'm trying to move beyond Emacs/Vim/Kate
>and was wondering if Eclipse is better and if it is the *best*
>IDE for Python.
>
>Should I leave Emacs and do Python coding in Eclipse?
>  
>
I've been a heavy Emacs user for several years, but recently switched to 
Eclipse for Python development.  I was skeptical at first, but I gave it 
a chance for a few days and was convinced.

The killer PyDev feature for me is pylint integration.  Being informed 
immediately when you mistype a variable name is a big timesaver.  Also 
nice is the refactoring support (although this it is possible to 
integrate BicycleRepairMan! with Emacs, I found it easier to use in 
Eclipse).  I still find the Eclipse editor awkward for some things that 
are easy in Emacs (Emacs is in my fingers), so I occasionally switch 
back to Emacs for a quick edit.

Eclipse performance is not a problem for me, but I have a beefy box.

Enjoy,

Aaron Bingham

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Eclipse best/good or bad IDE for Python?

[Aaron Bingham]

Paul Boddie wrote:

>[EMAIL PROTECTED] wrote:
>  
>
>>Eclipse, for example, performs like a dog on
>>my dual opteron workstation w/ 2GB of RAM, which is more than enough to
>>annoy me. I shouldn't have to wait more than about 1 second for an
>>editor to start and then open what is essentially a text file :-P.
>>
>>
>
>And then, due to the excessive "project management" screen furniture,
>have to edit it "through the keyhole"...
>  
>
Hi Paul,

Did you ever try double clicking the editor tab?  That zooms the editor 
to take up the whole window and hids all the "screen furniture".  You 
can double click the tab again to get the furniture back.

Aaron Bingham

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: search backward

[Aaron Bingham]

Robert wrote:
I need to find the location of a short string in a long string. The
problem however is that i need to search backward.
Does anybody know how to search in reverse direction?
>>> "foofoo".find("foo")
0
>>> "foofoo".rfind("foo")
3
>>> "foofoo".index("foo")
0
>>> "foofoo".rindex("foo")
3
--

Aaron Bingham
Application Developer
Cenix BioScience GmbH

--
http://mail.python.org/mailman/listinfo/python-list

Re: 2 versions of python on 1 machine

[Aaron Bingham]

flupke wrote:
I have version 2.3.4 and 2.4 installed on windows and i thought that by 
switching the PYTHONPATH parameter to the dir of the 2.4 version that 
that would make python 2.4 active.

However when i envoke python from the commandline, it still runs 2.3.4
Is it possible to have 2 versions installed and switching versions when
you need to?
How can i do that?
You need to put the path to the desired version in the PATH environment 
variable.  You shoudn't need to change PYTHONPATH at all.

--

Aaron Bingham
Application Developer
Cenix BioScience GmbH

--
http://mail.python.org/mailman/listinfo/python-list

Re: Fuzzy matching of postal addresses

[Aaron Bingham]

Andrew McLean wrote:
I have a problem that is suspect isn't unusual and I'm looking to see if 
there is any code available to help. I've Googled without success.

Basically, I have two databases containing lists of postal addresses and 
need to look for matching addresses in the two databases. More 
precisely, for each address in database A I want to find a single 
matching address in database B.
I had a similar problem to solve a while ago.  I can't give you my code, 
but I used this paper as the basis for my solution (BibTeX entry from 
http://citeseer.ist.psu.edu/monge00adaptive.html):

@misc{ monge-adaptive,
  author = "Alvaro E. Monge",
  title = "An Adaptive and Efficient Algorithm for Detecting 
Approximately Duplicate
Database Records",
  url = "citeseer.ist.psu.edu/monge00adaptive.html" }

There is a lot of literature--try a google search for "approximate 
string match"--but very little publically available code in this area, 
from what I could gather.  Removing punctuation, etc., as others have 
suggested in this thread, is _not_sufficient_.  Presumably you want to 
be able to match typos or phonetic errors as well.  This paper's 
algorithm deals with those problems quite nicely,

--
----
Aaron Bingham
Application Developer
Cenix BioScience GmbH

--
http://mail.python.org/mailman/listinfo/python-list

Re: Fuzzy matching of postal addresses

[Aaron Bingham]

Andrew McLean wrote:
Thanks for all the suggestions. There were some really useful pointers.
A few random points:
[snip]
4. You need to be careful doing an endswith search. It was actually my 
first approach to the house name issue. The problem is you end up 
matching "12 Acacia Avenue, ..." with "2 Acacia Avenue, ...".
Is that really a problem?  That looks like a likely typo to me.  I guess 
it depends on your data set.  In my case, the addresses were scattered 
all over the place, with relatively few in a given city, so the 
likelyhood of two addresses on the same street in the same town was very 
low.  We /wanted/ to check for this kind of 'duplication'.

Note that endswith will not deal with 'Avenue' vs. 'Ave.', but I supose 
a normalization phase could take care of this for you.  The Monge 
algorithm I pointed you to takes care of this pretty nicely.

--
----
Aaron Bingham
Application Developer
Cenix BioScience GmbH

--
http://mail.python.org/mailman/listinfo/python-list

Re: __eq__ on a dict

[Aaron Bingham]

"Steven D'Aprano" <[EMAIL PROTECTED]> writes:

> On Mon, 11 Jul 2005 12:42:55 +0200, Neil Benn wrote:
>
>> Hello,
>> 
>> I can't find the docs for __eq__ on a dict and I can't find 
>> a description on what the eq does (strangely it does implement > and < 
>> but I have no idea what that does).  Does anyone know (definitively) 
>> what the __eq__, __gt__, __lt__ methods do.

[snip]

> For any two objects x and y, when you call 
>
> x == y
>
> Python calls x.__eq__(y). That includes dicts:
>
> py> dictA = {0: "spam"}
> py> dictB = {0: "sp" + "am"}
> py> dictC = {1: "ham"}
> py> 
> py> dictA == dictB
> True
> py> dictA.__eq__(dictB)   # same as dictA == dictB
> True
> py> dictB == dictC   # calls dictB.__eq__(dictC)
> False
>
> Two dicts are equal if they have the same keys and the same values.

That is what I would expect, but where is that documented?  Also,
where is the behavior of the much less obvious dictionary methods
__ge__, __gt__, __le__, __lt__, and __cmp__ methods documented?

> In general, you should not call __eq__ directly, but use the == operator
> instead.

That is clear enough, the OP was seeking information about the
behavior of these operators when used with dictionaries.

Thanks,

-- 

Aaron Bingham
Senior Software Engineer
Cenix BioScience GmbH


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: __eq__ on a dict

[Aaron Bingham]

Hi Steven,

Thanks for digging into this.

"Steven D'Aprano" <[EMAIL PROTECTED]> writes:

> Replying to myself... how sad.
>
> On Tue, 12 Jul 2005 15:41:46 +1000, Steven D'Aprano wrote:
>
>> That wasn't clear from his post at all. If he had explained what he
>> wanted, I wouldn't have wasted my time explaining what he already knew.
>
> On reading it, that came across more snarky than I intended. Sorry.

It's ok.  I realize the OP did not make it crystal clear what he was
getting at in his post.  It's easier when you are sitting in the same
office ;-).

> However, I wasn't completely insane, since I came across this tidbit:
>
> http://python.active-venture.com/ref/comparisons.html
>
> "Mappings (dictionaries) compare equal if and only if their sorted (key,
> value) lists compare equal. Outcomes other than equality are resolved
> consistently, but are not otherwise defined."
>
> with a footnote leading to this comment:
>
> "Earlier versions of Python used lexicographic comparison of the sorted
> (key, value) lists, but this was very expensive for the common case of
> comparing for equality."

Ah, I missed that, thanks for the pointer.  Seems information of
dictionary comparisons should also appear in the Library Reference
under Mapping Types.

> I also suggested:
>
>> My second thought was that comparison is implemented by first comparing
>> keys, then values, ie cmp(dictA, dictB)
> [snip]
>> I don't think I can prove it though.

Looking at the source code links Simon posted (thanks Simon!) it is
clear that, in the current version of CPython, dictionaries are
ordered first by length and only if the lengths are equal are the
keys and values examined.

> Equality of dicts is guaranteed. Two dicts are equal if and only if their
> keys:value pairs are equal. Other orderings between dicts are calculated
> consistently but not in any documented way.
>
> One gotcha is that some dicts are unordered:
>
> py> {1:1j} < {1:2j}
> Traceback (most recent call last):
>   File "", line 1, in ?
> TypeError: cannot compare complex numbers using <, <=, >, >=
>
> but even that is special-cased up to the wazzoo:
>
> py> {1:1j} < {1:1j}
> False

Hmm... not sure I like that!

-- 

Aaron Bingham
Senior Software Engineer
Cenix BioScience GmbH


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [perl-python] text pattern matching, and expressiveness

[Aaron Sherman]

John Bokma wrote:
Xah Lee wrote:

Perl is a language of syntactical variegations. Python on the other
hand, does not even allow changes in code's indentation, but its
efficiency and power in expression, with respect to semantics (i.e.
algorithms), showcases Perl's poverty in specification.

Clarify :-D.
Well, I think it's pretty clearly troll-bait. Semantics != algorithms 
and Perl (just like Python) has no such poverty WRT specification.

Sounds like someone who learned Python and forgot that it wasn't a 
religion. Too bad, but a disease unique to no particular language. Perl, 
Python, Ruby, C, APL, LISP... they've all had their unpleasant zealots.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Faster way to do this...

[Aaron Bingham]

Harlin Seritt wrote:
I've got the following code:
nums = range(0)
for a in range(100):
   nums.append(a)
Is there a better way to have num initialized to a list of 100
consecutive int values?
You mean like this?
nums = range(100)
;-)
--

Aaron Bingham
Software Engineer
Cenix BioScience GmbH

--
http://mail.python.org/mailman/listinfo/python-list

metaclass error

[Aaron Steele]

hi,
i have a very simple package organized as follows:
!-!
bgp/
   __init__.py
   managers/
   __init__.py
   ManagerInterface.py
   TestManager.py
!-!
and here's ManagerInterface.py and TestManager.py:
!-!
# ManagerInterface.py
class ManagerInterface(object):
   def __init__(self): pass
   def process(self, recset, operation):
   print 'In ManagerInterface.process()...'
# TestManager.py
import ManagerInterface
class TestManager(ManagerInterface):
   def process(self, recset, operation):
   print 'In TestManager.process()...' 
super(TestManager,self).process(recset,operation)
!---!

when i try to import the TestManager module via the interpreter, i get 
the following error:

!---!
$ python
Python 2.4.1c1 (#1, Mar 14 2005, 10:28:18)
[GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-49)] on linux2
>>> import bgp.managers.TestManager
Traceback (most recent call last):
 File "", line 1, in ?
 File "bgp/managers/TestManager.py", line 2, in ?
   class TestManager(ManagerInterface):
TypeError: Error when calling the metaclass bases
   module.__init__() takes at most 2 arguments (3 given)
!---!
any thoughts? i think that when python executes the TestManager class 
statement, it collects the base class (ManagerInterface) into a tuple 
and then executes the class body in a dictionary... is this where the 
error is happening?

thanks!
aaron
--
http://mail.python.org/mailman/listinfo/python-list

Re: COM connection point

[Aaron Brady]

That appears to do just the job!  Much obliged,

-OY

- Original Message - 
From: "Stefan Schukat" <[EMAIL PROTECTED]>
To: "Oy Politics" <[EMAIL PROTECTED]>;

Sent: Friday, March 18, 2005 6:05 AM
Subject: RE: COM connection point


> Just use
> 
> obj = win32com.client.Dispatch(obj)
> 
>   Stefan
> 
> 
> > -Original Message-
> > From:
[EMAIL PROTECTED]
> >
[mailto:[EMAIL PROTECTED]
Behalf Of
> > Oy Politics
> > Sent: Wednesday, March 16, 2005 11:51 PM
> > To: python-list@python.org
> > Subject: COM connection point
> > 
> > 
> > Hello:
> > 
> > I am building a COM client, with the ability to be
called 
> > back by events.
> > The events can arrive independently from the
server.  The 
> > client method is
> > called at the right time, so that is working. 
However, one 
> > parameter is
> > itself a COM object, and I am having trouble with
accessing 
> > the properties
> > of the parameter object.
> > 
> > Here is the output of what I have currently.
> > 
> > -- Python execute --
> > event! 
> > event! 
> > event! 
> > 
> > etc.  Obj is supposed to be my intended parameter.
 However, 
> > when I try to
> > access its property, I get the following:
> > 
> > -- Python execute --
> > pythoncom error: Python error invoking COM method.
> > 
> > Traceback (most recent call last):
> >   File 
> >
"C:\PYTHON23\Lib\site-packages\win32com\server\policy.py",
line 283,
> > in _Invoke_
> > return self._invoke_(dispid, lcid, wFlags,
args)
> >   File 
> >
"C:\PYTHON23\Lib\site-packages\win32com\server\policy.py",
line 288,
> > in _invoke_
> > return S_OK, -1, self._invokeex_(dispid, lcid,
wFlags, 
> > args, None, None)
> >   File 
> >
"C:\PYTHON23\Lib\site-packages\win32com\server\policy.py",
line 581,
> > in _invokeex_
> > return func(*args)
> >   File
"E:\otsl\testprojects_folder\pythoncom\pyclihh2.py", 
> > line 26, in
> > OnMyEvent
> > print "event!", obj, obj.AProp
> > exceptions.AttributeError: 'PyIDispatch' object
has no 
> > attribute 'AProp'
> > 
> > "QueryInterface" with the target IID gives the
following:
> > 
> > exceptions.TypeError: There is no interface object
registered 
> > that supports
> > this IID
> > 
> > "CastTo" gives this error:
> > 
> > exceptions.ValueError: This object can not be cast
> > 
> > Thanks a lot,
> > -OY
> > 
> > 
> > 
> > 
> > -- 
> >
http://mail.python.org/mailman/listinfo/python-list
> > 
> 



__ 
Do you Yahoo!? 
Yahoo! Mail - Easier than ever with enhanced search. Learn more. 
http://info.mail.yahoo.com/mail_250
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: comment out more than 1 line at once?

[Aaron Bingham]

Riko Wichmann wrote:
Dear all,
is there a way in Python to comment out blocks of code without putting 
a # in front of each line? Somethings like C's

/*
block of code here is commented out
*/
No.  Why do you want this?  Any good programmer's editor will have a 
comment/uncomment feature to make it easy to comment out whole blocks of 
code at a time using the hash character ('#').

Aaron
--
http://mail.python.org/mailman/listinfo/python-list

Re: how to list files with extension .txt in windows?

[Aaron Bingham]

ed wrote:
I have used batch script to do it but it has a lot of issues with
access denied... errors I pretty much given up using batch to do this.
 

If you do not have permission to access the files, switching to a 
different language will not help you.  You need to determine the cause 
of your access errors first.

Aaron
--
http://mail.python.org/mailman/listinfo/python-list

Re: byte code generated under linux ==> bad magic number under windows

[Aaron Bingham]

Philippe C. Martin wrote:
I understand from my reading that a .pyc generated by python anywhere should 
run anywhere else - is that true ?

If I generate 'compile.all' a pyc with python 2.3.3 under Linux, I get a 'bad 
magic number' trying to execute it under windows (2.4).

What am I doing wrong ?
are the pyc plateform dependant ? and if so must I generate one version for 
each version of Linux, windows .. ?
 

.pyc files are platform-independant but are incompatible between major 
Python versions.  You can not use a .pyc file generated with Python 2.3 
with Python 2.4 or vice versa.

Aaron
--
http://mail.python.org/mailman/listinfo/python-list

How to implement autonomous timer (for FRP)?

[Aaron Leung]

Hi everyone,

I want to take a shot at implementing a simple framework for functional
reactive programming in Python.  (It will be loosely based on a paper I
read about a system called FrTime for DrScheme.)  However, I'm not sure
how to go about implementing certain components.

One basic component will be an object representing the current time
(preferably down to, say, 10 ms).  This time object should update
itself autonomously.  Other objects which somehow depend on the time
object will be kept in a dependency graph, and automatically updated as
the time object updates itself.

I'm not sure what's the best way to implement this time object.  I'm
guessing it should run in its own thread, and whenever it updates, it
should update the graph of its dependents in another thread.  But
naturally I'd like the timer to be efficient and precise (again, down
to ~10 ms), and I'm not sure what my options are.  I'd appreciate any
suggestions.

Thanks and regards,
Aaron

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Data problem

[Aaron Bingham]

Jeffrey Maitland wrote:
I am using the MySQLdb module and I am writing to a MySQL database.
My write statement is something like.
cursor.execute(“INSERT INTO Data(Name, X, Y, Z, Data)
   VALUES('%s', '%s', '%s', '%f', '%f', '%f', 
'%s')""" %

   (name, high_x, high_y, high_z, data))
This is not causing your problem, but why not take advantage of the 
DB-API value substitution support, e.g.:

cursor.execute("""INSERT INTO Data(Name, X, Y, Z, Data)
   VALUES(%s, %f, %f, %f, %s)""",
   (name, high_x, high_y, high_z, data))
This will take care of correctly escaping and quoting strings and 
formatting floats for you automatically.

I can write the data to a file as a check and all  variables are showing 
correctly. However when I extract the data from the database the X, Y 
are not coming out with the same number that “should be going” into the 
database but the Z is.  Any Ideas?
My first guess would be that the X and Y fields in the DB are actually 
defined as integers, not floats, but you probably already checked that.

Regards,
--

Aaron Bingham
Application Developer
Cenix BioScience GmbH

--
http://mail.python.org/mailman/listinfo/python-list

Re: Optimisation Hints (dict processing and strings)

[Aaron Bingham]

Peter Hansen <[EMAIL PROTECTED]> writes:

> You've misunderstood the comments about this area.
> String concatenation is *not* "time consuming".
> *Repeated* concatenations *will become very time
> consuming*, along an exponential curve.  That's
> what the discussions about O(n^2) are referring
> to.  

For the record, O(n^2) is /not/ exponential, but polynomial.  A
function with exponential complexity would have, e.g. O(A^n) for some
constant A, which would be /much/ worse than the behavior of repeated
string concatenation.

Regards,

-- 
----
Aaron Bingham
Software Engineer
Cenix BioScience GmbH


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Little Q: how to print a variable's name, not its value?

[Aaron Bingham]

"[EMAIL PROTECTED]" <[EMAIL PROTECTED]> writes:

> Restating:  I'm doing some debugging of some code.  I want to print out
> the value of two variables whose names are known.  Let's call them
> myTime and myPlace.
>
> #debug:
> if self.debug:
>print "myTime = %s, myPlace = %s" % (myTime, myPlace)
>
> Notice that I had to type the variable's name once as an object, and
> once as the string representation of that object, i.e. the object's
> name.
> I wondered whether it might not be possible to do something like this:
> if self.debug:
>print "%s = %s" % ( name(myTime), myTime )
> where 'name' is the method or trick I'm after.

I must admit this is a bit of a sore point with me: this could be
solved so easily with a nice little macro .  For the case where
you only want to print the name and value of a variable (as opposed to
that of an expression), you may find this little function helpful:

def print_variable(name, scope):
print "%s = %r" % (name, scope[name])

You can call it like this:

print_variable('myTime', locals())

Not perfect, because you have to repeat locals() every time, but it
does save the repetition of the variable name.

Printing expressions can also be accomplished in a similar but more
dangerous[*] fashion:

def print_expression(expression, local_scope, global_scope=None):
if global_scope is None:
global_scope = globals()
print "%s = %r" % (expression, 
   eval(expression, local_scope, global_scope))

Example:

print_expression('i+1', locals())

[*] The use of eval here would make allowing user-entered expressions
here very dangerous, but as long as as it is used as a debugging
aid, with the expression as a literal string in the program, there
should be no security issue.  I would never use either of these
functions except as debugging aids anyway.

Regards,

-- 

Aaron Bingham
Software Engineer
Cenix BioScience GmbH


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Little Q: how to print a variable's name, not its value?

[Aaron Bingham]

Stewart Midwinter <[EMAIL PROTECTED]> writes:

[snip]

> Taking this idea a little further, I'd like to build a 'variable
> inspector' for my GUI app, so that I don't have to resort to debug
> statements.  Instead, I could pop open a window that lists every
> variable that has an expected None, string, int or float value, and
> select any one in order to see its value at that time.  I believe your
> method would be useful in that situation as well, no?

For that, I would suggest taking a look at Komodo
(http://www.activestate.com/Products/Komodo/) and its remote debugging
functionality, instead of building your own tool.  There may be other
IDEs that provide equivalent functionality, but I am not familiar with
them.

Regards,

-- 
----
Aaron Bingham
Software Engineer
Cenix BioScience GmbH


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why Python does *SLICING* the way it does??

[Aaron Bingham]

Bernhard Herzog <[EMAIL PROTECTED]> writes:

> There are very good reasons for half-open intervals and starting at 0
> apart from memory organization.  Dijkstra explained this quite well in
> http://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF

Thanks for the excellent link!

-- 
----
Aaron Bingham
Software Engineer
Cenix BioScience GmbH


-- 
http://mail.python.org/mailman/listinfo/python-list

JSON translated into SQL by python

[Aaron G.]

I am new to programming python for JSON to SQL and I was wondering why this 
does not work. All the values for entering the DB are correct. The 
EnterpriseValue data is not entering the database.

 
#collect data from JSON source at Yahoo
url = ["db", "http://y.ahoo.it/wlB89";]
#check all sites
checkstatus(url[]);
#retrieve EnterpriseValue data from yahoo to DB
url = "http://y.ahoo.it/wlB89";
data = helper.openJSON_URL(url)
#store data
curObservation = data["EnterpriseValue"]

#connect to amazon and post data from Yahoo
conn = inti_psql_amazon("db name", "db user", "password", "db source")
query = "CREATE TABLE temp2 (ID int NOT NULL AUTO_INCREMENT, Enterprise_Value 
float, PRIMARY KEY(ID));"
query = "INSERT INTO TABLE temp2 (enterprise) VALUES("+ str(curObservation) 
+");"   
do_query_amazon(conn, query)
close_psql_amazon(conn)
-- 
https://mail.python.org/mailman/listinfo/python-list

In Python 3, how to append a nested dictionary to a shelve file with the added difficulty of using a for loop?

[Aaron Christensen]

Hello,

I am trying to figure out how to populate a shelve file with a nested
dictionary.

These are my requirements:

-Create shelve file called people.db
-Append the shelve file with new people (person_1, person_2, etc.).
-Use a for loop to iterate through 'attributes' so that I do not need to
write out the lengthy code line by line to populate to the shelve file.
-Need to reference shelve file data for future use

Here is the key/value format that I would like to append to the shelve
file.

person_1 = { 'name': 'Bob', 'type': 'employee', 'attributes':
[{'game': 'basketball', 'high score': '100', 'time': '3.34'},
{'game': 'bridge', 'high score': '10', 'time': '30.34'},
{'game': 'foosball', 'high score': '2', 'time': '24'}]
'''
50+ other attributes
'''
}

# Example: s['person_1]['attributes'][2]['time'] would call out '24'.
# 's' is from 's = shelve.open('people')'
I have a dictionary dictPeople.py file (created using pprint() that
contains the information of person_1, etc. And I am extracting only a small
percentage of the data that is needed.

I have tried the following, but I get an invalid key error.

import shelve, dictPeople
s = shelve.open('people')
person = 'person_1'
s[person]['name'] = dictPeople.person_1['name']
s[person]['type'] = dictPeople.person_1['type']
# I need to use this for loop because there are 50+ attributes.
for attribute in range(0, len(dictPeople['attributes'][attribute])):
s[person]['attributes'][attribute]['game'] = \
dictPeople['attributes'][attribute]['game']
s[person]['attributes'][attribute]['high score'] = \
dictPeople['attributes'][attribute]['high score']
s[person]['attributes'][attribute]['time'] = \
dictPeople['attributes'][attribute]['time']
It turns out, I get the key error because I am not allowed to reference a
key/value pair the same way that I can with a dictionary. However, the
crazy thing is that I can call values in the db file using the same exact
format when trying to write.

For example: I can read data from the .db file using:

x = s['person_1']['name']
print(x)
BUT! I cannot write to that .db file using that exact format or I get an
invalid key error. Makes no sense!

s['person_1']['name'] = 'Bob'
# Returns invalid key entry.  Makes no sense.
Therefore, I tried to extract data and populate the db file using the
following:

s[person] = {   'name': dictPeople.person_1['name'],
'type': dictPeople.person_1['type'],
for attribute in range(0, len(dictPeople['attributes'][attribute])):
['game':  dictPeople['attributes'][attribute]['game'],
'high score': dictPeople['attributes'][attribute]['high score'],
'time': dictPeople['attributes'][attribute]['time']]

}
But, this obvously doesn't work because of the for loop. How can I do this?

-I am trying to figure out how to extract data from the dictionary
dictPeople.py file and store it in the people.db file.
-I am trying to add new people to the people.db file as more people become
available.
-I need to use the for loop because of the 50+ attributes I need to add.
-My future steps would be to modify some of the data values that I extract
and used to populate the people.db file, but my first step is to, at least,
extract, and then populate the people.db file.

Any help or guidance is greatly appreciated.  Thank you for your time and
reading my question.

Thanks!
Aaron
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: In Python 3, how to append a nested dictionary to a shelve file with the added difficulty of using a for loop?

[Aaron Christensen]

Hi Peter,

Thanks for the response!  Several things you stated definitely got me
thinking.  I really appreciate the response.  I used what you said and I am
able to accomplish what I needed.

Thanks!
Aaron



On Mon, Dec 21, 2015 at 7:23 PM, Peter Otten <__pete...@web.de> wrote:

> Aaron Christensen wrote:
>
> > Hello,
> >
> > I am trying to figure out how to populate a shelve file with a nested
> > dictionary.
> >
> > These are my requirements:
> >
> > -Create shelve file called people.db
> > -Append the shelve file with new people (person_1, person_2, etc.).
> > -Use a for loop to iterate through 'attributes' so that I do not need to
> > write out the lengthy code line by line to populate to the shelve file.
> > -Need to reference shelve file data for future use
> >
> > Here is the key/value format that I would like to append to the shelve
> > file.
> >
> > person_1 = { 'name': 'Bob', 'type': 'employee', 'attributes':
> > [{'game': 'basketball', 'high score': '100', 'time': '3.34'},
> > {'game': 'bridge', 'high score': '10', 'time': '30.34'},
> > {'game': 'foosball', 'high score': '2', 'time': '24'}]
> > '''
> > 50+ other attributes
> > '''
> > }
> >
> > # Example: s['person_1]['attributes'][2]['time'] would call out '24'.
> > # 's' is from 's = shelve.open('people')'
> > I have a dictionary dictPeople.py file (created using pprint() that
> > contains the information of person_1, etc. And I am extracting only a
> > small percentage of the data that is needed.
> >
> > I have tried the following, but I get an invalid key error.
> >
> > import shelve, dictPeople
> > s = shelve.open('people')
> > person = 'person_1'
> > s[person]['name'] = dictPeople.person_1['name']
> > s[person]['type'] = dictPeople.person_1['type']
> > # I need to use this for loop because there are 50+ attributes.
> > for attribute in range(0, len(dictPeople['attributes'][attribute])):
> > s[person]['attributes'][attribute]['game'] = \
> > dictPeople['attributes'][attribute]['game']
> > s[person]['attributes'][attribute]['high score'] = \
> > dictPeople['attributes'][attribute]['high score']
> > s[person]['attributes'][attribute]['time'] = \
> > dictPeople['attributes'][attribute]['time']
> > It turns out, I get the key error because I am not allowed to reference a
> > key/value pair the same way that I can with a dictionary. However, the
> > crazy thing is that I can call values in the db file using the same exact
> > format when trying to write.
> >
> > For example: I can read data from the .db file using:
> >
> > x = s['person_1']['name']
> > print(x)
> > BUT! I cannot write to that .db file using that exact format or I get an
> > invalid key error. Makes no sense!
> >
> > s['person_1']['name'] = 'Bob'
> > # Returns invalid key entry.  Makes no sense.
> > Therefore, I tried to extract data and populate the db file using the
> > following:
> >
> > s[person] = {   'name': dictPeople.person_1['name'],
> > 'type': dictPeople.person_1['type'],
> > for attribute in range(0, len(dictPeople['attributes'][attribute])):
> > ['game':  dictPeople['attributes'][attribute]['game'],
> > 'high score': dictPeople['attributes'][attribute]['high score'],
> > 'time': dictPeople['attributes'][attribute]['time']]
> >
> > }
> > But, this obvously doesn't work because of the for loop. How can I do
> > this?
> >
> > -I am trying to figure out how to extract data from the dictionary
> > dictPeople.py file and store it in the people.db file.
> > -I am trying to add new people to the people.db file as more people
> become
> > available.
> > -I need to use the for loop because of the 50+ attributes I need to add.
> > -My future steps would be to modify some of the dat

Re: Unable to use python 3.5

[Aaron Christensen]

Wow, you can find cyber bullies just about anywhere...

On Wed, Dec 23, 2015 at 8:00 AM, Chris Warrick  wrote:

> On 23 December 2015 at 07:38, Ankit Deshmukh  wrote:
> > Hi there,
> >
> >
> >
> > I am maters student in India,
>
> We don’t care (expect that you made a typo there).
>
> > I have installed python 3.5 in my windows 10
> > 64bit machine. Everything works fine except package installing. When in
> use
> > “pip install numpy” is shows unable to find *‘vcvarsall.bat’* I don’t
> know
> > how to fix it. I tried several things nothing works.
>
> You clearly haven’t tried enough, as this question is answered by a
> simple Google search. You need to:
>
> (a) install Visual Studio 2015 and configure it; or
> (b) find a binary package, eg. here:
> http://www.lfd.uci.edu/~gohlke/pythonlibs/
>
> A student who wants to work in programming should be able to find
> answers to their questions online. And know better than putting a
> phone number in their e-mail signature for the whole world to see.
>
> --
> Chris Warrick 
> PGP: 5EAAEA16
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Python and multiple user access via super cool fancy website

[Aaron Christensen]

Hi all,

I am not sure if this is the correct venue for my question, but I'd like to
submit my question just in case.  I am not a programmer but I do have an
incredible interest in it, so please excuse my lack of understanding if my
question isn't very thorough.

As an example, a website backend is developed using Python.  Users can
submit their input through the website and PHP (or some other language)
transfers the user input from the website fields to a database such as
MySQL.  There is a main script called main_script.py which extracts the
user data from MySQL, processes it, stores output in MySQL and sends output
to the user (via webpage and email).

About main_script.py
# main_script.py extracts user input from MySQL, processes it, stores
output in MySQL and send output to user (via webpage and email).
# Inputs: User personal information such as age, dob, nationality, hobbies,
and 20 or 30 other fields
# Output: main_script.py is going to do something with it such as access
the database and some shelve files or other py scripts. I have no clue what
it's going to do, but my point is that the processing of the input to
output will take longer than simply a print('Hello, %r!' %user_name).

My question:  I am curious to know how Python handles something like this.
Let's say that there are 10, 20, 50, or even 1000 users accessing the
website.  They all put in their 20 to 30 pieces of input and are waiting on
some fancy magic output.  How exactly does that work?  Can multiple users
access the same script?  Does the Python programmer need to code in a
manner that supports this?  Are requests to the script handled serially or
in parallel?

I've tried some searches, but not getting much except for "appending to the
same file", etc.

I hope my question is a good question.  Thank you for your time!
Aaron
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python and multiple user access via super cool fancy website

[Aaron Christensen]

Hi Jason,

What gives you that idea?
On Dec 25, 2015 12:23 AM, "Jason Friedman"  wrote:

> > I am not sure if this is the correct venue for my question, but I'd like
> to
> > submit my question just in case.  I am not a programmer but I do have an
> > incredible interest in it, so please excuse my lack of understanding if
> my
> > question isn't very thorough.
> >
> > As an example, a website backend is developed using Python.  Users can
> > submit their input through the website and PHP (or some other language)
> > transfers the user input from the website fields to a database such as
> > MySQL.  There is a main script called main_script.py which extracts the
> > user data from MySQL, processes it, stores output in MySQL and sends
> output
> > to the user (via webpage and email).
> >
> > About main_script.py
> > # main_script.py extracts user input from MySQL, processes it, stores
> > output in MySQL and send output to user (via webpage and email).
> > # Inputs: User personal information such as age, dob, nationality,
> hobbies,
> > and 20 or 30 other fields
> > # Output: main_script.py is going to do something with it such as access
> > the database and some shelve files or other py scripts. I have no clue
> what
> > it's going to do, but my point is that the processing of the input to
> > output will take longer than simply a print('Hello, %r!' %user_name).
> >
> > My question:  I am curious to know how Python handles something like
> this.
> > Let's say that there are 10, 20, 50, or even 1000 users accessing the
> > website.  They all put in their 20 to 30 pieces of input and are waiting
> on
> > some fancy magic output.  How exactly does that work?  Can multiple users
> > access the same script?  Does the Python programmer need to code in a
> > manner that supports this?  Are requests to the script handled serially
> or
> > in parallel?
>
> I have a hunch that you do not want to write the program, nor do you
> want to see exactly how a programmer would write it?
>
> The question is more like asking a heart surgeon how she performs
> heart surgery:  you don't plan to do it yourself, but you want a
> general idea of how it is done?  How do you keep the patient from
> bleeding to death?  Does the heart stop while performing the surgery,
> and if yes, how does the patient stay alive?
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python and multiple user access via super cool fancy website

[Aaron Christensen]

Great thank you!  I will look into it. I watched some tutorials on Django
last night. It appears to be somewhat of a bootstrap but for the backend.
I'm not sure if my opinion is correct.
On Dec 25, 2015 5:27 AM,  wrote:

> > On 25 December 2015 at 05:02, Aaron Christensen
> >  wrote:
> >> Hi Chris,
> >>
> >> Thank you for your response and information.  I enjoy doing Python on my
> >> free time so when I get closer to some kind of web application, then I
> can
> >> provide more information.
> >>
> >> Thanks for pointing me in the right direction so far.  I will replace
> any
> >> shelve usage with the database.  I also started looking at WSGI servers
> and
> >> just found a great deal of information on fullstackpython.com.
> >
> > Full Stack Python is a good resource, which teaches many things
> > considered best practices in the Python web world (I personally
> > recommend uWSGI instead of Gunicorn, but that’s mainly just my
> > preference)
> >
> >> I have some experience in PHP (which is why I mentioned it).  I am
> >> unfamiliar with Django, Flask, or Pyramid.  I am looking into Django,
> but am
> >> a little hesitant because of all the time I spent learning PHP.
> >
> > I’m sorry, but you wasted your time. PHP is an awful language with
> > multiple problems and bad design decisions. Here’s some laughing
> > stock:
> >
> > http://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/
> > http://reddit.com/r/lolphp
> > http://phpsadness.com/
> >
> > On the other hand, Python web frameworks are really fun and easy to
> > learn, even though their general design is a bit different from PHP’s:
> >
> > https://docs.djangoproject.com/en/1.9/intro/tutorial01/
> > http://tutorial.djangogirls.org/en/
>
> If you want to get a good overview of what it means to use a python web
> framework, try Miguel Grinberg’s great flask tutorial [1]. It’s based on
> flask, a more lightweight web framework compared to django, but the basic
> principles are the same. He covers about anything you need to know in easy
> to follow lessons. Only basic python knowledge is required.
>
> [1]
> http://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world
>
> my 2 cts,
>
> oneman
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python and multiple user access via super cool fancy website

[Aaron Christensen]

LOL. Thanks!  PHP was definitely not very easy to pick up and I'm still
having some issues. Last night I watched some tutorials on Django and plan
on reading all of the links on the docs page of Django.  I will also look
at your recommendation.  I think that will give me a good understanding.
Hopefully Django isn't a copy/paste kinda "put your website together" like
WordPress because my objective is to actually learn Python.
On Dec 25, 2015 3:59 AM, "Chris Warrick"  wrote:

> [Forwarding your message to the mailing list — please use the Reply
> All (or Reply to List) button in the future]
>
> On 25 December 2015 at 05:02, Aaron Christensen
>  wrote:
> > Hi Chris,
> >
> > Thank you for your response and information.  I enjoy doing Python on my
> > free time so when I get closer to some kind of web application, then I
> can
> > provide more information.
> >
> > Thanks for pointing me in the right direction so far.  I will replace any
> > shelve usage with the database.  I also started looking at WSGI servers
> and
> > just found a great deal of information on fullstackpython.com.
>
> Full Stack Python is a good resource, which teaches many things
> considered best practices in the Python web world (I personally
> recommend uWSGI instead of Gunicorn, but that’s mainly just my
> preference)
>
> > I have some experience in PHP (which is why I mentioned it).  I am
> > unfamiliar with Django, Flask, or Pyramid.  I am looking into Django,
> but am
> > a little hesitant because of all the time I spent learning PHP.
>
> I’m sorry, but you wasted your time. PHP is an awful language with
> multiple problems and bad design decisions. Here’s some laughing
> stock:
>
> http://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/
> http://reddit.com/r/lolphp
> http://phpsadness.com/
>
> On the other hand, Python web frameworks are really fun and easy to
> learn, even though their general design is a bit different from PHP’s:
>
> https://docs.djangoproject.com/en/1.9/intro/tutorial01/
> http://tutorial.djangogirls.org/en/
>
> --
> Chris Warrick <https://chriswarrick.com/>
> PGP: 5EAAEA16
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python and multiple user access via super cool fancy website

[Aaron Christensen]

On Dec 25, 2015 12:22 PM, "Jason Friedman"  wrote:
>
> >> I have a hunch that you do not want to write the program, nor do you
> >> want to see exactly how a programmer would write it?
> >>
> >> The question is more like asking a heart surgeon how she performs
> >> heart surgery:  you don't plan to do it yourself, but you want a
> >> general idea of how it is done?  How do you keep the patient from
> >> bleeding to death?  Does the heart stop while performing the surgery,
> >> and if yes, how does the patient stay alive?
>
> > What gives you that idea?
>
> You said:  I am not a programmer but I do have an incredible interest in
it.
>
> Many people say:  I am not a programmer but I want to become one.
>
> I just wanted to make sure your actual question was being answered.
> Sounds like it is.
>
> Also, the general practice is to put your response at the bottom of your
reply.

Understood.  I say I'm not a programmer because my degrees and day job are
not programing.  Thanks for making sure my question was answered!  My
question was answered but it also made me think of several other different
things.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python and multiple user access via super cool fancy website

[Aaron Christensen]

On Dec 25, 2015 12:38 PM, "Chris Warrick"  wrote:
>
> On 25 December 2015 at 13:15, Aaron Christensen
>  wrote:
> > LOL. Thanks!  PHP was definitely not very easy to pick up and I'm still
> > having some issues. Last night I watched some tutorials on Django and
plan
> > on reading all of the links on the docs page of Django.  I will also
look at
> > your recommendation.  I think that will give me a good understanding.
> > Hopefully Django isn't a copy/paste kinda "put your website together"
like
> > WordPress because my objective is to actually learn Python.
>
> That’s not what WordPress is. WordPress is a blog engine that can be
> used as a general-purpose CMS, full stop. You don’t need any coding
> skills to build a website with WordPress. Many people have done that —
> especially on wordpress.com or shared hosting services with one-click
> WP installers; and even without an installer, setting up WordPress on
> shared hosting requires a FTP client and reading comprehension
> anyways.
>
> On the other hand, Django is nothing like this. Django can do anything
> you tell it to, and you need to write code. While Django handles some
> things for you (eg. the admin panel or the ORM), you still need to
> write models, views, URL configuration, etc. yourself. You need an
> understanding of relational databases, HTTP, HTML/CSS, the template
> engine, and you do need to write actual code.
>
> And while there are tons of ready-made blog applications for Django
> that you can install and use, you can (and should!) write your own in
> an hour or two.  And it’s a lot more fun to do that than lazily
> downloading something.
>
> --
> Chris Warrick <https://chriswarrick.com/>
> PGP: 5EAAEA16
> --
> https://mail.python.org/mailman/listinfo/python-list

That's excellent news. I am looking forward to learning and working with
Django. I imagine that I will have Django related questions in a few weeks
or so.  I appreciate your lengthy responses and explanations!

Thank you,
Aaron
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A newbie's doubt

[Aaron Christensen]

That's an awesome response!
On Jan 7, 2016 6:35 AM, "Chris Angelico"  wrote:

> On Thu, Jan 7, 2016 at 2:20 PM, Henrique Correa  wrote:
> > Is Python's Tutorial (by Guido) a good and complete reference for the
> > language? I mean, after reading it, should I have a good basis on Python?
> >
> > I've came from js and php, and already know the very basics of py.
> >
> > Thank you!
>
> If by "good and complete" you mean "enough to write code in", then
> yes, I would say it is.
>
> If you mean "enough to write applications that you can sell for
> money", then it's probably insufficient; you'll want to also learn a
> few libraries, possibly including third-party ones like Flask/Django
> (to write web applications) or numpy/pandas (to write computational
> code) or matplotlib (to crunch numbers and make graphs).
>
> If, on the other hand, you mean "enough to understand how Python works
> internally", then no, it's not. It's not meant to go into that kind of
> detail. But you don't need to know that anyway.
>
> I would recommend going through that tutorial. You'll get a decent
> handle on how Python works. As a general rule, Python's object model
> is similar to what you'll know from JS; the scoping rules are
> different (instead of "var x;" to declare that x is local, you would
> have "global x" to declare that x is global - but you need declare
> only those globals that you assign to, not those you reference). As
> you go through it, write down some notes of everything that interests
> or confuses you; once you've completed the tutorial, go through your
> notes again. Some of what you've written down will now make perfect
> sense, and you can delete it; some will still confuse you, but you'll
> understand more of *why* it confuses you. So then you come back here
> to python-list with the bits that confuse you, and we'll be happy to
> explain stuff!
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: good python tutorial

[Aaron Christensen]

Thanks for sharing!

On Tue, Feb 23, 2016 at 1:48 AM, Mike S via Python-list <
python-list@python.org> wrote:

> This site was recommended by a friend, it looks really well put together,
> I thought it might be of interest to people considering online tutorials.
>
> http://www.python-course.eu/index.php
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Network Simulator

[Aaron Christensen]

On Feb 23, 2016 9:55 PM, "nikhil amraotkar" 
wrote:
>
> Hi...I need help to design a network simulator consisting for 5 routers
in python...Any help would be appretiated...
> Thanks..
> --
> https://mail.python.org/mailman/listinfo/python-list

What is the purpose for designing it in Python? I'm a network engineer and
use GNS3 and Packet Tracer.  I could see the benefit of creating Python
scripts to manage networking devices but not sure why you would want to use
Python.  I use Python to generate configuration scripts but haven't used it
for any management yet.
-- 
https://mail.python.org/mailman/listinfo/python-list

Psycopg2 to create a record using a FK

[Aaron Christensen]

Hello,

I am running the following versions of software:

Python 3.5
psycopg2==2.6.1
Postgres 9.4.5

I have 2 tables.  Table User has UserId (serial PK), LastName, FirstName,
Gender, DateOfBirth, and DateEnrolled.  Table UserProfile has UserProfileId
(serial, PK), UserId (FK), DateEntered, FaveNumber, and Activity.  There is
a one-to-many relationship.

The following PostgreSQL works and ultimately creates a record in
UserProfile with an associated UserId (FK).

\set last_name '''Sara'''
\set first_name '''Jackson'''
\set gender '''F'''
\set dob '''1941-1-12'''
\set fave_number '''3'''
\set activity '''volleyball'''


WITH ins_user AS (
INSERT INTO User
(LastName, FirstName, Gender, DateOfBirth, DateEnrolled)
VALUES (:last_name, :first_name, :gender, :dob, now())
RETURNING UserId)
INSERT INTO UserProfile
(UserId, DateEntered, FaveNumber, Activity)
VALUES ( (SELECT UserId FROM ins_user), now(), :fave_number :activity);

How can I build a psycopg2 cur.execute query that will accomplish the above
PostgreSQL?  I've read documentation but can't seem to get a handle on how
I should structure this command.

My starting point is:

cur.execute( \
"""INSERT INTO User \
(LastName, FirstName, Gender, DateOfBirth, DateEnrolled) \
VALUES (%s, %s, %s, %s, %s) RETURNING UserId;""", \
(last_name, first_name, gender, date_of_birth, now(), ??...??)


Also, I have a second question.  Is it possible to extract that value
derived from "RETURNING UserId" so that it can be used in a later query?

Thank you for your time!
Aaron
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Psycopg2 to create a record using a FK

[Aaron Christensen]

On Sat, Mar 12, 2016 at 5:26 AM, Peter Otten <__pete...@web.de> wrote:

> Aaron Christensen wrote:
>
> > Hello,
> >
> > I am running the following versions of software:
> >
> > Python 3.5
> > psycopg2==2.6.1
> > Postgres 9.4.5
> >
> > I have 2 tables.  Table User has UserId (serial PK), LastName, FirstName,
> > Gender, DateOfBirth, and DateEnrolled.  Table UserProfile has
> > UserProfileId
> > (serial, PK), UserId (FK), DateEntered, FaveNumber, and Activity.  There
> > is a one-to-many relationship.
> >
> > The following PostgreSQL works and ultimately creates a record in
> > UserProfile with an associated UserId (FK).
> >
> > \set last_name '''Sara'''
> > \set first_name '''Jackson'''
> > \set gender '''F'''
> > \set dob '''1941-1-12'''
> > \set fave_number '''3'''
> > \set activity '''volleyball'''
> >
> >
> > WITH ins_user AS (
> > INSERT INTO User
> > (LastName, FirstName, Gender, DateOfBirth, DateEnrolled)
> > VALUES (:last_name, :first_name, :gender, :dob, now())
> > RETURNING UserId)
> > INSERT INTO UserProfile
> > (UserId, DateEntered, FaveNumber, Activity)
> > VALUES ( (SELECT UserId FROM ins_user), now(), :fave_number :activity);
> >
> > How can I build a psycopg2 cur.execute query that will accomplish the
> > above
> > PostgreSQL?  I've read documentation but can't seem to get a handle on
> how
> > I should structure this command.
>
> I have not tried it, but wouldn't the straight-forward
>
> cur.execute("""
> WITH ins_user AS (
> INSERT INTO User
> (LastName, FirstName, Gender, DateOfBirth, DateEnrolled)
> VALUES (:last_name, :first_name, :gender, :dob, now())
> RETURNING UserId)
> INSERT INTO UserProfile
> (UserId, DateEntered, FaveNumber, Activity)
> VALUES ( (SELECT UserId FROM ins_user), now(), :fave_number :activity);
> """,
> dict(
> first_name="Sara",
> last_name="Jackson",
> gender="F",
> dob=datetime.date(1941, 1, 12),
> fave_number=3,
> activity="volleyball"
> ))
>
> work?
>

Hi Peter,

Yes, thank you.  It turns out that I can pass a dictionary as an argument.
I have combined your response with Dieter's and am running into some issues
now.  I need to figure out how to pass a dictionary AND the returned Id.

>
> > My starting point is:
> >
> > cur.execute( \
> > """INSERT INTO User \
> > (LastName, FirstName, Gender, DateOfBirth, DateEnrolled) \
> > VALUES (%s, %s, %s, %s, %s) RETURNING UserId;""", \
> > (last_name, first_name, gender, date_of_birth, now(), ??...??)
> >
> >
> > Also, I have a second question.  Is it possible to extract that value
> > derived from "RETURNING UserId" so that it can be used in a later query?
> >
> > Thank you for your time!
> > Aaron
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Psycopg2 to create a record using a FK

[Aaron Christensen]

On Sat, Mar 12, 2016 at 5:03 AM, dieter  wrote:

> Aaron Christensen  writes:
> > I am running the following versions of software:
> >
> > Python 3.5
> > psycopg2==2.6.1
> > Postgres 9.4.5
> >
> > I have 2 tables.  Table User has UserId (serial PK), LastName, FirstName,
> > Gender, DateOfBirth, and DateEnrolled.  Table UserProfile has
> UserProfileId
> > (serial, PK), UserId (FK), DateEntered, FaveNumber, and Activity.  There
> is
> > a one-to-many relationship.
> >
> > The following PostgreSQL works and ultimately creates a record in
> > UserProfile with an associated UserId (FK).
> >
> > \set last_name '''Sara'''
> > \set first_name '''Jackson'''
> > \set gender '''F'''
> > \set dob '''1941-1-12'''
> > \set fave_number '''3'''
> > \set activity '''volleyball'''
> >
> >
> > WITH ins_user AS (
> > INSERT INTO User
> > (LastName, FirstName, Gender, DateOfBirth, DateEnrolled)
> > VALUES (:last_name, :first_name, :gender, :dob, now())
> > RETURNING UserId)
> > INSERT INTO UserProfile
> > (UserId, DateEntered, FaveNumber, Activity)
> > VALUES ( (SELECT UserId FROM ins_user), now(), :fave_number :activity);
> >
> > How can I build a psycopg2 cur.execute query that will accomplish the
> above
> > PostgreSQL?  I've read documentation but can't seem to get a handle on
> how
> > I should structure this command.
> >
> > My starting point is:
> >
> > cur.execute( \
> > """INSERT INTO User \
> > (LastName, FirstName, Gender, DateOfBirth, DateEnrolled) \
> > VALUES (%s, %s, %s, %s, %s) RETURNING UserId;""", \
> > (last_name, first_name, gender, date_of_birth, now(), ??...??)
>
> You can add "returning UserId" to this SQL command to get back
> the id of the created user in your Python program. You can
> then use this "UserId" to create the row in your dependent table.
>
> I use it like this in one of my programs:
>
>   cursor.execute("insert into service(...) "
>  "values (...) returning id",
>  (...)
>  )
>   id = cursor.fetchone()[0]
>   cursor.execute(
> "insert into product(..., f_service_id) "
> "values (..., %s) returning id",
> (..., id)
> )
>   id = cursor.fetchone()[0]
>
>
>
> Likely, there is also a way to bind the "UserId" inside SQL (maybe
> via "SET") and use it in a second "INSERT" in the same call
> to "cur.execute". Check the Postgres documentation for this.
>
>
> > Also, I have a second question.  Is it possible to extract that value
> > derived from "RETURNING UserId" so that it can be used in a later query?
>
> Sure -- see above.
>
> --
> https://mail.python.org/mailman/listinfo/python-list


Hi Dieter,

Thanks for the response.  I managed to get it working and also combined it
with Peter's suggestion of passing a dictionary as an argument.  However, I
am trying to figure out how I can get the RETURNING ID to be used in the
next cur.execute().  Here is what I have been working on but have found
many ways for it not to work.  My latest response is that the tuple indices
must be integers or slices.

# Here I initialize the dictionaries.  I will use each dictionary as an
input into each cur.execute()
user_input = dict(
last_name = 'Jackson',
first_name = 'Sara',
gender = 'F',
date_of_birth = '1941-1-12'
)

user_profile_input = dict(
fave_number = 3,
activity = 'volleyball'
)



# Create record in User // cur.execute(query, user_input)
cur.execute("""
INSERT INTO User
(LastName, FirstName, Gender, DateOfBirth)
VALUES (%(last_name)s, %(first_name)s, %(gender)s, %(date_of_birth))
RETURNING UserId""",
user_input)
conn.commit()
UserId = cur.fetchone()[0]#< --   HERE is the UserId
print("UserId = %s" % UserId)

# Create record in UserProfile // cur.execute(query, user_profile_input)
cur.execute("""
INSERT INTO UserProfile
(FaveNumber, Activity, UserId)
VALUES (%(fave_number)s, %(activity)s, %s)< I tried
following your format
RETURNING UserProfileId""",
(user_profile_input, UserId)  # < This is what I'm trying
to figure out..  How to pass the UserId.
)
conn.commit()
UserProfileId = cur.fetchone()[0]
print("UserProfileId = %s" % UserProfileId)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Psycopg2 to create a record using a FK

[Aaron Christensen]

On Sat, Mar 12, 2016 at 9:57 PM, Aaron Christensen <
aaron.christen...@gmail.com> wrote:

>
>
> On Sat, Mar 12, 2016 at 5:03 AM, dieter  wrote:
>
>> Aaron Christensen  writes:
>> > I am running the following versions of software:
>> >
>> > Python 3.5
>> > psycopg2==2.6.1
>> > Postgres 9.4.5
>> >
>> > I have 2 tables.  Table User has UserId (serial PK), LastName,
>> FirstName,
>> > Gender, DateOfBirth, and DateEnrolled.  Table UserProfile has
>> UserProfileId
>> > (serial, PK), UserId (FK), DateEntered, FaveNumber, and Activity.
>> There is
>> > a one-to-many relationship.
>> >
>> > The following PostgreSQL works and ultimately creates a record in
>> > UserProfile with an associated UserId (FK).
>> >
>> > \set last_name '''Sara'''
>> > \set first_name '''Jackson'''
>> > \set gender '''F'''
>> > \set dob '''1941-1-12'''
>> > \set fave_number '''3'''
>> > \set activity '''volleyball'''
>> >
>> >
>> > WITH ins_user AS (
>> > INSERT INTO User
>> > (LastName, FirstName, Gender, DateOfBirth, DateEnrolled)
>> > VALUES (:last_name, :first_name, :gender, :dob, now())
>> > RETURNING UserId)
>> > INSERT INTO UserProfile
>> > (UserId, DateEntered, FaveNumber, Activity)
>> > VALUES ( (SELECT UserId FROM ins_user), now(), :fave_number :activity);
>> >
>> > How can I build a psycopg2 cur.execute query that will accomplish the
>> above
>> > PostgreSQL?  I've read documentation but can't seem to get a handle on
>> how
>> > I should structure this command.
>> >
>> > My starting point is:
>> >
>> > cur.execute( \
>> > """INSERT INTO User \
>> > (LastName, FirstName, Gender, DateOfBirth, DateEnrolled) \
>> > VALUES (%s, %s, %s, %s, %s) RETURNING UserId;""", \
>> > (last_name, first_name, gender, date_of_birth, now(), ??...??)
>>
>> You can add "returning UserId" to this SQL command to get back
>> the id of the created user in your Python program. You can
>> then use this "UserId" to create the row in your dependent table.
>>
>> I use it like this in one of my programs:
>>
>>   cursor.execute("insert into service(...) "
>>  "values (...) returning id",
>>  (...)
>>  )
>>   id = cursor.fetchone()[0]
>>   cursor.execute(
>> "insert into product(..., f_service_id) "
>> "values (..., %s) returning id",
>> (..., id)
>> )
>>   id = cursor.fetchone()[0]
>>
>>
>>
>> Likely, there is also a way to bind the "UserId" inside SQL (maybe
>> via "SET") and use it in a second "INSERT" in the same call
>> to "cur.execute". Check the Postgres documentation for this.
>>
>>
>> > Also, I have a second question.  Is it possible to extract that value
>> > derived from "RETURNING UserId" so that it can be used in a later query?
>>
>> Sure -- see above.
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>
>
> Hi Dieter,
>
> Thanks for the response.  I managed to get it working and also combined it
> with Peter's suggestion of passing a dictionary as an argument.  However, I
> am trying to figure out how I can get the RETURNING ID to be used in the
> next cur.execute().  Here is what I have been working on but have found
> many ways for it not to work.  My latest response is that the tuple indices
> must be integers or slices.
>
> # Here I initialize the dictionaries.  I will use each dictionary as an
> input into each cur.execute()
> user_input = dict(
> last_name = 'Jackson',
> first_name = 'Sara',
> gender = 'F',
> date_of_birth = '1941-1-12'
> )
>
> user_profile_input = dict(
> fave_number = 3,
> activity = 'volleyball'
> )
>
>
>
> # Create record in User // cur.execute(query, user_input)
> cur.execute("""
> INSERT INTO User
> (LastName, FirstName, Gender, DateOfBirth)
> VALUES (%(last_name)s, %(first_name)s, %(gender)s, %(date_of_birth))
> RETURNING UserId""",
> user_input)
> conn.commit()
> UserId = cur.fetchone()[0]#< -

Can I trust downloading Python?

[Aaron Martin]

Hi, I am thinking about getting a software but it requires python, so that
brought up a few questions. Is it safe do download python, and does it come
with spam or advertisements? If it doesn't then should I get the latest
version? I mostly want to know if it is safe to download, because most of
the time downloading free stuff off the internet comes with spam and all
that, so I want to know if I can trust downloading it.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RPC client class?

[Aaron Lav]

In article <[EMAIL PROTECTED]>,
Dick Watson <[EMAIL PROTECTED]> wrote:
>"Fredrik Lundh" <[EMAIL PROTECTED]> wrote in message 
>news:[EMAIL PROTECTED]
>> if the RPC you're talking about is Sun RPC, the whole concept is pretty
>> dated (the original RFCs are from the late eighties).
>
>Just because it's dated doesn't meant it isn't still something that 
>occasionally needs done. Thanks for the advice! 

There's also several packages that I know of that include an ONC RPC
IDL compiler, which you might want to look at if the IDL is complex:
 - my Pinefs package (http://www.panix.com/~asl2/software/Pinefs/),
 - the Michigan CITI pynfs/newpynfs 
(http://www.citi.umich.edu/projects/nfsv4/pynfs/)
 - Wim Lewis's rpcgen.py
(http://people.omnigroup.com/wiml/soft/stale-index.html#python)

Aaron Lav (http://www.pobox.com/~asl2)


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Access from one class to methode of other class

[Aaron Bingham]

Dennis Lee Bieber <[EMAIL PROTECTED]> writes:

> On Thu, 26 May 2005 14:33:45 +0200, VK <"myname"@example.invalid>
> declaimed the following in comp.lang.python:
>
>> Hi, all!
>> 
>> In my programm i have to insert a variable from class 2 to class 1 and I 
>> get error NameError: global name 'd' is not defined. How do I get access 
>> to d.entry.insert() method of class 1
>> 
>> class 1:
>Does Python even allow numeric class names?

Fortunately, no:

>>> class 1:
  File "", line 1
class 1:
  ^
SyntaxError: invalid syntax

-- 

Aaron Bingham
Software Engineer
Cenix BioScience GmbH


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: bool behavior in Python 3000?

[aaron . watters]

On Jul 11, 3:37 am, Rob Wolfe <[EMAIL PROTECTED]> wrote:
>
> But `bools` are usefull in some contexts. Consider this:
>
> >>> 1 == 1
> True
> >>> cmp(1, 1)
> 0
> >>> 1 == 2
> False
> >>> cmp(1, 2)
>
> -1
>
> At first look you can see that `cmp` does not return boolean value
> what not for all newbies is so obvious.

Excellent point!  And as long as we have them I
agree with Alan that the boolean data type should
implement real boolean algebra with respect to +, *, and ~,
for example supporting operator precedence appropriately
(versus using and, or, not) and also correctly
implementing DeMorgan's laws and other property's of
boolean algebra

   ~(a*b) == ~a + ~b

etcetera.

1+True is bad practice and should be an error.

Anything else is false advertising
(and the java community has the patent on that
methodology :c) ).

  -- Aaron Watters

===
Why does a giraffe have such a long neck?
Because its head is so far from its body!

-- 
http://mail.python.org/mailman/listinfo/python-list

renaming an open file in nt like unix?

[aaron . watters]

Hi.  I'm writing an archival system which I'd like to be portable
to Windows.

The system relies on the property of Unix which allows a
process to keep a file open even if another process renames
it while it is open.  Neither process sees any anomaly or
error.

Do the NT file systems support this feature (which I think is
standard for POSIX systems)?

Inquiring minds want to know.  -- Aaron Watters

===
an apple every 8 hours
will keep 3 doctors away.  -- kliban

-- 
http://mail.python.org/mailman/listinfo/python-list