Re: Cannot run a single MySQLdb execute....

[Νίκος Γκρ33κ]

Τη Πέμπτη, 28 Μαρτίου 2013 10:54:55 μ.μ. UTC+2, ο χρήστης Alan Meyer έγραψε:
> On 3/27/2013 11:50 PM, Νίκος Γκρ33κ wrote:
> 
> > I'am about to go nuts with python 3.2.3
> 
> >
> 
> > Do you see somehtign wrong with the following statement?
> 
> >
> 
> > cur.execute( '''SELECT hits FROM counters WHERE url = ?''', (page,) )
> 
> > data = cur.fetchone()
> 
> >
> 
> > because as you can see by visiting my webpage at http://superhost.gr it 
> > produces an error and i dont have  aclue why.
> 
> >
> 
> > Please help. i'am using MySQLdb
> 
> >
> 
> Nikos,
> 
> 
> 
> When I try to connect to that web page I see the following error message:
> 
> 
> 
> "ImportError: No module named pymysql "
> 
> 
> 
> If that's what you're getting, there's nothing wrong with your SQL or 
> 
> your cur.execute statement.  The problem is that the web server is not 
> 
> finding the pymysql module.
> 
> 
> 
> Is pymysql installed on the computer that is running your application? 
> 
> Can the web server module find it?
> 
> 
> 
> I must be missing something because, if that's the problem, your object 
> 
> named "cur" could not have been created successfully.  Maybe what I'm 
> 
> seeing is a new problem?
> 
> 
> 
>  Alan

Hello Ala, well nto this is not the error, i uses to had MySQLdb as a conenctor 
but even withat that i still see a blank webpage without no errros. Then i had 
a suspision that perhaps MySQLdb isnt't supported in Python 3.2.3 so i decided 
to try pymysql but it wasnt there.

I asked at hstgator support to install that module for me since iam not 
familiar with linux very much or pip but i got this response just now.


Click here to rate this response.
>
> Hello again,
>
> Per your request, a virtual environment has been created for you. You can 
> find local Python binaries in:
>
> /home/nikos/bin
>
> Unfortunately the pymysql module cannot be installed using pip or 
> easy_install and must be installed manually. There is no indication that this 
> module in any way requires Python 3. Is there a particular reason you must 
> use Python 3 instead of a version which supports the modules that you need? 
> Given that the modules you need are better supported by earlier versions of 
> Python, it's not clear to me why we're trying to use Python 3 instead.
>
> If you would still like us to proceed with manually installing the pymysql 
> module for the local version of Python under the nikos account, we will be 
> happy to give it a try but there will be a one-time charge of $35.00 for the 
> installation. I would recommend using an earlier version of Python which is 
> compatible with the modules that you need, but it is certainly up to you. 
> Please let us know how you would like to proceed. 

So, just to make sure that MySQLdb isnt causeing the probkem can someone, 
perhaps you that is familiar with linxu conenct to my jailed shell account and 
install manually the 'pymysql' module because i dont know how to do it and i 
cannot afford to pay ath linux admin because iam unemployed.

please i can providr ou with user and pass for my jailed shell access for 
someone ti install it manually.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Doing both regex match and assignment within a If loop?

[Peter Otten]

Victor Hooi wrote:

> Hi,
> 
> I have logline that I need to test against multiple regexes. E.g.:
> 
> import re
> 
> expression1 = re.compile(r'')
> expression2 = re.compile(r'')
> 
> with open('log.txt') as f:
> for line in f:
> if expression1.match(line):
> # Do something - extract fields from line.
> elif expression2.match(line):
> # Do something else - extract fields from line.
> else:
> # Oh noes! Raise exception.
> 
> However, in the "Do something" section - I need access to the match object
> itself, so that I can strip out certain fields from the line.
> 
> Is it possible to somehow test for a match, as well as do assignment of
> the re match object to a variable?
> 
> if expression1.match(line) = results:
> results.groupsdict()...
> 
> Obviously the above won't work - however, is there a Pythonic way to
> tackle this?
> 
> What I'm trying to avoid is this:
> 
> if expression1.match(line):
> results = expression1.match(line)
> 
> which I assume would call the regex match against the line twice - and
> when I'm dealing with a huge amount of log lines, slow things down.

(1)
for line in f:
match = expression1.match(line)
if match:
# ...
continue
match = expression2.match(line)
if match:
# ...
continue
raise NothingMatches

(2)
import re

class Matcher:
def __call__(self, expr, line):
result = self.match = expr.match(line)
return result
def __getattr__(self, name):
return getattr(self.match, name)

match = Matcher()

for line in f:
if match(expression1, line):
print(match.groupdict())
elif match(expression2, line):
print(match.group(1))
else:
raise NothingMatches


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Doing both regex match and assignment within a If loop?

[Alain Ketterlin]

Victor Hooi  writes:

> expression1 = re.compile(r'')
> expression2 = re.compile(r'')
[...]

Just a quick remark: regular expressions are pretty powerful at
representing alternatives. You could just stick everything inside a
single re, as in '...|...'

Then use the returned match to check which alternative was recognized
(make sure you have at least one group in each alternative).

> Is it possible to somehow test for a match, as well as do assignment
> of the re match object to a variable?

Yes, use '...(...)...' and MatchObject.group(). See the other messages.

-- Alain.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Lazy evaluated

[Habibutsu]

On 03/29/2013 05:40 AM, Steven D'Aprano wrote:

On Thu, 28 Mar 2013 22:37:47 +0300, Habibutsu wrote:


For example, we have  following code:

01|def foo():
02|return 1
03|
04|value = foo()
05|
06|if value == 1:
07|print value,"- equal 1"
08|
09|if isinstance(value, int):
10|print value,"- is int"
11|else:
12|print value,"- is not int"

Task is to create lazy evaluation for function 'foo'. For decision this
task we create special function 'lazy' which turn original function into
a lazy evaluated function by means of creating proxy object that
evaluated value if needed. We add following code:


01|def lazy(func):
02|
03|class ProxyLazy(object):
04|_result_value = None
05|
06|@property
07|def result_value(self):

[...]


It is often useful for the reader to copy and paste code blocks like this
into the interactive interpreter. It is more friendly and useful if you
paste it in a format that makes that easy (no prompts, no line numbers,
no blank lines inside classes or functions). Otherwise the reader has to
copy your code, paste it into an editor, strip out the line numbers and
blank lines, copy and paste it into the interpreter, and THEN they can
finally test your code and see what it does.

Previously I not to do it, yesterday decide to experiment. It seemed to 
me more readable if to see in the email-clent. Sorry, I will keep in 
mind in the future.

30|lazy_foo = lazy(foo)
31|value = lazy_foo()

Unfortunately, this method not allow to use 'isinstance' for check type.

Correct. And so it should not, because it value is not an int, it is a
ProxyLazy object. You cannot use a ProxyLazy object where an int is
expected:

py> value + 1
Traceback (most recent call last):
   File "", line 1, in 
TypeError: unsupported operand type(s) for +: 'ProxyLazy' and 'int'


Why not?

def __add__(self, other):
return self._result_value + other

and then
>>> print value + 1
2


so it is inappropriate to claim that value is an int.


I can overload all arithmetic operators and get almost really int value, 
but I don't like to write many trivial code.



I must admit, I don't understand the value of this "lazy proxy" type you
have created. You get a proxy like this:

value = lazy(foo)()  # get a lazy proxy
# much later
value = value.func()  # now safe to use as an int
print(value + 1)

What benefit does the lazy proxy give? Why not just do this?
Because some people like to use "list comprehensions" and other not, 
some people like labmda and other not. Advantage is that I can write 
more expressive code and use different strategies for different cases. 
"value = value.func()" is looks sad while I just need the value.


value = foo()  # functions are already a lazy proxy
# much later
value = value()  # now safe to use as an int
print(value + 1)


[...]

And everything seems to work, but appear other questions. If original
function return different types - what to do in this case? Where i am
wrong? What other way to do that. Was no idea to create keyword 'lazy'
in Python?

Why should it be a keyword? Python has very few keywords, and most of
them are for programming flow control, like "if", "else", "for", "while",
"pass", etc.

Question not in why I can't use exist tools - I can. If i can write code 
without classes that it does not mean that classes need to delete from 
language. Question why in language not present this ability?


Thanks
--
http://mail.python.org/mailman/listinfo/python-list

Re: Doing both regex match and assignment within a If loop?

[Arnaud Delobelle]

On Friday, 29 March 2013, Alain Ketterlin wrote:

> Victor Hooi > writes:
>
> > expression1 = re.compile(r'')
> > expression2 = re.compile(r'')
> [...]
>
> Just a quick remark: regular expressions are pretty powerful at
> representing alternatives. You could just stick everything inside a
> single re, as in '...|...'
>
>
Then use the returned match to check which alternative was recognized
> (make sure you have at least one group in each alternative).
>
>
Yes, and for extra ease/clarity you can name these alternatives (
'(?Ppattern)').  Then you can do

if m.group('case1'):
...
elif m.group('case2'):
   ...

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: No errors displayed but i blank scren nstead.

[Jean-Michel Pichavant]

 
> I am trying my best with the little knowledge i have and i expect no
> help from you. You are more inclinded to criticize that to actually
> help. And if i pay someone that certainly not gonna be you.
> 
> And i told you about gethostbyaddr, tht its not an issue its because
> the script bein run form cmd that canot get hold of an address via
> browser tis ok.
> 
> something else is wrong here and the page is displayed blank.
> --
> http://mail.python.org/mailman/listinfo/python-list
> 

This is a traceback

Traceback (most recent call last):
  File "metrites.py", line 28, in 
host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
  File "/opt/python3/lib/python3.2/os.py", line 450, in __getitem__
value = self._data[self.encodekey(key)]
KeyError: b'REMOTE_ADDR'


It tells you that variable REMOTE_ADDR does not exist on your server 
environment.

But Chris already told you that.

JM


-- IMPORTANT NOTICE: 

The contents of this email and any attachments are confidential and may also be 
privileged. If you are not the intended recipient, please notify the sender 
immediately and do not disclose the contents to any other person, use it for 
any purpose, or store or copy the information in any medium. Thank you.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sudoku

[Chris Angelico]

On Fri, Mar 29, 2013 at 9:11 AM, Eric Parry  wrote:
> Thank you for that explanation.
> No, I do not understand recursion. It is missing from my Python manual. I 
> would be pleased to receive further explanation from anyone.

If you already know what recursion is, just remember the answer.
Otherwise, find someone who is standing closer to Douglas Hofstadter
than you are; then ask him or her what recursion is.

:)

Recursion is a form of self-referential code. Take this simple, and
rather silly, means of calculating the sum of numbers in a list (like
the sum() function):

# The sum of numbers in an empty list is 0.
# Otherwise it is the first number plus the sum of the rest of the list.
def list_sum(lst):
if not lst: return 0
return lst[0] + list_sum(lst[1:])

>>> list_sum([1,2,3,4,5,6])
21

Note how the function calls itself - but not always. That's critical
to recursion - a termination condition. In this case, it's quite
obvious that the list will eventually have nothing left in it, so the
function will terminate. Sometimes it's less obvious. Sometimes a bug
results in infinite recursion... and:

RuntimeError: maximum recursion depth exceeded in comparison

Hope that helps!

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: No errors displayed but i blank scren nstead.

[Joel Goldstick]

On Thu, Mar 28, 2013 at 3:05 PM, Νίκος Γκρ33κ  wrote:

> Well i dont like people taking to me this way espceially when iam tryign 2
> days for something and thats changing from 2.6 => 3.2.3
>
> I follow advice as long as i can understand whats being said to me.
>
> So if someone wants to help by asking me to try things please do.
> --
> http://mail.python.org/mailman/listinfo/python-list
>

I think you took an unwise approach to switch from 2.6 to 3.2.3 on a live
server.  I haven't tried 3.x yet, but I have seen lots of articles and
videos about what is involved in moving that way.  The unicode stuff seems
to be a big change.  Another issue is whether third party modules that work
for 2.x are available for 3.x.  Following along with your questions it
appears you just changed interpreters and then watched things break.  I
suggest that you go back to study what is different about 3.x from 2.x.
When you have a good understanding of that, go through your 2.x code and
identify all of the areas that seem likely not to work.  Then write some
small test programs in 3.x to see if you can make the various functions
work.  You will learn a lot, and when you have problems you will be able to
come back here and ask more informed questions, and likely will get more
willing help.

good luck



-- 
Joel Goldstick
http://joelgoldstick.com
-- 
http://mail.python.org/mailman/listinfo/python-list

trying to open index.py using Indico software......

[Avnesh Shakya]

hi, 
   please help me...
   I have installed indico software and apache2,when i try to run it using 
http://indico/index.py,then it's downloading index.py, when i put http://indico,
then it's showing all thing properly,but it's not showing index.py but 
downloading it it's not showing any error, even i have tried error.log..

Please help me, I m unable to get this thing
Thanks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: From Perl to Python: restructuring a HPC workflow

[rusi]

On Mar 27, 5:58 pm, Chris Angelico  wrote:
> On Wed, Mar 27, 2013 at 10:29 PM, neurino  wrote:
> > We are a small group of people (approx. 10), working separetely on their own
> > projects (each employee manages approx. 2-3 projects). We deal with high
> > loads of data everyday.
>
> > This workflow has been flawless now for at least 15 years. New generations
> > of employees have been given Perl scripts and they developed the tools
> > further.
>
> I would recommend making sure the tools can all interoperate
> regardless of language, and then you can change any one at any time.
> Chances are that's already the case - working with stdin/stdout is one
> of the easiest ways to do that, for instance. With a structure that
> lets anyone use any language, you can then switch some of your things
> to Python, and demonstrate the readability advantages (which would you
> rather code in, pseudocode or line noise?). Make the switch as smooth
> as possible, and people will take it when it feels right.
>
> ChrisA

What Chris says is fine in the technical sphere.
It seems to me however that your problems are as much human as
technical -- convincing entrenched old fogeys to change.
No I dont have any cooked answers for that… You just need to keep your
eyes and ears open to see where you want a smooth painless transition
and when you want to 'do it with a bang.'

If you look at some of the stuff here
http://blog.explainmydata.com/2012/07/expensive-lessons-in-python-performance.html
you may find that these packages do much of what you want (And add
matplotlib to the set)
This may add some pizzazz to your case.

Warning: In my experience, this can often backfire!
-- 
http://mail.python.org/mailman/listinfo/python-list

HTTPConnection.send

[dspublic]

Hi!

I have a problem with HTTPConnection object send() method (pyver3.3.1). I want 
to send data from file-like object with HTTPConnection.send( f ), and I get a 
"data should be a bytes-like object or an iterable, ..." exception. I have 
investigated a send method, and discovered a problem: if data has a read 
attribute, send it , and try send it again with self.sock.sendall(data). My 
opinion is need an "else" after the "if hasattr(data, "read")"

Please, somebody help me. Does it a real BUG or my mistake?

http.client.py >

if hasattr(data, "read") :
if self.debuglevel > 0:
print("sendIng a read()able")
encode = False
try:
mode = data.mode
except AttributeError:
# io.BytesIO and other file-like objects don't have a `mode`
# attribute.
pass
else:
if "b" not in mode:
encode = True
if self.debuglevel > 0:
print("encoding file using iso-8859-1")
while 1:
datablock = data.read(blocksize)
if not datablock:
break
if encode:
datablock = datablock.encode("iso-8859-1")
self.sock.sendall(datablock)
ELSE: # i guess missing 
try:
self.sock.sendall(data)
except TypeError:
if isinstance(data, collections.Iterable):
  for d in data:
  self.sock.sendall(d)
  else:
  raise TypeError("data should be a bytes-like object "
"or an iterable, got %r" % type(data))
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Doing both regex match and assignment within a If loop?

[Neil Cerutti]

On 2013-03-29, Alain Ketterlin  wrote:
> Victor Hooi  writes:
>
>> expression1 = re.compile(r'')
>> expression2 = re.compile(r'')
> [...]
>
> Just a quick remark: regular expressions are pretty powerful at
> representing alternatives. You could just stick everything
> inside a single re, as in '...|...'
>
> Then use the returned match to check which alternative was
> recognized (make sure you have at least one group in each
> alternative).

Yes, but in a Python program it's more straightforward to program
in Python. ;)

But this is from a grade A regex avoider, so take it with a small
chunk of sodium.

>> Is it possible to somehow test for a match, as well as do assignment
>> of the re match object to a variable?

One way to attack this problem that's not yet been explicitly
mentioned is to match using a generator function:

def match_each(s, re_seq):
   for r in re_seq:
   yield r.match(s)

And later something like:

for match in match_each(s, (expression1, expression2, expression3)):
if match:
print(match.groups()) # etc...

-- 
Neil Cerutti
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: HTTPConnection.send

[dspublic]

Problematical python versions: 3.2+ (2.x, 3.0, 3.1 ok)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Small program ideas

[Neil Cerutti]

On 2013-03-28, PMT  wrote:
> Em s?bado, 16 de fevereiro de 2013 03h22min41s UTC, eli m  escreveu:
>> Any small program ideas? I would prefer to stick to command
>> line ones. Thanks.
>
> What about this one? 
>
> Do you know how to do the elevator simulation?

Cribbed from an early chapter of the excellent computer science
manual, _Simply Scheme_: A command line program to convert a
sentence into Pig-Latin.

C:\> piglatin.py Look in the bag!
Ooklay in the agbay!

Start with a function that converts a single word to Pig-Latin,
and work your way up to the final program in small steps.

-- 
Neil Cerutti
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Screencast: Creating and deploying an advanced python web application in under 15 minutes!

[timothy crosley]

I apologize for the audio from the original screen cast, it was really sub-par. 
I bought a new microphone and re-recorded it: 
http://www.youtube.com/watch?v=0L8TsmrZPLg&feature=youtu.be

Thanks!

Timothy

On Tuesday, March 26, 2013 4:54:15 AM UTC-4, timothy crosley wrote:
> Hi,
> 
> 
> 
> I've created a screen cast showing how a message board with live-validation 
> and Ajax calls written in python can be built and deployed in under 15 
> minutes. You can view it here: http://www.youtube.com/watch?v=ucougrZK9wI
> 
> 
> 
> I hope some of you find it useful,
> 
> 
> 
> Thanks!
> 
> 
> 
> Timothy
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Doing both regex match and assignment within a If loop?

[Mitya Sirenef]

On 03/29/2013 04:27 AM, Peter Otten wrote:

(2)

> import re
>
> class Matcher:
> def __call__(self, expr, line):
> result = self.match = expr.match(line)
> return result
> def __getattr__(self, name):
> return getattr(self.match, name)


Perhaps it's a little simpler to do this?


self.match =  expr.match(line)

> return self.match


 -m


--
Lark's Tongue Guide to Python: http://lightbird.net/larks/

Frisbeetarianism is the belief that when you die, your soul goes up on
the roof and gets stuck.  George Carlin

--
http://mail.python.org/mailman/listinfo/python-list

Re: Screencast: Creating and deploying an advanced python web application in under 15 minutes!

[Karim]

On 29/03/2013 14:53, timothy crosley wrote:

I apologize for the audio from the original screen cast, it was really sub-par. I 
bought a new microphone and re-recorded it: 
http://www.youtube.com/watch?v=0L8TsmrZPLg&feature=youtu.be

Thanks!

Timothy

On Tuesday, March 26, 2013 4:54:15 AM UTC-4, timothy crosley wrote:

Hi,



I've created a screen cast showing how a message board with live-validation and 
Ajax calls written in python can be built and deployed in under 15 minutes. You 
can view it here: http://www.youtube.com/watch?v=ucougrZK9wI



I hope some of you find it useful,



Thanks!



Timothy

Hi Timothy,

Very interesting!

Thx a lot!
Cheers
Karim
--
http://mail.python.org/mailman/listinfo/python-list

How to find bad row with db api executemany()?

[Roy Smith]

I'm inserting a gazillion rows into a MySQL database using MySQLdb and 
cursor.executemany() for efficiency.  Every once in a while, I get a row which 
violates some kind of database constraint and raises Error.

I can catch the exception, but don't see any way to tell which row caused the 
problem.  Is this information obtainable, short of retrying each row one by one?

---
Roy Smith
r...@panix.com


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: flaming vs accuracy [was Re: Performance of int/long in Python 3]

[Grant Edwards]

On 2013-03-28, Ethan Furman  wrote:

> I cannot speak for the borg mind, but for myself a troll is anyone
> who continually posts rants (such as RR & XL) or who continuously
> hijacks threads to talk about their pet peeve (such as jmf).

Assuming jmf actually does care deeply and genuinely about Unicode
implementations, and his postings reflect his actual position/opinion,
then he's not a troll.  Traditionally, a troll is someone who posts
statements purely to provoke a response -- they don't really care
about the topic and often don't believe what they're posting.

-- 
Grant Edwards   grant.b.edwardsYow! BARBARA STANWYCK makes
  at   me nervous!!
  gmail.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: HTTPConnection.send

[Peter Otten]

dspub...@freemail.hu wrote:

> I have a problem with HTTPConnection object send() method (pyver3.3.1). I
> want to send data from file-like object with HTTPConnection.send( f ), and
> I get a "data should be a bytes-like object or an iterable, ..."
> exception. I have investigated a send method, and discovered a problem: if
> data has a read attribute, send it , and try send it again with
> self.sock.sendall(data). My opinion is need an "else" after the "if
> hasattr(data, "read")"
> 
> Please, somebody help me. Does it a real BUG or my mistake?

I think your analysis is correct. Please file a bug report on 
.
 
> http.client.py >
> 
> if hasattr(data, "read") :
> if self.debuglevel > 0:
> print("sendIng a read()able")
> encode = False
> try:
> mode = data.mode
> except AttributeError:
> # io.BytesIO and other file-like objects don't have a
> # `mode` attribute.
> pass
> else:
> if "b" not in mode:
> encode = True
> if self.debuglevel > 0:
> print("encoding file using iso-8859-1")
> while 1:
> datablock = data.read(blocksize)
> if not datablock:
> break
> if encode:
> datablock = datablock.encode("iso-8859-1")
> self.sock.sendall(datablock)
> ELSE: # i guess missing 
> try:
> self.sock.sendall(data)
> except TypeError:
> if isinstance(data, collections.Iterable):
>   for d in data:
>   self.sock.sendall(d)
>   else:
>   raise TypeError("data should be a bytes-like object "
> "or an iterable, got %r" % type(data))

-- 
http://mail.python.org/mailman/listinfo/python-list

Interlacing of images

[hesoyam16]

Hi,
I am new to python , and working on some basic stuff. I have separated the even 
and odd rows of a image into two separate images, now i want to combine them 
back, is there a simple way to do that using numpy or opencv commands or do I 
have to use a for loop. 

Thanks,
Eric
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: HTTPConnection.send

[Chris Angelico]

On Fri, Mar 29, 2013 at 11:27 PM,   wrote:
> I have a problem with HTTPConnection object send() method (pyver3.3.1). I 
> want to send data from file-like object with HTTPConnection.send( f ), and I 
> get a "data should be a bytes-like object or an iterable, ..." exception. I 
> have investigated a send method, and discovered a problem: if data has a read 
> attribute, send it , and try send it again with self.sock.sendall(data). My 
> opinion is need an "else" after the "if hasattr(data, "read")"
>
> Please, somebody help me. Does it a real BUG or my mistake?

Yeah, I think you may be right on that. Changeset 67046 added the try
block, and removed the else. I'd raise this on the tracker; I'd say
the removal of else was purely accidental.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Interlacing of images

[Mark Lawrence]

On 29/03/2013 14:52, hesoya...@gmail.com wrote:

Hi,
I am new to python , and working on some basic stuff. I have separated the even 
and odd rows of a image into two separate images, now i want to combine them 
back, is there a simple way to do that using numpy or opencv commands or do I 
have to use a for loop.

Thanks,
Eric



Use the builtin zip function?

--
If you're using GoogleCrap™ please read this 
http://wiki.python.org/moin/GoogleGroupsPython.


Mark Lawrence

--
http://mail.python.org/mailman/listinfo/python-list

Re: flaming vs accuracy [was Re: Performance of int/long in Python 3]

[Ethan Furman]

On 03/29/2013 07:52 AM, Grant Edwards wrote:

On 2013-03-28, Ethan Furman  wrote:


I cannot speak for the borg mind, but for myself a troll is anyone
who continually posts rants (such as RR & XL) or who continuously
hijacks threads to talk about their pet peeve (such as jmf).


Assuming jmf actually does care deeply and genuinely about Unicode
implementations, and his postings reflect his actual position/opinion,
then he's not a troll.  Traditionally, a troll is someone who posts
statements purely to provoke a response -- they don't really care
about the topic and often don't believe what they're posting.


Even if he does care deeply and genuinely he still hijacks threads, still refuses the challenges to try X or Y and 
report back, and (ISTM) still refuses to learn.


If that's not trollish behavior, what is it?

FWIW I don't think he does care deeply and genuinely (at least not genuinely) or he would do more than whine about micro 
benchmarks and make sweeping statements like "nobody here understands unicode" (paraphrased).


--
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: HTTPConnection.send

[dspublic]

Thanx for confirmations...

I have reported http://bugs.python.org/issue17575 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: flaming vs accuracy [was Re: Performance of int/long in Python 3]

[Grant Edwards]

On 2013-03-29, Ethan Furman  wrote:
> On 03/29/2013 07:52 AM, Grant Edwards wrote:
>> On 2013-03-28, Ethan Furman  wrote:
>>
>>> I cannot speak for the borg mind, but for myself a troll is anyone
>>> who continually posts rants (such as RR & XL) or who continuously
>>> hijacks threads to talk about their pet peeve (such as jmf).
>>
>> Assuming jmf actually does care deeply and genuinely about Unicode
>> implementations, and his postings reflect his actual
>> position/opinion, then he's not a troll.  Traditionally, a troll is
>> someone who posts statements purely to provoke a response -- they
>> don't really care about the topic and often don't believe what
>> they're posting.
>
> Even if he does care deeply and genuinely he still hijacks threads,
> still refuses the challenges to try X or Y and report back, and
> (ISTM) still refuses to learn.
>
> If that's not trollish behavior, what is it?

He might indeed be trolling.  But what defines a troll is
motive/intent, not behavior.  Those behaviors are all common in
non-troll net.kooks.  Maybe I'm being a bit too "old-school Usenet",
but being rude, ignorant (even stubbornly so), wrong, or irrational
doesn't make you a troll.  What makes you a troll is intent.  If you
don't actually care about the topic but are posting because you enjoy
poking people with a stick to watch them jump and howl, then you're a
troll.

> FWIW I don't think he does care deeply and genuinely (at least not
> genuinely) or he would do more than whine about micro benchmarks and
> make sweeping statements like "nobody here understands unicode"
> (paraphrased).

Perhaps he doesn't care about Unicode or Python performance.  If so
he's putting on a pretty good act -- if he's a troll, he's a good one
and he's running a long game.  Personally, I don't think he's a troll.
I think he's obsessed with what he percieves as an issue with Python's
string implementation.  IOW, if he's a troll, he's got me fooled.

-- 
Grant Edwards   grant.b.edwardsYow! It's a hole all the
  at   way to downtown Burbank!
  gmail.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Cannot run a single MySQLdb execute....

[Alan Meyer]

On 03/29/2013 03:40 AM, Νίκος Γκρ33κ wrote:
...

So, just to make sure that MySQLdb isnt causeing the probkem can
someone, perhaps you that is familiar with linxu conenct to my jailed
shell account and install manually the 'pymysql' module because i
dont know how to do it and i cannot afford to pay ath linux admin
because iam unemployed.

please i can providr ou with user and pass for my jailed shell access
for someone ti install it manually.


Sorry Nikos, I don't have time to do anything like that.  I'm going nuts 
with my own programming problems and deadlines.


However, MySQLdb is a well established module and what you're asking it 
to do is very simple and very standard.


I can think of several obvious possibilities for you to pursue.

The first one is, are you successfully connecting to the database at 
all?  I suspect that you are not.  You may have an error in your 
connection string - the host, userid, port number, database name, or 
password.  That's the most likely problem.


See if you can find some other program on your server that does 
successfully connect and look at the connection parameters.  Be sure 
that you're checking the return value from your connect() call.


If everything looks good, check to see if you have rights to the 
database and table you are trying to select from.


Good luck.

Alan
--
http://mail.python.org/mailman/listinfo/python-list

Export data from python to a txt file

[Ana Dionísio]

Hello!!!

I have this lists a=[1,3,5,6,10], b=[a,t,q,r,s] and I need to export it to a 
txt file and I can't use csv.

And I want the next format:

a 1 3 5 6 10
b a t q r s

I already have this code:

"f = open("test.txt", 'w')
 f.write("a")
 f.write("\n")
 f.write("b")
 f.write("\n")

 for i in xrange(len(a)):
LDFile.write("\t")
LDFile.write(str(a[i]))
LDFile.write("\t")
LDFile.write(str(b[i]))

 f.close()"

But it doesn't have the format I want. Can you help?

Thanks!

 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Cannot run a single MySQLdb execute....

[Alan Meyer]

On 03/29/2013 01:32 PM, Alan Meyer wrote:


However, MySQLdb is a well established module and what you're asking it
to do is very simple and very standard.


Oh, sorry, I see that you already said that mysqldb won't work with 
python 3.  My comments in the last message are irrelevant.


Sorry again.

Good luck.

Alan

--
http://mail.python.org/mailman/listinfo/python-list

Re: Export data from python to a txt file

[Jason Swails]

On Fri, Mar 29, 2013 at 1:33 PM, Ana Dionísio wrote:

> Hello!!!
>
> I have this lists a=[1,3,5,6,10], b=[a,t,q,r,s] and I need to export it to
> a txt file and I can't use csv.
>

It would help if you showed exactly what you had in your program or in the
Python interpreter.  For instance

a = [1, 3, 5, 6, 10]
b = [a, t, q, r, s]

won't work, since it interprets the letters in the b-list as variables
rather than strings.  It would need to be

b = ['a', 't', 'q', 'r', 's']

or something equivalent.  Assuming you had that part correct:


> And I want the next format:
>
> a 1 3 5 6 10
> b a t q r s
>
> I already have this code:
>
> "f = open("test.txt", 'w')
>  f.write("a")
>  f.write("\n")
>  f.write("b")
>  f.write("\n")
>
>  for i in xrange(len(a)):
> LDFile.write("\t")
> LDFile.write(str(a[i]))
> LDFile.write("\t")
> LDFile.write(str(b[i]))
>
>  f.close()"
>
> But it doesn't have the format I want. Can you help?
>

Walk through exactly what the computer is doing here.  First it prints a,
then a newline, then b, then a newline:

--begin--
a
b
--end--

Then your for loop executes (notice LDFile and f are different file
objects, so they won't write to the same location).  This will print a tab,
followed by the i'th element of a, followed by another tab, followed by the
i'th element of b.  So this:

a 1 3 5 6 10
b a t q r s

--begin-- [tab]  1 [tab]  a [tab]   3 [tab]   t [tab]   5 [tab]   q [tab]
6 [tab]   r [tab]   10 [tab]   s [tab]--end--

So the final file will look something like this:

a
b
 1 a 3 t 5 q 6 r 10 s


which is obviously not what you want.  The following code will do what you
want (although there are ways of doing this in less code, this is the most
straightforward):

f.write('a\t')
for element in a:
   f.write(str(element) + '\t')
f.write('\n')
f.write('b\t')
for element in b:
   f.write(str(element) + '\t')
f.write('\n')

HTH,
Jason
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Export data from python to a txt file

[Mark Lawrence]

On 29/03/2013 17:33, Ana Dionísio wrote:

Hello!!!

I have this lists a=[1,3,5,6,10], b=[a,t,q,r,s] and I need to export it to a 
txt file and I can't use csv.

And I want the next format:

a 1 3 5 6 10
b a t q r s

I already have this code:

"f = open("test.txt", 'w')
  f.write("a")


You'll have to write your data here.


  f.write("\n")


And here.


  f.write("b")
  f.write("\n")

  for i in xrange(len(a)):
 LDFile.write("\t")
 LDFile.write(str(a[i]))
 LDFile.write("\t")
 LDFile.write(str(b[i]))


You rarely need a loop like this in Python.
for x in a:
  doSomething
is the usual way of writing this.  But you don't need this anyway.  Just 
use the string join method as you can't use csv, which tells me it's 
homework, so I'll leave you to it :)




  f.close()"

But it doesn't have the format I want. Can you help?

Thanks!



--
If you're using GoogleCrap™ please read this 
http://wiki.python.org/moin/GoogleGroupsPython.


Mark Lawrence

--
http://mail.python.org/mailman/listinfo/python-list

Re: Export data from python to a txt file

[rusi]

On Mar 29, 10:33 pm, Ana Dionísio  wrote:
> Hello!!!
>
> I have this lists a=[1,3,5,6,10], b=[a,t,q,r,s] and I need to export it to a 
> txt file and I can't use csv.
>
> And I want the next format:
>
> a 1 3 5 6 10
> b a t q r s
>
> I already have this code:
>
> "f = open("test.txt", 'w')
>  f.write("a")
>  f.write("\n")
>  f.write("b")
>  f.write("\n")
>
>  for i in xrange(len(a)):
>     LDFile.write("\t")
>     LDFile.write(str(a[i]))
>     LDFile.write("\t")
>     LDFile.write(str(b[i]))
>
>  f.close()"
>
> But it doesn't have the format I want. Can you help?
>
> Thanks!

Dunno about LDFile. [Neither does google]

You can try:
>>> a=[1,3,7,10,100]
>>> "a " + " ".join(map(str,a))
'a 1 3 7 10 100'
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Surrogate pairs in new flexible string representation [was Re: flaming vs accuracy [was Re: Performance of int/long in Python 3]]

[Terry Reedy]

On 3/28/2013 10:37 PM, Steven D'Aprano wrote:


Under what circumstances will a string be created from a wchar_t string?
How, and why, would such a string be created? Why would Python still
support strings containing surrogates when it now has a nice, shiny,
surrogate-free flexible representation?


I believe because surrogates are legal codepoints and users may put them 
in strings even though python does not (except for surrogate_escape 
error handling).


I believe some of the internal complexity comes from supporting the old 
C-api so as to not immediately invalidate existing extensions.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Export data from python to a txt file

[Vincent Vande Vyvre]

Le 29/03/13 18:33, Ana Dionísio a écrit :
> Hello!!!
>
> I have this lists a=[1,3,5,6,10], b=[a,t,q,r,s] and I need to export it to a 
> txt file and I can't use csv.
>
> And I want the next format:
>
> a 1 3 5 6 10
> b a t q r s
>
> I already have this code:
>
> "f = open("test.txt", 'w')
>  f.write("a")
>  f.write("\n")
>  f.write("b")
>  f.write("\n")
>
>  for i in xrange(len(a)):
> LDFile.write("\t")
> LDFile.write(str(a[i]))
> LDFile.write("\t")
> LDFile.write(str(b[i]))
>
>  f.close()"
>
> But it doesn't have the format I want. Can you help?
>
> Thanks!
>
>  
Something like that:

Python 2.6.5 (r265:79063, Oct  1 2012, 22:07:21)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> a=[1,3,5,6,10]
>>> b=['a', 't', 'q', 'r', 's']
>>> sta = " ".join([str(i) for i in a])
>>> stb = " ".join(b)
>>> txt = "a " + sta + '\nb ' + stb
>>> f = open('test.txt', 'w')
>>> f.write(txt)
>>> f.close()
>>> f = open('test.txt', 'r')
>>> for line in f:
... print line
...
a 1 3 5 6 10

b a t q r s
>>>



-- 
Vincent V.V.
Oqapy  . Qarte
 . PaQager 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Export data from python to a txt file

[Peter Otten]

Ana Dionísio wrote:

> I have this lists a=[1,3,5,6,10], b=[a,t,q,r,s] and I need to export it to
> a txt file and I can't use csv.

What do you mean by "can't use csv"?

- You cannot get it to work with the csv module
- You are not allowed to use csv by your instructor
- Something else.

> And I want the next format:
> 
> a 1 3 5 6 10
> b a t q r s
> 
> I already have this code:
> 
> "f = open("test.txt", 'w')
>  f.write("a")
>  f.write("\n")
>  f.write("b")
>  f.write("\n")
> 
>  for i in xrange(len(a)):
> LDFile.write("\t")
> LDFile.write(str(a[i]))
> LDFile.write("\t")
> LDFile.write(str(b[i]))
> 
>  f.close()"
> 
> But it doesn't have the format I want. Can you help?
> 
> Thanks!


-- 
http://mail.python.org/mailman/listinfo/python-list

Generator hack to avoid an exception

[Kruno Saho]

I understand that 'with ignore(...)' will be coming out in Python 3.4 next 
year, but in the mean time I have added a yield at the end of a particular 
coroutine.

Here is the 'fixed' version:
http://pastebin.com/c8kbqc4w

Here is the original version, which I believe is more pythonic:
http://pastebin.com/38Fb7N0Y

The reason being is to have my code outside the generator clean and flat(er). I 
feel that this little hack, while irritating, is more pythonic than a 
try/except, which only breaks out of an infinite loop, which was not there to 
begin with. I could replace the infinite loop with an iteration over a range, 
but that in itself is doing more work unnecessarily, and is just as hackish. I 
would not do this if the coroutine controlled any resources, as they would need 
to be cleaned up. I have not seen this issue addressed in PEP8.

What is the consensus on this issue being more/less pythonic, and how 
acceptable is this practice?

Thanks.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Extended Call for Proposals for Presentations and Tutorials at PyCon SG 2013

[Anna Ravenscroft]

Just a followup -
Alex Martelli and I attended PyCon SG 2012 and it was a wonderful
experience. I highly encourage folks to consider submitting a proposal
to this conference. It's a worthwhile conference and a chance to visit
a fascinating place.

cordially,
Anna

On Fri, Mar 29, 2013 at 5:55 AM, George Goh  wrote:
> Hi,
>
> On behalf of the organizing committee of PyCon SG 2013, we are
> announcing our Extended Call for Proposals for Presentations and
> Tutorials for the 2013 PyCon Singapore Conference, to be held in
> Singapore from June 13 to 15, 2013.
>
> We are pleased to announce that our keynote speaker is Wes McKinney,
> main developer of pandas (http://pandas.pydata.org/) and author of the
> new book "Python for Data Analysis".
>
> To submit your proposal(s) for presentations and/or tutorials, please
> refer to the details found at https://pycon.sg/proposals/
>
> The submission deadlines for both have been extended to April 30, 2013
>
> For enquiries, pls direct them to conference at
> pycon.sg
>
> We look forward to receiving your proposals! And to a great conference
> this year.
>
> Best regards,
> George Goh
> On behalf of the PyCon SG 2013 Organizing Committee
> --
> http://mail.python.org/mailman/listinfo/python-announce-list
>
> Support the Python Software Foundation:
> http://www.python.org/psf/donations/



-- 
cordially,
Anna
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[Dave Angel]

On 03/29/2013 10:48 AM, Roy Smith wrote:

I'm inserting a gazillion rows into a MySQL database using MySQLdb and 
cursor.executemany() for efficiency.  Every once in a while, I get a row which 
violates some kind of database constraint and raises Error.

I can catch the exception, but don't see any way to tell which row caused the 
problem.  Is this information obtainable, short of retrying each row one by one?



I don't know the direct answer, or even if there is one (way to get 
MySQL to tell you which one failed), but ...


Assuming that executeMany is much cheaper than a million calls to 
executeOne (or whatever).


 -- single bad rows --
If you have a million items, and you know exactly one is different, you 
can narrow it down more quickly than just sequencing through them.  You 
can do half of them at a time, carefully choosing which subset of the 
total you use each time.  After 20 such calls, you can then calculate 
exactly which one is different.  Standard CS algorithm.


 -- sparse set of rows --
If you know that it's at least one, but still less than a dozen or so, 
it's a little trickier, but you should still converge on a final list 
pretty quickly.  Each time you do half, you also do the complementary 
half.  If either of them has no 'differences" you can then eliminate 
half the cases.


If you don't get a specific answer where MySQL can tell you the bad row, 
and if you don't know what I'm talking about, ask and I'll try to 
elaborate on one of the two above cases.


--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: Cannot run a single MySQLdb execute....

[nagia . retsina]

Τη Παρασκευή, 29 Μαρτίου 2013 7:39:20 μ.μ. UTC+2, ο χρήστης Alan Meyer έγραψε:
> On 03/29/2013 01:32 PM, Alan Meyer wrote:
> 
> 
> 
> > However, MySQLdb is a well established module and what you're asking it
> 
> > to do is very simple and very standard.
> 
> 
> 
> Oh, sorry, I see that you already said that mysqldb won't work with 
> 
> python 3.  My comments in the last message are irrelevant.
> 
> 
> 
> Sorry again.
> 
> 
> 
> Good luck.
> 
> 
> 
>  Alan

Thanks Alan i decided to work with 'pymysql' and iam struggling to egt thbis 
working but i still receive the same blank page for some unknown reason.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: No errors displayed but i blank scren nstead.

[nagia . retsina]

Τη Πέμπτη, 28 Μαρτίου 2013 4:19:47 μ.μ. UTC+2, ο χρήστης Νίκος Γκρ33κ έγραψε:
> Fianlly my cgi .py script doesnt produce any more errors, i think i ahve 
> correct them but it present a blank screen
> 
> 
> 
> http://superhost.gr
> 
> 
> 
> Any idea why?
> 
> What should i check?

Thants not the issue, that message uses to be shown in comamnd line even woith 
python 2.6

somehting else is wrong here and it shwos blank pages.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: No errors displayed but i blank scren nstead.

[nagia . retsina]

But now iam also receivein this error message as shown here when i switches to 
'pymysql'
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: No errors displayed but i blank scren nstead.

[nagia . retsina]

But now iam also receiving this error message as shown here when i switches to 
'pymysql'
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: No errors displayed but i blank scren nstead.

[Chris Angelico]

On Sat, Mar 30, 2013 at 6:27 AM,   wrote:
> But now iam also receivein this error message as shown here when i switches 
> to 'pymysql'

Why the change of email address? Are you trying to dodge killfiles?

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: No errors displayed but i blank scren nstead.

[Neil Cerutti]

On 2013-03-29, Chris Angelico  wrote:
> On Sat, Mar 30, 2013 at 6:27 AM,   wrote:
>> But now iam also receivein this error message as shown here when i switches 
>> to 'pymysql'
>
> Why the change of email address? Are you trying to dodge killfiles?

I had that one killfiled already. Must be he/she/it is posting
from a different sock puppet.

-- 
Neil Cerutti
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: No errors displayed but i blank scren nstead.

[nagia . retsina]

Τη Παρασκευή, 29 Μαρτίου 2013 9:34:07 μ.μ. UTC+2, ο χρήστης Chris Angelico 
έγραψε:
> On Sat, Mar 30, 2013 at 6:27 AM,   wrote:
> 
> > But now iam also receivein this error message as shown here when i switches 
> > to 'pymysql'
> 
> 
> 
> Why the change of email address? Are you trying to dodge killfiles?
> 
> 
> 
> ChrisA

No, someone else is using this computer too along with Chrome and Chrome 
associates his mail address with all google services and i was under the 
impression i was sposting from my gmail group account. Sorry for that. it 
happened many tiems alrready as i noticed days ago too.
-- 
http://mail.python.org/mailman/listinfo/python-list

dir() vs print(dir()) in the embedded mode

[Nick Gnedin]

Folks,

I have a newbie question: I am trying to embed Python into my 
application. While playing around, I noticed that the behavior of the 
interpreter in the embedded mode differs from the standalone one.


Namely, in the standalone mode if I type dir(), I get a list of build-in 
symbols. In the embedded mode only print(dir()) does that, while just 
dir() returns silently.


Is there a way to intercept the output of dir() (and all other commands) 
and display them to the user?


Here is an example code that illustrates the behavior (the first call to 
PyRun_SimpleString() returns silently).


Many thanks for your future hints,

Nick


#include 

int main()
{
  Py_Initialize();
  PyRun_SimpleString("dir()");
  printf("-\n");
  PyRun_SimpleString("print(dir())");
  Py_Finalize();

  return 0;
}
--
http://mail.python.org/mailman/listinfo/python-list

Python class and variable issue(newby question!)

[Sam Berry]

Hey, 

Im new to object orientated programming and have an issue with using classes. 
Im using the kivy module, a GUI creator , so posting the actual code may 
confuse. But an example of what im trying to achieve is below

class test()
s = 1

def test1()
global s
s = 2

def test2()
global s
s = 3

def test3()
global s
s = 4


class test4()

def printing()
print test().s

After been in the test()class  a choice of buttons allows the user to run 
either of the functions within it and sends us into the test4() class. Then 
another button runs printing().

However printing() always prints 1, even if the variable has been changed. Im 
guessing because  print test().s  redefines the variable. May be a very simple 
to solve problem, but i cant figure it out.

Any insights of how to get this working, or example code would be very helpful!

Thanks, Sam
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: No errors displayed but i blank scren nstead.

[Walter Hurry]

On Fri, 29 Mar 2013 20:14:16 +, Neil Cerutti wrote:

> On 2013-03-29, Chris Angelico  wrote:
>> On Sat, Mar 30, 2013 at 6:27 AM,   wrote:
>>> But now iam also receivein this error message as shown here when i
>>> switches to 'pymysql'
>>
>> Why the change of email address? Are you trying to dodge killfiles?
> 
> I had that one killfiled already. Must be he/she/it is posting from a
> different sock puppet.


Ditto. Just another entry for the bozo bin.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python class and variable issue(newby question!)

[Chris Angelico]

On Sat, Mar 30, 2013 at 8:12 AM, Sam Berry  wrote:
> class test()
> s = 1
>
> def test1()
> global s
> s = 2
>
> def test2()
> global s
> s = 3
>
> def test3()
> global s
> s = 4

That's not a global, that's a class variable. But to give any more
specific help, it'd be helpful to have some actual working code -
twiddle your code until it's a nice simple example:

http://sscce.org/

Most likely, what you want is to make these functions into either
static methods or instance methods. Beyond that, hard to say exactly.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: dir() vs print(dir()) in the embedded mode

[MRAB]

On 29/03/2013 21:04, Nick Gnedin wrote:


Folks,

I have a newbie question: I am trying to embed Python into my
application. While playing around, I noticed that the behavior of the
interpreter in the embedded mode differs from the standalone one.

Namely, in the standalone mode if I type dir(), I get a list of build-in
symbols. In the embedded mode only print(dir()) does that, while just
dir() returns silently.

Is there a way to intercept the output of dir() (and all other commands)
and display them to the user?

Here is an example code that illustrates the behavior (the first call to
PyRun_SimpleString() returns silently).

Many thanks for your future hints,


dir() doesn't print anything, it just returns a list.

It's the interactive interpreter that's printing the result, unless
it's None.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python class and variable issue(newby question!)

[Benjamin Kaplan]

On Fri, Mar 29, 2013 at 2:12 PM, Sam Berry  wrote:
> Hey,
>
> Im new to object orientated programming and have an issue with using classes. 
> Im using the kivy module, a GUI creator , so posting the actual code may 
> confuse. But an example of what im trying to achieve is below
>
> class test()
> s = 1
>
> def test1()
> global s
> s = 2
>
> def test2()
> global s
> s = 3
>
> def test3()
> global s
> s = 4
>
>
> class test4()
>
> def printing()
> print test().s
>
> After been in the test()class  a choice of buttons allows the user to run 
> either of the functions within it and sends us into the test4() class. Then 
> another button runs printing().
>
> However printing() always prints 1, even if the variable has been changed. Im 
> guessing because  print test().s  redefines the variable. May be a very 
> simple to solve problem, but i cant figure it out.
>
> Any insights of how to get this working, or example code would be very 
> helpful!
>
> Thanks, Sam
> --


There are three different namespaces here: global, class, and local.
You're assigning to the global s, which exists outside the class. Do
"print s" instead of "print test().s" and you'll see your changed
value of s. If you want to change the value of s inside the class,
it's called "test.s" (notice that there's no parenthesis- that's
because s here is an attribute of the test class, not each instance of
test. The same value will be shared by all instances of that class).
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: flaming vs accuracy [was Re: Performance of int/long in Python 3]

[rurpy]

On 03/28/2013 02:31 PM, Ethan Furman wrote:
> On 03/28/2013 12:54 PM, ru...@yahoo.com wrote:
>> On 03/28/2013 01:48 AM, Steven D'Aprano wrote:
>> For someone who delights in pointing out the logical errors of
>> others you are often remarkably sloppy in your own logic.
>> 
>> Of course language can be both helpful and excessively strong. That
>> is the case when language less strong would be equally or more
>> helpful.
> 
> It can also be the case when language less strong would be useless.

I don't get your point.
I was pointing out the fallacy in Steven's logic (which you cut).
How is your statement relevant to that?

>> Further, "liar" is both so non-objective and so pejoratively 
>> emotive that it is a word much more likely to be used by someone
>> interested in trolling than in a serious discussion, so most
>> sensible people here likely would not bite.
> 
> Non-objective?  If today poster B says X, and tomorrow poster B says
> s/he was unaware of X until just now, is not "liar" a reasonable
> conclusion?

Of course not.  People forget what they posted previously, change 
their mind, don't express what they intended perfectly, sometimes
express a complex thought that the reader inaccurately perceives
as contradictory, don't realize themselves that their thinking
is contradictory, ...
And of course who among us *not* a "liar" since we all lie from 
time to time.

Lying involves intent to deceive.  I haven't been following jmfauth's 
claims since they are not of interest to me, but going back and quickly
looking at the posts that triggered the "liar" and "idiot" posts, I 
did not see anything that made me think that jmfauth was not sincere 
in his beliefs.  Being wrong and being sincere are not exclusive.
Nor did Steven even try to justify the "liar" claim.  As to Mark
Lawrence, that seemed like a pure "I don't like you" insult whose 
proper place is /dev/null.

Even if the odds are 80% that the person is lying, why risk your
own credibility by making a nearly impossible to substantiate claim?
Someone may praise some company's product constantly online and be 
discovered to be a salesperson at that company.  Most of the time 
you would be right to accuse the person of dishonesty.  But I knew 
a person who was very young and naive, who really believed in the 
product and truly didn't see anything wrong in doing that.  That 
doesn't make it good behavior but those who claimed he was hiding 
his identity for personal gain were wrong (at least as far as I 
could tell, knowing the person personally.)  Just post the facts 
and let people draw their own conclusions; that's better than making
aggressive and offensive claims than can never be proven.

Calling people liars or idiots not only damages the reputation of 
the Python community in general [*1] but hurts your own credibility 
as well, since any sensible reader will wonder if other opinions 
you post are more influenced by your emotions than by your intelligence.

>>> I hope that we all agree that we want a nice, friendly,
>>> productive community where everyone is welcome.
>> 
>> I hope so too but it is likely that some people want a place to
>> develop and assert some sense of influence, engage in verbal duels,
>> instigate arguments, etc.  That can be true of regulars here as
>> well as drive-by posters.
>> 
>>> But some people simply cannot or will not behave in ways that are
>>> compatible with those community values. There are some people
>>> whom we *do not want here*
>> 
>> In other words, everyone is NOT welcome.
> 
> Correct.  Do you not agree?

Don't ask me, ask Steven.  He was the one who wrote two sentences 
earlier, "...we want a...community where everyone is welcome."

I'll snip the rest of your post because it is your opinions
and I've already said why I disagree.  Most people are smart enough
to make their own evaluations of posters here and if they are not,
and reject python based on what they read from a single poster 
who obviously has "strong" views, then perhaps that's for the 
best.  That possibility (which I think is very close to zero) is
a tiny price to pay to avoid all the hostility and noise.


[*1] See for example the blog post at 
  
http://joepie91.wordpress.com/2013/02/19/the-python-documentation-is-bad-and-you-should-feel-bad/
which was recently discussed in this list and in which the 
author wrote, "the community around Python is one of the most 
hostile and unhelpful communities around any programming-related
topic that I have ever seen".
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sudoku

[Eric Parry]

On Friday, March 29, 2013 9:15:36 AM UTC+10:30, Chris Angelico wrote:
> On Fri, Mar 29, 2013 at 9:11 AM, Eric Parry  wrote:
> 
> > Thank you for that explanation.
> 
> > No, I do not understand recursion. It is missing from my Python manual. I 
> > would be pleased to receive further explanation from anyone.
> 
> 
> 
> If you already know what recursion is, just remember the answer.
> 
> Otherwise, find someone who is standing closer to Douglas Hofstadter
> 
> than you are; then ask him or her what recursion is.
> 
> 
> 
> :)
> 
> 
> 
> Recursion is a form of self-referential code. Take this simple, and
> 
> rather silly, means of calculating the sum of numbers in a list (like
> 
> the sum() function):
> 
> 
> 
> # The sum of numbers in an empty list is 0.
> 
> # Otherwise it is the first number plus the sum of the rest of the list.
> 
> def list_sum(lst):
> 
> if not lst: return 0
> 
> return lst[0] + list_sum(lst[1:])
> 
> 
> 
> >>> list_sum([1,2,3,4,5,6])
> 
> 21
> 
> 
> 
> Note how the function calls itself - but not always. That's critical
> 
> to recursion - a termination condition. In this case, it's quite
> 
> obvious that the list will eventually have nothing left in it, so the
> 
> function will terminate. Sometimes it's less obvious. Sometimes a bug
> 
> results in infinite recursion... and:
> 
> 
> 
> RuntimeError: maximum recursion depth exceeded in comparison
> 
> 
> 
> Hope that helps!
> 
> 
> 
> ChrisA

Thank you for that example Chris.
That explains why the program keeps running after a solution is found.
Eric
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Surrogate pairs in new flexible string representation

[Christian Heimes]

Am 29.03.2013 07:22, schrieb Ian Kelly:
> Since the PEP specifically mentions ParseTuple string conversion, I am
> thinking that this is probably the motivation for caching it.  A
> string that is passed into a C function (that uses one of the various
> UTF-8 char* format specifiers) is perhaps likely to be passed into
> that function again at some point, so the UTF-8 representation is kept
> around to avoid the need to recompose it at on each call.

It's not just about caching but also about memory management. The
additional utf8 member is required for backward compatibility. The APIs
expect a pointer to an existing and shared block of memory. They don't
take ownership of the memory block and therefore don't free() it.

Christian

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sudoku

[Dave Angel]

On 03/29/2013 05:47 PM, Eric Parry wrote:



  


That explains why the program keeps running after a solution is found.


A recursive function can be designed to find all solutions, in which 
case it would (as you say) keep running.


The function you posted in the first place uses exit() to avoid keeping 
running.  It stops as soon as a solution is found.


Sometimes a problem cannot be solved in the number of stack entries 
supplied by Python.  So even though such a function will terminate, it 
may crash first if the problem is too big.  Example, the factorial 
problem I described earlier, if you pass it 2000 as a parameter.  If 
this is a problem, one can tell the Python to give you more stack entries.


Given a 9x9 matrix, and at least some of them filled in, the maximum 
depth your code can use is less than 81.  So it won't get a stack 
overflow in any implementation of Python I've seen.  Perhaps in an 8051.


Sometimes a bug in such a function will cause it to run indefinitely, 
and/or to overflow the stack.  I don't see such a bug in this function.



--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: dir() vs print(dir()) in the embedded mode

[Steven D'Aprano]

On Fri, 29 Mar 2013 16:04:24 -0500, Nick Gnedin wrote:

> Folks,
> 
> I have a newbie question: I am trying to embed Python into my
> application. While playing around, I noticed that the behavior of the
> interpreter in the embedded mode differs from the standalone one.
> 
> Namely, in the standalone mode if I type dir(), I get a list of build-in
> symbols. In the embedded mode only print(dir()) does that, while just
> dir() returns silently.

What do you mean by "standalone mode"? If you are talking about the 
interactive interpreter, that is expected behaviour. The interactive 
interpreter automatically prints the output of every line.

When running non-interactively as a script (whether embedded or not), if 
you want to print the output of a line, you will have to call print on 
that output.


> Is there a way to intercept the output of dir() (and all other commands)
> and display them to the user?

Automatically? I don't believe so. I think it is hard-coded behaviour of 
the interactive interpreter, and cannot by enabled for scripts.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python class and variable issue(newby question!)

[Sam Berry]

Thanks for the responses! My issue was sorted with Benjamins post, just 
printing s worked.

Cheers for the info though Chris, if i have any further issues il post them 
with some working code.

Sam
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python class and variable issue(newby question!)

[Chris Angelico]

On Sat, Mar 30, 2013 at 9:17 AM, Sam Berry  wrote:
> Thanks for the responses! My issue was sorted with Benjamins post, just 
> printing s worked.
>
> Cheers for the info though Chris, if i have any further issues il post them 
> with some working code.

Awesome! Always happy to help out.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python class and variable issue(newby question!)

[Steven D'Aprano]

On Sat, 30 Mar 2013 08:24:00 +1100, Chris Angelico wrote:

> On Sat, Mar 30, 2013 at 8:12 AM, Sam Berry 
> wrote:
>> class test()
>> s = 1
>>
>> def test1()
>> global s
>> s = 2

> That's not a global, that's a class variable.


/me thwacks Chris with a halibut.


Not only is "class variable" ambiguous, but Python uses different scoping 
rules for "variables" (name bindings) and attributes.

I don't know what language first decided to conflate object attributes 
and variables -- I suspect Java -- but it is actively harmful terminology 
(in Python at least, if not in general) and whoever invented it is bad 
and should feel bad.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python class and variable issue(newby question!)

[Chris Angelico]

On Sat, Mar 30, 2013 at 9:51 AM, Steven D'Aprano
 wrote:
> On Sat, 30 Mar 2013 08:24:00 +1100, Chris Angelico wrote:
>
>> On Sat, Mar 30, 2013 at 8:12 AM, Sam Berry 
>> wrote:
>>> class test()
>>> s = 1
>>>
>>> def test1()
>>> global s
>>> s = 2
>
>> That's not a global, that's a class variable.
>
>
> /me thwacks Chris with a halibut.

Ow ow ow, I give in... I am suitably and appropriately thwacked.

> Not only is "class variable" ambiguous, but Python uses different scoping
> rules for "variables" (name bindings) and attributes.

Yes, I should have said class ATTRIBUTE. Sorry all! (Can I blame the
whole 8AM and still up thing? No? Mea culpa, anyhow.)

> I don't know what language first decided to conflate object attributes
> and variables -- I suspect Java -- but it is actively harmful terminology
> (in Python at least, if not in general) and whoever invented it is bad
> and should feel bad.

Agreed. Unfortunately it permeates a lot of programmer's vocabularies,
mine included.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python class and variable issue(newby question!)

[Dave Angel]

On 03/29/2013 06:17 PM, Sam Berry wrote:

Thanks for the responses! My issue was sorted with Benjamins post, just 
printing s worked.

Cheers for the info though Chris, if i have any further issues il post them 
with some working code.


In that case, you probably should add a line like:


s = None

at the beginning of the code, along with a description of what s is 
supposed to represent (especially since the name doesn't reveal much).


And you should remove the class variable s.  It'll just confuse things.

I guess this isn't the place to rail about non-const globals.

--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: flaming vs accuracy [was Re: Performance of int/long in Python 3]

[Ethan Furman]

On 03/29/2013 02:26 PM, ru...@yahoo.com wrote:

On 03/28/2013 02:31 PM, Ethan Furman wrote:

On 03/28/2013 12:54 PM, ru...@yahoo.com wrote:

On 03/28/2013 01:48 AM, Steven D'Aprano wrote:
For someone who delights in pointing out the logical errors of
others you are often remarkably sloppy in your own logic.

Of course language can be both helpful and excessively strong. That
is the case when language less strong would be equally or more
helpful.


It can also be the case when language less strong would be useless.


I don't get your point.
I was pointing out the fallacy in Steven's logic (which you cut).
How is your statement relevant to that?


Ah.  I thought you were saying that in all cases helpful strong language would 
be even more helpful if less strong.



Further, "liar" is both so non-objective and so pejoratively
emotive that it is a word much more likely to be used by someone
interested in trolling than in a serious discussion, so most
sensible people here likely would not bite.


Non-objective?  If today poster B says X, and tomorrow poster B says
s/he was unaware of X until just now, is not "liar" a reasonable
conclusion?


Of course not.  People forget what they posted previously, change
their mind, don't express what they intended perfectly, sometimes
express a complex thought that the reader inaccurately perceives
as contradictory, don't realize themselves that their thinking
is contradictory, ...


I agree, which is why I resisted my own impulse to call him a liar; however, he has been harping on this subject for 
months now, so I would be suprised if he actually was surprised and had forgotten...




Lying involves intent to deceive.  I haven't been following jmfauth's
claims since they are not of interest to me, but going back and quickly
looking at the posts that triggered the "liar" and "idiot" posts, I
did not see anything that made me think that jmfauth was not sincere
in his beliefs.  Being wrong and being sincere are not exclusive.
Nor did Steven even try to justify the "liar" claim.  As to Mark
Lawrence, that seemed like a pure "I don't like you" insult whose
proper place is /dev/null.


After months of jmf's antagonist posts, I don't blame them.


I hope that we all agree that we want a nice, friendly,
productive community where everyone is welcome.


I hope so too but it is likely that some people want a place to
develop and assert some sense of influence, engage in verbal duels,
instigate arguments, etc.  That can be true of regulars here as
well as drive-by posters.


But some people simply cannot or will not behave in ways that are
compatible with those community values. There are some people
whom we *do not want here*


In other words, everyone is NOT welcome.


Correct.  Do you not agree?


Don't ask me, ask Steven.  He was the one who wrote two sentences
earlier, "...we want a...community where everyone is welcome."


Ah, right -- missed that!

--
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[Roy Smith]

In article ,
 Dennis Lee Bieber  wrote:

> If using MySQLdb, there isn't all that much difference... MySQLdb is
> still compatible with MySQL v4 (and maybe even v3), and since those
> versions don't have "prepared statements", .executemany() essentially
> turns into something that creates a newline delimited "list" of
> "identical" (but for argument substitution) statements and submits that
> to MySQL.

Shockingly, that does appear to be the case.  I had thought during my 
initial testing that I was seeing far greater throughput, but as I got 
more into the project and started doing some side-by-side comparisons, 
it the differences went away.

We're sucking in a pretty huge amount of data.  The source document is a 
7 GB gzipped XML file.  I'm not sure how big it is uncompressed (we use 
gzip.GzipFile to uncompress on the fly) but I'm guessing something like 
a 30x compression ratio so 200 GB?  The last time we imported the whole 
set, it ran for 21 days!

It turns out, the problems we were seeing were all inserts into a new 
table we added.   Apparently, the default charset is latin-1 and we 
didn't notice that when we altered the schema!  Once I noticed that all 
the other tables were utf-8 and changed this one to be that, the 
problems went away.

Sadly, I ended up resorting to a truly ugly hack to diagnose the 
problem.  I catch the exception and parse the text message.  Yuck.

try:
self.executemany(self.sql_statement, self.sql_params)
except MySQLdb.Error as ex:
code, message = ex.args
m = re.search(r".* at row (\d+)$", message)
if m:
i = int(m.group(1)) - 1  # Python is 0-index, SQL, 1-index


The other truly horrible part of the project was when I decided it was 
bad for my import script to have too much schema knowledge hard-wired 
in.  So, I decided to use SQLAlchemy to introspect the database and 
discover the column names, types, and defaults.  It turns out, if an 
integer column has a default (say, 0), the introspected data comes back 
with the default as the string, '0'.  WTF???

Does Postgress's Python adapter handle executemany() in a sane way?  
We're not wedded to MySQL in any way.  We use it for exactly this one 
process.  We get these XML dumps from a supplier's SQL-Server database.  
We stage the data in MySQL, then export what we need into MongoDB.  We 
could easily swap out the MySQL staging for Postgress if that worked 
better.

Hmmm, we do take advantage of REPLACE INTO, which I think is a 
non-standard MySQL addition.  Not sure if Postgress supports that.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[Chris Angelico]

On Sat, Mar 30, 2013 at 11:41 AM, Roy Smith  wrote:
> In article ,
>  Dennis Lee Bieber  wrote:
>
>> If using MySQLdb, there isn't all that much difference... MySQLdb is
>> still compatible with MySQL v4 (and maybe even v3), and since those
>> versions don't have "prepared statements", .executemany() essentially
>> turns into something that creates a newline delimited "list" of
>> "identical" (but for argument substitution) statements and submits that
>> to MySQL.
>
> Shockingly, that does appear to be the case.  I had thought during my
> initial testing that I was seeing far greater throughput, but as I got
> more into the project and started doing some side-by-side comparisons,
> it the differences went away.

How much are you doing per transaction? The two extremes (everything
in one transaction, or each line in its own transaction) are probably
the worst for performance. See what happens if you pepper the code
with 'begin' and 'commit' statements (maybe every thousand or ten
thousand rows) to see if performance improves.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Screencast: Creating and deploying an advanced python web application in under 15 minutes!

[timothy crosley]

Thanks Karim!

On Friday, March 29, 2013 10:47:41 AM UTC-4, Karim wrote:
> On 29/03/2013 14:53, timothy crosley wrote:
> 
> > I apologize for the audio from the original screen cast, it was really 
> > sub-par. I bought a new microphone and re-recorded it: 
> > http://www.youtube.com/watch?v=0L8TsmrZPLg&feature=youtu.be
> 
> >
> 
> > Thanks!
> 
> >
> 
> > Timothy
> 
> >
> 
> > On Tuesday, March 26, 2013 4:54:15 AM UTC-4, timothy crosley wrote:
> 
> >> Hi,
> 
> >>
> 
> >>
> 
> >>
> 
> >> I've created a screen cast showing how a message board with 
> >> live-validation and Ajax calls written in python can be built and deployed 
> >> in under 15 minutes. You can view it here: 
> >> http://www.youtube.com/watch?v=ucougrZK9wI
> 
> >>
> 
> >>
> 
> >>
> 
> >> I hope some of you find it useful,
> 
> >>
> 
> >>
> 
> >>
> 
> >> Thanks!
> 
> >>
> 
> >>
> 
> >>
> 
> >> Timothy
> 
> Hi Timothy,
> 
> 
> 
> Very interesting!
> 
> 
> 
> Thx a lot!
> 
> Cheers
> 
> Karim

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[Roy Smith]

In article ,
 Chris Angelico  wrote:

> On Sat, Mar 30, 2013 at 11:41 AM, Roy Smith  wrote:
> > In article ,
> >  Dennis Lee Bieber  wrote:
> >
> >> If using MySQLdb, there isn't all that much difference... MySQLdb is
> >> still compatible with MySQL v4 (and maybe even v3), and since those
> >> versions don't have "prepared statements", .executemany() essentially
> >> turns into something that creates a newline delimited "list" of
> >> "identical" (but for argument substitution) statements and submits that
> >> to MySQL.
> >
> > Shockingly, that does appear to be the case.  I had thought during my
> > initial testing that I was seeing far greater throughput, but as I got
> > more into the project and started doing some side-by-side comparisons,
> > it the differences went away.
> 
> How much are you doing per transaction? The two extremes (everything
> in one transaction, or each line in its own transaction) are probably
> the worst for performance. See what happens if you pepper the code
> with 'begin' and 'commit' statements (maybe every thousand or ten
> thousand rows) to see if performance improves.
> 
> ChrisA

We're doing it all in one transaction, on purpose.  We start with an 
initial dump, then get updates about once a day.  We want to make sure 
that the updates either complete without errors, or back out cleanly.  
If we ever had a partial daily update, the result would be a mess.

Hmmm, on the other hand, I could probably try doing the initial dump the 
way you describe.  If it fails, we can just delete the whole thing and 
start again.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[Chris Angelico]

On Sat, Mar 30, 2013 at 12:19 PM, Roy Smith  wrote:
> In article ,
>  Chris Angelico  wrote:
>
>> On Sat, Mar 30, 2013 at 11:41 AM, Roy Smith  wrote:
>> > In article ,
>> >  Dennis Lee Bieber  wrote:
>> >
>> >> If using MySQLdb, there isn't all that much difference... MySQLdb is
>> >> still compatible with MySQL v4 (and maybe even v3), and since those
>> >> versions don't have "prepared statements", .executemany() essentially
>> >> turns into something that creates a newline delimited "list" of
>> >> "identical" (but for argument substitution) statements and submits that
>> >> to MySQL.
>> >
>> > Shockingly, that does appear to be the case.  I had thought during my
>> > initial testing that I was seeing far greater throughput, but as I got
>> > more into the project and started doing some side-by-side comparisons,
>> > it the differences went away.
>>
>> How much are you doing per transaction? The two extremes (everything
>> in one transaction, or each line in its own transaction) are probably
>> the worst for performance. See what happens if you pepper the code
>> with 'begin' and 'commit' statements (maybe every thousand or ten
>> thousand rows) to see if performance improves.
>>
>> ChrisA
>
> We're doing it all in one transaction, on purpose.  We start with an
> initial dump, then get updates about once a day.  We want to make sure
> that the updates either complete without errors, or back out cleanly.
> If we ever had a partial daily update, the result would be a mess.
>
> Hmmm, on the other hand, I could probably try doing the initial dump the
> way you describe.  If it fails, we can just delete the whole thing and
> start again.

One transaction for the lot isn't nearly as bad as one transaction per
row, but it can consume a lot of memory on the server - or at least,
that's what I found last time I worked with MySQL. (PostgreSQL works
completely differently, and I'd strongly recommend doing it all as one
transaction if you switch.) It's not guaranteed to help, but if it
won't hurt to try, there's a chance you'll gain some performance.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[Roy Smith]

In article ,
 Roy Smith  wrote:

> In article ,
>  Dennis Lee Bieber  wrote:
> 
> > If using MySQLdb, there isn't all that much difference... MySQLdb is
> > still compatible with MySQL v4 (and maybe even v3), and since those
> > versions don't have "prepared statements", .executemany() essentially
> > turns into something that creates a newline delimited "list" of
> > "identical" (but for argument substitution) statements and submits that
> > to MySQL.
> 
> Shockingly, that does appear to be the case.  I had thought during my 
> initial testing that I was seeing far greater throughput, but as I got 
> more into the project and started doing some side-by-side comparisons, 
> it the differences went away.

OMG, this is amazing.

http://stackoverflow.com/questions/3945642/

It turns out, the MySQLdb executemany() runs a regex over your SQL and 
picks one of two algorithms depending on whether it matches or not.

restr = (r"\svalues\s*"
r"(\(((?http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[Chris Angelico]

On Sat, Mar 30, 2013 at 1:44 PM, Roy Smith  wrote:
> The slow way (i.e. "VALUES"), I'm inserting 1000 rows about every 2.4
> seconds.  When I switch to "values", I'm getting more like 1000 rows in
> 100 ms!
>
> A truly breathtaking bug.

*facepalm*

Doubly facepalm because a regex could easily have tested for mixed case.

Especially facepalm because there's some way to do this that's faster
than straight INSERT statements, and it's not clearly documented as
"hey, guys, if you want to dump loads of data in, use COPY instead"
(it might be that, I don't know, but usually COPY isn't directly
transliterable with INSERT).

I agree. Breathtaking.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[Roy Smith]

In article ,
 Chris Angelico  wrote:

> Especially facepalm because there's some way to do this that's faster
> than straight INSERT statements, and it's not clearly documented as
> "hey, guys, if you want to dump loads of data in, use COPY instead"
> (it might be that, I don't know, but usually COPY isn't directly
> transliterable with INSERT).

We're actually using REPLACE INTO.  For the initial data load, we could 
just as well do INSERT, but we need the REPLACE functionality as we roll 
in the daily incremental updates.

This also explains why, even after provisioning our RDS instance for 
2000 IOPS (that's AWS-speak for "we paid extra to get more disk 
bandwidth"), we didn't see any performance improvement!
-- 
http://mail.python.org/mailman/listinfo/python-list

take elance python test

[jianxiang.yi]

Hi,
Did any one take elance python test?
It is come from here 
"http://www.expertrating.com/certifications/Python-2-x-Test.asp";.
I want to know who have thoese subject, or demo, or souce code.
Sorry about my poor english.

reagrds
yi
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[Chris Angelico]

On Sat, Mar 30, 2013 at 2:09 PM, Roy Smith  wrote:
> In article ,
>  Chris Angelico  wrote:
>
>> Especially facepalm because there's some way to do this that's faster
>> than straight INSERT statements, and it's not clearly documented as
>> "hey, guys, if you want to dump loads of data in, use COPY instead"
>> (it might be that, I don't know, but usually COPY isn't directly
>> transliterable with INSERT).
>
> We're actually using REPLACE INTO.  For the initial data load, we could
> just as well do INSERT, but we need the REPLACE functionality as we roll
> in the daily incremental updates.
>
> This also explains why, even after provisioning our RDS instance for
> 2000 IOPS (that's AWS-speak for "we paid extra to get more disk
> bandwidth"), we didn't see any performance improvement!

Hmm. I heard around the forums that Amazon weren't that great at disk
bandwidth anyway, and that provisioning IO was often a waste of money.
But we never did all that much much research on Amazon I/O
performance; shortly after doing some basic benchmarking, we decided
that the cloud was a poor fit for our system model, and went looking
at dedicated servers with their own RAID storage right there on the
bus.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[rusi]

On Mar 30, 7:49 am, Chris Angelico  wrote:
> On Sat, Mar 30, 2013 at 1:44 PM, Roy Smith  wrote:
> > The slow way (i.e. "VALUES"), I'm inserting 1000 rows about every 2.4
> > seconds.  When I switch to "values", I'm getting more like 1000 rows in
> > 100 ms!
>
> > A truly breathtaking bug.
>
> *facepalm*
>
> Doubly facepalm because a regex could easily have tested for mixed case.
>
> Especially facepalm because there's some way to do this that's faster
> than straight INSERT statements, and it's not clearly documented as
> "hey, guys, if you want to dump loads of data in, use COPY instead"
> (it might be that, I don't know, but usually COPY isn't directly
> transliterable with INSERT).
>
> I agree. Breathtaking.
>
> ChrisA

I recently heard this:
A phone company needed to send out bulk-smses to its customers. It was
of the order of millions.
A (noob?) python programmer was assigned the task and used django with
whatever is the django orm.
It took of the order of weeks to send out the smses.
A python expert was called in.  He threw out the python and redid it
in SQL.
It was done in minutes.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[Tim Chase]

On 2013-03-29 21:19, Roy Smith wrote:
> We're doing it all in one transaction, on purpose.  We start with
> an initial dump, then get updates about once a day.  We want to
> make sure that the updates either complete without errors, or back
> out cleanly. If we ever had a partial daily update, the result
> would be a mess.

Having had to do some similarly-sized bulk data loads (in my case,
MSSqlServer at $JOB) couple other ideas occur to me:

1) I believe MySQL has a side-loading function (I'd have to go
digging for it; a quick google suggests a "LOAD DATA INFILE"
statement[1]) that allows you to load data from an external file such
as an XML or CSV file

2) Load into a temp table in testable batches, then do some sort of
batch insert into your main table.  Again, a quick google suggest the
"INSERT ... SELECT" syntax[2]

-tkc

[1]
http://dev.mysql.com/doc/refman/5.1/en/load-data.html

[2]
http://dev.mysql.com/doc/refman/5.0/en/insert-select.html
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[rusi]

On Mar 30, 8:13 am, rusi  wrote:

> It took of the order of weeks to send out the smses.

'Week' I think is more accurate.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()? (PS)

[Tim Chase]

On 2013-03-29 22:17, Tim Chase wrote:
> 2) Load into a temp table in testable batches, then do some sort of
> batch insert into your main table.  Again, a quick google suggest
> the "INSERT ... SELECT" syntax[2]

It looks like there's a corresponding "REPLACE INTO ... SELECT"
syntax[1], as you mention doing a REPLACE INTO rather than a straight
INSERT

-tkc

[1]
http://dev.mysql.com/doc/refman/5.0/en/replace.html
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[Roy Smith]

In article ,
 Chris Angelico  wrote:

> Hmm. I heard around the forums that Amazon weren't that great at disk
> bandwidth anyway, and that provisioning IO was often a waste of money.

Au, contraire.  I guess it all depends on what you're doing.  If you're 
CPU bound, increasing your I/O bandwidth won't help.  But, at least on 
our database (MongoDB) servers, we saw a huge performance boost when we 
started going for provisioned IO.

> But we never did all that much much research on Amazon I/O
> performance; shortly after doing some basic benchmarking, we decided
> that the cloud was a poor fit for our system model, and went looking
> at dedicated servers with their own RAID storage right there on the
> bus.

As far as I can tell, from a raw price/performance basis, they're pretty 
expensive.  But, from a convenience standpoint, it's hard to beat.

Case in point: We've been thinking about SSD as our next performance 
step-up.  One day, we just spun up some big honking machine, configured 
it with 2 TB of SSD, and played around for a while.  Wicked fast.  Then 
we shut it down.  That experiment probably cost us $10 or so, and we 
were able to run it on the spur of the moment.

Another example was last summer when we had a huge traffic spike because 
of a new product release.  Caught us by surprise how much new traffic it 
would generate.  Our site was in total meltdown.  We were able to spin 
up 10 new servers in an afternoon.  If we had to go out and buy 
hardware, have it shipped to us, figure out where we had rack space, 
power, network capacity, cooling, etc, we'd have been out of business 
before we got back on the air.

Yet another example.  We just (as in, while I've been typing this) had 
one of our servers go down.  Looks like the underlying hardware the VM 
was running on croaked, because when the instance came back up, it had a 
new IP address.  The whole event was over in a couple of minutes, with 
only minor disruption to the service.  And, presumably, there's some 
piece of hardware somewhere in Virginia that needs repairing, but that's 
not our problem.

The really big boys (Google, Facebook) run their own data centers.  But, 
some surprisingly large operations run out of AWS.  Netflix, for 
example.  The convenience and flexibility is worth a lot.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[Roy Smith]

In article 
,
 rusi  wrote:

> I recently heard this:
> A phone company needed to send out bulk-smses to its customers. It was
> of the order of millions.
> A (noob?) python programmer was assigned the task and used django with
> whatever is the django orm.
> It took of the order of weeks to send out the smses.
> A python expert was called in.  He threw out the python and redid it
> in SQL.
> It was done in minutes.

I'm not surprised.  It almost certainly wasn't the python that was the 
problem.  More than likely, he was doing some horribly inefficient 
database operations.

Certainly, in our case, performance is all about the database.  We 
mostly can't even measure the time we spend running Python code.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[Chris Angelico]

On Sat, Mar 30, 2013 at 2:36 PM, Roy Smith  wrote:
> In article ,
>  Chris Angelico  wrote:
>
>> Hmm. I heard around the forums that Amazon weren't that great at disk
>> bandwidth anyway, and that provisioning IO was often a waste of money.
>
> Au, contraire.  I guess it all depends on what you're doing.  If you're
> CPU bound, increasing your I/O bandwidth won't help.  But, at least on
> our database (MongoDB) servers, we saw a huge performance boost when we
> started going for provisioned IO.
>
> As far as I can tell, from a raw price/performance basis, they're pretty
> expensive.  But, from a convenience standpoint, it's hard to beat.

Yeah, I'm not saying you won't see performance go up - just that it's
going to be expensive compared to what you could do with a dedicated
server. The flexibility costs.

> Case in point: We've been thinking about SSD as our next performance
> step-up.  One day, we just spun up some big honking machine, configured
> it with 2 TB of SSD, and played around for a while.  Wicked fast.  Then
> we shut it down.  That experiment probably cost us $10 or so, and we
> were able to run it on the spur of the moment.

That is one thing the cloud is *awesome* for. "Hmm, I wonder.."
*half an hour later* "Now I know." *bill comes in* "That was cheap."

> Another example was last summer when we had a huge traffic spike because
> of a new product release.  Caught us by surprise how much new traffic it
> would generate.  Our site was in total meltdown.  We were able to spin
> up 10 new servers in an afternoon.  If we had to go out and buy
> hardware, have it shipped to us, figure out where we had rack space,
> power, network capacity, cooling, etc, we'd have been out of business
> before we got back on the air.

Yep. We're looking rather at server rental, from big honking data
centers (by the way, I never managed to figure this out - at what size
do things begin to honk? Larger than a bread box?), where hopefully
they would be able to deploy us more servers within a matter of hours.
Not as good as cloud (where you can have more servers in minutes), but
you described "in an afternoon" as your success story, so I'm guessing
most of the delay was an administrative one - the decision to actually
go ahead and spin those servers up. Unless you're automating the whole
thing (which is possible with the Amazon cloud, but not every client
will do it), that's always going to cost you.

> Yet another example.  We just (as in, while I've been typing this) had
> one of our servers go down.  Looks like the underlying hardware the VM
> was running on croaked, because when the instance came back up, it had a
> new IP address.  The whole event was over in a couple of minutes, with
> only minor disruption to the service.  And, presumably, there's some
> piece of hardware somewhere in Virginia that needs repairing, but that's
> not our problem.

Yeah, that's also a concern. And that works beautifully as long as
you're okay with that. It'll also happen more often with AWS than with
dedicated hardware, because there are more components to fail. That's
part of what wouldn't have fitted our server layout; we have a layer
on top of all that to manage monitoring (and, incidentally, we set up
a dozen home-grade laptops as a "server cluster" to test all those
systems, and knew within minutes of one of the laptops deciding that
it had crunched its last bits), and we really want stable IP addresses
for DNS and such. Cloud isn't bad, it was just a bad fit for us.

Side point: You mentioned SSDs. Are you aware of the fundamental risks
associated with them? Only a handful of SSD models are actually
trustworthy for databasing. Sure, they're fast, but can you afford
data corruption in the event of a power outage? Most SSDs will
cheerfully lie about fsync() and thus violate transactional integrity
(ACID compliance etc).

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[Roy Smith]

In article ,
 Chris Angelico  wrote:

> Side point: You mentioned SSDs. Are you aware of the fundamental risks
> associated with them? Only a handful of SSD models are actually
> trustworthy for databasing.

We haven't decided if we're going that route yet, but if we do, we will 
probably use do RAID SSD for added reliability.  We also have all our 
database servers in failover clusters, so we get added reliability that 
way too.

But, we have some runway left with more conventional technologies, so we 
don't need to decide for a while.  Ultimately, however, as reliability 
goes up and cost comes down, it's hard to imagine the SSD isn't going to 
be a big part of our lives at some point.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[Chris Angelico]

On Sat, Mar 30, 2013 at 3:10 PM, Roy Smith  wrote:
> In article ,
>  Chris Angelico  wrote:
>
>> Side point: You mentioned SSDs. Are you aware of the fundamental risks
>> associated with them? Only a handful of SSD models are actually
>> trustworthy for databasing.
>
> We haven't decided if we're going that route yet, but if we do, we will
> probably use do RAID SSD for added reliability.  We also have all our
> database servers in failover clusters, so we get added reliability that
> way too.

But will you know if you have corruption? Normally, transactional
integrity means:

1) If a transaction, from begin to commit, is not completely applied,
then it is completely not-applied; and
2) If a transaction that was affected by this one has been applied,
then so has this one.

SSDs that lie about fsync (and some hard disks lie too, as do some
operating systems and some file system drivers, but - under Linux at
least - it's possible to guarantee the OS and FS parts) can violate
both halves. Anything might have been written, anything might have
been missed. I did some tests with PostgreSQL on an SSD, and the
results were seriously scary.

> But, we have some runway left with more conventional technologies, so we
> don't need to decide for a while.  Ultimately, however, as reliability
> goes up and cost comes down, it's hard to imagine the SSD isn't going to
> be a big part of our lives at some point.

Yes. I hope that by then, the manufacturers will realize that TPS
isn't the only thing that matters. I'm sure SSDs will mature to the
point where we can trust all brands equally (or at least, most brands
- maybe it'll be like "server SSDs" and "desktop SSDs"?), but until
then, there aren't many options.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to find bad row with db api executemany()?

[Chris Angelico]

On Sat, Mar 30, 2013 at 3:19 PM, Dennis Lee Bieber
 wrote:
> I think MySQL is the only common DBMS with an extension on INSERT of
> allowing multiple records (I've not checked my Access 2010 docs, and my
> MSDE/SQL-Server books are in storage -- but SQLite3, Firebird, and
> PostgreSQL all seem to be "one INSERT = one record").

I don't know about performance, but syntactically at least, an INSERT
is certainly allowed to do multiple records. I do this all the time
for database dump/recreation, something like:

INSERT INTO table (field,field,field) VALUES
(value,value,value),(value,value,value);

I've done this in PostgreSQL, and I'm pretty sure also in MySQL. That
might be identical in performance to two separate statements, but at
least it's clearer.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

I wanna make wordlist cutter

[TheDigitalroot]

I imagine to make a wordlist cutter program with python pgrogramming . but I 
don't know too much with it . 

I think , first , I will make a user input as a wordlist with command 

eg , C:\john\Desktop\wordlist.txt

and than I will use a string to count

eg , Enter your count: 

at that time , my program assume count of numbers like notepad ++ which inlcude 
in wordlist.txt that i input with command .

How can i code for that , please guide me and help me .
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: I wanna make wordlist cutter

[TheDigitalroot]

nope , my bro i mean count number .

In notepad ++ , when we insert words like the follwoing ,
book
cook
meet
beat

we know 4 words because Notepad ++ show me 4 counts of words .

I mean that , bro 
thanks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: I wanna make wordlist cutter

[rusi]

On Mar 30, 10:09 am, TheDigitalroot  wrote:
> I imagine to make a wordlist cutter program with python pgrogramming . but I 
> don't know too much with it .
>
> I think , first , I will make a user input as a wordlist with command
>
> eg , C:\john\Desktop\wordlist.txt
>
> and than I will use a string to count
>
> eg , Enter your count:
>
> at that time , my program assume count of numbers like notepad ++ which 
> inlcude in wordlist.txt that i input with command .
>
> How can i code for that , please guide me and help me .

No idea what is 'wordlist cutter.'
Are you assuming that folks here should be upto speed with notepad++
vocabulary?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: I wanna make wordlist cutter

[TheDigitalroot]

 I mean , I wanna detect counts of words that i insert with command line .
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: I wanna make wordlist cutter

[rusi]

On Mar 30, 10:28 am, TheDigitalroot  wrote:
> nope , my bro i mean count number .
>
> In notepad ++ , when we insert words like the follwoing ,
> book
> cook
> meet
> beat
>
> we know 4 words because Notepad ++ show me 4 counts of words .
>
> I mean that , bro
> thanks

Still cant make out what you want.
Anyways heres a suggestion.
To start with forget about files; work with (triple quoted?) strings
And before that maybe a simple string representing a single line.
And before that maybe a list containing strings, where each string is
a word as you want.
Now do what you want with that list (Or tell us what you want done)
Then reverse-engineer back to files (Or if stuck tell us where)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: dir() vs print(dir()) in the embedded mode

[rusi]

On Mar 30, 2:04 am, Nick Gnedin  wrote:
> Folks,
>
> I have a newbie question: I am trying to embed Python into my
> application. While playing around, I noticed that the behavior of the
> interpreter in the embedded mode differs from the standalone one.
>
> Namely, in the standalone mode if I type dir(), I get a list of build-in
> symbols. In the embedded mode only print(dir()) does that, while just
> dir() returns silently.

This:
http://docs.python.org/2/library/custominterp.html
is (or can be conceptualized as providing) the interactive interpreter
behavior.
IOW the interactive interpreter (aka REPL) is a packaging of that.

>
> Is there a way to intercept the output of dir() (and all other commands)
> and display them to the user?

You are looking at it upside down (as MRAB hinted).
You dont want to intercept (aka monkeypatch or subtract) from a
functional ie value-returning dir etc
Instead you want to add (your own custom) REPL behavior.

>
> Here is an example code that illustrates the behavior (the first call to
> PyRun_SimpleString() returns silently).
>
> Many thanks for your future hints,
>
> Nick
>
> #include 
>
> int main()
> {
>    Py_Initialize();
>    PyRun_SimpleString("dir()");
>    printf("-\n");
>    PyRun_SimpleString("print(dir())");
>    Py_Finalize();
>
>    return 0;
>
>
>
>
>
>
>
> }

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: I wanna make wordlist cutter

[TheDigitalroot]

On Saturday, March 30, 2013 9:09:06 AM UTC+4, TheDigitalroot wrote:
> I imagine to make a wordlist cutter program with python pgrogramming . but I 
> don't know too much with it . 
> 
> 
> 
> I think , first , I will make a user input as a wordlist with command 
> 
> 
> 
> eg , C:\john\Desktop\wordlist.txt
> 
> 
> 
> and than I will use a string to count
> 
> 
> 
> eg , Enter your count: 
> 
> 
> 
> at that time , my program assume count of numbers like notepad ++ which 
> inlcude in wordlist.txt that i input with command .
> 
> 
> 
> How can i code for that , please guide me and help me .
thanks bro , all of the others can give me a dvice or sample codes 
-- 
http://mail.python.org/mailman/listinfo/python-list