[Twisted-Python] util.quote deprecated
Hi, what is a proper way to escape user input in database query strings? I've used quote from twisted.enterprise.util, but it is deprecated now. Is there any other module for this purpose? Thanks, Pet ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] util.quote deprecated
On Tue, Mar 3, 2009 at 13:17, Pet wrote:
> Hi,
>
> what is a proper way to escape user input in database query strings?
> I've used quote from twisted.enterprise.util, but it is deprecated now.
> Is there any other module for this purpose?
I can't do better than quote the sqlite documentation. I Hope this helps!
http://docs.python.org/library/sqlite3.html
"""
Usually your SQL operations will need to use values from Python
variables. You shouldn’t assemble your query using Python’s string
operations because doing so is insecure; it makes your program
vulnerable to an SQL injection attack.
Instead, use the DB-API’s parameter substitution. Put ? as a
placeholder wherever you want to use a value, and then provide a tuple
of values as the second argument to the cursor’s execute() method.
(Other database modules may use a different placeholder, such as %s or
:1.) For example:
# Never do this -- insecure!
symbol = 'IBM'
c.execute("... where symbol = '%s'" % symbol)
# Do this instead
t = (symbol,)
c.execute('select * from stocks where symbol=?', t)
# Larger example
for t in [('2006-03-28', 'BUY', 'IBM', 1000, 45.00),
('2006-04-05', 'BUY', 'MSOFT', 1000, 72.00),
('2006-04-06', 'SELL', 'IBM', 500, 53.00),
]:
c.execute('insert into stocks values (?,?,?,?,?)', t)
"""
--
Amaury Forgeot d'Arc
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] util.quote deprecated
On Tue, 3 Mar 2009 13:17:48 +0100, Pet wrote:
Hi,
what is a proper way to escape user input in database query strings?
I've used quote from twisted.enterprise.util, but it is deprecated now.
Is there any other module for this purpose?
The proper way is with "bind parameters". This keeps SQL separate from
data and removes the entire category of bugs due to misquoting. The way
to use bind parameters is to pass the SQL string as a separate argument
from the SQL data. Using DB-API 2.0, this means something like:
cursor.execute("SELECT foo FROM bar WHERE baz = ?", (3,))
Using ADBAPI, it means something very similar:
connpool.runQuery("SELECT foo FROM bar WHERE baz = ?", (3,))
Different database adapters use different syntaxes for the "?" part. The
`paramstyle´ attribute of the DB-API 2.0 module tells you what syntax a
particular module uses. See the DB-API 2.0 PEP
(< http://www.python.org/dev/peps/pep-0249/>) for details.
Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] util.quote deprecated
On Tue, Mar 03, 2009 at 01:17:48PM +0100, Pet wrote:
> what is a proper way to escape user input in database query strings?
> I've used quote from twisted.enterprise.util, but it is deprecated now.
> Is there any other module for this purpose?
The correct way to escape user input is not to do it at all, but rather
to leave it up to the DB-API module you're using:
from twisted.enterprise.adbapi import ConnectionPool
pool = ConnectionPool("psycopg2")
d = pool.runQuery("""
SELECT *
FROM students
WHERE name = %s
""", "Robert '); DROP TABLE students;--")
Note that although I've used "%s" in the query, this is not normal
Python string-formatting, the "%s" is just tells the DB-API module I'm
using (in this case, psycopg2 for PostgreSQL) to quote one of the extra
parameters and insert in that spot. Look up "paramstyle" in the DB-API
spec[1] and the documentation for the DB-API module you're using for
more details.
[1] http://www.python.org/dev/peps/pep-0249/
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] util.quote deprecated
On Tue, Mar 3, 2009 at 2:04 PM, Tim Allen wrote:
> On Tue, Mar 03, 2009 at 01:17:48PM +0100, Pet wrote:
> > what is a proper way to escape user input in database query strings?
> > I've used quote from twisted.enterprise.util, but it is deprecated now.
> > Is there any other module for this purpose?
>
> The correct way to escape user input is not to do it at all, but rather
> to leave it up to the DB-API module you're using:
>
>from twisted.enterprise.adbapi import ConnectionPool
>
>pool = ConnectionPool("psycopg2")
>d = pool.runQuery("""
>SELECT *
>FROM students
>WHERE name = %s
>""", "Robert '); DROP TABLE students;--")
>
> Note that although I've used "%s" in the query, this is not normal
> Python string-formatting, the "%s" is just tells the DB-API module I'm
> using (in this case, psycopg2 for PostgreSQL) to quote one of the extra
> parameters and insert in that spot. Look up "paramstyle" in the DB-API
> spec[1] and the documentation for the DB-API module you're using for
> more details.
>
> [1] http://www.python.org/dev/peps/pep-0249/
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
Thanks for all your answers! It works fine in that way. But what, if I
compose my query? For example:
def getData(self, type=''):
id = 1
if type:
str = " AND mytype = %s " % type
str = ''
query = "SELECT * FROM table WHERE id = %s %s " % (id,str)
cur.execute(query)
I mean, str part is not always there and I need escape it only if type is
passed to function
Thanks, Pet
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] too many file descriptors in select
I have started to get this message coming up on a fairly regular basis now. I have an application that is very simple and has run by and large without problem for about 7 months. But over the pas 4 - 6 weeks has started to produce this error 'too many file descriptors in select'. The application basically falls over at this point, since it can't service any more requests. I am running on Windows XP sp2, python 2.4.4, twisted 2.5. The application consists of about 150+ vehicles sending back gps tracks every 20 seconds. We collect these, log them, store them in a database and then update a current status. So nothing very elaborate. I have been through the archives to search for some help and have found some discussion of the problem going back to 2004, but nothing that looks like something I could get a handle on to solving my problem. In fact most of the information seems rather inconclusive especially with regard to Windows. If there is a 'so-called solution' with an update or some sort of workround- that would be great. Or is it a practical proposition to attempt to trap the error myself and potentially restart the service or the whole application. Or is there a different and better path to take. Of course I am not sure how I would get in to pick up the exception and deal with it. So if that is a starter I wouldn't object to some help on that. Anyway thanks for any feedback. John Aherne ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] too many file descriptors in select
John Aherne wrote: I have started to get this message coming up on a fairly regular basis now. I have an application that is very simple and has run by and large without problem for about 7 months. But over the pas 4 - 6 weeks has started to produce this error 'too many file descriptors in select'. The application basically falls over at this point, since it can't service any more requests. Assuming that you've made sure you aren't leaking any connections, you are still faced with the problem of that the select module on Windows is limited to 512 file descriptors. Your options: 1. Switch to Linux, where you can use other reactors with a much higher limit, e.g. epoll. 2. Use the IOCP reactor on Windows, which isn't as well tested and is missing some features, in particular SSL support. 3. Recompile the select module on Windows with a higher limit; basically it's some C constant you redefine, it's probably right at the top of the file. You just get the Python tarball, and rebuild the appropriate module (or everything and just extract the appropriate PYD). -Itamar, still living the voice-recognition life ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted developers: please donate your time!
On Tue, Mar 3, 2009 at 12:41 AM, wrote: > > On 2 Mar, 02:08 pm, ita...@itamarst.org wrote: >> >> On Tue, 2009-02-24 at 21:25 +1300, Michael Hudson wrote: >>> >>> 2009/2/24 Itamar Shtull-Trauring : >>> > As part of the TSF's fundraising efforts, we are trying to get upfront >>> > donations of time you will spend developing Twisted. It will then get >>> > matched by a donor, if we are successful in getting this grant. So if >>> > you're planning on working on Twisted anyway this year, your work will >>> > count twice as much! >>> > >>> > So, if you're interested, please reply, saying something like "I will >>> > spend two weeks working on Gopher support over the next year." >>> >>> I don't entirely understand, but I will commit to spending the >>> equivalent of two working weeks as a general review monkey over the >>> next year. Is that the sort of thing you wanted to hear? >> >> Yes - thanks to you and all thee rest of the people who are >> volunteering! Anyone else willing to step up? I will commit to spending 2 weeks on Conch in the coming year. -p -- Paul Swartz paulswartz at gmail dot com http://paulswartz.net/ AIM: z3penguin ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] too many file descriptors in select
Thanks for the information. Looks pretty conclusive. Thinking about it, I reckon I must be leaking connections. So I'll go and look. Thanks again John Aherne On Tue, Mar 3, 2009 at 5:24 PM, Itamar Shtull-Trauring wrote: > John Aherne wrote: > >> I have started to get this message coming up on a fairly regular basis >> now. >> >> I have an application that is very simple and has run by and large without >> problem for about 7 months. But over the pas 4 - 6 weeks has started to >> produce this error 'too many file descriptors in select'. The application >> basically falls over at this point, since it can't service any more >> requests. >> > Assuming that you've made sure you aren't leaking any connections, you are > still faced with the problem of that the select module on Windows is limited > to 512 file descriptors. Your options: > > 1. Switch to Linux, where you can use other reactors with a much higher > limit, e.g. epoll. > 2. Use the IOCP reactor on Windows, which isn't as well tested and is > missing some features, in particular SSL support. > 3. Recompile the select module on Windows with a higher limit; basically > it's some C constant you redefine, it's probably right at the top of the > file. You just get the Python tarball, and rebuild the appropriate module > (or everything and just extract the appropriate PYD). > > -Itamar, still living the voice-recognition life > > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
