Re: how to convert from Decimal('1.23456789') to Decimal('1.234')
On Mar 23, 4:40 pm, valpa wrote:
> I only need the 3 digits after '.'
>
> Is there any way other than converting from/to string?
I'm not sure if this is the canonical way but it works:
>>> d = Decimal('1.23456789')
>>> three_places = Decimal('0.001') # or anything that has the exponent depth
>>> you want
>>> d.quantize(three_places, 'ROUND_DOWN')
Decimal('1.234')
--
http://mail.python.org/mailman/listinfo/python-list
Re: Unicode problem in ucs4
On Mar 20, 5:47 pm, "M.-A. Lemburg" wrote:
> On 2009-03-20 12:13, abhi wrote:
>
>
>
>
>
> > On Mar 20, 11:03 am, "Martin v. Löwis" wrote:
> >>> Any idea on why this is happening?
> >> Can you provide a complete example? Your code looks correct, and should
> >> just work.
>
> >> How do you know the result contains only 't' (i.e. how do you know it
> >> does not contain 'e', 's', 't')?
>
> >> Regards,
> >> Martin
>
> > Hi Martin,
> > Here is the code:
> > unicodeTest.c
>
> > #include
>
> > static PyObject *unicode_helper(PyObject *self,PyObject *args){
> > PyObject *sampleObj = NULL;
> > Py_UNICODE *sample = NULL;
>
> > if (!PyArg_ParseTuple(args, "O", &sampleObj)){
> > return NULL;
> > }
>
> > // Explicitly convert it to unicode and get Py_UNICODE value
> > sampleObj = PyUnicode_FromObject(sampleObj);
> > sample = PyUnicode_AS_UNICODE(sampleObj);
> > wprintf(L"database value after unicode conversion is : %s\n",
> > sample);
>
> You have to use PyUnicode_AsWideChar() to convert a Python
> Unicode object to a wchar_t representation.
>
> Please don't make any assumptions on what Py_UNICODE maps
> to and always use the the Unicode API for this. It is designed
> to provide a portable interface and will not do more conversion
> work than necessary.
>
>
>
>
>
> > return Py_BuildValue("");
> > }
>
> > static PyMethodDef funcs[]={{"unicodeTest",(PyCFunction)
> > unicode_helper,METH_VARARGS,"test ucs2, ucs4"},{NULL}};
>
> > void initunicodeTest(void){
> > Py_InitModule3("unicodeTest",funcs,"");
> > }
>
> > When i install this unicodeTest on python ucs2 wprintf prints whatever
> > is passed eg
>
> > import unicodeTest
> > unicodeTest.unicodeTest("hello world")
> > database value after unicode conversion is : hello world
>
> > but it prints the following on ucs4 configured python:
> > database value after unicode conversion is : h
>
> > Regards,
> > Abhigyan
> > --
> >http://mail.python.org/mailman/listinfo/python-list
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source (#1, Mar 20 2009)>>>
> Python/Zope Consulting and Support ... http://www.egenix.com/
> >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
>
>
>
> ::: Try our new mxODBC.Connect Python Database Interface for free !
>
> eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
> Registered at Amtsgericht Duesseldorf: HRB 46611
> http://www.egenix.com/company/contact/- Hide quoted text -
>
> - Show quoted text -- Hide quoted text -
>
> - Show quoted text -
Hi Mark,
Thanks for the help. I tried PyUnicode_AsWideChar() but I am
getting the same result i.e. only the first letter.
sample code:
#include
static PyObject *unicode_helper(PyObject *self,PyObject *args){
PyObject *sampleObj = NULL;
wchar_t *sample = NULL;
int size = 0;
if (!PyArg_ParseTuple(args, "O", &sampleObj)){
return NULL;
}
// use wide char function
size = PyUnicode_AsWideChar(databaseObj, sample,
PyUnicode_GetSize(databaseObj));
printf("%d chars are copied to sample\n", size);
wprintf(L"database value after unicode conversion is : %s\n",
sample);
return Py_BuildValue("");
}
static PyMethodDef funcs[]={{"unicodeTest",(PyCFunction)
unicode_helper,METH_VARARGS,"test ucs2, ucs4"},{NULL}};
void initunicodeTest(void){
Py_InitModule3("unicodeTest",funcs,"");
}
This prints the following when input value is given as "test":
4 chars are copied to sample
database value after unicode conversion is : t
Any ideas?
-
Abhigyan
--
http://mail.python.org/mailman/listinfo/python-list
Re: How can I do bit operation on python float value
On Mar 23, 5:44 pm, valpa wrote: > I have a python float 1.2345678. I know that it is stored as a double > in C type. And I know it actually is 1010101010101 -like format. Then > I want to do some bit operation on it. How? > > Sure, I want a float output when I finish the operation. import struct pack it into a str [Python 2.X] using d format unpack it using Q format, gets you a 64-bit unsigned int do "some bit operation" using & | ^ ~ << >> operators pack it into a str using Q format unpack it into a float using d format What is "some bit operation"? Perhaps there is already a high-level way of doing some of what you want e.g. in the math module: frexp, ldexp, isnan, isinf, ... HTH John -- http://mail.python.org/mailman/listinfo/python-list
Re: how to convert from Decimal('1.23456789') to Decimal('1.234')
In that case, I usually use # when rounding is proper, s = '1.23456789' print round(float(s)) or # when cut out is proper, from math import floor print floor(float(s)*1000)/1000 Hyunchul valpa wrote: I only need the 3 digits after '.' Is there any way other than converting from/to string? -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: how to convert from Decimal('1.23456789') to Decimal('1.234')
On Sun, 22 Mar 2009 23:40:38 -0700, valpa wrote:
> I only need the 3 digits after '.'
>
> Is there any way other than converting from/to string?
You should Read the Fine Manual:
http://docs.python.org/library/decimal.html
[quote]
The quantize() method rounds a number to a fixed exponent. This method is
useful for monetary applications that often round results to a fixed
number of places:
>>> Decimal('7.325').quantize(Decimal('.01'), rounding=ROUND_DOWN)
Decimal('7.32')
>>> Decimal('7.325').quantize(Decimal('1.'), rounding=ROUND_UP)
Decimal('8')
[end quote]
In my opinion, that's hideously ugly, but you can create a helper
function very easily:
def round(dec, places, rounding=decimal.ROUND_HALF_UP):
return dec.quantize(decimal.Decimal(str(10**-places)), rounding)
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: splitting a large dictionary into smaller ones
On Sun, 22 Mar 2009 23:10:21 -0400, Terry Reedy wrote: > Searching for a key in, say, 10 dicts will be slower than searching for > it in just one. The only reason I would do this would be if the dict > had to be split, say over several machines. But then, you could query > them in parallel. That surely depends on the number of collisions? With millions of keys, the number of collisions could be high (although it almost certainly won't be). As I understand it, searching a dict is O(1) on average, but if there are collisions it can degrade to O(m) (where m is the number of items colliding for some hash). In principle, m could be large. Hypothetically speaking, if you could predict where collisions occur and split your data into multiple smaller dicts with fewer collisions, you could delegate to one of the smaller dicts and reduce O(m) lookups to 2*O(1) lookups, which of course is just O(1). Of course, this entirely assumes that collisions in Python's hash function are both predictable and frequent in the OP's data. If they're not both, then it's unlikely that this suggestion will be of practical use. Could be an interesting experiment though :) -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: lipo: can't figure out the architecture type of
On Mar 19, 4:48 pm, Vijayendra Bapte wrote: > Hi, > > I am getting an gcc compilation error while installing FSEvents > (http://pypi.python.org/packages/source/p/pyobjc-framework-FSEvents/ > pyobjc-framework-FSEvents-2.2b1.tar.gz) package on my Mac (OS X > 10.4.11, Intel Core Duo 32 bit processor, Python2.6.1, gcc: i686-apple- > darwin8-gcc-4.0.1) > > gcc failed while building the "FSEvents._callbacks" extension on $ > python setup.py install > > here is the error trace... > > running install_lib > running build_py > running build_ext > building 'FSEvents._callbacks' extension > gcc -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk - > fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -O3 -I/Library/ > Frameworks/Python.framework/Versions/2.6/include/python2.6 -c Modules/ > _callbacks.m -o build/temp.macosx-10.3-i386-2.6/Modules/_callbacks.o - > O0 > Modules/_callbacks.m:60: error: parse error before > 'm_python_context_template' > Modules/_callbacks.m:62: warning: excess elements in scalar > initializer > Modules/_callbacks.m:62: warning: (near initialization for > 'm_python_context_template') > Modules/_callbacks.m:63: warning: excess elements in scalar > initializer > . > . > . > . > (Error ending with) > Modules/_callbacks.m:133: error: previous definition of 'result' was > here > Modules/_callbacks.m:353: error: parse error before 'FSEventStreamRef' > lipo: can't figure out the architecture type of: /var/tmp// > cco6kalc.out > error: command 'gcc' failed with exit status 1 > > Could someone help me in solving this compilation error? Finally after doing lots of search on google, I found that there is no support of FSEvents for Tiger. It wraps some API's that aren't available on Tiger systems. Alternatives: (1) Using kqueue/kevent: It might be possible to emulate the behavior using kevent/kqueue. But the problem here is, it does not provide the granularity as FSEvents i.e I you cannot detect which file is modified/ added/deleted inside the marked directory. (2) The other alternative is polling for changes. -- http://mail.python.org/mailman/listinfo/python-list
Re: Unicode problem in ucs4
On Mar 23, 6:18 pm, abhi wrote:
[snip]
> Hi Mark,
> Thanks for the help. I tried PyUnicode_AsWideChar() but I am
> getting the same result i.e. only the first letter.
>
> sample code:
>
> #include
>
> static PyObject *unicode_helper(PyObject *self,PyObject *args){
> PyObject *sampleObj = NULL;
> wchar_t *sample = NULL;
> int size = 0;
>
> if (!PyArg_ParseTuple(args, "O", &sampleObj)){
> return NULL;
> }
>
> // use wide char function
> size = PyUnicode_AsWideChar(databaseObj, sample,
> PyUnicode_GetSize(databaseObj));
What is databaseObj??? Copy/paste the *actual* code that you compiled
and ran.
> printf("%d chars are copied to sample\n", size);
> wprintf(L"database value after unicode conversion is : %s\n",
> sample);
> return Py_BuildValue("");
>
> }
>
> static PyMethodDef funcs[]={{"unicodeTest",(PyCFunction)
> unicode_helper,METH_VARARGS,"test ucs2, ucs4"},{NULL}};
>
> void initunicodeTest(void){
> Py_InitModule3("unicodeTest",funcs,"");
>
> }
>
> This prints the following when input value is given as "test":
> 4 chars are copied to sample
> database value after unicode conversion is : t
[presuming littleendian] The ucs4 string will look like "\t\0\0\0e
\0\0\0s\0\0\0t\0\0\0" in memory. I suspect that your wprintf is
grokking only 16-bit doodads -- "\t\0" is printed and then "\0\0" is
end-of-string. Try your wprintf on sample[0], ..., sample[3] in a loop
and see what you get. Use bog-standard printf to print the hex
representation of each of the 16 bytes starting at the address sample
is pointing to.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Async serial communication/threads sharing data
"Nick Craig-Wood" wrote: > > I wrote a serial port to TCP proxy (with logging) with twisted. The > problem I had was that twisted serial ports didn't seem to have any > back pressure. By that I mean I could pump data into a 9600 baud > serial port at 10 Mbit/s. Twisted would then buffer the data for me > using 10s or 100s or Megabytes of RAM. No data would be lost, but > there would be hours of latency and my program would use up all my RAM > and explode. > > What I wanted to happen was for twisted to stop taking the data when > the serial port buffer was full and to only take the data at 9600 > baud. > > I never did solve that problem :-( > Not sure if this is Twisted's fault - do python sockets have automatic back pressure? Do Linux sockets have back pressure? - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Unicode problem in ucs4
On Mar 23, 6:41 pm, John Machin had a severe attack of backslashitis: > [presuming littleendian] The ucs4 string will look like "\t\0\0\0e > \0\0\0s\0\0\0t\0\0\0" in memory. I suspect that your wprintf is > grokking only 16-bit doodads -- "\t\0" is printed and then "\0\0" is > end-of-string. Try your wprintf on sample[0], ..., sample[3] in a loop > and see what you get. Use bog-standard printf to print the hex > representation of each of the 16 bytes starting at the address sample > is pointing to. and typed \t in two places where he should have typed t :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: splitting a large dictionary into smaller ones
On Mar 23, 1:32 pm, per wrote: > hi all, > > i have a very large dictionary object that is built from a text file > that is about 800 MB -- it contains several million keys. ideally i > would like to pickle this object so that i wouldnt have to parse this > large file to compute the dictionary every time i run my program. > however currently the pickled file is over 300 MB and takes a very > long time to write to disk - even longer than recomputing the > dictionary from scratch. What version of Python are you using, if 2.X are you using pickle or cPickle, on what platform, and what *exactly* is the code that you are using to call *ickle.dump() and *ickle.load()? What datatype(s) are the keys? the values? If str/unicode, what are average lengths? -- http://mail.python.org/mailman/listinfo/python-list
Re: how to convert from Decimal('1.23456789') to Decimal('1.234')
On Mar 23, 7:01 am, alex23 wrote:
> On Mar 23, 4:40 pm, valpa wrote:
>
> > I only need the 3 digits after '.'
>
> > Is there any way other than converting from/to string?
>
> I'm not sure if this is the canonical way but it works:
>
> >>> d = Decimal('1.23456789')
> >>> three_places = Decimal('0.001') # or anything that has the exponent depth
> >>> you want
> >>> d.quantize(three_places, 'ROUND_DOWN')
>
> Decimal('1.234')
Yes, that's the official 'right way'. There's
also a _rescale method that does this:
>>> Decimal('1.23456789')._rescale(-3, 'ROUND_HALF_EVEN')
Decimal('1.235')
... but as the leading underscore indicates, it's
private and undocumented, so you shouldn't rely on it
not to change or disappear in a future version.
The two methods are subtly different, in that the
quantize method respects the current context, while
the _rescale method ignores it. For example:
>>> getcontext().prec = 3
>>> Decimal('1.23456789').quantize(Decimal('0.001'))
Traceback (most recent call last):
File "", line 1, in
File "/Users/dickinsm/python_source/trunk/Lib/decimal.py", line
2364, in quantize
'quantize result has too many digits for current context')
File "/Users/dickinsm/python_source/trunk/Lib/decimal.py", line
3735, in _raise_error
raise error(explanation)
decimal.InvalidOperation: quantize result has too many digits for
current context
>>> Decimal('1.23456789')._rescale(-3, 'ROUND_DOWN')
Decimal('1.234')
[61114 refs]
Mark
--
http://mail.python.org/mailman/listinfo/python-list
Re: Obtaining the attributes and properties of a folder recursively.
venutaurus...@gmail.com wrote:
On Mar 21, 3:05 pm, Tim Golden wrote:
venutaurus...@gmail.com wrote:
Thank you Sir for your reply. It is working for me. But is failing if
I have Unicode characters in my path. I tried giving a 'u' in front of
the path but still it fails at f.createdat. Does it support Unicode
Characters?
This the traceback which I got while running the above python script
on my data:
---
\\?\C:\JPDump\AGIT_FSDM\Unicode\Arabic files\Arabicְדזךמאּשּׁתּ
פּפֿשּﭏכּטּךּאָשּׁבֿפֿװײ״טצתא
In fact, to give me something to test against, would you
be able to send me the result of running this code, please?
It's just listing the names of the files under that particular
directly, encoded as utf8 so they'll survive transit:
import os
from winsys import fs
PATH = r"\\?\C:\JPDump\AGIT_FSDM\Unicode\Arabic files"
f = open ("arabic-files.txt", "wb")
for filename in fs.listdir (PATH):
f.write (filename.encode ("utf8") + "\r\n")
f.close ()
os.startfile ("arabic-files.txt")
Thanks
TJG
This is the output for your code Sir. But I think os.stat is
sufficient to satisfy all my requirements. May be we just have to make
the path itself a Unicode path.
Arabicְדזךמאּשּׁתּפּפֿשּﭏכּטּךּאָשּׁבֿפֿװײ״טצתא
Arabicְדזךמאּשּׁתּפּפֿשּﭏכּטּךּאָשּׁבֿפֿװײ״טצתא.doc
Arabicְדזךמאּשּׁתּפּפֿשּﭏכּטּךּאָשּׁבֿפֿװײ״טצתאnew.doc
الأبجدي.txt
تعلم اللغة الإيطالية مجان.doc
دروس مجانية في اللغة الإنجليزي.txt
دروس مجانية في اللغة الإنجليزي1.txt
Yes, thanks to your question I discovered one or two small
bugs in the WinSys unicode handling, hopefully now fixed.
But, I agree, you can use os.stat to do pretty much all
you want by passing unicode filenames. You'll also need
to use the win32security functions to get the ACL stuff.
TJG
--
http://mail.python.org/mailman/listinfo/python-list
Re: How can I do bit operation on python float value
On 3月23日, 上午3时18分, John Machin wrote: > On Mar 23, 5:44 pm, valpa wrote: > > > I have a python float 1.2345678. I know that it is stored as a double > > in C type. And I know it actually is 1010101010101 -like format. Then > > I want to do some bit operation on it. How? > > > Sure, I want a float output when I finish the operation. > > import struct > pack it into a str [Python 2.X] using d format > unpack it using Q format, gets you a 64-bit unsigned int > do "some bit operation" using & | ^ ~ << >> operators > pack it into a str using Q format > unpack it into a float using d format > > What is "some bit operation"? Perhaps there is already a high-level > way of doing some of what you want e.g. in the math module: frexp, > ldexp, isnan, isinf, ... > > HTH > John Thanks John! Yes, I want to do a & operation. The pack way is to convert float to/from string, Is it efficient? valpa -- http://mail.python.org/mailman/listinfo/python-list
Re: python 3, subclassing TextIOWrapper.
For D. Murray's suggestion---I think that we programmers have to learn the idiom. We don't always control open, such as subprocess.Popen(). Thank you. I hope these thoughts help with issue 5513 and the related questions to follow about complete removal of file in python3. Opening the file in binary mode for text behavior was not obvious to me, but makes good sense now that you've explained the further nesting. -- http://mail.python.org/mailman/listinfo/python-list
Re: how to convert from Decimal('1.23456789') to Decimal('1.234')
On Mar 23, 6:40 am, valpa wrote:
> I only need the 3 digits after '.'
>
> Is there any way other than converting from/to string?
And in Python 3.0, just use the built-in round function:
>>> from decimal import Decimal
>>> round(Decimal('1.23456789'), 3)
Decimal('1.235')
This uses the rounding specified by the context, so if
you want a rounding mode other than the default
ROUND_HALF_EVEN, just set the context rounding
appropriately:
>>> from decimal import getcontext, ROUND_DOWN
>>> getcontext().rounding = ROUND_DOWN
>>> round(Decimal('1.23456789'), 3)
Decimal('1.234')
Mark
--
http://mail.python.org/mailman/listinfo/python-list
Re: How can I do bit operation on python float value
On Mar 23, 8:44 am, valpa wrote: > Yes, I want to do a & operation. Right, but what & operation, and why? Are you trying to do something that's mathematically meaningful? If so, there may be a better (more portable/faster/clearer) way than bit-twiddling. Mark -- http://mail.python.org/mailman/listinfo/python-list
Re: Unicode problem in ucs4
On 2009-03-23 08:18, abhi wrote:
> On Mar 20, 5:47 pm, "M.-A. Lemburg" wrote:
>>> unicodeTest.c
>>> #include
>>> static PyObject *unicode_helper(PyObject *self,PyObject *args){
>>>PyObject *sampleObj = NULL;
>>>Py_UNICODE *sample = NULL;
>>> if (!PyArg_ParseTuple(args, "O", &sampleObj)){
>>> return NULL;
>>> }
>>> // Explicitly convert it to unicode and get Py_UNICODE value
>>> sampleObj = PyUnicode_FromObject(sampleObj);
>>> sample = PyUnicode_AS_UNICODE(sampleObj);
>>> wprintf(L"database value after unicode conversion is : %s\n",
>>> sample);
>> You have to use PyUnicode_AsWideChar() to convert a Python
>> Unicode object to a wchar_t representation.
>>
>> Please don't make any assumptions on what Py_UNICODE maps
>> to and always use the the Unicode API for this. It is designed
>> to provide a portable interface and will not do more conversion
>> work than necessary.
>
> Hi Mark,
> Thanks for the help. I tried PyUnicode_AsWideChar() but I am
> getting the same result i.e. only the first letter.
>
> sample code:
>
> #include
>
> static PyObject *unicode_helper(PyObject *self,PyObject *args){
> PyObject *sampleObj = NULL;
> wchar_t *sample = NULL;
> int size = 0;
>
> if (!PyArg_ParseTuple(args, "O", &sampleObj)){
> return NULL;
> }
>
> // use wide char function
> size = PyUnicode_AsWideChar(databaseObj, sample,
> PyUnicode_GetSize(databaseObj));
The 3. argument is the buffer size in bytes, not code points.
The result will require sizeof(wchar_t) * PyUnicode_GetSize(databaseObj)
bytes without a trailing NUL, otherwise sizeof(wchar_t) *
(PyUnicode_GetSize(databaseObj) + 1).
You also have to allocate the buffer to store the wchar_t data in.
Passing in a NULL pointer will result in a seg fault. The function
does not allocate a buffer for you:
/* Copies the Unicode Object contents into the wchar_t buffer w. At
most size wchar_t characters are copied.
Note that the resulting wchar_t string may or may not be
0-terminated. It is the responsibility of the caller to make sure
that the wchar_t string is 0-terminated in case this is required by
the application.
Returns the number of wchar_t characters copied (excluding a
possibly trailing 0-termination character) or -1 in case of an
error. */
PyAPI_FUNC(Py_ssize_t) PyUnicode_AsWideChar(
PyUnicodeObject *unicode, /* Unicode object */
register wchar_t *w,/* wchar_t buffer */
Py_ssize_t size /* size of buffer */
);
> printf("%d chars are copied to sample\n", size);
> wprintf(L"database value after unicode conversion is : %s\n",
> sample);
> return Py_BuildValue("");
>
> }
>
>
> static PyMethodDef funcs[]={{"unicodeTest",(PyCFunction)
> unicode_helper,METH_VARARGS,"test ucs2, ucs4"},{NULL}};
>
> void initunicodeTest(void){
> Py_InitModule3("unicodeTest",funcs,"");
>
> }
>
> This prints the following when input value is given as "test":
> 4 chars are copied to sample
> database value after unicode conversion is : t
>
> Any ideas?
>
> -
> Abhigyan
> --
> http://mail.python.org/mailman/listinfo/python-list
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Mar 23 2009)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
2009-03-19: Released mxODBC.Connect 1.0.1 http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free !
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
--
http://mail.python.org/mailman/listinfo/python-list
Re: how to convert from Decimal('1.23456789') to Decimal('1.234')
On Mar 23, 2:24�am, Steven D'Aprano
wrote:
> On Sun, 22 Mar 2009 23:40:38 -0700, valpa wrote:
> > I only need the 3 digits after '.'
>
> > Is there any way other than converting from/to string?
>
> You should Read the Fine Manual:
>
> http://docs.python.org/library/decimal.html
>
> [quote]
> The quantize() method rounds a number to a fixed exponent. This method is
> useful for monetary applications that often round results to a fixed
> number of places:
>
> >>> Decimal('7.325').quantize(Decimal('.01'), rounding=ROUND_DOWN)
> Decimal('7.32')
> >>> Decimal('7.325').quantize(Decimal('1.'), rounding=ROUND_UP)
>
> Decimal('8')
>
> [end quote]
>
> In my opinion, that's hideously ugly,
In looking at this for the first time, it struck
me as funny why the first argument to quantize
even requires a number since you can pass a
Conext as an argument. But if you do, precision
is ignored. Quantize isn't the way to do that.
> but you can create a helper
> function very easily:
>
> def round(dec, places, rounding=decimal.ROUND_HALF_UP):
> � � return dec.quantize(decimal.Decimal(str(10**-places)), rounding)
Still ugly. I would do this:
>>> a = Decimal('1.23456789')
>>> for i in xrange(1,6):
print Context.create_decimal(Context(i,ROUND_DOWN),a)
1
1.2
1.23
1.234
1.2345
>>> print a
1.23456789
>
> --
> Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: Async serial communication/threads sharing data
Jean-Paul Calderone wrote:
> On Sun, 22 Mar 2009 12:30:04 -0500, Nick Craig-Wood
> wrote:
> >I wrote a serial port to TCP proxy (with logging) with twisted. The
> >problem I had was that twisted serial ports didn't seem to have any
> >back pressure. By that I mean I could pump data into a 9600 baud
> >serial port at 10 Mbit/s. Twisted would then buffer the data for me
> >using 10s or 100s or Megabytes of RAM. No data would be lost, but
> >there would be hours of latency and my program would use up all my RAM
> >and explode.
> >
> >What I wanted to happen was for twisted to stop taking the data when
> >the serial port buffer was full and to only take the data at 9600
> >baud.
>
> This is what Twisted's producers and consumers let you do. There's a
> document covering these features:
>
> http://twistedmatrix.com/projects/core/documentation/howto/producers.html
>
> In the case of a TCP to serial forwarder, you don't actually have to
> implement either a producer or a consumer, since both the TCP connection
> and the serial connection are already both producers and consumers. All
> you need to do is hook them up to each other so that when the send buffer
> of one fills up, the other one gets paused, and when the buffer is empty
> again, it gets resumed.
I eventually came up with this which seems to work, but I'm not sure
it is the best way of doing it as it had to mess about with the
twisted internals to get the number of bytes in the serial port
output buffer.
class SerialPort(protocol.Protocol):
"""Create a serial port connection and pass data from it to a known list of
TCP ports."""
def __init__(self, port, reactor, baudrate, log_name=None):
self.tcp_ports = []
self.serial = serialport.SerialPort(self, reactor, port, baudrate,
rtscts=0)
self.log = None
if log_name is not None:
self.log = file('%s-0' % log_name, 'w')
def write(self, data):
"""Write data to the serial port."""
self.serial.write(data)
if self.log:
self.log.write(data)
self.throttle()
def throttle(self):
"""
Pause the inputs if there is too much data in the output buffer
"""
bytes_in_buffer = len(self.serial.dataBuffer) + self.serial._tempDataLen
too_full = bytes_in_buffer > 1024
for tcp_port in self.tcp_ports:
if too_full:
tcp_port.transport.pauseProducing()
else:
tcp_port.transport.resumeProducing()
if too_full:
reactor.callLater(0.1, self.throttle)
--
Nick Craig-Wood -- http://www.craig-wood.com/nick
--
http://mail.python.org/mailman/listinfo/python-list
Re: How can I do bit operation on python float value
On Mar 23, 7:44 pm, valpa wrote: > On 3月23日, 上午3时18分, John Machin wrote: > > > > > On Mar 23, 5:44 pm, valpa wrote: > > > > I have a python float 1.2345678. I know that it is stored as a double > > > in C type. And I know it actually is 1010101010101 -like format. Then > > > I want to do some bit operation on it. How? > > > > Sure, I want a float output when I finish the operation. > > > import struct > > pack it into a str [Python 2.X] using d format > > unpack it using Q format, gets you a 64-bit unsigned int > > do "some bit operation" using & | ^ ~ << >> operators > > pack it into a str using Q format > > unpack it into a float using d format > > > What is "some bit operation"? Perhaps there is already a high-level > > way of doing some of what you want e.g. in the math module: frexp, > > ldexp, isnan, isinf, ... > > > HTH > > John > > Thanks John! > > Yes, I want to do a & operation. > The pack way is to convert float to/from string, pack converts various types to a str object, unpack converts from a str object to various types; please read the struct manual instead of guessing. > Is it efficient? It's relatively efficient, given that what I described takes 4 Python function calls. In absolute terms, it's a very inefficient way of doing one & operation on a float. You can try it and see if it is fast enough for your purpose. If you tell us what you are trying to do, we might be able to suggest a better way. Cheers, John -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I do bit operation on python float value
On Mon, Mar 23, 2009 at 2:44 AM, valpa wrote: > I have a python float 1.2345678. I know that it is stored as a double > in C type. And I know it actually is 1010101010101 -like format. Then > I want to do some bit operation on it. How? > > Sure, I want a float output when I finish the operation. Just for fun: >>> import ctypes >>> def cast(value, to_type): ... return ctypes.cast(ctypes.pointer(value), ctypes.POINTER(to_type))[0] ... >>> def float_to_int(x): ... return cast(ctypes.c_double(x), ctypes.c_uint64) ... >>> def int_to_float(x): ... return cast(ctypes.c_uint64(x), ctypes.c_double) ... >>> float_to_int(5) 4617315517961601024 >>> bin(float_to_int(5)) '0b1010100' >>> int_to_float(float_to_int(5) & float_to_int(10)) 2.5 -Miles -- http://mail.python.org/mailman/listinfo/python-list
Re: how to convert from Decimal('1.23456789') to Decimal('1.234')
On Mon, 23 Mar 2009 01:45:53 -0700, Mensanator wrote:
>> but you can create a helper
>> function very easily:
>>
>> def round(dec, places, rounding=decimal.ROUND_HALF_UP): � � return
>> dec.quantize(decimal.Decimal(str(10**-places)), rounding)
>
> Still ugly. I would do this:
>
a = Decimal('1.23456789')
>
for i in xrange(1,6):
> print Context.create_decimal(Context(i,ROUND_DOWN),a)
Well, that's hardly any less ugly.
And it also gives different results to my function: my function rounds to
decimal places, yours to digits. Very different things.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: Unicode problem in ucs4
On Mar 23, 3:04 pm, "M.-A. Lemburg" wrote:
> On 2009-03-23 08:18, abhi wrote:
>
>
>
> > On Mar 20, 5:47 pm, "M.-A. Lemburg" wrote:
> >>> unicodeTest.c
> >>> #include
> >>> static PyObject *unicode_helper(PyObject *self,PyObject *args){
> >>> PyObject *sampleObj = NULL;
> >>> Py_UNICODE *sample = NULL;
> >>> if (!PyArg_ParseTuple(args, "O", &sampleObj)){
> >>> return NULL;
> >>> }
> >>> // Explicitly convert it to unicode and get Py_UNICODE value
> >>> sampleObj = PyUnicode_FromObject(sampleObj);
> >>> sample = PyUnicode_AS_UNICODE(sampleObj);
> >>> wprintf(L"database value after unicode conversion is : %s\n",
> >>> sample);
> >> You have to use PyUnicode_AsWideChar() to convert a Python
> >> Unicode object to a wchar_t representation.
>
> >> Please don't make any assumptions on what Py_UNICODE maps
> >> to and always use the the Unicode API for this. It is designed
> >> to provide a portable interface and will not do more conversion
> >> work than necessary.
>
> > Hi Mark,
> > Thanks for the help. I tried PyUnicode_AsWideChar() but I am
> > getting the same result i.e. only the first letter.
>
> > sample code:
>
> > #include
>
> > static PyObject *unicode_helper(PyObject *self,PyObject *args){
> > PyObject *sampleObj = NULL;
> > wchar_t *sample = NULL;
> > int size = 0;
>
> > if (!PyArg_ParseTuple(args, "O", &sampleObj)){
> > return NULL;
> > }
>
> > // use wide char function
> > size = PyUnicode_AsWideChar(databaseObj, sample,
> > PyUnicode_GetSize(databaseObj));
>
> The 3. argument is the buffer size in bytes, not code points.
> The result will require sizeof(wchar_t) * PyUnicode_GetSize(databaseObj)
> bytes without a trailing NUL, otherwise sizeof(wchar_t) *
> (PyUnicode_GetSize(databaseObj) + 1).
>
> You also have to allocate the buffer to store the wchar_t data in.
> Passing in a NULL pointer will result in a seg fault. The function
> does not allocate a buffer for you:
>
> /* Copies the Unicode Object contents into the wchar_t buffer w. At
> most size wchar_t characters are copied.
>
> Note that the resulting wchar_t string may or may not be
> 0-terminated. It is the responsibility of the caller to make sure
> that the wchar_t string is 0-terminated in case this is required by
> the application.
>
> Returns the number of wchar_t characters copied (excluding a
> possibly trailing 0-termination character) or -1 in case of an
> error. */
>
> PyAPI_FUNC(Py_ssize_t) PyUnicode_AsWideChar(
> PyUnicodeObject *unicode, /* Unicode object */
> register wchar_t *w, /* wchar_t buffer */
> Py_ssize_t size /* size of buffer */
> );
>
>
>
> > printf("%d chars are copied to sample\n", size);
> > wprintf(L"database value after unicode conversion is : %s\n",
> > sample);
> > return Py_BuildValue("");
>
> > }
>
> > static PyMethodDef funcs[]={{"unicodeTest",(PyCFunction)
> > unicode_helper,METH_VARARGS,"test ucs2, ucs4"},{NULL}};
>
> > void initunicodeTest(void){
> > Py_InitModule3("unicodeTest",funcs,"");
>
> > }
>
> > This prints the following when input value is given as "test":
> > 4 chars are copied to sample
> > database value after unicode conversion is : t
>
> > Any ideas?
>
> > -
> > Abhigyan
> > --
> >http://mail.python.org/mailman/listinfo/python-list
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source (#1, Mar 23 2009)>>>
> Python/Zope Consulting and Support ... http://www.egenix.com/
> >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
>
>
> 2009-03-19: Released mxODBC.Connect 1.0.1 http://python.egenix.com/
>
> ::: Try our new mxODBC.Connect Python Database Interface for free !
>
> eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
> Registered at Amtsgericht Duesseldorf: HRB 46611
> http://www.egenix.com/company/contact/
Thanks Marc, John,
With your help, I am at least somewhere. I re-wrote the code
to compare Py_Unicode and wchar_t outputs and they both look exactly
the same.
#include
static PyObject *unicode_helper(PyObject *self,PyObject *args){
const char *name;
PyObject *sampleObj = NULL;
Py_UNICODE *sample = NULL;
wchar_t * w=NULL;
int size = 0;
int i;
if (!PyArg_ParseTuple(args, "O", &sampleObj)){
return NULL;
}
// Explicitly convert it to unicode and get Py_UNICODE value
sampleObj = PyUnicode_FromObject(sampleObj);
sample = PyUnicode_AS_UNICODE(sampleObj);
printf("size of sampleObj is : %d\n",PyUnicode_GET_SIZE
Re: Safe to call Py_Initialize() frequently?
On 23/03/2009 12:14 PM, Graham Dumpleton wrote: On Mar 21, 10:27 am, Mark Hammond wrote: Calling Py_Initialize and Py_Finalize multiple times does leak (Python 3 has mechanisms so this need to always be true in the future, but it is true now for non-trivial apps. Mark, can you please clarify this statement you are making. The grammar used makes it a bit unclear. Yes, sorry - s/this need to/this need not/ Are you saying, that effectively by design, Python 3.0 will always leak memory upon Py_Finalize() being called, or that it shouldn't leak memory and that problems with older versions of Python have been fixed up? The latter - kindof - py3k provides an enhanced API that *allows* extensions to be 'safe' in this regard, but it doesn't enforce it. Modules 'trivially' ported from py2k will not magically get this ability - they must explicitly take advantage of it. pywin32 is yet to do so (ie, it is a 'trivial' port...) I hope this clarifies... Mark -- http://mail.python.org/mailman/listinfo/python-list
Re: splitting a large dictionary into smaller ones
As others have said, a database is probably the right answer. There, the data is kept on disk, and only a few records at a time are read for each access, with modification transactions usually being synchronous. However, there are use cases where your approach makes more sense. And it shouldn't be too hard to do what you describe. So it's probably worth implementing. Use case: suppose most accesses are to a 'local' set of keys, in some sense. For example, if the keys were serial numbers, with the most recently created serial numbers likely to be accessed much more often than those a few years old. Then if you split the serial numbers up in the most obvious way (using the high digits of the number, for example to index which external file the data is in), you'd find nearly all accesses would be to the file with the highest index. So you create a wrapper class with an init and two member methods, read and write, plus a flush() method. read takes a key, and returns a value, while write takes both key and value, and saves it away. The init method creates a list of dictionaries (initially of length zero) and saves some information about the location and naming of the actual pickle files (or whatever you're using). When a read or write method is called, you do the divide (or whatever filter you use) to find an index into the list of dictionaries. If this index is beyond the end of the list, extend the list with None till it's big enough. Now, if the particular entry in the list is None, you need to read in the corresponding pickle file, and add the created dictionary object into this index of the list. At this point, you read or write your dictionary as usual. The only other tricky thing is to save a 'dirty bit' for each loaded dictionary. When your wrapper's flush() is called, you pickle each of the dirty dictionaries back to the file system. Lots of possible variants. For example, you could use a dictionary of dictionary objects instead of a list of dictionaries. That would be useful if the list will be large, and sparse. The main thing I don't know here is how to overload the [] operator, if you need that particular syntax for the read and write methods. per wrote: hi all, i have a very large dictionary object that is built from a text file that is about 800 MB -- it contains several million keys. ideally i would like to pickle this object so that i wouldnt have to parse this large file to compute the dictionary every time i run my program. however currently the pickled file is over 300 MB and takes a very long time to write to disk - even longer than recomputing the dictionary from scratch. i would like to split the dictionary into smaller ones, containing only hundreds of thousands of keys, and then try to pickle them. is there a way to easily do this? i.e. is there an easy way to make a wrapper for this such that i can access this dictionary as just one object, but underneath it's split into several? so that i can write my_dict[k] and get a value, or set my_dict[m] to some value without knowing which sub dictionary it's in. if there aren't known ways to do this, i would greatly apprciate any advice/examples on how to write this data structure from scratch, reusing as much of the dict() class as possible. thanks. large_dict[a] -- http://mail.python.org/mailman/listinfo/python-list
Re: Safe to call Py_Initialize() frequently?
On Mar 23, 10:00 pm, Mark Hammond wrote: > On 23/03/2009 12:14 PM, Graham Dumpleton wrote: > > > On Mar 21, 10:27 am, Mark Hammond wrote: > >> Calling > >> Py_Initialize and Py_Finalize multiple times does leak (Python 3 has > >> mechanisms so this need to always be true in the future, but it is true > >> now for non-trivial apps. > > > Mark, can you please clarify this statement you are making. The > > grammar used makes it a bit unclear. > > Yes, sorry - s/this need to/this need not/ > > > Are you saying, that effectively by design, Python 3.0 will always > > leak memory upon Py_Finalize() being called, or that it shouldn't leak > > memory and that problems with older versions of Python have been fixed > > up? > > The latter - kindof - py3k provides an enhanced API that *allows* > extensions to be 'safe' in this regard, but it doesn't enforce it. > Modules 'trivially' ported from py2k will not magically get this ability > - they must explicitly take advantage of it. pywin32 is yet to do so > (ie, it is a 'trivial' port...) > > I hope this clarifies... Yes, but ... There still may be problems. The issues is old, but suspect that comments in the issue: http://bugs.python.org/issue1856 maybe still hold true. That is, that there are some things that Python doesn't free up which are related to Python simplified GIL state API. Normally this wouldn't matter as another call to Py_Initialize() would see existing data and reuse it. So, doesn't strictly leak memory in that sense. In mod_wsgi however, Apache will completely unload the mod_wsgi module on a restart. This would also mean that the Python library is also unloaded from memory. When it reloads both, the global static variables where information was left behind have been lost and nulled out. Thus Python when initialised again, will recreate the data it needs. So, for case where Python library unloaded, looks like may well suffer a memory leak regardless. As to third party C extension modules, they aren't really an issue, because all that is done in Apache parent process is Py_Initialize() and Py_Finalize() and nothing else really. Just done to get interpreter setup before forking child processes. There is more detail on this analysis in that thread on mod_wsgi list at: Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: splitting a large dictionary into smaller ones
As others have said, a database is probably the right answer. There, the data is kept on disk, and only a few records at a time are read for each access, with modification transactions usually being synchronous. However, there are use cases where your approach makes more sense. And it shouldn't be too hard to do what you describe. So it's probably worth implementing. Use case: suppose most accesses are to a 'local' set of keys, in some sense. For example, if the keys were serial numbers, with the most recently created serial numbers likely to be accessed much more often than those a few years old. Then if you split the serial numbers up in the most obvious way (using the high digits of the number, for example to index which external file the data is in), you'd find nearly all accesses would be to the file with the highest index. So you create a wrapper class with an init and two member methods, read and write, plus a flush() method. read takes a key, and returns a value, while write takes both key and value, and saves it away. The init method creates a list of dictionaries (initially of length zero) and saves some information about the location and naming of the actual pickle files (or whatever you're using). When a read or write method is called, you do the divide (or whatever filter you use) to find an index into the list of dictionaries. If this index is beyond the end of the list, extend the list with None till it's big enough. Now, if the particular entry in the list is None, you need to read in the corresponding pickle file, and add the created dictionary object into this index of the list. At this point, you read or write your dictionary as usual. The only other tricky thing is to save a 'dirty bit' for each loaded dictionary. When your wrapper's flush() is called, you pickle each of the dirty dictionaries back to the file system. Lots of possible variants. For example, you could use a dictionary of dictionary objects instead of a list of dictionaries. That would be useful if the list will be large, and sparse. The main thing I don't know here is how to overload the [] operator, if you need that particular syntax for the read and write methods. per wrote: hi all, i have a very large dictionary object that is built from a text file that is about 800 MB -- it contains several million keys. ideally i would like to pickle this object so that i wouldnt have to parse this large file to compute the dictionary every time i run my program. however currently the pickled file is over 300 MB and takes a very long time to write to disk - even longer than recomputing the dictionary from scratch. i would like to split the dictionary into smaller ones, containing only hundreds of thousands of keys, and then try to pickle them. is there a way to easily do this? i.e. is there an easy way to make a wrapper for this such that i can access this dictionary as just one object, but underneath it's split into several? so that i can write my_dict[k] and get a value, or set my_dict[m] to some value without knowing which sub dictionary it's in. if there aren't known ways to do this, i would greatly apprciate any advice/examples on how to write this data structure from scratch, reusing as much of the dict() class as possible. thanks. large_dict[a] -- http://mail.python.org/mailman/listinfo/python-list
Re: Using python 3 for scripting?
Not too keen on working with Solaris either. Did some small configuration last time I worked there and it was all a mess. I'm trying to convince them to switch to OpenBSD :) Timo -- http://mail.python.org/mailman/listinfo/python-list
Re: Unicode problem in ucs4
On 2009-03-23 11:50, abhi wrote:
> On Mar 23, 3:04 pm, "M.-A. Lemburg" wrote:
> Thanks Marc, John,
> With your help, I am at least somewhere. I re-wrote the code
> to compare Py_Unicode and wchar_t outputs and they both look exactly
> the same.
>
> #include
>
> static PyObject *unicode_helper(PyObject *self,PyObject *args){
> const char *name;
> PyObject *sampleObj = NULL;
> Py_UNICODE *sample = NULL;
> wchar_t * w=NULL;
> int size = 0;
> int i;
>
> if (!PyArg_ParseTuple(args, "O", &sampleObj)){
> return NULL;
> }
>
>
> // Explicitly convert it to unicode and get Py_UNICODE value
> sampleObj = PyUnicode_FromObject(sampleObj);
> sample = PyUnicode_AS_UNICODE(sampleObj);
> printf("size of sampleObj is : %d\n",PyUnicode_GET_SIZE
> (sampleObj));
> w = (wchar_t *) malloc((PyUnicode_GET_SIZE(sampleObj)+1)*sizeof
> (wchar_t));
> size = PyUnicode_AsWideChar(sampleObj,w,(PyUnicode_GET_SIZE(sampleObj)
> +1)*sizeof(wchar_t));
> printf("%d chars are copied to w\n",size);
> printf("size of wchar_t is : %d\n", sizeof(wchar_t));
> printf("size of Py_UNICODE is: %d\n",sizeof(Py_UNICODE));
> for(i=0;i printf("sample is : %c\n",sample[i]);
> printf("w is : %c\n",w[i]);
> }
> return sampleObj;
> }
>
> static PyMethodDef funcs[]={{"unicodeTest",(PyCFunction)
> unicode_helper,METH_VARARGS,"test ucs2, ucs4"},{NULL}};
>
> void initunicodeTest(void){
> Py_InitModule3("unicodeTest",funcs,"");
> }
>
> This gives the following output when I pass "abc" as input:
>
> size of sampleObj is : 3
> 3 chars are copied to w
> size of wchar_t is : 4
> size of Py_UNICODE is: 4
> sample is : a
> w is : a
> sample is : b
> w is : b
> sample is : c
> w is : c
>
> So, both Py_UNICODE and wchar_t are 4 bytes and since it contains 3
> \0s after a char, printf or wprintf is only printing one letter.
> I need to further process the data and those libraries will need the
> data in UCS2 format (2 bytes), otherwise they fail. Is there any way
> by which I can force wchar_t to be 2 bytes, or can I convert this UCS4
> data to UCS2 explicitly?
Sure: just use the appropriate UTF-16 codec for this.
/* Generic codec based encoding API.
object is passed through the encoder function found for the given
encoding using the error handling method defined by errors. errors
may be NULL to use the default method defined for the codec.
Raises a LookupError in case no encoder can be found.
*/
PyAPI_FUNC(PyObject *) PyCodec_Encode(
PyObject *object,
const char *encoding,
const char *errors
);
encoding needs to be set to 'utf-16-le' for little endian, 'utf-16-be'
for big endian.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Mar 23 2009)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
2009-03-19: Released mxODBC.Connect 1.0.1 http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free !
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
--
http://mail.python.org/mailman/listinfo/python-list
Re: Style formating of multiline query, advise
On Mar 19, 10:26 am, Marco Mariani wrote: > someone wrote: > >> Also, for SQL, (A) why are you using nested joins?, and > > > inner select produce smaller set which is then joined with other > > table, kind a optimization > > Did you time it? I've did explain on both kinds of query (with nested select and without) and query plan is identical to both. So, there is no difference, well in PostGres > I've done some "kind of a optimization" that slowed queries by tenfold, > because postgres didn't need my advice, and knew better. RDBMS > performance is non-intuitive, really. If you're using mysql, YMMV, > because its optimizer is not as good. > > > Yes, my original question was about formatting. It's not original > > query (only a part). > > Try this: > > http://www.dpriver.com/pp/sqlformat.htm > > My 2c: I use textwrap.dedent to strip leading spaces from every line. -- http://mail.python.org/mailman/listinfo/python-list
Re: Unicode problem in ucs4
On Mar 23, 4:37 pm, "M.-A. Lemburg" wrote:
> On 2009-03-23 11:50, abhi wrote:
>
>
>
> > On Mar 23, 3:04 pm, "M.-A. Lemburg" wrote:
> > Thanks Marc, John,
> > With your help, I am at least somewhere. I re-wrote the code
> > to compare Py_Unicode and wchar_t outputs and they both look exactly
> > the same.
>
> > #include
>
> > static PyObject *unicode_helper(PyObject *self,PyObject *args){
> > const char *name;
> > PyObject *sampleObj = NULL;
> > Py_UNICODE *sample = NULL;
> > wchar_t * w=NULL;
> > int size = 0;
> > int i;
>
> > if (!PyArg_ParseTuple(args, "O", &sampleObj)){
> > return NULL;
> > }
>
> > // Explicitly convert it to unicode and get Py_UNICODE value
> > sampleObj = PyUnicode_FromObject(sampleObj);
> > sample = PyUnicode_AS_UNICODE(sampleObj);
> > printf("size of sampleObj is : %d\n",PyUnicode_GET_SIZE
> > (sampleObj));
> > w = (wchar_t *) malloc((PyUnicode_GET_SIZE(sampleObj)+1)*sizeof
> > (wchar_t));
> > size = PyUnicode_AsWideChar(sampleObj,w,(PyUnicode_GET_SIZE(sampleObj)
> > +1)*sizeof(wchar_t));
> > printf("%d chars are copied to w\n",size);
> > printf("size of wchar_t is : %d\n", sizeof(wchar_t));
> > printf("size of Py_UNICODE is: %d\n",sizeof(Py_UNICODE));
> > for(i=0;i > printf("sample is : %c\n",sample[i]);
> > printf("w is : %c\n",w[i]);
> > }
> > return sampleObj;
> > }
>
> > static PyMethodDef funcs[]={{"unicodeTest",(PyCFunction)
> > unicode_helper,METH_VARARGS,"test ucs2, ucs4"},{NULL}};
>
> > void initunicodeTest(void){
> > Py_InitModule3("unicodeTest",funcs,"");
> > }
>
> > This gives the following output when I pass "abc" as input:
>
> > size of sampleObj is : 3
> > 3 chars are copied to w
> > size of wchar_t is : 4
> > size of Py_UNICODE is: 4
> > sample is : a
> > w is : a
> > sample is : b
> > w is : b
> > sample is : c
> > w is : c
>
> > So, both Py_UNICODE and wchar_t are 4 bytes and since it contains 3
> > \0s after a char, printf or wprintf is only printing one letter.
> > I need to further process the data and those libraries will need the
> > data in UCS2 format (2 bytes), otherwise they fail. Is there any way
> > by which I can force wchar_t to be 2 bytes, or can I convert this UCS4
> > data to UCS2 explicitly?
>
> Sure: just use the appropriate UTF-16 codec for this.
>
> /* Generic codec based encoding API.
>
> object is passed through the encoder function found for the given
> encoding using the error handling method defined by errors. errors
> may be NULL to use the default method defined for the codec.
>
> Raises a LookupError in case no encoder can be found.
>
> */
>
> PyAPI_FUNC(PyObject *) PyCodec_Encode(
> PyObject *object,
> const char *encoding,
> const char *errors
> );
>
> encoding needs to be set to 'utf-16-le' for little endian, 'utf-16-be'
> for big endian.
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source (#1, Mar 23 2009)>>>
> Python/Zope Consulting and Support ... http://www.egenix.com/
> >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
>
>
> 2009-03-19: Released mxODBC.Connect 1.0.1 http://python.egenix.com/
>
> ::: Try our new mxODBC.Connect Python Database Interface for free !
>
> eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
> Registered at Amtsgericht Duesseldorf: HRB 46611
> http://www.egenix.com/company/contact/
Thanks, but this is returning PyObject *, whereas I need value in some
variable which can be printed using wprintf() like wchar_t (having a
size of 2 bytes). If I again convert this PyObject to wchar_t or
PyUnicode, I go back to where I started. :)
-
Abhigyan
--
http://mail.python.org/mailman/listinfo/python-list
Escaping optional parameter in WHERE clause
Hi,
as you can see below I have some optional parameter for my query (mf,
age). They are in WHERE clause only if not empty.
In this function they are not escaped as, for example, 'search'
parameter, cause I can't pass them to execute function, which does
escaping automatically.
I could write another if's block like that
if mf and not age:
db.execute(query, search, mf, limit)
if age and not mf:
db.execute(query, search, age, limit)
if age and mf:
db.execute(query, search, mf, age, limit)
Is there a better way to deal with optional WHERE clause?
Pet
def getData(self, db, params):
search = params.get('search','')
age = params.get('age','')
mf = params.get('mf','')
limit = params.get('limit',1)
if mf:
mf = " AND mf = %s " % mf
if age:
age = " AND age = %s " % age
query = """
SELECT * FROM mytable
WHERE class = 'P'
AND name = %s
""" + mf + """
""" + age + """
ORDER BY id DESC
LIMIT %s;
"""
db.execute(query, search, limit)
result = db.fetchall()
return result
--
http://mail.python.org/mailman/listinfo/python-list
Re: Async serial communication/threads sharing data
On Mon, 23 Mar 2009 05:30:04 -0500, Nick Craig-Wood wrote:
Jean-Paul Calderone wrote:
[snip]
In the case of a TCP to serial forwarder, you don't actually have to
implement either a producer or a consumer, since both the TCP connection
and the serial connection are already both producers and consumers. All
you need to do is hook them up to each other so that when the send buffer
of one fills up, the other one gets paused, and when the buffer is empty
again, it gets resumed.
I eventually came up with this which seems to work, but I'm not sure
it is the best way of doing it as it had to mess about with the
twisted internals to get the number of bytes in the serial port
output buffer.
This is sort of on the right track. Here's how to do it without
touching implementation details:
class SerialPort(protocol.Protocol):
"""Create a serial port connection and pass data from it to a known list of TCP
ports."""
def __init__(self, port, reactor, baudrate, log_name=None):
self.tcp_ports = []
self.serial = serialport.SerialPort(self, reactor, port, baudrate,
rtscts=0)
Here, register this object to receive buffer full and not-full events from
the serial port:
self.serial.registerProducer(self, True)
And an attribute to keep track of our state:
self.paused = False
self.log = None
if log_name is not None:
self.log = file('%s-0' % log_name, 'w')
I'm not exactly sure where `self.tcp_ports´ gets populated, so I'll make a
method to do it:
def addPort(self, port):
if self.paused:
port.transport.pauseProducing()
self.tcp_ports.append(port)
Now a method to handle serial port buffer full events:
def pauseProducing(self):
self.paused = True
for port in self.tcp_ports:
port.transport.pauseProducing()
And a method to handle buffer no longer full events:
def resumeProducing(self):
self.paused = False
for port in self.tcp_ports:
port.transport.resumeProducing()
With these, you can get rid of `throttle´ entirely.
def write(self, data):
"""Write data to the serial port."""
self.serial.write(data)
if self.log:
self.log.write(data)
self.throttle()
def throttle(self):
"""
Pause the inputs if there is too much data in the output buffer
"""
bytes_in_buffer = len(self.serial.dataBuffer) + self.serial._tempDataLen
too_full = bytes_in_buffer > 1024
for tcp_port in self.tcp_ports:
if too_full:
tcp_port.transport.pauseProducing()
else:
tcp_port.transport.resumeProducing()
if too_full:
reactor.callLater(0.1, self.throttle)
Jean-Paul
--
http://mail.python.org/mailman/listinfo/python-list
Looking for a Python Django Freelancer
For a Python Google Apps Project with Django we are looking for a technical developer offsite. IF you are interesting please send your CV and rate to daniel.jor...@fellow-consulting.de -- http://mail.python.org/mailman/listinfo/python-list
Re: Escaping optional parameter in WHERE clause
someone wrote:
Hi,
as you can see below I have some optional parameter for my query (mf,
age). They are in WHERE clause only if not empty.
In this function they are not escaped as, for example, 'search'
parameter, cause I can't pass them to execute function, which does
escaping automatically.
I could write another if's block like that
if mf and not age:
db.execute(query, search, mf, limit)
if age and not mf:
db.execute(query, search, age, limit)
if age and mf:
db.execute(query, search, mf, age, limit)
Is there a better way to deal with optional WHERE clause?
Pet
def getData(self, db, params):
search = params.get('search','')
age = params.get('age','')
mf = params.get('mf','')
limit = params.get('limit',1)
if mf:
mf = " AND mf = %s " % mf
if age:
age = " AND age = %s " % age
query = """
SELECT * FROM mytable
WHERE class = 'P'
AND name = %s
""" + mf + """
""" + age + """
ORDER BY id DESC
LIMIT %s;
"""
db.execute(query, search, limit)
result = db.fetchall()
return result
How about:
def getData(self, db, params):
search = params.get('search', '')
age = params.get('age', '')
mf = params.get('mf', '')
limit = params.get('limit', 1)
query = """
SELECT * FROM mytable
WHERE class = 'P'
AND name = %s
"""
values = [search]
if mf:
query += " AND mf = %s"
values.append(mf)
if age:
query += " AND age = %s"
values.append(age)
query += """
ORDER BY id DESC
LIMIT %s;
"""
values.append(limit)
db.execute(query, *values)
result = db.fetchall()
return result
--
http://mail.python.org/mailman/listinfo/python-list
Re: Escaping optional parameter in WHERE clause
note that your version is open to sql injection attacks, while mrab's reply isn't. andrew someone wrote: > if mf: > mf = " AND mf = %s " % mf > if age: > age = " AND age = %s " % age -- http://mail.python.org/mailman/listinfo/python-list
Re: splitting a large dictionary into smaller ones
i have a very large dictionary object that is built from a text file
that is about 800 MB -- it contains several million keys. ideally i
would like to pickle this object so that i wouldnt have to parse this
large file to compute the dictionary every time i run my program.
however currently the pickled file is over 300 MB and takes a very
long time to write to disk - even longer than recomputing the
dictionary from scratch.
i would like to split the dictionary into smaller ones, containing
only hundreds of thousands of keys, and then try to pickle them. is
there a way to easily do this?
While others have suggested databases, they may be a bit
overkill, depending on your needs. Python2.5+ supplies not only
the sqlite3 module, but older versions (at least back to 2.0)
offer the anydbm module (changed to "dbm" in 3.0), allowing you
to create an on-disk string-to-string dictionary:
import anydbm
db = anydbm.open("data.db", "c")
# populate some data
# using "db" as your dictionary
import csv
f = file("800megs.txt")
data = csv.reader(f, delimiter='\t')
data.next() # discard a header row
for key, value in data:
db[key] = value
f.close()
print db["some key"]
db.close()
The resulting DB object is a little sparsely documented, but for
the most part it can be treated like a dictionary. The advantage
is that, if the source data doesn't change, you can parse once
and then just use your "data.db" file from there out.
-tkc
--
http://mail.python.org/mailman/listinfo/python-list
Re: Escaping optional parameter in WHERE clause
ah, sorry, from title i guess you were aware of this. andrew andrew cooke wrote: > note that your version is open to sql injection attacks, while mrab's > reply isn't. andrew > > someone wrote: >> if mf: >> mf = " AND mf = %s " % mf >> if age: >> age = " AND age = %s " % age -- http://mail.python.org/mailman/listinfo/python-list
Re: Escaping optional parameter in WHERE clause
On Mar 23, 1:48 pm, MRAB wrote:
> someone wrote:
> > Hi,
>
> > as you can see below I have some optional parameter for my query (mf,
> > age). They are in WHERE clause only if not empty.
> > In this function they are not escaped as, for example, 'search'
> > parameter, cause I can't pass them to execute function, which does
> > escaping automatically.
>
> > I could write another if's block like that
>
> > if mf and not age:
> > db.execute(query, search, mf, limit)
> > if age and not mf:
> > db.execute(query, search, age, limit)
> > if age and mf:
> > db.execute(query, search, mf, age, limit)
>
> > Is there a better way to deal with optional WHERE clause?
>
> > Pet
>
> > def getData(self, db, params):
> > search = params.get('search','')
> > age = params.get('age','')
> > mf = params.get('mf','')
> > limit = params.get('limit',1)
>
> > if mf:
> > mf = " AND mf = %s " % mf
> > if age:
> > age = " AND age = %s " % age
>
> > query = """
> > SELECT * FROM mytable
> > WHERE class = 'P'
> > AND name = %s
> > """ + mf + """
> > """ + age + """
> > ORDER BY id DESC
> > LIMIT %s;
> > """
>
> > db.execute(query, search, limit)
> > result = db.fetchall()
> > return result
>
> How about:
>
> def getData(self, db, params):
> search = params.get('search', '')
> age = params.get('age', '')
> mf = params.get('mf', '')
> limit = params.get('limit', 1)
>
> query = """
> SELECT * FROM mytable
> WHERE class = 'P'
> AND name = %s
> """
> values = [search]
>
> if mf:
> query += " AND mf = %s"
> values.append(mf)
>
> if age:
> query += " AND age = %s"
> values.append(age)
>
> query += """
> ORDER BY id DESC
> LIMIT %s;
> """
> values.append(limit)
>
> db.execute(query, *values)
> result = db.fetchall()
> return result
Like it. Thanks, man!
--
http://mail.python.org/mailman/listinfo/python-list
Portable Python 1.1 released
Dear people, Portable Python 1.1 is released in three flavors: Python 2.5.4, 2.6.1 and 3.0.1. More information: Included in this release: - This release contains three different packages for three different Python versions – Python 2.5.4, Python 2.6.1 and Python 3.0.1. Packages are totally independent and can run side-by-side each other or any other Python installation. Software included (in alphabetical order per package): Portable Python 1.1 based on Python 2.5.4 - Django-1.0.2-final - IPython-0.9.1 - Matplotlib-0.98.5.2 - Numpy-1.2.1 - PIL-1.1.6 - Py2exe-0.6.9 - PyGame-1.8.1 - PyReadline-1.5 - PyScripter v1.9.9.6 - PyWin32-212 - Rpyc-2.60 - Scipy-0.7.0b1 - SPE-0.8.4.c - VPython-3.2.11 - wxPython-unicode-2.8.9.1 Portable Python 1.1 based on Python 2.6.1 - Django-1.0.2-final - IPython-0.9.1 - PIL-1.1.6 - Py2exe-0.6.9 - PyGame-1.8.1 - PyReadline-1.5 - PyScripter v1.9.9.6 - PyWin32-212 - Rpyc-2.60 - SPE-0.8.4.c - wxPython-unicode-2.8.9.1 Portable Python 1.1 based on Python 3.0.1 - PyScripter v1.9.9.6 - Rpyc-2.60 Installation and use: - After downloading entire distribution or specific Python version package, run the installer, select the target folder and you are done! In the main folder you will find shortcuts for selected applications in that package. Some of the most popular free Python IDE’s come preinstalled and preconfigured with Portable Python. How to use and configure them further please consult their documentation or project sites. All issues, suggestions or success stories you can post on the Portable Python Google group: http://groups.google.com/group/portablepython kind regards, Perica Zivkovic http://www.PortablePython.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Unicode problem in ucs4
On Mar 23, 4:57 pm, abhi wrote:
> On Mar 23, 4:37 pm, "M.-A. Lemburg" wrote:
>
>
>
> > On 2009-03-23 11:50, abhi wrote:
>
> > > On Mar 23, 3:04 pm, "M.-A. Lemburg" wrote:
> > > Thanks Marc, John,
> > > With your help, I am at least somewhere. I re-wrote the code
> > > to compare Py_Unicode and wchar_t outputs and they both look exactly
> > > the same.
>
> > > #include
>
> > > static PyObject *unicode_helper(PyObject *self,PyObject *args){
> > > const char *name;
> > > PyObject *sampleObj = NULL;
> > > Py_UNICODE *sample = NULL;
> > > wchar_t * w=NULL;
> > > int size = 0;
> > > int i;
>
> > > if (!PyArg_ParseTuple(args, "O", &sampleObj)){
> > > return NULL;
> > > }
>
> > > // Explicitly convert it to unicode and get Py_UNICODE value
> > > sampleObj = PyUnicode_FromObject(sampleObj);
> > > sample = PyUnicode_AS_UNICODE(sampleObj);
> > > printf("size of sampleObj is : %d\n",PyUnicode_GET_SIZE
> > > (sampleObj));
> > > w = (wchar_t *) malloc((PyUnicode_GET_SIZE(sampleObj)+1)*sizeof
> > > (wchar_t));
> > > size = PyUnicode_AsWideChar(sampleObj,w,(PyUnicode_GET_SIZE(sampleObj)
> > > +1)*sizeof(wchar_t));
> > > printf("%d chars are copied to w\n",size);
> > > printf("size of wchar_t is : %d\n", sizeof(wchar_t));
> > > printf("size of Py_UNICODE is: %d\n",sizeof(Py_UNICODE));
> > > for(i=0;i > > printf("sample is : %c\n",sample[i]);
> > > printf("w is : %c\n",w[i]);
> > > }
> > > return sampleObj;
> > > }
>
> > > static PyMethodDef funcs[]={{"unicodeTest",(PyCFunction)
> > > unicode_helper,METH_VARARGS,"test ucs2, ucs4"},{NULL}};
>
> > > void initunicodeTest(void){
> > > Py_InitModule3("unicodeTest",funcs,"");
> > > }
>
> > > This gives the following output when I pass "abc" as input:
>
> > > size of sampleObj is : 3
> > > 3 chars are copied to w
> > > size of wchar_t is : 4
> > > size of Py_UNICODE is: 4
> > > sample is : a
> > > w is : a
> > > sample is : b
> > > w is : b
> > > sample is : c
> > > w is : c
>
> > > So, both Py_UNICODE and wchar_t are 4 bytes and since it contains 3
> > > \0s after a char, printf or wprintf is only printing one letter.
> > > I need to further process the data and those libraries will need the
> > > data in UCS2 format (2 bytes), otherwise they fail. Is there any way
> > > by which I can force wchar_t to be 2 bytes, or can I convert this UCS4
> > > data to UCS2 explicitly?
>
> > Sure: just use the appropriate UTF-16 codec for this.
>
> > /* Generic codec based encoding API.
>
> > object is passed through the encoder function found for the given
> > encoding using the error handling method defined by errors. errors
> > may be NULL to use the default method defined for the codec.
>
> > Raises a LookupError in case no encoder can be found.
>
> > */
>
> > PyAPI_FUNC(PyObject *) PyCodec_Encode(
> > PyObject *object,
> > const char *encoding,
> > const char *errors
> > );
>
> > encoding needs to be set to 'utf-16-le' for little endian, 'utf-16-be'
> > for big endian.
>
> > --
> > Marc-Andre Lemburg
> > eGenix.com
>
> > Professional Python Services directly from the Source (#1, Mar 23 2009)>>>
> > Python/Zope Consulting and Support ... http://www.egenix.com/
> > >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
> > >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
>
> >
> > 2009-03-19: Released mxODBC.Connect 1.0.1 http://python.egenix.com/
>
> > ::: Try our new mxODBC.Connect Python Database Interface for free !
>
> > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
> > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
> > Registered at Amtsgericht Duesseldorf: HRB 46611
> > http://www.egenix.com/company/contact/
>
> Thanks, but this is returning PyObject *, whereas I need value in some
> variable which can be printed using wprintf() like wchar_t (having a
> size of 2 bytes). If I again convert this PyObject to wchar_t or
> PyUnicode, I go back to where I started. :)
>
> -
> Abhigyan
Hi Marc,
Is there any way to ensure that wchar_t size would always be 2
instead of 4 in ucs4 configured python? Googling gave me the
impression that there is some logic written in PyUnicode_AsWideChar()
which can take care of ucs4 to ucs2 conversion if sizes of Py_UNICODE
and wchar_t are different.
-
Abhigyan
--
http://mail.python.org/mailman/listinfo/python-list
Re: file.read() doesn't read the whole file
Sreejith K wrote:
> > Try and write an example that shows the problem in fifteen lines or
> > less. Much easier for us to focus on the issue that way.
>
> import os
> def read(length, offset):
> os.chdir('/mnt/gfs_local/')
> snap = open('mango.txt_snaps/snap1/0','r')
> snap.seek(offset)
> data = snap.read(length)
> print data
>
> read(4096,0)
>
> This code shows what actually happens inside the code I've written.
> This prints the 4096 bytes from the file '0' which is only 654 bytes.
> When we run the code we get the whole file. That's right. I also get
> it. But when this read() function becomes the file class read()
> function in fuse, the data printed is not the whole but only a few
> lines from the beginning. I usually use less to read a file, when
> 'less'ing a file (whose size is less than 4096bytes) a call to read
> (0,4096) is made and data is returned. 'less' use this data returned
> by my fuse read() function to display its contents. But it was
> supposed to be the whole lines in the file like the example, but its
> not This is the problem I'm facing. Did I do something wrong here ?
If I'm understanding you correctly, you are saying that when you use
this function as the fuse read function you don't get the whole file,
and you are verifying this by using 'less' to read the 'file' exposed
by fuse. Correct?
So you still have not decoupled the python read from the fuse read in
your debugging. You are focused on the fact that the python read "must
be failing", yet you still (as far as you have told us) not _proven_
that by logging the value returned from the read. Until you do that,
you can't even be sure where your problem is.
If you have done it, show us the logging output, please.
--
R. David Murray http://www.bitdance.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: file.read() doesn't read the whole file
Sreejith K wrote:
>> Try and write an example that shows the problem in fifteen lines or
>> less. Much easier for us to focus on the issue that way.
>
> import os
> def read(length, offset):
> os.chdir('/mnt/gfs_local/')
> snap = open('mango.txt_snaps/snap1/0','r')
> snap.seek(offset)
> data = snap.read(length)
> print data
>
> read(4096,0)
>
> This code shows what actually happens inside the code I've written.
> This prints the 4096 bytes from the file '0' which is only 654 bytes.
> When we run the code we get the whole file. That's right. I also get
> it. But when this read() function becomes the file class read()
> function in fuse, the data printed is not the whole but only a few
> lines from the beginning.
This is confusing. I presume you to mean that when you make this
function a method of some class it stops operating correctly?
But I am not sure.
I am still struggling to understand your problem. Sorry,it's just a
language thing. If we take our time we will understand each other in the
end.
regards
Steve
> I usually use less to read a file, when
> 'less'ing a file (whose size is less than 4096bytes) a call to read
> (0,4096) is made and data is returned. 'less' use this data returned
> by my fuse read() function to display its contents. But it was
> supposed to be the whole lines in the file like the example, but its
> not This is the problem I'm facing. Did I do something wrong here ?
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/
--
http://mail.python.org/mailman/listinfo/python-list
Re: what features would you like to see in 2to3?
Kay Schluehr wrote: > On 22 Mrz., 20:39, Benjamin Peterson wrote: >> It's GSoC time again, and I've had lots of interested students asking about >> doing on project on improving 2to3. What kinds of improvements and features >> would you like to see in it which student programmers could accomplish? > > It would suffice to write some proper 2to3 tutorial level > documentation that explains how to extend 2to3 i.e. how to write some > new fixers. Most of the stuff I came along was rather special and > cheap student labor is hardly ever available or just in time. GSoC projects are supposed to produce code primarily, so a documentation project might not get support. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ Want to know? Come to PyCon - soon! http://us.pycon.org/ -- http://mail.python.org/mailman/listinfo/python-list
Re: script files with python (instead of tcsh/bash)?
MRAB wrote: Two quick questions: As a replacement for grep I would use the re module and its methods? What about awk which I regularly use to extract fields based on position but not column number, what should I be using in Python to do the same? Just use string slicing. Would that be equivalent to splitting the string and then subscripting on the result? That seems to be the closest to awk .. wouldn't string slicing depend on column numbers for the line (so to speak)i.e, specific know index values? The other things I need to do consist of moving files, manipulating file names and piping outputs of one command to the next, so I'm digging into the documentation as much as I can. The 'os' and 'shutil' modules. Excellent, thanks for the pointers. I was aware of the os module, but didn't know about the shutil module. The more I dig into Python, the more I like it and the more I'm impressed. (I do miss bock comments, when you are trying out new things that is useful .. I know I can use """ as a substitute though). Thanks again, Esmail -- http://mail.python.org/mailman/listinfo/python-list
Re: splitting a large dictionary into smaller ones
per wrote: > hi all, > > i have a very large dictionary object that is built from a text file > that is about 800 MB -- it contains several million keys. ideally i > would like to pickle this object so that i wouldnt have to parse this > large file to compute the dictionary every time i run my program. > however currently the pickled file is over 300 MB and takes a very > long time to write to disk - even longer than recomputing the > dictionary from scratch. > > i would like to split the dictionary into smaller ones, containing > only hundreds of thousands of keys, and then try to pickle them. is > there a way to easily do this? i.e. is there an easy way to make a > wrapper for this such that i can access this dictionary as just one > object, but underneath it's split into several? so that i can write > my_dict[k] and get a value, or set my_dict[m] to some value without > knowing which sub dictionary it's in. > > if there aren't known ways to do this, i would greatly apprciate any > advice/examples on how to write this data structure from scratch, > reusing as much of the dict() class as possible. > You aren't by any chance running this on Python 3.0, are you? The I/O implementation for that release is known to be slow, and this would have its effect on pickle dump/load performance. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ Want to know? Come to PyCon - soon! http://us.pycon.org/ -- http://mail.python.org/mailman/listinfo/python-list
Re: script files with python (instead of tcsh/bash)?
Hi Gabriel, Gabriel Genellina wrote: En Sun, 22 Mar 2009 11:05:22 -0300, MRAB escribió: Esmail wrote: Nick Craig-Wood wrote: Esmail wrote: <..> As a replacement for grep I would use the re module and its methods? Perhaps; but strings have methods too (`"abc" in line` is easier to read, and faster, than the corresponding r.e.) I'm a big fan of readability (which is one reason I am attracted to Python) .. thanks for pointing out the alternative, good to have options. The other things I need to do consist of moving files, manipulating file names and piping outputs of one command to the next, so I'm digging into the documentation as much as I can. The 'os' and 'shutil' modules. And for executing external commands with piping, I'd add the subprocess module. I will take a look at the subprocess module, thanks! Esmail -- http://mail.python.org/mailman/listinfo/python-list
Re: Escaping optional parameter in WHERE clause
MRAB wrote:
> someone wrote:
>> Hi,
>>
>> as you can see below I have some optional parameter for my query (mf,
>> age). They are in WHERE clause only if not empty.
>> In this function they are not escaped as, for example, 'search'
>> parameter, cause I can't pass them to execute function, which does
>> escaping automatically.
>>
>> I could write another if's block like that
>>
>> if mf and not age:
>> db.execute(query, search, mf, limit)
>> if age and not mf:
>> db.execute(query, search, age, limit)
>> if age and mf:
>> db.execute(query, search, mf, age, limit)
>>
>> Is there a better way to deal with optional WHERE clause?
>>
>> Pet
>>
>> def getData(self, db, params):
>> search = params.get('search','')
>> age = params.get('age','')
>> mf = params.get('mf','')
>> limit = params.get('limit',1)
>>
>> if mf:
>> mf = " AND mf = %s " % mf
>> if age:
>> age = " AND age = %s " % age
>>
>> query = """
>> SELECT * FROM mytable
>> WHERE class = 'P'
>> AND name = %s
>> """ + mf + """
>> """ + age + """
>> ORDER BY id DESC
>> LIMIT %s;
>> """
>>
>> db.execute(query, search, limit)
>> result = db.fetchall()
>> return result
>>
> How about:
>
> def getData(self, db, params):
> search = params.get('search', '')
> age = params.get('age', '')
> mf = params.get('mf', '')
> limit = params.get('limit', 1)
>
> query = """
> SELECT * FROM mytable
> WHERE class = 'P'
> AND name = %s
> """
> values = [search]
>
> if mf:
> query += " AND mf = %s"
> values.append(mf)
>
> if age:
> query += " AND age = %s"
> values.append(age)
>
> query += """
> ORDER BY id DESC
> LIMIT %s;
> """
> values.append(limit)
>
> db.execute(query, *values)
db.execute(query, tuple(values))
> result = db.fetchall()
> return result
The .execute() method should take two arguments, the second being a
tuple of data values. Some interfaces don't like an empty tuple when the
query has no parameters.
regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/
--
http://mail.python.org/mailman/listinfo/python-list
Re: script files with python (instead of tcsh/bash)?
Hello again Nick,
thanks for the additional script example. I was able to put
something together where I read the whole file into a list
as a series of lines (via readlines()) and then loop through
the lines seeing if the target string was "in" the line .. seems
to have worked reasonably well.
I am sure over time I will pick up the more Python(ic?) ways of
doing things.
I presume you mean something like this
... | awk '{print $2}'
In python, assuming you've got the line in the "line" variable, then
In python an equivalent of the above would be
import fileinput
for line in fileinput.input():
print line.split()[1]
cool .. that is what I came up with too!
Note that the fileinput module is really useful for making shell
command replacements!
yes, I am going to have to process a number of files, so I'm also
looking at glob and os.walk(??)
The other things I need to do consist of moving files, manipulating file
names and piping outputs of one command to the next, so I'm digging into
the documentation as much as I can.
Read up on the os module and the subprocess module. You'll find you
need to do much less piping with python as with shell because it has
almost everything you'll need built in.
Using built in functions is much quicker than fork()-ing an external
command too.
Thanks again for all the useful info and examples, this really has been
a great help.
Esmail
--
http://mail.python.org/mailman/listinfo/python-list
Re: Lambda forms and scoping
"Gabriel Genellina" wrote:
> > However, I think that a Python closure is not quite the same thing as a
> > 'computer science' closure, for the same reason that people coming from a
> > language with variables-and-values as opposed to namespaces get confused
> > when dealing with Python function call semantics. Consider:
> >
> > http://en.wikipedia.org/wiki/Closure_(computer_science)
> >
> > That says that a closure can be used to provide a function with a private
> > set of variables that persist from one invocation to the next, so that
> > a value established in one call can be accessed in the next. The last
> > part of that sentence is not true in Python, since any assignment inside
> > a function affects only the local (per-invocation) namespace or (given
> > a global statement) the global namespace. A function cannot change the
> > thing pointed to by a name in the closure. Only the outer function,
> > for whom that name is in its local namespace, can do that.
>
> That's true in Python 2.x, but 3.x has the "nonlocal" keyword - so you can
> modify variables in outer scopes too:
>
> p3> z = 1
> p3> def o():
> ... z = 2
> ... def i():
> ... nonlocal z
> ... print("z in i:", z)
> ... z = 5
> ... print("z in o:", z)
> ... i()
> ... print("z in o:", z)
> ... z=3
> ... print("z in o at exit:", z)
> ... return i
> ...
> p3> i=o()
> z in o: 2
> z in i: 2
> z in o: 5
> z in o at exit: 3
> p3> z
> 1
> p3> i()
> z in i: 3
> p3> i()
> z in i: 5
>
> (Anyway I think the inability to "modify" a variable doesn't invalidate
> the "closure" concept...)
Invalidate, no, but it does mean that the word meant something slightly
different to a Python 2.x programmer than to, say, a Scheme programmer.
We could say that a Python 2.x closure is a "read-only closure".
But now with Python 3.x we can really have fun (thank you for that info):
>>> def g():
... def a(x):
... nonlocal z
... z = z + x
... def b(x):
... nonlocal z
... z = z - x
... def p():
... print(z)
... z = 1
... return a, b, p
...
>>> add, sub, pr = g()
>>> pr()
1
>>> add(10)
>>> pr()
11
>>> sub(5)
>>> pr()
6
So, as the wikipedia article says, we could, if we wanted to, use python 3
closures to reimplement objects, in a very confusing fashion :)
--
R. David Murray http://www.bitdance.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: loading program's global variables in ipython
Peter Otten wrote:
Use
%run -i myfile.py
or
execfile("myfile.py") # not ipython-specific
thanks for these suggestions Peter, I have had exactly the same problem
and was looking for a way around it -- this will be very helpful.
Esmail
ps: for some reason I am unable to post to the
gmane.comp.python.ipython.user group. For one I can't install the
Windows version of iPython (Linux works superfine :-)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Generator
John Posner wrote: > > [snip] > > > If you want next(g) to yield 3, you'd have to do something like: > > > > > > g = (x for x in s[:]) > > > > > > where s[:] makes a copy of s that is then iterated over. > > > BTW, this simpler statement works, too: > >g = iter(s[:]) Yes, but one presumes that in the real code that prompted the OP's question he wasn't just returning 'x'. -- R. David Murray http://www.bitdance.com -- http://mail.python.org/mailman/listinfo/python-list
Relative Imports, why the hell is it so hard?
Hi All, I'm fairly new to Python so I still have a lot to learn. But I'd like to know how to correectly use relative imports. Please, please... please! don't go off on rants about why you think relative imports should not be used. I've got 15+ years in C++ and relative inclusion of other sections of code has never been a problem. As far as I am concerned what I am trying to do is perfectly reasonable and valid. Thank you in advance to everyone who helps solve this, because I just don't get it. Example: \ App | main.py +--\subpack1 | | __init__.py | | module1.py | +--\subpack2 | | __init__.py | | module2.py Module1 needs to access functionality in Module2. #module1.py from ..subpack2 import module2 Seems reasonable to me... but it just does not work and I was so liking Python. :( -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mon, Mar 23, 2009 at 10:16 AM, CinnamonDonkey wrote: > Hi All, > > I'm fairly new to Python so I still have a lot to learn. But I'd like > to know how to correectly use relative imports. > > Please, please... please! don't go off on rants about why you think > relative imports should not be used. I've got 15+ years in C++ and > relative inclusion of other sections of code has never been a problem. > As far as I am concerned what I am trying to do is perfectly > reasonable and valid. > > Thank you in advance to everyone who helps solve this, because I just > don't get it. > > Example: > > \ App > | main.py > +--\subpack1 > | | __init__.py > | | module1.py > | > +--\subpack2 > | | __init__.py > | | module2.py > > > Module1 needs to access functionality in Module2. > > #module1.py > from ..subpack2 import module2 > > Seems reasonable to me... but it just does not work and I was so > liking Python. :( Relative imports are perfectly fine, in my opinion. Do you have "from __future__ import absolute_import" at the top of module1.py? Should work fine once you add that line. - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
>> Please, please... please! don't go off on rants about why you think >> relative imports should not be used. I've got 15+ years in C++ and >> relative inclusion of other sections of code has never been a >> problem. As far as I am concerned what I am trying to do is >> perfectly reasonable and valid. However, C++ != Python. Regardless whether or not you can "make it work", translating idioms from one language to another is often suboptimal. That may not be the case here, but it bears keeping in mind. >> Example: >> \ App >> | main.py >> +--\subpack1 >> | | __init__.py >> | | module1.py >> | >> +--\subpack2 >> | | __init__.py >> | | module2.py >> Module1 needs to access functionality in Module2. >> #module1.py >> from ..subpack2 import module2 >> Seems reasonable to me... but it just does not work and I was so >> liking Python. :( You might try removing a dot from your import statement. Python is also not Unix. Popping up one level in the package hierarchy is done with ".", not "..". -- Skip Montanaro - s...@pobox.com - http://www.smontanaro.net/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
Hi Guys, Thanx for the quick responses, it is very much appreciated! Skip, that's a good point about "C++ != Python" and I assure you I am very much aware of that ;-). Looking at http://www.python.org/dev/peps/pep-0328/#guido-s-decision would suggest, unless I am completely miss-understanding the example, that '.' refers to the current level and '..' pops up a level. First three uses: # Access moduleY in same level as ModuleX from .moduleY import spam from .moduleY import spam as ham from . import moduleY Just to be sure though I tried both ;): from ..subpack2 import module1 #ValueError: Attempted relative import beyond toplevel package from .subpack2 import module1 #ImportError: No module named subpack2 from . import subpack2 #ImportError: cannot import name subpack2 from .. import subpack2 #ValueError: Attempted relative import beyond toplevel package Max, thank you for the response... I tried adding "from __future__ import absolute_import" which made no difference. I still get exactly the same error messages. Perhaps I should have mentioned that I am using Python 2.5, which I understand alread supports relative imports out of the box. I'll keep this line in for now anyway though :-) Cheers! #subpack1.module1 from __future__ import absolute_import from .. import subpack2 def subpack1_module1_foo(): print "subpack1_module1_foo()" call_subpack2_module1() def call_subpack2_module1(): subpack2.module2.subpack2_module2_foo() #subpack2.module2 def subpack2_module2_foo(): print "subpack2_module2_foo()" #main.py import subpack1.module1 if __name__ == "__main__": subpack1.module1.subpack1_module1_foo() I am starting the sandbox app with the command line: python main.py with the current working directory in \app where main.py is located. Shaun >8) On 23 Mar, 14:44, s...@pobox.com wrote: > >> Please, please... please! don't go off on rants about why you think > >> relative imports should not be used. I've got 15+ years in C++ and > >> relative inclusion of other sections of code has never been a > >> problem. As far as I am concerned what I am trying to do is > >> perfectly reasonable and valid. > > However, C++ != Python. Regardless whether or not you can "make it work", > translating idioms from one language to another is often suboptimal. That > may not be the case here, but it bears keeping in mind. > > >> Example: > > >> \ App > >> | main.py > >> +--\subpack1 > >> | | __init__.py > >> | | module1.py > >> | > >> +--\subpack2 > >> | | __init__.py > >> | | module2.py > > >> Module1 needs to access functionality in Module2. > > >> #module1.py > >> from ..subpack2 import module2 > > >> Seems reasonable to me... but it just does not work and I was so > >> liking Python. :( > > You might try removing a dot from your import statement. Python is also not > Unix. Popping up one level in the package hierarchy is done with ".", not > "..". > > -- > Skip Montanaro - s...@pobox.com -http://www.smontanaro.net/ -- http://mail.python.org/mailman/listinfo/python-list
Re: [python-list] Re: Strange crash issue on Windows w/ PyGTK, Cairo...
CJ Kucera wrote: > Okay, I've got a reproducible testcase of this available up here: > http://apocalyptech.com/pygtk-zlib/ > > I'm no longer *totally* convinced that it's a zlib issue... zlib's call > actually returns a valid string, and the error happens later in the app. Hello, again, list. One last update on this, in case anyone happened to be curious about it. After talking with the PyCairo folks, it looks like this *was* a PyCairo issue after all, and the changes to the zlib call which had apparently fixed the issue were just incidental, basically, because of how the memory allocation ended up changing because of the modified vars. Anyway, a recent fix in PyCairo CVS seems to take care of it. zlib, as probably should have been expected, is vindicated! -CJ -- WOW: Flemmy| "Happiness isn't good enough for me! I p...@apocalyptech.com | demand euphoria!" 24.24.2.3171 | - Calvin -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mon, Mar 23, 2009 at 11:22 AM, CinnamonDonkey wrote: > Looking at http://www.python.org/dev/peps/pep-0328/#guido-s-decision > would suggest, unless I am completely miss-understanding the example, > that '.' refers to the current level and '..' pops up a level. That is correct, but you cannot jump beyond the parent package, which is why your code isn't working. > Max, thank you for the response... I tried adding "from __future__ > import absolute_import" which made no difference. I still get exactly > the same error messages. Perhaps I should have mentioned that I am > using Python 2.5, which I understand alread supports relative imports > out of the box. I'll keep this line in for now anyway though :-) > Cheers! Sorry, I use that line to avoid conflicts with standard modules, and forgot that relative imports are already enabled. Basically, the reason your code doesn't work is because you're trying to use relative imports between two separate packages. As far as I know, this isn't possible. What I did to get your code working was move main.py one directory up and create an empty __init__.py under \App. The following code should then work: # main.py import App.subpack1.module1 if __name__ == "__main__": App.subpack1.module1.subpack1_module1_foo() # App.subpack1.module1 from ..subpack2 import module2 def subpack1_module1_foo(): print "subpack1_module1_foo()" call_subpack2_module1() def call_subpack2_module1(): module2.subpack2_module2_foo() # App.subpack2.module2 def subpack2_module2_foo(): print "subpack2_module2_foo()" - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
En Mon, 23 Mar 2009 12:22:21 -0300, CinnamonDonkey escribió: >> \ App >> | main.py >> +--\subpack1 >> | | __init__.py >> | | module1.py >> | >> +--\subpack2 >> | | __init__.py >> | | module2.py >> Module1 needs to access functionality in Module2. >> #module1.py >> from ..subpack2 import module2 >> Seems reasonable to me... but it just does not work and I was so >> liking Python. :( Another name for relative imports is "intra-package imports". They work *inside* a package, and you cannot go out of the package. If App is not a package, then subpack1 and subpack2 are separate packages and you cannot use relative imports between them. So module1 must refer to module2 absolutely: from subpack2 import module2 from ..subpack2 import module1 #ValueError: Attempted relative import beyond toplevel package See the exception message. Max, thank you for the response... I tried adding "from __future__ import absolute_import" which made no difference. I still get exactly the same error messages. Perhaps I should have mentioned that I am using Python 2.5, which I understand alread supports relative imports out of the box. I'll keep this line in for now anyway though :-) That __future__ line is not to enable relative imports (since they have incompatible syntax, don't require anything special) but to ensure Python interprets "normal" imports (that is, without leading dots) always as absolute. The default behavior in 2.5 is to try *both* ways before failing. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: pickle.load() extremely slow performance
Benjamin Peterson wrote: Terry Reedy udel.edu> writes: 3.1a1 is out and I believe it has the io improvements. Massive ones, too. It'd be interesting to see your results on the alpha. On 3.1a1 the unpickle step takes 2.4 seconds, an 1875% improvement. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
My applogies if this is a silly question... but what makes something a package? and does that mean that what I am trying to do is not possible ? :( On 23 Mar, 15:53, "Gabriel Genellina" wrote: > En Mon, 23 Mar 2009 12:22:21 -0300, CinnamonDonkey > escribió: > > > > >> >> \ App > >> >> | main.py > >> >> +--\subpack1 > >> >> | | __init__.py > >> >> | | module1.py > >> >> | > >> >> +--\subpack2 > >> >> | | __init__.py > >> >> | | module2.py > > >> >> Module1 needs to access functionality in Module2. > > >> >> #module1.py > >> >> from ..subpack2 import module2 > > >> >> Seems reasonable to me... but it just does not work and I was so > >> >> liking Python. :( > > Another name for relative imports is "intra-package imports". They work > *inside* a package, and you cannot go out of the package. > If App is not a package, then subpack1 and subpack2 are separate packages > and you cannot use relative imports between them. So module1 must refer to > module2 absolutely: > > from subpack2 import module2 > > > from ..subpack2 import module1 #ValueError: Attempted relative import > > beyond toplevel package > > See the exception message. > > > Max, thank you for the response... I tried adding "from __future__ > > import absolute_import" which made no difference. I still get exactly > > the same error messages. Perhaps I should have mentioned that I am > > using Python 2.5, which I understand alread supports relative imports > > out of the box. I'll keep this line in for now anyway though :-) > > That __future__ line is not to enable relative imports (since they have > incompatible syntax, don't require anything special) but to ensure Python > interprets "normal" imports (that is, without leading dots) always as > absolute. The default behavior in 2.5 is to try *both* ways before failing. > > -- > Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mon, Mar 23, 2009 at 12:19 PM, CinnamonDonkey wrote: > My applogies if this is a silly question... but what makes something a > package? and does that mean that what I am trying to do is not > possible ? A package is a directory that has an __init__.py file. That file can be empty, or contain some initialization code. In your original example, subpack1 and subpack2 are packages. By adding an empty __init__.py file under \App, I made App into a package, which allowed me to execute "import App.subpack1.module1" in main.py. See the following url for additional info: http://docs.python.org/tutorial/modules.html - Max -- http://mail.python.org/mailman/listinfo/python-list
pylint 0.17.0 and astng 0.18.0 release
Hello, we are glad to announce the release of pylint 0.17.0 http://www.logilab.org/project/pylint/0.17.0 which is based on a major refactoring of astng (0.18.0) http://www.logilab.org/project/logilab-astng/0.18.0 . For python 2.5, pylint will now use python's _ast module which is much faster than the older compiler.ast module. See the ChangeLog files for more detailed information and our blogentry http://www.logilab.org/blogentry/8554 explaining how we support both compiler and _ast. -- Emile Anclin http://www.logilab.fr/ http://www.logilab.org/ Informatique scientifique & et gestion de connaissances -- http://mail.python.org/mailman/listinfo/python-list
Re: Using python 3 for scripting?
On 3/22/2009 12:41 PM Chris Rebert apparently wrote: 2.6.1, the latest non-3.x release is probably best. Most libraries haven't been ported to 3.x yet, so Python 3 has yet to become widespread. This seems slightly optimistic to me. Until a week ago, there was not a NumPy release for 2.6. There is still not a SciPy release for 2.6. Most dismaying, the SimpleParse guys need a little help compiling SimpleParse for 2.6 and have not yet gotten that help. (Is anyone listening?) So 2.5.4 is still perhaps the safest bet, even though it is more awkward for writing code close to Python 3 syntax. Alan Isaac -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
En Mon, 23 Mar 2009 13:19:51 -0300, CinnamonDonkey escribió: My applogies if this is a silly question... but what makes something a package? A package is a directory with an __init__.py file [that Python is aware of]. and does that mean that what I am trying to do is not possible ? You can do an "absolute" import of subpack1 and subpack2. But you cannot import them "relatively" - not using your current configuration, but see Maxim Khitrov post for an alternate disposition that works. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: pickle.load() extremely slow performance
On Mon, 23 Mar 2009 10:57:54 -0500, Jim Garrison wrote: Benjamin Peterson wrote: Terry Reedy udel.edu> writes: 3.1a1 is out and I believe it has the io improvements. Massive ones, too. It'd be interesting to see your results on the alpha. On 3.1a1 the unpickle step takes 2.4 seconds, an 1875% improvement. Surely you mean a 94.7% improvement? Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: script files with python (instead of tcsh/bash)?
On 3/21/2009 9:26 AM Esmail apparently wrote: I also write out some gnuplot scripts that later get executed to generate .jpg images. See Gnuplot.py Alan Isaac -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mar 23, 11:22 am, CinnamonDonkey wrote: > Hi Guys, > > Thanx for the quick responses, it is very much appreciated! > > Skip, that's a good point about "C++ != Python" and I assure you I am > very much aware of that ;-). > > Looking athttp://www.python.org/dev/peps/pep-0328/#guido-s-decision > would suggest, unless I am completely miss-understanding the example, > that '.' refers to the current level and '..' pops up a level. First > three uses: > > # Access moduleY in same level as ModuleX > from .moduleY import spam > from .moduleY import spam as ham > from . import moduleY > > Just to be sure though I tried both ;): > > from ..subpack2 import module1 #ValueError: Attempted relative import > beyond toplevel package > from .subpack2 import module1 #ImportError: No module named subpack2 > from . import subpack2 #ImportError: cannot import name subpack2 > from .. import subpack2 #ValueError: Attempted relative import beyond > toplevel package > > Max, thank you for the response... I tried adding "from __future__ > import absolute_import" which made no difference. I still get exactly > the same error messages. Perhaps I should have mentioned that I am > using Python 2.5, which I understand alread supports relative imports > out of the box. I'll keep this line in for now anyway though :-) > Cheers! I wrote http://code.google.com/p/pyimport-relative so I could have a simple and more reliable way to specify a *file* relative import in Python 2.5 (and it works with 2.6 as well). I'm not sure it's for everyone, or that it is flawless. Actually, it's not. But I use it for in pydbgr (http://code.google.com/p/pydbgr) so see that for examples of how to use. It allows me to run the development code out of the source tree while having possibly a different version installed as an egg. > > #subpack1.module1 > from __future__ import absolute_import > > from .. import subpack2 > > def subpack1_module1_foo(): > print "subpack1_module1_foo()" > call_subpack2_module1() > > def call_subpack2_module1(): > subpack2.module2.subpack2_module2_foo() > > #subpack2.module2 > def subpack2_module2_foo(): > print "subpack2_module2_foo()" > > #main.py > import subpack1.module1 > > if __name__ == "__main__": > subpack1.module1.subpack1_module1_foo() > > I am starting the sandbox app with the command line: python main.py > with the current working directory in \app where main.py is located. > > Shaun >8) > > On 23 Mar, 14:44, s...@pobox.com wrote: > > > >> Please, please... please! don't go off on rants about why you think > > >> relative imports should not be used. I've got 15+ years in C++ and > > >> relative inclusion of other sections of code has never been a > > >> problem. As far as I am concerned what I am trying to do is > > >> perfectly reasonable and valid. > > > However, C++ != Python. Regardless whether or not you can "make it work", > > translating idioms from one language to another is often suboptimal. That > > may not be the case here, but it bears keeping in mind. > > > >> Example: > > > >> \ App > > >> | main.py > > >> +--\subpack1 > > >> | | __init__.py > > >> | | module1.py > > >> | > > >> +--\subpack2 > > >> | | __init__.py > > >> | | module2.py > > > >> Module1 needs to access functionality in Module2. > > > >> #module1.py > > >> from ..subpack2 import module2 > > > >> Seems reasonable to me... but it just does not work and I was so > > >> liking Python. :( > > > You might try removing a dot from your import statement. Python is also not > > Unix. Popping up one level in the package hierarchy is done with ".", not > > "..". > > > -- > > Skip Montanaro - s...@pobox.com -http://www.smontanaro.net/ > > -- http://mail.python.org/mailman/listinfo/python-list
Re: how to convert from Decimal('1.23456789') to Decimal('1.234')
On Mar 23, 5:48 am, Steven D'Aprano wrote:
> On Mon, 23 Mar 2009 01:45:53 -0700, Mensanator wrote:
> >> but you can create a helper
> >> function very easily:
>
> >> def round(dec, places, rounding=decimal.ROUND_HALF_UP): return
> >> dec.quantize(decimal.Decimal(str(10**-places)), rounding)
>
> > Still ugly. I would do this:
>
> a = Decimal('1.23456789')
>
> for i in xrange(1,6):
> > print Context.create_decimal(Context(i,ROUND_DOWN),a)
>
> Well, that's hardly any less ugly.
I wouldn't say so since there are no strings attached. Didn't the
OP specifically ask for a solution that didn't involve strings?
>
> And it also gives different results to my function: my function rounds to
> decimal places, yours to digits. Very different things.
Yeah, I know all about that. I work in Environmental Remediation.
That's real science, where rounding to decimal places is strictly
forbidden, significant digits must be preserved. That means rounding
to digits. Do you know what kind of hoops I have to jump through to
get Access or Excel to round properly when doing unit conversion?
Surely you're not so maive that you think dividing by 1000 simply
moves
the decimal point three places?
>
> --
> Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: pickle.load() extremely slow performance
Jean-Paul Calderone wrote: > On Mon, 23 Mar 2009 10:57:54 -0500, Jim Garrison wrote: >> Benjamin Peterson wrote: >>> Terry Reedy udel.edu> writes: 3.1a1 is out and I believe it has the io improvements. >>> >>> Massive ones, too. It'd be interesting to see your results on the alpha. >> >> On 3.1a1 the unpickle step takes 2.4 seconds, an 1875% improvement. > > Surely you mean a 94.7% improvement? > Well, since it's now running almost twenty times faster, the speed has increased by 1875%. Not sure what the mathematics of improvement are ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ Want to know? Come to PyCon - soon! http://us.pycon.org/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Async serial communication/threads sharing data
Jean-Paul Calderone wrote:
> On Mon, 23 Mar 2009 05:30:04 -0500, Nick Craig-Wood
> wrote:
> >Jean-Paul Calderone wrote:
> > [snip]
> >>
> >> In the case of a TCP to serial forwarder, you don't actually have to
> >> implement either a producer or a consumer, since both the TCP connection
> >> and the serial connection are already both producers and consumers. All
> >> you need to do is hook them up to each other so that when the send buffer
> >> of one fills up, the other one gets paused, and when the buffer is empty
> >> again, it gets resumed.
> >
> >I eventually came up with this which seems to work, but I'm not sure
> >it is the best way of doing it as it had to mess about with the
> >twisted internals to get the number of bytes in the serial port
> >output buffer.
>
> This is sort of on the right track. Here's how to do it without
> touching implementation details:
Thank you for that!
See below for the complete prog with your suggested modifications.
That seems to solve the problem - I can see the writer pausing and
unpausing at the serial port rate.
write 16446
write 6430
pause producing
resume producing
write 65536
write 56724
pause producing
resume producing
write 65536
write 65536
pause producing
It has exposed a problem with the sender not throttling properly now,
but I guess that is progress!
Thanks for your help
Here is the complete program FYI with your suggested mods.
#!/usr/bin/python
"""Transfer data between a serial port and one (or more) TCP
connections.
options:
-h, --help:this help
-p, --port=PORT: port, a number, default = 0 or a device name
-b, --baud=BAUD: baudrate, default 115200
-t, --tcp=PORT: TCP port number, default 1234
-l, --log: log data streams to 'snifter-0', 'snifter-1'
-L, --log_name=NAME: log data streams to '-0', '-1'
"""
import sys
import getopt
from twisted.internet import reactor, protocol, serialport
from zope.interface import implements
from twisted.internet import protocol, interfaces
# FIXME set serial buffer size? SEND_LIMIT
class SerialPort(protocol.Protocol):
"""Create a serial port connection and pass data from it to a
known list of
TCP ports."""
def __init__(self, port, reactor, baudrate, log_name=None):
self.tcp_ports = []
self.serial = serialport.SerialPort(self, reactor, port,
baudrate, rtscts=0)
self.serial.registerProducer(self, True)
self.paused = False
self.log = None
if log_name is not None:
self.log = file('%s-0' % log_name, 'w')
def add_tcp(self, tcp_port):
"""Add a TCPPort to those receiving serial data."""
if self.paused:
tcp_port.transport.pauseProducing()
self.tcp_ports.append(tcp_port)
def del_tcp(self, tcp_port):
"""Remove a TCPPort from the those receiving serial data."""
self.tcp_ports.remove(tcp_port)
def write(self, data):
"""Write data to the serial port."""
self.serial.write(data)
if self.log:
self.log.write(data)
def pauseProducing(self):
"""Pause producing event"""
print "pause producing"
self.paused = True
for port in self.tcp_ports:
port.transport.pauseProducing()
def resumeProducing(self):
"""Resume producing event"""
print "resume producing"
self.paused = False
for port in self.tcp_ports:
port.transport.resumeProducing()
def stopProducing(self):
"""Stop producing event"""
print "stop producing"
def dataReceived(self, data):
"""Pass any received data to the list of TCPPorts."""
for tcp_port in self.tcp_ports:
tcp_port.write(data)
class TCPPort(protocol.Protocol):
"""Create a TCP server connection and pass data from it to the
serial port."""
def __init__(self, serial, log_name, index):
"""Add this TCPPort to the SerialPort."""
self.serial = serial
self.serial.add_tcp(self)
self.log = None
if log_name is not None:
self.log = file('%s-%d' % (log_name, index+1), 'w')
def __del__(self):
"""Remove this TCPPort from the SerialPort."""
self.serial.del_tcp(self)
def dataReceived(self, data):
"""Pass received data to the SerialPort."""
print "write", len(data)
self.serial.write(data)
def write(self, data):
"""Write data to the TCP port."""
self.transport.write(data)
if self.log is not None:
self.log.write(data)
class TCPPortFactory(protocol.ServerFactory):
"""Factory to create TCPPort protocol instances, an instanced
SerialPort must be passed in."""
def __init__(self, serial, log_name=None):
self.serial = serial
self.log_name = log_name
self.index = 0
def buildProtocol(self, addr):
"""Build a TCPPort, passing in the insta
mocking xmlrpclib
At the risk of sounding like I don't know what I'm doing, I must say that I am finding it rather difficult/tedious to mock the xmlrpclib interface using minimock. I refuse to believe that I'm the only developer to have tried this before, but google isn't being my friend and I can't seem to get it to work outside the interpreter (ie: in actual unit tests). Has anyone in c.l.p encountered this problem or have any tips? -- http://mail.python.org/mailman/listinfo/python-list
Syntax error when importing a file which starts with a number
Hello, all. I don't suppose anyone has any idea why it seems to be impossible to import any file which starts with a number? You get a syntax error, whether the file exists or not. Try it yourself: >>> import foo ImportError: No module named foo >>> import 1foo File "", line 1 import 1foo ^ SyntaxError: invalid syntax Is this just me, or has anyone else run into it? Is it a known bug? (If so, I can't find it on a bug tracker or in any Google searches). It's a bit annoying, as I have an enforced naming scheme. Any way round it? Thanks in advance! Simon -- http://mail.python.org/mailman/listinfo/python-list
Re: Syntax error when importing a file which starts with a number
Forgot to mention: I'm on Python 2.5.2, on Ubuntu 8.10. Simon -- http://mail.python.org/mailman/listinfo/python-list
Re: Disable automatic interning
George Sakkis writes:
> I'm working on some graph generation problem where the node identity
> is significant (e.g. "if node1 is node2: # do something) but ideally I
> wouldn't want to impose any constraint on what a node is
I'm not sure if it helps in your case, but you can easily turn off the
optimization by subclassing from the string type:
>>> class mystr(str):
... pass
...
>>> s = mystr('x')
>>> s
'x'
>>> s is 'x'
False
Since mystr can have additional functionality on top of str, the
caching doesn't apply to the subclass instances.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Syntax error when importing a file which starts with a number
On Mon, Mar 23, 2009 at 1:56 PM, wrote:
> Hello, all.
>
> I don't suppose anyone has any idea why it seems to be impossible to
> import any file which starts with a number? You get a syntax error,
> whether the file exists or not.
Identifiers can't start with a number.
http://docs.python.org/reference/lexical_analysis.html#identifiers
> It's a bit annoying, as I have an enforced naming scheme. Any way
> round it?
You could import it like so:
some_valid_identifer = __import__('1foo', globals(), locals())
http://docs.python.org/library/functions.html#__import__
But a far better solution is to fix your naming scheme—it's completely
illogical to have a Python module naming scheme where the names aren't
valid identifiers.
-Miles
--
http://mail.python.org/mailman/listinfo/python-list
Re: Syntax error when importing a file which starts with a number
En Mon, 23 Mar 2009 14:56:21 -0300, escribió: I don't suppose anyone has any idea why it seems to be impossible to import any file which starts with a number? You get a syntax error, whether the file exists or not. You don't import a file, you import a module. And a module name is an identifier: http://docs.python.org/reference/simple_stmts.html#the-import-statement Identifiers must begin with a letter or underscore. (Standalone scripts that aren't supposed to be imported don't have to obey this rule, like 2to3.py) Is this just me, or has anyone else run into it? Is it a known bug? (If so, I can't find it on a bug tracker or in any Google searches). It's not a bug. Suppose you have a file '1.py' and you could write: import 1 x = 1 Does x refer to the integer 1 or the module 1? It's a bit annoying, as I have an enforced naming scheme. Any way round it? You might use a prefix in all these modules. Or just an _ in front of the ones that start with a number. But remember that a module name must obey both your filesystem rules for file names and Python rules for identifiers: letters A-Z (unless you're using Python 3), digits, _. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Mail client in Python?
Idle curiosity: is there a (decent) IMAP mail client (web or local) written in Python? I've got a project that needs doing, and it just occurred to me that a mail client might be the ideal interface; I'd have to change some back-end stuff (to do database queries instead of IMAP or POP queries), but could save myself a huge amount of work re-creating the wheel. Thanks for any recommendations! -Ken -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- http://mail.python.org/mailman/listinfo/python-list
Re: Syntax error when importing a file which starts with a number
The grammar indicates that the module name is an identifier, and identifiers can't start with digits (you can't have a variable name that starts with a '1' either). This is probably quite fundamental (I guess the lexer will implement it) so suspect it is impossible to change. That means it is a bug, not a feature (and it's quite a reasonable restriction, since it reduces ambiguity). See http://docs.python.org/3.0/reference/simple_stmts.html#import http://docs.python.org/3.0/reference/lexical_analysis.html#grammar-token-identifier Andrew simon.wo...@gmail.com wrote: > Hello, all. > > I don't suppose anyone has any idea why it seems to be impossible to > import any file which starts with a number? You get a syntax error, > whether the file exists or not. > > Try it yourself: > import foo > ImportError: No module named foo > import 1foo > File "", line 1 > import 1foo >^ > SyntaxError: invalid syntax > > Is this just me, or has anyone else run into it? Is it a known bug? > (If so, I can't find it on a bug tracker or in any Google searches). > > It's a bit annoying, as I have an enforced naming scheme. Any way > round it? > > Thanks in advance! > > Simon > -- > http://mail.python.org/mailman/listinfo/python-list > > -- http://mail.python.org/mailman/listinfo/python-list
Re: Syntax error when importing a file which starts with a number
simon.wo...@gmail.com wrote: Hello, all. I don't suppose anyone has any idea why it seems to be impossible to import any file which starts with a number? You get a syntax error, whether the file exists or not. Try it yourself: import foo ImportError: No module named foo import 1foo File "", line 1 import 1foo ^ SyntaxError: invalid syntax Is this just me, or has anyone else run into it? Is it a known bug? (If so, I can't find it on a bug tracker or in any Google searches). It's a bit annoying, as I have an enforced naming scheme. Any way round it? A module name must be a valid identifier. For example, if you do: import foo then you can write: foo.bar() but if it allowed: import 1foo then you'd be stuck because you can't write: 1foo.bar() You also aren't allowed to have spaces in a name, so no, it's not a bug. I suppose that if there was a demand for this kind of thing then quoting could be used for the filename: import "1foo" as one_foo -- http://mail.python.org/mailman/listinfo/python-list
Re: Syntax error when importing a file which starts with a number
andrew cooke wrote: ffs. feature, not bug. sorry. > This is probably quite fundamental (I guess the lexer will implement it) > so suspect it is impossible to change. That means it is a bug, not a > feature (and it's quite a reasonable restriction, since it reduces > ambiguity). -- http://mail.python.org/mailman/listinfo/python-list
Re: Mail client in Python?
On Mar 23, 2009, at 7:14 PM, Ken D'Ambrosio wrote: Idle curiosity: is there a (decent) IMAP mail client (web or local) written in Python? I've got a project that needs doing, and it just occurred to me that a mail client might be the ideal interface; I'd have to change some back-end stuff (to do database queries instead of IMAP or POP queries), but could save myself a huge amount of work re- creating the wheel. You might want to take a look at Chandler: http://chandlerproject.org/ -- PA. http://alt.textdrive.com/nanoki/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Using python 3 for scripting?
Alan G Isaac wrote: > On 3/22/2009 12:41 PM Chris Rebert apparently wrote: > > 2.6.1, the latest non-3.x release is probably best. Most libraries > > haven't been ported to 3.x yet, so Python 3 has yet to become > > widespread. > > This seems slightly optimistic to me. Until a week ago, there was > not a NumPy release for 2.6. There is still not a SciPy release > for 2.6. Most dismaying, the SimpleParse guys need a little help > compiling SimpleParse for 2.6 and have not yet gotten that help. > (Is anyone listening?) > > So 2.5.4 is still perhaps the safest bet, even though it is more > awkward for writing code close to Python 3 syntax. I tend to target whatever is in Debian stable, which starting from this month is 2.5 (recently upgraded from 2.4). 2.6 or 3.x is nowhere to be seen in Debian stable, testing or unstable :-( -- Nick Craig-Wood -- http://www.craig-wood.com/nick -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
CinnamonDonkey: >what makes something a package? If you don't know what a package is, then maybe you don't need packages. In your project is it possible to avoid using packages and just use modules in the same directory? Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Portable Python 1.1 released
This is a great piece of work. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
udp package header
Hi all, I got a problem. İ want to send udp package and get this package (server and clinet ). it's easy to python but i want to look the udp header how can i do ? -- http://mail.python.org/mailman/listinfo/python-list
Wing IDE 3.1.8 released
Hi, Wingware has released version 3.1.8 of Wing IDE, a bug-fix release for all three product levels of Wing IDE. *Release Highlights* This release includes the following: * Fixed problems seen with Subversion 1.4+ * Properly ignore settrace exception on x64 systems * Fixed "perforce submit" for some Perforce versions * Show meaningful error when debug a main file within a zip or egg * Fixed cursor color in search fields when editor background is not white * Use .chm file on Windows for python manual, when available * Upgraded Wing's private copy of Python to 2.5.4 (includes security patches) * About 5 other bug fixes: see the change log for details: http://wingware.com/pub/wingide/3.1.8/CHANGELOG.txt *Downloads* Wing IDE Professional and Wing IDE Personal are commercial software and require a license to run. A free trial license can be obtained directly from the product when launched. Wing IDE Pro 3.1.8http://wingware.com/downloads/wingide/3.1 Wing IDE Personal 3.1.8 http://wingware.com/downloads/wingide-personal/3.1 Wing IDE 101 3.1.8http://wingware.com/downloads/wingide-101/3.1 *About Wing IDE* Wing IDE is an integrated development environment for the Python programming language. It provides powerful debugging, editing, code intelligence, testing, and search capabilities that reduce development and debugging time, cut down on coding errors, and make it easier to understand and navigate Python code. Wing IDE is available in three product levels: Wing IDE Professional is the full-featured Python IDE, Wing IDE Personal offers a reduced feature set at a low price, and Wing IDE 101 is a free simplified version designed for teaching entry level programming courses with Python. System requirements are Windows 2000 or later, OS X 10.3.9 or later for PPC or Intel (requires X11 Server), or a recent Linux system (either 32 or 64 bit). Wing IDE 3.1 supports Python versions 2.0.x through 2.5.x. *New Features in Wing 3.1* This release adds the following features not found in Wing 3.0.x: * Support for zip archives * Support for pkg_resources name spaces and eggs * Support for doctest and nose style unit tests (*) * Scan for sys.path changes such as those used in buildout * How-To and support for Google App Engine * Inline context appropriate templates/snippets integrated with autocompleter (*) * Word list driven auto-completion in non-Python files (**) * Quick navigation to files and symbols by typing a fragment (**) * Improved support for Stackless Python * Preference to strip trailing white space on save * Display gi_running and gi_frame for generators * Improved code analysis for Python 2.5 * Other minor features and bug fixes not found in Wing 3.0.x (*)'d items are available in Wing IDE Professional only. (**)'d items are available in Wing IDE Personal or Professional only. Please see the change log for a detailed list of changes: http://wingware.com/pub/wingide/3.1.8/CHANGELOG.txt *Purchasing and Upgrading* Wing 3.1 is a free upgrade for all Wing IDE 3.0 and 3.1 users. Any 2.x license sold after May 2nd 2006 is free to upgrade; others cost 1/2 the normal price to upgrade. Upgrade a 2.x license: https://wingware.com/store/upgrade Purchase a 3.x license:https://wingware.com/store/purchase -- The Wingware Team Wingware | Python IDE Advancing Software Development www.wingware.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Syntax error when importing a file which starts with a number
On Mar 24, 4:56 am, simon.wo...@gmail.com wrote: > It's a bit annoying, as I have an enforced naming scheme. Do you mean that some authority other than yourself is seriously insisting that the names of source files *must* start with one or more digits? What is the rationale for such a scheme? -- http://mail.python.org/mailman/listinfo/python-list
Re: pickle.load() extremely slow performance
Steve Holden wrote: Jean-Paul Calderone wrote: On Mon, 23 Mar 2009 10:57:54 -0500, Jim Garrison wrote: Benjamin Peterson wrote: Terry Reedy udel.edu> writes: 3.1a1 is out and I believe it has the io improvements. Massive ones, too. It'd be interesting to see your results on the alpha. On 3.1a1 the unpickle step takes 2.4 seconds, an 1875% improvement. Surely you mean a 94.7% improvement? Well, since it's now running almost twenty times faster, the speed has increased by 1875%. Not sure what the mathematics of improvement are ... regards Steve The arithmetic depends on whether you're looking at time or velocity, which are inverses of each other. If you double your velocity (100% increase) the time required goes down by 50%. A 1000% increase in velocity results in a 90% decrease in time... etc. I guess I equate "performance" to velocity. -- http://mail.python.org/mailman/listinfo/python-list
Re: Unicode problem in ucs4
On 2009-03-23 14:05, abhi wrote: > Hi Marc, >Is there any way to ensure that wchar_t size would always be 2 > instead of 4 in ucs4 configured python? Googling gave me the > impression that there is some logic written in PyUnicode_AsWideChar() > which can take care of ucs4 to ucs2 conversion if sizes of Py_UNICODE > and wchar_t are different. wchar_t is defined by your compiler. There's no way to change that. However, you can configure Python to use UCS2 (default) or UCS4 (used on most Unix platforms), so it's easy to customize for your needs. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 23 2009) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2009-03-19: Released mxODBC.Connect 1.0.1 http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Unicode problem in ucs4
On 2009-03-23 12:57, abhi wrote: >>> Is there any way >>> by which I can force wchar_t to be 2 bytes, or can I convert this UCS4 >>> data to UCS2 explicitly? >> Sure: just use the appropriate UTF-16 codec for this. >> >> /* Generic codec based encoding API. >> >>object is passed through the encoder function found for the given >>encoding using the error handling method defined by errors. errors >>may be NULL to use the default method defined for the codec. >> >>Raises a LookupError in case no encoder can be found. >> >> */ >> >> PyAPI_FUNC(PyObject *) PyCodec_Encode( >>PyObject *object, >>const char *encoding, >>const char *errors >>); >> >> encoding needs to be set to 'utf-16-le' for little endian, 'utf-16-be' >> for big endian. > > Thanks, but this is returning PyObject *, whereas I need value in some > variable which can be printed using wprintf() like wchar_t (having a > size of 2 bytes). If I again convert this PyObject to wchar_t or > PyUnicode, I go back to where I started. :) It will return a PyString object with the UTF-16 data. You can use PyString_AS_STRING() to access the data stored by it. Note that writing your own UCS2/UCS4 converter isn't all that hard either. Just have a look at the code in unicodeobject.c for PyUnicode_AsWideChar(). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 23 2009) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2009-03-19: Released mxODBC.Connect 1.0.1 http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ -- http://mail.python.org/mailman/listinfo/python-list
Re: what features would you like to see in 2to3?
Kay Schluehr gmx.net> writes: > > On 22 Mrz., 20:39, Benjamin Peterson wrote: > > It's GSoC time again, and I've had lots of interested students asking about > > doing on project on improving 2to3. What kinds of improvements and features > > would you like to see in it which student programmers could accomplish? > > It would suffice to write some proper 2to3 tutorial level > documentation that explains how to extend 2to3 i.e. how to write some > new fixers. Yes, documentation is somewhere down my todo list. I might get some of it done this summer. -- http://mail.python.org/mailman/listinfo/python-list
Re: Syntax error when importing a file which starts with a number
Many thanks to all for explanations. I'm going to take everyone's advice and ignore the naming scheme (especially as, on rereading, the naming scheme is apparently only mandatory if you're using C or Maple, for some reason). Thanks again. Simon (For those interested: > Do you mean that some authority other than yourself is seriously > insisting that the names of source files *must* start with one or more > digits? What is the rationale for such a scheme? The relevent section from the manual is > What to submit if you used C or Maple on the PWF > The first few characters of each file name should give the project number, > with the dot replaced by a minus sign or underscore ( _ ). For example, all > the programs written for project 1.2 should have file names beginning with > 1-2 or 1_2; for instance 1-2prog.c or 1_2equip.c... As "project" and "submit" suggest, this is for education rather than work, so a certain amount of educational bureaucracy is not unsurprising. Also, it does then go on to state that if you're unable to use that convention then you can just include a readme file that explains what you've done.) -- http://mail.python.org/mailman/listinfo/python-list
Re: script files with python (instead of tcsh/bash)?
Alan G Isaac wrote: On 3/21/2009 9:26 AM Esmail apparently wrote: I also write out some gnuplot scripts that later get executed to generate .jpg images. See Gnuplot.py Thanks Alan, I will! Esmail -- http://mail.python.org/mailman/listinfo/python-list
Re: decorators tutorials
Josiah Carlson wrote:
... I try to limit my use of decorators whenever possible, both because I
still have to support Python 2.3 (which doesn't support the syntax), and
because I find that they obfuscate what the code is doing more often
than not. I will admit that they are useful as a metaprogramming
technique. Just be careful.
I find them very useful in debugging (for example writing a decorator
'traced'), and have also used them to simplify code that puts a lot
of functions into a dictionary. In the latter case, the function can
be returned unadorned, or can be stored, and only the locater for the
function returned:
function_table = []
def enumerated(function):
function_table.append(function)
return len(function_table) - 1
@enumerated
def triple(a, b, c):
blah, blah, blah
@enumerated
def double(a, b):
blah, blah, blah
@enumerated
def another(a, b, c):
blah, blah, blah
After all of that, the identifiers triple, double, and another refer
to 0, 1, and 2 respectively, and you can call functions with the
following structure:
function_table[double]('pi', 3.14)
This can be very useful for certain kinds of table-driven code (where
you may want to be able to efficiently store lists of functions to call
in data files).
--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list
Re: Syntax error when importing a file which starts with a number
On Mar 24, 9:12 am, simon.wo...@gmail.com wrote: > Many thanks to all for explanations. I'm going to take everyone's > advice and ignore the naming scheme (especially as, on rereading, the > naming scheme is apparently only mandatory if you're using C or Maple, > for some reason). > > Thanks again. > > Simon > > (For those interested: > > > Do you mean that some authority other than yourself is seriously > > insisting that the names of source files *must* start with one or more > > digits? What is the rationale for such a scheme? > > The relevent section from the manual is > > > What to submit if you used C or Maple on the PWF > > The first few characters of each file name should give the project number, > > with the dot replaced by a minus sign or underscore ( _ ). For example, all > > the programs written for project 1.2 should have file names beginning with > > 1-2 or 1_2; for instance 1-2prog.c or 1_2equip.c... Gulp ... So in a sorted list of files, some of the project 1.2 files will appear under 1-2 and others miles away under 1_2 ? And even if the submitter is not so dopey, the submittee has two different possibilities when looking for project 1.2's files? Brilliant! And this is an educational institution ? Have they never heard of directories? Time to upgrade from MS-DOS 1.0 to MS-DOS 2.0! And what's wrong with a dot in a filename anyway? -- http://mail.python.org/mailman/listinfo/python-list
Re: Syntax error when importing a file which starts with a number
> So in a sorted list of files, some of the project 1.2 files will > appear under 1-2 and others miles away under 1_2 ? And even if > the submitter is not so dopey, the submittee has two different > possibilities when looking for project 1.2's files? Brilliant! And > this is an educational institution ? Have they never heard of > directories? Time to upgrade from MS-DOS 1.0 to MS-DOS 2.0! And what's > wrong with a dot in a filename anyway? It's not that bad, thankfully: you do submit into a dedicated username- allocated directory, so the files from everyone don't go all into one place. Barely anything's done with the submitted files anyway: I think they just run them through some software to make sure it's not a blatant copy&paste of anyone else's. The actual marking is done on your written report -- which (in addition to the electronic submission) has to contain printed listings of all your programs. Thanks again. Simon -- http://mail.python.org/mailman/listinfo/python-list
Re: What's the difference between generating a value and returning a value?
On 3/23/2009 6:12 PM grocery_stocker apparently wrote: http://openbookproject.net/thinkCSpy/ch05.xhtml#index15 "The built-in functions we have used, such as abs, pow, and max, have produced results. Calling each of these functions generates a value, which we usually assign to a variable or use as part of an expression. biggest = max(3, 7, 2, 5) x = abs(3 - 11) + 10 But so far, none of the functions we have written has returned a value. I do not think this distinction is intended. The key point is that the user-*written* functions up to then do not include and explicit return statement. (Actually, they still return None, which can in fact be bound to a name.) However it might (?) be that they are trying to allow that some callable objects are not function types. (E.g., creating an instance by calling the class, which some of the builtins do.) If so, this terminology would not be a standard way of making that distinction. Alan Isaac (*not* a CS type) -- http://mail.python.org/mailman/listinfo/python-list
Re: how to convert from Decimal('1.23456789') to Decimal('1.234')
On Mon, 23 Mar 2009 10:06:23 -0700, Mensanator wrote:
> On Mar 23, 5:48 am, Steven D'Aprano cybersource.com.au> wrote:
>> On Mon, 23 Mar 2009 01:45:53 -0700, Mensanator wrote:
>> >> but you can create a helper
>> >> function very easily:
>>
>> >> def round(dec, places, rounding=decimal.ROUND_HALF_UP): return
>> >> dec.quantize(decimal.Decimal(str(10**-places)), rounding)
>>
>> > Still ugly. I would do this:
>>
>> a = Decimal('1.23456789')
>>
>> for i in xrange(1,6):
>> > print Context.create_decimal(Context(i,ROUND_DOWN),a)
>>
>> Well, that's hardly any less ugly.
>
> I wouldn't say so since there are no strings attached. Didn't the OP
> specifically ask for a solution that didn't involve strings?
No, the OP asked for a solution that didn't involve converting the
decimal number to a string first.
Besides, I don't believe you can specify the rounding mode unless you use
strings *wink*
>>> type(ROUND_DOWN)
>> And it also gives different results to my function: my function rounds
>> to decimal places, yours to digits. Very different things.
>
> Yeah, I know all about that. I work in Environmental Remediation. That's
> real science, where rounding to decimal places is strictly forbidden,
> significant digits must be preserved. That means rounding to digits.
Okay, so in other words you solved your problem rather than the OP's
problem.
> Do
> you know what kind of hoops I have to jump through to get Access or
> Excel to round properly when doing unit conversion?
I feel your pain.
> Surely you're not so maive that you think dividing by 1000 simply moves
> the decimal point three places?
Of course it does, if you're using real numbers. If you're using floats,
no, not quite, there are rounding issues involved, and underflow.
I'm not sure why you raise this. Is this a general rant, or do you have a
specific criticism?
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
