a_list.count(a_callable) ?
Hi, I'm wondering if it is useful to extend the count() method of a list to accept a callable object? What it does should be quite intuitive: count the number of items that the callable returns True or anything logically equivalent (non-empty sequence, non-zero number, etc). This would return the same result as len(filter(a_callable, a_list)), but without constructing an intermediate list which is thrown away after len() is done. This would also be equivalent to n = 0 for i in a_list: if a_callable(i): n += 1 but with much shorter and easier-to-read code. It would also run faster. This is my first post and please bear with me if I'm not posting it in the right way. Regards, Ping -- http://mail.python.org/mailman/listinfo/python-list
Re: a_list.count(a_callable) ?
> > sum(1 for i in a_list if a_callable(i)) > > -- > Carsten Haesehttp://informixdb.sourceforge.net This works nicely but not very intuitive or readable to me. First of all, the generator expression makes sense only to trained eyes. Secondly, using sum(1 ...) to mean count() isn't very intuitive either. I would still prefer an expression like a_list.count(a_callable), which is short, clean, and easy to understand. :) However, it does produce ambiguities if a_list is a list of callables. Should the count() method match values or check return values of a_callable? There are several possible designs but I'm not sure which is better. Ping -- http://mail.python.org/mailman/listinfo/python-list
Re: a_list.count(a_callable) ?
On 6 15 , 11 17 , Dustan <[EMAIL PROTECTED]> wrote: > On Jun 15, 9:15 am, Ping <[EMAIL PROTECTED]> wrote: > > > > sum(1 for i in a_list if a_callable(i)) > > > > -- > > > Carsten Haesehttp://informixdb.sourceforge.net > > > This works nicely but not very intuitive or readable to me. > > > First of all, the generator expression makes sense only to > > trained eyes. Secondly, using sum(1 ...) to mean count() > > isn't very intuitive either. > > Then wrap it in a function: > def count(a_list, a_function): >return sum(1 for i in a_list if a_function(i)) > > And call the function. You can also give it a different name (although > I can't think of a concise name that would express it any better). > Hmm... This sounds like the best idea so far. It is efficient both in memory and time while exposes an easy-to-understand name. I would name the function count_items though. n = count_items(a_list, lambda x: x > 3) # very readable :) cheers, Ping -- http://mail.python.org/mailman/listinfo/python-list
Re: a_list.count(a_callable) ?
On 6 16 , 12 33 , Carsten Haese <[EMAIL PROTECTED]> wrote: > On Fri, 2007-06-15 at 14:15 +0000, Ping wrote: > > using sum(1 ...) to mean count() isn't very intuitive > > I find it very intuitive, but then again, my years of studying Math may > have skewed my intuition. > > > I would still prefer an expression like a_list.count(a_callable), > > which is short, clean, and easy to understand. > > Did you see my alternative example on this thread? It allows you to use > list.count in almost exactly that way, except that instead of passing > the callable directly, you pass an object that defers to your callable > in its __eq__ method. > > HTH, > > -- > Carsten Haesehttp://informixdb.sourceforge.net Yes, I read it. This works, but it may make lots of dummy classes with the special __eq__ method if I have many criteria to use for counting. I find the function design mentioned by Dustan most attractive. :) cheers, Ping -- http://mail.python.org/mailman/listinfo/python-list
Re: a_list.count(a_callable) ?
On 6 16 , 2 06 , "BJörn Lindqvist" <[EMAIL PROTECTED]> wrote:
>
> Maybe you could extend count() analogous to how sort() works:
>
> # L is a list of Person objects, each Person has a name attribute
> L.sort(key = attrgetter("name"))
>
> # How many olle are there?
> print L.count("olle", key = attrgetter("name"))
>
> # And index could be extended in the same way!
> # Whom has id 1234?
> print L.index(1234, key = attrgetter("id")).name
>
> All of these could be solved by giving Person an __eq__() method, but
> it fails when you need to search, sort or count on a different key.
>
> --
> mvh Björn
Wow! This jumps out of my screen! I like it very much.
How to get the extension into the language?
cheers,
Ping
p.s. By the way, I guess you meant
print L[L.index(1234, key = attrgetter("id"))].name
in the index example.
--
http://mail.python.org/mailman/listinfo/python-list
Re: a_list.count(a_callable) ?
Somehow I did not see my post sent about 10 hours ago.
I'm posting it again. I apologize if it showed up twice.
After seeing all the ideas tossed around, now I like
the proposal made by BJörn Lindqvist the best, and I
extend it further to match the sort() method:
L.count(value, cmp=None, key=None)
With this signature, the callable case is simply
L.count(True, cmp=a_callable),
although now a_callable must return True instead
of anything logical equivalent. I can live with that.
I made an implementation with subclassing and
Carsten Haese's sum(1 ...) method, see below.
It works fine for me. It would be great to see
it supported by the built-in list. :)
cheers,
Ping
$ cat slist.py
#!/usr/bin/env python
from operator import *
class slist (list):
def count(self, value, cmp=None, key=None):
if not cmp and not key: return list.count(self, value)
if not cmp: cmp = eq
if not key: # cmp given, no key
return sum(1 for i in self if cmp(i, value))
# both cmp and key are given
return sum(1 for i in self if cmp(key(i), value))
class Person:
def __init__(self, first_name, last_name, age, gender):
self.first_name, self.last_name, self.age, self.gender
= \
first_name, last_name, age, gender
a = slist([3, 5, 7, 3])
print "a =", a
print "a has", a.count(3), "3's and", a.count(4), "4's."
print "a has", a.count(4, cmp=gt), "elements > 4 and", \
a.count(5, cmp=le), "elements <= 5."
b = slist( [ Person("John", "Smith", 30, 'm'), Person("Claire", "Doe",
23, 'f'), \
Person("John", "Black", 43, 'm'), Person("Anne", "Jolie", 50,
'f') ] )
print "b has", b.count("John", key=attrgetter("first_name")), \
"elements with first_name == John."
print "b has", b.count(25, cmp=le, key=attrgetter("age")), \
"elements with age <= 25."
$ ./slist.py
a = [3, 5, 7, 3]
a has 2 3's and 0 4's.
a has 2 elements > 4 and 3 elements <= 5.
b has 2 elements with first_name == John.
b has 2 elements with age <= 30.
--
http://mail.python.org/mailman/listinfo/python-list
Re: a_list.count(a_callable) ?
> print "b has", b.count(25, cmp=le, key=attrgetter("age")), \
> "elements with age <= 25."
> [deleted]
> b has 2 elements with age <= 30.
Oops, I mixed up when copying and pasting at different times... :p
The output was of course
b has 1 elements with age <= 25.
Ping
--
http://mail.python.org/mailman/listinfo/python-list
A patch to support L.count(value, cmp=None, key=None)
Hi,
I patched Objects/listobject.c to support
L.count(value, cmp=None, key=None).
I tested it with the same script above by replacing slist
with built-in list. It worked correctly with this small
test. The patch is below (126 lines, I hope that's not
too big to be pasted here). This is the first time that
I modified CPython source, and I may very well make
mistakes in (lack of) reference counting or other things.
Comments and corrections are much appreciated!
Regards,
Ping
--- Objects/listobject.c.orig Sun Oct 29 05:39:10 2006
+++ Objects/listobject.cTue Jun 19 01:04:30 2007
@@ -919,12 +919,12 @@
/* Comparison function. Takes care of calling a user-supplied
* comparison function (any callable Python object), which must not
be
- * NULL (use the ISLT macro if you don't know, or call
PyObject_RichCompareBool
- * with Py_LT if you know it's NULL).
- * Returns -1 on error, 1 if x < y, 0 if x >= y.
+ * NULL.
+ * Returns -9 on error, otherwise return the result of the user-
supplied
+ * comparison.
*/
static int
-islt(PyObject *x, PyObject *y, PyObject *compare)
+custom_compare(PyObject *x, PyObject *y, PyObject *compare)
{
PyObject *res;
PyObject *args;
@@ -936,7 +936,7 @@
*/
args = PyTuple_New(2);
if (args == NULL)
- return -1;
+ return -9;
Py_INCREF(x);
Py_INCREF(y);
PyTuple_SET_ITEM(args, 0, x);
@@ -944,16 +944,28 @@
res = PyObject_Call(compare, args, NULL);
Py_DECREF(args);
if (res == NULL)
- return -1;
+ return -9;
if (!PyInt_Check(res)) {
Py_DECREF(res);
PyErr_SetString(PyExc_TypeError,
"comparison function must return
int");
- return -1;
+ return -9;
}
i = PyInt_AsLong(res);
Py_DECREF(res);
- return i < 0;
+ return i;
+}
+
+/* "less-than" Comparison function. Calls custom_compare to do the
+ * actual comparison.
+ * Returns -1 on error, 1 if x < y, 0 if x >= y.
+ */
+static int
+islt(PyObject *x, PyObject *y, PyObject *compare)
+{
+ int res = custom_compare(x, y, compare);
+ if (res == -9) return -1;
+ return res < 0;
}
/* If COMPARE is NULL, calls PyObject_RichCompareBool with Py_LT,
else calls
@@ -2232,16 +2244,44 @@
}
static PyObject *
-listcount(PyListObject *self, PyObject *v)
+listcount(PyListObject *self, PyObject * args, PyObject *kwds)
{
+ PyObject *v = NULL; /* value for counting */
+ PyObject *compare = NULL;
+ PyObject *keyfunc = NULL;
+ static char *kwlist[] = {"value", "cmp", "key", 0};
+ PyObject *item;
Py_ssize_t count = 0;
Py_ssize_t i;
+ int cmp;
+
+ assert(self != NULL);
+ assert (PyList_Check(self));
+ if (args != NULL) {
+ if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|
OO:count",
+ kwlist, &v, &compare, &keyfunc))
+ return NULL;
+ }
+ if (compare == Py_None)
+ compare = NULL;
+ if (keyfunc == Py_None)
+ keyfunc = NULL;
for (i = 0; i < self->ob_size; i++) {
- int cmp = PyObject_RichCompareBool(self->ob_item[i],
v, Py_EQ);
+ item = self->ob_item[i];
+ if (keyfunc != NULL) {
+ item = PyObject_CallFunctionObjArgs(keyfunc,
item,
+ NULL);
+ }
+
+ if (compare != NULL) {
+ cmp = custom_compare(item, v, compare);
+ } else {
+ cmp = PyObject_RichCompareBool(item, v,
Py_EQ);
+ }
if (cmp > 0)
count++;
- else if (cmp < 0)
+ else if (cmp == -9)
return NULL;
}
return PyInt_FromSsize_t(count);
@@ -2404,7 +2444,7 @@
PyDoc_STRVAR(index_doc,
"L.index(value, [start, [stop]]) -> integer -- return first index of
value");
PyDoc_STRVAR(count_doc,
-"L.count(value) -> integer -- return number of occurrences of
value");
+"L.count(value, cmp=None, key=None) -> integer -- return number of
occurrences of value [in the key] [with the cmp function].");
PyDoc_STRVAR(reverse_doc,
"L.reverse() -- reverse *IN PLACE*");
PyDoc_STRVAR(sort_doc,
@@ -2422,7 +2462,7 @@
{"pop", (PyCFunction)listpop, METH_VARARGS,
pop_doc},
{"remove", (PyCFunction)listremove, METH_O, remove_doc},
{"index", (PyCFunction)listindex, METH_VARARGS,
index_doc},
- {"count", (PyCFunction)listcount, METH_O, count_doc},
+
Asynchronous HTTP client
Hi,
I'm trying to find a way to create an asynchronous HTTP client so I
can get responses from web servers in a way like
async_http_open('http://example.com/', callback_func)
# immediately continues, and callback_func is called with response
as arg when it is ready
It seems twisted can do it, but I hesitate to bring in such a big
package as a dependency because my client should be light. Asyncore
and asynchat are lighter but they don't speak HTTP. The asynchttp
project on sourceforge is a fusion between asynchat and httplib, but
it hasn't been updated since 2001 and is seriously out of sync with
httplib.
I'd appreciate it if anyone can shed some lights on this.
Thanks,
Ping
--
http://mail.python.org/mailman/listinfo/python-list
Python compiling with third party tools
I am developing an open source software for optimization purpose. We have interface to a list of solvers or solver interfaces like Openopt, Cylp, GLPK, CBC, SCip, Cplex, etc. Now the sponsor asks us to compile the python code, and wrap all the third party tools with it so that they do not need to install any of those tools or even python itself. Does anyone know how to do this? Any advice is appreciated. Ping -- https://mail.python.org/mailman/listinfo/python-list
memory control in Python
Dear All, I am working on an optimization problem, where we are trying to minimize some indicators like energy usage, energy cost, CO2 emission. In this problem, we have a bunch of energy conversion technologies for electricity and thermal purpose, such as heat pump, boiler, chiller, etc.. We are trying to model it with a one year time period. the current time step is one hour. We have preselected the candidate technologies to exclude those we don't want to study so that the problem size could be reduced with a limited candidate technologies. In the current case study, we only analyze the electric chiller and heat pump to supply the cooling load, while power grid will supply the electricity for all electric loads. There are binary variables regarding installation decisions of technologies and continuous variables related to installation capacity and hourly operational decisions. For small cases, Python works well. But if we consider longer time period. then it would fail due to the memory usage issues. We have tested several case studies to check the memory use for different time period, including 1) 2 hours in one day, 2) 24 hours in one day, 3) 20 days with 24 hours each day, as well as 4) 30 days with 24 hours each day. The first 3 cases are feasible while the last case gives out the memory error. When we are testing the forth case, the memory error comes out while creating the inequality constraints. The problem size is 1) Aeq: 12 * 26, Aineq: 30 * 26; 2) Aeq: 144*268, Aineq:316*268; 3) Aeq: 2880*5284, Aineq: 6244*5284; 4) Aeq: 4320 * 7924, Aineq is unknown due to the memory error. The solver is CPLEX (academic). It turns out that the solver is taking a lot of memory as you can see in the memory test report. for the first three cases, different memory usage is observed, and it grows up dramatically with the increase of the time period. 1) solver memory usage: 25.6 MB, 2) 19.5 MB; 3) solver memory usage: 830.0742 MB. Based on my observations, I have some of the following questions regarding 1) In order to create the optimization problem (Aeq, Aineq), how can we reduce the memory usage in python programming? 2) how can I reduce the memory usage of different solvers in Python? Why would the memory decrease for the CPLEX solver within the 24-hours-in-one-day case compared with the case 1? Thanks in advance, Ping solver memory comparison.xlsx Description: MS-Excel 2007 spreadsheet -- https://mail.python.org/mailman/listinfo/python-list
Re: memory control in Python
On Saturday, August 15, 2015 at 11:56:22 AM UTC-7, Laura Creighton wrote: > If the problem is that Python is using too much memory, then PyPy may > be able to help you. PyPy is an alternative implementation of Python, > and by defaiult uses a minimark garbage collector. > https://pypy.readthedocs.org/en/release-2.4.x/garbage_collection.html > > You will have to write your own bindings for the CPLEX C library, though, > using cffi. http://cffi.readthedocs.org/en/latest/overview.html (since > the bindings you have assume the CPython ref counting gc). > > But if your C program is itself using too much memory, then this probably > won't help. > > Discuss this more on pypy-...@python.org or the #pypy channel on freenode. > People on pypy-dev would appreciate not getting libreoffice spreadsheet > attachments but just the figures as plain text. > > Laura Hi, Laura, Thank you for the advice. I see that some people also mention that PyPy may save some speed and memory than CPython. But since I am working on an optimization problem, I wonder whether PyPy would support NumPy and Scipy. In which progress have this integration been done? -- https://mail.python.org/mailman/listinfo/python-list
Re: memory control in Python
Hi, Dieter, If I move from Python to Jython or IronPython, do I need to retool whatever I have done? If so, that may take quite a long time. This may make the reimplementation impossible. -- https://mail.python.org/mailman/listinfo/python-list
Re: memory control in Python
Hi, Oscar, Your feedback is very valuable to me since you dig into the problem itself. Basically, we are trying to develop an open source software with multiple interface to several free solvers so that we can switch among them in case one of them is not working or so efficient. The optimization problem is developed on the basis of OPENOPT. the number of binary variables grows with the change of time period. The problem size (Aeq: equality constraints, Aineq: inequality constraints)and binary variables are as listed below. As you can imagine, it is hard to test each case of binary variables separately. Aeq Aineq Binary 1 hr6*1517*15 6 2 hr12*26 30*26 10 24 hr 144*268 316*26898 480 hr 2880*5284 6244*5284 1922 720 hr 4320*7924 9364*7924 unknown Our final goal is to test 8760 time steps in our problem. We can reduce it according to the purpose of optimization. For example, for planning, the time period would be one year with longer time step than 1 hr; for operational purpose, the time period would be 7 days, with 15 min time step. In the second case, the total time step comes down to 402. Actually, we have tried several solvers, including cplex, LPSOLVE, GLPK. We don't need to stick to CPLEX for the MIP problem. The performance of each solver is compared regarding memory usage(MB). 2hr24hr240hr 480hr GLPK 26 27.6222.6806 LPSOLVE 11.6 15.5 421.61557.3 CPLEX25.6 19.5 192.06 719.1 I think one way to reduce the problem size probably would be to categorize it into planning and operation problem. Another way would be to look into Cython for which I am not quite sure whether it would help. But I would like to try to reformulate the problem itself as you advised. -- https://mail.python.org/mailman/listinfo/python-list
struct unpack issue
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi,
I am writing a small program to decode MS bitmap image. When I use statements
as follow, it works fine:
header['sig'] = str(struct.unpack('2s', self.__read(src, 2))[0])
header['len'] = int(struct.unpack('1i', self.__read(src, 4))[0])
However, when I tried to rewrite them in short:
header = struct.unpack('2s1i', self.__read(src, 6))
The Python interpreter in my Linux box came up with an error:
...
header = struct.unpack('2s1i', self.__read(src, 6))
File "/usr/lib/python2.5/struct.py", line 87, in unpack
return o.unpack(s)
struct.error: unpack requires a string argument of length 8
It was weired that the required argument length increased to 8. Any idea on
this? I am using a 32bit pentium-m and the picture file was stored in
little-edian format.
Sincerely,
- --
Ping Zhao, PhD Student Dormitory: +82-(0)31-338-9503-0415
Information Engineering [EMAIL PROTECTED]
Myongji University http://oceanio.wordpress.com
GMT +09:00 (Seoul)
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAkhTHocACgkQyOZNd2jjOpvTxgCfYnJKZqf4d3Em3xKDcAcpvK8i
tx4AoI0WvEBv5tvoTQrQlMsj/3UO16t1
=SGnL
-END PGP SIGNATURE-
--
http://mail.python.org/mailman/listinfo/python-list
cheetah.
I am having trouble using cheetah and getting it to work the way i want (mvc) I have the following file. index.py : # imports BibleController and setting. BibleController.py that includes the Bible.py model to database (those works). At the end of it i have. print Template ( file = "BibleView.Tmpl", searchList = templateValue ) BibleView.tmpl. I generated something call BibleView.py using cheetah template. But when i run BibleView.py i get this error. File "BibleView.py", line 97, in respond __v = VFFSL(SL,"ver",True) ameMapper.NotFound: cannot find 'ver' Whats wrong? _ On the road to retirement? Check out MSN Life Events for advice on how to get there! http://lifeevents.msn.com/category.aspx?cid=Retirement -- http://mail.python.org/mailman/listinfo/python-list
converting mysql db into sqlite3.
hi, i have a simple problem of opening an existing mysql database with sqlite3. Is it possible? (I have an instinct that it is), however i don';t know the most easiest and straight forward approach to do so. Thanks for the help. _ View Athletes Collections with Live Search http://sportmaps.live.com/index.html?source=hmemailtaglinenov06&FORM=MGAC01 -- http://mail.python.org/mailman/listinfo/python-list
modifying iterator value.
Hi, I have something simple i want to do. i want to modify an iterator value. for x in someList x = 1 is it possible to do that..if not how do i modify a list using iteration. _ Express yourself instantly with MSN Messenger! Download today - it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/ -- http://mail.python.org/mailman/listinfo/python-list
python rounding problem.
Hey i have a stupid question. How do i get python to print the result in only three decimal place... Example>>> round (2.9954254, 3) 2.9951 but i want to get rid of all trailing 0's..how would i do that? _ Express yourself instantly with MSN Messenger! Download today - it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/ -- http://mail.python.org/mailman/listinfo/python-list
make two tables having same orders in both column and row names
Hi, I would like to compare values in two table with same column and row names, but with different orders in column and row names. For example, table_A in a file looks like the follows: AA100 AA109 AA101 AA103 AA102 BB1 2 9 2.3 1 28 BB3 12 9 2.3 1 28 BB9 0.5 2 2.3 1 28 BB2 2 9 21 1 20 Table_B in the other file looks like the follows: AA101 AA109 AA100 AA103 AA102 BB1 2 9 2.3 2 28 BB2 2 9 2.3 1 28 BB9 2 9 2.3 1 28 BB3 2 2 2 1 28 Can anyone give an efficient way to make the two tables having same orders in column and row names so I can easily and correctly compare the values in positions? Thanks, PingHsun -- http://mail.python.org/mailman/listinfo/python-list
