Re: Match items in large list

2009-02-15 Thread Fisherking
Thank you all for your answers! I must say I'm really impressed by the willing to help and the quick responses in this group (since it is my first post)! I solved the problem using SQL-queries. I added a id, the same for each item in a chain (two or more similar posts) and just updated the databas

Re: Match items in large list

2009-02-12 Thread Terry Reedy
Fisherking wrote: Hi, I hope you can help me with this optimizing problem! I have a large list of dictionaries (about 250 000 of them). Two or more of these dictionaries contains the same data (not more than five or six of them per item) e.g. [{'a':1,'b':'2','c':3} , {'d': 4,'e':'5','f':6},{'a':

Re: Match items in large list

2009-02-12 Thread MRAB
Fisherking wrote: Hi, I hope you can help me with this optimizing problem! I have a large list of dictionaries (about 250 000 of them). Two or more of these dictionaries contains the same data (not more than five or six of them per item) e.g. [{'a':1,'b':'2','c':3} , {'d': 4,'e':'5','f':6},{'a':

Re: Match items in large list

2009-02-12 Thread Peter Otten
Fisherking wrote: > I hope you can help me with this optimizing problem! > I have a large list of dictionaries (about 250 000 of them). Two or > more of these dictionaries contains the same data (not more than five > or six of them per item) e.g. [{'a':1,'b':'2','c':3} , {'d': > 4,'e':'5','f':6},{

Re: Match items in large list

2009-02-12 Thread Martin
Hi, 2009/2/12 Paul Rubin : > Fisherking writes: >> Which are the best way of searching through the list and extract the >> items that are the same. hmmm how about using sqlites in memory database and let SQL take care of finding that for you? hth Martin -- http://soup.alt.delete.co.at http:/

Re: Match items in large list

2009-02-12 Thread Paul Rubin
Fisherking writes: > Which are the best way of searching through the list and extract the > items that are the same. Sort the list, then scan through looking for duplicates, which will be adjacent to each other. -- http://mail.python.org/mailman/listinfo/python-list

Match items in large list

2009-02-12 Thread Fisherking
Hi, I hope you can help me with this optimizing problem! I have a large list of dictionaries (about 250 000 of them). Two or more of these dictionaries contains the same data (not more than five or six of them per item) e.g. [{'a':1,'b':'2','c':3} , {'d': 4,'e':'5','f':6},{'a':1,'b':'2','c':3} , {