On 2010-01-11 17:50 PM, wiso wrote:
The problem now is this: start reading file r1_200909.log start reading file r1_200910.log readen 488832 lines from file r1_200910.log readen 517247 lines from file r1_200909.logwith huge file (the real case) the program freeze. Is there a solution to avoid pickling/serialization, ... for example something like this: if __name__ == "__main__": file_names = ["r1_200909.log", "r1_200910.log"] pool = multiprocessing.Pool(len(file_names)) childrens = [Container(f) for f in file_names] pool.map(lambda c: c.read(), childrens) PicklingError: Can't pickle<type 'function'>: attribute lookup __builtin__.function failed
You can't pickle lambda functions. What information do you actually need back from the workers? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
