Astan Chee wrote:
> Sorry about that, here is a summary of my complete code. I haven't cleaned
> it up much or anything, but this is what it does:
>
> import time
> import multiprocessing
>
> test_constx =0
> test_consty =0
>
> def functionTester(x):
> global test_constx
You don't need to declare a variable as global unless you want to rebind
(assign to) it.
> global test_consty
> print "constx " + str(test_constx)
> print "consty " + str(test_consty)
> return (test_constx*x[0]-x[1]+test_consty*x[0]+x[2])
>
> def functionTesterMain(constx,consty):
> global test_constx
> global test_consty
> test_constx = constx
> test_consty = consty
> num_args =
> [(61,12,1),(61,12,2),(61,12,3),(61,11,4),(61,12,4),(62,33,4),(7,12,4),
(16,19,4),(35,36,4),(37,38,3),(55,56,3),(57,63,3)]
> num_processes = multiprocessing.cpu_count()
> pool = multiprocessing.Pool(num_processes)
I think you need to create the pool outside the function; in the current
configuration you get three not one Pool instance.
> rs = []
> start = time.time()
> rs = pool.map(functionTester,num_args)
> end = time.time()
> elapsed= end - start
> min = elapsed/60
> print "Took", elapsed, "seconds to run, which is the same as", min,
> "minutes"
> pos = 0
> high = 0
> n = None
> for r in rs:
> if r > high:
> n = num_args[pos]
> high = r
> pos+=1
> print "high " + str(high)
> print "n " + str(n)
>
> return high,n
>
> if __name__ == '__main__':
> for i in range(1,4):
> a,b = functionTesterMain(i,7)
> print "-----------"
> print "a " + str(a)
> print "b " + str(a)
>
>
> Which doesn't seem to work because the functionTester() needs to be
> simpler and not use global variables.
> I'm using global variables because I'm also trying to pass a few other
> variables and I tried using a class but that just gave me a unpickleable
> error. I tried using zip but I'm confused with how I can get to pass the
> data.
A simple approach would be to pass an index into a list
const_data = zip(range(1, 4), [7]*3)
> I know I can probably combine the data into tuples but that means that
> there is alot of data duplication, especially if the constx and consty are
> large dictionaries (or even custom objects), which might happen later.
> So it seems that map doesn't quite like functions like these. Anyway, I'll
> try and see if threads or something can substitute. I'd appriciate any
help. Thanks
Your code, slightly modified and cleaned up (yes, four-space indent improves
readability):
import time
import multiprocessing
const_data = zip(range(1, 4), [7]*3)
num_args = [(61, 12, 1), (61, 12, 2), (61, 12, 3), (61, 11, 4),
(61, 12, 4), (62, 33, 4), (7, 12, 4), (16, 19, 4),
(35, 36, 4), (37, 38, 3), (55, 56, 3), (57, 63, 3)]
def functionTester(args):
i, x, y, z = args
constx, consty = const_data[i]
print "constx", constx
print "consty", consty
return constx*x - y + consty*x + z
def functionTesterMain(pool, index):
start = time.time()
rs = pool.map(functionTester, (((index, ) + x) for x in num_args))
end = time.time()
elapsed = end - start
min = elapsed/60
print "Took", elapsed,
print "seconds to run, which is the same as", min, "minutes"
return max(zip(rs, num_args))
if __name__ == '__main__':
num_processes = multiprocessing.cpu_count()
pool = multiprocessing.Pool(num_processes)
for i, _ in enumerate(const_data):
a, b = functionTesterMain(pool, i)
print "-----------"
print "a", a
print "b", b
--
http://mail.python.org/mailman/listinfo/python-list
