Hello,
I'm experiencing really strange problem. I wrote data into cassandra
cluster. I'm trying to check if data inserted then fetched are equally to
source data (file). Code below is the task for celery that does
the comparison with sha1(). The problem is that celery worker returning
since time to time during the comparison output like that:
2011-04-14 17:24:33,225: INFO/PoolWorker-134]
tasks.insertData[f377efdb-33a2-48f4-ab00-52b1898e216c]: [Error/EmailTest]
Email corrupted.]
If i execute the task code manually the output is correct ,[Email data test:
OK].
I thought that possible bug is in multi threading but i start celery workers
with only one thread to remove that case.
Another problem that is occurring often is :
[2011-04-14 12:46:49,682: INFO/PoolWorker-1] Connection 17810000 (IP:9160)
in ConnectionPool (id = 15612176) failed: timed out
[2011-04-14 12:46:49,844: INFO/PoolWorker-1] Connection 17810000 (IP:9160)
in ConnectionPool (id = 15612176) failed: UnavailableException()
I'm using pycassa connection pooling with parameters pool_size=15 (5*
number of nodes), max_retries=30, max_overflow=5, timeout=4
Any ideas where should be problems? The client is pycassa 1.0.8, and I tried
it with 1.0.6 too.
Thanks
Best,
Peter
#######
@task(ignore_result=True)
def checkData(key):
logger = insertData.get_logger()
logger.info("Reading email %s" % key)
logger.info("Task id %s" % checkData.request.id)
f = open(key, 'r')
sEmail = f.readlines()
f.close()
m = hashlib.sha1()
m.update(''.join(sEmail))
sHash = m.hexdigest()
#fetch email from DB
email = rawEmail(key)
m = hashlib.sha1()
m.update(email)
dHash = m.hexdigest()
if sHash != dHash:
logger.info("[Error/EmailTest] Email corrupted.] < %s >" % key)
else:
logger.info("[Email data test: OK]")