Hello, I am finding that when I start my scheduler.py workers, they steadily consume more and more memory at a rate of about 100 KB every 3 seconds. After a day or so, the system grinds as all the memory is consumed. I suspected the MySQL driver, but I have verified that the leak occurs with both the pymysql and MySQLdb libaries. The leak seems slower when using pymysql but I haven't investigated thourougly.
One thing that's different in my application is I use the scheduler workers to do intensive computations on a remote server and then return results. To decrease the latency, I have modified the worker main loop to run every 100 ms. My guess is that one possible reason a memory leak hasn't been reported before is that the scheduler defaults to a much slower heartbeat. NOTE: I see the memory leak even when the workers are not doing anything other than looping, so my computational code can be ruled out. I have made a simple scheduler that has none of my tasks, and it still displays the same memory leak. Before I work on trying to demonstrate the problem with a standalone example, does anyone have any suggestions of things to check? I am surprised to find a memory leak since when using pymysql, everything is in python. Thanks, G
