Hi Brandon, I've tested variants of the mysql version with both _use_ and _store_ with no appreciable difference.
the dbi version appears heavily CPU-bound, with minimal memory usage (my trivial test cases don't actually -keep- the data anywhere). USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 27941 90.4 0.1 42556 17784 pts/2 S+ 23:55 0:16 ./dbi -Adam On Jul 3, 2011, at 10:47 PM, Brandon Phelps wrote: > Just a wild guess here but have you checked memory usage during the > job? Maybe it has something to do with libdbi using mysql_store_result > for such a large number of rows? Try switching to mysql_use_result and > see if the problem persists? > > Just a thought, could be way off. > > On 7/3/2011 10:14 PM, Adam Jacob Muller wrote: >> Hi, >> I'm curious if anyone knows of any specific issues with regards to libdbi >> and MySQL with RRDTool. I have a specific dataset where I pull a significant >> number of rows from MySQL to draw graphs (think, in the 100,000's of rows). >> And its extremely slow. >> >> I've specifically isolated this not to query execution time but to libdbi. >> >> to completely isolate the issue, this is a small program I wrote that uses >> libdbi to read a months worth of samples (~2*60*24*30 = 86400 rows): >> >> # ./dbi >> dbi_conn_queryf took 0.5319 >> dbi_result_next_row took 15.0172 >> looped through 71891 rows >> # >> >> >> This is the same thing, but using the native mysql c-bindings: >> # ./mysql >> mysql_query took 0.0021 >> mysql_fetch_row took 0.5352 >> looped through 71891 rows >> # >> >> >> >> This probably ultimately seems like a libdbi issue, but I thought i'd bring >> it up here because it seems to have an extraordinary impact upon rrdtool >> performance and i'm curious if anyone here has seen it. >> >> >> -Adam >> _______________________________________________ >> rrd-users mailing list >> [email protected] >> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users > > _______________________________________________ > rrd-users mailing list > [email protected] > https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users _______________________________________________ rrd-users mailing list [email protected] https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
