I've been poking at the scanner a bit using the large literal test case from the other day
http://archives.postgresql.org/pgsql-hackers/2002-04/msg00811.php I've been able to reduce the wall-clock run time of that test from 3:37 min to 2:13 min and the base_yylex() per-call time from 137ms to 24ms. I've used 'flex -8 -CFa' and restructured the code to avoid looping over and copying the input string half a dozen times. For instance, instead of scanstr(), the escape sequences are resolved as the input is scanned, and instead of the myinput() routine I use the function yy_scan_buffer() provided by flex for scanning in-memory strings. (This would make the code flex-dependent, but in reality it already is anyway.) The "before" profile was: % cumulative self self total time seconds seconds calls ms/call ms/call name 23.51 6.65 6.65 110 60.45 137.27 base_yylex 23.19 13.21 6.56 11 596.36 1089.69 pq_getstring 19.16 18.63 5.42 74882482 0.00 0.00 pq_getbyte 14.99 22.87 4.24 11 385.45 385.46 scanstr 9.61 25.59 2.72 23 118.26 118.26 yy_get_previous_state 3.78 26.66 1.07 34 31.47 31.47 myinput 3.64 27.69 1.03 22 46.82 46.82 textin 1.48 28.11 0.42 34 12.35 43.82 yy_get_next_buffer The "after" profile is: % cumulative self self total time seconds seconds calls ms/call ms/call name 40.30 5.65 5.65 11 513.64 943.64 pq_getstring 33.74 10.38 4.73 74882482 0.00 0.00 pq_getbyte 18.90 13.03 2.65 110 24.09 24.09 base_yylex 6.85 13.99 0.96 22 43.64 43.64 textin 0.07 14.00 0.01 86 0.12 0.12 heap_fetch -- Peter Eisentraut [EMAIL PROTECTED] ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
