Hi , On 2016-10-28 14:47:35 +0300, Dmitry Melnik wrote: > We'd like to present our work on adding LLVM JIT compilation of expressions > in SQL queries for PostgreSQL.
Great! I'm also working in the area, albeit with a, I think, a bit different approach[1]. Is your goal to integrate this work into postgres proper, or is this more an academic research project? If the former, lets try to collaborate. If the latter, lets talk, so we're not integrating something dumb ;) [1]. So far I've basically converted expression evaluation, and tuple deforming, into small interpreted (switch/computed goto opcode dispatch) mini-languages, which then can be JITed. Adding a small handwritten x86-64 JIT (out of fun, not because I think that's a good approach) also resulted in quite noticeable speedups. Did you experiment with JITing tuple deforming as well? The reason I was thinking of going in this direction, is that it's a lot faster to compile such mini pieces of code, and it already gives significant speedups. There still are function calls to postgres functions, but they're all direct function calls, instead of indirect ones. > Currently, our JIT is used to compile expressions in every query, so for > short-running queries JIT compilation may take longer than query execution > itself. We plan to address it by using planner estimate to decide whether > it worth JIT compiling, also we may try parallel JIT compilation. But for > now we recommend testing it on a large workload in order to pay off the > compilation (we've tested on 40GB database for Q1). Could you give some estimates about how long JITing takes for you, say for Q1? Different approaches here obviously have very different tradeoffs. > This work is a part of our greater effort on implementing full JIT compiler > in PostgreSQL, where along with JITting expressions we've changed the > iteration model from Volcano-style to push-model and reimplemented code > generation with LLVM for most of Scan/Aggregation/Join methods. That > approach gives much better speedup (x4-5 times on Q1), but it takes many > code changes, so we're developing it as PostgreSQL extension. FWIW, I think long term, we're going to want something like that in core. I'm currently working on a "gradual" transformation towards that, by *optionally* dealing with "batches" of tuples which get passed around. Noticeable speedups, but not in the 4-5x range. > Also we're going to give a lightning talk at upcoming PGConf.EU in Tallinn, > and discuss the further development with PostgreSQL community. We'd > appreciate any feedback! Cool, lets chat a bit, I'm also here. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
