Re: [HACKERS] Vitesse DB call for testing

2014-10-18 Thread CK Tan
Hi Mark, Vitesse DB won't be open-sourced, or it would have been a contrib module to postgres. We should take further discussions off this list. People should contact me directly if there is any questions. Thanks, ck...@vitessedata.com On Fri, Oct 17, 2014 at 10:55 PM, Mark Kirkwood wrote: >

Re: [HACKERS] Vitesse DB call for testing

2014-10-17 Thread Mark Kirkwood
On 18/10/14 07:13, Josh Berkus wrote: CK, Before we go any further on this, how is Vitesse currently licensed? last time we talked it was still proprietary. If it's not being open-sourced, we likely need to take discussion off this list. +1 Guys, you need to 'fess up on the licensing! Rega

Re: [HACKERS] Vitesse DB call for testing

2014-10-17 Thread CK Tan
Indeed! A big part of our implementation is based on the Neumann paper. There are also a few other papers that impacted our implemented: A. Ailamaki, D. DeWitt, M. Hill, D. Wood. DBMSs On A Modern Processor: Where Does Time Go? Peter Boncz, Marcin Zukowski, Niels Nes. MonetDB/X100: Hyper-Pipelini

Re: [HACKERS] Vitesse DB call for testing

2014-10-17 Thread David Gould
On Fri, 17 Oct 2014 13:12:27 -0400 Tom Lane wrote: > CK Tan writes: > > The bigint sum,avg,count case in the example you tried has some > > optimization. We use int128 to accumulate the bigint instead of numeric in > > pg. Hence the big speed up. Try the same query on int4 for the improvement

Re: [HACKERS] Vitesse DB call for testing

2014-10-17 Thread CK Tan
Happy to contribute to that decision :-) On Fri, Oct 17, 2014 at 11:35 AM, Tom Lane wrote: > Andres Freund writes: >> On 2014-10-17 13:12:27 -0400, Tom Lane wrote: >>> Well, that's pretty much cheating: it's too hard to disentangle what's >>> coming from JIT vs what's coming from using a differ

Re: [HACKERS] Vitesse DB call for testing

2014-10-17 Thread Tom Lane
Andres Freund writes: > On 2014-10-17 13:12:27 -0400, Tom Lane wrote: >> Well, that's pretty much cheating: it's too hard to disentangle what's >> coming from JIT vs what's coming from using a different accumulator >> datatype. If we wanted to depend on having int128 available we could >> get tha

Re: [HACKERS] Vitesse DB call for testing

2014-10-17 Thread Merlin Moncure
On Fri, Oct 17, 2014 at 1:21 PM, Peter Geoghegan wrote: > On Fri, Oct 17, 2014 at 11:08 AM, Feng Tian wrote: >> I agree using that using int128 in stock postgres will speed up things too. >> On the other hand, that is only one part of the equation. For example, if >> you look at TPCH Q1, the in

Re: [HACKERS] Vitesse DB call for testing

2014-10-17 Thread Andres Freund
On 2014-10-17 13:12:27 -0400, Tom Lane wrote: > Well, that's pretty much cheating: it's too hard to disentangle what's > coming from JIT vs what's coming from using a different accumulator > datatype. If we wanted to depend on having int128 available we could > get that speedup with a couple hours

Re: [HACKERS] Vitesse DB call for testing

2014-10-17 Thread Peter Geoghegan
On Fri, Oct 17, 2014 at 11:08 AM, Feng Tian wrote: > I agree using that using int128 in stock postgres will speed up things too. > On the other hand, that is only one part of the equation. For example, if > you look at TPCH Q1, the int128 "cheating" does not kick in at all, but we > are 8x faste

Re: [HACKERS] Vitesse DB call for testing

2014-10-17 Thread Josh Berkus
CK, Before we go any further on this, how is Vitesse currently licensed? last time we talked it was still proprietary. If it's not being open-sourced, we likely need to take discussion off this list. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailin

Fwd: [HACKERS] Vitesse DB call for testing

2014-10-17 Thread Feng Tian
Hi, Tom, Sorry for double post to you. Feng -- Forwarded message -- From: Feng Tian Date: Fri, Oct 17, 2014 at 10:29 AM Subject: Re: [HACKERS] Vitesse DB call for testing To: Tom Lane Hi, Tom, I agree using that using int128 in stock postgres will speed up things too. On

Re: [HACKERS] Vitesse DB call for testing

2014-10-17 Thread Tom Lane
CK Tan writes: > The bigint sum,avg,count case in the example you tried has some optimization. > We use int128 to accumulate the bigint instead of numeric in pg. Hence the > big speed up. Try the same query on int4 for the improvement where both pg > and vitessedb are using int4 in the executio

Re: [HACKERS] Vitesse DB call for testing

2014-10-17 Thread Merlin Moncure
On Fri, Oct 17, 2014 at 10:47 AM, CK Tan wrote: > Merlin, glad you tried it. > > We take the query plan exactly as given by the planner, decide whether to JIT > or to punt depending on the cost. If we punt, it goes back to pg executor. If > we JIT, and if we could not proceed (usually of some op

Re: [HACKERS] Vitesse DB call for testing

2014-10-17 Thread CK Tan
Merlin, glad you tried it. We take the query plan exactly as given by the planner, decide whether to JIT or to punt depending on the cost. If we punt, it goes back to pg executor. If we JIT, and if we could not proceed (usually of some operators we haven't implemented yet), we again punt. Once

Re: [HACKERS] Vitesse DB call for testing

2014-10-17 Thread Merlin Moncure
On Fri, Oct 17, 2014 at 8:14 AM, Merlin Moncure wrote: > On Fri, Oct 17, 2014 at 7:32 AM, CK Tan wrote: >> Hi everyone, >> >> Vitesse DB 9.3.5.S is Postgres 9.3.5 with a LLVM-JIT query executor >> designed for compute intensive OLAP workload. We have gotten it to a >> reasonable state and would l

Re: [HACKERS] Vitesse DB call for testing

2014-10-17 Thread Merlin Moncure
On Fri, Oct 17, 2014 at 7:32 AM, CK Tan wrote: > Hi everyone, > > Vitesse DB 9.3.5.S is Postgres 9.3.5 with a LLVM-JIT query executor > designed for compute intensive OLAP workload. We have gotten it to a > reasonable state and would like to open it up to the pg hackers > community for testing and

Re: [HACKERS] Vitesse DB call for testing

2014-10-17 Thread Andres Freund
Hi, On 2014-10-17 05:32:13 -0700, CK Tan wrote: > Vitesse DB 9.3.5.S is Postgres 9.3.5 with a LLVM-JIT query executor > designed for compute intensive OLAP workload. We have gotten it to a > reasonable state and would like to open it up to the pg hackers > community for testing and suggestions. >

[HACKERS] Vitesse DB call for testing

2014-10-17 Thread CK Tan
Hi everyone, Vitesse DB 9.3.5.S is Postgres 9.3.5 with a LLVM-JIT query executor designed for compute intensive OLAP workload. We have gotten it to a reasonable state and would like to open it up to the pg hackers community for testing and suggestions. Vitesse DB offers -- JIT Compilation for com