EBS but with new generation SSD not magnetic one.
On Sat, Sep 6, 2014 at 12:11 PM, Alex Kamil <[email protected]> wrote: > do you use EBS or ephemeral storage, I found EBS performance to be > somewhat unpredictable > > > On Sat, Sep 6, 2014 at 2:37 AM, Vikas Agarwal <[email protected]> > wrote: > >> Hbase is 0.98.0 >> Phoenix is 4.0 >> >> >> On Sat, Sep 6, 2014 at 12:04 PM, Vikas Agarwal <[email protected]> >> wrote: >> >>> Yes, that is why it is a trouble for me. However, on contrary, HBase >>> shell is also on the same machine and same environment, so if it is an >>> issue of resource (CPU or memory) it should have affected the HBase too, >>> but HBase is able to give me results within 0.0150 seconds. :( >>> >>> No, I haven't tested it outside AWS. I guess, it should not be the case >>> due to much better performance by native HBase query on HBase shell. >>> >>> >>> On Sat, Sep 6, 2014 at 11:59 AM, James Taylor <[email protected]> >>> wrote: >>> >>>> Something is up in your environment. What version of Phoenix and HBase >>>> are you using and in what environment? Have you tried this locally, >>>> outside of AWS to compare? >>>> >>>> Take a look at our perf numbers, generated more-or-less daily, and >>>> which run over more data that what you're testing against: >>>> >>>> http://phoenix-bin.github.io/client/performance/phoenix-20140904095313.htm >>>> >>>> Some of these are point queries and they take in the neighborhood of >>>> 0.01 seconds. >>>> >>>> Thanks, >>>> James >>>> >>>> On Fri, Sep 5, 2014 at 10:48 PM, Vikas Agarwal <[email protected]> >>>> wrote: >>>> > Missed to mention that count query (posted in my last mail) is also >>>> taking >>>> > very long time to return the count. >>>> > >>>> > >>>> > On Sat, Sep 6, 2014 at 11:17 AM, Vikas Agarwal <[email protected] >>>> > >>>> > wrote: >>>> >> >>>> >> As I mentioned, schema is nothing but bunch of fields (some being >>>> >> integers, longs and text) along with primary key (row key) and I am >>>> making >>>> >> simple query to get result for a particular primary key, nothing >>>> more than >>>> >> that. >>>> >> >>>> >> 0: jdbc:phoenix:localhost> SELECT count(1) FROM table_name; >>>> >> >>>> >> +------------+ >>>> >> >>>> >> | COUNT(1) | >>>> >> >>>> >> +------------+ >>>> >> >>>> >> | 4667515 | >>>> >> >>>> >> +------------+ >>>> >> >>>> >> 1 row selected (132.11 seconds) >>>> >> >>>> >> >>>> >> >>>> >> On Sat, Sep 6, 2014 at 11:09 AM, Puneet Kumar Ojha >>>> >> <[email protected]> wrote: >>>> >>> >>>> >>> If you can share the schema,data type,cardinality of each dimension >>>> and >>>> >>> usual queries, I can help to design a schema with performance of >>>> less than 1 >>>> >>> sec using Phoenix. >>>> >>> >>>> >>> >>>> >>> >>>> >>> Thanks >>>> >>> >>>> >>> >>>> >>> >>>> >>> >>>> >>> >>>> >>> ------ Original message------ >>>> >>> >>>> >>> From: James Taylor >>>> >>> >>>> >>> Date: Sat, Sep 6, 2014 10:15 AM >>>> >>> >>>> >>> To: user; >>>> >>> >>>> >>> Subject:Re: Phoenix response time >>>> >>> >>>> >>> >>>> >>> >>>> >>> Vikas, >>>> >>> Please post your schema and query. >>>> >>> Thanks, >>>> >>> James >>>> >>> >>>> >>> On Fri, Sep 5, 2014 at 9:18 PM, Vikas Agarwal < >>>> [email protected]> >>>> >>> wrote: >>>> >>> > Ours is also a single node setup right now and as of now there >>>> are less >>>> >>> > than >>>> >>> > 1 million rows which is expected to grow around 100m at minimum. >>>> >>> > >>>> >>> > I am aware of secondary indexes but when I am querying on >>>> primary/row >>>> >>> > key, >>>> >>> > why would it take so much time? >>>> >>> > >>>> >>> > I am directly querying using sqlline for Phoenix and hbase shell >>>> for >>>> >>> > HBase >>>> >>> > query. I am not expecting to do any fine tuning for such small >>>> dataset. >>>> >>> > I am >>>> >>> > assumimg a minimum performance level out of the box. >>>> >>> > >>>> >>> > On Friday, September 5, 2014, yeshwanth kumar < >>>> [email protected]> >>>> >>> > wrote: >>>> >>> >> >>>> >>> >> hi vikas, >>>> >>> >> >>>> >>> >> we used phoenix on a 4 core/23Gb machine, as a single node setup. >>>> >>> >> used HDP 2.1 >>>> >>> >> our table has 50-70M rows, >>>> >>> >> select on that table took less than 2 seconds. >>>> >>> >> Aggregation queries took less than 8 seconds. >>>> >>> >> for achieving good performance we created secondary index on the >>>> >>> >> table. >>>> >>> >> >>>> >>> >> make sure you finetuned hbase, >>>> >>> >> enabling compression on the data makes a difference in response. >>>> >>> >> if u distribute the data and load over all regions in hbase, >>>> >>> >> look at the performance tips mentioned in phoenix blog >>>> >>> >> >>>> >>> >> -yeshwanth >>>> >>> >> >>>> >>> >> >>>> >>> >> >>>> >>> >> Cheers, >>>> >>> >> Yeshwanth >>>> >>> >> >>>> >>> >> >>>> >>> >> >>>> >>> >> On Fri, Sep 5, 2014 at 5:42 PM, Vikas Agarwal < >>>> [email protected]> >>>> >>> >> wrote: >>>> >>> >>> >>>> >>> >>> Hi, >>>> >>> >>> >>>> >>> >>> Preface: We are testing phoenix using Hortonworks distribution >>>> for >>>> >>> >>> HBase >>>> >>> >>> on Amazon EC2 instance (r3.large, 2 CPU/15 GB RAM). >>>> >>> >>> >>>> >>> >>> With contrast to performance benchmarks, I found Phoenix to be >>>> very >>>> >>> >>> slow >>>> >>> >>> in querying even on primary key or row key. So, tried to >>>> increase the >>>> >>> >>> RAM >>>> >>> >>> for HBase and Phoenix and increasing the CPU and RAM by >>>> upgrading the >>>> >>> >>> EC2 >>>> >>> >>> machine type to r3.xlarge (4 CPU, 30 GB RAM). Results were like >>>> this: >>>> >>> >>> >>>> >>> >>> Time takes in returning result of query on row key: >>>> >>> >>> With Storm running and very less RAM available: 50 sec >>>> >>> >>> >>>> >>> >>> With Storm stopped and RAM available to Phoenix and HBase: 18 >>>> sec >>>> >>> >>> >>>> >>> >>> With new machine of next higher category (4 CPU and 30 GB RAM): >>>> 8 sec >>>> >>> >>> >>>> >>> >>> Pure HBase query by row key with Storm stopped and (2 CPU, 15 GB >>>> >>> >>> RAM): >>>> >>> >>> 0.0150 seconds. :) >>>> >>> >>> >>>> >>> >>> So, the difference seems to be many fold of what native HBase is >>>> >>> >>> providing to us. I am not able to understand how it can be >>>> possible? >>>> >>> >>> What I >>>> >>> >>> am missing here? >>>> >>> >>> >>>> >>> >>> -- >>>> >>> >>> Regards, >>>> >>> >>> Vikas Agarwal >>>> >>> >>> 91 – 9928301411 >>>> >>> >>> >>>> >>> >>> InfoObjects, Inc. >>>> >>> >>> Execution Matters >>>> >>> >>> http://www.infoobjects.com >>>> >>> >>> 2041 Mission College Boulevard, #280 >>>> >>> >>> Santa Clara, CA 95054 >>>> >>> >>> +1 (408) 988-2000 Work >>>> >>> >>> +1 (408) 716-2726 Fax >>>> >>> >> >>>> >>> >> >>>> >>> > >>>> >>> > >>>> >>> > -- >>>> >>> > Regards, >>>> >>> > Vikas Agarwal >>>> >>> > 91 – 9928301411 >>>> >>> > >>>> >>> > InfoObjects, Inc. >>>> >>> > Execution Matters >>>> >>> > http://www.infoobjects.com >>>> >>> > 2041 Mission College Boulevard, #280 >>>> >>> > Santa Clara, CA 95054 >>>> >>> > +1 (408) 988-2000 Work >>>> >>> > +1 (408) 716-2726 Fax >>>> >>> > >>>> >>> > >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> -- >>>> >> Regards, >>>> >> Vikas Agarwal >>>> >> 91 – 9928301411 >>>> >> >>>> >> InfoObjects, Inc. >>>> >> Execution Matters >>>> >> http://www.infoobjects.com >>>> >> 2041 Mission College Boulevard, #280 >>>> >> Santa Clara, CA 95054 >>>> >> +1 (408) 988-2000 Work >>>> >> +1 (408) 716-2726 Fax >>>> > >>>> > >>>> > >>>> > >>>> > -- >>>> > Regards, >>>> > Vikas Agarwal >>>> > 91 – 9928301411 >>>> > >>>> > InfoObjects, Inc. >>>> > Execution Matters >>>> > http://www.infoobjects.com >>>> > 2041 Mission College Boulevard, #280 >>>> > Santa Clara, CA 95054 >>>> > +1 (408) 988-2000 Work >>>> > +1 (408) 716-2726 Fax >>>> >>> >>> >>> >>> -- >>> Regards, >>> Vikas Agarwal >>> 91 – 9928301411 >>> >>> InfoObjects, Inc. >>> Execution Matters >>> http://www.infoobjects.com >>> 2041 Mission College Boulevard, #280 >>> Santa Clara, CA 95054 >>> +1 (408) 988-2000 Work >>> +1 (408) 716-2726 Fax >>> >>> >> >> >> -- >> Regards, >> Vikas Agarwal >> 91 – 9928301411 >> >> InfoObjects, Inc. >> Execution Matters >> http://www.infoobjects.com >> 2041 Mission College Boulevard, #280 >> Santa Clara, CA 95054 >> +1 (408) 988-2000 Work >> +1 (408) 716-2726 Fax >> >> > -- Regards, Vikas Agarwal 91 – 9928301411 InfoObjects, Inc. Execution Matters http://www.infoobjects.com 2041 Mission College Boulevard, #280 Santa Clara, CA 95054 +1 (408) 988-2000 Work +1 (408) 716-2726 Fax
