Is there any way to using appropriate index automatically?
Yeah, I mean no hint , use appropriate index automatically. I create a local index and a query with corresponding index column filter in where clause. But the query doesn’t use index, with index hint it uses it.
Re: Is there any way to using appropriate index automatically?
Phoenix-version: 4.14.3-HBase-1.4-SNAPSHOT hbase-version: 1.4.6 Table: CREATE TABLE test_phoenix.app ( dt integer not null, a bigint not null , b bigint not null , c bigint not null , d bigint not null , e bigint not null , f bigint not null , g bigint not null , h bigint not null , i bigint not null , j bigint not null , k bigint not null , m decimal(30,6) , n decimal(30,6) CONSTRAINT pk PRIMARY KEY (dt, a,b,c,d,e,f,g,h,i,j,k) ) SALT_BUCKETS = 3,UPDATE_CACHE_FREQUENCY = 30; Index: CREATE local INDEX local_c_h_index ON test_phoenix.app (c,h) ASYNC; (Has been filled data with bulkload and index is active) Query: select /*+ INDEX(test_phoenix.app local_c_h_index) */ * from TEST_PHOENIX.APP where c=2 and h = 1 limit 5; select * from TEST_PHOENIX.APP where c=2 and h = 1 limit 5; The first query will use index local_c_h_index and result shortly, the second query won’t , and response slowly. The explain plan is weird, all showing without using index. > On Aug 19, 2019, at 5:54 PM, Aleksandr Saraseka > wrote: > > We have no problems with that. I mean indexes are used even without hints, if > they're suitable for a query. > Maybe you can share your Phoenix version, query, index definition and exec > plan ? > > On Mon, Aug 19, 2019 at 12:46 PM you Zhuang <mailto:zhuangzixiao...@gmail.com>> wrote: > Yeah, I mean no hint , use appropriate index automatically. I create a local > index and a query with corresponding index column filter in where clause. > But the query doesn’t use index, with index hint it uses it. > > > -- > Aleksandr Saraseka > DBA > 380997600401 > • asaras...@eztexting.com > <mailto:asaras...@eztexting.com> • eztexting.com > <http://eztexting.com/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> > > > <http://facebook.com/eztexting?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> > > <http://linkedin.com/company/eztexting/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> > > <http://twitter.com/eztexting?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> > > <https://www.youtube.com/eztexting?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> > > <https://www.instagram.com/ez_texting/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> > > <https://www.facebook.com/alex.saraseka?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> > > <https://www.linkedin.com/in/alexander-saraseka-32616076/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
Re: Is there any way to using appropriate index automatically?
You are right, thanks so much, but it’s so limited that I can’t include all columns due to hundreds of columns existing in one table. I think covered columns must be queried in global indexes is reasonable, local index isn’t. Because we query rowkey from local index first , then get actual row from data table. Thus we have no necessity to limit local index usage including all queried columns. > On Aug 20, 2019, at 12:32 AM, Josh Elser wrote: > > http://phoenix.apache.org/faq.html#Why_isnt_my_secondary_index_being_used > <http://phoenix.apache.org/faq.html#Why_isnt_my_secondary_index_being_used> > > On 8/19/19 6:06 AM, you Zhuang wrote: >> Phoenix-version: 4.14.3-HBase-1.4-SNAPSHOT >> hbase-version: 1.4.6 >> Table: >> CREATE TABLE test_phoenix.app ( >> dt integer not null, >> a bigint not null , >> b bigint not null , >> c bigint not null , >> d bigint not null , >> e bigint not null , >> f bigint not null , >> g bigint not null , >> h bigint not null , >> i bigint not null , >> j bigint not null , >> k bigint not null , >> m decimal(30,6) , >> n decimal(30,6) >> CONSTRAINT pk PRIMARY KEY (dt, a,b,c,d,e,f,g,h,i,j,k) >> ) SALT_BUCKETS = 3,UPDATE_CACHE_FREQUENCY = 30; >> Index: >> CREATE local INDEX local_c_h_index ON test_phoenix.app (c,h) ASYNC; >> (Has been filled data with bulkload and index is active) >> Query: >> select /*+ INDEX(test_phoenix.app local_c_h_index) */ * from >> TEST_PHOENIX.APP where c=2 and h = 1 limit 5; >> select * from TEST_PHOENIX.APP where c=2 and h = 1 limit 5; >> The first query will use index local_c_h_index and result shortly, the >> second query won’t , and response slowly. >> The explain plan is weird, all showing without using index. >>> On Aug 19, 2019, at 5:54 PM, Aleksandr Saraseka >> <mailto:asaras...@eztexting.com> <mailto:asaras...@eztexting.com >>> <mailto:asaras...@eztexting.com>>> wrote: >>> >>> We have no problems with that. I mean indexes are used even without hints, >>> if they're suitable for a query. >>> Maybe you can share your Phoenix version, query, index definition and exec >>> plan ? >>> >>> On Mon, Aug 19, 2019 at 12:46 PM you Zhuang >> <mailto:zhuangzixiao...@gmail.com> <mailto:zhuangzixiao...@gmail.com >>> <mailto:zhuangzixiao...@gmail.com>>> wrote: >>> >>>Yeah, I mean no hint , use appropriate index automatically. I >>>create a local index and a query with corresponding index column >>>filter in where clause. But the query doesn’t use index, with >>>index hint it uses it. >>> >>> >>> >>> -- >>> Aleksandr Saraseka >>> DBA >>> 380997600401 >>> > *•* asaras...@eztexting.com >>> <mailto:asaras...@eztexting.com> <mailto:asaras...@eztexting.com >>> <mailto:asaras...@eztexting.com>> *•* eztexting.com >>> <http://eztexting.com/><http://eztexting.com/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>> >>> <http://eztexting.com/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>> >>> >>> >>> <http://facebook.com/eztexting?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>> >>> <http://facebook.com/eztexting?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>> >>> >>> <http://linkedin.com/company/eztexting/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>> >>> <http://linkedin.com/company/eztexting/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>> >>> >>> <http://twitter.com/eztexting?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>> >>> <http://twitter.com/eztexting?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>> >>> >>> <https://www.youtube.com/eztexting?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>> >>> <https://www.youtube.com/eztexting?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>> >>> >>> <https://www.instagram.com/ez_texting/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>> >>> <https://www.instagram.com/ez_texting/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>> >>> >>> <https://www.facebook.com/alex.saraseka?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>> >>> <https://www.facebook.com/alex.saraseka?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>> >>> >>> <https://www.linkedin.com/in/alexander-saraseka-32616076/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>> >>> <https://www.linkedin.com/in/alexander-saraseka-32616076/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>>
Re: Is there any way to using appropriate index automatically?
Er, I also read the sentence “Unlike global indexes, local indexes will use an index even when all columns referenced in the query are not contained in the index. This is done by default for local indexes because we know that the table and index data co-reside on the same region server thus ensuring the lookup is local.” I ‘m totally confused. > On Aug 20, 2019, at 12:32 AM, Josh Elser wrote: > > http://phoenix.apache.org/faq.html#Why_isnt_my_secondary_index_being_used > <http://phoenix.apache.org/faq.html#Why_isnt_my_secondary_index_being_used> > > On 8/19/19 6:06 AM, you Zhuang wrote: >> Phoenix-version: 4.14.3-HBase-1.4-SNAPSHOT >> hbase-version: 1.4.6 >> Table: >> CREATE TABLE test_phoenix.app ( >> dt integer not null, >> a bigint not null , >> b bigint not null , >> c bigint not null , >> d bigint not null , >> e bigint not null , >> f bigint not null , >> g bigint not null , >> h bigint not null , >> i bigint not null , >> j bigint not null , >> k bigint not null , >> m decimal(30,6) , >> n decimal(30,6) >> CONSTRAINT pk PRIMARY KEY (dt, a,b,c,d,e,f,g,h,i,j,k) >> ) SALT_BUCKETS = 3,UPDATE_CACHE_FREQUENCY = 30; >> Index: >> CREATE local INDEX local_c_h_index ON test_phoenix.app (c,h) ASYNC; >> (Has been filled data with bulkload and index is active) >> Query: >> select /*+ INDEX(test_phoenix.app local_c_h_index) */ * from >> TEST_PHOENIX.APP where c=2 and h = 1 limit 5; >> select * from TEST_PHOENIX.APP where c=2 and h = 1 limit 5; >> The first query will use index local_c_h_index and result shortly, the >> second query won’t , and response slowly. >> The explain plan is weird, all showing without using index. >>> On Aug 19, 2019, at 5:54 PM, Aleksandr Saraseka >> <mailto:asaras...@eztexting.com> <mailto:asaras...@eztexting.com >>> <mailto:asaras...@eztexting.com>>> wrote: >>> >>> We have no problems with that. I mean indexes are used even without hints, >>> if they're suitable for a query. >>> Maybe you can share your Phoenix version, query, index definition and exec >>> plan ? >>> >>> On Mon, Aug 19, 2019 at 12:46 PM you Zhuang >> <mailto:zhuangzixiao...@gmail.com> <mailto:zhuangzixiao...@gmail.com >>> <mailto:zhuangzixiao...@gmail.com>>> wrote: >>> >>>Yeah, I mean no hint , use appropriate index automatically. I >>>create a local index and a query with corresponding index column >>>filter in where clause. But the query doesn’t use index, with >>>index hint it uses it. >>> >>> >>> >>> -- >>> Aleksandr Saraseka >>> DBA >>> 380997600401 >>> > *•* asaras...@eztexting.com >>> <mailto:asaras...@eztexting.com> <mailto:asaras...@eztexting.com >>> <mailto:asaras...@eztexting.com>> *•* eztexting.com >>> <http://eztexting.com/><http://eztexting.com/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>> >>> <http://eztexting.com/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>> >>> >>> >>> <http://facebook.com/eztexting?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>> >>> <http://facebook.com/eztexting?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>> >>> >>> <http://linkedin.com/company/eztexting/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>> >>> <http://linkedin.com/company/eztexting/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>> >>> >>> <http://twitter.com/eztexting?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>> >>> <http://twitter.com/eztexting?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>> >>> >>> <https://www.youtube.com/eztexting?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>> >>> <https://www.youtube.com/eztexting?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>> >>> >>> <https://www.instagram.com/ez_texting/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>> >>> <https://www.instagram.com/ez_texting/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>> >>> >>> <https://www.facebook.com/alex.saraseka?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>> >>> <https://www.facebook.com/alex.saraseka?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>> >>> >>> <https://www.linkedin.com/in/alexander-saraseka-32616076/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>> >>> <https://www.linkedin.com/in/alexander-saraseka-32616076/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>>
Re: Is there any way to using appropriate index automatically?
No, the index can’t be used . > On Aug 21, 2019, at 2:38 AM, Ankit Singhal <mailto:ankitsingha...@gmail.com>> wrote: > > CREATE local INDEX local_c_h_index ON test_phoenix.app (c,h) ASYNC; > (Has been filled data with bulkload and index is active) > > Query: > select /*+ INDEX(test_phoenix.app local_c_h_index) */ * from > TEST_PHOENIX.APP where c=2 and h = 1 limit 5; > select * from TEST_PHOENIX.APP where c=2 and h = 1 limit 5; > > The first query will use index local_c_h_index and result shortly, the second > query won’t , and response slowly. > Yeah, the local index should be used in both the cases, looks like a bug to > me, can you please raise a JIRA in Phoenix project for the same. > QueryOptimizer.java may have a relevant code to fix the issue, so the patch > would really be appreciated. > > And, also can you try running "select a,b,c,d,e,f,g,h,i,j,k,m,n from > TEST_PHOENIX.APP where c=2 and h = 1 limit 5", and see if index is getting > used. > > Regards, > Ankit Singhal > > On Tue, Aug 20, 2019 at 1:49 AM you Zhuang <mailto:zhuangzixiao...@gmail.com>> wrote: > Er, I also read the sentence “Unlike global indexes, local indexes will use > an index even when all columns referenced in the query are not contained in > the index. This is done by default for local indexes because we know that the > table and index data co-reside on the same region server thus ensuring the > lookup is local.” > > I ‘m totally confused. > > >> On Aug 20, 2019, at 12:32 AM, Josh Elser > <mailto:els...@apache.org>> wrote: >> >> http://phoenix.apache.org/faq.html#Why_isnt_my_secondary_index_being_used >> <http://phoenix.apache.org/faq.html#Why_isnt_my_secondary_index_being_used> >> >> On 8/19/19 6:06 AM, you Zhuang wrote: >>> Phoenix-version: 4.14.3-HBase-1.4-SNAPSHOT >>> hbase-version: 1.4.6 >>> Table: >>> CREATE TABLE test_phoenix.app ( >>> dt integer not null, >>> a bigint not null , >>> b bigint not null , >>> c bigint not null , >>> d bigint not null , >>> e bigint not null , >>> f bigint not null , >>> g bigint not null , >>> h bigint not null , >>> i bigint not null , >>> j bigint not null , >>> k bigint not null , >>> m decimal(30,6) , >>> n decimal(30,6) >>> CONSTRAINT pk PRIMARY KEY (dt, a,b,c,d,e,f,g,h,i,j,k) >>> ) SALT_BUCKETS = 3,UPDATE_CACHE_FREQUENCY = 30; >>> Index: >>> CREATE local INDEX local_c_h_index ON test_phoenix.app (c,h) ASYNC; >>> (Has been filled data with bulkload and index is active) >>> Query: >>> select /*+ INDEX(test_phoenix.app local_c_h_index) */ * from >>> TEST_PHOENIX.APP where c=2 and h = 1 limit 5; >>> select * from TEST_PHOENIX.APP where c=2 and h = 1 limit 5; >>> The first query will use index local_c_h_index and result shortly, the >>> second query won’t , and response slowly. >>> The explain plan is weird, all showing without using index. >>>> On Aug 19, 2019, at 5:54 PM, Aleksandr Saraseka >>> <mailto:asaras...@eztexting.com> <mailto:asaras...@eztexting.com >>>> <mailto:asaras...@eztexting.com>>> wrote: >>>> >>>> We have no problems with that. I mean indexes are used even without hints, >>>> if they're suitable for a query. >>>> Maybe you can share your Phoenix version, query, index definition and exec >>>> plan ? >>>> >>>> On Mon, Aug 19, 2019 at 12:46 PM you Zhuang >>> <mailto:zhuangzixiao...@gmail.com> <mailto:zhuangzixiao...@gmail.com >>>> <mailto:zhuangzixiao...@gmail.com>>> wrote: >>>> >>>>Yeah, I mean no hint , use appropriate index automatically. I >>>>create a local index and a query with corresponding index column >>>>filter in where clause. But the query doesn’t use index, with >>>>index hint it uses it. >>>> >>>> >>>> >>>> -- >>>>Aleksandr Saraseka >>>> DBA >>>> 380997600401 >>>> > *•* asaras...@eztexting.com >>>> <mailto:asaras...@eztexting.com> <mailto:asaras...@eztexting.com >>>> <mailto:asaras...@eztexting.com>> *•* eztexting.com >>>> <http://eztexting.com/><http://eztexting.com/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature >>>> >>>> <http://eztexting.com/?utm_sour
Re: Is there any way to using appropriate index automatically?
Thanks, I will get a try > On Aug 21, 2019, at 5:24 AM, Vincent Poon wrote: > > check out PHOENIX-5109 , it likely fixes your issue. > Unfortunately it's targeted for 4.15.0 which hasn't been released yet. Maybe > you can backport and see if it works for your query. > > On Tue, Aug 20, 2019 at 11:38 AM Ankit Singhal <mailto:ankitsingha...@gmail.com>> wrote: > CREATE local INDEX local_c_h_index ON test_phoenix.app (c,h) ASYNC; > (Has been filled data with bulkload and index is active) > > Query: > select /*+ INDEX(test_phoenix.app local_c_h_index) */ * from > TEST_PHOENIX.APP where c=2 and h = 1 limit 5; > select * from TEST_PHOENIX.APP where c=2 and h = 1 limit 5; > > The first query will use index local_c_h_index and result shortly, the second > query won’t , and response slowly. > Yeah, the local index should be used in both the cases, looks like a bug to > me, can you please raise a JIRA in Phoenix project for the same. > QueryOptimizer.java may have a relevant code to fix the issue, so the patch > would really be appreciated. > > And, also can you try running "select a,b,c,d,e,f,g,h,i,j,k,m,n from > TEST_PHOENIX.APP where c=2 and h = 1 limit 5", and see if index is getting > used. > > Regards, > Ankit Singhal > > On Tue, Aug 20, 2019 at 1:49 AM you Zhuang <mailto:zhuangzixiao...@gmail.com>> wrote: > Er, I also read the sentence “Unlike global indexes, local indexes will use > an index even when all columns referenced in the query are not contained in > the index. This is done by default for local indexes because we know that the > table and index data co-reside on the same region server thus ensuring the > lookup is local.” > > I ‘m totally confused. > > >> On Aug 20, 2019, at 12:32 AM, Josh Elser > <mailto:els...@apache.org>> wrote: >> >> http://phoenix.apache.org/faq.html#Why_isnt_my_secondary_index_being_used >> <http://phoenix.apache.org/faq.html#Why_isnt_my_secondary_index_being_used> >> >> On 8/19/19 6:06 AM, you Zhuang wrote: >>> Phoenix-version: 4.14.3-HBase-1.4-SNAPSHOT >>> hbase-version: 1.4.6 >>> Table: >>> CREATE TABLE test_phoenix.app ( >>> dt integer not null, >>> a bigint not null , >>> b bigint not null , >>> c bigint not null , >>> d bigint not null , >>> e bigint not null , >>> f bigint not null , >>> g bigint not null , >>> h bigint not null , >>> i bigint not null , >>> j bigint not null , >>> k bigint not null , >>> m decimal(30,6) , >>> n decimal(30,6) >>> CONSTRAINT pk PRIMARY KEY (dt, a,b,c,d,e,f,g,h,i,j,k) >>> ) SALT_BUCKETS = 3,UPDATE_CACHE_FREQUENCY = 30; >>> Index: >>> CREATE local INDEX local_c_h_index ON test_phoenix.app (c,h) ASYNC; >>> (Has been filled data with bulkload and index is active) >>> Query: >>> select /*+ INDEX(test_phoenix.app local_c_h_index) */ * from >>> TEST_PHOENIX.APP where c=2 and h = 1 limit 5; >>> select * from TEST_PHOENIX.APP where c=2 and h = 1 limit 5; >>> The first query will use index local_c_h_index and result shortly, the >>> second query won’t , and response slowly. >>> The explain plan is weird, all showing without using index. >>>> On Aug 19, 2019, at 5:54 PM, Aleksandr Saraseka >>> <mailto:asaras...@eztexting.com> <mailto:asaras...@eztexting.com >>>> <mailto:asaras...@eztexting.com>>> wrote: >>>> >>>> We have no problems with that. I mean indexes are used even without hints, >>>> if they're suitable for a query. >>>> Maybe you can share your Phoenix version, query, index definition and exec >>>> plan ? >>>> >>>> On Mon, Aug 19, 2019 at 12:46 PM you Zhuang >>> <mailto:zhuangzixiao...@gmail.com> <mailto:zhuangzixiao...@gmail.com >>>> <mailto:zhuangzixiao...@gmail.com>>> wrote: >>>> >>>>Yeah, I mean no hint , use appropriate index automatically. I >>>>create a local index and a query with corresponding index column >>>>filter in where clause. But the query doesn’t use index, with >>>>index hint it uses it. >>>> >>>> >>>> >>>> -- >>>>Aleksandr Saraseka >>>> DBA >>>> 380997600401 >>>> > *•* asaras...@eztexting.com >>>> <mailto:asaras...@eztexting.com> <mailto:asaras...@eztexting.com >>>> <mailto:asaras...@eztexting
Is there a way to specify split num or reducer num when creating phoenix table ?
I have a chronological series of data. Data row like dt, r1 ,r2 ,r3 ,r4 ,r5 ,r6 ,d1 ,d2 ,d3 ,d4 , d5 … And dt is format as 20190829 , increasing monotonically, such as 20190830,20190831... The query pattern is some like select * from table where dt between 20180620 and 20190829 and r3 = ? And r6 = ?; Dt is mandatory, remain filter is some random combination of r1 to r6, selected columns are always all columns *. I have made dt,r1,r2,… r6 to be compound primary key. The create table clause is below: CREATE TABLE app.table( Dt integer not null , R1 integer not null, R2 integer not null, R3 integer not null, R4 integer not null, R5 integer not null, R6 integer not null, D1 decimal(30,6), D2 decimal(30,6), D3 decimal(30,6), D4 decimal(30,6), D5 decimal(30,6), D6 decimal(30,6) CONSTRAINT pk PRIMARY KEY (dt,r1,r2,r3,r4,r5,r6) ) SALT_BUCKETS = 3,UPDATE_CACHE_FREQUENCY = 30,COMPRESSION = 'SNAPPY', SPLIT_POLICY = 'org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy', MAX_FILESIZE = '50’; I have 3 region server so I determine SALT_BUCKETS = 3. But when I initially load table data with csvbulkload tool , the dt ranges from 20180620 to 20190829, data size is about 1T, Csvbulkload map reduce shows 3 partitions for reducer, It always failed due to so small partitions. I increase SALT_BUCKETS = 512, but max SALT_BUCKETS = 256, I set it to 256 but not works. I know I can split on (…) when creating table, but I don’t know how to determine the point , and hundreds of points is scaring. So Is there a way to specify split num or reducer num when creating phoenix table ? I will be expecting any advice for tuning this scenario.
Re: !indexes in sqlline shows empty
Awesome , it works , thanks. > On Aug 21, 2019, at 5:35 AM, Vincent Poon wrote: > > try this: > !indexes > > On Tue, Aug 20, 2019 at 12:29 AM you Zhuang <mailto:zhuangzixiao...@gmail.com>> wrote: > > > Does sqlline be not accurate ? > > When I query columns in index , explain plan shows not using the index.
Re: Is there a way to specify split num or reducer num when creating phoenix table ?
Sorry,i can’t get your words. when I set SALT_BUCKETS = 256, the region num is actually 256. Pre-split along date-boundaries will be hundreds of split points, it is a heavy work. The biggest problem I’m facing is how to bulkload 1T data to phoenix table. > On Aug 29, 2019, at 8:18 PM, Josh Elser wrote: > > inside
