Hi MIch, I would recommend you to use Phoenix API/Tools to write data to a Phoenix table so that it can handle secondary index seamlessly. Your approach of **Rebuilding** index after every bulkload will run into scalability problems as your primary table keeps growing.
~Anil On Sat, Oct 22, 2016 at 11:22 AM, Mich Talebzadeh <mich.talebza...@gmail.com > wrote: > A workout I deployed was to rebuild the index immediately after bulk load > of data into Hbase table > > ALTER INDEX MARKETDATAHBASE_IDX1 ON "marketDataHbase" REBUILD; > > > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On 22 October 2016 at 18:25, Mich Talebzadeh <mich.talebza...@gmail.com> > wrote: > >> Sorry Ted, >> >> This is the syntax for view >> >> create view "marketDataHbase" (PK VARCHAR PRIMARY KEY, >> “price_info”.”ticker” VARCHAR,"price_info"."timecreated" VARCHAR, >> "price_info"."price" VARCHAR); >> >> Thanks James for clarification. >> >> My understanding is that when one creates an index on a Phoenix view on >> an Hbase table, a local index file is created in Phoenix with data as is >> and that Phoenix table has no way of knowing if new data is added to Hbase >> table. >> >> From an operational point of view unless index in Phoenix is immediately >> updated after any append to Hbase, then it is pretty useless. >> >> In my case data is inserted to Hbase table. I am just using Phoenix for >> data queries (DQ) as opposed to inserts. >> >> Regards, >> >> >> >> >> >> >> >> Dr Mich Talebzadeh >> >> >> >> LinkedIn * >> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >> >> >> >> http://talebzadehmich.wordpress.com >> >> >> *Disclaimer:* Use it at your own risk. Any and all responsibility for >> any loss, damage or destruction of data or any other property which may >> arise from relying on this email's technical content is explicitly >> disclaimed. The author will in no case be liable for any monetary damages >> arising from such loss, damage or destruction. >> >> >> >> On 22 October 2016 at 17:59, James Taylor <jamestay...@apache.org> wrote: >> >>> Hi Mich, >>> Phoenix indexes are only updated if you use Phoenix APIs to input the >>> data. >>> Thanks, >>> James >>> >>> >>> On Saturday, October 22, 2016, Ted Yu <yuzhih...@gmail.com> wrote: >>> >>>> The first statement creates index, not view. >>>> >>>> Can you check ? >>>> >>>> Cheers >>>> >>>> > On Oct 22, 2016, at 1:51 AM, Mich Talebzadeh < >>>> mich.talebza...@gmail.com> wrote: >>>> > >>>> > Hi, >>>> > >>>> > I have a Hbase table that is populated via >>>> > org.apache.hadoop.hbase.mapreduce.ImportTsv >>>> > through bulk load ever 15 minutes. This works fine. >>>> > >>>> > In Phoenix I created a view on this table >>>> > >>>> > jdbc:phoenix:rhes564:2181> create index marketDataHbase_idx on >>>> > "marketDataHbase" ("price_info"."ticker", "price_info"."price", >>>> > "price_info"."timecreated"); >>>> > >>>> > This also does what is supposed to do and shows correct count. >>>> > >>>> > I then created an index in Phoenix as below >>>> > >>>> > create index index_dx1 on "marketDataHbase" >>>> > ("price_info"."timecreated","price_info"."ticker", >>>> "price_info"."price"); >>>> > >>>> > that showed the records OK at that time. I verified this using explain >>>> > >>>> > >>>> > 0: jdbc:phoenix:rhes564:2181> explain select count(1) from >>>> > "marketDataHbase"; >>>> > +---------------------------------------------------------+ >>>> > | PLAN | >>>> > +---------------------------------------------------------+ >>>> > | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER INDEX_DX1 | >>>> > | SERVER FILTER BY FIRST KEY ONLY | >>>> > | SERVER AGGREGATE INTO SINGLE ROW | >>>> > +---------------------------------------------------------+ >>>> > >>>> > Now the issue is that the above does not show new data since build in >>>> Hbase >>>> > table unless I do the following: >>>> > >>>> > 0: jdbc:phoenix:rhes564:2181> alter index INDEX_DX1 on >>>> "marketDataHbase" >>>> > rebuild; >>>> > >>>> > >>>> > Which is not what an index should do (The covered index should be >>>> > maintained automatically). >>>> > The simple issue is how to overcome this problem? >>>> > >>>> > As I understand the index in Phoenix ia another file independent of >>>> the >>>> > original phoenix view so I assume that this index file is not updated >>>> for >>>> > one reason or other? >>>> > >>>> > Thanks >>>> >>> >> > -- Thanks & Regards, Anil Gupta
