As you are also not seeing any cell having timestamp greater than 9 (in scan
'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00", STOPROW
=> "\x00SYSTEM\x00CATALOG\x01"}) then I think restarting your hbase cluster
will help as it will bounce the stale cache.
Nothing is required to be deleted from phoenix side.
On Sat, Apr 30, 2016 at 9:04 PM, Bavineni, Bharata <
bharata.bavin...@epsilon.com> wrote:
> Hi Ankit,
>
> We have removed all cells for these two columns and scan with raw =>
> true does not return any results:
>
> scan 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT\x00", STOPROW =>
> "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT_\x00"}
>
> scan 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x00", STOPROW =>
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP_\x00"}
>
> We still see the same error.
>
>
>
> Do we need to explicitly delete from phoenix as well?
>
>
>
> Thanks,
>
> Bharathi.
>
>
>
> *From:* Ankit Singhal [mailto:ankitsingha...@gmail.com]
> *Sent:* Saturday, April 30, 2016 3:00 AM
> *To:* user@phoenix.apache.org
> *Subject:* Re: Undefined column. columnName=IS_ROW_TIMESTAMP
>
>
>
> Do a raw scan on SYSTEM.CATALOG through hbase shell and confirm that there
> should not be any cell for SYSTEM.CATALOG row having timestamp greater than
> 9.
>
>
>
> On Sat, Apr 30, 2016 at 3:28 AM, Arun Kumaran Sabtharishi <
> arun1...@gmail.com> wrote:
>
> Ankit,
>
>
>
> Follwing the steps, the below exception is thrown. Can you let me know
> what could have gone wrong?
>
>
>
> ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP
> BOOLEAN;
>
> Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
> (state=42M04,code=1013)
>
> org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013
> (42M04): Table already exists. tableName=SYSTEM.CATALOG
>
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
>
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
>
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
>
> at sqlline.Commands.execute(Commands.java:822)
>
> at sqlline.Commands.sql(Commands.java:732)
>
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
>
> at sqlline.SqlLine.begin(SqlLine.java:681)
>
> at sqlline.SqlLine.start(SqlLine.java:398)
>
> at sqlline.SqlLine.main(SqlLine.java:292)
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Fri, Apr 29, 2016 at 4:47 AM, Ankit Singhal <ankitsingha...@gmail.com>
> wrote:
>
> Hi Arun,
>
>
>
> Actually , we missed the quotes as shell will consider everything after
> ";" as second command.
>
>
>
> bin/sqlline.py "localhost;currentSCN=9"
>
>
>
> So, can you fix the timestamp for these columns?
>
> Steps you may follow(but request you to try first on non production
> cluster as these are not tested by me).
>
> - Take snapshot of your system.catalog( for backup)
>
> - delete such cells from system.catalog at latest timestamp
>
> - set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should
> not have any query running and using schema in point in time snapshots).
>
> - do flush and major_compaction on SYSTEM.CATALOG
>
> - when you don't see those columns and open connection at currentSCN=9 and
> alter table to add both the columns.
>
> - you may set keep_deleted_cells back to true in SYSTEM.CATALOG
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
>
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
> On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <
> arun1...@gmail.com> wrote:
>
> James,
>
>
>
> Thanks for the answer. But, why was the currentSCN=9 on connection is not
> working in the first place? Do you have any pointers for us to look into?
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <jamestay...@apache.org>
> wrote:
>
> Hi Arun,
>
> Yes, this could cause problems, as we base the need to upgrade on the time
> stamp of the system catalog table.
>
> Thanks,
>
> James
>
>
>
> On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <arun1...@gmail.com>
> wrote:
>
> Hi Ankit,
>
>
>
> Just following with the question that when the alter statement was issued
> with CurrentSCN=9, the current timestamp was not set to 9.
>
> Will this cause an issue in the future if it has to compare the timestamps?
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <
> arun1...@gmail.com> wrote:
>
> Hi Ankit,
>
>
>
> Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
> IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it actually added
> the columns to the SYSTEM.CATALOG.
>
> But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current
> timestamp was not set to 9. Will this cause an issue in the future if it
> has to compare the timestamps?
>
>
>
> After the alter statement was executed and when queried from HBase, the
> following is the output.
>
>
>
> hbase(main):004:0> get
> 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
> COLUMN CELL
> 0 ATA_TYPE timestamp=1461362240655,
> value=\x80\x00\x00\x10
> 0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
> 0:NULLABLE timestamp=1461362240655,
> value=\x80\x00\x00\x01
> 0 RDINAL_POSITION timestamp=1461362240655,
> value=\x80\x00\x004
> 0 ORT_ORDER timestamp=1461362240655,
> value=\x80\x00\x00\x02
>
>
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <ankitsingha...@gmail.com>
> wrote:
>
> Yes Arun, it seems fine to me.
>
> Can you do it for BASE_COLUMN_COUNT too so that the dependent features
> work correctly.(Remember use correct INTEGER byte representation for
> DATA_TYPE column).
>
>
>
> And, can you also please share output of
>
> > scan 'SYSTEM.SEQUENCE'
>
>
>
> Regards,
>
> Ankit
>
>
>
> On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <
> arun1...@gmail.com> wrote:
>
> Ankit,
>
>
>
> Though the ALTER in pheonix failed, a bunch of puts in HBase did the job.
>
> Is it okay to do the following? These puts in HBase intends to do what the
> ALTER command in phoenix supposed to do.
>
>
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
> arun1...@gmail.com> wrote:
>
> Ankit,
>
>
>
> When tried executing the query, the following exception is thrown.
>
>
>
> Error: ERROR 301 (23000): Concurrent modification to table.
> tableName=SYSTEM.CATALOG (state=23000,code=301)
> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301
> (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
>
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
> at sqlline.Commands.execute(Commands.java:822)
> at sqlline.Commands.sql(Commands.java:732)
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
> at sqlline.SqlLine.begin(SqlLine.java:681)
> at sqlline.SqlLine.start(SqlLine.java:398)
>
>
>
> Thank,
> Arun
>
>
>
> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <ankitsingha...@gmail.com>
> wrote:
>
> Arun,
>
> As per the system.catalog ,there seems no way that upgrade code(adding new
> columns and view upgrade for version < v4.5) doesn't execute.
>
>
>
> But anyways, let's add the required columns manually by following below
> step;
>
>
>
> > ./sqlline.py localhost;CurrentSCN=9
>
> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
> IS_ROW_TIMESTAMP BOOLEAN;
>
> >!quit
>
>
>
> Quit the shell and start new session without CurrentSCN.
>
> > ./sqlline.py localhost
>
> > !describe system.catalog
>
>
>
> this should resolve the issue of missing column.
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
>
>
> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
> arun1...@gmail.com> wrote:
>
> Ankit,James,
>
>
>
> Please find the results you have asked for in the attachment text document.
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <ankitsingha...@gmail.com>
> wrote:
>
> Arun,
>
> the observations you had is understandable.
>
> So, that's why we again requested the output of
>
>
>
> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG*
>
>
>
> So that we can help you in adding ROW_TIMESTAMP column in system.catalog
> table so that your phoenix setup works properly and you can also see it in
> !describe command.
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
> arun1...@gmail.com> wrote:
>
> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>
>
>
> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
>
>
>
> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>
>
>
> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
>
>
>
> Is this an expected behavior?
>
>
>
> If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do?
> This issue is blocking us in the production environment. Any help to
> resolve or workaround is highly appreciated.
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <ankitsingha...@gmail.com>
> wrote:
>
> It's ok if you can just post after grep for CATALOG in a command output (scan
> 'SYSTEM.CATALOG', {RAW=>true}).
>
>
>
> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
> arun1...@gmail.com> wrote:
>
> One more question to add,
>
> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>
>
>
> I have limited the scan in HBase and here is the following data.
>
>
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655,
> value=\x7F\xFF\xFF\xFF
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655,
> value=\x80\x00\x00\x05
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
> timestamp=1460354090655, value=default
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
> timestamp=1460354090655, value=\x01
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655,
> value=\x80\x00\x00\x00\x00\x00\x00\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x04
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> value=\x80\x01
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x05
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x03
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
> arun1...@gmail.com> wrote:
>
> James,
>
>
>
> Table SYSTEM.CATALOG is ENABLED
>
> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
> '|org.apache
>
>
>
> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3
> => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>
>
>
> |', coprocessor$4 =>
> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>
>
>
> taEndpointImpl|1|', coprocessor$6 =>
> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
> {'SPLIT_POLICY' => 'org.apache.
>
>
>
> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>
> COLUMN FAMILIES DESCRIPTION
>
> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW',
> REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>
>
>
> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true',
> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>
>
>
> 1 row in 0.6060 seconds
>
>
>
> The above is for describe SYSTEM.CATALOG. The output for scan
> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <jamestay...@apache.org>
> wrote:
>
> Arun,
>
> Please run the command Ankit mentioned in an HBase shell and post the
> output back here.
>
> Thanks,
>
> James
>
>
>
>
> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <arun1...@gmail.com>
> wrote:
>
> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
>
> But we do see this column from select statement:
>
> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <ankitsingha...@gmail.com>
> wrote:
>
> Hi Arun,
>
>
>
> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
> !describe on system.catalog.
>
>
>
>
>
> if not,
>
> can you share the output of below command. As it seems SYSTEM.CATALOG was
> updated with timestamp greater v4.6 timestamp , and which stopping upgrade
> code to add a new column.
>
>
>
> scan 'SYSTEM.CATALOG', {RAW=>true}
>
>
>
>
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
> arun1...@gmail.com> wrote:
>
> After further investigation, we found that Phoenix Upsert query
> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
> before 4.6 upgrade?"
>
>
>
> We do see that clearCache() is being called for 4.7, and 4.7 upgrades from
> ConnectionQueryServicesImpl class, but not for 4.6
>
>
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
> arun1...@gmail.com> wrote:
>
> James,
>
>
>
> To add more information on this issue, this happens in new phoenix views
> associated with brand new tables as well. So, this cannot be an
> upgrade/migration issue. Not figured out a specific way to reproduce this
> issue yet. Could you throw some ideas on what direction this problem could
> be approached from this point?
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ------------------------------
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
