Re: Bits getting flipped in record value

2019-03-21 Thread Reid Chan
23:28 To: user@hbase.apache.org Subject: Re: Bits getting flipped in record value We're using FAST_DIFF for the block encoding. For completeness here's the rest: {TABLE_ATTRIBUTES => {coprocessor$1 => 's3://optix.rtb16/root/lib/geomesa-hbase-distributed-r

Re: Bits getting flipped in record value

2019-03-21 Thread Austin Heyne
, R.C From: aheyne Sent: 21 March 2019 10:06 To: Sean Busbey Cc: user@hbase.apache.org Subject: Re: Bits getting flipped in record value Correct, no records will ever be updated. We do have a custom coprocessor loaded but before I mention that you shou

Re: Bits getting flipped in record value

2019-03-20 Thread Reid Chan
Do your table has `DATA_BLOCK_ENCODING` set? -- Best regards, R.C From: aheyne Sent: 21 March 2019 10:06 To: Sean Busbey Cc: user@hbase.apache.org Subject: Re: Bits getting flipped in record value Correct, no records will

Re: Bits getting flipped in record value

2019-03-20 Thread aheyne
Correct, no records will ever be updated. We do have a custom coprocessor loaded but before I mention that you should have more context about what we're actually doing. We're running GeoMesa [1] on top of HBase. Practically what this means is that for every record we write, we write twice, onc

Re: Bits getting flipped in record value

2019-03-20 Thread Sean Busbey
So you're saying no records should ever be updated, right? Do you have any coprocessors loaded? On Wed, Mar 20, 2019, 20:32 aheyne wrote: > I don't have the WALs but due to the nature of the data each record/key > is unique. The keys for the data are generated using spatial-temporal > dimensi

Re: Bits getting flipped in record value

2019-03-20 Thread aheyne
I don't have the WALs but due to the nature of the data each record/key is unique. The keys for the data are generated using spatial-temporal dimensions of the observation. -Austin On 2019-03-20 21:25, Sean Busbey wrote: Have you examined the wals for writes to the impacted cells to verify an

Re: Bits getting flipped in record value

2019-03-20 Thread Sean Busbey
Have you examined the wals for writes to the impacted cells to verify an update wasn't written with the change to the value? On Wed, Mar 20, 2019, 17:47 Austin Heyne wrote: > Hey all, > > We're running HBase 1.4.8 on EMR 5.20 backed by S3 and we're seeing a > bit get flipped in some record value