Several things happened: 1) By far the biggest issue was running well in the cloud. HBase can be run in the cloud, but it’s hard to do it well in a cost efficient way. While each cloud does provide a native key/value store, these don’t all have the advanced functionality that we need (coprocessors, a transaction manager). And we would have had to write an intermediate layer to handle all the different cloud key/value interfaces. In constrast, the three major clouds all offer an option for a database Hive already supports.
2) Getting the transaction manager integrated proved challenging. We worked with Omid, but it ate a lot of cycles on the machines we used in our tests. Given that Omid and Tephra are both continuing to grow and be developed this issue could probably have been overcome in time. 3) Consistent feedback that we got from users as we talked to them about using HBase as the metastore was that they did not know how to administer HBase. Most Hive users are already familiar with an RDBMS and feel comfortable administering one. It was clear if we were going to include HBase it would have to be completely embedded and managed automatically by Hive. This is doable, but a lot of work. Daniel’s caching work is showing promising numbers. Given the above issues it makes more sense to pursue caching rather than keep going down the HBase route. This is particularly true in the cloud where we don’t always get great response times from the cloud native databases. Caching will radically improve our query planning performance. But it won’t address the test matrix issue. By supporting four different databases in production (Derby is only for test) we have to run all the Hive tests on four different systems to assure ourselves it works. I’m trying to mitigate this with the Dockerfile work I’m doing in the standalone-metastore. It will make it easier to do that testing, but the testing will still be required. Alan. Alan. On Tue, Nov 7, 2017 at 3:23 PM, Alexander Kolbasov <ak...@cloudera.com> wrote: > I am wondering what happened with HMS on HBase project? I saw a bunch of > slides telling about the progress and good POC results, but then looks like > it was abandoned for some reason. Were there any serious issues discovered > or benefits turned out to be insignificant? > > - Alex >
