Thanks Thejas for prompt response.
I should have elaborated the question a bit more. When I said ODBC / JDBC
is not an option, what I mean was, we have existing ODBC / JDBC
applications and they generate SQL statements adhering to the SQL92
standard. Since HiveQL is different from it and supports syntax for Load
etc, reuse of the existing ODBC / JDBC applications may not be feasible.
But, for writing a new module specific to the Hive, we can still use any
interface, ODBC / JDBC / Hive Thrift and currently weighing the best
options.
Here are the reasons for considering Thrift Hive.
1. While evaluating different JDBC drivers, we have ran into scalability
and performance issues with the JDBC Drivers.
2. During the evaluation of the JDBC drivers, ran into situations where,
not all the features that we were looking for are supported by any
particular driver.
3. The expectation was, thrift API would provide better performance over
the JDBC / ODBC.
Need to implement the following capabilities
1. Kerberos authentication
2. Support to generate native HiveQL and partitioning capabilities.
3. Performance is one of the key criteria.
Based on the above, after evaluating some of the ODBC / JDBC drivers, since
I have seen some performance issues and functionality issues, I am
exploring on the Thrift Hive client API with an assumption that some of the
current shortfalls with the ODBC / JDBC drivers can be addressed by using
the Client API. The initial hiccup that I ran into is with the
authentication.
Hope this gives enough background.
In case if I can get a better throughput using the JDBC option itself, then
I can still consider using the JDBC as opposed to the Thrift Hive. Right
now, evaluating the available options.
Regards
On Wed, Dec 30, 2015 at 11:45 PM, Thejas Nair <thejas.n...@gmail.com> wrote:
> Srinivas,
> Can you please elaborate on why ODBC/JDBC is not an option ?
> I didn't understand what you meant by "unable to generate the HiveQL"
> with those options. How does using thrift api directly help in that
> case ?
>
> ODBC/JDBC is the preferred API for users. There are many features
> implemented in those layers, including the security and also high
> availability features.
> Incorrect use of the thrift api can potentially lead to other issues
> like memory leaks in HiveServer2.
>
>
>
>
>
>
> On Wed, Dec 30, 2015 at 8:37 AM, Srinivas M <smudigo...@gmail.com> wrote:
> > Hi
> >
> > I am trying to develop a custom application using the Thrift Hive client
> > interface to access the Hive and read and write into the hive tables.
> ODBC
> > and JDBC are not option because of the inherent limitations with those
> > interfaces (i.e unable to generate the HiveQL etc).
> >
> > While using the Thrift Hive interface, the first challenge that I ran
> into
> > is with the authentication. I could able to connect to hive only when I
> set
> > the authentication of the hiveserver2 as NOSASL.
> > My application should be able to provide Kerberos authentication and all
> > other authentications supported by Hive.
> >
> > I had tried to look for some samples around implementing authentication
> in
> > the Thrift Hive client, but I could not find much details on that.
> > Can someone help me understand the options available for implementing
> > authentication while using the Thrift Hive client interfaces ?
> >
> > --
> > Srinivas
> > (*-*)
> >
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > You have to grow from the inside out. None can teach you, none can make
> you
> > spiritual.
> > -Narendra Nath Dutta(Swamy Vivekananda)
> >
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
--
Srinivas
(*-*)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
You have to grow from the inside out. None can teach you, none can make you
spiritual.
-Narendra Nath Dutta(Swamy Vivekananda)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
