Re: Save DataFrame to HBase

2016-05-11 Thread Ted Yu
Please note: The name of hbase table is specified in: def writeCatalog = s"""{ |"table":{"namespace":"default", "name":"table1"}, not by the: HBaseTableCatalog.newTable -> "5" FYI On Tue, May 10, 2016 at 3:11 PM, Ted Yu wrote: > I think so. > > Please refer to the table

Re: Save DataFrame to HBase

2016-05-10 Thread Ted Yu
I think so. Please refer to the table population tests in (master branch): hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala Cheers On Tue, May 10, 2016 at 2:53 PM, Benjamin Kim wrote: > Ted, > > Will the hbase-spark module allow for creating tables in Spark SQL

Re: Save DataFrame to HBase

2016-05-10 Thread Benjamin Kim
Ted, Will the hbase-spark module allow for creating tables in Spark SQL that reference the hbase tables underneath? In this way, users can query using just SQL. Thanks, Ben > On Apr 28, 2016, at 3:09 AM, Ted Yu wrote: > > Hbase 2.0 release likely would come after Spark 2.0 release. > > The

Re: Save DataFrame to HBase

2016-04-28 Thread Ted Yu
Hbase 2.0 release likely would come after Spark 2.0 release. There're other features being developed in hbase 2.0 I am not sure when hbase 2.0 would be released. The refguide is incomplete. Zhan has assigned the doc JIRA to himself. The documentation would be done after fixing bugs in hbase-s

Re: Save DataFrame to HBase

2016-04-27 Thread Benjamin Kim
Hi Ted, Do you know when the release will be? I also see some documentation for usage of the hbase-spark module at the hbase website. But, I don’t see an example on how to save data. There is only one for reading/querying data. Will this be added when the final version does get released? Thank

Re: Save DataFrame to HBase

2016-04-27 Thread Benjamin Kim
Daniel, If you can get the code snippet, that would be great! I’ve been trying to get it to work for me as well. The examples on the Phoenix website do not work for me. If you are willing to also, can you include your setup to make Phoenix work with Spark? Thanks, Ben > On Apr 27, 2016, at 11

Re: Save DataFrame to HBase

2016-04-27 Thread Paras sachdeva
Hi Daniel, Would you possibly be able to share the snipped to code you have used ? Thank you. On Wed, Apr 27, 2016 at 3:13 PM, Daniel Haviv < daniel.ha...@veracity-group.com> wrote: > Hi Benjamin, > Yes it should work. > > Let me know if you need further assistance I might be able to get the co

Re: Save DataFrame to HBase

2016-04-27 Thread Daniel Haviv
Hi Benjamin, Yes it should work. Let me know if you need further assistance I might be able to get the code I've used for that project. Thank you. Daniel > On 24 Apr 2016, at 17:35, Benjamin Kim wrote: > > Hi Daniel, > > How did you get the Phoenix plugin to work? I have CDH 5.5.2 installed

Re: Save DataFrame to HBase

2016-04-24 Thread Benjamin Kim
Hi Daniel, How did you get the Phoenix plugin to work? I have CDH 5.5.2 installed which comes with HBase 1.0.0 and Phoenix 4.5.2. Do you think this will work? Thanks, Ben > On Apr 24, 2016, at 1:43 AM, Daniel Haviv > wrote: > > Hi, > I tried saving DF to HBase using a hive table with hbase s

Re: Save DataFrame to HBase

2016-04-24 Thread Daniel Haviv
Hi, I tried saving DF to HBase using a hive table with hbase storage handler and hiveContext but it failed due to a bug. I was able to persist the DF to hbase using Apache Pheonix which was pretty simple. Thank you. Daniel > On 21 Apr 2016, at 16:52, Benjamin Kim wrote: > > Has anyone found

Re: Save DataFrame to HBase

2016-04-21 Thread Zhan Zhang
You can try this https://github.com/hortonworks/shc.git or here http://spark-packages.org/package/zhzhan/shc Currently it is in the process of merging into HBase. Thanks. Zhan Zhang On Apr 21, 2016, at 8:44 AM, Benjamin Kim mailto:bbuil...@gmail.com>> wrote: Hi Ted, Can this module be used

Re: Save DataFrame to HBase

2016-04-21 Thread Benjamin Kim
Hi Ted, Can this module be used with an older version of HBase, such as 1.0 or 1.1? Where can I get the module from? Thanks, Ben > On Apr 21, 2016, at 6:56 AM, Ted Yu wrote: > > The hbase-spark module in Apache HBase (coming with hbase 2.0 release) can do > this. > > On Thu, Apr 21, 2016 at

Re: Save DataFrame to HBase

2016-04-21 Thread Ted Yu
The hbase-spark module in Apache HBase (coming with hbase 2.0 release) can do this. On Thu, Apr 21, 2016 at 6:52 AM, Benjamin Kim wrote: > Has anyone found an easy way to save a DataFrame into HBase? > > Thanks, > Ben > > > - >

Save DataFrame to HBase

2016-04-21 Thread Benjamin Kim
Has anyone found an easy way to save a DataFrame into HBase? Thanks, Ben - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org