Re: unsubscribe)

Hi Uzi, To unsubscribe e-mail: user-unsubscr...@spark.apache.org *Daniel Lopes* Chief Data and Analytics Officer | OneMatch c: +55 (18) 99764-2733 | https://www.linkedin.com/in/dslopes www.onematch.com.br <http://www.onematch.com.br/?pk_campaign=EmailSignature&pk_kwd=daniel-lopes> O

Check out Kyper! Trying to be Uber of Data

I just signed up for Kyper and thought you might be interested, too! http://l.aunch.us/L7Ezb

Re: unsubscribe

please send to user-unsubscr...@spark.apache.org *Daniel Lopes* Chief Data and Analytics Officer | OneMatch c: +55 (18) 99764-2733 | https://www.linkedin.com/in/dslopes www.onematch.com.br <http://www.onematch.com.br/?utm_source=EmailSignature&utm_term=daniel-lopes> On Tue, Aug 2, 201

Re: Scala VS Java VS Python

--- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > > -- *Daniel Lopes, B.Eng* Data Scientist - BankFacil CREA/SP 5069410560 <http://edital.confea.org.br/Con

spark-csv on Amazon EMR

Hi, Some know how to use spark-csv in create-cluster statement of Amazon EMR CLI? Best, -- *Daniel Lopes, B.Eng* Data Scientist - BankFacil CREA/SP 5069410560 <http://edital.confea.org.br/ConsultaProfissional/cartao.aspx?rnp=2613651334> Mob +55 (18) 99764-2733 Ph +55 (11) 3522-800

UDF with 2 arguments

use? Best, -- *Daniel Lopes, B.Eng* Data Scientist - BankFacil CREA/SP 5069410560 <http://edital.confea.org.br/ConsultaProfissional/cartao.aspx?rnp=2613651334> Mob +55 (18) 99764-2733 Ph +55 (11) 3522-8009 http://about.me/dannyeuu Av. Nova Independência, 956, São Paulo, SP Bairro Brooklin Paulis

Re: UDF with 2 arguments

of Spark you have? > > >>> from pyspark.sql.functions import udf > >>> def f(a, b): pass > ... > >>> my_udf = udf(f) > >>> from pyspark.sql.types import * > >>> my_udf = udf(f, IntegerType()) > > > On Wed, Nov 25, 2015 at 12

Spark 1.5.2 + Hive 1.0.0 in Amazon EMR 4.2.0

onseProcessingTime=[0.169], HttpClientSendRequestTime=[0.145], 15/11/30 21:40:21 WARN RetryingMetaStoreClient: MetaStoreClient lost connection. Attempting to reconnect. org.apache.thrift.TApplication*Exception: Invalid method name: 'alter_table_with_cascade'* Thanks! -- *Daniel Lopes, B.Eng*

year out of range

quot;, line 1563, in func = lambda _, it: map(lambda x: returnType.toInternal(f(*x)), it) File "/usr/local/src/spark160master/spark-1.6.0-bin-2.6.0/python/lib/pyspark.zip/pyspark/sql/types.py", line 191, in toInternal else time.mktime(dt.timetuple())) *ValueError: year out of rang

Re: year out of range

ull|null| null| null| null| +-++-++--+---+-+-+--+--++-+-+--+++++--+++--+-+------+ - *Daniel Lopes* Chief Data and Analytics Officer | OneMatch c: +55 (18) 99764-2733 | https://www.linkedin.com/in/dslopes www.one

Re: year out of range

Thanks, I *tested* the function offline and works Tested too with select * from after convert the data and see the new data good *but* if I *register as temp table* to *join other table* stilll shows *the same error*. ValueError: year out of range Best, *Daniel Lopes* Chief Data and Analytics

Re: year out of range

Thanks Mike, A good way to debug! Was that already! Best, *Daniel Lopes* Chief Data and Analytics Officer | OneMatch c: +55 (18) 99764-2733 | https://www.linkedin.com/in/dslopes www.onematch.com.br <http://www.onematch.com.br/?utm_source=EmailSignature&utm_term=daniel-lopes> On

Re: year out of range

Thanks Ayan! *Daniel Lopes* Chief Data and Analytics Officer | OneMatch c: +55 (18) 99764-2733 | https://www.linkedin.com/in/dslopes www.onematch.com.br <http://www.onematch.com.br/?utm_source=EmailSignature&utm_term=daniel-lopes> On Thu, Sep 8, 2016 at 7:54 PM, ayan guha wrote: >

Spark + Parquet + IBM Block Storage at Bluemix

rted due to stage failure: Task 60 in stage 30.0 failed 10 times, most recent failure: Lost task 60.9 in stage 30.0 (TID 2556, yp-spark-dal09-env5-0039): org.apache.hadoop.fs.swift.exceptions.SwiftConfigurationException:* Missing mandatory configuration option: fs.swift.service.keystone.auth.url* at org.apache.hadoop.fs.swift.http.RestClientBindings.copy(RestClientBindings.java:223) at org.apache.hadoop.fs.swift.http.RestClientBindings.bind(RestClientBindings.java:147) *Daniel Lopes* Chief Data and Analytics Officer | OneMatch c: +55 (18) 99764-2733 | https://www.linkedin.com/in/dslopes www.onematch.com.br <http://www.onematch.com.br/?utm_source=EmailSignature&utm_term=daniel-lopes>

Re: Spark + Parquet + IBM Block Storage at Bluemix

Thanks Steve, But this error occurs only with parquet files, CSVs works. Best, *Daniel Lopes* Chief Data and Analytics Officer | OneMatch c: +55 (18) 99764-2733 | https://www.linkedin.com/in/dslopes www.onematch.com.br <http://www.onematch.com.br/?utm_source=EmailSignature&utm_term

Re: Fw: Spark + Parquet + IBM Block Storage at Bluemix

Hi Mario, Thanks for your help, so I will keeping using CSVs Best, *Daniel Lopes* Chief Data and Analytics Officer | OneMatch c: +55 (18) 99764-2733 | https://www.linkedin.com/in/dslopes www.onematch.com.br <http://www.onematch.com.br/?utm_source=EmailSignature&utm_term=daniel-lopes&g

Re: unsubscribe

Hi Chang, just send a e-mail to user-unsubscr...@spark.apache.org Best, *Daniel Lopes* Chief Data and Analytics Officer | OneMatch c: +55 (18) 99764-2733 | https://www.linkedin.com/in/dslopes www.onematch.com.br <http://www.onematch.com.br/?utm_source=EmailSignature&utm_term=daniel-lop

Re: unsubscribe

To unsubscribe e-mail: user-unsubscr...@spark.apache.org *Daniel Lopes* Chief Data and Analytics Officer | OneMatch c: +55 (18) 99764-2733 | http://www.daniellopes.com.br www.onematch.com.br <http://www.onematch.com.br/?utm_source=EmailSignature&utm_term=daniel-lopes> On Mon, Sep 26,