ode. 'com.snowplowanalytics:scala-maxmind-iplookups:0.2.0’ was
> imported in Gradle.
>
> spark version: 1.6.0
> scala: 2.10.4
> scala-maxmind-iplookups: 0.2.0
>
> I run my test, got the error as below:
> java.lang.NoClassDefFoundError:
> scala/collection/JavaConversions$JMapWr
plookups" % "0.2.0"
)
otherwise, to process streaming log I use logstash with kafka as input.
You can set kafka as output if you need to do some extra calculation
with spark.
Le 23/02/2016 15:07, Romain Sagean a écrit :
Hi,
I use maxmind geoip with spark (no streaming). To make i
Hi,
I use maxmind geoip with spark (no streaming). To make it work you should
use mapPartition. I don't know if something similar exist for spark
streaming.
my code for reference:
def parseIP(ip:String, ipLookups: IpLookups): List[String] = {
val lookupResult = ipLookups.performLookups(ip)
ns
> > Kryo is working as it’s meant to?
> >
> >
> >
> > Am I at the point where I should pull apart the source code and build my
> own
> > DateTime class? I hate reinventing the wheel though.
> >
> >
> >
> > Thanks,
> >
> &g
(), days);
>> Maybe catch the NPE and print out the value of currentDate to see if
>> there is more clue ?
>>
>> Cheers
>>
>> On Tue, Nov 10, 2015 at 12:55 PM, Romain Sagean
>> wrote:
>>
>>> see below a more complete version of the code.
>
2015-11-10 18:39 GMT+01:00 Ted Yu :
> Can you show the stack trace for the NPE ?
>
> Which release of Spark are you using ?
>
> Cheers
>
> On Tue, Nov 10, 2015 at 8:20 AM, romain sagean
> wrote:
>
>> Hi community,
>> I try to apply the function below duri
Hi community,
I try to apply the function below during a flatMapValues or a map but I
get a nullPointerException with the plusDays(1). What did I miss ?
def allDates(dateSeq: Seq[DateTime], dateEnd: DateTime): Seq[DateTime] = {
if (dateSeq.last.isBefore(dateEnd)){
allDates(dateSeq:+ d
hereby notified that any dissemination, distribution or
>> copying of this communication is strictly prohibited. If you have received
>> this transmission in error, do not read it. Please immediately reply to the
>> sender that you have received this communication in error and then delete
>> it.
>>
>> Esta mensagem e seus anexos se dirigem exclusivamente ao seu
>> destinatário, pode conter informação privilegiada ou confidencial e é para
>> uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o
>> destinatário indicado, fica notificado de que a leitura, utilização,
>> divulgação e/ou cópia sem autorização pode estar proibida em virtude da
>> legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos
>> o comunique imediatamente por esta mesma via e proceda a sua destruição
>>
>
>
--
Romain Sagean
