I forgot to mention that in this code my out.collect method is outputting a
tuple of Map[key,value] and the count
as Int.
On Fri, Apr 29, 2016 at 4:53 PM, Punit Naik wrote:
> Anyways, I fixed it. To you groupBy you should attach this:
>
> .reduceGroup {
>
t((k,v))
}
On Fri, Apr 29, 2016 at 2:58 PM, Punit Naik wrote:
> What if after grouping I wanted to count the occurrences of the key
> "ga_date"?
>
> On Fri, Apr 29, 2016 at 2:15 PM, Stefano Baghino <
> stefano.bagh...@radicalbit.io> wrote:
>
>> The `
> > map.get("ga_dateHour").get.toString().length()-2))) }}}
> >
> > val k=j.groupBy(_.get("ga_date"))
> >
> > But when I execute this, it throws an exception saying:
> >
> > org.apache.flink.api.common.InvalidProgramException: Return type
gt; ga("ga_date")) or
> val k=j.groupBy(_.apply("ga_date"))
>
> As a side note, I believe the user mailing list may be more appropriate for
> this kind of issues.
>
> [1]: https://issues.apache.org/jira/browse/FLINK-2673
> [2]:
> http://dataartisans.gith
mException: Return type
Option[String] of KeySelector class
org.apache.flink.api.scala.DataSet$$anon$12 is not a valid key type
Where am I going wrong?
--
Thank You
Regards
Punit Naik
I am so sorry. Please ignore my previous reply. Actually my input was too
big so it hung. So stupid of me. Thanks a lot! Your example worked!
On Fri, Apr 29, 2016 at 12:35 AM, Punit Naik wrote:
> I tried exactly what you told me. But when I execute this code, first of
> all it give
out
> each line preceded by an "arrow").
> I hope I've been helpful.
>
> On Thu, Apr 28, 2016 at 12:33 PM, Punit Naik
> wrote:
>
> > No Sir, its one json per line.
> >
> > On Thu, Apr 28, 2016 at 3:19 PM, Fabian Hueske
> wrote:
> >
> to point you to the excellent programming guide in the official
> documentation [1]. I'm sure you'll find the reading very interesting and
> worthwhile.
>
> [1]:
>
> https://ci.apache.org/projects/flink/flink-docs-master/apis/common/index.html
>
> On Thu, A
.
int, double, map, array etc. I have attached a small two line input file
and I request you to please implement the logic that I have explained above
using flink. It would be a great help.
On Thu, Apr 28, 2016 at 4:04 PM, Punit Naik wrote:
> I managed to fix this error. I basically had to do va
I managed to fix this error. I basically had to do val j=data.map { x => (
x.replaceAll("\"","\\\"")) } instead of val j=data.map { x => ("\"\"\""+x+
"\"\"\"") }
On Wed, Apr 27, 2016 at 4:05 PM, Punit Naik w
No Sir, its one json per line.
On Thu, Apr 28, 2016 at 3:19 PM, Fabian Hueske wrote:
> readTextFile reads a file line-wise.
>
> Is it possible, that your first line only contains "{"?
>
> 2016-04-28 8:06 GMT+02:00 Punit Naik :
>
> > I have a test file whic
This prints "{"
--
Thank You
Regards
Punit Naik
sed to it but it does not as it outputs a "None" string which means it
failed.
Why does this happen and how can I make it work?
On Wed, Apr 27, 2016 at 12:41 PM, Punit Naik wrote:
> I just tried it and it still cannot parse it. It still takes the input as
> a dataset object rather
I just tried it and it still cannot parse it. It still takes the input as a
dataset object rather than a string.
On Wed, Apr 27, 2016 at 12:36 PM, Punit Naik wrote:
> Okay Thanks a lot Fabian!
>
> On Wed, Apr 27, 2016 at 12:34 PM, Fabian Hueske wrote:
>
>> You should do th
JSON parser.
>
> 2016-04-27 6:40 GMT+02:00 Punit Naik :
>
> > Hi
> >
> > So I managed to do the map part. I stuc with the "import
> > scala.util.parsing.json._" library for parsing.
> >
> > First I read my JSON:
> >
> > val data=env.
a parser in the
> open() method.
>
> Best, Fabian
>
> [1]
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/batch/dataset_transformations.html#map
> [2]
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/common/index.html#specifying-transformation-
v.readTextFile("/path/to/jsonfile");
> DataSet json = text.map(new YourMapFunctionWhichParsesJSON());
>
> Best, Fabian
>
> 2016-04-26 8:06 GMT+02:00 Punit Naik :
>
> > Hi
> >
> > I am new to Flink. I was experimenting with the Dataset API and found out
>
Hi
I am new to Flink. I was experimenting with the Dataset API and found out
that there is no explicit method for loading a JSON file as input. Can
anyone please suggest me a workaround?
--
Thank You
Regards
Punit Naik
18 matches
Mail list logo
