Re: [ANNOUNCE] Introducing Apache Flink Taiwan User Group - Flink.tw

[Fabian Hueske]

Hi Gordon,

this is great news! I am very happy to hear that there is a new local Flink
community in Taiwan!
Translating blog posts, slide sets, and other documentation is very
valuable and makes the project known to a broader audience and accessible
to more people.
Please let us know, if you need help with anything.

Thanks for your efforts,
Fabian

2016-01-01 19:24 GMT+01:00 tzulitai :

> Hi Flink community,
>
> We are very excited to announce a new local Flink user group / meetup based
> in Taiwan:
>
> Facebook group:  http://www.facebook.com/groups/flink.tw
> 
> Blog:  http://blog.flink.tw 
>
> We're a group of Chinese-speaking people who finds Apache Flink truly
> amazing =)
>
> The blog will be mainly used for accumulating translated information from
> Flink Forward, the official Apache Flink website, Data Artisan blogs etc.,
> as well as any content about Flink and related-technology originated from
> the blog's co-authors.
>
> We will also be organizing meetups in Taipei! We're currently preparing for
> our first meetup.
> Please let us know if you're a Chinese speaker and would like to give a
> talk
> at our future meetups.
>
> You can subscribe to the blog, or join our Facebook group for any latest
> news.
> The group is open to anyone who loves Flink too!
>
> Thanks,
> Gordon
>
> P.S. We have made a slight derivation to the Flink logo for our group's
> use.
> If the community finds this inappropriate, please let me know and I will
> adjust it immediately.
>
>
>
> --
> View this message in context:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ANNOUNCE-Introducing-Apache-Flink-Taiwan-User-Group-Flink-tw-tp4136.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>

Re: RideCleansing example

[Fabian Hueske]

Hi Serkan,

yes this is expected. The reference implementations use DataStream.print()
to emit the resulting data stream to the standard output. If the program is
executed in an IDE, the standard output is goes to the IDE's console. If
you start Flink in a local (or cluster) setup, the standard output is
redirected to a .out file in the ./log folder. For local setups, this will
be the *-jobmanager-*.out file, for cluster setups the individual
*-taskmanager-*.out files.

Note that the behavior of .print() differs for the DataStream and the
DataSet API. For DataSet, the output of the task manager (worker) processes
is collected by the job manager (master) process and is shipped to the
client process that submitted the program and printed there. For
DataStream, the output is directly emitted by the task manager processes
and written to their .out log files.

Best,
Fabian

2016-01-01 19:06 GMT+01:00 Serkan Taş :

> Hi,
>
> I am working on lectures and examples to get used to on flink.
>
> For the RideCleansing example, in intelliJ it is working as expected. But
> after packaging submitting to local flink, there is no output.
>
> is it normal ?
>
> Here is the console
>
> 01/01/2016 19:55:31 Job execution switched to status RUNNING.
> 01/01/2016 19:55:31 Source: Custom Source -> Filter -> Sink:
> Unnamed(1/1) sw
> itched to SCHEDULED
> 01/01/2016 19:55:31 Source: Custom Source -> Filter -> Sink:
> Unnamed(1/1) sw
> itched to DEPLOYING
> 01/01/2016 19:55:32 Source: Custom Source -> Filter -> Sink:
> Unnamed(1/1) sw
> itched to RUNNING
>
> and the web screen
>
> (not-set) Source : Custom Source -> Filter -> Sink: Unnamed
>
> thanx in advance
>
>
> *Serkan Taş*
> Mobil : +90 532 250 07 71
> Likya Bilgi Teknolojileri
> ve İletişim Hiz. Ltd. Şti.
> www.likyateknoloji.com
>
> --
> Bu elektronik posta ve onunla iletilen bütün dosyalar gizlidir. Sadece
> yukarıda isimleri belirtilen kişiler arasında özel haberleşme amacını
> taşımaktadır. Size yanlışlıkla ulaşmışsa bu elektonik postanın içeriğini
> açıklamanız, kopyalamanız, yönlendirmeniz ve kullanmanız kesinlikle
> yasaktır. Lütfen mesajı geri gönderiniz ve sisteminizden siliniz. Likya
> Bilgi Teknolojileri ve İletişim Hiz. Ltd. Şti. bu mesajın içeriği ile
> ilgili olarak hiç bir hukuksal sorumluluğu kabul etmez.
>
> This electronic mail and any files transmitted with it are intended for
> the private use of  the persons named above. If you received this message
> in error, forwarding, copying or use of any of the information is strictly
> prohibited. Please immediately notify the sender and delete it from your
> system. Likya Bilgi Teknolojileri ve İletişim Hiz. Ltd. Şti. does not
> accept legal responsibility for the contents of this message.
> --
>
>
>
>
>
>
>
> P
> Bu e-postayı yazdırmadan önce, çevreye olan sorumluluğunuzu tekrar düşünün.
> Please consider your environmental responsibility before printing this
> e-mail.
>
>
>

Re: Unit testing support for flink application?

[lofifnc]

Hi,

I'm currently updating and improving the documentation[1] of flink-spector.
Regarding missing examples: I'm planning to include small examples showing
the behaviour of output matchers, as the documentation already includes
several demonstrating how to assemble test cases. Please let me know, if
you're having problems with a particular aspect of the framework or the
documentation or like to have an example for a special case. I will try to
work it into the next version.

At the moment the framework is lacking support to tests the scala api. As
you've experienced It's possible to  create simple cases but you'll soon
discover problems with hamcrest and scala types in general. I have written a
test environment for the scala api. And it should be easy to provide a nice
trait for scalatest and some wrappers around the java classes. What I'm
struggling with at the moment, is to utilize the scalatest matchers for
output verification. 

Best, 
Alex

[1]https://github.com/ottogroup/flink-spector/wiki



--
View this message in context: 
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Unit-testing-support-for-flink-application-tp4130p4140.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at 
Nabble.com.

Re: Unit testing support for flink application?

[Filipe Correia]

Hi Stephan,

Yes, both generating the datastreams and verifying expectations. Is
the recommended way to create custom data sources and data sinks? I've
meanwhile started down this road, but still hoping for a better way.

Filipe

On Thu, Dec 31, 2015 at 3:20 PM, Stephan Ewen  wrote:
> Hi!
>
> Are you referring to testing streaming programs?
> What is the main obstacle for you? Generating test data streams?
>
> Thanks,
> Stephan
>
>
> On Thu, Dec 31, 2015 at 12:43 PM, Filipe Correia 
> wrote:
>>
>> Hi all,
>>
>> I'm finding it hard to unit test my Flink application. Are there any
>> guidelines / best practices for unit testing a Flink application,
>> especially for programming for the streaming API with Scala?
>>
>> Having a few good examples would also help a lot.
>>
>> I'm aware of flink-spector[1], and it looks great. Although the API is
>> not ready to be used from Scala yet, I've created my tests with Java
>> and managed to get it working for some very simple cases . But it does
>> still lack documentation / examples, so I'm having trouble using it
>> for most of the unit tests that I would like to create.
>>
>> Thanks,
>>
>> Filipe
>>
>> [1] https://github.com/ottogroup/flink-spector
>
>

Re: Unit testing support for flink application?

[Filipe Correia]

Hello Alex,

Thanks for the reply.

On Sat, Jan 2, 2016 at 2:03 PM, lofifnc  wrote:
> I'm currently updating and improving the documentation[1] of flink-spector.
> Regarding missing examples: I'm planning to include small examples showing
> the behaviour of output matchers, as the documentation already includes
> several demonstrating how to assemble test cases.

Sounds good, I would find those useful!

> Please let me know, if
> you're having problems with a particular aspect of the framework or the
> documentation or like to have an example for a special case. I will try to
> work it into the next version.

Ok. Like you said, I quickly found problems trying to test my scala
code, so I didn't get very far. Scalatest support will probably help
with most issues that I've found. I will be sure to try flinkspector
again when it arrives

> At the moment the framework is lacking support to tests the scala api.

Thanks for your work on this. It's something that I'm interested in,
and I'm sure that I'm not the only one!

Regards,

Filipe

Re: [ANNOUNCE] Introducing Apache Flink Taiwan User Group - Flink.tw

[Matthias J. Sax]

Pretty cool!

On 01/02/2016 10:30 AM, Fabian Hueske wrote:
> Hi Gordon,
> 
> this is great news! I am very happy to hear that there is a new local
> Flink community in Taiwan!
> Translating blog posts, slide sets, and other documentation is very
> valuable and makes the project known to a broader audience and
> accessible to more people.
> Please let us know, if you need help with anything.
> 
> Thanks for your efforts,
> Fabian
> 
> 2016-01-01 19:24 GMT+01:00 tzulitai  >:
> 
> Hi Flink community,
> 
> We are very excited to announce a new local Flink user group /
> meetup based
> in Taiwan:
> 
> Facebook group:  http://www.facebook.com/groups/flink.tw
> 
> Blog:  http://blog.flink.tw 
> 
> We're a group of Chinese-speaking people who finds Apache Flink truly
> amazing =)
> 
> The blog will be mainly used for accumulating translated information
> from
> Flink Forward, the official Apache Flink website, Data Artisan blogs
> etc.,
> as well as any content about Flink and related-technology originated
> from
> the blog's co-authors.
> 
> We will also be organizing meetups in Taipei! We're currently
> preparing for
> our first meetup.
> Please let us know if you're a Chinese speaker and would like to
> give a talk
> at our future meetups.
> 
> You can subscribe to the blog, or join our Facebook group for any latest
> news.
> The group is open to anyone who loves Flink too!
> 
> Thanks,
> Gordon
> 
> P.S. We have made a slight derivation to the Flink logo for our
> group's use.
> If the community finds this inappropriate, please let me know and I will
> adjust it immediately.
> 
> 
> 
> --
> View this message in context:
> 
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ANNOUNCE-Introducing-Apache-Flink-Taiwan-User-Group-Flink-tw-tp4136.html
> Sent from the Apache Flink User Mailing List archive. mailing list
> archive at Nabble.com.
> 
> 



signature.asc
Description: OpenPGP digital signature