[GitHub] [flink] haf edited a comment on issue #6594: [FLINK-9311] [pubsub] Added PubSub source connector with support for checkpointing (ATLEAST_ONCE)

[GitBox]

haf edited a comment on issue #6594: [FLINK-9311] [pubsub] Added PubSub source 
connector with support for checkpointing (ATLEAST_ONCE)
URL: https://github.com/apache/flink/pull/6594#issuecomment-483549547
 
 
   @becketqin Hi! Any progress?
   
   @Xeli Can you e-mail me the other connector you've build for yourself, 
pretty please? henrik - at - haf dot se.
   
   Also @Xeli I'm successfully running this locally in the IDE, but when I 
submit it to a local "cluster" of a single TM and a single JM, I'm getting 
(tensor is the name of my laptop):
   
   ```
   ==> 
/usr/local/Cellar/apache-flink/1.7.2/libexec/log/flink-h-taskexecutor-1-tensor.out
 <==
        at 
com.google.api.gax.rpc.UnaryCallable$1.futureCall(UnaryCallable.java:126)
        at 
com.google.api.gax.rpc.UnaryCallable.futureCall(UnaryCallable.java:87)
        at okr.sources.PubSubSource.run(PubSubSource.java:126)
        at 
org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:94)
        at 
org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:58)
        at 
org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:99)
        at 
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:300)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:704)
        at java.lang.Thread.run(Thread.java:748)
   ```
   I think something is swallowing the exception. IRL I think it was an 
IllegalStateException caused by not including `META-INF/services` with 
   
   <img width="1053" alt="image" 
src="https://user-images.githubusercontent.com/193115/56228901-043e1e80-6079-11e9-969d-46266c662af4.png";>
   
   It ends up in the running state, without consuming any data, and with logs 
like this:
   
   ```
   ==> 
/usr/local/Cellar/apache-flink/1.7.2/libexec/log/flink-h-standalonesession-1-tensor.log
 <==
   2019-04-16 09:42:27,232 INFO  
org.apache.flink.runtime.checkpoint.CheckpointCoordinator     - Triggering 
checkpoint 93 @ 1555400547232 for job 52dae7a7fa6e9caec8e34f2f837f2e97.
   2019-04-16 09:42:27,243 INFO  
org.apache.flink.runtime.checkpoint.CheckpointCoordinator     - Completed 
checkpoint 93 for job 52dae7a7fa6e9caec8e34f2f837f2e97 (83957 bytes in 11 ms).
   2019-04-16 09:42:28,231 INFO  
org.apache.flink.runtime.checkpoint.CheckpointCoordinator     - Triggering 
checkpoint 94 @ 1555400548231 for job 52dae7a7fa6e9caec8e34f2f837f2e97.
   ```
   
   It also says:
   
   > 2019-04-16 09:40:54,419 INFO  
org.apache.flink.streaming.api.functions.source.MessageAcknowledgingSourceBase  
- No state to restore for the PubSubSource.
   
   Is that correct? Isn't the previously used checkpoint used, or would I need 
to start ZK for that?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services