Hi Caizhi,
Got it! Thanks for your clarification.
On Tue, Jan 18, 2022 at 11:45 PM Caizhi Weng wrote:
> Hi!
>
> "But flink application only has 10 task managers". I assume that you're
> talking about task slots instead of task managers.
>
> If there are more partitions than task slots, each tas
Hi Caizhi,
Thank you for your reply. The heap size is 512m. Fetching from the DB table
is the only costly operation. After fetching from DB, I simply ingested a
kafka topic. That should not be the bottleneck.
Here is the jdbc configuration. Is that correct config?
val query = String.format("SELEC
Hi!
This is not the desired behavior. As you have set fetchSize to 20 there
will be only 20 records in each parallelism of the source. How large is
your heap size? Does your job have any other operations which consume a lot
of heap memory?
Qihua Yang 于2022年1月19日周三 15:27写道：
> Here is the errors
Hi!
"But flink application only has 10 task managers". I assume that you're
talking about task slots instead of task managers.
If there are more partitions than task slots, each task slot may be
assigned more than one partition. By default this is a first come first
served assignment. After a tas
Here is the errors
Exception: java.lang.OutOfMemoryError thrown from the
UncaughtExceptionHandler in thread "server-timer"
Exception: java.lang.OutOfMemoryError thrown from the
UncaughtExceptionHandler in thread "I/O dispatcher 16"
Exception: java.lang.OutOfMemoryError thrown from the
UncaughtExcep
Hi,
I have a flink cluster(50 hosts, each host runs a task manager).
I am using Flink JDBC to consume data from a database. The db table is
pretty large, around 18734 rows. I configured the JDBC number of
partitions to 50. fetchSize is 20. Flink application has 50 task managers.
Anyone know wh
Hi,
I plan to use the database as flink source by using flink JDBC. I know
setNumPartitions can be used for parallelism in table reading.
If the num of task managers is less than numPartitions, what is the
behavior?
For example:
I config setNumPartitions(20), but flink application only has 10 task
Hi,Mason
I assign the jira to you.
Thanks for your contribution.
Best,
Guowei
On Wed, Jan 19, 2022 at 2:07 PM Mason Chen wrote:
> Hi all,
>
> There is some interest from our users to use prometheus push gateway
> reporter with a https endpoint. So, I've filed
> https://issues.apache.org/jira/br
Hi all,
There is some interest from our users to use prometheus push gateway reporter
with a https endpoint. So, I've filed
https://issues.apache.org/jira/browse/FLINK-25697 and I figured that it would
be acceptable since influxdb reporter supports something similar. Could someone
assign me th
Hi, summer
>>>Now I need to use a third-party jar in the Flink service, should I put
it under ${FLINK_HOME}/lib?
I think maybe an alternative way is to put the third-party jar into a fat
jar.
>>>How to enable Flink to automatically load third-party jars？
In general this is the JVM mechanism. It m
Hello
Now I need to use a third-party jar in the Flink service, should I put
it under ${FLINK_HOME}/lib? How to enable Flink to automatically load
third-party jars？
TH
I think I may know what is causing the issue... So I have 3 job managers.
1- I Navigated to a non leader UI and submitted a new job...
2- The UI timed out with grey lines
3- Some Internal Server error messages appeared.
4- Going back to the leader UI checking the running jobs, the job seems to
hav
Sorry, unhandled exception.
On Tue, 18 Jan 2022 at 15:16, John Smith wrote:
> Also that I handled exception is printed in job managers.
>
> On Tue., Jan. 18, 2022, 10:11 a.m. John Smith,
> wrote:
>
>> I actually mean the job manager. I run a total of three job managers for
>> HA.
>>
>> For exam
Also that I handled exception is printed in job managers.
On Tue., Jan. 18, 2022, 10:11 a.m. John Smith,
wrote:
> I actually mean the job manager. I run a total of three job managers for
> HA.
>
> For example of I click running jobs, it displays light grey boxes for a
> while and then top right
Hi Jessy,
Which approach is suitable for a standalone deployment in Kubernetes? Do we
> have some best practises for running Flink applications on K8s ?
I would deploy Flink in Application Mode using the standalone K8s
deployment:
https://nightlies.apache.org/flink/flink-docs-master/docs/deploym
Hi Razin,
No discussion thread has been started yet for a next Flink release for the
1.13 branch on the Dev mailing list. That means that it's currently not
known if and when this will be released. You can always ask on the Dev
mailing list if there's interest to create a new 1.13 release.
Best r
Thanks for your reply.
We can try this on the CLI, but what about the Web UI?
Thanks,
Jerry
---Original---
From: "Gabor Somogyi"
Hello Martijn,
When can we expect a release of version 1.13.6 with log4j 2.17.1?
Thank you,
Razin
On Tue, Jan 18, 2022 at 10:04 AM Martijn Visser
wrote:
> Hi Deepti,
>
> The announcement has not been made official yet, it's pending the
> publication of a couple of things. I expect it today or
I actually mean the job manager. I run a total of three job managers for HA.
For example of I click running jobs, it displays light grey boxes for a
while and then top right corner throws internal server error. But after if
I refresh it's ok I see the list. It seems to happen on the non leader job
Hi Deepti,
The announcement has not been made official yet, it's pending the
publication of a couple of things. I expect it today or at latest tomorrow.
Best regards,
Martijn
On Tue, 18 Jan 2022 at 15:21, Deepti Sharma S
wrote:
> Hello Martijn,
>
>
>
> Can you please confirm if Flink version
Hello Martijn,
Can you please confirm if Flink version 1.14.3 is released? I can see the
package on Github, under Tags. However not published on Flink website?
Regards,
Deepti Sharma
PMP® & ITIL
From: Martijn Visser
Sent: Friday, January 7, 2022 1:21 PM
To: Deepti Sharma S
Cc: user@flink.ap
Hi Alexis,
I believe you should be able to use the `ConnectedStreams#transform()`
method.
Best, Piotrek
wt., 18 sty 2022 o 14:20 Alexis Sarda-Espinosa <
alexis.sarda-espin...@microfocus.com> napisał(a):
> Hi again everyone,
>
>
>
> It’s been a while, so first of all happy new year :)
>
>
>
> I
Hi Team,
Any insights for below mail will be helpful.
Thanks
Jessy
On Fri, Jan 14, 2022, 11:09 PM Jessy Ping
wrote:
> Hi Team,
>
> We are planning to run the below pipeline as a standalone Flink
> application cluster on kubernetes. It will be better if the community can
> share their insights r
Hi again everyone,
It’s been a while, so first of all happy new year :)
I was revisiting this discussion and started looking at the code. However, it
seems that all of the overloads of ConnectedStreams#process expect a
CoProcessFunction or the Keyed counterpart, so I don’t think I can inject a
Hey Yang,
Thank you for confirming it.
IMO, a better approach is to change the order "log_setting" , "ARGS" and
"FLINK_ENV_JAVA_OPTS" in exec command.
In that way we prioritize user defined properties.
From:
exec "$JAVA_RUN" $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}"
-classpath "`ma
export SSL_PASSWORD=secret
flink run -yDsecurity.ssl.rest.*-password=$SSL_PASSWORD ... app.jar
Such way the code which starts the workload can store the passwords in a
centrally protected area.
This still can be hacked but at least not stored in plain text file.
BR,
G
On Tue, Jan 18, 2022 at 10
Hello,
security.ssl.rest.*-password and s3.secret.key are parameters in flink-conf.xml
that can only be set with plain text, which may lead to password leak. Can
anyone provide some other extended capabilities that allow developers to
configure cipher text and specify classes and methods to decr
27 matches
Mail list logo
