Hi Sun, As Dulce said, running in a cluster is typically recommended. However if for some reason you need to run in standalone mode, why do you recreate Cluster on each job, can you try to reuse the MiniCluster?
I happen to have a similar setup when we are running in standalone mode, and run multiple batch job throughout the day, I just initialize the cluster once at beginning of process and reuse it Best -----Original Message----- From: Jiabao Sun <jiabao....@xtransfer.cn> Sent: 18 January 2024 02:46 To: user@flink.apache.org Subject: RE: Flink Slow Execution Hi Dulce, MiniCluster is generally used for local testing and is limited by the resources of a single machine. When more tasks are executed, it may not be able to immediately acquire the resources needed to start the MiniCluster, resulting in slower startup times. If running Flink tasks in a production environment, it is recommended to use cluster deployment mode[1]. You can also use resource providers like Yarn[2] or Kubernetes[3]. Best, Jiabao [1] https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/deployment/resource-providers/standalone/overview/ [2] https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/deployment/resource-providers/yarn/ [3] https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/ On 2024/01/17 21:11:56 Dulce Morim wrote: > Hello, > > In a single JVM, I'm running multiple flink batch jobs locally using the > MiniCluster (Java 17 and Flink 1.18). > > At the beginning of the process, the Mini Cluster starts pretty much > instantly. However, I'm running into an issue where the more jobs I execute > the longer the MiniCluster takes to start. > > Here's an example: > > 2024-01-17 17:07:26.989 [INFO ] MiniCluster - Starting Flink Mini Cluster > 2024-01-17 17:07:27.165 [INFO ] MiniCluster - Starting Metrics Registry > 2024-01-17 17:07:33.801 [INFO ] MetricRegistryImpl - No metrics reporter > configured, no metrics will be exposed/reported. > 2024-01-17 17:07:33.801 [INFO ] MiniCluster - Starting RPC Service(s) > 2024-01-17 17:07:34.646 [INFO ] MiniCluster - Flink Mini Cluster started > successfully > > Has anyone faced this issue? > > Thanks. > This e-mail and any attachments are confidential to the addressee(s) and may contain information that is legally privileged and/or confidential. If you are not the intended recipient of this e-mail you are hereby notified that any dissemination, distribution, or copying of its content is strictly prohibited. If you have received this message in error, please notify the sender by return e-mail and destroy the message and all copies in your possession. To find out more details about how we may collect, use and share your personal information, please see https://www.mwam.com/privacy-policy. This includes details of how calls you make to us may be recorded in order for us to comply with our legal and regulatory obligations. To the extent that the contents of this email constitutes a financial promotion, please note that it is issued only to and/or directed only at persons who are professional clients or eligible counterparties as defined in the FCA Rules. Any investment products or services described in this email are available only to professional clients and eligible counterparties. Persons who are not professional clients or eligible counterparties should not rely or act on the contents of this email. Marshall Wace LLP is authorised and regulated by the Financial Conduct Authority. Marshall Wace LLP is a limited liability partnership registered in England and Wales with registered number OC302228 and registered office at George House, 131 Sloane Street, London, SW1X 9AT. If you are receiving this e-mail as a client, or an investor in an investment vehicle, managed or advised by Marshall Wace North America L.P., the sender of this e-mail is communicating with you in the sender's capacity as an associated or related person of Marshall Wace North America L.P. (“MWNA”), which is registered with the US Securities and Exchange Commission (“SEC”) as an investment adviser. Registration with the SEC does not imply that MWNA or its employees possess a certain level of skill or training.
