[
https://issues.apache.org/jira/browse/FLINK-39947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18094137#comment-18094137
]
Weiqing Yang commented on FLINK-39947:
--------------------------------------
Took a look at the linked CI run (AdaptiveScheduler / Test, module {{core}}).
The failure signature is worth separating from a normal test failure:
{code}
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 239
[ERROR] Crashed tests:
[ERROR] org.apache.flink.runtime.taskexecutor.TaskExecutorPartitionLifecycleTest
{code}
A few observations from the log:
* This is a forked-JVM termination ({{reuseForks=true}},
{{forkCountUnitTest=4}}, {{-Xmx768m}}), not an assertion or error inside a test
method — the fork died at the VM level while running {{jvmRun3}}.
* {{TaskExecutorPartitionLifecycleTest}} never reports completion anywhere in
the log — there is no {{Tests run: ... -- in
...TaskExecutorPartitionLifecycleTest}} line. Surefire lists it under "Crashed
tests" because it was the class assigned to the fork that died, which does not
by itself mean the crash originated in this class.
* Immediately before the termination, a burst of unrelated classes
({{EdgeManagerBuildUtilTest}} ... {{SingleJobApplicationTest}}) log "Running"
with no matching completion lines — the buffered stdout of the dying fork being
flushed.
I also checked the class for a resource leak that could accumulate over a
reused fork: the {{NettyShuffleEnvironment}} instances it builds are all closed
via {{RpcUtils.terminateRpcEndpoint(taskExecutor)}} ->
{{TaskManagerServices.shutDown()}}, and there's no {{System.exit}} on the
exercised paths. So I don't see an obvious logic bug or leak in the test itself.
Given that, the evidence points to a VM-level fork crash (resource pressure on
the runner being the most likely trigger) with this class as the assigned
victim, rather than a defect in {{TaskExecutorPartitionLifecycleTest}}. I also
don't see it recurring — this appears to be the only ticket referencing a
"forked VM terminated" crash in a while, from the single run above.
FLINK-39414 / [PR #27930|https://github.com/apache/flink/pull/27930] separately
reduces the long ({{15s}}/{{30s}}) {{willNotCompleteWithin}} windows in
{{testEnableBatchJobRecoveryAnd*}} — that lowers how long this class occupies a
reused fork, though it wouldn't address a VM crash on its own.
Leaving these notes here as context for future reference.
> Fix the flaky test case in class TaskExecutorPartitionLifecycleTest
> -------------------------------------------------------------------
>
> Key: FLINK-39947
> URL: https://issues.apache.org/jira/browse/FLINK-39947
> Project: Flink
> Issue Type: Bug
> Components: Tests
> Reporter: Yuepeng Pan
> Priority: Major
> Fix For: 2.4.0
>
>
> [https://github.com/apache/flink/actions/runs/27591116646/job/81577296085]
> {code:java}
> Error: 04:05:20 04:05:20.118 [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:3.2.2:test (default-test) on
> project flink-runtime:
> Error: 04:05:20 04:05:20.118 [ERROR]
> Error: 04:05:20 04:05:20.118 [ERROR] Please refer to
> /root/flink/flink-runtime/target/surefire-reports for the individual test
> results.
> Error: 04:05:20 04:05:20.118 [ERROR] Please refer to dump files (if any
> exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
> Error: 04:05:20 04:05:20.118 [ERROR] ExecutionException The forked VM
> terminated without properly saying goodbye. VM crash or System.exit called?
> Error: 04:05:20 04:05:20.118 [ERROR] Command was /bin/sh -c cd
> '/root/flink/flink-runtime' && '/usr/lib/jvm/jdk-17.0.7+7/bin/java'
> '-XX:+UseG1GC' '-Xms256m' '-XX:+IgnoreUnrecognizedVMOptions'
> '--add-opens=java.base/java.util=ALL-UNNAMED'
> '--add-opens=java.base/java.lang=ALL-UNNAMED'
> '--add-opens=java.base/java.net=ALL-UNNAMED'
> '--add-opens=java.base/java.io=ALL-UNNAMED'
> '--add-opens=java.base/java.util.concurrent=ALL-UNNAMED'
> '-Djunit.platform.reflection.search.useLegacySemantics=true' '-Xmx768m'
> '-jar'
> '/root/flink/flink-runtime/target/surefire/surefirebooter-20260616035942380_81.jar'
> '/root/flink/flink-runtime/target/surefire'
> '2026-06-16T03-57-55_665-jvmRun3' 'surefire-20260616035942380_78tmp'
> 'surefire_23-20260616035942380_80tmp'
> Error: 04:05:20 04:05:20.118 [ERROR] Error occurred in starting fork, check
> output in log
> Error: 04:05:20 04:05:20.118 [ERROR] Process Exit Code: 239
> Error: 04:05:20 04:05:20.118 [ERROR] Crashed tests:
> Error: 04:05:20 04:05:20.118 [ERROR]
> org.apache.flink.runtime.taskexecutor.TaskExecutorPartitionLifecycleTest
> Error: 04:05:20 04:05:20.118 [ERROR]
> org.apache.maven.surefire.booter.SurefireBooterForkException:
> ExecutionException The forked VM terminated without properly saying goodbye.
> VM crash or System.exit called?
> Error: 04:05:20 04:05:20.118 [ERROR] Command was /bin/sh -c cd
> '/root/flink/flink-runtime' && '/usr/lib/jvm/jdk-17.0.7+7/bin/java'
> '-XX:+UseG1GC' '-Xms256m' '-XX:+IgnoreUnrecognizedVMOptions'
> '--add-opens=java.base/java.util=ALL-UNNAMED'
> '--add-opens=java.base/java.lang=ALL-UNNAMED'
> '--add-opens=java.base/java.net=ALL-UNNAMED'
> '--add-opens=java.base/java.io=ALL-UNNAMED'
> '--add-opens=java.base/java.util.concurrent=ALL-UNNAMED'
> '-Djunit.platform.reflection.search.useLegacySemantics=true' '-Xmx768m'
> '-jar'
> '/root/flink/flink-runtime/target/surefire/surefirebooter-20260616035942380_81.jar'
> '/root/flink/flink-runtime/target/surefire'
> '2026-06-16T03-57-55_665-jvmRun3' 'surefire-20260616035942380_78tmp'
> 'surefire_23-20260616035942380_80tmp'
> Error: 04:05:20 04:05:20.118 [ERROR] Error occurred in starting fork, check
> output in log
> Error: 04:05:20 04:05:20.118 [ERROR] Process Exit Code: 239
> Error: 04:05:20 04:05:20.118 [ERROR] Crashed tests:
> Error: 04:05:20 04:05:20.118 [ERROR]
> org.apache.flink.runtime.taskexecutor.TaskExecutorPartitionLifecycleTest
> Error: 04:05:20 04:05:20.118 [ERROR] at
> org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:456)
> Error: 04:05:20 04:05:20.118 [ERROR] at
> org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkOnceMultiple(ForkStarter.java:358)
> Error: 04:05:20 04:05:20.118 [ERROR] at
> org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:296)
> Error: 04:05:20 04:05:20.118 [ERROR] at
> org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:250)
> Error: 04:05:20 04:05:20.118 [ERROR] at
> org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1240)
> Error: 04:05:20 04:05:20.118 [ERROR] at
> org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1089)
> Error: 04:05:20 04:05:20.118 [ERROR] at
> org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:905)
> Error: 04:05:20 04:05:20.118 [ERROR] at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
> Error: 04:05:20 04:05:20.118 [ERROR] at
> org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2(MojoExecutor.java:370)
> Error: 04:05:20 04:05:20.118 [ERROR] at
> org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:351)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:215)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:171)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:163)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:294)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.cli.MavenCli.execute(MavenCli.java:960)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.cli.MavenCli.doMain(MavenCli.java:293)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.cli.MavenCli.main(MavenCli.java:196)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> java.base/java.lang.reflect.Method.invoke(Method.java:568)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> java.base/java.lang.reflect.Method.invoke(Method.java:568)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:52)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.wrapper.WrapperExecutor.execute(WrapperExecutor.java:161)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.wrapper.MavenWrapperMain.main(MavenWrapperMain.java:73)
> Error: 04:05:20 04:05:20.119 [ERROR] Caused by:
> org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM
> terminated without properly saying goodbye. VM crash or System.exit called?
> Error: 04:05:20 04:05:20.119 [ERROR] Command was /bin/sh -c cd
> '/root/flink/flink-runtime' && '/usr/lib/jvm/jdk-17.0.7+7/bin/java'
> '-XX:+UseG1GC' '-Xms256m' '-XX:+IgnoreUnrecognizedVMOptions'
> '--add-opens=java.base/java.util=ALL-UNNAMED'
> '--add-opens=java.base/java.lang=ALL-UNNAMED'
> '--add-opens=java.base/java.net=ALL-UNNAMED'
> '--add-opens=java.base/java.io=ALL-UNNAMED'
> '--add-opens=java.base/java.util.concurrent=ALL-UNNAMED'
> '-Djunit.platform.reflection.search.useLegacySemantics=true' '-Xmx768m'
> '-jar'
> '/root/flink/flink-runtime/target/surefire/surefirebooter-20260616035942380_81.jar'
> '/root/flink/flink-runtime/target/surefire'
> '2026-06-16T03-57-55_665-jvmRun3' 'surefire-20260616035942380_78tmp'
> 'surefire_23-20260616035942380_80tmp'
> Error: 04:05:20 04:05:20.119 [ERROR] Error occurred in starting fork, check
> output in log
> Error: 04:05:20 04:05:20.119 [ERROR] Process Exit Code: 239
> Error: 04:05:20 04:05:20.119 [ERROR] Crashed tests:
> Error: 04:05:20 04:05:20.119 [ERROR]
> org.apache.flink.runtime.taskexecutor.TaskExecutorPartitionLifecycleTest
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:643)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> org.apache.maven.plugin.surefire.booterclient.ForkStarter.lambda$runSuitesForkOnceMultiple$3(ForkStarter.java:350)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> Error: 04:05:20 04:05:20.119 [ERROR] at
> java.base/java.lang.Thread.run(Thread.java:833)
> Error: 04:05:20 04:05:20.119 [ERROR] -> [Help 1]
> Error: 04:05:20 04:05:20.119 [ERROR]
> Error: 04:05:20 04:05:20.119 [ERROR] To see the full stack trace of the
> errors, re-run Maven with the -e switch.
> Error: 04:05:20 04:05:20.119 [ERROR] Re-run Maven using the -X switch to
> enable full debug logging.
> Error: 04:05:20 04:05:20.119 [ERROR]
> Error: 04:05:20 04:05:20.119 [ERROR] For more information about the errors
> and possible solutions, please read the following articles:
> Error: 04:05:20 04:05:20.119 [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> Error: 04:05:20 04:05:20.119 [ERROR]
> Error: 04:05:20 04:05:20.119 [ERROR] After correcting the problems, you can
> resume the build with the command {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
