nicusX commented on code in PR #28863: URL: https://github.com/apache/flink/pull/28863#discussion_r3820958758
########## docs/content/docs/concepts/glossary.md: ########## @@ -72,165 +167,363 @@ Java, this corresponds to the definition of *Instance* or *Object* in Java. In t Flink, the term *parallel instance* is also frequently used to emphasize that multiple instances of the same [Operator](#operator) or [Function](#function) type are running in parallel. -#### Flink Application +#### Flink Job -A Flink application is a Java Application that submits one or multiple [Flink -Jobs](#flink-job) from the `main()` method (or by some other means). Submitting -jobs is usually done by calling `execute()` on an execution environment. +A Flink Job is the unit of data processing execution in Flink: a Job as a whole is submitted, +started, stopped and resumed, although under some conditions Flink may restart a Job only partially. -The jobs of an application can either be submitted to a long running [Flink -Session Cluster](#flink-session-cluster), to a dedicated [Flink Application -Cluster](#flink-application-cluster), or to a [Flink Job -Cluster](#flink-job-cluster). +A Job is submitted either by a [Flink Application](#flink-application), by calling `execute()` on an +execution environment, or as a single [Flink SQL Statement](#flink-sql-statement) or [Statement +Set](#statement-set). -#### Flink Job +A Flink Job is the runtime representation of a [Logical Graph](#logical-graph) (also often called +*Dataflow Graph*). The Logical Graph is optimized into a [Job Graph](#job-graph), from which the +[Physical Graph](#physical-graph) that actually runs in a [Flink Cluster](#flink-cluster) is derived. -A Flink Job is the runtime representation of a [logical graph](#logical-graph) -(also often called dataflow graph) that is created and submitted by calling -`execute()` in a [Flink Application](#flink-application). +#### Flink Job Cluster -#### JobGraph +A Flink Job Cluster is a dedicated [Flink Cluster](#flink-cluster) that only +executes a single [Flink Job](#flink-job). The lifetime of the +[Flink Cluster](#flink-cluster) is bound to the lifetime of the Flink Job. +This deployment mode has been deprecated since Flink 1.15. -see [Logical Graph](#logical-graph) +#### Job Graph + +Also *JobGraph*. + +A Job Graph is the optimized representation of a [Logical Graph](#logical-graph), and the +representation that a [Flink Application](#flink-application) submits to the [Flink +Cluster](#flink-cluster). + +Producing the Job Graph is mainly a matter of chaining: consecutive [Operators](#operator) that are +not separated by a repartitioning are merged into a single [Task](#task). The nodes of a Job Graph +are therefore [Tasks](#task), each implementing one Operator or one [Operator +Chain](#operator-chain). + +The Job Graph is translated into a [Physical Graph](#physical-graph) for execution. + +Job Graph is sometimes referred to as *Optimized Dataflow*. #### Flink JobManager -The JobManager is the orchestrator of a [Flink Cluster](#flink-cluster). It contains three distinct -components: Flink Resource Manager, Flink Dispatcher and one [Flink JobMaster](#flink-jobmaster) -per running [Flink Job](#flink-job). +Also *Job Manager*. + +The JobManager is the orchestrator of a [Flink Cluster](#flink-cluster). It does not process any +data itself: it translates the submitted [Job Graph](#job-graph) into a [Physical +Graph](#physical-graph), schedules the resulting [Sub-Tasks](#sub-task) on the +[TaskManagers](#flink-taskmanager), and coordinates [Checkpoints](#checkpoint) and +[Savepoints](#savepoint). It contains three distinct components: Flink Resource Manager, Flink +Dispatcher and one [Flink JobMaster](#flink-jobmaster) per running [Flink Job](#flink-job). #### Flink JobMaster JobMasters are one of the components running in the [JobManager](#flink-jobmanager). A JobMaster is -responsible for supervising the execution of the [Tasks](#task) of a single job. +responsible for supervising the execution of the [Sub-Tasks](#sub-task) of a single Job. It derives +the [Physical Graph](#physical-graph) from the Job's [Job Graph](#job-graph), requests the slots +needed to run it, deploys the Sub-Tasks to the [TaskManagers](#flink-taskmanager), and triggers the +Job's [Checkpoints](#checkpoint). #### JobResultStore The JobResultStore is a Flink component that persists the results of globally terminated -(i.e. finished, cancelled or failed) jobs to a filesystem, allowing the results to outlive -a finished job. Each result contains the job's identifier, final state, name, the application it -belongs to, etc. These results are then used by Flink to determine whether jobs should -be subject to recovery in highly-available clusters. - -#### ApplicationResultStore +(i.e. finished, cancelled or failed) Jobs to a filesystem, allowing the results to outlive +a finished Job. Each result contains the Job's identifier, final state, name, the Application it +belongs to, etc. These results are then used by Flink to determine whether Jobs should +be subject to recovery in highly-available Clusters. -The ApplicationResultStore is a Flink component that persists the results of terminated -(i.e. finished, cancelled or failed) applications to a filesystem, allowing the results to outlive -a terminated application. Each result contains the application's identifier, final state, name, -etc. These results are then used by Flink to determine whether applications should -be subject to recovery in highly-available clusters. +#### Key Group -#### History Server +A Key Group is the atomic unit of key distribution and state assignment across parallel Review Comment: @alpinegizmo I added Key Group, as you suggested. I deliberately omitted saying that this why changing `maxParallelism` breaks the ability to restore from a snapshot. It looked too much for a Glossary -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
