mattcasters opened a new pull request, #7377:
URL: https://github.com/apache/hop/pull/7377
# Generalized Variable Scopes Walkthrough
We have generalized and hardened variable scope propagation across
nested/mixed execution hierarchies (Workflow -> Pipeline -> Workflow) in Apache
Hop for issue #3232
## Changes Made
### 1. Unified Parent Engine Crawling
*
**[SetVariable.java](file:///home/matt/git/mattcasters/hop/plugins/transforms/setvariable/src/main/java/org/apache/hop/pipeline/transforms/setvariable/SetVariable.java)**:
* Implemented `getParentEngine(IVariables engine)` to handle both
`IPipelineEngine` and `IWorkflowEngine` parent lookup dynamically.
* Replaced pipeline-only and workflow-only loops with a unified crawled
list of all parent engines in the execution tree.
*
**[ActionSetVariables.java](file:///home/matt/git/mattcasters/hop/plugins/actions/setvariables/src/main/java/org/apache/hop/workflow/actions/setvariables/ActionSetVariables.java)**:
* Implemented the same unified parent crawl logic and updated scope
handling (JVM, ROOT_WORKFLOW, PARENT_WORKFLOW) to propagate variables correctly
across alternating scopes.
### 2. Runtime Variable Inheritance for Sub-engines
To ensure that sub-pipelines and sub-workflows inherit dynamically updated
variables from parent pipeline execution contexts, we modified the creation and
initialization of child engines to reference `getPipeline()` (current pipeline
instance) instead of `this` (startup-time frozen transform variables):
*
**[WorkflowExecutor.java](file:///home/matt/git/mattcasters/hop/plugins/transforms/workflowexecutor/src/main/java/org/apache/hop/pipeline/transforms/workflowexecutor/WorkflowExecutor.java)**
*
**[PipelineExecutor.java](file:///home/matt/git/mattcasters/hop/plugins/transforms/pipelineexecutor/src/main/java/org/apache/hop/pipeline/transforms/pipelineexecutor/PipelineExecutor.java)**
*
**[SimpleMapping.java](file:///home/matt/git/mattcasters/hop/plugins/transforms/mapping/src/main/java/org/apache/hop/pipeline/transforms/mapping/SimpleMapping.java)**
### 3. User-Facing Wording
* Updated i18n English resource files to use generic wording (e.g. "Valid in
the parent" and "Valid in the root") while maintaining full backward
compatibility in serialized XML metadata:
* **[messages_en_US.properties
(SetVariable)](file:///home/matt/git/mattcasters/hop/plugins/transforms/setvariable/src/main/resources/org/apache/hop/pipeline/transforms/setvariable/messages/messages_en_US.properties)**
* **[messages_en_US.properties
(ActionSetVariables)](file:///home/matt/git/mattcasters/hop/plugins/actions/setvariables/src/main/resources/org/apache/hop/workflow/actions/setvariables/messages/messages_en_US.properties)**
### 4. Integration Tests Created & Modified
We added tests covering both old flat workflow behavior and new nested/mixed
hierarchy capabilities:
* **Transform SetVariables (0064 group)**:
*
[0064-set-variables-child-pipeline.hpl](file:///home/matt/git/mattcasters/hop/integration-tests/transforms/0064-set-variables-child-pipeline.hpl)
*
[0064-set-variables-child-workflow.hwf](file:///home/matt/git/mattcasters/hop/integration-tests/transforms/0064-set-variables-child-workflow.hwf)
*
[0064-set-variables-parent-pipeline.hpl](file:///home/matt/git/mattcasters/hop/integration-tests/transforms/0064-set-variables-parent-pipeline.hpl)
*
[0064-set-variables-verify-nested.hpl](file:///home/matt/git/mattcasters/hop/integration-tests/transforms/0064-set-variables-verify-nested.hpl)
*
[main-0064-set-variables.hwf](file:///home/matt/git/mattcasters/hop/integration-tests/transforms/main-0064-set-variables.hwf)
*(modified to chain the new nested verification)*
* **Action SetVariables (0018 group)**:
*
[main-0018-set-variables.hwf](file:///home/matt/git/mattcasters/hop/integration-tests/actions/main-0018-set-variables.hwf)
*
[0018-set-variables-pipeline.hpl](file:///home/matt/git/mattcasters/hop/integration-tests/actions/0018-set-variables-pipeline.hpl)
*
[0018-set-variables-nested-workflow.hwf](file:///home/matt/git/mattcasters/hop/integration-tests/actions/0018-set-variables-nested-workflow.hwf)
*
[0018-set-variables-verify.hpl](file:///home/matt/git/mattcasters/hop/integration-tests/actions/0018-set-variables-verify.hpl)
---
## Verification Results
1. **Compilation and Unit Tests**:
- Executed Maven tests on all modified plugins:
```bash
./mvnw test -pl
plugins/transforms/setvariable,plugins/transforms/workflowexecutor,plugins/transforms/pipelineexecutor,plugins/transforms/mapping,plugins/actions/setvariables
```
- Result: **BUILD SUCCESS**, all tests passed.
2. **Formatting Compliance Check**:
- Spotless check ran on all modified plugins:
```bash
./mvnw spotless:check -pl
plugins/transforms/setvariable,plugins/transforms/workflowexecutor,plugins/transforms/pipelineexecutor,plugins/transforms/mapping,plugins/actions/setvariables
```
- Result: **BUILD SUCCESS**, formatting is fully clean.
3. **Integration tests**:
- Ran transforms integration test 0064 (with success)
- Ran actions integration test 0018 (with success)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
