[
https://issues.apache.org/jira/browse/NIFI-16353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sevag Keosseian updated NIFI-16353:
-----------------------------------
Description:
StandardConnectorTestRunner uses shared mutable directories under target/work,
while its NiFi properties use shared repository and storage paths under
target/nifi-storage.
When multiple runners execute concurrently, they create separate mock NiFi
instances but can write to the same:
- FlowFile, content, database, and NAR repositories
- WAL and state files
- Assets and connector assets
- Unpacked NAR working directories
- Jetty working directories
This can cause cross-writes, repository corruption, startup failures, or one
runner’s shutdown affecting another runner.
NIFI-16183 increased the graceful shutdown timeout, which widened the collision
window but did not cause the underlying shared-filesystem problem. Its shutdown
behavior should remain unchanged.
Proposed behavior
Add an optional API:
```
StandardConnectorTestRunner.Builder.instanceDirectory(Path)
```
When supplied, all mutable NiFi paths must be derived from that instance root.
The configured NAR library directory remains shared and read-only.
When no instance directory is supplied, preserve the existing directory
behavior for backward compatibility.
Continue using OS-assigned HTTP ports by default.
Caller-provided instance directories remain caller-owned and must not be
deleted when the runner closes or bootstrap fails.
Acceptance criteria
- Separate instance roots isolate all mutable paths.
- The NAR library directory remains shared.
- Two runners can start and stop concurrently without cross-writing.
- Closing one runner does not affect the other.
- Caller-provided directories are retained after shutdown and bootstrap
failure.
- Existing builder APIs remain compatible.
- Default port selection remains OS-assigned.
- Existing behavior is preserved when instanceDirectory is omitted.
- NIFI-16183 shutdown timeout behavior is unchanged.
Verification
```bash
./mvnw \
-pl
nifi-connector-mock-bundle/nifi-connector-mock-test-bundle/nifi-connector-mock-integration-tests
\
-am clean verify \
-DforkCount=2 \
-DreuseForks=true
```
Implementation branch:
[https://github.com/skeossei/nifi/tree/standard-connector-runner-isolation]
was:
StandardConnectorTestRunner uses shared mutable directories under target/work,
while its NiFi properties use shared repository and storage paths under
target/nifi-storage.
When multiple runners execute concurrently, they create separate mock NiFi
instances but can write to the same:
- FlowFile, content, database, and NAR repositories
- WAL and state files
- Assets and connector assets
- Unpacked NAR working directories
- Jetty working directories
This can cause cross-writes, repository corruption, startup failures, or one
runner’s shutdown affecting another runner.
NIFI-16183 increased the graceful shutdown timeout, which widened the collision
window but did not cause the underlying shared-filesystem problem. Its shutdown
behavior should remain unchanged.
Proposed behavior
Add an optional API:
```java
StandardConnectorTestRunner.Builder.instanceDirectory(Path)
```
When supplied, all mutable NiFi paths must be derived from that instance root.
The configured NAR library directory remains shared and read-only.
When no instance directory is supplied, preserve the existing directory
behavior for backward compatibility.
Continue using OS-assigned HTTP ports by default.
Caller-provided instance directories remain caller-owned and must not be
deleted when the runner closes or bootstrap fails.
Acceptance criteria
- Separate instance roots isolate all mutable paths.
- The NAR library directory remains shared.
- Two runners can start and stop concurrently without cross-writing.
- Closing one runner does not affect the other.
- Caller-provided directories are retained after shutdown and bootstrap failure.
- Existing builder APIs remain compatible.
- Default port selection remains OS-assigned.
- Existing behavior is preserved when instanceDirectory is omitted.
- NIFI-16183 shutdown timeout behavior is unchanged.
Verification
```bash
./mvnw \
-pl
nifi-connector-mock-bundle/nifi-connector-mock-test-bundle/nifi-connector-mock-integration-tests
\
-am clean verify \
-DforkCount=2 \
-DreuseForks=true
```
Implementation branch:
https://github.com/skeossei/nifi/tree/standard-connector-runner-isolation
> Support filesystem isolation for StandardConnectorTestRunner instances
> ----------------------------------------------------------------------
>
> Key: NIFI-16353
> URL: https://issues.apache.org/jira/browse/NIFI-16353
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Reporter: Sevag Keosseian
> Assignee: Sevag Keosseian
> Priority: Minor
> Labels: Connector, test-infrastructure
>
> StandardConnectorTestRunner uses shared mutable directories under
> target/work, while its NiFi properties use shared repository and storage
> paths under target/nifi-storage.
> When multiple runners execute concurrently, they create separate mock NiFi
> instances but can write to the same:
> - FlowFile, content, database, and NAR repositories
> - WAL and state files
> - Assets and connector assets
> - Unpacked NAR working directories
> - Jetty working directories
> This can cause cross-writes, repository corruption, startup failures, or one
> runner’s shutdown affecting another runner.
> NIFI-16183 increased the graceful shutdown timeout, which widened the
> collision window but did not cause the underlying shared-filesystem problem.
> Its shutdown behavior should remain unchanged.
> Proposed behavior
> Add an optional API:
> ```
> StandardConnectorTestRunner.Builder.instanceDirectory(Path)
> ```
> When supplied, all mutable NiFi paths must be derived from that instance
> root. The configured NAR library directory remains shared and read-only.
> When no instance directory is supplied, preserve the existing directory
> behavior for backward compatibility.
> Continue using OS-assigned HTTP ports by default.
> Caller-provided instance directories remain caller-owned and must not be
> deleted when the runner closes or bootstrap fails.
> Acceptance criteria
> - Separate instance roots isolate all mutable paths.
> - The NAR library directory remains shared.
> - Two runners can start and stop concurrently without cross-writing.
> - Closing one runner does not affect the other.
> - Caller-provided directories are retained after shutdown and bootstrap
> failure.
> - Existing builder APIs remain compatible.
> - Default port selection remains OS-assigned.
> - Existing behavior is preserved when instanceDirectory is omitted.
> - NIFI-16183 shutdown timeout behavior is unchanged.
> Verification
> ```bash
> ./mvnw \
> -pl
> nifi-connector-mock-bundle/nifi-connector-mock-test-bundle/nifi-connector-mock-integration-tests
> \
> -am clean verify \
> -DforkCount=2 \
> -DreuseForks=true
> ```
> Implementation branch:
> [https://github.com/skeossei/nifi/tree/standard-connector-runner-isolation]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
