Mihaly Szjatinya has uploaded a new patch set (#2). (
http://gerrit.cloudera.org:8080/23745 )
Change subject: IMPALA-14477: [WIP] Speed up test execution (Part 2)
......................................................................
IMPALA-14477: [WIP] Speed up test execution (Part 2)
This continues the effort in IMPALA-6070 and consequent tickets.
Still mostly WIP to agree on basic concepts, some features not
finalized / for debugging purposes. Tested locally and on Jenkins,
mostly for individual test suites.
1. Container Infrastructure
- Cgroup v1/v2 Compatibility: Added smart detection for both cgroup
v1 and v2 systems
- Support for cgroup v2 unified hierarchy
(/sys/fs/cgroup/cgroup.controllers)
- Upgraded default base image from Ubuntu 16.04 to Ubuntu 22.04
- Enhanced container startup logging and diagnostic information
- Fixed IMPALA_BUILD_THREADS detection for BE_TEST
- Docker attach readiness improvements with timing optimizations
- CPU workload integration hooks for performance testing
- Better error reporting and container lifecycle management
- Added per-container prefixes for tailed multi-suite logs
- Increased suite timeouts to reflect the current suite volumes
- Added hardcoded configs section to unload the script parameters
2. Improved Monitoring and Timeline UI
Timeline Improvements
- 6-graph comprehensive timeline: CPU/Memory/Disk usage for both
Container and Host
- Enhanced JSON data structure for rich metrics collection
- Timeline generation in case of job failure
- Collapsible test results per Test Suite
- Added min_memory_usage_gb and max_memory_usage_gb attributes to
ContainerMonitor
- Added Host Machine Information section to timeline
CPU Workload Testing System
- Build and test phase CPU load simulation for realistic metrics
- cpu_workloads.sh: Configurable CPU stress testing utilities for
performance benchmarking
3. Docker Registry Support
- Complete registry push/pull automation
- Two options for passing credentials: config file vs Jenkins envvars
- Human-readable tagging system (repo:tag format instead of
digest-only)
- Docker registry authentication with robot account support
- SHA256 digest handling for image validation and integrity
- Multi-format registry support (standard Docker + Harbor-style
registries)
4. Python 2.7 Compatibility
[Perhaps not needed in the future]
- Replace f-string syntax with .format() method calls for string
formatting
- Replace subprocess.DEVNULL with open(os.devnull, 'w') for
compatibility
- Remove Python 3+ only subprocess.TimeoutExpired exception handling
- Fix various import and syntax compatibility issues
- Ensure all Docker scripts work across Python 2.7+ environments
5. Bugfixing
TODO:
- Test concurrent runs / tune for better performance
- Fix individual tests
- Auto-mode for most optimal test run
- Finalize test report in Timeline
- Incremental build support
- Move to Python3
Stats so far:
SUITE | Result
---------------------|--------------------
BE_TEST | ✅ PASS
| 50m
---------------------|--------------------
FE_TEST | ❌ 4 failures - jwtAuth related
| 47m
---------------------|--------------------
EE_TEST_PARALLEL | ❌ 2 failures
| 1h
---------------------|--------------------
EE_TEST_SERIAL (2) | ✅ PASS
| 47m
---------------------|--------------------
CLUSTER_TEST (3) | ❌ 35 failures - ssl/ipv6 related
| 2h 7m
---------------------|--------------------
JDBC_TEST | ✅ PASS
build image - 1h5m
docker push - 36min (23GB)
docker pull - 5min
Change-Id: I7a126d135ad0425c60ce7c7cd6857b455e655fd9
---
A docker/cpu_workloads.sh
M docker/entrypoint.sh
M docker/monitor.py
M docker/test-with-docker.py
M docker/timeline.html.template
5 files changed, 1,697 insertions(+), 113 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/23745/2
--
To view, visit http://gerrit.cloudera.org:8080/23745
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7a126d135ad0425c60ce7c7cd6857b455e655fd9
Gerrit-Change-Number: 23745
Gerrit-PatchSet: 2
Gerrit-Owner: Mihaly Szjatinya <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Mihaly Szjatinya <[email protected]>
