pythaac opened a new pull request, #10562:
URL: https://github.com/apache/gravitino/pull/10562
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
Examples:
- "[#123] feat(operator): Support xxx"
- "[#233] fix: Check null before access result in xxx"
- "[MINOR] refactor: Fix typo in variable name"
- "[MINOR] docs: Fix typo in README"
- "[#255] test: Fix flaky test NameOfTheTest"
Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->
### What changes were proposed in this pull request?
1. **Enhanced ClickHouse index capabilities:** Added support to configure
the `GRANULARITY` of minmax and bloom filter secondary indexes for ClickHouse.
Users can now pass `"granularity"` inside the index properties map.
2. **Robust Validation:** Added logic to validate granularity user inputs.
It accurately rejects case-insensitive properties (e.g., `"Granularity"`),
negative, or non-numeric strings, safely falling back to defaults if the
property is absent.
3. **Core API / DTO Upgrades:** To carry index properties across core
layers, introduced the properties parameter to index creation operations
covering api, common, client, and server REST modules.
### Why are the changes needed?
To fix hardcoded limitations on ClickHouse tables. Previously, the index
granularity for minmax and bloom filter indexes was strictly hardcoded to 1 and
3. Users working with heavy analytical datasets need the ability to optimize
these granularities manually. This PR introduces the required mechanism while
maintaining backward compatibility with the existing hardcoded defaults.
Fix: #10446
### Does this PR introduce _any_ user-facing change?
Yes.
1. Adding table indexes containing the `"granularity"` property is now
allowed and properly translated into the ClickHouse DDL script format `INDEX
<name> ... GRANULARITY <value>`.
2. Changed core index APIs (added new constructors allowing developers to
pass `Map<String, String> properties`).
### How was this patch tested?
Local unit tests and newly added comprehensive integration tests have been
run to verify the correctness.
- **Unit tests:**
- Verified `CREATE` and `ALTER` DDL scripts accurately consume properties
or fallback to defaults. Confirmed that invalid string parsing guarantees
`IllegalArgumentException`.
- Expanded server module tests ensuring REST serialization of DTO
properties works correctly.
- **Integration tests:**
- Full E2E testing ensures custom granularities persist correctly
dynamically inside the ClickHouse container and returns the correct index
properties upon table load.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
