laserninja opened a new issue, #13576: URL: https://github.com/apache/gravitino/issues/13576
### Describe the feature Add a provider-neutral integration for automatic PII classification results on tables and columns, including provenance, review, and reconciliation with Gravitino's existing tags. ### Motivation We are developing a solution for automatic PII classification. Gravitino already has tagging and proposals for automatic tagging and discovery; an integration contract would allow existing classifiers to contribute results without coupling the catalog to a particular scanning implementation. ### Describe the solution Start with one external classifier integration: - Define an authenticated, idempotent result-ingestion contract containing asset/column identity, classification, confidence where available, detector/version, scan time, and review state. - Agree the findings representation with the related discussion, then map accepted findings to existing tags. - Preserve manual decisions and define behavior for conflicting findings, rescans, stale results, and schema changes. Rejected findings must not be reapplied silently on retry. - Expose provenance, freshness, and review status through permission-aware APIs and the asset UI. Do not store raw PII samples in catalog metadata. - Make accepted classifications available to existing tag-based lookup and future discovery consumers. Acceptance should cover ingesting a finding, reviewing it, applying the corresponding tag, and locating the classified asset. Repeated ingestion must not duplicate findings; a rescan must respect a manual override; inaccessible assets and findings must remain hidden. A new generic search engine, scanner runtime, and query-time policy enforcement are outside this issue's scope. ### Additional context Discussion: [Integrate automatic PII classification with trusted discovery](https://github.com/apache/gravitino/discussions/13575). Refines the integration aspect of [automatic tagging #9385](https://github.com/apache/gravitino/issues/9385) and [external PII labeling #7579](https://github.com/apache/gravitino/issues/7579). Coordinate discovery with [metadata search #2172](https://github.com/apache/gravitino/issues/2172) and [MCP discovery PR #12114](https://github.com/apache/gravitino/pull/12114); policy consumption is tracked separately in [#12758](https://github.com/apache/gravitino/issues/12758). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
