[
https://issues.apache.org/jira/browse/HIVE-27186?focusedWorklogId=858539&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-858539
]
ASF GitHub Bot logged work on HIVE-27186:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Apr/23 02:28
Start Date: 22/Apr/23 02:28
Worklog Time Spent: 10m
Work Description: henrib commented on code in PR #4194:
URL: https://github.com/apache/hive/pull/4194#discussion_r1174258092
##########
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/properties/PropertyManager.java:
##########
@@ -0,0 +1,576 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.metastore.properties;
+
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlException;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.JexlFeatures;
+import org.apache.commons.jexl3.introspection.JexlPermissions;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.lang.reflect.Constructor;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Properties;
+import java.util.TreeMap;
+import java.util.UUID;
+import java.util.function.Function;
+
+/**
+ * A property manager.
+ * <p>
+ * This handles operations at the higher functional level; an instance is
created per-session and
+ * drives queries and updates in a transactional manner.
+ * </p>
+ * <p>
+ * The manager ties the property schemas into one namespace; all property maps
it handles must and will use
+ * one of its known schema.
+ * </p>
+ */
+public abstract class PropertyManager {
+ /** The logger. */
+ public static final Logger LOGGER =
LoggerFactory.getLogger(PropertyManager.class);
+ /** The set of dirty maps. */
+ protected Map<String, PropertyMap> dirtyMaps = new HashMap<>();
+ /** This manager namespace. */
+ protected final String namespace;
+ /** The property map store. */
+ protected final PropertyStore store;
+ /** A Jexl engine for convenience. */
+ static final JexlEngine JEXL;
+ static {
+ JexlFeatures features = new JexlFeatures()
+ .sideEffect(false)
+ .sideEffectGlobal(false);
+ JexlPermissions p = JexlPermissions.RESTRICTED
+ .compose("org.apache.hadoop.hive.metastore.properties.*");
+ JEXL = new JexlBuilder()
+ .features(features)
+ .permissions(p)
+ .create();
+ }
+
+ /**
+ * The map of defined managers.
+ */
+ private static final Map<String, Constructor<? extends PropertyManager>>
NSMANAGERS = new HashMap<>();
Review Comment:
Because the property manager owns the property schema(s) and different
implementations are expected; we 'replace' modifying the db schema by allowing
different property manager namespaces and manager implementations. Next feature
that needs to persist metadata properties may create its own (think partitions
metadata).
Issue Time Tracking
-------------------
Worklog Id: (was: 858539)
Time Spent: 7.5h (was: 7h 20m)
> A persistent property store
> ----------------------------
>
> Key: HIVE-27186
> URL: https://issues.apache.org/jira/browse/HIVE-27186
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 4.0.0-alpha-2
> Reporter: Henri Biestro
> Assignee: Henri Biestro
> Priority: Major
> Labels: pull-request-available
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> WHAT
> A persistent property store usable as a support facility for any metadata
> augmentation feature.
> WHY
> When adding new meta-data oriented features, we usually need to persist
> information linking the feature data and the HiveMetaStore objects it applies
> to. Any information related to a database, a table or the cluster - like
> statistics for example or any operational data state or data (think rolling
> backup) - fall in this use-case.
> Typically, accommodating such a feature requires modifying the Metastore
> database schema by adding or altering a table. It also usually implies
> modifying the thrift APIs to expose such meta-data to consumers.
> The proposed feature wants to solve the persistence and query/transport for
> these types of use-cases by exposing a 'key/(meta)value' store exposed as a
> property system.
> HOW
> A property-value model is the simple and generic exposed API.
> To provision for several usage scenarios, the model entry point is a
> 'namespace' that qualifies the feature-component property manager. For
> example, 'stats' could be the namespace for all properties related to the
> 'statistics' feature.
> The namespace identifies a manager that handles property-groups persisted as
> property-maps. For instance, all statistics pertaining to a given table would
> be collocated in the same property-group. As such, all properties (say number
> of 'unique_values' per columns) for a given HMS table 'relation0' would all
> be stored and persisted in the same property-map instance.
> Property-maps may be decorated by an (optional) schema that may declare the
> name and value-type of allowed properties (and their optional default value).
> Each property is addressed by a name, a path uniquely identifying the
> property in a given property map.
> The manager also handles transforming property-map names to the property-map
> keys used to persist them in the DB.
> The API provides inserting/updating properties in bulk transactionally. It
> also provides selection/projection to help reduce the volume of exchange
> between client/server; selection can use (JEXL expression) predicates to
> filter maps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
