[
https://issues.apache.org/jira/browse/HIVE-27186?focusedWorklogId=858552&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-858552
]
ASF GitHub Bot logged work on HIVE-27186:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Apr/23 03:56
Start Date: 22/Apr/23 03:56
Worklog Time Spent: 10m
Work Description: henrib commented on code in PR #4194:
URL: https://github.com/apache/hive/pull/4194#discussion_r1174278092
##########
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/properties/PropertyManager.java:
##########
@@ -0,0 +1,576 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.metastore.properties;
+
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlException;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.JexlFeatures;
+import org.apache.commons.jexl3.introspection.JexlPermissions;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.lang.reflect.Constructor;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Properties;
+import java.util.TreeMap;
+import java.util.UUID;
+import java.util.function.Function;
+
+/**
+ * A property manager.
+ * <p>
+ * This handles operations at the higher functional level; an instance is
created per-session and
+ * drives queries and updates in a transactional manner.
+ * </p>
+ * <p>
+ * The manager ties the property schemas into one namespace; all property maps
it handles must and will use
+ * one of its known schema.
+ * </p>
+ */
+public abstract class PropertyManager {
+ /** The logger. */
+ public static final Logger LOGGER =
LoggerFactory.getLogger(PropertyManager.class);
+ /** The set of dirty maps. */
+ protected Map<String, PropertyMap> dirtyMaps = new HashMap<>();
+ /** This manager namespace. */
+ protected final String namespace;
+ /** The property map store. */
+ protected final PropertyStore store;
+ /** A Jexl engine for convenience. */
+ static final JexlEngine JEXL;
+ static {
+ JexlFeatures features = new JexlFeatures()
+ .sideEffect(false)
+ .sideEffectGlobal(false);
+ JexlPermissions p = JexlPermissions.RESTRICTED
+ .compose("org.apache.hadoop.hive.metastore.properties.*");
+ JEXL = new JexlBuilder()
+ .features(features)
+ .permissions(p)
+ .create();
+ }
+
+ /**
+ * The map of defined managers.
+ */
+ private static final Map<String, Constructor<? extends PropertyManager>>
NSMANAGERS = new HashMap<>();
+
+ /**
+ * Declares a property manager class.
+ * @param ns the namespace
+ * @param pmClazz the property mamanger class
+ */
+ public static boolean declare(String ns, Class<? extends PropertyManager>
pmClazz) {
+ try {
+ synchronized(NSMANAGERS) {
+ Constructor<? extends PropertyManager> ctor = NSMANAGERS.get(ns);
+ if (ctor == null) {
+ ctor = pmClazz.getConstructor(String.class, PropertyStore.class);
+ NSMANAGERS.put(ns, ctor);
+ return true;
+ } else {
+ if (!Objects.equals(ctor.getDeclaringClass(), pmClazz)) {
+ LOGGER.error("namespace {} is already declared for {}", ns,
pmClazz.getCanonicalName());
+ }
+ }
+ }
+ } catch(NoSuchMethodException xnom ) {
+ LOGGER.error("namespace declaration failed: " + ns + ", " +
pmClazz.getCanonicalName(),
+ xnom);
+ }
+ return false;
+ }
+
+ /**
+ * Creates an instance of manager using its declared namespace.
+ * @param namespace the manager"s namespace
+ * @param store the property store
+ * @return a property manager instance
+ */
+ public static PropertyManager create(String namespace, PropertyStore store) {
+ final Constructor<? extends PropertyManager> ctor;
+ synchronized(NSMANAGERS) {
+ ctor = NSMANAGERS.get(namespace);
+ }
+ if (ctor != null) {
+ try {
+ return ctor.newInstance(namespace, store);
+ } catch(Exception xany) {
+ LOGGER.error("property manager creation failed "+ namespace, xany);
+ }
+ } else {
+ LOGGER.error("no such property manager namespace is declared " +
namespace);
Review Comment:
An error is not silent :-) ; it also should only occur during development of
a new PropertyManager since it means the constructor failed. I'm trying to
avoid throwing an exception that could crash the server or avoid it to start.
Issue Time Tracking
-------------------
Worklog Id: (was: 858552)
Time Spent: 9h 40m (was: 9.5h)
> A persistent property store
> ----------------------------
>
> Key: HIVE-27186
> URL: https://issues.apache.org/jira/browse/HIVE-27186
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 4.0.0-alpha-2
> Reporter: Henri Biestro
> Assignee: Henri Biestro
> Priority: Major
> Labels: pull-request-available
> Time Spent: 9h 40m
> Remaining Estimate: 0h
>
> WHAT
> A persistent property store usable as a support facility for any metadata
> augmentation feature.
> WHY
> When adding new meta-data oriented features, we usually need to persist
> information linking the feature data and the HiveMetaStore objects it applies
> to. Any information related to a database, a table or the cluster - like
> statistics for example or any operational data state or data (think rolling
> backup) - fall in this use-case.
> Typically, accommodating such a feature requires modifying the Metastore
> database schema by adding or altering a table. It also usually implies
> modifying the thrift APIs to expose such meta-data to consumers.
> The proposed feature wants to solve the persistence and query/transport for
> these types of use-cases by exposing a 'key/(meta)value' store exposed as a
> property system.
> HOW
> A property-value model is the simple and generic exposed API.
> To provision for several usage scenarios, the model entry point is a
> 'namespace' that qualifies the feature-component property manager. For
> example, 'stats' could be the namespace for all properties related to the
> 'statistics' feature.
> The namespace identifies a manager that handles property-groups persisted as
> property-maps. For instance, all statistics pertaining to a given table would
> be collocated in the same property-group. As such, all properties (say number
> of 'unique_values' per columns) for a given HMS table 'relation0' would all
> be stored and persisted in the same property-map instance.
> Property-maps may be decorated by an (optional) schema that may declare the
> name and value-type of allowed properties (and their optional default value).
> Each property is addressed by a name, a path uniquely identifying the
> property in a given property map.
> The manager also handles transforming property-map names to the property-map
> keys used to persist them in the DB.
> The API provides inserting/updating properties in bulk transactionally. It
> also provides selection/projection to help reduce the volume of exchange
> between client/server; selection can use (JEXL expression) predicates to
> filter maps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
