Can anyone give me some suggestions? I have been stuck in this problem for
several days. Need help!!
> 在 2018年4月22日，下午9:38，侯宗田 <zongtian...@icloud.com> 写道：
>
>
> Hi,
>
> I am writing a application which needs the metastore about hive tables. I
> have used webhcat to get the information about tables and process them. But a
> simple request takes over eight seconds to respond on localhost. Why is this
> so slow, and how can I fix it or is there other way I can extract the
> metadata in C?
>
> $ time curl -s
> 'http://localhost:50111/templeton/v1/ddl/database/default/table/haha?user.name=ctdean
>
> <http://localhost:50111/templeton/v1/ddl/database/default/table/haha?user.name=ctdean>'
> {"columns":
> [{"name":"id","type":"int"}],
> "database":"default",
> "table":"haha"}
>
> real 0m8.400s
> user 0m0.053s
> sys 0m0.019s
> it seems to run a hcat.py, and it create a bunch of things then clear them,
> it takes very long time, does anyone have some ideas about it?? Any
> suggestions will be very appreciated!
>
> $hcat.py -e "use default; desc haha; "
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings
> <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 18/04/21 16:38:13 INFO conf.HiveConf: Found configuration file
> file:/usr/local/hive/conf/hive-site.xml
> 18/04/21 16:38:15 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 18/04/21 16:38:16 INFO session.SessionState: Created HDFS directory:
> /tmp/hive/kousouda/05096382-f9b6-4dae-aee2-dfa6750c0668
> 18/04/21 16:38:16 INFO session.SessionState: Created local directory:
> /tmp/hive/java/kousouda/05096382-f9b6-4dae-aee2-dfa6750c0668
> 18/04/21 16:38:16 INFO session.SessionState: Created HDFS directory:
> /tmp/hive/kousouda/05096382-f9b6-4dae-aee2-dfa6750c0668/_tmp_space.db
> 18/04/21 16:38:16 INFO ql.Driver: Compiling
> command(queryId=kousouda_20180421163816_58c38a44-25e3-4665-8bb5-a9b17fdf2d62):
> use default
> 18/04/21 16:38:17 INFO metastore.HiveMetaStore: 0: Opening raw store with
> implementation class:org.apache.hadoop.hive.metastore.ObjectStore
> 18/04/21 16:38:17 INFO metastore.ObjectStore: ObjectStore, initialize called
> 18/04/21 16:38:18 INFO DataNucleus.Persistence: Property
> hive.metastore.integral.jdo.pushdown unknown - will be ignored
> 18/04/21 16:38:18 INFO DataNucleus.Persistence: Property
> datanucleus.cache.level2 unknown - will be ignored
> 18/04/21 16:38:18 INFO metastore.ObjectStore: Setting MetaStore object pin
> classes with
> hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
> 18/04/21 16:38:20 INFO metastore.MetaStoreDirectSql: Using direct SQL,
> underlying DB is MYSQL
> 18/04/21 16:38:20 INFO metastore.ObjectStore: Initialized ObjectStore
> 18/04/21 16:38:20 INFO metastore.HiveMetaStore: Added admin role in metastore
> 18/04/21 16:38:20 INFO metastore.HiveMetaStore: Added public role in metastore
> 18/04/21 16:38:20 INFO metastore.HiveMetaStore: No user is added in admin
> role, since config is empty
> 18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: get_all_functions
> 18/04/21 16:38:20 INFO HiveMetaStore.audit: ugi=kousouda
> ip=unknown-ip-addr cmd=get_all_functions
> 18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: get_database: default
> 18/04/21 16:38:20 INFO HiveMetaStore.audit: ugi=kousouda
> ip=unknown-ip-addr cmd=get_database: default
> 18/04/21 16:38:20 INFO ql.Driver: Semantic Analysis Completed
> 18/04/21 16:38:20 INFO ql.Driver: Returning Hive schema:
> Schema(fieldSchemas:null, properties:null)
> 18/04/21 16:38:20 INFO ql.Driver: Completed compiling
> command(queryId=kousouda_20180421163816_58c38a44-25e3-4665-8bb5-a9b17fdf2d62);
> Time taken: 3.936 seconds
> 18/04/21 16:38:20 INFO ql.Driver: Concurrency mode is disabled, not creating
> a lock manager
> 18/04/21 16:38:20 INFO ql.Driver: Executing
> command(queryId=kousouda_20180421163816_58c38a44-25e3-4665-8bb5-a9b17fdf2d62):
> use default
> 18/04/21 16:38:20 INFO sqlstd.SQLStdHiveAccessController: Created
> SQLStdHiveAccessController for session context : HiveAuthzSessionContext
> [sessionString=05096382-f9b6-4dae-aee2-dfa6750c0668, clientType=HIVECLI]
> 18/04/21 16:38:20 WARN session.SessionState: METASTORE_FILTER_HOOK will be
> ignored, since hive.security.authorization.manager is set to instance of
> HiveAuthorizerFactory.
> 18/04/21 16:38:20 INFO hive.metastore: Mestastore configuration
> hive.metastore.filter.hook changed from
> org.apache.hadoop.hive.metastore.DefaultMetaStoreFilterHookImpl to
> org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook
> 18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: Cleaning up thread local
> RawStore...
> 18/04/21 16:38:20 INFO HiveMetaStore.audit: ugi=kousouda
> ip=unknown-ip-addr cmd=Cleaning up thread local RawStore...
> 18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: Done cleaning up thread
> local RawStore
> 18/04/21 16:38:20 INFO HiveMetaStore.audit: ugi=kousouda
> ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore
> 18/04/21 16:38:20 INFO ql.Driver: Starting task [Stage-0:DDL] in serial mode
> 18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: get_database: default
> 18/04/21 16:38:20 INFO HiveMetaStore.audit: ugi=kousouda
> ip=unknown-ip-addr cmd=get_database: default
> 18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: Opening raw store with
> implementation class:org.apache.hadoop.hive.metastore.ObjectStore
> 18/04/21 16:38:20 INFO metastore.ObjectStore: ObjectStore, initialize called
> 18/04/21 16:38:20 INFO metastore.MetaStoreDirectSql: Using direct SQL,
> underlying DB is MYSQL
> 18/04/21 16:38:20 INFO metastore.ObjectStore: Initialized ObjectStore
> 18/04/21 16:38:20 INFO metastore.HiveMetaStore: 0: get_database: default
> 18/04/21 16:38:20 INFO HiveMetaStore.audit: ugi=kousouda
> ip=unknown-ip-addr cmd=get_database: default
> 18/04/21 16:38:20 INFO ql.Driver: Completed executing
> command(queryId=kousouda_20180421163816_58c38a44-25e3-4665-8bb5-a9b17fdf2d62);
> Time taken: 0.202 seconds
> OK
