Thanks Mich,
ANALYZE TABLE PARTITION(dt='2017-08-20, bar='hello'') COMPUTE STATISTICS
indeed make count(*) returns correct value (for the partition only).
but my hive table was not able to get data from those pre-existed json file
unless I insert one record for the partition AND run ANALYZE TABLE ... COMPUTE
STATISTICS for the partition. I must have missed something.
How to make those preexisted json visible in hive table?
On Mon, Oct 30, 2017 at 4:53 PM, Mich Talebzadeh <mich.talebza...@gmail.com>
wrote:
> have you analyzed table for the partition?
>
> ANALYZE TABLE test_table PARTITION('2017-08-20, bar='hello'') COMPUTE
> STATISTICS;
>
> and do count(*) from table
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn *
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
> On 30 October 2017 at 22:29, Jiewen Shao <fifistorm...@gmail.com> wrote:
>
>> Hi, I have persisted lots of JSON files on S3 under partitioned
>> directories such as /bucket/table1/dt=2017-10-28/bar=hello/*
>>
>> 1. Now I created a hive table:
>> CREATE EXTERNAL TABLE table1 (.... )
>> PARTITIONED BY (dt string, bar string) ROW FORMAT serde
>> 'org.apache.hive.hcatalog.data.JsonSerDe' LOCATION 's3://bucket/table1';
>>
>> 2. Under hive commandline
>> select * from table1; // return nothing
>>
>> 3. INSERT INTO TABLE table1 PARTITION (dt='2017-08-28', bar='hello')
>> select ....;
>>
>> 4. now select * from table1; // return all the data from that partition
>>
>> 5. select count(*) from table1; // returns 1
>>
>> Can someone explain what did I miss?
>>
>> Thanks a lot!
>>
>
>
