Hi Navis,
one scan task with the correct filter given would be most efficient I
guess. I think the Filter Pushdown mechanism is not working correctly in
this case. It cannot be a too exotic use case to have two views on a table
that when querying yields two different select statements towards this
I thinks it's back to original problem.
What you wanted is separated scan(task) for different view. But hive does
not work like that. If two tables or views (or mix of them) has same
location, it's regarded as same table with same table description (will be
overridden by lastly visited table or vi
I'm using Hive 0.10 (the version bundled with CDH4.4).
The explain at my end looks similar to yours. I guess my real concern is
around the way I have implemented the filters.
This is how I have done it:
- In the constructor of my RecordReader I read the property
hive.io.filter.expr.serialized and
In my try, it worked (and should be).
CREATE EXTERNAL TABLE MasterTable (
column1 STRING, column2 STRING)
LOCATION 'hdfs://localhost:9000/home/navis/my_location';
CREATE VIEW IF NOT EXISTS View1 (column1, column2) AS SELECT column1,
column2 FROM MasterTable WHERE column1<30;
CREATE VIEW IF N
Hi Navis (and others),
seems like my solution with views does not work after all. That is, it
works fine as long as I do not use filter pushdown. My setup is something
like below:
CREATE EXTERNAL TABLE MasterTable (
column1 STRING,
column2 STRING,
column3 STRING
column4 STRING)
PARTITIO
Hi Navis,
thank you for sorting this out! I have tried getting around this by using
views towards a single master table instead in combination with UDFs
instead . Seems to work so far.
/Petter
2013/12/18 Navis류승우
> Hive uses path to table(or partition) mapping internally (you can see that
>
Hive uses path to table(or partition) mapping internally (you can see that
in MapredWork, etc.), which might caused first table overwritten by other.
I didn't tried symlink on hdfs, which could be a solution.
2013/12/12 Petter von Dolwitz (Hem)
> Hi,
>
> I have declared several external table
