On Sun, Mar 20, 2011 at 11:20 AM, Ted Yu <yuzhih...@gmail.com> wrote: > How about link to http://imageshack.us/ or TinyPic ? > > Thanks > > On Sun, Mar 20, 2011 at 7:56 AM, Edward Capriolo <edlinuxg...@gmail.com> > wrote: >> >> On Sun, Mar 20, 2011 at 10:30 AM, Ted Yu <yuzhih...@gmail.com> wrote: >> > Can someone re-attach the missing figures for that wiki ? >> > >> > Thanks >> > >> > On Sun, Mar 20, 2011 at 7:15 AM, bharath vissapragada >> > <bharathvissapragada1...@gmail.com> wrote: >> >> >> >> Hi Igor, >> >> >> >> See http://wiki.apache.org/hadoop/Hive/JoinOptimization and see the >> >> jira 1642 which automatically converts a normal join into map-join >> >> (Otherwise you can specify the mapjoin hints in the query itself.). >> >> Because your 'S' table is very small , it can be replicated across all >> >> the mappers and the reduce phase can be avoided. This can greatly >> >> reduce the runtime .. (See the results section in the page for >> >> details.). >> >> >> >> Hope this helps. >> >> >> >> Thanks >> >> >> >> >> >> On Sun, Mar 20, 2011 at 6:37 PM, Jov <zhao6...@gmail.com> wrote: >> >> > 2011/3/20 Igor Tatarinov <i...@decide.com>: >> >> >> I have the following join that takes 4.5 hours (with 12 nodes) >> >> >> mostly >> >> >> because of a single reduce task that gets the bulk of the work: >> >> >> SELECT ... >> >> >> FROM T >> >> >> LEFT OUTER JOIN S >> >> >> ON T.timestamp = S.timestamp and T.id = S.id >> >> >> This is a 1:0/1 join so the size of the output is exactly the same >> >> >> as >> >> >> the >> >> >> size of T (500M records). S is actually very small (5K). >> >> >> I've tried: >> >> >> - switching the order of the join conditions >> >> >> - using a different hash function setting (jenkins instead of >> >> >> murmur) >> >> >> - using SET set hive.auto.convert.join = true; >> >> > >> >> > are you sure your query convert to mapjoin? if not,try use explicit >> >> > mapjoin hint. >> >> > >> >> > >> >> >> - using SET hive.optimize.skewjoin = true; >> >> >> but nothing helped :( >> >> >> Anything else I can try? >> >> >> Thanks! >> >> > >> >> >> >> >> >> >> >> -- >> >> Regards, >> >> Bharath .V >> >> w:http://research.iiit.ac.in/~bharath.v >> > >> > >> >> The wiki does not allow images, confluence does but we have not moved >> their yet. > >
Not a bad option, but lame that the foundation with the most popular web server in the world has to host our images elsewhere :(
