Hey guys, As soon as the Ivy and Avro version changes are in, let's start the branching process?
-Abe On Wed, Jul 23, 2014 at 1:02 AM, Venkat Ranganathan < vranganat...@hortonworks.com> wrote: > Generally the number of tasks is a hint to InputFormat.getSplits() > based on number of splits wanted and it can potentially return > different number of splits. > > If you see ExportInputFormat, the split size is determined by > combined file sizes divided by requested num mappers. This is a > integer division and there is a potential for getting additional > splits. > > Thanks > > Venkat > > On Tue, Jul 22, 2014 at 11:41 PM, David Robson > <david.rob...@software.dell.com> wrote: > > Yes OraOop is not enabled by default - and also this particular issue is > only when using partitioning - so they would have turned on some extra > flags. Then they also have to be using 1 more mapper than requested as you > said - so it's a pretty specific combination of events - so I don't think > we need to hold up the release for it. > > > > Do you recall what the behaviour should be in regards to this - the > documentation leads me to believe if we request 4 mappers we should get 4 > mappers? If this is only a request and we may get more - perhaps we should > update the documentation about this. Otherwise we could change the code to > guarantee no more than 4 mappers? > > > > -----Original Message----- > > From: Abraham Elmahrek [mailto:a...@cloudera.com] > > Sent: Wednesday, 23 July 2014 3:16 PM > > To: dev@sqoop.apache.org > > Subject: Re: Final Jiras and Release Date > > > > Hey guys, > > > > AFAIK we can fall back onto the original connector? With that being > said, OraOop is an awesome connector... So i'm completely open to waiting > for a fix if it isn't too large. > > > > David, I'm assuming you're referring to the "-m" option? I do recall > some cases where there may be one more split than desired if splitting is > not clean. Hopefully Venkat has more insight! > > > > -Abe > > > > > > On Tue, Jul 22, 2014 at 9:42 PM, David Robson < > david.rob...@software.dell.com> wrote: > > > >> Hey Venkat, > >> > >> I had a look into that issue - it seems the problem is even though you > >> request 4 mappers - Sqoop runs 5 mappers. Is the number of mappers > >> meant to be guaranteed? For example if I say 4 mappers, should I get 4 > >> mappers? I guess there could be potential to get less mappers if the > >> data was for example 1 row - then that would only be processed by 1 > >> mapper. But should it be possible to get more mappers than requested? > >> > >> OraOop is assuming there will be no more mappers than requested - so > >> if this is a valid scenario we would need to modify OraOop. On the > >> other hand if this is unexpected behaviour then we should look at why > >> the num mappers is not working? > >> > >> Either way I don't think we need to fix it for this release - as it > >> seems customers would be unlikely to hit this issue. > >> > >> David > >> > >> -----Original Message----- > >> From: Venkat Ranganathan [mailto:vranganat...@hortonworks.com] > >> Sent: Wednesday, 23 July 2014 1:04 PM > >> To: dev@sqoop.apache.org > >> Subject: Re: Final Jiras and Release Date > >> > >> Abe > >> > >> Do we want to target SQOOP-1388 also for 1.4.5 - the one that Vidya > >> has raised for one of the Oracle connector export failure? > >> > >> My avro patch is in RB. > >> > >> Thanks > >> > >> Venkat > >> > >> On Tue, Jul 22, 2014 at 6:12 PM, Abraham Elmahrek <a...@cloudera.com> > >> wrote: > >> > Hey guys, > >> > > >> > It looks like we still have a couple of Jiras still open. Let's see > >> > how things look tomorrow, but let's have these be the last two Jiras > >> > slotted for this release. > >> > > >> > -Abe > >> > > >> > > >> > On Fri, Jul 18, 2014 at 1:20 PM, Venkat Ranganathan < > >> > vranganat...@hortonworks.com> wrote: > >> > > >> >> Sounds good. I have already uploaded the patch for SQOOP-1358. It > >> >> needs to be reviewed by a committer and then committed if no > >> >> further changes are needed (or more work needed otherwise). > >> >> > >> >> Thanks for driving the release! > >> >> > >> >> Venkat > >> >> > >> >> On Fri, Jul 18, 2014 at 10:58 AM, Jarek Jarcec Cecho > >> >> <jar...@apache.org> > >> >> wrote: > >> >> > Seems reasonable to me, thank you for driving the release Abe! > >> >> > > >> >> > Jarcec > >> >> > > >> >> > On Jul 18, 2014, at 10:40 AM, Abraham Elmahrek <a...@cloudera.com> > >> wrote: > >> >> > > >> >> >> Hey guys, > >> >> >> > >> >> >> I haven't seen a -1 on this suggestion. So let's create a > >> >> >> release branch come Wednesday July 23rd. > >> >> >> > >> >> >> -Abe > >> >> >> > >> >> >> > >> >> >> On Wed, Jul 16, 2014 at 11:38 AM, Abraham Elmahrek > >> >> >> <a...@cloudera.com> > >> >> wrote: > >> >> >> > >> >> >>> Are there any objections to this branch date? > >> >> >>> > >> >> >>> -Abe > >> >> >>> > >> >> >>> > >> >> >>> On Mon, Jul 14, 2014 at 1:22 PM, Abraham Elmahrek > >> >> >>> <a...@apache.org> > >> >> wrote: > >> >> >>> > >> >> >>>> Hey guys, > >> >> >>>> > >> >> >>>> It looks like there are a couple of Jiras that are in progress > >> >> >>>> and slotted for the 1.4.5 release. If we aim for July 23rd to > >> >> >>>> create a > >> >> release > >> >> >>>> branch, does that give every one enough time to finish up what > >> >> >>>> they're working on? > >> >> >>>> > >> >> >>>> See https://issues.apache.org/jira/browse/SQOOP-1353 for more > >> >> >>>> information. > >> >> >>>> > >> >> >>>> -Abe > >> >> >>>> > >> >> >>> > >> >> >>> > >> >> > > >> >> > >> >> -- > >> >> CONFIDENTIALITY NOTICE > >> >> NOTICE: This message is intended for the use of the individual or > >> >> entity to which it is addressed and may contain information that is > >> >> confidential, privileged and exempt from disclosure under > >> >> applicable law. If the reader of this message is not the intended > >> >> recipient, you are hereby notified that any printing, copying, > >> >> dissemination, distribution, disclosure or forwarding of this > >> >> communication is strictly prohibited. If you have received this > >> >> communication in error, please contact the sender immediately and > >> >> delete it from your > >> system. Thank You. > >> >> > >> > >> -- > >> CONFIDENTIALITY NOTICE > >> NOTICE: This message is intended for the use of the individual or > >> entity to which it is addressed and may contain information that is > >> confidential, privileged and exempt from disclosure under applicable > >> law. If the reader of this message is not the intended recipient, you > >> are hereby notified that any printing, copying, dissemination, > >> distribution, disclosure or forwarding of this communication is > >> strictly prohibited. If you have received this communication in error, > >> please contact the sender immediately and delete it from your system. > Thank You. > >> > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
