Okay, Thanks. I'll make a draft CR that sets up Savanna for dotted names, and one that uses dotted names with streaming.
Best, Trevor On Wed, 2014-02-05 at 15:58 +0400, Sergey Lukjanov wrote: > I like the dot-separated name. There are several reasons for it: > > > * it'll not require changes in all Savanna subprojects; > * eventually we'd like to use not only Oozie for EDP (for example, if > we'll support Twitter Storm) and this new tools could require > additional 'subtypes'. > > > Thanks for catching this. > > > On Tue, Feb 4, 2014 at 10:47 PM, Trevor McKay <tmc...@redhat.com> > wrote: > Thanks Andrew. > > My author thought, which is in between, is to allow dotted > types. > "MapReduce.streaming" for example. > > This gives you the subtype flavor but keeps all the APIs the > same. > We just need a wrapper function to separate them when we > compare types. > > Best, > > Trevor > > On Mon, 2014-02-03 at 14:57 -0800, Andrew Lazarev wrote: > > I see two points: > > * having Savanna types mapped to Oozie action types is > intuitive for > > hadoop users and this is something we would like to keep > > * it is hard to distinguish different kinds of one job type > > > > > > Adding 'subtype' field will solve both problems. Having it > optional > > will not break backward compatibility. Adding database > migration > > script is also pretty straightforward. > > > > > > Summarizing, my vote is on "subtype" field. > > > > > > Thanks, > > Andrew. > > > > > > On Mon, Feb 3, 2014 at 2:10 PM, Trevor McKay > <tmc...@redhat.com> > > wrote: > > > > I was trying my best to avoid adding extra job types > to > > support > > mapreduce variants like streaming or mapreduce with > pipes, but > > it seems > > that adding the types is the simplest solution. > > > > On the API side, Savanna can live without a specific > job type > > by > > examining the data in the job record. > Presence/absence of > > certain > > things, or null values, etc, can provide adequate > indicators > > to what > > kind of mapreduce it is. Maybe a little bit subtle. > > > > But for the UI, it seems that explicit knowledge of > what the > > job is > > makes things easier and better for the user. When a > user > > creates a > > streaming mapreduce job and the UI is aware of the > type later > > on at job > > launch, the user can be prompted to provide the > right configs > > (i.e., the > > streaming mapper and reducer values). > > > > The explicit job type also supports validation > without having > > to add > > extra flags (which impacts the savanna client, and > the JSON, > > etc). For > > example, a streaming mapreduce job does not require > any > > specified > > libraries so the fact that it is meant to be a > streaming job > > needs to be > > known at job creation time. > > > > So, to that end, I propose that we add a > MapReduceStreaming > > job type, > > and probably at some point we will have > MapReducePiped too. > > It's > > possible that we might have other job types in the > future too > > as the > > feature set grows. > > > > There was an effort to make Savanna job types > parallel Oozie > > action > > types, but in this case that's just not possible > without > > introducing a > > "subtype" field in the job record, which leads to a > database > > migration > > script and savanna client changes. > > > > What do you think? > > > > Best, > > > > Trevor > > > > > > > > _______________________________________________ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > _______________________________________________ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > -- > Sincerely yours, > Sergey Lukjanov > Savanna Technical Lead > Mirantis Inc. > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
