On Mon, Sep 13, 2010 at 1:04 PM, Owen O'Malley <omal...@apache.org> wrote:
> On Mon, Sep 13, 2010 at 11:10 AM, Todd Lipcon <t...@cloudera.com> wrote:
> > Yep, but there are plenty of 10 node clusters out there that do important
> > work at small startups or single-use-case installations, too. We need to
> > provide scalability and security features that work for the 100+ node
> > clusters but also not leave the beginners in the dust.
>
> 10 node clusters are an important use case, but creating the user
> accounts on those clusters is very easy because of the few users.
> Futhermore, if the accounts aren't there it just means the users have
> no groups. Which for a single use system with security turned off
> isn't the end of the world.
>
> > But I think there are plenty of people out there who have built small
> > webapps, shell scripts, cron jobs, etc that use hadoop.job.ugi on some
> > shared account to impersonate other users.
>
> I'd be surprised. At Yahoo, the primary problem came with people
> screen scraping the jobtracker http. With security turned off that
> isn't an issue. Again, it isn't hard, just the evolving interface of
> UserGroupInformation changed. With security, we tried really hard to
> maintain backwards compatibility and succeeded for the vast (99%+)
> majority of the users.
>
> > Perhaps I am estimating
> > incorrectly - that's why I wanted this discussion on a user-facing list
> > rather than a dev-facing list.
>
> Obviously the pointer is there for them to follow into the rabbit hole
> of the dev lists. *grin*
>
> > Another example use case that I do a lot on non-secure clusters is:
> hadoop
> > fs -Dhadoop.job.ugi=hadoop,hadoop <something I want to do as a
> superuser>.
> > The permissions model we have in 0.20 obviously isn't secure, but it's
> nice
> > to avoid accidental mistakes, and making it easy to "sudo" like that is
> > handy.
>
> It might make sense to add a new switch ( -user ?) to hadoop fs that
> does a doAs before doing the
> shell command. You could even make it fancy and try to be a proxy user
> if security is turned on.
>
Yep, I agree - I think either (ab)using proxyuser functionality or adding
some new "sudoers" like configuration would be very handy and we should do
it.
>
> > Regardless of our particular opinions, isn't our policy that we cannot
> break
> > API compatibility between versions without a one-version deprecation
> period?
>
> There wasn't a way to keep UGI stable. It was a broken design before
> the security work. It is marked evolving so we try to minimize
> breakage, but it isn't prohibited.
>
>
I agree that keeping API compatibility for UGI was probably impossible, and
respect that. But it would certainly be very easy to do a patch like the
following:
JobClient(Configuration conf) {
if (conf.get("hadoop.job.ugi") != null &&
UserGroupInformation.isSecurityEnabled()) {
LOG.warn("Stop being evil. Don't use hadoop.job.ugi! RAAWR");
UserGroupInformation.createRemoteUser(...).doAs() { create proxy }
} else {
create normal RPC proxy;
}
}
... and the same on the HDFS side.
Would you -1 such a compatibility layer?
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
