Hi Abe, Thanks for the quick reply. Currently, the postgres instance in all that is kerberized. When I add the kerberosServerName I get:
java.sql.SQLException: No suitable driver found for jdbc:postgresql://postgres.lyleco.com:5432;kerberosServerName=postgres/[email protected] or java.sql.SQLException: No suitable driver found for jdbc:postgresql://postgres.lyleco.com:5432&kerberosServerName=postgres/[email protected] (I tried both). I'm also troubled by the repeated kinit: Generic preauthentication failure while getting initial credentials at the very top. I don't understand how sqoop is delegating, but the kinit is surprising. -D... On Thu, Nov 20, 2014 at 6:15 PM, Abraham Elmahrek <[email protected]> wrote: > This is a long shot... but I thought I'd say something. > > If you have Hadoop configured for Kerberos, then Sqoop should automatically > use your kerberos credentials to run a MapReduce job. That MapReduce job > will run with its own Kerberos principal (normally yarn or mapred). This > means that Hadoop has access to postgresql as long as it requests a session > from the KDC, assuming postgresql and Hadoop are part of the same realm. My > understanding is the JDBC driver will handle this for you if you provide the > postgresql service name via "kerberosServerName" in the JDBC connection > string: > "jdbc:postgresql://postgres.lyleco.com:5432;kerberosServerName=postgres/<host>@<REALM>". > > Ref: http://jdbc.postgresql.org/documentation/84/connect.html > > -Abe > > On Thu, Nov 20, 2014 at 2:53 PM, David Lyle <[email protected]> wrote: >> >> Hi, >> >> I'm trying to use Sqoop to connect to a kerberized Postgres instance. >> At this time, I can connect from psql, but my sqoop session gives me >> the log below. Any idea where I'm going wrong? >> >> Thanks! >> >> -David... >> >> >> sqoop list-databases -Djavax.security.auth.debug=true --connect >> "jdbc:postgresql://postgres.lyleco.com:5432" -verbose >> >> kinit: Generic preauthentication failure while getting initial credentials >> kinit: Generic preauthentication failure while getting initial credentials >> >> 14/11/20 17:26:37 INFO sqoop.Sqoop: Running Sqoop version: >> 1.4.4.2.1.7.0-784 >> 14/11/20 17:26:37 DEBUG tool.BaseSqoopTool: Enabled debug logging. >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Loaded manager factory: >> com.cloudera.sqoop.manager.DefaultManagerFactory >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Trying ManagerFactory: >> com.cloudera.sqoop.manager.DefaultManagerFactory >> 14/11/20 17:26:37 DEBUG manager.DefaultManagerFactory: Trying with >> scheme: jdbc:postgresql: >> 14/11/20 17:26:37 INFO manager.SqlManager: Using default fetchSize of 1000 >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Instantiated ConnManager >> org.apache.sqoop.manager.PostgresqlManager@1663108 >> 14/11/20 17:26:37 DEBUG manager.SqlManager: No connection paramenters >> specified. Using regular API for making connection. >> 14/11/20 17:26:37 ERROR manager.CatalogQueryManager: Failed to list >> databases >> org.postgresql.util.PSQLException: GSS Authentication failed >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49) >> at >> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380) >> at >> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) >> at >> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) >> at >> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) >> at >> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) >> at >> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) >> at >> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) >> at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) >> at org.postgresql.Driver.makeConnection(Driver.java:393) >> at org.postgresql.Driver.connect(Driver.java:267) >> at java.sql.DriverManager.getConnection(DriverManager.java:571) >> at java.sql.DriverManager.getConnection(DriverManager.java:233) >> at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824) >> at >> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) >> at >> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57) >> at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49) >> at org.apache.sqoop.Sqoop.run(Sqoop.java:147) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) >> at org.apache.sqoop.Sqoop.main(Sqoop.java:240) >> Caused by: javax.security.auth.login.LoginException: No LoginModules >> configured for pgjdbc >> at javax.security.auth.login.LoginContext.init(LoginContext.java:273) >> at javax.security.auth.login.LoginContext.<init>(LoginContext.java:418) >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42) >> ... 22 more >> 14/11/20 17:26:37 ERROR sqoop.Sqoop: Got exception running Sqoop: >> java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS >> Authentication failed >> java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS >> Authentication failed >> at >> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:73) >> at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49) >> at org.apache.sqoop.Sqoop.run(Sqoop.java:147) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) >> at org.apache.sqoop.Sqoop.main(Sqoop.java:240) >> Caused by: org.postgresql.util.PSQLException: GSS Authentication failed >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49) >> at >> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380) >> at >> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) >> at >> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) >> at >> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) >> at >> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) >> at >> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) >> at >> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) >> at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) >> at org.postgresql.Driver.makeConnection(Driver.java:393) >> at org.postgresql.Driver.connect(Driver.java:267) >> at java.sql.DriverManager.getConnection(DriverManager.java:571) >> at java.sql.DriverManager.getConnection(DriverManager.java:233) >> at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824) >> at >> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) >> at >> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57) >> ... 7 more >> Caused by: javax.security.auth.login.LoginException: No LoginModules >> configured for pgjdbc >> at javax.security.auth.login.LoginContext.init(LoginContext.java:273) >> at javax.security.auth.login.LoginContext.<init>(LoginContext.java:418) >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42) >> ... 22 more > >
