Daniel,
Instead of using localhost:19998, you may want to use the real ip address
TachyonMaster is configured. You should be able to see more info in
Tachyon's UI as well. More info could be found here:
http://tachyon-project.org/master/Running-Tachyon-on-EC2.html
Best,
Haoyuan
On Fri, Apr 24, 2015 at 12:21 PM, Daniel Mahler <dmah...@gmail.com> wrote:
> I have a cluster launched with spark-ec2.
> I can see a TachyonMaster process running,
> but I do not seem to be able to use tachyon from the spark-shell.
>
> if I try
>
> rdd.saveAsTextFile("tachyon://localhost:19998/path")
> I get
>
> 15/04/24 19:18:31 INFO TaskSetManager: Starting task 12.2 in stage 1.0
> (TID 216, ip-10-63-69-48.ec2.internal, PROCESS_LOCAL, 1383 bytes)
> 15/04/24 19:18:31 WARN TaskSetManager: Lost task 32.2 in stage 1.0 (TID
> 177, ip-10-63-69-48.ec2.internal): java.io.IOException: Failed to connect
> to master localhost/127.0.0.1:19998 after 5 attempts
> at tachyon.client.TachyonFS.connect(TachyonFS.java:293)
> at tachyon.client.TachyonFS.getUnderfsAddress(TachyonFS.java:1224)
> at tachyon.hadoop.TFS.initialize(TFS.java:289)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2262)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:86)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2296)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2278)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:316)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:194)
> at
> org.apache.spark.SparkHadoopWriter.open(SparkHadoopWriter.scala:83)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$13.apply(PairRDDFunctions.scala:1068)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$13.apply(PairRDDFunctions.scala:1059)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: tachyon.org.apache.thrift.TException: Failed to connect to
> master localhost/127.0.0.1:19998 after 5 attempts
> at tachyon.master.MasterClient.connect(MasterClient.java:178)
> at tachyon.client.TachyonFS.connect(TachyonFS.java:290)
> ... 17 more
> Caused by: tachyon.org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused
> at
> tachyon.org.apache.thrift.transport.TSocket.open(TSocket.java:185)
> at
> tachyon.org.apache.thrift.transport.TFramedTransport.open(TFramedTransport.java:81)
> at tachyon.master.MasterClient.connect(MasterClient.java:156)
> ... 18 more
> Caused by: java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:579)
> at
> tachyon.org.apache.thrift.transport.TSocket.open(TSocket.java:180)
> ... 20 more
>
>
> What do I need to do before I can use tachyon?
>
> thanks
> Daniel
>
--
Haoyuan Li
CEO, Tachyon Nexus <http://www.tachyonnexus.com/>
AMPLab, EECS, UC Berkeley http://www.cs.berkeley.edu/~haoyuan/
