> thing will occur.
>
> --
> bit1...@163.com
>
>
> *From:* Saisai Shao
> *Date:* 2015-04-29 10:54
> *To:* Vadim Bichutskiy
> *CC:* bit1...@163.com; lokeshkumar ; user
>
> *Subject:* Re: Re: Spark streaming - textFileStream/fileStream - Get file
&
ur.
>>
>> --
>> bit1...@163.com
>>
>>
>> *From:* Saisai Shao
>> *Date:* 2015-04-29 10:54
>> *To:* Vadim Bichutskiy
>> *CC:* bit1...@163.com; lokeshkumar ; user
>>
>> *Subject:* Re: Re: Spark stre
15:50
To: Akhil Das
CC: bit1...@163.com; Vadim Bichutskiy; lokeshkumar; user
Subject: Re: Re: Spark streaming - textFileStream/fileStream - Get file name
Yes, looks like a solution but quite tricky. You have to parse the debug string
to get the file name, also relies on HadoopRDD to get the file
: Spark streaming - textFileStream/fileStream - Get file name
I think it might be useful in Spark Streaming's file input stream, but not sure
is it useful in SparkContext#textFile, since we specify the file by our own, so
why we still need to know the file name.
I will open up a JIRA to me
o way in RDD
>> to figure out the file information where the data in RDD is from
>>
>> --
>> bit1...@163.com
>>
>>
>> *From:* Saisai Shao
>> *Date:* 2015-04-29 10:10
>> *To:* lokeshkumar
>> *CC:* spark users
>
ta in RDD is from
>
> --
> bit1...@163.com
>
>
> *From:* Saisai Shao
> *Date:* 2015-04-29 10:10
> *To:* lokeshkumar
> *CC:* spark users
> *Subject:* Re: Spark streaming - textFileStream/fileStream - Get file name
> I think currently the
: Re: Spark streaming - textFileStream/fileStream - Get file name
I think currently there's no API in Spark Streaming you can use to get the file
names for file input streams. Actually it is not trivial to support this, may
be you could file a JIRA with wishes you want the community to suppor
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Spark-streaming-textFileStream-fileStream-Get-file-name-tp22692.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---
-textFileStream-fileStream-Get-file-name-tp22692.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h
