Hi Steve,
Thanks again for your in depth replies, we found your comments quite
useful. A few responses inline:
On 8/9/13 10:31 AM, Steve Loughran wrote:
On 8 August 2013 21:51, Matevz Tadel wrote:
We already do fallback to xrootd on open failures from our application
framework ... the job
On 8 August 2013 21:51, Matevz Tadel wrote:
> Hi Steve,
>
> Thank you very much for the reality check! Some more answers inline ...
>
>
> On 8/8/13 1:30 PM, Steve Loughran wrote:
>
>> On 7 August 2013 10:59, Jeff Dost wrote:
>>
>> Hello,
>>>
>>> We work in a software development team at the UCS
Hi Steve,
Thank you very much for the reality check! Some more answers inline ...
On 8/8/13 1:30 PM, Steve Loughran wrote:
On 7 August 2013 10:59, Jeff Dost wrote:
Hello,
We work in a software development team at the UCSD CMS Tier2 Center. We
would like to propose a mechanism to allow one
This is being targeted for release 2.3.
2.1.x release stream is for stabilizing. When it reaches stability, 2.2 GA
will be released. The current features in development will make it to 2.3,
including HDFS-2832.
On Thu, Aug 8, 2013 at 2:04 PM, Matevz Tadel wrote:
> Thanks Colin, I subscribed to
Thanks Colin, I subscribed to HDFS-2832 so that I can follow the development
there. I assume this is targeting release 2.1.
Best,
Matevz
On 08/08/13 12:10, Colin McCabe wrote:
There is work underway to decouple the block layer and the namespace
layer of HDFS from each other. Once this is don
On 7 August 2013 10:59, Jeff Dost wrote:
> Hello,
>
> We work in a software development team at the UCSD CMS Tier2 Center. We
> would like to propose a mechanism to allow one to subclass the
> DFSInputStream in a clean way from an external package. First I'd like to
> give some motivation on wh
There is work underway to decouple the block layer and the namespace
layer of HDFS from each other. Once this is done, block behaviors
like the one you describe will be easy to implement. It's a use case
very similar to the hierarchical storage management (HSM) use case
that we've discussed befor
Hi everybody,
I'm jumping in as Jeff is away due to an unexpected annoyance involving
Californian wildlife.
On 8/7/13 7:47 PM, Andrew Wang wrote:
Blocks are supposed to be an internal abstraction within HDFS, and aren't an
inherent part of FileSystem (the user-visible class used to access all
Blocks are supposed to be an internal abstraction within HDFS, and aren't
an inherent part of FileSystem (the user-visible class used to access all
Hadoop filesystems).
Is it possible to instead deal with files and offsets? On a read failure,
you could open a stream to the same file on the backup
Thank you for the suggestion, but we don't see how simply wrapping a
FileSystem object would be sufficient in our use case. The reason why
is we need to catch and handle read exceptions at the block level.
There aren't any public methods available in the high level FileSystem
abstraction laye
I don't think exposing DFSClient and DistributedFileSystem members is
necessary to achieve what you're trying to do. We've got wrapper
FileSystems like FilterFileSystem and ViewFileSystem which you might be
able to use for inspiration, and the HCFS wiki lists some third-party
FileSystems that might
Hi Jeff,
Do you need to subclass or could you simply wrap? Generally composition as
opposed to inheritance is a lot safer way of integrating software written
by different parties, since inheritance exposes all the implementation
details which are subject to change.
-Todd
On Wed, Aug 7, 2013 at 1
Hello Jeff
Is it something that could go under HCFS project?
http://wiki.apache.org/hadoop/HCFS
(I might be wrong?)
Joe
On 8/7/13 10:59 AM, "Jeff Dost" wrote:
>Hello,
>
>We work in a software development team at the UCSD CMS Tier2 Center. We
>would like to propose a mechanism to allow one to
Hello,
We work in a software development team at the UCSD CMS Tier2 Center. We
would like to propose a mechanism to allow one to subclass the
DFSInputStream in a clean way from an external package. First I'd like
to give some motivation on why and then will proceed with the details.
We hav
14 matches
Mail list logo
