Nikhil, You should search the mailing list archives, but I'm not aware of any discussion around that. If you wanted to try something like that, you might be able to accomplish it via FUSE or similar. For example, this page lists ways you can mount HDFS as a normal filesystem, including fuse-based solutions: https://wiki.apache.org/hadoop/MountableHDFS
-Ewen On Tue, Jan 26, 2016 at 5:24 PM, Steve Brandon <st...@sparkcentral.com> wrote: > I would give this a look and see if it works, since you mention HDFS > > https://github.com/linkedin/gobblin > > On Tue, Jan 26, 2016 at 3:49 PM, Nikhil Joshi <toughsh...@gmail.com> > wrote: > > > Hi, > > > > I'm new to the Kafka community. Has there been any discussion around > > plugging-in external filesystems (like HDFS) for Kafka persistence? > Though > > local filesystem gives the best throughput for the append-only Kafka log > > data-structure, other filesystems might be able to provide better storage > > efficiency, scale or geo-replication at the cost of throughput. > > > > Any pointers here? > > > > Nikhil > > > -- Thanks, Ewen
