Re: [PATCH v2] Support for write stream IDs

2015-05-11 Thread Martin K. Petersen
> "Jens" == Jens Axboe writes: Jens, Jens> There are actual technical challenges on the device side that Jens> sometimes interferes. [...] Right now we use the same protocol to speak to USB keys and million dollar storage arrays. That's because the protocol was designed to be abstract and c

Re: [PATCH v2] Support for write stream IDs

2015-05-08 Thread Jens Axboe
On 05/07/2015 01:19 PM, Martin K. Petersen wrote: "Jens" == Jens Axboe writes: Jens> This wont solve the problem of devices having too few streams. But Jens> it'll work regardless, we'll just have to push them separately to Jens> do that. It's not an easy problem for them either, resource Jens

Re: [PATCH v2] Support for write stream IDs

2015-05-07 Thread Martin K. Petersen
> "Jens" == Jens Axboe writes: Jens> This wont solve the problem of devices having too few streams. But Jens> it'll work regardless, we'll just have to push them separately to Jens> do that. It's not an easy problem for them either, resource Jens> constraints on the device side could exclude

Re: [PATCH v2] Support for write stream IDs

2015-05-06 Thread Jens Axboe
On 05/06/2015 08:26 AM, Peter Zijlstra wrote: On Tue, May 05, 2015 at 06:09:18PM -0400, Martin K. Petersen wrote: "Jens" == Jens Axboe writes: Jens> I'm not trying to make a shortcut. I deliberately do not want to Jens> make ID generation/assignment part of the kernel. There's no Jens> reason

Re: [PATCH v2] Support for write stream IDs

2015-05-06 Thread Jens Axboe
On 05/05/2015 04:09 PM, Martin K. Petersen wrote: "Jens" == Jens Axboe writes: Jens> I'm not trying to make a shortcut. I deliberately do not want to Jens> make ID generation/assignment part of the kernel. There's no Jens> reason that can't exist outside of the kernel, in a libstreamid or Jens

Re: [PATCH v2] Support for write stream IDs

2015-05-06 Thread Boaz Harrosh
On 05/06/2015 01:09 AM, Martin K. Petersen wrote: >> "Jens" == Jens Axboe writes: <> > > The only sensible solution is for the kernel to manage the stream > IDs. And for them to be plentiful. The storage device is free to ignore > them, do LRU or whatever it pleases to manage them if it has a

Re: [PATCH v2] Support for write stream IDs

2015-05-06 Thread Peter Zijlstra
On Tue, May 05, 2015 at 06:09:18PM -0400, Martin K. Petersen wrote: > > "Jens" == Jens Axboe writes: > > Jens> I'm not trying to make a shortcut. I deliberately do not want to > Jens> make ID generation/assignment part of the kernel. There's no > Jens> reason that can't exist outside of the k

Re: [PATCH v2] Support for write stream IDs

2015-05-05 Thread Martin K. Petersen
> "Jens" == Jens Axboe writes: Jens> I'm not trying to make a shortcut. I deliberately do not want to Jens> make ID generation/assignment part of the kernel. There's no Jens> reason that can't exist outside of the kernel, in a libstreamid or Jens> similar. That just perpetuates the broken mo

Re: [PATCH v2] Support for write stream IDs

2015-05-05 Thread Jens Axboe
On 05/05/2015 03:39 PM, Martin K. Petersen wrote: "Jens" == Jens Axboe writes: Jens> The kernel patches deal only with ensuring that the stream Jens> information gets passed down. If the device requires explicit Jens> stream open/close actions, then that needs to be handled on the Jens> side.

Re: [PATCH v2] Support for write stream IDs

2015-05-05 Thread Martin K. Petersen
> "Jens" == Jens Axboe writes: Jens> The kernel patches deal only with ensuring that the stream Jens> information gets passed down. If the device requires explicit Jens> stream open/close actions, then that needs to be handled on the Jens> side. Well, I'm deeply concerned about that "on the

Re: [PATCH v2] Support for write stream IDs

2015-05-05 Thread Jens Axboe
On 05/05/2015 02:51 PM, Jeff Moyer wrote: Jens Axboe writes: Hi, Changes since the last posting: - Added a specific per-file fadvise setting. POSIX_FADV_STREAMID sets the inode and file stream ID, POSIX_FADV_STREAMID_FILE sets just the file stream ID. - Addressed review comments. I'v

Re: [PATCH v2] Support for write stream IDs

2015-05-05 Thread Jeff Moyer
Jens Axboe writes: > Hi, > > Changes since the last posting: > > - Added a specific per-file fadvise setting. POSIX_FADV_STREAMID sets > the inode and file stream ID, POSIX_FADV_STREAMID_FILE sets just the > file stream ID. > > - Addressed review comments. > > I've since run some testing with

Re: [PATCH v2] Support for write stream IDs

2015-05-05 Thread Christoph Hellwig
On Tue, May 05, 2015 at 02:31:34PM -0600, Jens Axboe wrote: > There is a user, we are using it. And there's no data structure bloating, > both the file and inode additions are filling existing holes. FYI, they do increase the size on any 32-bit architecture, which is where it hurts most. -- To uns

Re: [PATCH v2] Support for write stream IDs

2015-05-05 Thread Christoph Hellwig
On Tue, May 05, 2015 at 02:31:34PM -0600, Jens Axboe wrote: > Even outside of that, there are use cases for caching that need not have > hardware assist. Could, would. But in the meantime you'd adding dead wood kernel code, and even worse user interfaces. > >Merging infrastructure without any us

Re: [PATCH v2] Support for write stream IDs

2015-05-05 Thread Jens Axboe
On 05/05/2015 02:20 PM, Christoph Hellwig wrote: On Tue, May 05, 2015 at 02:12:01PM -0600, Jens Axboe wrote: We can't merge the NVMe bits until the proposal is included/finalized. But this is a problem. I don't want to add this to the Facebook kernel until we know the API is stable, while I have

Re: [PATCH v2] Support for write stream IDs

2015-05-05 Thread Christoph Hellwig
On Tue, May 05, 2015 at 02:12:01PM -0600, Jens Axboe wrote: > We can't merge the NVMe bits until the proposal is included/finalized. But > this is a problem. I don't want to add this to the Facebook kernel until we > know the API is stable, while I have no problem adding experimental NVMe > changes

Re: [PATCH v2] Support for write stream IDs

2015-05-05 Thread Jens Axboe
On 05/05/2015 02:07 PM, Christoph Hellwig wrote: On Tue, May 05, 2015 at 02:02:54PM -0600, Jens Axboe wrote: Unless there are any grave concerns here, I'd like to merge this for 4.2. Without a driver implementing the feature I don't see any reason to even consider merging it. We can't merge

Re: [PATCH v2] Support for write stream IDs

2015-05-05 Thread Christoph Hellwig
On Tue, May 05, 2015 at 02:02:54PM -0600, Jens Axboe wrote: > Unless there are any grave concerns here, I'd like to merge this for > 4.2. Without a driver implementing the feature I don't see any reason to even consider merging it. -- To unsubscribe from this list: send the line "unsubscribe linux

[PATCH v2] Support for write stream IDs

2015-05-05 Thread Jens Axboe
Hi, Changes since the last posting: - Added a specific per-file fadvise setting. POSIX_FADV_STREAMID sets the inode and file stream ID, POSIX_FADV_STREAMID_FILE sets just the file stream ID. - Addressed review comments. I've since run some testing with write streams. Test case was a RocksDB

[PATCH v2] Support for write stream IDs

2015-04-18 Thread Jens Axboe
Hi, v2 of this posting. Changes since v1: - Rebased on top of current master. - Fix EINVAL -> -EINVAL typo. - Cleanup up BIO_STREAM_OFFSET definition. - Pack i_streamid and f_streamid better into struct file and struct inode. - Add a separate per-file hint, FADV_FILE_STREAMID. This only set