[ANNOUNCE] Three things.

2019-08-29 Thread Daniel Phillips
Hi folks, how's it going? Over here, we have been rather busy lately, and for the last five years or so to be honest. Today it is my pleasure to be able to announce three GPL open source projects: 1) Shardmap Shardmap is the next generation directory index developed for Tux3, and which we are now

Re: [FYI] tux3: Core changes

2015-07-31 Thread Daniel Phillips
On Friday, July 31, 2015 5:00:43 PM PDT, Daniel Phillips wrote: Note: Hirofumi's email is clear, logical and speaks to the question. This branch of the thread is largely pointless, though it essentially says the same thing in non-technical terms. Perhaps your next response should be to Hir

Re: [FYI] tux3: Core changes

2015-07-31 Thread Daniel Phillips
On Friday, July 31, 2015 3:27:12 PM PDT, David Lang wrote: On Fri, 31 Jul 2015, Daniel Phillips wrote: On Friday, July 31, 2015 11:29:51 AM PDT, David Lang wrote: ... you weren't asking about any particular feature of Tux, you were asking if we were still willing to push out stuff

Re: [FYI] tux3: Core changes

2015-07-31 Thread Daniel Phillips
On Friday, July 31, 2015 11:29:51 AM PDT, David Lang wrote: We, the Linux Community have less tolerance for losing people's data and preventing them from operating than we used to when it was all tinkerer's personal data and secondary systems. So rather than pushing optimizations out to everyo

Re: [FYI] tux3: Core changes

2015-07-31 Thread Daniel Phillips
On Friday, July 31, 2015 11:29:51 AM PDT, David Lang wrote: If you define this as "loosing our mojo", then yes we have. A pity. There remains so much to do that simply will not get done in the absence of mojo. Regards, Daniel -- To unsubscribe from this list: send the line "unsubscribe linux

Re: [FYI] tux3: Core changes

2015-07-31 Thread Daniel Phillips
On Friday, July 31, 2015 8:37:35 AM PDT, Raymond Jennings wrote: Returning ENOSPC when you have free space you can't yet prove is safer than not returning it and risking a data loss when you get hit by a write/commit storm. :) Remember when delayed allocation was scary and unproven, because pro

Re: Tux3 Report: How fast can we fail?

2015-05-27 Thread Daniel Phillips
On 05/27/2015 02:39 PM, Pavel Machek wrote: > On Wed 2015-05-27 11:28:50, Daniel Phillips wrote: >> On Tuesday, May 26, 2015 11:41:39 PM PDT, Mosis Tembo wrote: >>> On Tue, May 26, 2015 at 6:03 PM, Pavel Machek wrote: >>> >>>> >>>>> We ident

Re: [FYI] tux3: Core changes

2015-05-27 Thread Daniel Phillips
On 05/27/2015 02:37 PM, Pavel Machek wrote: > On Wed 2015-05-27 11:09:25, Daniel Phillips wrote: >> On Wednesday, May 27, 2015 12:41:37 AM PDT, Pavel Machek wrote: >>> On Fri 2015-05-15 02:38:33, Daniel Phillips wrote: >>>> On 05/14/2015 08:06 PM, Rik van Riel wrot

Re: Tux3 Report: How fast can we fail?

2015-05-27 Thread Daniel Phillips
On Tuesday, May 26, 2015 11:41:39 PM PDT, Mosis Tembo wrote: On Tue, May 26, 2015 at 6:03 PM, Pavel Machek wrote: We identified the following quality metrics for this algorithm: 1) Never fails to detect out of space in the front end. 2) Always fills a volume to 100% before reporting out o

Re: [FYI] tux3: Core changes

2015-05-27 Thread Daniel Phillips
On Wednesday, May 27, 2015 12:41:37 AM PDT, Pavel Machek wrote: On Fri 2015-05-15 02:38:33, Daniel Phillips wrote: On 05/14/2015 08:06 PM, Rik van Riel wrote: ... Umm. Why do you think it is only issue for executable files? I meant: files with code in them, that will be executed. Please

Re: [FYI] tux3: Core changes

2015-05-26 Thread Daniel Phillips
On 05/26/2015 02:36 PM, Rik van Riel wrote: > On 05/26/2015 04:22 PM, Daniel Phillips wrote: >> On 05/26/2015 02:00 AM, Jan Kara wrote: >>> So my opinion is: Don't fork the page if page_count is elevated. You can >>> just wait for the IO if you need stable p

Re: [FYI] tux3: Core changes

2015-05-26 Thread Daniel Phillips
On 05/26/2015 02:00 AM, Jan Kara wrote: > On Tue 26-05-15 01:08:56, Daniel Phillips wrote: >> On Tuesday, May 26, 2015 12:09:10 AM PDT, Jan Kara wrote: >>> E.g. video drivers (or infiniband or direct IO for that matter) which >>> have buffers in user memory (may be mma

Re: [FYI] tux3: Core changes

2015-05-26 Thread Daniel Phillips
Hi Sergey, On 05/26/2015 03:22 AM, Sergey Senozhatsky wrote: > > Hello, > > is it possible to page-fork-bomb the system by some 'malicious' app? Not in any new way. A page fork can happen either in the front end, where it has to wait for memory like any other normal memory user, or in the backe

Re: [FYI] tux3: Core changes

2015-05-26 Thread Daniel Phillips
On Monday, May 25, 2015 11:13:46 PM PDT, David Lang wrote: I'm assuming that Rik is talking about whatever has the reference to the page via one of the methods that he talked about. This would be a good moment to provide specifics. Regards, Daniel -- To unsubscribe from this list: send the l

Re: [FYI] tux3: Core changes

2015-05-26 Thread Daniel Phillips
On Tuesday, May 26, 2015 12:09:10 AM PDT, Jan Kara wrote: E.g. video drivers (or infiniband or direct IO for that matter) which have buffers in user memory (may be mmapped file), grab references to pages and hand out PFNs of those pages to the hardware to store data in them... If you fork a pag

Re: [FYI] tux3: Core changes

2015-05-25 Thread Daniel Phillips
On Monday, May 25, 2015 11:04:39 PM PDT, David Lang wrote: if the page gets modified again, will that cause any issues? what if the page gets modified before the copy gets written out, so that there are two dirty copies of the page in the process of being written? David Lang How is the page

Re: [FYI] tux3: Core changes

2015-05-25 Thread Daniel Phillips
On Monday, May 25, 2015 9:25:44 PM PDT, Rik van Riel wrote: On 05/21/2015 03:53 PM, Daniel Phillips wrote: On Wednesday, May 20, 2015 8:51:46 PM PDT, David Lang wrote: how do you prevent it from continuing to interact with the old version of the page and never see updates or have it's ch

Re: [FYI] tux3: Core changes

2015-05-21 Thread Daniel Phillips
On Wednesday, May 20, 2015 8:51:46 PM PDT, David Lang wrote: how do you prevent it from continuing to interact with the old version of the page and never see updates or have it's changes reflected on the current page? Why would it do that, and what would be surprising about it? Did you have a

[WIP][PATCH] tux3: preliminatry nospace handling

2015-05-21 Thread Daniel Phillips
Hi Josef, This is a rollup patch for preliminary nospace handling in Tux3, in line with my post here: http://lkml.iu.edu/hypermail/linux/kernel/1505.1/03167.html You still have ENOSPC issues. Maybe it would be helpful to look at what we have done. I saw a reproducible case with 1,000 tasks

Re: [FYI] tux3: Core changes

2015-05-20 Thread Daniel Phillips
On 05/20/2015 03:51 PM, Daniel Phillips wrote: > On 05/20/2015 12:53 PM, Rik van Riel wrote: >> How does tux3 prevent a user of find_get_page() from reading from >> or writing into the pre-COW page, instead of the current page? > > Careful control of the dirty bits (we have

Re: [FYI] tux3: Core changes

2015-05-20 Thread Daniel Phillips
On 05/20/2015 12:53 PM, Rik van Riel wrote: > On 05/20/2015 12:22 PM, Daniel Phillips wrote: >> On 05/20/2015 07:44 AM, Jan Kara wrote: >>> On Tue 19-05-15 13:33:31, David Lang wrote: > >>> Yeah, that's what I meant. If you create a function which manipula

Re: [FYI] tux3: Core changes

2015-05-20 Thread Daniel Phillips
On 05/20/2015 07:44 AM, Jan Kara wrote: > On Tue 19-05-15 13:33:31, David Lang wrote: >> On Tue, 19 May 2015, Daniel Phillips wrote: >> >>>> I understand that Tux3 may avoid these issues due to some other mechanisms >>>> it internally has but if page fo

Re: [FYI] tux3: Core changes

2015-05-19 Thread Daniel Phillips
Hi Jan, On 05/19/2015 07:00 AM, Jan Kara wrote: > On Thu 14-05-15 01:26:23, Daniel Phillips wrote: >> Hi Rik, >> >> Our linux-tux3 tree currently currently carries this 652 line diff >> against core, to make Tux3 work. This is mainly by Hirofumi, except >> the fs

Re: [FYI] tux3: Core changes

2015-05-18 Thread Daniel Phillips
On 05/17/2015 07:20 PM, Rik van Riel wrote: > On 05/17/2015 09:26 AM, Boaz Harrosh wrote: >> On 05/14/2015 03:59 PM, Rik van Riel wrote: >>> The issue is that things like ptrace, AIO, infiniband >>> RDMA, and other direct memory access subsystems can take >>> a reference to page A, which Tux3 clone

Re: [FYI] tux3: Core changes

2015-05-15 Thread Daniel Phillips
On 05/15/2015 01:09 AM, Mel Gorman wrote: > On Thu, May 14, 2015 at 11:06:22PM -0400, Rik van Riel wrote: >> On 05/14/2015 08:06 PM, Daniel Phillips wrote: >>>> The issue is that things like ptrace, AIO, infiniband >>>> RDMA, and other direct memory access subsys

Re: [FYI] tux3: Core changes

2015-05-15 Thread Daniel Phillips
On 05/14/2015 08:06 PM, Rik van Riel wrote: > On 05/14/2015 08:06 PM, Daniel Phillips wrote: >>> The issue is that things like ptrace, AIO, infiniband >>> RDMA, and other direct memory access subsystems can take >>> a reference to page A, which Tux3 clones into a n

Re: [FYI] tux3: Core changes

2015-05-14 Thread Daniel Phillips
Hi Rik, Added Mel, Andrea and Peterz to CC as interested parties. There are probably others, please just jump in. On 05/14/2015 05:59 AM, Rik van Riel wrote: > On 05/14/2015 04:26 AM, Daniel Phillips wrote: >> Hi Rik, >> >> Our linux-tux3 tree currently currently carri

[FYI] tux3: Core changes

2015-05-14 Thread Daniel Phillips
Hi Rik, Our linux-tux3 tree currently currently carries this 652 line diff against core, to make Tux3 work. This is mainly by Hirofumi, except the fs-writeback.c hook, which is by me. The main part you may be interested in is rmap.c, which addresses the issues raised at the 2013 Linux Storage File

[WIP] tux3: Optimized fsync

2015-05-14 Thread Daniel Phillips
elta_ref) sb->delta_pending++; wake_up_all(&sb->delta_transition_wq); } + +#ifdef __KERNEL__ +#include "commit_fsync.c" +#endif diff --git a/fs/tux3/commit_fsync.c b/fs/tux3/commit_fsync.c new file mode 100644 index 000..9a59c59 --- /dev/null +++ b/fs/tux3/

Re: Tux3 Report: How fast can we fail?

2015-05-13 Thread Daniel Phillips
Addendum to that post... On 05/12/2015 10:46 AM, I wrote: > ...For example, we currently > overestimate the cost of a rewrite because we would need to go poking > around in btrees to do that more accurately. Fixing that will be quite > a bit of work... Ah no, I was wrong about that, it will not b

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-13 Thread Daniel Phillips
On Wednesday, May 13, 2015 1:25:38 PM PDT, Martin Steigerwald wrote: Am Mittwoch, 13. Mai 2015, 12:37:41 schrieb Daniel Phillips: On 05/13/2015 12:09 PM, Martin Steigerwald wrote: ... Daniel, if you want to change the process of patch review and inclusion into the kernel, model an example

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-13 Thread Daniel Phillips
On Wednesday, May 13, 2015 1:02:34 PM PDT, Jeremy Allison wrote: On Wed, May 13, 2015 at 12:37:41PM -0700, Daniel Phillips wrote: On 05/13/2015 12:09 PM, Martin Steigerwald wrote: ... Daniel, please listen to Martin. He speaks a fundamental truth here. As you know, I am also interested in

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-13 Thread Daniel Phillips
On 05/13/2015 12:09 PM, Martin Steigerwald wrote: > Daniel, what are you trying to achieve here? > > I thought you wanted to create interest for your filesystem and acceptance > for merging it. > > What I see you are actually creating tough is something different. > > Is what you see after you

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-13 Thread Daniel Phillips
On 05/13/2015 06:08 AM, Mike Galbraith wrote: > On Wed, 2015-05-13 at 04:31 -0700, Daniel Phillips wrote: >> Third possibility: build from our repository, as Mike did. > > Sorry about that folks. I've lost all interest, it won't happen again. Thanks for your valuable

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-13 Thread Daniel Phillips
On 05/13/2015 04:31 AM, Daniel Phillips wrote: Let me be the first to catch that arithmetic error > Let's say our delta size is 400MB (typical under load) and we leave > a "nice big gap" of 112 MB after flushing each one. Let's say we do > two thousand of those

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-13 Thread Daniel Phillips
On 05/13/2015 12:25 AM, Pavel Machek wrote: > On Mon 2015-05-11 16:53:10, Daniel Phillips wrote: >> Hi Pavel, >> >> On 05/11/2015 03:12 PM, Pavel Machek wrote: >>>>> It is a fact of life that when you change one aspect of an intimately >>>>> inte

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-12 Thread Daniel Phillips
On 05/12/2015 03:35 PM, David Lang wrote: > On Tue, 12 May 2015, Daniel Phillips wrote: >> On 05/12/2015 02:30 PM, David Lang wrote: >>> You need to get out of the mindset that Ted and Dave are Enemies that you >>> need to overcome, they are >>> friendly com

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-12 Thread Daniel Phillips
On 05/12/2015 02:30 PM, David Lang wrote: > On Tue, 12 May 2015, Daniel Phillips wrote: >> Phoronix published a headline that identifies Dave Chinner as >> someone who takes shots at other projects. Seems pretty much on >> the money to me, and it ought to be obvious why he d

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-12 Thread Daniel Phillips
On 05/12/2015 02:30 PM, David Lang wrote: > On Tue, 12 May 2015, Daniel Phillips wrote: >> Phoronix published a headline that identifies Dave Chinner as >> someone who takes shots at other projects. Seems pretty much on >> the money to me, and it ought to be obvious why he d

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-12 Thread Daniel Phillips
On 05/12/2015 11:39 AM, David Lang wrote: > On Mon, 11 May 2015, Daniel Phillips wrote: >>> ...it's the mm and core kernel developers that need to >>> review and accept that code *before* we can consider merging tux3. >> >> Please do not say "we" w

Tux3 Report: How fast can we fail?

2015-05-12 Thread Daniel Phillips
ally was. Regards, Daniel [1] Overhead of an uncontended bus locked add is about 6 nanoseconds on my i5, and about ten times higher when contended. /* * Blurt v0.0 * * A trivial multitasking filesystem load generator * * Daniel Phillips, June 2015 * * to build: c99 -Wall blurt.c -oblurt * to

Re: Tux3 Report: How fast can we fsync?

2015-05-12 Thread Daniel Phillips
ally was. Regards, Daniel [1] Overhead of an uncontended bus locked add is about 6 nanoseconds on my i5, and about ten times higher when contended. /* * Blurt v0.0 * * A trivial multitasking filesystem load generator * * Daniel Phillips, June 2015 * * to build: c99 -Wall blurt.c -oblurt * to

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-12 Thread Daniel Phillips
On 05/12/2015 02:03 AM, Pavel Machek wrote: > On Mon 2015-05-11 19:34:34, Daniel Phillips wrote: >> On 05/11/2015 04:17 PM, Theodore Ts'o wrote: >>> and another way that people >>> doing competitive benchmarking can screw up and produce misleading >>> num

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-11 Thread Daniel Phillips
On Monday, May 11, 2015 10:38:42 PM PDT, Dave Chinner wrote: > I think Ted and I are on the same page here. "Competitive > benchmarks" only matter to the people who are trying to sell > something. You're trying to sell Tux3, but By "same page", do you mean "transparently obvious about obstruct

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-11 Thread Daniel Phillips
Hi David, On 05/11/2015 05:12 PM, David Lang wrote: > On Mon, 11 May 2015, Daniel Phillips wrote: > >> On 05/11/2015 03:12 PM, Pavel Machek wrote: >>>>> It is a fact of life that when you change one aspect of an intimately >>>>> interconnected system

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-11 Thread Daniel Phillips
On 05/11/2015 04:17 PM, Theodore Ts'o wrote: > On Tue, May 12, 2015 at 12:12:23AM +0200, Pavel Machek wrote: >> Umm, are you sure. If "some areas of disk are faster than others" is >> still true on todays harddrives, the gaps will decrease the >> performance (as you'll "use up" the fast areas mor

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-11 Thread Daniel Phillips
Hi Pavel, On 05/11/2015 03:12 PM, Pavel Machek wrote: >>> It is a fact of life that when you change one aspect of an intimately >>> interconnected system, >>> something else will change as well. You have naive/nonexistent free space >>> management now; when you >>> design something workable ther

Re: Tux3 Report: How fast can we fsync?

2015-05-02 Thread Daniel Phillips
On Friday, May 1, 2015 6:07:48 PM PDT, David Lang wrote: On Fri, 1 May 2015, Daniel Phillips wrote: On Friday, May 1, 2015 8:38:55 AM PDT, Dave Chinner wrote: Well, yes - I never claimed XFS is a general purpose filesystem. It is a high performance filesystem. Is is also becoming more

Re: Tux3 Report: How fast can we fsync?

2015-05-01 Thread Daniel Phillips
On Friday, May 1, 2015 8:38:55 AM PDT, Dave Chinner wrote: > > Well, yes - I never claimed XFS is a general purpose filesystem. It > is a high performance filesystem. Is is also becoming more relevant > to general purpose systems as low cost storage gains capabilities > that used to be considered

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-04-30 Thread Daniel Phillips
Hi Ted, On 04/30/2015 07:57 AM, Theodore Ts'o wrote: > This is one of the reasons why I find head-to-head "competitions" > between file systems to be not very helpful for anything other than > benchmarketing. It's almost certain that the benchmark won't be > "fair" in some way, and it doesn't rea

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-04-30 Thread Daniel Phillips
On 04/30/2015 07:33 AM, Mike Galbraith wrote: > Well ok, let's forget bad blood, straw men... and answering my question > too I suppose. Not having any sexy IO gizmos in my little desktop box, > I don't care deeply which stomps the other flat on beastly boxen. I'm with you, especially the forget

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-04-30 Thread Daniel Phillips
On 04/30/2015 07:28 AM, Howard Chu wrote: > Daniel Phillips wrote: >> >> >> On 04/30/2015 06:48 AM, Mike Galbraith wrote: >>> On Thu, 2015-04-30 at 05:58 -0700, Daniel Phillips wrote: >>>> On Thursday, April 30, 2015 5:07:21 AM PDT, Mike Galbraith wrot

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-04-30 Thread Daniel Phillips
On 04/30/2015 06:48 AM, Mike Galbraith wrote: > On Thu, 2015-04-30 at 05:58 -0700, Daniel Phillips wrote: >> On Thursday, April 30, 2015 5:07:21 AM PDT, Mike Galbraith wrote: >>> On Thu, 2015-04-30 at 04:14 -0700, Daniel Phillips wrote: >>> >>>> Lovely sou

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-04-30 Thread Daniel Phillips
On Thursday, April 30, 2015 5:07:21 AM PDT, Mike Galbraith wrote: On Thu, 2015-04-30 at 04:14 -0700, Daniel Phillips wrote: Lovely sounding argument, but it is wrong because Tux3 still beats XFS even with seek time factored out of the equation. Hm. Do you have big-storage comparison numbers

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-04-30 Thread Daniel Phillips
On Wednesday, April 29, 2015 5:20:08 PM PDT, Dave Chinner wrote: It's easy to be fast on empty filesystems. XFS does not aim to be fast in such situations - it aims to have consistent performance across the life of the filesystem. In this case, ext4, btrfs and tux3 have optimal allocation fillin

Re: Tux3 Report: How fast can we fsync?

2015-04-30 Thread Daniel Phillips
On Wednesday, April 29, 2015 8:50:57 PM PDT, Mike Galbraith wrote: On Wed, 2015-04-29 at 13:40 -0700, Daniel Phillips wrote: That order of magnitude latency difference is striking. It sounds good, but what does it mean? I see a smaller difference here, maybe because of running under KVM

Re: Tux3 Report: How fast can we fsync?

2015-04-30 Thread Daniel Phillips
On Wednesday, April 29, 2015 6:46:16 PM PDT, Dave Chinner wrote: I measured fsync performance using a 7200 RPM disk as a virtual drive under KVM, configured with cache=none so that asynchronous writes are cached and synchronous writes translate into direct writes to the block device. Yup, a slo

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-04-29 Thread Daniel Phillips
On Wednesday, April 29, 2015 12:05:26 PM PDT, Mike Galbraith wrote: Here's something that _might_ interest xfs folks. cd git (source repository of git itself) make clean echo 3 > /proc/sys/vm/drop_caches time make -j8 test ext42m20.721s xfs 6m41.887s <-- ick btrfs 1m32.038s tux31m

Re: Tux3 Report: How fast can we fsync?

2015-04-29 Thread Daniel Phillips
On Wednesday, April 29, 2015 9:42:43 AM PDT, Mike Galbraith wrote: [dbench bakeoff] With dbench v4.00, tux3 seems to be king of the max_latency hill, but btrfs took throughput on my box. With v3.04, tux3 took 1st place at splashing about in pagecache, but last place at dbench -S. Hohum, curio

Re: Tux3 Report: How fast can we fsync?

2015-04-29 Thread Daniel Phillips
On Tuesday, April 28, 2015 11:33:33 PM PDT, Mike Galbraith wrote: On Tue, 2015-04-28 at 23:01 -0700, Daniel Phillips wrote: On Tuesday, April 28, 2015 7:21:11 PM PDT, Mike Galbraith wrote: Where does tux3 live? What I found looked abandoned. Current work is here: https://github.com

Re: Tux3 Report: How fast can we fsync?

2015-04-28 Thread Daniel Phillips
On Tuesday, April 28, 2015 11:20:08 PM PDT, Richard Weinberger wrote: On Wed, Apr 29, 2015 at 8:01 AM, Daniel Phillips wrote: On Tuesday, April 28, 2015 7:21:11 PM PDT, Mike Galbraith wrote: ... Where can I find the fsync code? IOW how to reproduce your results? :) Hi Richard, If you can

Re: Tux3 Report: How fast can we fsync?

2015-04-28 Thread Daniel Phillips
On Tuesday, April 28, 2015 7:21:11 PM PDT, Mike Galbraith wrote: Where does tux3 live? What I found looked abandoned. Current work is here: https://github.com/OGAWAHirofumi/linux-tux3 Note, the new fsync code isn't pushed to that tree yet, however Hirofumi's optimized syncfs is already in

Tux3 Report: How fast can we fsync?

2015-04-28 Thread Daniel Phillips
Greetings, This post is dedicated to Ted, who raised doubts a while back about whether Tux3 can ever have a fast fsync: https://lkml.org/lkml/2013/5/11/128 "Re: Tux3 Report: Faster than tmpfs, what?" Ted suggested that Tux3's inherently asynchronous design would be a limitation when it c

Re: [RFC] Tux3 for review

2014-06-24 Thread Daniel Phillips
On Tuesday, June 24, 2014 4:52:15 AM PDT, James Bottomley wrote: On Tue, 2014-06-24 at 04:27 -0700, Daniel Phillips wrote: I emphatically disagree that it is premature for asking Tux3 to be merged. You might think so, but I do not. While I do not begrudge you your opinion, Linux did not get to

Re: [RFC] Tux3 for review

2014-06-24 Thread Daniel Phillips
On Tuesday, June 24, 2014 3:59:40 AM PDT, Theodore Ts'o wrote: On Tue, Jun 24, 2014 at 02:10:52AM -0700, Daniel Phillips wrote: That makes sense, because the patches to transform our workarounds into shiny new kernel hooks are still in progress, as I said. I would appreciate the courte

Re: [RFC] Tux3 for review

2014-06-24 Thread Daniel Phillips
On Saturday, June 21, 2014 6:06:00 PM PDT, Dave Chinner wrote: BTW, it's worth noting that reviewers are *allowed* to change their mind at any time during a discussion or during review cycles. Indeed, this occurs quite commonly. It's no different to multiple reviewers disagreeing on what the best

Re: [RFC] Tux3 for review

2014-06-24 Thread Daniel Phillips
On Monday, June 23, 2014 9:41:30 PM PDT, James Bottomley wrote: [rhetoric snipped] ... I'm still arguing the facts: proving that page forking can be integrated into writeback without adding to the maintenance burden is a big issue for tux3. Sorry, I must have missed those facts, I only saw

Re: [RFC] Tux3 for review

2014-06-23 Thread Daniel Phillips
On Sunday, June 22, 2014 11:34:50 AM PDT, Theodore Ts'o wrote: On Sat, Jun 21, 2014 at 08:32:03PM -0700, Daniel Phillips wrote: That's a bit disingenuous: the concern has always been how page forking interacted with writeback. It's not new, it was one of the major things br

Re: [RFC] Tux3 for review

2014-06-23 Thread Daniel Phillips
On Saturday, June 21, 2014 12:29:01 PM PDT, James Bottomley wrote: On Thu, 2014-06-19 at 14:58 -0700, Daniel Phillips wrote: On Thursday, June 19, 2014 2:26:48 AM PDT, Lukáš Czerner wrote: ... the concern has always been how page forking interacted with writeback. More accurately, that is

Re: [RFC] Tux3 for review

2014-06-21 Thread Daniel Phillips
On Saturday, June 21, 2014 12:29:01 PM PDT, James Bottomley wrote: On Thu, 2014-06-19 at 14:58 -0700, Daniel Phillips wrote: We already removed 450 lines of core kernel workarounds from Tux3 with an approach that was literally cut and pasted from one of Dave's emails. Then Dave change

Re: [RFC] Tux3 for review

2014-06-19 Thread Daniel Phillips
On Thursday, June 19, 2014 9:24:10 AM PDT, Josef Bacik wrote: On 05/16/2014 05:50 PM, Daniel Phillips wrote: We would like to offer Tux3 for review for mainline merge. We have prepared a new repository suitable for pulling: https://urldefense.proofpoint.com/v1/url?u=https://git.kernel.org

Re: [RFC] Tux3 for review

2014-06-19 Thread Daniel Phillips
On Thursday, June 19, 2014 2:26:48 AM PDT, Lukáš Czerner wrote: On Thu, 19 Jun 2014, Pavel Machek wrote: Date: Thu, 19 Jun 2014 10:21:29 +0200 From: Pavel Machek To: James Bottomley Cc: Daniel Phillips , Dave Chinner , linux-kernel@vger.kernel.org, linux-fsde...@vger.kernel.org

Re: [RFC] Tux3 for review

2014-06-15 Thread Daniel Phillips
On Friday, June 13, 2014 1:20:39 PM PDT, Pavel Machek wrote: Hi! On Fri 2014-06-13 10:49:39, Daniel Phillips wrote: Hi Pavel, On Friday, June 13, 2014 3:32:16 AM PDT, Pavel Machek wrote: ... Actually, would it make sense to have staging/fs/? That makes sense to me, if a suitably expert

Re: [RFC] Tux3 for review

2014-06-13 Thread Daniel Phillips
Hi Pavel, On Friday, June 13, 2014 3:32:16 AM PDT, Pavel Machek wrote: Hmm, it seems that merging filesystems is getting harder over time. Soon, it will be impossible to merge new filesystem. My thought exactly, but it carries more weight coming from you. It is getting more unpleasant to discu

Re: [RFC][PATCH 1/2] Add a super operation for writeback

2014-06-03 Thread Daniel Phillips
On Tuesday, June 3, 2014 8:21:55 AM PDT, Jan Kara wrote: On Tue 03-06-14 07:14:44, Christoph Hellwig wrote: On Tue, Jun 03, 2014 at 04:05:31PM +0200, Jan Kara wrote: ... So I agree per-bdi / per-sb matters only in simple setups but machines with single rotating disk with several partitions a

Re: [RFC][PATCH 1/2] Add a super operation for writeback

2014-06-03 Thread Daniel Phillips
Hi Dave, Here is a non-incremental patch. This implements your suggestion from yesterday, except that the wb list lock is dropped before calling ->writeback(). Regards, Daniel >From d030d328757b160b39b252e82811a94843513cfc Mon Sep 17 00:00:00 2001 From: Daniel Phillips Date: Tue, 3 Ju

Re: [RFC][PATCH 1/2] Add a super operation for writeback

2014-06-03 Thread Daniel Phillips
Hi Dave, On 06/02/2014 08:33 PM, Dave Chinner wrote: > On Mon, Jun 02, 2014 at 01:02:29PM -0700, Daniel Phillips wrote: >> >> Redirty_tail nearly works, but "if (!list_empty(&wb->b_dirty))" is >> not correct because the inode needs to end up on the dirty list

Re: [RFC][PATCH 2/2] tux3: Use writeback hook to remove duplicated core code

2014-06-02 Thread Daniel Phillips
On 06/01/2014 08:30 PM, Dave Chinner wrote: > I get very worried whenever I see locks inside inode->i_lock. In > general, i_lock is supposed to be the innermost lock that is taken, > and there are very few exceptions to that - the inode LRU list is > one of the few. I generally trust Hirofumi to

Re: [RFC][PATCH 1/2] Add a super operation for writeback

2014-06-02 Thread Daniel Phillips
On 06/01/2014 08:15 PM, Dave Chinner wrote: > On Sun, Jun 01, 2014 at 02:41:02PM -0700, I wrote: >> + >> +/* >> + * Add inode to writeback dirty list with current time. >> + */ >> +void inode_writeback_touch(struct inode *inode) >> +{ >> +struct backing_dev_info *bdi = inode->i_sb->s_bdi; >> +

[RFC][PATCH 2/2] tux3: Use writeback hook to remove duplicated core code

2014-06-01 Thread Daniel Phillips
ars young to fs-writeback and does not trigger repeated ->writeback flushes. Signed-off-by: Daniel Phillips --- fs/tux3/Makefile | 2 +- fs/tux3/commit.c | 1 - fs/tux3/commit_flusher.c | 180 ++ fs/tux3/commit_flusher.h | 16 -- f

[RFC][PATCH 1/2] Add a super operation for writeback

2014-06-01 Thread Daniel Phillips
nificant compared to the convoluted code path that is avoided. Regards, Daniel --- From: Daniel Phillips Subject: [PATCH] Add a super operation for writeback Add a "writeback" super operation to be called in the form: progress = sb->s_op->writeback(sb, &wbc, &am

Re: [RFC] Tux3 for review

2014-05-23 Thread Daniel Phillips
Hi Dongsu, On Thursday, May 22, 2014 2:52:27 AM PDT, Dongsu Park wrote: First of all, thank you for trying to merge it to mainline. Maybe I cannot say the code is clean enough, but basically the filesystem seems to work at least. Thank you for confirming that. We test Tux3 extensively so we kn

Re: [RFC] Tux3 for review

2014-05-20 Thread Daniel Phillips
Hi Dave, This is to address your concern about theoretical interaction between direct IO and Tux3 page fork. On Monday, May 19, 2014 10:41:40 PM PDT, I wrote: Except that Direct IO impacts on the design of the page forking code (because of how things like get_user_pages() need to be aware of

Re: [RFC] Tux3 for review

2014-05-19 Thread Daniel Phillips
On Friday, May 16, 2014 10:29:43 PM PDT, I wrote: Hirofumi is the one who deserves congratulations, recognition for providing more than half the code including most of the hard parts, and thanks for bringing Tux3 back to life. An epilogue... one gentleman took that suggestion seriously and sen

Re: [RFC] Tux3 for review

2014-05-19 Thread Daniel Phillips
On Monday, May 19, 2014 8:18:02 PM PDT, Dave Chinner wrote: On Mon, May 19, 2014 at 05:55:30PM -0700, Daniel Phillips wrote: On 05/18/2014 04:55 PM, Dave Chinner wrote: ... I'm not commenting on the c99 comment style, I'm passing comment on the fact that a filesystem that has com

Re: [RFC] Tux3 for review

2014-05-19 Thread Daniel Phillips
On 05/18/2014 04:55 PM, Dave Chinner wrote: On Fri, May 16, 2014 at 05:50:59PM -0700, Daniel Phillips wrote: We would like to offer Tux3 for review for mainline merge. We have prepared a new repository suitable for pulling: https://git.kernel.org/cgit/linux/kernel/git/daniel/linux-tux3.git

Re: [RFC] Tux3 for review

2014-05-16 Thread Daniel Phillips
On Friday, May 16, 2014 10:09:50 PM PDT, Martin Steigerwald wrote: Hi Daniel! Am Freitag, 16. Mai 2014, 17:50:59 schrieb Daniel Phillips: We would like to offer Tux3 for review for mainline merge. We have prepared a new repository suitable for pulling: At long last! Congrats for arriving at

[RFC] Tux3 for review

2014-05-16 Thread Daniel Phillips
We would like to offer Tux3 for review for mainline merge. We have prepared a new repository suitable for pulling: https://git.kernel.org/cgit/linux/kernel/git/daniel/linux-tux3.git/ Tux3 kernel module files are here: https://git.kernel.org/cgit/linux/kernel/git/daniel/linux-tux3.git/tree/fs/t

Subject: Tux3 May Day Festivities

2014-05-01 Thread Daniel Phillips
Today is May Day, an ancient festival day. A time for dancing, singing, feasting and refactoring repository topology. We are preparing Tux3 now for review with a view to mainline merge. Our repository is currently on Github: https://github.com/OGAWAHirofumi/tux3 This repository needs some refac

Re: Tux3 Report: Untar Unleashed

2014-04-25 Thread Daniel Phillips
Yesterday I wrote: > When we checked read performance on the untarred tree, we immediately saw > mixed results. Re-tarring the kernel tree is faster than Ext4, but directory listing is > slower by a multiple. So we need to analyze and fix ls without breaking the good tar > and untar behavior. Th

Tux3 Report: Untar Unleashed

2014-04-24 Thread Daniel Phillips
Hi, It is about time to post Tux3 patches for review. Almost. Running the classic kernel untar test on an older 60 MB/sec hard disk turned up a performance gap versus Ext4 by a factor of six. Hmm, that is in the range where it could be a basic design bug, we need to do something. Tux3 perf

Tux3 Report: Our blocker list

2014-02-03 Thread Daniel Phillips
At Korea Linux Forum last fall, Linus asked, haven't I been hearing about Tux3 for ten years? I said, no, that was Tux2, completely different. You only heard about Tux3 for six years. Well, that is about long enough to keep hearing about an out of tree filesystem. Last time we talked about mergin

Re: [Ksummit-2013-discuss] [ATTEND] How to act on LKML

2013-07-25 Thread Daniel Phillips
Hi Willy, I understand completely. I don't blame you. Filter the thread. Done. I am not tired of the subject, quite the contrary. Please do not speak for me in that regard. After many years of wandering in the toxic wasteland, finally some actual progress. Regards, Daniel -- To unsubscribe fr

Re: [Ksummit-2013-discuss] [ATTEND] How to act on LKML

2013-07-25 Thread Daniel Phillips
On 07/25/2013 02:34 PM, Willy Tarreau wrote: > Guys, could we please stop this endless boring thread ? Willy, I believe we are on the same side of the civility debate, but I somehow got the feeling that you just characterized my comment re "open and honest" as "endless and boring". I agree that

Re: [Ksummit-2013-discuss] [ATTEND] How to act on LKML

2013-07-24 Thread Daniel Phillips
On 07/24/2013 12:51 AM, Felipe Contreras wrote: > Your mistaken fallacy seems to be that you think one can *always* be > both A (open and honest), and B (polite)... You are are right, I do think that you can *always* be both open and honest, and polite. I do not believe that I am mistaken. And I

Re: [Ksummit-2013-discuss] Maybe it's time to shut this thread down (Was: Re: [ 00/19] 3.10.1-stable review)

2013-07-22 Thread Daniel Phillips
On 07/22/2013 09:02 PM, Luck, Tony wrote: > Some thoughts on the format of the discussion at KS: > > ... > 5) Volunteers are under-represented at Kernel Summit Volunteers are the "dark matter" of Linux Kernel contribution. They are not the "usual suspects" who nearly all have full time jobs now,

Re: [Ksummit-2013-discuss] [ATTEND] How to act on LKML

2013-07-20 Thread Daniel Phillips
On 07/20/2013 12:36 PM, Felipe Contreras wrote: > I think you need more than "hope" to change one of the fundamental > rules of LKML; be open and honest, even if that means expressing your > opinion in a way that others might consider offensive and colorful. Logical fallacy type: bifurcation. You

Re: [Ksummit-2013-discuss] Maybe it's time to shut this thread down (Was: Re: [ 00/19] 3.10.1-stable review)

2013-07-20 Thread Daniel Phillips
On 07/18/2013 03:54 PM, Sarah Sharp wrote: > Let's shift this discussion away from the terms "abuse" and > "professionalism" to "respect" and "civility". Brilliant, and +1 for a session at KS. In the mean time, why don't we all try to demonstrate the real meaning of respect and civility, by prac

Re: [PATCH] Optimize wait_sb_inodes()

2013-06-27 Thread Daniel Phillips
Hi Ted, On Thu, Jun 27, 2013 at 11:36 AM, Theodore Ts'o wrote: > If the goal is to optimize file system freeze operations, sure. But > that's also not a common operation, so if it burns a little extra CPU > time, it wouldn't cause me to think that it was a high priority issue. > Decreasing the w

Re: [PATCH] Optimize wait_sb_inodes()

2013-06-27 Thread Daniel Phillips
On Wed, Jun 26, 2013 at 10:18 PM, OGAWA Hirofumi wrote: > ...vfs can't know the data is whether after sync point or before > sync point, and have to wait or not. FS is using the behavior like > data=journal has tracking of those already, and can reuse it. Clearly showing why the current interface

  1   2   3   4   5   6   7   8   >