Re: Large disk drives
On Wed, Nov 05, 2014 at 05:07:48PM +0100, James Bottomley wrote: > > OK, but I still don't understand how windows gets the partition table on > there in the first place ... that must surely be some sort of guess the > disk size hack. 99% of the time the partition table was provided by the drive manufacturer; I would assume they have a way of doing that as a part of their manufacturing processes that don't require an OS individually running the equivalent of fdisk or gdisk on said drive. - Ted -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: SPDX-License-Identifier
On Fri, Feb 21, 2014 at 09:57:20AM -0800, Greg Kroah-Hartman wrote: > > But shouldn't we at least write somewhere > > that it has connection to spdx.org where you can find out that licenses. > > Why? Are these licenses so unknown that no one knows what they are? > And, as part of the kernel-as-a-whole-work, they all resolve to GPLv2 > anyway, and we have that license in the source tree, so nothing else > should be needed. Note that not all lawyers are in agreement about this, so if this is a driver being developed by a company, you may want to ask your corporate counsel if they have an opinion about this. I've received advice of the form that it's not obvious that regardless of whether or not us *engineers* understand what all of the licensing terms mean, what's important is whether someone who is accused of "borrowing" GPL'ed code and dropping it in a driver for some other OS can convince a judge whether or not it's considered "obvious" from a legal perspective what an SPDX header means, and what is implied by an SPDX license identifer. Also note that with the advent of web sites that allow people to do web searches and turn up a singleton file via some gitweb interface, the fact that the full license text is distributed alongside the tarball might or might have as much legal significance as it once had. But of course, I'm not a lawyer, and if your company has is paying for the development of the driver, the Golden Rule applies (he who has the Gold, makes the Rules), and each of our respective corporate lawyers may have different opinions about what might happen if the question was ever to be adjudicated in court. Cheers, - Ted -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: SPDX-License-Identifier
On Mon, Feb 24, 2014 at 11:12:53AM +0100, Michal Simek wrote: > > But of course, I'm not a lawyer, and if your company has is paying for > > the development of the driver, the Golden Rule applies (he who has the > > Gold, makes the Rules), and each of our respective corporate lawyers > > may have different opinions about what might happen if the question > > was ever to be adjudicated in court. > > Aren't all these points already answered by SPDX project? > I believe that they should know how this should be handled properly. The SPDX can not give legal advice; not to you, and not to your company. One lawyer might believe that /* * SPDX-License-Identifier: GPL-2.0 */ Might be sufficient. Others might believe you need to do: /* * Copyright Ty Coon, 2012. * * SPDX-License-Identifier: GPL-2.0 */ Still others might believe you need at the very least: /* * Copyright Ty Coon, 2012. * * All Rights Reserved. * * SPDX-License-Identifier: GPL-2.0 */ As far as I know, there is no case law on point about whether or not SPDX-License-Identifier has legal significance, or whether the court would consider that to be a valid copyright permission statement. So any "answers" made by any lawyer would be guesses. Of course, an guess by a lawyer which is retained by *you* or your company and fully informed with the unique parameters of your situation would constitute legal advice. Anything else, including anything any of us could say on this mailing list, would be biovating. :-) Cheers, - Ted -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: First kernel patch (optimization)
On Wed, Sep 16, 2015 at 05:03:39PM +0100, Eric Curtin wrote: > Hi Greg, > > As I said in the subject of the mail (which I have been since told I > shouldn't have done this), I'm a noob to kernel code. I tried to pick > off something super simple to just see what the process of getting a > patch in is. Youtube videos and documentation only get you so far. > > From reading your response, should I refrain from sending in these > micro-optimizations in future? Getting in smaller patches is easier > for me as I only do this in my spare time, which I don't have a lot > of! What I'd ask you to consider is what your end goal? Is it just to collect a scalp (woo hoo! I've gotten a patch into the kernel)? Or is it to actually make things better for yourself or other users? Or are you trying to get make your self more employable, etc. Micro-optimizations is often not particularly useful for anything other than the first goal, and it really doesn't help anyone. If you're just doing this in your spare time, then hopefully I hope you are being choosy about what's the best way to use your spare time, so the question of what your goals are going to be is a very important thing for you to figure out. Regardless of whether it's worthwhile to get this patch into the kernel, doing any *more* micro-optimizations is probably not a good use of your time or anyone else's. I'd strongly encourage you to move on to something more than just micro-optimizations as quickly as possible. Best regards, - Ted -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: First kernel patch (optimization)
On Wed, Sep 16, 2015 at 01:26:51PM -0400, Josh Boyer wrote: > > That isn't true. It helps the submitter understand the workflow and > expectations. What you meant to say is that it doesn't help you. The problem is that workflow isn't the hard part. It's the part that can be taught most easily, sure. But people seem to get really hung up on it, and I fear that we have people who never progress beyond sending trivial patches and spelling fixes and white space fixes and micro-optimizations. If the "you too can be a kernel developer" classes and web sites and tutorials also taught people how to take performance measurements, and something about the scientific measurement, that would be something. Or if it taught people how to create tests and to run regression testing. Or if it taught people how to try to do fuzz testing, and then once they find a sequence which causes crash, how to narrow down the failure to a specific part of the kernel, and how to fix and confirm that the kernel no longer crashes with the fix --- that would be useful. If they can understand kernel code; if they can understand the scientific measurement; if they can understand how to do performance measurements --- being able to properly format patches is something which most kernel developers can very easily guide a new contributor to do correctly. Or in the worst case, it doesn't take much time for me to fix a whitespace problem and just tell the contributor --- by the way, I fixed up this minor issue; could you please make sure you do this in the future? But if a test hasn't been tested, or if the contributor things it's a micro-optimization, but it actually takes more CPU time and/or more stack space and/or bloats the kernel --- that's much more work for the kernel maintainer to have to deal with when reviewing a patch. So I have a very strong disagreement with the belief that teaching people the workflow is the more important thing. In my mind, that's like first focusing on the proper how to properly fill out a golf score card, and the ettiquette and traditions around handicaps, etc --- before making sure the prospective player is good at putting and driving. Personally, I'm terrible at putting and driving, so spending a lot of time learning how to fill out a golf score card would be a waste of my time. A good kernel programmer has to understand systems thinking; how to figure out abstractions and when it's a good thing to add a new layer of abstraction and when it's better to rework an exsting abstraction layer. If we have someone who knows the workflow, but which doesn't understand systems thinking, or how to do testing, then what? Great, we've just created another Nick Krause. Do you think encouraging a Nick Krause helps anyone? If people really are hung up on learning the workflow, I don't mind if they want to learn that part and send some silly micro-optimization or spelling fix or whitespace fix. But it's really, really important that they move beyond that. And if they aren't capable of moving beyond that, trying to inflate are recruitment numbers by encouraging someone who can only do trivial fixes means that we may be get what we can easily measure --- but it may not be what we really need as a community. Cheers, - Ted -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: First kernel patch (optimization)
On Fri, Sep 18, 2015 at 12:42:48AM -0700, Greg KH wrote: > So don't take cleanup patches for your code, that's fine, and I totally > understand why _you_ don't want to do that. But to blow off the effort > as being somehow trivial and not worthy of us, that's totally missing > the point, and making things harder on yourself in the end. It's not that I think cleanup patches are "trivial and not worthy of us", although I'll note that cleanup patches can end up causing real bug fixes (including possibly fixes that address security problems) to not apply to stable kernels, which means someone needs to deal with failed patches --- or what is much more likely, the failed patch gets bounced back to the overworked maintainer who then drops the patch because he or she doesn't have the bandwidth to manually backport the patch to N different stable trees, and then we all suffer. So cleanup patches *do* have a cost. Rather, what concerns me is that we aren't pushing people to go *beyond* cleanup patches. We have lots of tutorials about how to create perfectly formed patches; but we don't seem to have patches about how to do proper benchmarking. Or how to check system call and ioctl interfaces to make sure the code is doing appropriate input checking. So I don't want to pre-reject these people; but to rather push them and to help them to try their hand at more substantive work. What surprises me is the apparent assumption that people need a huge amount of hand-holding on how to properly form a patch, but that people will be able to figure out how to do proper benchmarking, or design proper abstractions, etc., as skills that will magically appear full-formed into the new kernel programmer's mind without any help or work on our part. > So don't be a jerk, and spout off as to how we only need "skilled" > people contributing. Of course we need that, but the way we get those > people is to help them become skilled, not requiring them to show up > with those skills automatically. I think where we disagree is in how to make them become skilled. I don't think just focusing on contents of SubmittingPatches is sufficient, and there seems to be a "Step 2: And then a miracle occurs" between the SubmittingPatches step and the "skilled contributor" end goal. Cheers, - Ted -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: First kernel patch (optimization)
On Fri, Sep 18, 2015 at 10:18:27PM -0700, Greg KH wrote:
> And again, don't knock the basic whitespace patch. It is non-trivial,
> see the tutorials for proof of that.
>
> And please, NEVER chide someone for contributing whitespace patches,
> it's a sure way to ensure that this person never contributes again,
> something that I hope you agree is toxic to our future.
Not less than 24 hours ago, I received a "my second kernel patch[1]"
contribution which doesn't even compile (and no, it wasn't from Nick
Krause). It was a hundred+ lines of diffs of the form:
- printf("Hello, world\n");
+ printf("Hello,
+ world\n");
and
- foo(a, b, c, d); /* nowhere near 80 characters */
+ foo(a, b,/* Okay; so what was the point? */
+ c, d);
(Reference ommitted because I don't want to embarass the contributor;
people who really want to find it should have no trouble doing so.)
I didn't childe the contributor for submitting a whitespace patch. I
childed him on creating a patch which didn't build after you applied
it. And I pointed him at kvm-xfstests and gce-xfstests, for which I
have spent a lot of time making them as turn key and as easy to use as
possible, specifically because I *am* trying to make life easier for
newcomers. (Although if we have people who aren't clear on the
concept of valid C code, I'm not sure how much any tutorials or making
test suites more accessible is really going to help. :-)
I am sure that it is indeed very hard for a certain class of people to
create a basic whitespace patch. What I haven't seen the data for is
that which addresses the question: for the people for which creating a
whitespace patch is difficult, what percentage of them have the
capability to eventually develop the advanced skills we need for
kernel development? And how much effort do we need to invest for them
to accomplish this, and how does the amount of investment required to
get them to that stage relate to amount of return these people will
contribute to the community --- that is, is this a charitable donation
where we are asking maintainers to invest in something that will
ultimately not benefit them, or is this a wise long-term investment?
People may still be willing to devote energy if it achieves some other
desirable long-term goal (i.e., increasing the representation of
under-represented groups), even if it doesn't help their subsystem
directly. But it's useful to know how to pitch the "ask" in terms of
what their expectations should be.
> I have been saying for years that we have a lack of real projects /
> tasks / ideas for people who are skilled, yet have no idea what to do.
> I know of well over a hundred people I have email addresses of that have
> asked me for these types of things, and have patches in the kernel that
> are non-trivial to prove that they have the skill to do real things.
>
> It's a real problem, and one that I don't have an answer for. We need
> these types of tasks, and I don't have them, and every maintainer I ask
> about it also doesn't have them. What we are asking for is people to
> somehow come up with tasks on their own, as if they know what needs to
> be done.
How about maintaining a list of people who are in this state? It
could be as simple as a list of git commit ID's --- their e-mail
addresses will be in the git commit. Republish it once a month, with
some kind of auto-expiry mechanism unless (a) the author of said git
commit requests that it be removed, or (b) after some time period
unless the author is continuing to submit at least some non-trivial
patch.
At least that way maintainers who are looking for some minions will at
least have a starting point.
As far as creating tasks for people who _are_ skilled, the question is
how picky are these people, and how much time does it take to delegate
work to them? If it takes 8 hours of effort to delegate 16 hours (a
weekend's worth) of effort, with a 50% probability that the person
won't flaky out, and another 4 hours of work afterwards cleaning up
and debugging the patch. It has negative overall value to the
maintainer.
Or if the work isn't directly kernel programming, but instead work in
support of kernel programming (i.e., improving documentation, fixing
up regression tests, auditing interfaces looking for security holes),
is this work that "graduates" from the "my first patch" programs
willing to do?
There are almost certainly "bottle-washing" duty style tasks
available, but the question is are they tasks that people will want to
do? And how much work is a "week-end task" really? If it takes two
hours to delegate a task the maintainer can do in an hour, but it
takes the newcomer a weekend, then it only makes sense for the
maintainer to do this if (a) they are doing it as a cherity, or (b)
there is a sufficiently high probability the newcomer to stick around.
> Remember, when we started, the number of things that needed to be done
> was h
Re: First kernel patch (optimization)
On Sat, Sep 19, 2015 at 02:52:06PM +0200, Alexander Holler wrote: > > I've recently posted a proof of concept for wiping files, or in other words > to really delete files, And it was a disaster because if someone posts > imperfect pathhes on this list, people have fun trying to eat you (because > they seem bored or whatever). People gave you feedback on how what would be necessary to make the patch acceptable, and you rejected the advice complaining that it would take you months of "unpaid time". You were then complaining that the people who gave you feedback wouldn't fix the patch for you, and that you threatened that if we didn't integrate your racy patch into the core VFS, you would go switch to FreeBSD. > Even posting perfect patches is a game, because there exists always a space, > newline or variable name which might be used to start annoying discussions. Trust me, the issues with your patch went *way* beyond extra spaces or newlines. > So, there is no reason to wonder about the lack of such tasks. Greg was specifically looking for patches that could be done in a weekend. If you want to raise the argument that we should lower the standards for accepting patches so that more patches can accomplished within a weekend's worth of work, we can have that discussion --- but I'm not sure it will have the end result that you are hoping for. Best regards, - Ted -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: First kernel patch (optimization)
On Sat, Sep 19, 2015 at 07:47:22PM +0200, Alexander Holler wrote: > No. I don't want to lower the standards. Maybe in regard to silly style > stuff, but not in regard to code quality (and I mean real bugs like races, > deadlocks or such, and not if a line has more than 80 characters). I would > have liked some comments like "good or bad idea" but this list is imho the > wrong place to search for such useful comments. I haven't searched for > comments on the code, as I was FULLY aware that the code is ugly and NOT > ready for inclusion), That was asked and answered on the original thread, but perhaps it got lost amongst some of the noise --- some of which was contributed by yourself, but be that as it may. The *feature* is in and of itself not an insane idea. "Secure delete" is something that Linux has had before (in fact I did the original implementation for ext2 a decade+ ago), and indeed the interface, using the FS_SECRM_FL flag, is still present in Linux even if it is not supported by any of the file systems in the tree at the moment. It got ripped out because doing it right in the face of ongoing interface changes and performance improvements (going back to the pre-2.0, and possibly the pre-1.2 days), it was simpler to rip it out rather than to reimplement it. I wasn't responsible for ripping it out, back in mists of pre-history, but I didn't care enough to reimplement it --- or complain about it, either. And no one else cared enough to complain to Linus about this being a user-visible change that broke them. So it's a great example of how a feature and functionality that no one cares about *can* get ripped out of the kernel. Perhaps not so surprisingly, over a decade later, it is not currently at the top of the priority list of any of the current file system or VFS developers, as far as I know. One of the reasons for that is that there are a number of other ways of achieving the same functionality. These include using tmpfs, or using file system level encryption. They require a bit more system administrator setup than just being able to set the FS_SECR_FL flag, true, but just because it's more convenient doesn't mean that it's worth doing. So this is a feature request. It's a reasonable feature request, in that if someone would like to pay $$$ for some consultant to implement it in a way that is bug-free, I suspect it could go upstream. Someone who was very motivated and with the sufficient skills could also invest their own effort to make a patch that can go upstream too. You've elected not, to because you believe it would take you months of "unpaid time". That's purely within your rights to do. But you don't have the right to try to tell other people what work to do on their behalf --- not unless you are paying their salary. And of course, if you want to maintain an out-of-tree patch, there's nothing wrong with that. I've implemented code that has never gone upstream because the weight of the other file system developers were afraid that Enterise Linux customers would use the feature incorrectly, and it would cause potentially cause a security failure if used inappropriately, which would be a PR and support nightmare for them. Seeing that the weight of the other file system developers are against the patch, it's never gone into the mainline Linux kernel, even though I could have forced the feature into ext4. However, this patch is in active use in practically every single data center kernel for Google, and it's in use in at least one other very large publically traded company that uses cluster file systems such as Hadoopfs. And if someone wants a copy of the FALLOC_FL_NO_HIDE_STALE patch for ext4, I'm happy to give it to them. But it hasn't gone upstream, and I'm OK with that. As far as what you want to do next, you have a personal "proof of concept" patch that seems to work well enough for you. Great! I'm sure you can keep using it for your own purposes. If you can convince someone with the skills to get the patch to an upstreamable state, it is my judgement that this is doable, so this puts your feature in a much better state than the FALLOC_FL_NO_HIDE_STALE flag. However, there is still a non-trivial amount of work left to do to turn your "proof of concept" patch into something that is upstremable, including changing the interface to using the FS_SECRM_FL flag. And your whining that other people should change *their* priorities to match *yours* is not likely to help. - Ted -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 72/73] xfs: Convert mru cache to XArray
On Wed, Dec 06, 2017 at 06:06:48AM -0800, Matthew Wilcox wrote: > > Unfortunately for you, I don't find arguments along the lines of > > "lockdep will save us" at all convincing. lockdep already throws > > too many false positives to be useful as a tool that reliably and > > accurately points out rare, exciting, complex, intricate locking > > problems. > > But it does reliably and accurately point out "dude, you forgot to take > the lock". It's caught a number of real problems in my own testing that > you never got to see. The problem is that if it has too many false positives --- and it's gotten *way* worse with the completion callback "feature", people will just stop using Lockdep as being too annyoing and a waste of developer time when trying to figure what is a legitimate locking bug versus lockdep getting confused. I can't even disable the new Lockdep feature which is throwing lots of new false positives --- it's just all or nothing. Dave has just said he's already stopped using Lockdep, as a result. - Ted -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Lockdep is less useful than it was
On Thu, Dec 07, 2017 at 02:38:03PM -0800, Matthew Wilcox wrote:
> I think it was a mistake to force these on for everybody; they have a
> much higher false-positive rate than the rest of lockdep, so as you say
> forcing them on leads to fewer people using *any* of lockdep.
>
> The bug you're hitting isn't Byungchul's fault; it's an annotation
> problem. The same kind of annotation problem that we used to have with
> dozens of other places in the kernel which are now fixed.
The question is whose responsibility is it to annotate the code? On
another thread it was suggested it was ext4's responsibility to add
annotations to avoid the false positives --- never the mind the fact
that every single file system is going to have add annotations.
Also note that the documentation for how to add annotations is
***horrible***. It's mostly, "try to figure out how other people
added magic cargo cult code which is not well defined (look at the
definitions of lockdep_set_class, lockdep_set_class_and_name,
lockdep_set_class_and_subclass, and lockdep_set_subclass, and weep) in
other subsystems and hope and pray it works for you."
And the explanation when there are failures, either false positives,
or not, are completely opaque. For example:
[ 16.190198] ext4lazyinit/648 is trying to acquire lock:
[ 16.191201] ((gendisk_completion)1 << part_shift(NUMA_NO_NODE)){+.+.}, at:
[<8a1ebe9d>] wait_for_completion_io+0x12/0x20
Just try to tell me that:
((gendisk_completion)1 << part_shift(NUMA_NO_NODE)){+.+.}
is human comprehensible with a straight face. And since the messages
don't even include the subclass/class/name key annotations, as lockdep
tries to handle things that are more and more complex, I'd argue it
has already crossed the boundary where unless you are a lockdep
developer, good luck trying to understand what is going on or how to
add annotations.
So if you are adding complexity to the kernel with the argument,
"lockdep will save us", I'm with Dave --- it's just not a believable
argument.
Cheers,
- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
