On 5 Mar 2007 10:31:33 -0800, Martin Unsal <[EMAIL PROTECTED]> wrote: > On Mar 5, 9:15 am, "Chris Mellon" <[EMAIL PROTECTED]> wrote: > > That's actually the exact benefit of unit testing, but I don't feel > > that you've actually made a case that this workflow is error prone. > > You often have multiple developers working on the same parts of the > > same module? > > Protecting your head is the exact benefit of bike helmets, that > doesn't mean you should bike more more recklessly just because you're > wearing a helmet. :) > > Doing text merges is more error prone than not doing them. :) > > There are myriad other benefits of breaking up large files into > functional units. Integration history, refactoring, reuse, as I > mentioned. Better clarity of design. Easier communication and > coordination within a team. What's the down side? What's the advantage > of big files with many functional units? >
I never advocated big files with many functional units - just files that are "just big enough". You'll know you've broken them down small enough when you stop having to do text merges every time you commit. > > If you don't do this, you aren't really testing your changes, you're > > testing your reload() machinery. > > Only because reload() is hard in Python! ;) > > > You seem to have a lot of views about > > what the "Python way" should be and those are at odds with the actual > > way people work with Python. I'm not (necessarily) saying you're > > wrong, but you seem to be coming at this from a confrontational > > standpoint. > > When I refer to "Pythonic" all I'm talking about is what I've read > here and observed in other people's code. I'm here looking for more > information about how other people work, to see if there are good > solutions to the problems I see. > > However when I talk about what I think is "wrong" with the Pythonic > way, obviously that's just my opinion formed by my own experience. > > > Your claim, for example, that the language shouldn't place constraints > > on how you manage your modules is questionable. I think it's more > > likely that you've developed a workflow based around the constraints > > (and abilities) of other languages and you're now expecting Python to > > conform to that instead of its own. > > I don't think so; I'm observing things that are common to several > projects in several languages. > .... languages with similar runtime semantics and perhaps common ancestry? All languages place limitations on how you handle modules, either because they have infrastructure you need to use or because they lack it and you're left on your own. > > I wonder if you've ever asked yourself why this is the case. I know > > from my own experience why it's done in traditional C++/C environments > > - it's because compiling is slow and breaking things into as many > > files (with as few interdependencies) as possible speeds up the > > compilation process. > > I don't think that's actually true. Fewer, bigger compilation units > actually compile faster in C, at least in my experience. > If you're doing whole project compilation. When you're working, though, you want to be able to do incremental compilation (all modern compilers I know of support this) so you just recompile the files you've changed (and dependencies) and relink. Support for this is why we have stuff like precompiled headers, shadow headers like Qt uses, and why C++ project management advocates single class-per-file structures. Fewer dependencies between compilation units means a faster rebuild-test turnaround. > > Absent this need (which doesn't exist in Python), > > Python still takes time to load & "precompile". That time is becoming > significant for me even in a modest sized project; I imagine it would > be pretty awful in a multimillion line project. > > No matter how fast it is, I'd rather reload one module than exit my > interpreter and reload the entire world. > Sure, but whats your goal here? If you're just testing something as you work, then this works fine. If you're testing large changes, that affect many modules, then you *need* to reload your world, because you want to make sure that what you're testing is clean. I think this might be related to your desire to have everything in lots of little files. The more modules you load, the harder it is to track your dependencies and make sure that the reload is correct. > This is not a problem for Python as scripting language. This is a real > problem for Python as world class application development language. > Considering that no other "world class application development language" supports reload even as well as Python does, I'm not sure I can agree here. A perfect reload might be a nice thing to have, but lack of it hardly tosses Python (or any language) out of the running. > > In a package __init__, which exists expressly > > for the purpose of exposing it's interior namespaces as a single flat > > one, it makes perfect sense. > > OK! That's good info, thanks. > > > Nobody I know uses reload() for anything more than trivial "as > > you work" testing in the interpreter. It's not reliable or recommended > > for anything other than that. > > That too... although I think that's unfortunate. If reload() were > reliable, would you use it? Do you think it's inherently unreliable, > that is, it couldn't be fixed without fundamentally breaking the > Python language core? > The semantics of exactly what reload should do are tricky. Pythons reload works in a sensible but limited way. More complicated reloads are generally considered more trouble than they are worth. I've wanted different things from reload() at different times, so I'm not even sure what I would consider it being "reliable". Here's a trivial example - if you rename a class in a module and then reload it, what should happen to instances of the class you renamed? > > This is > > still a much faster way than compiling any but the most trivial of > > C/C++ modules. > > I'm with you there! I love Python and I'd never go back to C/C++. That > doesn't change my opinion that Python's import mechanism is an > impediment to developing large projects in the language. > > > If you don't like working with explicit namespaces, you've probably > > chosen the wrong language. > > I never said that. I like foo.Bar(), I just don't like typing > foo.Foo() and bar.Bar(), which is a waste of space; syntax without > semantics. > There's nothing that prevents there being a bar.Foo, the namespace makes it clear where you're getting the object. This is again a consequence of treating modules like classes. Some modules only expose a single class (StringIO/cStringIO in the standardlib is a good example), but it's more common for them to expose a single set of "functionality". That said, nothing prevents you from using "from foo import Foo" if Foo is all you need (or need most - you can combine this with import foo). > > I propose that the technique most amenable to source code management > > is for a single file (or RCS level module, if you have a locking RCS) > > to have everything that it makes sense to edit or change for a > > specific feature. > > Oh, I agree completely. I think we're using the exact same criterion. > A class is a self-contained feature with a well defined interface, > just what you'd want to put in it's own file. (Obviously there are > trivial classes which don't implement features, and they don't need > their own files.) > Sure, if all your classes are that. But very few classes exist in isolation - there's external and internal dependencies, and some classes are tightly bound. There's no reason for these tightly bound classes to be in external files (or an external namespace), because when you work on one you'll need to work on them all. > > You're also placing far too much emphasis on reload. Focus yourself on > > unit tests and environment scripts instead. These are more reliable > > and easier to validate than reload() in a shell. > > I think this is the crux of my frustration. I think reload() is > unreliable and hard to validate because Python's package management is > broken. I appreciate your suggestion of alternatives and I think I > need to come to terms with the fact that reload() is just broken. That > doesn't mean it has to be that way or that Python is blameless in this > problem. > I wonder what environments you worked in before that actually had a reliable and gotcha free version of reload? I actually don't know of any - Smalltalk is closest. It's not really "broken" when you understand what it does. There's just an expectation that it does something else, and when it doesn't meet that expectation it's assumed to be broken. Now, thats a fair definition of "broken", but replacing running instances in a live image is a very hard problem to solve generally. Limiting reload() to straightforward, reliable behavior is a reasonable design decision. -- http://mail.python.org/mailman/listinfo/python-list
