On Tuesday, June 17, 2014 4:33:24 PM UTC+2, Volker Braun wrote: > I've spent some time looking at hashdist which is probably the closest to > what we need, but I don't think its the way to go for us right now. First, > Sage depends on the LD_LIBRARY_PATH hack on too many places. Before that is > fixed its hard to do anything with a real package management system. At the > same time, hashdist isn't ready. Its version 0.2, which is incompatible to > 0.1, there is no documentation for writing builder plugins / API. And much > of the source is, let's say, lightly documented with doctests being almost > completely absent. And no parallel build support, ... But in the future we > should definitely reconsider that, when these issues have been addressed on > both sides. >
There's quite a lot of nosetests and everything was definitely developed by unit tests...I'd argue that doctests vs. nosetests depends a little bit on taste and a lot of what kind of code (and how complicated test fixtures) you need. I won't contest that code documentation is light. That's mainly my fault, and mainly because Hashdist was (perhaps still is) an experiment -- it doesn't make so much sense to document things thoroughly if they may change at any moment, and the number of coders touching the code are only a couple. This must definitely change. > > What I don't like in hashdist is that the packages don't have a clear > backing as Python classes in the code, which is related to not being able > to customize the heck out of it. There is the school of thought that says > you shouldn't be allowed to do anything that doesn't lead to provably > correct hash->build maps, and everything must be stateless functional. > Thats nice if you want a package manager for end users only, but that is > IMHO not good > enough for development purposes. I'd rather have to do an occasional "make > clean" than an extra 10k file ops every time I test a change. > Fundamentally, what does a package define? Its how to get the source code, > the version of the source code, and how to turn the source into a binary. > Right now, hashdist packages only have a hook into the third part. Why > can't we override how hashing works, or where to get the source from? Yes > that might allow you to break the promise of provably correct builds, but > we are all adults here. As long as you build stuff in my home directory I > ought to be able to break anything I want. > One of the reasons Hashdist came about was actually because "we're all adults" and to remove the strictness of Nix. Perhaps it's still too strict, but at least we said the exact same thing about Nix at one point :-) A lot of what you say is fully correct, and is simply an effect of Hashdist still being in development and still being developed. Many of the things you asked for are almost there (e.g., package builds already happen breadth-first and turning that into parallel is a mostly trivial matter). Others may simply be that we didn't think about them, and I think you'll find we're very open to somebody coming in and change how things are done. We are definitely not purists. You can definitely build a stack relying solely on LD_LIBRARY_PATH. And so on. We very much have the same goal here. What you have to ask yourself is: Will starting from scratch get you where you want faster than building on and improving Hashdist. Either way, there's work to be done. If you are interested in cooperating perhaps the quickest way to move forward is a Google Hangout session; I think a lot of this is really about what software architecture would fit the package manager of Sage, and that all the smaller issues discussed so far in the thread is more on the background noise level (as they are things easily changed about Hashdist), and it may be difficult to kick the discussion to the level we need in an email thread. Dag Sverre -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at http://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
