Re: Stringification, numification, and booleanification of pairs
* wolverian ([EMAIL PROTECTED]) [050925 11:57]: > On Sun, Sep 25, 2005 at 12:52:08PM +0200, Juerd wrote: > > Hackers on this list, what do you think? > > I think separating the two is extremely confusing. I do not see any uses > for it, but maybe I am not thinking hard enough. Of course, having "$thing".'x' produce something else than $thing.'x' is very confusion, however, they have a different purpose: Stringification/Numification should be used to let an object play its natural role within the program. For instance, some "Temperature" object producing 273 when compared to the melting point of water. Interpolation should be used on places where output is generated, where the outside consumer has less knowledge about the internals. Like "273'C" for the same Temperature object. Sorry, my program uses Celcius ;-) I like to have this behavior. The question is: will the two different uses be visible enough that the average programmer will understand it after a while, or will this become one of those nasty dark corners of the language which no-one dares to touch? The more constructs (like pairs, junctions, and hashes) produce different results for both cases, the better the choice for distinction is, because people get aware of it without reading a book. -- MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Perl 6 design wiki?
* Yuval Kogman ([EMAIL PROTECTED]) [060304 00:40]: > I think Perl 6 could use a wiki for several things: Last week, I was looking for ideas/plans about the required changes in CPAN, and it was so increadibly hard to find the documents. People tend to use SVN as substitute means of publication and everything is spread-out over too many websites and directories. The danger is that just another website is started, making the confusion even larger: good intergration is crucial. Commitment of the verious design groups. It would be nice to have the CPAN-ID's and password for authenication (like RT does) to reduce the maintenance effort. One thing which is playing in my head already for some time is: Do we really want a translation from Perl5 modules into Perl6 on a one-to-one basis? There are so many deceased modules occupying beautiful name-spaces! Can we please re-arrange the name-spaces? Could we try to kind-of pre-register name-spaces for perl6 modules? Maybe like the european new TLD .eu is being sold. If you "own" a perl5 name-space on this moment, you can claim the same name with preference for perl6, but otherwise people may suggest replacements. The Wiki as name-space market. Wouldn't it be nice when -for instance- the various XML-module implementators tried to design the best new XML interface together? IMO, a wiki is excelent for that, because discussion threads on mailinglists are not usable as design documents. I was unhappy so see some of MY MODULES be already translated into Perl6 by someone. Although it is nice that people put effort in building a library for Perl6, it is IMHO incorrect to suppose that the Perl5 (sometimes even perl4) herritage should be preserved for our nice new language. Perl5 modules should run as Perl5 on Parrot. Perl6 modules should use the new power of Perl6: not only inside the module itself, but especially in the interface presented to the users. I do second your idea for perl6 design wiki! Together we can work-out new interfaces and at the same time learn how to use the new features we got from each other. It's quite a big leap we can -and have to- make! NOW is THE chance to improve the quality of our code base. -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Perl 6 design wiki?
* Stevan Little ([EMAIL PROTECTED]) [060305 02:49]: > On 3/4/06, Mark Overmeer <[EMAIL PROTECTED]> wrote: > > Could we try to kind-of pre-register name-spaces for perl6 modules? > There is no need to do such a thing, we have the 3 level naming scheme > in Perl 6 now. > Foo-0.0.1-cpan:JRANDOM I know about the naming scheme, but I am not really looking forward to the two new perl books "Perl DBI-(Any)-cpan:TIMB" and "Perl DBI-(Any)-mailto:[EMAIL PROTECTED]" That you have the possibility to work your way out in namespace clashes shouldn't directly mean that you let them happen easily. I would really like to maintain a certain hierarchical name-space structure on CPAN, where we strive for unique names, although can work around accidental collissions. An other reason to have a kind of module/namespace pre-registration is to see who is (planning to go) working on what. I think that's needed on the moment. -- MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Perl 6 design wiki?
* Ruud H.G. van Tol ([EMAIL PROTECTED]) [060305 18:11]: > Juerd schreef: > > hierarchical names make less > > and less sense by the day > > I don't oversee the field yet, but maybe: > Introduce aliases (or hardlinks, in file-system-speak). > Likely in a separate top branch, such as "@alias::". > The @alias-prefix is only necessary when there is a collision. Once upon a time, I had to categorize the subjects of 20k websites in some logical structure, and then I discovered why the yellow pages have a flat organization... Of course, hierarchical names get less and less useful when your set grows larger. But it is better than nothing. > @alias::HTTPD::Session -> Apache::Session > HTTPD::Session -> Apache::Session Well, that's a technical solutions... your fill the name-space even more: the larger the harder to grasp. In general, it has many benefit to pick one name, and solve everything else via a searching mechanism like search.cpan.org. -- MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Perl 6 design wiki?
* Stevan Little ([EMAIL PROTECTED]) [060305 16:53]: > > to the two new perl books "Perl DBI-(Any)-cpan:TIMB" > > and "Perl DBI-(Any)-mailto:[EMAIL PROTECTED]" > Well, to start with, there is no > C6PAN/SixPan/Whatever-it-will-be-called yet, so there is nothing to > pre-register for. On the German Perl Workshop last week, I stirred things up a little with a BoF session about the subject. Amongst others, Andreas and Juerd where present. > Second, Perl 6 is still (at the very least) a year away from a > (mostly) complete implementation, > So pre-registration for namespaces in an unfinished language > is a litte premature IMO. So, in your opinion, it is easy to extend CPAN to contain perl6/parrot/ etc? Depends on the targets you set. Perl is especially known for its extended library: it is the main reason we still exist. To make Perl6 a success, we have to take care of the library! Just like new languages (like ruby) and ~features challenged Larry to redesign our favorit language, should CPAN be evaluated against new developments. Can we learn from SVN/SVK? Linux/GIT? Packaging like rpm's, etc? The library world grew much more complex with languages, character-sets, webdav, and super-fast networks at home. We do not only need to host Perl6, but also perl5, pir, pasm, partcl, etc etc languages. Quite a large number of people I spoke last week saw the need for major changes in the general set-up of CPAN. But certainly the current one is there to stay as well: CPAN is great. Maybe we can make it greater. Say Perl6 is still 2 years away, than we need to hurry with new plans for CPAN: when the currently eagerly waiting hurd of Perl programmers jump in, it is too late... > And lastly, I really don't like the idea anyway. It reminds me of the > domain-name squatters of the mid-90s. No, what I suggest is just like how cpan works now. But with a little planning ahead. > Some things may just need to stay the same, namespace intact. Anything > else would IMO further the Perl 5/6 gap that will inevitably exist > because of the depths of the language changes. For me, it feels as a big mistake when our new archiving needs are seen as a simple extra glue to the existing CPAN. If we need something smarter over two years, we have to start designing it NOW, and building it tomorrow! Before this becomes a real discussion about the future of CPAN: this thread started with Yuval asking about having a Perl Wiki. It would be nice to have a place where the community can be build, where the wide variety of projects can be organized which are required for a good and large new library. Were we can store presentations and teaching materials for the new languages, and so on. -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: comment scope
* Damian Conway ([EMAIL PROTECTED]) [060314 23:03]: > Larry wrote: > > >On Tue, Mar 14, 2006 at 06:36:22PM +0100, Ruud H.G. van Tol wrote: > >: Or POD-ish. > > > >Or POD-ish is what A2 proposes. I still like > > > >=begin FOO > > ... > >=end FOO > > And I'm in the midst of writing S26 (Documentation), > wherein I propose a standard: > > =begin comment > ... > =end comment > > form. > (And, yes, they nest correctly). Looking very much forward to it... as you know. But don't you think that the term "comment" is too general? The comment text has a purpose, for instance - comment to explain code - comment for future implementation ideas - references to design documents - etc etc The "comment" keyword is telling us nothing more than =begin skip_this ... =end F.i: =explain code or =begin explain code -- MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: comment scope
* Damian Conway ([EMAIL PROTECTED]) [060315 01:52]: > That's why the general Perl 6 Pod syntax allows any block construct to be > labelled: > > =begin comment (explanation) > The devil made me write this code > =end comment > > =begin comment (future implementation) > Add a :devil option > =end comment > > =begin comment (design) > See www.dev.il/design/S25.pod > =end comment > > =begin comment (etc. etc.) > >:-) > =end comment But are we still applying huffman encoding here? I mean: do we really write that little comments that long constructs are acceptable? What about =comment explain The devil made me write this code =comment future Add a :devil option =comment design See www.dev.il/design/S25.pod =comment etc. etc. >:-) =cut Much cleaner, and I don't like Pascal. -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: UNIX commands: chmod
* Gabor Szabo ([EMAIL PROTECTED]) [060325 09:58]: > So we had chmod in Perl5, I would like to make sure it works in > Perl6 as well with slight modifications. > > LIST = chmod MODE, LIST My feeling is that this function design is a bit of a mistake. Usually, one perl function maps on one operating-system function, but in this case it doesn't: it simulated the common chmod(1) user command. User commands have a different purpose and application than program calls. If chmod complains to a used about "no permission", the used can understand that, filter out the files and handle accordingly. However, applications should have a thorough error-handling. > > 1) In list context it would return the names of the files successfully > changed . > In scalar context it should be the number of files successfully changed > as it is in Perl5 Perl's chmod can better simulate chmod(2) than chmod(1), and see the latter as something derived. If anything is returned, it should be an error-object, not the file name. So, the "real" core chmod is sub chmod($mode, $file) Of course, besides that you may implement a version which accepts a list, but that is certainly not needed except for perl5 compatibility... > 2) In addition to the octet representation it would also be ready to receive > the unix style mode representation which is string > one or more of the letters ugoa, > one of the symbols +-= > one or more of the letters rwxXstugo would be nice... but platform independent? > 3) While some of the modes are UNIX specific, it would be nice to find similar > modes in other operating system and do the right thing there too. > e.g. "all" in UNIX would map to "Everyone" in Windows/NTFS Then we come into the dangerous dungeons of File::Spec. It would be so nice to redesign that: my Dir $top .= new($root); my Dir $etc = $top.cd('etc'); $etc.chmod('g+w'); my Stat $s = $etc.file('passwd').stat; for $cwd.glob('*.c') -> .chmod('a+w'); my File $f .= unix('/etc/passwd'); When it is easy to manupulate dirs and files as objects, than we can hide all these OS-specific calls about directories, paths, modes and stuff in the objects, outside the core language. > 4) "filename".chmod(MODE) should also work and I guess > .chmod(MODE) should also work on those 3 files Should all methods which accept a string as argument have an alternative method in the string class? Why then in this case? -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: UNIX commands: chmod
* Gabor Szabo ([EMAIL PROTECTED]) [060326 19:26]: > On 3/26/06, Mark Overmeer <[EMAIL PROTECTED]> wrote: > > > LIST = chmod MODE, LIST > > > > My feeling is that this function design is a bit of a mistake. Usually, > > one perl function maps on one operating-system function, but in this case > > it doesn't: it simulated the common chmod(1) user command. > > If we look at Perl just as a high-level language than you might be right, > and I really liked the object being returned idea, but I think Perl should > also keep serving the system administrators with much simpler needs. "Simple tasks must be simple, and complex ones must be possible" Once upon a time, handling file(name)s was simple. But not anymore: os-specifics and character-sets are a pain. One of Perl's targets is to be platform independent: with only a few letters more you can achieve that wrt file- and directory handling. -- MarkOv [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: UNIX commands: chmod
* Larry Wall ([EMAIL PROTECTED]) [060327 01:07]: > On Sun, Mar 26, 2006 at 02:40:03PM -0800, Larry Wall wrote: > : On the original question, I see it more as a junctional issue. > : Assuming we have only chmod($,$), this sould autothread: > : > : unless chmod MODE, all(@files) -> $oops { > : ???; > : profit(); > : } > > Except that junctional logic is allowed to fail as soon as it can be > falsified, $oops being the filename or the error? To produce a good error report, you need both. To be compatible with Perl5, the order of the @files must be preserved. Is it really worth it to design a new syntax to avoid the use of 'for' with chmod? In more characters as well? -- MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: weak roles
* Yuval Kogman ([EMAIL PROTECTED]) [060808 01:04]: > The CPAN has many reinvented wheels. This happened since we have so > many authors (== opinions) over so many years. > > For example, we have Mail::Message to Email::Simple, to > Mail::Internet to They all do the same thing differently. Sometimes, the external interface of a module looks the same, and there are cases where the internals behave as such as well. In general, however, the internals are more different then the user code shows. That makes your proposal unworkable. In this example: Mail::Internet is deprecated: does not even support multiparts Email::Simple does e-mail as simple as possible Mail::Message does e-mail as RFC-compliant as possible > Then there is Email::Abstract, which provides a simplified API for > all of these together, so that modules like Mail::Thread can work > with either. For different approaches to tackle the (in this case e-mail) problems you need different internal helper objects, different methods, different attributes. It is in general hard to force an extra interface definition inside the same class. > Perhaps an invokation of an ambiguous method is an error for the > caller ($object responds to N flavours of the "header" method, you > must explicitly say which flavour you prefer right now). But that's not all. A header contains fields, which are usually objects as well. How would you include variation in those in the definition? And different versions of one module which use your Email::Abstract module? You must not try to understand the structure of other modules into such a detail, because you cannot maintain it. > Delegates are another way around this but let's face it: Delegates are not sufficient to implement such couplings between "unrelated" modules: you commonly need serious parameter rewrites. Delegates are nice within a homogeneous concept. The only way I have found to work is with "wrapper" objects which translate between the two modules. The advantanges are that you have localized the hassle of interfacing to one class, and you hide the internal complexity of both sides (OO is about abstraction!). Of course, we could use the Email::Abstract interface as a base- class to all email related modules, but you know that this wouldn't work for the Perl community... -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: [svn:perl6-synopsis] r14421 - doc/trunk/design/syn
* Damian Conway ([EMAIL PROTECTED]) [070613 22:46]: > My underlying model is that documentation and the source it's documenting > should be entirely orthogonal. So, to me, it would be very surprising if a > programming construct (block comments) interacted with documentation. Or > vice versa. > > * At the Perl 6 level, all Pod is syntactically and semantically > invisible and therefore already equivalent to a comment. > > * At the Pod 6 level, all Perl code is merely meaningless ambient > background noise and therefore already equivalent to whitespace. We had a private discussion about this already three years ago: I fully disagree! If the code and the documentation are not entangled, why do you want to put them in the same file? Why do you put them in the same distribution even? No, the documentation is all about the code. The docs present everything what the user should know about the code. The docs are the user's view of the implementation, and the code is the computer's view on the same. Realizing that, there are a few problems: - is everything in the code documented? - is everything documented correctly? - during development, the interfaces change. Do we update the docs? Getting people to document (and maintain the documentation) is not easy. For some people, it's just not in their system. So, if you can make it easier to do, if you can avoid the need for duplication (write it in code, write the same again in the docs), if you can save some typing, you probably end-up with better code. Last week, I restructured the code of MailTools... published in 1995, one of the oldest modules around. Also one of the most used modules and described in many books. Changing the manual-page organization from "at-end-of-file" to "interleaved" put the docs close to the respective code. I found many mistakes in the docs, during this process: methods which were not documented, methods which were not implemented, parameters which were not or wrongly described... As some people know, I wrote OODoc which extends the syntax of POD(5) with logical markup and some simple avoidance of replication. For the MailBox distribution (which is quite large), that saved me typing of 700.000 characters (35%)! Do you know how long it takes to type those? So how much time did I gain to spend on code quality? I had suggested syntax like this, in Perl6: class Mail::Message { `base class for message types .has $msgid; `The (internet wide) unique string which identifies this `message object. May be undef as long as the message is `begin composed. .has $content_type = 'text/plain'; } You see that there is little room for errors in the documentation. Before people start flame-wars: the syntax is just to start a discussion, to show how simple it could be. I am not stuck on that. This can be automatically translated into (traditional POD like this) (leaving out some blank lines for shortness sake) =head1 NAME Mail::Mesage - base class for message types =head1 INHERITANCE Mail::Message isa Object Mail::Message is extended by Mail::Box::Message =head1 METHODS =head2 Attributes =over 4 =item C<$msgid> The (internet wide) unique string which identifies this message object. May be undef as long as the message is begin composed. =item C<$content_type> (default 'text/plain') =back Damian, can you show how you would document the same code in POD6 syntax to get a comparible short man-page? IMO: code and docs are two representations on one thinking processes, named "programming". They are the opposit not orthogonal: parallel developments with a little shifted focus. The separation between State and Church is only about power: that the Pope can't tell the President how to rule a country. But the people need to merge their religious believes with their social duties. Are you designing for "The Power" or "The People"? -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: POD <-> Code entanglement
* Thomas Wittek ([EMAIL PROTECTED]) [070614 15:18]: > So maybe directives like method, sub, attribute, class etc. might be a > better choice regarding semantics. See OODoc::Parser::Markov > It's a bit like HTML<->XML, where the former lacks most of the semantics > and makes the information processing - not to speak about a consistent > look over several documents - a lot harder. In HTML, you have logical markup as well: EM, STRONG, KEY, CODE, etc With id and class, you can make any annotation you like: my_method OPTIONS This is the description of a method. I love the power of CSS. > I could imagine a semantic documentation in Perl6, that could be > translated to XML/HTML+CSS or to POD(6) for formatting it. The nicest thing would be that the semantic docs become part of the parse tree, which then (using standard introspection) can be used to generate manual pages, natively into POD, roff, HTML, whatever. I see no reason why entangled docs are so hard to parse for Perl6, as Damian arguments. Even it being difficult is not a good reason to make the life of all programmers harder. -- MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: POD <-> Code entanglement
* Thom Boyer ([EMAIL PROTECTED]) [070614 15:49]: > the existing S26, say things like: > > =Method the method synopsis goes here > =begin Parameters > =item foo is the fooiest parameter > =item bar is the barstest parameter > =end Parameters Where is the link with the code? That's the point: there is no automatic checking/avoidance of repetition. Besides, your example is not defined by S26: one can decide to use the tags, someone else chooses other names, and they then cannot be combined into one nice homogeneous set of pages. That's a horror! And if you really like above syntax, why not define =method the method synopsis goes here =option foo is the fooiest parameter =default foo 10 =requires bar is the barstest parameter Which is close to how OODoc is extending POD for Perl5. IMO We can (should) do better for Perl6. -- MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: [svn:perl6-synopsis] r14421 - doc/trunk/design/syn
mmentary) - Use expressions where expressions are good for (calculation) - Use regexes where regexes are good for (matching) ... i.e. it is not a valid argument: expressions and regexes are integrated. >- Then generate unified documentation by extracting information from > wherever its available (source or mark-up), according to what the > reader asks for The semantics differ per module, so the reader cannot (for instance) use her own style sheet: the superset of definitions is not known. >- The issue is not having sufficiently powerful Pod syntax; > the issue is having sufficiently powerful documentation-generating > *tools* IMO it is: the issue is to have sufficiently integrety in code and markup to be able to create documentation-generating tools which produce enough quality. And at the same time give the programmer the mimimal burden on writing documentation, to increase the chance that it is present and good. [this last sentence is my only design criterium] > To answer Mark's specific question: > > > class Mail::Message { > > `base class for message types > > > > .has $msgid; > > `The (internet wide) unique string which identifies this > > `message object. May be undef as long as the message is > > `begin composed. > > > > .has $content_type = 'text/plain'; > > } > > > > Damian, can you show how you would document the same code in POD6 > > syntax to get a comparible short man-page? > > Like so: > >class Mail::Message { >=PURPOSE Base class for message types > >has $msgid; >=for PURPOSE >The (internet wide) unique string which identifies this >message object. May be undef as long as the message is >begin composed. > >has $content_type = 'text/plain'; >} This is just a syntax transformation, where I can live with. No problem. But it is not the whole story. "PURPOSE" is not in S26. So: you need at least a few lines more in above counter-example to make this work. - the semantic pod - in the purely structural pod - in pod2whatever more? In my "vision", the example is complete. Everything else is determined by the processing tools and style sheets. > PS: I agree that there needs to be a mechanism for abstracting names >within Pod and for extracting those names from ambient code. I will >propose such a mechanism and the syntax for it in a subsequent email >(probably early next week). I am looking forward to it. Do not understand me wrong: for the choosen approach to produce documentation, you did an excellent job with the specification. It is well written, well documented, and even implemented. But I do not see how this approach contributes to the homogeneous set of manual-pages for Perl modules that the end-user prefers. And I feel that its freedom makes my life as programmer any easier. [ Damian, we didn't change our opinions a thing since the last debate on this subject, last year, haven't we. Probably just another holy war ] -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: [svn:perl6-synopsis] r14421 - doc/trunk/design/syn
* Smylers ([EMAIL PROTECTED]) [070616 09:09]: > > * Damian Conway ([EMAIL PROTECTED]) [070615 00:17]: > > > * Pod 6 is both a structural and a semantic scheme; you can specify > > > both the structure of a document, and the meaning of its various > > > components > > > > Yes, and that is one of the things which worries me most *You can*. > > It's full freedom, > > You're concerned that an aspect of Perl 6 might have too much freedom? > Isn't Perl all about giving users freedom to choose their own way of > doing something? Why treat documentation as a second-class citizen all the time? Why have a standard syntax for regexes, and not for docs? Aren't you glad that at last we get a standard for OO programming and named parameters? The boundary between freedom and anacharchy is faint. > Yes. But in reality many people will follow what others do, or look to > follow best practices. With Perl 5 you have complete freedom as to the > names of C<=head1> sections in the Pod for modules, yet in browsing Cpan > it's clear that there are conventions and many people use the same > headings. So not mandating a convention isn't much of a problem. Well, the you are talking about the top three headers, the most. And those have a good example in standard UNIX manual-pages. So: there is a definitions for them, which most people have seen. Look at how people document how methods should be called. We, as programmers, don't worry: we can read between the lines. But most "normal" programmers we hardly ever meet, have a hard time. Is "freedom" (in some definition) so important that consistency must suffer? > Do you really think that people can now, before Perl 6 has gained > anything approaching the usage we expect, make policy for how things > should be documented, such that that policy will be the best possible > way of documenting everything written in Perl 6, for ever? Or even a > good way? There is no need to think that a documentation syntax develops differently than a programming language. So when Perl is developing, POD can develop in parallel. And, of course, documentation is for most people a well known area: at least, every program I wrote last 30 years had some form of documentation. To find a kind of common base of features you need does not require any knowledge about the language involved is quite simple. > That strikes me as incredibly shortsighted, verging on arrogance by > whoever comes up with the rules, and doomed to failure. Sorry? Not only you insult me, but you also ignore all these other languages which do have a nice and standard way of documenting. Insignificant languages, like Java, which we tend to ignore. Also Python, to name a closer neighbour. It is incredibly shortsighted to think that any experienced programmer would suggest a standard which cannot evolve, and adapt to variance in needs. What I suggest is at least some kind of suggestion how to denote the things everyone certainly needs. As I understand now, Damian is working on that. > If a particular convention gains widespread approval then peer pressure > should encourage its use (in the same way that strict and warnings are > currently optional in Perl 5, but in the absence of a good reason it's > expected that they be used). Well, "man Perl/BUGS" says: The -w switch is not mandatory. So at least some people agree that there may need some more guidance. Warnings default "on" is a good idea, just as giving people a good default documentation strategy. -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: = at Start of Line ([svn:perl6-synopsis] r14421 - doc/trunk/design/syn)
> brian d foy writes: >> I doubt other languages will want to deal with this situation. * Smylers ([EMAIL PROTECTED]) [070616 08:44]: > With these new Pod rules it's possible to entirely remove Pod from a > file without knowing _anything_ about the host language. (It could > straightforwardly be done as an editor macro, for example.) That > permits Pod to be used to document just about anything; all you need to > allow it is a filter that strips off all Pod before you do anything else > with the file. And then the main point: if you write documentation which is not related to Perl6 coding itself, do we really need to create just another text processor? There are already so many sofisticated text processors available! Well, ok, if you "get it for free", like with POD(5), then take that opportunity. But if the life of ordinary programmers who write documentation is complicated just to provide this feature, we are IMO on the wrong path. And why do you want easy to remove docs? Perl6 is even nicer: you can distribute it compiled; no program text and no docs! Docs in code files are much less of a "burden" than in Perl5, so the need to strip them from the code has deminished. > If Pod were to take notice of the host language's context throughout the > file then this would not be possible: every language which wished to > have Pod support would require its own Pod parser embedded within the > languge parser. _That_ is orders of magnitude more complex than the > simplicity of filtering off all Pod first, and strikes me as something > other languages are much less likely to be bothered to do. Other languages already have their own documentation system. Why do you expect them to use POD6? Why should we design POD6 with this endlessly open requirement in mind? Let's rule the World! All I try to achieve is the best possible set of documentation for end-users: consistent in structure and correct in content. To achieve this, I wish to keep the freedom and avoid the anacharchy. Correctness automatically improves where redundancy is removed. But that requires a closer doc/code entanglement than a lot of the experienced Perl people like (==are used to). -- MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: [svn:perl6-synopsis] r14421 - doc/trunk/design/syn
* Damian Conway ([EMAIL PROTECTED]) [070616 08:29]: > No. There *will* be conventions, which will be defined by the behaviour > of the standard documentation tools that we create. man-page Perl6 secion BUGS: The "Damian Documentation Conventions" should have been mandatory. [Careful: this is intented to be a pun] > >[ ... Probably just another holy war ] > ... I'm merely frustrated by the fact that I can't seem to convince > you that they're actually being addressed. Well, let's have a drink in Vienna to ease our mutual frustrations ;-) -- MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: [svn:perl6-synopsis] r14421 - doc/trunk/design/syn
* Jonathan Scott Duff ([EMAIL PROTECTED]) [070616 20:15]: > You mention OOP. For Perl 5 we have a standard, if very general, > syntax and "open" semantics that have allowed people to implement OOP > in a variety of ways. This was all well and good for a while until we > realized that there should be some more reasonable defaults (in both > syntax and semantics) for common operations in OOP. OOP in Perl5 has a mechanism, but not a standard in the use of those features. Full OO languages all choose a direction, but there are serious culture differences. Perl uses them all. There are many ways how to instantiate an object, which is lot of fun as programmer but a hassle for "the average" programmer. Within one program, you may very well need to use different instantiation techniques... Of course, this could have been forseen (perl is not the first language which implements OO), and some advice on the convention to use could have avoided wild collection of approaches we see now. This is also why "Perl Best Practices" is a good book [shameless plug] although I would have welcomed it 11 years earlier. > I think it's the same thing with POD6. It's "open" enough that many > documentation systems can be built from it (man pages, books, magazines, > wikis, etc.) For some of those documentation systems we'll have nice > conventions and other conventions will grow as needed. If we find that > convention isn't enough in specific areas, ... Without any doubt, there are thousands of documentation systems around. At least ten percent of them are considered "the best ever made" by its developer or developer community. Just by simple math, the chance that the system developed really is the best is less than one percent. Gladly, there are many areas of application for documentation systems, and there is a large variety in taste by the users. So per user/usage, a different doc-system can be the best. Still, its a muddy terrain. POD6 has the unique opportunity to become "the best documentation system" for a specific large group of users: Perl programmers. Tenth of thousands of people will use POD6 in their Perl6 programs, every day. IMO, any argument that POD6 is good because it can be used to write books or express complex mathematical expressions is really frightning me. Damian is correctly avoiding this, in his argument. Is this another area where our community is trying to incorporate every thinkable feature in some design? And in the process, producing such a complex design that it deserves an extra volume in "Programming Perl 6"? (Next to the 1000 page volumes for each of basic syntax, grammars, and OO). IMO, the needs for POD6 are much simpler: a way to document the features of the related Perl6 code, and smooth that into (parts of) manual pages. If you write a book, than use pod6-to-docbook or pod6-to-pod, or whatever, to get some fragments in your some documentation system which is specialized in books. Every single complication added to the doc syntax will make it not to understand for a large percentage of the primar target community, as every teacher can tell you from experience. > Also, I don't think that documentation is being treated as > second-class at all. It's being treated as first-class but different. It's treated second-class in the Perl6 sense: the tools are not an integral part of the Perl6 environemnt. When I create a program, it starts with a goal. To reach that goal, I have to write some code, some docs, some tests. For me, it is all part of the program. I do not see one to be of more importance than the other: all three are first-class sitizens in my program. And what I really wish for is that those three work together in een KISS way; to achieve my goal in no time so I can drink (more) beer. Yeh, on the subject of tests,... :-b -- MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Pod 6: ease of implementation vs easy of use
* Damian Conway ([EMAIL PROTECTED]) [070616 23:21]: > I will, however, take a moment to answer the accusation that I appear to > have redesigned Pod the way I did in order to make implementation > easier... The opposit: your work is known to seek the corners of the language which hurt most. So please ignore those callers in the dark, which are apparently poorly informed. Recently, I also had a few encounters with grumpy mongers, which, simply based on the fact that they feel more important, feel the need to insult or ignore other people. Especially, the word "Freedom" is used a lot in these flames, although the interpretation of that term is quite different, World-wide. What is acceptable for the sake of "Freedom"? -- MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Referring to source code within Perldoc: the new A<> code
(Perl6's structure is probably much more extended), the more levels of hierarchy you get. Are users able to understand this? Are developers able to maintain manual interface description lists without mistakes? Is the shown syntax sufficient for tools to create it automatically? As decided, of course without looking at the perl code itself. Your design goal of A<> is to avoid replication of code information, in which you succeeded. Now your write method eat(Food $meal) {...} =for DESCRIPTION The A() method has the following argument list: A<(..)> In stead of method eat(Food $meal) {...} =for DESCRIPTION The eat() method has the following argument list: Food $meal. What I would like is to get rit of the replication of that description line as well, using back-end specific templates/style-sheets. What about: =definition method eat(Food $meal) {...} =for DESCRIPTION ... In this case, the Perl and POD are using the text in the file in an overlapping way, but still the Perl6 and POD6 parsers are fully seperate. This will open a whole new realm of possible simplifications. -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Referring to source code within Perldoc: the new A<> code
* Damian Conway ([EMAIL PROTECTED]) [070621 01:45]: > Mark Overmeer wrote: > >This is exactly the form of documentation you do *not* want the > >user to write, for various reasons: > Well, I agree it is the form that "you" (singular, specific) do not want; > but I'm not sure it's bad for "you" (plural, generic) to write in all > cases. ;-) My idea was "you" as lazy Perl programmers (most of us), against "you" as orthodox orthogonalists :-b > > * The explicit naming of the class name in method and attribute > >descriptions is in general a bad idea: by inheritance, you get > >sub-classes which also provide this method. > A dedicated OO documentation tool could certainly do a better job in that > case, I heartily agree. I'm looking forward to using one. OODoc achieves this with POD(5)syntax with extra logical markup tags. It's a simple tool, with simple syntax. So, I would say: try it! Read OODoc::Parser::Markov first. It sounds as if you think that OO/Grammar will be rarely used. That there is no need to help users document it. > > * How do you see this syntactically work in combination with the > >item list? At least the POD(5) needed that. I need a combined > >example. > I'm not sure I understand what you're asking here. Can you describe the > example you'd like to see? In POD(5) you do =over 4 =item myfunction1 some description =item myfunction2 some description =back How would this small example (blanks removed) work in POD6? > > * Using ambient back-references this way probably requires a > >two-pass document generator. AFAIK this is not required for the > >design of POD6 so far. > > Actually a two-pass generator is already required for Pod. A second pass is > always necessary whenever there are any forward-targetted L<#local link> > formatting codes in a document. The is a simple strategy to avoid a second pass in this case. Probably you (signular form) found a special case that this does not work. > > method eat(Food $meal) { > > if($manger.isFull) { $manger.clean } > >=for DESCRIPTION > >The A() method has the following argument list: A<(..)> > > Oops... now the argument list became (without warning) "$manger.isFull" > > Sure. But this is true of any scheme that doesn't do full parsing of the > ambient source code. Which Pod isn't going to do That's doesn't make this less of a problem. I think it is a wide open user trap. > > * In the manual-page of my sub-class, I want to refer to the > >documentation of specific attributes and methods. How? > >Can I also refer to elements in distributions which are not > >mine, so where I cannot add X<> or such? For instance, > >when I inherit from a core Perl class? > > This is not possible, using this mechanism, due to the lexical scoping of > the aliases. An automatic documentation-generator tool (that produces Pod) > is more appropriate for this task. Of course, I can make OODoc produce POD6. However, in a new generation of the documentation syntax, I would really have liked to have higher level features, for instance with which we can produce docs with many links (like HTML), not poor section references like traditional manual pages. > > * In my sub-class, I want to enlist automatically the methods > >and attributes which are inherited. > > An automatic documentation-generator tool is more appropriate for > this task too. For each little bit more complicated than the straight-forward task, you say: let some external tool do that. Each of these spots is a weak point in your orthodox orthogonal approach, where you ask people to install and learn more tools, adding needless complications to the development process. The major disadvantage of external tools is, of course, their incompatible results. Perl's set of manual-pages will continue to be messy. > =Definition >method eat(Food $meal) {...} > > But with that slight change in syntax you could then use the standard Pod > parser to segment your document into mark-up and ambient code, search the > resulting object tree for 'Definition' nodes, grab the immediately > following ambient code block, parse it using Perl 6's own grammar, then > autogenerate content for the next 'DESCRIPTION' node in whatever form you > would like it to be, using information extracted from the parsed code. Wow, sounds really simple to implement. No. > Or, since the Perl 6 grammar can preserve Perl comments in the parse tree, > you could even create a tool that understood an abbreviated notation such > as
Re: Referring to source code within Perldoc: the new A<> code
* Smylers ([EMAIL PROTECTED]) [070621 20:33]: > Documentation, unlike code, doesn't have to be backwards compatible: if > Perl 6.0.1 changes the API of a standard function that will break > existing code; but if Perl 6.0.1 has documentation with a different > structure from Perl 6.0.0, that won't break anything. When 6.0.1 changes the doc structure, then all 6.0.0 docs are unusuable, or need conversion. Although this doesn't cause code breaking, it does have maintenance and usability issues. Don't forget that Perl code is so compact that we (at least me) have more lines of doc than of code in our files. If you take documentation serious enough, you certainly do want (at least some) backwards compatibility too. And as much as possible automatic consistency checks/warrants. -- MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Referring to source code within Perldoc: the new A<> code
> Juerd wrote: >> This dedicated OO documentation must be core, because Perl itself is >> heavily OO. * Damian Conway ([EMAIL PROTECTED]) [070621 23:54]: > Yes. I completely agree that such a tool not be standard and universally Do you mean "must be" i.s.o. "not be"? > available. Just as the pod-to-whatever back-ends of the documentation > chain must also be. -- MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Referring to source code within Perldoc: the new A<> code
* Damian Conway ([EMAIL PROTECTED]) [070621 08:07]: > Mark Overmeer wrote: > [...yet another honest and heartfelt plea for Pod 6 to be something > entirely different from what it is currently designed to be.] > The solution is simple, you know, Mark. Why not just write up your own > alternate S26, redesigning Pod 6 the way you think it should work, and > then publish your proposal for consideration here? You may remember that I repeatedly asked @Larry not to forget the documentation aspect in the redesign of Perl, in person during various YAPCs and Workshops. Then, when you finally took the challenge, I have send you a extensive email showing various alternative syntaxes for condensed, integrated documentation strategies. (2 nov 2005) Those design examples are far from a detailed design, because that takes weeks, not hours. Besides, there are a few good alternatives when the code/doc separation doctrine is left. For instance, Juerd's "is documented" traits. I have already proven that adding some simple logical markup to the POD(5) syntax can simplify the documentation process enormously, with my OODoc (::Parser::Markov). As I already reported in one of the initial messages of this (long) thread, the tool saved me to type 700.000 characters of (needed dupplicated) text for my MailBox suite alone. That was a simple gain within the classical POD dogma; with real integration, we can reduce our documentation efforts much further. [Shall I give a lightning talk on OODoc in action, at upcoming YAPC::EU?] > After all, Larry's track record is clear: he's never once allowed > someone's reputation or status (even his own!) to deter him from > replacing an existing design with someone else's superior one. True. However, when the common @Larry believe is that POD6 should build on POD(5) ideas of orthodox orthogonalism, then it is a waste of my sparse time. I am not afraid to take such challange, opposit of that: otherwise this discussion probably had died out days ago. But I do have a number of large Perl(5) projects on my hands already. -- MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: [svn:perl6-synopsis] r14421 - doc/trunk/design/syn
I fully agree with David's response to this mail. The only thing I would like to add: * Smylers ([EMAIL PROTECTED]) [070621 18:02]: > [*0] Consider a function C. I'd document it along > the lines of: > > valid_postcode > > Returns whether the specified postcode is valid, for example: > > if (valid_postcode $postcode) { > > Javadoc-style systems seem to insist on documentation like: > > valid_postcode > Description: Returns whether the specified postcode is valid. > Parameters: > $postcode: (string) The postcode to test for validity > Returns: (boolean) Whether $postcode is valid > Exceptions: none > Side Effects: none Of course, you can write horrible documentation in any syntax: that's up to the authors. But now, just try to write above documentation in the new POD6 syntax... in that case, it is not only horrible documentation, but also 2 pages long. In my idea, it suffices to write: method isValidPostalCode(str $postalcode) returns bool {...} By introspection during manual-page creation, it can collect sufficient information to create this documentation item (controlled by a (user-provided) template). With the POD back-end, something like (blank lines removed) =head1 METHODS =over 4 =item $obj->isValidPostalCode(str $postalcode) returns bool =back Then, when you want to add some docs to the method, to help the correct use, add it, for instance like: method isValidPostalCode(str $postalcode) returns bool {...} ` Check wether the postal code confirms to the standards `$postalcode: a string with blanks trimmed. `return: the string is not cleaned-up. or maybe (exact syntax open to discussion) method isValidPostalCode(str $postalcode) returns bool {...} #= Check wether the postal code confirms to the standards #=$postalcode #= a string with blanks trimmed. #=return #= the string is not cleaned-up. or method isValidPostalCode(str $postalcode) returns bool {...} // Check wether the postal code confirms to the standards // // $postalcodea string with blanks trimmed. // return the string is not cleaned-up, if you need // that, call M. or maybe at the bottom of your file, whatever you like __DOC__ =doc isValidPostalCode Check wether the postal code confirms to the standards $postalcodea string with blanks trimmed. return the string is not cleaned-up, if you need that, call M. There is so much user-friendliness to gain. Very condensed documentation. Of course, you will get simple ways to change the default mark-up of the parameters, for instance for the case of MMD's, huge parameter lists, or where the parser cannot figure-out info automatically. -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Referring to source code within Perldoc: the new A<> code
* Damian Conway ([EMAIL PROTECTED]) [070622 08:38]: > And, no, I don't consider the pointers to your excellent module to be > suitable specific examples of what we're not giving you...mainly because I > believe that the Pod 6 documentation language I've designed (in conjunction > with the ability for Perl 6 to parse Perl 6) *does* give you what you need > to build such tools. Well, we tried to avoid the stale-mate discussion, but it's back again. IMO, POD6 should not provide the possibility to build such tools: it should *be* the tool. With a nice (compact) standard definition how to document each of the designed features in Perl6, and in attachment C some details which explain how Ben Smylers can live in anarchy ;-) > So it seems we're still at an impasse. Nah, at least a lot more people are thinking about the subject now. > I fully respect your decision not to > attempt a full alternative design (if anything, your estimate of it only > taking "weeks" is optimistic ;-), but unless someone is willing to step up > and suggest some specific improvements to the current proposal, how can we > move forward towards the best possible result? If you read it well, I say: "it's a waste of time if the idea of orthogonalism (full code and doc separation) cannot be discussed". Because my plans are exactly the opposite: optimally merging doc and code. So, it is only a "no" when @Larry says "no". -- MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: [svn:perl6-synopsis] r14421 - doc/trunk/design/syn
* Moritz Lenz ([EMAIL PROTECTED]) [070622 09:16]: > Damian Conway wrote: > > Would the following syntax suffice? > > > > method isValidPostalCode(str $postalcode) returns bool {...} > > =PURPOSE Check weather the postal code confirms to the standards > > =ARG $postalcode > > a string with blanks trimmed. > > =RETURN > > the string is not cleaned-up. > > > > Because you can already do precisely that in the current design of Pod 6. > > I really like that example. > And it makes me think that some of this discussion results from > insufficient Pod 6 knowledge, simply because we haven't read enough real > Perl 6 that is annotated with Pod 6. No, no... although you can create lines of text which are alike, the whole meaning of these lines and needs for the processing tools behind this is very different. Our discussion has always been about that: how much info can the tools produce automatically: is POD6 integrated with Perl6, or only (accidentally) sharing the same file. -- MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: [svn:perl6-synopsis] r14421 - doc/trunk/design/syn
* Damian Conway ([EMAIL PROTECTED]) [070622 09:02]: > Mark Overmeer wrote: > >Then, when you want to add some docs to the method, to help the > >correct use, add it, for instance like: > > > > method isValidPostalCode(str $postalcode) returns bool {...} > > ` Check wether the postal code confirms to the standards > > `$postalcode: a string with blanks trimmed. > > `return: the string is not cleaned-up. > > > >or maybe (exact syntax open to discussion) > > > > method isValidPostalCode(str $postalcode) returns bool {...} > > #= Check wether the postal code confirms to the standards > > #=$postalcode > > #= a string with blanks trimmed. > > #=return > > #= the string is not cleaned-up. > > Would the following syntax suffice? > >method isValidPostalCode(str $postalcode) returns bool {...} >=PURPOSE Check weather the postal code confirms to the standards >=ARG $postalcode >a string with blanks trimmed. >=RETURN >the string is not cleaned-up. > > Because you can already do precisely that in the current design of Pod 6. This is syntactically what can be done with the current design, but is semantically very different. > And having done it, the current Pod::Parser will extract all of that > information (including the source code), preserving the sequential and > hierarchical relationships, and present you with an internal object > representation that your documentation tool can then analyze and > convert to whatever content and layout you prefer (including > non-semantic Pod that you can then feed into any of a dozen > back-ends). Besides the point that I do not really like the YELLING all the time, it still has the complication to reconstruct Perl code from the Pod::Parser tree. The tools I wish for need to able to match the tree created by Perl with the tree created by your Pod::Parser, in some safe and simple way. The needs for my kind of documentation gerenators are very simple: the Perl6 parser needs to collect all lines of docs which follow a certain line with code into the AST node of that line or item. The Perl6 parser itself doesn't need to do anything more than that. Tools doing introspection on the AST do the rest. You gave the hint that comments are also in the parse tree. So, that is suffiencent for me: sub greetings() { say "Hello, World!" } #= the standard first program<-- doc for user # we like Perl6 new features<-- additional doc on internals. or #=sub greetings #= the standard first program sub greetings() { say "Hello, World!" } # we like Perl6 new features <-- additional doc on internals. [ I think that the doc block should always start with the item what you are describing, so if the block is before the item (or anywhere else in the file), then it needs to be referenced to (as simple as possible) ] [ now I have to study the introspection features of the AST ] -- MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: [svn:perl6-synopsis] r14421 - doc/trunk/design/syn
> >> Mark Overmeer wrote: > >> >Then, when you want to add some docs to the method, to help the > >> >correct use, add it, for instance like: > >> > > >> > method isValidPostalCode(str $postalcode) returns bool {...} > >> > ` Check wether the postal code confirms to the standards > >* Damian Conway ([EMAIL PROTECTED]) [070622 09:02]: > >>method isValidPostalCode(str $postalcode) returns bool {...} > >>=PURPOSE Check weather the postal code confirms to the standards * Jonathan Lang ([EMAIL PROTECTED]) [070622 10:41]: > For the record, I find Damian's example to be considerably more readable. Noted. Of course, it is just a syntactical difference, so not really significant (not significant in the sense of our debate). But, I think we should produce a few real code files and then compare the visual friendliness of the various markup alternatives. CAPITALS do not really attract me, but a design for Perl6 is not about me, but about the average programmer (used to the DOS filesystem?) > Please forgive my ignorance: what does "AST" stand for? The Abstract Syntax Tree, the result of the code parser, un-interpreted. -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Documenting Perl6
=begin INTRO Mongers, I must say, I am a bit disappointed that the discussions about the future of documentation in Perl has died. Or was everyone fully occupied by YAPC::NA? I spent last week with my family on a stormy island, without sufficient internet access, so was unable to stirr things up again, but maybe this email will bring the focus back. Damian challenged me by asking what I think how Perl6's documentation should be done. When I think about documentation, I do not (immediately) think about some mark-up language; that is just a minor component. My focus is on the whole process: from writer to reader. Although Damain says to have studied OODoc, this most import features of that system were ignored: simplifying the documentation process, improving the documentation quality. Each time I re-read the list below, there are things I wish to change or add: it is neither complete nor final. But I cannot wait longer to post it. It's not my wish to extend this into a detailed requirements document, just to set a focus of discussion. Maybe someone wants to comment on it? MarkOv =end INTRO Documentation of code (i.e. Perl6) In this text, we try to determine the environment for the optimal documentation system for Perl6, but applicable to any other programming language. Everyone will have his/her own weights on different aspects, and you may even totally not agree with some of the listed remarks: it is open to discussion. Some of the wishes contradict an other as well. === Goal The sole goal: the best documentation for Perl6 It is very important to keep in mind that documentation is made for some target community to be read. Write-only texts are useless. === Target communities Documentation is added to code, to provide additional information about the code to inform some target community. There are different target communities possible for the same piece of code, which should all be served as good as possible. Traditionally, you see 1) code comments, for maintainers of the software 2) manual-pages, for everyone else But more specific user groups can be defined: 1) code comments, for maintainers 2) developer manuals (the complete interface) 3) user manuals (distribution external interfaces) 4) selective look-up (for perldoc -f or IDE) === Fundamentals There are a few fundaments for good documentation: - it must be written - it must be correct - it must be consistent in structure - it must be consistent in content - it must be accessible (find back/pleasant to read) [Writing] The best way to get people into writing documentation, is to make it as simple as possible. This means: - reduce the need to read man-pages or books to be able to create it, - reduce the amount of text to be typed, - avoid the need for additional tools to be installed, - reduce the need for configuration. All for the sake of laziness. The less time people need to manage their documentation environment, the more time they have to write quality texts. [Writing] To ease the burden of writing docs, documentation generating tools should use as much information from the code as is useful for the target community. Replication between code and docs make changes a double effort. Manual replicated of text between files (like the inclusion of the license text in each file) during programming is an avoidable burden. [Writing] Each documentation fragment belongs to some part of the implementation. This may be a distribution, a file, a class or grammar or package, a method, rule or sub, a positional or named parameter, and so on. This relation comes natural (because of the mixture of code and doc), or enforced (via some reference syntax). [Writing] The documentation fragments need some markup. Many mark-up languages exist, which do have more or less the same features. Two of those are POD and PDD S26. Within one distribution, it is useful to use the same kind of markup syntax. Document generators should only get a minimal abstract interface to collect the results of the markup parser, for instance a $markup->produceHtml($fragment, ...) [Writing] The markup language used should be capable of addressing the things that a document writer wishes to express, not on what certain output back-end can handle (those can always ignore things they cannot handle) [Correctness] The documentation and the related code must be cross checkable, on matters they overlap. Better to avoid replication, in which case there is no overlap to be checked. [Correctness] The user should be stimulated to write in a good style. One of the ways to achieve this, is to avoid the need to write the same sentence over and over again. For instance: "This method returns a boolean, to indicate success" is a sentence to avoid. (Template based) auto- generation could be used introduce abbreviations for often used constructs. [Correctness] Produced manuals should by default be checked f
Re: Documenting Perl6
* Richard Hainsworth ([EMAIL PROTECTED]) [070702 17:06]: > Having read this posting, I find it VERY hard to understand exactly how > Damian and Mark differ fundamentally. They both seem to be after the > same thing in the end. Damain defines a markup language, and says that "tools" (to be defined) will do the actual work. I try to define the needs to create good tools first. The unspoken conclusion from my list must be that POD6 is not bringing us closer to the goal than POD(5) does, because most of the needs are not answered in that design. > A rhetorical question of my own and an answer. > Q) what is the real difference between comments and documentation? > - Syntactically for the perl parser they are both white space. > - Both are used to explain the code. Docs are for end-users, which do want to use the code but were the internal organization is a black box. Comments explain code where it is not obvious for maintainers, looking at the guts. > Questions: > 1) Would it be possible to unify all comments and documentation into a > single paradigm, such as a sort of bracketting convention with optional > extra information for other tools to use, eg., layout information, > grouping information, context (viz., for a tutorial, or reference) > information? All are forms of text, so yes. However, probably you loose more than you win, because, as you said yourself: their purpose is very different. When you forcefully merge unrelated things, you add complexity. > 2) Would it be possible for the brackets to have different 'opacities' > for the perl parser, so that code can be recognised both as code and as > documentation with extra information? In this way, code would be > included in some forms of documentation (eg., function signatures in > reference manuals) just by placing "transparent" documentation brackets > around them. Actually, for commenting code you probably already use invisible brackets: you put comments on seperate lines before a piece of code, and when the next comment block starts, the previous ends. > 3) Would it be possible for the brackets to have 'opacities' for the > perl parser that can be set by the value of some variable, hence > debugging / tracing code could be switched on by setting that variable > in the code? Are you merging the idea of assertions with documentation here? > 4) Is there a means for specifying the way documentation is reassembled, > eg., a means for defining for say a tutorial a different structure than > the linear structure of the documentation strings within the software? I would like to put this responsibility in the back-ends, with a big "YES". > c) Am I wrong in thinking that implementing the suggestions in the > questions should not require a substantial redesign of perl6? There is no need for any redesign. > d) If what I have suggested can already be done with Pod, could some > examples be shown. I have asked for more extensive demonstrations how POD6 would be used, but uptill now only saw very small documentation fragments. But IMO, POD6 is not the problem: it's just not the solution. -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Documenting Perl6 part 2
Hi Mongers, Part one of this story, which I released last week on this list, was really provoking people to rethink the way work work now (not!) The challenge was made by Damian, to design a documentation system which is easier to use: where code and doc work together, in stead of being orthogonally separated. Well, the gauntlet has been taken, resulting in this 0.0 version; of course incomplete. Most of the non-inlined features of this design are available for Perl5 via the OODoc distribution. It is a pity that Perl5 subroutines provide so little information about how they are to be used: therefore you have to specify much more explicitly than required for Perl6. As I have asked twice before, I would really like to see a demo by Damian on how to document a small Perl6 class (say 30 lines) using his S26 specs. Including everything what is needed to get it configured in a way that tools may swallow it. In return, I will use the design described here to do the same... for comparison. MarkOv Documentation of Perl6 === part2: general syntax, version 0.0 The crucial difference between Damian's S26 design and my opinion about documentation, is that IMO the focus must on being able to create tools which produce the best end-user documentation (see part1 of this story) and the markup language is just one (minor) issue. Damians Synopsis S26 does design a markup language, but does not want to define code inter- connection nor predefined markup tags to help the tools. === Choosing Symbols Let's try to avoid reinventing the syntax wheel. In Perl5 we use POD for end-user documentation (text surrounded by lines which start with a '=') and comments to explain the program (texts with a leading '#'). In Perl6, the same symbols are allocated, and people are used to that. Let's stick with them, for now. Defined are two ways to add documentation fragments to the code: (1) the POD(5) way: =over 4 =item . one =back =cut As in POD(5), any line which starts with a '=' will start a documentation block, and '=cut' ends it. Extensions to POD(5) - "internal" blank lines are optional; only the line before the first line of a documentation fragment must be blank. This will make the documentation much more compact to write, so code will get more visible in the file. - each tag is matched with /^\=+(\w+)[\s+|$]/ By permitting more than one '=' as leader, the author can make his/her own visual helpers. For instance, the '=head1' in the file looks as heavy as the '=over', but is of course of much more importance. So: the author may decide to say 'head1'. The extra '=' are not significant. - a (long) list of logical markup tags will be used to add information about what is being documented. We will not write =item print OPTIONS but =sub print OPTIONS or sub print OPTIONS (if we need to) - the tag /^\=+end\s+$1/ will also terminate a pod block, like =cut, but then with a semantic role as well. (2) Inlined documentation, like comment Many (small) features in the program need to get some documentation, for instance, class attributes. To start a documentation block, as described in (1) for each of them is both a lot of work to type and makes it harder to get an overview on the code. Without extension of the available symbols for documentation in Perl6, our hands are bound. So, therefore (for the moment) I use /[^|\s|\;]\#\=/ for user documentation, where /[^|\s|;]\#[^=]/ is programmers comment. Inlined docs are automatically linked to *declaration* above or before them: .has Point $center; #= center coordinate of the universe method move(float $dx, float $dy, float $dz) { #= Jump to a different location, relative to the #= current position. #=param $dx in parsec # of course, time should be included in this # interface. We will do that later. [normal comment] } The last two lines are code comments. === Producing Document fragments Both inlined and block documentation provides the same features. The fragments start in one of three ways: (a) method print() {...} # implementation anywhere in the file #=method print =method print #=description=description #= line1 line1 #= line2 line2 #=end method =end method (b) method print() {...} # implementation anywhere in the file #=method print =method print #= line1 line1 #= line2 line2 #=end method =end method (c) method print() {...} method print() {...} #= line1 #= line2 =description line1 line2 =cut When fo
Re: Documenting Perl6 part 2
* Damian Conway ([EMAIL PROTECTED]) [070711 11:12]: > Just a brief note to reassure Mark--and everyone else who's interested-- > that I'm not ignoring his post...I'm just fully occupied at the moment > with other (paying) work. In the meantime I'm thinking very carefully > about what Mark suggested. I'll reply properly as soon as I am able. I am glad to hear that you are still alive and kicking, and that my suggestions are not simply ignored. But when I read this in the "Perl 6 Design Minutes for 11 July 2007" on use.perl.org, I sincerely hope that the summerizor is wrong: . in the last week, the main people who are not happy with what I've done have put out a document indicated what they'd like. I'm digesting that and I'm going to try to explain how it can be made to fit in with what we already have. I'm in the process of stealing as much as I can to result in a better whole Both part1 and part2 of my story prove that the only thing you do NOT need to standardize in some documentation system is the markup language (like S26); you need to standardize the semantics. Merging two totally disjunct approaches: this will probably produce the worst of both, and makes no-one happy. (*) Sorry. -- Regards, MarkOv (*) that said by a Dutchman, from a culture which is renowned to discuss the middle way as solution for all conflicts. ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: pluralization idea that keeps bugging me
* Larry Wall ([EMAIL PROTECTED]) [080126 16:58]: > Last night I got a message entitled: "yum: 1 Updates Available". > After a recent exchange on PerlMonks about join, I've been thinking > about the problem of pluralization in interpolated strings, where we > get things like: > > say "Received $m message{ 1==$m ?? '' !! 's' }." > > Any other cute ideas? I totally agree with many responses, that special support for the English language is not preferred, certainly when it bothers developers for other natural languages. Imagin that you wrote your code this way for a website, and then your boss (always blame your boss) decides that the site must be ported to Chinese for expansion... It would be nice if Perl joined nearly all other Open Source applications, in being multi-languaged. During the lightningtalks of last YAPC::EU, I called for localization of error messages in Perl 5.12, but Perl6 improvements are welcomed as well. My idea: Recently, I released Log::Report, which is a new translation framework. It combines exception-handling with report dispatch and translations. What's new: some module produces a text, but that module was found on CPAN. Only the author of the main program knows how to handle the text. So, delay translations until an output layer is reached. Locale::TextDomain and gettext translate immediately, as does $! They translate on the location where the report emerges. Log::Dispatch and Log::Log4perl cannot influence the text production process. What my new Log::Report does, is delaying translations to the moment it reaches the dispatcher. Like this: package main; dispatcher SYSLOG => 'syslog', language => 'en-US', charset => 'ascii', facility => 'local4'; dispatcher STDOUT => 'website', language => 'cn', charset => 'utf8'; run_some_code(); # text both to syslog and stdout package Someone::Elses::Package; use Log::Report 'translation-table-namespace'; sub run_some_code() { # Locale::TextDomain compatible syntax, info ~ print info __nx"Received {m} messages", $m, m => $m; } To syslog in English (what I understand), and to the website in Chinese (what I do not understand) Of course, there are quite some more features in the module. The translation tables can have gettext syntax, database driven, or maybe a module with Perl routines from complex languages. (Only the first is implemented on the moment, but the framework is present). The provided try() is also implemented as dispatcher, which collects the messages from the block, and has not yet been translated: try { error __"help!" }; if($@) # an Log::Report::Dispatcher::Try object { my $exception = [EMAIL PROTECTED]>wasFatal; $exception->throw # re-cast if $ex->message !~ m/help/; # ignore call for help } When someone starts coding, it is more and more uncertain in which languages it will be used later. So, it would be nice to help people to avoid mistakes which may block an easy conversion. For instance, best if texts are produced in as large blocks as possible, outside the program file. We know how to do that: a template system. Templates themselves are easily translatable. About a zillion or two CPAN modules implement a Locale::TextDomain-like HASH-based substitution system in templates. Translations are impossible for syntaxes like this: print "Received $m messages" because the $m is already filled-in before print is called. For this reason, a lookup in the translation table is impossible. It would be nice to not translate above string into print 'Received '.$m.' messages' but report info => 'Received {m} messages', m => $m, linenr => __LINE__, ..etc.. (of course, some \Q\E like meta-syntax, not {}) Print() works internally more like printf(). No problem. Without translation tables defined, it just takes what it got as first argument. In the infrastructure, we need a reason for each message, like syslog levels. Print, warn, and die have implied reasons (resp info, warn and error). Everyone is tricking trace and verbose levels, so we need a few more useful levels. Concluding: - hopefully, there is a way to simplify the work for all of us who do need to support many languages within one application - create one standard, so all CPAN modules integrate in the same way - let's try to get Perl to handle languages! -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: pluralization idea that keeps bugging me
* David Green ([EMAIL PROTECTED]) [080131 08:48]: > I've always wanted a magic-S (and I don't think the anglocentrism > matters, because Perl is already pretty anglocentric -- more so than > plural S's, which apply to some other languages anyway). In "the good old days" all computer OSes were anglo-centric. They are not like that anymore. But Perl still is. > use Locale::Lingua::EN; > say "There was\s {3} ox\s";# There were 3 oxen > > use Locale::Lingua::Romana::Perligata; > say "{3} bos\s erat\s";# 3 boves erant > > Although calling it "\s" loses its impact in other languages But > I think the underlying idea to seize on is a way to grab interpolated > values so that there's a nice way to do tricks like that. Preferably > in a way that doesn't look symmetrical so you can "point" it before or > behind. As I suggested in a previous mail, we can do it by making say/print a bit smarter. Instead of interpolating before they are called, we let them interpolate themselves, and optionally translate. > say "I've got $bid dollar\s, do I hear {$< + 1}?" pre-parse standard call to (s)print(f)/say from say "I've got $bid dollar, do I hear ", $bid+1, "?" into print "I've got \Ibid\E dollar, do I hear \I__ANON1__\E?\n", bid => $bid, __ANON1 => $bid+1, __LINE => __LINE__; (introducting \I \E as interpolation indicators) Isn't the (usual, existing) translation syntax a lot simpler than you suggest? (the rewrite will take place as first step within the print/say. Translations must be implemented in the output layers, because only there we know enough about character-set and end-user. Within the program, you do not want to be bothered with translated strings) The default interpolation implementation for print() can be very simple. However, now we can also make translation modules which use external tables or databases to do the optional intelligent work. I do not think that your use Locale::Lingua::Romana::Perligata; is usable, because the translation (in general) adapts to the language of the user of the module, not the likings of the author. A more general use is: setlocale('lat') open OUT, ">:language('lat'):encoding('latin1')", $f -- MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Files, Directories, Resources, Operating Systems
* Richard Hainsworth ([EMAIL PROTECTED]) [081126 08:21]: > The S16: chown, chmod thread seems to be too unix-focussed. > > To be portable, the minimum assumptions need to be made about the > environment in which a program operates. Alternatively, the software > needs to be able to determine whether the environment it is operating in > meets a minimum set of conditions. > ... > Thus I would suggest that the perl6 specifications should be written in > an abstract way, one not related to a specific operating system and in a > way that can be adapted by an implementor to specific systems. I fully agree with you: the way the design is going is making the same "mistakes" of Perl5 again. Where we were able to release the Perl5 syntax more and more when the design of Perl6 made more progress, so should we do with the way we use modules. S16 is not doing that. Also Rafael's suggestion to focus on POSIX is not the way a nice interface should work. POSIX calls (and non-POSIX means) are ways to implement the interface to the Operating System, which can be different from the most practical interface on implementation level. We should focus on OS abstraction. For instance, if a file is represented in an object, then the most friendly interface names would be like: $file->owner($user); my $user = $file->owner; under the hood, we use chown and stat. I really would like to see a "standard" object oriented view on the system, which mainly autodetects the environment. I am really fed-up using File::Spec and Path::Class explicitly all the time. Also, I get data from a CD which was written case-insensitive and then copied to my Linux box. It would be nice to be able to say: "treat this directory case insensitive" (even when the implementation is slow) Shared with Windows default behavioral interface. So, I would like a radical change... trying to be as much general (non UNIX specific) as possible: (sorry, my Perl6 syntax is still very sloppy) some global $*OS # may be different per parallel instance of the program # Maybe an OS function which returns $*OS my $dir = $*OS.dir($*PROGRAM.arg[0]); # above maybe hidden with a functional wrappers: dir $argv[0] $dir.case_sensitive(0); if $dir.entry('xyz').is_file {} my $f = $dir.file('xyz'); $f.owner($*OS.user); $*OS.system('ls | lpr'); print $*OS.family; print $*OS.kernel_version; my $pid = $*OS.process.label; We should also be aware that we design Perl6 for parallelism. Do we require all nodes to run the same OS (~version)? Besides, I would really like to get a incremental growth path to do things we cannot do yet. Some things are currently difficult to realize under UNIX/Linux because there is not kernel interface defined for it. For instance, you do not know in which character-set the filename is; that is file-system dependent. So, we treat filenames as raw bytes. This does cause dangers (a UTF-8 codepoint in the filename with a \x2F ('/') byte in it, for instance) But as long as the OS cannot provide the user with this information, we should still give the author a way to specify it. $*OS.filesystem('/home', type => 'xfs', name_encoding => 'latin1' , text_content_encoding => 'utf-8,bom', illegal_chars => "/\x0" , case_sensitive => 1, max_path => 1024); I have been working on such a module for Perl5 (which has a much wider field than Path::Class) but (as many other of my projects) did not complete it to a usable/publishable level (yet). It is all NOT too difficult to implement (we do share this knowledge), but the design of this needs to be free from historical mistakes. That's a challenge. -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Files, Directories, Resources, Operating Systems
* Leon Timmermans ([EMAIL PROTECTED]) [081126 15:43]: > On Wed, Nov 26, 2008 at 12:40 PM, Mark Overmeer <[EMAIL PROTECTED]> wrote: > That is a task for the operating system, not Perl. You're trying to > solve the problem at the wrong end here IMHO. In my (and your) case, the operating system is not helping at all and there is no chance in having that changed. So... My remark was just one example, and I can give many more, where I would like to see more abstraction in the OS interface to avoid the need for each user to re-invent the wheel of interoperability. > > For instance, you do not know in which character-set the filename is; > > that is file-system dependent. So, we treat filenames as raw bytes. > > On native file-system types (like ext3fs), character-set is not > file-system dependent but non-existent. It really is raw bytes. Not on the presentation level to the user. This makes it even more horrifying. It depends on the setting of an environment variable of the actual user how the bytes of the filename are interpreted. On the OS filesystem implementation you are probably correct (in the UNIX/Linux case), but programs are used for end-user results. > > This does cause dangers (a UTF-8 codepoint in the filename with > > a \x2F ('/') byte in it, for instance) > A \x2F always means a '/'. UTF-8 was designed to be backwards > compatible like that. Yes, you are right on this. ASCII does not suffer from UTF-8, so my example was flawed. The second 128 does cause problems. How can glob() sort filenames, for instance? UTF-16 names should not enter the Perl program unless you are aware of it, because those can hurt badly. Please comment on the big picture in the debate: there are all kinds of OS dependent things I really would like to see hidden in a (large) abstraction layer to simplify the development of portable scripts. I don't say I know all the answers, but I do feel a lot of pain in each module for CPAN the same thing again. -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Files, Directories, Resources, Operating Systems
* Tom Christiansen ([EMAIL PROTECTED]) [081126 23:55]: > On "Wed, 26 Nov 2008 11:18:01 PST."--or, for backwards compatibility, > at 7:18:01 p.m. hora Romae on a.d. VI Kal. Dec. MMDCCLXI AUC, > Larry Wall <[EMAIL PROTECTED]> wrote: > > SUMMARY: I've been looking into this sort of thing lately (see p5p), > and there may not even *be* **a** "right" answer. The reasons >why take us into an area we've traditionally avoided. What a long message... > Mark>> We should focus on OS abstraction. > Mark>> [...] the design of this needs to be free from historical mistakes. > ... It cannot be > done in an automated fashion, since you can't know a filesystem that knew > *locale* each filename was created under, and without that, you have to > guess--almost always wrongly. Exactly. This is an historical mistake, understandable to have at least a path of growth from the current system open() interface. Only users which have the same locale can see the names the same. If you change your locale your filenames break! Say you change from cyrillic into English. In my suggestion, the programmer (who is ofter local on the system) can at least say what the locale was when the filenames where created. On some OS, that OS can tell you. What I would like is an object model which does allow us at least to abstract these problems away... whether it can be resolved automatically or only with help is for later. > There is ABSOLUTELY NO WAY I've found to tell whether these utf-8 > string should test equal, and when, nor how to order them, without > knowing the locale: > > "RESUME", > "Resume" > "resume" > "Resum\x{e9}" > "r\x{E9}sum\x{E9}" > "r\x{E9}sume\x{301}" > "Re\x{301}sume\x{301}" This is done by the locale of the user of the script, as usual for ls(1). So, I do not see your problem here. I don't mind if problems with unicode are not solved or solvable. Could be discuss about a buildin File::Spec/Path::Class? And we allow us the same limitations as these have, for the moment. -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: S16: chown, chmod
* Martin D Kealey ([EMAIL PROTECTED]) [081202 04:37]: > On Tue, 25 Nov 2008, Dave Whipp wrote: > > sub setstat(String|File $filename, StatBuf $stat) { > ... > if $caps.CAP_FOWNER { > # we're privileged, so it *should* just work. > POSIX::chown $filename, $stat.uid, $stat.gid; > POSIX::chmod $filename, Fcntl::ST_PERM($stat.mode); > POSIX::utime $filename, $stat.mtime, $stat.atime; > return; > ... > if catching_exception(IO_error) { > throw IO_error(EPERM, $filename, "Can't give file away"); Implementing things this way is as big a mistake as doing this: (Perl5 syntax) if(-r $fn) { open READ, '<', $fn; while( ) For the two reasons: (1) you have a race condition: the permission may change between the test and the actual open(). In very rare cases, which makes those bug hard to trace (2) the check itself is flawed: whether you can chown() does not only depend on the OS, but also on the disk: might be a CD-ROM. The only correct way to implement it, is simply try chown() and handle the error. Yes, you need quite more code for that. -- MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Files, Directories, Resources, Operating Systems
* Aristotle Pagaltzis ([EMAIL PROTECTED]) [081204 14:38]: > Furthermore, from the point of view of the OS, even treating file > names as opaque binary blobs is actually fine! Programs don’t > care after all. In fact, no problem shows up until the point > where you try to show filenames to a user; that is when the > headaches start, not any sooner. So, they start when - you have users pick filenames (with Tk) for a graphical applications. You have to know the right codeset to be able to display them correctly. - you have XML-files with meta-data on files which are being distributed. (I have a lot of those) - when you start doing path manipulation on (UTF-16) "blob"s, and so forth. I have been fighting these problems for a long time, and they worry me more and more because we see Unicode being introduced on the OS-level. The mess is growing by the day. > To that, the right solution is simply nt to roundtrip filenames > through the user interface; instead, keep both the original octet > sequence as well as the decoded version, and use the decoded > version in UI but refer back to the pristine original when the > user elects, via UI, to operate on that file. But now you simply say "decode it". But to be able to decode it, you must known in which charset it is in the first place. So: where do we start guessing? An educated guess at OS level, or on each user program again? > decoding issues. The higher-level problems like sorting names in > a locale-aware fashion will be solved by the CPAN collective much > better than any boil-the-ocean abstract interface design that the > Perl 6 cabal would produce – if indeed these are real problems at > all in practice. Why? Are CPAN programmers smarter than Perl6 Cabal people? What I whould like to be designed is an object model for OS, processes directories, and files. We will not be able to solve all problems for each OS. Maybe people need to install additional CPAN modules to get smarter behavior. But I would really welcome it if platform independent coding is the default behavior, without need for File::Spec, Class::Path and such. Once, we have made the step from FILEHANDLES to IO::File. Let's make it go a little further. The discussion is stuck in "filenames", which are a problematic area. But we started with chown and friends. It really would like to be able to write: $file = File.new($filename); $file.owner($user); if $file.owner eq $user {} $file.open() over if($has_POSIX) { chown $filename, $user; if((stat $filename)[4]==getpwuid $user) {} } else { die "Sorry, do not understand your system"; } -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Files, Directories, Resources, Operating Systems
* Aristotle Pagaltzis ([EMAIL PROTECTED]) [081204 16:57]: > * Mark Overmeer <[EMAIL PROTECTED]> [2008-12-04 16:50]: > > * Aristotle Pagaltzis ([EMAIL PROTECTED]) [081204 14:38]: > > > Furthermore, from the point of view of the OS, even treating file > > > names as opaque binary blobs is actually fine! Programs don’t > > > care after all. In fact, no problem shows up until the point > > > where you try to show filenames to a user; that is when the > > > headaches start, not any sooner. > > > > So, they start when > > - you have users pick filenames (with Tk) for a graphical > > applications. You have to know the right codeset to be able > > to display them correctly. > > Yes, but you can afford imperfection because presumably you know > which displayed filename corresponds to which stored octet > sequence, so even if the name displays incorrectly, you still > operate on the right file if the user picks it. With all these different encodings, it is easy to show filenames which are not a little-bit incorrect, but which are unrecognizably corrupted. In the whole debate, it look like there are only two groups of developers involved: the programming language authors and the end-application developers. But do not forget that there are also CPAN library authors and maintainers (my main involvement) When you create a good library, you have to support multiple (unpredicatable) platformas and languages. Each time you say: "oh, just let the end-user figure that out", you add complexity and distribute implementation horrors. Good, generally available libraries are crucial for any language. > > - you have XML-files with meta-data on files which are > > being distributed. (I have a lot of those) > Use URI encoding unless you like a world of pain. You are looking at it from the wrong point of view: Perl is used as a glue language: other people determine what kind of data we have to process. So, also in my case, the content of these XML structures is totally out of my hands: no influence on the definitions at all. I think that is the more common situation. > NTFS seems to say it’s all Unicode and comes back as either > CP1252 or UTF-16 depending on which API you use, so I guess you > could auto-decode those. But FAT is codepage-dependent, and I > don’t know if Windows has a good way of distinguishing when you > are getting what. So Windows seems marginally more consistent > than Unix, but possibly only apparently. (What happens if you zip > a file with random binary garbage for a name on Unix and then > unzip it on Windows?) > > I have no idea what other systems do. Well, the nice thing about File::Spec/Class::Path is that someone did know how those systems work and everyone can benefit from it. So why are you all so hessitating in making each other's life easier? There is no 100% solution, but 0% is even worse! Once upon a time, Perl people where eager for good DWIMming and powerful programming. Nowadays, I see so much fear in our community to attempt simpler/better/other ways of programming. We get a brand new language, with a horribly outdated documentation system and very traditional OS approach. As if everyone prefers to stick to Perl's 22 years and Unixes 39 years old choices, where the world around us saw huge development and change in needs. Are "we" just getting old, grumpy and tired? Where is the new blood to stir us up? - MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Files, Directories, Resources, Operating Systems
* Aristotle Pagaltzis ([EMAIL PROTECTED]) [081208 19:16]: > * Mark Overmeer <[EMAIL PROTECTED]> [2008-12-07 14:20]: > > So why are you all so hessitating in making each other's life > > easier? There is no 100% solution, but 0% is even worse! > > It looks like Python 3000 just tried that. > People are not happy about it: > http://utcc.utoronto.ca/~cks/space/blog/python/OsListdirProblem I thought we were having a serious discussion. We all know that considering all names as Unicode is a stupid presumption. It seems that some bright minds got stuck in a deep recursion about codesets in file- and directory names. A pitty that we do not focus on the general concept of OS abstraction (knowing that some problems are only partially solvable (on the moment)). -- MarkOv ---- Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Files, Directories, Resources, Operating Systems
* Aristotle Pagaltzis ([EMAIL PROTECTED]) [081210 00:06]: > * Mark Overmeer <[EMAIL PROTECTED]> [2008-12-08 21:20]: > > A pitty that we do not focus on the general concept of OS > > abstraction (knowing that some problems are only partially > > solvable (on the moment)). > > Well go on. Explain how you would, f.ex., provide an abstract > API over file ownership and access permissions between Win32 > and Unix? I don’t see such a thing being possible at all: there > are too many differences with pervasive consequences. The most > you can reasonably do (AFAICT) is map Win32-style owner/access > info to a Unix-style API for reading only. (I do not have time today for long emails... paying work to do :-( The short answer: Just like Path::Class or IO::File, I suggest an OO interface. That means that you may share methods between different OSes but it also may not be possible. Within this OO interface, you could design two abstraction levels: one which maps directly to the OS calls, like supports "chown" via some POSIX mix-in. On an other level, we attempt to unify environments. For the latter, you can think of methods like "owner" getter and setter, "os_family" or "size". Even more to my likings is an additional super-level. In this case, the actual platform-dependent implementation does its best... Maybe something like: (still Perl5 style) $file->change_attributes(owner => $user, group => $group, readable => 1, ...); The core implementation tries as good and as bad as it goes to unify various kinds of attributes onto OS specific features, taking care of nastiness like change-order limitations. Typically becoming smarter over time. Real DWIMming, exploiting our joint knowledge and share this. -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: 6PAN idea
ers and presentation from various YAPCs, for instance the relatively global level http://cpan6.org/papers/2008yapceu/ There is a lot of code on my disk (unreleased), and Log::Report and XML::Compile are the most important released general purpose components for the moment. A few more to follow. -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: 6PAN Spec question
* Daniel Ruoso (dan...@ruoso.com) [081218 13:39]: > Em Qui, 2008-12-18 às 13:08 +1100, Timothy S. Nelson escreveu: > > My question is, what sort of information actually belongs in a final > > version of the 6PAN spec? I'm assuming it will at least include 6PAN > > Package > > format (layout, metadata, etc), and I'd suggest that it also include the > > layout of packages on the 6PAN server. > > 2) How source packages are uploaded, indexed, mirrored, searched, > downloaded etc (6PAN/CPAN6) If you understand my explanation of CPAN6, then you certainly must be ware that 6PAN and CPAN6 have nothing to do with each other. Please do not use them in combination. It is as silly as saying TCP/Linux -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: 6PAN Spec question
* jerry gay (jerry@gmail.com) [081218 14:01]: > > If you understand my explanation of CPAN6, then you certainly must be > > ware that 6PAN and CPAN6 have nothing to do with each other. Please do > > not use them in combination. It is as silly as saying TCP/Linux > there's a difference? that was never clear to me (but i haven't read > any docs). at best, they are incredibly, confusingly, similarly named. Well, 6PAN is about downloading the right Perl6 module and installing it. Like Perl5s CPAN.pm module in combination with MakeMaker or Module::Build, interpreting the content of the released packages. It defines how authors have to release Perl6 material so end-user can install and use them. CPAN6 is a distribution network; the PAUSE uploader, CPAN package indexer and CPAN ftp-server network (ftp.cpan.org). CPAN6 can be used to distribute any kind of data: perl5 distributions, 6pan distributions, rpms, word documents, and photo's... as long as there is some meta-data available to distinguish two seperate releases. It is about upload and download with access rules, name and version label restrictions, namespace restrictions, data integrety, distributed search, redundant servers, release-under-embargo, etc etc. In general: CPAN6 implements everything you need to create data collections for any purpose. Kind of very smart network directory. AFAIK, there is no comparible project on the moment. In "CPAN" name is used for different things, so that has caused this inconvenient name look-alike. -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: 6PAN Spec question
* Timothy S. Nelson (wayl...@wayland.id.au) [081220 03:45]: > Btw, looks like I was wrong about the terminology of > CPAN6/6PAN/whatever. See link below for details (the new Terminology > section). > > http://svn.pugscode.org/pugs/docs/Perl6/Spec/S22-package-format.pod I do not understand the relevance of ???A: just abstract away the actual distribution mechanism. FTP or CD is also not mentioned as possible transporters, so why is that for CPAN6 important? For ???B. In the current set-up, you use CPAN.pm to download, and then install. The 'cpan' script is a wrapper around CPAN.pm. CPAN.pm starts the install tool. A more convient structure would be to have an user application (maybe the same 'cpan' script), which calls some download backend to retreive the distribution and then calls the right install tool per distribution. I would say: ???B is "cpan-NG" -- MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Perl 6 pod processor in Perl 5
* Gabor Szabo (szab...@gmail.com) [090212 06:44]: > As an experiment to check how we could reuse CPAN to distribute Perl 6 > packages. > > Not surprisingly neither of them can handle the perl pod. > I contacted both maintainers asking to look into it suggesting > to use Perl6::Perldoc of Damian but it is quite old. The reason for that may be that (after a very long discussion) Damian promissed me to have a serious look at a more powerful documentation systems, like there is in python or Java. And he never found time to look into that. -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: S16 and File::Spec/Cwd/PathTools
* Timothy S. Nelson (wayl...@wayland.id.au) [090217 08:13]: > Should the functionality of File::Spec and Cwd be integrated into > the IO modules? I'm not advocating the interface, but the functionality > might be useful. We had a very long discussion about this subject on this list, last November. And yes, I agree with this closer code relationship between IO objects and operations on them. Starting point: http://www.mail-archive.com/perl6-language@perl.org/msg28566.html -- MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: IO, Trees, and Time/Date
* Timothy S. Nelson (wayl...@wayland.id.au) [090218 03:08]: > On Tue, 17 Feb 2009, Richard Hainsworth wrote: > >Moreover, if perl6 distinguishes between a location and a file, then the > >spec can distinguish between a .children() method that provides a list of > >child locations (viz., sub-directories) and .files(), which provides a > >list of the contents of the location. > > Keep in mind that files and directories are also not the only things > in a filesystem. There are links, devices, pipes, and others to worry > about. Which is why I prefer my solution. A directory contains "entries" (not children). A $directory->list() could return these children. Unless you ask for the type, mtime, or such (hidden stat) it should stay abstract for reasons of performance. On the other hand, you are also not allowed to cache the knowledge without regular verification, because facts may change. The designed interface should focus on what you want to do with the OS, on (as abstract as possible) actions. -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Docstrings in Perl 6
* Damian Conway (dam...@conway.org) [090504 00:03]: > Hinrik ?rn Sigur?sson wrote: > > I've been thinking lately about how Perl 6 might offer functionality > > similar to Python's docstrings. That is, documentation which is tied > > directly to a particular routine, class or module[1]. This is > > something which would is very useful in a REPL, and for documentation > > readers[2]. We had long and fierse fights about it, back in July 2007. Challenged by Damian, I can up with http://www.nntp.perl.org/group/perl.perl6.language/2007/07/msg27894.html And never heard from it since. (I know you are busy) > For the latest S26 proposal that I'm (very quietly) working on, I'm > considering two possible mechanisms to support tying docs to specific > components of a program. > > The first is an C trait: > method reverse ( >Bool $recursive is docs too> > ) > is doc with the order of elements reversed.> > > The second is a generalized Pod comment form: > method reverse #={ Returns copy of L with order of elems reversed. } > ( Bool $recursive #={ reverse nested Ls too } > ) Does this mean that it now will be acceptable to interlink documentation with information from the code? I would certainly appreciate that change. > > One advantage to using Pod blocks in place of actual strings a la > > Python, is that the documentation is still independent of the source > > code, and need not be in the same file. > > That's certainly true of your proposal. However, many might argue that > one *disadvantage* of using Pod blocks plus :name<> that way is that the > documentation is independent of the source code, and need not be in the > same file. ;-) The "=begin METHOD" is the starter of the method implementation (docs are as important to programs as code is), so is a big advantage to read the documented subject on the initial line. Certainly if the documentation gets longer. However, for my taste, the focus is attracted to the wrong components of the line. I would prefer =begin METHOD :name=method reverse Acts on a L. Returns a copy Acts on a LIST. Returns a copy =end METHOD =cut method reverse() { method reverse() { (=end METHOD suggests that the method ends there, but of course, the coding part still has to come. That's confusion) Or, simpler (alternative syntaxes) method reverse() { method reverse() { #= Acts on a LIST ` Acts on a LIST. #= Returns a copy ` Returns a copy. Documentation and comments are alike: Documentation for external module users, Comments for internal module users. Therefore, I think that a '#= ' comes naturally next to the '# ' I suggest a syntax abbreviation from m/^\s*\`/ to "is doc<>" -- MarkOv Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Docstrings in Perl 6
* Hinrik ??rn Sigur??sson (hinrik@gmail.com) [090504 10:45]: > In Perl 5 I hate having Pod interleaved with code, since it is usually > mostly /user/ documentation and so just gets in the way. Having normal > Pod elsewhere (at end-of-file or in a different file) and keeping API > Pod docs inline (and having tools which might merge the two into nice > HTML if desired) would be the best of both world. I also maintain old and often used modules written by experienced programmers, which did have the POD and the end of the file and in seperate files. Transforming that into interleaved documentation highlighted many mistakes: often, over time, the interface gets extended or deprecated. When the documentation is far away from the code, people forget to update it. That is a fact. Besides, user-doc and code-doc have quite some overlap. By interleaving user-doc with the code, you need less code-doc to achieve the same clarity. On the other hand, you also need a global API description, which demonstrates how the various functions/methods work together. So in my personal favorit style, I document the nasty details of each method close to the code of the method, and after all the code I add a long text on all the main subjects of the code, on how to use the module as a whole. In that large text, I do not wish to detail all the options because that frustrates the learning process. -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: New CPAN
* Daniel Carrera (daniel.carr...@theingots.org) [090528 14:18]: > There was some talk on IRC about a new version of CPAN to match the new > version of Perl. In March 2006, Sam Vilain and I started to think about a new CPAN what we named CPAN6. There is a lot of information about the project on http://cpan6.org You can see there that a lot of work has been doen. Core Perl people did really discourage me by continuously debating about the name of the project, instead of helping to implement it. But now other people start to wake up, it might get back on track. > Recap: wayland76 wants to integrate CPAN with the local package manager > (RPM, DEB). He proposed using Software::Package for that (which is > incomplete). Be aware that there are confusing entanglements in the current set-up. Firstly, you have the CPAN archive, which uses "Pause" as entry point and ftp-servers to distribute the accepted "distributions". Secondly, you need local configuration tools to get distributions installed on the right spot. That differs per platform. And sometimes people install for instance under usernames, not system-wide. This is the playfield of CPAN.pm and CPANPLUS. For instance compiling XS, discovering libraries, ... Then, you have the administration on what is installed. Basically, that is what RPM/DEB packages think is interesting. CPAN6 is a possible alternative to the first (the CPAN archive), but improving on the second by allowing advanced (server side) searches. Perl's installation tools have a very limited knowledge... much less than search.cpan.org. > A) Can we add "version", "target" flags to CPAN metadata? I was thinking > that current modules would all be version=1 and target=perl5. CPAN does not have an official concept of version numbers on distributions, but only on the pm files which are inside the distribution. The 02packages list which is used to lookup dependencies converts pm-file $VERSION into distribution names which "accidentally" contain a number. The problem is more serious. Perl6 installation needs to have multiple versions of the same module installed in parallel (and even run within the same program!). So the whole concept of installation has to change into something more modern. > D) In addition to target=perl5 and target=perl6 we could have target=parrot. IMO, we need many more. What you call "target" is actually a namespace. The current CPAN archive has only one target. But with Perl6, we have a need for perl6 distributions, but also the pre-compiled versions of these modules, for PIR, pasm, pieton, and whatever languages we will develop based on Parrot. And... maybe deb and rpm archives which (semi-)automatically provide repackaged versions of modules which arrive in the perl6 archive. CPAN6 can do all that in a rather simple way. > E) With all this done, we can talk about the new CPAN package format. We > can use the "alien" utility (written in Perl) to finish the > Software::Package distribution (which currently only supports RPM). CPAN6's opinion: it distributes releases (Perl5 terminologie: a distribution) which simply is a set of files. During transport, they may get packed together (with f.i. tar) and compressed (with f.i. gzip), but that is unimportant. Whether only a single file is shipped (like a single tar.gz prepared by the author) or many does not matter. So: a minimal installation tool does not need Archive::Tar and a zillion of other core modules in my design. > F) In summary, we have a possible course of action: There is a lot more structurally problematic. Please read one of my papers on the cpan6.org website. I am looking forward to meet people who have read my design documents, and understand that there is more to it than "let's discuss about the project name". There is at least 40k lines of Perl code already waiting to be used for this task. -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: New CPAN
* Daniel Carrera (daniel.carr...@theingots.org) [090528 21:46]: > use Whiteness:from; > So we have to give some thought to how the modules are going to be > stored in the system. Actually, I have made some progress to implement this for Perl5. The problem is that you need parallel trees of the same version. So naturally you get something like: /usr/lib/perl5.10.1/File-Slurp-0.98/File/Slurp.pm /usr/lib/perl5.10.1/File-Slurp-0.99/File/Slurp.pm And put the right File-Slurp-0.9[89] in your INC path. Even: we can just untar the distribution on that spot, run make, and then use the blib tree in the INC path. But what is the right version? One of the problems companies often have, is that they have a stable system with perl applications. Then, they add a new application which requires newer versions of modules which are already in use. By upgrading, they may break their stable code. So, I came up with the idea to have a index file of used module versions per script. It defines which distributions are used for that script. The first time you run the script, it collects all the newest distribution versions and "freezes" that list for all next runs. The list of newest modules, as is used in the initial run of a script, only accepts a new version if the tests succeed and also all the tests from modules which depend on that have been re-run successfully. The index file also solves the performance hazard of scanning huge INC paths, which makes runtime look-up for module locations slow. I have started to implement above as part of CPAN::Site for Perl5, more as feasability study. Not releasable yet. Quite straight forward with an INC code-hook. -- MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: New CPAN
* Daniel Carrera (daniel.carr...@theingots.org) [090528 16:07]: > Mark Overmeer wrote: >> In March 2006, Sam Vilain and I started to think about a new CPAN >> what we named CPAN6. There is a lot of information about the project on >> http://cpan6.org > > I know about CPAN6, thanks. It's come up a couple of times on IRC. > Perhaps you could hang out on the IRC channel so we don't have different > people going off in different directions with CPAN. It's not a discussion like "let's make a change to the current set-up", so IRC doesn't seem to me a good spot to discuss it. I gave various talks on YAPC's and interviewd a few of the CPAN maintainers. Workshops, Hackathons and YAPCs are more suitable. Trying to keep-up with normal email is already a daily struggle. Besides, having 9 hours time-difference with the West Coast doesn't help: when they wake-up at 8, I am just getting my 2 yr old son from Kindergarten back home. And when he finally sleeps, I am tired ;-) >>> A) Can we add "version", "target" flags to CPAN metadata? I was >>> thinking that current modules would all be version=1 and >>> target=perl5. >> >> CPAN does not have an official concept of version numbers on distributions, >> but only on the pm files which are inside the distribution. > > I know, thanks. I was suggesting a change. I had a discussion of two days about "versions" with Sam Vilain. What we came up with, is that there is only one solution: versions need to be strings without meaning. Of course, you can help yourself using some kind of logic sequencing these strings, but everyone has different ideas about how to do that. Who (besides Perl people) expect 5.10 after 5.8? The solution is simple: each release has a unique version string. The release also lists as meta-data what kind of follow-up it is to which existing release. For instance: 5.10.0 diverts from 5.8.8 (upgrade) 5.8.9replaces 5.8.8 (update) 5.10.1-blead diverts from 5.10.0 (devel) 5.10.1 replaces 5.10.0 and 5.10.1-blead (merge) In upgrade paths, replacements are safe upgrades and diversion may break code interfaces, where upgrades need more care (and probably explicit user approval) Which this approach, you can safely create all the weird versioning needs that Perl6 has designed. A picture could be presented to the users, to help making choices: | 5.8.8 | \ | \ 5.8.9 5.10.0 | \ | \ |5.10.1-blead | / 5.10.1 >>> D) In addition to target=perl5 and target=perl6 we could have target=parrot. >> >> IMO, we need many more. What you call "target" is actually a namespace. >> The current CPAN archive has only one target. > > Target, namespace, same difference. It's an identifier to divide modules > into categories. Well... do you need all archives to share one infrastructure? I think it is nicer to have people being able to open-up CPAN-like archives with additional features. For instance, if someone wants to publish pre-compiled PIR versions of modules, then s/he should not need the assistence of the core CPAN. With an URI namespace, you address an archive where you can find the data. A part of that namespace (URI fragment) could be a target within the archive. >>> F) In summary, we have a possible course of action: >> There is a lot more structurally problematic. Please read one of my >> papers on the cpan6.org website. > I have scanned through the first one. It's 30 pages... Oh, that's the small one. -- MarkOv Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: [RFC] CPAN6 requirements analysis
* Alex Elsayed (eternal...@gmail.com) [090528 22:17]: > While lurking in IRC, I've seen several discussions of what CPAN 6 should > look like. I would really like to see a split in terminology being used for the various seperate problems. The traditional confusion about what CPAN is: an archive or an install tool. Package manager discussions are in the process AFTER the install tool: to distribute OS changes to be made. In the messages on the list, I see people merge requirements of these very independent tasks. I think that package managers are not a "CPAN" related problem at all. The Perl install tool decides which files it wants to have within some file-system tree and versioned environment, and then package managers distribute those files and meta-data. Also, there are various different package managers around for Linux distributions, and they tend to be replaced every few years. If you want people to use Perl modules on their Linux systems in a convenient way, you have to distribute each perl module in all of the existing formats. Of course, a tool like "alien" can be used to simplify the task of creating all these flavors. IMO, that discussion should go in the direction of additional services: the CPAN archive distributes what authors publish. The install tools (CPAN.pm/CPANPLUS/successors) make that code fit in specific operating systems. As a service, other people can publish the results of their specific module installation via package-managers to the world, such that those people can use they platform native software management tools. Just like search.cpan.org is an independent additional service on the CPAN archive. > I personally believe that there are a few requirements for a package format > that is sufficient for Perl 6: > * It must enable packaging for both binary- and source-based distros > * It must enable automatic generation of packages for supported systems > (although it may well not be capable of it out of the box) > * It must permit (or preferably help with) attempts to support new systems > * It must be simple to submit packages in the correct format > * It must enable the design and building of an automatic testing system The worst flaws in software design are based on the idea that you can organize the outside world. The Perl community will never be able to push its packaging mechanism into Linux distributions. We may be able to select the ideal packaging mechanism, and then they will wrap that in their own packaging mechanism. IMO, as Perl community, we should focus on the things we are good in: have people contribute free Perl code in a simple and platform independent way. Let package manager specialists do their job! They will NEVER be satisfied by the choices we make for them. To help package builders do their job simple and correctly, we have to supply them with as good metadata as we can. Which means: . as much as possible (sufficient kwantity) . very explicitly defined structure and content (predictable quality) . added with minimal hassle for the (often lazy) authors of the code (usability) . extensible meta-data structure (future grow path by design) To get these things right is already an increadible amount of work... and no-one else does that for us. There are so many improvements in this area to be made in our current set-up already, which no-one picks up. Does anyone like to see my wishlist? See http://cpan6.org/papers/2008yapceu/ In the last example http://cpan6.org/papers/2008yapceu/cache-release.xml I show much more additional meta-data information than currently provided my Meta.YML, but all very usefull for packaging tools and other services. (I have used XML as syntax, because with XML-schemas I can precisely describe the structure and content: quality!, but you can also transmit the data in YAML, JSON, Data::Dumper,... that is just unimportant syntax!!!) One thing to mention: I have separated some meta-data into own releases as well. For instance, in stead of including all kinds of personal information (like email address) inside the distribution, I let people make a release which contains that information just like a software release. The released software only contains a reference to the personal information. That means that people can upgrade their personal information (make a new release of themselves) when the email address changes. Of course, that exact content of "personal information" is not an Perl6 issue, but it is solved in my CPAN6. PACKAGING IS NOT OUR BALLGAME!!! -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: New CPAN
* Daniel Carrera (daniel.carr...@theingots.org) [090529 08:17]: >> Workshops, Hackathons and YAPCs are more suitable. > But those venues are not available on a day-to-day basis. At least, you get the time to discuss it in depth. Some even basic meta- data issues are just too complex for the short size of email, let alone IRC. Will you be at YAPC::EU in Lisbon? Let's BoF. >> I had a discussion of two days about "versions" with Sam Vilain. What >> we came up with, is that there is only one solution: > Where did that discussion happen? Here? Could you send me the link to > the right page on the list archives? He (from NZ) stayed at my place (in NL) for a few days before YAPC::EU 2007 (UK) where we gave a presentation about the subject. The results are in the initial paper page 22-24 http://www.cpan6.org/papers/cpan6-design.pdf A lot of the content of that design paper comes from discussions with many, many people. >> Oh, that's the small one. > In that case, I don't think I have time to read your papers. I'm sorry. > I have other work to do. Reading a paper is much less work than following IRC. When the brain- storm on IRC is over, someone will have to structurize the ideas into pages with comparible size and complexity. > One other problem with a long paper is that it is not a conversation. > It is a one-way communication medium, so it is less likely to build > consensus (unless the paper itself was written through a consensus > process). The paper describes many small components which need to be addressed, with limited detail. Of course, most is open for discussion. Most did have a lot of discussion already before it was written down. IMO, it never happens that things get implemented as described in the initial design. So, there is so much what can be changed when people come up with better plans. But nothing in there is about Perl6 installation tools or Linux Distribution packaging, where most of the discussion seems to be about. The overlap is meta-data definition and data transport. -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: New CPAN
* Daniel Carrera (daniel.carr...@theingots.org) [090529 09:38]: >> He (from NZ) stayed at my place (in NL) for a few days before YAPC::EU >> 2007 (UK) where we gave a presentation about the subject. The results >> are in the initial paper page 22-24 > > The discussion did not happen on this mailing list? Was it just you and > Sam? I was hoping to see the discussion thread (rather than just "this > is what we decided"). Two days of personal discussion is more than just an email thread. And "decided" is too strong: for well-thought reasons it went into the design this way. Until someone has a better idea with good arguments. >> Reading a paper is much less work than following IRC. When the brain- >> storm on IRC is over, someone will have to structurize the ideas into >> pages with comparible size and complexity. > > I'm sorry, but I am not going to read your 30-page paper. Even the > Synopses are not that long, and I can be fairly confident that the > Synopses are the product of community consensus, so at least I have a > good reason to read those. Do not be blind folded by the formatting. A PDF of 30 pages with a few images easily looks larger than a plain text POD file. For instance, S02-bits.pod (the first real Synopsis) has 24078 words on 3790 lines. My plain-text TeX files have 11229 words on 1764... so the whole CPAN6 project is described in less than half the size of one Synopsis chapter. Synopses where written by Larry, of course after discussions but mainly based on his great personal design capabilities. And then polished further and further by the community. I do not see a difference with my approach... other than that there is not yet a community. Congratulated in advance with your marriage, next month. Of course, we can meet at some other moment because you live close-by. Or you can join the long list of people who guess that they understand what the CPAN6 project proposes and comment on that weak assumption. Anyone interested in a CPAN6 hackathon? -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: New CPAN
maintainers will not be needed anymore. Maintainers can focus on the quality of the content. Perl6, Perl5, parrot, pieton, python, PHP, family pictures, business documents, ... And yes, I you really want to make 1 huge archive where everything can be put in, as suggested here, then it is still possible. That is how ftp-servers are organized. -- MarkOv Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: New CPAN
* Daniel Carrera (daniel.carr...@theingots.org) [090529 11:42]: > Timothy S. Nelson wrote: >> While I've no objection to building the end-user software to >> support multiple repositories, I know that there are certain segments >> of the community who are very very keen to keep everything in the one >> repository. > > After reading the Zen of Comprehensive Archive Networks (ZCAN), I think > there is very good reason for retaining the current infrastructure with > the current, large, set of mirrors. That is not to say that we can't > upgrade the packages and metadata. It also says: "Another important credo is: Avoid bottlenecks and interdependencies. Decentralize. Create and encourage alternatives." When reading the ZCAN, be very aware that it discusses CPAN. As users of CPAN, we all know that it works (for the moment). But that should never be a reason the think outside that box of daily practice! When making a step in development, first figure out which components of CPAN are really valuable, which are outdated, and which are missing. Then keep the good, salvage the outdated and fill-in the missing. Just to give you something to think of: with the current speed of networks and price of hardware, you do not need a huge network of daily synchronizing mirrors but a sufficiently large network of very up-to-date mirrors. Preferrably smart mirrors, which can answer (Ajax-like) some simple questions about distributions, such that you do not have to download the immensely growing 02packages.details files anymore. Ftp-mirror sync trees. If you add subtrees on your disk which contain different archives or targets, then these will get mirrored as well. > "CPAN shall not piggyback another language" -- from ZCAN. > Judging from the ZCAN page, I don't expect that uploading Ruby modules > to CPAN will go well, even if that module can be compiled to Parrot. The > ZCAN page gave good reasons for this. Agreed: do not merge sets of unrelated data! Perl6 and Perl5 are unrelated sets of data. The only relation is the people who use it. -- MarkOv Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: [RFC] CPAN6 requirements analysis
* Timothy S. Nelson (wayl...@wayland.id.au) [090529 12:02]: > On Fri, 29 May 2009, Mark Overmeer wrote: > * CPAN6; this is a piece of software for managing an archive network (such as > Pause6, below). This is not specified in this document; see > http://cpan6.org/ Yes. It facilitates organization of archives: exchanging releases (f.i. software distributions), without the knowledge about the internal organization of the archive. This layer ensures that various servers do not need to run the same version of the archiving software: alternative implementations can exist as long as they have a minimal subset of features. It contains things like . protocol negotiation . XML, YAML, Data::Dumper syntax? . packaged with tar, cpio, zip, none? . compressed with gzip, bzip2, none? . transport protocol http, ftp, rsync, .. . release addresses: archive name (URI), product name, version name Quite simple. > * PAUSE6; this is an actual network based on the CPAN6 software (see > above). It also is not documented here. Pause6 is one implementation of archive maintenance software. In the first version written in Perl5, it implements things like . archive maintenance rules . uploaded release inspection . actual storage on disk . trust, licenses, authors . sub-setting and merging archives . mirror synchronization . simple meta-data searches . end-user downloads These are all server-side things: the client code is very small! Client- side, it is more like ftp (downloads are done via ftp, http, etc. The simplest CPAN6 client is this: wget http://cpan6.org/perl5/get?product=XML-LibXML which gives you a copy of the seperate files in the last version of XML-LibXML without packaging or compression via http from a mirror nearby you. Then cd XML-LibXML-0.69/payload/; make install Something like that. So: Pause6 is the logic behind our current Pause upload interface plus all the maintenance tasks from CPAN maintainers. > * CPAN6.pm; this is a piece of software that starts with what it can get > on PAUSE6, and attempts to give you an installed perl module (this is a > replacement for CPANPLUS/cpan2dist) Well, in my current implementation, CPAN6.pm is a faint layer on top of CPAN.pm. It will search a little differently (search on the web or local disk when using a CD, no 02packages.details), but downloads are still managed by LWP. >> I think that package managers are not a "CPAN" related problem at all. >> The Perl install tool decides which files it wants to have within some >> file-system tree and versioned environment, and then package managers >> distribute those files and meta-data. > > The terms you've used here are so general that they could describe any > one of a number of things. I'd like some concretification or something. > Example? CPAN archive and Perl install tools are fields of the software author: his/her attempt to get software from the own disk usable on someone else's system. Package managers are used by the end-user to have control over their system. This is a different problem, each OS(-version) has other solutions. > Have you seen Software::Packager? It's a Perl5 module that's designed > to create a "source package"* for any supported package manager. What I see in the module is two-fold: . some meta-data . some wrapping (tar) On the meta-data side, it is much too UNIX oriented and far too little. It totally lacks any support for platform dependent problems like . which charsets are supported by filesystems . which characters have a special meaning on various OSes, and how do install tools work around encountered problems. (for instance a ':' in a name, unicode package names, ...) . all kinds of important additional meta-data are missing, like license. The Meta.YML is much more complete. . the implementation is very UNIX centric As indicated before, I think the transport wrapper, like tar, is totally unimportant: it should be abstracted as a transport layer, hidden on the application level of installation tools. > Just for the record, I'm interested in having a Software::Packager > port/rewrite be part of the installation tool; overlapping what you're > calling CPAN6.pm. My CPAN6.pm implementation transports a directory of files from the source (author) to the end-user, without knowledge about the content of that directory: whether it is one perl5.tar.gz or a dozen pictures. In parallel, it uses a meta-data file for such release, which contains cpan6, pause6, and release specific meta-data. For Perl5, it will capture all of the Meta.YML info and more. For instance, it will add checksums for all files, and crypto-signatures. > I'd like to suggest a further terminology split: > - 6PAN: command-line install tool The term 6pan has also b
Re: New CPAN
* Nicholas Clark (n...@ccl4.org) [090529 14:07]: > On Fri, May 29, 2009 at 02:43:13PM +0200, Mark Overmeer wrote: > > > "CPAN shall not piggyback another language" -- from ZCAN. > > > Judging from the ZCAN page, I don't expect that uploading Ruby modules > > > to CPAN will go well, even if that module can be compiled to Parrot. The > > > ZCAN page gave good reasons for this. > > > > Agreed: do not merge sets of unrelated data! Perl6 and Perl5 are > > unrelated sets of data. The only relation is the people who use it. > > I disagree. > Strongly. ;-) > CPAN is the Comprehensive Perl Archive Network. > Not the Comprehensive Perl 5 Archive Network. What's in a name. Is it also CPAN is the Comprehensive Parrot Archive Network CPAN is the Comprehensive Pieton Archive Network CPAN is the Comprehensive Pony Archive Network CPAN is the Comprehensive PHPArchive Network CPAN is the Comprehensive PRuby Archive Network So, where do you stop? Perl6 and Perl5 have some things in common, just like PHP and Perl5. Some people say that Perl6 is a different language, not a next generation of Perl5. Do we need to install Perl5 on our system to get access to the install tools to install Perl6 modules? -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: New CPAN
* Daniel Carrera (daniel.carr...@theingots.org) [090529 14:24]: > I think that it would be a good idea to put Perl 5 and Perl 6 modules in > the same CPAN. I have a very cowardous reply on this. My CPAN6 design supports both sub-setting and super-setting archives. So, it can produce three access-points: pure-perl6, pure-perl5 and combined. If they physically share a disk "store", you even do not even need copies of the releases. And... I do not really care what kind of information people keep inside the archive. That is for the founding "board" of the archive to decide. As long as it has meta-data, it is ok to my scope. -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: New CPAN
* Daniel Carrera (daniel.carr...@theingots.org) [090529 14:39]: > Much, actually. As the ZCAN document explains, the set of mirrors are > donated to Perl by various donors who agreed to hold *Perl* modules. > These computers do not belong to us. If the donors agreed to hold Perl > modules, it would be an abuse to use it to upload Ruby and Python > modules as well. Actually, there are various kinds of ftp-servers. Some are huge: they really don't care. Other are small: companies who use Perl and have made some diskspace available. They like to have a copy, and use it also for there own local installations. With Perl6 and other derivates, with the new version needs, the size of the archive will grow with factors. The current CPAN is tiny (fits on your photo-camera) "The big guys" really don't care about diskspace. The smaller may say: I am not using Perl6, so I don't want this 50GB on my ftp server. I would like to host Perl5, not Perl6/Parrot. Should we care to lose some ftp-mirrors? Ah, no, not really. Technology improvements made this a non-issue. One nicely connected ftp-server per continent may very well be sufficient now-adays. >> So, where do you stop? > > You stop at the point where you start breaching your verbal agreement > with the owners of the computers you are using. I don't know: did they agree on Perl or Perl5? That's unclear. And will resolve itself some way or the other when either solution gets implemented. >> Perl6 and Perl5 have some things in common, just like PHP and Perl5. > > Perl 6 is the next version of Perl 5 and Perl 6 comes with a Perl 5 > compatibility mode and Perl 6 is intended to be able to use Perl 5 > modules. That makes Perl 5 different from PHP. The parrot-based PHP implementation will be able to link Perl6 and PHP code, if I understand correctly. And the same for Python. And Lua. and so on. I don't know. I am less convinced about a strong bond than you. Ok, enough stirring-up for today. Let's eat diner. -- MarkOv Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: New CPAN
* Daniel Carrera (daniel.carr...@theingots.org) [090529 19:55]: > Btw, if we do go ahead with this "meta CPAN" idea, it'll be important to > divide the network into self-contained groups. Earlier I used the word > "target". Alternatively we could say "platform". Example platforms could > include: > > Perl 5 <-- Perl 5 source code. > Perl 6 <-- Perl 6 source code. > Parrot <-- Parrot assembly. > Lua<-- Lua source code. > Nokia.bin <-- Compiled binary for the Nokia handheld. > Elf-ia64 <-- Compiled binary in ELF format for the IA64 architecture. > > You get the idea... Mirrors pick which platforms they'll hold. Now we are getting somewhere. A "mirror" doing only a part is a bit strange. A mirror doing half-work is more a filter. So, if we replace your term "platform" into my term "archive", and your "meta CPAN archive" into my "CPAN6", then we are on the same track... I call it "CPAN6 servers invite selected archives on their system", where you like to say "CPAN mirrors copy selected platforms/targets of the main archive" And the next consideration: when we have a piece of software which administers Perl5 or Perl6 or Nokia.bin or Elf. Why stop there? What is the overlap? It is basically all just some blob of data with some associated meta-data to search and retreive the blobs. It is only the client side install tool which looks into the content of the package. Why not allow pure pod releases? A small step to documents in any other format. Why not share holiday pictures? Also just a blob of data with some meta-data. Those final steps changed my world. Where Windows has a "My Pictures" directory, how do we organize "Our Pictures"? Just a hack: Flickr or a shared NFS disk. And "Our Software?": CPAN. And there are document management systems, ftp-servers, linux distributions, ... etc etc... And all facilitate creating collections of a certain type. People are collectors by evolution, for their own or for a group. It is simple to organize a private collection on computers, but for all group collections on computers, we only have hacks. Studying different kinds of collections have led me to a relatively small set of features we need. What will we share, with whom, how do we distribute, where do we store it what does it need additionally. Add some modern needs as security, fail-over and license tracking and see my meta-data design. The surprise is that I have not been able to find any software where you can start collections as general basic infrastructure. On the level of network file-system. A totally new application of Internet. The beauty of our community collection in CPAN can be extended to benefit all computer users. That is where my CPAN6 is about. CPAN to the power 6. Oh, maybe it will also be used for the Perl6 module collection. That's to the Perl people to decide. -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: [RFC] CPAN6 requirements analysis
* Timothy S. Nelson (wayl...@wayland.id.au) [090530 02:15]: >>> * PAUSE6; this is an actual network based on the CPAN6 software (see >>> above). It also is not documented here. >> Pause6 is one implementation of archive maintenance software. In the >> first version written in Perl5, it implements things like > > I'm still not sure we're on the same page. Let me give an example: > - Perl6 is a specification > - Rakudo is an implementation of that specification > - www.example.com runs a copy of Rakudo (to serve their site) > > So we have: > - Specification > - Implementation > - Installation > > Which of those is CPAN6? Which is Pause6? Your use of the word > "implementation" is confusing me; I'm saying Pause6 is the installation > (accessible to the world). Is that what you're saying? CPAN6 is like IP: routing and exchanging packages. Quite small and stupid. Pause6 is like TCP: knows how to get the right packages. Install tools are like HTTP: understand the content of packages. You should avoid ever changing the IP level. But next to TCP, you have UDP and so on, for different package handling needs. (On that level, I define connections to external archive-like applications, like OpenID) So: CPAN6 has a spec and an implementation Pause6 has a spec and an implementation Install tools may get a spec and implementation (part of this discussion thread, but not of my project) CPAN6 will run on all servers with this group-sharing concept. It enables playing with large number of archives and mirrors: required to move from the solo namespace of current CPAN into multiple namespaces. Pause6 may be replaced eventually, gradulary, with other implementations. One CPAN6 server can run different implementations and configurations in parallel. Install tools for Perl will be installed where people run Perl and use an archive which contains Perl. Of course, other kinds of applications can use the archive data. -- MarkOv Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: New CPAN
* Andrew Whitworth (wknight8...@gmail.com) [090530 00:24]: > I agree. Doing one thing well is so much better for everybody then > doing a million things poorly. An assorted "blob of data" repository > is far less valuable to the Perl5, Perl6, and Parrot communities then > a dedicated library repository is. Why? Ever heart of extensible meta-data. Can be done in two ways. The idea is tha, on the three levels of implementation (see recent email in other thread), you have some meta-data. 1 On the CPAN6 level (transport layer) you have uniqueness information: give the release a name: . source archive URI or target name . product or package name . version and some standard data, like size and data of upload. 2 On the Pause6 level, you have adminstrative facts: . validation . access . searching, etc 3 On the application level (install tools etc) You may also need some facts. So: the core of the CPAN6 design is meta-data which accompany blocks. But I do not say that the Pause6 administration only accepts meta-data for level 1 and 2. It also transports meta-data on level 3. It does not UNDERSTAND that meta-data, but it does provide that meta-data. Ignoring information does not make the archives implementation harder. The ONLY difference between a "specialized" implementation of an archive and a flexible one like I propose, is that in the latter you are forced to clearly assign meta-data facts to one of these three levels. But there is no limitiation in the amount and content of the meta-data you can collect. Well, so your worries are unjustified. And the two simple solutions as I promissed in the first line of my comment: 1 Add three seperate meta-data fragments to one blob 2 Create one meta-data file which contains all three components. As simple as that. -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: [RFC] CPAN6 requirements analysis
* Timothy S. Nelson (wayl...@wayland.id.au) [090530 03:11]: > On Fri, 29 May 2009, Alex Elsayed wrote: >> Instead, it would go to the distributions, who are already well-prepared to >> handle packaging. We'd just be providing the tools and material they need to >> do so. > > Let me reiterate that, while I'd love to have the complete CPAN > available pre-packaged for my distro, it doesn't seem like that's > necessarily going to happen. Certainly not if you need recent releases of modules, because someone fixed a bug for you. It can easily take a year before a distribution picks that up. > Thus, what I want is a tool that will allow me to download the source, > turn it into a package for my distro, and install it. This is why I > want the Software::Packager solution. Empower the user :). But that module does not contain anything which is required to produce packages for Linux distributions. Well, ok, I should say: it does not handle any of the complex things. Yesterday, I drew a picture of how I see services to happen. Having an automated service which is specialized in building debs and such. Now, such tools do exist, but are usually not published by the distribution makers. Maybe, you can collect those (and unify them) Could be far more complex than you wish for. -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: CPAN -- moving forward
* Daniel Carrera (daniel.carr...@theingots.org) [090530 20:54]: > 3) A high-level install tool, analogous to yum or apt, that uses the > CPAN network and resolves dependencies. > Mark O. is most interested in (3). These are all things which I do *not* play in the layers I want to build. Although I do have an opinion on these subjects (if you know me: I have opinions on everything) but they are all not included in my design. Wrong abstraction layer. A pity you didn't want to read the paper. My Archiver implementation defines ways how to specify the meta-data for dependencies, because that is a common need, but does not resolve dependencies. It provide sufficient means of search so you can investigate the dependency details before you start downloading releases, but in general, there are nasty application dependent interpretations in these details. Interpreting dependency information sometimes requires human intervension. For instance: do you want to upgrade to a higher version which breaks api (and triggers the upgrade of a dozen other modules on your system) or stay in the maintenance track of the older api. (example is Mail:: SpamAssassin 2 vs 3) These problems cannot be resolved server-side. Apt or yum are just like CPAN install tools. Nothing more. So outside my personal scope. (But important to Perl6) Please clarify in your spec what you mean with "package" in a dependency. Is it about a single distributed version (what I call a "release" into the archive), or a release with a certain product name, or with a certain tag. Or any of those: how would you specify that? And how would you denote ranges of conflicting packages? Well, maybe start with putting that in your wishlist. And conflicting/dependencies in licenses? How would you resolve dependency conflicts? Design human intervention in this process of dependency processing. How do you handle external dependencies to Linux distribution components like database software, which change name with each new release of each linux distribution? Will we lockin Perl to a few well known distributions, or do week keep our trandition open mind. So: please pick from http://lwn.net/Distributions/ (currently 548 distros listed) which you want to support... and wait for other people to add more. Perl is one of the last pure English open source projects. When you install a localized KDE or Gnome *everything* (>95%) gets translated. But not Perl! You must include description and keywords in various languages. Can other people contribute translations? How will they add those to CPAN? The Meta.YML specs are already further than what you propose. Go to a Perl QA Hackathon and see how hard people work to get Meta.YML extended! (last year Oslo, last March in Birmingham) Reuse that. > I propose that we work on (1) and (2) first and do (3) later. (3) is > quite independent of (1) and (2). Mark might want to check that the > package format includes enough meta-data to not conflict with is ideas. There is no way that it can conflict, as long as it is well defined. Have a look at the Kwalify module, or use the horribly (but more widely known) XML Schema's to make it well-defined. My very extended meta-data design probably gives you enough to shop for your wishlist. But I may lack a few things and can add those easily on this stage of the project. Or you can put those in the application specific meta-data block which Pause6/CPAN6 supports. (Which is a pity: unification of meta- data has many advantanges) As examples of how you could adapt your design to my plans: . you do not need to organize file signatures, because those are (far more flexible) already provided on the transport layer of CPAN6. (where it fits logically) . you do not need any packing or unpacking of trees, because that is also part of my transport layer: sender and receiver decide whether they pack temporarily or not. . I offer nice standard solutions for identity management (authors) and licenses. But these are probably automatically translatable. I will keep an eye on your growing list of meta-data needs, and see whether there appear things which are not covered in my specs yet. We need more and better meta-data, not a reinvented wheel. -- MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Module naming conventions
> Currently in CPAN you have modules like: > Digest::MD5 > Digest::SHA > Digest::MD5::Perl > Digest::SHA::PurePerl > The difference is that the first two are implemented in C and the later > two in Perl. This is comparible to adding a "target" to each of the modules, a suggestion when you started this long thread. Either in the name or with the name, we need some kind of separation when you want to leave everything in one archive. With the idea of "target", I think that the intention was to have them added to distributions. But above seems to me applied to pm files. Or is that not your intention? Currently, these problems are resolved in the Makefile.PL: some modules try to autodetect whether to install a binary or a pure perl implementation based on the availability of a C compile. That isn't simple. The main problem with this kind of solutions, is that you do not know beforehand which tags you need to distinguish. For instance, you may also want to release a different version for VMS (so need an OS component), for small systems, and so forth. Multiple classes of tags/targets/keywords? There are at least 2 different perspectives: . the author must be able to provide such special selection criterium in the name to avoid name-space collissions. . the user needs to know which selections there are: which possible values s/he can expect and in which combinations. An alternative solution is: . do not standardize on name extensions: simply have the author choose one. For instance a version number with his PauseID added. . have the install tools decide the best option: it should be able to search in the meta-data of the distribution for the optimal match with the platform it is on. The install tools need to be a bit smarter in this respect that what we currently use. For instance, a dependency could be: requires: Some-Module version 2.*-DANIEL (very explicit) requires: Some-Module version * (easy for user, best fit) There are (in any solution) various complications: . how do we know which versions are compatible? Other people can release incompatible modules in the same product name. . new releases by the same autor sometimes break the interface as well, with a major version number increment. . if the user does not specify an explicit version, which one to take? Who decides the preference? . will Perl6 version on PM files, like perl5, or on distributions (like all other projects do)? . how does the client get the info to do the best match? The 02packages.details file is already far too large. Server-side? A few of these problems need to be tackled in the versioning scheme, some are automatable, and some may require user intervention at install time. Some new balance between them, Perl5 is very simple compared to the need requirements. -- MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: More flexible POD
simple /^\s*\`\s+/, but can think of many alternatives. The link mentioned before refers to a more details spec with examples to be able to create the DocTree. -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Gripes about Pod6 (S26)
* John Gabriele (jmg3...@gmail.com) [100209 14:31]: > [Markdown]: http://daringfireball.net/projects/markdown/ > [Pandoc]: http://johnmacfarlane.net/pandoc/ > [reST]: http://docutils.sourceforge.net/rst.html Or, more Perl like: [OODoc] http://perl.overmeer.net/oodoc/ http://perl.overmeer.net/oodoc/html/jump.cgi?OODoc_Parser_Markov&62 -- MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Progressively Overhauling Documentation
* Juerd ([EMAIL PROTECTED]) [040823 19:46]: > David Green skribis 2004-08-23 11:30 (-0600): > > One of the selling features (or one of the features that is always sold) > > of POD is that you can mix it with your code. Except nobody does, at > > least I can't recall that last time I saw a module that did that, and I > > don't think I've ever really done that myself. The POD usually sits in > > a lump at the end of the file. Oops, someone starts the holy war (again). Wether you put the docs in begin or end of the file, or intermixed with the code has a lot to do with your personal background. > I also think POD should be overhauled completely. I've been thinking > about proposing something like: > sub foo ( > ) description { > Calculates the foo-intersection of $bar and $xyzzy, optionally > blah blah blah... > } returns Array | undef { > # real code here > } I found a few suggestions on the list, but... it's oversimplifying things. Let's state 1) Code is doing things 2) Documentation explains how to use the code (some people explain the code itself, but that is where the '#' comments are for) To be able to write code easily, it should be compact enough to get a good overview. Intermixing code and comment is a bad idea. For instance in my programs, docs are about 5 times the volume of the code which would be ugly in the proposed solution. I prefer to put documentation just before the code. That's the only way my sloppy mind can preoduce docs which stay in sync with the intention of the code. Certainly in larger programs --and always in OO programs-- the functions methods in a user's document can be organized better than the order of the code. So: why not support that? What do you need to be able to do that? In my case, I had to document my set of distributions. It's 125 packages on the moment, all in OO with up to 4 levels of hierarchy. In total 988 methods, 174 diagnics explained (errors and warnings) and 242 examples included. Do you think that you can document this in POD? The thing which POD lacks the most, is that it is visual markup (like B and I in HTML) not logical markup (like STRONG and EM). So, I added logical markup markers, like =method =overload =function. Then, =error and =warning clears that up, and =example starts some docs. =option, =default, =requires describe the OPTIONS A full method in my code looks like this: =method showSomeText TEXT, FORMAT, OPTIONS Print the TEXT in the specified FORMAT. =option linewidth INTEGER =default linewidth 70 The maximum number of characters. =requires language STRING ISO indication of the languages =example $obj->showSomeText('hoi!', '%s', language => 'nl') =error language required The language is not use on the moment, but still must be specified. =cut sub showSomeText { my ($self, $text, $format, %option) = @_; my $lw = $option{linewidth} || 70; my $lang = $option{language} or die "language required"; printf $format, $text; } The number of options tend to grow when the program gets bigger. Especially the constructors (like new()) can have many. On some levels of hierarchy you may want to add new options, or change their default, or fixate it. OODoc is the module which processes the MANIFEST file of one or more modules to produce real documentation out of this. POD is a simplification, but there is also enough info to produce real HTML (which optimal hyperlinks). With HTML, you can create templates to organize the info in the pages (the OODoc distribution come with examples for very small and for a huge setup). The documentation fragments are split, joined, grouped, sorted, formatted and reformatted in various ways to produce the optimal result for users of the module. As example, the HTML output for MailBox (and related): http://perl.overmeer.net/mailbox/html/ Hey! Before you say anything: it is very difficult to give any form of documentation on close to 1000 methods! Verbosity on various spots is configurable in the templates. TO MAKE MY POINT I do not want to turn people over into using OODoc, that's not the point. But I would really like Perl6 to have a logical markup (which may or may not have code syntax, like description{} introduced above). We (I) need more than only a discription of some options and the method as a whole. Let's try to get an integrated solution, including the defaults, diagnistics, and examples. IMO, user documentation should stay outside the main part of the code or discription, to keep the code readible. Any nice solutions for Perl6? -- MarkOv drs Mark A.C.J. OvermeerMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
Re: [S29] uniq
* Ingo Blechschmidt ([EMAIL PROTECTED]) [050519 16:52]: > Should it be possible to give an own comparator block, similar as with > grep? E.g. > uniq ; # > > uniq:{ abs $^a == abs $^b } 42, 23, -23, 23, 42 > # 42, 23, 42 'uniq' differs from 'sort' because there is no order relationship between the elements. A quick algorithm for finding the unique elements in perl5 is sub uniq(@) { my %h = map { ($_ => 1) } @elements; keys %h; } or sub uniq(@) { my %h; $h{$_}++ for @elements; keys %h; } anyway: O(n), which is much better than sort can do. For your proposal, the requested code only requires this uniq:{ abs $^a } 42, 23, -23, 23, 42, 42, 23, 42 uniq:{lc} @words -- MarkOv drs Mark A.C.J. OvermeerMARKOV Solutions [EMAIL PROTECTED] [EMAIL PROTECTED] http://Mark.Overmeer.net http://solutions.overmeer.net
To :D or not to :D
Hi all, Liz and Tux demonstrate powerful Perl6 code at each monthly meeting of our Mongers in Amsterdam. Looking at their examples, I collected a few questions of which I want to discuss the first one in this thread. When I look at Merijns (Tux') code, I see a huge number of :D attributes. https://github.com/Tux/CSV/blob/master/lib/Text/CSV.pm Close to all scalar positional parameter (51x) carry the :D flag. I count only 3 where the parameter does accept undef and the method is able to handle it. I count another 3 where the :D is missing, but the method is not able the handle it. The same for examples Liz shows us in our core code: most scalar positional parameters have :D. Writing a sub which is able to handle undef is usually more work than implementing "I expect sane values for all of the parameters". Questions: . are they using :D correctly? . the simpelest code does not handle the undef case... but now needs the more complex prototype to be correct. . it feels like the wrong default: usually you have to do something extra for the unusual cases, not the 90%+ usual cases. . :D looks really ugly, don't you think? Try to explain to students to add this smiley everywhere. Can someone explain this to me? (Or point me to the correct place) -- Thanks in advance MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: To :D or not to :D
* Moritz Lenz (mor...@faui2k3.org) [151012 15:32]: > > . are they using :D correctly? > > Yes, though not everybody uses :D as much as they do. Do you check that > all the parameters that your Perl 5 methods/subs receive are defined? If > not, you wouldn't use :D in Perl 6 either. In Perl5, you get slower code when you test for definedness... in Perl6 you get faster (better optimized) code. That's a big difference. > FWIW you can now (as of a few days ago) control the default with > use invocant :D; How can de invocant not be defined? > use parameters :D; The new "use warnings"/"use strict"... > which means all those :D annotations can go away, and you have to use :_ > explicitly if you want to allow all. Oh, use :U for that. Ehhh already in use. > That said, I agree that it's the wrong default. And the design documents > even mandate a default to :D, though at the time it was written, it > wasn't clear how to switch off that default, nor how to avoid having to > write > > method new(MyClassHere:U: *@args) { ... } > > in the constructor, which would be quite hostile to newbies. It's still > not clear to me how to avoid that. It is also unclear to me what this means. It is a method which requires and undef parameter? > And I don't know if we can change it now without creating a huge havoc > in the existing ecosystem. There shouldn't be a problem making :D a superfluous option. Of swiftly add "use parameters $_;" to all modules. And there still quite a number of other crucial changes going in anyway... > Another concern is that if "everything" defaults to :D, then classes > (and other type objects) aren't really first class objects anymore, > which is a really neat thing to have. Can you give me an example? Many other languages are capable to live without undef and have first class type objects. In the old days, I had to implement the Algol68 style during compile construction course ;-) > > . :D looks really ugly, don't you think? Try to explain to students > > to add this smiley everywhere. > > It's not uglier than a 'die "Must be defined" unless defined $x' Much too expensive in Perl5. -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: To :D or not to :D
* Patrick R. Michaud (pmich...@pobox.com) [151012 20:25]: > > > method new(MyClassHere:U: *@args) { ... } > Keep in mind that what Perl 6 calls a "type object" isn't quite the > same as class objects in other languages -- a Perl 6 typename is > really an undefined instance of a class. In other words, the > identifiers C, C, C etc. refer to instances of > those classes just like the literals C<3>, C<4/5>, and C<[1,2,3]> are > instances of those classes. They share the same method spaces. Yes, that what I started realizing when I saw all the pain Perl6 goes to ignore the existence of a real "undef" in the language. (I follow Perl6 from a short distance since its start, read all original designs, but this penny did not drop before, sorry) The reasoning behind the output of my $a; $a.say; $a = 1; $a.saybeing (Any) \n 1 \n Which actually means "unstantiated (Any)" versus "instantiated (Int) value=1" for me personally painfully imbalanced. my Car $a; my Car $b; Now I have two different types. Eh, wait they are both Cars. No, not the same because types are real objects. Or maybe a bit the same, because $a===$b is made to work. Ehhh... Confused. Is there any precedence in a succesful programming language where types and values get mixed-up this way? There must be a hidden advantange, which I do not see. Of course, syntactically this works, but Why? For me, a "Car" is not "a vehicle which is not yet there", but is a featural and functional template of a thing. The template describes the interface of a car, not the instance of a car. A type is scientific, an object is dull facts. Is that an old-fashioned, traditional idea to be abandoned? -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: To :D or not to :D
* Moritz Lenz (mor...@faui2k3.org) [151013 07:18]: > >In Perl5, you get slower code when you test for definedness... in Perl6 > >you get faster (better optimized) code. That's a big difference. > > Do you? Did you actually measure that? For Perl6? Well, Liz tells us that it can be optimized better (you limit the range of choices) Whether this is already the case does not really matter here. And at least it produces the error at the side of the caller, not inside your function. > >How can de invocant not be defined? > > Well, if you call a constructor, you call it on the type object. Hence the > type object is the class, and not defined. Yes, that's the other thread which emerged from my :D question. When you call .new, the type is well defined (maybe Any). Trickery links that type to some missing value "undef". And then the programmer is expected to understand that :U actually means "it's a class method". I would be able to understand it better if that whole type==undef stayed hidden from the story told to the programmer. For instance, by calling this flag :T (from Type) The freed up flag :U may say "non-defined for positional allowed", and :D can become the default. > >There shouldn't be a problem making :D a superfluous option. Of swiftly > >add "use parameters $_;" to all modules. > > Why shouldn't this be a problem? If you make :D the default behavior, then adding "use parameters $_;" to existing code would avoid the urgency to scan through all code to see what needs to be changed. > >>> . :D looks really ugly, don't you think? Try to explain to students > >>> to add this smiley everywhere. > >> > >>It's not uglier than a 'die "Must be defined" unless defined $x' > >Much too expensive in Perl5. > > Then don't do in Perl 6 either. If you can argue away the need for safety > based on the need for performance, you can also argue away the need for > safety based on the need for cleaner code. Remember that I only ask for :D to be the default. That's an generic request which does not depend on my personal programming style. For my personal Perl5 code, no: I do not check the definedness of all parameters because it is a lot of work to code and slows-down all subs. I have the intention to use the definedness checks on all parameters in Perl6 because the brievety of ":D" gives me the impression that it is fast, produces a standardized clean message, and easy to use. So, my code would look like Tux's work: cluttered with :D Within a code block, variables are often temporary undefined. But as positional parameters of functions that's rarely useful. For named parameters, the situation is different. In their case, you want to choose between: "undef value means missing parameter", "undef value means illegal value" (:D), "undef value is allowed". For my coding style, the first case is usually true. I am not sure, but probably Perl6 makes do difference between missing existence and undefinedness of named parameters... where hashes and XML (minOccurs=0/nillable) do. So probably merged to "undef value is allowed". Moritz, Patrick, thanks for your patience and detailed answers! -- Regards, MarkOv Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Backwards compatibility and release 1.0
* Moritz Lenz (mor...@faui2k3.org) [151014 09:54]: > In Practice, there's a small number of people who try to update modules to > match when the compiler changed. Most module authors don't hang out in > #perl6, eager to update their modules to the lastest rakudo change. With the relatively small number of P6 modules out there, and the enormous impact on the number of bug-reports we will see once Perl6 is out, it may be useful to pull everything "in", into a central archive where a number of people can change everything. Of course, it is not an ideal situation. There are no ideal solutions. Like it is done in the Linux kernel: once your driver gets accepted, any change in core which affects the driver will be corrected by the one who caused the change. We need that ability for Perl6 syntax, I think. -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: To :D or not to :D
Hi Patrick, thank you for your thoughts. I needed a bit more time for the response ;-) * Patrick R. Michaud (pmich...@pobox.com) [151013 01:05]: > On Tue, Oct 13, 2015 at 12:32:01AM +0200, Mark Overmeer wrote: > > Yes, that what I started realizing when I saw all the pain Perl6 goes to > > ignore the existence of a real "undef" in the language. (I follow Perl6 > > from a short distance since its start, read all original designs, but > > this penny did not drop before, sorry) > > Actually, it's the other way around. Perl 6 is not trying to ignore > the existence of an "undef", it's recognizing that there are in fact > many different kinds of undef. It's the languages that think there's > only one "real undef" that are confused. :) Yes, I like the typed undef... as real undef. Let's call it "Car:undef" when it fits in a Car typed variable which is not used yet. Easy to understand for Perl5 people. > > my Car $a; my Car $b; Now I have two different types. > > Not really, you have two variables, both of which have been > initialized to the same (undefined) Car object. It's not > much different than > >my Int $a; my Int $b; $a = $b = 1; > > where $a and $b end up being variables that contain the same > Int object. This is a different view on the syntax layer: implementor vs programmer. As programmer, knowing of optimizations of this kind adds to the confusion, not to the understanding. As long as my Int $a; my Int $b; $a = $b = 1; $b = 2; $a.say, $b.say produces 1\n2\n, I'm fine with any implementation. I don't want to know how it is done internally. > I think Self would be the (pardon the pun) prototypical example, and > it's also a common feature of JavaScript. As to whether there is > precedence in other "successful" programming languages I use it in JavaScript. I reread a few tutorials about OO/prototype in Javascript yesterday. They smoothly speak about adding methods to objects etc. They speak about types and objects in a very natural way. They don't speak about undef objects to contain types, which are used as object factory. I have not found the need for :U there. > Perl 6 still has classes and instances, but Perl 6 also adds the > ability to represent undefined instances of a type (or, rephrased, > to have typed "undefs"). Perl 6 then binds the commonly-used > identifier (e.g., C) to an undefined instance rather than the > class object that defines the instances. Once you accept that model, > it seems to work out very well in practice. Trying to reformulate this without spoilers of internal details: Perl6 has classes, instances, and prototypes. Commonly used types, like Int, are prototypes based on classes. Therefore, they can easily absorb features via Roles and Mixins. Note about the internals: we are not keeping a separate administration of type features, but keep that inside the instantiated objects. As all prototype-based languages do. Acceptable? > If Research refers instead to the class definition, then the > method namespaces for instances and classes begin to clash, and > we have to introduce all sorts of disambiguation for that. Yes, disambiguation either via a keyword (class method/instance method) or via the invocant (being a type :T or instance... multi methods to the rescue) For me, prototypes in JavaScript work well, because all graphical elements on the screen are slightly different in behavior. I am a bit afraid to get into swamps when programs are large, where it is already hard to keep track on all the cleanly organized classes. So... my hessitation seems to be limited to the explanation and name of the flag :U, which tell me that a type is an undef. Type defining features are loaded into a base object which does not carry a value: yes. Via specialization you get objects which carry a value. Maybe this comes back to the distiction between implicit and explicit undefinedness. -- MarkOv Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Re: Backwards compatibility and release 1.0
* Elizabeth Mattijsen (l...@dijkmat.nl) [151015 10:43]: > FWIW, I’m with FROGGS on this. > use variables :D; In the first response to this message, Moritz spoke about use invocant :D; and use parameters :D; Three different things? > at the top of the scope of your code, and then you’re set. I admit > it feels a *little* like the “use strict” boilerplate of Perl 5. It is. > On the other hand, I think by just specifying a type *without* smiley, > is already so much better than the situation in Perl 5 that the lacking > strictness of :D will not be needed much to catch programming / garbage > in type of errors anyway. Much better, of course. Programming languages are used by people of different taste. Some may find "much better" enough, other people want more. -- Regards, MarkOv ---- Mark Overmeer MScMARKOV Solutions m...@overmeer.net soluti...@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net