This and other RFCs are available on the web at http://dev.perl.org/rfc/ =head1 TITLE Replace C<format> built-in with C<format> function =head1 VERSION Maintainer: Damian Conway <[EMAIL PROTECTED]> Date: 15 Sep 2000 Last Modified: 20 Sep 2000 Mailing List: [EMAIL PROTECTED] Number: 230 Version: 3 Status: Frozen =head1 ABSTRACT This RFC proposes that Perl's existing C<format> mechanism be replaced with a standard module based on parts of the Text::Autoformat module. =head1 DESCRIPTION I can never remember how formats work. The specification syntax is confusing to me. And I usually don't want the formatted text going straight down some magical output stream. It all came to a head when I was building Text::Autoformat. The smart text recognition was easy -- trying to do the final formatting with C<formline> and $^A was just too painful. So I created the C<Text::Autoformat::form> subroutine. It uses a template specification than fits my brain better, it's deeply DWIMical, it's highly configurable, and it's re-entrant (so you can use a C<form> to format another C<form>'s headers and footers, for example). I propose that the existing C<format> mechanism be removed from Perl 6 and be replaced with an add-in function from a standard module (based on the semantics of C<Text::Autoformat::form>) as described in the following sections. =head2 The C<Format::format> function The function itself would be called C<format> and would be imported with a C<use Format> call. It takes a format (or "picture" or "template") string followed by one or more replacement values. It then interpolates those values into each picture string, and either: =over 4 =item * returns the result as a single multi-line string (in a scalar context) =item * returns the result as a list of single-line strings (in a list context) =back In a void context, format would emit a warning: "Useless use of format in void context". A picture string consists of sequences of the following characters: =over 4 =item <<<<<<<< Left-justified field indicator. A series of two or more sequential <'s specify a left-justified field to be filled by a subsequent value. =item >>>>>>>> Right-justified field indicator. A series of two or more sequential >'s specify a right-justified field to be filled by a subsequent value. =item ^^^^^^^^ Centre-justified field indicator. A series of two or more sequential ^'s specify a centred field to be filled by a subsequent value. =item >>>.<<<< A numerically formatted field with the specified number of digits to either side of the decimal place. See L<Numerical formatting> below. =item [[[[[[[[ Left-justified block field indicator. Just like a <<<<<<<<< field, except it repeats as required on subsequent lines. See below. =item ]]]]]]]] Right-justified block field indicator. Just like a >>>>>>>> field, except it repeats as required on subsequent lines. See below. =item |||||||| Centre-justified block field indicator. Just like a ^^^^^^^^ field, except it repeats as required on subsequent lines. See below. =item ]]].[[[[ A numerically formatted block field with the specified number of digits to either side of the decimal place. Just like a >>>.<<<< field, except it repeats as required on subsequent lines. See below. =item ~ A single character field. Interpolates a single character from a subsequent data value. Repeats on subsequent lines as required. =item \0 A zero-width field separator. A null character in the template field is ignored (i.e. I<not> included in the formatted text). This allows two fields to be adjacent. For example: "[[[[[[[[[[[\0[[[[[[[[[[[". To put a literal "\0" in a formatted text, use a '~' field and pass it the data string "\0". =item \ Literal escape of next character (e.g. C<\~> is formatted as a literal '~', not a one character wide field). =item Any other character That literal character. =back Hence a typical use of C<format> might look like this: $formatted = format "<[[[[[[[[[[[[[[[[> >>>>>>>>>>>>>>>>", $aphorism, "page $page_num"; and might produce an output like this: <Like a camel > page 123 <through the eye > <of a needle, so > <are the days of > <our lives > Note that, because every field (except a C<~> field) must be at least two characters wide, the single C<E<lt>> and C<E<gt>> brackets in the format string are correctly interpreted as literals. =head3 Multi-line filling As the previous example indicates, any line with a block field continues interpolation of that field on subsequent lines until all block fields in the format have consumed all their data. Non-block fields on these lines are replaced by the appropriate number of spaces. For example: $title = "On The Evil That Is Spam"; $text1 = "How many times have you longed to smash..."; $text2 = "...the bedevilment that is spam?"; print format "<<<<<<<<<<< [[[[[[[[[[[[[[[ [[[[[[[[[[", $title, $text1, $text2; # prints: # # On The Evil How many times ...the be- # have you longed devilment # to smash... that is # spam =head3 Hyphenation and line-breaking As the previous example indicates, within a block field, words are wrapped whole, unless they will not fit into the field at all, in which case they are broken and hyphenated. Simple hyphenation is used (i.e. break at the I<N-1>th character and insert a '-'), unless a suitable alternative subroutine is specified instead. Words will not be broken if the break would leave less than 2 characters of the word on the current line. This minimum can be varied by setting the 'minbreak' option to some positive numeric value. This value indicates the minumum total broken characters (I<including> hyphens) that must be left on the current line. Alternative word breaking subroutines can be specified using the "break" option in a configuration hash. For example: print format { break => \&my_line_breaker }, $template, @data; C<format> expects any user-defined line-breaking subroutine to take three arguments (the string to be broken, the maximum permissible length of the initial section, and the total width of the field being filled). The subroutine must return a list of two strings: the initial (broken) section of the word, and the remainder of the string respectively). For example: sub tilde_break = sub($$$) { (substr($_[0],0,$_[1]-1).'~', substr($_[0],$_[1]-1)); } print format { break => \&tilde_break }, $template, @data; This causes '~' to be used as the hyphenation character. The standard Format.pm module provides two exportable functions to simplify the use of variant hyphenation schemes. The exportable function C<Format::break_with> takes a single string argument and returns a reference to a sub which hyphenates with that string. Hence the previous example could be rewritten: use Format qw( break_wrap ); print format { break => break_with('~') }, $template, @data; The exportable function C<Format::break_TeX> takes an optional argument specifying a TeX hyphenation file and returns a reference to a sub which hyphenates using Jan Pazdziora's TeX::Hyphen module (assuming it is installed). For example: use Format qw( break_TeX ); print format { break => break_TeX("lithuanian.hy") } $template, @data; Note that in the previous examples there is no leading '\&' before C<break_with> or C<break_TeX>, since each is being directly I<called>, arther than referred to. They each return a reference to some other suitable word-breaking subroutine. Specifying C<{break => undef}> reverts C<format> to its default line-breaking behaviour (see L<A note about C<format> options> and L<Lexically permanent options> below). The Format.pm module might also provide other predefined breaking subroutines; for example, to cope with localization and two-byte character issues such as those addressed by existing modifications to the Perl 5 C<format> mechanism in JPerl. =head3 Controlling line filling C<format> preserves the original whitespace (including newlines) from an interpolated string, unless called with certain options. The "squeeze" option (when specified with a true value) causes any interpolated sequence of spaces and/or tabs (but not newlines) to be replaced with a single space. The "fill" option independently causes newlines to be "squeezed". Hence: $frmt = "# [[[[[[[[[[[[[[[[[[[[["; $data = "h e\t \tl lo\nworld\t\t\t\t\t"; print format $frmt, $data; # h e l lo # world print format {squeeze=>1}, $frmt, $data; # h e l lo # world print format {fill=>1}, $frmt, $data; # h e l lo world print format {squeeze=>1, fill=>1}, $frmt, $data; # h e l lo world Whether or not filling or squeezing is in effect, C<format> can also be directed to trim any extra whitespace from the I<end> of each line it formats, using the "trim" option. If this option is specified with a true value, every line returned by C<format> will automatically have the substitution C<s/[ \t]+$//gm> applied to it. Hence: print length format "[[[[[[[[[[", "short"; # 11 print length format {trim=>1}, "[[[[[[[[[[", "short"; # 6 =head3 A note about C<format> options Like templates and data, option sets (passed via hash references) can be interleaved within a single C<format> call. For example, to leave the text in $quote unsqueezed, unfilled, and unhyphenated, whilst squeezing, filling, and hyphenating the text in $intro and $commentary: print format { squeeze=>1, fill=>1, break=>Format::break_TeX }, '[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]', $intro, { squeeze=>0, fill=>0, break=>undef }, ' > [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[', $quote, { squeeze=>1, fill=>1, break=>Format::break_TeX }, '[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]', $commentary; or, more cleanly: $TIDY = { squeeze=>1, fill=>1, break=>Format::break_TeX }; $RAW = { squeeze=>0, fill=>0, break=>undef }; print format $TIDY, '[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]', $intro, $RAW, ' > [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[', $quote; $TIDY, '[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]', $commentary; Note that -- in C<Text::Autoformat::form> -- changes in options cannot currently be made I<between> interpolated data arguments (i.e. they apply to the next complete interpolation template and all the data interpolated into it). However, the ability to apply options to a single interpolated field might be a useful feature in the proposed C<Format::format>. For example, to create two columns where the left column is (only) filled and the right is (only) squeezed: print format '[[[[[[[[[[[[[[[[[[[ [[[[[[[[[[[[[[[[[[[', { squeeze=>0, fill=>1 }, $column1_text, { squeeze=>1, fill=>0 }, $column2_text; =head3 How C<format> consumes strings Within a single call to C<format> fields consume data text as they format it, so the following: $text = "a line of text to be formatted over three lines"; print format "<<<<<<<<<<\n <<<<<<<<\n <<<<<<\n", $text, $text, $text; produces: a line of text to be fo- not: a line of a line a line To achieve the latter effect, the variable arguments must be converted to independent literals (by double-quoted interpolation): print format "<<<<<<<<<<\n <<<<<<<<\n <<<<<<\n", "$text", "$text", "$text"; Although values passed from variable arguments are progressively consumed I<within> C<format>, the values of the original variables passed to C<format> are I<not> altered. In other words, internally C<format> operates on a copy of each variable's string. Hence: print format "<<<<<<<<<<\n <<<<<<<<\n <<<<<<\n", $text, $text, $text; print $text, "\n"; will print: a line of text to be fo- a line of text to be formatted over three lines To cause C<format> to I<visibly> consume the values of the original variables passed to it, they must be passed as references. Thus: print format "<<<<<<<<<<\n <<<<<<<<\n <<<<<<\n", \$text, \$text, \$text; print $text, "\n"; will print: a line of text to be fo- rmatted over three lines Note that, for safety, the "non-consuming" behaviour takes precedence, so if a variable is passed to C<format> both by reference I<and> by value, its final value will be unchanged. =head3 Numerical formatting The ">>>.<<<" and "]]].[[[" field specifiers may be used to format numeric values about a fixed decimal place marker. For example: print format '(>>>>>.<<)', <<EONUMS; 1 1.0 1.001 1.009 123.456 1234567 one two EONUMS would print: ( 1.0 ) ( 1.0 ) ( 1.00) ( 1.01) ( 123.46) (#####.##) (?????.??) (?????.??) Fractions are rounded (a la C<sprintf>) to the specified number of places after the decimal, but only the minimal number of significant digits are shown. That's why, in the above example, 1 and 1.0 are formatted as "1.0", whilst 1.001 is formatted as "1.00". It is possible to specify that the I<maximal> number of decimal places always be used, by giving the configuration option 'numeric' the value 'AllPlaces'. For example: print format { numeric => 'AllPlaces' }, '(>>>>>.<<)', <<'EONUMS'; 1 1.0 EONUMS would print: ( 1.00) ( 1.00) Note that although decimal digits are rounded to fit the specified width, the integral part of a number is never modified. If there are not enough places before the decimal place to represent the number, the entire number is replaced with hashes. If a I<non-numeric> sequence is passed as data for a numeric field, it is formatted as a series of question marks. This querulous behaviour can be changed by giving the configuration option 'numeric' the value 'SkipNaN', in which case, any invalid numeric data is simply ignored. For example: print format { numeric => 'SkipNaN' }, '(>>>>>.<<)', <<EONUMS; 1 two three 4 EONUMS would print: ( 1.0 ) ( 4.0 ) Other values for the "numeric" option might also be provided, such as: =over 4 =item 'Commas' Locale-sensitive commification of numbers. For example print format {numeric=>'Commas'}, ']]]]]]]]].[', [100,1000,10000,1000000,10000000]; # 100.0 # 1,000.0 # 10,000.0 # 1,000,000.0 # #########.# =item 'Currency' Cause the locale's currency sign to be properly inserted as part of each interpolation. This option would also imply 'AllPlaces'. For example: print format {numeric=>'Currency'}, ']]]]].[[', [1,100,1000,10000]; # $1.00 # $100.00 # $1000.00 # #####.## =item 'Fill(I<char>)' Use the specified character as the fill character, rather than spaces. For example: print format {numeric=>'Fill(*)'}, ']]]]].[[[[', [1,10.01,100.001,1000.0001,10000.00001], {numeric=>'Fill(+)'}, ']]]]].[[[[', [10,100,10000]; # ****1.0*** # ***10.01** # **100.001* # *1000.0001 # 10000.0000 # +++10.0+++ # ++100.0+++ # 10000.0+++ =back =head3 Filling block fields with lists of values If the data provided for a particular field is an array reference, then C<format> automatically joins the elements of the array into a single string, separating each element with a newline character. As a result, a call like this: @values = qw( 1 10 100 1000 ); print format "(]]]].[[)", \@values; will print out ( 1.00) ( 10.00) ( 100.00) (1000.00) as might be expected. Note that, because arrays must be passed using a reference, their original contents are consumed by C<format>, just like the contents of scalars passed by reference. To avoid having an array consumed by C<format>, pass it as an anonymous array: print format "(]]]].[[)", [@values]; =head3 Headers, footers, and pages The C<format> subroutine can also insert headers, footers, and page-feeds as it formats. These features are controlled by the "pagenum", "pagelen", "header", "footer", and "pagefeed" options. The "pagenum" option takes a scalar value or a reference to a scalar variable and starts page numbering at that value. If a reference to a scalar variable is specified, the value of that variable is updated as the formatting proceeds, so that the next page number is available in the variable after formatting. This can be useful for multi-part reports. The "pagelen" option specifies the total number of lines in a page (including headers and footers, but I<not> page-feeds). If the "header" option is specified with a string value, that string is used as the header of every page generated. If it is specified as a reference to a subroutine, that subroutine is called at the start of every page and its return value used as the header string. When called, the subroutine is passed the current page number. Likewise, if the "footer" option is specified with a string value, that string is used as the footer of every page generated. If it is specified as a reference to a subroutine, that subroutine is called at the I<start> of every page and its return value used as the footer string. When called, the footer subroutine is passed the current page number. Both the header and footer options can -- alternatively -- be specified as hash references. In this case the hash entries for the keys "left", "centre" (or "center"), and "right" specify what is to appear on the left, centre, and right of the header/footer. The entry for the key "width" specifies how wide the footer is to be. The "left", "centre", and "right" values may be literal strings or subroutines (just as a normal header/footer specification may be.) See the second example below. The "pagefeed" option acts in exactly the same way as "header" and "footer", using a literal string or a subroutine to produce a pagefeed string (but not necessarily "\014"), which is appended after the footer. Note however that the pagefeed is not counted as part of the page length, nor can it be specified as a left-centre-right hash. The header, footer, and pagefeed page components are recomputed at the start of each new page, before the page contents are formatted (recomputing the header and footer makes it possible to determine how many lines of data to format so as to adhere to the specified page length). When the call to C<format> is complete and the data has been fully formatted, the footer subroutine is called one last time, with an extra argument of 1. The string returned by this final call is used as the final footer. So for example, a series of 60-line pages with appropriate headers and footers might be formatted like so: print format { header => sub { "Page $_[0]\n\n" }, footer => sub { return "" if $_[1]; "-"x50."\n".format ">"x50", "...".($_[0]+1); }, pagefeed => "\n\n", pagelen => 60 }, $template, @data; Note the recursive use of C<format> within the "footer" option. As a second example, to set up headers and footers such that the running head is right justified in the header and the page number (starting at 7) is centred in the footer: print format { header => { right => "Running head" }, footer => { centre => sub { "Page $_[0]" } }, pagelen => 60, pagenum => 7, }, $template, @data; =head3 Lexically permanent options If C<use Format> is called with a hash reference as an argument, the entries of that hash specify options that are to be made the defaults until the end of the current lexical scope. For example, to cause C<format> to always squeeze and trim whitespace but respect newlines, the following line would be placed at the start of the source file: use Format { squeeze=>1, fill=>0, trim=>1 }; With these defaults in effect, filling could be turned on and trimming disabled within a particular subroutine like so: sub particular { use Format { fill => 1, trim => 0 }; # do filled, untrimmed formatting here } # format's defaults revert to previous values at end of scope =head3 C<format> examples As an example of the use of C<format>, the following: $count = 1; $text = "A big long piece of text to be formatted exquisitely"; print format q{ |||| <<<<<<<<<< ---------------- ^^^^ ]]]]]]]]]]\| = ]]].[[[ }, $count, $text, $count+11, $text, "123 123.4\n123.456789"; produces the following output: 1 A big long ---------------- 12 piece of| text to be| formatted| exquisite-| ly| = 123.0 123.4 123.456 Alternatively, picture strings and replacement values can be interleaved. For example: $report = format 'Name Rank Serial Number', '==== ==== =============', '<<<<<<<<<<<<< ^^^^ <<<<<<<<<<<<<', $name, $rank, $serial_number, '' 'Age Sex Description', '=== === =====================', '^^^ ^^^ [[[[[[[[[[[]]]]]]]]]]', $age, $sex, $description; =head1 MIGRATION ISSUES Non-trivial, but do-able. I also intend to extract C<Text::Autoformat::form> from Text::Autoformat and create a new Perl 5 module: Text::Reformat.pm. This would provide the same functionality (and a migration path) for Perl 5 users. =head1 IMPLEMENTATION See Text::Autoformat module. The proposed built-in/add-in ought to be reimplemented in C for speed, with Text::Autoformat::form (or Text::Reformat::format) as a reference. =head1 REFERENCES None.
