This and other RFCs are available on the web at
http://dev.perl.org/rfc/
=head1 TITLE
Replace C<format> built-in with pragmatically-induced C<format> function
=head1 VERSION
Maintainer: Damian Conway <[EMAIL PROTECTED]>
Date: 15 Sep 2000
Last Modified: 18 Sep 2000
Mailing List: [EMAIL PROTECTED]
Number: 230
Version: 2
Status: Developing
=head1 ABSTRACT
This RFC proposes that Perl's existing C<format> mechanism be replaced
with a standard module based on parts of the Text::Autoformat module.
=head1 DESCRIPTION
I can never remember how formats work. The specification syntax is confusing
to me. And I usually don't want the formatted text going straight down some
magical output stream.
It all came to a head when I was building Text::Autoformat. The smart text
recognition was easy -- trying to do the final formatting with C<formline>
and $^A was just too painful.
So I created the C<Text::Autoformat::form> subroutine. It uses a template
specification than fits my brain better, it's deeply DWIMical, it's highly
configurable, and it's re-entrant (so you can use a C<form> to format
another C<form>'s headers and footers, for example).
I propose that the existing C<format> mechanism be removed from Perl 6
and be replaced with a pragma-induced add-in function, based on
the semantics of C<Text::Autoformat::form>, as described in
the following sections.
=head2 The C<Format::format> function
The function itself would be called C<format> and would be
imported with a C<use Format> pragma.
It takes a format (or "picture" or "template") string followed by one or
more replacement values. It then interpolates those values into each picture
string, and either:
=over 4
=item *
returns the result as a single multi-line string (in a scalar context)
=item *
returns the result as a list of single-line strings (in a list context)
=back
In a void context, format would emit a warning: "Useless use of format
in void context".
A picture string consists of sequences of the following characters:
=over 4
=item <<<<<<<<
Left-justified field indicator. A series of two or more sequential <'s
specify a left-justified field to be filled by a subsequent value.
=item >>>>>>>>
Right-justified field indicator. A series of two or more sequential >'s
specify a right-justified field to be filled by a subsequent value.
=item ^^^^^^^^
Centre-justified field indicator. A series of two or more sequential ^'s
specify a centred field to be filled by a subsequent value.
=item >>>.<<<<
A numerically formatted field with the specified number of digits to
either side of the decimal place. See L<Numerical formatting> below.
=item [[[[[[[[
Left-justified block field indicator.
Just like a <<<<<<<<< field, except it repeats as required on subsequent
lines. See below.
=item ]]]]]]]]
Right-justified block field indicator.
Just like a >>>>>>>> field, except it repeats as required on subsequent
lines. See below.
=item ||||||||
Centre-justified block field indicator.
Just like a ^^^^^^^^ field, except it repeats as required on subsequent
lines. See below.
=item ]]].[[[[
A numerically formatted block field with the specified number of digits to
either side of the decimal place.
Just like a >>>.<<<< field, except it repeats as required on
subsequent lines. See below.
=item ~
A single character field. Interpolates a single character from a subsequent
data value. Repeats on subsequent lines as required.
=item \
Literal escape of next character (e.g. C<\~> is formatted as a literal '~',
not a one character wide field).
=item Any other character
That literal character.
=back
Hence a typical use of C<format> might look like this:
$formatted = format "<[[[[[[[[[[[[[[[[> >>>>>>>>>>>>>>>>",
$aphorism, "page $page_num";
and might produce an output like this:
<Like a camel > page 123
<through the eye >
<of a needle, so >
<are the days of >
<our lives >
Note that, because every field (except a C<~> field) must be at least
two characters wide, the single C<E<lt>> and C<E<gt>> brackets in the
format string are correctly interpreted as literals.
=head3 Multi-line filling
As the previous example indicates, any line with a block field continues
interpolation of that field on subsequent lines until all block fields
in the format have consumed all their data. Non-block fields on these
lines are replaced by the appropriate number of spaces.
For example:
$title = "On The Evil That Is Spam";
$text1 = "How many times have you longed to smash...";
$text2 = "...the bedevilment that is spam?";
print format "<<<<<<<<<<< [[[[[[[[[[[[[[[ [[[[[[[[[[",
$title, $text1, $text2;
# prints:
#
# On The Evil How many times ...the be-
# have you longed devilment
# to smash... that is
# spam
=head3 Hyphenation
As the previous example indicates, within a block field, words are
wrapped whole, unless they will not fit into the field at all, in which
case they are broken and hyphenated. Simple hyphenation is
used (i.e. break at the I<N-1>th character and insert a '-'), unless a
suitable alternative subroutine is specified instead.
Words will not be broken if the break would leave less than 2 characters
of the word on the current line. This minimum can be varied by setting the
'minbreak' option to some positive numeric value. This value indicates
the minumum total broken characters (I<including> hyphens) that must be
left on the current line.
Alternative word breaking subroutines can be specified using the "break"
option in a configuration hash. For example:
print format { break => \&my_line_breaker },
$template,
@data;
C<format> expects any user-defined line-breaking subroutine to take three
arguments (the string to be broken, the maximum permissible length of
the initial section, and the total width of the field being filled).
The subroutine must return a list of two strings: the initial
(broken) section of the word, and the remainder of the string
respectively).
For example:
sub tilde_break = sub($$$)
{
(substr($_[0],0,$_[1]-1).'~', substr($_[0],$_[1]-1));
}
print format { break => \&tilde_break },
$template,
@data;
This causes '~' to be used as the hyphenation character.
The C<use Format> pragma provides two exportable functions to
simplify the use of variant hyphenation schemes.
The exportable function C<Format::break_with> takes a single string
argument and returns a reference to a sub which hyphenates with that
string. Hence the previous example could be rewritten:
use Format qw( break_wrap );
print format { break => break_with('~') },
$template,
@data;
The exportable function C<Format::break_TeX> takes an optional argument
specifying a TeX hyphenation file and returns a reference to a sub which
hyphenates using Jan Pazdziora's TeX::Hyphen module (assuming it is
installed). For example:
use Format qw( break_TeX );
print format { break => break_TeX("lithuanian.hy") }
$template,
@data;
Note that in the previous examples there is no leading '\&' before
C<break_with> or C<break_TeX>, since each is being directly I<called>,
arther than referred to. They each return a reference to some other
suitable word-breaking subroutine.
=head3 Controlling line filling
C<format> preserves the original whitespace (including newlines) from an
interpolated string, unless called with certain options.
The "squeeze" option (when specified with a true value) causes any
interpolated sequence of spaces and/or tabs (but not newlines) to be
replaced with a single space.
The "fill" option independently causes newlines to be "squeezed".
Hence:
$frmt = "# [[[[[[[[[[[[[[[[[[[[[";
$data = "h e\t \tl lo\nworld\t\t\t\t\t";
print format $frmt, $data;
# h e l lo
# world
print format {squeeze=>1}, $frmt, $data;
# h e l lo
# world
print format {fill=>1}, $frmt, $data;
# h e l lo world
print format {squeeze=>1, fill=>1}, $frmt, $data;
# h e l lo world
Whether or not filling or squeezing is in effect, C<format> can also be
directed to trim any extra whitespace from the I<end> of each line it
formats, using the "trim" option. If this option is specified with a
true value, every line returned by C<format> will automatically have the
substitution C<s/[ \t]+$//gm> applied to it.
Hence:
print length format "[[[[[[[[[[", "short";
# 11
print length format {trim=>1}, "[[[[[[[[[[", "short";
# 6
=head3 How C<format> consumes strings
Within a single call to C<format> fields consume data
text as they format it, so the following:
$text = "a line of text to be formatted over three lines";
print format "<<<<<<<<<<\n <<<<<<<<\n <<<<<<\n",
$text, $text, $text;
produces:
a line of
text to
be fo-
not:
a line of
a line
a line
To achieve the latter effect, the variable arguments must be converted
to independent literals (by double-quoted interpolation):
print format "<<<<<<<<<<\n <<<<<<<<\n <<<<<<\n",
"$text", "$text", "$text";
Although values passed from variable arguments are progressively consumed
I<within> C<format>, the values of the original variables passed to C<format>
are I<not> altered. In other words, internally C<format> operates on a
copy of each variable's string. Hence:
print format "<<<<<<<<<<\n <<<<<<<<\n <<<<<<\n",
$text, $text, $text;
print $text, "\n";
will print:
a line of
text to
be fo-
a line of text to be formatted over three lines
To cause C<format> to I<visibly> consume the values of the original
variables passed to it, they must be passed as references. Thus:
print format "<<<<<<<<<<\n <<<<<<<<\n <<<<<<\n",
\$text, \$text, \$text;
print $text, "\n";
will print:
a line of
text to
be fo-
rmatted over three lines
Note that, for safety, the "non-consuming" behaviour takes precedence,
so if a variable is passed to C<format> both by reference I<and> by value,
its final value will be unchanged.
=head3 Numerical formatting
The ">>>.<<<" and "]]].[[[" field specifiers may be used to format
numeric values about a fixed decimal place marker. For example:
print format '(>>>>>.<<)', <<EONUMS;
1
1.0
1.001
1.009
123.456
1234567
one two
EONUMS
would print:
( 1.0 )
( 1.0 )
( 1.00)
( 1.01)
( 123.46)
(#####.##)
(?????.??)
(?????.??)
Fractions are rounded (a la C<sprintf>) to the specified number of
places after the decimal, but only the minimal number of significant
digits are shown. That's why, in the above example, 1 and 1.0 are
formatted as "1.0", whilst 1.001 is formatted as "1.00".
It is possible to specify that the I<maximal> number of decimal places
always be used, by giving the configuration option 'numeric' the value
'AllPlaces'. For example:
print format { numeric => 'AllPlaces' },
'(>>>>>.<<)', <<'EONUMS';
1
1.0
EONUMS
would print:
( 1.00)
( 1.00)
Note that although decimal digits are rounded to fit the specified width, the
integral part of a number is never modified. If there are not enough places
before the decimal place to represent the number, the entire number is
replaced with hashes.
If a I<non-numeric> sequence is passed as data for a numeric field, it is
formatted as a series of question marks. This querulous behaviour can be
changed by giving the configuration option 'numeric' the value 'SkipNaN',
in which case, any invalid numeric data is simply ignored. For example:
print format { numeric => 'SkipNaN' },
'(>>>>>.<<)',
<<EONUMS;
1
two three
4
EONUMS
would print:
( 1.0 )
( 4.0 )
=head3 Filling block fields with lists of values
If the data provided for a particular field is an array reference,
then C<format> automatically joins the elements of the array into a single
string, separating each element with a newline character. As a result, a
call like this:
@values = qw( 1 10 100 1000 );
print format "(]]]].[[)", \@values;
will print out
( 1.00)
( 10.00)
( 100.00)
(1000.00)
as might be expected.
Note that, because arrays must be passed using a reference, their
original contents are consumed by C<format>, just like the contents of
scalars passed by reference.
To avoid having an array consumed by C<format>, pass it as an anonymous
array:
print format "(]]]].[[)", [@values];
=head3 Headers, footers, and pages
The C<format> subroutine can also insert headers, footers, and page-feeds
as it formats. These features are controlled by the "pagenum", "pagelen",
"header", "footer", and "pagefeed" options.
The "pagenum" option takes a scalar value or a reference to a scalar
variable and starts page numbering at that value. If a reference to a
scalar variable is specified, the value of that variable is updated as
the formatting proceeds, so that the next page number is available in
the variable after formatting. This can be useful for multi-part reports.
The "pagelen" option specifies the total number of lines in a page
(including headers and footers, but I<not> page-feeds).
If the "header" option is specified with a string value, that string is
used as the header of every page generated. If it is specified as a reference
to a subroutine, that subroutine is called at the start of every page and
its return value used as the header string. When called, the subroutine is
passed the current page number.
Likewise, if the "footer" option is specified with a string value, that
string is used as the footer of every page generated. If it is specified
as a reference to a subroutine, that subroutine is called at the I<start>
of every page and its return value used as the footer string. When called,
the footer subroutine is passed the current page number.
Both the header and footer options can -- alternatively -- be
specified as hash references. In this case the hash entries for the keys
"left", "centre" (or "center"), and "right" specify what is to appear
on the left, centre, and right of the header/footer. The entry for the
key "width" specifies how wide the footer is to be. The "left",
"centre", and "right" values may be literal strings or subroutines
(just as a normal header/footer specification may be.) See the second
example below.
The "pagefeed" option acts in exactly the same way as "header" and
"footer", using a literal string or a subroutine to produce a pagefeed
string (but not necessarily "\014"), which is appended after the footer.
Note however that the pagefeed is not counted as part of the page length,
nor can it be specified as a left-centre-right hash.
The header, footer, and pagefeed page components are recomputed at the
start of each new page, before the page contents are formatted
(recomputing the header and footer makes it possible to determine
how many lines of data to format so as to adhere to the specified
page length).
When the call to C<format> is complete and the data has been fully formatted,
the footer subroutine is called one last time, with an extra argument of 1.
The string returned by this final call is used as the final footer.
So for example, a series of 60-line pages with appropriate headers and
footers might be formatted like so:
print format
{ header => sub { "Page $_[0]\n\n" },
footer => sub { return "" if $_[1];
"-"x50."\n".format ">"x50", "...".($_[0]+1);
},
pagefeed => "\n\n",
pagelen => 60
},
$template,
@data;
Note the recursive use of C<format> within the "footer" option.
As a second example, to set up headers and footers such that the running
head is right justified in the header and the page number (starting at 7)
is centred in the footer:
print format
{ header => { right => "Running head" },
footer => { centre => sub { "Page $_[0]" } },
pagelen => 60,
pagenum => 7,
},
$template,
@data;
=head3 Lexically permanent options
If C<use Format> is called with a hash reference as an argument, the
entries of that hash specify options that are to be made the defaults
until the end of the current lexical scope. For example, to cause
C<format> to always squeeze and trim whitespace but respect newlines,
the following line would be placed at the start of the source file:
use Format { squeeze=>1, fill=>0, trim=>1 };
With these defaults in effect, filling could be turned on and trimming disabled
within a particular subroutine like so:
sub particular {
use Format { fill => 1, trim => 0 };
# do filled, untrimmed formatting here
} # format's defaults revert to previous values at end of scope
=head3 C<format> examples
As an example of the use of C<format>, the following:
$count = 1;
$text = "A big long piece of text to be formatted exquisitely";
print format q{
|||| <<<<<<<<<<
----------------
^^^^ ]]]]]]]]]]\|
=
]]].[[[
}, $count, $text, $count+11, $text, "123 123.4\n123.456789";
produces the following output:
1 A big long
----------------
12 piece of|
text to be|
formatted|
exquisite-|
ly|
=
123.0
123.4
123.456
Alternatively, picture strings and replacement values can be interleaved.
For example:
$report = format
'Name Rank Serial Number',
'==== ==== =============',
'<<<<<<<<<<<<< ^^^^ <<<<<<<<<<<<<',
$name, $rank, $serial_number,
''
'Age Sex Description',
'=== === =====================',
'^^^ ^^^ [[[[[[[[[[[]]]]]]]]]]',
$age, $sex, $description;
=head1 MIGRATION ISSUES
Non-trivial, but do-able.
=head1 IMPLEMENTATION
See Text::Autoformat module.
The proposed built-in/add-in ought to be reimplemented in C for speed.
=head1 REFERENCES
None.
