Branch: refs/heads/blead
Home: https://github.com/Perl/perl5
Commit: 6db85f435326a989f2f06c83366caa94eaea50c5
https://github.com/Perl/perl5/commit/6db85f435326a989f2f06c83366caa94eaea50c5
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M toke.c
Log Message:
-----------
toke.c: Rmv unnecessary SV
This code created an SV simply to get SVfARG() to print it out. But
nowadays there is UTF8fARG which does the right thing on strings, so the
SV is unnecessary.
The code also has a fix where non-ASCII but Latin1 delimiters when 'use
utf8' is in effect would potentially come out as garbage This commit
also adds a clause in a conditional to prevent that.
Commit: 6ab7f65a8ce9d58ece5539d19d84ae00487894c5
https://github.com/Perl/perl5/commit/6ab7f65a8ce9d58ece5539d19d84ae00487894c5
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M toke.c
Log Message:
-----------
toke.c: Split a variable into two for clarity
It can have two different meanings; split and rename to clarify what
meaning it refers to.
Commit: 72358bbed002bb654b9158224ec350252ac8716e
https://github.com/Perl/perl5/commit/72358bbed002bb654b9158224ec350252ac8716e
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M toke.c
Log Message:
-----------
toke.c: Rename some variables; terminology in comment
Previously in places, things were called variously delimiter and
terminator, or variants thereof. This makes things consistent, and
clearer.
Commit: cf398451a5926ec75c6c5773c571b66dd670c2cb
https://github.com/Perl/perl5/commit/cf398451a5926ec75c6c5773c571b66dd670c2cb
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M toke.c
Log Message:
-----------
toke.c: Rmv unnecessary conditionals
Cheaper to just redo a simple assignment than to test if you've already
done it and skipping it if you had.
Commit: 908ddb65a3d72cfc75ac23c08a1db5e52e22e688
https://github.com/Perl/perl5/commit/908ddb65a3d72cfc75ac23c08a1db5e52e22e688
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M toke.c
Log Message:
-----------
toke.c: white-space only
This outdents some code that the next commit will remove an enclosing
block from
Commit: 32b87797e986f5d99836e16ea6b9d9ff5a56d3be
https://github.com/Perl/perl5/commit/32b87797e986f5d99836e16ea6b9d9ff5a56d3be
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M toke.c
Log Message:
-----------
toke.c: merge loops, multi-byte delim
The code in toke.c had two closely related loops; one for unmirrored
delimiters (same on both ends of the string) that could take UTF-8
delimiters, and one that allowed a mirrored closing delimiter, which
could take only a single byte. I found that it was just easiest to
collapse these into one loop in preparation to allow multi-byte
mirroring.
Commit: e80ffedaa6971ee7c9654d690bc70f67b7de2669
https://github.com/Perl/perl5/commit/e80ffedaa6971ee7c9654d690bc70f67b7de2669
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
regen/unicode_constants.pl: White space only
Align the output of this bit vertically with surrounding output.
Commit: 63cd44e4d01aafda8bc32c13f34dbab0035ac382
https://github.com/Perl/perl5/commit/63cd44e4d01aafda8bc32c13f34dbab0035ac382
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M regen/unicode_constants.pl
Log Message:
-----------
regen/unicode_constants.pl: Extract code into a fcn
This is in preparation for it to be used in multiple places in a future
commit.
Commit: c7b32e72c6b0a4931897121ac865a0fbc7445f17
https://github.com/Perl/perl5/commit/c7b32e72c6b0a4931897121ac865a0fbc7445f17
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
unicode_constants.pl: Generate paired string delimiters
This commit causes several C strings to be generated containing bytes
that match paired string delimiters beyond the four that have
traditionally been used in Perl. This will allow a future commit to
accept more matching delimiters around strings than those four.
The code explains how the added delimiters are chosen.
Commit: dce1e563ba694cbba6b45aaa354e0e0bacb26f69
https://github.com/Perl/perl5/commit/dce1e563ba694cbba6b45aaa354e0e0bacb26f69
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M regen/unicode_constants.pl
Log Message:
-----------
regen/unicode_constants.pl: List paired delimiters
This adds the capability to temporarily change a scalar to true to cause
this to print on stderr a list of the paired string delimiters, suitable
for pasting into a pod.
Commit: 9c9853e81d56a8abb664dd3e6332722675cc9a7c
https://github.com/Perl/perl5/commit/9c9853e81d56a8abb664dd3e6332722675cc9a7c
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M cpan/experimental/lib/experimental.pm
M feature.h
M lib/feature.pm
M lib/warnings.pm
M pod/perldelta.pod
M pod/perldiag.pod
M regen/feature.pl
M regen/warnings.pl
M t/lib/croak/toke
M t/lib/warnings/toke
M t/op/lex.t
M toke.c
M warnings.h
Log Message:
-----------
Add 'extra paired delimiters' feature
When this feature is enabled, one can use many more string delimiters
that have an opening version and a mirrored closing one.
Commit: 835f2666d2ae366f7af912303f061f066b8376c4
https://github.com/Perl/perl5/commit/835f2666d2ae366f7af912303f061f066b8376c4
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M t/lib/croak/toke
M t/lib/warnings/toke
M t/op/lex.t
M toke.c
M unicode_constants.h
Log Message:
-----------
Allow reversal of some paired delimiters; deprecations
Unicode says certain opening punctuation characters may be used as
closing ones in some languages; and their mirror is instead the opening
one.
This commit changes to allow either one of each such set to be the
opening one.
It also deprecates the use of any of the new mirrored delimiters to be
used outside the feature as an unmirrored delimiter, and the normal
closing delimiter from being used as an unpaired opening one while in
the feature. This gives us the freedom to make some or all of the new
paired delimiters be reversible.
Commit: 1df06faef81d6e2c5662cfb5d6cfc0844c38766e
https://github.com/Perl/perl5/commit/1df06faef81d6e2c5662cfb5d6cfc0844c38766e
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
unicode_constants.pl: Refactor to catch more paired delims
Previously, only characters that Unicode included in its bidirectional
algorithm have been eligible to be found by this program to be mirrored
string delimiters.
This commit adds 5 quotation marker character pairs that
are omitted from the bidirectional algorithm, as most quotes are,
because, as the Standard says, their "directionality and pairing status
is less predictable than paired brackets."
But we're not particularly interested in those semantics, most string
delimiters will be selected only for their visual appearance.
Because they aren't in the bidi algorithm, there is no property that
maps one member of a pair to its mate. However, Two characters whose
names pair only by LEFT vs RIGHT are almost certainly a mirrored pair.
This doesn't catch all possibilities; future commits will expand the
ones caught.
The commit refactors things so as to make future commits easier which
look at even more delimiter possibilities.
Commit: 9dfe4b75f459937c2299c80ec51fb4708616a240
https://github.com/Perl/perl5/commit/9dfe4b75f459937c2299c80ec51fb4708616a240
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M regen/unicode_constants.pl
Log Message:
-----------
unicode_constants.pl: Output why chars not chosen
This script now examines all punctuation characters to see if there is a
mirrored character for it, suitable for use as a Perl string delimiter.
Some don't qualify, and some do qualify but the script doesn't catch
them.
This commit adds the ability to output which characters it doesn't think
qualify, and why. This enables a maintainer to easily check and know
what its deficiencies are, or that there is a good reason that a
particular character gets rejected.
Commit: 565fbe1bf153b300641c08e2ec3668869422cb4d
https://github.com/Perl/perl5/commit/565fbe1bf153b300641c08e2ec3668869422cb4d
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
unicode_constants.pl: Add REVERSED punctuation
Besides LEFT/RIGHT, horizontal directionality can be specified by
Unicode in names by the presence or absence of REVERSED.
Enhancing the algorithm to take this into account adds 2 pairs or
mirrored delimiters that were previously overlooked.
Commit: 7e4a71c64c41ee74930b339d65a6d0ecdea70316
https://github.com/Perl/perl5/commit/7e4a71c64c41ee74930b339d65a6d0ecdea70316
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
Add < > variants to paired delimiters
Perl considers '< >' to be delimiters for strings; this commit adds
most of the Unicode variants of these to also be string delimiters. The
ones that are combinations of both < and >, aren't included, as that
would be visually confusing.
Commit: 210ad843ab44bc50a506b9ac9c6713b00b27ce44
https://github.com/Perl/perl5/commit/210ad843ab44bc50a506b9ac9c6713b00b27ce44
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
unicode_constants.pl: Consider all \pP for delims
Previously, only the punctuation characters that Unicode had classed as
being opening/closing were considered in looking for suitable paired
delimiters.
This commit looks at all punctuation characters. There are actually
only 7 new pairs found.
This gives us ꧁ ꧂ as string delimiterss, if your font allows,
which are Javanese and used to surround an honorific title, according to
Wikipedia.
Commit: 36327261879f8aa3b3cf2cde3bb43251392f774a
https://github.com/Perl/perl5/commit/36327261879f8aa3b3cf2cde3bb43251392f774a
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
Add SMALLER THAN to paired string delimiters
This commit adds 2 pairs of symbols that are variants on SMALLER THAN.
These look a lot like <>, so makes sense to make them paired delimiters.
Commit: 6763e244a01413dca6205be4f5d40f15a83bfff3
https://github.com/Perl/perl5/commit/6763e244a01413dca6205be4f5d40f15a83bfff3
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
Add PRECEDES/SUCCEEDS to paired string delimiters
This commit adds 15 pairs of symbols that are variants on PRECEDES.
These look a lot like <>, so makes sense to make them paired delimiters.
Commit: 0a4ec1c02bfc1e8d7a7141c2335d20f2f61f7112
https://github.com/Perl/perl5/commit/0a4ec1c02bfc1e8d7a7141c2335d20f2f61f7112
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
Add SUBSET/SUPERSET to paired string delimiters
This commit adds 20 pairs of symbols that are variants on SUBSET
These make nice paired delimiters in the vein of < >
Commit: c1b67e77ce8e2d80062d1f23dfe4c8e0cdbfd590
https://github.com/Perl/perl5/commit/c1b67e77ce8e2d80062d1f23dfe4c8e0cdbfd590
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
Add 'ELEMENT OF'/CONTAINS to paired string delimiters
This commit adds 8 pairs of symbols that are variants on ELEMENT OF
These make nice paired delimiters in the vein of < >
Commit: 50b179151ab7c50465ad9dcd16636805978a0ada
https://github.com/Perl/perl5/commit/50b179151ab7c50465ad9dcd16636805978a0ada
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
unicode_constants.pl: Prepare for examining Symbols
Heretofore, the code looking for paired string delimiters has looked at
punctuation, and a few symbols that Unicode gives a mirror for. But
there are many more suitable-for-pairing characters in Unicode.
This commit generalizes things so as to handle the extra complexities of
the way symbols are named beyond the punctuation names. For example,
RIGHTWARDS is sometimes used; it turns out that it also is used in one
punctuation character, which was previously overlooked by this script.
The generalization introduced by this commit handles almost all current
Unicode symbols properly.
But some symbols are barely distinguishable from their mirrors, such as
a tilde and a reversed tilde. The scheme adopted here, then, makes the
default for a symbol pair to not be marked as paired delimiters. The
code explicitly has to specify that a given pair is to be included.
The next few commits are mostly for adding ones that I thought were
good.
Commit: 9153861ee6e7388524ed557a08b6498ce3d8988e
https://github.com/Perl/perl5/commit/9153861ee6e7388524ed557a08b6498ce3d8988e
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M regen/unicode_constants.pl
Log Message:
-----------
Directionality pres/abs-ence can mean paired delimiters
Another way Unicode indicates that a character has horizontal
directionality is by adding LEFT or RIGHT to the name of a base
character. Hence we get RIGHT SPEAKER vs just plain SPEAKER.
Presumably this comes about when they didn't consider directionality at
first, and then realized later it was needed.
This commit makes the script look for these kinds of character pairs.
Because the current Unicode version only has this characteristic for
Symbols, and symbols must be included explicitly, no changes in what
gets paired ensues. But if you turn on the outputting of characters not
chosen, that list will now include things meeting this new criteria.
Less than a handful actually are like this.
Commit: c7c4369b7fc926018c3c32ca50f9649de7794726
https://github.com/Perl/perl5/commit/c7c4369b7fc926018c3c32ca50f9649de7794726
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
Add TACK paired delimiters
The bidi-aware characters containing this word are visually suitable for
being mirrored delimiters.
Commit: 84c1ecba3cd8905c63a30bead51f310457de1a0a
https://github.com/Perl/perl5/commit/84c1ecba3cd8905c63a30bead51f310457de1a0a
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
Add TURNSTILE paired delimiters
The bidi-aware characters containing this word are visually suitable for
being mirrored delimiters.
Commit: 4a4b745599392d7fdd61168bdcdc9e61b6c40a25
https://github.com/Perl/perl5/commit/4a4b745599392d7fdd61168bdcdc9e61b6c40a25
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
Add INDEX paired delimiters
The bidi-aware characters containing this word are visually suitable for
being mirrored delimiters. The 'index' refers to the index finger
in a hand pointing at the delimited string
Commit: 3e9e4fd83c55cfce5dc933d71f51d46e3c25be61
https://github.com/Perl/perl5/commit/3e9e4fd83c55cfce5dc933d71f51d46e3c25be61
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M pod/perldelta.pod
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
Add musical score paired delimiters
The characters that signify the beginning and ending of Western music
scores serve as good delimiters
Commit: e73a9e146014cf8337a76fe3f2f796935b0e84f6
https://github.com/Perl/perl5/commit/e73a9e146014cf8337a76fe3f2f796935b0e84f6
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
Add THREE RAYS paired delimiters
The characters with this name look good as mirrored delimiters.
Commit: a29825fb7d24f59dafbdfc07fb7481838acdef41
https://github.com/Perl/perl5/commit/a29825fb7d24f59dafbdfc07fb7481838acdef41
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
Add DOUBLE TRIANGLEs paired delimiters
The characters with this name look good as mirrored delimiters.
Commit: 806eaaf9dee151d9a00021704239209934b72563
https://github.com/Perl/perl5/commit/806eaaf9dee151d9a00021704239209934b72563
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
Add ERASE paired delimiters
The characters with this name look good as mirrored delimiters.
Commit: ad558b4e118927df1ab4d8ec02d8c73b5a282ce3
https://github.com/Perl/perl5/commit/ad558b4e118927df1ab4d8ec02d8c73b5a282ce3
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
Add TELEPHONE RECEIVER paired delimiters
The characters with this name look good as mirrored delimiters.
Commit: a863e562cae5ce949a7fa5f01ab4dcd5b999be02
https://github.com/Perl/perl5/commit/a863e562cae5ce949a7fa5f01ab4dcd5b999be02
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
Add SPEAKERs paired delimiters
The characters with this name look good as mirrored delimiters.
Commit: 0b6e3da14b7eeef5c4eca585e0c4c8911a8ecae7
https://github.com/Perl/perl5/commit/0b6e3da14b7eeef5c4eca585e0c4c8911a8ecae7
Author: Karl Williamson <[email protected]>
Date: 2022-03-19 (Sat, 19 Mar 2022)
Changed paths:
M lib/feature.pm
M regen/feature.pl
M regen/unicode_constants.pl
M unicode_constants.h
Log Message:
-----------
Add arrows to paired string delimiters
Unicode has lots of arrows of various shapes, sizes, and directions.
None of them were of consequence to the Bidirectional algorithm, so none
were specified as being mirrored pairs. This commit uses the
generalizations already in place from previous commits to examine arrow
symbols and choose which are mirrored pairs.
As previously, it rejects arrows with contrary directionality, and ones
without horizontal directionality.
Compare: https://github.com/Perl/perl5/compare/42a429a3564e...0b6e3da14b7e
