https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67224
--- Comment #31 from Joseph S. Myers <jsm28 at gcc dot gnu.org> --- Author: jsm28 Date: Thu Sep 19 19:56:11 2019 New Revision: 275979 URL: https://gcc.gnu.org/viewcvs?rev=275979&root=gcc&view=rev Log: Support extended characters in C/C++ identifiers (PR c/67224) libcpp/ChangeLog 2019-09-19 Lewis Hyatt <lhy...@gmail.com> PR c/67224 * charset.c (_cpp_valid_utf8): New function to help lex UTF-8 tokens. * internal.h (_cpp_valid_utf8): Declare. * lex.c (forms_identifier_p): Use it to recognize UTF-8 identifiers. (_cpp_lex_direct): Handle UTF-8 in identifiers and CPP_OTHER tokens. Do all work in "default" case to avoid slowing down typical code paths. Also handle $ and UCN in the default case for consistency. gcc/Changelog 2019-09-19 Lewis Hyatt <lhy...@gmail.com> PR c/67224 * doc/cpp.texi: Document support for extended characters in identifiers. * doc/cppopts.texi: Likewise. gcc/testsuite/ChangeLog 2019-09-19 Lewis Hyatt <lhy...@gmail.com> PR c/67224 * c-c++-common/cpp/ucnid-2011-1-utf8.c: New test. * g++.dg/cpp/ucnid-1-utf8.C: New test. * g++.dg/cpp/ucnid-2-utf8.C: New test. * g++.dg/cpp/ucnid-3-utf8.C: New test. * g++.dg/cpp/ucnid-4-utf8.C: New test. * g++.dg/other/ucnid-1-utf8.C: New test. * gcc.dg/cpp/ucnid-1-utf8.c: New test. * gcc.dg/cpp/ucnid-10-utf8.c: New test. * gcc.dg/cpp/ucnid-11-utf8.c: New test. * gcc.dg/cpp/ucnid-12-utf8.c: New test. * gcc.dg/cpp/ucnid-13-utf8.c: New test. * gcc.dg/cpp/ucnid-14-utf8.c: New test. * gcc.dg/cpp/ucnid-15-utf8.c: New test. * gcc.dg/cpp/ucnid-2-utf8.c: New test. * gcc.dg/cpp/ucnid-3-utf8.c: New test. * gcc.dg/cpp/ucnid-4-utf8.c: New test. * gcc.dg/cpp/ucnid-6-utf8.c: New test. * gcc.dg/cpp/ucnid-7-utf8.c: New test. * gcc.dg/cpp/ucnid-9-utf8.c: New test. * gcc.dg/ucnid-1-utf8.c: New test. * gcc.dg/ucnid-10-utf8.c: New test. * gcc.dg/ucnid-11-utf8.c: New test. * gcc.dg/ucnid-12-utf8.c: New test. * gcc.dg/ucnid-13-utf8.c: New test. * gcc.dg/ucnid-14-utf8.c: New test. * gcc.dg/ucnid-15-utf8.c: New test. * gcc.dg/ucnid-16-utf8.c: New test. * gcc.dg/ucnid-2-utf8.c: New test. * gcc.dg/ucnid-3-utf8.c: New test. * gcc.dg/ucnid-4-utf8.c: New test. * gcc.dg/ucnid-5-utf8.c: New test. * gcc.dg/ucnid-6-utf8.c: New test. * gcc.dg/ucnid-7-utf8.c: New test. * gcc.dg/ucnid-8-utf8.c: New test. * gcc.dg/ucnid-9-utf8.c: New test. Added: trunk/gcc/testsuite/c-c++-common/cpp/ucnid-2011-1-utf8.c trunk/gcc/testsuite/g++.dg/cpp/ucnid-1-utf8.C trunk/gcc/testsuite/g++.dg/cpp/ucnid-2-utf8.C trunk/gcc/testsuite/g++.dg/cpp/ucnid-3-utf8.C trunk/gcc/testsuite/g++.dg/cpp/ucnid-4-utf8.C trunk/gcc/testsuite/g++.dg/other/ucnid-1-utf8.C trunk/gcc/testsuite/gcc.dg/cpp/ucnid-1-utf8.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-10-utf8.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-11-utf8.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-12-utf8.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-13-utf8.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-14-utf8.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-15-utf8.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-2-utf8.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-3-utf8.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-4-utf8.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-6-utf8.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-7-utf8.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-9-utf8.c trunk/gcc/testsuite/gcc.dg/ucnid-1-utf8.c trunk/gcc/testsuite/gcc.dg/ucnid-10-utf8.c trunk/gcc/testsuite/gcc.dg/ucnid-11-utf8.c trunk/gcc/testsuite/gcc.dg/ucnid-12-utf8.c trunk/gcc/testsuite/gcc.dg/ucnid-13-utf8.c trunk/gcc/testsuite/gcc.dg/ucnid-14-utf8.c trunk/gcc/testsuite/gcc.dg/ucnid-15-utf8.c trunk/gcc/testsuite/gcc.dg/ucnid-16-utf8.c trunk/gcc/testsuite/gcc.dg/ucnid-2-utf8.c trunk/gcc/testsuite/gcc.dg/ucnid-3-utf8.c trunk/gcc/testsuite/gcc.dg/ucnid-4-utf8.c trunk/gcc/testsuite/gcc.dg/ucnid-5-utf8.c trunk/gcc/testsuite/gcc.dg/ucnid-6-utf8.c trunk/gcc/testsuite/gcc.dg/ucnid-7-utf8.c trunk/gcc/testsuite/gcc.dg/ucnid-8-utf8.c trunk/gcc/testsuite/gcc.dg/ucnid-9-utf8.c Modified: trunk/gcc/ChangeLog trunk/gcc/doc/cpp.texi trunk/gcc/doc/cppopts.texi trunk/gcc/testsuite/ChangeLog trunk/libcpp/ChangeLog trunk/libcpp/charset.c trunk/libcpp/internal.h trunk/libcpp/lex.c
