XeTeX with 8-bit TeX-fonts.

[Guenter Milde]

On 2017-01-11, Guenter Milde wrote:
> On 2017-01-11, Jürgen Spitzmüller wrote:
>> 2017-01-11 17:12 GMT+01:00 Scott Kostyshak <skost...@lyx.org>:
>>> On Wed, Jan 11, 2017 at 10:27:34AM +0100, Kornel Benko wrote:



>>> > Now we have unwanted messages e.g. for the test
>>> "export/doc/de/UserGuide_pdf4_texF".
...
>> This is related to http://www.lyx.org/trac/ticket/9740
...
> The warnings (and a subsequent error) are there, since "pfd4_texF"
> (XeTeX and 8-bit TeX-Fonts) uses the ASCII input encoding (see the
> ticket for the reasons).

> Therefore, "invertedTests" contains the pattern

> # Non-ASCII char in verbatim environment.
> # Fails with ASCII (and hence also with XeTeX).
> export/doc/(|de/|es/|fr/)UserGuide_pdf4_texF


> However, the same warnings and error occure with PDFLaTeX when
> setting Document>Settings>Language>Encoding to ASCII.

> The real problem is bad support for ASCII inputenc.
> (But our tests don't test different encodings.)

Besides fixing and testing the ASCII input encoding, we could also use
"xetex-inputenc.sty", a sized down version of the original inputenc.sty
that allows XeTeX with TeX fonts for all encodings supported by
inputenc.sty (see below).

+1 better support for XeTeX with TeX fonts

-1 fragile regarding included files and auxiliary files

-1 who needs XeTeX with TeX fonts?

Alternatively,
> We could drop support for XeTeX with TeX fonts (except the user
> explicitely sets Language>Encoding to ASCII)

Günter



%% This is file `xetex-inputenc.sty',
%% a variant of `inputenc.sty' that works with XeTeX in 8-bit mode.
%% (by removing the test for XeTeX/LuaTeX introduced in
%% recent versions of inputenc.sty).
%%
%% Copyright © 2017 Günter Milde
%%
%% This file may be distributed and/or modified under the
%% conditions of the LaTeX Project Public License, either version 1.3c
%% of this license or (at your option) any later version.
%% The latest version of this license is in
%%    http://www.latex-project.org/lppl.txt
%% and version 1.3c or later is part of all distributions of LaTeX
%% version 2005/12/01 or later.
%%
%% Usage:
%%   In the preamble of the file, write \XeTeXinputencoding "bytes".
%%
%% XeTeX will change the interpretation of the input bytes beginning at the
%% *next* line of the input file.
%%
%% There is also \XeTeXdefaultencoding "bytes" which changes the initial
%% encoding for any files that are *subsequently* opened. Therefore, you can
%% use this to get XeTeX to read an existing file in a legacy encoding, without
%% having to edit that file itself -- just set the default encoding from a
%% "driver" file before doing \input.
%%
%% However, with \XeTeXdefaultencoding "bytes", errors occure with auxiliary
%% files (tableofcontents, index, ...) because xetex *always* writes auxiliary
%% files using utf-8, regardless of the \XeTeXdefaultencoding setting.
%% https://www.tug.org/pipermail/xetex/2010-April/016452.html
%
\NeedsTeXFormat{LaTeX2e}[1995/12/01]
\ProvidesPackage{xetex-inputenc}
   [2015/03/17 v1.2c XeTeX Input encoding file]
%
% \AtBeginDocument{
%      \PackageWarning{xetex-inputenc}%
%        {`xetex-inputenc' expects XeTeXinputencoding "bytes".\MessageBreak
%        }
% }
\def\DeclareInputMath#1{%
   \@inpenc@test
   \bgroup
      \uccode`\~#1%
      \uppercase{%
   \egroup
      \def~%
   }%
}
\def\DeclareInputText#1#2{%
   \def\reserved@a##1 ${}%
   \def\reserved@b{#2}%
   \ifcat_\expandafter\reserved@a\meaning\reserved@b$ $_%
      \DeclareInputMath{#1}{#2}%
   \else
      \DeclareInputMath{#1}{\IeC{#2}}%
   \fi
}
\def\IeC{%
  \ifx\protect\@typeset@protect
    \expandafter\@firstofone
  \else
    \noexpand\IeC
  \fi
}
\def\inputencoding#1{%
  \the\inpenc@prehook
  \gdef\@inpenc@test{\global\let\@inpenc@test\relax}%
  \edef\@inpenc@undefined{\noexpand\protect\noexpand\@inpenc@undefined@{#1}}%
  \edef\inputencodingname{#1}%
  \@inpenc@loop\^^A\^^H%
  \@inpenc@loop\^^K\^^K%
  \@inpenc@loop\^^N\^^_%
  \@inpenc@loop\^^?\^^ff%
  \xdef\saved@endlinechar@code{\the\endlinechar}%
  \endlinechar\m@ne
  \xdef\saved@space@catcode{\the\catcode`\ }%
  \catcode`\ 9\relax
  \xdef\saved@at@catcode{\the\catcode`\@}%
  \makeatletter
  \xdef\saved@percent@catcode{\the\catcode`\%}%
  \catcode`\%14\relax
  \input{#1.def}%
  \endlinechar\saved@endlinechar@code\relax
  \catcode`\ \saved@space@catcode\relax
  \catcode`\@\saved@at@catcode\relax
  \catcode`\%\saved@percent@catcode\relax
  \ifx\@inpenc@test\relax\else
    \PackageWarning{xetex-inputenc}%
             {No characters defined\MessageBreak
              by input encoding change to `#1'\MessageBreak}%
  \fi
  \the\inpenc@posthook
}
\newtoks\inpenc@prehook
\newtoks\inpenc@posthook
\def\@inpenc@undefined@#1{\PackageError{xetex-inputenc}%
        {Keyboard character used is undefined\MessageBreak
         in inputencoding `#1'}%
       {You need to provide a  definition with
        \noexpand\DeclareInputText\MessageBreak or
        \noexpand\DeclareInputMath before using this key.}}%
\def\@inpenc@loop#1#2{%
  \@tempcnta`#1\relax
  \loop
    \catcode\@tempcnta\active
    \bgroup
      \uccode`\~\@tempcnta
      \uppercase{%
    \egroup
         \let~\@inpenc@undefined
      }%
  \ifnum\@tempcnta<`#2\relax
     \advance\@tempcnta\@ne
  \repeat}
\DeclareOption*{\inputencoding{\CurrentOption}}
\ProcessOptions
\endinput
%%
%% End of file `xetex-inputenc.sty'.


\documentclass[]{article}

\usepackage{lmodern}
\usepackage[T1]{fontenc}
\usepackage[utf8]{xetex-inputenc}
\usepackage{textcomp}
\XeTeXinputencoding "bytes"

\begin{document}

\section{test}

Grüße
äöü

µm

\end{document}