Hi,
The unistr/* modules and, with them, libunistring contain code for parsing
byte streams to Unicode characters. Currently it is optimized for speed
at the expense of safety (garbage in - garbage out). The code for safer parsing
(garbage in - error code) is already present for years, but needs to be
explicitly enabled through -DCONFIG_UNICODE_SAFETY.
I'm proposing to make this the default. It's time to favour safety over speed:
* We've seen so many CVEs in the past years, due to loose checking.
There is a theoretical possibility that such loose checking in libunistring
might be used to construct security attacks.
* No one nowadays will say "let's go back from UTF-8 to ASCII" because of
the small performance overhead of a few added machine instructions.
libunistring passes "make check" with these changes.
2016-11-17 Bruno Haible <br...@clisp.org>
Enable Unicode decoder safety unconditionally.
* lib/unistr.in.h (u32_mbtouc_unsafe): Assume CONFIG_UNICODE_SAFETY.
* lib/unistr/u8-mblen.c (u8_mblen): Likewise.
* lib/unistr/u8-mbtouc-unsafe.c (u8_mbtouc_unsafe): Likewise.
* lib/unistr/u8-mbtouc-unsafe-aux.c (u8_mbtouc_unsafe_aux): Likewise.
* lib/unistr/u8-prev.c (u8_prev): Likewise.
* lib/unistr/u8-strmblen.c (u8_strmblen): Likewise.
* lib/unistr/u8-strmbtouc.c (u8_strmbtouc): Likewise.
* lib/unistr/u16-mblen.c (u16_mblen): Likewise.
* lib/unistr/u16-mbtouc-unsafe.c (u16_mbtouc_unsafe): Likewise.
* lib/unistr/u16-mbtouc-unsafe-aux.c (u16_mbtouc_unsafe_aux): Likewise.
* lib/unistr/u16-prev.c (u16_prev): Likewise.
* lib/unistr/u16-strmblen.c (u16_strmblen): Likewise.
* lib/unistr/u16-strmbtouc.c (u16_strmbtouc): Likewise.
* lib/unistr/u32-mblen.c (u32_mblen): Likewise.
* lib/unistr/u32-mbtouc-unsafe.c (u32_mbtouc_unsafe): Likewise.
* lib/unistr/u32-prev.c (u32_prev): Likewise.
* lib/unistr/u32-next.c (u32_next): Likewise.
* lib/unistr/u32-strmblen.c (u32_strmblen): Likewise.
* lib/unistr/u32-strmbtouc.c (u32_strmbtouc): Likewise.
* lib/uniconv/u8-conv-to-enc.c (u8_conv_to_encoding): Likewise.
* lib/uniconv/u8-strconv-to-enc.c (u8_strconv_to_encoding): Likewise.
* tests/unistr/test-u16-prev.c (check_invalid): Enable the
CONFIG_UNICODE_SAFETY tests unconditionally.
* tests/unistr/test-u32-mblen.c (main): Likewise.
* tests/unistr/test-u32-mbtouc.h (test_function): Likewise.
* tests/unistr/test-u32-prev.c (check_invalid): Likewise.
* tests/unistr/test-u32-next.c (main): Likewise.
* tests/unistr/test-u32-strmblen.c (main): Likewise.
* tests/unistr/test-u32-strmbtouc.c (main): Likewise.
* lib/unistr/u8-check.c (u8_check): Remove old dead code.
* lib/unistr/u8-mbtouc.c (u8_mbtouc): Likewise.
* lib/unistr/u8-mbtouc-aux.c (u8_mbtouc_aux): Likewise.
* lib/unistr/u8-mbtoucr.c (u8_mbtoucr): Likewise.
* lib/unistr/u8-uctomb.c (u8_uctomb): Likewise.
* lib/unistr/u8-uctomb-aux.c (u8_uctomb_aux): Likewise.
* lib/unistr/u16-check.c (u16_check): Update comment.
* NEWS: Mention the changes that callers should be aware of.
>From 64bf594e0ac0050dfd84f4b81ec364bd535482b9 Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Thu, 17 Nov 2016 11:51:55 +0100
Subject: [PATCH] Enable Unicode decoder safety unconditionally.
* lib/unistr.in.h (u32_mbtouc_unsafe): Assume CONFIG_UNICODE_SAFETY.
* lib/unistr/u8-mblen.c (u8_mblen): Likewise.
* lib/unistr/u8-mbtouc-unsafe.c (u8_mbtouc_unsafe): Likewise.
* lib/unistr/u8-mbtouc-unsafe-aux.c (u8_mbtouc_unsafe_aux): Likewise.
* lib/unistr/u8-prev.c (u8_prev): Likewise.
* lib/unistr/u8-strmblen.c (u8_strmblen): Likewise.
* lib/unistr/u8-strmbtouc.c (u8_strmbtouc): Likewise.
* lib/unistr/u16-mblen.c (u16_mblen): Likewise.
* lib/unistr/u16-mbtouc-unsafe.c (u16_mbtouc_unsafe): Likewise.
* lib/unistr/u16-mbtouc-unsafe-aux.c (u16_mbtouc_unsafe_aux): Likewise.
* lib/unistr/u16-prev.c (u16_prev): Likewise.
* lib/unistr/u16-strmblen.c (u16_strmblen): Likewise.
* lib/unistr/u16-strmbtouc.c (u16_strmbtouc): Likewise.
* lib/unistr/u32-mblen.c (u32_mblen): Likewise.
* lib/unistr/u32-mbtouc-unsafe.c (u32_mbtouc_unsafe): Likewise.
* lib/unistr/u32-prev.c (u32_prev): Likewise.
* lib/unistr/u32-next.c (u32_next): Likewise.
* lib/unistr/u32-strmblen.c (u32_strmblen): Likewise.
* lib/unistr/u32-strmbtouc.c (u32_strmbtouc): Likewise.
* lib/uniconv/u8-conv-to-enc.c (u8_conv_to_encoding): Likewise.
* lib/uniconv/u8-strconv-to-enc.c (u8_strconv_to_encoding): Likewise.
* tests/unistr/test-u16-prev.c (check_invalid): Enable the
CONFIG_UNICODE_SAFETY tests unconditionally.
* tests/unistr/test-u32-mblen.c (main): Likewise.
* tests/unistr/test-u32-mbtouc.h (test_function): Likewise.
* tests/unistr/test-u32-prev.c (check_invalid): Likewise.
* tests/unistr/test-u32-next.c (main): Likewise.
* tests/unistr/test-u32-strmblen.c (main): Likewise.
* tests/unistr/test-u32-strmbtouc.c (main): Likewise.
* lib/unistr/u8-check.c (u8_check): Remove old dead code.
* lib/unistr/u8-mbtouc.c (u8_mbtouc): Likewise.
* lib/unistr/u8-mbtouc-aux.c (u8_mbtouc_aux): Likewise.
* lib/unistr/u8-mbtoucr.c (u8_mbtoucr): Likewise.
* lib/unistr/u8-uctomb.c (u8_uctomb): Likewise.
* lib/unistr/u8-uctomb-aux.c (u8_uctomb_aux): Likewise.
* lib/unistr/u16-check.c (u16_check): Update comment.
* NEWS: Mention the changes that callers should be aware of.
---
ChangeLog | 41 +++++++++++
NEWS | 3 +
lib/uniconv/u8-conv-to-enc.c | 2 -
lib/uniconv/u8-strconv-to-enc.c | 2 -
lib/unistr.in.h | 4 -
lib/unistr/u16-check.c | 2 +-
lib/unistr/u16-mblen.c | 9 +--
lib/unistr/u16-mbtouc-unsafe-aux.c | 4 -
lib/unistr/u16-mbtouc-unsafe.c | 4 -
lib/unistr/u16-prev.c | 4 -
lib/unistr/u16-strmblen.c | 7 +-
lib/unistr/u16-strmbtouc.c | 8 +-
lib/unistr/u32-mblen.c | 2 -
lib/unistr/u32-mbtouc-unsafe.c | 4 -
lib/unistr/u32-next.c | 2 -
lib/unistr/u32-prev.c | 2 -
lib/unistr/u32-strmblen.c | 4 -
lib/unistr/u32-strmbtouc.c | 4 -
lib/unistr/u8-check.c | 30 +-------
lib/unistr/u8-mblen.c | 42 +----------
lib/unistr/u8-mbtouc-aux.c | 104 +-------------------------
lib/unistr/u8-mbtouc-unsafe-aux.c | 121 -------------------------------
lib/unistr/u8-mbtouc-unsafe.c | 121 -------------------------------
lib/unistr/u8-mbtouc.c | 104 +-------------------------
lib/unistr/u8-mbtoucr.c | 145 +------------------------------------
lib/unistr/u8-prev.c | 16 +---
lib/unistr/u8-strmblen.c | 42 +----------
lib/unistr/u8-strmbtouc.c | 57 +--------------
lib/unistr/u8-uctomb-aux.c | 13 ----
lib/unistr/u8-uctomb.c | 13 ----
tests/unistr/test-u16-prev.c | 2 -
tests/unistr/test-u32-mblen.c | 2 -
tests/unistr/test-u32-mbtouc.h | 2 -
tests/unistr/test-u32-next.c | 2 -
tests/unistr/test-u32-prev.c | 2 -
tests/unistr/test-u32-strmblen.c | 2 -
tests/unistr/test-u32-strmbtouc.c | 2 -
37 files changed, 59 insertions(+), 871 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 5527191..b3ded4f 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,44 @@
+2016-11-17 Bruno Haible <br...@clisp.org>
+
+ Enable Unicode decoder safety unconditionally.
+ * lib/unistr.in.h (u32_mbtouc_unsafe): Assume CONFIG_UNICODE_SAFETY.
+ * lib/unistr/u8-mblen.c (u8_mblen): Likewise.
+ * lib/unistr/u8-mbtouc-unsafe.c (u8_mbtouc_unsafe): Likewise.
+ * lib/unistr/u8-mbtouc-unsafe-aux.c (u8_mbtouc_unsafe_aux): Likewise.
+ * lib/unistr/u8-prev.c (u8_prev): Likewise.
+ * lib/unistr/u8-strmblen.c (u8_strmblen): Likewise.
+ * lib/unistr/u8-strmbtouc.c (u8_strmbtouc): Likewise.
+ * lib/unistr/u16-mblen.c (u16_mblen): Likewise.
+ * lib/unistr/u16-mbtouc-unsafe.c (u16_mbtouc_unsafe): Likewise.
+ * lib/unistr/u16-mbtouc-unsafe-aux.c (u16_mbtouc_unsafe_aux): Likewise.
+ * lib/unistr/u16-prev.c (u16_prev): Likewise.
+ * lib/unistr/u16-strmblen.c (u16_strmblen): Likewise.
+ * lib/unistr/u16-strmbtouc.c (u16_strmbtouc): Likewise.
+ * lib/unistr/u32-mblen.c (u32_mblen): Likewise.
+ * lib/unistr/u32-mbtouc-unsafe.c (u32_mbtouc_unsafe): Likewise.
+ * lib/unistr/u32-prev.c (u32_prev): Likewise.
+ * lib/unistr/u32-next.c (u32_next): Likewise.
+ * lib/unistr/u32-strmblen.c (u32_strmblen): Likewise.
+ * lib/unistr/u32-strmbtouc.c (u32_strmbtouc): Likewise.
+ * lib/uniconv/u8-conv-to-enc.c (u8_conv_to_encoding): Likewise.
+ * lib/uniconv/u8-strconv-to-enc.c (u8_strconv_to_encoding): Likewise.
+ * tests/unistr/test-u16-prev.c (check_invalid): Enable the
+ CONFIG_UNICODE_SAFETY tests unconditionally.
+ * tests/unistr/test-u32-mblen.c (main): Likewise.
+ * tests/unistr/test-u32-mbtouc.h (test_function): Likewise.
+ * tests/unistr/test-u32-prev.c (check_invalid): Likewise.
+ * tests/unistr/test-u32-next.c (main): Likewise.
+ * tests/unistr/test-u32-strmblen.c (main): Likewise.
+ * tests/unistr/test-u32-strmbtouc.c (main): Likewise.
+ * lib/unistr/u8-check.c (u8_check): Remove old dead code.
+ * lib/unistr/u8-mbtouc.c (u8_mbtouc): Likewise.
+ * lib/unistr/u8-mbtouc-aux.c (u8_mbtouc_aux): Likewise.
+ * lib/unistr/u8-mbtoucr.c (u8_mbtoucr): Likewise.
+ * lib/unistr/u8-uctomb.c (u8_uctomb): Likewise.
+ * lib/unistr/u8-uctomb-aux.c (u8_uctomb_aux): Likewise.
+ * lib/unistr/u16-check.c (u16_check): Update comment.
+ * NEWS: Mention the changes that callers should be aware of.
+
2016-11-15 Pedro Alves <pal...@redhat.com>
sys_time: add gnulib::timeval for C++
diff --git a/NEWS b/NEWS
index ac302b5..cd30859 100644
--- a/NEWS
+++ b/NEWS
@@ -3,6 +3,9 @@ Important notes
Date Modules Changes
+2016-11-17 unistr/u32-strmblen The function u32_strmblen can now return -1.
+2016-11-17 unistr/u32-strmbtouc The function u32_strmbtouc can now return -1.
+
2015-04-24 acl This module no longer defines file_has_acl.
Use the new file-has-acl module for that.
Using only the latter module makes for fewer
diff --git a/lib/uniconv/u8-conv-to-enc.c b/lib/uniconv/u8-conv-to-enc.c
index 1057de7..bfd42b1 100644
--- a/lib/uniconv/u8-conv-to-enc.c
+++ b/lib/uniconv/u8-conv-to-enc.c
@@ -41,13 +41,11 @@ u8_conv_to_encoding (const char *tocode,
char *result;
/* Conversion from UTF-8 to UTF-8. No need to go through iconv(). */
-#if CONFIG_UNICODE_SAFETY
if (u8_check (src, srclen))
{
errno = EILSEQ;
return NULL;
}
-#endif
/* Memory allocation. */
if (resultbuf != NULL && *lengthp >= srclen)
diff --git a/lib/uniconv/u8-strconv-to-enc.c b/lib/uniconv/u8-strconv-to-enc.c
index e2c3660..5380306 100644
--- a/lib/uniconv/u8-strconv-to-enc.c
+++ b/lib/uniconv/u8-strconv-to-enc.c
@@ -41,13 +41,11 @@ u8_strconv_to_encoding (const uint8_t *string,
{
/* Conversion from UTF-8 to UTF-8. No need to go through iconv(). */
length = u8_strlen (string) + 1;
-#if CONFIG_UNICODE_SAFETY
if (u8_check (string, length))
{
errno = EILSEQ;
return NULL;
}
-#endif
result = (char *) malloc (length);
if (result == NULL)
{
diff --git a/lib/unistr.in.h b/lib/unistr.in.h
index 2a64b31..f5f5744 100644
--- a/lib/unistr.in.h
+++ b/lib/unistr.in.h
@@ -197,15 +197,11 @@ u32_mbtouc_unsafe (ucs4_t *puc,
{
uint32_t c = *s;
-# if CONFIG_UNICODE_SAFETY
if (c < 0xd800 || (c >= 0xe000 && c < 0x110000))
-# endif
*puc = c;
-# if CONFIG_UNICODE_SAFETY
else
/* invalid multibyte character */
*puc = 0xfffd;
-# endif
return 1;
}
# endif
diff --git a/lib/unistr/u16-check.c b/lib/unistr/u16-check.c
index 7bd5d3e..8210715 100644
--- a/lib/unistr/u16-check.c
+++ b/lib/unistr/u16-check.c
@@ -27,7 +27,7 @@ u16_check (const uint16_t *s, size_t n)
while (s < s_end)
{
- /* Keep in sync with unistr.h and utf16-ucs4.c. */
+ /* Keep in sync with unistr.h and u16-mbtouc-aux.c. */
uint16_t c = *s;
if (c < 0xd800 || c >= 0xe000)
diff --git a/lib/unistr/u16-mblen.c b/lib/unistr/u16-mblen.c
index 17cb846..9a42c18 100644
--- a/lib/unistr/u16-mblen.c
+++ b/lib/unistr/u16-mblen.c
@@ -26,24 +26,17 @@ u16_mblen (const uint16_t *s, size_t n)
{
if (n > 0)
{
- /* Keep in sync with unistr.h and utf16-ucs4.c. */
+ /* Keep in sync with unistr.h and u16-mbtouc-aux.c. */
uint16_t c = *s;
if (c < 0xd800 || c >= 0xe000)
return (c != 0 ? 1 : 0);
-#if CONFIG_UNICODE_SAFETY
if (c < 0xdc00)
{
if (n >= 2
&& s[1] >= 0xdc00 && s[1] < 0xe000)
return 2;
}
-#else
- {
- if (n >= 2)
- return 2;
- }
-#endif
}
/* invalid or incomplete multibyte character */
return -1;
diff --git a/lib/unistr/u16-mbtouc-unsafe-aux.c b/lib/unistr/u16-mbtouc-unsafe-aux.c
index 48d2bce..500d8c1 100644
--- a/lib/unistr/u16-mbtouc-unsafe-aux.c
+++ b/lib/unistr/u16-mbtouc-unsafe-aux.c
@@ -27,15 +27,11 @@ u16_mbtouc_unsafe_aux (ucs4_t *puc, const uint16_t *s, size_t n)
{
uint16_t c = *s;
-#if CONFIG_UNICODE_SAFETY
if (c < 0xdc00)
-#endif
{
if (n >= 2)
{
-#if CONFIG_UNICODE_SAFETY
if (s[1] >= 0xdc00 && s[1] < 0xe000)
-#endif
{
*puc = 0x10000 + ((c - 0xd800) << 10) + (s[1] - 0xdc00);
return 2;
diff --git a/lib/unistr/u16-mbtouc-unsafe.c b/lib/unistr/u16-mbtouc-unsafe.c
index 03cf255..c5d7a16 100644
--- a/lib/unistr/u16-mbtouc-unsafe.c
+++ b/lib/unistr/u16-mbtouc-unsafe.c
@@ -38,15 +38,11 @@ u16_mbtouc_unsafe (ucs4_t *puc, const uint16_t *s, size_t n)
*puc = c;
return 1;
}
-#if CONFIG_UNICODE_SAFETY
if (c < 0xdc00)
-#endif
{
if (n >= 2)
{
-#if CONFIG_UNICODE_SAFETY
if (s[1] >= 0xdc00 && s[1] < 0xe000)
-#endif
{
*puc = 0x10000 + ((c - 0xd800) << 10) + (s[1] - 0xdc00);
return 2;
diff --git a/lib/unistr/u16-prev.c b/lib/unistr/u16-prev.c
index 2c3e45a..e9beb73 100644
--- a/lib/unistr/u16-prev.c
+++ b/lib/unistr/u16-prev.c
@@ -33,16 +33,12 @@ u16_prev (ucs4_t *puc, const uint16_t *s, const uint16_t *start)
*puc = c_1;
return s - 1;
}
-#if CONFIG_UNICODE_SAFETY
if (c_1 >= 0xdc00)
-#endif
if (s - 1 != start)
{
uint16_t c_2 = s[-2];
-#if CONFIG_UNICODE_SAFETY
if (c_2 >= 0xd800 && c_2 < 0xdc00)
-#endif
{
*puc = 0x10000 + ((c_2 - 0xd800) << 10) + (c_1 - 0xdc00);
return s - 2;
diff --git a/lib/unistr/u16-strmblen.c b/lib/unistr/u16-strmblen.c
index dd507c7..20acb8c 100644
--- a/lib/unistr/u16-strmblen.c
+++ b/lib/unistr/u16-strmblen.c
@@ -24,21 +24,16 @@
int
u16_strmblen (const uint16_t *s)
{
- /* Keep in sync with unistr.h and utf16-ucs4.c. */
+ /* Keep in sync with unistr.h and u16-mbtouc-aux.c. */
uint16_t c = *s;
if (c < 0xd800 || c >= 0xe000)
return (c != 0 ? 1 : 0);
-#if CONFIG_UNICODE_SAFETY
if (c < 0xdc00)
{
if (s[1] >= 0xdc00 && s[1] < 0xe000)
return 2;
}
-#else
- if (s[1] != 0)
- return 2;
-#endif
/* invalid or incomplete multibyte character */
return -1;
}
diff --git a/lib/unistr/u16-strmbtouc.c b/lib/unistr/u16-strmbtouc.c
index f1945fe..affa09b 100644
--- a/lib/unistr/u16-strmbtouc.c
+++ b/lib/unistr/u16-strmbtouc.c
@@ -24,7 +24,7 @@
int
u16_strmbtouc (ucs4_t *puc, const uint16_t *s)
{
- /* Keep in sync with unistr.h and utf16-ucs4.c. */
+ /* Keep in sync with unistr.h and u16-mbtouc-aux.c. */
uint16_t c = *s;
if (c < 0xd800 || c >= 0xe000)
@@ -32,15 +32,9 @@ u16_strmbtouc (ucs4_t *puc, const uint16_t *s)
*puc = c;
return (c != 0 ? 1 : 0);
}
-#if CONFIG_UNICODE_SAFETY
if (c < 0xdc00)
-#endif
{
-#if CONFIG_UNICODE_SAFETY
if (s[1] >= 0xdc00 && s[1] < 0xe000)
-#else
- if (s[1] != 0)
-#endif
{
*puc = 0x10000 + ((c - 0xd800) << 10) + (s[1] - 0xdc00);
return 2;
diff --git a/lib/unistr/u32-mblen.c b/lib/unistr/u32-mblen.c
index a1d5823..57a88cf 100644
--- a/lib/unistr/u32-mblen.c
+++ b/lib/unistr/u32-mblen.c
@@ -27,9 +27,7 @@ u32_mblen (const uint32_t *s, size_t n)
{
uint32_t c = *s;
-#if CONFIG_UNICODE_SAFETY
if (c < 0xd800 || (c >= 0xe000 && c < 0x110000))
-#endif
return (c != 0 ? 1 : 0);
}
/* invalid or incomplete multibyte character */
diff --git a/lib/unistr/u32-mbtouc-unsafe.c b/lib/unistr/u32-mbtouc-unsafe.c
index 921b6d1..4b8dd3c 100644
--- a/lib/unistr/u32-mbtouc-unsafe.c
+++ b/lib/unistr/u32-mbtouc-unsafe.c
@@ -33,15 +33,11 @@ u32_mbtouc_unsafe (ucs4_t *puc, const uint32_t *s, size_t n)
{
uint32_t c = *s;
-#if CONFIG_UNICODE_SAFETY
if (c < 0xd800 || (c >= 0xe000 && c < 0x110000))
-#endif
*puc = c;
-#if CONFIG_UNICODE_SAFETY
else
/* invalid multibyte character */
*puc = 0xfffd;
-#endif
return 1;
}
diff --git a/lib/unistr/u32-next.c b/lib/unistr/u32-next.c
index cc04a9f..d2c7c32 100644
--- a/lib/unistr/u32-next.c
+++ b/lib/unistr/u32-next.c
@@ -30,10 +30,8 @@ u32_next (ucs4_t *puc, const uint32_t *s)
return s + count;
else
{
-#if CONFIG_UNICODE_SAFETY
if (count < 0)
*puc = 0xfffd;
-#endif
return NULL;
}
}
diff --git a/lib/unistr/u32-prev.c b/lib/unistr/u32-prev.c
index ad7f678..e44d82a 100644
--- a/lib/unistr/u32-prev.c
+++ b/lib/unistr/u32-prev.c
@@ -27,9 +27,7 @@ u32_prev (ucs4_t *puc, const uint32_t *s, const uint32_t *start)
{
uint32_t c_1 = s[-1];
-#if CONFIG_UNICODE_SAFETY
if (c_1 < 0xd800 || (c_1 >= 0xe000 && c_1 < 0x110000))
-#endif
{
*puc = c_1;
return s - 1;
diff --git a/lib/unistr/u32-strmblen.c b/lib/unistr/u32-strmblen.c
index 396a0dd..ea6c334 100644
--- a/lib/unistr/u32-strmblen.c
+++ b/lib/unistr/u32-strmblen.c
@@ -25,12 +25,8 @@ u32_strmblen (const uint32_t *s)
{
uint32_t c = *s;
-#if CONFIG_UNICODE_SAFETY
if (c < 0xd800 || (c >= 0xe000 && c < 0x110000))
-#endif
return (c != 0 ? 1 : 0);
-#if CONFIG_UNICODE_SAFETY
/* invalid multibyte character */
return -1;
-#endif
}
diff --git a/lib/unistr/u32-strmbtouc.c b/lib/unistr/u32-strmbtouc.c
index e3b22b3..c88c1a5 100644
--- a/lib/unistr/u32-strmbtouc.c
+++ b/lib/unistr/u32-strmbtouc.c
@@ -25,15 +25,11 @@ u32_strmbtouc (ucs4_t *puc, const uint32_t *s)
{
uint32_t c = *s;
-#if CONFIG_UNICODE_SAFETY
if (c < 0xd800 || (c >= 0xe000 && c < 0x110000))
-#endif
{
*puc = c;
return (c != 0 ? 1 : 0);
}
-#if CONFIG_UNICODE_SAFETY
/* invalid multibyte character */
return -1;
-#endif
}
diff --git a/lib/unistr/u8-check.c b/lib/unistr/u8-check.c
index b4f217c..37e7471 100644
--- a/lib/unistr/u8-check.c
+++ b/lib/unistr/u8-check.c
@@ -63,40 +63,12 @@ u8_check (const uint8_t *s, size_t n)
&& (s[1] ^ 0x80) < 0x40 && (s[2] ^ 0x80) < 0x40
&& (s[3] ^ 0x80) < 0x40
&& (c >= 0xf1 || s[1] >= 0x90)
-#if 1
- && (c < 0xf4 || (c == 0xf4 && s[1] < 0x90))
-#endif
- )
+ && (c < 0xf4 || (c == 0xf4 && s[1] < 0x90)))
{
s += 4;
continue;
}
}
-#if 0
- else if (c < 0xfc)
- {
- if (s + 5 <= s_end
- && (s[1] ^ 0x80) < 0x40 && (s[2] ^ 0x80) < 0x40
- && (s[3] ^ 0x80) < 0x40 && (s[4] ^ 0x80) < 0x40
- && (c >= 0xf9 || s[1] >= 0x88))
- {
- s += 5;
- continue;
- }
- }
- else if (c < 0xfe)
- {
- if (s + 6 <= s_end
- && (s[1] ^ 0x80) < 0x40 && (s[2] ^ 0x80) < 0x40
- && (s[3] ^ 0x80) < 0x40 && (s[4] ^ 0x80) < 0x40
- && (s[5] ^ 0x80) < 0x40
- && (c >= 0xfd || s[1] >= 0x84))
- {
- s += 6;
- continue;
- }
- }
-#endif
}
/* invalid or incomplete multibyte character */
return s;
diff --git a/lib/unistr/u8-mblen.c b/lib/unistr/u8-mblen.c
index ba37161..c88f4c9 100644
--- a/lib/unistr/u8-mblen.c
+++ b/lib/unistr/u8-mblen.c
@@ -36,62 +36,26 @@ u8_mblen (const uint8_t *s, size_t n)
if (c < 0xe0)
{
if (n >= 2
-#if CONFIG_UNICODE_SAFETY
- && (s[1] ^ 0x80) < 0x40
-#endif
- )
+ && (s[1] ^ 0x80) < 0x40)
return 2;
}
else if (c < 0xf0)
{
if (n >= 3
-#if CONFIG_UNICODE_SAFETY
&& (s[1] ^ 0x80) < 0x40 && (s[2] ^ 0x80) < 0x40
&& (c >= 0xe1 || s[1] >= 0xa0)
- && (c != 0xed || s[1] < 0xa0)
-#endif
- )
+ && (c != 0xed || s[1] < 0xa0))
return 3;
}
else if (c < 0xf8)
{
if (n >= 4
-#if CONFIG_UNICODE_SAFETY
&& (s[1] ^ 0x80) < 0x40 && (s[2] ^ 0x80) < 0x40
&& (s[3] ^ 0x80) < 0x40
&& (c >= 0xf1 || s[1] >= 0x90)
-#if 1
- && (c < 0xf4 || (c == 0xf4 && s[1] < 0x90))
-#endif
-#endif
- )
+ && (c < 0xf4 || (c == 0xf4 && s[1] < 0x90)))
return 4;
}
-#if 0
- else if (c < 0xfc)
- {
- if (n >= 5
-#if CONFIG_UNICODE_SAFETY
- && (s[1] ^ 0x80) < 0x40 && (s[2] ^ 0x80) < 0x40
- && (s[3] ^ 0x80) < 0x40 && (s[4] ^ 0x80) < 0x40
- && (c >= 0xf9 || s[1] >= 0x88)
-#endif
- )
- return 5;
- }
- else if (c < 0xfe)
- {
- if (n >= 6
-#if CONFIG_UNICODE_SAFETY
- && (s[1] ^ 0x80) < 0x40 && (s[2] ^ 0x80) < 0x40
- && (s[3] ^ 0x80) < 0x40 && (s[4] ^ 0x80) < 0x40
- && (s[5] ^ 0x80) < 0x40
- && (c >= 0xfd || s[1] >= 0x84)
-#endif
- )
- return 6;
- }
-#endif
}
}
/* invalid or incomplete multibyte character */
diff --git a/lib/unistr/u8-mbtouc-aux.c b/lib/unistr/u8-mbtouc-aux.c
index d546ae7..940f67e 100644
--- a/lib/unistr/u8-mbtouc-aux.c
+++ b/lib/unistr/u8-mbtouc-aux.c
@@ -95,10 +95,7 @@ u8_mbtouc_aux (ucs4_t *puc, const uint8_t *s, size_t n)
if ((s[3] ^ 0x80) < 0x40)
{
if ((c >= 0xf1 || s[1] >= 0x90)
-#if 1
- && (c < 0xf4 || (c == 0xf4 && s[1] < 0x90))
-#endif
- )
+ && (c < 0xf4 || (c == 0xf4 && s[1] < 0x90)))
{
*puc = ((unsigned int) (c & 0x07) << 18)
| ((unsigned int) (s[1] ^ 0x80) << 12)
@@ -132,105 +129,6 @@ u8_mbtouc_aux (ucs4_t *puc, const uint8_t *s, size_t n)
return 3;
}
}
-#if 0
- else if (c < 0xfc)
- {
- if (n >= 5)
- {
- if ((s[1] ^ 0x80) < 0x40)
- {
- if ((s[2] ^ 0x80) < 0x40)
- {
- if ((s[3] ^ 0x80) < 0x40)
- {
- if ((s[4] ^ 0x80) < 0x40)
- {
- if (c >= 0xf9 || s[1] >= 0x88)
- {
- *puc = ((unsigned int) (c & 0x03) << 24)
- | ((unsigned int) (s[1] ^ 0x80) << 18)
- | ((unsigned int) (s[2] ^ 0x80) << 12)
- | ((unsigned int) (s[3] ^ 0x80) << 6)
- | (unsigned int) (s[4] ^ 0x80);
- return 5;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 5;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 4;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 3;
- }
- /* invalid multibyte character */
- return 2;
- }
- /* invalid multibyte character */
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return n;
- }
- }
- else if (c < 0xfe)
- {
- if (n >= 6)
- {
- if ((s[1] ^ 0x80) < 0x40)
- {
- if ((s[2] ^ 0x80) < 0x40)
- {
- if ((s[3] ^ 0x80) < 0x40)
- {
- if ((s[4] ^ 0x80) < 0x40)
- {
- if ((s[5] ^ 0x80) < 0x40)
- {
- if (c >= 0xfd || s[1] >= 0x84)
- {
- *puc = ((unsigned int) (c & 0x01) << 30)
- | ((unsigned int) (s[1] ^ 0x80) << 24)
- | ((unsigned int) (s[2] ^ 0x80) << 18)
- | ((unsigned int) (s[3] ^ 0x80) << 12)
- | ((unsigned int) (s[4] ^ 0x80) << 6)
- | (unsigned int) (s[5] ^ 0x80);
- return 6;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 6;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 5;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 4;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 3;
- }
- /* invalid multibyte character */
- return 2;
- }
- /* invalid multibyte character */
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return n;
- }
- }
-#endif
}
/* invalid multibyte character */
*puc = 0xfffd;
diff --git a/lib/unistr/u8-mbtouc-unsafe-aux.c b/lib/unistr/u8-mbtouc-unsafe-aux.c
index 61783e4..4fab7bd 100644
--- a/lib/unistr/u8-mbtouc-unsafe-aux.c
+++ b/lib/unistr/u8-mbtouc-unsafe-aux.c
@@ -33,17 +33,13 @@ u8_mbtouc_unsafe_aux (ucs4_t *puc, const uint8_t *s, size_t n)
{
if (n >= 2)
{
-#if CONFIG_UNICODE_SAFETY
if ((s[1] ^ 0x80) < 0x40)
-#endif
{
*puc = ((unsigned int) (c & 0x1f) << 6)
| (unsigned int) (s[1] ^ 0x80);
return 2;
}
-#if CONFIG_UNICODE_SAFETY
/* invalid multibyte character */
-#endif
}
else
{
@@ -56,21 +52,18 @@ u8_mbtouc_unsafe_aux (ucs4_t *puc, const uint8_t *s, size_t n)
{
if (n >= 3)
{
-#if CONFIG_UNICODE_SAFETY
if ((s[1] ^ 0x80) < 0x40)
{
if ((s[2] ^ 0x80) < 0x40)
{
if ((c >= 0xe1 || s[1] >= 0xa0)
&& (c != 0xed || s[1] < 0xa0))
-#endif
{
*puc = ((unsigned int) (c & 0x0f) << 12)
| ((unsigned int) (s[1] ^ 0x80) << 6)
| (unsigned int) (s[2] ^ 0x80);
return 3;
}
-#if CONFIG_UNICODE_SAFETY
/* invalid multibyte character */
*puc = 0xfffd;
return 3;
@@ -80,7 +73,6 @@ u8_mbtouc_unsafe_aux (ucs4_t *puc, const uint8_t *s, size_t n)
return 2;
}
/* invalid multibyte character */
-#endif
}
else
{
@@ -96,7 +88,6 @@ u8_mbtouc_unsafe_aux (ucs4_t *puc, const uint8_t *s, size_t n)
{
if (n >= 4)
{
-#if CONFIG_UNICODE_SAFETY
if ((s[1] ^ 0x80) < 0x40)
{
if ((s[2] ^ 0x80) < 0x40)
@@ -104,11 +95,8 @@ u8_mbtouc_unsafe_aux (ucs4_t *puc, const uint8_t *s, size_t n)
if ((s[3] ^ 0x80) < 0x40)
{
if ((c >= 0xf1 || s[1] >= 0x90)
-#if 1
&& (c < 0xf4 || (c == 0xf4 && s[1] < 0x90))
-#endif
)
-#endif
{
*puc = ((unsigned int) (c & 0x07) << 18)
| ((unsigned int) (s[1] ^ 0x80) << 12)
@@ -116,7 +104,6 @@ u8_mbtouc_unsafe_aux (ucs4_t *puc, const uint8_t *s, size_t n)
| (unsigned int) (s[3] ^ 0x80);
return 4;
}
-#if CONFIG_UNICODE_SAFETY
/* invalid multibyte character */
*puc = 0xfffd;
return 4;
@@ -130,7 +117,6 @@ u8_mbtouc_unsafe_aux (ucs4_t *puc, const uint8_t *s, size_t n)
return 2;
}
/* invalid multibyte character */
-#endif
}
else
{
@@ -144,113 +130,6 @@ u8_mbtouc_unsafe_aux (ucs4_t *puc, const uint8_t *s, size_t n)
return 3;
}
}
-#if 0
- else if (c < 0xfc)
- {
- if (n >= 5)
- {
-#if CONFIG_UNICODE_SAFETY
- if ((s[1] ^ 0x80) < 0x40)
- {
- if ((s[2] ^ 0x80) < 0x40)
- {
- if ((s[3] ^ 0x80) < 0x40)
- {
- if ((s[4] ^ 0x80) < 0x40)
- {
- if (c >= 0xf9 || s[1] >= 0x88)
-#endif
- {
- *puc = ((unsigned int) (c & 0x03) << 24)
- | ((unsigned int) (s[1] ^ 0x80) << 18)
- | ((unsigned int) (s[2] ^ 0x80) << 12)
- | ((unsigned int) (s[3] ^ 0x80) << 6)
- | (unsigned int) (s[4] ^ 0x80);
- return 5;
- }
-#if CONFIG_UNICODE_SAFETY
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 5;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 4;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 3;
- }
- /* invalid multibyte character */
- return 2;
- }
- /* invalid multibyte character */
-#endif
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return n;
- }
- }
- else if (c < 0xfe)
- {
- if (n >= 6)
- {
-#if CONFIG_UNICODE_SAFETY
- if ((s[1] ^ 0x80) < 0x40)
- {
- if ((s[2] ^ 0x80) < 0x40)
- {
- if ((s[3] ^ 0x80) < 0x40)
- {
- if ((s[4] ^ 0x80) < 0x40)
- {
- if ((s[5] ^ 0x80) < 0x40)
- {
- if (c >= 0xfd || s[1] >= 0x84)
-#endif
- {
- *puc = ((unsigned int) (c & 0x01) << 30)
- | ((unsigned int) (s[1] ^ 0x80) << 24)
- | ((unsigned int) (s[2] ^ 0x80) << 18)
- | ((unsigned int) (s[3] ^ 0x80) << 12)
- | ((unsigned int) (s[4] ^ 0x80) << 6)
- | (unsigned int) (s[5] ^ 0x80);
- return 6;
- }
-#if CONFIG_UNICODE_SAFETY
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 6;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 5;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 4;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 3;
- }
- /* invalid multibyte character */
- return 2;
- }
- /* invalid multibyte character */
-#endif
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return n;
- }
- }
-#endif
}
/* invalid multibyte character */
*puc = 0xfffd;
diff --git a/lib/unistr/u8-mbtouc-unsafe.c b/lib/unistr/u8-mbtouc-unsafe.c
index b73e99a..477d555 100644
--- a/lib/unistr/u8-mbtouc-unsafe.c
+++ b/lib/unistr/u8-mbtouc-unsafe.c
@@ -44,17 +44,13 @@ u8_mbtouc_unsafe (ucs4_t *puc, const uint8_t *s, size_t n)
{
if (n >= 2)
{
-#if CONFIG_UNICODE_SAFETY
if ((s[1] ^ 0x80) < 0x40)
-#endif
{
*puc = ((unsigned int) (c & 0x1f) << 6)
| (unsigned int) (s[1] ^ 0x80);
return 2;
}
-#if CONFIG_UNICODE_SAFETY
/* invalid multibyte character */
-#endif
}
else
{
@@ -67,21 +63,18 @@ u8_mbtouc_unsafe (ucs4_t *puc, const uint8_t *s, size_t n)
{
if (n >= 3)
{
-#if CONFIG_UNICODE_SAFETY
if ((s[1] ^ 0x80) < 0x40)
{
if ((s[2] ^ 0x80) < 0x40)
{
if ((c >= 0xe1 || s[1] >= 0xa0)
&& (c != 0xed || s[1] < 0xa0))
-#endif
{
*puc = ((unsigned int) (c & 0x0f) << 12)
| ((unsigned int) (s[1] ^ 0x80) << 6)
| (unsigned int) (s[2] ^ 0x80);
return 3;
}
-#if CONFIG_UNICODE_SAFETY
/* invalid multibyte character */
*puc = 0xfffd;
return 3;
@@ -91,7 +84,6 @@ u8_mbtouc_unsafe (ucs4_t *puc, const uint8_t *s, size_t n)
return 2;
}
/* invalid multibyte character */
-#endif
}
else
{
@@ -107,7 +99,6 @@ u8_mbtouc_unsafe (ucs4_t *puc, const uint8_t *s, size_t n)
{
if (n >= 4)
{
-#if CONFIG_UNICODE_SAFETY
if ((s[1] ^ 0x80) < 0x40)
{
if ((s[2] ^ 0x80) < 0x40)
@@ -115,11 +106,8 @@ u8_mbtouc_unsafe (ucs4_t *puc, const uint8_t *s, size_t n)
if ((s[3] ^ 0x80) < 0x40)
{
if ((c >= 0xf1 || s[1] >= 0x90)
-#if 1
&& (c < 0xf4 || (c == 0xf4 && s[1] < 0x90))
-#endif
)
-#endif
{
*puc = ((unsigned int) (c & 0x07) << 18)
| ((unsigned int) (s[1] ^ 0x80) << 12)
@@ -127,7 +115,6 @@ u8_mbtouc_unsafe (ucs4_t *puc, const uint8_t *s, size_t n)
| (unsigned int) (s[3] ^ 0x80);
return 4;
}
-#if CONFIG_UNICODE_SAFETY
/* invalid multibyte character */
*puc = 0xfffd;
return 4;
@@ -141,7 +128,6 @@ u8_mbtouc_unsafe (ucs4_t *puc, const uint8_t *s, size_t n)
return 2;
}
/* invalid multibyte character */
-#endif
}
else
{
@@ -155,113 +141,6 @@ u8_mbtouc_unsafe (ucs4_t *puc, const uint8_t *s, size_t n)
return 3;
}
}
-#if 0
- else if (c < 0xfc)
- {
- if (n >= 5)
- {
-#if CONFIG_UNICODE_SAFETY
- if ((s[1] ^ 0x80) < 0x40)
- {
- if ((s[2] ^ 0x80) < 0x40)
- {
- if ((s[3] ^ 0x80) < 0x40)
- {
- if ((s[4] ^ 0x80) < 0x40)
- {
- if (c >= 0xf9 || s[1] >= 0x88)
-#endif
- {
- *puc = ((unsigned int) (c & 0x03) << 24)
- | ((unsigned int) (s[1] ^ 0x80) << 18)
- | ((unsigned int) (s[2] ^ 0x80) << 12)
- | ((unsigned int) (s[3] ^ 0x80) << 6)
- | (unsigned int) (s[4] ^ 0x80);
- return 5;
- }
-#if CONFIG_UNICODE_SAFETY
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 5;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 4;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 3;
- }
- /* invalid multibyte character */
- return 2;
- }
- /* invalid multibyte character */
-#endif
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return n;
- }
- }
- else if (c < 0xfe)
- {
- if (n >= 6)
- {
-#if CONFIG_UNICODE_SAFETY
- if ((s[1] ^ 0x80) < 0x40)
- {
- if ((s[2] ^ 0x80) < 0x40)
- {
- if ((s[3] ^ 0x80) < 0x40)
- {
- if ((s[4] ^ 0x80) < 0x40)
- {
- if ((s[5] ^ 0x80) < 0x40)
- {
- if (c >= 0xfd || s[1] >= 0x84)
-#endif
- {
- *puc = ((unsigned int) (c & 0x01) << 30)
- | ((unsigned int) (s[1] ^ 0x80) << 24)
- | ((unsigned int) (s[2] ^ 0x80) << 18)
- | ((unsigned int) (s[3] ^ 0x80) << 12)
- | ((unsigned int) (s[4] ^ 0x80) << 6)
- | (unsigned int) (s[5] ^ 0x80);
- return 6;
- }
-#if CONFIG_UNICODE_SAFETY
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 6;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 5;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 4;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 3;
- }
- /* invalid multibyte character */
- return 2;
- }
- /* invalid multibyte character */
-#endif
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return n;
- }
- }
-#endif
}
/* invalid multibyte character */
*puc = 0xfffd;
diff --git a/lib/unistr/u8-mbtouc.c b/lib/unistr/u8-mbtouc.c
index bfb0e4d..21e20a5 100644
--- a/lib/unistr/u8-mbtouc.c
+++ b/lib/unistr/u8-mbtouc.c
@@ -105,10 +105,7 @@ u8_mbtouc (ucs4_t *puc, const uint8_t *s, size_t n)
if ((s[3] ^ 0x80) < 0x40)
{
if ((c >= 0xf1 || s[1] >= 0x90)
-#if 1
- && (c < 0xf4 || (c == 0xf4 && s[1] < 0x90))
-#endif
- )
+ && (c < 0xf4 || (c == 0xf4 && s[1] < 0x90)))
{
*puc = ((unsigned int) (c & 0x07) << 18)
| ((unsigned int) (s[1] ^ 0x80) << 12)
@@ -142,105 +139,6 @@ u8_mbtouc (ucs4_t *puc, const uint8_t *s, size_t n)
return 3;
}
}
-#if 0
- else if (c < 0xfc)
- {
- if (n >= 5)
- {
- if ((s[1] ^ 0x80) < 0x40)
- {
- if ((s[2] ^ 0x80) < 0x40)
- {
- if ((s[3] ^ 0x80) < 0x40)
- {
- if ((s[4] ^ 0x80) < 0x40)
- {
- if (c >= 0xf9 || s[1] >= 0x88)
- {
- *puc = ((unsigned int) (c & 0x03) << 24)
- | ((unsigned int) (s[1] ^ 0x80) << 18)
- | ((unsigned int) (s[2] ^ 0x80) << 12)
- | ((unsigned int) (s[3] ^ 0x80) << 6)
- | (unsigned int) (s[4] ^ 0x80);
- return 5;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 5;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 4;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 3;
- }
- /* invalid multibyte character */
- return 2;
- }
- /* invalid multibyte character */
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return n;
- }
- }
- else if (c < 0xfe)
- {
- if (n >= 6)
- {
- if ((s[1] ^ 0x80) < 0x40)
- {
- if ((s[2] ^ 0x80) < 0x40)
- {
- if ((s[3] ^ 0x80) < 0x40)
- {
- if ((s[4] ^ 0x80) < 0x40)
- {
- if ((s[5] ^ 0x80) < 0x40)
- {
- if (c >= 0xfd || s[1] >= 0x84)
- {
- *puc = ((unsigned int) (c & 0x01) << 30)
- | ((unsigned int) (s[1] ^ 0x80) << 24)
- | ((unsigned int) (s[2] ^ 0x80) << 18)
- | ((unsigned int) (s[3] ^ 0x80) << 12)
- | ((unsigned int) (s[4] ^ 0x80) << 6)
- | (unsigned int) (s[5] ^ 0x80);
- return 6;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 6;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 5;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 4;
- }
- /* invalid multibyte character */
- *puc = 0xfffd;
- return 3;
- }
- /* invalid multibyte character */
- return 2;
- }
- /* invalid multibyte character */
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return n;
- }
- }
-#endif
}
/* invalid multibyte character */
*puc = 0xfffd;
diff --git a/lib/unistr/u8-mbtoucr.c b/lib/unistr/u8-mbtoucr.c
index 5fd45b5..5586e5f 100644
--- a/lib/unistr/u8-mbtoucr.c
+++ b/lib/unistr/u8-mbtoucr.c
@@ -92,10 +92,7 @@ u8_mbtoucr (ucs4_t *puc, const uint8_t *s, size_t n)
{
if ((s[1] ^ 0x80) < 0x40
&& (c >= 0xf1 || s[1] >= 0x90)
-#if 1
- && (c < 0xf4 || (c == 0xf4 && s[1] < 0x90))
-#endif
- )
+ && (c < 0xf4 || (c == 0xf4 && s[1] < 0x90)))
{
if (n >= 3)
{
@@ -138,146 +135,6 @@ u8_mbtoucr (ucs4_t *puc, const uint8_t *s, size_t n)
return -2;
}
}
-#if 0
- else if (c < 0xfc)
- {
- if (n >= 2)
- {
- if ((s[1] ^ 0x80) < 0x40
- && (c >= 0xf9 || s[1] >= 0x88))
- {
- if (n >= 3)
- {
- if ((s[2] ^ 0x80) < 0x40)
- {
- if (n >= 4)
- {
- if ((s[3] ^ 0x80) < 0x40)
- {
- if (n >= 5)
- {
- if ((s[4] ^ 0x80) < 0x40)
- {
- *puc = ((unsigned int) (c & 0x03) << 24)
- | ((unsigned int) (s[1] ^ 0x80) << 18)
- | ((unsigned int) (s[2] ^ 0x80) << 12)
- | ((unsigned int) (s[3] ^ 0x80) << 6)
- | (unsigned int) (s[4] ^ 0x80);
- return 5;
- }
- /* invalid multibyte character */
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return -2;
- }
- }
- /* invalid multibyte character */
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return -2;
- }
- }
- /* invalid multibyte character */
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return -2;
- }
- }
- /* invalid multibyte character */
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return -2;
- }
- }
- else if (c < 0xfe)
- {
- if (n >= 2)
- {
- if ((s[1] ^ 0x80) < 0x40
- && (c >= 0xfd || s[1] >= 0x84))
- {
- if (n >= 3)
- {
- if ((s[2] ^ 0x80) < 0x40)
- {
- if (n >= 4)
- {
- if ((s[3] ^ 0x80) < 0x40)
- {
- if (n >= 5)
- {
- if ((s[4] ^ 0x80) < 0x40)
- {
- if (n >= 6)
- {
- if ((s[5] ^ 0x80) < 0x40)
- {
- *puc = ((unsigned int) (c & 0x01) << 30)
- | ((unsigned int) (s[1] ^ 0x80) << 24)
- | ((unsigned int) (s[2] ^ 0x80) << 18)
- | ((unsigned int) (s[3] ^ 0x80) << 12)
- | ((unsigned int) (s[4] ^ 0x80) << 6)
- | (unsigned int) (s[5] ^ 0x80);
- return 6;
- }
- /* invalid multibyte character */
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return -2;
- }
- }
- /* invalid multibyte character */
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return -2;
- }
- }
- /* invalid multibyte character */
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return -2;
- }
- }
- /* invalid multibyte character */
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return -2;
- }
- }
- /* invalid multibyte character */
- }
- else
- {
- /* incomplete multibyte character */
- *puc = 0xfffd;
- return -2;
- }
- }
-#endif
}
/* invalid multibyte character */
*puc = 0xfffd;
diff --git a/lib/unistr/u8-prev.c b/lib/unistr/u8-prev.c
index b609acf..1abbb41 100644
--- a/lib/unistr/u8-prev.c
+++ b/lib/unistr/u8-prev.c
@@ -33,9 +33,7 @@ u8_prev (ucs4_t *puc, const uint8_t *s, const uint8_t *start)
*puc = c_1;
return s - 1;
}
-#if CONFIG_UNICODE_SAFETY
if ((c_1 ^ 0x80) < 0x40)
-#endif
if (s - 1 != start)
{
uint8_t c_2 = s[-2];
@@ -46,38 +44,28 @@ u8_prev (ucs4_t *puc, const uint8_t *s, const uint8_t *start)
| (unsigned int) (c_1 ^ 0x80);
return s - 2;
}
-#if CONFIG_UNICODE_SAFETY
if ((c_2 ^ 0x80) < 0x40)
-#endif
if (s - 2 != start)
{
uint8_t c_3 = s[-3];
if (c_3 >= 0xe0 && c_3 < 0xf0
-#if CONFIG_UNICODE_SAFETY
&& (c_3 >= 0xe1 || c_2 >= 0xa0)
- && (c_3 != 0xed || c_2 < 0xa0)
-#endif
- )
+ && (c_3 != 0xed || c_2 < 0xa0))
{
*puc = ((unsigned int) (c_3 & 0x0f) << 12)
| ((unsigned int) (c_2 ^ 0x80) << 6)
| (unsigned int) (c_1 ^ 0x80);
return s - 3;
}
-#if CONFIG_UNICODE_SAFETY
if ((c_3 ^ 0x80) < 0x40)
-#endif
if (s - 3 != start)
{
uint8_t c_4 = s[-4];
if (c_4 >= 0xf0 && c_4 < 0xf8
-#if CONFIG_UNICODE_SAFETY
&& (c_4 >= 0xf1 || c_3 >= 0x90)
- && (c_4 < 0xf4 || (c_4 == 0xf4 && c_3 < 0x90))
-#endif
- )
+ && (c_4 < 0xf4 || (c_4 == 0xf4 && c_3 < 0x90)))
{
*puc = ((unsigned int) (c_4 & 0x07) << 18)
| ((unsigned int) (c_3 ^ 0x80) << 12)
diff --git a/lib/unistr/u8-strmblen.c b/lib/unistr/u8-strmblen.c
index 49ba0e2..5cb7214 100644
--- a/lib/unistr/u8-strmblen.c
+++ b/lib/unistr/u8-strmblen.c
@@ -33,64 +33,24 @@ u8_strmblen (const uint8_t *s)
{
if (c < 0xe0)
{
-#if CONFIG_UNICODE_SAFETY
if ((s[1] ^ 0x80) < 0x40)
-#else
- if (s[1] != 0)
-#endif
return 2;
}
else if (c < 0xf0)
{
-#if CONFIG_UNICODE_SAFETY
if ((s[1] ^ 0x80) < 0x40 && (s[2] ^ 0x80) < 0x40
&& (c >= 0xe1 || s[1] >= 0xa0)
&& (c != 0xed || s[1] < 0xa0))
-#else
- if (s[1] != 0 && s[2] != 0)
-#endif
return 3;
}
else if (c < 0xf8)
{
-#if CONFIG_UNICODE_SAFETY
if ((s[1] ^ 0x80) < 0x40 && (s[2] ^ 0x80) < 0x40
&& (s[3] ^ 0x80) < 0x40
&& (c >= 0xf1 || s[1] >= 0x90)
-#if 1
- && (c < 0xf4 || (c == 0xf4 && s[1] < 0x90))
-#endif
- )
-#else
- if (s[1] != 0 && s[2] != 0 && s[3] != 0)
-#endif
+ && (c < 0xf4 || (c == 0xf4 && s[1] < 0x90)))
return 4;
}
-#if 0
- else if (c < 0xfc)
- {
-#if CONFIG_UNICODE_SAFETY
- if ((s[1] ^ 0x80) < 0x40 && (s[2] ^ 0x80) < 0x40
- && (s[3] ^ 0x80) < 0x40 && (s[4] ^ 0x80) < 0x40
- && (c >= 0xf9 || s[1] >= 0x88))
-#else
- if (s[1] != 0 && s[2] != 0 && s[3] != 0 && s[4] != 0)
-#endif
- return 5;
- }
- else if (c < 0xfe)
- {
-#if CONFIG_UNICODE_SAFETY
- if ((s[1] ^ 0x80) < 0x40 && (s[2] ^ 0x80) < 0x40
- && (s[3] ^ 0x80) < 0x40 && (s[4] ^ 0x80) < 0x40
- && (s[5] ^ 0x80) < 0x40
- && (c >= 0xfd || s[1] >= 0x84))
-#else
- if (s[1] != 0 && s[2] != 0 && s[3] != 0 && s[4] != 0 && s[5] != 0)
-#endif
- return 6;
- }
-#endif
}
/* invalid or incomplete multibyte character */
return -1;
diff --git a/lib/unistr/u8-strmbtouc.c b/lib/unistr/u8-strmbtouc.c
index c8094d9..746d972 100644
--- a/lib/unistr/u8-strmbtouc.c
+++ b/lib/unistr/u8-strmbtouc.c
@@ -36,11 +36,7 @@ u8_strmbtouc (ucs4_t *puc, const uint8_t *s)
{
if (c < 0xe0)
{
-#if CONFIG_UNICODE_SAFETY
if ((s[1] ^ 0x80) < 0x40)
-#else
- if (s[1] != 0)
-#endif
{
*puc = ((unsigned int) (c & 0x1f) << 6)
| (unsigned int) (s[1] ^ 0x80);
@@ -49,13 +45,9 @@ u8_strmbtouc (ucs4_t *puc, const uint8_t *s)
}
else if (c < 0xf0)
{
-#if CONFIG_UNICODE_SAFETY
if ((s[1] ^ 0x80) < 0x40 && (s[2] ^ 0x80) < 0x40
&& (c >= 0xe1 || s[1] >= 0xa0)
&& (c != 0xed || s[1] < 0xa0))
-#else
- if (s[1] != 0 && s[2] != 0)
-#endif
{
*puc = ((unsigned int) (c & 0x0f) << 12)
| ((unsigned int) (s[1] ^ 0x80) << 6)
@@ -65,17 +57,10 @@ u8_strmbtouc (ucs4_t *puc, const uint8_t *s)
}
else if (c < 0xf8)
{
-#if CONFIG_UNICODE_SAFETY
if ((s[1] ^ 0x80) < 0x40 && (s[2] ^ 0x80) < 0x40
&& (s[3] ^ 0x80) < 0x40
&& (c >= 0xf1 || s[1] >= 0x90)
-#if 1
- && (c < 0xf4 || (c == 0xf4 && s[1] < 0x90))
-#endif
- )
-#else
- if (s[1] != 0 && s[2] != 0 && s[3] != 0)
-#endif
+ && (c < 0xf4 || (c == 0xf4 && s[1] < 0x90)))
{
*puc = ((unsigned int) (c & 0x07) << 18)
| ((unsigned int) (s[1] ^ 0x80) << 12)
@@ -84,46 +69,6 @@ u8_strmbtouc (ucs4_t *puc, const uint8_t *s)
return 4;
}
}
-#if 0
- else if (c < 0xfc)
- {
-#if CONFIG_UNICODE_SAFETY
- if ((s[1] ^ 0x80) < 0x40 && (s[2] ^ 0x80) < 0x40
- && (s[3] ^ 0x80) < 0x40 && (s[4] ^ 0x80) < 0x40
- && (c >= 0xf9 || s[1] >= 0x88))
-#else
- if (s[1] != 0 && s[2] != 0 && s[3] != 0 && s[4] != 0)
-#endif
- {
- *puc = ((unsigned int) (c & 0x03) << 24)
- | ((unsigned int) (s[1] ^ 0x80) << 18)
- | ((unsigned int) (s[2] ^ 0x80) << 12)
- | ((unsigned int) (s[3] ^ 0x80) << 6)
- | (unsigned int) (s[4] ^ 0x80);
- return 5;
- }
- }
- else if (c < 0xfe)
- {
-#if CONFIG_UNICODE_SAFETY
- if ((s[1] ^ 0x80) < 0x40 && (s[2] ^ 0x80) < 0x40
- && (s[3] ^ 0x80) < 0x40 && (s[4] ^ 0x80) < 0x40
- && (s[5] ^ 0x80) < 0x40
- && (c >= 0xfd || s[1] >= 0x84))
-#else
- if (s[1] != 0 && s[2] != 0 && s[3] != 0 && s[4] != 0 && s[5] != 0)
-#endif
- {
- *puc = ((unsigned int) (c & 0x01) << 30)
- | ((unsigned int) (s[1] ^ 0x80) << 24)
- | ((unsigned int) (s[2] ^ 0x80) << 18)
- | ((unsigned int) (s[3] ^ 0x80) << 12)
- | ((unsigned int) (s[4] ^ 0x80) << 6)
- | (unsigned int) (s[5] ^ 0x80);
- return 6;
- }
- }
-#endif
}
/* invalid or incomplete multibyte character */
return -1;
diff --git a/lib/unistr/u8-uctomb-aux.c b/lib/unistr/u8-uctomb-aux.c
index a41ffcf..aa4dead 100644
--- a/lib/unistr/u8-uctomb-aux.c
+++ b/lib/unistr/u8-uctomb-aux.c
@@ -37,17 +37,8 @@ u8_uctomb_aux (uint8_t *s, ucs4_t uc, int n)
else
return -1;
}
-#if 0
- else if (uc < 0x200000)
- count = 4;
- else if (uc < 0x4000000)
- count = 5;
- else if (uc <= 0x7fffffff)
- count = 6;
-#else
else if (uc < 0x110000)
count = 4;
-#endif
else
return -1;
@@ -56,10 +47,6 @@ u8_uctomb_aux (uint8_t *s, ucs4_t uc, int n)
switch (count) /* note: code falls through cases! */
{
-#if 0
- case 6: s[5] = 0x80 | (uc & 0x3f); uc = uc >> 6; uc |= 0x4000000;
- case 5: s[4] = 0x80 | (uc & 0x3f); uc = uc >> 6; uc |= 0x200000;
-#endif
case 4: s[3] = 0x80 | (uc & 0x3f); uc = uc >> 6; uc |= 0x10000;
/* fallthrough */
case 3: s[2] = 0x80 | (uc & 0x3f); uc = uc >> 6; uc |= 0x800;
diff --git a/lib/unistr/u8-uctomb.c b/lib/unistr/u8-uctomb.c
index d62ad0f..4db2576 100644
--- a/lib/unistr/u8-uctomb.c
+++ b/lib/unistr/u8-uctomb.c
@@ -52,17 +52,8 @@ u8_uctomb (uint8_t *s, ucs4_t uc, int n)
else
return -1;
}
-#if 0
- else if (uc < 0x200000)
- count = 4;
- else if (uc < 0x4000000)
- count = 5;
- else if (uc <= 0x7fffffff)
- count = 6;
-#else
else if (uc < 0x110000)
count = 4;
-#endif
else
return -1;
@@ -70,10 +61,6 @@ u8_uctomb (uint8_t *s, ucs4_t uc, int n)
{
switch (count) /* note: code falls through cases! */
{
-#if 0
- case 6: s[5] = 0x80 | (uc & 0x3f); uc = uc >> 6; uc |= 0x4000000;
- case 5: s[4] = 0x80 | (uc & 0x3f); uc = uc >> 6; uc |= 0x200000;
-#endif
case 4: s[3] = 0x80 | (uc & 0x3f); uc = uc >> 6; uc |= 0x10000;
case 3: s[2] = 0x80 | (uc & 0x3f); uc = uc >> 6; uc |= 0x800;
case 2: s[1] = 0x80 | (uc & 0x3f); uc = uc >> 6; uc |= 0xc0;
diff --git a/tests/unistr/test-u16-prev.c b/tests/unistr/test-u16-prev.c
index a1479d1..65e9ab3 100644
--- a/tests/unistr/test-u16-prev.c
+++ b/tests/unistr/test-u16-prev.c
@@ -84,7 +84,6 @@ check_invalid (const uint16_t *input, size_t input_length)
if (uc != 0xBADFACE)
return 2;
-#if CONFIG_UNICODE_SAFETY
/* Test recognition when preceded by a 1-unit character. */
{
uint16_t buf[100];
@@ -121,7 +120,6 @@ check_invalid (const uint16_t *input, size_t input_length)
if (uc != 0xBADFACE)
return 6;
}
-#endif
return 0;
}
diff --git a/tests/unistr/test-u32-mblen.c b/tests/unistr/test-u32-mblen.c
index a58f2d8..e707e21 100644
--- a/tests/unistr/test-u32-mblen.c
+++ b/tests/unistr/test-u32-mblen.c
@@ -68,14 +68,12 @@ main ()
ASSERT (ret == 1);
}
-#if CONFIG_UNICODE_SAFETY
/* Test incomplete/invalid 1-unit input. */
{
static const uint32_t input[] = { 0x340000 };
ret = u32_mblen (input, 1);
ASSERT (ret == -1);
}
-#endif
return 0;
}
diff --git a/tests/unistr/test-u32-mbtouc.h b/tests/unistr/test-u32-mbtouc.h
index 22d5e0b..d332125 100644
--- a/tests/unistr/test-u32-mbtouc.h
+++ b/tests/unistr/test-u32-mbtouc.h
@@ -64,7 +64,6 @@ test_function (int (*my_u32_mbtouc) (ucs4_t *, const uint32_t *, size_t))
ASSERT (uc == 0x1D51F);
}
-#if FULL_SAFETY || CONFIG_UNICODE_SAFETY
/* Test incomplete/invalid 1-unit input. */
{
static const uint32_t input[] = { 0x340000 };
@@ -73,5 +72,4 @@ test_function (int (*my_u32_mbtouc) (ucs4_t *, const uint32_t *, size_t))
ASSERT (ret == 1);
ASSERT (uc == 0xFFFD);
}
-#endif
}
diff --git a/tests/unistr/test-u32-next.c b/tests/unistr/test-u32-next.c
index b7116b3..3b87804 100644
--- a/tests/unistr/test-u32-next.c
+++ b/tests/unistr/test-u32-next.c
@@ -71,7 +71,6 @@ main ()
ASSERT (uc == 0x1D51F);
}
-#if CONFIG_UNICODE_SAFETY
/* Test incomplete/invalid 1-unit input. */
{
static const uint32_t input[] = { 0x340000, 0 };
@@ -80,7 +79,6 @@ main ()
ASSERT (ret == NULL);
ASSERT (uc == 0xFFFD);
}
-#endif
return 0;
}
diff --git a/tests/unistr/test-u32-prev.c b/tests/unistr/test-u32-prev.c
index 5f9cf43..cdcda94 100644
--- a/tests/unistr/test-u32-prev.c
+++ b/tests/unistr/test-u32-prev.c
@@ -56,7 +56,6 @@ check (const uint32_t *input, size_t input_length, ucs4_t *puc)
static int
check_invalid (const uint32_t *input, size_t input_length)
{
-#if CONFIG_UNICODE_SAFETY
ucs4_t uc;
/* Test recognition when at the beginning of the string. */
@@ -83,7 +82,6 @@ check_invalid (const uint32_t *input, size_t input_length)
if (uc != 0xBADFACE)
return 4;
}
-#endif
return 0;
}
diff --git a/tests/unistr/test-u32-strmblen.c b/tests/unistr/test-u32-strmblen.c
index ae87b9b..80626b8 100644
--- a/tests/unistr/test-u32-strmblen.c
+++ b/tests/unistr/test-u32-strmblen.c
@@ -62,14 +62,12 @@ main ()
ASSERT (ret == 1);
}
-#if CONFIG_UNICODE_SAFETY
/* Test incomplete/invalid 1-unit input. */
{
static const uint32_t input[] = { 0x340000, 0 };
ret = u32_strmblen (input);
ASSERT (ret == -1);
}
-#endif
return 0;
}
diff --git a/tests/unistr/test-u32-strmbtouc.c b/tests/unistr/test-u32-strmbtouc.c
index 258c491..fa5311e 100644
--- a/tests/unistr/test-u32-strmbtouc.c
+++ b/tests/unistr/test-u32-strmbtouc.c
@@ -71,7 +71,6 @@ main ()
ASSERT (uc == 0x1D51F);
}
-#if CONFIG_UNICODE_SAFETY
/* Test incomplete/invalid 1-unit input. */
{
static const uint32_t input[] = { 0x340000, 0 };
@@ -80,7 +79,6 @@ main ()
ASSERT (ret == -1);
ASSERT (uc == 0xBADFACE);
}
-#endif
return 0;
}
--
2.6.4
