>> Hi Danny. Please could you enhance your tests so that they can detect
>> this error (so that they will fail if this error is present).
>
> Sometimes I should use more words.
>
> What I meant to say is: I'm trying to devote as much of my time as
> possible to other things (WC-NG specifically) so if you could help me
> out as much as possible with reducing the review effort needed, I'd
> really appreciate it. I assume your test didn't pick up this error
> already, and I can't see at a glance whether you have extended the test
> so that it does. If you do that, then that'll save me (and anyone now
> and forever) from having to scrutinize the code to see whether it
> handles this properly. That precisely what a regression test is for.
>
> So if you already extended it, apologies as I haven't tried it or
> examined it, as I wanted to ask you first.
>
> Also please include the log message each time you attach a patch.
> Please could you re-post the patch and log message together even if no
> further changes are needed.
>
> Thanks,
> - Julian
Hi Julian,
I have already looked into adding a regression test for this, but it
is not possible even after applying the patch. The only way that the
problem would manifest is if one of the other svn_subst_translate*
functions that allow EOL_STR to be specified were also extended to
accept a TRANSLATED_LINE_ENDINGS parameter. For example, if
svn_subst_translate_cstring2() were extended in this manner, "\r" was
used as the EOL string, and the value to be translated included only
Windows-style line endings, then the problem would occur. With this
patch, only svn_subst_translate_string() is extended, and
svn_subst_translate_string() always uses "\n" for EOL_STR.
Attached is the patch and log message.
Index: build.conf
===================================================================
--- build.conf (revision 1050825)
+++ build.conf (working copy)
@@ -865,6 +865,14 @@ sources = target-test.c
install = test
libs = libsvn_test libsvn_subr apriconv apr
+[subst_translate-test]
+description = Test the svn_subst_translate* functions
+type = exe
+path = subversion/tests/libsvn_subr
+sources = subst_translate-test.c
+install = test
+libs = libsvn_test libsvn_subr apriconv apr
+
[translate-test]
description = Test eol conversion and keyword substitution routines
type = exe
Index: subversion/include/svn_subst.h
===================================================================
--- subversion/include/svn_subst.h (revision 1050825)
+++ subversion/include/svn_subst.h (working copy)
@@ -592,19 +592,46 @@ svn_subst_stream_detranslated(svn_stream_t **strea
/* EOL conversion and character encodings */
+/** Similar to svn_subst_translate_string2(), except that the information about
+ * whether re-encoding or line ending translation were performed is discarded.
+ *
+ * @deprecated Provided for backward compatibility with the 1.6 API.
+ */
+SVN_DEPRECATED
+svn_error_t *svn_subst_translate_string(svn_string_t **new_value,
+ const svn_string_t *value,
+ const char *encoding,
+ apr_pool_t *pool);
+
/** Translate the string @a value from character encoding @a encoding to
* UTF8, and also from its current line-ending style to LF line-endings. If
* @a encoding is @c NULL, translate from the system-default encoding.
*
+ * If @a translated_to_utf8 is not @c NULL, then set @a *translated_to_utf8
+ * to @c TRUE if at least one character of @a value in the source character
+ * encoding was translated to UTF-8, or to @c FALSE otherwise.
+ *
+ * If @a translated_line_endings is not @c NULL, then set @a
+ * *translated_line_endings to @c TRUE if at least one line ending was
+ * changed to LF, or to @c FALSE otherwise.
+ *
* Recognized line endings are LF, CR, CRLF. If @a value has inconsistent
* line endings, return @c SVN_ERR_IO_INCONSISTENT_EOL.
*
- * Set @a *new_value to the translated string, allocated in @a pool.
+ * Set @a *new_value to the translated string, allocated in @a result_pool.
+ *
+ * @a scratch_pool is used for temporary allocations.
+ *
+ * @since New in 1.7.
*/
-svn_error_t *svn_subst_translate_string(svn_string_t **new_value,
- const svn_string_t *value,
- const char *encoding,
- apr_pool_t *pool);
+svn_error_t *
+svn_subst_translate_string2(svn_string_t **new_value,
+ svn_boolean_t *translated_to_utf8,
+ svn_boolean_t *translated_line_endings,
+ const svn_string_t *value,
+ const char *encoding,
+ apr_pool_t *result_pool,
+ apr_pool_t *scratch_pool);
/** Translate the string @a value from UTF8 and LF line-endings into native
* character encoding and native line-endings. If @a for_output is TRUE,
Index: subversion/libsvn_subr/subst.c
===================================================================
--- subversion/libsvn_subr/subst.c (revision 1050825)
+++ subversion/libsvn_subr/subst.c (working copy)
@@ -606,11 +606,34 @@ translate_keyword(char *buf,
return FALSE;
}
+/* A boolean expression that evaluates to true if the first STR_LEN characters
+ of the string STR are one of the end-of-line strings LF, CR, or CRLF;
+ to false otherwise. */
+#define STRING_IS_EOL(str, str_len) \
+ (((str_len) == 2 && (str)[0] == '\r' && (str)[1] == '\n') || \
+ ((str_len) == 1 && ((str)[0] == '\n' || (str)[0] == '\r')))
+/* A boolean expression that evaluates to true if the end-of-line string EOL1,
+ having length EOL1_LEN, and the end-of-line string EOL2, having length
+ EOL2_LEN, are different, assuming that EOL1 and EOL2 are both from the
+ set {"\n", "\r", "\r\n"}; to false otherwise.
+
+ Given that EOL1 and EOL2 are either "\n", "\r", or "\r\n", then if
+ EOL1_LEN is not the same as EOL2_LEN, then EOL1 and EOL2 are of course
+ different. If EOL1_LEN and EOL2_LEN are both 2 then EOL1 and EOL2 are both
+ "\r\n" and *EOL1 == *EOL2. Otherwise, EOL1_LEN and EOL2_LEN are both 1.
+ We need only check the one character for equality to determine whether
+ EOL1 and EOL2 are different in that case. */
+#define DIFFERENT_EOL_STRINGS(eol1, eol1_len, eol2, eol2_len) \
+ (((eol1_len) != (eol2_len)) || (*(eol1) != *(eol2)))
+
+
/* Translate the newline string NEWLINE_BUF (of length NEWLINE_LEN) to
the newline string EOL_STR (of length EOL_STR_LEN), writing the
result (which is always EOL_STR) to the stream DST.
+ This function assumes that NEWLINE_BUF is either "\n", "\r", or "\r\n".
+
Also check for consistency of the source newline strings across
multiple calls, using SRC_FORMAT (length *SRC_FORMAT_LEN) as a cache
of the first newline found. If the current newline is not the same
@@ -620,6 +643,11 @@ translate_keyword(char *buf,
newline in the file, and copy it to {SRC_FORMAT, *SRC_FORMAT_LEN} to
use for later consistency checks.
+ If TRANSLATED_EOL is not NULL, then set *TRANSLATED_EOL to TRUE if the
+ newline string that was written (EOL_STR) is not the same as the newline
+ string that was translated (NEWLINE_BUF), otherwise leave *TRANSLATED_EOL
+ untouched.
+
Note: all parameters are required even if REPAIR is TRUE.
### We could require that REPAIR must not change across a sequence of
calls, and could then optimize by not using SRC_FORMAT at all if
@@ -633,17 +661,19 @@ translate_newline(const char *eol_str,
const char *newline_buf,
apr_size_t newline_len,
svn_stream_t *dst,
+ svn_boolean_t *translated_eol,
svn_boolean_t repair)
{
+ SVN_ERR_ASSERT(STRING_IS_EOL(newline_buf, newline_len));
+
/* If we've seen a newline before, compare it with our cache to
check for consistency, else cache it for future comparisons. */
if (*src_format_len)
{
/* Comparing with cache. If we are inconsistent and
we are NOT repairing the file, generate an error! */
- if ((! repair) &&
- ((*src_format_len != newline_len) ||
- (strncmp(src_format, newline_buf, newline_len))))
+ if ((! repair) && DIFFERENT_EOL_STRINGS(src_format, *src_format_len,
+ newline_buf, newline_len))
return svn_error_create(SVN_ERR_IO_INCONSISTENT_EOL, NULL, NULL);
}
else
@@ -653,8 +683,16 @@ translate_newline(const char *eol_str,
strncpy(src_format, newline_buf, newline_len);
*src_format_len = newline_len;
}
+
/* Write the desired newline */
- return translate_write(dst, eol_str, eol_str_len);
+ SVN_ERR(translate_write(dst, eol_str, eol_str_len));
+
+ if (translated_eol != NULL &&
+ (eol_str_len != newline_len ||
+ memcmp(eol_str, newline_buf, eol_str_len) != 0))
+ *translated_eol = TRUE;
+
+ return SVN_NO_ERROR;
}
@@ -765,10 +803,12 @@ svn_subst_keywords_differ2(apr_hash_t *a,
return FALSE;
}
+
/* Baton for translate_chunk() to store its state in. */
struct translation_baton
{
const char *eol_str;
+ svn_boolean_t *translated_eol;
svn_boolean_t repair;
apr_hash_t *keywords;
svn_boolean_t expand;
@@ -813,6 +853,7 @@ struct translation_baton
*/
static struct translation_baton *
create_translation_baton(const char *eol_str,
+ svn_boolean_t *translated_eol,
svn_boolean_t repair,
apr_hash_t *keywords,
svn_boolean_t expand,
@@ -826,6 +867,7 @@ create_translation_baton(const char *eol_str,
b->eol_str = eol_str;
b->eol_str_len = eol_str ? strlen(eol_str) : 0;
+ b->translated_eol = translated_eol;
b->repair = repair;
b->keywords = keywords;
b->expand = expand;
@@ -888,6 +930,9 @@ eol_unchanged(struct translation_baton *b,
* To finish a series of chunk translations, flush all buffers by calling
* this routine with a NULL value for BUF.
*
+ * If B->translated_eol is not NULL, then set *B->translated_eol to TRUE if
+ * an end-of-line sequence was changed, otherwise leave it untouched.
+ *
* Use POOL for temporary allocations.
*/
static svn_error_t *
@@ -924,7 +969,8 @@ translate_chunk(svn_stream_t *dst,
SVN_ERR(translate_newline(b->eol_str, b->eol_str_len,
b->src_format,
&b->src_format_len, b->newline_buf,
- b->newline_off, dst, b->repair));
+ b->newline_off, dst, b->translated_eol,
+ b->repair));
b->newline_off = 0;
}
@@ -1070,7 +1116,8 @@ translate_chunk(svn_stream_t *dst,
b->src_format,
&b->src_format_len,
b->newline_buf,
- b->newline_off, dst, b->repair));
+ b->newline_off, dst,
+ b->translated_eol, b->repair));
b->newline_off = 0;
break;
@@ -1086,7 +1133,7 @@ translate_chunk(svn_stream_t *dst,
SVN_ERR(translate_newline(b->eol_str, b->eol_str_len,
b->src_format, &b->src_format_len,
b->newline_buf, b->newline_off,
- dst, b->repair));
+ dst, b->translated_eol, b->repair));
b->newline_off = 0;
}
@@ -1349,14 +1396,20 @@ svn_subst_read_specialfile(svn_stream_t **stream,
return SVN_NO_ERROR;
}
-
-svn_stream_t *
-svn_subst_stream_translated(svn_stream_t *stream,
- const char *eol_str,
- svn_boolean_t repair,
- apr_hash_t *keywords,
- svn_boolean_t expand,
- apr_pool_t *result_pool)
+/* Same as svn_subst_stream_translated(), except for the following.
+ *
+ * If TRANSLATED_EOL is not NULL, then reading and/or writing to the stream
+ * will set *TRANSLATED_EOL to TRUE if an end-of-line sequence was changed,
+ * otherwise leave it untouched.
+ */
+static svn_stream_t *
+stream_translated(svn_stream_t *stream,
+ const char *eol_str,
+ svn_boolean_t *translated_eol,
+ svn_boolean_t repair,
+ apr_hash_t *keywords,
+ svn_boolean_t expand,
+ apr_pool_t *result_pool)
{
struct translated_stream_baton *baton
= apr_palloc(result_pool, sizeof(*baton));
@@ -1398,9 +1451,11 @@ svn_subst_read_specialfile(svn_stream_t **stream,
/* Setup the baton fields */
baton->stream = stream;
baton->in_baton
- = create_translation_baton(eol_str, repair, keywords, expand, result_pool);
+ = create_translation_baton(eol_str, translated_eol, repair, keywords,
+ expand, result_pool);
baton->out_baton
- = create_translation_baton(eol_str, repair, keywords, expand, result_pool);
+ = create_translation_baton(eol_str, translated_eol, repair, keywords,
+ expand, result_pool);
baton->written = FALSE;
baton->readbuf = svn_stringbuf_create("", result_pool);
baton->readbuf_off = 0;
@@ -1417,15 +1472,32 @@ svn_subst_read_specialfile(svn_stream_t **stream,
return s;
}
+svn_stream_t *
+svn_subst_stream_translated(svn_stream_t *stream,
+ const char *eol_str,
+ svn_boolean_t repair,
+ apr_hash_t *keywords,
+ svn_boolean_t expand,
+ apr_pool_t *result_pool)
+{
+ return stream_translated(stream, eol_str, NULL, repair, keywords, expand,
+ result_pool);
+}
-svn_error_t *
-svn_subst_translate_cstring2(const char *src,
- const char **dst,
- const char *eol_str,
- svn_boolean_t repair,
- apr_hash_t *keywords,
- svn_boolean_t expand,
- apr_pool_t *pool)
+/* Same as svn_subst_translate_cstring2(), except for the following.
+ *
+ * If TRANSLATED_EOL is not NULL, then set *TRANSLATED_EOL to TRUE if an
+ * end-of-line sequence was changed, or to FALSE otherwise.
+ */
+static svn_error_t *
+translate_cstring(const char **dst,
+ svn_boolean_t *translated_eol,
+ const char *src,
+ const char *eol_str,
+ svn_boolean_t repair,
+ apr_hash_t *keywords,
+ svn_boolean_t expand,
+ apr_pool_t *pool)
{
svn_stringbuf_t *dst_stringbuf;
svn_stream_t *dst_stream;
@@ -1442,9 +1514,12 @@ svn_subst_read_specialfile(svn_stream_t **stream,
dst_stringbuf = svn_stringbuf_create("", pool);
dst_stream = svn_stream_from_stringbuf(dst_stringbuf, pool);
+ if (translated_eol)
+ *translated_eol = FALSE;
+
/* Another wrapper to translate the content. */
- dst_stream = svn_subst_stream_translated(dst_stream, eol_str, repair,
- keywords, expand, pool);
+ dst_stream = stream_translated(dst_stream, eol_str, translated_eol, repair,
+ keywords, expand, pool);
/* Jam the text into the destination stream (to translate it). */
SVN_ERR(svn_stream_write(dst_stream, src, &len));
@@ -1456,6 +1531,19 @@ svn_subst_read_specialfile(svn_stream_t **stream,
return SVN_NO_ERROR;
}
+svn_error_t *
+svn_subst_translate_cstring2(const char *src,
+ const char **dst,
+ const char *eol_str,
+ svn_boolean_t repair,
+ apr_hash_t *keywords,
+ svn_boolean_t expand,
+ apr_pool_t *pool)
+{
+ return translate_cstring(dst, NULL, src, eol_str, repair, keywords, expand,
+ pool);
+}
+
/* Given a special file at SRC, generate a textual representation of
it in a normal file at DST. Perform all allocations in POOL. */
/* ### this should be folded into svn_subst_copy_and_translate3 */
@@ -1768,14 +1856,16 @@ svn_subst_stream_from_specialfile(svn_stream_t **s
/*** String translation */
svn_error_t *
-svn_subst_translate_string(svn_string_t **new_value,
- const svn_string_t *value,
- const char *encoding,
- apr_pool_t *pool)
+svn_subst_translate_string2(svn_string_t **new_value,
+ svn_boolean_t *translated_to_utf8,
+ svn_boolean_t *translated_line_endings,
+ const svn_string_t *value,
+ const char *encoding,
+ apr_pool_t *result_pool,
+ apr_pool_t *scratch_pool)
{
const char *val_utf8;
const char *val_utf8_lf;
- apr_pool_t *scratch_pool = svn_pool_create(pool);
if (value == NULL)
{
@@ -1793,16 +1883,19 @@ svn_error_t *
SVN_ERR(svn_utf_cstring_to_utf8(&val_utf8, value->data, scratch_pool));
}
- SVN_ERR(svn_subst_translate_cstring2(val_utf8,
- &val_utf8_lf,
- "\n", /* translate to LF */
- FALSE, /* no repair */
- NULL, /* no keywords */
- FALSE, /* no expansion */
- scratch_pool));
+ if (translated_to_utf8)
+ *translated_to_utf8 = (strcmp(value->data, val_utf8) != 0);
- *new_value = svn_string_create(val_utf8_lf, pool);
- svn_pool_destroy(scratch_pool);
+ SVN_ERR(translate_cstring(&val_utf8_lf,
+ translated_line_endings,
+ val_utf8,
+ "\n", /* translate to LF */
+ FALSE, /* no repair */
+ NULL, /* no keywords */
+ FALSE, /* no expansion */
+ scratch_pool));
+
+ *new_value = svn_string_create(val_utf8_lf, result_pool);
return SVN_NO_ERROR;
}
Index: subversion/libsvn_subr/deprecated.c
===================================================================
--- subversion/libsvn_subr/deprecated.c (revision 1050825)
+++ subversion/libsvn_subr/deprecated.c (working copy)
@@ -250,6 +250,16 @@ svn_subst_stream_translated_to_normal_form(svn_str
}
svn_error_t *
+svn_subst_translate_string(svn_string_t **new_value,
+ const svn_string_t *value,
+ const char *encoding,
+ apr_pool_t *pool)
+{
+ return svn_subst_translate_string2(new_value, NULL, NULL, value,
+ encoding, pool, pool);
+}
+
+svn_error_t *
svn_subst_stream_detranslated(svn_stream_t **stream_p,
const char *src,
svn_subst_eol_style_t eol_style,
Index: subversion/tests/libsvn_subr/subst_translate-test.c
===================================================================
--- subversion/tests/libsvn_subr/subst_translate-test.c (revision 0)
+++ subversion/tests/libsvn_subr/subst_translate-test.c (revision 0)
@@ -0,0 +1,313 @@
+/*
+ * subst_translate-test.c -- test the svn_subst_translate* functions
+ *
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ */
+
+#include <stddef.h>
+#include <string.h>
+
+#include <apr.h>
+#include <apr_errno.h>
+#include <apr_general.h>
+
+#include "../svn_test.h"
+
+#include "svn_types.h"
+#include "svn_error_codes.h"
+#include "svn_error.h"
+#include "svn_pools.h"
+#include "svn_string.h"
+#include "svn_subst.h"
+
+static char s_buf[2053];
+
+/**
+ * Converts the string STR to C code for a string literal that represents it.
+ *
+ * Not thread safe. The returned pointer is to a static buffer, so it does
+ * not need to be freed.
+ */
+static char*
+strtocsrc(const char *str)
+{
+ char *p = s_buf;
+ *p++ = '"';
+
+ size_t i;
+ for (i = 0; str[i] != '\0' && i < 512; ++i) {
+ sprintf(p, "\\x%02x", (int) str[i]);
+ p += 4;
+ }
+
+ if (i < 512) {
+ *p++ = '"';
+ } else {
+ *p++ = '.';
+ *p++ = '.';
+ *p++ = '.';
+ // no terminating double quote character
+ }
+
+ *p = '\0';
+
+ return s_buf;
+}
+
+static svn_error_t *
+test_svn_subst_translate_string2(apr_pool_t *pool)
+{
+ svn_string_t *new_value;
+ svn_boolean_t translated_to_utf8, translated_line_endings;
+
+ /* No reencoding, no translation of line endings */
+ const char data0[] = "abcdefz";
+ svn_string_t *string0 = svn_string_create(data0, pool);
+ new_value = NULL;
+ translated_line_endings = TRUE;
+ SVN_ERR(svn_subst_translate_string2(&new_value,
+ NULL, &translated_line_endings,
+ string0, "UTF-8", pool, pool));
+ if (strcmp(new_value->data, data0) != 0)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "svn_subst_translate_string2() on STRING0 should "
+ "yield \"abcdefz\".");
+ if (translated_line_endings)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "svn_subst_translate_string2() on STRING0 should "
+ "reset TRANSLATED_LINE_ENDINGS to FALSE because "
+ "the string does not contain a new line to "
+ "translate.");
+ new_value = NULL;
+ translated_to_utf8 = TRUE;
+ translated_line_endings = TRUE;
+ SVN_ERR(svn_subst_translate_string2(&new_value, &translated_to_utf8,
+ &translated_line_endings,
+ string0, "ISO-8859-1", pool, pool));
+ if (strcmp(new_value->data, data0) != 0)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "(2) svn_subst_translate_string2() on STRING0 "
+ "should yield \"abcdefz\".");
+ if (translated_to_utf8)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "svn_subst_translate_string2() on STRING0 should "
+ "reset TRANSLATED_TO_UTF8 to FALSE because "
+ "the string should not be changed as a result of "
+ "reencoding to UTF-8.");
+ if (translated_line_endings)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "(2) svn_subst_translate_string2() on STRING0 "
+ "should reset TRANSLATED_LINE_ENDINGS to FALSE "
+ "because the string does not contain a new line "
+ "to translate.");
+
+ /* No reencoding, translation of line endings */
+ const char data1[] = " \r\n\r\n \r\n \r\n";
+ svn_string_t *string1 = svn_string_create(data1, pool);
+ const char expected_result1[] = " \n\n \n \n";
+ new_value = NULL;
+ translated_line_endings = FALSE;
+ SVN_ERR(svn_subst_translate_string2(&new_value,
+ NULL, &translated_line_endings,
+ string1, "UTF-8", pool, pool));
+ if (strcmp(new_value->data, expected_result1) != 0)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "svn_subst_translate_string2() on STRING1 should "
+ "yield \" \\n\\n \\n \\n\".");
+ if (! translated_line_endings)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "svn_subst_translate_string2() on STRING1 should "
+ "set TRANSLATED_LINE_ENDINGS to TRUE because the "
+ "string has Windows-style line endings that "
+ "were translated.");
+ new_value = NULL;
+ translated_to_utf8 = TRUE;
+ translated_line_endings = FALSE;
+ SVN_ERR(svn_subst_translate_string2(&new_value, &translated_to_utf8,
+ &translated_line_endings,
+ string1, "ISO-8859-1", pool, pool));
+ if (strcmp(new_value->data, expected_result1) != 0)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "(2) svn_subst_translate_string2() on STRING1 "
+ "should yield \" \\n\\n \\n \\n\""
+ ".");
+ if (translated_to_utf8)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "svn_subst_translate_string2() on STRING1 should "
+ "reset TRANSLATED_TO_UTF8 to FALSE because "
+ "the string should not be changed as a result of "
+ "reencoding to UTF-8.");
+ if (! translated_line_endings)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "(2) svn_subst_translate_string2() on STRING1 "
+ "should set TRANSLATED_LINE_ENDINGS to TRUE "
+ "because the string has Windows-style line "
+ "endings that were translated.");
+
+ /* Reencoding, no translation of line endings */
+ const char data2[] = "\xc7\xa9\xf4\xdf";
+ svn_string_t *string2 = svn_string_create(data2, pool);
+ const char expected_result2[] = "\xc3\x87\xc2\xa9\xc3\xb4\xc3\x9f";
+ new_value = NULL;
+ translated_to_utf8 = FALSE;
+ SVN_ERR(svn_subst_translate_string2(&new_value,
+ &translated_to_utf8, NULL,
+ string2, "ISO-8859-1", pool, pool));
+ if (strcmp(new_value->data, expected_result2) != 0)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "svn_subst_translate_string2() on STRING2 should "
+ "yield \"\\xc3\\x87\\xc2\\xa9\\xc3\\xb4\\xc3\\x9f"
+ "\". Instead, got %s.",
+ strtocsrc(new_value->data));
+ if (! translated_to_utf8)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "svn_subst_translate_string2() on STRING2 should "
+ "set TRANSLATED_TO_UTF8 to TRUE.");
+ new_value = NULL;
+ translated_to_utf8 = FALSE;
+ translated_line_endings = TRUE;
+ SVN_ERR(svn_subst_translate_string2(&new_value, &translated_to_utf8,
+ &translated_line_endings,
+ string2, "ISO-8859-1", pool, pool));
+ if (strcmp(new_value->data, expected_result2) != 0)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "(2) svn_subst_translate_string2() on STRING2 "
+ "should yield \"\\xc3\\x87\\xc2\\xa9\\xc3\\xb4"
+ "\\xc3\\x9f\". Instead, got %s.",
+ strtocsrc(new_value->data));
+ if (! translated_to_utf8)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "(2) svn_subst_translate_string2() on STRING2 "
+ "should set TRANSLATED_TO_UTF8 to TRUE.");
+ if (translated_line_endings)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "svn_subst_translate_string2() on STRING2 should "
+ "reset TRANSLATED_LINE_ENDINGS to FALSE.");
+
+ /* Reencoding, translation of line endings */
+ const char data3[] = "\xc7\xa9\xf4\xdf\r\n";
+ svn_string_t *string3 = svn_string_create(data3, pool);
+ const char expected_result3[] = "\xc3\x87\xc2\xa9\xc3\xb4\xc3\x9f\n";
+ new_value = NULL;
+ translated_to_utf8 = FALSE;
+ SVN_ERR(svn_subst_translate_string2(&new_value,
+ &translated_to_utf8, NULL,
+ string3, "ISO-8859-1", pool, pool));
+ if (strcmp(new_value->data, expected_result3) != 0)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "svn_subst_translate_string2() on STRING3 should "
+ "yield \"\\xc3\\x87\\xc2\\xa9\\xc3\\xb4\\xc3\\x9f"
+ "\\x0a\". Instead, got %s.",
+ strtocsrc(new_value->data));
+ if (! translated_to_utf8)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "svn_subst_translate_string2() on STRING3 should "
+ "set TRANSLATED_TO_UTF8 to TRUE.");
+ new_value = NULL;
+ translated_to_utf8 = FALSE;
+ translated_line_endings = FALSE;
+ SVN_ERR(svn_subst_translate_string2(&new_value, &translated_to_utf8,
+ &translated_line_endings,
+ string3, "ISO-8859-1", pool, pool));
+ if (strcmp(new_value->data, expected_result3) != 0)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "(2) svn_subst_translate_string2() on STRING3 "
+ "should yield \"\\xc3\\x87\\xc2\\xa9\\xc3\\xb4"
+ "\\xc3\\x9f\\x0a\". Instead, got %s.",
+ strtocsrc(new_value->data));
+ if (! translated_to_utf8)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "(2) svn_subst_translate_string2() on STRING3 "
+ "should set TRANSLATED_TO_UTF8 to TRUE.");
+ if (! translated_line_endings)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "svn_subst_translate_string2() on STRING3 should "
+ "set TRANSLATED_LINE_ENDINGS to TRUE.");
+
+ return SVN_NO_ERROR;
+}
+
+/*static svn_error_t *
+test_svn_subst_copy_and_translate4(apr_pool_t *pool)
+{
+ return SVN_NO_ERROR;
+}
+
+static svn_error_t *
+test_svn_subst_stream_translated(apr_pool_t *pool)
+{
+ return SVN_NO_ERROR;
+}*/
+
+static svn_error_t *
+test_svn_subst_translate_cstring2(apr_pool_t *pool)
+{
+ /* Test the unusual case where EOL_STR is an empty string. */
+ const char src0[] = " \r \n\r\n \n\n\n";
+ const char *dest0 = NULL;
+ const char expected_result0[] = " ";
+ SVN_ERR(svn_subst_translate_cstring2(src0, &dest0, "", TRUE, NULL, FALSE,
+ pool));
+ if (strcmp(dest0, expected_result0) != 0)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "svn_subst_translate_cstring2() on SRC0 should "
+ "yield \" \".");
+
+ /* Test the unusual case where EOL_STR is not a standard EOL string. */
+ const char src1[] = " \r \n\r\n \n\n\n";
+ const char *dest1 = NULL;
+ const char expected_result1[] = " z zz zzz";
+ SVN_ERR(svn_subst_translate_cstring2(src1, &dest1, "z", TRUE, NULL, FALSE,
+ pool));
+ if (strcmp(dest1, expected_result1) != 0)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "svn_subst_translate_cstring2() on SRC1 should "
+ "yield \" z zz zzz\".");
+ const char src2[] = " \n \n ";
+ const char *dest2 = NULL;
+ const char expected_result2[] = " buzz buzz ";
+ SVN_ERR(svn_subst_translate_cstring2(src2, &dest2, "buzz", FALSE, NULL,
FALSE,
+ pool));
+ if (strcmp(dest2, expected_result2) != 0)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "svn_subst_translate_cstring2() on SRC2 should "
+ "yield \" buzz buzz \".");
+ const char src3[] = " \r\n \n";
+ const char *dest3 = NULL;
+ const char expected_result3[] = " buzz buzz";
+ SVN_ERR(svn_subst_translate_cstring2(src3, &dest3, "buzz", TRUE, NULL, FALSE,
+ pool));
+ if (strcmp(dest3, expected_result3) != 0)
+ return svn_error_createf(SVN_ERR_TEST_FAILED, NULL,
+ "svn_subst_translate_cstring2() on SRC3 should "
+ "yield \" buzz buzz\".");
+
+ return SVN_NO_ERROR;
+}
+
+struct svn_test_descriptor_t test_funcs[] =
+ {
+ SVN_TEST_NULL,
+ SVN_TEST_PASS2(test_svn_subst_translate_string2,
+ "test svn_subst_translate_string2()"),
+ SVN_TEST_PASS2(test_svn_subst_translate_cstring2,
+ "test svn_subst_translate_cstring2()"),
+ SVN_TEST_NULL
+ };
Property changes on: subversion/tests/libsvn_subr/subst_translate-test.c
___________________________________________________________________
Added: svn:eol-style
+ native
[[[
Add a public API function, svn_subst_translate_string2(), an extension of
svn_subst_translate_string(), that has two additional output parameters for
determining whether re-encoding and/or line ending translation were performed.
As discussed at:
http://thread.gmane.org/gmane.comp.version-control.subversion.devel/122550
http://thread.gmane.org/gmane.comp.version-control.subversion.devel/123020
The essential changes are to the translate_newline() function, which now takes
an svn_boolean_t pointer, the value at which is set to TRUE if the pointer is
non-NULL and a different newline is written out. Most other changes are to pass
the svn_boolean_t pointer through to translate_newline().
* build.conf
Configure subversion/tests/libsvn_subr/subst_translate-test
* subversion/include/svn_subst.h
(svn_subst_translate_string2): New function.
(svn_subst_translate_string): Deprecate in favor of
svn_subst_translate_string2().
* subversion/libsvn_subr/subst.c
(STRING_IS_EOL): New macro that tests whether a string is an end-of-line
string ("\n", "\r", "\r\n").
(DIFFERENT_EOL_STRINGS): New macro that tests whether two end-of-line strings
are different.
(translate_newline): Add the TRANSLATED_EOL parameter. If the function
writes out a different newline, then it sets TRANSLATED_EOL to TRUE.
(translation_baton): Add the TRANSLATED_EOL field.
(create_translation_baton): Add a new parameter TRANSLATED_EOL that is
passed to the resulting translation_baton.
(translate_chunk): When calling translate_newline(), pass TRANSLATED_EOL from
the translation_baton.
(stream_translated): New static function. Its implementation is the old
implementation of svn_subst_stream_translated(), but accepting another
parameter, TRANSLATED_EOL, that is passed to the in/out translation batons
that it creates.
(svn_subst_stream_translated): Now a wrapper for stream_translated().
(translate_cstring): New static function. Its implementation is the old
implementation of svn_subst_translate_cstring2(), but modified to accept
another parameter, TRANSLATED_EOL, that is passed to stream_translated().
(svn_subst_translate_cstring2): Now a wrapper for translate_cstring().
(svn_subst_translate_string): Move to deprecated.c.
(svn_subst_translate_string2): New function. It takes three additional
parameters: TRANSLATED_TO_UTF8, TRANSLATED_LINE_ENDINGS, and another pool
parameter. The task of recording whether it translates a line ending is
delegated to translate_cstring().
* subversion/libsvn_subr/deprecated.c
(svn_subst_translate_string): Now a wrapper for svn_subst_translate_string2().
* subversion/tests/libsvn_subr/subst_translate-test.c
New test suite for the svn_subst_translate_* functions.
]]]
