Re: [GENERAL] invalid UTF-8 byte sequence detected

2006-03-15 Thread Wes
On 3/16/06 12:41 AM, "Michael Glaesemann" <[EMAIL PROTECTED]> wrote: > I hope you didn't take my comments as a suggestion: I think you > *should* preprocess your data and use UTF8 as the default encoding > (or another encoding that includes all of the characters you hope to > use) for your databas

Re: [GENERAL] invalid UTF-8 byte sequence detected

2006-03-15 Thread Michael Glaesemann
On Mar 16, 2006, at 15:39 , Wes wrote: That might be worth a shot. I don't really understand the ramifications, though, especially given Tom's warning. I guess as long as I don't care about things like sort order for those fields, it may not matter much. I hope you didn't take my commen

Re: [GENERAL] invalid UTF-8 byte sequence detected

2006-03-15 Thread Wes
On 3/16/06 12:13 AM, "Michael Glaesemann" <[EMAIL PROTECTED]> wrote: > SQL_ASCII is *not* ASCII. See the "What's a good default encoding?" > thread on this same list from today. > > http://archives.postgresql.org/pgsql-general/2006-03/msg00685.php > > I don't believe it will discard anything on

Re: [GENERAL] invalid UTF-8 byte sequence detected

2006-03-15 Thread Michael Glaesemann
On Mar 16, 2006, at 14:42 , mike wrote: If you don't care to store those characters then maybe you should change the database character set to use SQL_ASCII instead of UTF-8. I believe ASCII will quietly discard those characters when converting from UTF-8. SQL_ASCII is *not* ASCII. See

Re: [GENERAL] invalid UTF-8 byte sequence detected

2006-03-15 Thread Wes
On 3/15/06 11:42 PM, "mike" <[EMAIL PROTECTED]> wrote: > If you don't care to store those characters then maybe you should change > the database character set to use SQL_ASCII instead of UTF-8. I believe > ASCII will quietly discard those characters when converting from UTF-8. I thought about th

Re: [GENERAL] invalid UTF-8 byte sequence detected

2006-03-15 Thread mike
If you don't care to store those characters then maybe you should change the database character set to use SQL_ASCII instead of UTF-8. I believe ASCII will quietly discard those characters when converting from UTF-8. Mike On Wed, 2006-03-15 at 23:20 -0600, Wes wrote: > Based on a couple of othe

[GENERAL] invalid UTF-8 byte sequence detected

2006-03-15 Thread Wes
Based on a couple of other posts, I think I'm out of luck, but I'm hoping something might have changed recently. I'm loading a very high volume of data with COPY using libpq - about 100+ million rows per day. The problem is that the COPY sometimes aborts with invalid UTF-8 byte sequence detecte

Re: [GENERAL] invalid UTF-8 byte sequence detected

2005-11-16 Thread Markus Wollny
Hi! > -Ursprüngliche Nachricht- > Von: Bruce Momjian [mailto:[EMAIL PROTECTED] > Gesendet: Dienstag, 15. November 2005 19:46 > An: Markus Wollny > Cc: pgsql-general@postgresql.org > Betreff: Re: [GENERAL] invalid UTF-8 byte sequence detected > > > I am als

Re: [GENERAL] invalid UTF-8 byte sequence detected

2005-11-15 Thread Bruce Momjian
I am also confused how invalid UTF8 sequences got into your database. It shouldn't have been possible. --- Markus Wollny wrote: > Hello! > > I am currently testdriving migration of our PostgreSQL 8.0 databases to 8.1; > i

[GENERAL] invalid UTF-8 byte sequence detected

2005-11-09 Thread Markus Wollny
Hello! I am currently testdriving migration of our PostgreSQL 8.0 databases to 8.1; in this process I have stumbled a couple of times over certain errors in text-fields that lead to error-messages during import of the dump like these: <2005-11-09 14:57:34 CET - 9354: [EMAIL PROTECTED]>ERROR: i