Re: [GENERAL] invalid byte sequence for encoding "UTF8": 0x00

2011-11-20 Thread hubert depesz lubaczewski
On Sat, Nov 19, 2011 at 09:32:12AM -0800, pawel_kukawski wrote: > Is there any way I can store NULL character (\u) in string ? > > Or there is only one option that I have change every text field to bytea. correct question is: why do you want to store \u in text field? Best regards, depe

[GENERAL] invalid byte sequence for encoding "UTF8": 0x00

2011-11-19 Thread pawel_kukawski
Hi, Is there any way I can store NULL character (\u) in string ? Or there is only one option that I have change every text field to bytea. Regards, Paweł -- View this message in context: http://postgresql.1045698.n5.nabble.com/invalid-byte-sequence-for-encoding-UTF8-0x00-tp5007173p5007173.

Re: [GENERAL] Invalid byte sequence for encoding "UTF8": 0xedbebf

2011-06-17 Thread Albe Laurenz
BRUSSER Michael wrote: >>> Is there a way to find the records with the text field containing Unicode bytes "0xedbebf"? >>> Unfortunately this is a very old version 7.3.10 >> >> This should work on 7.3 (according to the documentation): >> SELECT id FROM nlsdata WHERE position('\360\235\204\236'::byt

Re: [GENERAL] Invalid byte sequence for encoding "UTF8": 0xedbebf

2011-06-16 Thread BRUSSER Michael
-Original Message- From: Albe Laurenz [mailto:laurenz.a...@wien.gv.at] Sent: Thursday, June 16, 2011 5:16 AM To: BRUSSER Michael; pgsql-general@postgresql.org Subject: RE: [GENERAL] Invalid byte sequence for encoding "UTF8": 0xedbebf BRUSSER Michael wrote: > Is there a wa

Re: [GENERAL] Invalid byte sequence for encoding "UTF8": 0xedbebf

2011-06-16 Thread Albe Laurenz
BRUSSER Michael wrote: > Is there a way to find the records with the text field containing Unicode bytes "0xedbebf"? > > Unfortunately this is a very old version 7.3.10 This should work on 7.3 (according to the documentation): SELECT id FROM nlsdata WHERE position('\360\235\204\236'::bytea IN val

Re: [GENERAL] Invalid byte sequence for encoding "UTF8": 0xedbebf

2011-06-15 Thread BRUSSER Michael
-Original Message- From: pgsql-general-ow...@postgresql.org [mailto:pgsql-general-ow...@postgresql.org] On Behalf Of Alan Hodgson Sent: Wednesday, June 15, 2011 5:37 PM To: pgsql-general@postgresql.org Subject: Re: [GENERAL] Invalid byte sequence for encoding "UTF8": 0xedbebf

Re: [GENERAL] Invalid byte sequence for encoding "UTF8": 0xedbebf

2011-06-15 Thread Alan Hodgson
On June 15, 2011 01:18:27 PM BRUSSER Michael wrote: > Unless there's no other options I don't want to use sed or break file into > pieces, if possible, iconv loads everything into RAM. You can use "split", convert the pieces, and then recombine, I did that when converting a large database to utf-

[GENERAL] Invalid byte sequence for encoding "UTF8": 0xedbebf

2011-06-15 Thread BRUSSER Michael
This is a follow-up on my previous message http://archives.postgresql.org/pgsql-general/2011-06/msg00054.php I think I have now some understanding of what's causing the problem, but I don't have a good solution, instead more questions. The release notes for v8.1 at http://www.postgresql.org/doc

Re: [GENERAL] invalid byte sequence for encoding "UTF8"

2011-06-02 Thread Derrick Rice
That specific character sequence is a result of Unicode implementations prior to 6.0 mixing with later implementations. See here: http://en.wikipedia.org/wiki/Specials_%28Unicode_block%29#Replacement_character You could replace that sequence with the correct 0xFFFD sequence with `sed` for exampl

[GENERAL] invalid byte sequence for encoding "UTF8"

2011-06-02 Thread BRUSSER Michael
We upgrading some old database (7.3.10 to 8.4.4). This involves running pg_dump on the old db and loading the datafile to the new db. If this matters we do not use pg_restore, the dump file is just sourced with psql, and this is where I ran into problem: psql: .../postgresql_archive.src/...

Re: [GENERAL] invalid byte sequence for encoding "UTF8": 0xf1612220

2011-05-12 Thread Cédric Villemain
2011/5/12 Craig Ringer : > On 05/11/2011 03:16 PM, AI Rumman wrote: >> >> I am trying to migrate a database from Postgresql 8.2 to Postgresql 8.3 >> and getting the following error: >> >> pg_restore: [archiver (db)] Error from TOC entry 2764; 0 29708702 TABLE >> DATA originaldata postgres >> pg_res

Re: [GENERAL] invalid byte sequence for encoding "UTF8": 0xf1612220

2011-05-11 Thread Craig Ringer
On 05/11/2011 03:16 PM, AI Rumman wrote: I am trying to migrate a database from Postgresql 8.2 to Postgresql 8.3 and getting the following error: pg_restore: [archiver (db)] Error from TOC entry 2764; 0 29708702 TABLE DATA originaldata postgres pg_restore: [archiver (db)] COPY failed: ERROR: in

[GENERAL] invalid byte sequence for encoding "UTF8": 0xf1612220

2011-05-11 Thread AI Rumman
I am trying to migrate a database from Postgresql 8.2 to Postgresql 8.3 and getting the following error: pg_restore: [archiver (db)] Error from TOC entry 2764; 0 29708702 TABLE DATA originaldata postgres pg_restore: [archiver (db)] COPY failed: ERROR: invalid byte sequence for encoding "UTF8": 0x

Re: [GENERAL] invalid byte sequence for encoding "UTF8": 0xab

2009-06-08 Thread Dimitri Fontaine
"Grand, Mark D." writes: > It turns out that my problem was that the editor I was using (emacs) > does not properly support utf8 encoding. Emacs does support utf8 properly. http://www.emacswiki.org/emacs/ChangingEncodings It could be I'm biased because I use emacs from CVS, which is going to

Re: [GENERAL] invalid byte sequence for encoding "UTF8": 0xab

2009-06-08 Thread Grand, Mark D.
] invalid byte sequence for encoding "UTF8": 0xab Mark D. Grand wrote: > I am having a vexing problem with a script I am writing to > populate reference tables in a new database. > > I am running postgreSQL 8.3 with psql 8.3.7. > > Psql reads this SQL st

Re: [GENERAL] invalid byte sequence for encoding "UTF8": 0xab

2009-06-08 Thread Albe Laurenz
Mark D. Grand wrote: > I am having a vexing problem with a script I am writing to > populate reference tables in a new database. > > I am running postgreSQL 8.3 with psql 8.3.7. > > Psql reads this SQL statement: > > INSERT INTO META_AUTH.DOMAIN_META_ASSERTION (TITLE, DESCRIPTION, > META_

Re: [GENERAL] invalid byte sequence for encoding "UTF8": 0xab

2009-06-05 Thread Vick Khera
On Fri, Jun 5, 2009 at 9:57 AM, Tom Lane wrote: > The ASCII code for '<' is 0x3c, not 0xab.  I am not sure what you are > actually typing; although it's suggestive that the LATIN1 code 0xab > corresponds to a symbol that looks approximately like '<<'.  The most > likely bet is that you are typing t

Re: [GENERAL] invalid byte sequence for encoding "UTF8": 0xab

2009-06-05 Thread Tom Lane
"Grand, Mark D." writes: > ... I get this message: > ERROR: invalid byte sequence for encoding "UTF8": 0xab > HINT: This error can also happen if the byte sequence does not match the > encoding expected by the server, which is controlled by "client_encoding". > It is complaining about the '<'

[GENERAL] invalid byte sequence for encoding "UTF8": 0xab

2009-06-05 Thread Grand, Mark D.
I am having a vexing problem with a script I am writing to populate reference tables in a new database. I am running postgreSQL 8.3 with psql 8.3.7. Psql reads this SQL statement: INSERT INTO META_AUTH.DOMAIN_META_ASSERTION (TITLE, DESCRIPTION, META_ASSERTION) VALUES ('Super-User Aut

Re: [GENERAL] invalid byte sequence for encoding "UTF8"

2007-11-30 Thread Albe Laurenz
Glyn Astill wrote: > I've setup a postgres 8.2 server and have a database setup with UTF8 > encoding. I intend to read some of our legacy data into the table, > this legacy data is in ASCII format, and as far as I know is 8 bit > ASCII. > > We have a migration tool from mertechdata.com to convert

Re: [GENERAL] invalid byte sequence for encoding "UTF8"

2007-11-30 Thread Martijn van Oosterhout
On Fri, Nov 30, 2007 at 09:44:36AM +, Glyn Astill wrote: > I've setup a postgres 8.2 server and have a database setup with UTF8 > encoding. I intend to read some of our legacy data into the table, > this legacy data is in ASCII format, and as far as I know is 8 bit > ASCII. Your problem is tha

Re: [GENERAL] invalid byte sequence for encoding "UTF8"

2007-11-30 Thread Gregory Stark
[Generally it's not a good idea to start a new thread by responding to an existing one, it confuses people and makes it more likely for your question to be missed.] "Glyn Astill" <[EMAIL PROTECTED]> writes: > Hi People, > > I've setup a postgres 8.2 server and have a database setup with UTF8 >

Re: [GENERAL] invalid byte sequence for encoding "UTF8"

2007-11-30 Thread Usama Dar
On 11/30/07, Glyn Astill <[EMAIL PROTECTED]> wrote: > > Hi People, > > I've setup a postgres 8.2 server and have a database setup with UTF8 > encoding. I intend to read some of our legacy data into the table, > this legacy data is in ASCII format, and as far as I know is 8 bit > ASCII. > > We have

[GENERAL] invalid byte sequence for encoding "UTF8"

2007-11-30 Thread Glyn Astill
Hi People, I've setup a postgres 8.2 server and have a database setup with UTF8 encoding. I intend to read some of our legacy data into the table, this legacy data is in ASCII format, and as far as I know is 8 bit ASCII. We have a migration tool from mertechdata.com to convert these files that ar

Re: [GENERAL] invalid byte sequence for encoding "UTF8": 0xff

2007-09-03 Thread Ashish Karalkar
- Original Message - From: "Albe Laurenz" <[EMAIL PROTECTED]> To: "Ashish Karalkar *EXTERN*" <[EMAIL PROTECTED]> Cc: Sent: Monday, September 03, 2007 4:54 PM Subject: RE: [GENERAL] invalid byte sequence for encoding "UTF8": 0xff Ashish K

Re: [GENERAL] invalid byte sequence for encoding "UTF8": 0xff

2007-09-03 Thread Albe Laurenz
Ashish Karalkar wrote: >>> I have a data script which runs fine from PgAdmin SQL >>> Editor,but when I run this from command prompt I get >>> following error: >>> >>> test=# \i /usr/local/pgsql/qsweb1/QSWEB_100_4_Default_Data.sql >>> >>> psql:/usr/local/pgsql/qsweb1/QSWEB_100_4_Default_Data.sql:1

Re: [GENERAL] invalid byte sequence for encoding "UTF8": 0xff

2007-09-03 Thread Ashish Karalkar
- Original Message - From: "Ashish Karalkar" <[EMAIL PROTECTED]> To: "Albe Laurenz" <[EMAIL PROTECTED]> Sent: Monday, September 03, 2007 4:09 PM Subject: Re: [GENERAL] invalid byte sequence for encoding "UTF8": 0xff - Original M

Re: [GENERAL] invalid byte sequence for encoding "UTF8": 0xff

2007-09-03 Thread Albe Laurenz
Ashish Karalkar wrote: > I have a data script which runs fine from PgAdmin SQL > Editor,but when I run this from command prompt I get > following error: > > > test=# \i /usr/local/pgsql/qsweb1/QSWEB_100_4_Default_Data.sql > > psql:/usr/local/pgsql/qsweb1/QSWEB_100_4_Default_Data.sql:1: >

Re: [GENERAL] invalid byte sequence for encoding "UTF8": 0xff

2007-09-03 Thread Martijn van Oosterhout
On Mon, Sep 03, 2007 at 01:36:58PM +0530, Ashish Karalkar wrote: > Hello All, > > I have a data script which runs fine from PgAdmin SQL Editor,but when I run > this from command prompt I get following error: > test=# \i /usr/local/pgsql/qsweb1/QSWEB_100_4_Default_Data.sql > > psql:/usr/local/p

[GENERAL] invalid byte sequence for encoding "UTF8": 0xff

2007-09-03 Thread Ashish Karalkar
Hello All, I have a data script which runs fine from PgAdmin SQL Editor,but when I run this from command prompt I get following error: test=# \i /usr/local/pgsql/qsweb1/QSWEB_100_4_Default_Data.sql psql:/usr/local/pgsql/qsweb1/QSWEB_100_4_Default_Data.sql:1: ERROR: invalid byt e sequence fo

Re: [GENERAL] invalid byte sequence for encoding "UTF8"

2007-03-21 Thread Martijn van Oosterhout
On Wed, Mar 21, 2007 at 09:54:41AM -0700, Alan Hodgson wrote: > iconv needs to read the whole file into RAM. What you can do is use the > UNIX split utility to split the dump file into smaller segments, use iconv > on each segment, and then cat all the converted segments back together into > a

Re: [GENERAL] invalid byte sequence for encoding "UTF8"

2007-03-21 Thread Alan Hodgson
On Wednesday 21 March 2007 04:17, "Fuzzygoth" <[EMAIL PROTECTED]> wrote: > I've searched the forums and found people with similar problems but > not much > on a way to remedy it. I did try using iconv which was suggested in a > thread > but it returned an error saying even the 22GB file was too la

[GENERAL] invalid byte sequence for encoding "UTF8"

2007-03-21 Thread Fuzzygoth
Hi, I am trying currently trying to setup our new database sever, we have upgraded to PostgreSQL 8.1.8. When I try to restore the backup (which is stored as a set of SQL statements that my restore script feeds into PSQL to execute) it returns the following error. psql:/mnt/tmp/app/application_dat

Re: [GENERAL] invalid byte sequence for encoding "UTF8"

2007-01-16 Thread Chad Wagner
On 1/16/07, Gary Benade <[EMAIL PROTECTED]> wrote: I used shp2pgsql.exe to create an import sql for my gis database. The resultant sql has data like this in it.INSERT INTO "gis"."sa_area" ("label","type","level",the_geom) VALUES ('MÔRELIG','0x2','2','01060001000'); The Ô is ascii char 21

Re: [GENERAL] invalid byte sequence for encoding "UTF8"

2007-01-16 Thread Martijn van Oosterhout
On Tue, Jan 16, 2007 at 03:40:52PM +0200, Gary Benade wrote: > I used shp2pgsql.exe to create an import sql for my gis database. > The resultant sql has data like this in it.INSERT INTO "gis"."sa_area" > ("label","type","level",the_geom) VALUES > ('MÔRELIG','0x2','2','01060001000'); > The

[GENERAL] invalid byte sequence for encoding "UTF8"

2007-01-16 Thread Gary Benade
I used shp2pgsql.exe to create an import sql for my gis database. The resultant sql has data like this in it.INSERT INTO "gis"."sa_area" ("label","type","level",the_geom) VALUES ('MÔRELIG','0x2','2','01060001000'); The Ô is ascii char 212. This wont import, PSQL returns ERROR: invalid byt