Re: [GENERAL] encoding confusion with \copy command

2014-09-17 Thread John R Pierce
On 9/17/2014 6:08 PM, John DeSoi wrote: On Sep 17, 2014, at 11:52 AM, Martin Waite wrote: >You are right that I need an intermediate step. I will probably use a CSV parser that is liberal in what it accepts, but writes out strict CSV data suitable for postgres. > If you find such a utility

Re: [GENERAL] encoding confusion with \copy command

2014-09-17 Thread Adrian Klaver
On 09/17/2014 06:08 PM, John DeSoi wrote: On Sep 17, 2014, at 11:52 AM, Martin Waite wrote: You are right that I need an intermediate step. I will probably use a CSV parser that is liberal in what it accepts, but writes out strict CSV data suitable for postgres. If you find such a utili

Re: [GENERAL] encoding confusion with \copy command

2014-09-17 Thread John DeSoi
On Sep 17, 2014, at 11:52 AM, Martin Waite wrote: > You are right that I need an intermediate step. I will probably use a CSV > parser that is liberal in what it accepts, but writes out strict CSV data > suitable for postgres. > If you find such a utility, please share. My clients love Exce

Re: [GENERAL] encoding confusion with \copy command

2014-09-17 Thread Martin Waite
You are right that I need an intermediate step. I will probably use a CSV parser that is liberal in what it accepts, but writes out strict CSV data suitable for postgres. Thanks for the help. On 17 September 2014 15:40, Adrian Klaver wrote: > On 09/17/2014 06:48 AM, Martin Waite wrote: > >> Hi

Re: [GENERAL] encoding confusion with \copy command

2014-09-17 Thread Adrian Klaver
On 09/17/2014 06:48 AM, Martin Waite wrote: Hi Adrian, I apologise - I meant 9.4 Looks like you will need an intermediate step. A quick search found the below which might offer a solution: http://www.excel-sql-server.com/sql-server-export-to-excel-using-bcp-sqlcmd-csv.htm FYI the good stuf

Re: [GENERAL] encoding confusion with \copy command

2014-09-17 Thread Adrian Klaver
On 09/17/2014 03:03 AM, Martin Waite wrote: Hi, I have a postgresql 7.4 server and client on Centos 6.4. The database server is using UTF-8 encoding. First I think we need to establish what version of Postgres you using. Are you really using 7.4? I have been exploring the use of the \cop

Re: [GENERAL] encoding confusion with \copy command

2014-09-17 Thread Martin Waite
Hi Adrian, I apologise - I meant 9.4 regards, Martin On 17 September 2014 14:35, Adrian Klaver wrote: > On 09/17/2014 03:03 AM, Martin Waite wrote: > >> Hi, >> >> I have a postgresql 7.4 server and client on Centos 6.4. The database >> server is using UTF-8 encoding. >> > > First I think we n

[GENERAL] encoding confusion with \copy command

2014-09-17 Thread Martin Waite
Hi, I have a postgresql 7.4 server and client on Centos 6.4. The database server is using UTF-8 encoding. I have been exploring the use of the \copy command for importing CSV data generated by SQL Server 2008. SQL Server 2008 export tool does not escape quotes that are in the content of fields,

Re: [GENERAL] ENCODING = 'LATIN1' LC_CTYPE?

2014-08-08 Thread Albe Laurenz
Sylvia Preuß wrote: > I’d like to create a database with ENCODING LATIN1 . > > CREATE DATABASE z_latin1 > WITH OWNER = admin >ENCODING = 'LATIN1' >TABLESPACE = pg_default >LC_COLLATE = 'German_Germany.1252' >LC_CTYPE = 'German_Germany.1252' >CONNECTION LIM

[GENERAL] ENCODING = 'LATIN1' LC_CTYPE?

2014-08-07 Thread Preuß , Sylvia
Dear list, I'd like to create a database with ENCODING LATIN1 . CREATE DATABASE z_latin1 WITH OWNER = admin ENCODING = 'LATIN1' TABLESPACE = pg_default LC_COLLATE = 'German_Germany.1252' LC_CTYPE = 'German_Germany.1252' CONNECTION LIMIT = -1; FEHLER: Kodierung

Re: [GENERAL] encoding and LC_COLLATE

2011-11-15 Thread LPlateAndy
Hi Adrian/Mark Thanks again for your help, i have now got the load working by setting the encoding to WIN1252. I had been assuming i was setting it to UTF8 SET CLIENT_ENCODING TO 'WIN1252'; Andy -- View this message in context: http://postgresql.1045698.n5.nabble.com/encoding-and-LC-COLLATE-tp

Re: [GENERAL] encoding and LC_COLLATE

2011-11-15 Thread LPlateAndy
Hi Mark (and Adrian), As as update i've now found the same data fails on my postgres 8 which doesn't seem to have the LC_COLLATE etc setting and is just UTF-8 so i guess there is possibly just something about the way the data is getting passed in. This is the error message from postgres 9

Re: [GENERAL] encoding and LC_COLLATE

2011-11-14 Thread Mark Watson
De : pgsql-general-ow...@postgresql.org [mailto:pgsql-general-ow...@postgresql.org] De la part de Adrian Klaver >Envoyé : 14 novembre 2011 13:03 >... > >Second is the data coming in actually UTF8 or some other encoding? >...  Hi Andy, I have to agree with Adrian in that the data may be coming in

Re: [GENERAL] encoding and LC_COLLATE

2011-11-14 Thread Adrian Klaver
On Mon, Nov 14, 2011 at 8:25 AM, LPlateAndy wrote: > Hi Adrian, > > ** ** > > You’re right, i’m trying to get the copy command to put a load of data > into a table. It’s now working fine except for any instances with an e acute > > > ** ** > > I tried putting “ SET CLIENT_ENCODING TO ‘UT

Re: [GENERAL] encoding and LC_COLLATE

2011-11-14 Thread LPlateAndy
Hi Adrian, You're right, i'm trying to get the copy command to put a load of data into a table. It's now working fine except for any instances with an e acute I tried putting " SET CLIENT_ENCODING TO 'UTF-8'; " but still got the error. I guess that just because i'm verifying what's incoming

Re: [GENERAL] encoding and LC_COLLATE

2011-11-14 Thread Adrian Klaver
On Monday, November 14, 2011 3:03:32 am LPlateAndy wrote: > Hi, > > I set up my postgres 9.0 install 6 months ago and generally everything is > fine but a recent data load with an e acute character failed which an > unsupported message which surprised me as we're using UTF-8. > > However, i can

Re: [GENERAL] encoding and LC_COLLATE

2011-11-14 Thread LPlateAndy
Hi, In response to my own question i have now read the following: http://stackoverflow.com/questions/6579621/lc-collate-and-lc-ctype-suport-for-utf-8-in-postgresql It seems to show that apart from a complete reinstall i should just use a Locale of C (as i'm on Windows and POSIX wont work). This

[GENERAL] encoding and LC_COLLATE

2011-11-14 Thread LPlateAndy
Hi, I set up my postgres 9.0 install 6 months ago and generally everything is fine but a recent data load with an e acute character failed which an unsupported message which surprised me as we're using UTF-8. However, i can now see that the listing for the database set up show a restriction unde

Re: [GENERAL] Encoding Conversion

2011-02-03 Thread Tom Lane
Wilton Wonrath writes: > When I´m doing a conversion PostgreSQL returns SCAPES to me besides the > desire > caracter. Why ? Somewhere between 8.2 and 9.0 convert() was redefined to take and return bytea, not text, to reflect the fact that the data it deals in isn't necessarily encoded in the da

[GENERAL] Encoding Conversion

2011-02-03 Thread Wilton Wonrath
Hi, When I´m doing a conversion PostgreSQL returns SCAPES to me besides the desire caracter. Why ? Using Postgresql 8.2: SELECT CONVERT('Rejeição: Duplicidade de NF-e, com diferença na Chave de Acesso [35110100608804000178550010001009471840996034]','UTF8','LATIN1') Result: "Rejeição: Du

Re: [GENERAL] Encoding change question...

2010-08-16 Thread Karl Denninger
Peter C. Lai wrote: > The doublequotes isn't UTF8 it's people copying and pasting from Microsoft > stuff, which is WIN-1252. So try to use that with iconv instead of utf8 > > On 2010-08-16 12:40:03PM -0500, Karl Denninger wrote: > >> So I have myself a nice pickle here. >> >> I've got a data

Re: [GENERAL] Encoding change question...

2010-08-16 Thread Peter C. Lai
The doublequotes isn't UTF8 it's people copying and pasting from Microsoft stuff, which is WIN-1252. So try to use that with iconv instead of utf8 On 2010-08-16 12:40:03PM -0500, Karl Denninger wrote: > So I have myself a nice pickle here. > > I've got a database which was originally created

[GENERAL] Encoding change question...

2010-08-16 Thread Karl Denninger
So I have myself a nice pickle here. I've got a database which was originally created with SQL_ASCII for the encoding (anything goes text fields) Unfortunately, I have a bunch of data that was encoded in UTF-8 that's in an RSS feed that I need to load into said database. iconv barfs all over

Re: [GENERAL] Encoding using the Frontend/Backend Protocol TCP/IP

2009-11-19 Thread Raimon Fernandez
On 19/11/2009, at 21:21, Kovalevski Andrei wrote: > Hi, > > the string is ok, but the problem is inside the message. The length of the > message is incorrect: > > your message: > 510046557064617465207472616E73616374696F6E7320736574206465736372697074696F6E3D27546573742056616C75657364C387272

Re: [GENERAL] Encoding using the Frontend/Backend Protocol TCP/IP

2009-11-19 Thread Kovalevski Andrei
Hi, the string is ok, but the problem is inside the message. The length of the message is incorrect: your message: 5100*46*557064617465207472616E73616374696F6E7320736574206465736372697074696F6E3D27546573742056616C75657364C387272077686572652069643D313133 it should be: 5100*45*5570646

Re: [GENERAL] Encoding using the Frontend/Backend Protocol TCP/IP

2009-11-19 Thread Raimon Fernandez
On 19/11/2009, at 18:13, Raimon Fernandez wrote: > > On 19/11/2009, at 17:27, Kovalevski Andrei wrote: > >> Hi >> >> could it be that you have errors in your UTF8 string? For example you might >> use UTF16 encoding, it can explain why some characters force errors but >> others are not. > >

Re: [GENERAL] Encoding using the Frontend/Backend Protocol TCP/IP

2009-11-19 Thread Raimon Fernandez
On 19/11/2009, at 17:27, Kovalevski Andrei wrote: > Hi > > could it be that you have errors in your UTF8 string? For example you might > use UTF16 encoding, it can explain why some characters force errors but > others are not. It only happens with values like àéïçñ I think UTF8 can handle thi

Re: [GENERAL] Encoding using the Frontend/Backend Protocol TCP/IP

2009-11-19 Thread Kovalevski Andrei
Hi could it be that you have errors in your UTF8 string? For example you might use UTF16 encoding, it can explain why some characters force errors but others are not. Can you post here the string and its' encoded version? Raimon Fernandez wrote: Hello, I'm trying to send some strings that

[GENERAL] Encoding using the Frontend/Backend Protocol TCP/IP

2009-11-19 Thread Raimon Fernandez
Hello, I'm trying to send some strings that have chars outside from standar ascii, like çñàèó Once I'm connected, the client and server both uses UT8Encoding. And I'm sending all the strings encoded in UTF8. At least the received ones are working, as I get the text exactly as it is, with spe

Re: [GENERAL] Encoding question when dumping/restoring databases for upgrade

2009-08-13 Thread Tom Lane
a...@archie.netg.se writes: > I am sitting on version 7.4.x and am going to upgrade to version 8.3.x. >> From all I can read I should have no problem with actual format of the > pgdump file (for actual dumping and restoring purposes) but I am having > problems with encoding (which I was fairly s

[GENERAL] Encoding question when dumping/restoring databases for upgrade

2009-08-13 Thread arsi
Hello, I am sitting on version 7.4.x and am going to upgrade to version 8.3.x. From all I can read I should have no problem with actual format of the pgdump file (for actual dumping and restoring purposes) but I am having problems with encoding (which I was fairly sure I would). I have search

Re: [GENERAL] encoding of PostgreSQL messages

2009-03-05 Thread Hiroshi Saito
Hi. Karsten-san. Yeah, It was a problem unsolvable by the driver to relay. although perseverance keeping without giving up! -- arigatougozaimasu:-) Regards, Hiroshi Saito - Original Message - From: "Karsten Hilbert" On Tue, Mar 03, 2009 at 12:35:37AM +0900, Hiroshi Saito wrote:

Re: [GENERAL] encoding of PostgreSQL messages

2009-03-04 Thread Karsten Hilbert
On Tue, Mar 03, 2009 at 12:35:37AM +0900, Hiroshi Saito wrote: > Sorry very late reaction. > I desire problem solution. So do I :-) Ganbatte ! Karsten -- GPG key ID E4071346 @ wwwkeys.pgp.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 -- Sent via pgsql-general mailing list (pgsql-gene

Re: [GENERAL] encoding of PostgreSQL messages

2009-03-02 Thread Hiroshi Saito
Hi. Sorry very late reaction. I desire problem solution.Therefore, one evidence I tried jdbc program. http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/connect_problem/jdbctestx.java C:\home\HIROSHI>java jdbctestx org.postgresql.util.PSQLException: FATAL: ???[??"postgres"??

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-12 Thread Karsten Hilbert
On Wed, Feb 11, 2009 at 02:20:47PM +0900, Hiroshi Inoue wrote: > Tom Lane wrote: > > Hiroshi Inoue writes: > >> I'm thinking of the following steps in the backend code. > > > >> 1.Set LC_MESSAGES to "C" until the client_encoding is > >> determined. I have tried that but it didn't work out for

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-12 Thread Karsten Hilbert
On Thu, Feb 12, 2009 at 10:28:38PM +0900, Hiroshi Inoue wrote: > >>> Reflecting on the bigger picture ... I would imagine that the vast > >>> majority of existing applications depend on client_encoding settings > >>> that come from postgresql.conf, ALTER USER SET, ALTER DATABASE SET, or > >>> jus

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-12 Thread Hiroshi Inoue
Tom Lane wrote: > Peter Eisentraut writes: >> Tom Lane wrote: >>> Reflecting on the bigger picture ... I would imagine that the vast >>> majority of existing applications depend on client_encoding settings >>> that come from postgresql.conf, ALTER USER SET, ALTER DATABASE SET, or >>> just the def

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-11 Thread Peter Eisentraut
On Wednesday 11 February 2009 18:00:31 Tom Lane wrote: > Peter Eisentraut writes: > > Tom Lane wrote: > >> Reflecting on the bigger picture ... I would imagine that the vast > >> majority of existing applications depend on client_encoding settings > >> that come from postgresql.conf, ALTER USER S

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-11 Thread Tom Lane
Peter Eisentraut writes: > Tom Lane wrote: >> Reflecting on the bigger picture ... I would imagine that the vast >> majority of existing applications depend on client_encoding settings >> that come from postgresql.conf, ALTER USER SET, ALTER DATABASE SET, or >> just the default (== database encod

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-10 Thread Peter Eisentraut
Tom Lane wrote: I believe the only real "fix" is to guarantee that messages are sent as untranslated ASCII until we have sent an encoding indicator at the end of the startup sequence. Which has its own pretty clear downside: no more translation of authorization failures. We should process the

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-10 Thread Peter Eisentraut
Tom Lane wrote: Reflecting on the bigger picture ... I would imagine that the vast majority of existing applications depend on client_encoding settings that come from postgresql.conf, ALTER USER SET, ALTER DATABASE SET, or just the default (== database encoding). I don't think a solution that p

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-10 Thread Hiroshi Inoue
Tom Lane wrote: > Hiroshi Inoue writes: >> I'm thinking of the following steps in the backend code. > >> 1.Set LC_MESSAGES to "C" until the client_encoding is >> determined. >> 2.When a client_encoding is specifed in the startup >> message, bind the corrsponding codeset to the >> textdomain

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-10 Thread Tom Lane
Hiroshi Inoue writes: > I'm thinking of the following steps in the backend code. > 1.Set LC_MESSAGES to "C" until the client_encoding is > determined. > 2.When a client_encoding is specifed in the startup > message, bind the corrsponding codeset to the > textdomain and set LC_MESSAGES to th

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-10 Thread Hiroshi Inoue
Tom Lane wrote: > Peter Eisentraut writes: >> Bruce Momjian wrote: >>> Can someone comment on this? > >> I think we have discussed more proper solutions earlier in this thread. >> IMO the best approach would be for the client to include the client >> encoding in the startup package. > > Huh?

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-08 Thread Hiroshi Inoue
Tom Lane wrote: > Peter Eisentraut writes: >> Bruce Momjian wrote: >>> Can someone comment on this? > >> Looks like a horrible hack to me. Recoding stuff to the client encoding >> in the server outside the existing recoding mechanism looks pretty evil >> to me. Plus, it does not address the p

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-08 Thread Karsten Hilbert
On Sun, Feb 08, 2009 at 10:38:16AM -0500, Tom Lane wrote: > I believe the only real "fix" is to guarantee that messages are sent > as untranslated ASCII until we have sent an encoding indicator at > the end of the startup sequence. Which has its own pretty clear > downside: no more translation of

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-08 Thread Tom Lane
Peter Eisentraut writes: > Bruce Momjian wrote: >> Can someone comment on this? > Looks like a horrible hack to me. Recoding stuff to the client encoding > in the server outside the existing recoding mechanism looks pretty evil > to me. Plus, it does not address the problem of what happens to

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-08 Thread Peter Eisentraut
Bruce Momjian wrote: Can someone comment on this? Looks like a horrible hack to me. Recoding stuff to the client encoding in the server outside the existing recoding mechanism looks pretty evil to me. Plus, it does not address the problem of what happens to messages sent before this, it ju

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-08 Thread Hiroshi Inoue
Peter Eisentraut wrote: Bruce Momjian wrote: Can someone comment on this? Looks like a horrible hack to me. Recoding stuff to the client encoding in the server outside the existing recoding mechanism looks pretty evil to me. > Plus, it does not address the problem of what happens to messa

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-07 Thread Bruce Momjian
Can someone comment on this? --- Hiroshi Inoue wrote: > Hi, > > This topic seems to be related to the bug report > [ODBC] Localized error messages, wrong charset > . > > Bruce Momjian wrote: > > Added to TODO: > > > >

Re: [GENERAL] encoding of PostgreSQL messages

2009-01-30 Thread Hiroshi Inoue
Hi, This topic seems to be related to the bug report [ODBC] Localized error messages, wrong charset . Bruce Momjian wrote: Added to TODO: Improve encoding of connection startup messages sent to the client Currently some authentication error messages are sent in

Re: [GENERAL] Encoding problem using pg_dumpall

2009-01-29 Thread Magnus Hagander
Tom Lane wrote: > Magnus Hagander writes: >> Tom Lane wrote: >>> (Hmm, actually it looks like pg_dumpall hasn't got a -E switch, >>> which seems like an oversight. So you need to fix your locale, >>> or else use pg_dump directly.) > >> IIRC, you can't set the windows console to be UTF8. > > Ugh

Re: [GENERAL] Encoding problem using pg_dumpall

2009-01-29 Thread Tom Lane
Magnus Hagander writes: > Tom Lane wrote: >> (Hmm, actually it looks like pg_dumpall hasn't got a -E switch, >> which seems like an oversight. So you need to fix your locale, >> or else use pg_dump directly.) > IIRC, you can't set the windows console to be UTF8. Ugh. That seems to raise the pr

Re: [GENERAL] Encoding problem using pg_dumpall

2009-01-29 Thread Magnus Hagander
Tom Lane wrote: > "Moshe Ben-Shoham" writes: >> C:\Program Files\PostgreSQL\8.3\bin>pg_dumpall -U admint > >> c:\temp\dbdump.sql >> pg_dump: SQL command failed >> pg_dump: Error message from server: ERROR: character 0xd595 of encoding >> "UTF8" has no equivalent in "WIN1252" > > Apparently you h

Re: [GENERAL] Encoding problem using pg_dumpall

2009-01-29 Thread Tom Lane
"Moshe Ben-Shoham" writes: > C:\Program Files\PostgreSQL\8.3\bin>pg_dumpall -U admint > > c:\temp\dbdump.sql > pg_dump: SQL command failed > pg_dump: Error message from server: ERROR: character 0xd595 of encoding > "UTF8" has no equivalent in "WIN1252" Apparently you have WIN1252 set as the defa

[GENERAL] Encoding problem using pg_dumpall

2009-01-29 Thread Moshe Ben-Shoham
Hi, I have a database with encoding UTF-8 installed on Windows, and I try to dump it using pg_dumpall, on the machine on which the database is installed. I get the following error message: C:\Program Files\PostgreSQL\8.3\bin>pg_dumpall -U admint > c:\temp\dbdump.sql pg_dump: SQL command fa

Re: [GENERAL] encoding of PostgreSQL messages

2009-01-21 Thread Bruce Momjian
lient > by current means. > > Thanks, > Karsten > > On Thu, Jan 01, 2009 at 08:33:56PM +0200, Peter Eisentraut wrote: > > Subject: Re: [GENERAL] encoding of PostgreSQL messages > > User-Agent: KMail/1.9.9 > > > > On Wednesday 31 December 2008 20:23:47 Tom L

Re: [GENERAL] encoding of PostgreSQL messages

2009-01-07 Thread Karsten Hilbert
sages in 7-bit English until server_encoding can be retrieved by the client by current means. Thanks, Karsten On Thu, Jan 01, 2009 at 08:33:56PM +0200, Peter Eisentraut wrote: > Subject: Re: [GENERAL] encoding of PostgreSQL messages > User-Agent: KMail/1.9.9 > > On Wednesday 31 December

Re: [GENERAL] encoding of PostgreSQL messages

2009-01-01 Thread Peter Eisentraut
On Wednesday 31 December 2008 20:23:47 Tom Lane wrote: > > The proper fix is probably to include the client encoding in the > > connection startup message. > > What of errors occurring before such an option could be applied? Connection errors are handled by the client, which knows the client encod

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-31 Thread Karsten Hilbert
> Hm, so maybe both Peter and Alvaro are right: > > 1) Setting the translation wrapper to a NOOP as early as possible. > > Thus, the first messages are sent in 7-bit ASCII English. Despite being *marked* for translation and a translation to exist in the .po file, that is. Karsten -- Sensat

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-31 Thread Karsten Hilbert
> > The proper fix is probably to include the client encoding in the > connection > > startup message. > > What of errors occurring before such an option could be applied? > > I think that ultimately it's necessary to accept that there will be some > window during connection startup where sendin

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-31 Thread Tom Lane
Peter Eisentraut writes: > On Wednesday 31 December 2008 18:57:29 Karsten Hilbert wrote: >> The solution is to find the right layer to take control of the encoding but >> this is eventually only possible if the encoding is *known*. Thus the plea >> for "7-bit-ascii English by default until the enc

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-31 Thread Karsten Hilbert
> On Wednesday 31 December 2008 18:57:29 Karsten Hilbert wrote: > > The solution is to find the right layer to take control of the encoding > but > > this is eventually only possible if the encoding is *known*. Thus the > plea > > for "7-bit-ascii English by default until the encoding *can* be kno

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-31 Thread Peter Eisentraut
On Wednesday 31 December 2008 18:57:29 Karsten Hilbert wrote: > The solution is to find the right layer to take control of the encoding but > this is eventually only possible if the encoding is *known*. Thus the plea > for "7-bit-ascii English by default until the encoding *can* be known". > Going

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-31 Thread Karsten Hilbert
> Karsten Hilbert writes: > > On Mon, Dec 29, 2008 at 09:07:14AM -0300, Alvaro Herrera wrote: > >> And I'm now wondering if we should delay initializing the translation > >> stuff until after client_encoding has been reported. > > > Or else > > > - just don't pass those messages through gettext

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-31 Thread Tom Lane
Karsten Hilbert writes: > On Mon, Dec 29, 2008 at 09:07:14AM -0300, Alvaro Herrera wrote: >> And I'm now wondering if we should delay initializing the translation >> stuff until after client_encoding has been reported. > Or else > - just don't pass those messages through gettext so they are >

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-31 Thread Karsten Hilbert
On Mon, Dec 29, 2008 at 09:07:14AM -0300, Alvaro Herrera wrote: > > That would not quite be enough -- I am talking about > > messages reported *during* auth, say > > > > FATAL: password authentication failed for user "postgres" > > > > or > > > > fe_sendauth: no password supplied > > >

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-29 Thread Alvaro Herrera
Karsten Hilbert wrote: > On Tue, Dec 23, 2008 at 06:45:17PM -0300, Alvaro Herrera wrote: > > Hmm, isn't client_encoding reported in the startup packet sent by the > > server, after auth? > > That would not quite be enough -- I am talking about > messages reported *during* auth, say > > FAT

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-23 Thread John DeSoi
On Dec 23, 2008, at 4:24 PM, Karsten Hilbert wrote: How can I programmatically detect which encoding a PostgreSQL server I am trying to connect to sends back messages -- before I connect (so client_encoding and the pg_settings table are flat out). I don't think there is a way because you can'

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-23 Thread Karsten Hilbert
On Tue, Dec 23, 2008 at 06:45:17PM -0300, Alvaro Herrera wrote: > > How can I programmatically detect which encoding a > > PostgreSQL server I am trying to connect to sends back > > messages -- before I connect (so client_encoding and > > the pg_settings table are flat out). > > Hmm, isn't client

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-23 Thread Alvaro Herrera
Karsten Hilbert wrote: > Hi all ! > > How can I programmatically detect which encoding a > PostgreSQL server I am trying to connect to sends back > messages -- before I connect (so client_encoding and > the pg_settings table are flat out). Hmm, isn't client_encoding reported in the startup packet

[GENERAL] encoding of PostgreSQL messages

2008-12-23 Thread Karsten Hilbert
Hi all ! How can I programmatically detect which encoding a PostgreSQL server I am trying to connect to sends back messages -- before I connect (so client_encoding and the pg_settings table are flat out). Thanks, Karsten -- GPG key ID E4071346 @ wwwkeys.pgp.net E167 67FD A291 2BEA 73BD 4537 78B

Re: [GENERAL] Encoding

2008-11-19 Thread Alvaro Herrera
Gustavo Rosso escribió: > This error give me to load a file into postgres: > ERROR: invalid byte sequence for encoding "UTF8": 0xa4 > > I must force load data. > How must to do? Set client_encoding to latin1 (or latin9). How you do this is client-dependent. Note that you can post questions in s

[GENERAL] Encoding

2008-11-19 Thread Gustavo Rosso
This error give me to load a file into postgres: ERROR: invalid byte sequence for encoding "UTF8": 0xa4 I must force load data. How must to do? -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-ge

Re: [GENERAL] encoding confusion

2008-06-11 Thread Alban Hertroys
On Jun 11, 2008, at 9:03 AM, Richard Huxton wrote: Sim Zacks wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The data in the longblob field might be text, which could be causing the confusion. For example, when I look at the data in the longblob field, I see /n for a newline and when

Re: [GENERAL] encoding confusion

2008-06-11 Thread Richard Huxton
Sim Zacks wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The data in the longblob field might be text, which could be causing the confusion. For example, when I look at the data in the longblob field, I see /n for a newline and when I look at the bytea it is 012. That's right - newline i

Re: [GENERAL] encoding confusion

2008-06-10 Thread Sim Zacks
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The data in the longblob field might be text, which could be causing the confusion. For example, when I look at the data in the longblob field, I see /n for a newline and when I look at the bytea it is 012. I can only tell you what happened in the cli

Re: [GENERAL] encoding confusion

2008-06-10 Thread Albe Laurenz
Sim Zacks wrote: > We originally tested it on mysql and now we are migrating it > to postgresql. > > The messages are stored in a longblob field on mysql and a bytea field > in postgresql. > > I set the database up as UTF-8, even though we get emails that are not > UTF encoded, mostly because I

Re: [GENERAL] encoding confusion

2008-06-10 Thread Richard Huxton
Sim Zacks wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 {BACKGROUND] I am testing dbmail for our corporate email solution. We originally tested it on mysql and now we are migrating it to postgresql. The messages are stored in a longblob field on mysql and a bytea field in postgresql. I

[GENERAL] encoding confusion

2008-06-10 Thread Sim Zacks
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 {BACKGROUND] I am testing dbmail for our corporate email solution. We originally tested it on mysql and now we are migrating it to postgresql. The messages are stored in a longblob field on mysql and a bytea field in postgresql. I set the database u

Re: [GENERAL] encoding does not match server's locale

2008-02-04 Thread Tom Lane
mikie <[EMAIL PROTECTED]> writes: > However, I used different encodings in previous 8.2 within a cluster > and now I don't know how to restore my databases. initdb in C locale. regards, tom lane ---(end of broadcast)--- TIP

[GENERAL] encoding does not match server's locale

2008-02-04 Thread mikie
Hi, I just wanted to upgrade from 8.2 to 8.3. So I performed pg_dump_all and now when I try to feed psql with the dump I have problems with some databases: psql:c:/Program Files/PostgreSQL/8.2/bin/dump.sql:32: ERROR: encoding LATIN2 does not match server's locale Polish_Poland.1250 OK, I have alr

Re: [GENERAL] Encoding questions

2007-07-08 Thread Michael Glaesemann
On Jul 8, 2007, at 14:09 , Pat Maddox wrote: Can I just convert it, or do I need to dump it, drop it, recreate with UTF-8, and then load the data? It depends on what is currently in your (SQL_ASCII) database. If it's only ASCII data or all non-ASCII data is UTF-8, you should be okay just

[GENERAL] Encoding questions

2007-07-08 Thread Pat Maddox
My databases all have the default SQL_ASCII encoding. I read the docs and it says that you shouldn't use it if you're going to have any non-ASCII data. I need to start supporting UTF-8, so it seems I need to convert my database. Can I just convert it, or do I need to dump it, drop it, recreate

Re: [GENERAL] Encoding Sort order

2007-05-31 Thread CAJ CAJ
On 5/31/07, Richard Broersma Jr <[EMAIL PROTECTED]> wrote: I understand that it is possible for tables from different databases that have the exact same data but different Encoding can sort rows in differet orders. Could this allow affect the order that triggers and rules are fired? This mi

Re: [GENERAL] Encoding Sort order

2007-05-31 Thread Tom Lane
Richard Broersma Jr <[EMAIL PROTECTED]> writes: > I understand that it is possible for tables from different databases that > have the exact same data > but different Encoding can sort rows in differet orders. > Could this allow affect the order that triggers and rules are fired? AFAIK, no --- t

[GENERAL] Encoding Sort order

2007-05-31 Thread Richard Broersma Jr
I understand that it is possible for tables from different databases that have the exact same data but different Encoding can sort rows in differet orders. Could this allow affect the order that triggers and rules are fired? Regards, Richard Broersma Jr. ---(end of broad

Re: [GENERAL] encoding problem at restore

2007-02-18 Thread Michael Fuhr
On Sat, Feb 17, 2007 at 03:12:44AM -0800, Bob Hunter wrote: > ERROR: invalid byte sequence for encoding "UTF8": > 0xe02031 > HINT: This error can also happen if the byte sequence > does not match the encoding expected by the server, > which is controlled by "client_encoding". > CONTEXT: COPY , l

[GENERAL] encoding problem at restore

2007-02-18 Thread Bob Hunter
Hello, I have just updated to postgres8.1 and have the following problem. The first line of the "PostgreSQL database dump" says: SET client_encoding = 'SQL_ASCII'; which is correct. However, the restore says: ERROR: invalid byte sequence for encoding "UTF8": 0xe02031 HINT: This error can als

Re: [GENERAL] encoding advice requested

2006-11-14 Thread Martijn van Oosterhout
On Tue, Nov 14, 2006 at 12:01:44AM +0100, Daniel Verite wrote: > Also, you'll find this extensively and better explained in this article, for > example: > http://ppewww.ph.gla.ac.uk/~flavell/charset/form-i18n.html This is a *really* good article about character sets and form submission. Especially

Re: [GENERAL] encoding advice requested

2006-11-13 Thread Daniel Verite
Rick Schumeyer wrote: > I will have to try the WIN1252 encoding. > > On the client side, my application is a web browser. On the server > side, it is php scripts on a linux box. The data comes from copying > data from a browser window (pointing to another web site) and pasting it > i

Re: [GENERAL] encoding advice requested

2006-11-13 Thread Marcus Engene
Rick Schumeyer skrev: I will have to try the WIN1252 encoding. On the client side, my application is a web browser. On the server side, it is php scripts on a linux box. The data comes from copying data from a browser window (pointing to another web site) and pasting it into an html textare

Re: [GENERAL] encoding advice requested

2006-11-13 Thread Rick Schumeyer
Albe Laurenz wrote: My database locale is en_US, and by default my databases are UTF8. My application code allows the user to paste text into a box and submit it to the database. Sometimes the pasted text contains non UTF8 characters, typically the "fancy" forms of quotes and apostr

Re: [GENERAL] encoding advice requested

2006-11-13 Thread Albe Laurenz
>> My database locale is en_US, and by default my databases are UTF8. >> >> My application code allows the user to paste text into a box and submit >> it to the database. Sometimes the pasted text contains non UTF8 >> characters, typically the "fancy" forms of quotes and apostrophes. The >> da

Re: [GENERAL] encoding advice requested

2006-11-12 Thread Daniel Verite
Rick Schumeyer wrote: > My database locale is en_US, and by default my databases are UTF8. > > My application code allows the user to paste text into a box and submit > it to the database. Sometimes the pasted text contains non UTF8 > characters, typically the "fancy" forms of quotes a

Re: [GENERAL] encoding advice requested

2006-11-12 Thread Shoaib Mir
Using the convert function might be of help here as well:convert(string using conversion_name)Change encoding using specified conversion name. Conversions can be defined by CREATE CONVERSION. Also there are some pre-defined conversion names ( http://www.postgresql.org/docs/current/static/functions-

Re: [GENERAL] encoding advice requested

2006-11-11 Thread Martijn van Oosterhout
On Sat, Nov 11, 2006 at 02:45:00PM -0500, Rick Schumeyer wrote: > My database locale is en_US, and by default my databases are UTF8. > > My application code allows the user to paste text into a box and submit > it to the database. Sometimes the pasted text contains non UTF8 > characters, typica

[GENERAL] encoding advice requested

2006-11-11 Thread Rick Schumeyer
My database locale is en_US, and by default my databases are UTF8. My application code allows the user to paste text into a box and submit it to the database. Sometimes the pasted text contains non UTF8 characters, typically the "fancy" forms of quotes and apostrophes. The database does not

Re: [GENERAL] Encoding, Unicode, locales, etc.

2006-11-01 Thread Tom Lane
Carlos Moreno <[EMAIL PROTECTED]> writes: > ... The one that does the case conversion "correctly" (read: as I > expect it as per Spanish or French rules) is 8.1.4 with en_US locale > (LC_CTYPE and LC_COLLATE both showing en_US.UTF-8). PG 7.4.14, *even > with locale es_ES*, does not do the case con

Re: [GENERAL] Encoding, Unicode, locales, etc.

2006-11-01 Thread Karsten Hilbert
On Wed, Nov 01, 2006 at 08:50:30PM +0100, Martijn van Oosterhout wrote: > > Could this paragraph be put into the docs and/or the FAQ, > > please ? Along with the recommendation that if you require > > multiple encodings for your databases you better had your OS > > locale configured properly for U

  1   2   >