Peter Eisentraut writes:
> Alternative consideration: We could allow this in CSV format if we made
> users quote the first value if it starts with a BOM. This might be a
> reasonable way to get MS compatibility.
I don't think we can get away with a retroactive restriction on the
contents of data
On mån, 2011-09-26 at 21:49 +0300, Peter Eisentraut wrote:
> If I store a BOM in row 1, column 1 of my table, because,
> well, maybe it's an XML document or something, then it needs to be
> able to survive a copy out and in. The only way we could proceed with
> this would be if we prohibited BOMs
Brar Piening wrote:
It's a pity that the Unicode standard actually allows something that
can cause problems but blaming the non-platform again doesn't solve
the existing issues.
To put in a more humoruos but actually correct way:
M$ has found a standard conforming way of preventing users to
Tom Lane wrote:
Note that the reference to byte order betrays the implicit context
assumption: that we're talking about UTF16 or UTF32 representation.
Note that there is no implicit context assumption in the Unicode FAQ.
It's equally covering UTF-8, UTF-16 and UTF-32.
Another quote:
Q: Can a UT
Brar Piening writes:
> Citing from the Unicode FAQ again:
> Q: Where is a BOM useful?
> A: A BOM is useful at the beginning of files that are typed as text, but
> for which it is not known whether they are in big or little endian
> formatit can also serve as a hint indicating that the file is
Robert Haas wrote:
The thing that makes me doubt that is this comment from Tatsuo Ishii:
TI> COPY explicitly specifies the encoding (to be UTF-8 in this case). So
TI> I think we should not regard U+FEFF as "BOM" in COPY, rather we should
TI> regard U+FEFF as "ZERO WIDTH NO-BREAK SPACE".
If
Tom Lane wrote:
Yeah, that's a reasonable argument for rejecting the patch altogether.
I'm not qualified to decide whether it outweighs the "we need to be able
to read Notepad output" argument.
Actually it's not only notepad.
I quite often find myself doing something like the following when mo
Tom Lane wrote:
Putting a BOM into UTF8 data is flat out invalid per spec --- the fact
that Microsloth does it does not make it standards-conformant.
Could you share a pointer to the spec?
All I've ever heard is that a BOM is optional for UTF-8 but not forbidden.
The Unicode FAQ (http://unicod
On mån, 2011-09-26 at 14:44 -0400, Robert Haas wrote:
> > We did recently accept a patch for psql -f to skip over a UTF-8
> > byte-order mark. We had a lot of this same discussion there.
>
> But that case is different, because zero-width, non-breaking space has
> no particular meaning in an SQL s
On 09/26/2011 02:38 PM, Peter Eisentraut wrote:
On mån, 2011-09-26 at 13:19 -0400, Robert Haas wrote:
The thing that makes me doubt that is this comment from Tatsuo Ishii:
TI> COPY explicitly specifies the encoding (to be UTF-8 in this case).
So
TI> I think we should not regard U+FEFF as "B
On Mon, Sep 26, 2011 at 2:38 PM, Peter Eisentraut wrote:
> On mån, 2011-09-26 at 13:19 -0400, Robert Haas wrote:
>> The thing that makes me doubt that is this comment from Tatsuo Ishii:
>>
>> TI> COPY explicitly specifies the encoding (to be UTF-8 in this case).
>> So
>> TI> I think we should not
On mån, 2011-09-26 at 13:19 -0400, Robert Haas wrote:
> The thing that makes me doubt that is this comment from Tatsuo Ishii:
>
> TI> COPY explicitly specifies the encoding (to be UTF-8 in this case).
> So
> TI> I think we should not regard U+FEFF as "BOM" in COPY, rather we
> should
> TI> regard
On tis, 2011-09-27 at 00:09 +0900, Tatsuo Ishii wrote:
> Suppose a user uses brain-dead editor, which does not accept UTF-8
> without BOM.
I would first like to see evidence that such an editor exists.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your
Robert Haas writes:
> On Mon, Sep 26, 2011 at 1:28 PM, Tom Lane wrote:
>> Robert Haas writes:
>>> The thing that makes me doubt that is this comment from Tatsuo Ishii:
>>> TI> COPY explicitly specifies the encoding (to be UTF-8 in this case). So
>>> TI> I think we should not regard U+FEFF as "B
On Mon, Sep 26, 2011 at 1:28 PM, Tom Lane wrote:
> Robert Haas writes:
>> The thing that makes me doubt that is this comment from Tatsuo Ishii:
>> TI> COPY explicitly specifies the encoding (to be UTF-8 in this case). So
>> TI> I think we should not regard U+FEFF as "BOM" in COPY, rather we shou
Robert Haas writes:
> The thing that makes me doubt that is this comment from Tatsuo Ishii:
> TI> COPY explicitly specifies the encoding (to be UTF-8 in this case). So
> TI> I think we should not regard U+FEFF as "BOM" in COPY, rather we should
> TI> regard U+FEFF as "ZERO WIDTH NO-BREAK SPACE".
Robert Haas writes:
> On Mon, Sep 26, 2011 at 11:09 AM, Tatsuo Ishii wrote:
>> Suppose a user uses brain-dead editor, which does not accept UTF-8
>> without BOM.
> Maybe this needs to be an optional behavior, controlled by some COPY option.
I'm not excited about emitting non-standards-conforman
On Mon, Sep 26, 2011 at 1:15 PM, Tom Lane wrote:
> Robert Haas writes:
>> On Mon, Sep 26, 2011 at 11:09 AM, Tatsuo Ishii wrote:
>>> Suppose a user uses brain-dead editor, which does not accept UTF-8
>>> without BOM.
>
>> Maybe this needs to be an optional behavior, controlled by some COPY option
On Mon, Sep 26, 2011 at 11:09 AM, Tatsuo Ishii wrote:
>> "David E. Wheeler"
>> writes:
>>> On Sep 25, 2011, at 9:58 PM, Itagaki Takahiro wrote:
I'm thinking about only COPY FROM for reads, but if someone wants to add
BOM in COPY TO, we might also support COPY TO WITH BOM for writes.
>
> "David E. Wheeler"
> writes:
>> On Sep 25, 2011, at 9:58 PM, Itagaki Takahiro wrote:
>>> I'm thinking about only COPY FROM for reads, but if someone wants to add
>>> BOM in COPY TO, we might also support COPY TO WITH BOM for writes.
>
>> I think it would have to be optional, since "some recip
> I'd like to support UTF-8 text or csv files that has BOM (byte order mark)
> in COPY FROM command. BOM will be automatically detected and ignored
> if the file encoding is UTF-8. WIP patch attached.
>From RFC3629(http://tools.ietf.org/html/rfc3629#section-6):
o A protocol SHOULD forbid use of
"David E. Wheeler"
writes:
> On Sep 25, 2011, at 9:58 PM, Itagaki Takahiro wrote:
>> I'm thinking about only COPY FROM for reads, but if someone wants to add
>> BOM in COPY TO, we might also support COPY TO WITH BOM for writes.
> I think it would have to be optional, since "some recipients of U
On 09/26/2011 07:12 AM, Magnus Hagander wrote:
On Mon, Sep 26, 2011 at 06:58, Itagaki Takahiro
wrote:
Hi,
I'd like to support UTF-8 text or csv files that has BOM (byte order mark)
in COPY FROM command. BOM will be automatically detected and ignored
if the file encoding is UTF-8. WIP patch
On Mon, Sep 26, 2011 at 13:36, Itagaki Takahiro
wrote:
> On Mon, Sep 26, 2011 at 20:12, Magnus Hagander wrote:
>> I like it in general. But if we're looking at the BOM, shouldn't we
>> also look and *reject* the file if it's a BOM for a non-UTF8 file? Say
>> if the BOM claims it's UTF16?
>
> -1 b
On Mon, Sep 26, 2011 at 20:12, Magnus Hagander wrote:
> I like it in general. But if we're looking at the BOM, shouldn't we
> also look and *reject* the file if it's a BOM for a non-UTF8 file? Say
> if the BOM claims it's UTF16?
-1 because we're depending on manual configuration for now.
It would
On Mon, Sep 26, 2011 at 06:58, Itagaki Takahiro
wrote:
> Hi,
>
> I'd like to support UTF-8 text or csv files that has BOM (byte order mark)
> in COPY FROM command. BOM will be automatically detected and ignored
> if the file encoding is UTF-8. WIP patch attached.
>
> I'm thinking about only COPY F
On Sep 25, 2011, at 9:58 PM, Itagaki Takahiro wrote:
> I'd like to support UTF-8 text or csv files that has BOM (byte order mark)
> in COPY FROM command. BOM will be automatically detected and ignored
> if the file encoding is UTF-8. WIP patch attached.
By my reading of http://unicode.org/faq/utf
Hi,
I'd like to support UTF-8 text or csv files that has BOM (byte order mark)
in COPY FROM command. BOM will be automatically detected and ignored
if the file encoding is UTF-8. WIP patch attached.
I'm thinking about only COPY FROM for reads, but if someone wants to add
BOM in COPY TO, we might
28 matches
Mail list logo
