On Sat, Nov 9, 2024, at 08:07, Joel Jacobson wrote: > Here's a draft of an idea I'm considering (not yet implemented):
I realize the last part about optional quoting is unnecessary, since if quoting is desired, users could just use the 'csv' format. Revised draft of the idea (not yet implemented): - Fast path for newline-free types: For the list of built-in types where we know the ::text representation cannot contain newlines, we take the fast path in NextCopyFromRawFields(), pointing cstate->raw_fields[0] directly to cstate->line_buf.data. - Handling newlines for other types: For any other types, we would need to scan the string for newline characters. If a newline is encountered, it results in an error. This brings up the question on what to offer users wanting to export text values containing newlines. To address this need, I think that's out of scope for the 'list' format, and is better handled by a separate 'value' format: - Such a format would be specialized for exporting a value "as is" to a file, or importing en entire file as a single value. - Such a value could be a physical single-column single-row, but could also be constructed using e.g. string_agg(). - The 'value' format could also easily support import/export binary data (bytea), to e.g. allow importing/exporting images, etc. Dimensionality perspective on formats: 2D formats: 'text', 'csv', 'binary' (tabular formats) 1D format: 'list' (single-column) 0D format: 'value' (single-column, single-row) /Joel
