Re: Speed up COPY FROM text/CSV parsing using SIMD

2025-08-21 Thread KAZAR Ayoub
> On Thu, 14 Aug 2025 at 18:00, KAZAR Ayoub wrote: > >> Thanks for running that benchmark! Would you mind sharing a reproducer > >> for the regression you observed? > > > > Of course, I attached the sql to generate the text and csv test files. > > If having a 1/3 of line length of special characte

Re: Speed up COPY FROM text/CSV parsing using SIMD

2025-08-21 Thread Andrew Dunstan
On 2025-08-19 Tu 10:14 AM, Nazir Bilal Yavuz wrote: Hi, On Tue, 19 Aug 2025 at 15:33, Nazir Bilal Yavuz wrote: I am able to reproduce the regression you mentioned but both regressions are %20 on my end. I found that (by experimenting) SIMD causes a regression if it advances less than 5 chara

Re: Speed up COPY FROM text/CSV parsing using SIMD

2025-08-19 Thread Nazir Bilal Yavuz
Hi, On Tue, 19 Aug 2025 at 15:33, Nazir Bilal Yavuz wrote: > > I am able to reproduce the regression you mentioned but both > regressions are %20 on my end. I found that (by experimenting) SIMD > causes a regression if it advances less than 5 characters. > > So, I implemented a small heuristic. I

Re: Speed up COPY FROM text/CSV parsing using SIMD

2025-08-19 Thread Nazir Bilal Yavuz
Hi, On Thu, 14 Aug 2025 at 18:00, KAZAR Ayoub wrote: >> Thanks for running that benchmark! Would you mind sharing a reproducer >> for the regression you observed? > > Of course, I attached the sql to generate the text and csv test files. > If having a 1/3 of line length of special characters can

Re: Speed up COPY FROM text/CSV parsing using SIMD

2025-08-19 Thread Ants Aasma
On Thu, 7 Aug 2025 at 14:15, Nazir Bilal Yavuz wrote: > I have a couple of ideas that I was working on: > --- > > + * However, SIMD optimization cannot be applied in the following > cases: > + * - Inside quoted fields, where escape sequences and closing quotes > + * requ

Re: Speed up COPY FROM text/CSV parsing using SIMD

2025-08-14 Thread KAZAR Ayoub
> Hi, > > On Thu, 14 Aug 2025 at 05:25, KAZAR Ayoub wrote: > > > > Following Nazir's findings about 4096 bytes being the performant line > length, I did more benchmarks from my side on both TEXT and CSV formats > with two different cases of normal data (no special characters) and data > with many

Re: Speed up COPY FROM text/CSV parsing using SIMD

2025-08-14 Thread Nazir Bilal Yavuz
Hi, On Thu, 14 Aug 2025 at 05:25, KAZAR Ayoub wrote: > > Following Nazir's findings about 4096 bytes being the performant line length, > I did more benchmarks from my side on both TEXT and CSV formats with two > different cases of normal data (no special characters) and data with many > specia

Re: Speed up COPY FROM text/CSV parsing using SIMD

2025-08-13 Thread KAZAR Ayoub
Following Nazir's findings about 4096 bytes being the performant line length, I did more benchmarks from my side on both TEXT and CSV formats with two different cases of normal data (no special characters) and data with many special characters. Results are con good as expected and similar to previ

Re: Speed up COPY FROM text/CSV parsing using SIMD

2025-08-12 Thread Shinya Kato
On Tue, Aug 12, 2025 at 4:25 PM Shinya Kato wrote: > > + * However, SIMD optimization cannot be applied in the following > > cases: > > + * - Inside quoted fields, where escape sequences and closing > > quotes > > + * require sequential processing to handle correctly.

Re: Speed up COPY FROM text/CSV parsing using SIMD

2025-08-12 Thread Shinya Kato
On Thu, Aug 7, 2025 at 8:15 PM Nazir Bilal Yavuz wrote: > > Hi, > > Thank you for working on this! > > On Thu, 7 Aug 2025 at 04:49, Shinya Kato wrote: > > > > Hi hackers, > > > > I have implemented SIMD optimization for the COPY FROM (FORMAT {csv, > > text}) command and observed approximately a 5

Re: Speed up COPY FROM text/CSV parsing using SIMD

2025-08-11 Thread Nazir Bilal Yavuz
Hi, On Thu, 7 Aug 2025 at 14:15, Nazir Bilal Yavuz wrote: > > On Thu, 7 Aug 2025 at 04:49, Shinya Kato wrote: > > > > I have implemented SIMD optimization for the COPY FROM (FORMAT {csv, > > text}) command and observed approximately a 5% performance > > improvement. Please see the detailed test

Re: Speed up COPY FROM text/CSV parsing using SIMD

2025-08-07 Thread Nazir Bilal Yavuz
Hi, Thank you for working on this! On Thu, 7 Aug 2025 at 04:49, Shinya Kato wrote: > > Hi hackers, > > I have implemented SIMD optimization for the COPY FROM (FORMAT {csv, > text}) command and observed approximately a 5% performance > improvement. Please see the detailed test results below. I h