On Sun, Jul 12, 2020 at 10:42 PM Gobin Sougrakpam
<gobinsougrak...@gmail.com> wrote:
>
> I encountered this error but was able to fix it after setting the
> scanner.Buffer size to a rather large number.
>
> bufio.Scanner: token too long
...
> Now, the question is when I run this function successfully, the first token
> that is generated from the split function is 637 bytes.
> What is taking up the buffer that I am getting the error when I set the
> buffer to smaller values?
Please include code as plain text or as a link to the Go playground,
not in reverse. Plain text is easier to read in general. Thanks.
I don't understand the split function that you showed:
splitFunc := func(data []byte, atEOF bool) (advance int, token []byte,
err error) {
for i := 0; i < len(data); i++ {
if data[i] == ',' {
return i + 1, data[:i], nil
}
if !atEOF {
return 0, nil, nil
}
}
return 0, data, bufio.ErrFinalToken
}
Unless the very first character is a comma, that function is going to
keep returning 0, nil, nil until all the data is pulled into the
buffer. Perhaps you meant to put the atEOF check outside of the loop
over data.
Ian
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXUHd8%2BpdWVJfgoK5qo2ww52zfV_esYgFmGkOtTeCazdw%40mail.gmail.com.
