2020. július 13., hétfő 7:42:13 UTC+2 időpontban Gobin Sougrakpam a
következőt írta:
>
> Hi Folks,
>
>
> I encountered this error but was able to fix it after setting the
> scanner.Buffer size to a rather large number.
>
> *bufio.Scanner: token too long*
>
> Here is my fixed function:
>
> func scanFile(f *os.File) error {
> scanner := bufio.NewScanner(f)
> scanner.Buffer(make([]byte, 0, 819200), 819200)
> splitFunc := func(data []byte, atEOF bool) (advance int, token []byte,
> err error) {
> for i := 0; i < len(data); i++ {
> if data[i] == ',' {
> return i + 1, data[:i], nil
> }
> if !atEOF {
> return 0, nil, nil
> }
> }
> return 0, data, bufio.ErrFinalToken
> }
> scanner.Split(splitFunc)
> newfile, err := os.Create("test/new.txt")
> if err != nil {
> return err
> }
> writer := bufio.NewWriter(newfile)
> for scanner.Scan() {
> n, err := writer.Write(scanner.Bytes())
> if err != nil {
> return err
> }
> fmt.Printf("%d bytes written\n", n)
> }
>
> if err := scanner.Err(); err != nil {
> return err
> }
> return nil
> }
>
>
> Now, the question is when I run this function successfully, the first
> token that is generated from the split function is 637 bytes.
> What is taking up the buffer that I am getting the error when I set the
> buffer to smaller values?
>
> Thanks.
>
>
>
Your splitFunc always returns "0, nil, nil" if data[0]!=','.
Use "i := bytes.IndexByte(data, ',')" instead of this for cycle.
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/9600e82c-379b-40fc-99f8-de7e6105b291o%40googlegroups.com.
