> Token should be called at most once per Next call, Ah, that's exactly what my mistake was -- I use Token() in the loop the standard way, but in the element handling, I used function calls, within which I use the passed tokenizer to get the Token() again.
Not being able to do that, I have to save all the Token() info to different variables, then pass all those variables to my function *separately*, instead of passing merely a *single* tokenizer. Thanks for helping! On Sat, Jan 13, 2018 at 5:19 PM, Nigel Tao <nigel...@golang.org> wrote: > On Fri, Jan 12, 2018 at 2:24 PM, Tong Sun <suntong...@gmail.com> wrote: > > but I really suggest that you give it a test. > > It is already tested directly by func TestTokenizer in token_test.go > > The Tokenizer.Text method implementation does not call Tokenizer.Next, > and only the Next method moves on to the next token. It is simply a > misleading comment, possibly a relic from an earlier version of the > API. > > If you can reproduce your code that gave you unexpected results, I'm > happy to look at it. Keep in mind that the package doc comment (the > top of https://godoc.org/golang.org/x/net/html ) details the intended > programming model. For example, near "In EBNF notation", it states > that Token should be called at most once per Next call, so the > https://play.golang.org/p/7ZwQB98kHE example is mis-using the API. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
