On Sunday, January 14, 2018 at 12:33:46 AM UTC-5, Tong Sun wrote: > > > Token should be called at most once per Next call, > > Ah, that's exactly what my mistake was -- I use Token() in the loop the > standard way, but in the element handling, I used function calls, within > which I use the passed tokenizer to get the Token() again. > > Not being able to do that, I have to save all the Token() info to > different variables, then pass all those variables to my function > *separately*, instead of passing merely a *single* tokenizer. >
Actually, found out I only called Token() once: https://play.golang.org/p/HtevQ3RbQsi reader := strings.NewReader("<div class=\"hello\">SomeText</div>") tokenizer := html.NewTokenizer(reader) tokenizer.Next() fmt.Println(tokenizer.TagName()) fmt.Println(tokenizer.Token()) I.e., what I used *in the loop* the standard way was `TagName()`: case html.StartTagToken, html.EndTagToken: tn, _ := z.TagName() tag := strings.ToLower(string(tn)) if tt == html.StartTagToken { if tag == "body" { depth = 0 } printElmt(depth, z) ... But by the time I need to call Token() *within my function (*printElmt*)* to get the full token info, it's already impossible. Thanks for helping! > > > On Sat, Jan 13, 2018 at 5:19 PM, Nigel Tao wrote: > >> On Fri, Jan 12, 2018 at 2:24 PM, Tong Sun wrote: >> > but I really suggest that you give it a test. >> >> It is already tested directly by func TestTokenizer in token_test.go >> >> The Tokenizer.Text method implementation does not call Tokenizer.Next, >> and only the Next method moves on to the next token. It is simply a >> misleading comment, possibly a relic from an earlier version of the >> API. >> >> If you can reproduce your code that gave you unexpected results, I'm >> happy to look at it. Keep in mind that the package doc comment (the >> top of https://godoc.org/golang.org/x/net/html ) details the intended >> programming model. For example, near "In EBNF notation", it states >> that Token should be called at most once per Next call, so the >> https://play.golang.org/p/7ZwQB98kHE example is mis-using the API. >> > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
