[jira] [Created] (CAMEL-19094) Tokenizer ignores includeTokens

[Pasquale Congiusti (Jira)]

Pasquale Congiusti created CAMEL-19094:
------------------------------------------

             Summary: Tokenizer ignores includeTokens
                 Key: CAMEL-19094
                 URL: https://issues.apache.org/jira/browse/CAMEL-19094
             Project: Camel
          Issue Type: Bug
    Affects Versions: 3.20.1
            Reporter: Pasquale Congiusti


Hi folks,
maybe I am using it in a wrong way, but it seems to me that the Tokenizer is 
not working as it should when we try to use IncludeTokens option. I'm testing 
the following:

{code}
        TokenizeLanguage lan = new TokenizeLanguage();
        lan.setIncludeTokens(true);
        lan.setToken("el");

        from("timer:java?period={{time:1000}}").routeId("java")
            .setBody()
                .simple("Hello Camel from ${routeId}")
            //.split(body().tokenize("el"))
            .split(lan.createExpression())
            .log("${body}");
{code}

But I get

{code}
2023-02-27 15:13:27.462  INFO 596799 --- [ - timer://java] Test.java:22         
               : H
2023-02-27 15:13:27.464  INFO 596799 --- [ - timer://java] Test.java:22         
               : lo Cam
2023-02-27 15:13:27.464  INFO 596799 --- [ - timer://java] Test.java:22         
               :  from java
2023-02-27 15:13:28.428  INFO 596799 --- [ - timer://java] Test.java:22         
               : H
2023-02-27 15:13:28.430  INFO 596799 --- [ - timer://java] Test.java:22         
               : lo Cam
2023-02-27 15:13:28.431  INFO 596799 --- [ - timer://java] Test.java:22         
               :  from java
2023-02-27 15:13:29.428  INFO 596799 --- [ - timer://java] Test.java:22         
               : H
2023-02-27 15:13:29.430  INFO 596799 --- [ - timer://java] Test.java:22         
               : lo Cam
2023-02-27 15:13:29.431  INFO 596799 --- [ - timer://java] Test.java:22         
               :  from java
{code}

I'd expect there to get "el" token as well.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)