Hi Rayees,
which version of Graylog and Elasticsearch are you running?
Cheers,
Jochen
On Saturday, 18 February 2017 01:46:16 UTC+1, Rayees Namathponnan wrote:
>
> Hi All,
>
> I created “Copy Input” extractor to get key value pair, here is my
> message, trying to extract *level* and *status* from all the messages
>
>
>
> 2016-09-28 19:21:52,466 level=INFO tag="run_workflow.py" msg="Run complete
> for appname=cils, job_date=20160912, status=Passed starttime=Wed Sep 28
> 19:15:25 2016, endtime=Wed Sep 28 19:21:47 2016, duration=0:6:21,
> inputs=[{"path": "/esss/srg/20160912_1473688239855_4f29bbb6efdb3c39",
> "tag": "cilPurge", "stats": {"size": "13.47MB"}}, {"path":
> "/compressed/cil/20160912", "tag": "cil", "stats": {"size": "580.16MB"}}],
> outputs=[{"path": "/processed/test/parse//cil", "tag": "cil.output.folder",
> "stats": {"diffSize": "645.78MB", "newFiles": [], "endSize": "18.93GB",
> "startSize": "18.30GB"}}]”
>
>
>
> Its failing with below error
>
> 0]: index [graylog_8], type [message], id
> [c0e7ea80-f572-11e6-b21e-5254007b267d], message
> [java.lang.IllegalArgumentException: Document contains at least one immense
> term in field="level" (whose UTF8 encoding is longer than the max length
> 32766), all of which were skipped. Please correct the analyzer to not
> produce such terms. The prefix of the first immense term is: '[70, 101,
> 98, 32, 49, 55, 32, 49, 57, 58, 51, 57, 58, 51, 53, 32, 115, 106, 101, 108,
> 107, 51, 51, 32, 115, 121, 115, 116, 101, 109]...', original message: bytes
> can be at most 32766 in length; got 34944]
> 2017-02-17T19:39:56.795-05:00 ERROR [Extractor] Could not apply converter
> [tokenizer] of extractor [77e451d0-f3b9-11e6-b21e-5254007b267d].
> java.lang.IllegalArgumentException: Multiple entries with same key:
> id=4038, and id=4038,
> at
> com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:136)
> ~[graylog.jar:?]
> at
> com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:98)
>
> ~[graylog.jar:?]
> at
> com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:84)
>
> ~[graylog.jar:?]
> at
> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:295)
> ~[graylog.jar:?]
> at
> org.graylog2.inputs.converters.TokenizerConverter.convert(TokenizerConverter.java:55)
>
> ~[graylog.jar:?]
> at org.graylog2.plugin.inputs.Extractor.runConverters(Extractor.java:242)
> [graylog.jar:?]
> at org.graylog2.plugin.inputs.Extractor.runExtractor(Extractor.java:228)
> [graylog.jar:?]
> at org.graylog2.filters.ExtractorFilter.filter(ExtractorFilter.java:73)
> [graylog.jar:?]
> at
> org.graylog2.messageprocessors.MessageFilterChainProcessor.process(MessageFilterChainProcessor.java:100)
>
> [graylog.jar:?]
> at
> org.graylog2.buffers.processors.ServerProcessBufferProcessor.handleMessage(ServerProcessBufferProcessor.java:56)
>
> [graylog.jar:?]
> at
> org.graylog2.shared.buffers.processors.ProcessBufferProcessor.dispatchMessage(ProcessBufferProcessor.java:82)
>
> [graylog.jar:?]
> at
> org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:61)
>
> [graylog.jar:?]
> at
> org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:35)
>
> [graylog.jar:?]
> at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143)
> [graylog.jar:?]
> at
> com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66)
>
> [graylog.jar:?]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]
> 2017-02-17T19:39:57.677-05:00 ERROR [Messages] Failed to index [1]
> messages. Please check the index error log in your web interface for the
> reason. Error: failure in bulk execution:
> [0]: index [graylog_8], type [message], id
> [c560a160-f572-11e6-b21e-5254007b267d], message
> [java.lang.IllegalArgumentException: Document contains at least one immense
> term in field="level" (whose UTF8 encoding is longer than the max length
> 32766), all of which were skipped. Please correct the analyzer to not
> produce such terms. The prefix of the first immense term is: '[70, 101,
> 98, 32, 49, 55, 32, 49, 57, 58, 51, 57, 58, 52, 53, 32, 115, 106, 101, 108,
> 107, 51, 51, 32, 115, 121, 115, 116, 101, 109]...', original message: bytes
> can be at most 32766 in length; got 34944]
> ^Z
>
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/013d028e-f053-4b5f-8da6-18ded9bdd073%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
