Hi All,
I created “Copy Input” extractor to get key value pair, here is my message,
trying to extract level and status from all the messages
2016-09-28 19:21:52,466 level=INFO tag="run_workflow.py" msg="Run complete for
appname=cils, job_date=20160912, status=Passed starttime=Wed Sep 28 19:15:25
2016, endtime=Wed Sep 28 19:21:47 2016, duration=0:6:21, inputs=[{"path":
"/esss/srg/20160912_1473688239855_4f29bbb6efdb3c39", "tag": "cilPurge",
"stats": {"size": "13.47MB"}}, {"path": "/compressed/cil/20160912", "tag":
"cil", "stats": {"size": "580.16MB"}}], outputs=[{"path":
"/processed/test/parse//cil", "tag": "cil.output.folder", "stats": {"diffSize":
"645.78MB", "newFiles": [], "endSize": "18.93GB", "startSize": "18.30GB"}}]”
Its failing with below error
0]: index [graylog_8], type [message], id
[c0e7ea80-f572-11e6-b21e-5254007b267d], message
[java.lang.IllegalArgumentException: Document contains at least one immense
term in field="level" (whose UTF8 encoding is longer than the max length
32766), all of which were skipped. Please correct the analyzer to not produce
such terms. The prefix of the first immense term is: '[70, 101, 98, 32, 49,
55, 32, 49, 57, 58, 51, 57, 58, 51, 53, 32, 115, 106, 101, 108, 107, 51, 51,
32, 115, 121, 115, 116, 101, 109]...', original message: bytes can be at most
32766 in length; got 34944]
2017-02-17T19:39:56.795-05:00 ERROR [Extractor] Could not apply converter
[tokenizer] of extractor [77e451d0-f3b9-11e6-b21e-5254007b267d].
java.lang.IllegalArgumentException: Multiple entries with same key: id=4038,
and id=4038,
at
com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:136)
~[graylog.jar:?]
at
com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:98)
~[graylog.jar:?]
at
com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:84)
~[graylog.jar:?]
at
com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:295)
~[graylog.jar:?]
at
org.graylog2.inputs.converters.TokenizerConverter.convert(TokenizerConverter.java:55)
~[graylog.jar:?]
at
org.graylog2.plugin.inputs.Extractor.runConverters(Extractor.java:242)
[graylog.jar:?]
at
org.graylog2.plugin.inputs.Extractor.runExtractor(Extractor.java:228)
[graylog.jar:?]
at org.graylog2.filters.ExtractorFilter.filter(ExtractorFilter.java:73)
[graylog.jar:?]
at
org.graylog2.messageprocessors.MessageFilterChainProcessor.process(MessageFilterChainProcessor.java:100)
[graylog.jar:?]
at
org.graylog2.buffers.processors.ServerProcessBufferProcessor.handleMessage(ServerProcessBufferProcessor.java:56)
[graylog.jar:?]
at
org.graylog2.shared.buffers.processors.ProcessBufferProcessor.dispatchMessage(ProcessBufferProcessor.java:82)
[graylog.jar:?]
at
org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:61)
[graylog.jar:?]
at
org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:35)
[graylog.jar:?]
at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143)
[graylog.jar:?]
at
com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66)
[graylog.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]
2017-02-17T19:39:57.677-05:00 ERROR [Messages] Failed to index [1] messages.
Please check the index error log in your web interface for the reason. Error:
failure in bulk execution:
[0]: index [graylog_8], type [message], id
[c560a160-f572-11e6-b21e-5254007b267d], message
[java.lang.IllegalArgumentException: Document contains at least one immense
term in field="level" (whose UTF8 encoding is longer than the max length
32766), all of which were skipped. Please correct the analyzer to not produce
such terms. The prefix of the first immense term is: '[70, 101, 98, 32, 49,
55, 32, 49, 57, 58, 51, 57, 58, 52, 53, 32, 115, 106, 101, 108, 107, 51, 51,
32, 115, 121, 115, 116, 101, 109]...', original message: bytes can be at most
32766 in length; got 34944]
^Z
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/3475055D-8B53-48B8-8A48-61887A5A1EC1%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
