I've noticed another error. The timestamp field is being replaced
correctly, but the "gl2_processing_error" field is showing the following
error (on all messages):
For rule 'WO-CS-RAS': In call to function 'parse_date' at 8:15 an exception
was thrown: Invalid format: "2017-02-08 15:05:59,170" is malformed at "17-02-08
15:05:59,170"
It doesn't seem to have any adverse effects, but I'm curious as to what
might be causing it?
On Wednesday, February 8, 2017 at 1:56:17 PM UTC-5, Al Reynolds wrote:
>
> Figured it out--parse_date needed the timestamp . New rule looks like this:
> rule "WO-CS-RAS"
> when
>
> contains(to_string($message.file),"centralserver\\ras-server\\log\\ras_cs_")
> then
> set_field("WO_Log_Source","RAS-CS");
> let matches = grok(pattern: "%{WO_CS_RAS_CS_MESSAGE}", value:
> to_string($message.message));
> set_fields(matches);
> let date = parse_date(to_string($message.WO_Timestamp), "YYYY-MM-dd
> HH:mm:ss,SSS", "EST");
> set_field("timestamp", date);
> route_to_stream("WideOrbit Logs");
> end
>
> I was under the impression that the timezone was optional?
>
> Thanks for all your help with this Jochen--it's greatly appreciated!
>
> Cheers,
> Al
>
> On Wednesday, February 8, 2017 at 11:05:22 AM UTC-5, Al Reynolds wrote:
>>
>> That's what I get for typing it out...thank you for catching that!
>> Unfortunately, even after correcting for the incorrect milliseconds value,
>> it's still not replacing timestamp value. I sent the parsed date to a new
>> field (in this case, "log_timestamp") to verify that the output data was in
>> the correct format, which it is now, but it still won't replace the
>> timestamp field.
>>
>> Message sample with "log_timestamp" field:
>> WO_CS_RAS_CS_MESSAGE
>> 2017-02-08 11:00:34,980 WARN [Task 'ATLANTA-FS' FS timer.1]
>> FriendshipTasksServiceImpl = Could not obtain task info for:
>> 2c95ac8e-57e3-91b2-0158-495b880b24e8REQUEST FAILED ==> STATUS CODE: 404,
>> RESPONSE BODY:
>> WO_LogLevel
>> WARN
>> WO_Log_Source
>> RAS-CS
>> WO_Message
>> Could not obtain task info for: 2c95ac8e-57e3-91b2-0158-
>> 495b880b24e8REQUEST FAILED ==> STATUS CODE: 404, RESPONSE BODY:
>> WO_Process
>> Task 'ATLANTA-FS' FS timer.1
>> WO_SubProcess
>> FriendshipTasksServiceImpl
>> WO_Timestamp
>> 2017-02-08 11:00:34,980
>> facility
>> filebeat
>> file
>> d:\centralserver\ras-server\log\ras_cs_WO-ATL-CS.log
>> input_type
>> log
>> log_timestamp
>> 2017-02-08T11:00:34.980Z
>> message
>> 2017-02-08 11:00:34,980 WARN [Task 'ATLANTA-FS' FS timer.1]
>> FriendshipTasksServiceImpl = Could not obtain task info for:
>> 2c95ac8e-57e3-91b2-0158-495b880b24e8REQUEST FAILED ==> STATUS CODE: 404,
>> RESPONSE BODY:
>> name
>> WO-ATL-CS
>> offset
>> 2372156
>> source
>> WO-ATL-CS
>> timestamp
>> 2017-02-08T16:00:35.864Z
>> type
>> log
>>
>> Corrected rule:
>> rule "WO-CS-RAS"
>> when
>>
>> contains(to_string($message.file),"centralserver\\ras-server\\log\\ras_cs_")
>> then
>> set_field("WO_Log_Source","RAS-CS");
>> let matches = grok(pattern: "%{WO_CS_RAS_CS_MESSAGE}", value:
>> to_string($message.message));
>> set_fields(matches);
>> let date = parse_date(to_string($message.WO_Timestamp), "YYYY-MM-dd
>> HH:mm:ss,SSS");
>> set_field("timestamp", date);
>> route_to_stream("WideOrbit Logs");
>> end
>>
>> Thanks!
>>
>> Cheers,
>> Al
>>
>> On Wednesday, February 8, 2017 at 10:55:03 AM UTC-5, Jochen Schalanda
>> wrote:
>>>
>>> Hi Al,
>>>
>>> On Wednesday, 8 February 2017 15:46:07 UTC+1, Al Reynolds wrote:
>>>>
>>>> WO_Timestamp
>>>> 2017-02-08 09:42:30,056
>>>>
>>>> Those messages are with the date parsing disabled. I'm attempting to
>>>> replace "timestamp" with the "WO_Timestamp" field.
>>>>
>>>
>>> The string in WO_Timestamp doesn't match the pattern "YYYY-MM-dd
>>> HH:mm:ss,sss" used in parse_date(). See
>>> http://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html
>>>
>>> for details.
>>>
>>> Hint: 's' and 'S' are not the same thing.
>>>
>>>
>>> Side note: The full_message field is empty on my filebeat inputs--is
>>>> that expected behavior?
>>>>
>>>
>>> Yes, that's expected.
>>>
>>> What would you expect to find in the (optional) full_message field?
>>>
>>> Cheers,
>>> Jochen
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/34c557b5-8552-405b-8892-04c8eb1ad2b1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
