Figured it out--parse_date needed the timestamp . New rule looks like this:
rule "WO-CS-RAS"
when
contains(to_string($message.file),"centralserver\\ras-server\\log\\ras_cs_")
then
set_field("WO_Log_Source","RAS-CS");
let matches = grok(pattern: "%{WO_CS_RAS_CS_MESSAGE}", value:
to_string($message.message));
set_fields(matches);
let date = parse_date(to_string($message.WO_Timestamp), "YYYY-MM-dd
HH:mm:ss,SSS", "EST");
set_field("timestamp", date);
route_to_stream("WideOrbit Logs");
end
I was under the impression that the timezone was optional?
Thanks for all your help with this Jochen--it's greatly appreciated!
Cheers,
Al
On Wednesday, February 8, 2017 at 11:05:22 AM UTC-5, Al Reynolds wrote:
>
> That's what I get for typing it out...thank you for catching that!
> Unfortunately, even after correcting for the incorrect milliseconds value,
> it's still not replacing timestamp value. I sent the parsed date to a new
> field (in this case, "log_timestamp") to verify that the output data was in
> the correct format, which it is now, but it still won't replace the
> timestamp field.
>
> Message sample with "log_timestamp" field:
> WO_CS_RAS_CS_MESSAGE
> 2017-02-08 11:00:34,980 WARN [Task 'ATLANTA-FS' FS timer.1]
> FriendshipTasksServiceImpl = Could not obtain task info for:
> 2c95ac8e-57e3-91b2-0158-495b880b24e8REQUEST FAILED ==> STATUS CODE: 404,
> RESPONSE BODY:
> WO_LogLevel
> WARN
> WO_Log_Source
> RAS-CS
> WO_Message
> Could not obtain task info for: 2c95ac8e-57e3-91b2-0158-
> 495b880b24e8REQUEST FAILED ==> STATUS CODE: 404, RESPONSE BODY:
> WO_Process
> Task 'ATLANTA-FS' FS timer.1
> WO_SubProcess
> FriendshipTasksServiceImpl
> WO_Timestamp
> 2017-02-08 11:00:34,980
> facility
> filebeat
> file
> d:\centralserver\ras-server\log\ras_cs_WO-ATL-CS.log
> input_type
> log
> log_timestamp
> 2017-02-08T11:00:34.980Z
> message
> 2017-02-08 11:00:34,980 WARN [Task 'ATLANTA-FS' FS timer.1]
> FriendshipTasksServiceImpl = Could not obtain task info for:
> 2c95ac8e-57e3-91b2-0158-495b880b24e8REQUEST FAILED ==> STATUS CODE: 404,
> RESPONSE BODY:
> name
> WO-ATL-CS
> offset
> 2372156
> source
> WO-ATL-CS
> timestamp
> 2017-02-08T16:00:35.864Z
> type
> log
>
> Corrected rule:
> rule "WO-CS-RAS"
> when
>
> contains(to_string($message.file),"centralserver\\ras-server\\log\\ras_cs_")
> then
> set_field("WO_Log_Source","RAS-CS");
> let matches = grok(pattern: "%{WO_CS_RAS_CS_MESSAGE}", value:
> to_string($message.message));
> set_fields(matches);
> let date = parse_date(to_string($message.WO_Timestamp), "YYYY-MM-dd
> HH:mm:ss,SSS");
> set_field("timestamp", date);
> route_to_stream("WideOrbit Logs");
> end
>
> Thanks!
>
> Cheers,
> Al
>
> On Wednesday, February 8, 2017 at 10:55:03 AM UTC-5, Jochen Schalanda
> wrote:
>>
>> Hi Al,
>>
>> On Wednesday, 8 February 2017 15:46:07 UTC+1, Al Reynolds wrote:
>>>
>>> WO_Timestamp
>>> 2017-02-08 09:42:30,056
>>>
>>> Those messages are with the date parsing disabled. I'm attempting to
>>> replace "timestamp" with the "WO_Timestamp" field.
>>>
>>
>> The string in WO_Timestamp doesn't match the pattern "YYYY-MM-dd
>> HH:mm:ss,sss" used in parse_date(). See
>> http://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html
>>
>> for details.
>>
>> Hint: 's' and 'S' are not the same thing.
>>
>>
>> Side note: The full_message field is empty on my filebeat inputs--is that
>>> expected behavior?
>>>
>>
>> Yes, that's expected.
>>
>> What would you expect to find in the (optional) full_message field?
>>
>> Cheers,
>> Jochen
>>
>
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/a3b908a8-4993-4107-87f8-55445605cc66%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
