Hi Al,
you might want to try to use your pattern with lower case 'y' for the year
component of the date pattern.
Cheers,
Jochen
On Wednesday, 8 February 2017 21:09:19 UTC+1, Al Reynolds wrote:
>
> I've noticed another error. The timestamp field is being replaced
> correctly, but the "gl2_processing_error" field is showing the following
> error (on all messages):
> For rule 'WO-CS-RAS': In call to function 'parse_date' at 8:15 an
> exception was thrown: Invalid format: "2017-02-08 15:05:59,170" is
> malformed at "17-02-08 15:05:59,170"
>
>
> It doesn't seem to have any adverse effects, but I'm curious as to what
> might be causing it?
>
> On Wednesday, February 8, 2017 at 1:56:17 PM UTC-5, Al Reynolds wrote:
>>
>> Figured it out--parse_date needed the timestamp . New rule looks like
>> this:
>> rule "WO-CS-RAS"
>> when
>>
>> contains(to_string($message.file),"centralserver\\ras-server\\log\\ras_cs_")
>> then
>> set_field("WO_Log_Source","RAS-CS");
>> let matches = grok(pattern: "%{WO_CS_RAS_CS_MESSAGE}", value:
>> to_string($message.message));
>> set_fields(matches);
>> let date = parse_date(to_string($message.WO_Timestamp), "YYYY-MM-dd
>> HH:mm:ss,SSS", "EST");
>> set_field("timestamp", date);
>> route_to_stream("WideOrbit Logs");
>> end
>>
>> I was under the impression that the timezone was optional?
>>
>> Thanks for all your help with this Jochen--it's greatly appreciated!
>>
>> Cheers,
>> Al
>>
>> On Wednesday, February 8, 2017 at 11:05:22 AM UTC-5, Al Reynolds wrote:
>>>
>>> That's what I get for typing it out...thank you for catching that!
>>> Unfortunately, even after correcting for the incorrect milliseconds value,
>>> it's still not replacing timestamp value. I sent the parsed date to a new
>>> field (in this case, "log_timestamp") to verify that the output data was in
>>> the correct format, which it is now, but it still won't replace the
>>> timestamp field.
>>>
>>> Message sample with "log_timestamp" field:
>>> WO_CS_RAS_CS_MESSAGE
>>> 2017-02-08 11:00:34,980 WARN [Task 'ATLANTA-FS' FS timer.1]
>>> FriendshipTasksServiceImpl = Could not obtain task info for:
>>> 2c95ac8e-57e3-91b2-0158-495b880b24e8REQUEST FAILED ==> STATUS CODE: 404,
>>> RESPONSE BODY:
>>> WO_LogLevel
>>> WARN
>>> WO_Log_Source
>>> RAS-CS
>>> WO_Message
>>> Could not obtain task info for: 2c95ac8e-57e3-91b2-0158-
>>> 495b880b24e8REQUEST FAILED ==> STATUS CODE: 404, RESPONSE BODY:
>>> WO_Process
>>> Task 'ATLANTA-FS' FS timer.1
>>> WO_SubProcess
>>> FriendshipTasksServiceImpl
>>> WO_Timestamp
>>> 2017-02-08 11:00:34,980
>>> facility
>>> filebeat
>>> file
>>> d:\centralserver\ras-server\log\ras_cs_WO-ATL-CS.log
>>> input_type
>>> log
>>> log_timestamp
>>> 2017-02-08T11:00:34.980Z
>>> message
>>> 2017-02-08 11:00:34,980 WARN [Task 'ATLANTA-FS' FS timer.1]
>>> FriendshipTasksServiceImpl = Could not obtain task info for:
>>> 2c95ac8e-57e3-91b2-0158-495b880b24e8REQUEST FAILED ==> STATUS CODE: 404,
>>> RESPONSE BODY:
>>> name
>>> WO-ATL-CS
>>> offset
>>> 2372156
>>> source
>>> WO-ATL-CS
>>> timestamp
>>> 2017-02-08T16:00:35.864Z
>>> type
>>> log
>>>
>>> Corrected rule:
>>> rule "WO-CS-RAS"
>>> when
>>>
>>> contains(to_string($message.file),"centralserver\\ras-server\\log\\ras_cs_")
>>> then
>>> set_field("WO_Log_Source","RAS-CS");
>>> let matches = grok(pattern: "%{WO_CS_RAS_CS_MESSAGE}", value:
>>> to_string($message.message));
>>> set_fields(matches);
>>> let date = parse_date(to_string($message.WO_Timestamp), "YYYY-MM-dd
>>> HH:mm:ss,SSS");
>>> set_field("timestamp", date);
>>> route_to_stream("WideOrbit Logs");
>>> end
>>>
>>> Thanks!
>>>
>>> Cheers,
>>> Al
>>>
>>> On Wednesday, February 8, 2017 at 10:55:03 AM UTC-5, Jochen Schalanda
>>> wrote:
>>>>
>>>> Hi Al,
>>>>
>>>> On Wednesday, 8 February 2017 15:46:07 UTC+1, Al Reynolds wrote:
>>>>>
>>>>> WO_Timestamp
>>>>> 2017-02-08 09:42:30,056
>>>>>
>>>>> Those messages are with the date parsing disabled. I'm attempting to
>>>>> replace "timestamp" with the "WO_Timestamp" field.
>>>>>
>>>>
>>>> The string in WO_Timestamp doesn't match the pattern "YYYY-MM-dd
>>>> HH:mm:ss,sss" used in parse_date(). See
>>>> http://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html
>>>>
>>>> for details.
>>>>
>>>> Hint: 's' and 'S' are not the same thing.
>>>>
>>>>
>>>> Side note: The full_message field is empty on my filebeat inputs--is
>>>>> that expected behavior?
>>>>>
>>>>
>>>> Yes, that's expected.
>>>>
>>>> What would you expect to find in the (optional) full_message field?
>>>>
>>>> Cheers,
>>>> Jochen
>>>>
>>>
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/561bf48b-bfdc-4ee6-b0a0-f7e706dc1e5c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
