Date

Mutate Elastic Compatible

Synopsis

Parses dates from a field in various formats into a standardized timestamp.

Schema

date:
  - field: <ident>
  - formats: <enum[]>
  - description: <text>
  - if: <script>
  - ignore_failure: <boolean>
  - locale: <string>
  - on_failure: <processor[]>
  - on_success: <processor[]>
  - output_format: <string>
  - tag: <string>
  - target_field: <ident>
  - timezone: <string>

Configuration

Field	Required	Default	Description
field	Y	-	Field containing the date to parse
formats	N	-	Array of expected date formats. Supports: standard Go time layouts, ISO8601, UNIX, UNIX_MS
description	N	-	Explanatory note
if	N	-	Condition to run
ignore_failure	N	false	Continue processing if parsing fails
locale	N	-	Locale for date parsing (e.g., "en_US")
on_failure	N	-	See Handling Failures
on_success	N	-	See Handling Success
output_format	N	2006-01-02T15:04:05.000Z07:00	Format pattern for the output timestamp
tag	N	-	Identifier
target_field	N	@timestamp	Field to store the parsed date
timezone	N	UTC	Timezone for parsing (e.g., "Europe/Amsterdam")

Details

The processor features an advanced timestamp parser that can automatically detect and parse a wide variety of date/time formats without requiring explicit format specification. It supports nearly all timestamp formats, including:

Years	YYYY, YY, yyyy, yy
Months	MMMM (full name), MMM (abbreviated), MM (zero-padded), M
Days	DD, D, dd, d
Hours	HH (24-hour), hh (12-hour), h
Minutes / Seconds	mm, m, ss, s
Milliseconds	S, SSS, SSSSSS, SSSSSSSSS
Timezones	Z, ZZ, ZZZ
AM / PM	A, a

Human-readable and machine timestamps, as well as Java time patterns and Go time layouts are supported. Numeric timestamps (i.e. Unix epoch in seconds/milliseconds) are handled.

Output formats and timezones are customizable. Also, templates can be used for dynamic timezone and locale selection.

note

By default, the parsed date is stored in the @timestamp field, but this can be customized using target_field.

warning

Date parsing will fail if none of the specified formats match the input. Set ignore_failure to true if errors should be tolerated.

Examples

Standard Dates

Parsing a date using the standard format...	{ "initial_date": "25/04/2016 12:02:01" } date: - field: initial_date - target_field: timestamp - formats: ["02/01/2006 15:04:05"] - timezone: "Europe/Amsterdam"
converts it to the ISO format with timezone:	{ "initial_date": "25/04/2016 12:02:01", "timestamp": "2016-04-25T12:02:01.000+02:00" }

Automatic Detection

Parsing various timestamps without explicit specs...	{ "date1": "2023-12-29_15:04:05", "date2": "29/Dec/2023:15:04:05 +0000", "date3": "Fri Dec 29 15:04:05 2023", "date4": "1461578521", "date5": "1461578521000" } date: - field: date1 - target_field: time1 date: - field: date2 - target_field: time2 date: - field: date3 - target_field: time3 date: - field: date4 - target_field: time4 date: - field: date5 - target_field: time5
detects and normalizes all dates automatically:	{ "time1": "2023-12-29T15:04:05.000Z", "time2": "2023-12-29T15:04:05.000Z", "time3": "2023-12-29T15:04:05.000Z", "time4": "2016-04-25T12:02:01.000Z", "time5": "2016-04-25T12:02:01.000Z" }

Nested Fields with ISO8601

Processing dates in nested objects...	{ "user": { "login_date": "2016-04-25T12:02:01.789Z" } } date: - field: user.login_date - target_field: timestamp - formats: ["2006-01-02T15:04:05.000Z07:00"]
maintains the correct timestamp:	{ "user": { "login_date": "2016-04-25T12:02:01.789Z" }, "timestamp": "2016-04-25T12:02:01.789Z" }

Output Formats

Specifying a custom format...	{ "initial_date": "25/04/2016 12:02:01" } date: - field: initial_date - target_field: timestamp - formats: ["02/01/2006 15:04:05"] - output_format: "2006-01-02 15:04:05"
generates the specified output:	{ "initial_date": "25/04/2016 12:02:01", "timestamp": "2016-04-25 12:02:01" }

Dynamic Timezone and Locale

Template fields for timezone and locale...	{ "event_date": "2016-04-25 12:02:01", "tz": "Europe/London", "lang": "en_GB" } date: - field: event_date - target_field: timestamp - formats: ["2006-01-02 15:04:05"] - timezone: "{{{tz}}}" - locale: "{{{lang}}}"
can be used for parsing:	{ "event_date": "2016-04-25 12:02:01", "tz": "Europe/London", "lang": "en_GB", "timestamp": "2016-04-25T12:02:01.000+01:00" }