Body Required

• docs array[object] Required

  Sample documents to test in the pipeline.

  Hide docs attributes Show docs attributes object

  • _id string
  • _index string
  • _source object Required

    JSON body for the document.

• pipeline object Additional properties
  Hide pipeline attributes Show pipeline attributes object

  • description string

    Description of the ingest pipeline.

  • on_failure array[object]

    Processors to run immediately after a processor failure.

    Hide on_failure attributes Show on_failure attributes object

    • append object
      Hide append attributes Show append attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • value object | array[object] Required

        The value to be appended. Supports template snippets.

        One of:

        object-1 object array-2 array[object]
      • allow_duplicates boolean

        If false, the processor does not append values already present in the field.

        Default value is true.

    • attachment object
      Hide attachment attributes Show attachment attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • indexed_chars number

        The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

        Default value is 100000.

      • indexed_chars_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • properties array[string]

        Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • remove_binary boolean

        If true, the binary field will be removed from the document

        Default value is false.

      • resource_name string

        Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

    • bytes object
      Hide bytes attributes Show bytes attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • circle object
      Hide circle attributes Show circle attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • error_distance number Required

        The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • shape_type string Required

        Values are geo_shape or shape.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • community_id object
      Hide community_id attributes Show community_id attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • source_ip string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • source_port string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • destination_ip string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • destination_port string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • iana_number string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • icmp_type string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • icmp_code string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • transport string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • seed number

        Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

        Default value is 0.

      • ignore_missing boolean

        If true and any required fields are missing, the processor quietly exits without modifying the document.

        Default value is true.

    • convert object
      Hide convert attributes Show convert attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • type string Required

        Values are integer, long, double, float, boolean, ip, string, or auto.

    • csv object
      Hide csv attributes Show csv attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • empty_value object

        Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • quote string

        Quote used in CSV, has to be single character string.

        Default value is ".

      • separator string

        Separator used in CSV, has to be single character string.

        Default value is ,.

      • target_fields string | array[string] Required
      • trim boolean

        Trim whitespaces in unquoted fields.

    • date object
      Hide date attributes Show date attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • formats array[string] Required

        An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

      • locale string

        The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

        Default value is ENGLISH.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • timezone string

        The timezone to use when parsing the date. Supports template snippets.

        Default value is UTC.

      • output_format string

        The format to use when writing the date to target_field. Must be a valid java time pattern.

        Default value is yyyy-MM-dd'T'HH:mm:ss.SSSXXX.

    • date_index_name object
      Hide date_index_name attributes Show date_index_name attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • date_formats array[string]

        An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

      • date_rounding string Required

        How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • index_name_format string

        The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

        Default value is yyyy-MM-dd.

      • index_name_prefix string

        A prefix of the index name to be prepended before the printed date. Supports template snippets.

      • locale string

        The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

        Default value is ENGLISH.

      • timezone string

        The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

        Default value is UTC.

    • dissect object
      Hide dissect attributes Show dissect attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • append_separator string

        The character(s) that separate the appended fields.

        Default value is "".

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • pattern string Required

        The pattern to apply to the field.

    • dot_expander object
      Hide dot_expander attributes Show dot_expander attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • override boolean

        Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

        Default value is false.

      • path string

        The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

    • drop object
      Hide drop attributes Show drop attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

    • enrich object
      Hide enrich attributes Show enrich attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • max_matches number

        The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

        Default value is 1.

      • override boolean

        If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        Default value is true.

      • policy_name string Required

        The name of the enrich policy to use.

      • shape_relation string

        Values are intersects, disjoint, within, or contains.

      • target_field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • fail object
      Hide fail attributes Show fail attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • message string Required

        The error message thrown by the processor. Supports template snippets.

    • fingerprint object
      Hide fingerprint attributes Show fingerprint attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • fields string | array[string] Required
      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • salt string

        Salt value for the hash function.

      • method string

        Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

      • ignore_missing boolean

        If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

        Default value is false.

    • foreach object
      Hide foreach attributes Show foreach attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true, the processor silently exits without changing the document if the field is null or missing.

        Default value is false.

      • processor object Required
    • ip_location object
      Hide ip_location attributes Show ip_location attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • database_file string

        The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        Default value is GeoLite2-City.mmdb.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • first_only boolean

        If true, only the first found IP location data will be returned, even if the field contains an array.

        Default value is true.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • properties array[string]

        Controls what properties are added to the target_field based on the IP location lookup.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • download_database_on_pipeline_creation boolean

        If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

    • geo_grid object
      Hide geo_grid attributes Show geo_grid attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        The field to interpret as a geo-tile.= The field format is determined by the tile_type.

      • tile_type string Required

        Values are geotile, geohex, or geohash.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • parent_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • children_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • non_children_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • precision_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • target_format string

        Values are geojson or wkt.

    • geoip object
      Hide geoip attributes Show geoip attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • database_file string

        The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        Default value is GeoLite2-City.mmdb.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • first_only boolean

        If true, only the first found geoip data will be returned, even if the field contains an array.

        Default value is true.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • properties array[string]

        Controls what properties are added to the target_field based on the geoip lookup.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • download_database_on_pipeline_creation boolean

        If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

    • grok object
      Hide grok attributes Show grok attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • ecs_compatibility string

        Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

        Default value is disabled.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • pattern_definitions object

        A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

        Hide pattern_definitions attribute Show pattern_definitions attribute object

        • * string Additional properties
      • patterns array[string] Required

        An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

      • trace_match boolean

        When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

        Default value is false.

    • gsub object
      Hide gsub attributes Show gsub attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • pattern string Required

        The pattern to be replaced.

      • replacement string Required

        The string to replace the matching patterns with.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • html_strip object
      Hide html_strip attributes Show html_strip attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document,

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • inference object
      Hide inference attributes Show inference attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • model_id string Required
      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • field_map object

        Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

        Hide field_map attribute Show field_map attribute object

        • * object Additional properties
      • inference_config object
        Hide inference_config attributes Show inference_config attributes object

        • regression object
        • classification object

      • input_output object | array[object]

        Input fields for inference and output (destination) fields for the inference results. This option is incompatible with the target_field and field_map options.

        One of:

        InputConfig object array-2 array[object]
      • ignore_missing boolean

        If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

    • join object
      Hide join attributes Show join attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • separator string Required

        The separator character.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • json object
      Hide json attributes Show json attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • add_to_root boolean

        Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

        Default value is false.

      • add_to_root_conflict_strategy string

        Values are replace or merge.

      • allow_duplicate_keys boolean

        When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

        Default value is false.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • kv object
      Hide kv attributes Show kv attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • exclude_keys array[string]

        List of keys to exclude from document.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • field_split string Required

        Regex pattern to use for splitting key-value pairs.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • include_keys array[string]

        List of keys to filter and insert into document. Defaults to including all keys.

      • prefix string

        Prefix to be added to extracted keys.

        Default value is null.

      • strip_brackets boolean

        If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • trim_key string

        String of characters to trim from extracted keys.

      • trim_value string

        String of characters to trim from extracted values.

      • value_split string Required

        Regex pattern to use for splitting the key from the value within a key-value pair.

    • lowercase object
      Hide lowercase attributes Show lowercase attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • network_direction object
      Hide network_direction attributes Show network_direction attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • source_ip string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • destination_ip string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • internal_networks array[string]

        List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

      • internal_networks_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and any required fields are missing, the processor quietly exits without modifying the document.

        Default value is true.

    • pipeline object
      Hide pipeline attributes Show pipeline attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • name string Required
      • ignore_missing_pipeline boolean

        Whether to ignore missing pipelines instead of failing.

        Default value is false.

    • redact object
      Hide redact attributes Show redact attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • patterns array[string] Required

        A list of grok expressions to match and redact named captures with

      • pattern_definitions object
        Hide pattern_definitions attribute Show pattern_definitions attribute object

        • * string Additional properties
      • prefix string

        Start a redacted section with this token

        Default value is <.

      • suffix string

        End a redacted section with this token

        Default value is >.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • skip_if_unlicensed boolean

        If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

        Default value is false.

      • trace_redact boolean Generally available; Added in 8.16.0

        If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

        Default value is false.

    • registered_domain object
      Hide registered_domain attributes Show registered_domain attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and any required fields are missing, the processor quietly exits without modifying the document.

        Default value is true.

    • remove object
      Hide remove attributes Show remove attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string | array[string] Required
      • keep string | array[string]
      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

    • rename object
      Hide rename attributes Show rename attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • target_field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • reroute object
      Hide reroute attributes Show reroute attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • destination string

        A static value for the target. Can’t be set when the dataset or namespace option is set.

      • dataset string | array[string]

        Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.

        Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

        default {{data_stream.dataset}}

        One of:

        string-1 string array-2 array[string]

      • namespace string | array[string]

        Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.

        Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

        default {{data_stream.namespace}}

        One of:

        string-1 string array-2 array[string]
    • script object
      Hide script attributes Show script attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • id string

      • lang string

        Any of:

        string-1 string string-2 string

        Values are painless, expression, mustache, or java.
      • params object

        Object containing parameters for the script.

        Hide params attribute Show params attribute object

        • * object Additional properties

      • source string | object

        One of:

        string-1 string SearchRequestBody object
    • set object
      Hide set attributes Show set attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • copy_from string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_empty_value boolean

        If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

        Default value is false.

      • media_type string

        The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

      • override boolean

        If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        Default value is true.

      • value object

        The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

    • set_security_user object
      Hide set_security_user attributes Show set_security_user attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • properties array[string]

        Controls what user related properties are added to the field.

    • sort object
      Hide sort attributes Show sort attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • order string

        Values are asc or desc.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • split object
      Hide split attributes Show split attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • preserve_trailing boolean

        Preserves empty trailing fields, if any.

        Default value is false.

      • separator string Required

        A regex which matches the separator, for example, , or \s+.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • terminate object
      Hide terminate attributes Show terminate attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

    • trim object
      Hide trim attributes Show trim attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • uppercase object
      Hide uppercase attributes Show uppercase attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • urldecode object
      Hide urldecode attributes Show urldecode attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • uri_parts object
      Hide uri_parts attributes Show uri_parts attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • keep_original boolean

        If true, the processor copies the unparsed URI to <target_field>.original.

        Default value is true.

      • remove_if_successful boolean

        If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • user_agent object
      Hide user_agent attributes Show user_agent attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • regex_file string

        The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • properties array[string]

        Controls what properties are added to target_field.

        Values are name, os, device, original, or version. Default value is ["name", "major", "minor", "patch", "build", "os", "os_name", "os_major", "os_minor", "device"].

      • extract_device_type boolean Beta; Added in 8.9.0

        Extracts device type from the user agent string on a best-effort basis.

        Default value is false.

  • processors array[object]

    Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.

    Hide processors attributes Show processors attributes object

    • append object
      Hide append attributes Show append attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • value object | array[object] Required

        The value to be appended. Supports template snippets.

        One of:

        object-1 object array-2 array[object]
      • allow_duplicates boolean

        If false, the processor does not append values already present in the field.

        Default value is true.

    • attachment object
      Hide attachment attributes Show attachment attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • indexed_chars number

        The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

        Default value is 100000.

      • indexed_chars_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • properties array[string]

        Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • remove_binary boolean

        If true, the binary field will be removed from the document

        Default value is false.

      • resource_name string

        Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

    • bytes object
      Hide bytes attributes Show bytes attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • circle object
      Hide circle attributes Show circle attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • error_distance number Required

        The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • shape_type string Required

        Values are geo_shape or shape.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • community_id object
      Hide community_id attributes Show community_id attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • source_ip string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • source_port string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • destination_ip string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • destination_port string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • iana_number string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • icmp_type string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • icmp_code string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • transport string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • seed number

        Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

        Default value is 0.

      • ignore_missing boolean

        If true and any required fields are missing, the processor quietly exits without modifying the document.

        Default value is true.

    • convert object
      Hide convert attributes Show convert attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • type string Required

        Values are integer, long, double, float, boolean, ip, string, or auto.

    • csv object
      Hide csv attributes Show csv attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • empty_value object

        Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • quote string

        Quote used in CSV, has to be single character string.

        Default value is ".

      • separator string

        Separator used in CSV, has to be single character string.

        Default value is ,.

      • target_fields string | array[string] Required
      • trim boolean

        Trim whitespaces in unquoted fields.

    • date object
      Hide date attributes Show date attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • formats array[string] Required

        An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

      • locale string

        The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

        Default value is ENGLISH.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • timezone string

        The timezone to use when parsing the date. Supports template snippets.

        Default value is UTC.

      • output_format string

        The format to use when writing the date to target_field. Must be a valid java time pattern.

        Default value is yyyy-MM-dd'T'HH:mm:ss.SSSXXX.

    • date_index_name object
      Hide date_index_name attributes Show date_index_name attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • date_formats array[string]

        An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

      • date_rounding string Required

        How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • index_name_format string

        The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

        Default value is yyyy-MM-dd.

      • index_name_prefix string

        A prefix of the index name to be prepended before the printed date. Supports template snippets.

      • locale string

        The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

        Default value is ENGLISH.

      • timezone string

        The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

        Default value is UTC.

    • dissect object
      Hide dissect attributes Show dissect attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • append_separator string

        The character(s) that separate the appended fields.

        Default value is "".

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • pattern string Required

        The pattern to apply to the field.

    • dot_expander object
      Hide dot_expander attributes Show dot_expander attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • override boolean

        Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

        Default value is false.

      • path string

        The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

    • drop object
      Hide drop attributes Show drop attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

    • enrich object
      Hide enrich attributes Show enrich attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • max_matches number

        The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

        Default value is 1.

      • override boolean

        If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        Default value is true.

      • policy_name string Required

        The name of the enrich policy to use.

      • shape_relation string

        Values are intersects, disjoint, within, or contains.

      • target_field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • fail object
      Hide fail attributes Show fail attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • message string Required

        The error message thrown by the processor. Supports template snippets.

    • fingerprint object
      Hide fingerprint attributes Show fingerprint attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • fields string | array[string] Required
      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • salt string

        Salt value for the hash function.

      • method string

        Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

      • ignore_missing boolean

        If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

        Default value is false.

    • foreach object
      Hide foreach attributes Show foreach attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true, the processor silently exits without changing the document if the field is null or missing.

        Default value is false.

      • processor object Required
    • ip_location object
      Hide ip_location attributes Show ip_location attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • database_file string

        The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        Default value is GeoLite2-City.mmdb.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • first_only boolean

        If true, only the first found IP location data will be returned, even if the field contains an array.

        Default value is true.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • properties array[string]

        Controls what properties are added to the target_field based on the IP location lookup.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • download_database_on_pipeline_creation boolean

        If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

    • geo_grid object
      Hide geo_grid attributes Show geo_grid attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        The field to interpret as a geo-tile.= The field format is determined by the tile_type.

      • tile_type string Required

        Values are geotile, geohex, or geohash.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • parent_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • children_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • non_children_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • precision_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • target_format string

        Values are geojson or wkt.

    • geoip object
      Hide geoip attributes Show geoip attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • database_file string

        The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        Default value is GeoLite2-City.mmdb.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • first_only boolean

        If true, only the first found geoip data will be returned, even if the field contains an array.

        Default value is true.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • properties array[string]

        Controls what properties are added to the target_field based on the geoip lookup.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • download_database_on_pipeline_creation boolean

        If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

    • grok object
      Hide grok attributes Show grok attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • ecs_compatibility string

        Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

        Default value is disabled.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • pattern_definitions object

        A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

        Hide pattern_definitions attribute Show pattern_definitions attribute object

        • * string Additional properties
      • patterns array[string] Required

        An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

      • trace_match boolean

        When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

        Default value is false.

    • gsub object
      Hide gsub attributes Show gsub attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • pattern string Required

        The pattern to be replaced.

      • replacement string Required

        The string to replace the matching patterns with.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • html_strip object
      Hide html_strip attributes Show html_strip attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document,

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • inference object
      Hide inference attributes Show inference attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • model_id string Required
      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • field_map object

        Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

        Hide field_map attribute Show field_map attribute object

        • * object Additional properties
      • inference_config object
        Hide inference_config attributes Show inference_config attributes object

        • regression object
        • classification object

      • input_output object | array[object]

        Input fields for inference and output (destination) fields for the inference results. This option is incompatible with the target_field and field_map options.

        One of:

        InputConfig object array-2 array[object]
      • ignore_missing boolean

        If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

    • join object
      Hide join attributes Show join attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • separator string Required

        The separator character.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • json object
      Hide json attributes Show json attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • add_to_root boolean

        Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

        Default value is false.

      • add_to_root_conflict_strategy string

        Values are replace or merge.

      • allow_duplicate_keys boolean

        When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

        Default value is false.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • kv object
      Hide kv attributes Show kv attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • exclude_keys array[string]

        List of keys to exclude from document.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • field_split string Required

        Regex pattern to use for splitting key-value pairs.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • include_keys array[string]

        List of keys to filter and insert into document. Defaults to including all keys.

      • prefix string

        Prefix to be added to extracted keys.

        Default value is null.

      • strip_brackets boolean

        If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • trim_key string

        String of characters to trim from extracted keys.

      • trim_value string

        String of characters to trim from extracted values.

      • value_split string Required

        Regex pattern to use for splitting the key from the value within a key-value pair.

    • lowercase object
      Hide lowercase attributes Show lowercase attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • network_direction object
      Hide network_direction attributes Show network_direction attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • source_ip string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • destination_ip string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • internal_networks array[string]

        List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

      • internal_networks_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and any required fields are missing, the processor quietly exits without modifying the document.

        Default value is true.

    • pipeline object
      Hide pipeline attributes Show pipeline attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • name string Required
      • ignore_missing_pipeline boolean

        Whether to ignore missing pipelines instead of failing.

        Default value is false.

    • redact object
      Hide redact attributes Show redact attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • patterns array[string] Required

        A list of grok expressions to match and redact named captures with

      • pattern_definitions object
        Hide pattern_definitions attribute Show pattern_definitions attribute object

        • * string Additional properties
      • prefix string

        Start a redacted section with this token

        Default value is <.

      • suffix string

        End a redacted section with this token

        Default value is >.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • skip_if_unlicensed boolean

        If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

        Default value is false.

      • trace_redact boolean Generally available; Added in 8.16.0

        If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

        Default value is false.

    • registered_domain object
      Hide registered_domain attributes Show registered_domain attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and any required fields are missing, the processor quietly exits without modifying the document.

        Default value is true.

    • remove object
      Hide remove attributes Show remove attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string | array[string] Required
      • keep string | array[string]
      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

    • rename object
      Hide rename attributes Show rename attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • target_field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • reroute object
      Hide reroute attributes Show reroute attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • destination string

        A static value for the target. Can’t be set when the dataset or namespace option is set.

      • dataset string | array[string]

        Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.

        Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

        default {{data_stream.dataset}}

        One of:

        string-1 string array-2 array[string]

      • namespace string | array[string]

        Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.

        Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

        default {{data_stream.namespace}}

        One of:

        string-1 string array-2 array[string]
    • script object
      Hide script attributes Show script attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • id string

      • lang string

        Any of:

        string-1 string string-2 string

        Values are painless, expression, mustache, or java.
      • params object

        Object containing parameters for the script.

        Hide params attribute Show params attribute object

        • * object Additional properties

      • source string | object

        One of:

        string-1 string SearchRequestBody object
    • set object
      Hide set attributes Show set attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • copy_from string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_empty_value boolean

        If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

        Default value is false.

      • media_type string

        The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

      • override boolean

        If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        Default value is true.

      • value object

        The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

    • set_security_user object
      Hide set_security_user attributes Show set_security_user attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • properties array[string]

        Controls what user related properties are added to the field.

    • sort object
      Hide sort attributes Show sort attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • order string

        Values are asc or desc.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • split object
      Hide split attributes Show split attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • preserve_trailing boolean

        Preserves empty trailing fields, if any.

        Default value is false.

      • separator string Required

        A regex which matches the separator, for example, , or \s+.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • terminate object
      Hide terminate attributes Show terminate attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

    • trim object
      Hide trim attributes Show trim attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • uppercase object
      Hide uppercase attributes Show uppercase attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • urldecode object
      Hide urldecode attributes Show urldecode attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • uri_parts object
      Hide uri_parts attributes Show uri_parts attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • keep_original boolean

        If true, the processor copies the unparsed URI to <target_field>.original.

        Default value is true.

      • remove_if_successful boolean

        If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

        Default value is false.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • user_agent object
      Hide user_agent attributes Show user_agent attributes object

      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if object
        Hide if attributes Show if attributes object

        • source
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        • lang
        • options object
      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.
      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

        Default value is false.

      • regex_file string

        The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • properties array[string]

        Controls what properties are added to target_field.

        Values are name, os, device, original, or version. Default value is ["name", "major", "minor", "patch", "build", "os", "os_name", "os_major", "os_minor", "device"].

      • extract_device_type boolean Beta; Added in 8.9.0

        Extracts device type from the user agent string on a best-effort basis.

        Default value is false.

  • version number
  • deprecated boolean

    Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.

    Default value is false.

  • _meta object
    Hide _meta attribute Show _meta attribute object

    • * object Additional properties