Add fields connected to rules and alerts #355

tomsonpl · 2023-03-20T12:06:43Z

Closes: https://github.com/elastic/security-team/issues/5929

Not sure if we need rule_name, but from what I remember, it was agreed that we wanted it.

elasticmachine · 2023-03-20T12:12:20Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-03-28T07:58:29.773+0000
Duration: 8 min 42 sec

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

tomsonpl · 2023-03-20T19:56:16Z

@pzl any idea if rule_name should be a keyword or just text ?

pzl · 2023-03-20T20:06:59Z

any idea if rule_name should be a keyword or just text ?

are we ever going to filter/search via name? I would imagine in filtering scenarios we probably know the rule_id and will be exact matching on that, right?

If for some reason we will filter on the name, do we know the full string to match (e.g. find rule by name "Unusual Network Activity")? then keyword. If we expect to filter on partial word matches (e.g. match the previous rule name based on the search string "network") then text could be ok.

If we don't need to filter by name, then I would set it to keyword, and index: false. Or just don't map it at all

# Conflicts: # custom_schemas/custom_base.yml # package/endpoint/data_stream/actions/fields/fields.yml

tomsonpl · 2023-03-22T15:39:03Z

@pzl @dasansol92 could you take a look please? Thank you!

pzl

looks okay, I think things are added in the right place. Added @ashokaditya to review the intent of whats being added

ashokaditya

I added some suggestions to update the mappings. Let me know if you need help or have questions.

ashokaditya · 2023-03-23T15:35:39Z

custom_schemas/custom_base.yml

+    - name: data.rule_id
+      type: alias
+      path: EndpointActions.data.rule_id
+      level: custom
+      short: EndpointActions.data.rule_id
+      description: >
+        ID of the rule that triggered the action
+
+    - name: data.rule_name
+      type: alias
+      path: EndpointActions.data.rule_name
+      level: custom
+      short: EndpointActions.data.rule_name
+      description: >
+        Name of the rule that triggered the action


The rules fields should be updated to use ECS names. That would mean they don't stay under EndpointActions. They would be at the same level as agent and user which are ECS fields.

So the field definitions should not appear here.

EndpointActions is a grouping we created for storing fields that are not available in ECS. Consequently, you'd also need to update the custom_subsets/elastic_endpoint/actions/actions.yaml file to update the action doc mappings. That would end up looking like

... ... user: fields: id: {} rule: fields: id: {} name: {}

By the way, there's also a rule.uuid. Not sure if we want to use that instead of id

https://www.elastic.co/guide/en/ecs/current/ecs-rule.html

ashokaditya · 2023-03-23T15:48:49Z

custom_schemas/custom_base.yml

+    - name: data.alert_ids
+      type: alias
+      path: EndpointActions.data.alert_ids
+      level: custom
+      short: EndpointActions.data.alert_ids
+      description: >
+        List of alert ids that triggered the action


This is good as alert_ids is a custom field.

ashokaditya · 2023-03-23T15:58:23Z

custom_schemas/custom_endpoint_actions.yml

+    - name: data.rule_id
+      type: keyword
+      level: custom
+      short: rule id
+      description: >
+        ID of the rule that triggered the action
+
+    - name: data.rule_name
+      type: keyword
+      index: false
+      level: custom
+      short: rule name
+      description: >
+        Name of the rule that triggered the action


rule_id and rule_name should not be here as they are not custom fields as I mentioned earlier.

ashokaditya · 2023-03-23T16:04:31Z

custom_schemas/custom_endpoint_actions.yml

+    - name: data.alert_ids
+      type: keyword
+      level: custom
+      short: alert ids
+      description: >
+        List of alert ids that triggered the action


You would also need to edit the custom_subsets/elastic_endpoint/actions/actions.yaml file instead. If you want the fields to also exist on the response doc then you need to edit the custom_subsets/elastic_endpoint/action_responses/action_responses.yaml.

ashokaditya · 2023-03-24T07:52:28Z

custom_schemas/custom_base.yml

@@ -52,6 +52,14 @@
      description: >
        The action request information

+    - name: data.alert_ids


This should be named with a singular form, i.e. data.alert_id. https://www.elastic.co/guide/en/ecs/8.6/ecs-guidelines.html

ashokaditya

Took another pass at reviewing. I have a question and suggestions.

ashokaditya · 2023-03-27T15:14:21Z

schemas/v1/action_responses/action_responses.yaml

  path: EndpointActions.data.alert_ids
  short: EndpointActions.data.alert_ids


These lines also need updates. 😅

ashokaditya · 2023-03-27T15:19:37Z

custom_subsets/elastic_endpoint/actions/actions.yaml

+  rule:
+    fields:
+      id: {}
+      name: {}


Q: Do we need these fields also in the response index? If yes then we need to add this to the action_responses/action_responses.yml file too.

ashokaditya · 2023-03-28T08:01:57Z

custom_schemas/custom_endpoint_actions.yml

+    - name: data.alert_id
+      type: keyword
+      level: custom
+      short: alert id
+      description: >
+        List of alert ids that triggered the action


Thanks for the changes. The fact that this is a keyword we can store an array of strings here s, so you can have multiple alert ids in the doc. Just that we should conform to ECS naming best practices.

tomsonpl · 2023-03-28T10:25:58Z

@ashokaditya Thank you for your feedback and bearing with me fixing it so many times :P

elasticmachine · 2023-04-25T21:11:02Z

Package endpoint - 8.8.0 containing this change is available at https://epr.elastic.co/search?package=endpoint

Add fields connected to rules and alerts

1467f5c

tomsonpl requested a review from pzl March 20, 2023 12:06

tomsonpl requested a review from a team as a code owner March 20, 2023 12:06

tomsonpl requested a review from dasansol92 March 20, 2023 12:06

tomsonpl and others added 2 commits March 20, 2023 13:42

Add changes to schema

c0c1222

update schemas

4c72d1e

tomsonpl added 5 commits March 22, 2023 15:01

add index:false to rule_name

27fdfb6

Merge remote-tracking branch 'origin/alert-fields' into alert-fields

fbbfd93

# Conflicts: # custom_schemas/custom_base.yml # package/endpoint/data_stream/actions/fields/fields.yml

add index:false to rule_name

6fa6940

add index:false to rule_name

156e22a

add index:false to rule_name

fefc7c2

pzl requested a review from ashokaditya March 22, 2023 15:46

pzl approved these changes Mar 22, 2023

View reviewed changes

ashokaditya requested changes Mar 23, 2023

View reviewed changes

ashokaditya reviewed Mar 24, 2023

View reviewed changes

apply comments from Ash

92f73cc

tomsonpl requested a review from ashokaditya March 27, 2023 14:30

ashokaditya reviewed Mar 27, 2023

View reviewed changes

tomsonpl added 2 commits March 28, 2023 09:45

rename alert_ids to alert_id

f1a39de

remove alert it from responses

707eac3

tomsonpl requested a review from ashokaditya March 28, 2023 07:59

ashokaditya approved these changes Mar 28, 2023

View reviewed changes

tomsonpl merged commit 8ad9c63 into main Mar 28, 2023

pzl deleted the alert-fields branch August 14, 2023 20:39

		path: EndpointActions.data.alert_ids
		short: EndpointActions.data.alert_ids

Add fields connected to rules and alerts #355

Add fields connected to rules and alerts #355

Uh oh!

Conversation

tomsonpl commented Mar 20, 2023

Uh oh!

elasticmachine commented Mar 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

🤖 GitHub comments

Uh oh!

tomsonpl commented Mar 20, 2023

Uh oh!

pzl commented Mar 20, 2023

Uh oh!

tomsonpl commented Mar 22, 2023

Uh oh!

pzl left a comment

Choose a reason for hiding this comment

Uh oh!

ashokaditya left a comment

Choose a reason for hiding this comment

Uh oh!

ashokaditya Mar 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ashokaditya Mar 23, 2023

Choose a reason for hiding this comment

Uh oh!

ashokaditya Mar 23, 2023

Choose a reason for hiding this comment

Uh oh!

ashokaditya Mar 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ashokaditya Mar 24, 2023

Choose a reason for hiding this comment

Uh oh!

ashokaditya left a comment

Choose a reason for hiding this comment

Uh oh!

ashokaditya Mar 27, 2023

Choose a reason for hiding this comment

Uh oh!

ashokaditya Mar 27, 2023

Choose a reason for hiding this comment

Uh oh!

ashokaditya Mar 28, 2023

Choose a reason for hiding this comment

Uh oh!

tomsonpl commented Mar 28, 2023

Uh oh!

elasticmachine commented Apr 25, 2023

Uh oh!

Uh oh!

elasticmachine commented Mar 20, 2023 •

edited

Loading

ashokaditya Mar 23, 2023 •

edited

Loading

ashokaditya Mar 23, 2023 •

edited

Loading