feat(aci): create WorkflowFireHistory objects when workflows fire for a group #88495

cathteng · 2025-04-01T21:07:22Z

Refactor the process of firing actions to:

Create WorkflowFireHistory objects for each unique workflow being fired
Grab the correct notification_uuid from the WorkflowFireHistory objects to fire each Action with
Update the WorkflowEventData correctly with workflow_env and workflow_id to fire each Action with

Refactors into a single function that can be used in process_workflows AND delayed workflow processing 😇

codecov · 2025-04-01T21:45:22Z

Codecov Report

All modified and coverable lines are covered by tests ✅

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #88495      +/-   ##
==========================================
+ Coverage   84.49%   87.76%   +3.26%     
==========================================
  Files       10032    10032              
  Lines      567910   567979      +69     
  Branches    22270    22270              
==========================================
+ Hits       479856   498472   +18616     
+ Misses      87634    69087   -18547     
  Partials      420      420

cathteng · 2025-04-03T01:23:42Z

src/sentry/workflow_engine/processors/workflow.py

+    return workflow_id_to_fire_history
+
+
+def fire_actions(actions: list[Action], detector: Detector, event_data: WorkflowEventData) -> None:


not sure the correct home for this 🤔 could live in sentry/workflow_engine/processors/action.py too?

yeah, i think this belongs in that file rather than here. the focus is on triggering the actions.

saponifi3d

i feel like there is a bit of cleanup to happen here, we should try not to add everything to the workflow file and split it up based on what we're processing.

i'm also curious how much of this code we might need.. we have a trigger_actions method and now a fire_actions method, we also already have a set list of actions to trigger and a list of the triggered workflows, but we're re-querying for that data here.

saponifi3d · 2025-04-03T17:43:13Z

src/sentry/workflow_engine/models/action.py

@@ -66,7 +66,7 @@ def get_handler(self) -> ActionHandler:
        action_type = Action.Type(self.type)
        return action_handler_registry.get(action_type)

-    def trigger(self, event_data: WorkflowEventData, detector: Detector) -> None:
+    def trigger(self, event_data: WorkflowEventData, detector: Detector, uuid: str) -> None:


doesn't seem like we are using the uuid, can we remove it?

saponifi3d · 2025-04-03T17:44:52Z

src/sentry/workflow_engine/processors/workflow.py

+    return workflow_id_to_fire_history
+
+
+def fire_actions(actions: list[Action], detector: Detector, event_data: WorkflowEventData) -> None:


yeah, i think this belongs in that file rather than here. the focus is on triggering the actions.

saponifi3d · 2025-04-03T17:46:29Z

src/sentry/workflow_engine/processors/workflow.py

+def get_actions_to_workflows(actions_to_fire: list[Action]) -> dict[int, Workflow]:
+    action_ids = {action.id for action in actions_to_fire}
+    workflow_actions = (
+        WorkflowDataConditionGroup.objects.select_related("workflow", "condition_group")
+        .filter(condition_group__dataconditiongroupaction__action_id__in=action_ids)
+        .values_list("condition_group__dataconditiongroupaction__action_id", "workflow_id")
+    )
+
+    action_to_workflows: dict[int, int] = {
+        action_id: workflow_id for action_id, workflow_id in workflow_actions
+    }
+    workflow_ids = set(action_to_workflows.values())
+
+    workflows = Workflow.objects.filter(id__in=workflow_ids)
+    workflow_ids_to_workflows = {workflow.id: workflow for workflow in workflows}
+
+    # TODO: account for actions being attached to multiple workflows?
+    # should you even send multiple notifications then? :think:
+    actions_to_workflows: dict[int, Workflow] = {
+        action_id: workflow_ids_to_workflows[workflow_id]
+        for action_id, workflow_id in action_to_workflows.items()
+    }
+    return actions_to_workflows
+
+
+def create_workflow_fire_histories(
+    actions_to_workflows: dict[int, Workflow], event_data: WorkflowEventData
+) -> dict[int, WorkflowFireHistory]:
+    workflows = set(actions_to_workflows.values())
+
+    workflow_fire_histories = [
+        WorkflowFireHistory(
+            workflow=workflow,
+            group=event_data.event.group,
+            event_id=event_data.event.event_id,
+        )
+        for workflow in workflows
+    ]
+    fire_histories = WorkflowFireHistory.objects.bulk_create(workflow_fire_histories)
+
+    workflow_id_to_fire_history: dict[int, WorkflowFireHistory] = {
+        workflow_fire_history.workflow_id: workflow_fire_history
+        for workflow_fire_history in fire_histories
+    }
+    return workflow_id_to_fire_history


i feel like we should make a processors/workflow_fire_history.py to encapsulate hold this logic, that way we can keep this file focused on the pure workflow processing.

saponifi3d · 2025-04-03T17:47:11Z

src/sentry/workflow_engine/processors/workflow.py

@@ -129,7 +196,9 @@ def evaluate_workflows_action_filters(
    return filter_recently_fired_workflow_actions(filtered_action_groups, event_data.event.group)


-def log_fired_workflows(log_name: str, actions: list[Action], job: WorkflowEventData) -> None:
+def log_fired_workflows(


should we move this into process/workflow_fire_history file i proposed? 🤔

saponifi3d · 2025-04-03T17:48:48Z

tests/sentry/workflow_engine/handlers/action/test_action_handlers.py

@@ -20,7 +21,7 @@ def setUp(self):
    def test_execute_without_group_type(self):
        """Test that execute does nothing when detector has no group_type"""
        self.detector.type = ""
-        self.action.trigger(self.event_data, self.detector)
+        self.action.trigger(self.event_data, self.detector, str(uuid4()))


this uuid wasn't being used, can we remove this stuff?

curious about the plan around this uuid as well, i wonder if this is something we should encapsulate inside of self.action.trigger rather than exposing it in the params

saponifi3d · 2025-04-03T17:52:26Z

src/sentry/workflow_engine/processors/workflow.py

+    # TODO: account for actions being attached to multiple workflows?
+    # should you even send multiple notifications then? :think:


no, we should de-dupe the actions and only trigger them once. there was some stuff around that in trigger_actions that i turned the action list for all the workflows into a set.

saponifi3d · 2025-04-03T17:52:58Z

src/sentry/workflow_engine/processors/workflow.py

@@ -34,6 +36,71 @@ class WorkflowDataConditionGroupType(StrEnum):
    WORKFLOW_TRIGGER = "workflow_trigger"


+def get_actions_to_workflows(actions_to_fire: list[Action]) -> dict[int, Workflow]:


curious about this method at a high level, we already have a list of the actions that have been filtered to what will trigger. do we need to do this lookup?

wedamija · 2025-04-03T17:17:51Z

src/sentry/workflow_engine/processors/workflow.py

+def get_actions_to_workflows(actions_to_fire: list[Action]) -> dict[int, Workflow]:
+    action_ids = {action.id for action in actions_to_fire}
+    workflow_actions = (
+        WorkflowDataConditionGroup.objects.select_related("workflow", "condition_group")


I think that select_related is unnecessary, since you're already joining via values_list.

wedamija · 2025-04-03T17:49:33Z

src/sentry/workflow_engine/processors/workflow.py

+                event_data, workflow_env=workflow.environment, workflow_id=workflow.id
+            )
+
+            notification_uuid = workflow_id_to_fire_history[workflow.id].notification_uuid


Just checking - is the notification uuid meant to be shared between actions?

i didn't really understand why we did this because CTR should be tracked per action (unique uuid per action), but today we have it implemented as 1 UUID for all actions that fire for an issue alert

sentry/src/sentry/rules/processing/processor.py

Lines 414 to 418 in 342bab4

notification_uuid = str(uuid.uuid4())

rule_fire_history = history.record(rule, self.group, self.event.event_id, notification_uuid)

grouped_futures = activate_downstream_actions(

rule, self.event, notification_uuid, rule_fire_history

)

Hmm... Do we use something else to differentiate the CTR by combining the uuid with the action name? Or are we just tracking incorrectly when multiple actions are set up?

wedamija · 2025-04-03T17:56:43Z

src/sentry/workflow_engine/processors/workflow.py

+    # TODO: account for actions being attached to multiple workflows?
+    # should you even send multiple notifications then? :think:


Can actions actually be attached to multiple workflows? I mean I know the data model supports it, but are there plans to support that?

The thought is this is to support the eventual Notification Center center project, which is pretty far down the line. we shouldn't be worrying tooo much about it today.

cathteng · 2025-04-07T16:19:04Z

Replacing with #88775

github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Apr 1, 2025

vercel bot deployed to Preview April 1, 2025 21:10 View deployment

vercel bot deployed to Preview April 1, 2025 22:56 View deployment

cathteng force-pushed the cathy/aci/use-workflow-fire-history branch from 22e6926 to 0a3780b Compare April 2, 2025 16:04

vercel bot deployed to Preview April 2, 2025 16:08 View deployment

Base automatically changed from cathy/aci/workflow-fire-history-model to master April 2, 2025 17:50

cathteng added 9 commits April 2, 2025 18:17

add WorkflowFireHistory model

af9b9b2

update migration

879a370

update migration again

35759c4

update migration again again

44513f0

update migratino

aaa1c58

create WorkflowFireHistory objects when workflows fire for a group

131730a

use list

55b5a0e

refactor firing actions to also update WorkflowFireHistory

fc8f2db

fix migration

6f3b932

cathteng force-pushed the cathy/aci/use-workflow-fire-history branch from 0a3780b to 6f3b932 Compare April 3, 2025 01:18

cathteng added 2 commits April 2, 2025 18:21

fix migration again

801be63

fix migration again

e288085

cathteng commented Apr 3, 2025

View reviewed changes

cathteng marked this pull request as ready for review April 3, 2025 01:24

cathteng requested a review from a team as a code owner April 3, 2025 01:24

vercel bot deployed to Preview April 3, 2025 01:26 View deployment

cathteng requested a review from iamrajjoshi April 3, 2025 01:27

fix test

342bab4

vercel bot deployed to Preview April 3, 2025 02:03 View deployment

cathteng requested a review from saponifi3d April 3, 2025 17:41

saponifi3d reviewed Apr 3, 2025

View reviewed changes

wedamija approved these changes Apr 3, 2025

View reviewed changes

cathteng requested a review from GabeVillalobos April 3, 2025 18:37

cathteng marked this pull request as draft April 3, 2025 20:17

cathteng closed this Apr 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(aci): create WorkflowFireHistory objects when workflows fire for a group #88495

feat(aci): create WorkflowFireHistory objects when workflows fire for a group #88495

cathteng commented Apr 1, 2025 •

edited

Loading

codecov bot commented Apr 1, 2025 •

edited

Loading

cathteng Apr 3, 2025

saponifi3d Apr 3, 2025

saponifi3d left a comment

saponifi3d Apr 3, 2025

saponifi3d Apr 3, 2025

saponifi3d Apr 3, 2025

saponifi3d Apr 3, 2025

saponifi3d Apr 3, 2025

saponifi3d Apr 3, 2025

saponifi3d Apr 3, 2025

wedamija Apr 3, 2025

wedamija Apr 3, 2025

cathteng Apr 3, 2025

wedamija Apr 3, 2025

wedamija Apr 3, 2025

saponifi3d Apr 3, 2025

cathteng commented Apr 7, 2025

		return workflow_id_to_fire_history


		def fire_actions(actions: list[Action], detector: Detector, event_data: WorkflowEventData) -> None:

		# TODO: account for actions being attached to multiple workflows?
		# should you even send multiple notifications then? :think:

		@@ -34,6 +36,71 @@ class WorkflowDataConditionGroupType(StrEnum):
		WORKFLOW_TRIGGER = "workflow_trigger"


		def get_actions_to_workflows(actions_to_fire: list[Action]) -> dict[int, Workflow]:

	notification_uuid = str(uuid.uuid4())
	rule_fire_history = history.record(rule, self.group, self.event.event_id, notification_uuid)
	grouped_futures = activate_downstream_actions(
	rule, self.event, notification_uuid, rule_fire_history
	)

feat(aci): create WorkflowFireHistory objects when workflows fire for a group #88495

feat(aci): create WorkflowFireHistory objects when workflows fire for a group #88495

Conversation

cathteng commented Apr 1, 2025 • edited Loading

codecov bot commented Apr 1, 2025 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saponifi3d left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cathteng commented Apr 7, 2025

cathteng commented Apr 1, 2025 •

edited

Loading

codecov bot commented Apr 1, 2025 •

edited

Loading