feat(aci): setup workflow preview abstraction #87553

cathteng · 2025-03-20T22:33:59Z

Get group_ids from the past 14 days that would have been triggered by a Workflow, which is represented by its trigger condition group + triggers and a list of filter condition groups + their filters. We also manually pass in the detectors that we want to think of as "attached" to the workflow.

We need to pass in all these arguments because we want the workflow preview to update when the user updates the form, so the Workflow may not be saved in the db yet.

preview_workflow logic

Find Groups from the last 14 days with types corresponding to the detectors
Filter groups based on workflow triggers (see A)
Filter groups based on each workflow filter groups (see A; the word filter is overused, sorry 😞)
Return group ids that meet both the triggers and any of the filter groups

A. How to find groups that would have fired a data condition group:

Split fast and slow conditions
Check fast conditions for a list of groups (see B)
Check slow conditions with groups that did not meet the fast conditions according to the data condition group logic_type (see B + "continue checking")

B. How to find groups that met a list of conditions

Call get_preview_groups for each condition with the group_ids, each condition will have this implemented their own way. This will return the group_ids that meet the condition
For logic_type=DataConditionGroup.Type.ALL: groups that met all conditions, continue checking groups that met all conditions
For logic_type=DataConditionGroup.Type.ANY / logic_type=DataConditionGroup.Type.ANY_SHORT_CIRCUIT: groups that met any condition, continue checking groups that did not
For logic_type=DataConditionGroup.Type.NONE: groups that met no conditions, continue checking groups that met no conditions
If evaluating a list of fast conditions, the list of "continue checking" groups = groups we will evaluate slow conditions on

cathteng · 2025-03-20T22:34:43Z

src/sentry/workflow_engine/processors/workflow_preview.py

+    # check fast conditions first
+    fast_conditions, slow_conditions = split_conditions_by_speed(conditions)
+
+    # TODO: early return if they are invalid conditon pairs? see src/sentry/rules/history/preview.py VALID_CONDITION_PAIRS


sentry/src/sentry/rules/history/preview.py

Lines 53 to 59 in 6a4af04

# Most of the ISSUE_STATE_CONDITIONS are mutually exclusive, except for the following pairs.

VALID_CONDITION_PAIRS = {

"sentry.rules.conditions.first_seen_event.FirstSeenEventCondition": "sentry.rules.conditions.high_priority_issue.NewHighPriorityIssueCondition",

"sentry.rules.conditions.high_priority_issue.NewHighPriorityIssueCondition": "sentry.rules.conditions.first_seen_event.FirstSeenEventCondition",

"sentry.rules.conditions.reappeared_event.ReappearedEventCondition": "sentry.rules.conditions.high_priority_issue.ExistingHighPriorityIssueCondition",

"sentry.rules.conditions.high_priority_issue.ExistingHighPriorityIssueCondition": "sentry.rules.conditions.reappeared_event.ReappearedEventCondition",

}

a lot of other early return conditions in the existing implementation

sentry/src/sentry/rules/history/preview.py

Lines 74 to 95 in 6a4af04

issue_state_conditions, frequency_conditions = categorize_conditions(conditions)

# must have at least one condition to filter activity

if not issue_state_conditions and not frequency_conditions:

return None

elif len(issue_state_conditions) > 1 and condition_match == "all":

# Of the supported conditions, any more than two would be mutually exclusive

if len(issue_state_conditions) > 2:

return {}

condition_ids = {condition["id"] for condition in issue_state_conditions}

# all the issue state conditions are mutually exclusive

if not any(condition in VALID_CONDITION_PAIRS for condition in condition_ids):

return {}

# if there are multiple issue state conditions, they must be one of the valid pairs

for condition in condition_ids:

if (

condition in VALID_CONDITION_PAIRS

and VALID_CONDITION_PAIRS[condition] not in condition_ids

):

return {}

These conditions are generally quite complicated - might be a good time to see if we can simplify it a bit or remove some restrictions

maybe we could not call the endpoint from the frontend at all if these exist 🤔

cathteng · 2025-03-20T22:37:13Z

src/sentry/workflow_engine/processors/workflow_preview.py

+    if not groups_to_check:
+        return groups_meeting_fast_conditions
+
+    _, groups_meeting_conditions = preview_conditions(logic_type, slow_conditions, groups_to_check)


the current implementation estimates slow conditions for the groups with the top 10 most events in the past two weeks to approximate, since we don't want to blow up Snuba with queries

sentry/src/sentry/rules/history/preview.py

Lines 248 to 278 in 6a4af04

"""

Filters the activity to contain only groups that have the most events (out of the given groups) in the past 2 weeks.

If no groups are provided because there are no issue state change conditions, returns the top groups overall.

Since frequency conditions require snuba query(s), we need to limit the number groups we process.

"""

if has_issue_state_condition:

datasets = {dataset_map.get(group) for group in condition_activity.keys()}

else:

# condition_activity will be empty because there are no issue state conditions.

# So, we look to find top groups over all datasets

datasets = set(DATASET_TO_COLUMN_NAME.keys())

group_ids = list(condition_activity.keys())

# queries each dataset for top x groups and then gets top x overall

query_params = []

for dataset in datasets:

kwargs = get_update_kwargs_for_groups(

dataset,

group_ids,

{

"dataset": dataset,

"start": start,

"end": end,

"filter_keys": {"project_id": [project.id]},

"aggregations": [("count", "group_id", "groupCount")],

"groupby": ["group_id"],

"orderby": "-groupCount",

"selected_columns": ["group_id", "groupCount"],

"limit": FREQUENCY_CONDITION_GROUP_LIMIT,

},

sentry/src/sentry/rules/history/preview.py

Line 39 in 6a4af04

FREQUENCY_CONDITION_GROUP_LIMIT = 10

Can't this all be done with one query though, similar to how we do batching? It has to go through all of the data here anyway. We could limit it to 10k results, the max snuba will return. Maybe a good limit for earlier group query too?

yeah i need to figure out how slow conditions will be batched properly. similarly for tagged event filter and event attribute filter that need to query snuba

cathteng · 2025-03-20T22:38:39Z

src/sentry/workflow_engine/models/data_condition.py

@@ -169,6 +169,9 @@ def evaluate_value(self, value: T) -> DataConditionResult:
        result = handler.evaluate_value(value, self.comparison)
        return self.get_condition_result() if result else None

+    def get_preview_groups(self, group_ids: set[int]) -> set[int]:


placeholder, should implement for every DataCondition type, via the condition_handler_registry?

codecov · 2025-03-20T23:05:09Z

Codecov Report

All modified and coverable lines are covered by tests ✅

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff            @@
##           master   #87553    +/-   ##
========================================
  Coverage   87.74%   87.75%            
========================================
  Files       10014    10016     +2     
  Lines      567496   567640   +144     
  Branches    22297    22297            
========================================
+ Hits       497977   498108   +131     
- Misses      69102    69115    +13     
  Partials      417      417

wedamija · 2025-03-28T19:35:27Z

src/sentry/workflow_engine/processors/workflow_preview.py

+        Group.objects.filter(
+            project__in=projects,
+            last_seen__gte=timezone.now() - timedelta(days=14),
+            type__in=group_types,
+        ).values_list("id", flat=True)


We didn't have this in the old code, but might be a good idea to set an upper bound for number of ids we return here. Something high is fine, 10k, 100k, not sure... If there were 100k groups I think it'd likely time out

wedamija · 2025-03-28T19:37:46Z

src/sentry/workflow_engine/processors/workflow_preview.py

+    for condition in conditions:
+        conditions_to_triggered_groups[condition.id] = condition.get_preview_groups(group_ids)


How will we get the preview conditions here? Is each condition going to need to do some querying?

yes they will do their own querying and return the group_ids that meet the condition. this could be implemented on each DataConditionHandler

This might end up being a bit more inefficient compared to the old implementation, since it uses activity and I think most conditions are able to infer what happened from that.

I guess in most cases, we won't have a lot of conditions, and we can make batch queries within the conditions as needed

the handlers can use their choice of whatever filtering they need to do with the group id to filter correctly -- some use Activity and others filter on Group

i was going to filter on Activity or Group just like the existing implementation, which also does a query per condition and pulls the Activity or Group result into memory via values_list

wedamija · 2025-03-28T19:42:57Z

src/sentry/workflow_engine/processors/workflow_preview.py

+    if not groups_to_check:
+        return groups_meeting_fast_conditions
+
+    _, groups_meeting_conditions = preview_conditions(logic_type, slow_conditions, groups_to_check)


Can't this all be done with one query though, similar to how we do batching? It has to go through all of the data here anyway. We could limit it to 10k results, the max snuba will return. Maybe a good limit for earlier group query too?

cathteng · 2025-04-07T20:08:07Z

this requires a whole tech spec of its own. punting until after V0

github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Mar 20, 2025

cathteng commented Mar 20, 2025

View reviewed changes

cathteng requested a review from a team March 20, 2025 22:37

cathteng commented Mar 20, 2025

View reviewed changes

cathteng changed the title ~~wip(aci): workflow preview~~ spike(aci): workflow preview Mar 20, 2025

cathteng force-pushed the cathy/aci/workflow-preview branch from 6a4af04 to 1776798 Compare March 26, 2025 19:41

cathteng changed the title ~~spike(aci): workflow preview~~ feat(aci): workflow preview abstraction Mar 26, 2025

cathteng marked this pull request as ready for review March 26, 2025 19:45

vercel bot deployed to Preview March 26, 2025 19:47 View deployment

cathteng changed the title ~~feat(aci): workflow preview abstraction~~ feat(aci): setup workflow preview abstraction Mar 26, 2025

vercel bot deployed to Preview March 26, 2025 20:24 View deployment

cathteng requested a review from snigdhas March 27, 2025 17:28

wedamija reviewed Mar 28, 2025

View reviewed changes

cathteng marked this pull request as draft March 31, 2025 16:06

cathteng added 3 commits April 2, 2025 09:06

workflow preview

45b42c2

add tests for the abstraction

0562781

fix test

38850f2

cathteng force-pushed the cathy/aci/workflow-preview branch from df4ae4b to 38850f2 Compare April 2, 2025 16:06

vercel bot deployed to Preview April 2, 2025 16:12 View deployment

cathteng closed this Apr 7, 2025

github-actions bot locked and limited conversation to collaborators Apr 23, 2025

	# Most of the ISSUE_STATE_CONDITIONS are mutually exclusive, except for the following pairs.
	VALID_CONDITION_PAIRS = {
	"sentry.rules.conditions.first_seen_event.FirstSeenEventCondition": "sentry.rules.conditions.high_priority_issue.NewHighPriorityIssueCondition",
	"sentry.rules.conditions.high_priority_issue.NewHighPriorityIssueCondition": "sentry.rules.conditions.first_seen_event.FirstSeenEventCondition",
	"sentry.rules.conditions.reappeared_event.ReappearedEventCondition": "sentry.rules.conditions.high_priority_issue.ExistingHighPriorityIssueCondition",
	"sentry.rules.conditions.high_priority_issue.ExistingHighPriorityIssueCondition": "sentry.rules.conditions.reappeared_event.ReappearedEventCondition",
	}

	issue_state_conditions, frequency_conditions = categorize_conditions(conditions)
	# must have at least one condition to filter activity
	if not issue_state_conditions and not frequency_conditions:
	return None
	elif len(issue_state_conditions) > 1 and condition_match == "all":
	# Of the supported conditions, any more than two would be mutually exclusive
	if len(issue_state_conditions) > 2:
	return {}

	condition_ids = {condition["id"] for condition in issue_state_conditions}

	# all the issue state conditions are mutually exclusive
	if not any(condition in VALID_CONDITION_PAIRS for condition in condition_ids):
	return {}

	# if there are multiple issue state conditions, they must be one of the valid pairs
	for condition in condition_ids:
	if (
	condition in VALID_CONDITION_PAIRS
	and VALID_CONDITION_PAIRS[condition] not in condition_ids
	):
	return {}

	"""
	Filters the activity to contain only groups that have the most events (out of the given groups) in the past 2 weeks.
	If no groups are provided because there are no issue state change conditions, returns the top groups overall.

	Since frequency conditions require snuba query(s), we need to limit the number groups we process.
	"""
	if has_issue_state_condition:
	datasets = {dataset_map.get(group) for group in condition_activity.keys()}
	else:
	# condition_activity will be empty because there are no issue state conditions.
	# So, we look to find top groups over all datasets
	datasets = set(DATASET_TO_COLUMN_NAME.keys())
	group_ids = list(condition_activity.keys())

	# queries each dataset for top x groups and then gets top x overall
	query_params = []
	for dataset in datasets:
	kwargs = get_update_kwargs_for_groups(
	dataset,
	group_ids,
	{
	"dataset": dataset,
	"start": start,
	"end": end,
	"filter_keys": {"project_id": [project.id]},
	"aggregations": [("count", "group_id", "groupCount")],
	"groupby": ["group_id"],
	"orderby": "-groupCount",
	"selected_columns": ["group_id", "groupCount"],
	"limit": FREQUENCY_CONDITION_GROUP_LIMIT,
	},

		for condition in conditions:
		conditions_to_triggered_groups[condition.id] = condition.get_preview_groups(group_ids)

Uh oh!

feat(aci): setup workflow preview abstraction #87553

feat(aci): setup workflow preview abstraction #87553

Uh oh!

Conversation

cathteng commented Mar 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

snigdhas Mar 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Mar 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cathteng Mar 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cathteng commented Apr 7, 2025

Uh oh!

Uh oh!

cathteng commented Mar 20, 2025 •

edited

Loading

snigdhas Mar 26, 2025 •

edited

Loading

codecov bot commented Mar 20, 2025 •

edited

Loading

cathteng Mar 28, 2025 •

edited

Loading