Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

migrations(shared-views): backfill existing views with org visibility #88632

Merged
merged 1 commit into from
Apr 3, 2025

Conversation

MichaelSun48
Copy link
Member

@MichaelSun48 MichaelSun48 commented Apr 2, 2025

As part of transitioning to a shared-by-default model for views, we need to make all existing views have Organization level visibility. This PR updates all existing views to have org visibility.

This visibility column is not used by any public frontend pages, so this will not have any visible effects for users.

All entry-points to creating views have been plugged in this PR

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Apr 2, 2025
Copy link
Contributor

github-actions bot commented Apr 2, 2025

This PR has a migration; here is the generated SQL for src/sentry/migrations/0856_backfill_groupsearchviews_with_org_visibility.py ()

--
-- Raw Python operation
--
-- THIS OPERATION CANNOT BE WRITTEN AS SQL

@MichaelSun48 MichaelSun48 force-pushed the msun/sharedViews/backfillViewsToOrgVisibility branch from ac2486b to 877cf7a Compare April 2, 2025 23:39
Copy link
Contributor

github-actions bot commented Apr 2, 2025

This PR has a migration; here is the generated SQL for src/sentry/migrations/0858_backfill_groupsearchviews_with_org_visibility.py ()

--
-- Raw Python operation
--
-- THIS OPERATION CANNOT BE WRITTEN AS SQL

@MichaelSun48 MichaelSun48 marked this pull request as ready for review April 2, 2025 23:43
@MichaelSun48 MichaelSun48 requested a review from a team as a code owner April 2, 2025 23:43
@MichaelSun48 MichaelSun48 requested a review from a team April 2, 2025 23:44
Comment on lines +17 to +19
for gsv in RangeQuerySetWrapperWithProgressBar(GroupSearchView.objects.all()):
gsv.visibility = GroupSearchViewVisibility.ORGANIZATION
gsv.save()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will result in N queries, N being number of GroupSearchView entries in the db.

Can we do instead:

GroupSearchView.objects.all().update(visibility=GroupSearchViewVisibility.ORGANIZATION)

Copy link
Member

@JoshFerge JoshFerge Apr 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

during a backfill, doing updates one by one is ideal, as to not make too many updates at once which could overwhelm our database / cause lots of row-level-locks at once. some relevant docs at https://develop.sentry.dev/backend/application-domains/database-migrations/#filters. can also look at some of our other backfill migrations, this is generally the pattern used, e.g. https://github.com/getsentry/sentry/blob/7fefef3f4518d5410e928306d70646fae2c8f667/src/sentry/migrations/0704_backfill_rule_user_team.py#L20

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another reason to do row level updates instead of big batches is statement timeouts. Queries that update 10k+ rows run the risk of getting terminated by query timeouts.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, ok, makes sense! TIL :)

Comment on lines +17 to +19
for gsv in RangeQuerySetWrapperWithProgressBar(GroupSearchView.objects.all()):
gsv.visibility = GroupSearchViewVisibility.ORGANIZATION
gsv.save()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another reason to do row level updates instead of big batches is statement timeouts. Queries that update 10k+ rows run the risk of getting terminated by query timeouts.

@MichaelSun48 MichaelSun48 merged commit 5f52d0d into master Apr 3, 2025
51 checks passed
@MichaelSun48 MichaelSun48 deleted the msun/sharedViews/backfillViewsToOrgVisibility branch April 3, 2025 16:29
andrewshie-sentry pushed a commit that referenced this pull request Apr 8, 2025
…#88632)

As part of transitioning to a shared-by-default model for views, we need
to make all existing views have Organization level visibility. This PR
updates all existing views to have org visibility.

This visibility column is not used by any public frontend pages, so this
will not have any visible effects for users.

All entry-points to creating views have been plugged in [this
PR](#88630)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Scope: Backend Automatically applied to PRs that change backend components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants