Skip to content

refactor(s3): Move all cache files to the cache/ folder #833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 20, 2024

Conversation

ViBiOh
Copy link
Contributor

@ViBiOh ViBiOh commented Aug 7, 2024

What does this PR do?

Change the location of caches files under a cache/ folder prefix. So that it's allowed by the Policy and will result in a NotFoundException instead of AccessDeniedException, as explained here.

Motivation

avoid CloudTrail pollution for AccessDeniedException

Testing Guidelines

Additional Notes

Types of changes

  • Bug fix
  • New feature
  • Breaking change
  • Misc (docs, refactoring, dependency upgrade, etc.)

Check all that apply

  • This PR's description is comprehensive
  • This PR contains breaking changes that are documented in the description
  • This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
  • This PR impacts documentation, and it has been updated (or a ticket has been logged)
  • This PR's changes are covered by the automated tests
  • This PR collects user input/sensitive content into Datadog
  • This PR passes the integration tests (ask a Datadog member to run the tests)
  • This PR passes the unit tests
  • This PR passes the installation tests (ask a Datadog member to run the tests)

@github-actions github-actions bot added the aws label Aug 7, 2024
@ViBiOh ViBiOh marked this pull request as ready for review August 8, 2024 14:32
@ViBiOh ViBiOh requested a review from a team as a code owner August 8, 2024 14:32
@ViBiOh ViBiOh changed the title refactor(s3): Prefer s3 for checking file existence refactor(s3): Prefer s3 list for checking file existence Aug 8, 2024
@ViBiOh ViBiOh force-pushed the vibioh/refine_access_denied_exception branch from a0ce784 to 7270fe0 Compare August 8, 2024 14:41
@klivan
Copy link
Contributor

klivan commented Aug 12, 2024

My concern is that List* operations are 10x more expensive than Get*, so we'll 10x customers' costs for these checks.

@tyrcho
Copy link
Contributor

tyrcho commented Aug 13, 2024

My concern is that List* operations are 10x more expensive than Get*, so we'll 10x customers' costs for these checks.

If this cost becomes a concern, I guess we could offer both alternatives, configurable with an env variable.
By default we should keep the existing behaviour, and if the variable is set we would switch to List operations to reduce CloudTrail pollution with some extra S3 cost.

@ViBiOh ViBiOh force-pushed the vibioh/refine_access_denied_exception branch 2 times, most recently from 2639383 to f6ae413 Compare August 20, 2024 11:17
@ViBiOh ViBiOh changed the title refactor(s3): Prefer s3 list for checking file existence refactor(s3): Move all cache files to the cache/ folder Aug 20, 2024
@ViBiOh ViBiOh force-pushed the vibioh/refine_access_denied_exception branch from 9f81c97 to 8d5969e Compare August 20, 2024 12:04
@ViBiOh ViBiOh merged commit ef89890 into master Aug 20, 2024
13 checks passed
@ViBiOh ViBiOh deleted the vibioh/refine_access_denied_exception branch August 20, 2024 12:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants