s3-notifications: Unable to update the s3 event notifications on an existing S3 bucket. #31303
Description
Describe the bug
I am facing an error when trying to update the s3 event notification configuration on an existing S3 bucket using CDK.
Created a stack to add s3 event notification on an existing s3 bucket. Create operation goes through successfully.
When I update the cdk stack to add event notifications on the same s3 bucket, the Update operation fails with below error.
Error:
Received response status [FAILED] from custom resource. Message returned: Error: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.. See the details in CloudWatch Log Stream:
Here the lambda code backing the BucketNotifications custom resource is unable to identify that the existing event notifications were also managed by the same cdk stack thereby creating duplicate notifications resulting in error.
Regression Issue
- Select this option if this issue appears to be a regression.
Last Known Working CDK Version
No response
Expected Behavior
Should be able to add additional notification configuration on the existing s3 bucket.
Current Behavior
Issue
Created a cdk stack and added an event notification on an existing s3 bucket. Create operation is successful.
Updated the same stack to add a new event notification along with the existing ones. The update operation fails with below error.
Error:
Received response status [FAILED] from custom resource. Message returned: Error: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.. See the details in CloudWatch Log Stream:
Reproduction Steps
Steps:
1. Create a stack by commenting out the 2nd event notification (notificationfilter2)
2. once stack is created, update the stack after uncommenting the 2nd event notification.
my_lambda = _lambda.Function(
self, 'HelloHandler',
runtime=_lambda.Runtime.PYTHON_3_8,
code=_lambda.Code.from_asset('lambda'),
handler='hello.handler',
)
bucket = _s3.Bucket.from_bucket_name(self, "Bucket", "bucketname")
notification = _s3_notify.LambdaDestination(my_lambda)
notificationfilter1 = _s3.NotificationKeyFilter(prefix="foo/", suffix="bar/")
bucket.add_event_notification(_s3.EventType.OBJECT_CREATED, notification, notificationfilter1
)
# notificationfilter2 = _s3.NotificationKeyFilter(prefix="fo1/",suffix="ba1/",)
# bucket.add_event_notification(_s3.EventType.OBJECT_CREATED, notification, notificationfilter2
# )
Possible Solution
Analysis details:
Create a cdk stack to add below event notification to an existing s3 bucket. The Create operation is successful.
'Events': ['s3:ObjectCreated:*'], 'Filter': {'Key': {'FilterRules': [{'Name': 'Prefix', 'Value': 'bar/'}, {'Name': 'Suffix', 'Value': 'foo/'}]}}}
In the notifications-resource-handler code
For unmanaged buckets, there is a get_id function used to evaluate the hash for each event notification to confirm if this is created by the stack or is an existing external configuration:
def get_id(n):
n['Id'] = ''
strToHash=json.dumps(n, sort_keys=True).replace('"Name": "prefix"', '"Name": "Prefix"').replace('"Name": "suffix"', '"Name": "Suffix"')
return f"{stack_id}-{hash(strToHash)}"
During creation, this goes fine as it treats all existing configurations as external.
Then it appends incoming + external and creates the final notification configuration as expected.
During an update operation, the lambda code will first get the existing event notifications on the s3 bucket.
Then it validates if the existing event notifications matches the notification in the incoming request from Cloudformation.
It does this by evaluating the hash for each existing event notification and validating if this matches with the hash of the incoming event notifications from Cloudformation.
When there is a match, it identifies this existing event configuration as managed by the stack.
This helps in eliminating the duplicates.
There is an issue with this hash evaluation, due to a change in order of the prefix and suffix within the filter rules between the existing notification and the notification coming from Cloudformation.
Sample event notification from Cloudformation request
{
"Events": [
"s3:ObjectCreated:*"
],
"Filter": {
"Key": {
"FilterRules": [
{
"Name": "Suffix",
"Value": "foo/"
},
{
"Name": "Prefix",
"Value": "bar/"
}
]
}
},
"Id": "",
"LambdaFunctionArn": "arn:aws:lambda:<aws-region>:<aws-account-id>:function:<FunctionName>"
}
Sample existing S3 event notification:
"Events": [
"s3:ObjectCreated:*"
],
"Filter": {
"Key": {
"FilterRules": [
{
"Name": "Prefix",
"Value": "bar/"
},
{
"Name": "Suffix",
"Value": "foo/"
}
]
}
},
"Id": "",
"LambdaFunctionArn": "arn:aws:lambda:<aws-region>:<aws-account-id>:function:<FunctionName>"
}
Note: Even though the content of the above jsons are the same, the order of the filter rules (prefix and suffix) are different.
Due to this, the hash evaluated will also be different. So, it is unable to identify that the existing configuration was also added by the stack itself.
This ends up in a configuration which includes the duplicates (existing configurations on s3 + incoming event configuration from Cloudformation stack)
resulting in below error:
Received response status [FAILED] from custom resource. Message returned: Error: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.
This can be workaround by changing the order of filters in the cdk synthesized stack template, by passing the prefix first and then suffix within the filter rules array.
Additional Information/Context
No response
CDK CLI Version
2.152.0
Framework Version
No response
Node.js Version
v20.16.0
OS
Linux
Language
Python
Language Version
No response
Other information
No response