Bug: DynamoDBStreamChangedRecordModel doesn't deserialize ApproximateCreationDateTime with time components when using Pydantic V2 #3048
Description
Expected Behaviour
DynamoDB Stream events with ApproximateCreationDateTime with time components should be deserialized.
Current Behaviour
In v2, pydantic raises the validation error date_from_datetime_inexact. In v1, pydantic drops time components silently when parsing timestamps into date objects.
Code snippet
from aws_lambda_powertools.utilities.parser.models import DynamoDBStreamModel
# This is an example from the official AWS documention:
# https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.Lambda.Tutorial.html#Streams.Lambda.Tutorial.LambdaFunction
event = {
"Records": [
{
"eventID": "7de3041dd709b024af6f29e4fa13d34c",
"eventName": "INSERT",
"eventVersion": "1.1",
"eventSource": "aws:dynamodb",
"awsRegion": "region",
"dynamodb": {
"ApproximateCreationDateTime": 1479499740,
"Keys": {
"Timestamp": {"S": "2016-11-18:12:09:36"},
"Username": {"S": "John Doe"},
},
"NewImage": {
"Timestamp": {"S": "2016-11-18:12:09:36"},
"Message": {"S": "This is a bark from the Woofer social network"},
"Username": {"S": "John Doe"},
},
"SequenceNumber": "13021600000000001596893679",
"SizeBytes": 112,
"StreamViewType": "NEW_IMAGE",
},
"eventSourceARN": "arn:aws:dynamodb:region:account ID:table/BarkTable/stream/2016-11-16T20:42:48.104",
}
]
}
DynamoDBStreamModel.model_validate(event)Possible Solution
The simplest solution probably is to promote ApproximateCreationDateTime to an Optional[datetime] instead of Optional[date].
The test events in dynamoStreamEvent.json don't contain the attribute which is probably why this wasn't caught during the pydantic v2 upgrade.
Steps to Reproduce
- Put the code snippet into a Python script.
- Install latest aws-lambda-powertools + pydantic v2 versions.
- Execute the Python script.
Powertools for AWS Lambda (Python) version
latest
AWS Lambda function runtime
3.11
Packaging format used
PyPi
Debugging logs
./python-3.11.4/lib/python3.11/site-packages/aws_lambda_powertools/package_logger.py:20: UserWarning: POWERTOOLS_DEBUG environment variable is enabled. Setting logging level to DEBUG.
if powertools_debug_is_set():
Traceback (most recent call last):
File "dynamodb_fail.py", line 30, in <module>
DynamoDBStreamModel.model_validate(event)
File "./python-3.11.4/lib/python3.11/site-packages/pydantic/main.py", line 504, in model_validate
return cls.__pydantic_validator__.validate_python(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for DynamoDBStreamModel
Records.0.dynamodb.ApproximateCreationDateTime
Datetimes provided to dates should have zero time - e.g. be exact dates [type=date_from_datetime_inexact, input_value=1479499740, input_type=int]
For further information visit https://errors.pydantic.dev/2.3/v/date_from_datetime_inexact