Bug: `APIGatewayRestResolver(enable_validation=True)` incompatible with `from __future__ import annotations`

[jmahlik]

Expected Behaviour

Using the data validation feature of the event resolvers doesn't appear compatible with from __future__ import annotations. There's undefined refs when trying to rebuild the model in the TypeAdapter.

It would be ideal to import annotations so type defs are ignored at runtime. Additionally, when using boto stubs they really shouldn't be shipped at runtime since they can be pretty large.

Current Behaviour

See details for stack trace.

Traceback (most recent call last):
  File "\venv\Lib\site-packages\pydantic\type_adapter.py", line 277, in _init_core_attrs
    self._core_schema = _getattr_no_parents(self._type, '__pydantic_core_schema__')
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\pydantic\type_adapter.py", line 119, in _getattr_no_parents
    raise AttributeError(attribute)
AttributeError: __pydantic_core_schema__

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "\venv\Lib\site-packages\pydantic\_internal\_generate_schema.py", line 724, in _resolve_forward_ref
    obj = _typing_extra.eval_type_backport(obj, globalns=self._types_namespace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\pydantic\_internal\_typing_extra.py", line 264, in eval_type_backport
    return typing._eval_type(  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Python\Python312\Lib\typing.py", line 415, in _eval_type
    return t._evaluate(globalns, localns, type_params, recursive_guard=recursive_guard)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Python\Python312\Lib\typing.py", line 947, in _evaluate
    eval(self.__forward_code__, globalns, localns),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 1, in <module>
NameError: name 'Input' is not defined. Did you mean: 'input'?

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "\f.py", line 44, in <module>
    print(app.resolve(event, {}))
          ^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\aws_lambda_powertools\event_handler\api_gateway.py", line 2100, in resolve
    response = self._resolve().build(self.current_event, self._cors)
               ^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\aws_lambda_powertools\event_handler\api_gateway.py", line 2209, in _resolve
    return self._call_route(route, route_keys)  # pass fn args
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\aws_lambda_powertools\event_handler\api_gateway.py", line 2316, in _call_route
    route(router_middlewares=self._router_middlewares, app=self, route_arguments=route_arguments),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\aws_lambda_powertools\event_handler\api_gateway.py", line 450, in __call__
    return self._middleware_stack(app)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\aws_lambda_powertools\event_handler\api_gateway.py", line 1436, in __call__
    return self.current_middleware(app, self.next_middleware)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\aws_lambda_powertools\event_handler\middlewares\base.py", line 121, in __call__
    return self.handler(app, next_middleware)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\aws_lambda_powertools\event_handler\middlewares\openapi_validation.py", line 80, in handler
    route.dependant.path_params,
    ^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\aws_lambda_powertools\event_handler\api_gateway.py", line 499, in dependant
    self._dependant = get_dependant(path=self.openapi_path, call=self.func, responses=self.responses)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\aws_lambda_powertools\event_handler\openapi\dependant.py", line 192, in get_dependant
    param_field = analyze_param(
                  ^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\aws_lambda_powertools\event_handler\openapi\params.py", line 965, in analyze_param
    field = _create_model_field(field_info, type_annotation, param_name, is_path_param)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\aws_lambda_powertools\event_handler\openapi\params.py", line 1098, in _create_model_field
    return create_response_field(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\aws_lambda_powertools\event_handler\openapi\params.py", line 1067, in create_response_field
    return ModelField(**kwargs)  # type: ignore[arg-type]
           ^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 6, in __init__
  File "\venv\Lib\site-packages\aws_lambda_powertools\event_handler\openapi\compat.py", line 86, in __post_init__
    self._type_adapter: TypeAdapter[Any] = TypeAdapter(
                                           ^^^^^^^^^^^^
  File "\venv\Lib\site-packages\pydantic\type_adapter.py", line 264, in __init__
    self._init_core_attrs(rebuild_mocks=False)
  File "\venv\Lib\site-packages\pydantic\type_adapter.py", line 142, in wrapped
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\pydantic\type_adapter.py", line 284, in _init_core_attrs
    self._core_schema = _get_schema(self._type, config_wrapper, parent_depth=self._parent_depth)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\pydantic\type_adapter.py", line 102, in _get_schema
    schema = gen.generate_schema(type_)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\pydantic\_internal\_generate_schema.py", line 512, in generate_schema
    schema = self._generate_schema_inner(obj)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\pydantic\_internal\_generate_schema.py", line 768, in _generate_schema_inner
    return self._annotated_schema(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\pydantic\_internal\_generate_schema.py", line 1818, in _annotated_schema
    source_type, *annotations = self._get_args_resolving_forward_refs(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\pydantic\_internal\_generate_schema.py", line 746, in _get_args_resolving_forward_refs
    args = tuple([self._resolve_forward_ref(a) if isinstance(a, ForwardRef) else a for a in args])
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\venv\Lib\site-packages\pydantic\_internal\_generate_schema.py", line 726, in _resolve_forward_ref
    raise PydanticUndefinedAnnotation.from_name_error(e) from e
pydantic.errors.PydanticUndefinedAnnotation: name 'Input' is not defined

For further information visit https://errors.pydantic.dev/2.8/u/undefined-annotation

Code snippet

# Works if removed
from __future__ import annotations

import json
from typing import TYPE_CHECKING

from aws_lambda_powertools.event_handler import APIGatewayRestResolver
from pydantic import BaseModel

if TYPE_CHECKING:
    # Actual use case imports boto stubs
    # These are giant pyi files that shouldn't be shipped at runtime
    from typing import Iterable


class Input(BaseModel):
    email: str


# Tried this as well
# Input = ForwardRef("Input")


class Output(BaseModel):
    response: str


app = APIGatewayRestResolver(enable_validation=True)


@app.post("/hello")
def hello(event: Input) -> Output:
    return Output(response=f"Hello {event.email}")


def func(a: Iterable[int]): ...


event = {
    "path": "/hello",
    "httpMethod": "POST",
    "requestContext": {
        "requestId": "227b78aa-779d-47d4-a48e-ce62120393b8",
    },
    "body": json.dumps(
        {
            "email": "hello@test.com",
        }
    ),
}

print(app.resolve(event, {}))

Possible Solution

The closest thing I could find was this fastapi issue fastapi/fastapi#10007. I've tried Input = ForwardRef("Input") from the pydantic docs but that doesn't appear to resolve the issue.

One workaround is to stringify the annotations not required at runtime i.e.

def func(a: "Iterable[int]"): 
    ...

This isn't ideal but it does work.

Steps to Reproduce

Run the file.

Environment:

aws-lambda-powertools    2.43.1
pydantic                             2.8.2
pydantic_core                    2.20.1

Powertools for AWS Lambda (Python) version

latest

AWS Lambda function runtime

3.12

Packaging format used

PyPi

Debugging logs

No response

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

[leandrodamascena]

Looking at this now.

[leandrodamascena]

Hey @jmahlik! I can confirm that we have something weird here when we try to use TypeAdapter to convert the model. I know that Pydantic has some limitations when working with __future__ annotations but I'll need a time to see what is going on with our integration.

It would be ideal to import annotations so type defs are ignored at runtime. Additionally, when using boto stubs they really shouldn't be shipped at runtime since they can be pretty large.

I totally agree that you shouldn't ship boto stubs to production, they are more suitable for development environments. In the meantime, you can use quotes ("Iterable[int]") for forward references and avoid issues.

Thanks.