Add support for AWS Secrets Manager #532

mavwolverine · 2025-02-07T01:49:56Z

Working on the unit tests

mavwolverine · 2025-02-07T17:00:24Z

Not sure why the lint is failing for ruff. Locally it says everything is fine.

mavwolverine · 2025-02-11T06:04:41Z

boto3 dependency on urllib3 conflicts with requests on Python < 3.10.
So this feature will have to be available only python >=3.10

mavwolverine · 2025-02-11T14:51:06Z

The tests failing are not related to the change

hramezani · 2025-02-19T11:05:19Z

Thanks @mavwolverine for this PR.

I think the test failures are related to this PR's package update.

Also, you need to document this new settings source and provide an example of how to use it.

mavwolverine · 2025-03-07T05:49:36Z

Will work on the docs and sample over the weekend

mavwolverine · 2025-03-07T19:43:56Z

@hramezani Hopefully this is the last time I have to rework the PR. Had to refactor it due to dependency changes in upstream and then UV changes.
I believe the PR is ready

hramezani · 2025-03-10T08:55:34Z

Thanks @mavwolverine for the update.

I have a refactoring PR on my list to review. I will then return to this PR. You will probably need to rebase it later, and I will let you know.

I also can help you if you don't have time to work

pyproject.toml

docs/index.md

hyperlint-ai · 2025-03-11T17:18:58Z

PR Change Summary

Added support for AWS Secrets Manager in the documentation, including usage instructions and example code.

Introduced a new section for AWS Secrets Manager in the documentation
Provided usage instructions for setting parameters and naming conventions
Included example code for integrating AWS Secrets Manager with Pydantic settings

Modified Files

docs/index.md

How can I customize these reviews?

Check out the Hyperlint AI Reviewer docs for more information on how to customize the review.

If you just want to ignore it on this PR, you can add the hyperlint-ignore label to the PR. Future changes won't trigger a Hyperlint review.

Note specifically for link checks, we only check the first 30 links in a file and we cache the results for several hours (for instance, if you just added a page, you might experience this). Our recommendation is to add hyperlint-ignore to the PR to ignore the link check for this PR.

What is Hyperlint?

Hyperlint is an AI agent that helps you write, edit, and maintain your documentation.

Learn more about the Hyperlint AI reviewer and the checks that we can run on your documentation.

mavwolverine · 2025-03-11T18:56:25Z

Finally!

@hramezani please check. Thanks

hramezani · 2025-03-12T14:13:47Z

Thanks @mavwolverine for the update!

This is the first PR on my list for review/merge. I will get back to it later.

hramezani · 2025-03-17T11:59:28Z

tests/test_source_aws_secrets_manager.py

+if not yaml:
+    pytest.skip('PyYAML is not installed', allow_module_level=True)


Do we need pyyaml to be installed for this test?

Yes. Moto the mocking library for boto3 has a dependency on responses which imports yaml. The second tests run in ci was failing and tests won't even start.

Hence the import moto is inside try catch after trying to import yaml.

And the skip module is because it fails for the @mock_aws decorator.

hramezani · 2025-03-17T12:28:10Z

@mavwolverine I am not familiar with AWS Secret Manager. is secret_id equal to the secret name we define in AWS Secret Manager? it seems the value of secret has to be a json dump(plain text). please mention this in the doc. probably plain-text is the only type of secrets that this settings source can handle. if so, please update the doc

Can you explain how I can test your PR?

tests/test_source_aws_secrets_manager.py

hramezani · 2025-03-17T12:38:50Z

tests/test_source_aws_secrets_manager.py

+        class AWSSecretsManagerSettings(BaseSettings):
+            """AWSSecretsManager settings."""
+
+            SqlServerUser: str


do we need this field? because the alias of the next field is SqlServerUser

@hramezani I kept the tests similar to Azure.

https://github.com/pydantic/pydantic-settings/blob/main/tests/test_source_azure_key_vault.py#L54

I think we don't need SqlServerUser: str and we can remove it

mavwolverine · 2025-03-17T14:59:13Z

@mavwolverine I am not familiar with AWS Secret Manager. is secret_id equal to the secret name we define in AWS Secret Manager? it seems the value of secret has to be a json dump(plain text). please mention this in the doc. probably plain-text is the only type of secrets that this settings source can handle. if so, please update the doc

Can you explain how I can test your PR?

Secret id is the secret name identifier in AWS.
The PR support Key Value pairs in Secrets Manager.

The key value when retrieved is returned as a json string.

To test in real AWS account, login to AWS account, go to AWS Secrets Manager, add new secret.
Select secret type as "Other type of secret".
Add key value pairs with similar naming as the variables in code.
Configure AWS env variables for credentials (on EC2 instances this is done using Instance role and is automatic).

hramezani · 2025-03-17T15:57:51Z

Thanks @mavwolverine for explaining how to test it.

Please address two small comment on tests and then we can merge the PR

tests/test_source_aws_secrets_manager.py

hramezani · 2025-03-19T11:08:33Z

Thanks @mavwolverine

mavwolverine · 2025-03-19T14:37:00Z

@hramezani Thanks for taking care of the changes. Got busy with work and was planning on looking into the changes over the weekend.

mavwolverine force-pushed the aws_secrets_manager branch 2 times, most recently from 12534c7 to 6b927c3 Compare February 7, 2025 15:15

mavwolverine force-pushed the aws_secrets_manager branch from 6b927c3 to 74b09c0 Compare February 7, 2025 17:50

mavwolverine marked this pull request as ready for review February 11, 2025 06:04

mavwolverine force-pushed the aws_secrets_manager branch from bdc4bd0 to 2742887 Compare February 11, 2025 14:53

mavwolverine force-pushed the aws_secrets_manager branch from 2742887 to eefb492 Compare March 7, 2025 00:33

mavwolverine force-pushed the aws_secrets_manager branch from 4436bba to dd3965d Compare March 7, 2025 19:40

hramezani reviewed Mar 11, 2025

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

hramezani reviewed Mar 11, 2025

View reviewed changes

docs/index.md Outdated Show resolved Hide resolved

Add support for AWS Secrets Manager

fa6f3a0

mavwolverine force-pushed the aws_secrets_manager branch from 3ab9a32 to fa6f3a0 Compare March 11, 2025 17:54

fix imports

5a76f48

hramezani reviewed Mar 17, 2025

View reviewed changes

pydantic-hooky bot added the awaiting author revision label Mar 17, 2025

pydantic-hooky bot assigned mavwolverine Mar 17, 2025

hramezani reviewed Mar 17, 2025

View reviewed changes

tests/test_source_aws_secrets_manager.py Outdated Show resolved Hide resolved

hramezani reviewed Mar 17, 2025

View reviewed changes

hramezani reviewed Mar 19, 2025

View reviewed changes

tests/test_source_aws_secrets_manager.py Outdated Show resolved Hide resolved

Update tests/test_source_aws_secrets_manager.py

a2f7478

hramezani reviewed Mar 19, 2025

View reviewed changes

tests/test_source_aws_secrets_manager.py Outdated Show resolved Hide resolved

Update tests/test_source_aws_secrets_manager.py

6134b0a

hramezani reviewed Mar 19, 2025

View reviewed changes

tests/test_source_aws_secrets_manager.py Outdated Show resolved Hide resolved

Update tests/test_source_aws_secrets_manager.py

e00cd49

hramezani approved these changes Mar 19, 2025

View reviewed changes

hramezani merged commit c8a0df4 into pydantic:main Mar 19, 2025
19 checks passed

		if not yaml:
		pytest.skip('PyYAML is not installed', allow_module_level=True)

Uh oh!

Add support for AWS Secrets Manager #532

Add support for AWS Secrets Manager #532

Uh oh!

Conversation

mavwolverine commented Feb 7, 2025

Uh oh!

mavwolverine commented Feb 7, 2025

Uh oh!

mavwolverine commented Feb 11, 2025

Uh oh!

mavwolverine commented Feb 11, 2025

Uh oh!

hramezani commented Feb 19, 2025

Uh oh!

mavwolverine commented Mar 7, 2025

Uh oh!

mavwolverine commented Mar 7, 2025

Uh oh!

hramezani commented Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hyperlint-ai bot commented Mar 11, 2025

PR Change Summary

Uh oh!

mavwolverine commented Mar 11, 2025

Uh oh!

hramezani commented Mar 12, 2025

Uh oh!

hramezani Mar 17, 2025

Choose a reason for hiding this comment

Uh oh!

mavwolverine Mar 17, 2025

Choose a reason for hiding this comment

Uh oh!

hramezani commented Mar 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

hramezani Mar 17, 2025

Choose a reason for hiding this comment

Uh oh!

mavwolverine Mar 17, 2025

Choose a reason for hiding this comment

Uh oh!

hramezani Mar 17, 2025

Choose a reason for hiding this comment

Uh oh!

mavwolverine commented Mar 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hramezani commented Mar 17, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hramezani commented Mar 19, 2025

Uh oh!

Uh oh!

mavwolverine commented Mar 19, 2025

Uh oh!

Uh oh!

hramezani commented Mar 10, 2025 •

edited

Loading

hramezani commented Mar 17, 2025 •

edited

Loading

mavwolverine commented Mar 17, 2025 •

edited

Loading