Skip to content

Fix concurrency issues in Azure credential refresh. #13730

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
nastra merged 2 commits into from
Aug 6, 2025
Merged

Fix concurrency issues in Azure credential refresh. #13730

nastra merged 2 commits into from
Aug 6, 2025

Conversation

@msawyer-palantir
Copy link
Contributor

@msawyer-palantir msawyer-palantir commented Aug 4, 2025 •
edited
Loading

Fixes #13733

The first commit fixes a crash when trying to use the pipeline async on a parallel Reactor thread:

Caused by: java.lang.IllegalStateException: block()/blockFirst()/blockLast() are blocking, which is not supported in thread parallel-1
	at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:83)
	at reactor.core.publisher.Mono.block(Mono.java:1742)
	at org.apache.iceberg.azure.adlsv2.VendedAdlsCredentialProvider.credentialForAccount(VendedAdlsCredentialProvider.java:78)
	at org.apache.iceberg.azure.adlsv2.VendedAzureSasCredentialPolicy.maybeUpdateCredential(VendedAzureSasCredentialPolicy.java:58)
	at org.apache.iceberg.azure.adlsv2.VendedAzureSasCredentialPolicy.process(VendedAzureSasCredentialPolicy.java:46)
	at com.azure.core.http.HttpPipelineNextPolicy.process(HttpPipelineNextPolicy.java:81)
	at com.azure.storage.common.policy.MetadataValidationPolicy.process(MetadataValidationPolicy.java:45)
	at com.azure.core.http.HttpPipelineNextPolicy.process(HttpPipelineNextPolicy.java:81)
	at com.azure.core.http.policy.AddDatePolicy.lambda$process$0(AddDatePolicy.java:43)
	at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:45)
	... 12 more

The second commit fixes some other thread-safety issues I noticed in the relevant code, which indeed caused failures in our tests.

cc @nastra

@msawyer-palantir
Azure: Fix inappropriate Mono#block call in credential vending.
2b7a0bd 
These calls are only allowed in a non-blocking context, whereas previously one was used in the async version of `process`.
@github-actions github-actions bot added the AZURE label Aug 4, 2025
@msawyer-palantir msawyer-palantir changed the title Fix concurrency issues in Azure credential vending. Fix concurrency issues in Azure credential refresh. Aug 4, 2025
@nastra
Copy link
Contributor

nastra commented Aug 5, 2025

/cc @ChaladiMohanVamsi

@msawyer-palantir
Azure: Fix assorted thread-safety issues with credential vending.
5878ab9 
`HttpPipelinePolicy` implementations should be thread-safe because pipelines can be used to send multiple requests at once.
@msawyer-palantir msawyer-palantir force-pushed the fix/azure-credential-vending-concurrency branch from 6cd8d15 to 5878ab9 Compare August 5, 2025 11:09
@msawyer-palantir msawyer-palantir requested a review from nastra August 5, 2025 11:09
@nastra nastra merged commit 5ce63ab into apache:main Aug 6, 2025
42 checks passed
@PhillHenry PhillHenry mentioned this pull request Sep 14, 2025
Closed
3 tasks
tom-s-powell pushed a commit to tom-s-powell/iceberg that referenced this pull request Oct 6, 2025
@msawyer-palantir @rshkv
Azure: Fix concurrency issues in Azure credential refresh. (apache#13730
6fee0d7 
)

* Azure: Fix inappropriate `Mono#block` call in credential vending.

These calls are only allowed in a non-blocking context, whereas previously one was used in the async version of `process`.

* Azure: Fix assorted thread-safety issues with credential vending.

`HttpPipelinePolicy` implementations should be thread-safe because pipelines can be used to send multiple requests at once.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nastra nastra nastra approved these changes

Assignees

No one assigned

Labels

AZURE

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Azure Credential Refresh does not work in most cases

2 participants

@msawyer-palantir @nastra