Skip to content

Conversation

@kevinjqliu
Copy link
Contributor

@kevinjqliu kevinjqliu commented Aug 15, 2025

Closes #13818

This PR adds support for passing Microsoft Entra access token to ADLSFileIO using the newly added ADLS_TOKEN property. The access token is used to create a TokenCredential class used to configure the DataLakeFileSystemClientBuilder.

See a similar implementation in apache doris

The token expiration is set for 1 hour for now, which is the default lifetime of an access token. We can make this configurable in the future.

Testing

Built and tested locally with spark

./gradlew -DsparkVersions=3.5 :iceberg-spark:iceberg-spark-runtime-3.5_2.12:jar 

Pass in token string

    .config("spark.sql.catalog.onelake.io-impl", "org.apache.iceberg.azure.adlsv2.ADLSFileIO")
    .config("spark.sql.catalog.onelake.adls.token", token)

@github-actions github-actions bot added the AZURE label Aug 15, 2025
@kevinjqliu
Copy link
Contributor Author

cc @bryanck @mrcnc @NikitaMatskevich since yall have experience working with ADLSFileIO :)

@pvary
Copy link
Contributor

pvary commented Aug 18, 2025

@kevinjqliu: Could we add a test case for the new feature?

@kevinjqliu kevinjqliu requested a review from pvary August 19, 2025 04:26
@kevinjqliu
Copy link
Contributor Author

@pvary added! could you take another look?

@pvary pvary merged commit ede5b5a into apache:main Aug 19, 2025
42 checks passed
@pvary
Copy link
Contributor

pvary commented Aug 19, 2025

Merged to main.
Thanks for the PR @kevinjqliu!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Allow ADLSFileIO to Accept Arbitrary Access Tokens for Authentication

2 participants