[ONNX] Improve verify_onnx_program to use VerificationInterpreter #148706
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/148706
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New Failure, 59 PendingAs of commit 291183d with merge base 96176e3 ( NEW FAILURE - The following job has failed:
UNSTABLE - The following jobs are marked as unstable, possibly due to flakiness on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
| onnx_outputs = onnx_program(*args, **kwargs) | ||
|
|
||
| # Flatten args for ONNX program and the VerificationInterpreter | ||
| flat_args, _ = exported_program._get_flat_args_with_check(args, kwargs) |
There was a problem hiding this comment.
No but I figured this is more robust when the kwargs are not in order
ghstack-source-id: b273d2d Pull Request resolved: pytorch#148706 Signed-off-by: Justin Chu <[email protected]>
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 mandatory check(s) failed. The first few are: Dig deeper by viewing the failures on hud |
|
@pytorchbot merge -i |
|
@pytorchbot merge -f "ONNX tests passed" |
Merge startedYour change will be merged while ignoring the following 1 checks: pull / win-vs2022-cpu-py3 / build Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
|
The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command |
Merge startedYour change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Use a simple try catch to handle onnx runtime errors in the verification interpreter when that happens. One example is ort will sometimes produce a list of None for some nodes. I am not sure how that happens yet. Pull Request resolved: #148730 Approved by: https://github.com/titaiwangms ghstack dependencies: #148706
Stack from ghstack (oldest at bottom):
I realized we can just extend
verify_onnx_programto return intermediate values. There is no need for us to expose the VerificationInterpreter to users.I added a
compare_intermediatesoption toverify_onnx_program.