-
Notifications
You must be signed in to change notification settings - Fork 127
Introduce a triton version checking #1608
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Summary: to avoid confusion after a recent triton upgrade in #1585
torchinductor/utils.py
Outdated
triton_compiler_version = triton.runtime.version_key().split("-")[6] | ||
if triton_compiler_version != "7929002797455b30efce6e41eddc6b57": | ||
log.error( | ||
"Please update your triton version to af76c989eb4799b015f8b288ccd8421558772e56" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's include the pip command needed to install it in the error message.
torchinductor/utils.py
Outdated
@@ -33,6 +35,15 @@ def has_triton(): | |||
try: | |||
import triton | |||
|
|||
if triton is not None: | |||
# This is temp fix to hint people upgrad triton after https://github.com/pytorch/torchdynamo/pull/1585 | |||
triton_compiler_version = triton.runtime.version_key().split("-")[6] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we disable this check in FBCODE?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When we sync to fbcode this time, we will have to update trition. So I guess it should be fine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, but then when we want to update the next time this will prevent that upgrade.
torchinductor/utils.py
Outdated
if triton is not None: | ||
# This is temp fix to hint people upgrad triton after https://github.com/pytorch/torchdynamo/pull/1585 | ||
triton_compiler_version = triton.runtime.version_key().split("-")[6] | ||
if triton_compiler_version != "7929002797455b30efce6e41eddc6b57": |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Requirering a specific version sounds a bit harsh? If you only want to prevent that error, you can just detect that the function takes more arguments now.
Also you can have an env variable to override this check so that devs that know what they're doing can set it to skip this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Switched to checking launch-time exception instead. Should be general enough unless the next time triton upgrade triggers exactly the same error. This can also address Jason's FBCODE concern.
torchinductor/utils.py
Outdated
log.error( | ||
"Please update your triton version to af76c989eb4799b015f8b288ccd8421558772e56" | ||
) | ||
exit() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not raise RuntimeError()
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SGTM
Summary: to avoid confusion after a recent triton upgrade in #1585
Summary: to avoid confusion after a recent triton upgrade in #1585