-
-
Notifications
You must be signed in to change notification settings - Fork 196
DEV: Add nbval to CI. #132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Co-authored-by: Mridul Seth <[email protected]>
strategy: | ||
max-parallel: 12 | ||
matrix: | ||
os: [Ubuntu-20.04, macOS-latest] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any reasons to not add windows, too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only because I expected it would require special windows-specific handling to make it work... I just tried it and it seems that's the case, see: https://github.com/numpy/numpy-tutorials/runs/7309090446?check_suite_focus=true
My vote is for incremental adoption: get linux/macos in sooner since they're working now, and open a follow-up issue for anyone who wants to figure out what magic combo of workarounds is necessary to get it to work on windows :)
I'm still not 100% happy with my CI solution here: https://github.com/NASA-NAVO/navo-workshop, but I think adding tox helps a lot with point no3 in your list, and it also helps to manage and run the same command through the different CI providers and by contributors. |
This reverts commit 6716218.
I'm +1 for a tox.ini (or whatever testing config file) as well. I'd also vote that for that to be done in a follow-up PR so that it's self-contained, but if you'd like to add it here please feel free to do so! |
@rossbar - it's your PR, so whatever works for you sounds good. I can put doing some incremental stuff for this as part of my scipy sprint, but no promises that I actually manage to go through that list. |
I'm going to go ahead and put this in as-is to get the additional testing in place. I'll open issues for the follow-up tasks that were identified above. Thanks @bsipocz ! |
Closes #51 .
Add a basic
nbval
workflow to the CI, borrowing heavily from @MridulS 's recipe in nx-guides. Currently only smoketests the notebooks withpytest --nbval-lax
. More rigorous testing (i.e. comparison with saved outputs) can always be explored in followup PRs.A couple of potential improvements:
conftest.py
or similar mechanismpytest
andnbval
as testing requirements and update the contributor guide with a quick note on how to run locally.