[iOS] added workflows for libtorchvision_ops.a binary build #3582
Conversation
[ghstack-poisoned]
There was a problem hiding this comment.
I had a look at the PR and it looks great, thanks!
I have a couple of suggestions, let me know what you think.
Additionally, linter is failing https://app.circleci.com/pipelines/github/pytorch/vision/6972/workflows/02a89cd4-2c6b-453a-b859-f63895f63f03/jobs/470076 , could you fix those as well?
| cd ${VISION_IOS_ROOT}/build | ||
| cmake -DLIBTORCH_HEADER_ROOT=${LIBTORCH_HEADER_ROOT} \ | ||
| -DCMAKE_TOOLCHAIN_FILE=${VISION_IOS_ROOT}/../cmake/iOS.cmake \ | ||
| -DIOS_ARCH=${IOS_ARCH} \ | ||
| -DIOS_PLATFORM=${IOS_PLATFORM} \ | ||
| .. | ||
| make | ||
| rm -rf ${VISION_IOS_ROOT}/build |
There was a problem hiding this comment.
Should we use ios/build_ios.sh from torchvision here instead of manually replicating that script here?
There was a problem hiding this comment.
Yeah, this is a good idea, fixed.
| # clone main pytorch repo | ||
| mkdir -p ${VISION_IOS_ROOT}/lib | ||
| mkdir -p ${VISION_IOS_ROOT}/build | ||
| git clone --recursive https://github.com/pytorch/pytorch.git | ||
| TORCH_ROOT="${VISION_IOS_ROOT}/pytorch" | ||
|
|
||
| # run build script | ||
| chmod a+x ${TORCH_ROOT}/scripts/build_ios.sh | ||
| echo "########################################################" | ||
| cat ${TORCH_ROOT}/scripts/build_ios.sh | ||
| echo "########################################################" | ||
| echo "IOS_ARCH: ${IOS_ARCH}" | ||
| echo "IOS_PLATFORM: ${IOS_PLATFORM}" | ||
| export IOS_ARCH=${IOS_ARCH} | ||
| export IOS_PLATFORM=${IOS_PLATFORM} | ||
| unbuffer ${TORCH_ROOT}/scripts/build_ios.sh 2>&1 | ts | ||
|
|
||
| LIBTORCH_HEADER_ROOT="${TORCH_ROOT}/build_ios/install/include" |
There was a problem hiding this comment.
Do you think there would be a way of pulling the pre-compiled PyTorch binaries for iOS here, so that we avoid compiling PyTorch? Compiling PyTorch in our CI significantly increases our CI time (the ios jobs are taking 30min to run)
What we currently do for our other torchvision CIs is that we take the latest PyTorch-nightly build (that has been uploaded to conda / pip) and use it to compile torchvision. Could we do something similar here? Like taking the latest PyTorch-ios that has been uploaded to S3 and start from there?
There was a problem hiding this comment.
Yes, we can do that, but I remember you left a comment under the design doc as following:
One thing to keep in mind when building the binaries is that the pytorch used to build torchvision should be the same as the one we make available via the pytorch binaries, otherwise there can be symbol issues.
That is the reason that I and Tao decided to build PyTorch from the source code. It would be even easier if we download the PyTorch iOS binary from S3.
There was a problem hiding this comment.
If we use the PyTorch iOS binaries from S3 to build torchvision (and the CI job times are aligned), this means that the nightlies from both torchvision and PyTorch will work, while if we pull from master this guarantee doesn't apply anymore (except if the git hashes are the same).
The comment in the doc was to try to point out exactly this: if we make available in the PyTorch nightlies a slightly different version than the one that torchvision was compiled with, there can be symbol issues.
[ghstack-poisoned]
[ghstack-poisoned]
Fixed. |
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
…3582) Summary: * [iOS] added workflows for libtorchvision_ops.a binary build [ghstack-poisoned] * Update on "[iOS] added workflows for libtorchvision_ops.a binary build" [ghstack-poisoned] * Update on "[iOS] added workflows for libtorchvision_ops.a binary build" [ghstack-poisoned] * Update on "[iOS] added workflows for libtorchvision_ops.a binary build" [ghstack-poisoned] * Update on "[iOS] added workflows for libtorchvision_ops.a binary build" [ghstack-poisoned] * Update on "[iOS] added workflows for libtorchvision_ops.a binary build" [ghstack-poisoned] * Update on "[iOS] added workflows for libtorchvision_ops.a binary build" [ghstack-poisoned] Reviewed By: fmassa Differential Revision: D27433924 fbshipit-source-id: 2f7b07ae21d6f57a022a4748b8ae15cd71968818 Co-authored-by: Francisco Massa <[email protected]>
Stack from ghstack: