* [gentoo-soc] Weekly Report: Portage Powered Android
@ 2020-07-20 7:40 Gunwant Jain
2020-07-24 14:08 ` Benda Xu
0 siblings, 1 reply; 49+ messages in thread
From: Gunwant Jain @ 2020-07-20 7:40 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1571 bytes --]
Hello Everyone,
This week, I started managing the structure for the modified LineageOS
fork we would keep. Namely the modified `repo-manifest` [1] and
`llvm_android` [2], for now. No patches have been applied to them yet
for the reasons discussed below.
My plan for this week was to start with patching the `llvm_android`
build scripts. Android uses a prebuilt Clang (bundled with the repo)
among other stuff to build the toolchain. So I had to emerge Clang on my
phone.
Emerging Clang turned out not to be a breeze when I have a phone with 3
gigs of RAM. So for about the longest time, I was busy trying to make up
for it. I tried setting up distcc, increased swap/zram, but still the
OOM killer got me.
Later last night(/day), I came across the tips for building clang/gcc in
an embedded Android environment at [3]. So I followed that and as of
now, me writing the mail, OOM has not got me, Clang is emerging.
While I considering the patching part easy, I guess the major bottleneck
in the process of building Android natively would be the compilation.
Essentially I will have to trade time for memory. I will also give
another shot to distcc while I am at this.
Regardless of problems I faced, I made "TODO" comments in the source and
wrote notes about where to patch the build scripts locally. I avidly
wait for Clang to compile and then start testing the patched build
script.
Regards,
Gunwant
[1] https://github.com/WantGuns/manifest
[2] https://github.com/WantGuns/toolchain_llvm_android
[3] https://wiki.gentoo.org/wiki/Project:Android/tarball#Out_of_Memory
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-20 7:40 [gentoo-soc] Weekly Report: Portage Powered Android Gunwant Jain
@ 2020-07-24 14:08 ` Benda Xu
2020-07-24 15:02 ` Gunwant Jain
0 siblings, 1 reply; 49+ messages in thread
From: Benda Xu @ 2020-07-24 14:08 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1928 bytes --]
Gunwant Jain <[email protected]> writes:
> This week, I started managing the structure for the modified LineageOS
> fork we would keep. Namely the modified `repo-manifest` [1] and
> `llvm_android` [2], for now. No patches have been applied to them yet
> for the reasons discussed below.
>
> My plan for this week was to start with patching the `llvm_android`
> build scripts. Android uses a prebuilt Clang (bundled with the repo)
> among other stuff to build the toolchain. So I had to emerge Clang on my
> phone.
> Emerging Clang turned out not to be a breeze when I have a phone with 3
> gigs of RAM. So for about the longest time, I was busy trying to make up
> for it. I tried setting up distcc, increased swap/zram, but still the
> OOM killer got me.
> Later last night(/day), I came across the tips for building clang/gcc in
> an embedded Android environment at [3]. So I followed that and as of
> now, me writing the mail, OOM has not got me, Clang is emerging.
Did it finally go through?
> While I considering the patching part easy, I guess the major bottleneck
> in the process of building Android natively would be the compilation.
> Essentially I will have to trade time for memory. I will also give
> another shot to distcc while I am at this.
>
> Regardless of problems I faced, I made "TODO" comments in the source and
> wrote notes about where to patch the build scripts locally. I avidly
> wait for Clang to compile and then start testing the patched build
> script.
Yes, the compilers are the biggest burden of the building process. You
will be facing both the RAM and disk space shortages.
Luckily the HPC industry is testing out the arm64 architecture. We
could bet our luck with the more powerful arm64 servers.
Benda
> [1] https://github.com/WantGuns/manifest
> [2] https://github.com/WantGuns/toolchain_llvm_android
> [3] https://wiki.gentoo.org/wiki/Project:Android/tarball#Out_of_Memory
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-24 14:08 ` Benda Xu
@ 2020-07-24 15:02 ` Gunwant Jain
2020-07-24 17:07 ` UnderSampled
2020-07-25 3:00 ` Benda Xu
0 siblings, 2 replies; 49+ messages in thread
From: Gunwant Jain @ 2020-07-24 15:02 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1256 bytes --]
On 20/07/24 10:08PM, Benda Xu wrote:
> Gunwant Jain <[email protected]> writes:
>
> > This week, I started managing the structure for the modified LineageOS
> > fork we would keep. Namely the modified `repo-manifest` [1] and
> > `llvm_android` [2], for now. No patches have been applied to them yet
> > for the reasons discussed below.
> >
> > My plan for this week was to start with patching the `llvm_android`
> > build scripts. Android uses a prebuilt Clang (bundled with the repo)
> > among other stuff to build the toolchain. So I had to emerge Clang on my
> > phone.
> > Emerging Clang turned out not to be a breeze when I have a phone with 3
> > gigs of RAM. So for about the longest time, I was busy trying to make up
> > for it. I tried setting up distcc, increased swap/zram, but still the
> > OOM killer got me.
> > Later last night(/day), I came across the tips for building clang/gcc in
> > an embedded Android environment at [3]. So I followed that and as of
> > now, me writing the mail, OOM has not got me, Clang is emerging.
>
> Did it finally go through?
Yes it did.
I can also report that llvm_android's stage1 clang is compiled for
aarch64, I am on the stage2 step currently.
Regards,
Gunwant
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-24 15:02 ` Gunwant Jain
@ 2020-07-24 17:07 ` UnderSampled
2020-07-24 18:12 ` Gunwant Jain
2020-07-25 3:00 ` Benda Xu
1 sibling, 1 reply; 49+ messages in thread
From: UnderSampled @ 2020-07-24 17:07 UTC (permalink / raw
To: gentoo-soc, Gunwant Jain
[-- Attachment #1: Type: text/plain, Size: 1809 bytes --]
You may find this useful: https://glodroid.github.io/
This is a modern AOSP tree for OrangePi, RaspberryPi, and Pine64 devices. That includes the Pinebook and PinePhone: https://www.pine64.org/.
I can get you one of these devices if you think it could help.
We should definitely consider targeting a build of Shark Bait for the PinePhone.
On July 24, 2020 11:02:57 AM EDT, Gunwant Jain <[email protected]> wrote:
>On 20/07/24 10:08PM, Benda Xu wrote:
>> Gunwant Jain <[email protected]> writes:
>>
>> > This week, I started managing the structure for the modified
>LineageOS
>> > fork we would keep. Namely the modified `repo-manifest` [1] and
>> > `llvm_android` [2], for now. No patches have been applied to them
>yet
>> > for the reasons discussed below.
>> >
>> > My plan for this week was to start with patching the `llvm_android`
>> > build scripts. Android uses a prebuilt Clang (bundled with the
>repo)
>> > among other stuff to build the toolchain. So I had to emerge Clang
>on my
>> > phone.
>> > Emerging Clang turned out not to be a breeze when I have a phone
>with 3
>> > gigs of RAM. So for about the longest time, I was busy trying to
>make up
>> > for it. I tried setting up distcc, increased swap/zram, but still
>the
>> > OOM killer got me.
>> > Later last night(/day), I came across the tips for building
>clang/gcc in
>> > an embedded Android environment at [3]. So I followed that and as
>of
>> > now, me writing the mail, OOM has not got me, Clang is emerging.
>>
>> Did it finally go through?
>
>Yes it did.
>I can also report that llvm_android's stage1 clang is compiled for
>aarch64, I am on the stage2 step currently.
>
>
>Regards,
>Gunwant
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
[-- Attachment #2: Type: text/html, Size: 2450 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-24 17:07 ` UnderSampled
@ 2020-07-24 18:12 ` Gunwant Jain
0 siblings, 0 replies; 49+ messages in thread
From: Gunwant Jain @ 2020-07-24 18:12 UTC (permalink / raw
To: UnderSampled; +Cc: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 3323 bytes --]
Hey Stephen,
I took a brief look at GloDroid. It looks like just another AOSP tree but
with support for more devices like you mentioned.
But SharkBait aims to build Android on the device itself, the flavour of
Android should not matter. And for the same reasons, I chose Lineage's
tree because of the massive device support it has.
To implement SharkBait, we will have to swap the toolchains AOSP offers
with the ones supporting aarch64 host. Once that is sorted out, we could,
in theory, implement any flavour of AOSP, be it glodroid, or any other
fork, just by tracking SharkBait's toolchain sources.
On top of that SharkBait aims also to separate the AOSP modules and
provide updates via Portage. But I predict that this would be SharkBait
specific.
Targeting newer devices like the PinePhone would be like the normal way
AOSP forks like Lineage extend support to newer devices. I agree on a
PinePhone target.
Once again, I thank you for offering me help in getting such devices.
But I would politely ask for you to wait until I implement SharkBait on
a phone first (the Lineage sources for my phone are easily available).
Benda offered me an arm64 server for the project, I think that would be
enough to work on, once I succeed in building the toolchain (or fail to
do so because of hardware limitations).
Thanks,
Gunwant
On 20/07/24 01:07PM, UnderSampled wrote:
> You may find this useful: https://glodroid.github.io/
> This is a modern AOSP tree for OrangePi, RaspberryPi, and Pine64 devices. That includes the Pinebook and PinePhone: https://www.pine64.org/.
>
> I can get you one of these devices if you think it could help.
>
> We should definitely consider targeting a build of Shark Bait for the PinePhone.
>
>
> On July 24, 2020 11:02:57 AM EDT, Gunwant Jain <[email protected]> wrote:
> >On 20/07/24 10:08PM, Benda Xu wrote:
> >> Gunwant Jain <[email protected]> writes:
> >>
> >> > This week, I started managing the structure for the modified
> >LineageOS
> >> > fork we would keep. Namely the modified `repo-manifest` [1] and
> >> > `llvm_android` [2], for now. No patches have been applied to them
> >yet
> >> > for the reasons discussed below.
> >> >
> >> > My plan for this week was to start with patching the `llvm_android`
> >> > build scripts. Android uses a prebuilt Clang (bundled with the
> >repo)
> >> > among other stuff to build the toolchain. So I had to emerge Clang
> >on my
> >> > phone.
> >> > Emerging Clang turned out not to be a breeze when I have a phone
> >with 3
> >> > gigs of RAM. So for about the longest time, I was busy trying to
> >make up
> >> > for it. I tried setting up distcc, increased swap/zram, but still
> >the
> >> > OOM killer got me.
> >> > Later last night(/day), I came across the tips for building
> >clang/gcc in
> >> > an embedded Android environment at [3]. So I followed that and as
> >of
> >> > now, me writing the mail, OOM has not got me, Clang is emerging.
> >>
> >> Did it finally go through?
> >
> >Yes it did.
> >I can also report that llvm_android's stage1 clang is compiled for
> >aarch64, I am on the stage2 step currently.
> >
> >
> >Regards,
> >Gunwant
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-24 15:02 ` Gunwant Jain
2020-07-24 17:07 ` UnderSampled
@ 2020-07-25 3:00 ` Benda Xu
2020-07-25 8:02 ` Gunwant Jain
1 sibling, 1 reply; 49+ messages in thread
From: Benda Xu @ 2020-07-25 3:00 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 263 bytes --]
Gunwant Jain <[email protected]> writes:
> Yes it did.
> I can also report that llvm_android's stage1 clang is compiled for
> aarch64, I am on the stage2 step currently.
Good. Could you please remind me of the meanings of llvm stage1 and
stage2?
Benda
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-25 3:00 ` Benda Xu
@ 2020-07-25 8:02 ` Gunwant Jain
2020-07-25 8:21 ` Gunwant Jain
0 siblings, 1 reply; 49+ messages in thread
From: Gunwant Jain @ 2020-07-25 8:02 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1720 bytes --]
On 20/07/25 11:00AM, Benda Xu wrote:
> Gunwant Jain <[email protected]> writes:
>
> > Yes it did.
> > I can also report that llvm_android's stage1 clang is compiled for
> > aarch64, I am on the stage2 step currently.
>
> Good. Could you please remind me of the meanings of llvm stage1 and
> stage2?
Sure. AOSP provides its own distribution of LLVM. So most of what stage1
and stage2 I will be describing could be taken from [1].
Basically, we build LLVM stage1 using our own host toolchain. Then we
use the newly built stage1 compiler to build stage2, the ready to use
toolchain.
Coming to how this goes in our use-case, if we were to build the LLVM
toolchain using llvm_android (basically just build scripts), we will use
a prebuilt Clang toolchain, bundled with llvm_android to build stage1.
Also AOSP targets only X86 architechture in stage1.
After stage1 is built, the build script proceeds to start building
stage2. This is where we use the newly built stage1 compilers to build
stage2 compilers. AOSP targets AArch64, ARM, X86 and BPF when building
stage2. Then the build script packages stage2 install and dumps the tar
for our use.
In the cycle of AOSP development, this newly packaged stage2 will be
provided as the prebuilt LLVM toolchain in [2]. This repository is used:
- In the AOSP tree, when someone builds a new custom ROM.
- In llvm_android as the prebuilt toolchain used to build a version
bumped stage1.
For us, we will have to target AArch64 in building both stage1 and
stage2.
Regards,
Gunwant
[1] https://llvm.org/docs/BuildingADistribution.html
[2] https://android.googlesource.com/platform/prebuilts/clang/host/linux-x86/
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-25 8:02 ` Gunwant Jain
@ 2020-07-25 8:21 ` Gunwant Jain
0 siblings, 0 replies; 49+ messages in thread
From: Gunwant Jain @ 2020-07-25 8:21 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 2328 bytes --]
I am sorry to add on like this, but if it was not mentioned, I am not
using any prebuilt repository AOSP bundled with llvm_android. This is
the reason why I spent the time emerging Clang on my phone; for using it
to build stage1.
Although if advised otherwise, I could potentially break the AOSP
development cycle for our use and start the same cycle but with prebuilt
LLVM toolchain for AArch64 host instead of X86. I could expand on how to
do this, if you would like.
On 20/07/25 01:33PM, Gunwant Jain wrote:
> On 20/07/25 11:00AM, Benda Xu wrote:
> > Gunwant Jain <[email protected]> writes:
> >
> > > Yes it did.
> > > I can also report that llvm_android's stage1 clang is compiled for
> > > aarch64, I am on the stage2 step currently.
> >
> > Good. Could you please remind me of the meanings of llvm stage1 and
> > stage2?
>
> Sure. AOSP provides its own distribution of LLVM. So most of what stage1
> and stage2 I will be describing could be taken from [1].
> Basically, we build LLVM stage1 using our own host toolchain. Then we
> use the newly built stage1 compiler to build stage2, the ready to use
> toolchain.
>
> Coming to how this goes in our use-case, if we were to build the LLVM
> toolchain using llvm_android (basically just build scripts), we will use
> a prebuilt Clang toolchain, bundled with llvm_android to build stage1.
> Also AOSP targets only X86 architechture in stage1.
> After stage1 is built, the build script proceeds to start building
> stage2. This is where we use the newly built stage1 compilers to build
> stage2 compilers. AOSP targets AArch64, ARM, X86 and BPF when building
> stage2. Then the build script packages stage2 install and dumps the tar
> for our use.
>
> In the cycle of AOSP development, this newly packaged stage2 will be
> provided as the prebuilt LLVM toolchain in [2]. This repository is used:
> - In the AOSP tree, when someone builds a new custom ROM.
> - In llvm_android as the prebuilt toolchain used to build a version
> bumped stage1.
>
> For us, we will have to target AArch64 in building both stage1 and
> stage2.
>
>
> Regards,
> Gunwant
>
> [1] https://llvm.org/docs/BuildingADistribution.html
> [2] https://android.googlesource.com/platform/prebuilts/clang/host/linux-x86/
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* [gentoo-soc] Weekly Report: Portage Powered Android
@ 2020-08-17 18:05 Gunwant Jain
2020-08-17 23:15 ` Benda Xu
0 siblings, 1 reply; 49+ messages in thread
From: Gunwant Jain @ 2020-08-17 18:05 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 818 bytes --]
Hello everyone,
As I planned in my previous report, this week I was working on the
llvm_android build script. I can finally report that it works and we can
build AOSP's llvm for AArch64 Host now[1].
I wasn't able to handle the prebuilt GCC dependency libraries the
aforementioned LLVM toolchain requires. This would take longer time to
deal with because AOSP's build scripts[2] are all over the place.
And so I prioritize building GCC below the main goal of compiling
Android on AArch64 for the time being.
Coming to which, I have started testing building Lineage on AArch64
today. I believe this should take another week of efforts, if not more.
Regards,
Gunwant
[1] https://github.com/WantGuns/toolchain_llvm_android
[2] https://android.googlesource.com/toolchain/gcc/+/refs/heads/master
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-08-17 18:05 Gunwant Jain
@ 2020-08-17 23:15 ` Benda Xu
2020-08-20 6:21 ` Gunwant Jain
0 siblings, 1 reply; 49+ messages in thread
From: Benda Xu @ 2020-08-17 23:15 UTC (permalink / raw
To: gentoo-soc
Hi Gunwant,
Gunwant Jain <[email protected]> writes:
> As I planned in my previous report, this week I was working on the
> llvm_android build script. I can finally report that it works and we can
> build AOSP's llvm for AArch64 Host now[1].
>
> I wasn't able to handle the prebuilt GCC dependency libraries the
> aforementioned LLVM toolchain requires. This would take longer time to
> deal with because AOSP's build scripts[2] are all over the place.
> And so I prioritize building GCC below the main goal of compiling
> Android on AArch64 for the time being.
No problem. It looks like the prebuilt GCC dependency libraries serves
as a seed for LLVM. Is that correct?
> Coming to which, I have started testing building Lineage on AArch64
> today. I believe this should take another week of efforts, if not more.
Good.
Cheers,
Benda
^ permalink raw reply [flat|nested] 49+ messages in thread
* [gentoo-soc] Weekly Report: Portage Powered Android
@ 2020-08-12 1:41 Gunwant Jain
2020-08-12 7:34 ` EBo
0 siblings, 1 reply; 49+ messages in thread
From: Gunwant Jain @ 2020-08-12 1:41 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1409 bytes --]
Hi all,
This and the previous week, I have been working on building AOSP's LLVM
toolchain for aarch64. tldr is that I have succeeded in doing so, but I
have yet to commit my progress at the (yet to be) SharkBait manifest[1].
Really there are just a couple of things to work on before we are ready
to build Android on arm64:
1. The build process is a bit immature. I will commit the polished
changes in a day or two at the LLVM source[2].
2. While trying to minimize dependencies that AOSP bundles for building
LLVM, I noticed that building LLVM requires using libraries from a
prebuilt gcc[3]. This is unwanted because we aim not to use any of the
prebuilt dependencies that AOSP ships.
Therefore this week I am working upon building AOSP's gcc[4] for
aarch64.
Once the GCC dependency is sorted, I will start testing building Lineage
on aarch64. I have set a personal goal to atleast start these tests by
this week, since we do not have much time left.
Finally, I would like to thank my mentor, Heroxbd for providing the
arm64 server, due to which I could pick up the pace and build AOSP's
LLVM on aarch64.
Regards,
Gunwant
[1] https://github.com/WantGuns/manifest/tree/lineage-17.1
[2] https://github.com/WantGuns/toolchain_llvm_android
[3] https://github.com/WantGuns/manifest/blob/llvm-r353983/default.xml#L39
[4] https://android.googlesource.com/toolchain/gcc
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 484 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-08-12 1:41 Gunwant Jain
@ 2020-08-12 7:34 ` EBo
2020-08-14 7:26 ` Gunwant Jain
0 siblings, 1 reply; 49+ messages in thread
From: EBo @ 2020-08-12 7:34 UTC (permalink / raw
To: gentoo-soc
I thought I would also mention that recently I learned about the Proxy
Maintainers program[1]. It might be another venue to support the
efforts. As mentioned before, if this is of interest you should take it
up with your mentor and others.
EBo --
[1] https://wiki.gentoo.org/wiki/Project:Proxy_Maintainers
On Aug 11 2020 7:41 PM, Gunwant Jain wrote:
> Hi all,
>
> This and the previous week, I have been working on building AOSP's
> LLVM
> toolchain for aarch64. tldr is that I have succeeded in doing so, but
> I
> have yet to commit my progress at the (yet to be) SharkBait
> manifest[1].
>
> Really there are just a couple of things to work on before we are
> ready
> to build Android on arm64:
>
> 1. The build process is a bit immature. I will commit the polished
> changes in a day or two at the LLVM source[2].
>
> 2. While trying to minimize dependencies that AOSP bundles for
> building
> LLVM, I noticed that building LLVM requires using libraries from a
> prebuilt gcc[3]. This is unwanted because we aim not to use any of
> the
> prebuilt dependencies that AOSP ships.
> Therefore this week I am working upon building AOSP's gcc[4] for
> aarch64.
>
> Once the GCC dependency is sorted, I will start testing building
> Lineage
> on aarch64. I have set a personal goal to atleast start these tests
> by
> this week, since we do not have much time left.
>
> Finally, I would like to thank my mentor, Heroxbd for providing the
> arm64 server, due to which I could pick up the pace and build AOSP's
> LLVM on aarch64.
>
>
> Regards,
> Gunwant
>
> [1] https://github.com/WantGuns/manifest/tree/lineage-17.1
> [2] https://github.com/WantGuns/toolchain_llvm_android
> [3]
> https://github.com/WantGuns/manifest/blob/llvm-r353983/default.xml#L39
> [4] https://android.googlesource.com/toolchain/gcc
^ permalink raw reply [flat|nested] 49+ messages in thread
* [gentoo-soc] Weekly Report: Portage Powered Android
@ 2020-07-30 15:00 Gunwant Jain
0 siblings, 0 replies; 49+ messages in thread
From: Gunwant Jain @ 2020-07-30 15:00 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1667 bytes --]
Hello everyone,
I am sending my weekly report this late because I figured that I had
already discussed my findings this week on the mailing list. But my
mentor asked me to write it nevertheless.
My task was to build AOSP's LLVM toolchain, but for AArch64 host.
Patching the build scripts for that was the easy part. But running the
script over and over became tedious at some point. Many times, my sshfs
caused problems with the CMakeCache, essentially forcing me to clean
build. Other times, the OOM killer got me.
Nevertheless I can report that llvm's stage1 builds for the AArch64 host
and CMake is accepting the stage1 compilers to build the final llvm
stage2. I have yet to fully build llvm stage2, but I am sure that my
Mentor's kind offering of an arm64 server would check off that task.
I have already explained llvm stage1 and stage2, in some depth, in the
previous weekly report's reply chain. I'll put a brief explanation here
anyways.
Bascially, AOSP packages its own LLVM distribution. The idiomatic way to
build LLVM in this case would be to build a "stage 1" compiler with your
host toolchain and then build the "stage 2" compiler using the "stage 1"
compiler. In AOSP's case, that stage2 compiler is used :
- in the AOSP tree as the prebuilt toolchain, used to build AOSP.
- as the stage1 compiler to build a version bumped LLVM toolchain
(later used as the prebuilt toolchain in the AOSP tree).
Therefore my next task would be to use that llvm toolchain (built for
AArch64 host) in the AOSP tree and build Android on arm64 unlike the
conventional cross-compiling method.
Regards,
Gunwant
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread* [gentoo-soc] Weekly Report: Portage Powered Android
@ 2020-07-13 23:09 Gunwant Jain
2020-07-14 0:17 ` Benda Xu
2020-07-14 4:05 ` Pengcheng Xu
0 siblings, 2 replies; 49+ messages in thread
From: Gunwant Jain @ 2020-07-13 23:09 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 634 bytes --]
Hello Everyone,
Like I mentioned in my previous off-timed weekly report, I had been
working on studying Android's Build Process this week.
I have published a blog [1] regarding the same (most of it is just
confirmed workflows). I also laid a plan on how to approach building
Android on aarch64 host.
The next two weeks will be focused on that. I will be setting up an ARM
based VPS. Once we are through with natively compiling Android, I plan
to separate LLVM/Clang toolchain along with Bionic from the AOSP
codebase, Bionic here being the tricky part.
Regards,
Gunwant
[1] https://wantguns.gitlab.io/blog/android/build_android/
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-13 23:09 Gunwant Jain
@ 2020-07-14 0:17 ` Benda Xu
2020-07-14 5:01 ` Gunwant Jain
2020-07-14 4:05 ` Pengcheng Xu
1 sibling, 1 reply; 49+ messages in thread
From: Benda Xu @ 2020-07-14 0:17 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 725 bytes --]
Gunwant Jain <[email protected]> writes:
> Like I mentioned in my previous off-timed weekly report, I had been
> working on studying Android's Build Process this week.
>
> I have published a blog [1] regarding the same (most of it is just
> confirmed workflows). I also laid a plan on how to approach building
> Android on aarch64 host.
That's an excellent write-up.
> The next two weeks will be focused on that. I will be setting up an
> ARM based VPS.
What kind of VPS are you planning to set up?
> Once we are through with natively compiling Android, I plan to
> separate LLVM/Clang toolchain along with Bionic from the AOSP
> codebase, Bionic here being the tricky part.
Good.
Benda
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-14 0:17 ` Benda Xu
@ 2020-07-14 5:01 ` Gunwant Jain
2020-07-15 1:04 ` Benda Xu
0 siblings, 1 reply; 49+ messages in thread
From: Gunwant Jain @ 2020-07-14 5:01 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1141 bytes --]
On 20/07/14 08:17AM, Benda Xu wrote:
> Gunwant Jain <[email protected]> writes:
>
> > Like I mentioned in my previous off-timed weekly report, I had been
> > working on studying Android's Build Process this week.
> >
> > I have published a blog [1] regarding the same (most of it is just
> > confirmed workflows). I also laid a plan on how to approach building
> > Android on aarch64 host.
>
> That's an excellent write-up.
Thanks !
> > The next two weeks will be focused on that. I will be setting up an
> > ARM based VPS.
>
> What kind of VPS are you planning to set up?
I don't know actually, I was looking for AWS's EC2 A1
https://aws.amazon.com/ec2/instance-types/a1/
but then I realised that the credits which we get from Github's Student
Pack are for _AWS Educate_. AWS does offer a free tier variant, but it
only provides credits worth 30$ at the moment.
> > Once we are through with natively compiling Android, I plan to
> > separate LLVM/Clang toolchain along with Bionic from the AOSP
> > codebase, Bionic here being the tricky part.
>
> Good.
>
> Benda
Regards,
Gunwant
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-14 5:01 ` Gunwant Jain
@ 2020-07-15 1:04 ` Benda Xu
2020-07-15 1:09 ` UnderSampled
` (2 more replies)
0 siblings, 3 replies; 49+ messages in thread
From: Benda Xu @ 2020-07-15 1:04 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 735 bytes --]
Gunwant Jain <[email protected]> writes:
>> > The next two weeks will be focused on that. I will be setting up an
>> > ARM based VPS.
>>
>> What kind of VPS are you planning to set up?
>
> I don't know actually, I was looking for AWS's EC2 A1
> https://aws.amazon.com/ec2/instance-types/a1/
> but then I realised that the credits which we get from Github's Student
> Pack are for _AWS Educate_. AWS does offer a free tier variant, but it
> only provides credits worth 30$ at the moment.
AWS free/education tier will be less powerful than your phone. Please
go ahead and try but don't bat much on it.
Does your phone have an SD card slot? Is it possible to set up NFS or
iSCSI on you phone?
Benda
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-15 1:04 ` Benda Xu
@ 2020-07-15 1:09 ` UnderSampled
2020-07-15 4:55 ` Gunwant Jain
2020-07-15 4:45 ` Pengcheng Xu
2020-07-15 4:52 ` Gunwant Jain
2 siblings, 1 reply; 49+ messages in thread
From: UnderSampled @ 2020-07-15 1:09 UTC (permalink / raw
To: gentoo-soc, Benda Xu
[-- Attachment #1: Type: text/plain, Size: 951 bytes --]
Perhaps a Raspberry Pi would be helpful?
On July 14, 2020 9:04:49 PM EDT, Benda Xu <[email protected]> wrote:
>Gunwant Jain <[email protected]> writes:
>
>>> > The next two weeks will be focused on that. I will be setting up
>an
>>> > ARM based VPS.
>>>
>>> What kind of VPS are you planning to set up?
>>
>> I don't know actually, I was looking for AWS's EC2 A1
>> https://aws.amazon.com/ec2/instance-types/a1/
>> but then I realised that the credits which we get from Github's
>Student
>> Pack are for _AWS Educate_. AWS does offer a free tier variant, but
>it
>> only provides credits worth 30$ at the moment.
>
>AWS free/education tier will be less powerful than your phone. Please
>go ahead and try but don't bat much on it.
>
>Does your phone have an SD card slot? Is it possible to set up NFS or
>iSCSI on you phone?
>
>Benda
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
[-- Attachment #2: Type: text/html, Size: 1723 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* RE: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-15 1:04 ` Benda Xu
2020-07-15 1:09 ` UnderSampled
@ 2020-07-15 4:45 ` Pengcheng Xu
2020-07-15 4:58 ` Gunwant Jain
2020-07-15 4:52 ` Gunwant Jain
2 siblings, 1 reply; 49+ messages in thread
From: Pengcheng Xu @ 2020-07-15 4:45 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1245 bytes --]
I think NFS should be feasible to get storage on your phone; I've done that back in 2018 for my Nexus 6P. Just remember to use robust Wi-Fi, or the NFS client will hang and mess things up quickly.
Regards,
--
Pengcheng Xu
https://jsteward.moe
> -----Original Message-----
> From: Benda Xu <[email protected]>
> Sent: Wednesday, July 15, 2020 9:05 AM
> To: [email protected]
> Subject: Re: [gentoo-soc] Weekly Report: Portage Powered Android
>
> Gunwant Jain <[email protected]> writes:
>
> >> > The next two weeks will be focused on that. I will be setting up an
> >> > ARM based VPS.
> >>
> >> What kind of VPS are you planning to set up?
> >
> > I don't know actually, I was looking for AWS's EC2 A1
> > https://aws.amazon.com/ec2/instance-types/a1/
> > but then I realised that the credits which we get from Github's Student
> > Pack are for _AWS Educate_. AWS does offer a free tier variant, but it
> > only provides credits worth 30$ at the moment.
>
> AWS free/education tier will be less powerful than your phone. Please
> go ahead and try but don't bat much on it.
>
> Does your phone have an SD card slot? Is it possible to set up NFS or
> iSCSI on you phone?
>
> Benda
[-- Attachment #2: openpgp-digital-signature.asc --]
[-- Type: application/pgp-signature, Size: 834 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-15 1:04 ` Benda Xu
2020-07-15 1:09 ` UnderSampled
2020-07-15 4:45 ` Pengcheng Xu
@ 2020-07-15 4:52 ` Gunwant Jain
2 siblings, 0 replies; 49+ messages in thread
From: Gunwant Jain @ 2020-07-15 4:52 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1282 bytes --]
On 20/07/15 09:04AM, Benda Xu wrote:
> Gunwant Jain <[email protected]> writes:
>
> >> > The next two weeks will be focused on that. I will be setting up an
> >> > ARM based VPS.
> >>
> >> What kind of VPS are you planning to set up?
> >
> > I don't know actually, I was looking for AWS's EC2 A1
> > https://aws.amazon.com/ec2/instance-types/a1/
> > but then I realised that the credits which we get from Github's Student
> > Pack are for _AWS Educate_. AWS does offer a free tier variant, but it
> > only provides credits worth 30$ at the moment.
>
> AWS free/education tier will be less powerful than your phone. Please
> go ahead and try but don't bat much on it.
Okay.
> Does your phone have an SD card slot? Is it possible to set up NFS or
> iSCSI on you phone?
Yes, my phone officially supports expandable storage upto 256 gigs. That
should suffice building Lineage. I am looking into the NFS based
solution first though.
One more thing I should point out is that building Android, atleast
Android Q, generally requires atleast 16 gigs of RAM. I have encountered
people trying to circumvent this with ZRAM. My phone only has 3 gigs of
RAM, so I will have to do the same. Let's see how this fairs out.
Regards,
Gunwant
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* RE: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-13 23:09 Gunwant Jain
2020-07-14 0:17 ` Benda Xu
@ 2020-07-14 4:05 ` Pengcheng Xu
2020-07-14 5:05 ` Gunwant Jain
1 sibling, 1 reply; 49+ messages in thread
From: Pengcheng Xu @ 2020-07-14 4:05 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1362 bytes --]
Outlook just crashed while sending the reply, so bear with me if you see multiple copies of this; I'll try to find something more robust once I have the time.
> -----Original Message-----
> From: Gunwant Jain <[email protected]>
> Sent: Tuesday, July 14, 2020 7:09 AM
> To: [email protected]
> Subject: [gentoo-soc] Weekly Report: Portage Powered Android
>
> Hello Everyone,
>
> Like I mentioned in my previous off-timed weekly report, I had been
> working on studying Android's Build Process this week.
>
> I have published a blog [1] regarding the same (most of it is just
> confirmed workflows). I also laid a plan on how to approach building
> Android on aarch64 host.
>
> The next two weeks will be focused on that. I will be setting up an ARM
> based VPS. Once we are through with natively compiling Android, I plan
> to separate LLVM/Clang toolchain along with Bionic from the AOSP
> codebase, Bionic here being the tricky part.
Just to confirm, you're trying to get a ARM machine for something that's more commodious and faster to work with, right? Otherwise you could just use the phone itself...
>
> Regards,
> Gunwant
>
> [1] https://wantguns.gitlab.io/blog/android/build_android/
You're generally in good shape; keep up the awesome work!
Regards,
--
Pengcheng Xu
https://jsteward.moe
[-- Attachment #2: openpgp-digital-signature.asc --]
[-- Type: application/pgp-signature, Size: 834 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-14 4:05 ` Pengcheng Xu
@ 2020-07-14 5:05 ` Gunwant Jain
0 siblings, 0 replies; 49+ messages in thread
From: Gunwant Jain @ 2020-07-14 5:05 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 840 bytes --]
On 20/07/14 12:05PM, Pengcheng Xu wrote:
> Outlook just crashed while sending the reply, so bear with me if you see multiple copies of this; I'll try to find something more robust once I have the time.
> > The next two weeks will be focused on that. I will be setting up an ARM
> > based VPS. Once we are through with natively compiling Android, I plan
> > to separate LLVM/Clang toolchain along with Bionic from the AOSP
> > codebase, Bionic here being the tricky part.
>
> Just to confirm, you're trying to get a ARM machine for something that's more commodious and faster to work with, right? Otherwise you could just use the phone itself...
Yes, exactly. The only breaking factor is the storage I have in my
phone, which is not even capabale of syncing the llvm_android repo
(around 15+ gigs due to dependencies).
Regards,
Gunwant
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* [gentoo-soc] Weekly Report: Portage Powered Android
@ 2020-07-08 11:39 Gunwant Jain
2020-07-08 13:53 ` Pengcheng Xu
2020-07-09 1:04 ` Benda Xu
0 siblings, 2 replies; 49+ messages in thread
From: Gunwant Jain @ 2020-07-08 11:39 UTC (permalink / raw
To: gentoo-soc; +Cc: heroxbd
[-- Attachment #1: Type: text/plain, Size: 1391 bytes --]
Hey Everyone !
I was about to give a coherent report for what I have been doing for 2
weeks next week, but my mentor suggested to explain my findings briefly.
I found out that AOSP has actually been phasing out of using GCC in
favour to Clang/LLVM based toolchains. And so, from the last time I
gave in my weekly report, I have been checking up on llvm_android [1],
the source of the toolchain which AOSP uses to build itself.
I built the toolchain for x86 host using the original scripts AOSP
provides and found out that they in terms of Bionic (Android's libc),
they only copy Bionic's headers, compiles Clang for different targets
using a prebuilt toolchain bundled with the entire package.
Later, when we build Android, the prebuilt toolchain compiles Bionic and
uses it as intermediatories to compile the other bulk of Android
(frameworks, etc.).
My main objective I think would be to introduce a aarch64 host config
for llvm_android, and integrate the toolchain in the AOSP build process.
Take this with a grain of salt, as I believe I should put more research
into this first, which is why I was taking another week.
I would probably write a detailed blog documenting AOSP's build process
by the end of this working week, covering everything we need to proceed.
Regards,
Gunwant
[1] https://android.googlesource.com/toolchain/llvm_android/
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* RE: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-08 11:39 Gunwant Jain
@ 2020-07-08 13:53 ` Pengcheng Xu
2020-07-09 1:04 ` Benda Xu
1 sibling, 0 replies; 49+ messages in thread
From: Pengcheng Xu @ 2020-07-08 13:53 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1855 bytes --]
Nice progress. Looking forward to your elaboration on the LLVM/Clang toolchain for aarch64 host!
Regards,
--
Pengcheng Xu
https://jsteward.moe
> -----Original Message-----
> From: Gunwant Jain <[email protected]>
> Sent: Wednesday, July 8, 2020 7:39 PM
> To: [email protected]
> Cc: [email protected]
> Subject: [gentoo-soc] Weekly Report: Portage Powered Android
>
> Hey Everyone !
>
> I was about to give a coherent report for what I have been doing for 2
> weeks next week, but my mentor suggested to explain my findings briefly.
>
> I found out that AOSP has actually been phasing out of using GCC in
> favour to Clang/LLVM based toolchains. And so, from the last time I
> gave in my weekly report, I have been checking up on llvm_android [1],
> the source of the toolchain which AOSP uses to build itself.
>
> I built the toolchain for x86 host using the original scripts AOSP
> provides and found out that they in terms of Bionic (Android's libc),
> they only copy Bionic's headers, compiles Clang for different targets
> using a prebuilt toolchain bundled with the entire package.
>
> Later, when we build Android, the prebuilt toolchain compiles Bionic and
> uses it as intermediatories to compile the other bulk of Android
> (frameworks, etc.).
>
> My main objective I think would be to introduce a aarch64 host config
> for llvm_android, and integrate the toolchain in the AOSP build process.
> Take this with a grain of salt, as I believe I should put more research
> into this first, which is why I was taking another week.
>
> I would probably write a detailed blog documenting AOSP's build process
> by the end of this working week, covering everything we need to proceed.
>
>
> Regards,
> Gunwant
>
> [1] https://android.googlesource.com/toolchain/llvm_android/
[-- Attachment #2: openpgp-digital-signature.asc --]
[-- Type: application/pgp-signature, Size: 834 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-08 11:39 Gunwant Jain
2020-07-08 13:53 ` Pengcheng Xu
@ 2020-07-09 1:04 ` Benda Xu
1 sibling, 0 replies; 49+ messages in thread
From: Benda Xu @ 2020-07-09 1:04 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1541 bytes --]
Hi Gunwant,
Gunwant Jain <[email protected]> writes:
> I was about to give a coherent report for what I have been doing for 2
> weeks next week, but my mentor suggested to explain my findings
> briefly.
>
> I found out that AOSP has actually been phasing out of using GCC in
> favour to Clang/LLVM based toolchains. And so, from the last time I
> gave in my weekly report, I have been checking up on llvm_android [1],
> the source of the toolchain which AOSP uses to build itself.
>
> I built the toolchain for x86 host using the original scripts AOSP
> provides and found out that they in terms of Bionic (Android's libc),
> they only copy Bionic's headers, compiles Clang for different targets
> using a prebuilt toolchain bundled with the entire package.
>
> Later, when we build Android, the prebuilt toolchain compiles Bionic and
> uses it as intermediatories to compile the other bulk of Android
> (frameworks, etc.).
>
> My main objective I think would be to introduce a aarch64 host config
> for llvm_android, and integrate the toolchain in the AOSP build process.
> Take this with a grain of salt, as I believe I should put more research
> into this first, which is why I was taking another week.
>
> I would probably write a detailed blog documenting AOSP's build process
> by the end of this working week, covering everything we need to proceed.
Good.
Please take a look into https://wiki.debian.org/AndroidTools, Debian
might have achieved some of the goals of us.
Yours,
Benda
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* [gentoo-soc] Weekly Report: Portage Powered Android
@ 2020-06-29 20:12 Gunwant Jain
2020-06-29 21:27 ` EBo
` (2 more replies)
0 siblings, 3 replies; 49+ messages in thread
From: Gunwant Jain @ 2020-06-29 20:12 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 959 bytes --]
Hello Everyone,
This week, I had relatively less work done.
I spent most of my time perusing the LFS guide and osdev.org learning
about building toolchains. Then I went onto building the
aarch64-linux-android toolchain as described in [1]. I updated the
kernel sources for my phone [2]. And in general tidied up my kernel
repository.
But while building the toolchain, I was not able to compile `gcc` as
I was riddled with errors along the way. Most of them are related to the
newer `gcc` (version 9.x) finding incompatibility in the sources of the
`gcc` I am trying to build (version 4.9).
I will continue to build the toolchain and hopefully move onto
separating `Bionic` from the other side of Android mid-week.
Thanks,
Gunwant
[1] https://wiki.gentoo.org/wiki/Android/SharkBait/Building_a_toolchain_for_aarch64-linux-android
[2] https://github.com/WantGuns/overlay/blob/master/sys-kernel/linux-headers/linux-headers-4.4.222.ebuild
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-06-29 20:12 Gunwant Jain
@ 2020-06-29 21:27 ` EBo
2020-06-30 21:42 ` Gunwant Jain
2020-07-01 7:17 ` Benda Xu
2020-07-01 14:07 ` Pengcheng Xu
2 siblings, 1 reply; 49+ messages in thread
From: EBo @ 2020-06-29 21:27 UTC (permalink / raw
To: gentoo-soc
Getting everything set up to move forward is sometimes the hardest
thing... You will have to discuss this with your mentor, but a few
posts in some appropriate indexed forum would be very useful for others.
Also, if you can document what steps you went through to make it work.
Speaking for myself, once I get something like this to work I start off
with a clean dir/disk and follow my notes step by step so that I know I
did not miss anything when writing it all down.
Anyway, like I said, talk to your mentor and see what is reasonable to
add along these lines, but I would love to see this documented.
Keep up the good work, and best of success.
On Jun 29 2020 2:12 PM, Gunwant Jain wrote:
> Hello Everyone,
>
> This week, I had relatively less work done.
> I spent most of my time perusing the LFS guide and osdev.org learning
> about building toolchains. Then I went onto building the
> aarch64-linux-android toolchain as described in [1]. I updated the
> kernel sources for my phone [2]. And in general tidied up my kernel
> repository.
> But while building the toolchain, I was not able to compile `gcc` as
> I was riddled with errors along the way. Most of them are related to
> the
> newer `gcc` (version 9.x) finding incompatibility in the sources of
> the
> `gcc` I am trying to build (version 4.9).
> I will continue to build the toolchain and hopefully move onto
> separating `Bionic` from the other side of Android mid-week.
>
> Thanks,
> Gunwant
>
> [1]
>
> https://wiki.gentoo.org/wiki/Android/SharkBait/Building_a_toolchain_for_aarch64-linux-android
> [2]
>
> https://github.com/WantGuns/overlay/blob/master/sys-kernel/linux-headers/linux-headers-4.4.222.ebuild
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-06-29 21:27 ` EBo
@ 2020-06-30 21:42 ` Gunwant Jain
2020-07-01 2:37 ` EBo
2020-07-01 12:40 ` Lucas Ramage
0 siblings, 2 replies; 49+ messages in thread
From: Gunwant Jain @ 2020-06-30 21:42 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1091 bytes --]
Thanks for your concern Ebo.
On 20/06/29 03:27PM, EBo wrote:
> Getting everything set up to move forward is sometimes the hardest thing...
> You will have to discuss this with your mentor, but a few posts in some
> appropriate indexed forum would be very useful for others. Also, if you can
> document what steps you went through to make it work. Speaking for myself,
> once I get something like this to work I start off with a clean dir/disk and
> follow my notes step by step so that I know I did not miss anything when
> writing it all down.
I am not quite sure what you mean by using a forum for posts, if it is
regarding having a public announcement of sorts, I plan to do it when we
finish implemententing the basic idea of SharkBait.
I agree that documenting will play a crucial role in my journey. And for
that I had set up a personal blog earlier [1]. I am actively writing new
blogs, thoroughly explaining whatever I do from scratch.
I am also open to any suggestions to a better medium you might have for
the documentation.
Regards,
Gunwant
[1] https://wantguns.gitlab.io/
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-06-30 21:42 ` Gunwant Jain
@ 2020-07-01 2:37 ` EBo
2020-07-01 3:15 ` Gunwant Jain
2020-07-01 12:40 ` Lucas Ramage
1 sibling, 1 reply; 49+ messages in thread
From: EBo @ 2020-07-01 2:37 UTC (permalink / raw
To: gentoo-soc
On Jun 30 2020 3:42 PM, Gunwant Jain wrote:
> On 20/06/29 03:27PM, EBo wrote:
>> Getting everything set up to move forward is sometimes the hardest
>> thing...
>> You will have to discuss this with your mentor, but a few posts in
>> some
>> appropriate indexed forum would be very useful for others. Also, if
>> you can
>> document what steps you went through to make it work. Speaking for
>> myself,
>> once I get something like this to work I start off with a clean
>> dir/disk and
>> follow my notes step by step so that I know I did not miss anything
>> when
>> writing it all down.
>
> I am not quite sure what you mean by using a forum for posts, if it
> is
> regarding having a public announcement of sorts, I plan to do it when
> we
> finish implemententing the basic idea of SharkBait.
> I agree that documenting will play a crucial role in my journey. And
> for
> that I had set up a personal blog earlier [1]. I am actively writing
> new
> blogs, thoroughly explaining whatever I do from scratch.
There are many places that people post questions/answers. I know of
one group that almost exclusively use StackOverflow. Others various
gentoo forums (see: https://forums.gentoo.org/). It is really up to you
and your preferences. That said it needs to be indexed and searchable.
Can you guarantee that your personal blog be around for the next decade
(a reasonable life expectancy for such a project)? I am simply
suggesting posting overviews/FAQ's in venues that we can expect to have
around for many years and be searchable by google, etc.
> I am also open to any suggestions to a better medium you might have
> for
> the documentation.
I have to be very careful here. Even moderately decent documentation
is a LOT of work, and I know of people literally writing documents full
time for very good pay. I could make some suggestions, and it could
take so much time and derail other efforts that would be your mentor's
priority. I do not want to cause any issues by interjecting comments
from time to time. Also, it will be months or more before I will have
enough breakout time to give this a real shot for install/use.
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-01 2:37 ` EBo
@ 2020-07-01 3:15 ` Gunwant Jain
0 siblings, 0 replies; 49+ messages in thread
From: Gunwant Jain @ 2020-07-01 3:15 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1762 bytes --]
On 20/06/30 08:37PM, EBo wrote:
> There are many places that people post questions/answers. I know of one
> group that almost exclusively use StackOverflow. Others various gentoo
> forums (see: https://forums.gentoo.org/). It is really up to you and your
> preferences. That said it needs to be indexed and searchable. Can you
> guarantee that your personal blog be around for the next decade (a
> reasonable life expectancy for such a project)? I am simply suggesting
> posting overviews/FAQ's in venues that we can expect to have around for many
> years and be searchable by google, etc.
I can assure you that my blogs are not going anywhere anytime soon.
<I am avidly thinking of going on a complete self-hosting journey once
I am done with GSoC>
Besides that, I even plan to use those blogs later for a better
sustained medium like the Gentoo wiki. If necessary, we could also have
some community interaction like you suggested on the Gentoo forums.
> I have to be very careful here. Even moderately decent documentation is a
> LOT of work, and I know of people literally writing documents full time for
> very good pay. I could make some suggestions, and it could take so much
> time and derail other efforts that would be your mentor's priority. I do
> not want to cause any issues by interjecting comments from time to time.
> Also, it will be months or more before I will have enough breakout time to
> give this a real shot for install/use.
I think I understand your concern, and let me reassure you that I am not
getting done with this project after GSoC ends. That being said, I think
it would be best if we focus on the community work once I get over with
my GSoC obligations.
I'll be happy to respond to more queries.
Regards,
Gunwant
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-06-30 21:42 ` Gunwant Jain
2020-07-01 2:37 ` EBo
@ 2020-07-01 12:40 ` Lucas Ramage
1 sibling, 0 replies; 49+ messages in thread
From: Lucas Ramage @ 2020-07-01 12:40 UTC (permalink / raw
To: gentoo-soc
For GSoC, it is typical for the student to write a personal blog.
However, we also have a SharkbaitOS website, as well as the Gentoo Wiki to address these concerns.
Once you have everything sorted for your implementation, then we can look at getting this content where it needs to go.
On June 30, 2020 9:42:43 PM UTC, Gunwant Jain <[email protected]> wrote:
>Thanks for your concern Ebo.
>
>On 20/06/29 03:27PM, EBo wrote:
>> Getting everything set up to move forward is sometimes the hardest
>thing...
>> You will have to discuss this with your mentor, but a few posts in
>some
>> appropriate indexed forum would be very useful for others. Also, if
>you can
>> document what steps you went through to make it work. Speaking for
>myself,
>> once I get something like this to work I start off with a clean
>dir/disk and
>> follow my notes step by step so that I know I did not miss anything
>when
>> writing it all down.
>
>I am not quite sure what you mean by using a forum for posts, if it is
>regarding having a public announcement of sorts, I plan to do it when
>we
>finish implemententing the basic idea of SharkBait.
>I agree that documenting will play a crucial role in my journey. And
>for
>that I had set up a personal blog earlier [1]. I am actively writing
>new
>blogs, thoroughly explaining whatever I do from scratch.
>
>I am also open to any suggestions to a better medium you might have for
>the documentation.
>
>Regards,
>Gunwant
>
>
>[1] https://wantguns.gitlab.io/
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-06-29 20:12 Gunwant Jain
2020-06-29 21:27 ` EBo
@ 2020-07-01 7:17 ` Benda Xu
2020-07-01 16:12 ` Gunwant Jain
2020-07-01 14:07 ` Pengcheng Xu
2 siblings, 1 reply; 49+ messages in thread
From: Benda Xu @ 2020-07-01 7:17 UTC (permalink / raw
To: Gunwant Jain; +Cc: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1335 bytes --]
Gunwant Jain <[email protected]> writes:
> This week, I had relatively less work done.
> I spent most of my time perusing the LFS guide and osdev.org learning
> about building toolchains. Then I went onto building the
> aarch64-linux-android toolchain as described in [1]. I updated the
> kernel sources for my phone [2]. And in general tidied up my kernel
> repository.
So the newly compile kernel works. Well done.
> But while building the toolchain, I was not able to compile `gcc` as I
> was riddled with errors along the way. Most of them are related to the
> newer `gcc` (version 9.x) finding incompatibility in the sources of
> the `gcc` I am trying to build (version 4.9). I will continue to
> build the toolchain and hopefully move onto separating `Bionic` from
> the other side of Android mid-week.
Why do you need to build gcc-4.9?
>
> [1]
> https://wiki.gentoo.org/wiki/Android/SharkBait/Building_a_toolchain_for_aarch64-linux-android
> [2]
> https://github.com/WantGuns/overlay/blob/master/sys-kernel/linux-headers/linux-headers-4.4.222.ebuild
Does the version number "4.4.222" coresspond to the kernel of your
phone? Generally, when compiling a toolchain, you don't need to match
the linux-header version to the running kernel. The newest linux-header
will do.
Benda
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-01 7:17 ` Benda Xu
@ 2020-07-01 16:12 ` Gunwant Jain
0 siblings, 0 replies; 49+ messages in thread
From: Gunwant Jain @ 2020-07-01 16:12 UTC (permalink / raw
To: Benda Xu; +Cc: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1115 bytes --]
On 20/07/01 03:17PM, Benda Xu wrote:
> > But while building the toolchain, I was not able to compile `gcc` as I
> > was riddled with errors along the way. Most of them are related to the
> > newer `gcc` (version 9.x) finding incompatibility in the sources of
> > the `gcc` I am trying to build (version 4.9). I will continue to
> > build the toolchain and hopefully move onto separating `Bionic` from
> > the other side of Android mid-week.
>
> Why do you need to build gcc-4.9?
I assumed we need to build the gcc which is used by the Android platform
(and as an extension NDK as well). The latest gcc which they ship is
version 4.9 [1].
> Does the version number "4.4.222" coresspond to the kernel of your
> phone?
Yes.
> Generally, when compiling a toolchain, you don't need to match
> the linux-header version to the running kernel. The newest linux-header
> will do.
I was not aware of that, thanks. But then again, I don't see why I
shouldn't continue using the "4.4.222" headers.
Regards,
Gunwant
[1] https://android.googlesource.com/toolchain/gcc/+/refs/heads/master
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* RE: [gentoo-soc] Weekly Report: Portage Powered Android
2020-06-29 20:12 Gunwant Jain
2020-06-29 21:27 ` EBo
2020-07-01 7:17 ` Benda Xu
@ 2020-07-01 14:07 ` Pengcheng Xu
2020-07-01 16:53 ` Gunwant Jain
2 siblings, 1 reply; 49+ messages in thread
From: Pengcheng Xu @ 2020-07-01 14:07 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 3309 bytes --]
Hi Gunwant,
Glad to see your progress regarding the toolchain. Stupid Microsoft Outlook won't do inline replies correctly, so bear with me if you see the following folded in your email client. And yes, I'm aware that the line breaks are completely messed up (i.e. super-long lines). I've planned to fix it some time later, just don't have much time to spare recently...
Anyway, find my comments below.
> -----Original Message-----
> From: Gunwant Jain <[email protected]>
> Sent: Tuesday, June 30, 2020 4:12 AM
> To: [email protected]
> Subject: [gentoo-soc] Weekly Report: Portage Powered Android
>
> Hello Everyone,
>
> This week, I had relatively less work done.
> I spent most of my time perusing the LFS guide and osdev.org learning
> about building toolchains. Then I went onto building the
> aarch64-linux-android toolchain as described in [1]. I updated the
> kernel sources for my phone [2]. And in general tidied up my kernel
> repository.
It's good that you chose to get familiar with the GNU build system (autotools); it will save you large amounts of time when meddling with toolchain-related work. I would also suggest taking a look at the "GNU Hello" project; it serves as a genuine example of the basics for a GNU project, which GCC is as well. Get familiar with how to read the configuration and compilation logs, especially when they're littered with libtool calls instead of directly calling the compiler.
> But while building the toolchain, I was not able to compile `gcc` as
> I was riddled with errors along the way. Most of them are related to the
> newer `gcc` (version 9.x) finding incompatibility in the sources of the
> `gcc` I am trying to build (version 4.9).
> I will continue to build the toolchain and hopefully move onto
> separating `Bionic` from the other side of Android mid-week.
Just as Benda pointed out, 4.9 may be the one I chose two years ago, but it's not a hard requirement: most Android things should compile fine with any newer versions of GCC as long as the toolchain is correctly configured for an Android target; in many occasions even Clang works perfectly fine. You may try to fix the problems in the 4.9 tree, tweak the options for the "build" compiler (gcc-9) to accept older, non-compliant sources, or try to find a newer version of GCC source tree with Android (i.e. Bionic) support. You may have better luck looking for such a source tree in the AOSP codebase.
>
> Thanks,
> Gunwant
>
> [1]
> https://wiki.gentoo.org/wiki/Android/SharkBait/Building_a_toolchain_for_aa
> rch64-linux-android
> [2]
> https://github.com/WantGuns/overlay/blob/master/sys-kernel/linux-headers/l
> inux-headers-4.4.222.ebuild
And, we'd love to see a link to your weekly writeup of your efforts and problems encountered in greater detail. We can find out more about your progress and perhaps provide assistance if we can see the exact problem; for example, it would be helpful if you quoted or linked to the exact error message in the gcc build process you just mentioned. It's also a good idea to keep these in track for archival purposes, such that future contributors can have an idea of what's going on if they run into the same problem.
Regards,
--
Pengcheng Xu
https://jsteward.moe
[-- Attachment #2: openpgp-digital-signature.asc --]
[-- Type: application/pgp-signature, Size: 834 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-01 14:07 ` Pengcheng Xu
@ 2020-07-01 16:53 ` Gunwant Jain
2020-07-02 3:50 ` UnderSampled
0 siblings, 1 reply; 49+ messages in thread
From: Gunwant Jain @ 2020-07-01 16:53 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 2900 bytes --]
Thanks for your comments Pengcheng.
On 20/07/01 10:07PM, Pengcheng Xu wrote:
> It's good that you chose to get familiar with the GNU build system (autotools); it will save you large amounts of time when meddling with toolchain-related work. I would also suggest taking a look at the "GNU Hello" project; it serves as a genuine example of the basics for a GNU project, which GCC is as well. Get familiar with how to read the configuration and compilation logs, especially when they're littered with libtool calls instead of directly calling the compiler.
Thanks for the tip, I will definitely look into it.
> > But while building the toolchain, I was not able to compile `gcc` as
> > I was riddled with errors along the way. Most of them are related to the
> > newer `gcc` (version 9.x) finding incompatibility in the sources of the
> > `gcc` I am trying to build (version 4.9).
> > I will continue to build the toolchain and hopefully move onto
> > separating `Bionic` from the other side of Android mid-week.
>
> Just as Benda pointed out, 4.9 may be the one I chose two years ago, but it's not a hard requirement: most Android things should compile fine with any newer versions of GCC as long as the toolchain is correctly configured for an Android target; in many occasions even Clang works perfectly fine. You may try to fix the problems in the 4.9 tree, tweak the options for the "build" compiler (gcc-9) to accept older, non-compliant sources, or try to find a newer version of GCC source tree with Android (i.e. Bionic) support. You may have better luck looking for such a source tree in the AOSP codebase.
I agree, and I just realised that even though NDK requires gcc-4.9,
Android 8.0 and above uses Clang for compilation. This differs from my
days of compiling custom ROMs (pre Android 8). I am sorry for the
confusion created with gcc, I should have known better.
Let me switch to building a clang based toolchain now onwards.
> And, we'd love to see a link to your weekly writeup of your efforts and problems encountered in greater detail. We can find out more about your progress and perhaps provide assistance if we can see the exact problem; for example, it would be helpful if you quoted or linked to the exact error message in the gcc build process you just mentioned. It's also a good idea to keep these in track for archival purposes, such that future contributors can have an idea of what's going on if they run into the same problem.
I wasn't planning on keeping the logs of errors I face, but since it is
requested, I will provide them in a new segment of "weekly blogs" now
onwards (keeping in mind that I have switched to using clang based
toolchains). Furthermore, I will be actively indicating the errors in
the IRC chatroom, which is logged as well.
Please let me know if you have more comments.
Regards,
Gunwant
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-07-01 16:53 ` Gunwant Jain
@ 2020-07-02 3:50 ` UnderSampled
2020-07-02 12:14 ` Gunwant Jain
0 siblings, 1 reply; 49+ messages in thread
From: UnderSampled @ 2020-07-02 3:50 UTC (permalink / raw
To: gentoo-soc, Gunwant Jain
[-- Attachment #1: Type: text/plain, Size: 3284 bytes --]
Where is the IRC logged? I generally only catch up on my phone, and haven't been able to check the IRC after the Gitter bridge failed.
Stephen
On July 1, 2020 12:53:54 PM EDT, Gunwant Jain <[email protected]> wrote:
>Thanks for your comments Pengcheng.
>
>On 20/07/01 10:07PM, Pengcheng Xu wrote:
>> It's good that you chose to get familiar with the GNU build system
>(autotools); it will save you large amounts of time when meddling with
>toolchain-related work. I would also suggest taking a look at the "GNU
>Hello" project; it serves as a genuine example of the basics for a GNU
>project, which GCC is as well. Get familiar with how to read the
>configuration and compilation logs, especially when they're littered
>with libtool calls instead of directly calling the compiler.
>
>Thanks for the tip, I will definitely look into it.
>
>> > But while building the toolchain, I was not able to compile `gcc`
>as
>> > I was riddled with errors along the way. Most of them are related
>to the
>> > newer `gcc` (version 9.x) finding incompatibility in the sources of
>the
>> > `gcc` I am trying to build (version 4.9).
>> > I will continue to build the toolchain and hopefully move onto
>> > separating `Bionic` from the other side of Android mid-week.
>>
>> Just as Benda pointed out, 4.9 may be the one I chose two years ago,
>but it's not a hard requirement: most Android things should compile
>fine with any newer versions of GCC as long as the toolchain is
>correctly configured for an Android target; in many occasions even
>Clang works perfectly fine. You may try to fix the problems in the 4.9
>tree, tweak the options for the "build" compiler (gcc-9) to accept
>older, non-compliant sources, or try to find a newer version of GCC
>source tree with Android (i.e. Bionic) support. You may have better
>luck looking for such a source tree in the AOSP codebase.
>
>I agree, and I just realised that even though NDK requires gcc-4.9,
>Android 8.0 and above uses Clang for compilation. This differs from my
>days of compiling custom ROMs (pre Android 8). I am sorry for the
>confusion created with gcc, I should have known better.
>
>Let me switch to building a clang based toolchain now onwards.
>
>> And, we'd love to see a link to your weekly writeup of your efforts
>and problems encountered in greater detail. We can find out more about
>your progress and perhaps provide assistance if we can see the exact
>problem; for example, it would be helpful if you quoted or linked to
>the exact error message in the gcc build process you just mentioned.
>It's also a good idea to keep these in track for archival purposes,
>such that future contributors can have an idea of what's going on if
>they run into the same problem.
>
>I wasn't planning on keeping the logs of errors I face, but since it is
>requested, I will provide them in a new segment of "weekly blogs" now
>onwards (keeping in mind that I have switched to using clang based
>toolchains). Furthermore, I will be actively indicating the errors in
>the IRC chatroom, which is logged as well.
>
>Please let me know if you have more comments.
>
>Regards,
>Gunwant
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
[-- Attachment #2: Type: text/html, Size: 4126 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* [gentoo-soc] Weekly Report: Portage Powered Android
@ 2020-06-22 19:15 Gunwant Jain
2020-06-23 4:45 ` Pengcheng Xu
2020-07-01 7:14 ` Benda Xu
0 siblings, 2 replies; 49+ messages in thread
From: Gunwant Jain @ 2020-06-22 19:15 UTC (permalink / raw
To: gentoo-soc; +Cc: heroxbd
[-- Attachment #1: Type: text/plain, Size: 1751 bytes --]
Hello Everyone,
I was supposed to integrate `bootstrap-init` and `sar-preinit` into
SharkBait this week. I couldn't quite do that.
`bootstrap-init` was initially cross-compiled for aarch64 using NDK. So
the first job was to get rid of NDK and use a different "build-system".
I opted for CMake and ported the project to use it [1]. But the binary
compiled by CMake on my phone (arm64 Gentoo), would not boot Android.
The build rules can be inspected on [1].
I believe this is because of the fact that I am not using a Bionic based
toolchain. So, I started working on building such a toolchain mid-week.
And this is where my current task starts coinciding with my next
scheduled task of providing a aarch64-linux-android toolchain where we
build Bionic as well. The crux of this task was already performed by
KireinaHoro in his GSoC [2]. Since my next scheduled task is required to
be finished for my current task, I propose to merge `bootstrap-init`
along with `sar-preinit` together later in the SharkBait workflow.
And to cover the installing process for users right now, I will write a
blog and post it on [3] within 24 hours of writing this mail. The blog
will provide a guide on assembling the components required for
System-As-Root based Android devices and installing SharkBaitOS. Keep in
mind that this process will indeed be simplified for the user later.
And I propose to edit SharkBait's official wiki later in my GSoC tenure
explaining the simplified procedure to end-users and porters.
Thanks,
Gunwant
[1] https://gitlab.com/WantGuns/bootstrap-init/-/tree/cmake
[2] https://wiki.gentoo.org/wiki/Android/SharkBait/Building_a_toolchain_for_aarch64-linux-android
[3] https://wantguns.gitlab.io/
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* RE: [gentoo-soc] Weekly Report: Portage Powered Android
2020-06-22 19:15 Gunwant Jain
@ 2020-06-23 4:45 ` Pengcheng Xu
2020-06-23 11:44 ` Gunwant Jain
2020-07-01 7:14 ` Benda Xu
1 sibling, 1 reply; 49+ messages in thread
From: Pengcheng Xu @ 2020-06-23 4:45 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 3027 bytes --]
Sorry for not getting back for some time; I had some other affairs that kept my hands busy. I have some comments for this week's report.
> -----Original Message-----
> From: Gunwant Jain <[email protected]>
> Sent: Tuesday, June 23, 2020 3:16 AM
> To: [email protected]
> Cc: [email protected]
> Subject: [gentoo-soc] Weekly Report: Portage Powered Android
>
> Hello Everyone,
>
> I was supposed to integrate `bootstrap-init` and `sar-preinit` into SharkBait
> this week. I couldn't quite do that.
> `bootstrap-init` was initially cross-compiled for aarch64 using NDK. So the
> first job was to get rid of NDK and use a different "build-system".
> I opted for CMake and ported the project to use it [1]. But the binary compiled
> by CMake on my phone (arm64 Gentoo), would not boot Android.
> The build rules can be inspected on [1].
Have you figured out _why_ Android is not booting? I can think of a bunch of reasons, such as missing dependencies for a dynamically-linked executable, crashing on execution, etc. Simply saying "would not boot Android" does not convey any information.
Besides, I don't think you need NDK for something as low-level as an init; you shouldn’t need one as long as the code does not use Android-specific functions or call into Android service (e.g. vold). You should be able to get away with something standard, i.e. newlib/musl/glibc. I haven't read the source code though, and I might be wrong here; do your survey.
>
> I believe this is because of the fact that I am not using a Bionic based toolchain.
> So, I started working on building such a toolchain mid-week.
> And this is where my current task starts coinciding with my next scheduled task
> of providing a aarch64-linux-android toolchain where we build Bionic as well.
> The crux of this task was already performed by KireinaHoro in his GSoC [2].
> Since my next scheduled task is required to be finished for my current task,
> I propose to merge `bootstrap-init` along with `sar-preinit` together later
> in the SharkBait workflow.
You may try with bionic toolchain, but just as I said, so long as no Android-specific APIs are involved, libc should not be the problem.
>
> And to cover the installing process for users right now, I will write a blog
> and post it on [3] within 24 hours of writing this mail. The blog will provide
> a guide on assembling the components required for System-As-Root based Android
> devices and installing SharkBaitOS. Keep in mind that this process will indeed
> be simplified for the user later.
> And I propose to edit SharkBait's official wiki later in my GSoC tenure explaining
> the simplified procedure to end-users and porters.
>
> Thanks,
> Gunwant
>
> [1] https://gitlab.com/WantGuns/bootstrap-init/-/tree/cmake
> [2]
> https://wiki.gentoo.org/wiki/Android/SharkBait/Building_a_toolchain_for_aa
> rch64-linux-android
> [3] https://wantguns.gitlab.io/
Regards,
--
Pengcheng Xu
https://jsteward.moe
[-- Attachment #2: openpgp-digital-signature.asc --]
[-- Type: application/pgp-signature, Size: 834 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-06-23 4:45 ` Pengcheng Xu
@ 2020-06-23 11:44 ` Gunwant Jain
0 siblings, 0 replies; 49+ messages in thread
From: Gunwant Jain @ 2020-06-23 11:44 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1912 bytes --]
Thanks for your query.
On 20/06/23 12:45PM, Pengcheng Xu wrote:
> Have you figured out _why_ Android is not booting? I can think of a bunch of reasons, such as missing dependencies for a dynamically-linked executable, crashing on execution, etc. Simply saying "would not boot Android" does not convey any information.
The binary exits after execution in the container. Also, it is a static
binary.
> Besides, I don't think you need NDK for something as low-level as an init; you shouldn’t need one as long as the code does not use Android-specific functions or call into Android service (e.g. vold). You should be able to get away with something standard, i.e. newlib/musl/glibc. I haven't read the source code though, and I might be wrong here; do your survey.
I had the same thoughts/arguments regarding using NDK for init. Also as
far as I can tell, it does not depend any Android-specific functions
either. But even then, using Gentoo's libc does not produce a working
binary. Couple of things to keep in mind regarding the NDK setup:
- It uses clang instead of gcc
- Uses a different sysroot, which is provided/bundled in NDK itself
(it is bundled according to the API level).
- It uses a different STL (again, bundled with NDK).
> You may try with bionic toolchain, but just as I said, so long as no Android-specific APIs are involved, libc should not be the problem.
I would argue the same actually, but using Bionic is the closest we can
get to have a NDK like environment without actually using NDK.
If required, I could also have this week devoted entirely in rewriting
`bootstrap-init`, but this would be counter-productive since the same
code-base produces working binaries using a different build-system.
I am open to any new advices on this topic.
Regards,
Gunwant
>
> Regards,
> --
> Pengcheng Xu
> https://jsteward.moe
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-06-22 19:15 Gunwant Jain
2020-06-23 4:45 ` Pengcheng Xu
@ 2020-07-01 7:14 ` Benda Xu
1 sibling, 0 replies; 49+ messages in thread
From: Benda Xu @ 2020-07-01 7:14 UTC (permalink / raw
To: Gunwant Jain; +Cc: gentoo-soc, heroxbd
[-- Attachment #1: Type: text/plain, Size: 654 bytes --]
Hi Gunwant,
Gunwant Jain <[email protected]> writes:
> I was supposed to integrate `bootstrap-init` and `sar-preinit` into
> SharkBait this week. I couldn't quite do that. `bootstrap-init` was
> initially cross-compiled for aarch64 using NDK. So the first job was
> to get rid of NDK and use a different "build-system". I opted for
> CMake and ported the project to use it [1]. But the binary compiled by
> CMake on my phone (arm64 Gentoo), would not boot Android. The build
> rules can be inspected on [1].
Could you please remind me why we need a bootstrap-init? Can we just
exec the Android init when the lxc-guest starts?
Yours,
Benda
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* [gentoo-soc] Weekly Report: Portage Powered Android
@ 2020-06-15 8:03 Gunwant Jain
2020-07-01 7:10 ` Benda Xu
0 siblings, 1 reply; 49+ messages in thread
From: Gunwant Jain @ 2020-06-15 8:03 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 744 bytes --]
Hello everyone,
Most of my work in this week revolved around bringing a `bootstrap-init`
[1] for, as the name suggests, sets up the environment for
System-as-Root Android inside the LXC container and finally exec's the
Android init.
What this essentially means is that now we can use SharkBait on SAR
devices (Android 9+).
The process of setting SharkBait up for such devices is a bit different
and I am currently in discussion with my mentors for possible design
flows in order to make it easier for Porters and Users.
I will be integrating all my work in SharkBait and quickly transition
into working on Android's build systems mid-week, in this week.
Thanks,
Gunwant
[1] https://gitlab.com/WantGuns/bootstrap-init
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* [gentoo-soc] Weekly Report: Portage Powered Android
@ 2020-06-09 11:39 Gunwant Jain
2020-07-01 7:08 ` Benda Xu
0 siblings, 1 reply; 49+ messages in thread
From: Gunwant Jain @ 2020-06-09 11:39 UTC (permalink / raw
To: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1396 bytes --]
Hello,
This week, I had quite some progress on my GSoC project.
The task scheduled was to provide a `preinit` which could boot Gentoo in System-As-Root enabled phones.
I completed that for good using the `magiskboot` tool, by hexpatching the kernel for such devices [1].
I faced some issues after patching the kernel as well, resolving them without the access of a serial console was inconvenient but achieved nevertheless. I have written extensive blogs, explaining the entire journey of doing it [2]. Overall, the process of using preinit is now streamlined and simple.
Apart from this, I have written a blog on the boot-mechanisms different versions of Android have. My blogs would be the primary source of watching my activity on GSoC, apart from the irc of-course.
The next task scheduled for week 1-2 was to boot Android in a container in the Gentoo land. I am pleased to inform that we have achieved that goal as well. It is in barebones presently, but everything (RIL, camera, etc.) works inside the Android container.
For the upcoming week, I will be writing a new `init` for bringing support to almost every boot-mechanism Google has experimented with. This would also help the project in being very scalable because we would have a single point of maintainence.
Regards,
Gunwant Jain
[1] https://gitlab.com/WantGuns/sar-preinit
[2] https://wantguns.gitlab.io
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: [gentoo-soc] Weekly Report: Portage Powered Android
2020-06-09 11:39 Gunwant Jain
@ 2020-07-01 7:08 ` Benda Xu
0 siblings, 0 replies; 49+ messages in thread
From: Benda Xu @ 2020-07-01 7:08 UTC (permalink / raw
To: Gunwant Jain; +Cc: gentoo-soc
[-- Attachment #1: Type: text/plain, Size: 1584 bytes --]
Hi Gunwant,
Gunwant Jain <[email protected]> writes:
> This week, I had quite some progress on my GSoC project. The task
> scheduled was to provide a `preinit` which could boot Gentoo in
> System-As-Root enabled phones. I completed that for good using the
> `magiskboot` tool, by hexpatching the kernel for such devices [1]. I
> faced some issues after patching the kernel as well, resolving them
> without the access of a serial console was inconvenient but achieved
> nevertheless. I have written extensive blogs, explaining the entire
> journey of doing it [2]. Overall, the process of using preinit is now
> streamlined and simple.
Very good to see that you have forward ported sharkbait to bleeding edge
Android versions. That's a major achievement. Well done!
> Apart from this, I have written a blog on the boot-mechanisms
> different versions of Android have. My blogs would be the primary
> source of watching my activity on GSoC, apart from the irc of-course.
>
> The next task scheduled for week 1-2 was to boot Android in a
> container in the Gentoo land. I am pleased to inform that we have
> achieved that goal as well. It is in barebones presently, but
> everything (RIL, camera, etc.) works inside the Android container.
> For the upcoming week, I will be writing a new `init` for bringing
> support to almost every boot-mechanism Google has experimented
> with. This would also help the project in being very scalable because
> we would have a single point of maintainence.
Good, please keep us updated on the status of the new `init`.
Yours,
Benda
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread
end of thread, other threads:[~2020-08-20 6:21 UTC | newest]
Thread overview: 49+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-07-20 7:40 [gentoo-soc] Weekly Report: Portage Powered Android Gunwant Jain
2020-07-24 14:08 ` Benda Xu
2020-07-24 15:02 ` Gunwant Jain
2020-07-24 17:07 ` UnderSampled
2020-07-24 18:12 ` Gunwant Jain
2020-07-25 3:00 ` Benda Xu
2020-07-25 8:02 ` Gunwant Jain
2020-07-25 8:21 ` Gunwant Jain
-- strict thread matches above, loose matches on Subject: below --
2020-08-17 18:05 Gunwant Jain
2020-08-17 23:15 ` Benda Xu
2020-08-20 6:21 ` Gunwant Jain
2020-08-12 1:41 Gunwant Jain
2020-08-12 7:34 ` EBo
2020-08-14 7:26 ` Gunwant Jain
2020-07-30 15:00 Gunwant Jain
2020-07-13 23:09 Gunwant Jain
2020-07-14 0:17 ` Benda Xu
2020-07-14 5:01 ` Gunwant Jain
2020-07-15 1:04 ` Benda Xu
2020-07-15 1:09 ` UnderSampled
2020-07-15 4:55 ` Gunwant Jain
2020-07-15 4:45 ` Pengcheng Xu
2020-07-15 4:58 ` Gunwant Jain
2020-07-15 4:52 ` Gunwant Jain
2020-07-14 4:05 ` Pengcheng Xu
2020-07-14 5:05 ` Gunwant Jain
2020-07-08 11:39 Gunwant Jain
2020-07-08 13:53 ` Pengcheng Xu
2020-07-09 1:04 ` Benda Xu
2020-06-29 20:12 Gunwant Jain
2020-06-29 21:27 ` EBo
2020-06-30 21:42 ` Gunwant Jain
2020-07-01 2:37 ` EBo
2020-07-01 3:15 ` Gunwant Jain
2020-07-01 12:40 ` Lucas Ramage
2020-07-01 7:17 ` Benda Xu
2020-07-01 16:12 ` Gunwant Jain
2020-07-01 14:07 ` Pengcheng Xu
2020-07-01 16:53 ` Gunwant Jain
2020-07-02 3:50 ` UnderSampled
2020-07-02 12:14 ` Gunwant Jain
2020-06-22 19:15 Gunwant Jain
2020-06-23 4:45 ` Pengcheng Xu
2020-06-23 11:44 ` Gunwant Jain
2020-07-01 7:14 ` Benda Xu
2020-06-15 8:03 Gunwant Jain
2020-07-01 7:10 ` Benda Xu
2020-06-09 11:39 Gunwant Jain
2020-07-01 7:08 ` Benda Xu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox