使用docker编译OpenHarmony e2000记录
0、环境准备
参考device_board_phytium/e2000/README.md · Phytium嵌入式软件/Phytium-OpenHarmony-device - Gitee.com
Windows11 安装Ubuntu20.04 amd64虚拟机。
虚拟机配置12核+10G内存。
*建议虚拟机新建一块硬盘，大小150G以上，把这个硬盘挂载在ubuntu系统目录下面。
否则硬盘容量太小编译过程中系统可能会爆掉。
我这里使用100G硬盘编译过程中提示空间不足，还删掉了repo的缓存目录一共14G左右，而且还是标准系统，如果是全量系统建议200G硬盘。
1、安装docker
卸载自带的docker，使用下面脚本编译。
wget -qO- https://get.docker.com/ | sh
rayu@rayu-ubuntu:/work/projects/docker-images$ sudo apt-get remove docker.io
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
bridge-utils containerd pigz runc ubuntu-fan
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
docker.io
0 upgraded, 0 newly installed, 1 to remove and 28 not upgraded.
After this operation, 113 MB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 204962 files and directories currently installed.)
Removing docker.io (24.0.5-0ubuntu1~20.04.1) ...
'/usr/share/docker.io/contrib/nuke-graph-directory.sh' -> '/var/lib/docker/nuke-graph-directory.sh'
Processing triggers for man-db (2.9.1-1) ...
rayu@rayu-ubuntu:/work/projects/docker-images$ wget -qO- https://get.docker.com/ | sh
# Executing docker install script, commit: e5543d473431b782227f8908005543bb4389b8de
+ sudo -E sh -c 'apt-get update -qq >/dev/null'
+ sudo -E sh -c 'DEBIAN_FRONTEND=noninteractive apt-get install -y -qq apt-transport-https ca-certificates curl >/dev/null'
+ sudo -E sh -c 'install -m 0755 -d /etc/apt/keyrings'
+ sudo -E sh -c 'curl -fsSL "https://download.docker.com/linux/ubuntu/gpg" | gpg --dearmor --yes -o /etc/apt/keyrings/docker.gpg'
gpg: WARNING: unsafe ownership on homedir '/home/rayu/.gnupg'
+ sudo -E sh -c 'chmod a+r /etc/apt/keyrings/docker.gpg'
+ sudo -E sh -c 'echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu focal stable" > /etc/apt/sources.list.d/docker.list'
+ sudo -E sh -c 'apt-get update -qq >/dev/null'
+ sudo -E sh -c 'DEBIAN_FRONTEND=noninteractive apt-get install -y -qq docker-ce docker-ce-cli containerd.io docker-compose-plugin docker-ce-rootless-extras docker-buildx-plugin >/dev/null'
+ sudo -E sh -c 'docker version'
Client: Docker Engine - Community
Version: 24.0.7
API version: 1.43
Go version: go1.20.10
Git commit: afdd53b
Built: Thu Oct 26 09:08:01 2023
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 24.0.7
API version: 1.43 (minimum version 1.12)
Go version: go1.20.10
Git commit: 311b9ff
Built: Thu Oct 26 09:08:01 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.26
GitCommit: 3dd1e886e55dd695541fdcd67420c2888645a495
runc:
Version: 1.1.10
GitCommit: v1.1.10-0-g18a0cb0
docker-init:
Version: 0.19.0
GitCommit: de40ad0
================================================================================
To run Docker as a non-privileged user, consider setting up the
Docker daemon in rootless mode for your user:
dockerd-rootless-setuptool.sh install
Visit https://docs.docker.com/go/rootless/ to learn about rootless mode.
To run the Docker daemon as a fully privileged service, but granting non-root
users access, refer to https://docs.docker.com/go/daemon-access/
WARNING: Access to the remote API on a privileged Docker daemon is equivalent
to root access on the host. Refer to the 'Docker daemon attack surface'
documentation for details: https://docs.docker.com/go/attack-surface/
================================================================================
安装完成后，非root用户不能直接输入docker images查看镜像，要加sudo，上面安装完成也输出了如何解决。解决方法如下：
使用dockerd-rootless-setuptool.sh install安装，这个sh是上一步安装docker的时候出现的。
rayu@rayu-ubuntu:/work/projects/docker-images$ ll
total 36
drwxr-xr-x 2 rayu rayu 4096 Dec 16 10:45 ./
drwxr-xr-x 13 rayu rayu 4096 Dec 16 10:35 ../
-rwxrwxr-x 1 rayu rayu 21927 Dec 16 10:45 1.sh*
-rwxr--r-- 1 rayu rayu 2781 Dec 16 10:34 Dockerfile*
rayu@rayu-ubuntu:/work/projects/docker-images$ dockerd-rootless-setuptool.sh install
[ERROR] Missing system requirements. Run the following commands to
[ERROR] install the requirements and run this tool again.
########## BEGIN ##########
sudo sh -eux <<EOF
# Install newuidmap & newgidmap binaries
apt-get install -y uidmap
EOF
########## END ##########
rayu@rayu-ubuntu:/work/projects/docker-images$ sudo apt-get install -y uidmap
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
bridge-utils ubuntu-fan
Use 'sudo apt autoremove' to remove them.
The following NEW packages will be installed:
uidmap
0 upgraded, 1 newly installed, 0 to remove and 28 not upgraded.
Need to get 26.4 kB of archives.
After this operation, 172 kB of additional disk space will be used.
Get:1 http://mirrors.aliyun.com/ubuntu focal-updates/universe amd64 uidmap amd64 1:4.8.1-1ubuntu5.20.04.4 [26.4 kB]
Fetched 26.4 kB in 0s (155 kB/s)
Selecting previously unselected package uidmap.
(Reading database ... 204939 files and directories currently installed.)
Preparing to unpack .../uidmap_1%3a4.8.1-1ubuntu5.20.04.4_amd64.deb ...
Unpacking uidmap (1:4.8.1-1ubuntu5.20.04.4) ...
Setting up uidmap (1:4.8.1-1ubuntu5.20.04.4) ...
Processing triggers for man-db (2.9.1-1) ...
rayu@rayu-ubuntu:/work/projects/docker-images$ dockerd-rootless-setuptool.sh install
[INFO] Creating /home/rayu/.config/systemd/user/docker.service
[INFO] starting systemd service docker.service
+ systemctl --user start docker.service
+ sleep 3
+ systemctl --user --no-pager --full status docker.service
● docker.service - Docker Application Container Engine (Rootless)
Loaded: loaded (/home/rayu/.config/systemd/user/docker.service; disabled; vendor preset: enabled)
Active: active (running) since Sat 2023-12-16 10:51:55 CST; 3s ago
Docs: https://docs.docker.com/go/rootless/
Main PID: 826246 (rootlesskit)
CGroup: /user.slice/user-1000.slice/user@1000.service/docker.service
├─826246 rootlesskit --net=slirp4netns --mtu=65520 --slirp4netns-sandbox=auto --slirp4netns-seccomp=auto --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/run --propagation=rslave /usr/bin/dockerd-rootless.sh
├─826257 /proc/self/exe --net=slirp4netns --mtu=65520 --slirp4netns-sandbox=auto --slirp4netns-seccomp=auto --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/run --propagation=rslave /usr/bin/dockerd-rootless.sh
├─826275 slirp4netns --mtu 65520 -r 3 --disable-host-loopback --enable-sandbox --enable-seccomp 826257 tap0
├─826282 dockerd
└─826302 containerd --config /run/user/1000/docker/containerd/containerd.toml
Dec 16 10:51:54 rayu-ubuntu dockerd-rootless.sh[826302]: time="2023-12-16T10:51:54.794366885+08:00" level=info msg="containerd successfully booted in 0.016137s"
Dec 16 10:51:54 rayu-ubuntu dockerd-rootless.sh[826282]: time="2023-12-16T10:51:54.866038197+08:00" level=info msg="Loading containers: start."
Dec 16 10:51:54 rayu-ubuntu dockerd-rootless.sh[826282]: time="2023-12-16T10:51:54.866262121+08:00" level=info msg="skipping firewalld management for rootless mode"
Dec 16 10:51:54 rayu-ubuntu dockerd-rootless.sh[826282]: time="2023-12-16T10:51:54.973219568+08:00" level=info msg="Loading containers: done."
Dec 16 10:51:55 rayu-ubuntu dockerd-rootless.sh[826282]: time="2023-12-16T10:51:55.011690682+08:00" level=warning msg="Not using native diff for overlay2, this may cause degraded performance for building images: running in a user namespace" storage-driver=overlay2
Dec 16 10:51:55 rayu-ubuntu dockerd-rootless.sh[826282]: time="2023-12-16T10:51:55.011861657+08:00" level=warning msg="WARNING: Running in rootless-mode without cgroups. To enable cgroups in rootless-mode, you need to boot the system in cgroup v2 mode."
Dec 16 10:51:55 rayu-ubuntu dockerd-rootless.sh[826282]: time="2023-12-16T10:51:55.011880961+08:00" level=info msg="Docker daemon" commit=311b9ff graphdriver=overlay2 version=24.0.7
Dec 16 10:51:55 rayu-ubuntu dockerd-rootless.sh[826282]: time="2023-12-16T10:51:55.012005473+08:00" level=info msg="Daemon has completed initialization"
Dec 16 10:51:55 rayu-ubuntu dockerd-rootless.sh[826282]: time="2023-12-16T10:51:55.042043697+08:00" level=info msg="API listen on /run/user/1000/docker.sock"
Dec 16 10:51:55 rayu-ubuntu systemd[1159]: Started Docker Application Container Engine (Rootless).
+ DOCKER_HOST=unix:///run/user/1000/docker.sock
+ /usr/bin/docker version
Client: Docker Engine - Community
Version: 24.0.7
API version: 1.43
Go version: go1.20.10
Git commit: afdd53b
Built: Thu Oct 26 09:08:01 2023
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 24.0.7
API version: 1.43 (minimum version 1.12)
Go version: go1.20.10
Git commit: 311b9ff
Built: Thu Oct 26 09:08:01 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.26
GitCommit: 3dd1e886e55dd695541fdcd67420c2888645a495
runc:
Version: 1.1.10
GitCommit: v1.1.10-0-g18a0cb0
docker-init:
Version: 0.19.0
GitCommit: de40ad0
rootlesskit:
Version: 1.1.1
ApiVersion: 1.1.1
NetworkDriver: slirp4netns
PortDriver: builtin
StateDir: /tmp/rootlesskit1655015090
slirp4netns:
Version: 0.4.3
GitCommit: 2244b9b6461afeccad1678fac3d6e478c28b4ad6
+ systemctl --user enable docker.service
Created symlink /home/rayu/.config/systemd/user/default.target.wants/docker.service → /home/rayu/.config/systemd/user/docker.service.
[INFO] Installed docker.service successfully.
[INFO] To control docker.service, run: `systemctl --user (start|stop|restart) docker.service`
[INFO] To run docker.service on system startup, run: `sudo loginctl enable-linger rayu`
[INFO] Creating CLI context "rootless"
Successfully created context "rootless"
[INFO] Using CLI context "rootless"
Current context is now "rootless"
[INFO] Make sure the following environment variable(s) are set (or add them to ~/.bashrc):
export PATH=/usr/bin:$PATH
[INFO] Some applications may require the following environment variable too:
export DOCKER_HOST=unix:///run/user/1000/docker.sock
rayu@rayu-ubuntu:/work/projects/docker-images$ ll
total 36
drwxr-xr-x 2 rayu rayu 4096 Dec 16 10:45 ./
drwxr-xr-x 13 rayu rayu 4096 Dec 16 10:35 ../
-rwxrwxr-x 1 rayu rayu 21927 Dec 16 10:45 1.sh*
-rwxr--r-- 1 rayu rayu 2781 Dec 16 10:34 Dockerfile*
rayu@rayu-ubuntu:/work/projects/docker-images$
修改完测试不用sudo也可以查看镜像和容器了。
2、编译docker镜像
$ docker build -f Dockerfile -t rayuuu/embedded-dev:v1 .
镜像编译完成：
rayu@rayu-ubuntu:/work/projects/docker-images$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
rayuuu/embedded-dev v1 b75d2552a44d 38 minutes ago 4.2GB
ubuntu 20.04 83a4bf3bb050 2 weeks ago 72.8MB
3、创建docker容器
docker run -h ohos --name ohos -p 2022:22 --privileged=true -v /work1/OpenHarmony/:/home/user/src \
-it rayuuu/embedded-dev:latest bash
如果进入容器没有权限，检查一下环境变量。实在不行切换为root用户，或者创建容器的时候添加-uroot，即：
docker run -h ohos --name ohos -p 2022:22 --privileged=true -uroot -v /work1/OpenHarmony/:/home/user/src \
-it rayuuu/embedded-dev:latest bash
这里用之前在虚拟机下载完的源码直接调用编译命令，编译完成如下：
4、docker编译系统
操作命令记录：
#下载repo，如果docker image编译时候忘记了repo用下面命令
#sudo curl -s https://gitee.com/oschina/repo/raw/fork_flow/repo-py3 > /home/user/repo
#cp /home/user/repo /usr/local/bin/
#sudo chmod +x /usr/local/bin/repo
#创建文件夹
cd /home/user/src
#替换成自己定义的workspace路径
export WORK_SPACE=/home/user/src/4.0
export PROJ_ROOT=$WORK_SPACE/4.0-release
mkdir -p /home/user/src/4.0/4.0-release
#dash -> bash dockerfile里面操作报错
sudo ln -sf /bin/bash /bin/sh
#git config
git config --global user.name "rayu"
git config --global user.email "rayu@rayuu.com"
git config --global credential.helper store
#下载openharmony源码 standard
cd $PROJ_ROOT
repo init -u https://gitee.com/openharmony/manifest -b refs/tags/OpenHarmony-v4.0-Release --no-repo-verify -g ohos:standard
repo sync -c
repo forall -c 'git lfs pull'
#预编译
cd $PROJ_ROOT
bash build/prebuilts_download.sh
#patch
export PHY_DEV=$WORK_SPACE/phytium_device
mkdir -p /home/user/src/4.0/phytium_device
cd $PHY_DEV
git clone https://gitee.com/phytium_embedded/phytium-openharmony-device.git
cd phytium-openharmony-device
./phytium_env.sh $PROJ_ROOT 1
#编译
cd $PROJ_ROOT
./build.sh --product-name e2000 --ccache --target-cpu arm64
测试docker使用普通用户编译也能成功，对应的文件夹给权限和环境变量配置正确即可。
5、镜像推送
docker镜像测试编译没问题后，推送镜像到hub.docker.com。
rayu@rayu-ubuntu:~$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
rayuuu/embedded-dev latest b3de5382d281 2 days ago 4.23GB
ubuntu 20.04 83a4bf3bb050 2 weeks ago 72.8MB
rayu@rayu-ubuntu:~$ docker push rayuuu/embedded-dev:latest
The push refers to repository [docker.io/rayuuu/embedded-dev]
74dbe038662c: Pushed
5f70bf18a086: Layer already exists
9458c9ede548: Pushed
8b4a1884050b: Pushed
5ff3cfc2dcbe: Pushed
2f53e5429b35: Pushed
2cd03a585ace: Pushed
9fe78d0e72d2: Layer already exists
fb7485264019: Layer already exists
d3fa9d362c05: Layer already exists
latest: digest: sha256:26baa65d95205fbe30f645e24959fd1024c9b37a104a8270d02d3672503dd960 size: 2411
rayu@rayu-ubuntu:~$ docker iamges
docker: 'iamges' is not a docker command.
See 'docker --help'
rayu@rayu-ubuntu:~$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
rayuuu/embedded-dev latest b3de5382d281 2 days ago 4.23GB
ubuntu 20.04 83a4bf3bb050 2 weeks ago 72.8MB
rayu@rayu-ubuntu:~$ docker tag rayuuu/embedded-dev:latest rayuuu/embedded-dev:v1
rayu@rayu-ubuntu:~$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
rayuuu/embedded-dev latest b3de5382d281 2 days ago 4.23GB
rayuuu/embedded-dev v1 b3de5382d281 2 days ago 4.23GB
ubuntu 20.04 83a4bf3bb050 2 weeks ago 72.8MB
rayu@rayu-ubuntu:~$ docker push rayuuu/embedded-dev:v1
The push refers to repository [docker.io/rayuuu/embedded-dev]
74dbe038662c: Layer already exists
5f70bf18a086: Layer already exists
9458c9ede548: Layer already exists
8b4a1884050b: Layer already exists
5ff3cfc2dcbe: Layer already exists
2f53e5429b35: Layer already exists
2cd03a585ace: Layer already exists
9fe78d0e72d2: Layer already exists
fb7485264019: Layer already exists
d3fa9d362c05: Layer already exists
v1: digest: sha256:26baa65d95205fbe30f645e24959fd1024c9b37a104a8270d02d3672503dd960 size: 2411
rayu@rayu-ubuntu:~$
dockerhub可以看到已经推送成功，压缩的docker大小1.21GB。
6、镜像拉取
后续想直接编译鸿蒙，可以直接docker pull rayuuu/embedded-dev:latest。
镜像拉取完成：
创建容器命令：
docker run -h ohos --name ohos -p 2022:22 --privileged=true -v /work1/OpenHarmony/:/home/user/src \
-it rayuuu/embedded-dev:latest bash
/work1/OpenHarmony/为主机目录，替换为自己的工作空间。
进入docker，sudo ln -sf /bin/bash /bin/sh 切换为bash。
如果/home/user/src没有权限，chown为user，chown -R user:user /home/user/src以此类推。