移植VMK180 TRD到VCK190的软件编译问题

作者: 付汉杰 hankf@xilinx.com ，文章来源：博客园

1. 概述
所有工具和参考设计使用2021.2。X86编译主机的操作系统是Ubuntu 18.04.6 LTS。

编译记录里的井号，由于和Markdown语法有冲突，把超过3个以上的连续井号全部替换成了星号。有些软件打印的记录非常长，于是把其中部分内容替换成了“......”。
硬件移植、和axi interrupt controller的设备树工作由季茂林（maolinj@xilinx.com）完成。

2. PetaLinux工程
2.1. VMK180 TRD的PetaLinux工程
直接编译vmk180-trd/petalinux/xilinx-vmk180-trd中的VMK180 TRD的PetaLinux工程，能成功编译。

2.2. VMK180 TRD XSA的PetaLinux工程
从VMK180 TRD带的XSA文件(vmk180-trd/platforms/vivado/vmk180_trd/project/vmk180_trd.xsa)创建PetaLinux工程，直接编译，会出错。部分错误信息如下：

ERROR: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function:
0001:
*** 0002:devicetree_do_compile(d)
0003:
File: '/proj/hankf/v212/vmk180-trd/vmk180-trd-xsa-peta/components/yocto/layers/core/meta/classes/devicetree.bbclass', lineno: 131, function: devicetree_do_compile
0127: if not(os.path.isfile(dtspath)) or not(dts.endswith(".dts") or devicetree_source_is_overlay(dtspath)):
0128: continue # skip non-.dts files and non-overlay files
0129: except:
0130: continue # skip if can't determine if overlay
*** 0131: devicetree_compile(dtspath, includes, d)
0132:}
0133:
0134:devicetree_do_install() {
0135: for DTB_FILE in `ls *.dtb *.dtbo`; do
.....................
'/proj/hankf/v212/vmk180-trd/vmk180-trd-xsa-peta/build/tmp/work/versal_generic-xilinx-linux/device-tree/xilinx-v2021.2+gitAUTOINC+c0acd8f064-r0', '-o', 'system-top.dtb', '-I', 'dts', '-O', 'dtb', 'system-top.dts.pp']' returned non-zero exit status 2.

Subprocess output:
/proj/hankf/v212/vmk180-trd/vmk180-trd-xsa-peta/project-spec/configs/../../components/plnx_workspace/device-tree/device-tree/pl.dtsi:364.86-366.8: ERROR (phandle_references): /amba_pl@0/mipi_csi2_rx_subsystem@a4060000/ports/port@1/endpoint: Reference to non-existent node or label "mipi_capture_pipe_cap_pipe_ISPPipeline_accel_0mipi_capture_pipe_mipi_csi_rx_ss_mipi_csi2_rx_subsyst_0"

ERROR: Input tree has errors, aborting (use -f to force output)

DEBUG: Python function devicetree_do_compile finished

把文件system-user.dtsi做下列更改后，才能编译通过。
/include/ "system-conf.dtsi"
/// {
//};

/delete-node/ &axi_intc_0;
/delete-node/ &mipi_csirx_outmipi_capture_pipe_mipi_csi_rx_ss_mipi_csi2_rx_subsyst_0;

/ {
amba_pl: amba_pl@0 {
axi_intc_0: interrupt-controller@a42e0000 {
interrupt-names = "irq";
interrupt-parent = <&gic>;
interrupts = <0 99 4>;
};
};
};

2.3. VCK190 XSA的PetaLinux工程
把VMK180 TRD的工程，移植到VCK190，再根据导出的XSA创建PetaLinux工程，也需要做上述更改，才能成功编译PetaLinux工程。

2.4. 移植VMK180 TRD的PetaLinux工程
直接在VMK180 TRD的PetaLinux工程，导入上述VCK190的XSA（移植到VCK190的VMK180 TRD的XSA），能成功编译PetaLinux工程。

2.5. 基于VMK180 TRD的BSP的PetaLinux工程
把VMK180 TRD的PetaLinux工程打包成BSP，再创建PetaLinux工程，并导入上述VCK190的XSA（移植到VCK190的VMK180 TRD的XSA），能成功编译。

2. X86-Host软件编译
2.1. libaio.h
编译vmk180-trd/pcie_host_package/qdma/apps/dma-util时，得到错误“libaio.h: No such file or directory”。

cc -g -I. -I../include -I../dma-utils -c -std=c99 -o ../dma-utils/dmaxfer.o ../dma-utils/dmaxfer.c -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D_LARGE_FILE_SOURCE -D_AIO_AIX_SOURCE
../dma-utils/dmaxfer.c:34:10: fatal error: libaio.h: No such file or directory
#include
^~~~~~~~~~
compilation terminated.
Makefile:23: recipe for target '../dma-utils/dmaxfer.o' failed

安装libaio1和libaio-dev，再编译就能能成功。

/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app$ sudo apt install libaio1 libaio-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
linux-hwe-5.4-headers-5.4.0-90 linux-hwe-5.4-headers-5.4.0-91 linux-hwe-5.4-headers-5.4.0-92 linux-hwe-5.4-headers-5.4.0-94
Use 'sudo apt autoremove' to remove them.
The following NEW packages will be installed:
libaio-dev libaio1
0 upgraded, 2 newly installed, 0 to remove and 18 not upgraded.
Need to get 19.3 kB of archives.
After this operation, 87.0 kB of additional disk space will be used.
Get:1 http://mirrors.aliyun.com/ubuntu bionic-updates/main amd64 libaio1 amd64 0.3.110-5ubuntu0.1 [6,476 B]
Get:2 http://mirrors.aliyun.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB]
Fetched 19.3 kB in 1s (13.1 kB/s)
Selecting previously unselected package libaio1:amd64.
(Reading database ... 383659 files and directories currently installed.)
Preparing to unpack .../libaio1_0.3.110-5ubuntu0.1_amd64.deb ...
Unpacking libaio1:amd64 (0.3.110-5ubuntu0.1) ...
Selecting previously unselected package libaio-dev:amd64.
Preparing to unpack .../libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ...
Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ...
Setting up libaio1:amd64 (0.3.110-5ubuntu0.1) ...
Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for libc-bin (2.27-3ubuntu1.4) ...

2.2. PF src ‘.tmp_30090’: Permission denied
编译QDMA驱动时，得到错误“mkdir: cannot create directory ‘.tmp_30090’: Permission denied”。

make -C driver
make[1]: Entering directory '/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/driver'
"distro=, dmajor= dminor= "
5.4.1575.4.0-96-generic: GIT,,, -DGITSP -DGIT
CROSS_COMPILE_FLAG = aarch64-linux-gnu-.
ARCH = x86_64.
rm: cannot remove '/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/driver/src/libqdma': Is a directory
ln: /proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/driver/src/libqdma: cannot overwrite directory
***********************
**** PF src ****
***********************
make[2]: Entering directory '/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/driver/src'
srcdir = /proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/driver/src.
KSRC = /lib/modules/5.4.0-96-generic/build.
VF = 0.
CROSS_COMPILE_FLAG = aarch64-linux-gnu-.
ARCH = x86_64.
EXTRA_FLAGS = .
ccflags-y = -D__READ_ONCE_DEFINED__.
make[3]: Entering directory '/usr/src/linux-headers-5.4.0-96-generic'
mkdir: cannot create directory ‘.tmp_30090’: Permission denied
mkdir: cannot create directory ‘.tmp_30092’: Permission denied
............

把所有文件的所有者改为当前用户，编译也有错误。

/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app$ whoami
hankf
/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app$ sudo chown -R hankf *

/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app$ make clean
***********************
**** apps ****
***********************
make -C apps clean;
............

/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app$ make
***********************
**** apps ****
***********************
make -C apps
make[1]: Entering directory '/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/apps'
*****************************
**** dma-from-device ****
*****************************
make -C dma-from-device clean;
make[2]: Entering directory '/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/apps/dma-from-device'
rm -rf *.o *.bin dma-from-device
make[2]: Leaving directory '/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/apps/dma-from-device'
............
make[3]: Entering directory '/usr/src/linux-headers-5.4.0-96-generic'
mkdir: cannot create directory ‘.tmp_34383’: Permission denied
mkdir: cannot create directory ‘.tmp_34385’: Permission denied
..........

使用sudo命令，以root权限编译，能编译成功。

/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app$ sudo make
***********************
**** apps ****
***********************
make -C apps
make[1]: Entering directory '/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/apps'
*****************************
**** dma-from-device ****
*****************************
make -C dma-from-device clean;
make[2]: Entering directory '/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/apps/dma-from-device'
rm -rf *.o *.bin dma-from-device
make[2]: Leaving directory '/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/apps/dma-from-device'
...................
srcdir = /proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/driver/src.
KSRC = /lib/modules/5.4.0-96-generic/build.
VF = 1.
CROSS_COMPILE_FLAG = .
ARCH = x86_64.
EXTRA_FLAGS = -D__QDMA_VF__.
ccflags-y = -D__READ_ONCE_DEFINED__.
Building modules, stage 2.
MODPOST 1 modules
CC [M] /proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/driver/src/qdma-vf.mod.o
LD [M] /proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/driver/src/qdma-vf.ko
make[3]: Leaving directory '/usr/src/linux-headers-5.4.0-96-generic'
make[2]: Leaving directory '/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/driver/src'
make[1]: Leaving directory '/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/driver'

2.3. Build host application pcie_app
编译应用程序pcie_app，得到错误“undefined reference to cv::namedWindow”。

/proj/hankf/vck190/v212/vck190-from-vmk180-trd-bsp-peta/project-spec/qdma-x86-host-app/pcie_app$ ./do_compile.sh
g++ -Wl,-O1 -o app1 main.o mainwindow.o videofrm.o pcie_host.o moc_mainwindow.o moc_videofrm.o -L/usr/lib -L/usr/lib/x86_64-linux-gnu/glib-2.0 -lopencv_core -lopencv_imgproc -lopencv_highgui -lopencv_ml -lopencv_video -lopencv_features2d -lopencv_calib3d -lopencv_objdetect -lopencv_imgcodecs -lrt -lglib-2.0 -lQt5Widgets -lQt5Gui -lQt5Concurrent -lQt5Core -lGL -lpthread
videofrm.o: In function `videofrm::config_frame()':
videofrm.cpp:(.text+0x43a): undefined reference to `cv::namedWindow(std::__cxx11::basic_string, std::allocator > const&, int)'
videofrm.o: In function `videofrm::convert_yuv_to_rgb_buffer(unsigned char*, unsigned char*, unsigned int, unsigned int)':
videofrm.cpp:(.text+0x776): undefined reference to `cv::imshow(std::__cxx11::basic_string, std::allocator > const&, cv::_InputArray const&)'
collect2: error: ld returned 1 exit status
Makefile:167: recipe for target 'app1' failed
make: *** [app1] Error 1
mv: cannot stat 'app1': No such file or directory

使用命令pkg-config opencv --cflags --libs，查看opencv相关库的编译选项，并加入Makefile，能编译成功。

/proj/hankf/vmk180/v212/vmk180-trd$ pkg-config opencv --cflags --libs | grep -i gui
-I/usr/local/include/opencv -I/usr/local/include -L/usr/local/lib -lopencv_dnn -lopencv_highgui -lopencv_ml -lopencv_objdetect -lopencv_shape -lopencv_stitching -lopencv_superres -lopencv_videostab -lopencv_calib3d -lopencv_videoio -lopencv_imgcodecs -lopencv_features2d -lopencv_video -lopencv_photo -lopencv_imgproc -lopencv_flann -lopencv_viz -lopencv_core

主要是在Makefile中加入"-L/usr/local/lib"，指定库的目录。

LIBS = $(SUBLIBS) -L/usr/lib -L/usr/lib/x86_64-linux-gnu/glib-2.0 -L/usr/local/lib -lopencv_core -lopencv_imgproc -lopencv_highgui -lopencv_ml -lopencv_video -lopencv_features2d -lopencv_calib3d -lopencv_objdetect -lopencv_imgcodecs -lrt -lglib-2.0 -lQt5Widgets -lQt5Gui -lQt5Concurrent -lQt5Core -lGL -lpthread

脚本do_compile.sh会执行qmake, 重新生成Makefile，导致Makefile的修改被丢失。所以修改do_compile.sh，屏蔽其中qmake。
if [ -f "$QMAKE_PATH" ]; then
# qmake create Makefile.
# $QMAKE_PATH
make
mv app1 pcie_host_app
else
echo "PLEASE SET QMAKE_PATH to installed qt's qmake path"
fi