The PYNQ-Z1 board is the hardware platform for the PYNQ open-source framework. It features a Zynq-7000 (XC7Z020-1CLG400C) All Programmable System-On-Chip (APSoC), integrating a feature-rich dual-core Cortex-A9 based processing system (PS) and Xilinx programmable logic (PL) in a single device. Figure 1 shows an image of the PYNQ-Z1 development board by Digilent.
Figure 1: The PYNQ-Z1 board
Digging into the hardware architecture of this board is out of scope of this text, so if you are interested in more details, have a look at the Zynq Technical Reference manual.
We will go through a brief description of the software components needed to bring up the board We describe the process to install a linux distribution and verify that the board can execute a simple example.
Software components Link to heading
Bringing up the PYNQ-Z1 board is fairly straight-forward:
- Option 1: use the vendor-provided SD card image for PYNQ-Z1 v3.0.1 from the official website.
- Option 2: use a custom linux distribution; build u-boot, the linux kernel/modules and a generic rootfs of the preferred linux distro, using the tools you are most familiar with.
We will go with Option 2, following the guide provided by Ichiro Kawazome.
Install a linux distribution on the PYNQ-Z1 board Link to heading
First, we will create a target directory that will host the contents of the SD card image: the early boot files, the linux kernel and drivers, and the user-space utilities.
1mkdir -p target
2mkdir -p target/boot
- Step 1: Build
u-boot. - Step 2: Build the Linux kernel
- Step 3: Build the rootfs (Debian 11)
- Step 4: Build Drivers and Services
Step 1: Build u-boot Link to heading
Get the patches/scripts:
1git clone --depth=1 -b v2016.03-1 https://github.com/ikwzm/FPGA-SoC-U-Boot-PYNQ-Z1.git
We need to build u-boot-spl.sfp and u-boot.img. The steps are outlined below:
- Download U-boot Source
1git clone git://git.denx.de/u-boot.git u-boot-2016.03-zynq-pynq-z1
Pick the supported version:
1cd u-boot-2016.03-zynq-pynq-z1
2git checkout -b u-boot-2016.03-zynq-pynq-z1 refs/tags/v2016.03
Apply a number of patches, specific to PYNQ-Z1:
1patch -p0 < ../files/u-boot-2016.03-zynq-pynq-z1.diff
2git add --update
3git add arch/arm/dts/zynq-pynqz1.dts
4git add board/xilinx/zynq/pynqz1_hw_platform/*
5git add configs/zynq_pynqz1_defconfig
6git add include/configs/zynq_pynqz1.h
7git commit -m "patch for zynq-pynq-z1"
8git tag -a v2016.03-zynq-pynq-z1-1 -m "Release v2016.03-1 for PYNQ-Z1"
Patch for the breaking changes of OpenSSL v1.1.x:
1wget https://blog.cloudkernels.net/pynq/0001-fix-build-with-OpenSSL-1.1.x.patch
2git am 0001-fix-build-with-OpenSSL-1.1.x.patch
Setup for Build:
1cd u-boot-2016.03-zynq-pynq-z1
2export ARCH=arm
3export CROSS_COMPILE=arm-linux-gnueabihf-
4make zynq_pynqz1_defconfig
Build u-boot & extract binaries:
1make
2# Copy u-boot.img, u-boot.elf and u-boot-spl.sfp to root directory
3cp spl/boot.bin ../boot.bin
4cp u-boot.img ../u-boot.img
5cp u-boot ../u-boot.elf
6cd ..
Copy boot.bin and u-boot.img to target/boot/:
1shell$ cp boot.bin ../target/zynq-pynqz1/boot/
2shell$ cp u-boot.img ../target/zynq-pynqz1/boot/
Step 2: Build the Linux Kernel Link to heading
Let’s go back to our working director (cd ../) and get the repo sources (we
will need some PYNQ-Z1 specific patches):
1wget https://github.com/ikwzm/FPGA-SoC-Linux/archive/refs/tags/v2.1.1.tar.gz
2tar -zxvf v2.1.1.tar.gz
3cd FPGA-SoC-Linux-2.1.1
Get the kernel sources (v5.10.109):
1git clone --depth 1 -b v5.10.109 git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux-5.10.109-armv7-fpga
2cd linux-5.10.109-armv7-fpga
3git checkout -b linux-5.10.109-armv7-fpga refs/tags/v5.10.109
Patch for armv7:
1patch -p1 < ../files/linux-5.10.109-armv7-fpga.diff
2git add --update
3git add arch/arm/configs/armv7_fpga_defconfig
4git add arch/arm/boot/dts/zynq-pynqz1.dts
5git commit -m "patch for armv7-fpga"
Patch for usb chipidea driver:
1patch -p1 < ../files/linux-5.10.109-armv7-fpga-patch-usb-chipidea.diff
2git add --update
3git commit -m "patch for usb chipidea driver for issue #3"
Patch for build debian package script:
1patch -p1 < ../files/linux-5.10.109-armv7-fpga-patch-builddeb.diff
2git add --update
3git commit -m "patch for scripts/package/builddeb to add tools/include and postinst script to header package"
Create tag and .version:
1git tag -a v5.10.109-armv7-fpga -m "release v5.10.109-armv7-fpga"
2echo 1 > .version
Setup for Build:
1cd linux-5.10.109-armv7-fpga
2export ARCH=arm
3export CROSS_COMPILE=arm-linux-gnueabihf-
4make armv7_fpga_defconfig
Build Linux Kernel and device tree:
1export DTC_FLAGS=--symbols
2make deb-pkg
3make zynq-zybo.dtb
4make zynq-zybo-z7.dtb
5make zynq-pynqz1.dtb
6make socfpga_cyclone5_de0_nano_soc.dtb
Copy zImage to vmlinuz-5.10.109-armv7-fpga:
1cp arch/arm/boot/zImage ../vmlinuz-5.10.109-armv7-fpga
Copy devicetree to target/boot/:
1cp arch/arm/boot/dts/zynq-pynqz1.dtb ../target/zynq-pynqz1/boot/devicetree-5.10.109-zynq-pynqz1.dtb
2./scripts/dtc/dtc -I dtb -O dts -o ../target//boot/devicetree-5.10.109-zynq-pynqz1.dts arch/arm/boot/dts/zynq-pynqz1.dtb
Step 3: Build the rootfs (Debian 11) Link to heading
We go back to our sources directory (cd ../) and setup a couple of host-side utilities:
1apt-get install qemu-user-static debootstrap binfmt-support
2export targetdir=debian11-rootfs
3export distro=bullseye
Build Debian RootFS first-step in $targetdir(=debian11-rootfs):
1mkdir $PWD/$targetdir
2sudo chown root $PWD/$targetdir
3sudo debootstrap --arch=armhf --foreign $distro $PWD/$targetdir
4sudo cp /usr/bin/qemu-arm-static $PWD/$targetdir/usr/bin
5sudo cp /etc/resolv.conf $PWD/$targetdir/etc
6sudo cp scripts/build-debian11-rootfs-with-qemu.sh $PWD/$targetdir
7sudo cp linux-image-5.10.109-armv7-fpga_5.10.109-armv7-fpga-1_armhf.deb $PWD/$targetdir
Build Debian RootFS second-step with QEMU:
Change Root to debian11-rootfs:
1sudo chroot $PWD/$targetdir
Run Second Stage (in the chroot):
1distro=bullseye
2export LANG=C
3/debootstrap/debootstrap --second-stage
Setup APT:
1cat <<EOT > /etc/apt/sources.list
2deb http://ftp.jp.debian.org/debian bullseye main contrib non-free
3deb-src http://ftp.jp.debian.org/debian bullseye main contrib non-free
4deb http://ftp.jp.debian.org/debian bullseye-updates main contrib non-free
5deb-src http://ftp.jp.debian.org/debian bullseye-updates main contrib non-free
6deb http://security.debian.org bullseye-security main contrib non-free
7deb-src http://security.debian.org bullseye-security main contrib non-free
8EOT
1cat <<EOT > /etc/apt/apt.conf.d/71-no-recommends
2APT::Install-Recommends "0";
3APT::Install-Suggests "0";
4EOT
1apt-get update -y
2apt-get upgrade -y
1# Install applications:
2apt-get install -y locales dialog
3dpkg-reconfigure locales
4apt-get install -y net-tools openssh-server ntpdate resolvconf sudo less hwinfo ntp tcsh zsh file
5
6# Setup hostname
7echo debian-fpga > /etc/hostname
8# Setup root password
9passwd
10# This time, we set the "admin" at the root' password.
11
12# To be able to login as root from Zynq serial port.
13
14cat <<EOT >> /etc/securetty
15# Seral Port for Xilinx Zynq
16ttyPS0
17EOT
18
19# Add a new guest user
20adduser fpga
21# This time, we set the "fpga" at the fpga'password.
22
23echo "fpga ALL=(ALL:ALL) ALL" > /etc/sudoers.d/fpga
24# Setup sshd config
25sed -i -e 's/#PasswordAuthentication/PasswordAuthentication/g' /etc/ssh/sshd_config
26# Setup Time Zone
27echo "Etc/UTC" > /etc/timezone
28dpkg-reconfigure -f noninteractive tzdata
29
30# Setup fstab
31cat <<EOT > /etc/fstab
32/dev/mmcblk0p1 /mnt/boot auto defaults 0 0
33none /config configfs defaults 0 0
34EOT
35
36# Setup Network
37apt-get install -y ifupdown
38cat <<EOT > /etc/network/interfaces.d/eth0
39allow-hotplug eth0
40iface eth0 inet dhcp
41EOT
42
43# Setup /lib/firmware
44mkdir /lib/firmware
45# Install Wireless tools and firmware
46apt-get install -y wireless-tools
47apt-get install -y wpasupplicant
48apt-get install -y firmware-realtek
49apt-get install -y firmware-ralink
50
51# Install Development applications
52apt-get install -y build-essential
53apt-get install -y git git-lfs
54apt-get install -y u-boot-tools device-tree-compiler
55apt-get install -y libssl-dev
56apt-get install -y socat
57apt-get install -y ruby rake ruby-msgpack ruby-serialport
58apt-get install -y python3 python3-dev python3-setuptools python3-wheel python3-pip python3-numpy
59pip3 install msgpack-rpc-python
60apt-get install -y flex bison pkg-config
61
62# Install Other applications
63apt-get install -y samba
64apt-get install -y avahi-daemon
65
66# Install Linux Modules
67mkdir /mnt/boot
68dpkg -i linux-image-5.10.109-armv7-fpga_5.10.109-armv7-fpga-1_armhf.deb
69
70# Clean Cache
71apt-get clean
72
73# Create Debian Package List
74dpkg -l > dpkg-list.txt
75
76# Finish
77exit
Now, we are not in the chroot. Let’s cleanup:
1sudo rm -f $PWD/$targetdir/usr/bin/qemu-arm-static
2sudo rm -f $PWD/$targetdir/build-debian11-rootfs-with-qemu.sh
3sudo rm -f $PWD/$targetdir/linux-image-5.10.109-armv7-fpga_5.10.109-armv7-fpga-1_armhf.deb
4sudo mv $PWD/$targetdir/dpkg-list.txt files/debian11-dpkg-list.txt
We can build a tgz with the above files:
1# Build debian11-rootfs-vanilla.tgz
2cd $PWD/$targetdir
3sudo tar cfz ../debian11-rootfs-vanilla.tgz *
4cd ..
5
6# Build debian11-rootfs-vanilla.tgz.files
7mkdir debian11-rootfs-vanilla.tgz.files
8cd debian11-rootfs-vanilla.tgz.files
9split -d --bytes=40M ../debian11-rootfs-vanilla.tgz
10cd ..
Putting it all together Link to heading
Now that we have a working linux distro on the board, let’s login and build a simple example. We will use Ichiro’s example.
Get the code Link to heading
First, let’s get the code:
1git clone FPGA-SoC-Linux-Example-1-PYNQ-Z1
2cd FPGA-SoC-Linux-Example-1-PYNQ-Z1
Install to FPGA Link to heading
1sudo rake install
2dmesg
The output should be something like the following:
1# sudo rake install
2dtbocfg.rb --install uio_irq_sample --dts uio_irq_sample.dts
3<stdin>:22.13-27.20: Warning (unit_address_vs_reg): /fragment@1/__overlay__/pump-uio: node has a reg or ranges property, but no unit name
4<stdin>:9.13-41.4: Warning (avoid_unnecessary_addr_size): /fragment@1: unnecessary #address-cells/#size-cells without "ranges" or child "reg" property
5# dmesg
6[..]
7[1767436.795386] fpga_manager fpga0: writing pump_axi4.bin to Xilinx Zynq FPGA Manager
8[1767436.966789] OF: overlay: WARNING: memory leak will occur if overlay removed, property: /amba/fpga-region0/firmware-name
9[1767437.047568] fclkcfg amba:fclk0: driver version : 1.7.2
10[1767437.054395] fclkcfg amba:fclk0: device name : amba:fclk0
11[1767437.060294] fclkcfg amba:fclk0: clock name : fclk0
12[1767437.066766] fclkcfg amba:fclk0: clock rate : 100000000
13[1767437.072665] fclkcfg amba:fclk0: clock enabled : 1
14[1767437.077652] fclkcfg amba:fclk0: driver installed.
15[1767437.093380] u-dma-buf udmabuf4: driver version = 3.2.5
16[1767437.098701] u-dma-buf udmabuf4: major number = 243
17[1767437.107763] u-dma-buf udmabuf4: minor number = 0
18[1767437.113390] u-dma-buf udmabuf4: phys address = 0x1f100000
19[1767437.119151] u-dma-buf udmabuf4: buffer size = 1048576
20[1767437.128990] u-dma-buf amba:pump-udmabuf4: driver installed.
21[1767437.147643] u-dma-buf udmabuf5: driver version = 3.2.5
22[1767437.153475] u-dma-buf udmabuf5: major number = 243
23[1767437.158803] u-dma-buf udmabuf5: minor number = 1
24[1767437.164453] u-dma-buf udmabuf5: phys address = 0x1f200000
25[1767437.170388] u-dma-buf udmabuf5: buffer size = 1048576
26[1767437.175881] u-dma-buf amba:pump-udmabuf5: driver installed.
Run the samples Link to heading
1# ./sample1
2time = 0.006068 sec
3time = 0.006053 sec
4time = 0.006064 sec
5time = 0.006040 sec
6time = 0.006060 sec
7time = 0.006052 sec
8time = 0.006061 sec
9time = 0.006077 sec
10time = 0.006019 sec
11time = 0.006048 sec
1# ./sample2
2time = 0.006066 sec
3time = 0.006052 sec
4time = 0.006050 sec
5time = 0.006068 sec
6time = 0.006165 sec
7time = 0.006120 sec
8time = 0.006079 sec
9time = 0.006088 sec
10time = 0.006101 sec
11time = 0.006090 sec
1# python3 ./sample.py
2elapsed_time:6.238[msec]
3elapsed_time:6.284[msec]
4elapsed_time:6.229[msec]
5elapsed_time:6.24[msec]
6elapsed_time:6.195[msec]
7elapsed_time:6.279[msec]
8elapsed_time:6.223[msec]
9elapsed_time:6.2[msec]
10elapsed_time:6.216[msec]
11average_time:6.234[msec]
12thougput :168.21[MByte/sec]
13udmabuf4 == udmabuf5 : OK
Note
You might find useful to browse through the udmabuf
repo. The samples above use udmabuf4 &
udmabuf5, but more elaborate examples could make use of other
source/destination buffers (eg udmabuf0 or udmabuf1). To enable these,
remember to remove the module and re-load it, specifying the relevant buffers:
1rmmod u_dma_buf
2modprobe u_dma_buf udmabuf0=$[ 2 * 1048576 ] udmabuf1=$[ 2* 1048576] udmabuf2=$[ 2 * 1048576 ] udmabuf3=$[ 2 * 1048576 ]
Simple Vector operation examples Link to heading
To demonstrate the added value of running on the FPGA fabric, we use simple vector operation examples, such as vector add, matrix-to-matrix multiplication etc.
The code can be found at github.
Future steps Link to heading
Give us a shout at team@cloudkernels.net if you liked it! The plan is to use this board to expose acceleration functionality over the network with vAccel. Stay tuned for the next post where we describe how we did it!