VisionFive 2 RISC-V - Building the Linux kernel

Introduction

The Linux community has been working on RISC-V versions of the Linux kernel and other software for years and there are Stackoverflow posts from 2015 with people attempting to compile Linux with RISC-V gcc.

Building a RISC-V version of the Linux kernel has gotten easier as tools such as compilers have matured. There are a number of RISC-V devices supported by the Linux kernel development community with manufacturers working to include their RISC-V devices. The VisionFive 2 by StarFive is not fully supported yet so StarFive has its own Github repository from which the development team is pushing updates into the Linux development team.

The StarFive Github area, starfive-tech · GitHub, has several repositories with the one containing the build materials for the VisionFive 2 SBC, GitHub - starfive-tech/VisionFive2. The VisionFive2 repository contains the software components for the Linux kernel. BusyBox, U-Boot loader, and other components for the VisionFive 2, which uses the JH7110 SOC, and the VisionFive, which uses the JH7100 SOC.

Background

If you have little to no experience, I highly recommend that you prepare yourself for the experience of building the Linux kernel by getting an understanding of the basic procedure. The first thing that I did was to spend some time watching various YouTube videos about the process. A list of ones that I found helpful:

• Making Simple Linux Distro from Scratch
• Linux Kernel Programming 01: Compile and Boot
• Embedded Linux Booting Process

Here are a few documents that may be helpful if you are interested in the internals of making an image and what happens during the booting of a Linux device from turning on the power to getting the login screen:

• Working with Linux and u-boot (embeddedartists.com)

What I found was the procedure for building the Linux kernel is quite intimidating in concept and fairly straightforward in execution if you follow the procedure correctly. The most important first step is to make sure that you have all of the cross compile and build tools for the RISC-V architecture installed.

I suggest that you develop understanding of the Git version control system. There is a free electronic version of the Pro Git book by Scott Chacon and Ben Straub that covers everything needed for using Git.

The most common problem I had was copy/paste errors when stepping through the procedures. I was selecting and copying text from a document displayed in a browser in Windows and pasting the text into the Ubuntu terminal window command line of the WSL virtual machine.

I have only tried to build images that are put on a microSD card. I have not tried to use the M.2 NVMe card nor build an image for such a card. My understanding is the image built for the SD card is the basis for the image for the eMMC and the NVMe drive. Once you understand the steps of the SD card image, you pretty much understand the build procedure for any version of the device.

There appear to be monthly Debian builds available through the StarFive Debian Repository from the Releases tab of the repository. In the Assets list are a number of Debian package files for applications such as FireFox.

Build and Test Environment

I used a Windows 11 Pro desktop using Windows Subsystem for Linux running Ubuntu. I have a Ubuntu Linux server with Subversion however I did not want to clutter it up with the tools and source for a Linux build when I assumed that I could easily make mistakes that would be most easily addressed by destroying a WSL virtual machine and restarting from scratch.

The PC I use has the following hardware:

• i5-11400 11th Generation Intel processor
• 24 GB of RAM (was 32 GB until a stick failed)
• 1 TB NVMe solid state drive
• an LG 46" 4K display

I purchased a VisionFive 2 with a bundle from Amazon.com that contained the following parts:

• VisionFive 2 version 1.2A SBC
• TTY to USB adapter cable
• 32 GB microSD card
• USB to microSD card adapter

I have been using Windows Subsystem for Linux, WSL, with Windows 11 to do my builds. I have run into only one issue, a PATH environment variable problem, that was easily resolved. With WSL, I'm using Ubuntu and doing everything from the command line in a terminal window.

Since the build documentation uses the command line interface, a terminal window with WSL works fine. The only Windows applications I use other than WSL are Chrome web browser and the Raspberry Pi Imager utility. I have found that moving files between the WSL file space and the Windows file space with the Windows File Manager works smoothly.

When Windows spins up the Ubuntu virtual machine, the PATH environment variable contains pathnames from Windows containing spaces such as /mnt/c/Program Files/Microsoft SQL Server/110/Tools/Binn/. Due to this difference, when I first tried to make the kernel, I received the following error message:

Your PATH contains spaces, TABs, and/or newline (\n) characters.
This doesn't work. Fix you PATH.

To resolve this, I echoed the $PATH variable to a file then created a simple shell script where I used the export command to set my PATH environment variable with a modified path taking out all of the pathnames with spaces. I then used the source command from the command line to set my PATH to the new value.

A TTY to USB adapter is highly recommended to provide access to the UART pins on the 40 pin connector. When the desktop was unavailable or I needed additional logs and other debug information, I used a terminal emulator as user root to look at what was happening. A terminal emulator will normally have some sort of logging capability allowing you to have a text log to review for information if needed or to include in a forum post if you are seeking assistance.

There are several applications that you will use when getting source code and components.

• a web browser to traverse the StarFive Github repository and to access various documents
• apt tool or other package manager for installing software tools
• git source management tool, sudo apt install git
• git large file tool, sudo apt install git-lfs
• curl tool, sudo apt install curl
• wget tool, sudo apt install wget

The documentation for the two alternative builds provides the apt commands needed to set up your build environment. The two methods do have many tools in common as well as a few that are not shared so to prevent wasting time tracking down errors due to missing tools, take the time to set up your build environment properly.

Source Tree Clone and Pulls

I have both build environments with the source of each image in a different file hierarchy. This way I can build either version by just going to the root folder of the file hierarchy for that image and then performing the appropriate procedure.

Since it is a lot of source to download and it has been a couple of weeks since I did the git clone of the various repositories, I'm now working out how to keep my local Git repositories updated.

The two different alternative image builds use Git in different ways. The local version of the StarFive VisionFive2 repository is created with a single git clone command which then pulls down several linked repositories including the Linux source which is physically stored in a separate repository on the StarFive GitHub. The local version of the Debian Image from Scratch is built by using several git clone commands along with several wget commands to manually build directory and file tree holding the source and the other components that are part of the Debian desktop image.

This Stackoverflow post, Can "git pull --all" update all my local branches?, describes some of the difficulties in managing a complex Git structure involving branches. What I am using currently is to go to the root directory of the tree where I did the git clone of VisionFive2 repository and do a git pull to update that tree then I go to the root directory of the manually built tree for the Debian Image from Scratch and run a bash script which goes to the several Git repositories that were cloned and perform a git pull in that directory.

Two Alternative Builds

There are two different documented methods for doing a Linux build using the StarFive VisionFive materials that I have found thus far. I will provide my experience on each below.

The first method uses a procedure specified in the README.md file in the StarFive VisionFive2 repository. That procedure builds a Linux kernel with BusyBox for a user interface using Buildroot. This version does not boot with a desktop. The image from that procedure requires using a TTY to USB adapter connected to the UART pins on the 40 pin connector with a terminal emulator on another device allowing access to a command line as the root user.

The second method is documented in Building the Debian Image from scratch. The procedure requires various manual procedures to download files, create files using the cat and echo commands, and manually creates the image file using a loop device. The second method is involves a fair amount of time and is quite tedious however the result is a desktop displayed on an HDMI screen connected to the VisionFive 2. This method has two users, root and user (can be elevated using sudo) with the username user the default for the sign in prompt.

In the rest of this article, I will refer to the first method as the VisionFive2 README.md version and the second method as the Debian Scratch Build.

The products created by these two different methods are different. The first method, VisionFive2 README.md method, appears to create a Linux kernel with BusyBox without most of the software content of the official StarFive Debian build whose latest version as of this writing is 202310. The second method, Debian Scratch Build, appears to create a complete replica of the 202310 build with all the software of that build.

The first major difference between the two is that the first method requires a terminal using a TTY to USB adapter connecting the terminal to the VisionFive 2 UART pins on the 40 pin connector. The second method provides a complete desktop that can be used with an HDMI monitor with keyboard and mouse attached to the VisionFive 2 device ports.

There are several steps during this procedure that can take a long time to complete so be prepared to start some tasks running with a command and then going off to do something else for a few hours.

After struggling with build errors for days by installing missing packages, I discovered that when I compared the suggested app install list from the README.md file with the list of things I had installed, I realized that the app install list from the README.md file contains most of these. When I retried using the same list by copying and pasting the text from the README.md file, I discovered that the paste operation inserted some kind of a line break character and when I ran the apt install pasted into the Ubuntu command line, the list of packages processed by the apt command was not the complete list I thought I was specifying.

Beware of copy and paste errors!

On the other hand, I had another error complaining the sha256 checksum of the GPU driver file did not match what was expected. It appeared the error was due to an incorrect GPU driver file but it was actually due to Git LFS not being installed before doing the Git repository clone with git clone https://github.com/starfive-tech/VisionFive2.git. I thought I had performed the step for installing Git LFS but I had downloaded the package but had not installed it with apt install.

Using the code

Setting up the build tools for the README.md alternative

The README.md file in the VisionFive 2 repository contains a fairly good list of steps. However during my first experience doing a kernel build, I ran into a couple of missing packages that caused the build to fail. However I later discovered that the packages were missing due to a copy/paste error on my part of an apt command in the procedure.

The README.md file in the directory VisionFive2 mentions the use of Git LFS (Large File Support) which is used to pull down some Third Party components such as the GPU drivers. What I have seen is that not having Git LFS installed can result in errors such as sha256 checksums  not matching that appear to have nothing to do with Git LFS not being available. 

When I ran the curl command to obtain Git LFS, it took a long time to run and seemed to hang. The only output that I saw was a message printed that mentioned the Ubuntu version I was using. I finally pressed the return key on my keyboard and it seemed to finish. You can double check that git-lfs is available for installation with apt-cache search git-lfs. Then run the apt install.

Cloning and setting up the VisionFive 2 Linux build source tree requires that Git be installed and Git LFS be installed first. Before using the git command to clone the source tree, you must have both installed or you will get an incomplete source and Third Party components that are incomplete yet the build will have few errors indicating the issue.

The VisionFive 2 repository README.md file specifies a number of steps including the apt command line used to set up the cross compile environment. 

Setting up the build tools for the Building from Scratch alternative

After using the Buildroot system from the VisionFive 2 repository, I changed to the procedure from the Building from scratch document. This required me to do a sudo apt install gcc-riscv64-linux-gnu since that process expects that tool which appears to be a different compiler than the VisionFive2 repository procedure uses. I thought that was the only difference until I ran into another missing application needed, at which point I did what I should have done at the beginning and copied and pasted the sudo apt install needed for the tools as specified in the Building from scratch document and also did the step to export an environment variable VF2_WORK_DIR for the build working directory.

As I worked my way through the steps, the step Create bootstrap rootfs in the Create Debian Root Filesystem section with the debootstrap command took a long time and appeared to hang before starting to slowly display lines of progress. In my first attempt using the command, I did get the error "gpg may be incorrect or out of date." Following the documentation, which mentioned this possible error, I then retried the command with the suggested --no-check-gpg parameter option and it ran fine though it took a long time as in over night in its retrieving and validating.

For the most part the rest of the procedure of Building from Scratch was a somewhat tedious process of copying and pasting text making sure that everything was properly selected when copying and that the paste was the text copied. My procedure was to select a line at a time for a copy and paste to ensure that the text was correct.

There is a point at which several files are created or have text lines appended using the cat command to copy text entered in the terminal command line window and place it into a file. Most of these were short files that could be done with a single copy/paste however there were a couple that had to be copy/pasted a section at a time.

Two important notes

One important note about the step in the Build SD Image section when the /dev/loop30 device is mentioned in the commands. You must replace the text of /dev/loop30 with the specific loop device that is displayed by the losetup command. In the terminal window I used the command as written in the procedure rather than substituting with my device, /dev/loop0, in place of /dev/loop30 resulting in an error. See the following excerpt.

rick@rchamber2:~/Documents/starfive/github/VF2/create_sd_image$ sudo losetup --partscan --show --find starfive-jh7110-SD-minimal-desktop.img
/dev/loop0
rick@rchamber2:~/Documents/starfive/github/VF2/create_sd_image$ mkdir boot rootfs
rick@rchamber2:~/Documents/starfive/github/VF2/create_sd_image$ sudo mkfs.vfat /dev/loop30p3
mkfs.fat 4.2 (2021-01-31)
mkfs.vfat: unable to open /dev/loop30p3: No such file or directory
rick@rchamber2:~/Documents/starfive/github/VF2/create_sd_image$ sudo mkfs.vfat /dev/loop0p3
mkfs.fat 4.2 (2021-01-31)
rick@rchamber2:~/Documents/starfive/github/VF2/create_sd_image$ sudo mkfs.ext4 -m 0 -L root /dev/loop0p4
mke2fs 1.46.5 (30-Dec-2021)
Discarding device blocks: done
Creating filesystem with 996091 4k blocks and 249488 inodes
Filesystem UUID: ff03fc59-02e2-4ade-b517-779a498d4c63
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

rick@rchamber2:~/Documents/starfive/github/VF2/create_sd_image$

Secondly, chose the correct image build procedure for the type of storage device - SD card, eMMC, or NVMe card - you are using. I used the eMMC procedure when I should have used the SD card procedure  and I found errors when booting from the SD card in my testing and the image failed to boot to the desktop.

The HDMI display showed the following message when booting with the first image I created that used the eMMC procedure with an extra step using sed command to modify the storage device name rather than the SD card image procedure (this is just the last part showing the error):

Begin: Running /scripts/local-block
Begin: Running /scripts/local-block… done. done.
Gave up waiting for root file system device. Common problems: Boot args (cat /proc/cmdline)
Check rootdelay= (did the system wait long enough?) - Missing modules (cat /proc/modules: ls /dev)
ALERT! /dev/mmcblk0p4 does not exist. Dropping to a shell!

Fortunately, the TTY interface showed the initramfs prompt and I was able to do some looking about using the TTY interface. When I looked in /dev at the initramfs prompt with ls /dev/m* I saw mmcblk1 and each of the expected partitions mmcblk1p4, etc. but not mmcblk0.

I took a look at the three files that were modified using the sed command to change mmcblk1 to mmcblk0 in the eMMC procedure. The file /etc/fstab had nothing more than a comment. The other two had directives that referenced mmcblk0 so I modified those back to mmcblk1. I then redid the SD card image build procedure, leaving out the sed command step, and created an image which I then burned to a microSD card using the Raspberry Pi Imager utility.

I placed the microSD card into the VisionFive 2 drive and booted the device by pressing the power button. After a short time, messages started scrolling on the display followed by a clearing of the screen, a pause, and then the desktop appeared with the sign in prompt.

I signed in and set up WiFi then tried out FireFox. Things looked similar to the 202310 Debian build.

Adding applications to the Debian image

Looking at the pathnames used in the wget commands in the Copy applications step of the Create Debian Root Filesystem section of the document, it appears that you could use the most recent version of those applications by using a pathname from the Assets in the StarFive Debian Repository Releases in the wget command line.

Some of the Assets available may not be working properly due to a still not corrected issue in a dependency. For example, as of this writing (Dec-02-2023) Chromium is not working properly and will not open correctly.

As part of investigating building an image containing applications that I have created, I wrote a simple HelloWorld.c application. I cross compiled the source file and put the executable into the /usr/bin folder of the Linux image area. I then built the image and tried it out on the VisionFive 2 and the program was there and it did run.

I also tried including Chromium from the list of Assets that were part of the October 2023 build of the Debian repository into my image. This involved using wget to pull down an Asset from the Debian Releases link. Chromium appeared as one of the applications in my desktop when I booted the image but unfortunately, there is currently, as of Dec-04-2023, a bug that is causing Chromium to crash.

Since I was modifying an already existing image build area, I used the following procedure adapted from the Building Debian from Scratch procedure to add Chromum:

sudo chroot $VF2_WORK_DIR/riscv-chroot-snapshots
cd /opt
wget https://github.com/starfive-tech/Debian/releases/download/v0.11.0-engineering-release-wayland/chromium-103.0.5060.114.deb
dpkg -i chromium-103.0.5060.114.deb
rm chromium-103.0.5060.114.deb
apt clean
history -c
sync
exit

I haven't tried but I would assume that you could use a similar procedure for any additional packages you wanted to add to a Debian build so long as the package was a Debian package compiled for the StarFive VisionFive 2 and a RISC-V compiler or cross compiler.

Points of Interest

I was happy to discover that I could use Windows Subsystem for Linux on a Windows PC to perform the VisionFive 2 image builds. There were a few bumps in the road such as the PATH environment variable that contained spaces as well as some copy/paste errors that caused build failures but it all worked out well in the end.

I was not using the TTY to USB adapter connected to the 40 pin connector on the VisionFive 2 in order to see the system terminal output when I first tried my image build. So what I saw on my HDMI monitor was that the image boot was failing when in reality the image I had built did not have support for the desktop. Once I connected up a Linux tablet with the TTY to USB adapter to the VisionFive 2 and was able to see what was happening during the boot, to deduce the incorrect device specified and correct the configuration files and try again.

The YouTube videos about building a Linux kernel were very helpful as they gave me some idea as to what to expect. Those videos also helped me to recognize that the first image I was building did not have a desktop for the HDMI display and was just the Linux kernel with BusyBox that had to be accessed through the system terminal using the TTY to USB adapter.

I've had to learn more about Git version control system and was happy to find the free online book.

When pulling down the source for building the images, it can take a lot of time especially for the Debian Image build. There were a number of times during when executing the procedures for building the images that there was a long wait with no feedback that anything was actually happening.