Busting FPGA Blockchain Myths

Now we’ll start driving the new AMD Vitis tool!  This section will cover the installation of the tool for the use-case at hand.  That is, for the acceleration of an algorithm on a AMD Alveo™ Data Center accelerator card.  Then the Ethash code will be brought into the tool, refactored as necessary, and built.  It’s important to note up front that we are not attempting to optimize the code in this article as this is left as an exercise for interested readers!

Installation of Vitis

This section provides quick overview of installing Vitis technology.  For complete details, including system requirements, refer to the <installation/release notes> (UG<####>).

The AMD Vitis installer can be downloaded from here:

• https://www.xilinx.com/support/download/index.html/content/xilinx/en/downloadNav/vitis.html

The Vitis tool can be used for various applications (e.g. embedded development, AI edge development,  high-performance compute acceleration) but for our acceleration use-case it will need to be installed on a Linux box with a fair bit of available disk space (~50GB).

Downloading and installing the tool will require AMD user credentials so if you don’t already have a free AMD user account you will want to create one here: 

• https://www.amd.com/en/registration/create-account.html.

Once you’ve downloaded the installer (*.bin file) simply run it as an executable.  If the file doesn’t have execution privileges, you’ll need to add it.  For example:

chmod +x <Xilinx_Installer_file>.bin

./<Xilinx_Installer_file>.bin

Alternatively, if you downloaded the complete offline installer you just run the setup file:

./xsetup

The installer will prompt you to select optional components.  The breakdown of these components will look something like:

Design Tools

• IDE
• DocNav

Devices

• Install devices for Alveo and Xilinx edge acceleration platforms
• Devices for Custom Platforms
• Engineering Sample Devices

Installation Options

These options can impact the install size quite a bit so to minimize the install footprint you can deselect the “Devices for Custom Platforms” device component.  Otherwise the remaining default options will be fine.

Additional Dependencies

Vitis technology leverages the OpenCL framework for acceleration flows and to compile OpenCL applications the appropriate header files are required.  For Ubuntu this is handled by the ocl-icd-opencl-dev package.

sudo apt install ocl-icd-opencl-dev

Although it was worth highlighting this particular package because of the important role OpenCL will play, this step can be skipped because it will be installed along with other XRT dependencies as covered next.

The Vitis application acceleration development flow also relies on a Xilinx Runtime (XRT) framework.  XRT is simply a software stack for the deployment environment that established an interface between the host and the targeted acceleration device.  Even if your development environment is separate from your deployment environment XRT will still be needed to build acceleration applications.  To ensure all the dependencies of XRT are met before installing the framework a shell script is provided in the XRT GitHub repo.  This can be downloaded and run as follows.

$ wget https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/tools/scripts/xrtdeps.sh

$ chmod +x xrtdeps.sh

$ sudo ./xrtdeps.sh

Lastly, we install XRT itself and some board-specific packages.  In this article we will target the AMD Alveo U200 so these final packages can be downloaded from the U200 Getting Started page: 

https://www.amd.com/en/support/downloads/alveo-downloads.html/accelerators/alveo/u200.html

Three separate packages will be needed from the site above.  They will have names along the lines of:

1. xrt_*.deb
2. xilinx-u200-xdma-*.deb
3. xilinx-u200-xdma-*-dev-*.deb

Install each of these in sequence.  It’s also worth noting that despite the naming of the packages both the “Deployment Shell” and the “Development Shell” are needed on the development/build environment.  An example of installing a particular version of these packages is given below.

$ sudo apt install ./xrt_201910.2.2.2173_18.04-xrt.deb

$ sudo apt install ./xilinx-u200-xdma-201830.2-2580015_18.04.deb

$ sudo apt install ./xilinx-u200-xdma-201830.2-dev-2580015_18.04.deb


Creating a New Acceleration Application

Time to launch the Vitis tool and get our hands dirty!  If launching the tool from the command line the shell environment must first be setup correctly using a supplied script.  For example:

$ source /opt/Xilinx/Vitis/2019.2/settings64.sh

$ vitis&

From the Vitis IDE we create a new Application Project (e.g. File > New > Application Project…).  After naming the project you’ll be asked to select a platform.  Which platforms are available will depend upon which board-specific packages that were installed.  Since we installed packages for the U200 at least this platform should be available, in addition to possibly some pre-installed defaults.

Moving forward with the U200 platform selected we then choose to create the “Vector Addition” example.  Once the application is created the project should resemble the screenshot below.

Part 3: Importing Ethash code

Earlier in this article we built and benchmarked the Ethash algorithm, but we didn’t actually take a close look at the code.  Should you take the time to inspect the benchmark code you would find that the following line calls the Ethash function:

ethash_full(&hash, full_mem, &params, previous_hash, nonce);

This, in-turn, calls upon the Ethash library and falls upon the following function in libethash/internal.c:

static void ethash_hash(

        ethash_return_value *ret,

        node const *full_nodes,

        ethash_cache const *cache,

        ethash_params const *params,

        const uint8_t header_hash[32],

        const uint64_t nonce) {

...

This function is effectively a C representation of the PoW algorithm given in the Ethereum yellow paper.  Programmers will take lite in the fact that the C code is significantly easier to follow than the paper.  Additionally, there’s some architecture-specific code in the function that can be ignored for the sake of this exercise as well as some code that assumes the presence of cached data.  Removing all such code simplified the function to a purely raw representation of the PoW algorithm as given below.

Now’s a good time to run the code and just confirm that it’s still doing what we expect after all the important changes.  Note the code assumes the availability of the DAG file in the Emulation-SW folder of the project (named ‘dataset’ in the screenshot below).  This is simply the file generated by the Python script covered earlier.

This is also a good time to review the two key output products of the build process.  Also located in the applicable Emulation-* folder is a *.xclbin file and a *.exe file.  The *.xclbin file represents the kernel.  At this stage of software emulation, its contents are far from its final format, but eventually, this will be the file actually loaded into the Alveo board.  This file will customize the board to run the applicable kernel in hardware.  The *.exe file represents the host code that will eventually run on the system hosting the Alveo board.  At a minimum, it will take the *.xclbin file as an argument since it will be responsible for loading the kernel.

Running the code (e.g. Run > Run) should print some output on the Console window.  This includes confirmation that the *.xclbin was loaded, confirmation that the DAG was loaded, and then finally the results of running Ethash as shown below.

Loading: '../binary_container_1.xclbin' 

dag loaded

mix: a3676e668d4a4a9de2d4688dfd9cc84c65464990fc58f2885e1b8f8bfd028b5a

hsh: a7ea1de3a8007134900cd2c86f7e55af68a1d3e4537438a0a966b6cbafa23c90

The output shows both hashes returned by the PoW, but if we look at the last ‘hsh’ value and compare it with the output from running the benchmark program on a PC we see that they match.  That confirms the code is still functionally correct!

Estimating Performance

Now we will move to the next stage of building/running: hardware emulation.  Switching to this configuration can be done from a drop-down menu at the top-right of the Application Project Settings as shown below.

Now we simply build and run again, just as we did for software emulation.  The difference, however, is that after running a hardware emulation will view some reports that estimate performance.  In fact, there are several reports available, but here we will look like just two:

• Profile Summary
• Application Timeline

Although these reports are available within the project folders, the Assistant window is provided for easy access to all reports.  An example of the two we’ll look at here is given below.  Note that these two particular reports will only become available after both building and running the application.

Opening the Profile Summary will provide something similar to the screenshot below.

It’s important to remember that these are just estimates since nothing was run on real hardware, but they none-the-less provide some nice sanity checks that the system is working as expected.  For example, we see that the kernel incurred the highest number of data transfers (67 in this case), which makes sense because the PoW is intended to be heavily memory bound.  Meaning it’s accessing memory a lot (making this a limiting factor for performance) as it grabs and processes chunks of the DAG.  We can also see the large memory transfer of the DAG from host memory to (global) on-board memory at address 0x4000002000.  And again, don’t be discouraged by the skewed emulation times for data transfers.  The reported time of almost 13s to transfer 1GB of data is a far cry from what will actually happen on hardware.

Next, we’ll look at the Application Timeline, which really helps visualize the various operations that take place to run the application.  Below is an example screenshot:

Here the top-left most bar represents the loading of the kernel.  Next, at around center, we see the DAG getting loaded.  Then immediately following the DAG are several bars representing different aspects of the kernel executing.  Near the bottom, we can also see a multitude of pulses representing dispersed memory reads as the kernel fetches data from the DAG.

This is a great exercise to confirm that the algorithm in functionally correct and the hardware will do as is expected, but it also demonstrates that running the code as-is is not an ideal solution for the sake of acceleration.  For example, the report indicates a relatively long latency estimate (which tends to be ball-park accurate) for the kernel at 0.044ms and the memory utilization is very low at just 1.872% (as taken from the Data Transfers tab of the Profile Summary).  This is because we took C code designed to run in a fairly serial manner on CPU clocked at a very high frequency and applied it to an FPGA, which is very good at leveraging parallelism, and ran it in a still serial manner at a much lower clock rate (300MHz).

None-the-less, we have a working kernel to use as a starting point towards additional optimizations that can yield higher levels of memory utilization and parallelism.  A nice exercise for the interested reader!  …or maybe a future article.

In the meantime, let's take what we have and run it on real hardware.

Read Busting FPGA Blockchain Myths Part 4: Deploying the Application

Part 4: Deploying the Application

Now we’ll start driving the new AMD Vitis tool!  This section will cover the installation of the tool for the use-case at hand.  That is, for the acceleration of an algorithm on a AMD Alveo™ Data Center accelerator card.  Then the Ethash code will be brought into the tool, refactored as necessary, and built.  It’s important to note up front that we are not attempting to optimize the code in this article as this is left as an exercise for interested readers!

Installation of Vitis

This section provides quick overview of installing Vitis technology.  For complete details, including system requirements, refer to the <installation/release notes> (UG<####>).

The AMD Vitis installer can be downloaded from here:

• https://www.xilinx.com/support/download/index.html/content/xilinx/en/downloadNav/vitis.html

The Vitis tool can be used for various applications (e.g. embedded development, AI edge development,  high-performance compute acceleration) but for our acceleration use-case it will need to be installed on a Linux box with a fair bit of available disk space (~50GB).

Downloading and installing the tool will require AMD user credentials so if you don’t already have a free AMD user account you will want to create one here: 

• https://www.amd.com/en/registration/create-account.html.

Once you’ve downloaded the installer (*.bin file) simply run it as an executable.  If the file doesn’t have execution privileges, you’ll need to add it.  For example:

chmod +x <Xilinx_Installer_file>.bin

./<Xilinx_Installer_file>.bin

Alternatively, if you downloaded the complete offline installer you just run the setup file:

./xsetup

The installer will prompt you to select optional components.  The breakdown of these components will look something like:

Design Tools

• IDE
• DocNav

Devices

• Install devices for Alveo and Xilinx edge acceleration platforms
• Devices for Custom Platforms
• Engineering Sample Devices

Installation Options

These options can impact the install size quite a bit so to minimize the install footprint you can deselect the “Devices for Custom Platforms” device component.  Otherwise the remaining default options will be fine.

Additional Dependencies

Vitis technology leverages the OpenCL framework for acceleration flows and to compile OpenCL applications the appropriate header files are required.  For Ubuntu this is handled by the ocl-icd-opencl-dev package.

sudo apt install ocl-icd-opencl-dev

Although it was worth highlighting this particular package because of the important role OpenCL will play, this step can be skipped because it will be installed along with other XRT dependencies as covered next.

The Vitis application acceleration development flow also relies on a Xilinx Runtime (XRT) framework.  XRT is simply a software stack for the deployment environment that established an interface between the host and the targeted acceleration device.  Even if your development environment is separate from your deployment environment XRT will still be needed to build acceleration applications.  To ensure all the dependencies of XRT are met before installing the framework a shell script is provided in the XRT GitHub repo.  This can be downloaded and run as follows.

$ wget https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/tools/scripts/xrtdeps.sh

$ chmod +x xrtdeps.sh

$ sudo ./xrtdeps.sh

Lastly, we install XRT itself and some board-specific packages.  In this article we will target the AMD Alveo U200 so these final packages can be downloaded from the U200 Getting Started page: 

https://www.amd.com/en/support/downloads/alveo-downloads.html/accelerators/alveo/u200.html

Three separate packages will be needed from the site above.  They will have names along the lines of:

1. xrt_*.deb
2. xilinx-u200-xdma-*.deb
3. xilinx-u200-xdma-*-dev-*.deb

Install each of these in sequence.  It’s also worth noting that despite the naming of the packages both the “Deployment Shell” and the “Development Shell” are needed on the development/build environment.  An example of installing a particular version of these packages is given below.

$ sudo apt install ./xrt_201910.2.2.2173_18.04-xrt.deb

$ sudo apt install ./xilinx-u200-xdma-201830.2-2580015_18.04.deb

$ sudo apt install ./xilinx-u200-xdma-201830.2-dev-2580015_18.04.deb


Creating a New Acceleration Application

Time to launch the Vitis tool and get our hands dirty!  If launching the tool from the command line the shell environment must first be setup correctly using a supplied script.  For example:

$ source /opt/Xilinx/Vitis/2019.2/settings64.sh

$ vitis&

From the Vitis IDE we create a new Application Project (e.g. File > New > Application Project…).  After naming the project you’ll be asked to select a platform.  Which platforms are available will depend upon which board-specific packages that were installed.  Since we installed packages for the U200 at least this platform should be available, in addition to possibly some pre-installed defaults.

Moving forward with the U200 platform selected we then choose to create the “Vector Addition” example.  Once the application is created the project should resemble the screenshot below.