Zybo - AXI DMA Inside Embedded Linux

by MiguelR73

11K714

Intro: Zybo - AXI DMA Inside Embedded Linux

As the title says, this tutorial explains how I did in order to be able to use the AXI DMA inside the embedded Linux on a Zybo board. Several other tutorials exist in order to install Linux on the Zybo platform (see references in the end of tutorial), so I won't cover that with much detail. However, I had to look through lots of forums and posts spread around the web in order to use the AXI DMA, and that is the reason why I am publishing this tutorial. If anything is wrong or if following the steps is not working for you, please comment and I'll do my best to help you out :)

As a side note, I did this with Vivado 2015.4 installed on Ubuntu 14.04 but I believe it is similar for other versions. Let's start then :)

STEP 1: Install Linux on the Zybo

Follow this tutorial with the following exceptions:

  1. In Step 2, beyond modifying the indicated line of code, modify fdt_high and initrd_high addresses to 0x10000000.
  2. In Step 7, additionally add to the bootargs line of code this -> uio_pdrv_genirq.of_id=generic-uio mem=256M

What this does is make Linux see only 256MB of DDR memory (instead of all the 512MB capacity that the DDR has). This is required in order to have a chunk of memory (in this case, 512-256=256MB) reserved for using the AXI DMA engine. This chunk of memory is our way of sharing data between the PS and the PL in a fast way.

The generic-uio part is required for our peripherals to be recognized and will be explained later on.

STEP 2: Install Required Tools in Zybo

First, run password command and define your password:

  1. passwd

It is not very safe playing around in the web with a root user without having a password. Then connect an Ethernet cable to connect to the web.

My way of communicating with the Zybo is via ssh so we have to install it. Additionally, we have to install the device-tree-compiler.

  1. sudo apt-get install openssh-server
  2. sudo apt-get install device-tree-compiler

The device-tree-compiler is required in order to enable us to change the device tree and make the Linux kernel to be aware of our peripherals. We will make the Linux kernel see our AXI DMA engine and our chunk of shared memory as peripherals.

STEP 3: Create a Simple AXI DMA Loopback Vivado Project

Now, in our host computer that has Vivado installed, we will create a simple AXI DMA loopback project. Follow this link for instructions with the following exceptions:

  1. Don't use the FIFO and connect directly the MM2S port to the S2MM port in the AXI DMA block.
  2. Double click in the AXI DMA block and uncheck Scather-Gather engine.
  3. Stop after generating the bitstream, we do not need the SDK.

STEP 4: Patch the Device Tree

Now, open a terminal and connect to the Zybo board via ssh (you will know your Zybo IP using ifconfig). We will patch the device tree.

  1. ssh root@YOUR_ZYBO_IP
  2. dtc -I fs -O dts /proc/device-tree -o devicetree.dts

This will dump the device tree in a human readable form. We will patch it in order for our DMA to be seen by the Linux kernel. Use the nano text editor and create "pl.dtsi" with the following content:

/ {
scratch_mem@10000000 {

#address-cells = <1>;

#size-cells = <1>;

reg = <0x10000000 0x10000000>;

compatible = "generic-uio";

interrupts = < 0 58 0 >;

interrupt-parent = <0x1>;

};

amba_pl: amba_pl {

ranges;

#size-cells = <0x1>;

#address-cells = <0x1>;

compatible = "simple-bus";

dma@40400000 {

#dma-cells = <1>;

compatible = "generic-uio";

interrupt-parent = <0x1>;

interrupts = <0 29 4 0 30 4>;

reg = <0x40400000 0x10000>;

};

};

};

Also, add the following line to the "devicetree.dts" file -> /include/ "pl.dtsi"

Now, we will recompile the device tree into binary form and substitute the original device tree. Then, our scratch_mem peripheral (located at 0x10000000) and our DMA engine peripheral (located at 0x40400000) will be recognized. Do the following:

  1. sudo mount /dev/mmcblk0p1 /boot
  2. sudo cp /boot/devicetree.dtb /boot/devicetree.dtb.orig
  3. sudo cp devicetree.dtb /boot/devicetree.dtb
  4. sudo sync
  5. sudo umount /boot
  6. reboot

Your ssh connection will close and your Zybo will reboot. Your device tree changes will be applied.

STEP 5: Substitute the Bitstream and Enjoy :)

After the reboot, re-connect to Zybo via ssh. You can see that your new peripherals are being recognized by using the following commands:

  1. ls -l /dev/uio?
  2. ls -l /sys/class/uio/uio?/
  3. cat /sys/class/uio/uio?/name

Now, we will need to transfer our DMA loopback bitstream to the Zybo. Open another terminal and type:

  1. scp /PATH_TO_BITSTREAM_FOLDER/BITFILE_NAME.bit root@YOUR_ZYBO_IP:/root/

Back to your ssh session (the other terminal), you will replace the bitstream being used. Run the following command

  1. sudo dd if=BITFILE_NAME.bit of=/dev/xdevcfg

Now the bitstream being used is the one containing the DMA loopback design and you can use it with an application. I made use of this files: file1, file2 and file3. In order to use them you have to install python and numpy:

  1. sudo apt-get install python python-numpy

Finally, one more thing before just running test_dma.py. The interrupts were not working for me, so I used polling. In order to do so, I substituted this piece of code:

if dma.wait():

print('DMA Transfer Completed')

else:

print('DMA Transfer FAILED')

by

while(1):

if(dma.idle()):

break

Now, you can run test_dma.py and verify that the AXI DMA is working. You will not have cache problems since your scratch_mem is being used as a peripheral. After you are able to use this example, you may break the loopback and insert your own custom IP.

STEP 6: (Optional) Change Clock Frequency

If your design needs a different PL clock frequency than 100MHz, you will need to change it separately, since this changes are not reflected in the bitstream. This is because the PL clocks are generated by the PS. Thus, in order to change the clock frequency of FCLK0 you run the following commands:

  1. echo fclk0 into /sys/{whatever_path_on_your_system}/fclk_export
  2. echo 1 into /sys/class/fclk/fclk0/enable
  3. echo {needed_frequency_in_Hz} into /sys/class/fclk/fclk0/set_rate

The same is applied to other clocks.

STEP 7: (Mandatory) Read the References

In order to make it short, this tutorial is very simplistic. For you to know what you're doing, you need to read a lot. This references are what made me put the AXI DMA working:

  1. Embedded Linux Tutorial #1
  2. Embedded Linux Tutorial #2
  3. Embedded Linux Tutorial #3
  4. Divide the DDR memory in Linux
  5. Create an AXI device driver
  6. Create an AXI DMA device driver
  7. Dynamically change the clock frequency

Good luck and lots of patience for you all, this is not for the weak :)

11 Comments

shekhawatraj1711 4 years ago

a very informative tutrotial ,but the test_dmay.py gets stuck in dma.idle() also .pl help

SergiuC15 4 years ago

Hello! The link is broken at step 1. Can you update it please?
Thanks:D

garryc33 6 years ago

I ran all your steps above and I can see uio0 and uio1 in /sys/class/uio/ but when I run your python scripts it just stalls forever. I used your code for polling the DMA but it also just stalls, any idea what could be causing this? Is there anyway to check if the DMA is being read correctly?

JeremieMelo 6 years ago

Is there anything I should change in vivado about the address of dma?

I directly use the bitstream of DMA project to generate BOOT.bin, and modify the zynq-zybo.h and zynq-zybo.dts in u-boot-Digilent-master-next and linux-Digilent-Dev-master. But I cannot boot Linux on zybo, it stuck there showing "starting kernel..."

do I have to modify something in vivado project address edit or change sth before I make the uImage?

MiguelR73 7 years ago

I don't know what happened to the other comments, hopefully you read this one.

The relative path for the zynq_zybo.h file is:
u-boot-Digilent-Dev/include/configs

This can be seen in Digilent Github in u-boot-Digilent-Dev repository.

Regarding your other question, the FPGA clock may be changed manually by following the instructions. By default, it will have the frequency you specified in the bitstream with which you created the FSBL.

Does this help?

VibishnaB 7 years ago

Hi,

Thank you for getting back to me. In step 1, you are asking to make the following changes:

"In Step 2, beyond modifying the indicated line of code, modify fdt_high and initrd_high addresses to 0x10000000"

I need some more clarification. I downloaded the u-boot-Digilent-Dev from the Digilent Github, just to see how to incorporate the changes. But I could not find the "zynq_zybo.h" file. Can you share the relative path of the file, please?

The link to Parallela discussion forum talks about changing the FPGA fabric clock. So do you recommend to do it manually?

MiguelR73 7 years ago

Ok, several thoughts about this:

Your board has 1GB of DDR
memory. Hence, you do not need to use mem=256MB, you could use mem=512MB
and this will limit Linux to use the first 512MB of the memory map.
This is will probably solve it.

If you did not change fdt_high and initrd_high addresses in Step 1, this could be the problem. See the last post in this page:

https://parallella.org/forums/viewtopic.php?f=51&t=3419

Tell me if this works ;)

VibishnaB 7 years ago

Hi,

Thank you for the detailed instructions.

I tried to replicate the steps in my zynq Zc702 board with linux image from analog devices with hdmi tx etc. I could not figure out the instruction given in step 1. basically modifying the zynq_zybo.h. I assumed it is going to be fine and skipped it. Then I proceeded with the subsequent steps.

Now I am stuck at step 5. there are no uio entry under /dev.

What could be wrong? any idea?

MiguelR73 7 years ago

Sorry for the late reply, I will try to help you :)

Typically, whenever the /dev/uio does not show up, even though you told in the device tree that you peripheral had the "compatible = generic-uio" entry, it is because in the bootargs line of code you did not put this -> "uio_pdrv_genirq.of_id=generic-uio".

Hopefully, you do not have to re-do all steps :) You can edit your device tree and add this at this stage. Then the /dev/uio entries should appear.

If this does not work, please post your device tree .dts file and I will have a look

VibishnaB 7 years ago

I have definitely added the bootargs in my device tree.

-------------------------------------------------------

chosen {

bootargs = "console=ttyPS0,115200 root=/dev/mmcblk0p2 rw earlyprintk rootfstype=ext4 rootwait uio_pdrv_genirq.of_id=generic-uio mem=256M";

linux,stdout-path = "/amba@0/uart@E0001000";

};

--------------------

I am attaching the entire device tree file.