The goal of this project is to make a prototype CCTV and thereby get exposure to video integration and image processing utilizing FPGA integration and a Real-Time Operating System on a Digilant Zybo Board. The Zynq -7000 Development Board has a Zynq-7000 SoC with two ARM cores in addition to an FPGA, making it a powerful system capable of many tasks that normal microprocessors cannot handle.
Step 1: Setup and Requirements: Software
We did all of our development using the WebPack edition of the Vivado Design Suite (download here). Getting Vivado correctly set up on your machine at home can be a bit tedious, but there are many helpful online tutorials for that.
The project uses FreeRTOS, which can be downloaded here. This download is an executable and can extract the FreeRTOS source files, feel free to put these wherever your want on your computer.
Source code for the project can be found here. This is a Github project containing all of the custom hardware we put on the FPGA, and any additions we made to FreeRTOS. The README file for this project will walk you through downloading the project, give a description of what is in each file, and walk you through adding source code to FreeRTOS.
Step 2: Setup and Requirements: Hardware
The required hardware for the project consists of the following:
- Zybo Board - Can be found on DigiKey or other online retailers
- OV7670 Camera Module - We sourced this module from Amazon, see References section for lots of good information about this module.
- Hobby Servo - No particular model necessary, we used this one from Amazon.
- A few jumper wires to connect PMOD pins - our simplest test setup needed 18 male-to-female breadboard wires for the camera module, but we made a custom proto-board to make operation much easier.
- A gimbal that can mount your camera/zybo board along with the servo. We created a custom one but a stock kit is a great option as well. For our custom gimbal we used:
- Thin plywood
- Metal standoffs
- Low Profile wood screws
Step 3: Create a New Vivado Project
Once Vivado is installed correctly, we can go ahead and begin our project. First, start Vivado and click "Create Project." This will open up a series of windows to guide you through picking a location for your project (feel free to put it wherever you want in your file tree) and a board the project is associated with. We will not specify any sources when creating the project, so make sure to click that check box when you see it. The board we are developing on is the Zybo board, and the specific board part we used for our project can be found in the image for this step.
Once the project is successfully created, click "Create Block Design" in the left-hand toolbar of Vivado. You may name your design whatever you want, but we used the default of "design_1" for ours. Store this block design local to the project, the default option in most cases. Once finished, these operations should open up a new window in Vivado containing an empty block diagram.
Step 4: Package Custom IP for Block Diagram Import
Before we create the block diagram for this project, we need to tell Vivado about a few custom modules we'll be using in the design. Click "Tools->Create and Package New IP..." to open a window that helps us do this! We will be packaging a specified directory for this task (well, actually a few, but we'll get to that). When Vivado asks for the directory, navigate to any of the folders contained in the 'hdl' section of the GitHub project, then click "Next." This should open a new, temporary Vivado project to package up all of our VHDL code. In this project, navigate to "Review and Package," then click "Package IP." This will close the project, and our custom VHDL module will be ready for use. Repeat this process with every submodule folder in the Github project, as we'll need them all for our block diagram.
Once this is finisehd, we just need to tell Vivado where to find these newly-packaged IP blocks. Click "Tools->Settings..." and navigate to "IP->Repository". Then, add the entire 'hdl' folder of the Github project as an IP repository. This will tell Vivado to look for custom IP blocks in this folder, making them available for use in our block diagram.
Step 5: Create Block Diagram and HDL Wrapper
After generating blocks from the VHDL files last step, it's time to add everything into our block diagram and connect it together. The goal is to generate an overall block diagram matching the image included here. Locate the "add IP button" that looks like a small blue plus on the block diagram editor. You can search for IP blocks in here and automatically add them to the block design. First, add a ZYNQ7 Processing System block. After it pops up, double-click on the block and scroll through the parameters making sure they all match with the included images of this step.
You will additionally need to add a clock wizard, axi timer, all vdma-related IP blocks, and the custom IP blocks we generated in the previous step. The configuration of all these blocks should be straightforward. After all blocks have been added to the block diagram, go back through and connect wires to match the reference diagram. Also, ensure that all wires and signals have the same names as in the reference block diagram.
Following block diagram design, navigate to Soruces>Design Soruces>Design Wrapper>Design and right click on your design. Selected "Create HDL Wrapper" from the dropdown that appears. A newly created HDL wrapper file should pop up on your screen and appear in the "Sources" pane of Vivado. We want to make a few edits the default wrapper, so double-click the newly-created wrapper in the Sources pane if it didn't open automatically. Then, replace all of the code in the default wrapper file with the code found in "design_1_wrapper.v" from the hdl folder of the Github project.
Once done with this, we are ready to attach constraints to the project and let Vivado generate the hardware that will go on the Zybo board.
Step 6: Adding Constraints and Generating Bitstream
Once our modifications to the block design wrapper have been implemented, we need to add a constraints file to Vivado. Click the "Add Sources" button in the Sources window, and select "Add or create constraints." In the next window, add "CCTV_Constraints.xdc" from the Github project's hdl folder. This will import the file into Vivado, and give the environment information about which pins from our block diagram correspond to each physical pin on the Zybo board.
To finalize our hardware design, click "Generate Bitstream" in the left-hand toolbar. Vivado will run through Synthesis, and Implementation before generating the bitstream, so this process may take some time. Once the bitstream is generated, click "File->Export->Export Hardware...", and export both the hardware description and the bitstream file generated into the local project location.
We are now finished generating the hardware that will be put on the FPGA portion of the Zynq chip, now we can move on to developing code for the ARM processor using the Xilinx SDK.
Step 7: Starting the SDK
When the SDK first starts up, it will take a little bit to setup the project and import the hardware description we just sent out of Vivado. Once this is finished, click "File->New->Board Support Package" to create the files necessary to interact with our hardware from a C program. During this process, make sure to include the lwip driver required by FreeRTOS.
After the Board Support Package has been created, we need to import the main source code for the project. Click "File->Import", then select "General->Existing Project", then point the SDK to the folder containing the FreeRTOS demo code and the code from the project Github repository.
Once the project is imported, we just need to tell the project to reference the Board Support Package we just generated. Right-click the FreeRTOS project folder, and click "Properties". Then, navigate to Project References and make sure that the newly-created board support package is the only box checked. Click "OK," and the project will build to completion.
Note:some users have reported a bug in the SDK that will result in the project not seeing the header files in the first BSP created. We have managed to fix this by generating a second BSP in the same way as before, right-clicking the project and selecting "Change Referenced BSP" to select the second package. After this, the SDK can correctly find the header files provided by the BSP.
Step 8: Build and Export SDK Project
Once the code has built successfully, we are ready to load it onto the board! Click "Debug->Debug Configurations..." and create a new System Debugger Configuration. We want this to be a Standalone Application Debug, and want to reset our entire system at the beginning of each run. In the Application tab, enable core0 on the processor, and set breakpoints at program start and main entry for easy debugging. Finally, hit "Apply", then "Debug" to send the compiled C project and hardware bitstream to the Zybo board.
Now all of the software required for the project is finished, let's get building!
Step 9: Custom Hardware: Camera Proto Board
We created a custom protoboard connector for the camera to plug into the PMOD connector, but jumper wires can work just fine if you don't mind messy wires. Zybo pinouts and our camera wiring diagram are included for reference.
Step 10: Custom Hardware: Servos & Gimbal Setup
Building the Gimbal:
As previously mentioned, we used a custom gimbal for this setup. In order to build the gimbal, two rectangular plywood profiles were cut out loosely matching the footprint of the Zybo board. A rectangular slot was cut in the middle of one of the plywood rectangles where the base of the servo was mounted with wood screws.
Next, the servo arm was connected the second plywood rectangle with additional screws. Last, metal standoff legs were added to all four corners of the gimbal, allowing for it to sit upright on a table or mount to a ceiling/wall. A dimensioned drawing of the Zybo board is included if you're thinking of designing anything.
Hobby Servo Control:
Simple hobby servos usually all operate at 50 Hz PWM frequencies (though check the servo's datasheet to make sure). A duty cycle of 5% is equivalent to a rotation of zero degrees, while a duty cycle of 10% is equal to a rotation of 180 degrees. Therefore, any angle between 0 and 180 degrees can be reached with an intermediate duty cycle. Example code for running the servo motor can be found on the project's Github repository.
Step 11: Custom Hardware: 3D Printed Case
We 3-D printed a case for our zybo board to protect it and mount it to our final gimbal easier. We used an open source design linked here: link
Step 12: Final Touches
The final device is capable of being mounted to a surface such as a ceiling or table. We left the servo on a panning loop where it will rotate 30 degrees, stop for 5 seconds to look at a space, and then rotate another 30 degrees until it turns back to zero. However, this is certainly not the only way to pan the camera.
VGA is outputted out the back port of the zybo board to a monitor while a micro USB cable links it to our computer in order to interface and program it. The servo is powered by a 3.3V port on the Zybo board, grounded, and hooked to pin T20 for signal.
Overall this project works well for viewing images and panning, as well as serving as a foundation for image processing. Integrated image processing is left for an exercise is desired.
Step 13: Sources and Further Readings
Some references we have used and further sources on the Zybo Board:
Step 14: Notes on Project Challenges
It's important to note that there are a number of challenges to this project to be aware of. Some of the things we ran into include:
- Vivado is quite prone to crashing. Make sure you save frequently and are patient when Vivado is not functioning correctly.
- The camera module we used uses I2C communication to set internal DSP settings. We found getting a communication driver working with the camera was a little more challenging than expected.
- Vdma is complex and will take some troubleshooting to get operating correctly. Hopefully, our guide should help avoid most of these issues.
Step 15: Notes on Accessibility
This guide was aimed to be used by a broad audience by streamlining some of the VHDL integration, RTOS code, and block design that we have already developed. Furthermore, all code and files are provided on the GitHub repository. Additional sources have been provided in case you get stuck on something that this guide can not help you with or you want to add complexity.
Additional modifications we recommend trying include adding a custom background averager and subtractor to allow for simplistic motion detection.