Introduction: Augmented Windshield
This project hints towards a smart windshield for use in the automotive industry, which could augment the driver’s perception of the world in front of him by displaying relevant data on a transparent LCD screen attached to the car’s windscreen. The inter-laid screen could allow more information to be given to the user regarding the objects in front of the car, and even alarms in certain situations, such as a detected risk of an accident. To capture the world in front, and also the user’s position relative to the screen, two sets of stereoscopic (or 3d imaging) devices are used.
It is a work in progress contest entry in the 2017 edition of the Digilent Design Contest.
Step 1: Hardware
To materialize the concept into a hardware design that is feasible both in terms of cost-effectiveness and also ease of implementation, the smart windshield screen is represented physically by means of a modified LCD screen (pic2). The mod in question consist of removing from a consumer-grade PC monitor the back plate, backlight assembly and optic filters, so that only the TFT layer remains. This gives a semi-transparent screen that allows the control of light passed through it, thus the user’s vision of the objects in front of him. The monitor case is used as a mechanical support (pic2) and case for the other boards (pic3).
The bulk of the hardware components are encased underneath metal shielding to reduce radio interference due to the internal high frequency signals. Inside the case (pic4), the main device is the Digilent Nexys Video development board (pic5), which was chosen due to its Xilinx Artix7 FPGA chip’s high number of internal and external resources.
The imaging devices consist of four MT9D111 video capture sensor boards (pic6), two in the front (pic7) and two in the back (pic8), each set allowing stereoscopical processing of visual data. The interface between the cameras and the FPGA devboard is done via a homemade FMC breakout board (pic9,10,11), given the high number of pins necessary. A 7 segment display (pic12) is also implemented for easy status monitoring.
Other hardware components incorporated (pic13,14) are the LCD screen’s driver board and power supply, and also the 12V power supply used by the FPGA board and cameras.
Step 2: Step 2: Software
This part, i.e. the data processing logic, will be implemented both in hard-coded hardware description language and also by means of a soft-core running embedded Linux.
The first main step in realizing this project will be the display of a light filter on the screen that will locally limit the light that is emitted by a source (for ex. a flashlight), that passes through into the eyes of the user. This will simulate a functionality that a real-life smart windshield could have in terms of protecting the vision of the driver from the visual interference given by the headlights of incoming cars, thus not hindering his vision of the road in front of him at night given by the high difference in light intensity.
This aspect of the project is still in progress due the high amount of work involved, so check by for more details and updates.