This is a project focused on the smart city context. In this matter, there is three major problem we are solving:

1 – energy saving in public lighting; 2 – improve city security; 3 – improve traffic flow.

1 – By using LEDs lights on streets, the savings are already up to 50%, and with the addition of Telemanagement, we can have 30% more savings.

2 – With the use of smart cameras, we can control the lights to dim down where the people flow is absent and make the street section brighter where people are walking. It will not only save energy but increase the feeling of being watched, thus, intimidate bad intentioned people. Furthermore, visual alarms (blinking the lamps for example), could be used in case of suspicious behavior.

3 – The smart camera will watch the traffic, process locally its conditions, and control the light signals in order to best manage the traffic. In this way, traffic jams could be avoided, cars would not have to wait for long time red signals when there is no flow in the crossing, and so on. Regarding technological problems, we are also solving what are common issues in IoT such as robust connectivity in city scale and camera integration for IoT Network, by using edge processing to transmit only relevant information.

See our publication on Embarcados and GitHub

Also on YouTube

Our team:

Milton Felipe Souza Santos

Gustavo Retuci Pinheiro

Eduardo Caldas Cardoso

Jonathas Baker

(Contact information at the bottom)

Detalhes_do_projeto.pdf
Download

Step 1: System Block Diagram

This is an overview of the solution architecture.

The system is composed of a Camera-Gateway that uses RFmesh on the FAN interface, WiFi on LAN, and also CAT-M for WAN connectivity. It also contains smart photocells, Smart Cameras, and light signals.

All devices in the networks, mainly the smart camera, are sending data through 6lowpan to the smart gateway, so it can take the decisions regarding public lighting and light signals control.

The gateway is also connected to our server via VPN. In this manner, we have access to the FAN and LAN, bot for checking the status or control the devices.

Step 2: Components for This Project

Smart Cam

- DragonBoard410C/DragonBoard820C

- USB Camera

- OneRF NIC

Camera Gateway

- DragonBoard410C/DragonBoard820C

- USB Camera

- OneRF NIC

- Cat-M/3G modem

Smart Light Signal

Step 3: Step 2: Circuit Diagram and Connections

Smart Cam

- Camera on USB port

- OneRF NIC at UART port

Camera Gateway

- Camera on USB port

- OneRF NIC at UART port

- 3G/Cat-M modem at USB port

(All connected by the IoT Mezzanine)

Smart Stree Light

- Conventional streetlight

- Relay board (3 channels)

- OneRF NIC

Smart Photocell

- OneRF NIC

- Power Meter

Step 4: Install Os on the DragonBoards

Installing Debian on Dragonboard820C (Fastboot Method)

Using a Linux OS, install the packages listed in: https://www.96boards.org/documentation/consumer/d...

On the dragonboard:

make s4 OFF,OFF,OFF,OFF

Turn on pressing vol (-)

If you are using a serial monitor (highly recommended), you will get the message “fastboot: processing commands” (serial monitor at 115200) Connect the micro-usb (J4) on the PC

On host PC:
Download (and unzip) from https://www.96boards.org/documentation/consumer/d...

$ sudo fastboot devices

452bb893 fastboot (example)

$ sudo fastboot flash boot boot-linaro-buster-dragonboard-820c-BUILD.img

$ sudo fastboot flash rootfs linaro-buster-alip-dragonboard-820c-BUILD.img

Installing Debian on Dragonboard410C

Steps on the computer (Linux)

1 – Download the image

$ cd ~

$ mkdir Debian_SD_Card_Install_image

$ cd Debian_SD_Card_Install_image

$ wget http://builds.96boards.org/releases/dragonboard41...

2 – Unzip the files

$ cd ~/Debian_SD_Card_Install_image

$ unzip dragonboard410c_sdcard_install_debian-233.zip

3 – Insert the microSD on your computer and check if it mounted

$ df -h

/dev/sdb1 7.4G 32K 7.4G 1% /media/3533-3737

4 – Unmount the microSD and burn the image

$ umount /dev/sdb1

$ sudo dd if=db410c_sd_install_debian.img of=/dev/sdb bs=4M oflag=sync status=noxfer

5 – Remove the microSD from you PC

Steps on the computer (Windows)
Download – SD Card image – (Option 1) SD Card Image – Install and boot from eMMC

https://www.96boards.org/documentation/consumer/dr...

Unzip SD Card Install Image

Download and install the Win32DiskImager tool

https://sourceforge.net/projects/win32diskimager/f...

Open Win32DiskImager tool

Insert the SD card in the computer

Find the extracted .img file

Click on Write

Steps on the Dragonboard
Make sure DragonBoard™ 410c is unplugged from power

Set S6 switch on DragonBoard™ 410c to 0-1-0-0, “SD Boot switch” should be set to “ON”.

Connect an HDMI

Plug a USB keyboard

Insert the microSD

Plug power adapter

Select the image to install and click “Install”

wait for the installation to finish

Remove power adapter

Remove the microSD

Set S6 switch to 0-0-0-0

DONE!

Step 5: Connectivity Interfaces

Installing Cat-m and 3G

Apply the following AT commands using a host machine:

AT#SIMDET? // check SIM presence
#SIMDET: 2,0 // sim not inserted

#SIMDET: 2,1 // sim inserted

AT+CREG? //check if it is registered

+CREG: 0,1 //(disable network registration unsolicited result code (factory default), registered home network)

AT+COPS?

+COPS: 0,0,”VIVO”,2 // (mode = automatic choice, format = alphanumeric, oper, ?)

AT+CPAS //Phone Activity Status

+CPAS: 0 //ready

AT+CSQ // check quality of service

+CSQ: 16,3 //(rssi, bit error rate)

AT+CGATT? //state of GPRS attachment

+CGATT: 1 //attached

AT+CGDCONT=1,”IP”,”zap.vivo.com.br”,,0,0 // configure context

AT+CGDCONT? // check context

+CGDCONT: 1,”IP”,”zap.vivo.com.br”,””,0,0

AT#SGACT=1,1 //Context Activation

#SGACT: 100.108.48.30

Setup the Interface

Using graphical environment

Connect the modem (oneRF_Modem_v04 – HE910)

Open Network Connections

Click on + to add new connection

Select Mobile Broadband

Select the correct device

Select the Country

Select the provider

Select the plan and Save

Remove the Modem

Reconnect the Modem

Using terminal
apt-get install pppconfig

pppconfig

provider = vivo

dinamico

CHAP

vivo

115200

Tone

*99#

no (manual)

/dev/ttyUSB0

save

cat /etc/ppp/peers/vivo

cat /etc/chatscripts/vivo

pon vivo

If you are using Cat-M module, just use the following commands before:

echo 1bc7 1101 > /sys/bus/usb-serial/drivers/option1/new_id

apt-get install comgt

comgt -d /dev/ttyUSB0 comgt info -d /dev/ttyUSB0

Step 6: Installing Essential Software Modules

On the development computer

Note that some steps are hardware-dependent and should be adjusted to meet your actual computer specifications. The libraries may be installed with a single command.

sudo apt install build-essential git libatlas libgoogle-glog-dev libiomp-dev libleveldb-dev liblmdb-dev libopencv-dev libopenmpi-dev libsnappy-dev libprotobuf-dev libatlas libboost libgflags2 hdf5 openmpi-bin opnempi-doc protobuf-compiler python-dev python-pip python-numpy python-scipy python-matplotlib python-future python-protobuf python-typing python-hypotesis python-yaml

OpenCV

This framework is used to develop image based statistical algorithms on the development machine. Since most of our code is written in Python, the easiest installation method is to just

pip install opencv-python

Note, however, that these wheels will not use anything apart from your CPU and may not even use all of its cores, so you may want to compile from source to achieve maximum performance. To build the package in Linux, for example, you donwload the zip file form the OpenCV Releases page and unzip it. From the unzipped folder:

mkdir build && cd build
cmake .. make all -j4

sudo make install

The -j4 command instructs make to use four threads. Use as many as your CPU has!

Caffe

To setup the Caffe framework from sources:

git clone https://github.com/BVLC/caffe.git && cd caffe
mkdir build

cmake ..

make all

make test make runtest

If all tests run successfully then you are all set.

TensorFlow

Google does not let you compile TensorFlow with ordinary tools. It requires Bazel for it and chances are it won’t work, so avoid compiling it and just grab the pre-compiled module with:

pip install tensorflow

If your computer is bit old and doesn’t have AVX instructions, get the last non-AVX tensorflow with

pip install tensorflow==1.5

And you’re done.

SNPE – Snapdragon™ Neural Processing Engine

Setting Snappy up, as our Qualcomm friends call SNPE, is not hard but the steps should be followed closely. The installation outline is:

clone the neural network frameworks’ git repositories

Caffe
Caffe2

TensorFlow

ONNX

run the scripts to check for dependencies
snpe/bin/dependencies.sh

snpe/bin/check_python_depends.sh

for each installed framework run snpe/bin/envsetup.sh

source $SNPE/bin/envsetup.sh -c $CAFFE_GIT

source $SNPE/bin/envsetup.sh -f $CAFFE2_GIT

source $SNPE/bin/envsetup.sh -t $TENSORFLOW_GIT

source $SNPE/bin/envsetup.sh -o $ONNX_GIT

To source SNPE in every terminal instance you open, append step three’s four lines to the end of your ~/.bashrc file.

On the target board

Moving to arm64 from amd64 is not an effortless task, as many libraries will take advantage of x86 instructions to boost their performance. Luckily, it is possible to compile most of the necessary resources on the board itself. The needed libraries may be installed with a single command.

Install them with apt and move on. Note that this step may take some time, as apt calls make to build the code that’s not pre-compiled.

OpenCV

Download the release from the OpenCV repository, unzip it somewhere and from the unzipped folder:

mkdir build && cd build
cmake ..

make all -j3

sudo make install

Note that we used the -j3 option. If you access the board via ssh, having all cores fully loaded may be enough to drop the connection. That’s not desirable. By limiting thread usage to three, we will always have at least one free thread to cope with ssh connections and general system housekeeping.

This is for the Dragonboard 820 and Inforce 6640 with the APQ8096 chip. On the Dragonboard 410 you will want to have some free virtual memory or limit the compile threads to one, since it has less physical RAM available.

It is also of note that cooling the chip will help increase performance by limiting thermal throttling. A heatsink does the trick at small loads but you will want a proper fan for compiling and other CPU-intensive loads.

Why not install OpenCV with apt or pip? Because compiling it in the target machine makes every available processor instruction visible to the compiler, improving execution performance.

SNPE – Snapdragon™ Neural Processing Engine

We installed Snappy just like it was on a desktop computer, even though there was no actual neural network framework installed (SNPE only needs the git repos, not the actual binaries).

However, since all we need are the binaries and headers for the snpe-net-run command, there’s a possibility that just having the following files on a folder and adding this folder to the PATH works:

Neural network binary
snpe/bin/aarch64-linux-gcc4.9/snpe-net-run

CPU libraries

snpe/lib/aarch64-linux-gcc4.9/libSNPE.so

snpe/lib/aarch64-linux-gcc4.9/libsymphony-cpu.so

/usr/lib/aarch64-linux-gnu/libatomic.so.1

DSP libraries

snpe/lib/dsp/libsnpe_dsp_skel.so

snpe/lib/aarch64-linux-gcc4.9/libsnpe_adsp.so

Results viewer

snpe/models/alexnet/scripts/show_alexnet_classifications.py

The bold item, /usr/lib/aarch64-linux-gnu/libatomic.so.1, is provided with Linaro on this path and must be copied to this hypothetical minimal folder.

Other imprtant packages:

sudo apt-get install net-tools
sudo apt-get install gedit

sudo apt install nodejs

sudo apt install openvpn