# Ball Follower Using Vision Processing

16,760

33

1

Video processing has always been challenging to robot builders as the coding is complex and people who don’t have the knowledge in MATLAB can’t do much, but in this tutorial we will be learning how to program a robot to follow a yellow coloured ball, using image processing, which works much faster than MATLAB programming and is much easier.

• For this I used VIZ bot kit from TRI, or you can make your our board which consist of a driver board or(L293D)that takes inputs from pin 5 and pin 2 of the parallel port.

• You would also need a WEB cam for capturing the video.

### Teacher Notes

Teachers! Did you use this instructable in your classroom?
Add a Teacher Note to share how you incorporated it into your lesson.

## Step 1: Set Up Roborealm

Once you have set the camera, you will be able to see the video, now place an object in front of the screen.

## Step 2: Blur!!

Now let’s add filters >Gaussian blur from left column to the command list to reduce noise. When we increase the blur the noise from the background is reduced as shown in the diagram below.

## Step 3: Color Filter

Next to filter out the ball, we add colours>RBG filter form the left column, now select the colour you want to filter.

## Step 4: COG

Now to calculate the position of the object on the screen we use analysis>centre of gravity, now we can select the option for stop tracking if pixels below 10% as shown below. And click ok.

## Step 5: More on COG...

Now we can see the blue box around the object showing the x and y axis values (i.e. at the centre the box). The values are stored in two variables COG_X and COG_Y and the size of the box is COG_BOX_SIZE which we can use in the next module, the VB script module. Modules can even be searched from the search tab available on the left column.

## Step 6: VB Module

Open the VB module and type in the code below, after selecting use the following text option.
xaxis=Getvariable("COG_X")
objsize=GetVariable("COG_BOX_SIZE")

if (objsize>10) then

if (xaxis>160) then
SetVariable "pin2",0
SetVariable "pin5",1

Else if(x axis<160) then
Set Variable "pin2",1
Set Variable "pin5",0

end if

Explanation of the code:
Get Variable is used to get the COG_X and COG_BOX_SIZE and store in variables “x axis” and “objsize” now by placing the ball in the centre of the screen we can find out the value of the object on the x-axis and size of the box. Next we have a nested ‘if loop’. The first loop checks box sixe value. If the box size is very big it means the object is very near. So we stop; but if it is a small value we check the next two ‘if conditions’, where if the x axis is greater than 160(approx centre) we declare two new variables pin5 and pin2 and make it 1 and 0 respectively and if lesser than 160 we make pin5 and pin2 as ‘0’ and ‘1’. Now click on reload and run to check if the program you wrote is functioning properly. You can even see the variables you have created on the tables in the right hand. You can now move the ball from left to right and see the variables change.

## Step 7: Connecting Your BOT...

Now to control your robot we use a parallel port module found in the path control>others>parallel-port now at pin5 and pin2. Select the variables “pin5” and “pin2” that we created. Then click on ok. Add it. Now connect the VIZ bot to the parallel port.
Alas! You have your own Vision processing robot.

if you dont have a LPT port u can use the serial port to transmit a particular string that can be processed on a Microcontroller...

## Discussions

Hello i am using this code to
control motor using arduino and matlab. But after running this program
output is always stop moving even if i move the ball in all directions.

clear;
clc % Clearing Matlab desktop
vid=videoinput('winvideo',1,'YUY2_320x240'); % Defining the video input object
set(vid,'FramesPerTrigger',1); % Setting frames per trigger
preview(vid); %////// Showing the video of the moving Ball(TO BE USED %
% WHILE TESTING)
pause(10);% Waiting for a certain time for the system to get initialised
rgb_image = getsnapshot(vid); % Storing Image in an array variable
[a b c]= size(rgb_image); % Determining the size of the captured frame.
y=a;
x=b;

% Defining Boundaries
x1=x/2-120;
x2=x/2+120;
y1=y/2-30;
y2=y/2+30;
ser=serial('COM34'); % Defining the specified COM Port to be used
fopen(ser); % starting serial Communication,opening serial port

while(1)
rgb_image = getsnapshot(vid); % storing image in an array variable
flushdata(vid); %Flushing the buffer
rbar=0;
cbar=0;
e=0;
fR=rgb_image(:,:,1);fG=rgb_image(:,:,2);fB=rgb_image(:,:,3);% Storing RGB components of the image in seperate arrays
I=((fR<=70)
& (fG>=80) & (fB<=70)); % Converting the RGB Image into
binary image///Detecting only the red component
% Following are the steps For Detecting the red ball
se=strel('disk',20);
B=imopen(I,se);
final=imclose(B,se);
[L,n]=bwlabel(final);
%imshow(rgb_image); %////THIS IS TO BE USED ONLY WHILE TESTING
%hold on % ////THIS IS TO BE USED ONLY WHILE TESTING
for k=1:n
[r,c]=find(L==k);
rbar=mean(r);
cbar=mean(c);
%plot(cbar,rbar,'Marker','*','MarkerEdgeColor','B' ,'MarkerSize',20) %////THIS IS TO BE USED ONLY WHILE TESTING
e=(((cbar>=x1)*2*2*2) + ((cbar<=x2)*2*2) + ((rbar>=y1)*2) + (rbar<=y2)) % Converting to decimal number
end
% Decision Making Conditions
switch (e)
case 5
disp('Move left'),fprintf(ser,'L');
case 6
disp('Move left'),fprintf(ser,'L');
case 7
disp('Move left'),fprintf(ser,'L');
case 9
disp('Move right'),fprintf(ser,'R');
case 10
disp('Move right'),fprintf(ser,'R');
case 11
disp('Move right'),fprintf(ser,'R');
case 13
disp('Move forward'),fprintf(ser,'F');
case 14
disp('Move back'),fprintf(ser,'B');
otherwise
disp('Stop Moving'),fprintf(ser,'S');
end
end
fclose(ser); % closing serial port