Introduction: A Simple Android UI Joystick
Hey all! This is an instructable about creating a simple joystick in Android using just a SurfaceView. I was planning on using something like this to control a robot via Bluetooth, but unfortunately couldn't really find a whole lot online about how exactly to put one of these together so I decided to make one from scratch. It's pretty bare-bones, but it gets the job done.
This Instructable assumes that you have some basic knowledge of Java, Object-Oriented Programming and some previous experience with Android development.
The Instructable will go in-depth about how all of the code functions while providing you with the instructions for creating your own joystick. If you just want a simple, completed version you can integrate into your app, you can download my completed version at: https://github.com/efficientisoceles/JoystickView
Just follow the steps on the GitHub page to test it out.
This is my first instructable, and it might not be too clear, so if there's any questions that you may have, feel free to leave a comment and I'll respond as soon as I can!
Step 1: Setting Up
This "Setup" portion is basically meant for Android studio; adapt it for your own compiler should you choose to use it.
As with any new app, give it an appropriate name. I called mine "JoystickTest", since, well, that's exactly what it was. You'll be able to export the joystick into other projects anyways after you're done by copying the class over.
For the version, I believe that anything past 2.2 would work fine. I used 4.2 though, as it's the same version as the app I was adding the Joystick to.
Finally, it'll ask you for an Activity to put in. I used an empty activity, as there's no need for any fancy extra components when the layout XML will basically not at all be needed. (We'll go into this later on). Name it MainActivity for easy reference.
Once the project has been generated, you should have a single java class called MainActivity.
Step 2: SurfaceView Constructors/More Setup
The SurfaceView widget is the base of the joystick that we will build upon. It provides access to a drawing surface and methods that we can use to detect the screen being touched.
Start by creating a new object. Name it to reflect the use - I named mine JoystickView. Once created, have the class extend SurfaceView. This allows the class to access all the fields and methods of SurfaceView, while also allowing it to be added to the UI just like a SurfaceView. After adding the "extends", your development environment should give you a code error about having incorrect constructors. You should override all three constructors of the SurfaceView class. Generally, you won't need to add anything extra to them. Just call "super" with the arguments inside the constructor for now; we'll add more later as we implement features.
The constructors you'll need to override are:
public SurfaceView (Context c)
public SurfaceView (Context c, AttributeSet a, int style)
public SurfaceView (Context c, AttributeSet a)
Note that it's not 100% necessary to override all three - you can get away with only overriding the first one. By overriding all three though, you'll be able to directly add the joystick into an XML layout, making it easier to implement into an UI.
We will also need a few callbacks to initialize stuff at the proper moment in the SurfaceView's lifecycle. Callbacks are a way that an object within a class can notify the class that a specific event has occurred within it. This can be that the object has completed loading (Say, if it were a UI element) or that it has completed a specific function. In Android, this is commonly done through Interfaces. The class simply implements the interface that the object has, adding the methods within the interface, and then the object calls said method when some event occurs. We need to implement the SurfaceView.CallBack interface. Add "implements SurfaceView.Callback" to the class declaration of your java file, right past the "extends SurfaceView". This will add the following three callback methods:
public void surfaceCreated(SurfaceHolder holder)
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height)
public void surfaceDestroyed(SurfaceHolder holder) If you're using Android Studio, it should prompt you to add the methods immediately. In each of the three constructors we setup previously, also add: getHolder().addCallback(this); Which will set the callback methods in this class to be the ones to be called when those events happen.
Out of these three callback methods, we only really care about surfaceCreated, since this method is called when the SurfaceView has been fully created and has all the dimensions laid out and is ready for drawing.
We'll have to change the MainActivity.java file a bit in order to easily debug the JoystickView. Create a new instance of JoystickView in the onCreate method of MainActivity, using the SurfaceView(Context c) constructor (Setting the context as "this"). Then, replace the stuff in brackets of the "setContentView" method with the name of your new joystick instance. This just makes it show only the JoystickView by default when your test app boots up.
Step 3: Drawing the Joystick
First off, a brief explanation of how drawing on a SurfaceView in Android actually works:
- Drawing on SurfaceViews in Android is done through the Canvas object, which holds all the shapes/images to be displayed to the user.
- Drawing on the canvas is like drawing on a Cartesian plane. It has an X and a Y axis, which increases to the left and down, respectively, and use pixels as units. Shapes are placed at coordinate points on this plane.
- Drawing on the canvas is ADDITIVE, meaning that when two shapes overlap, the shape that was drawn the latest will hide the shape that was drawn with an earlier command. This means you must always draw the base first.
- Pixels on the canvas are given relative to the size of your device screen. For example, a canvas that takes up the width of the device horizontally and 1/2 the height vertically on a 1280 * 720 screen would NOT have the same width and height in pixels as a 1920 * 1080 screen. The first one would have WxH 1280 * 360 while the second would have 1920 * 540, even though they take up the same amount of the screen on both devices.
Now onto the actual programming.
Global Variables and Dimension Setup
Since canvases have their length and width defined in terms of how many pixels they occupy on the screen, it would be unwise to hard-code the coordinates of the joystick or its radius as it'll maintain the same position and size on the canvas even if the SurfaceView changes in size. To make it automatically fit with the size of the canvas, we have to use global variables and calculate the joystick positions and sizes at runtime.
We will need four global variables, as follows:
We will also need a method, void setupDimensions(), to assign the variables their values. Within it, put the following:
centerX = getWidth() / 2;
centerY = getHeight() / 2;
baseRadius = Math.min(getWidth(), getHeight()) / 3;
hatRadius = Math.min(getWidth(), getHeight()) / 5;
This code simply assigns values to each of the variables as a ratio of the width and height of the SurfaceView, accessed using the methods "getWidth()" and "getHeight()". The Math.min ensures that the joystick will always fit inside the SurfaceView, even if one dimension is greatly smaller than the other. You can go ahead and assign your own ratios for each of these, the ones here are just suggested for a completely centered joystick.
Once this is done, add the setupDimensions() method to the surfaceCreated method to get the proper placement of your joystick. We call setupDimensions() in this method because at this point, we know that the SurfaceView has been initialized and thus trying to get its properties would not result in any exceptions.
The Drawing Method
The method to draw the joystick is actually pretty simple. Create a new method accepting two float arguments - newX and newY:
private void drawJoystick(float newX, float newY)
newX and newY will be used to specify the location of the hat (the top of the joystick) on the SurfaceView.
Now, remember that all drawing on the SurfaceView is done through its Canvas object. You'll need to get access to this Canvas in order to draw the joystick. You can do so by calling:
Canvas myCanvas = this.getHolder().lockCanvas();
This returns a Canvas object, which you should assign to a variable for easier reference.
You'll also need a Paint object, which represents the color that you're currently drawing with. Just create and instantiate it.
Paint colors = new Paint();
Now you're ready to draw. First, clear the canvas of everything on it:
Next, we draw the base. This is a simple circle that always stays in the same place. First, set the color by calling:
colors.setARGB(int alpha, int red, int green, int blue);
Each of those variables must be ints between 0 and 255. The higher, the more of that color will be in. Alpha determines transparency, with 255 being solid and 0 being invisible. For a more visual version of this, just google "RGB Color Picker". For my joystick, I used a light gray, with the aRGB values (255, 50, 50, 50), but feel free to use your own.
Now you can draw the base. Call the drawCircle method of your Canvas. This method requires 4 arguments in this order: (x position of center as float, y position of center as float, radius of the circle as float, color of the circle as a paint object). We already defined three of these in the previous section:
myCanvas.drawCircle(centerX, centerY, baseRadius, color)
Next, draw the hat. Call the setARGB method of your Paint object again to set a different color for the hat. I used a pure blue, with the aRGB combo (255, 0, 0, 255). Then call the drawCircle method, only this time use newX and newY as the coordinates to draw the hat and use the global variable for the radius of the hat.
myCanvas.drawCircle(newX, newY, radiusHat, color)
The joystick has now been drawn, but it has not been printed to the SurfaceView and is not visible to the user. To do this, call:
where myCanvas is the name of your Canvas variable.
Finally, you should surround the entire method with the following "if" statement:
This if statement prevents the drawing method from executing when the SurfaceView has not been created on-screen, preventing exceptions at runtime.
Once this method is done, append it to the surfaceCreated method that we overrode in the previous section, with newX = centerX and newY = centerY. This will draw a joystick with the top in the neutral position as soon as the app starts. Without this, you would just see a black box until the method is called.
Congratulations! You've drawn a joystick. However, it won't really do anything until we configure the panel to accept user input.
Step 4: Adding Interactivity
To allow the user to interact and move the joystick, we will have to implement the OnTouchListener interface. This interface forces us to add the onTouch method, which is automatically called when the user touches the screen (Or really interacts with it in any way). Using it, you can respond to the touch event however we need to. In our case, it would be moving the joystick to an appropriate position.
First, implement the OnTouchListener by adding ", View.OnTouchListener" to the class declaration past the "implements". In java, you can implement as many interfaces as you want, but they must be separated by commas. Add the following implement method as well:
public boolean onTouch(View v, MotionEvent e)
This will override the onTouch method in the OnTouchListener interface. This method will be called whenever the user touches the screen, giving us the View that the user touched and the way they touched it (e.g. tapped the screen, moved their finger while on the screen, let go of the screen, etc) through its two arguments. Within the method, we'll have to add our own code to respond to the user touching it.
Within the method, first add the following "if" statement
This makes sure that the touch listener only accepts touches coming from this SurfaceView. Within that if statement, add another one:
if(e.getAction != e.ACTION_UP)
This checks that the touch event is NOT the user lifting their finger off the touch screen. We need to do this to make sure that the joystick only moves as long as the user is touching the screen, and resets to its original position when the user lets go (Just like a real joystick). Within this "if" condition, call your joystick drawing method that we made in the last step:
The getX() and getY() methods give you the X and Y coordinates, respectively, in pixels where the user touched the screen. Sending them to the drawJoystick method causes the hat of the joystick to be drawn at those positions.
Next, add a "else" statement after the if. This else statement only executes if the opposite of the if statement is true, that is if "e.getAction == e.ACTION_UP", aka the joystick is released. Within it, place:
This resets the joystick to its center position when the user lets go.
Finally, add "return true;" to the end of the method, outside the if statements. We return true as returning false prevents the onTouch method from receiving future touches.
That's about it for this method for now. Go back to each of the constructors, and add the following line to them:
Adding this line will cause the SurfaceView to use the onTouch method from this class to handle user screen touches from now on.
At this point, when you run your app, you should be able to tap on the screen and move the joystick around.
Step 5: Constraining the Joystick
If you tested the app, you'll probably have noticed that the top of the joystick can fly way off the base of the joystick, which is obviously not how a real joystick should behave! The joystick hat should never leave the base, otherwise it looks unrealistic and might overlap with other views you have in the app.
Checking for Out of Bounds... Mathematically
To fix this, we'll have to add a few things to the onTouch method to check if the user is clicking OUTSIDE the bounds of the base of the joystick, and if so draw the joystick hat so that it's within the bounds of the base but still in the same direction the user is tapping (so that tapping slightly outside the joystick base doesn't make the thing not register). If this doesn't make sense, have a look at some of the attached pictures.
Checking that the user is within bounds is the easy part of this. We can simply calculate the total displacement of the joystick from its resting position, and compare that to the radius of the joystick. If the displacement is larger than the radius, then the user must be moving the joystick outside the bounds of the base. This is as the base is a circle, and the displacement of the edge of the base relative to the center of the circle at any point along it is always constant. Once again, take a look at the diagrams for more info.
To calculate this displacement, we have to use the Pythagorean theorem:
a^2 + b^2 = c^2, a b and c form a triangle.
Let's say the user clicked points have the co-ordinates (x', y') and the center (x, y). Let's also say a is the change in the x position and b is the change in the y position. c would therefore be the net displacement from the center. c could therefore be given as:
sqrt((x'-x) ^ 2 + (y'-y)^2)
Or in Java terms, specific for our program:
float displacement = (float) Math.sqrt(Math.pow(e.getX() - centerX, 2) + Math.pow(e.getY() - centerY, 2));
Where e is the MotionEvent in our onTouch method. Add that bit of code at the top of the (e.getAction() != e.ACTION_UP) if statement. We're placing it here because we only need to test that the user is moving the stick within bounds only when the user is actually moving it, which means only when the user is touching the SurfaceView.
Add another if statement before the drawJoystick, to check if the net displacement is less than the base radius. If it is, then the click is valid and we don't have to constrain the joystick hat.
if(displacement < baseRadius)
Add an else statement as well. This handles the case where the displacement is greater than the base radius, meaning we have to constrain the joystick to the base radius.
Constraining the Joystick
To do the constraining, we will use the parallel triangles identity. This basically says that if you have two triangles with two parallel sides and one of the same angle, the ratios of all three sides will be the same. A better explanation can be found here:
Since we want to keep the angle for the constrained joystick the same as it would be for the out-of-bounds joystick, we know the angle will have to be the same. We also know that both of the formed triangles will have to be right triangles, as the arms will always be formed out of (x' - x) and (y' - y), aka the displacements along the x and y axis relative to the center point. Therefore, two sides at the minimum will be parallel and the identity will take effect. Now we can find the lengths of the other two displacements by dividing the known displacements of the hypotenuses.
The ratio of the hypotenuses can be found by:
float ratio = baseRadius / displacement;
We divide by displacement since we want the other to be the new net displacement as it constrains the joystick to the base. Now multiply the values of the x and y displacements to get the new displacements so that the joystick is constrained to the base. Make sure you add centerX and centerY to the appropriate value, since these are the new displacements relative to the center.
float constrainedX = centerX + (e.getX() - centerX) * ratio;
float constrainedY = centerY + (e.getY() - centerY) * ratio;
Then call the drawJoystick method on the constrained values to draw the new joystick. If you did everything correctly, clicking outside the borders of the joystick base will still cause the joystick to move, but it will always remain within the base.
If any of this is confusing, hopefully the attached diagrams will show it better visually. After adding the drawJoystick, test out your app. The joystick should stick within the base now no matter where you touch the screen.
Step 6: Interacting With Other Activities
Okay, great, so now you have a basic but fully functional joystick that you can move around! But, you can't do anything yet. There's no way you can interact with it from another activity if you needed to. That's where adding the callback methods come in.
Remember when we went over callback methods a few steps ago? We need one now so that the joystick can report it being touched, and how its being touched to whatever other thing you need to use it in. To create this callback method, you'll first have to declare an Interface within the JoystickView:
public interface JoystickListener
void onJoystickMoved(float xPercent, float yPercent, int source);
You can really put anything you want in the arguments. You can put in the position it was clicked, the total click displacement, etc. I put in the percentage it was moved out of its total possible displacement as I was planning on using it to control motor power. You need to have the "source" int at the end if you're planning on having multiple joysticks, since that way you can differentiate between them.
Next, create a global instance of JoystickListener called "joystickCallback". In each of the constructors, add:
if(context instanceof JoystickListener)
joystickCallback = (JoystickListener) context;
Where joystickCallback is the global JoystickListener and context is the context parameter in the constructor.
This allows us to call the onJoystickMoved method in the class representing the activity that contains this joystick, provided it has implemented the JoystickListener and has the appropriate onJoystickMoved method. At any time, you can call
joystickCallback.onJoystickMoved(xPercent, yPercent, getId());
for example, and it would call the onJoystickMoved method in whatever happens to be implementing the listener. It would be most effective to place this in the onTouch method, as then it can relay information about how the joystick has moved to its parent method. You'll have to make three different calls to this method. Note that for this section, I will be using my implementation of returning a percentage.
In if(displacement < baseRadius), you should call it as:
joystickCallback.onJoystickMoved((e.getX() - centerX) / baseRadius, (e.getY() - centerY) / baseRadius, getId());
Since the distance that the stick is pulled on each axis will vary. In the else statement following, I would put:
joystickCallback.onJoystickMoved((constrainedX - centerX) / baseRadius, (constrainedY - centerY) / baseRadius, getId());
This is for the same reason as above. In the else statement directly after that, I would put:
joystickCallback.onJoystickMoved(0, 0, getId());
since the else condition only executes when the user lets go of the screen and the joystick moves back to neutral position.
The joystick is now ready to be used inside Activities. To test it, go back to your MainActivity class. Add "implements JoystickView.JoystickListener" and add the callback method to it. Inside the callback method, add
Log.d("Main Method", "X percent: " + xPercent + " Y percent: " + yPercent);
xPercent, yPercent are the arguments passed from JoystickView. Now when you move the joystick in the test app, you should see in the Android debugger percentages of how far the joystick is pushed in each axis. Y percentage increases going downward, and X percentage should increase going to the right. Negative percentages just mean you're pushing it in the other direction (left instead of right, for example). If everything looks correct, congratulations! You've succeeded in creating an Android joystick.
Step 7: Adding the JoystickView Via XML
One of the really great things about having this joystick extend a SurfaceView is that you can add it in directly through an XML. You can also set info about it just like any other view, setting the name, id, etc. We can test this out by going to the activity_main.xml and editing the layout.
If you're using Android Studio, the activity_main.xml should by default just be a blank RelativeLayout. I changed this to a LinearLayout with android:orientation="vertical" for the sake of simplicity. We can add the JoystickView to the layout via XML by adding:
within the LinearLayout, where your_package_name is the name of the package that you specified on creating the project. Like any other view, to make it work with Android you'll have to add a layout_width and a layout_height field.
I just used 300dp because it seemed reasonable. Of course, you could use layout weights and such since this is a LinearLayout, but for the sake of simplicity I'm hardcoding the size. In the XML preview, you should now see a large gray box that is labelled "JoystickView" (That is, if you're using Android Studio). Now you can go back to your MainActivity.java file and replace the parameter in the setContentView with your XML layout (In Android Studio, R.layout.activity_main). Running your app now will display the joystick, only scaled down to fit the dimensions you specified in the layout.
At this point, you're basically done. Congratulations! You've managed to create an Android joystick. To use this joystick in other apps, simply copy over the JoystickView java file and change the package name so that it fits with the rest of your new app. To use the joystick in other Activity layouts, just repeat what we did in this section in the new activity XML.
The rest of the steps after this is just non-essential stuff. We'll be going over how to do shading with the joystick, and how to implement multiple joysticks in one activity.
Step 8: Extras: Multiple Joysticks, 1 Activity
When you have a single joystick in your activity, receiving input is simple. When you have multiple joysticks, it gets a bit more complex. Because of the way we implemented the joystick callback, ALL the joysticks in your activity will communicate with the activity by calling the same onJoystickMoved method. To handle them correctly, you'll have to differentiate between them.
Fear not! This is actually super simple. When adding the joysticks in the XML, you simply have to give each joystick a value for its id tag. In this example, I added two joysticks. I gave the first joystick the id of:
And the second the id of:
Then, in the onJoystickMoved method, add a switch statement to differentiate between the joysticks and handle them:
Log.d("Right Joystick", "X: " + xPercent + " Y: " + yPercent);
Log.d("Left Joystick", "X: " + xPercent + " Y: " + yPercent);
Android assigns each id a specific int value, which can be accessed by calling "R.id.view_name_here". Calling getId() in a view with an assigned name returns the value specific to itself as well. Since we programmed the callback earlier to send this specific value to the Activity it's attached to, and we already know all the ids of the joysticks that we assigned, we can just use a switch statement to parse through them. That's it! Now give the app a run, and try moving each of the joysticks about.
Step 9: Extras: Prettier, Shaded Joysticks
Although at this point the joystick works, it does look rather.... dull. We can quickly remedy that by modifying our drawing method so that it does a bit of shading. The end product of this is a joystick that not only has a nice color gradient on it, but also has a proper stalk!
Changing Colors - The Hat
To begin, we'll have to surround the setARGB and the drawCircle parts of our drawJoystick method, both for the base and the hat, with "for" loops. They should take the following form:
for(int i = 1; i <= baseRadius / RATIO; i++)
for(int i = 1; i <= hatRadius / RATIO; i++)
RATIO is a constant that you should declare. It allows us later on to adjust the amount of shading to add to the joystick. Remember that more shading = higher load on the phone! We make the length of the loop relative to the radius of the base/hat as larger joysticks will require more shading to look proper.
Next, we'll gradually adjust the color for both of these to create a sort of shading effect. We'll start with the Joystick Hat, as it is simpler. Here, we want to shift the color from the original deep blue of the joystick to a lighter white, in order to emulate the light shining off the top. To do this, we simply make it so that as i increases, so the the red and green values, while the value of blue stays the same at 255. The makes the blue become lighter and lighter. Replace the setARGB method in the loop with:
colors.setARGB(255, (int) (i * (255 * RATIO/hatRadius)), (int) (i * (255 * RATIO/hatRadius)), 255);
Why multiply i by (RATIO/hatRadius)? This ensures that at the end of the loop, a value of 255 will be produced for both red and green, resulting an overall color of white. The loop ends when i = hatRadius/RATIO, so the final value of i will be hatRadius/RATIO which cancels out with (RATIO/hatRadius), resulting in just 255.
Changing Colors - The Base
This time, we use an overlapping shading technique. We reduce the value of alpha so that the drawn shapes are somewhat seethrough, and then layer the shapes on top of each other. The shapes that overlap will blend, and create a hue that is closer to the shading color. Replace the setARGB method in this loop with:
colors.setARGB(150/i, 255, 0, 0);
You can set the 3 latter arguments to whatever you want, they just define the color of the shading hue. We're using this loop to draw a stalk for the joystick anyhow, and the color doesn't matter for that. I used a red stalk. The 150/i ensures that as the loop nears completion, the alpha approaches zero and the bits closest to the end of the stalk become less visible
Shading - The Hat
Just replace the third argument in the drawCircle method with:
hatRadius - (float) i * (ratio) / 3
All this does is gradually decrease the drawing radius of the shading, until it reaches its minimum of 1/3 the radius of the joystick hat. This 1/3 radius is the shiny white portion at the top.
Shading - The Stalk
The joystick stalk is a bit more complex. We want to keep the base of the joystick stalk exactly on the center of the joystick base, while at the same time making a sort of a perspective effect with the shading to create the stalk. We also want to make it so that more or less of the stalk is visible depending on how far the user pushes the joystick, and to make it draw the stalk so that it angles the same way as the joystick hat.
To solve the angle problem, I added the following three lines of code:
float hypotenuse = (float) Math.sqrt(Math.pow(newX - centerX, 2) + Math.pow(newY - centerY, 2));
float sin = (newY - centerY) / hypotenuse; //sin = o/h
float cos = (newX - centerX) / hypotenuse; //cos = a/h
This calculates the sine and cos of the angle that the hat of the joystick makes with the center of the base. We can multiply a number by the cos and sin to get the x and y end coordinates that has a different length than the line the joystick base makes with the hat, but the same angle. This comes in handy inside the for loop, as we need to gradually shift the shading circles from beneath the joystick stalk towards the center, while keeping the center of all the circles along the same line.
The final drawCircle method in the for loop to draw the stalk looks like:
myCanvas.drawCircle(newX - cos * hypotenuse * (ratio/baseRadius) * i, newY - sin * hypotenuse * (ratio/baseRadius) * i, i * (hatRadius * ratio / baseRadius), colors);
We use the cos and sin that we calculated earlier to center the circles across the imaginary line between the joystick center and the hat. We multiply that by the hypotenuse, and then by i to ensure that the circles will be drawn at equal sections upon the line, and then multiply that by (ratio/baseRadius) * i so as to ensure that the final circle will be drawn at the base of the circle (Remember that the loop goes up to baseRadius/RATIO, so it'll cancel out to just hypotenuse * cos or hypotenuse * sin)
We also gradually grow the shading as we get closer to the center of the base, by multiplying hatRadius * ratio / baseRadius by i. Eventually i will equal baseRadius / ratio, so we just end up with a maximum size for a shading circle of hatRadius. Of course, you can resize this maximum size as needed. I just found that the hatRadius ended up making a nice-looking circle. Word of cauation; if you increase the size too much though (i.e. to an amount bigger than baseRadius), it WILL appear outside the base.
You can just go ahead and replace the drawCircle in the loop for drawing the base with the one shown above.
That's about it! Go ahead and launch the app to test your now prettily-shaded joystick.
This will also conclude our instructable. If you completed all of this, you should have a really quite nice-looking joystick that works pretty well too. I would like to thank you for sticking through this instructable, and I hope you learned something!