Introduction: CORE A.I. VOICE ASSISTANT (Android Version)
CHECK OUT THE WINDOWS VERSION AS WELL: HOME/LAB VOICE CONTROLLED ASSISTANT
Step 1: SETUP
The program is written in android studio
*WARNING* Before continuing you should know that this instructable won't learn you how to program in android studio, it will help you to develop your own voice controlled android application only if you already have some experience on programming!
Download and install Android Studio
Step 2: Libraries
After creating a new blank application you need to setup the libraries that we are gonna use on this project. "A library is a collection of non-volatile resources used by computer programs, often for software development. These may include configuration data, documentation, help data, message templates, pre-written code and subroutines, classes, values or typespecifications."~Wikipedia
import android.content.ActivityNotFoundException; import android.content.Intent; import android.media.MediaPlayer; import android.os.Bundle; import android.speech.RecognizerIntent; import android.speech.tts.TextToSpeech; import android.support.v7.app.AppCompatActivity; import android.view.KeyEvent; import android.view.View; import android.widget.Button; import android.widget.TextView; import android.widget.Toast; import java.io.File; import java.text.SimpleDateFormat; import java.util.ArrayList; import java.util.Arrays; import java.util.Calendar; import java.util.Locale; import java.util.regex.Pattern;
Step 3: Variables
After the libraries we should set the variables up! "A variable or scalar is a storage location (identified by a memory address) paired with an associated symbolic name (an identifier), which contains some known or unknown quantity of information referred to as a value."~Wikipedia
private Button openMic; private TextView showVoiceText; private final int REQ_CODE_SPEECH_OUTPUT = 143; String path = "/storage/sdcard1/Music/"; //Change the path to whatever path your music is located String[] fileArray; public int songNumber = 0; MediaPlayer mPlayer = new MediaPlayer(); public boolean isPlaying = false; public boolean isPaused = false; int result; TextToSpeech t1;
Step 4: Void OnCreate()
Let's take a look on the picture... As you can see void onCreate() is the very first part of the code that is getting executed!
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_home);
showVoiceText = (TextView) findViewById(R.id.showVoiceOutput);
/*Use the following command:
showVoiceText.setVisibility(View.INVISIBLE);
to make the text box invisible*/
musicScan();
openMic = (Button) findViewById(R.id.button);
openMic.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View view) {
btnToOpenMic();
}
});
t1 = new TextToSpeech(getApplicationContext(), new android.speech.tts.TextToSpeech.OnInitListener(){
@Override
public void onInit(int status){
if(status == TextToSpeech.SUCCESS){
result = t1.setLanguage(Locale.getDefault());
}else{
Toast.makeText(getApplicationContext(), "Feature not supported", Toast.LENGTH_SHORT).show();
}
}
});}Step 5: Void MusicScan()
On the previous step inside the void onCreate() we set up a void called musicScan. This void is gonna scan every file inside the "path" variable that we set on the "Variable" step.
private void musicScan(){
File f = new File(path);
File file[] = f.listFiles();
fileArray = new String[file.length];
for (int i = 0; i < file.length; ++i){
fileArray[i] = file[i].getName();
}
}*COMMENT*: IF YOU HAVE A NON MUSIC FILE IN THE PATH FOLDER THIS WOULD CAUSE THE APPLICATION TO CRASH THIS COULD BE EASILY SOLVED BY CHECKING WHETHER THE FILE ENDS WITH ".mp3" BEFORE ADDING IT IN THE ARRAY!
Step 6: Void SpeechStandby()
We want to open the google's voice services so that we can have access to it's enormous data base of words that google provides!
private void btnToOpenMic() {speechStandby();}
void speechStandby (){
if(isPlaying){
mPlayer.pause();
isPaused = true;
}
Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);</p><p> intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault());
intent.putExtra(RecognizerIntent.EXTRA_PROMPT, "Speak Now...");
try {
startActivityForResult(intent, REQ_CODE_SPEECH_OUTPUT);
} catch (ActivityNotFoundException tim) { }
}Step 7: Void ToSpeak()
In this part of the code we are making the application more interactive, by giving it the ability to speak
public void toSpeak(String toSpeak){
Toast.makeText(getApplicationContext(), toSpeak, Toast.LENGTH_SHORT).show();
int speak = t1.speak(toSpeak, TextToSpeech.QUEUE_FLUSH, null);
}Step 8: Void OnActivityResult()
This void is pretty complicated..."Starting another activity doesn't have to be one-way. You can also start another activity and receive a result back. To receive a result, call startActivityForResult() (instead of startActivity()).
For example, your app can start a camera app and receive the captured photo as a result. Or, you might start the People app in order for the user to select a contact and you'll receive the contact details as a result. Of course, the activity that responds must be designed to return a result. When it does, it sends the result as another Intent object. Your activity receives it in the onActivityResult() callback." ~developer.android.com
The text "Core A.I." is getting replaced with the voice command, so this voice is checking what the new text is and executes a piece of code based on what we said. *You can make the text box invisible if you want to however having it visible is an easier way to debug the application*
@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data){
super.onActivityResult(requestCode, resultCode, data);
restartMusic();
switch (requestCode){
case REQ_CODE_SPEECH_OUTPUT:{
if (resultCode == RESULT_OK && null != data){
ArrayList<string> voiceInText = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);
showVoiceText.setText(voiceInText.get(0));
String command = showVoiceText.getText().toString();
if(command.equals("exit")) {
finish();
System.exit(0);
}else if(command.equals("music")){
if(!isPlaying) {
mPlayer.reset();
playMusic(path + fileArray[songNumber]);
showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", ""));
}
}else if(command.equals("stop music")){
if(isPlaying) {
stopMusic();
}
}else if(command.equals("next track") || (command.equals("next song"))){
if(isPlaying) {
songNumber = songNumber + 1;
if(songNumber > fileArray.length - 1){
songNumber = 0;
}
mPlayer.reset();
isPlaying = false;
playMusic(path + fileArray[songNumber]);
showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", ""));
}
}else if(command.equals("previous track") || (command.equals("previous song"))){
if(isPlaying) {
songNumber = songNumber - 1;
if(songNumber < 0){
songNumber = fileArray.length - 1;
}
mPlayer.reset();
isPlaying = false;
playMusic(path + fileArray[songNumber]);
showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", ""));
}
}else if(command.equals("music list")){
showVoiceText.setText(Arrays.toString(fileArray) + " ");
}else if(command.equals("what time is it")){
time();
}else if(command.equals("what day is today")){
date();
}else if(command.toLowerCase().contains("play")){
String inputSong = (command.substring(command.lastIndexOf("play") + 5));
String inputSongLowerCase = inputSong.toLowerCase();
searchSong(inputSongLowerCase);
}
}
}break;
}
}Step 9: Void Time() and Void Date()
These voids are getting the time and the date whenever they are call from the void onActivityResult()!
public void time(){
Calendar calendar = Calendar.getInstance();
SimpleDateFormat mdformat = new SimpleDateFormat("HH:mm");
String strTime = "The time is : " + mdformat.format(calendar.getTime());
toSpeak(strTime);
}public void date(){
Calendar calendar = Calendar.getInstance();
SimpleDateFormat mdformat = new SimpleDateFormat("dd-MMM-yyyy");
String strDate = "Today is: " + mdformat.format(calendar.getTime());
toSpeak(strDate);
}Step 10: Void SearchSong()
Once the program gets the voice command "PLAY" it will expect from you to tell it the song that you want it to play. Before playing it should search if the specific song actually exists on the song array that we set up previously.
public void searchSong(String songToSearch){
try { Toast.makeText(getApplicationContext(), songToSearch, Toast.LENGTH_SHORT).show(); for (int i = 0; i < fileArray.length; i++) { String value = fileArray[i].toLowerCase().replace(".mp3", ""); if (Pattern.compile(Pattern.quote(songToSearch), Pattern.CASE_INSENSITIVE).matcher(value).find()) { songNumber = i; isPlaying = false; mPlayer.reset(); playMusic(path + fileArray[songNumber]); showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", "")); break; } } } catch (IllegalArgumentException e) { e.printStackTrace(); } catch (Exception e) { System.out.println("Exception of type : " + e.toString()); e.printStackTrace(); } }
Step 11: Void PlayMusic()
Now last but not least, we need the void that will play the song that we asked for!
private void playMusic(String song){
if(!isPlaying) { try { mPlayer.setOnCompletionListener(new MediaPlayer.OnCompletionListener() { @Override public void onCompletion(MediaPlayer mPlayer) { isPlaying = false; mPlayer.reset(); if(songNumber > fileArray.length - 1){ songNumber = 0; }else { songNumber = songNumber + 1; } playMusic(path + fileArray[songNumber]); showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", "")); }}); Toast.makeText(getApplicationContext(), song, Toast.LENGTH_SHORT).show(); mPlayer.setDataSource(song); mPlayer.prepare(); mPlayer.start(); isPlaying = true; } catch (IllegalArgumentException e) { e.printStackTrace(); } catch (Exception e) { System.out.println("Exception of type : " + e.toString()); e.printStackTrace(); } //showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", "")); }else{ stopMusic(); } }
Step 12: Void StopMusic() and Void RestartMusic()
We need a void that will stop the music once we ask for it. On the void speechStandBy we were pausing the music so that we could as for the next command we need a void to restart the music after the application gets the void command!
void restartMusic(){
if(isPlaying && isPaused){ mPlayer.start(); isPaused = false; } }private void stopMusic(){
if(isPlaying) { songNumber = 0; isPaused = false; isPlaying = false; mPlayer.pause(); } }
Step 13: Void OnDestroy()
This is a pretty simple code that will prevent the application from crashing when we are closing it
public void onDestroy(){
if(t1 != null){
t1.stop();
t1.shutdown();
}
super.onDestroy();
}
Step 14: Raw Code (Copy-Paste Version)
import android.content.ActivityNotFoundException;
import android.content.Intent; import android.media.MediaPlayer; import android.os.Bundle; import android.speech.RecognizerIntent; import android.speech.tts.TextToSpeech; import android.support.v7.app.AppCompatActivity; import android.view.KeyEvent; import android.view.View; import android.widget.Button; import android.widget.TextView; import android.widget.Toast;import java.io.File; import java.text.SimpleDateFormat; import java.util.ArrayList; import java.util.Arrays; import java.util.Calendar; import java.util.Locale; import java.util.regex.Pattern;
public class HomeActivity extends AppCompatActivity {
private Button openMic; private TextView showVoiceText; private final int REQ_CODE_SPEECH_OUTPUT = 143;
String path = "/storage/sdcard1/Music/"; String[] fileArray; public int songNumber = 0;
MediaPlayer mPlayer = new MediaPlayer(); public boolean isPlaying = false; public boolean isPaused = false; int result; TextToSpeech t1;
@Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_home);
showVoiceText = (TextView) findViewById(R.id.showVoiceOutput); musicScan();
openMic = (Button) findViewById(R.id.button);
openMic.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View view) { btnToOpenMic(); } });
t1 = new TextToSpeech(getApplicationContext(), new android.speech.tts.TextToSpeech.OnInitListener(){ @Override public void onInit(int status){ if(status == TextToSpeech.SUCCESS){ result = t1.setLanguage(Locale.getDefault()); }else{ Toast.makeText(getApplicationContext(), "Feature not supported", Toast.LENGTH_SHORT).show(); } } });
}
private void btnToOpenMic() { speechStandby(); }
void speechStandby (){ if(isPlaying){ mPlayer.pause(); isPaused = true; } Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH); intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault()); intent.putExtra(RecognizerIntent.EXTRA_PROMPT, "Speak Now...");
try { startActivityForResult(intent, REQ_CODE_SPEECH_OUTPUT); } catch (ActivityNotFoundException tim) {
} }
void restartMusic(){ if(isPlaying && isPaused){ mPlayer.start(); isPaused = false; } }
@Override protected void onActivityResult(int requestCode, int resultCode, Intent data){ super.onActivityResult(requestCode, resultCode, data); restartMusic(); switch (requestCode){ case REQ_CODE_SPEECH_OUTPUT:{ if (resultCode == RESULT_OK && null != data){ ArrayList voiceInText = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS); showVoiceText.setText(voiceInText.get(0)); String command = showVoiceText.getText().toString(); if(command.equals("exit")) { finish(); System.exit(0); }else if(command.equals("music")){ if(!isPlaying) { mPlayer.reset(); playMusic(path + fileArray[songNumber]); showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", "")); } }else if(command.equals("stop music")){ if(isPlaying) { stopMusic(); } }else if(command.equals("next track") || (command.equals("next song"))){ if(isPlaying) { songNumber = songNumber + 1; if(songNumber > fileArray.length - 1){ songNumber = 0; } mPlayer.reset(); isPlaying = false; playMusic(path + fileArray[songNumber]); showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", "")); }
}else if(command.equals("previous track") || (command.equals("previous song"))){ if(isPlaying) { songNumber = songNumber - 1; if(songNumber < 0){ songNumber = fileArray.length - 1; } mPlayer.reset(); isPlaying = false; playMusic(path + fileArray[songNumber]); showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", "")); }
}else if(command.equals("music list")){ showVoiceText.setText(Arrays.toString(fileArray) + " "); }else if(command.equals("what time is it")){ time(); }else if(command.equals("what day is today")){ date(); }else if(command.toLowerCase().contains("play")){ String inputSong = (command.substring(command.lastIndexOf("play") + 5)); String inputSongLowerCase = inputSong.toLowerCase(); searchSong(inputSongLowerCase); } } }break; } }
private void playMusic(String song){ if(!isPlaying) { try { mPlayer.setOnCompletionListener(new MediaPlayer.OnCompletionListener() { @Override public void onCompletion(MediaPlayer mPlayer) { isPlaying = false; mPlayer.reset(); if(songNumber > fileArray.length - 1){ songNumber = 0; }else { songNumber = songNumber + 1; } playMusic(path + fileArray[songNumber]); showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", "")); }
}); Toast.makeText(getApplicationContext(), song, Toast.LENGTH_SHORT).show(); mPlayer.setDataSource(song); mPlayer.prepare(); mPlayer.start(); isPlaying = true; } catch (IllegalArgumentException e) { e.printStackTrace(); } catch (Exception e) { System.out.println("Exception of type : " + e.toString()); e.printStackTrace(); } //showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", "")); }else{ stopMusic(); } }
private void stopMusic(){ if(isPlaying) { songNumber = 0; isPaused = false; isPlaying = false; mPlayer.pause(); } }
private void musicScan(){ File f = new File(path); File file[] = f.listFiles(); fileArray = new String[file.length]; for (int i = 0; i < file.length; ++i){ fileArray[i] = file[i].getName(); } }
public void toSpeak(String toSpeak){ Toast.makeText(getApplicationContext(), toSpeak, Toast.LENGTH_SHORT).show(); int speak = t1.speak(toSpeak, TextToSpeech.QUEUE_FLUSH, null); }
public void time(){ Calendar calendar = Calendar.getInstance(); SimpleDateFormat mdformat = new SimpleDateFormat("HH:mm"); String strTime = "The time is : " + mdformat.format(calendar.getTime()); toSpeak(strTime); }
public void date(){ Calendar calendar = Calendar.getInstance(); SimpleDateFormat mdformat = new SimpleDateFormat("dd-MMM-yyyy"); String strDate = "Today is: " + mdformat.format(calendar.getTime()); toSpeak(strDate); }
public void onDestroy(){ if(t1 != null){ t1.stop(); t1.shutdown(); } super.onDestroy(); }
public void searchSong(String songToSearch){ try { Toast.makeText(getApplicationContext(), songToSearch, Toast.LENGTH_SHORT).show(); for (int i = 0; i < fileArray.length; i++) { String value = fileArray[i].toLowerCase().replace(".mp3", ""); if (Pattern.compile(Pattern.quote(songToSearch), Pattern.CASE_INSENSITIVE).matcher(value).find()) { songNumber = i; isPlaying = false; mPlayer.reset(); playMusic(path + fileArray[songNumber]); showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", "")); break; } } } catch (IllegalArgumentException e) { e.printStackTrace(); } catch (Exception e) { System.out.println("Exception of type : " + e.toString()); e.printStackTrace(); } }
Step 15: Thanks for You Time!
It is really important to leave a feedback in the comment section to help me improve!
