Google Voice Recognition

From Openmoko

(Difference between revisions)
Jump to: navigation, search
(New page: Because the performance of your Freerunner is too poor for voice recognition, the Google Voice API can be used to convert an recorded Audio file into a text string. Be aware that the audi...)
 
(Future Development)
 
(34 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
Convert an Audio-File into text file via voice recognition.
  
Because the performance of your Freerunner is too poor for voice recognition, the Google Voice API can be used to convert an recorded Audio file into a text string. Be aware that the audio file will be transmitted to Google, so  need to have Internet access on your freerunner FR to submit the audio file. You must be aware of the fact, that the follow script is running on your freerunner but it is not a standalone voice recognition software and so you might not want to use this tool for personal audio files
+
{{Note|The shell script mentioned below can be used on any Linux-Operating System with some software requirements, because the speech recognition is not performed on the local machine.}}
 +
 
 +
Because the performance of your Freerunner is too poor for voice recognition, the Google Voice API can be used to convert a [http://wiki.openmoko.org/wiki/Applications#Audio recorded Audio file] into a text file. Be aware that the audio file will be transmitted to Google and the recognition is not performed on your FR. This implies, that you need to have Internet access on your freerunner FR to submit the audio file.  
 +
 
 +
{{Note|You must be aware of the fact, that the follow script is running on your freerunner but it is not a standalone voice recognition software and so you might not want to use this tool for private audio files.}}
  
 
==Google Voice API==
 
==Google Voice API==
 +
For using the Google Voice API and the script you need to have the following package installed on your freerunner:
 +
* '''SoX''' [http://sox.sourceforge.net http://sox.sourceforge.net]for converting WAV into FLAC files
 +
* '''WGET''' [http://www.gnu.org/s/wget/ http://www.gnu.org/s/wget/] for submitting the FLAC file to the Google Voice API
 +
* '''SED''' [http://sed.sourceforge.net/ http://sed.sourceforge.net/] for extracting the recognized text in the returned string of the Google Voice API
 +
Install the packages from the repositories of the freerunner [[Distributions]].
 +
 +
==Script Usage==
 +
* The script <tt>googlevoice.sh</tt> uses a the audio file <tt>message.wav</tt> in the directory of the script. All files are stored in the same directory, so you need write permissions for that directory.
 +
* SoX converts <tt>message.wav</tt>  into <tt>message.flac</tt>
 +
* wget submits the file <tt>message.flac</tt> to the Google Voice API and writes the return message to <tt>message.ret</tt>. The language variable in the script is set to German by  <tt>lang=de-de</tt>. If you want to submit a recorded file in US-English use <tt>lang=en-us</tt> instead.
 +
* SED extracts the recognized text <tt>message.ret</tt> by regular expressions and writes the text into <tt>message.txt</tt>.
 +
* Temporary files <tt>message.flac</tt> and <tt>message.ret</tt> will be deleted after the process.
 +
 +
{{Note|The return code of German audio files needs capitalization of nouns, because all words are return in small caps. A <tt>ispell</tt> or <tt>aspell</tt> correction of the <tt>message.txt</tt> might improve the recognized text.}}
 +
 +
==Basic Script Code==
 +
The script code <tt>googlevoice.sh</tt> can be tested on any Linux machine with SoX, SED, WGET installed. Modifiy the script according to your demands and storage of your audio files
 +
 +
  #!/bin/sh
 +
  echo "1 SoX Sound Exchange - Convert WAV to FLAC with 16000"
 +
  sox message.wav message.flac rate 16k
 +
  echo "2 Submit to Google Voice Recognition"
 +
  wget -q -U "Mozilla/5.0" --post-file message.flac --header="Content-Type: audio/x-flac; rate=16000" -O - "http://www.google.com/speech-api/v1/recognize?lang=de-de&client=chromium" > message.ret
 +
  echo "3 SED Extract recognized text"
 +
  cat message.ret | sed 's/.*utterance":"//' | sed 's/","confidence.*//' > message.txt
 +
  echo "4 Remove Temporary Files"
 +
  rm message.flac
 +
  rm message.ret
 +
  echo "5 Show Text "
 +
  cat message.txt
 +
 +
The parameter <tt>lang=de-de</tt> is indicating, that the Google Voice API is expecting a German language audio file. Replace <tt>lang=de-de</tt> by <tt>lang=en-us</tt> to submit an audio file in US-English.
 +
 +
== Script with Language Setting and Command Line Parameter ==
 +
The script <tt>googlevoicepar.sh</tt> with a command line parameter can be used if you want to use multiple input files for batch file recognition. You will call this script with the basename e.g. <tt>message0, message1,...</tt> by
 +
* <tt>googlevoicepar.sh message0</tt> converts <tt>message0.wav</tt> into <tt>message0.txt</tt>
 +
* <tt>googlevoicepar.sh message1</tt> converts <tt>message1.wav</tt> into <tt>message1.txt</tt>
 +
* <tt>googlevoicepar.sh message2</tt> converts <tt>message2.wav</tt> into <tt>message2.txt</tt>
 +
* ....
 +
 +
  #!/bin/sh
 +
  LANGUAGE="de-de"
 +
  echo "1 SoX Sound Exchange - Convert $1.wav to $1.flacC with 16000"
 +
  sox $1.wav $1.flac rate 16k
 +
  echo "2 Submit to Google Voice Recognition $1.flac"
 +
  wget -q -U "Mozilla/5.0" --post-file $1.flac --header="Content-Type: audio/x-flac; rate=16000" -O - "http://www.google.com/speech-api/v1/recognize?lang=${LANGUAGE}&client=chromium" > $1.ret
 +
  echo "3 SED Extract recognized text"
 +
  cat $1.ret | sed 's/.*utterance":"//' | sed 's/","confidence.*//' > $1.txt
 +
  echo "4 Remove Temporary Files"
 +
  rm $1.flac
 +
  rm $1.ret
 +
  echo "5 Show Text "
 +
  cat $1.txt
 +
 +
== Compare Performance and Results of Google Voice API with Standalone Speech Recognition on Linux ==
 +
 +
As mentioned above is speech recognition will not be performed on your Freerunner, the audio file will be submitted to a remote server and the the remote server will return an XML-File with info on accuracy and the recognized text. To compare the performance with a standalone speech recognition software, you have to test that on a Linux Box (e.g. Ubuntu)
 +
* as a text environment of a Large Vocabulary Speech Recognition [https://launchpad.net/ubuntu/maverick/+package/julius Julius] was used with the Julius-VoxForge Accoutic Model for English from the repository on Ubuntu 10.10.
 +
* Install the PPA for Ubuntu 10.10 by
 +
  sudo add-apt-repository ppa:grasch-simon-listens/simon
 +
* Update repository data
 +
  sudo apt-get update
 +
* Install OpenSource Speech Recognition Software  [http://sourceforge.net/projects/speech2text/ Simon] via Ubuntu Software-Center and configure Simon (see [http://simon.gibolles.com/doc/0.3/simon/en/index.pdf PDF-Simon-Tutorial]).
 +
  sudo apt-get install simon
 +
* Start Simon from Commandline and test a few words with your mic and record it to an audio file.
 +
  simon
 +
* Compare results with same audio file by Google Voice API and Simon.
 +
 +
{{Note|These test might help to create a remote Speech Recognition Solution based on Julius and Simon for usage on a Freerunnner. Simon can run as server. Tests can be used to connect your freerunner to Simon.}}
 +
 +
==Future Development==
 +
Google API will always need internet access like other speech recognition application for smart phones.
 +
 +
{{Note|Also other Speech recognition apps on Android (FlexT9) or on the iPhone (Dragon Natural Speaking for iPhone) are performing the transcription of an audio sample on the server and returning the transcript to the client mobile phone. That has to be done because of the limitations of the hardware on a mobile phone. This will improve in the future.}}
 +
 +
In analogy to the commercial apps for the development of [http://en.wikipedia.org/wiki/Speech_recognition_in_Linux OpenSource standalone Software on Linux] it might be good to have an OpenSource-Webinterface or an Android app to collect Audio Samples for improving the user independent Speech Recognition Profiles HMM for Speech Recognition of large vocabulary and different languages.
 +
* Users will get the speech recognition result on the freerunner or any other linux box,
 +
* The can correct the speech recognition result (transcript) and submit the correction back to the server
 +
* By the Audio-File-Submit and Text-File-Return of a server based speech recognition the Open Source Speech Recognition on Linux can be improved.
 +
* Most advanced candidate seems to [http://www.simon-listens.org Simon Listens] with server based backend (Tested on Ubuntu).
 +
'''Google Streaming Remote Engine''' - look over the chrome [http://src.chromium.org/viewvc/chrome/trunk/src/content/browser/speech/google_streaming_remote_engine.h?view=markup source]. As you can see in the long introductory comment from the sourcecode from the link, the javascript api work includes a separate streaming feature where there is a full-duplex Http connection. This allows a long running connection for chunking audio data on the submit side. No longer do you have to segment audio clips longer than 15 seconds as mentioned earlier. While there is more data, you just use the Http connection to  write another chunk on the upload channel. The download stream returns the json containing hypotheses and utterance elements.
 +
 +
'''TBD''' -  beta URL at google for the streaming-speech engine  ___________
 +
 +
==Links==
 +
* The WGET code was extended from the article on [http://www.commandlinefu.com/commands/view/8043/google-voice-recognition-api http://www.commandlinefu.com].
 +
* With your [[Android]] phone 2.2 the Google Voice API can be used by installation with Android Market.
 +
* with the [http://www.google.com/chrome Chrome Browser] on a Linux Box you can test the Google Voice API [http://slides.html5rocks.com/#speech-input http://slides.html5rocks.com/#speech-input] via a Web-Interface.
 +
* For standalone speech recognition on Linux Systems see [http://en.wikipedia.org/wiki/Speech_recognition_in_Linux Wikipedia-Article].
 +
* Web Speech API - W3C Spec [http://dvcs.w3.org/hg/speech-api/raw-file/tip/speechapi.html here]
 +
[[Category:Audio Applications]]

Latest revision as of 00:17, 23 October 2012

Convert an Audio-File into text file via voice recognition.

NOTE: The shell script mentioned below can be used on any Linux-Operating System with some software requirements, because the speech recognition is not performed on the local machine.


Because the performance of your Freerunner is too poor for voice recognition, the Google Voice API can be used to convert a recorded Audio file into a text file. Be aware that the audio file will be transmitted to Google and the recognition is not performed on your FR. This implies, that you need to have Internet access on your freerunner FR to submit the audio file.

NOTE: You must be aware of the fact, that the follow script is running on your freerunner but it is not a standalone voice recognition software and so you might not want to use this tool for private audio files.


Contents

[edit] Google Voice API

For using the Google Voice API and the script you need to have the following package installed on your freerunner:

Install the packages from the repositories of the freerunner Distributions.

[edit] Script Usage

  • The script googlevoice.sh uses a the audio file message.wav in the directory of the script. All files are stored in the same directory, so you need write permissions for that directory.
  • SoX converts message.wav into message.flac
  • wget submits the file message.flac to the Google Voice API and writes the return message to message.ret. The language variable in the script is set to German by lang=de-de. If you want to submit a recorded file in US-English use lang=en-us instead.
  • SED extracts the recognized text message.ret by regular expressions and writes the text into message.txt.
  • Temporary files message.flac and message.ret will be deleted after the process.
NOTE: The return code of German audio files needs capitalization of nouns, because all words are return in small caps. A ispell or aspell correction of the message.txt might improve the recognized text.


[edit] Basic Script Code

The script code googlevoice.sh can be tested on any Linux machine with SoX, SED, WGET installed. Modifiy the script according to your demands and storage of your audio files

 #!/bin/sh
 echo "1 SoX Sound Exchange - Convert WAV to FLAC with 16000" 
 sox message.wav message.flac rate 16k
 echo "2 Submit to Google Voice Recognition"
 wget -q -U "Mozilla/5.0" --post-file message.flac --header="Content-Type: audio/x-flac; rate=16000" -O - "http://www.google.com/speech-api/v1/recognize?lang=de-de&client=chromium" > message.ret
 echo "3 SED Extract recognized text" 
 cat message.ret | sed 's/.*utterance":"//' | sed 's/","confidence.*//' > message.txt
 echo "4 Remove Temporary Files"
 rm message.flac
 rm message.ret
 echo "5 Show Text "
 cat message.txt

The parameter lang=de-de is indicating, that the Google Voice API is expecting a German language audio file. Replace lang=de-de by lang=en-us to submit an audio file in US-English.

[edit] Script with Language Setting and Command Line Parameter

The script googlevoicepar.sh with a command line parameter can be used if you want to use multiple input files for batch file recognition. You will call this script with the basename e.g. message0, message1,... by

  • googlevoicepar.sh message0 converts message0.wav into message0.txt
  • googlevoicepar.sh message1 converts message1.wav into message1.txt
  • googlevoicepar.sh message2 converts message2.wav into message2.txt
  • ....
 #!/bin/sh
 LANGUAGE="de-de"
 echo "1 SoX Sound Exchange - Convert $1.wav to $1.flacC with 16000" 
 sox $1.wav $1.flac rate 16k
 echo "2 Submit to Google Voice Recognition $1.flac"
 wget -q -U "Mozilla/5.0" --post-file $1.flac --header="Content-Type: audio/x-flac; rate=16000" -O - "http://www.google.com/speech-api/v1/recognize?lang=${LANGUAGE}&client=chromium" > $1.ret
 echo "3 SED Extract recognized text" 
 cat $1.ret | sed 's/.*utterance":"//' | sed 's/","confidence.*//' > $1.txt
 echo "4 Remove Temporary Files"
 rm $1.flac
 rm $1.ret
 echo "5 Show Text "
 cat $1.txt

[edit] Compare Performance and Results of Google Voice API with Standalone Speech Recognition on Linux

As mentioned above is speech recognition will not be performed on your Freerunner, the audio file will be submitted to a remote server and the the remote server will return an XML-File with info on accuracy and the recognized text. To compare the performance with a standalone speech recognition software, you have to test that on a Linux Box (e.g. Ubuntu)

  • as a text environment of a Large Vocabulary Speech Recognition Julius was used with the Julius-VoxForge Accoutic Model for English from the repository on Ubuntu 10.10.
  • Install the PPA for Ubuntu 10.10 by
 sudo add-apt-repository ppa:grasch-simon-listens/simon
  • Update repository data
 sudo apt-get update
  • Install OpenSource Speech Recognition Software Simon via Ubuntu Software-Center and configure Simon (see PDF-Simon-Tutorial).
 sudo apt-get install simon
  • Start Simon from Commandline and test a few words with your mic and record it to an audio file.
 simon
  • Compare results with same audio file by Google Voice API and Simon.
NOTE: These test might help to create a remote Speech Recognition Solution based on Julius and Simon for usage on a Freerunnner. Simon can run as server. Tests can be used to connect your freerunner to Simon.


[edit] Future Development

Google API will always need internet access like other speech recognition application for smart phones.

NOTE: Also other Speech recognition apps on Android (FlexT9) or on the iPhone (Dragon Natural Speaking for iPhone) are performing the transcription of an audio sample on the server and returning the transcript to the client mobile phone. That has to be done because of the limitations of the hardware on a mobile phone. This will improve in the future.


In analogy to the commercial apps for the development of OpenSource standalone Software on Linux it might be good to have an OpenSource-Webinterface or an Android app to collect Audio Samples for improving the user independent Speech Recognition Profiles HMM for Speech Recognition of large vocabulary and different languages.

  • Users will get the speech recognition result on the freerunner or any other linux box,
  • The can correct the speech recognition result (transcript) and submit the correction back to the server
  • By the Audio-File-Submit and Text-File-Return of a server based speech recognition the Open Source Speech Recognition on Linux can be improved.
  • Most advanced candidate seems to Simon Listens with server based backend (Tested on Ubuntu).

Google Streaming Remote Engine - look over the chrome source. As you can see in the long introductory comment from the sourcecode from the link, the javascript api work includes a separate streaming feature where there is a full-duplex Http connection. This allows a long running connection for chunking audio data on the submit side. No longer do you have to segment audio clips longer than 15 seconds as mentioned earlier. While there is more data, you just use the Http connection to write another chunk on the upload channel. The download stream returns the json containing hypotheses and utterance elements.

TBD - beta URL at google for the streaming-speech engine ___________

[edit] Links

Personal tools

Because the performance of your Freerunner is too poor for voice recognition, the Google Voice API can be used to convert an recorded Audio file into a text string. Be aware that the audio file will be transmitted to Google, so need to have Internet access on your freerunner FR to submit the audio file. You must be aware of the fact, that the follow script is running on your freerunner but it is not a standalone voice recognition software and so you might not want to use this tool for personal audio files

Google Voice API