Aphasia

From Openmoko

(Difference between revisions)
Jump to: navigation, search
(Why Erlang?)
m (Are you a User)
 
(111 intermediate revisions by one user not shown)
Line 1: Line 1:
Three years ago Jane had a stroke which made her unable to [http://en.wikipedia.org/wiki/Aphasia speak]. Her intelligence is unaffected, but she cannot find the words.
+
Three years ago Jane had a [http://www.strokeforbundet.org/show.asp?hv=10&pl=U&si=520&la=10&un=520&tx=1&sp=147 stroke] which made her unable to [http://en.wikipedia.org/wiki/Aphasia speak]. Her intelligence is unaffected, but she cannot find the words.
 
   
 
   
How can she use a mobile phone to communicate with her husband when the words are gone? When her husband is on the phone, asking - What are you doing? Do you want to have lunch with me?
+
How can she use an Openmoko mobile phone to communicate with her husband when the words are gone? When her husband is on the phone, asking - What are you doing? Do you want to have lunch with me?
 
   
 
   
To answer his first question about what she is doing, she’s sending him a small picture, showing a cup of tea. And then, a second picture with a heart, and he answers he loves her too. Then he asks if they can meet, he want to have a cup of tea as well, and she answers by sending him her GPS location and he can hear the street address of the small restaurant where she is right now. If there was a phone that made all this possible, I bet she would want one. A short story, capturing what this project is all about -- to build a very special mobile phone.
+
To answer his first question about what she is doing, she’s using the Openmoko phone to send him a small picture, showing a cup of tea. And then, a second picture with a heart, and he answers he loves her too. Then he asks if they can meet, he want to have a cup of tea as well, and she answers by sending him her GPS location (the Openmoko phone is a GPS reciever) and he can hear the street address of the small restaurant where she is right now. If there was a phone that made all this possible, I bet she would want one. You've read a short story, capturing what this project is all about -- to build a very special mobile phone.
  
==Basics==
 
  
The goal of a clean and uncluttered finger friendly GUI without menus is not an easy task, but after some [http://www.youtube.com/watch?v=hOotkglKZ94 trial and error] (the 3d-flip source is [http://projects.openmoko.org/frs/?group_id=59 here]) I think I got the general idea. The metaphor of moving around in 2 dimensions by swiping a finger from one corner of the screen to another gave me the space I needed for the different applications such as Dialer, SMS, GPS, SketchApp and a Keyboard. I also made a DXF2ERL converter. With it I can draw GUI components in a CAD program and from the vector file automatically generate [[Erlang]] source code. DXF2ERL is [http://projects.openmoko.org/frs/?group_id=59 here].
+
===Concept===
 +
This page describes the concept of the Aphasia Openmoko. Why and how it started. How it can help.
 +
Doctor's comments. Ideas and general brainstorming. Feel free to add and change this page at will.
 +
----
  
==Why Erlang?==
+
====[http://www.dreamsongs.com/IHE/ Innovation happens elsewhere]====
1. Erlang is Smalltalk as Alan Kay wanted it! And apparently Kay himself early on described his early conception of objects as being "little computers" that would communicate with each other via messages.
+
It all started in the 90's, a friend of mine asked if I could make a Windows-program that he could use to connect his home pc to his office pc so that he could communicate with his son, suffering from Downs syndrome and unable to use an ordinary phone. His son knew [http://en.wikipedia.org/wiki/Pictogram pictograms] and could communicate by sending small pictures to his dad. I made a system which I also took to [http://www.hi.se/en/Swedish-Institute-of-Assistive-Technology-/ HI]. They found it interesting, but not mobile enough (a PC was a heavy box at that time) and restrictive because you had to have 2 PC's with the same software installed on both. The idea though was brilliant, and here we go again, with a new incarnation.
 +
----
  
2. Hot Code loading.
+
===Are you a Developer===
 +
Do you want to help create a unique phone? Start learning [[Erlang]]. The language is tailored to telecom applications by [http://www.erlang.org Ericsson]. Erlang supports programming "soft" real-time systems, which require response times in the order of milliseconds. It has support for concurrency and [http://en.wikipedia.org/wiki/High_availability 5 nines]. When you have a basic understanding of Erlang, follow the [[Aphasia Development Tutorial]] to get acquainted with the overall architecture, and how to write a new service and add it to the Aphasia userland. Sign-up as a project member [http://projects.openmoko.org/projects/aphasia/ here].
 +
----
  
3. Support of multi core processors.
+
===Are you a User===
 +
You want to take a look at what the phone can do for you, besides the trivial picture_to_sound service, which - as I said - is an old idea in a new package.
  
==The first step, the dialer==
+
On top of the four basic services; GSM, GPS, Sound and Haptic, there are already quite a few High Order Services and more to come. What is most important here is not if this or that service is implemented, but that '''your doctor can install and remove services on your phone without you even have to know about it'''. And you know what, he/she can do it over the network so you don't even have to plan a visit. Just pick up the phone and the new functions are there. What we are offering here is applications available on the network, but made especially for you, as opposed to general web applications that should scale en masse and are made for nobody. Clay Shirky put a name on this, it's [http://www.shirky.com/writings/situated_software.html situated software].
  
[[FSO]] takes care of most of the functionality when it comes to dial, answer, ring-tone etc. All I had to do was to create a clean dialer GUI. It sits in the space you are at when you start fresh. There is no presentation of the number you are dialing, just the buttons with figures and * and #. To place the call you dial the number and press_and_hold the green button. To hang up you press_and_hold the red button. When there is an incoming call the GUI will move to this space, and you press_and_hold the green button. Source code is [http://projects.openmoko.org/frs/?group_id=59 here].
 
  
[[Image:Dialer.jpeg]]
+
<gallery>
 +
Image:Sound2.jpeg|''output service''
 +
Image:Dialer2.jpeg|''dialer service''
 +
Image:Clock2.jpeg|''clock''
 +
Image:Mimic2.jpeg|''mimic service''
 +
Each service has a location. Moving around is done by swiping in different directions, movie clip below.|[http://www.kvamme.se/Aphasia/Images/erlang_neo.mpg]
 +
Another, cooler way to change service is to take a walk down the street. The GPS service is doing the swipin' for you, and the services for a certain location are showing up on screen when you move around.
 +
When you walk up to the bus stop the GPS service makes sure you have the clock and the bus_traveling service at hand. What the bus_traveling service should do is up to you and your doktor.
 +
</gallery>
  
Swipe the screen to the right and you will go to the...
+
As you can see, lots of new services still remains to invent/implement. If you are a programmer or want to learn, please read the developer section above and get going. People with Aphasia, and their relatives, are probably the most valuable resources we could get as developers, so please sign on.
 +
----
  
==Clock, Battery and GSM Signal==
+
===Q&A===
[[Image:clock.jpeg]]
+
You, who is reading this page, please write your question here and I'll try to give you an answer. Never mind if you don't manage formatting your question, I'll do that for you.
  
Some nice-to-have widgets. Push_and_hold the clock face and it adjusts to [[GPS]]-time. Touch the battery and it will force fast_charge, even from a dumb [[USB_charger]]. The green bar in the antenna shows 0-100% [[GSM]] signal strength. Source code is [http://projects.openmoko.org/frs/?group_id=59 here].
+
====='''Q:''' How did Jane find a picture of a cup of tea so quickly?=====
 +
''From an almost unlimited amount of small pictures she could find the cup_of_tea picture in only a few seconds. How did she do that?''
  
Swipe the screen to the right and you will go to the...
+
'''A:''' The pictures available on the Openmoko screen depends on where you are. When Jane opened the door and stepped inside the little tea house the pictures on her Openmoko changed to a set of pictures with some relevance to the situation she had to handle at that location. Tea brands, Buy, Pay the bill, Restroom, Taxi, Sugar, Lemon, you got he idea. And the heart is always there for her loved ones.
  
==GPS reticle==
+
====='''Q:''' So everybody 's got to have an Openmoko phone then?=====
[[Image:reticle.jpeg]]
+
''Can Jane talk to a regular mobile/house phone or does it have to be an Openmoko at both ends?''
  
Each button can hold a location (waypoint). It could be home, school, your hotel or your favourite Chanterelle (Cantharellus cibarius) spot. Push_and_hold a button to store your current location. The reticle goes red when you have satellites, and a target-point slides at the reticle horizontal line. Keep it in center and you are heading at the target. Close enough (within 100 square meters) your position will show on the reticle surface with the target locked in reticle center. Source code is [http://projects.openmoko.org/frs/?group_id=59 here].
+
'''A:''' Jane can use her Openmoko to call or pick up a call from any phone - any brand. When Jane is touching a picture on the Openmoko screen a sound player in the phone plays a sound snippet associated to that picture. Pointing at the picture is starting a sound player in the handset. The person Jane is talking to can hear the sound from the file being played.
  
Make sure you have [[Gllin]], else Install gllin_1.1+r931-r0_om-gta01.ipk and gpsd.
+
====='''Q:''' How could Jane's husband hear the street address of the small restaurant?=====
 +
''How could Jane's Openmoko phone "know" the street address when all it had was the GPS position?''
  
==SketchApp==
+
'''A:'''  If you are familiar with Google Maps, you know that you can type in a street address and get the map and exact GPS location. Use that database the other way around and you get the street address if you handle in a GPS location, it's called Reverse Geocoding. A simple text2speech application converts the address to sound.  
The *current* idea for this application is as follows:
+
 
+
During phone call, sketch a simple stick-figure (like a heart or a cup of tea to illustrate the short story on top of this page). Press an ok-button and a database-application pattern-matches the sketch and returns the sound-clip that is sent to your partner.
+
 
+
==SMS==
+
Send and receive SMS. Address book included.
+
 
+
==Finger Keyboard==
+
During a phone call, type a word and press OK, and a database-application takes the word and returns the sound-clip that is sent to your partner.
+
 
+
This keyboard should also work as a normal keyboard when typing SMS.
+
 
+
==Reminder==
+
Host-based application is used for composing events that could be geograpic and/or time triggered. Host application and phone reminder are synchronized with erlang hot code loading.
+
  
  
 
[[Category:System Applications]]
 
[[Category:System Applications]]

Latest revision as of 16:25, 30 August 2009

Three years ago Jane had a stroke which made her unable to speak. Her intelligence is unaffected, but she cannot find the words.

How can she use an Openmoko mobile phone to communicate with her husband when the words are gone? When her husband is on the phone, asking - What are you doing? Do you want to have lunch with me?

To answer his first question about what she is doing, she’s using the Openmoko phone to send him a small picture, showing a cup of tea. And then, a second picture with a heart, and he answers he loves her too. Then he asks if they can meet, he want to have a cup of tea as well, and she answers by sending him her GPS location (the Openmoko phone is a GPS reciever) and he can hear the street address of the small restaurant where she is right now. If there was a phone that made all this possible, I bet she would want one. You've read a short story, capturing what this project is all about -- to build a very special mobile phone.


Contents

[edit] Concept

This page describes the concept of the Aphasia Openmoko. Why and how it started. How it can help. Doctor's comments. Ideas and general brainstorming. Feel free to add and change this page at will.


[edit] Innovation happens elsewhere

It all started in the 90's, a friend of mine asked if I could make a Windows-program that he could use to connect his home pc to his office pc so that he could communicate with his son, suffering from Downs syndrome and unable to use an ordinary phone. His son knew pictograms and could communicate by sending small pictures to his dad. I made a system which I also took to HI. They found it interesting, but not mobile enough (a PC was a heavy box at that time) and restrictive because you had to have 2 PC's with the same software installed on both. The idea though was brilliant, and here we go again, with a new incarnation.


[edit] Are you a Developer

Do you want to help create a unique phone? Start learning Erlang. The language is tailored to telecom applications by Ericsson. Erlang supports programming "soft" real-time systems, which require response times in the order of milliseconds. It has support for concurrency and 5 nines. When you have a basic understanding of Erlang, follow the Aphasia Development Tutorial to get acquainted with the overall architecture, and how to write a new service and add it to the Aphasia userland. Sign-up as a project member here.


[edit] Are you a User

You want to take a look at what the phone can do for you, besides the trivial picture_to_sound service, which - as I said - is an old idea in a new package.

On top of the four basic services; GSM, GPS, Sound and Haptic, there are already quite a few High Order Services and more to come. What is most important here is not if this or that service is implemented, but that your doctor can install and remove services on your phone without you even have to know about it. And you know what, he/she can do it over the network so you don't even have to plan a visit. Just pick up the phone and the new functions are there. What we are offering here is applications available on the network, but made especially for you, as opposed to general web applications that should scale en masse and are made for nobody. Clay Shirky put a name on this, it's situated software.


As you can see, lots of new services still remains to invent/implement. If you are a programmer or want to learn, please read the developer section above and get going. People with Aphasia, and their relatives, are probably the most valuable resources we could get as developers, so please sign on.


[edit] Q&A

You, who is reading this page, please write your question here and I'll try to give you an answer. Never mind if you don't manage formatting your question, I'll do that for you.

[edit] Q: How did Jane find a picture of a cup of tea so quickly?

From an almost unlimited amount of small pictures she could find the cup_of_tea picture in only a few seconds. How did she do that?

A: The pictures available on the Openmoko screen depends on where you are. When Jane opened the door and stepped inside the little tea house the pictures on her Openmoko changed to a set of pictures with some relevance to the situation she had to handle at that location. Tea brands, Buy, Pay the bill, Restroom, Taxi, Sugar, Lemon, you got he idea. And the heart is always there for her loved ones.

[edit] Q: So everybody 's got to have an Openmoko phone then?

Can Jane talk to a regular mobile/house phone or does it have to be an Openmoko at both ends?

A: Jane can use her Openmoko to call or pick up a call from any phone - any brand. When Jane is touching a picture on the Openmoko screen a sound player in the phone plays a sound snippet associated to that picture. Pointing at the picture is starting a sound player in the handset. The person Jane is talking to can hear the sound from the file being played.

[edit] Q: How could Jane's husband hear the street address of the small restaurant?

How could Jane's Openmoko phone "know" the street address when all it had was the GPS position?

A: If you are familiar with Google Maps, you know that you can type in a street address and get the map and exact GPS location. Use that database the other way around and you get the street address if you handle in a GPS location, it's called Reverse Geocoding. A simple text2speech application converts the address to sound.

Personal tools

Three years ago Jane had a stroke which made her unable to speak. Her intelligence is unaffected, but she cannot find the words.

How can she use a mobile phone to communicate with her husband when the words are gone? When her husband is on the phone, asking - What are you doing? Do you want to have lunch with me?

To answer his first question about what she is doing, she’s sending him a small picture, showing a cup of tea. And then, a second picture with a heart, and he answers he loves her too. Then he asks if they can meet, he want to have a cup of tea as well, and she answers by sending him her GPS location and he can hear the street address of the small restaurant where she is right now. If there was a phone that made all this possible, I bet she would want one. A short story, capturing what this project is all about -- to build a very special mobile phone.

Basics

The goal of a clean and uncluttered finger friendly GUI without menus is not an easy task, but after some trial and error (the 3d-flip source is here) I think I got the general idea. The metaphor of moving around in 2 dimensions by swiping a finger from one corner of the screen to another gave me the space I needed for the different applications such as Dialer, SMS, GPS, SketchApp and a Keyboard. I also made a DXF2ERL converter. With it I can draw GUI components in a CAD program and from the vector file automatically generate Erlang source code. DXF2ERL is here.

Why Erlang?

1. Erlang is Smalltalk as Alan Kay wanted it! And apparently Kay himself early on described his early conception of objects as being "little computers" that would communicate with each other via messages.

2. Hot Code loading.

3. Support of multi core processors.

The first step, the dialer

FSO takes care of most of the functionality when it comes to dial, answer, ring-tone etc. All I had to do was to create a clean dialer GUI. It sits in the space you are at when you start fresh. There is no presentation of the number you are dialing, just the buttons with figures and * and #. To place the call you dial the number and press_and_hold the green button. To hang up you press_and_hold the red button. When there is an incoming call the GUI will move to this space, and you press_and_hold the green button. Source code is here.

Dialer.jpeg

Swipe the screen to the right and you will go to the...

Clock, Battery and GSM Signal

Clock.jpeg

Some nice-to-have widgets. Push_and_hold the clock face and it adjusts to GPS-time. Touch the battery and it will force fast_charge, even from a dumb USB_charger. The green bar in the antenna shows 0-100% GSM signal strength. Source code is here.

Swipe the screen to the right and you will go to the...

GPS reticle

Reticle.jpeg

Each button can hold a location (waypoint). It could be home, school, your hotel or your favourite Chanterelle (Cantharellus cibarius) spot. Push_and_hold a button to store your current location. The reticle goes red when you have satellites, and a target-point slides at the reticle horizontal line. Keep it in center and you are heading at the target. Close enough (within 100 square meters) your position will show on the reticle surface with the target locked in reticle center. Source code is here.

Make sure you have Gllin, else Install gllin_1.1+r931-r0_om-gta01.ipk and gpsd.

SketchApp

The *current* idea for this application is as follows:

During phone call, sketch a simple stick-figure (like a heart or a cup of tea to illustrate the short story on top of this page). Press an ok-button and a database-application pattern-matches the sketch and returns the sound-clip that is sent to your partner.

SMS

Send and receive SMS. Address book included.

Finger Keyboard

During a phone call, type a word and press OK, and a database-application takes the word and returns the sound-clip that is sent to your partner.

This keyboard should also work as a normal keyboard when typing SMS.

Reminder

Host-based application is used for composing events that could be geograpic and/or time triggered. Host application and phone reminder are synchronized with erlang hot code loading.