User:Quicksand

From Openmoko

(Difference between revisions)
Jump to: navigation, search
(Current Projects)
(zedlock)
 
(6 intermediate revisions by 2 users not shown)
Line 5: Line 5:
 
I have a degree in mathematics, and I have interests in natural language processing and in electronic music.  Accordingly, as time permits, I have some ideas for a free-form handwriting recognition engine for Linux that I would like to at least prototype (the currently available options, for example as in the Nokia 770/800, are proprietary).  And I'd love to have a simple softsynth and sequencer combination available on the Neo for sketching out musical ideas while traveling.
 
I have a degree in mathematics, and I have interests in natural language processing and in electronic music.  Accordingly, as time permits, I have some ideas for a free-form handwriting recognition engine for Linux that I would like to at least prototype (the currently available options, for example as in the Nokia 770/800, are proprietary).  And I'd love to have a simple softsynth and sequencer combination available on the Neo for sketching out musical ideas while traveling.
  
My intellectual property law practice is online at wixonlaw.com [http://www.wixonlaw.com]
+
My intellectual property law practice is online at [http://www.wixonlaw.com wixonlaw.com]
  
 
== Current Projects ==
 
== Current Projects ==
  
'''Free42''' [http://home.planet.nl/~demun000/thomas_projects/free42/]is an excellent GPL2 RPN calculator by Thomas Okken modeled after the HP42 series of calculators.  I have been using it for several years on my VGA PocketPC, and before that on a PalmOS device.
+
=== zedlock ===
 +
 
 +
'''Info moved to own page. See [[Zedlock]]'''
 +
 
 +
 
 +
Screen locking program.  Currently in PROTOTYPE form.
 +
 
 +
Download [http://wixonlaw.com/public/zedlock-0.0.1.tar.bz2 zedlock 0.0.1]
 +
 
 +
[[Image:Zedlock-screenshot-1.png]]
 +
 
 +
'''How it works'''
 +
 
 +
This is a screen-lock program in prototype form.  To run it, just run zedlock.py from a terminal in the directory where it's installed.
 +
 
 +
It's really simple.  To unlock, just trace a large 'Z' with a single stroke of your finger or stylus, anywhere on the screen.  It must be BIG ENOUGH (about half the smallest screen dimension in both height and width, approx. 2cm on the Neo), it must be drawn in the CORRECT SEQUENCE (from top-left to bottom-right), and it must be QUICK ENOUGH (completed in less than one second).  Within these parameters, a fair amount of variation (i.e., sloppiness) is tolerated.  (As a result, you can make some non-Z patterns that fool it.)
 +
 
 +
'''Who's Zed?'''
 +
 
 +
Zed's dead, baby, Zed's dead.
 +
 
 +
'''No, really, why Zed?'''
 +
 
 +
There are a bunch of things out there already called "zlock" or the like.  I selected the 'Z' glyph because it's fast and easy to make without looking, and its four points are easy to pull out of a stream of touchscreen data -- they are simply the maxima and minima of the functions (x+y) and (x-y).  I don't need to keep a linked list of intermediate points or anything like that; these points are updated on-the-fly.
 +
 
 +
'''Why would I want this?'''
 +
 
 +
You don't need to be looking at the screen to unlock it.  It is pretty unlikely to unlock accidentally (by your face while you are on the phone, in your pocket, etc.).  But it's simple and fast enough to perform frequently and easily, and to become second-nature.
 +
 
 +
Plus it looks nice: gtk+ and cairo are used for fluid anti-aliased graphics.  Visual feedback is provided after a stroke is completed: an openmoko-orange line shows where you made your stroke, and if it finds the 'Z' pattern, it is highlighted in white for a moment.
 +
 
 +
Success:
 +
 
 +
[[Image:Zedlock-screenshot-2.png]]
 +
 
 +
If your stroke doesn't meet the criteria, you get a big red 'X' on the screen and a three-second lockout.
 +
 
 +
Too small:
 +
 
 +
[[Image:Zedlock-screenshot-3.png]]
 +
 
 +
Wrong sequence:
 +
 
 +
[[Image:Zedlock-screenshot-4.png]]
 +
 
 +
With the iPhone and the Qtopia phone platform, you need to be looking at the touchscreen to unlock it.  Their sliders and animated keys are cool, but this is better.
 +
 
 +
'''Remember, this is a prototype'''
 +
 
 +
To install it, copy zedlock.py and zedlock.png to your home directory (or a subdirectory), make sure zedlock.py is executable, and run it from the directory you installed it in.  The program depends on python-pygtk and python-pycairo.  Note that it doesn't actually do anything all that functional yet -- it is simply a demonstration program.  This prototype doesn't actually lock or unlock anything; it just clears the screen after each stroke (and the subsequent visual feedback) so you can try it again and again and again.
 +
 
 +
The program should be more-or-less QVGA-friendly (well, almost, just change two global variables) and orientation-friendly already, but YMMV.
 +
 
 +
Yes, it's slow, especially to start up.  But that's embedded python for ya.  Plus I had to do some weird hacky things in cairo to get it to render predictably.  This is my first cairo project and my first python program, SO DON'T GIVE ME ANY GRIEF!  There are probably terrible, ugly, spaghetti-code-monsters in there if you care to look.  They'll go away over time.  :)
 +
 
 +
'''Future Plans'''
 +
 
 +
After the featureset solidifies a bit more, I will rewrite it in C (which I'm much more comfortable with), optimize it (a lot!), and add some features:
 +
 
 +
'''1)''' It will be rotationally insensitive so you don't need to look at it to determine portrait vs. landscape before you unlock.  This will be pretty easy to add.
 +
 
 +
'''2)''' The text on there is statically-coded right now, and fixing that will be trivial, so don't complain about it now.  It doesn't actually tell you the GSM/GPRS status or even the real time of day.  Yes, wise guy, IT'S ALWAYS 5:35 pm HERE.
 +
 
 +
'''3)''' Eventually it will get a battery/charging state icon, GSM signal strength bars, and a ringer volume indicator, so you can see those things while the screen remains locked.  I don't want to make the interface too cluttered, but those items seem essential.
 +
 
 +
'''4)''' Ideally this will slip right into the OpenMoko power management scheme.  Preferably neod and/or the dialer will *lock* the screen and require an unlock program (such as zedlock) in the following circumstances:
 +
 
 +
  * immediately after an incoming call is answered
 +
    (to prevent touchscreen input from the user's face or ear)
 +
 
 +
  * immediately after an outgoing call is dialed
 +
    (ditto)
 +
 
 +
  * on request
 +
    (via a lock button or menu, before you put it in your pocket)
 +
 
 +
  * upon an automatic return-from-suspend
 +
    (e.g., a datebook alarm, incoming phone call, SMS, etc.)
 +
 
 +
  * but *not* following a manual return-from-suspend
 +
 
 +
Note that this concept of screen-lock is different from the power-management concept of screen blanking.  Screen LOCK should probably occur more often, and is intended primarily to prevent accidental user-interface manipulation while the touchscreen is against the user's face, in the user's pocket, or otherwise subject to spurious inputs.  It is not intended to save power.
 +
 
 +
=== Password Safe ===
 +
 
 +
Unless someone beats me to it, I would like to implement or port an encrypted password safe for the Neo, preferably one that is able to exchange data with both a Linux desktop application and a Windows desktop application.  A good candidate for this would be '''KeePass''' and '''KeePassX''' but I need to investigate alternatives.  KeePassX compiles successfully but never opens a window on the Neo, and I'm not sure why -- other QT applications seem to work fine.
 +
 
 +
=== Xournal ===
 +
 
 +
[http://xournal.sourceforge.net Xournal] is a very nice scribble-notes package inspired by the Microsoft Journal application provided with Tablet PC versions of Windows.  It uses a vector-based ink model and supports multi-colored ink, different kinds of erasers, highlighters, and (in the new version) typed text.  It runs very well on the desktop, and marginally well on the Neo at this time, but with some fine-tuning and user-interface consolidation I think it will be a very useful addition to our suite of applications.  I will be working on it in the coming days.
 +
 
 +
== Finished Projects ==
 +
 
 +
=== Free42 ===
 +
 
 +
[http://home.planet.nl/~demun000/thomas_projects/free42/ Free42] is an excellent GPL2 RPN calculator by Thomas Okken modeled after the HP42 series of calculators.  I have been using it for several years on my VGA PocketPC, and before that on a PalmOS device.
  
 
[[Image:Image-free42.jpg]]
 
[[Image:Image-free42.jpg]]
Line 15: Line 110:
 
Unfortunately, there was no bitbake recipe for it in the OE database, so I had to make one.  This being my first attempt at such, it might be a bit of an ugly hack, but it works!  And it runs beautifully on the Neo.
 
Unfortunately, there was no bitbake recipe for it in the OE database, so I had to make one.  This being my first attempt at such, it might be a bit of an ugly hack, but it works!  And it runs beautifully on the Neo.
  
I have submitted this to the OE people upstream for inclusion into their metadata packageFor the impatient, the .bb recipe and patches are available via their bugtracker. [http://bugs.openembedded.org/show_bug.cgi?id=2920]
+
It's now available in the OE repositoryThe package name is free42-vga
  
'''Xournal''' [http://xournal.sourceforge.net] is a very nice scribble-notes package inspired by the Microsoft Journal application provided with Tablet PC versions of Windows.  It uses a vector-based ink model and supports multi-colored ink, different kinds of erasers, highlighters, and (in the new version) typed text.  It runs very well on the desktop, and marginally well on the Neo at this time, but with some fine-tuning and user-interface consolidation I think it will be a very useful addition to our suite of applications.  I will be working on it in the coming days.
+
=== Notecase ===
  
'''Notecase''' [http://notecase.sourceforge.net] is a hierarchical text note manager with rich text formatting capabilities.  It already runs well on the Linux desktop, the Windows desktop, and on the linux-based Zauruses.  There's a .bb recipe for an old version of Notecase in the OE metadata, and in the coming days I will be bringing that up to date and tweaking the user interface for small-screen devices such as the Neo.
+
[http://notecase.sourceforge.net Notecase] is a hierarchical text note manager with rich text formatting capabilities.  It already runs well on the Linux desktop, the Windows desktop, and on the linux-based Zauruses.  I have uploaded a .bb file and patch for the latest version (1.6.6) to the OE bugtracker.
 
+
'''Password Safe''' - Unless someone beats me to it, I would like to implement or port an encrypted password safe for the Neo, preferably one that is able to exchange data with both a Linux desktop application and a Windows desktop application.  A good candidate for this would be '''KeePass''' and '''KeePassX''' but I need to investigate alternatives.
+
  
 
== Future Projects ==
 
== Future Projects ==
  
'''Handwriting Recognition''' - Handwriting recognition does pretty well these days, both on the Windows platform (XP Tablet PC Edition and Vista), and on Mac OSX (Inkwell, I believe it is called, which is based on work originally done for the Newton).  Even Microsoft PocketPC platform has reasonably good recognition (through the Calligrapher third-party application, which Microsoft has licensed in an older version as the Transcriber built into Windows Mobile).
+
=== Handwriting Recognition ===
 +
 
 +
Handwriting recognition does pretty well these days, both on the Windows platform (XP Tablet PC Edition and Vista), and on Mac OSX (Inkwell, I believe it is called, which is based on work originally done for the Newton).  Even Microsoft PocketPC platform has reasonably good recognition (through the Calligrapher third-party application, which Microsoft has licensed in an older version as the Transcriber built into Windows Mobile).
  
 
The Nokia 770/800 apparently has a handwriting recognition engine that is proprietary, without source available.
 
The Nokia 770/800 apparently has a handwriting recognition engine that is proprietary, without source available.
Line 33: Line 128:
 
This will be a spare-time project at best, so it might not see the light of day for quite a while.
 
This will be a spare-time project at best, so it might not see the light of day for quite a while.
  
'''Music Platform''' - I had a General Midi sound generator that attached to my ancient Palm Vx, and a rudimentary step sequencer to go with it.  It was polyphonic and multitimbral, but it wasn't really useful for anything more than skething out ideas because of the constraints of the old PalmOS 3.x user interface.  But it was a load of fun, and useful within its constraints, so I would love to see something like this available in the OpenMoko project.
+
* Check out Tomoe, a FOSS handwriting recognition engine: http://tomoe.sourceforge.jp/ [[User:Bugmenot|Bugmenot]] 09:56, 25 October 2007 (CEST)
 +
 
 +
=== Music Platform ===
 +
 
 +
I had a General Midi sound generator that attached to my ancient Palm Vx, and a rudimentary step sequencer to go with it.  It was polyphonic and multitimbral, but it wasn't really useful for anything more than skething out ideas because of the constraints of the old PalmOS 3.x user interface.  But it was a load of fun, and useful within its constraints, so I would love to see something like this available in the OpenMoko project.
  
 
This is another project that might take a long while to come to fruition, but it is one I have had in mind for a while.
 
This is another project that might take a long while to come to fruition, but it is one I have had in mind for a while.

Latest revision as of 21:14, 17 July 2008

Clarke A. Wixon ("Quicksand") is a U.S. (California) intellectual property attorney by day, OpenMoko hacker by night.

I hope to bring some useful PDA-style applications to the OpenMoko platform, either by porting existing desktop applications, by fine-tuning existing bitbake recipes and user interface definitions in the OpenEmbedded database to better take advantage of the OpenMoko form factor and capabilities, or by coding something up to fit my own personal needs.

I have a degree in mathematics, and I have interests in natural language processing and in electronic music. Accordingly, as time permits, I have some ideas for a free-form handwriting recognition engine for Linux that I would like to at least prototype (the currently available options, for example as in the Nokia 770/800, are proprietary). And I'd love to have a simple softsynth and sequencer combination available on the Neo for sketching out musical ideas while traveling.

My intellectual property law practice is online at wixonlaw.com

Contents

[edit] Current Projects

[edit] zedlock

Info moved to own page. See Zedlock


Screen locking program. Currently in PROTOTYPE form.

Download zedlock 0.0.1

Zedlock-screenshot-1.png

How it works

This is a screen-lock program in prototype form. To run it, just run zedlock.py from a terminal in the directory where it's installed.

It's really simple. To unlock, just trace a large 'Z' with a single stroke of your finger or stylus, anywhere on the screen. It must be BIG ENOUGH (about half the smallest screen dimension in both height and width, approx. 2cm on the Neo), it must be drawn in the CORRECT SEQUENCE (from top-left to bottom-right), and it must be QUICK ENOUGH (completed in less than one second). Within these parameters, a fair amount of variation (i.e., sloppiness) is tolerated. (As a result, you can make some non-Z patterns that fool it.)

Who's Zed?

Zed's dead, baby, Zed's dead.

No, really, why Zed?

There are a bunch of things out there already called "zlock" or the like. I selected the 'Z' glyph because it's fast and easy to make without looking, and its four points are easy to pull out of a stream of touchscreen data -- they are simply the maxima and minima of the functions (x+y) and (x-y). I don't need to keep a linked list of intermediate points or anything like that; these points are updated on-the-fly.

Why would I want this?

You don't need to be looking at the screen to unlock it. It is pretty unlikely to unlock accidentally (by your face while you are on the phone, in your pocket, etc.). But it's simple and fast enough to perform frequently and easily, and to become second-nature.

Plus it looks nice: gtk+ and cairo are used for fluid anti-aliased graphics. Visual feedback is provided after a stroke is completed: an openmoko-orange line shows where you made your stroke, and if it finds the 'Z' pattern, it is highlighted in white for a moment.

Success:

Zedlock-screenshot-2.png

If your stroke doesn't meet the criteria, you get a big red 'X' on the screen and a three-second lockout.

Too small:

Zedlock-screenshot-3.png

Wrong sequence:

Zedlock-screenshot-4.png

With the iPhone and the Qtopia phone platform, you need to be looking at the touchscreen to unlock it. Their sliders and animated keys are cool, but this is better.

Remember, this is a prototype

To install it, copy zedlock.py and zedlock.png to your home directory (or a subdirectory), make sure zedlock.py is executable, and run it from the directory you installed it in. The program depends on python-pygtk and python-pycairo. Note that it doesn't actually do anything all that functional yet -- it is simply a demonstration program. This prototype doesn't actually lock or unlock anything; it just clears the screen after each stroke (and the subsequent visual feedback) so you can try it again and again and again.

The program should be more-or-less QVGA-friendly (well, almost, just change two global variables) and orientation-friendly already, but YMMV.

Yes, it's slow, especially to start up. But that's embedded python for ya. Plus I had to do some weird hacky things in cairo to get it to render predictably. This is my first cairo project and my first python program, SO DON'T GIVE ME ANY GRIEF! There are probably terrible, ugly, spaghetti-code-monsters in there if you care to look. They'll go away over time.  :)

Future Plans

After the featureset solidifies a bit more, I will rewrite it in C (which I'm much more comfortable with), optimize it (a lot!), and add some features:

1) It will be rotationally insensitive so you don't need to look at it to determine portrait vs. landscape before you unlock. This will be pretty easy to add.

2) The text on there is statically-coded right now, and fixing that will be trivial, so don't complain about it now. It doesn't actually tell you the GSM/GPRS status or even the real time of day. Yes, wise guy, IT'S ALWAYS 5:35 pm HERE.

3) Eventually it will get a battery/charging state icon, GSM signal strength bars, and a ringer volume indicator, so you can see those things while the screen remains locked. I don't want to make the interface too cluttered, but those items seem essential.

4) Ideally this will slip right into the OpenMoko power management scheme. Preferably neod and/or the dialer will *lock* the screen and require an unlock program (such as zedlock) in the following circumstances:

 * immediately after an incoming call is answered
   (to prevent touchscreen input from the user's face or ear)
 * immediately after an outgoing call is dialed
   (ditto)
 * on request
   (via a lock button or menu, before you put it in your pocket)
 * upon an automatic return-from-suspend
   (e.g., a datebook alarm, incoming phone call, SMS, etc.)
 * but *not* following a manual return-from-suspend

Note that this concept of screen-lock is different from the power-management concept of screen blanking. Screen LOCK should probably occur more often, and is intended primarily to prevent accidental user-interface manipulation while the touchscreen is against the user's face, in the user's pocket, or otherwise subject to spurious inputs. It is not intended to save power.

[edit] Password Safe

Unless someone beats me to it, I would like to implement or port an encrypted password safe for the Neo, preferably one that is able to exchange data with both a Linux desktop application and a Windows desktop application. A good candidate for this would be KeePass and KeePassX but I need to investigate alternatives. KeePassX compiles successfully but never opens a window on the Neo, and I'm not sure why -- other QT applications seem to work fine.

[edit] Xournal

Xournal is a very nice scribble-notes package inspired by the Microsoft Journal application provided with Tablet PC versions of Windows. It uses a vector-based ink model and supports multi-colored ink, different kinds of erasers, highlighters, and (in the new version) typed text. It runs very well on the desktop, and marginally well on the Neo at this time, but with some fine-tuning and user-interface consolidation I think it will be a very useful addition to our suite of applications. I will be working on it in the coming days.

[edit] Finished Projects

[edit] Free42

Free42 is an excellent GPL2 RPN calculator by Thomas Okken modeled after the HP42 series of calculators. I have been using it for several years on my VGA PocketPC, and before that on a PalmOS device.

Image-free42.jpg

Unfortunately, there was no bitbake recipe for it in the OE database, so I had to make one. This being my first attempt at such, it might be a bit of an ugly hack, but it works! And it runs beautifully on the Neo.

It's now available in the OE repository. The package name is free42-vga

[edit] Notecase

Notecase is a hierarchical text note manager with rich text formatting capabilities. It already runs well on the Linux desktop, the Windows desktop, and on the linux-based Zauruses. I have uploaded a .bb file and patch for the latest version (1.6.6) to the OE bugtracker.

[edit] Future Projects

[edit] Handwriting Recognition

Handwriting recognition does pretty well these days, both on the Windows platform (XP Tablet PC Edition and Vista), and on Mac OSX (Inkwell, I believe it is called, which is based on work originally done for the Newton). Even Microsoft PocketPC platform has reasonably good recognition (through the Calligrapher third-party application, which Microsoft has licensed in an older version as the Transcriber built into Windows Mobile).

The Nokia 770/800 apparently has a handwriting recognition engine that is proprietary, without source available.

I would love to see a free handwriting recognition engine that is unencumbered by commercial license restrictions and patents, and can be used with OpenMoko. With my background in mathematics and natural language processing (which I had the privilege to work on at Xerox PARC some years ago), I have some ideas on how to go about this in a relatively "processor-light" manner.

This will be a spare-time project at best, so it might not see the light of day for quite a while.

[edit] Music Platform

I had a General Midi sound generator that attached to my ancient Palm Vx, and a rudimentary step sequencer to go with it. It was polyphonic and multitimbral, but it wasn't really useful for anything more than skething out ideas because of the constraints of the old PalmOS 3.x user interface. But it was a load of fun, and useful within its constraints, so I would love to see something like this available in the OpenMoko project.

This is another project that might take a long while to come to fruition, but it is one I have had in mind for a while.

Personal tools

Clarke A. Wixon ("Quicksand") is a U.S. (California) intellectual property attorney by day, OpenMoko hacker by night.

I hope to bring some useful PDA-style applications to the OpenMoko platform, either by porting existing desktop applications, by fine-tuning existing bitbake recipes and user interface definitions in the OpenEmbedded database to better take advantage of the OpenMoko form factor and capabilities, or by coding something up to fit my own personal needs.

I have a degree in mathematics, and I have interests in natural language processing and in electronic music. Accordingly, as time permits, I have some ideas for a free-form handwriting recognition engine for Linux that I would like to at least prototype (the currently available options, for example as in the Nokia 770/800, are proprietary). And I'd love to have a simple softsynth and sequencer combination available on the Neo for sketching out musical ideas while traveling.

My intellectual property law practice is online at wixonlaw.com [1]

Current Projects

Free42 [2]is an excellent GPL2 RPN calculator by Thomas Okken modeled after the HP42 series of calculators. I have been using it for several years on my VGA PocketPC, and before that on a PalmOS device.

Image-free42.jpg

Unfortunately, there was no bitbake recipe for it in the OE database, so I had to make one. This being my first attempt at such, it might be a bit of an ugly hack, but it works! And it runs beautifully on the Neo.

I have submitted this to the OE people upstream for inclusion into their metadata package. For the impatient, the .bb recipe and patches are available via their bugtracker. [3]

Xournal [4] is a very nice scribble-notes package inspired by the Microsoft Journal application provided with Tablet PC versions of Windows. It uses a vector-based ink model and supports multi-colored ink, different kinds of erasers, highlighters, and (in the new version) typed text. It runs very well on the desktop, and marginally well on the Neo at this time, but with some fine-tuning and user-interface consolidation I think it will be a very useful addition to our suite of applications. I will be working on it in the coming days.

Notecase [5] is a hierarchical text note manager with rich text formatting capabilities. It already runs well on the Linux desktop, the Windows desktop, and on the linux-based Zauruses. There's a .bb recipe for an old version of Notecase in the OE metadata, and in the coming days I will be bringing that up to date and tweaking the user interface for small-screen devices such as the Neo.

Password Safe - Unless someone beats me to it, I would like to implement or port an encrypted password safe for the Neo, preferably one that is able to exchange data with both a Linux desktop application and a Windows desktop application. A good candidate for this would be KeePass and KeePassX but I need to investigate alternatives.

Future Projects

Handwriting Recognition - Handwriting recognition does pretty well these days, both on the Windows platform (XP Tablet PC Edition and Vista), and on Mac OSX (Inkwell, I believe it is called, which is based on work originally done for the Newton). Even Microsoft PocketPC platform has reasonably good recognition (through the Calligrapher third-party application, which Microsoft has licensed in an older version as the Transcriber built into Windows Mobile).

The Nokia 770/800 apparently has a handwriting recognition engine that is proprietary, without source available.

I would love to see a free handwriting recognition engine that is unencumbered by commercial license restrictions and patents, and can be used with OpenMoko. With my background in mathematics and natural language processing (which I had the privilege to work on at Xerox PARC some years ago), I have some ideas on how to go about this in a relatively "processor-light" manner.

This will be a spare-time project at best, so it might not see the light of day for quite a while.

Music Platform - I had a General Midi sound generator that attached to my ancient Palm Vx, and a rudimentary step sequencer to go with it. It was polyphonic and multitimbral, but it wasn't really useful for anything more than skething out ideas because of the constraints of the old PalmOS 3.x user interface. But it was a load of fun, and useful within its constraints, so I would love to see something like this available in the OpenMoko project.

This is another project that might take a long while to come to fruition, but it is one I have had in mind for a while.