PIM Storage

From Openmoko

Revision as of 10:13, 19 January 2009 by Niebert (Talk | contribs)

Jump to: navigation, search

Contents

Overview

PIM storage describes a means of storing data in an extensible and cross-referencable way.

Goals & Objectives

  • Method to exchange data between all applications completely transparent
  • Personal data security / encryption
  • Great bidirectional sync of personal data
  • Easy method for backup (ideally this should be automatic)
  • Real-time incremental search
  • Autocomplete of personal data information

Use Cases

  • I would like to attach arbitrary data to a contact (AKA: custom fields)
  • I would like some level of integration with automatic switching of profiles. For example, it would be cool to automatically backup my personal data to my desktop PC when I get home.
  • I want to type the first few numbers of a phone number and be presented a list of matching contacts.
    • I'd rather take a person, choose home/work, choose land-line/mobile and not bother with numbers anymore. Useful when I'm on my way to meet that person, or the person has his/her birthday, i.e. the person is already listed on my screen!
  • I want to add a contact to multiple groups (AKA: categories)
  • I want to be able to send my contact info to other Openmoko devices over bluetooth
  • I want to be able to two-way sync all calendar, contacts (and tasks) with a GroupDAV server (e.g. Citadel) over-the-air

Constraints

(TBD)

Implementation Recommendations

  • This will be based on Embedded EDS, this way we will can directly use Evolution data, get OpenSync for free, and eventually be able to talk to a Microsoft Exchange Server using the Novell Connector.
  • Alternatively: Go for a radically different approach: Use Topic Maps to store PIM data, use Topic Map's integrated merging features for syncing. --xbaldauf
    • Topic Maps allow multi-way sync, storage of arbitrary data (including categories, and even a hierarchy of categories), integration of PIM data which is separated otherwise (calendar, addressbook, personal notes, representing networks who knows whom and who works for whom and on which project, ...), multiple entries for each field (e.g. multiple addresses, multiple names, multiple phone numbers, ...)

Interactions

  • Embedded EDS needs libglade2, D-Bus, and libdb.

Unresolved Issues

  • Synchronizing w/ Outlook
  • Synchronizing w/ MacOS X - most Macs have bluetooth builtin and OSX addressbook supports BT sync.
  • Synchronizing w/ Google services

Possible solutions:

  • mokod - An Openmoko daemon to run on host operating systems
  • Syncevolution can sync with Scheduleworld which can sync with Google, and other programs.
  • Hack the libsyncml stack to work in OpenMoko and sync with OpenSync. Solve the connectivity problem and move the supporting-the-other-peers to modular, plugin based OpenSync.

Import/Export of Calendar Data for PIM-Storage

This Import/Export can be realized by syncing an ics Calendar File (e.g. std.ics from Kontact-KDE, ...) by parsing this ICS-file (ICAL) and import the to the PIM-Storage on OpenMoko device. Make sure you have sqlite3 client installed

opkg install sqlite3

The PIM database file on the OpenMoko device is located at

/home/root/Applications/Qtopia/qtopia_db.sqlite

You can handle the file with sqlite (see Import Vcf Contacts):

  sqlite3 ~/Applications/Qtopia/qtopia_db.sqlite

For parsing the ics-File (iCal) on the OpenMoko device you need to install Perl

opkg install perl

Remark: The following desciption shows a possbible additional concept for PIM storage for import/export by application of OpenSource perl libraries that can be used to realized the concept.

Lets name the perl scripts ics2qtcal.pl and qtcal2ics.pl.

  • ics2qtcal.pl will parse the ics-file and import it to the qtopia PIM storage. The script should have 2 parameters: (Par1) Input ics-file e.g. std.ics (Par2) Ouput sqlite-file - e.g. qtopia_db.sqlite.
  • qtcal2ics.pl will export the qtopia PIM storage to an ics-file. The script should have 2 parameters: (Par1) Input sqlite-file - e.g. qtopia_db.sqlite (Par2) Ouput ics-file - e.g. std.ics.

For parsing the ICS-file there are several CPAN perl libraries available

After parsing a file with SQL-Commands can be generated for the ics input file of that are used by sqlite3 to import the dates in the qtopia PIM storage ics2qtcal.pl. In a first step of realisation of ics2qtcal.pl the old qtopia-db can be backuped and a full import of.

When all scripts are in place generate an icon in /usr/share/application that call and rsync command from to Openmoko device to copy the kontact-KDE ics-file parse it and copy it to the qtopia database.

 #!/usr/bin/sh
 cd ~/Applications/Qtopia
 rsync -avz -e ssh user@192.168.0.200:/home/myhome/.kde/share/apps/korganizer/std.ics std.ics
 mv qtopia_db.sqlite qtopia_db.sqlite.backup
 perl /path_to_the_script/ics2qtcal.pl std.ics qtopia_db.sqlite

192.168.0.200 is the IP of the desktop computer. It can be replaced by any IP you have your home directory with the ICS file. You need to enter a password so far. Automate this process see http://troy.jdmz.net/rsync/index.html for using rsync and ssh without authentication.

The perl script can call the sqlite3 and import the calendar data in the sqlite-database. To avoid to have the same events 2 times in the PIM storage the existing OpenMoko database should be exported first via qtcal2ics.pl removing events from the SQL imput that already exist in the SQL database. Records that have changed should be updated via SQL commands.

 ....
 mv qtopia_db.sqlite qtopia_db.sqlite.backup
 perl /path_to_the_script/qtcal2ics.pl qtopia_db.sqlite qtopia.ics 
 perl /path_to_the_script/ics2qtcal.pl std.ics qtopia_db.sqlite qtopia.ics

Remark: The perl scripts mentioned above are not programmed yet (19.01.2009), they show a solution for syncing. The development of the perl scripts ics2qtcal.pl and qtcal2ics.pl mentioned above can be helpful in converting ical formats to and from the OpenMoko device anyway.

Questions and Answers

  • Q: When will an Embedded EDS specification be available? i.e. How I can get applications to store and retrieve data from in it, in a compatible way?
  • Q: Will it out of the box synchronize with the Kontact PIM suite (KDE)?
Personal tools

Overview

PIM storage describes a means of storing data in an extensible and cross-referencable way.

Goals & Objectives

  • Method to exchange data between all applications completely transparent
  • Personal data security / encryption
  • Great bidirectional sync of personal data
  • Easy method for backup (ideally this should be automatic)
  • Real-time incremental search
  • Autocomplete of personal data information

Use Cases

  • I would like to attach arbitrary data to a contact (AKA: custom fields)
  • I would like some level of integration with automatic switching of profiles. For example, it would be cool to automatically backup my personal data to my desktop PC when I get home.
  • I want to type the first few numbers of a phone number and be presented a list of matching contacts.
    • I'd rather take a person, choose home/work, choose land-line/mobile and not bother with numbers anymore. Useful when I'm on my way to meet that person, or the person has his/her birthday, i.e. the person is already listed on my screen!
  • I want to add a contact to multiple groups (AKA: categories)
  • I want to be able to send my contact info to other Openmoko devices over bluetooth
  • I want to be able to two-way sync all calendar, contacts (and tasks) with a GroupDAV server (e.g. Citadel) over-the-air

Constraints

(TBD)

Implementation Recommendations

  • This will be based on Embedded EDS, this way we will can directly use Evolution data, get OpenSync for free, and eventually be able to talk to a Microsoft Exchange Server using the Novell Connector.
  • Alternatively: Go for a radically different approach: Use Topic Maps to store PIM data, use Topic Map's integrated merging features for syncing. --xbaldauf
    • Topic Maps allow multi-way sync, storage of arbitrary data (including categories, and even a hierarchy of categories), integration of PIM data which is separated otherwise (calendar, addressbook, personal notes, representing networks who knows whom and who works for whom and on which project, ...), multiple entries for each field (e.g. multiple addresses, multiple names, multiple phone numbers, ...)

Interactions

  • Embedded EDS needs libglade2, D-Bus, and libdb.

Unresolved Issues

  • Synchronizing w/ Outlook
  • Synchronizing w/ MacOS X - most Macs have bluetooth builtin and OSX addressbook supports BT sync.
  • Synchronizing w/ Google services

Possible solutions:

  • mokod - An Openmoko daemon to run on host operating systems
  • Syncevolution can sync with Scheduleworld which can sync with Google, and other programs.
  • Hack the libsyncml stack to work in OpenMoko and sync with OpenSync. Solve the connectivity problem and move the supporting-the-other-peers to modular, plugin based OpenSync.

Import/Export of Calendar Data for PIM-Storage

This Import/Export can be realized by syncing an ics Calendar File (e.g. std.ics from Kontact-KDE, ...) by parsing this ICS-file (ICAL) and import the to the PIM-Storage on OpenMoko device. Make sure you have sqlite3 client installed

opkg install sqlite3

The PIM database file on the OpenMoko device is located at

/home/root/Applications/Qtopia/qtopia_db.sqlite

You can handle the file with sqlite (see Import Vcf Contacts):

  sqlite3 ~/Applications/Qtopia/qtopia_db.sqlite

For parsing the ics-File (iCal) on the OpenMoko device you need to install Perl

opkg install perl

Remark: The following desciption shows a possbible additional concept for PIM storage for import/export by application of OpenSource perl libraries that can be used to realized the concept.

Lets name the perl scripts ics2qtcal.pl and qtcal2ics.pl.

  • ics2qtcal.pl will parse the ics-file and import it to the qtopia PIM storage. The script should have 2 parameters: (Par1) Input ics-file e.g. std.ics (Par2) Ouput sqlite-file - e.g. qtopia_db.sqlite.
  • qtcal2ics.pl will export the qtopia PIM storage to an ics-file. The script should have 2 parameters: (Par1) Input sqlite-file - e.g. qtopia_db.sqlite (Par2) Ouput ics-file - e.g. std.ics.

For parsing the ICS-file there are several CPAN perl libraries available

After parsing a file with SQL-Commands can be generated for the ics input file of that are used by sqlite3 to import the dates in the qtopia PIM storage ics2qtcal.pl. In a first step of realisation of ics2qtcal.pl the old qtopia-db can be backuped and a full import of.

When all scripts are in place generate an icon in /usr/share/application that call and rsync command from to Openmoko device to copy the kontact-KDE ics-file parse it and copy it to the qtopia database.

 #!/usr/bin/sh
 cd ~/Applications/Qtopia
 rsync -avz -e ssh user@192.168.0.200:/home/myhome/.kde/share/apps/korganizer/std.ics std.ics
 mv qtopia_db.sqlite qtopia_db.sqlite.backup
 perl /path_to_the_script/ics2qtcal.pl std.ics qtopia_db.sqlite

192.168.0.200 is the IP of the desktop computer. It can be replaced by any IP you have your home directory with the ICS file. You need to enter a password so far. Automate this process see http://troy.jdmz.net/rsync/index.html for using rsync and ssh without authentication.

The perl script can call the sqlite3 and import the calendar data in the sqlite-database. To avoid to have the same events 2 times in the PIM storage the existing OpenMoko database should be exported first via qtcal2ics.pl removing events from the SQL imput that already exist in the SQL database. Records that have changed should be updated via SQL commands.

 ....
 mv qtopia_db.sqlite qtopia_db.sqlite.backup
 perl /path_to_the_script/qtcal2ics.pl qtopia_db.sqlite qtopia.ics 
 perl /path_to_the_script/ics2qtcal.pl std.ics qtopia_db.sqlite qtopia.ics

Remark: The perl scripts mentioned above are not programmed yet (19.01.2009), they show a solution for syncing. The development of the perl scripts ics2qtcal.pl and qtcal2ics.pl mentioned above can be helpful in converting ical formats to and from the OpenMoko device anyway.

Questions and Answers

  • Q: When will an Embedded EDS specification be available? i.e. How I can get applications to store and retrieve data from in it, in a compatible way?
  • Q: Will it out of the box synchronize with the Kontact PIM suite (KDE)?