Opimd redesign

From Openmoko

(Difference between revisions)
Jump to: navigation, search
 
(19 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
'''STATUS UPDATE:''' Finished (TAsn) working on it. (It's in opimd-redesign branch if you want), removed the backend system completely, now using only sqlite. Types are now stored in db. You can now pass multiple options per field, i.e query for all the contacts that their name is Tom, Dick or Harry. You can now query by type using the special prefix $. You just use $type_name as if it was a field_name and everything else is the same. There's also a convert script that converts from the old opimd db system to the new one. Contact me if you need any help. Works flawlessly, and very fast.
 +
 +
I marked features in page in two ways, DONE, and PRODUCTION, DONE means code is there and works, PRODUCTION means it's committed in master branch.
 +
 
This page will gather all the suggestions (and will be a decent design sketch) for the redesign of [[opimd]]. After many discussions in IRC we decided opimd needs a redesign.
 
This page will gather all the suggestions (and will be a decent design sketch) for the redesign of [[opimd]]. After many discussions in IRC we decided opimd needs a redesign.
  
Line 7: Line 11:
  
 
Info about those changes will be shown here as soon as we'll have time to write it down.
 
Info about those changes will be shown here as soon as we'll have time to write it down.
 +
 +
= Backends =
 +
DONE
 +
 +
Remove the backend system, use only sqlite. Let 3rd party software (pisi?) handle backends (importing and exporting). This will be a lot faster and a lot esaier to handle. Furthermore, it'll finally let us have a permanent contact id to use internally (for linking between contacts for example).
  
 
= Changes in fields =
 
= Changes in fields =
−
ATM opimd just lets you add fields as you wish, this is bad, since you can't really categorize fields and have to use weird hacks like the "tel:" prefix.
+
'''UPDATE: What we decided to go with: PRODUCTION'''
  
−
I'm suggesting a move to a configuration you'll have the following dbus methods:
+
'''IMPORTANT:''' This change invalidates the uri prefixing ("tel:" in phone numbers) that was used in opimd up until now.
−
* ListFields
+
−
* AddFiled
+
−
* DelField
+
−
* GetField
+
  
−
each field will have a type (phone number would be a type for instance) that will guide opimd on how to compare items of this type, again, in the case of phone numbers, use normalization.
+
Per domain:
  
−
The user of the API will be able to add fields and their types whenever he wants (and also remove such) and that will let us control opimd in a sane manner.
+
org.freesmartphone.PIM.Type.Add(name, type) <--- Adds the name and type
  
−
I think we also need a couple of default fields of special types:
+
org.freesmartphone.PIM.Type.List() <---- returns the list of available fields, format name:type.
−
* Phone - returns the default phone (of all the cells with type phone)
+
−
* Name - returns  "First Last, nick" or something like that.
+
−
* SIP - returns default contact info for sip
+
−
* SMS - returns default phone number for SMS etc etc.
+
  
−
This will make everything easier for the users of the API to handle and will be sane!
+
org.freesmartphone.PIM.Type.Delete(name) <-- returns nothing
 +
 
 +
org.freesmartphone.PIM.Type.Get(name) <--- returns the type of the field.
 +
 
 +
General for all of opimd:
 +
 
 +
org.freesmartphone.PIM.Types.List() <---- returns the list of available types in a tuple.
 +
 
 +
Types.List() returns the same thing for all the domains.
 +
 
 +
==API==
 +
This is the complete API (in freesmartphone docs).
 +
There are a couple of default fields, those fields can be overwritten, but if no value exists for them, they are used (also listed in the docs)
 +
 
 +
* http://git.freesmartphone.org/?p=specs.git;a=blob_plain;f=html/org.freesmartphone.PIM.Types.html;hb=HEAD
 +
* http://git.freesmartphone.org/?p=specs.git;a=blob_plain;f=html/org.freesmartphone.PIM.Fields.html;hb=HEAD
 +
 
 +
===Existing Types===
 +
# objectpath
 +
# phonenumber
 +
# address
 +
# email
 +
# name
 +
# date
 +
# uri
 +
# photo
 +
# text
 +
# longtext
 +
# boolean
 +
# timezone
 +
# number
 +
# integer
 +
# generic
 +
 
 +
 
 +
How to use:
 +
You can just use opimd like you have used it until now. '''BUT''' if you want opimd to work with number resolving (or any other field handling) you should add a matching type.
 +
So let's say to add support for the "Cellphone" field, we'll just have to:
 +
 
 +
Fields.Add("Cellphone", "phonenumber")
 +
 
 +
or for supporting "Nickname" we'll have to:
 +
 
 +
Fields.Add("Nickname", "name")
 +
 
 +
This will let opimd assume things about the values inside (for instance assume that phonenumber is important and should probably be an indexed property). But also will let you (and more importantly the UI apps) work in a sane number knowing the types (and let's say, adding icons per type).
 +
 
 +
The Types.List() is just for helping you know what's currently supported in opimd.
 +
 
 +
As I said, you can still use fields without types, but then the apps (and opimd) will not be able to search them correctly.
 +
 
 +
==Future development==
 +
DONE
 +
 
 +
* Enable querying by type, i.e, query where "$phonenumber" = 123 will search in all the fields that have the type phonenumber.
  
 
= Changes in query method =
 
= Changes in query method =
−
I don't get why we use this querying method, it jsut makes everything weird hard to handle.
+
NOTE: keep the old way, but also implement this:
  
 +
I don't get why we use this querying method, it just makes everything weird hard to handle.
  
 
= Internal design =
 
= Internal design =
−
opimd should internally use sqlite for caching as it's faster, easier to handle and generally better.
+
DONE
 +
 
 +
opimd should internally use sqlite and sqlite only.
  
 
= Query improvements =
 
= Query improvements =
 +
DONE
 +
 
It should really be possible to ask opimd for several contacts at once, i.e ask him for the contacts "Tom, dick and harry" or for the numbers "123, 435 and 546546" at one query, because querying opimd for 70 different contacts at one time (phonelog) is terribly slow.
 
It should really be possible to ask opimd for several contacts at once, i.e ask him for the contacts "Tom, dick and harry" or for the numbers "123, 435 and 546546" at one query, because querying opimd for 70 different contacts at one time (phonelog) is terribly slow.
  
 
Furthermore it should be able to ask for a "join" of backends when requesting from one.
 
Furthermore it should be able to ask for a "join" of backends when requesting from one.
 
i.e, I want to be able to ask opimd for the messages and the corresponding contacts for each message in one query, or even ask opimd for all the messages that match all the numbers from the contact "Charley".  
 
i.e, I want to be able to ask opimd for the messages and the corresponding contacts for each message in one query, or even ask opimd for all the messages that match all the numbers from the contact "Charley".  
 +
 +
Query reply should be possible to sort differently depending on the national character encoding.
 +
 
[[Category:FSO]]
 
[[Category:FSO]]
 
[[Category:Middleware]]
 
[[Category:Middleware]]
 
[[Category:PIM]]
 
[[Category:PIM]]

Latest revision as of 23:30, 21 February 2010

STATUS UPDATE: Finished (TAsn) working on it. (It's in opimd-redesign branch if you want), removed the backend system completely, now using only sqlite. Types are now stored in db. You can now pass multiple options per field, i.e query for all the contacts that their name is Tom, Dick or Harry. You can now query by type using the special prefix $. You just use $type_name as if it was a field_name and everything else is the same. There's also a convert script that converts from the old opimd db system to the new one. Contact me if you need any help. Works flawlessly, and very fast.

I marked features in page in two ways, DONE, and PRODUCTION, DONE means code is there and works, PRODUCTION means it's committed in master branch.

This page will gather all the suggestions (and will be a decent design sketch) for the redesign of opimd. After many discussions in IRC we decided opimd needs a redesign.

Main topics that came up are: switching opimd to use sqlite for internal operations and adding configuration for types instead of the use of URIs. Many of the ideas that came up concerning the second idea are represented in this IRC log: http://pastebin.com/mb7fbf10 and in the logs from #openmoko-cdevel@Freenode on 09-10 of october 2009.

Furthermore, it has been suggested than a change in the sqlite backend db design is also needed.


Info about those changes will be shown here as soon as we'll have time to write it down.

Contents

[edit] Backends

DONE

Remove the backend system, use only sqlite. Let 3rd party software (pisi?) handle backends (importing and exporting). This will be a lot faster and a lot esaier to handle. Furthermore, it'll finally let us have a permanent contact id to use internally (for linking between contacts for example).

[edit] Changes in fields

UPDATE: What we decided to go with: PRODUCTION

IMPORTANT: This change invalidates the uri prefixing ("tel:" in phone numbers) that was used in opimd up until now.

Per domain:

org.freesmartphone.PIM.Type.Add(name, type) <--- Adds the name and type

org.freesmartphone.PIM.Type.List() <---- returns the list of available fields, format name:type.

org.freesmartphone.PIM.Type.Delete(name) <-- returns nothing

org.freesmartphone.PIM.Type.Get(name) <--- returns the type of the field.

General for all of opimd:

org.freesmartphone.PIM.Types.List() <---- returns the list of available types in a tuple.

Types.List() returns the same thing for all the domains.

[edit] API

This is the complete API (in freesmartphone docs). There are a couple of default fields, those fields can be overwritten, but if no value exists for them, they are used (also listed in the docs)

[edit] Existing Types

  1. objectpath
  2. phonenumber
  3. address
  4. email
  5. name
  6. date
  7. uri
  8. photo
  9. text
  10. longtext
  11. boolean
  12. timezone
  13. number
  14. integer
  15. generic


How to use: You can just use opimd like you have used it until now. BUT if you want opimd to work with number resolving (or any other field handling) you should add a matching type. So let's say to add support for the "Cellphone" field, we'll just have to:

Fields.Add("Cellphone", "phonenumber")

or for supporting "Nickname" we'll have to:

Fields.Add("Nickname", "name")

This will let opimd assume things about the values inside (for instance assume that phonenumber is important and should probably be an indexed property). But also will let you (and more importantly the UI apps) work in a sane number knowing the types (and let's say, adding icons per type).

The Types.List() is just for helping you know what's currently supported in opimd.

As I said, you can still use fields without types, but then the apps (and opimd) will not be able to search them correctly.

[edit] Future development

DONE

  • Enable querying by type, i.e, query where "$phonenumber" = 123 will search in all the fields that have the type phonenumber.

[edit] Changes in query method

NOTE: keep the old way, but also implement this:

I don't get why we use this querying method, it just makes everything weird hard to handle.

[edit] Internal design

DONE

opimd should internally use sqlite and sqlite only.

[edit] Query improvements

DONE

It should really be possible to ask opimd for several contacts at once, i.e ask him for the contacts "Tom, dick and harry" or for the numbers "123, 435 and 546546" at one query, because querying opimd for 70 different contacts at one time (phonelog) is terribly slow.

Furthermore it should be able to ask for a "join" of backends when requesting from one. i.e, I want to be able to ask opimd for the messages and the corresponding contacts for each message in one query, or even ask opimd for all the messages that match all the numbers from the contact "Charley".

Query reply should be possible to sort differently depending on the national character encoding.

Retrieved from "http://wiki.openmoko.org/index.php?title=Opimd_redesign&oldid=79736"
Personal tools

This page will gather all the suggestions (and will be a decent design sketch) for the redesign of opimd. After many discussions in IRC we decided opimd needs a redesign.

Main topics that came up are: switching opimd to use sqlite for internal operations and adding configuration for types instead of the use of URIs. Many of the ideas that came up concerning the second idea are represented in this IRC log: http://pastebin.com/mb7fbf10 and in the logs from #openmoko-cdevel@Freenode on 09-10 of october 2009.

Furthermore, it has been suggested than a change in the sqlite backend db design is also needed.


Info about those changes will be shown here as soon as we'll have time to write it down.

Changes in fields

ATM opimd just lets you add fields as you wish, this is bad, since you can't really categorize fields and have to use weird hacks like the "tel:" prefix.

I'm suggesting a move to a configuration you'll have the following dbus methods:

  • ListFields
  • AddFiled
  • DelField
  • GetField

each field will have a type (phone number would be a type for instance) that will guide opimd on how to compare items of this type, again, in the case of phone numbers, use normalization.

The user of the API will be able to add fields and their types whenever he wants (and also remove such) and that will let us control opimd in a sane manner.

I think we also need a couple of default fields of special types:

  • Phone - returns the default phone (of all the cells with type phone)
  • Name - returns "First Last, nick" or something like that.
  • SIP - returns default contact info for sip
  • SMS - returns default phone number for SMS etc etc.

This will make everything easier for the users of the API to handle and will be sane!

Changes in query method

I don't get why we use this querying method, it jsut makes everything weird hard to handle.


Internal design

opimd should internally use sqlite for caching as it's faster, easier to handle and generally better.

Query improvements

It should really be possible to ask opimd for several contacts at once, i.e ask him for the contacts "Tom, dick and harry" or for the numbers "123, 435 and 546546" at one query, because querying opimd for 70 different contacts at one time (phonelog) is terribly slow.

Furthermore it should be able to ask for a "join" of backends when requesting from one. i.e, I want to be able to ask opimd for the messages and the corresponding contacts for each message in one query, or even ask opimd for all the messages that match all the numbers from the contact "Charley".