Opimd redesign

From Openmoko

(Difference between revisions)
Jump to: navigation, search
(→‎Changes in fields)
 
(10 intermediate revisions by one user not shown)
Line 1: Line 1:
−
'''STATUS UPDATE:''' Got mickey and dos1's agreement (they are the "leaders" of opimd). Will start (actually will help dos1) working on it asap. If you want to add anything, please add now.
+
'''STATUS UPDATE:''' Finished (TAsn) working on it. (It's in opimd-redesign branch if you want), removed the backend system completely, now using only sqlite. Types are now stored in db. You can now pass multiple options per field, i.e query for all the contacts that their name is Tom, Dick or Harry. You can now query by type using the special prefix $. You just use $type_name as if it was a field_name and everything else is the same. There's also a convert script that converts from the old opimd db system to the new one. Contact me if you need any help. Works flawlessly, and very fast.
 +
 
 +
I marked features in page in two ways, DONE, and PRODUCTION, DONE means code is there and works, PRODUCTION means it's committed in master branch.
  
 
This page will gather all the suggestions (and will be a decent design sketch) for the redesign of [[opimd]]. After many discussions in IRC we decided opimd needs a redesign.
 
This page will gather all the suggestions (and will be a decent design sketch) for the redesign of [[opimd]]. After many discussions in IRC we decided opimd needs a redesign.
Line 10: Line 12:
 
Info about those changes will be shown here as soon as we'll have time to write it down.
 
Info about those changes will be shown here as soon as we'll have time to write it down.
  
−
= Changes in fields =
+
= Backends =
−
ATM opimd just lets you add fields as you wish, this is bad, since you can't really categorize fields and have to use weird hacks like the "tel:" prefix.
+
DONE
  
−
I'm suggesting a move to a configuration you'll have the following dbus methods:
+
Remove the backend system, use only sqlite. Let 3rd party software (pisi?) handle backends (importing and exporting). This will be a lot faster and a lot esaier to handle. Furthermore, it'll finally let us have a permanent contact id to use internally (for linking between contacts for example).
−
* ListFields
+
−
* AddFiled
+
−
* DelField
+
−
* GetField
+
−
 
+
−
each field will have a type (phone number would be a type for instance) that will guide opimd on how to compare items of this type, again, in the case of phone numbers, use normalization.
+
−
 
+
−
The user of the API will be able to add fields and their types whenever he wants (and also remove such) and that will let us control opimd in a sane manner.
+
−
 
+
−
I think we also need a couple of default fields of special types:
+
−
* Phone - returns the default phone (of all the cells with type phone)
+
−
* Name - returns  "First Last, nick" or something like that.
+
−
* SIP - returns default contact info for sip
+
−
* SMS - returns default phone number for SMS etc etc.
+
−
 
+
−
This will make everything easier for the users of the API to handle and will be sane!
+
  
 +
= Changes in fields =
 +
'''UPDATE: What we decided to go with: PRODUCTION'''
  
−
'''Explanation of exactly what I meant (TAsn)'''
+
'''IMPORTANT:''' This change invalidates the uri prefixing ("tel:" in phone numbers) that was used in opimd up until now.
  
−
/org/freesmartphone/PIM/Contacts
+
Per domain:
  
 
org.freesmartphone.PIM.Type.Add(name, type) <--- Adds the name and type
 
org.freesmartphone.PIM.Type.Add(name, type) <--- Adds the name and type
Line 40: Line 28:
 
org.freesmartphone.PIM.Type.List() <---- returns the list of available fields, format name:type.
 
org.freesmartphone.PIM.Type.List() <---- returns the list of available fields, format name:type.
  
−
org.freesmartphone.PIM.Type.Delete(name) <-- return nothing
+
org.freesmartphone.PIM.Type.Delete(name) <-- returns nothing
  
−
org.freesmartphone.PIM.Type.Get(name) <--- returns type
+
org.freesmartphone.PIM.Type.Get(name) <--- returns the type of the field.
  
−
org.freesmartphone.PIM.Types.List() <---- returns the list of available fields, format name:type.
+
General for all of opimd:
  
−
Where Types.List() returns the same thing for all the domains.
+
org.freesmartphone.PIM.Types.List() <---- returns the list of available types in a tuple.
  
 +
Types.List() returns the same thing for all the domains.
  
−
'''Next section may be outdated:'''
+
==API==
 +
This is the complete API (in freesmartphone docs).
 +
There are a couple of default fields, those fields can be overwritten, but if no value exists for them, they are used (also listed in the docs)
  
−
I want anyone to be able to add future types (to search by) with no special opimd handling, i.e I want to be able to add IRC although you don't know how to handle it, name of the type will be irc, type of the type will be text(limited to X chars, he should state that when making the type, or maybe unlimited if he wants) or he can also set the type to "numeric", "int" etc etc.
+
* http://git.freesmartphone.org/?p=specs.git;a=blob_plain;f=html/org.freesmartphone.PIM.Types.html;hb=HEAD
 +
* http://git.freesmartphone.org/?p=specs.git;a=blob_plain;f=html/org.freesmartphone.PIM.Fields.html;hb=HEAD
  
−
This way he'll bind a type, called "IRC" that can contain text values only, this means that if he'll want to add a field called "Main-nick" he'll be able to set the type to "irc" (the one that we set earlier) that'll let him search in all the irc nicks easily and will also let opimd know what to expect in this type. (string limited to 300 chars for instance).
+
===Existing Types===
 +
# objectpath
 +
# phonenumber
 +
# address
 +
# email
 +
# name
 +
# date
 +
# uri
 +
# photo
 +
# text
 +
# longtext
 +
# boolean
 +
# timezone
 +
# number
 +
# integer
 +
# generic
  
  
 +
How to use:
 +
You can just use opimd like you have used it until now. '''BUT''' if you want opimd to work with number resolving (or any other field handling) you should add a matching type.
 +
So let's say to add support for the "Cellphone" field, we'll just have to:
  
−
Continuing with the IRC example I'll be able to add to the conact tom the following fields:
+
Fields.Add("Cellphone", "phonenumber")
  
−
Main-nick : "TAsn"
+
or for supporting "Nickname" we'll have to:
  
−
Alt-nick  : "TAsnn"
+
Fields.Add("Nickname", "name")
  
 +
This will let opimd assume things about the values inside (for instance assume that phonenumber is important and should probably be an indexed property). But also will let you (and more importantly the UI apps) work in a sane number knowing the types (and let's say, adding icons per type).
  
 +
The Types.List() is just for helping you know what's currently supported in opimd.
  
−
Though I won't be able to do that before I add the type (like this:
+
As I said, you can still use fields without types, but then the apps (and opimd) will not be able to search them correctly.
  
−
org.freesmartphone.PIM.Fields.Add("Alt-nick", "IRC")
+
==Future development==
 +
DONE
  
−
And I won't be able to do that before I do:
+
* Enable querying by type, i.e, query where "$phonenumber" = 123 will search in all the fields that have the type phonenumber.
−
 
+
−
org.freesmartphone.PIM.Types.Add("IRC", "text") (phonenumber/text/date/etc will be predefined by opimd, maybe even add a method to expose what's available there, but that can not be changed by API the "text" and etc types are set by opimd and opimd only this should probably also include the calculated type type, i.e, create a type that calculates concat(Name, SurName) or something like that, like explained in this document.)
+
−
 
+
−
 
+
−
 
+
−
This lets the APPS to easily set types for fields and for opimd to easily know where to search (When looking for IRC), what to expect when searching (a text field, not a number that's needed to be normalized, etc).
+
−
 
+
−
So, in our example in order to know who of the contacts posses the nick "TAsn" we'll just need to search all of the fields with the type IRC (user of the API will know to ask it). That's about it, I think.
+
−
 
+
−
I hope it's clean enough, let me know.
+
  
 
= Changes in query method =
 
= Changes in query method =
Line 85: Line 88:
  
 
I don't get why we use this querying method, it just makes everything weird hard to handle.
 
I don't get why we use this querying method, it just makes everything weird hard to handle.
−
 
−
Furthermore, it'll be easier and faster to get only "phone number" fields for matching (in opimd), the user of the API will just be able to tell it "find me a phone_number field that matches this number" and will not force us to check each field like we do atm.
 
  
 
= Internal design =
 
= Internal design =
−
opimd should internally use sqlite for caching as it's faster, easier to handle and generally better.
+
DONE
 +
 
 +
opimd should internally use sqlite and sqlite only.
  
 
= Query improvements =
 
= Query improvements =
 +
DONE
 +
 
It should really be possible to ask opimd for several contacts at once, i.e ask him for the contacts "Tom, dick and harry" or for the numbers "123, 435 and 546546" at one query, because querying opimd for 70 different contacts at one time (phonelog) is terribly slow.
 
It should really be possible to ask opimd for several contacts at once, i.e ask him for the contacts "Tom, dick and harry" or for the numbers "123, 435 and 546546" at one query, because querying opimd for 70 different contacts at one time (phonelog) is terribly slow.
  

Latest revision as of 23:30, 21 February 2010

STATUS UPDATE: Finished (TAsn) working on it. (It's in opimd-redesign branch if you want), removed the backend system completely, now using only sqlite. Types are now stored in db. You can now pass multiple options per field, i.e query for all the contacts that their name is Tom, Dick or Harry. You can now query by type using the special prefix $. You just use $type_name as if it was a field_name and everything else is the same. There's also a convert script that converts from the old opimd db system to the new one. Contact me if you need any help. Works flawlessly, and very fast.

I marked features in page in two ways, DONE, and PRODUCTION, DONE means code is there and works, PRODUCTION means it's committed in master branch.

This page will gather all the suggestions (and will be a decent design sketch) for the redesign of opimd. After many discussions in IRC we decided opimd needs a redesign.

Main topics that came up are: switching opimd to use sqlite for internal operations and adding configuration for types instead of the use of URIs. Many of the ideas that came up concerning the second idea are represented in this IRC log: http://pastebin.com/mb7fbf10 and in the logs from #openmoko-cdevel@Freenode on 09-10 of october 2009.

Furthermore, it has been suggested than a change in the sqlite backend db design is also needed.


Info about those changes will be shown here as soon as we'll have time to write it down.

Contents

[edit] Backends

DONE

Remove the backend system, use only sqlite. Let 3rd party software (pisi?) handle backends (importing and exporting). This will be a lot faster and a lot esaier to handle. Furthermore, it'll finally let us have a permanent contact id to use internally (for linking between contacts for example).

[edit] Changes in fields

UPDATE: What we decided to go with: PRODUCTION

IMPORTANT: This change invalidates the uri prefixing ("tel:" in phone numbers) that was used in opimd up until now.

Per domain:

org.freesmartphone.PIM.Type.Add(name, type) <--- Adds the name and type

org.freesmartphone.PIM.Type.List() <---- returns the list of available fields, format name:type.

org.freesmartphone.PIM.Type.Delete(name) <-- returns nothing

org.freesmartphone.PIM.Type.Get(name) <--- returns the type of the field.

General for all of opimd:

org.freesmartphone.PIM.Types.List() <---- returns the list of available types in a tuple.

Types.List() returns the same thing for all the domains.

[edit] API

This is the complete API (in freesmartphone docs). There are a couple of default fields, those fields can be overwritten, but if no value exists for them, they are used (also listed in the docs)

[edit] Existing Types

  1. objectpath
  2. phonenumber
  3. address
  4. email
  5. name
  6. date
  7. uri
  8. photo
  9. text
  10. longtext
  11. boolean
  12. timezone
  13. number
  14. integer
  15. generic


How to use: You can just use opimd like you have used it until now. BUT if you want opimd to work with number resolving (or any other field handling) you should add a matching type. So let's say to add support for the "Cellphone" field, we'll just have to:

Fields.Add("Cellphone", "phonenumber")

or for supporting "Nickname" we'll have to:

Fields.Add("Nickname", "name")

This will let opimd assume things about the values inside (for instance assume that phonenumber is important and should probably be an indexed property). But also will let you (and more importantly the UI apps) work in a sane number knowing the types (and let's say, adding icons per type).

The Types.List() is just for helping you know what's currently supported in opimd.

As I said, you can still use fields without types, but then the apps (and opimd) will not be able to search them correctly.

[edit] Future development

DONE

  • Enable querying by type, i.e, query where "$phonenumber" = 123 will search in all the fields that have the type phonenumber.

[edit] Changes in query method

NOTE: keep the old way, but also implement this:

I don't get why we use this querying method, it just makes everything weird hard to handle.

[edit] Internal design

DONE

opimd should internally use sqlite and sqlite only.

[edit] Query improvements

DONE

It should really be possible to ask opimd for several contacts at once, i.e ask him for the contacts "Tom, dick and harry" or for the numbers "123, 435 and 546546" at one query, because querying opimd for 70 different contacts at one time (phonelog) is terribly slow.

Furthermore it should be able to ask for a "join" of backends when requesting from one. i.e, I want to be able to ask opimd for the messages and the corresponding contacts for each message in one query, or even ask opimd for all the messages that match all the numbers from the contact "Charley".

Query reply should be possible to sort differently depending on the national character encoding.

Retrieved from "http://wiki.openmoko.org/index.php?title=Opimd_redesign&oldid=79736"
Personal tools

STATUS UPDATE: Got mickey and dos1's agreement (they are the "leaders" of opimd). Will start (actually will help dos1) working on it asap. If you want to add anything, please add now.

This page will gather all the suggestions (and will be a decent design sketch) for the redesign of opimd. After many discussions in IRC we decided opimd needs a redesign.

Main topics that came up are: switching opimd to use sqlite for internal operations and adding configuration for types instead of the use of URIs. Many of the ideas that came up concerning the second idea are represented in this IRC log: http://pastebin.com/mb7fbf10 and in the logs from #openmoko-cdevel@Freenode on 09-10 of october 2009.

Furthermore, it has been suggested than a change in the sqlite backend db design is also needed.


Info about those changes will be shown here as soon as we'll have time to write it down.

Changes in fields

ATM opimd just lets you add fields as you wish, this is bad, since you can't really categorize fields and have to use weird hacks like the "tel:" prefix.

I'm suggesting a move to a configuration you'll have the following dbus methods:

  • ListFields
  • AddFiled
  • DelField
  • GetField

each field will have a type (phone number would be a type for instance) that will guide opimd on how to compare items of this type, again, in the case of phone numbers, use normalization.

The user of the API will be able to add fields and their types whenever he wants (and also remove such) and that will let us control opimd in a sane manner.

I think we also need a couple of default fields of special types:

  • Phone - returns the default phone (of all the cells with type phone)
  • Name - returns "First Last, nick" or something like that.
  • SIP - returns default contact info for sip
  • SMS - returns default phone number for SMS etc etc.

This will make everything easier for the users of the API to handle and will be sane!


Explanation of exactly what I meant (TAsn)

/org/freesmartphone/PIM/Contacts

org.freesmartphone.PIM.Type.Add(name, type) <--- Adds the name and type

org.freesmartphone.PIM.Type.List() <---- returns the list of available fields, format name:type.

org.freesmartphone.PIM.Type.Delete(name) <-- return nothing

org.freesmartphone.PIM.Type.Get(name) <--- returns type

org.freesmartphone.PIM.Types.List() <---- returns the list of available fields, format name:type.

Where Types.List() returns the same thing for all the domains.


Next section may be outdated:

I want anyone to be able to add future types (to search by) with no special opimd handling, i.e I want to be able to add IRC although you don't know how to handle it, name of the type will be irc, type of the type will be text(limited to X chars, he should state that when making the type, or maybe unlimited if he wants) or he can also set the type to "numeric", "int" etc etc.

This way he'll bind a type, called "IRC" that can contain text values only, this means that if he'll want to add a field called "Main-nick" he'll be able to set the type to "irc" (the one that we set earlier) that'll let him search in all the irc nicks easily and will also let opimd know what to expect in this type. (string limited to 300 chars for instance).


Continuing with the IRC example I'll be able to add to the conact tom the following fields:

Main-nick : "TAsn"

Alt-nick  : "TAsnn"


Though I won't be able to do that before I add the type (like this:

org.freesmartphone.PIM.Fields.Add("Alt-nick", "IRC")

And I won't be able to do that before I do:

org.freesmartphone.PIM.Types.Add("IRC", "text") (phonenumber/text/date/etc will be predefined by opimd, maybe even add a method to expose what's available there, but that can not be changed by API the "text" and etc types are set by opimd and opimd only this should probably also include the calculated type type, i.e, create a type that calculates concat(Name, SurName) or something like that, like explained in this document.)


This lets the APPS to easily set types for fields and for opimd to easily know where to search (When looking for IRC), what to expect when searching (a text field, not a number that's needed to be normalized, etc).

So, in our example in order to know who of the contacts posses the nick "TAsn" we'll just need to search all of the fields with the type IRC (user of the API will know to ask it). That's about it, I think.

I hope it's clean enough, let me know.

Changes in query method

NOTE: keep the old way, but also implement this:

I don't get why we use this querying method, it just makes everything weird hard to handle.

Furthermore, it'll be easier and faster to get only "phone number" fields for matching (in opimd), the user of the API will just be able to tell it "find me a phone_number field that matches this number" and will not force us to check each field like we do atm.

Internal design

opimd should internally use sqlite for caching as it's faster, easier to handle and generally better.

Query improvements

It should really be possible to ask opimd for several contacts at once, i.e ask him for the contacts "Tom, dick and harry" or for the numbers "123, 435 and 546546" at one query, because querying opimd for 70 different contacts at one time (phonelog) is terribly slow.

Furthermore it should be able to ask for a "join" of backends when requesting from one. i.e, I want to be able to ask opimd for the messages and the corresponding contacts for each message in one query, or even ask opimd for all the messages that match all the numbers from the contact "Charley".

Query reply should be possible to sort differently depending on the national character encoding.