diff options
-rw-r--r-- | docs/applicationdomaintypes.md | 86 | ||||
-rw-r--r-- | docs/clientapi.md | 30 | ||||
-rw-r--r-- | docs/design.md | 23 | ||||
-rw-r--r-- | docs/extendingakonadi.md | 11 | ||||
-rw-r--r-- | docs/storage.md | 16 |
5 files changed, 115 insertions, 51 deletions
diff --git a/docs/applicationdomaintypes.md b/docs/applicationdomaintypes.md new file mode 100644 index 0000000..b0097a6 --- /dev/null +++ b/docs/applicationdomaintypes.md | |||
@@ -0,0 +1,86 @@ | |||
1 | ## Domain Types | ||
2 | A set of standardized domain types is defined. This is necessary to decouple applications from resources (so a calendar can access events from all resources), and to have a "language" for queries. | ||
3 | |||
4 | The definition of the domain model directly affects: | ||
5 | * granularity for data retrieval (email property, or individual subject, date, ...) | ||
6 | * queriable properties for filtering and sorting (sender, id, ...) | ||
7 | |||
8 | The purpose of these domain types is strictly to be the interface and the types are not meant to be used by applications directly, or to be restricted by any other specifications (such as ical). By nature these types will be part of the evolving interface, and will need to be adjusted for every new property that an application accesses. | ||
9 | |||
10 | ### Application Domain Types | ||
11 | This is a proposed set of types that we will need to evolve into what we actually require. Hierarchical types are required to be able to query for a result set of mixed types. | ||
12 | |||
13 | * Entity | ||
14 | * Domain Object | ||
15 | * Incidence | ||
16 | * Event | ||
17 | * Todo | ||
18 | * Journal | ||
19 | * Freebusy | ||
20 | * Note | ||
21 | |||
22 | * Contact | ||
23 | * Collection | ||
24 | * Akonadi Resource | ||
25 | * Mail Folder | ||
26 | * Calendar | ||
27 | * Todolist | ||
28 | * Journal | ||
29 | * Address Book | ||
30 | * Relation | ||
31 | * Tag | ||
32 | * Contact Group | ||
33 | * Thread | ||
34 | |||
35 | #### Properties | ||
36 | ```no-highlight | ||
37 | Entity: The smallest unit in the akonadi universe | ||
38 | id: unique identifier in the akonadi storage. Not persistant over db recreations and can therefore only be referenced from within the akonadi database. | ||
39 | revision: revision of the entity | ||
40 | resource: reference to AkonadiResource:id | ||
41 | ``` | ||
42 | ```no-highlight | ||
43 | Domain Object: | ||
44 | uid: unique identifier of the domain object. | ||
45 | ``` | ||
46 | ```no-highlight | ||
47 | Event: | ||
48 | summary: A string containing a short summary of the event. | ||
49 | startDate: The start date of the event. | ||
50 | startTime: The start time of the event. Optional. | ||
51 | ``` | ||
52 | |||
53 | ### References/Hierachies | ||
54 | Some domain objects reference others, and that is often used to build hierarchies. | ||
55 | Examples are folder hierachies, tags, todo hierarchies, mail threads, contact groups, etc. | ||
56 | |||
57 | These references can be built on two levels: | ||
58 | * On the akonadi entity level: The referenced object *must* be available in local storage, and we're only linking to that specific instance. If the referenced entity is removed, the reference breaks. The reference always only references a single akonadi entity. | ||
59 | * On the domain object level: The reference can remain also if no object currently matches the reference. The reference automatically applies to new entities containing an object with the referenced uid. More than one entity can be matched if they contain the same domain object. | ||
60 | |||
61 | #### Examples | ||
62 | The following hierachies exist among others: | ||
63 | |||
64 | * Parent Collection | ||
65 | * Given by the source (path of the folder in IMAP) | ||
66 | * Parent folder "owns" the sub entity | ||
67 | * Link exists on the akonadi entity level: We specify where the entity lives, this MUST always be a single parent entity. | ||
68 | * Subtodos | ||
69 | * Given by the todo itself | ||
70 | * Not necessarly owning (though often implemented as such, similar to threading) | ||
71 | * Link exists on domain object level. | ||
72 | * Mail Threads | ||
73 | * Non owning, but a mail always only belongs to one thread. | ||
74 | * Hierarchy given by message-id references in headers and subject + date. | ||
75 | * Link exists on domain object level. | ||
76 | * Contact Groups | ||
77 | * A contact can belong to many groups and the reference is non-owning. | ||
78 | * Link exists on domain object level. | ||
79 | * Tags | ||
80 | * An entity can have many tags | ||
81 | * The tag references the entity, not the other way around. | ||
82 | * Link exists on domain object level. | ||
83 | |||
84 | #### Example queries: | ||
85 | * All mail folders of a resource | ||
86 | * All threads within date-time range | ||
diff --git a/docs/clientapi.md b/docs/clientapi.md index 59998ca..58fbb13 100644 --- a/docs/clientapi.md +++ b/docs/clientapi.md | |||
@@ -17,35 +17,11 @@ The client API consists of: | |||
17 | A set of standardized domain types is defined. This is necessary to decouple applications from resources (so a calendar can access events from all resources), and to have a "language" for queries. | 17 | A set of standardized domain types is defined. This is necessary to decouple applications from resources (so a calendar can access events from all resources), and to have a "language" for queries. |
18 | 18 | ||
19 | The definition of the domain model directly affects: | 19 | The definition of the domain model directly affects: |
20 | * granularity for data retrievel (email property, or individual subject, date, ...) | 20 | * granularity for data retrieval (email property, or individual subject, date, ...) |
21 | * queriable properties (sender, id, ...) | 21 | * queriable properties for filtering and sorting (sender, id, ...) |
22 | * properties used for sorting (10 latest email) | ||
23 | 22 | ||
24 | The purpose of these domain types is strictly to be the interface and the types are not meant to be used by applications directly, or to be restricted by any other specifications (such as ical). By nature these types will be part of the evolving interface, and will need to be adjusted for every new property that an application must understand. | 23 | The purpose of these domain types is strictly to be the interface and the types are not meant to be used by applications directly, or to be restricted by any other specifications (such as ical). By nature these types will be part of the evolving interface, and will need to be adjusted for every new property that an application must understand. |
25 | 24 | ||
26 | ### Application Domain Types | ||
27 | This is a proposed set of types that we will need to evolve into what we actually require. Hierarchical types are required to be able to query for a result set of mixed types. | ||
28 | |||
29 | * Entity | ||
30 | * Item | ||
31 | * Incidence | ||
32 | * Event | ||
33 | * Todo | ||
34 | * Journal | ||
35 | * Freebusy | ||
36 | * Note | ||
37 | |||
38 | * Contact | ||
39 | * Collection | ||
40 | * Mail Folder | ||
41 | * Calendar | ||
42 | * Tasklist | ||
43 | * Journal | ||
44 | * Contact Group | ||
45 | * Address Book | ||
46 | * Relation | ||
47 | * Tag | ||
48 | |||
49 | ## Store Facade | 25 | ## Store Facade |
50 | The store is always accessed through a store specific facade, which hides: | 26 | The store is always accessed through a store specific facade, which hides: |
51 | * store access (one store could use a database, and another one plain files) | 27 | * store access (one store could use a database, and another one plain files) |
@@ -72,7 +48,7 @@ The changeset can be recorded by the domain object adapter while the properties | |||
72 | Each modification is associated with a specific revision, which allows the synchronizer to do automatic conflict resolution. | 48 | Each modification is associated with a specific revision, which allows the synchronizer to do automatic conflict resolution. |
73 | 49 | ||
74 | ### Conflict Resolution | 50 | ### Conflict Resolution |
75 | Conflicts can occur at two points in the client: | 51 | Conflicts can occur at two points: |
76 | 52 | ||
77 | * While i.e. an editor is open and we receive an update for the same entity | 53 | * While i.e. an editor is open and we receive an update for the same entity |
78 | * After a modification is sent to the synchronizer but before it's processed | 54 | * After a modification is sent to the synchronizer but before it's processed |
diff --git a/docs/design.md b/docs/design.md index 59cdf74..fe0d214 100644 --- a/docs/design.md +++ b/docs/design.md | |||
@@ -1,4 +1,4 @@ | |||
1 | # Goals | 1 | # Design Goals |
2 | ## Axioms | 2 | ## Axioms |
3 | 1. Personal information is stored in multiple sources (address books, email stores, calendar files, ...) | 3 | 1. Personal information is stored in multiple sources (address books, email stores, calendar files, ...) |
4 | 2. These sources may local, remote or a mix of local and remote | 4 | 2. These sources may local, remote or a mix of local and remote |
@@ -39,27 +39,18 @@ Implications of the above: | |||
39 | 39 | ||
40 | # Overview | 40 | # Overview |
41 | 41 | ||
42 | # Types | 42 | ## Types |
43 | ## Domain Type | 43 | ### Domain Type |
44 | The domain types exposed in the public interface. | 44 | The domain types exposed in the public interface. |
45 | 45 | ||
46 | ## Buffer Type | 46 | ### Buffer Type |
47 | The individual buffer types as specified by the resource. The are internal types that don't necessarily have a 1:1 mapping to the domain types, although that is the default case that the default implementations expect. | 47 | The individual buffer types as specified by the resource. The are internal types that don't necessarily have a 1:1 mapping to the domain types, although that is the default case that the default implementations expect. |
48 | 48 | ||
49 | ## Steps to add support for new types | 49 | ### Change Replay |
50 | * Add new type to applicationdomaintypes.h and implement `getTypenName()` | ||
51 | * Implement `TypeImplementation<>` for updating indexes etc. | ||
52 | * Add a type.fbs default schema for the type | ||
53 | |||
54 | ## Steps for adding support for a type to a resource | ||
55 | * Add a TypeAdaptorFactory, which can either register resource specific mappers, or stick to what the default implementation in TypeImplementation provides | ||
56 | * Add a TypeFacade that injects the TypeAdaptorFactory in the GenericFacade | ||
57 | * Register the facade in the resource | ||
58 | * Add synchronization code that creates the relevant objects | ||
59 | |||
60 | # Change Replay | ||
61 | The change replay is based on the revisions in the store. Clients (and also the write-back mechanism), are informed that a new revision is available. Each client can then go through all new revisions (starting from the last seen revision), and thus update it's state to the latest revision. | 50 | The change replay is based on the revisions in the store. Clients (and also the write-back mechanism), are informed that a new revision is available. Each client can then go through all new revisions (starting from the last seen revision), and thus update it's state to the latest revision. |
62 | 51 | ||
52 | # Client API | ||
53 | |||
63 | # Tradeoffs/Design Decisions | 54 | # Tradeoffs/Design Decisions |
64 | * Key-Value store instead of relational | 55 | * Key-Value store instead of relational |
65 | * `+` Schemaless, easier to evolve | 56 | * `+` Schemaless, easier to evolve |
diff --git a/docs/extendingakonadi.md b/docs/extendingakonadi.md new file mode 100644 index 0000000..fe38e87 --- /dev/null +++ b/docs/extendingakonadi.md | |||
@@ -0,0 +1,11 @@ | |||
1 | ## Steps to add support for new types | ||
2 | * Add new type to applicationdomaintypes.h and implement `getTypenName()` | ||
3 | * Implement `TypeImplementation<>` for updating indexes etc. | ||
4 | * Add a type.fbs default schema for the type | ||
5 | |||
6 | ## Steps for adding support for a type to a resource | ||
7 | * Add a TypeAdaptorFactory, which can either register resource specific mappers, or stick to what the default implementation in TypeImplementation provides | ||
8 | * Add a TypeFacade that injects the TypeAdaptorFactory in the GenericFacade | ||
9 | * Register the facade in the resource | ||
10 | * Add synchronization code that creates the relevant objects | ||
11 | |||
diff --git a/docs/storage.md b/docs/storage.md index 3a8d74a..26469a7 100644 --- a/docs/storage.md +++ b/docs/storage.md | |||
@@ -163,7 +163,7 @@ Other useful properties: | |||
163 | * No updates since September 2013 | 163 | * No updates since September 2013 |
164 | * http://unqlite.org | 164 | * http://unqlite.org |
165 | * bad performance with large database (looks like O(n)) | 165 | * bad performance with large database (looks like O(n)) |
166 | * like lmdb roughly 2*datasize | 166 | * like lmdb roughly 2\*datasize |
167 | * includes a document store | 167 | * includes a document store |
168 | * mmapped ready access | 168 | * mmapped ready access |
169 | * reading about 30% the speed of lmdb | 169 | * reading about 30% the speed of lmdb |
@@ -181,9 +181,9 @@ Since indexes always need to be updated they directly affect how fast we can wri | |||
181 | * fast fulltext searching | 181 | * fast fulltext searching |
182 | * No MVCC concurrency | 182 | * No MVCC concurrency |
183 | * Only supports one writer at a time | 183 | * Only supports one writer at a time |
184 | * If a reader is reading blocks that have now been changed by a writer, it throws a DatabaseModifiedException. This means most of the Xapian code needs to be in wihle (1) { try { .. } catch () } blocks and needs to be able to start from scratch. | 184 | * If a reader is reading blocks that have now been changed by a writer, it throws a DatabaseModifiedException. This means most of the Xapian code needs to be in `while (1) { try { .. } catch () }` blocks and needs to be able to start from scratch. |
185 | * Wildcard searching (as of 2015-01) isn't ideal. It works by expanding the word into all other words in the query and that typically makes the query size huge. This huge query is then sent to the database. Baloo has had to configure this expanding of terms so that it consumes less memory. | 185 | * Wildcard searching (as of 2015-01) isn't ideal. It works by expanding the word into all other words in the query and that typically makes the query size huge. This huge query is then sent to the database. Baloo has had to configure this expanding of terms so that it consumes less memory. |
186 | * Non existent UTF support - It does not support text normalization and splitting the terms at custom characters such as '_'. | 186 | * Non existent UTF support - It does not support text normalization and splitting the terms at custom characters such as '\_'. |
187 | * lmdb: | 187 | * lmdb: |
188 | * sorted keys | 188 | * sorted keys |
189 | * sorted duplicate keys | 189 | * sorted duplicate keys |
@@ -200,9 +200,9 @@ Since indexes always need to be updated they directly affect how fast we can wri | |||
200 | 200 | ||
201 | ## Useful Resources | 201 | ## Useful Resources |
202 | * LMDB | 202 | * LMDB |
203 | * Wikipedia for a good overview: https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database | 203 | * Wikipedia for a good overview: <https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database> |
204 | * Benchmarks: http://symas.com/mdb/microbench/ | 204 | * Benchmarks: <http://symas.com/mdb/microbench/> |
205 | * Tradeoffs: http://symas.com/is-lmdb-a-leveldb-killer/ | 205 | * Tradeoffs: <http://symas.com/is-lmdb-a-leveldb-killer/> |
206 | * Disk space benchmark: http://symas.com/mdb/ondisk/ | 206 | * Disk space benchmark: <http://symas.com/mdb/ondisk/> |
207 | * LMDB instead of Kyoto Cabinet as redis backend: http://www.anchor.com.au/blog/2013/05/second-strike-with-lightning/ | 207 | * LMDB instead of Kyoto Cabinet as redis backend: <http://www.anchor.com.au/blog/2013/05/second-strike-with-lightning/> |
208 | 208 | ||