summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorChristian Mollekopf <chrigi_1@fastmail.fm>2015-12-23 12:18:54 +0100
committerChristian Mollekopf <chrigi_1@fastmail.fm>2015-12-23 12:18:54 +0100
commit4210e5a24f700e5104333ed014c3269262b66bec (patch)
tree15bd4881d67b18265729fb2fa25fed0af6cf56af
parentb259728a4f63e022526ef86e6b5d6c62d9938d13 (diff)
downloadsink-4210e5a24f700e5104333ed014c3269262b66bec.tar.gz
sink-4210e5a24f700e5104333ed014c3269262b66bec.zip
Docs
-rw-r--r--docs/applicationdomaintypes.md8
-rw-r--r--docs/storage.md40
2 files changed, 42 insertions, 6 deletions
diff --git a/docs/applicationdomaintypes.md b/docs/applicationdomaintypes.md
index 502a349..9a50940 100644
--- a/docs/applicationdomaintypes.md
+++ b/docs/applicationdomaintypes.md
@@ -31,18 +31,19 @@ This is a proposed set of types that we will need to evolve into what we actuall
31 * Tag 31 * Tag
32 * Contact Group 32 * Contact Group
33 * Thread 33 * Thread
34 * Akonadi Resource 34 * Akonadi Resource
35 * Maildir Resource
35 36
36#### Properties 37#### Properties
37```no-highlight 38```no-highlight
38Entity: The smallest unit in the akonadi universe 39Entity: The smallest unit in the akonadi universe
39 id: unique identifier in the akonadi storage. Not persistant over db recreations and can therefore only be referenced from within the akonadi database. 40 id: unique identifier in the akonadi storage. Not persistant over db recreations and can therefore only be referenced from within the akonadi database.
40 revision: revision of the entity
41 resource: reference to AkonadiResource:id
42``` 41```
43```no-highlight 42```no-highlight
44Domain Object: 43Domain Object:
45 uid: unique identifier of the domain object. 44 uid: unique identifier of the domain object.
45 revision: revision of the entity
46 resource: reference to AkonadiResource:id of the parent resource.
46``` 47```
47```no-highlight 48```no-highlight
48Event: 49Event:
@@ -61,6 +62,7 @@ Mail:
61```no-highlight 62```no-highlight
62Akonadi Resource: 63Akonadi Resource:
63 type: The type of the resource. 64 type: The type of the resource.
65 name: The name of the resource.
64``` 66```
65```no-highlight 67```no-highlight
66Maildir Resource: 68Maildir Resource:
diff --git a/docs/storage.md b/docs/storage.md
index 26469a7..b6d73fe 100644
--- a/docs/storage.md
+++ b/docs/storage.md
@@ -86,7 +86,7 @@ Storage is split up in multiple named databases that reside in the same database
86``` 86```
87 87
88The resource can be effectively removed from disk (besides configuration), 88The resource can be effectively removed from disk (besides configuration),
89by deleting the `$RESOURCE_IDENTIFIER` directory and everything it contains. 89by deleting the directories matching `$RESOURCE_IDENTIFIER*` and everything they contain.
90 90
91#### Design Considerations 91#### Design Considerations
92* The stores are split by buffertype, so a full scan (which is done by type), doesn't require filtering by type first. The downside is that an additional lookup is required to get from revision to the data. 92* The stores are split by buffertype, so a full scan (which is done by type), doesn't require filtering by type first. The downside is that an additional lookup is required to get from revision to the data.
@@ -95,9 +95,43 @@ by deleting the `$RESOURCE_IDENTIFIER` directory and everything it contains.
95Every operation (create/delete/modify), leads to a new revision. The revision is an ever increasing number for the complete store. 95Every operation (create/delete/modify), leads to a new revision. The revision is an ever increasing number for the complete store.
96 96
97#### Design Considerations 97#### Design Considerations
98By having one revision for the complete store instead of one per type, the change replay always works across all types. This is especially important in the write-back 98By having one revision for the complete store instead of one per type, the change replay always works across all types. This is especially important in the write-back mechanism that replays the changes to the source.
99mechanism that replays the changes to the source.
100 99
100### BLOB properties
101Files are used to handle opaque large properties that should not end up in memory. BLOB properties are in their nature never queriable (extract parts of it to other properties if indexes are required).
102
103For reading:
104
105Resources...
106* store the file in ~/akonadi2/storage/$RESOURCE_IDENTIFIER_files/
107* store the filename in the blob property.
108* delete the file when the corresponding entity is deleted.
109
110Queries...
111* Copy the requested property to /tmp/akonadi2/client_files/ and provide the path in the property
112* The file is guaranteed to exist for the lifetime of the query result.
113
114Clients..
115* Load the file from disk and use it as they wish (moving is fine too)
116
117For writing:
118
119Clients..
120* Request a path from akonadi2 and store the file there.
121* Store the path of the written file in the property.
122
123Resources..
124* move the file to ~/akonadi2/storage/$RESOURCE_IDENTIFIER_files/
125* store the new path in the entity
126
127#### Design Considerations
128Using regular files as the interface has the advantages:
129* Existing mechanisms can be used to stream data directly to disk.
130* The necessary file operations can be efficiently handled depending on OS and filesystem.
131* We avoid reinventing the wheel.
132
133The copy is necessary to guarantee that the file remains for the client/resource even if the resource removes the file on it's side as part of a sync.
134The copy could be optimized by using hardlinks, which is not a portable solution though. For some next-gen copy-on-write filesystems copying is a very cheap operation.
101 135
102### Database choice 136### Database choice
103By design we're interested in key-value stores or perhaps document databases. This is because a fixed schema is not useful for this design, which makes 137By design we're interested in key-value stores or perhaps document databases. This is because a fixed schema is not useful for this design, which makes