summaryrefslogtreecommitdiffstats
path: root/docs/clientapi.md
diff options
context:
space:
mode:
authorChristian Mollekopf <chrigi_1@fastmail.fm>2016-02-08 15:38:45 +0100
committerChristian Mollekopf <chrigi_1@fastmail.fm>2016-02-08 15:38:45 +0100
commitb8200434209c317ebc4883b9f87513991bae33e3 (patch)
tree00164031c5b39eec7a5969289de21651d328d038 /docs/clientapi.md
parent0376079b862cf38973a59336f3843bca2788c576 (diff)
downloadsink-b8200434209c317ebc4883b9f87513991bae33e3.tar.gz
sink-b8200434209c317ebc4883b9f87513991bae33e3.zip
Documentation
Diffstat (limited to 'docs/clientapi.md')
-rw-r--r--docs/clientapi.md129
1 files changed, 2 insertions, 127 deletions
diff --git a/docs/clientapi.md b/docs/clientapi.md
index 219f972..be8ff19 100644
--- a/docs/clientapi.md
+++ b/docs/clientapi.md
@@ -13,16 +13,6 @@ The client API consists of:
13* property-level on-demand loading of data 13* property-level on-demand loading of data
14* streaming support for large properties (attachments) 14* streaming support for large properties (attachments)
15 15
16## Domain Types
17A set of standardized domain types is defined. This is necessary to decouple applications from resources (so a calendar can access events from all resources), and to have a "language" for queries.
18
19The definition of the domain model directly affects:
20
21* granularity for data retrieval (email property, or individual subject, date, ...)
22* queriable properties for filtering and sorting (sender, id, ...)
23
24The purpose of these domain types is strictly to be the interface and the types are not necessarily meant to be used by applications directly, or to be restricted by any other specifications (such as ical). By nature these types will be part of the evolving interface, and will need to be adjusted for every new property that an application must understand.
25
26## Store Facade 16## Store Facade
27The store is always accessed through a store specific facade, which hides: 17The store is always accessed through a store specific facade, which hides:
28 18
@@ -52,118 +42,12 @@ Each modification is associated with a specific revision, which allows the synch
52### Conflict Resolution 42### Conflict Resolution
53Conflicts can occur at two points: 43Conflicts can occur at two points:
54 44
55* While i.e. an editor is open and we receive an update for the same entity 45* In the client: While i.e. an editor is open and we receive an update for the same entity
56* After a modification is sent to the synchronizer but before it's processed 46* In the synchronizer: After a modification is sent to the synchronizer but before it's processed
57 47
58In the first case the client is repsonsible to resolve the conflict, in the latter case it's the synchronizer's responsibility. 48In the first case the client is repsonsible to resolve the conflict, in the latter case it's the synchronizer's responsibility.
59A small window exists where the client has already started the modification (i.e. command is in socket), and a notification has not yet arrived that the same entity has been changed. In such a case the synchronizer may reject the modification because it has the revision the modification refers to no longer available. 49A small window exists where the client has already started the modification (i.e. command is in socket), and a notification has not yet arrived that the same entity has been changed. In such a case the synchronizer may reject the modification because it has the revision the modification refers to no longer available.
60 50
61This design allows the synchronizer to be in control of the revisions, and keeps it from having to wait for all clients to update until it can drop revisions.
62
63## Query System
64The query system should allow for efficient retrieval for just the amount of data required by the client. Efficient querying is supported by the indexes provided by the resources.
65
66The query always retrieves a set of entities matching the query, while not necessarily all properties of the entity need to be populated.
67
68Queries should are declarative to keep the specification simple and to allow the implementation to choose the most efficient execution.
69
70Queries can be kept open (live) to receive updates as the store changes.
71
72### Query
73The query consists of:
74
75* a set of filters to match the wanted entities
76* the set of properties to retrieve for each entity
77
78Queryable properties are defined by the [[Domain Types]] above.
79
80### Query Result
81The result is returned directly after running the query in form of a QAbstractItemModel. Each row in the model represents a matching entity.
82
83The model allows to access the domain object directly, or to access individual properties directly via the rows columns.
84
85The model is always populated asynchronously. It is therefore initially empty and will then populate itself gradually, through the regular update mechanisms (rowsInserted).
86
87Tree Queries allow the application to query for i.e. a folder hierarchy in a single query. This is necessary for performance reasons to avoid recursive querying in large hierarchies. To avoid on the other hand loading large hierchies directly into memory, the model only populates the toplevel rows automatically, all other rows need to be populated by calling `QAbstractItemModel::fetchMore(QModelIndex);`. This way the resource can deal efficiently with the query (i.e. by avoiding the many roundtrips that would be necessary with recursive queries), while keeping the amount of data in memory to a minimum (i.e. if the majority of the folder tree is collapsed in the view anyways). A tree result set can therefore be seen as a set of sets, where every subset corresponds to the children of one parent.
88
89If the query is live, the model updates itself if the update applies to one of the already loaded subsets (otherwise it's currently irrelevant and will load once the subset is loaded).
90
91#### Enhancements
92* Asynchronous loading of entities/properties can be achieved by returning an invalid QVariant initially, and emitting dataChanged once the value is loaded.
93* To avoid loading a large list when not all data is necessary, a batch size could be defined to guarantee for instance that there is sufficient data to fill the screen, and the fetchMore mechanism can be used to gradually load more data as required when scrolling in the application.
94
95#### Filter
96A filter consists of:
97
98* a property to filter on as defined by the [[Domain Types]]
99* a comparator to use
100* a value
101
102The available comparators are:
103
104* equal
105* greater than
106* less than
107* inclusive range
108
109Value types include:
110
111* Null
112* Bool
113* Regular Expression
114* Substring
115* A type-specific literal value (e.g. string, number, date, ..)
116
117Filters can be combined using AND, OR, NOT.
118
119#### Example
120```
121query = {
122 offset: int
123 limit: int
124 filter = {
125 and {
126 collection = foo
127 or {
128 resource = res1
129 resource = res2
130 }
131 }
132 }
133}
134```
135
136possible API:
137
138```
139query.filter().and().property("collection") = "foo"
140query.filter().and().or().property("resource") = "res1"
141query.filter().and().or().property("resource") = "res2"
142query.filter().and().property("start-date") = InclusiveRange(QDateTime, QDateTime)
143```
144
145The problem is that it is difficult to adjust an individual resource property like that.
146
147### Usecases ###
148Mail:
149
150* All mails in folder X within date-range Y that are unread.
151* All mails (in all folders) that contain the string X in property Y.
152
153Todos:
154
155* Give me all the todos in that collection where their RELATED-TO field maps to no other todo UID field in the collection
156* Give me all the todos in that collection where their RELATED-TO field has a given value
157* Give me all the collections which have a given collection as parent and which have a descendant matching a criteria on its attributes;
158
159Events:
160
161* All events of calendar X within date-range Y.
162
163Generic:
164* entity with identifier X
165* all entities of resource X
166
167### Lazy Loading ### 51### Lazy Loading ###
168The system provides property-level lazy loading. This allows i.e. to defer downloading of attachments until the attachments is accessed, at the expense of having to have access to the source (which could be connected via internet). 52The system provides property-level lazy loading. This allows i.e. to defer downloading of attachments until the attachments is accessed, at the expense of having to have access to the source (which could be connected via internet).
169 53
@@ -173,12 +57,3 @@ Note: We should perhaps define a minimum set of properties that *must* be availa
173 57
174### Data streaming ### 58### Data streaming ###
175Large properties such as attachments should be streamable. An API that allows to retrieve a single property of a defined entity in a streamable fashion is probably enough. 59Large properties such as attachments should be streamable. An API that allows to retrieve a single property of a defined entity in a streamable fashion is probably enough.
176
177### Indexes ###
178Since only properties of the domain types can be queried, default implementations for commonly used indexes can be provided. These indexes are populated by generic preprocessors that use the domain-type interface to extract properties from individual entites.
179
180## Notifications ##
181A notification mechanism is required to inform clients about changes. Running queries will automatically update the result-set if a notification is received.
182
183Note: A notification could supply a hint on what changed, allowing clients to ignore revisions with irrelevant changes.
184A running query can do all of that transparently behind the scenes. Note that the hints should indeed only hint what has changed, and not supply the actual changeset. These hints should be tailored to what we see as useful, and must therefore be easy to modify.