Documentation

1 files changed, 21 insertions, 1 deletions

diff --git a/docs/resource.md b/docs/resource.md
index aa263e8..c8f58e9 100644
--- a/docs/resource.md
+++ b/docs/resource.md
@@ -9,6 +9,13 @@ The synchronizer process is responsible for processing all commands, executing s
 
 Processing of commands happens in the pipeline which executes all preprocessors ebfore the entity is persisted.
 
+The synchronizer process has the following primary components:
+
+* Command Queues: Queues that hold all incoming commands. Persisted over reboots.
+* Command Processor: A processor that empties the command queues by pushing commands through the pipeline.
+* Listener: Opens a socket and listens for incoming connections. On connection all incoming commands are read and entered into command queues. Control commands (i.e. a sync) don't require persistency and are therefore processed directly.
+* Synchronization: Handles synchronization to the source, as well as change-replay to the source. The modification commands generated by the synchronization enter the command queue as well.
+
 # Preprocessors
 Preprocessors are small processors that are guaranteed to be processed before an new/modified/deleted entity reaches storage. They can therefore be used for various tasks that need to be executed on every entity.
 
@@ -119,5 +126,18 @@ A pipeline is an assembly of a set of preprocessors with a defined order. A modi
     * Generate a full diff directly on top of the db. The diffing process can work against a single revision/snapshot (using transactions). It then generates a necessary changeset for the store.
     * If the source supports incremental changes the changeset can directly be generated from that information.
 
-The changeset is then simply inserted in the regular modification queue and processed like all other modifications. The synchronizer has to ensure only changes are replayed to the source that didn't come from it already.
+The changeset is then simply inserted in the regular modification queue and processed like all other modifications. The synchronizer has to ensure only changes are replayed to the source that didn't come from it already. This is done by marking changes that don't require changereplay to the source.
+
+## Synchronization Store
+To track progress of the synchronization, the synchronizer needs to maintain a separate store. It needs to be separate from the main store to properly separate the the synchronization from the Command Processor, which enables the two parts to run concurrently (We can't have two threads writing to the same store).
+
+While the synchronization store can contain any useful information for a resource to synchronize a typical example looks like this:
+
+* changereplay: Contains the last replayed revision. Used by the change replay to know what has been replayed to the source already.
+* remoteid.mapping.$BUFFERTYPE: Contains the mapping of a remote identifier to a local identifier. Necessary to track what has already been synchronized, and to replay changes to the remote entity.
+* localid.mapping.$BUFFERTYPE: Reverse mapping of the remoteid.mapping.
+
+The remoteid mapping has to be updated in two places:
 
+* New entities that are synchronized immediately get a localid assinged, that is then recorded together with the remoteid. This is required to be able to reference other entities directly in the command queue (i.e. for parent folders).
+* Entities created by clients get a remoteid assigned during change replay, so the entity can be recognized during the next sync.