| Commit message (Collapse) | Author | Age |
|---|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The case we ran into is the following:
* Fetching the full payload and marking all messages of a thread as read
happens simultaneously.
* The local modification to mark as read gets immediately overwritten
when the full payload arrives.
* Eventually the modification gets replayed to the server though (and
the reversal isn't because coming from the source), so on next sync the
situation fixes itself.
To be able to improve this we try to protect local modifications in that
properties that have been modified since baseRevision (which currently
isn't, but should be equal to the last to the server replayed revision)
are not overwritten. This conflict resolution strategy thus always
prefers local modifications. baseRevision is currently set to the
current maximum revision of the store at the time when the resource
creates the modification.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
Notes:
- Introduces the concept of queries on multiple properties (which meant changing query's internals a bit)
- Dates are stored as well as the "reference" in the index to allow quick filtering without fetching the whole entity
- Buckets are weeks starting on Monday (guaranteed by the use of the Julian calendar)
- Some size improvements are definitely possible (dates are padded numbers again, not using integer databases, Julian calendar starts at a very old date, etc.)
Test Plan: Tested in querytest
Reviewers: cmollekopf
Reviewed By: cmollekopf
Tags: #sink
Differential Revision: https://phabricator.kde.org/D13477
|
| | |
|
| |
|
|
|
| |
Otherwise we easily end up copying it and then have an entity that
points into nowhere. Callback -> no copy, no callback -> copy.
|
| |
|
|
| |
version when creating it
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A single QueryRunner should never have multiple workers running at the
same time. We did not properly enforce this in case of incremental
updates coming in.
The only way I managed to reproduce the crash:
* Open a large folder with lots of unread mail in kube
* Select a mail in the maillist and hold the down button
* This will:
* Repeatedly call fetch more
* Trigger lot's of mark as read modifications that result in
notifications.
* Eventually it crashes somewhere in EntityStore, likely because
of concurrent access of the filter structure which is shared through
the state.
We now ensure in the single threaded portion of the code that we only
ever run one worker at a time. If we did receive an update during,
we remember that change and fetch more once we're done.
To be able to call fetch again that portion was also factored out into a
separate function.
|
| | |
|
| |
|
|
|
| |
This is where this really belongs, only the indexing is part of storage.
This is necessary so preprocessors can move entities as well.
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
To have hierarchical debug output we have to pass around something at
run-time, there is no reasonable alternative. Log::Context provides the
identifier to do just that and largely replaces the debug component
idea.
|
| |
|
|
|
|
| |
This is really part of the storage, and will help us to cleanly
implement features like moving properties into a temporary place when
reading in a clean way as well.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
We have to access properties, so we need the mapper anyways, and the
ApplicationDomainType type shouldn't be a large overhead anyways.
|
|
|
This is the initial refactoring to improve how we deal with the storage.
It does a couple of things:
* Rename Sink::Storage to Sink::Storage::DataStore to free up the
Sink::Storage namespace
* Introduce a Sink::ResourceContext to have a single object that can be
passed around containing everything that is necessary to operate on a
resource. This is a lot better than the multiple separate parameters
that we used to pass around all over the place, while still allowing
for dependency injection for tests.
* Tie storage access together using the new EntityStore that directly
works with ApplicationDomainTypes. This gives us a central place where
main storage, indexes and buffer adaptors are tied together, which
will also give us a place to implement external indexes, such as a
fulltextindex using xapian.
* Use ApplicationDomainTypes as the default way to pass around entities.
Instead of using various ways to pass around entities (buffers,
buffer adaptors, ApplicationDomainTypes), only use a single way.
The old approach was confusing, and was only done as:
* optimization; really shouldn't be necessary and otherwise I'm sure
we can find better ways to optimize ApplicationDomainType itself.
* a way to account for entities that have multiple buffers, a concept
that I no longer deem relevant.
While this commit does the bulk of the work to get there, the following
commits will refactor more stuff to get things back to normal.
|
