summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAge
* Increment database versionkey-for-indexesMinijackson2018-07-06
|
* Use key API in indexesMinijackson2018-07-04
|
* Minor fixeskey-storage-apiMinijackson2018-07-04
|
* Fix changereplay + comment special case of readLatestMinijackson2018-07-04
|
* Fix pipelinetest + remove useless commented linesMinijackson2018-07-04
|
* Move Key API into own files + some fixesMinijackson2018-07-04
|
* [Storage] Implement Key APIMinijackson2018-07-04
|
* Test overlap live queriesMinijackson2018-07-04
|
* Contains with an empty key doesn't make much sense.Christian Mollekopf2018-06-26
|
* Only modifications have changed propertiesChristian Mollekopf2018-06-26
|
* Avoid overwriting local changes with remote modifications.Christian Mollekopf2018-06-26
| | | | | | | | | | | | | | | | | | | The case we ran into is the following: * Fetching the full payload and marking all messages of a thread as read happens simultaneously. * The local modification to mark as read gets immediately overwritten when the full payload arrives. * Eventually the modification gets replayed to the server though (and the reversal isn't because coming from the source), so on next sync the situation fixes itself. To be able to improve this we try to protect local modifications in that properties that have been modified since baseRevision (which currently isn't, but should be equal to the last to the server replayed revision) are not overwritten. This conflict resolution strategy thus always prefers local modifications. baseRevision is currently set to the current maximum revision of the store at the time when the resource creates the modification.
* Fixed the lost update scenarioChristian Mollekopf2018-06-25
| | | | | | | If we get a fetchMore right between when the revision was updated and the incrementalQuery actually running, we ended up loosing the update because the result provider ended up with a too recent revision after the additional initial query.
* Try harder to avoid storing a revision that is too high in the resultChristian Mollekopf2018-06-22
| | | | | | | | | | | | set. We might miss some updates. This should not normally ever happen if we assume that we have a transaction from start to finish of the query (the maxRevision() call should be equivalent. We do have some cornercases in our lmdb implementation that breaks transactions when new databases are opened, so we try to be extra safe this way.... Let's see if it works.
* Log query resultsChristian Mollekopf2018-06-22
|
* Fixed the thread index.Christian Mollekopf2018-06-19
| | | | | | | | * Modifications could result in index changes because we lost the threadId due to remove + add. A modify was necessary (although we can ignore it for the email case). * The ThreadIndexer would try to lookup and potentially index threads for empty parent ids, which is clearly wrong.
* Asserts, debug messages and other cleanupChristian Mollekopf2018-06-19
|
* Implement Overlap queriesRémi Nicole2018-06-19
| | | | | | | | | | | | | | | | | | | | Summary: Notes: - Introduces the concept of queries on multiple properties (which meant changing query's internals a bit) - Dates are stored as well as the "reference" in the index to allow quick filtering without fetching the whole entity - Buckets are weeks starting on Monday (guaranteed by the use of the Julian calendar) - Some size improvements are definitely possible (dates are padded numbers again, not using integer databases, Julian calendar starts at a very old date, etc.) Test Plan: Tested in querytest Reviewers: cmollekopf Reviewed By: cmollekopf Tags: #sink Differential Revision: https://phabricator.kde.org/D13477
* Debug outputChristian Mollekopf2018-06-18
|
* Reduce the maximum database size to 200MB until we have fixed the lmdbChristian Mollekopf2018-06-01
| | | | | | | | | | | issues. https://phabricator.kde.org/T8723 With 200MB we can both deal with the 200MB files on disk, and we could even load all of them (the 5 databases the resource uses), into memory. Once the open problems are resolved we should be able to bump it back to at least 20GB.
* Avoid trying to remove an item that we don't have in the model.Christian Mollekopf2018-06-01
| | | | This triggered the assert in createIndexFromId.
* CleanupChristian Mollekopf2018-05-29
|
* Log to qDebug which we then funnel to Sink::LogChristian Mollekopf2018-05-29
|
* CleanupChristian Mollekopf2018-05-29
|
* Reduce the maximum database size to 10GB on windows.Christian Mollekopf2018-05-29
|
* Only call the result handler if reading succeededChristian Mollekopf2018-05-29
|
* Avoid building tests that link to the resource on windows.Christian Mollekopf2018-05-28
| | | | | Because we'd have to properly export the symbols for the linking on windows to work.
* -DQT_PLUGIN seems to be a remnant of the past.Christian Mollekopf2018-05-28
| | | | Or at least I don't know what it is supposed to do.
* Cleaned up some unnecessary dependencies.Christian Mollekopf2018-05-28
|
* Fixed build with extra warnings.Christian Mollekopf2018-05-28
|
* Implement ranged queriesRémi Nicole2018-05-28
| | | | | | | | | | | | | | | Summary: Notes: - For now, only for QDateTime indexes - Invalid QDateTimes are stored in the index (subject to change) - Should be a drop-in replacement from ValueIndexes (except for `In` and `Contains` queries) Reviewers: cmollekopf Tags: #sink Differential Revision: https://phabricator.kde.org/D13105
* Document why we might be creating a db several times.Christian Mollekopf2018-05-27
|
* A wrapper for linenoiseChristian Mollekopf2018-05-27
| | | | | Because global variables in header files don't work if included from multiple places.
* Don't store references to temporarily valid data.Christian Mollekopf2018-05-25
|
* We can't recover from not being able to open the database.Christian Mollekopf2018-05-24
| | | | | We'll only end up with defunctional processes that may or may not do anything useful.
* Get the mapsize from a separate function and check the error whenChristian Mollekopf2018-05-24
| | | | setting it.
* Fixed linenoise warningsChristian Mollekopf2018-05-24
|
* Fixed a bunch of warningsChristian Mollekopf2018-05-24
|
* Return number of retrieved values.Christian Mollekopf2018-05-24
|
* Add findAllInRange function in the storage layerRémi Nicole2018-05-24
| | | | | | | | | | | | | | | | | Summary: In preparation of the support for ranged queries. Notes: Since they are pretty similar, it could be nice to refactor `scan` and `findAllInRange` to use common 3rd function Test Plan: This is tested in storagetest.cpp Reviewers: cmollekopf Tags: #sink Differential Revision: https://phabricator.kde.org/D13066
* We were off by a factor of 10Christian Mollekopf2018-05-24
| | | | | It's possible that we therefore went over the virtual address space limit on windows which is 128GB.
* Opening dbis may fail in read-only modeChristian Mollekopf2018-05-24
|
* I think those variables should just be staticChristian Mollekopf2018-05-24
|
* We need an extern declaration for non-static variables.Christian Mollekopf2018-05-23
| | | | Or so says the compiler.
* linenoise: Use inline instead of staticChristian Mollekopf2018-05-23
|
* Replaced readline with cpp-linenoiseChristian Mollekopf2018-05-23
| | | | | | | | | ... a single header readline replacement that works on all linux, osx and windows (or so they claim). Besides cleaning up the code considerably, it should help us build sinksh on windows where readline is not really (there are some ancient broken readline ports) available. cpp-readline comes from here: https://github.com/yhirose/cpp-linenoise
* Skip revisions that we can't replay.Christian Mollekopf2018-05-23
|
* CleanupChristian Mollekopf2018-05-23
|
* DbLayout support for indexesChristian Mollekopf2018-05-23
|
* Fixed use of mdb_dbi_openChristian Mollekopf2018-05-23
| | | | | | | | | | | | | | | | | | | | There can only ever be one transaction using mdb_dbi_open running, and that transaction must commit or abort before any other transaction attempts to use mdb_dbi_open. Use delayed dbi merging with write transactions and a temporary transaction for read transactions. We now protect dbi initialization with a mutex and immediately update the sDbis hash. This assumes that the created dbis are indeed We can still violate the only one transaction may use mdb_dbi_open rule if we start a read-only transaction after the write transaction, before the write transaction commits. It does not seem to be something we actually do though. Opening dbis on environment init is further separated out, so we don't end up in the regular openDatabase codepath at all.
* Assert that we have a QGuiApplication if we crash otherwise.Christian Mollekopf2018-05-20
|