You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've had a sense that electrified tables were behaving differently than expected.
This lead to writing a test that shows non-atomic transactions with intermediate reads are common.
The schema:
-- lww append only list registerCREATETABLEIF NOT EXISTS lww (
k INTEGERPRIMARY KEY,
v TEXT
);
-- electrify tableALTERTABLE lww ENABLE ELECTRIC;
Read and append transactions are generated:
[[:r9 [910111218]] [:append53]]
The better-sqlite3 driver is used:
constupsert=txn_conn.prepare('INSERT INTO lww (k,v) VALUES (@k,@v) ON CONFLICT (k) DO UPDATE SET v = v || \' \' || @v');constselect=txn_conn.prepare('SELECT k,v FROM lww WHERE k = @k');
electricsql/electric:latest Docker images and "electric-sql": "^0.10.1" client are used in a cluster of:
Running the same test on a non-electrified db shows no anomalies.
The test finds a variety of cycles and anomalies.
There are cases where the process will not read its writes or writes are lost.
Those will be covered in a future test more suited to illustrate that class of anomalies.
After doing some research on SQLite3 and isolation,
I did try using two separate distinct db connections, one for ElectricSQL and one for transactions.
This did not change the results.
Dumped the pragma values of the connections and didn't see anything of interest? connections-pragma.json
same test but with a non-electrified local SQLite3 db
will pass
The test creates two artifacts, a result summary and the full test results.
In the results, look for :G1b anomalies with :reads-of-final values.
These are specifically intermediate reads.
The full results:
have complete transaction, node, sync service, postgres, etc. logs.
The causal directory has further cycle and graph analysis.
Intermediate reads will be graphed as a G-single-item.
The graph for the example given above's intermediate read cycle:
I'm sure there will be questions so please ask.
I also hope to make the Jepsen Docker environment as usable as possible so feedback welcome.
Thanks.
The text was updated successfully, but these errors were encountered:
Hi,
I've had a sense that electrified tables were behaving differently than expected.
This lead to writing a test that shows non-atomic transactions with intermediate reads are common.
The schema:
Read and append transactions are generated:
The
better-sqlite3
driver is used:electricsql/electric:latest
Docker images and"electric-sql": "^0.10.1"
client are used in a cluster of:Here's an example of an intermediate read:
Running the same test on a non-electrified db shows no anomalies.
The test finds a variety of cycles and anomalies.
There are cases where the process will not read its writes or writes are lost.
Those will be covered in a future test more suited to illustrate that class of anomalies.
After doing some research on SQLite3 and isolation,
I did try using two separate distinct db connections, one for ElectricSQL and one for transactions.
This did not change the results.
Dumped the pragma values of the connections and didn't see anything of interest? connections-pragma.json
The test framework is Jepsen.
A docker environment and GitHub action have been created that reliably reproduce the behavior.
The GitHub action is a good place to start, https://github.com/nurturenature/jepsen-causal-consistency/actions
The test creates two artifacts, a result summary and the full test results.
In the results, look for
:G1b
anomalies with:reads-of-final
values.These are specifically intermediate reads.
The full results:
have complete transaction, node, sync service, postgres, etc. logs.
The causal directory has further cycle and graph analysis.
Intermediate reads will be graphed as a
G-single-item
.The graph for the example given above's intermediate read cycle:
I'm sure there will be questions so please ask.
I also hope to make the Jepsen Docker environment as usable as possible so feedback welcome.
Thanks.
The text was updated successfully, but these errors were encountered: