Synchronization is not matching

[leokaynan-bayer]

PGSync version: 3.0.0

Postgres version: PostgreSQL 15 Aurora RDS

Elasticsearch/OpenSearch version: OpenSearch v 2.9.0

Redis version: LATEST

Python version: 3.10.12

Problem Description:
PGSYNC is being run every 5 minutes with the "pgsync" command.

It works very well, however after a while the synchronization does not match. There is usually a difference of 1 or 2 records.

This difference in records happens after a few days.

And when that happens, I have to reindex all the records again.

this happens with both -deamon and --polling.

environment:

ELASTICSEARCH_STREAMING_BULK=True
ELASTICSEARCH=False
OPENSEARCH=True
SCHEMA=schema.json
CHECKPOINT_PATH=checkpoint
ELASTICSEARCH_TIMEOUT=20000

[toluaina]

One thing to mention, the state file is very important
This is checkpoint file starts with a .<db_name>_<index_name>
are you by any chance deleting this file/is this checkpoint path always accessible?

[leokaynan-bayer]

One thing to mention, the state file is very important This is checkpoint file starts with a .<db_name>_<index_name> are you by any chance deleting this file/is this checkpoint path always accessible?

The checkpoint file has not been deleted or changed.

The difference in records is random.

And when it happens, it shows that there is no data to be synchronized.