Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] [Source][MySQL-CDC] compatible debezium json format npe issue #6776

Closed
3 tasks done
liunaijie opened this issue Apr 29, 2024 · 1 comment · Fixed by #6809
Closed
3 tasks done

[Bug] [Source][MySQL-CDC] compatible debezium json format npe issue #6776

liunaijie opened this issue Apr 29, 2024 · 1 comment · Fixed by #6809
Labels

Comments

@liunaijie
Copy link
Contributor

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

mysql cdc when use default format, the update will generate 2 record (1 delete, 1 update)
when use compatible_debezium_json format, the update will only generate 1 record
But when set key_convert_schemas.enable to false, will get NPE

SeaTunnel Version

dev

SeaTunnel Config

env {
  parallelism = 1
  job.mode = "STREAMING"
  checkpoint.interval = 10000
}

source {
  MySQL-CDC {
    base-url = "jdbc:mysql://localhost:3306/st_cdc"
    username = "root"
    password = "123456"
    table-names = ["st_cdc.tb01"]
    startup.mode = "initial"
    format = compatible_debezium_json
    debezium = {
        key.converter.schemas.enable = false
        value.converter.schemas.enable = false
    }
  }
}

sink {
  Console {
  }
}


### Running Command

```shell
example

Error Exception

Null point exception

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

when set to true, the body will be empty.
截屏2024-04-29 18 35 50

when set to false, get npe
截屏2024-04-29 18 39 22

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@liunaijie liunaijie added the bug label Apr 29, 2024
@liunaijie
Copy link
Contributor Author

another thing i noticed, when i use mysql cdc to mysql cdc with default format, the update will insert new record, but can't delete the old record.

when i change to compatible_debezium_json format, the data will be debezium format, can't sink to mysql directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant