Skip to content

2.3.3 Release
Compare
Choose a tag to compare
@ic4y ic4y released this 22 Aug 08:21
· 607 commits to dev since this release
2.3.3
6977e9f

2.3.3 Release Note

Bug fix

Core

  • [Translation] [Spark] Fix SeaTunnelRowConvertor fail to convert when schema contains row type (#5170)

Connector

  • [Connector] [Jdbc] Fix the problem of JdbcOutputFormat database connection leak (#4802)
  • [Connector-V2] [AmazonDynamoDB] Fix the problem that all table data cannot be obtained (#5146)
  • [Connector-V2] [CDC-base] Fix cdc connection does not close (#4922)
  • [Connector-V2] [Clickhouse] Fix clickhouse source connector read Nullable() type is not null,example:Nullable(Float64) while value is null the result is 0.0 (#5080)
  • [Connector-V2] [ClickhouseFile] Fix ClickhouseFile write file failed when field value is null (#4937)
  • [Connector-V2] [Doris] Add streamload Content-type for doris URLdecode error (#4880)
  • [Connector-V2] [E2E] Fix maven scope (#4901)
  • [Connector-V2] [File-Base] Fix Hadoop Kerberos authentication related issues (#5171)
  • [Connector-V2] [Flink-runtime] Flink register table Environment The running mode is set tojob.mode (#4826)
  • [Connector-V2] [HbaseSink] Fix default timestamp (#4958)
  • [Connector-V2] [Hive] Support user-defined hive-site.xml (#4965)
  • [Connector-V2] [Jdbc] Support postgresql json type (#5194)
  • [Connector-V2] [Jdbc] Fix XA Transaction bug (#5020)
  • [Connector-V2] [Jdbc] Modify error comment in JdbcSourceConfigFactory (#5088)
  • [Connector-V2] [Jdbc-Oracle] Fix the Oracle number type mapping problem (#5209)
  • [Connector-V2] [Jdbc-Postgres] PostgreSQL versions below 9.5 are compatible use cdc sync problem (#5120)
  • [Connector-V2] [JindoOssFile] Fix plugin-mapping.properties (#5215)
  • [Connector-V2] [LocalFileSink] Fix LocalFile Sink file_format_type (#5118)
  • [Connector-V2] [Mongodb] Fix document error content and remove redundant code (#4982)
  • [Connector-V2] [Mongodb] Compatible with historical parameters (#4997)
  • [Connector-V2] [MongoDB-CDC] Incremental data kind error in snapshot phase (#5184)
  • [Connector-V2] [MongoDB-CDC] Solve startup resume token is negative (#5143)
  • [Connector-V2] [MongoDB-CDC] Refine data format to adapt to universal logic (#5162)
  • [Connector-V2] [MongoDB-CDC] Fix array index anomalies caused by #5057 (#5195)
  • [Connector-V2] [MySql-CDC] Fix jdbc connection leak for mysql (#5037)
  • [Connector-V2] [MySql-CDC] Fix mysql bit column to java byte (#4817)
  • [Connector-V2] [MySql-CDC] Fix listener not released when BinlogClient reuse (#5011)
  • [Connector-V2] [MySql-CDC] ServerId from int to long (#5033) (#5035)
  • [Connector-V2] [Paimon] Bump paimon-bundle version to 0.4.0-incubating (#5219)
  • [Connector-V2] [Redis] Fix redis nodes format error (#4981)
  • [Connector-V2] [StarRocks] Fix StarRocksJsonSerializer will transform array/map/row to string (#5281)
  • [Connector-V2] [StarRocks] Fix code style (#4966)

Zeta(ST-Engine)

  • [Zeta] Resolved the issue causing checkpoints to halt on tolerable-failure=0 (#5263)
  • [Zeta] Fix the issue of two identical IDs appearing when executing seatunnel.sh -l as the job resumes (#5191)
  • [Zeta] Fix SinkFlowLifeCycle without init lastCommitInfo (#5152)
  • [Zeta] Fix MultipleTableJobConfigParser ignore env option (#5067)
  • [Zeta] Fix checkpoint error report without msg (#5137)
  • [Zeta] [REST-API] Fix class not found exception (#5063)
  • [Zeta] [REST-API] Fix browser long type intercept (#5267)
  • [Zeta] Fix task state memory leak (#5139)
  • [Zeta] Fix when execute seatunnel.sh, console may print error (#4984)
  • [Zeta] Fixed the issue of duplicated metrics caused by job fault tolerance or restore (#5214)
  • [Zeta] Avoid Redundant Job Submissions by Checking Job Status (#5229)
  • [Zeta] Fix the serialization issue of GetMetricsOperation during multi-node operation (#5206)
  • [Zeta] Fix after the savepoint job is restored, the checkpoint file cannot be generated #4985 (#5051)
  • [Zeta] Fixed multi-table job data loss and latency issues (#149) (#5031)
  • [Zeta] Fix the HdfsStorage can not delete checkpoint file #5046 (#5054)
  • [Zeta] Fix if the initialization task throws an error and the task can not return to master (#4888)

TEST

  • [Test] Split updated modules integration test for part 5 (#5208)
  • [Test] Add code style check when docs changed (#5183)
  • [Test] Turn on the testSavepoint test (#5199)
  • [Test] Fix the CI Job name error (#5032)
  • [Test] [E2E] Fix testGetErrorInfo case error (#5282)
  • [Test] [Kafka] Kafka e2e error (#5200)
  • [Test] [MongoDB-CDC] Fix mongodb cdc e2e instability (#5128)
  • [Test] Declare files that will always have UNIX line endings on checkout (#5221)
  • [Test] Fixed a minor bug (#5274)
  • [Test] Fixed IMap file storage e2e bug (#5237)
  • [Test] Split updated modules integration test for part 4 (#5028)
  • [Test] Fix zeta ci error (#5254)

Others

  • Update the project description (#4967)
  • Update version to 2.3.3-SNAPSHOT (#5043)
  • Fix com.google.common.base.Preconditions to seatunnel shade one (#5284)
  • Fix style error (#5280)
  • Fix code style (#5092)

Improve

Core

  • [API & Zeta] Using connector custom serializer encode/decode states (#5238)
  • [API]Improve savemode api (#4767)

Connector

  • [CDC Base] Add some split parameters to the optionRule (#5161)
  • [CDC Base] Support exactly-once of cdc and fix the BinlogOffset comparing bug (#5057)
  • [CDC Base] Add a fast sampling method that supports character types (#5179)
  • [CDC Base] Change driver scope to provider (#5002)
  • [CDC Base] Remove driver for cdc connector (#4952)
  • [Connector-V2] [File] Optimize files commit order (#5045)
  • [Connector-V2] [HiveSink] Remove drop partition when abort (#4940)
  • [Connector-V2] [Jdbc] Remove scheduler in JDBC sink #4736 (#5168)
  • [Connector-V2] [Jdbc] Check url not null throw friendly message (#5097)
  • [Connector-V2] [Jdbc] Support for Decimal types as splict keys (#4634)
  • [Connector-V2] [Mongodb] Sink support transaction update/writing (#5034)
  • [Connector-V2] [Mongodb] Optimize reading logic (#5001)
  • [Connector-V2] [Neo4j] Supports neo4j sink batch write and update docs (#4841)
  • [Connector-V2] [OSS-Jindo] Optimize jindo oss connector (#4964)
  • [Connector-V2] [Tablestore sink] Remove scheduler in Tablestore sink (#5272)

Transform

  • [Transform-V2] [SQL] Support use catalogTableName as SQL expression (#5273)

CI

  • [CI/CD] Remove 'paths-ignore', enable the code style check for markdown files (#5286)

Zeta(ST-Engine)

  • [Zeta] Upgrade the key log output level (#4993)
  • [Zeta] Don't trigger handleSaveMode when restore (#5192)
  • [Zeta] Move checkpoint notify complete in checkpoint stage (#5185)
  • [Zeta] Add sleep for Task to reduce CPU cost (#5117)
  • [Zeta] Add an interface for batch retrieval of JobMetrics (#4576)
  • [Zeta] Move imap storage file dependency packages to submodules (#5218)
  • [Zeta] Improve CheckpointCoordinator log error when report error from task (#178) (#5134)
  • [Zeta] Improve CheckpointCoordinator notify complete when restore (#5136)

Feature

Connector

  • [Connector-V2] [CDC-base] Support tables without primary keys (with unique keys) (#163) (#5150)
  • [Connector-V2] [CDC-base] Support disable/enable exactly once for INITIAL (#4921)
  • [Connector-V2] [CDC-base] Change the time zone to the default time zone (#5030)
  • [Connector-V2] [CDC-base] Support string type shard fields (#5147)
  • [Connector-V2] [Clickhouse] Add clickhouse connector time zone key,default system time zone (#5078)
  • [Connector-V2] [Clickhouse] Clickhouse writes with checkpoints (#4999)
  • [Connector-V2] [Doris] Add doris factory type convertor (#5061)
  • [Connector-V2] [File] Add config of 'file_filter_pattern', which used for filtering files (#5153)
  • [Connector-V2] [File] Add cos source&sink (#4979)
  • [Connector-V2] [Hive] Hive add option because web need (#5154)
  • [Connector-V2] [Jdbc] Add JDBC Catalog auto create table (#4917)
  • [Connector-V2] [Jdbc] Add oceanbase dialect factory (#4989)
  • [Connector-V2] [Jdbc] JDBC source support string type as partition key (#4947)
  • [Connector-V2] [Jdbc-DM] Add DM upsert support (#5073)
  • [Connector-V2] [Kafka] Support read debezium format message from kafka (#5066)
  • [Connector-V2] [Kafka] Connector-kafka source support data conversion extracted by kafka connect source (#4516)
  • [Connector-V2] [Mongodb] Mongodb support cdc sink (#4833)
  • [Connector-V2] [Mongodb-CDC] Support source mongodb cdc (#4923)
  • [Connector-V2] [Mysql-CDC] Conversion of tinyint(1) to bool is supported (#5105)
  • [Connector-V2] [Redis] Support redis to set expiration time (#4975)

Zeta(ST-Engine)

  • [Zeta] [CDC-base] Support schema evolution framework(DDL) (#5125)
  • [Zeta] [REST-API] Add REST API To Submit Job (#5107)
  • [Zeta] Checkpoint support hdfs ha mode (#4942)
  • [Zeta] The expiration time of a historical Job can be config (#5180)
  • [Zeta] Support history service record job execute error (#5114)
  • [Zeta] Configuration files support user variable replacement (#4969)

Transform

  • [Transform-V2] [SQL] Support 'select *' and 'like' clause for SQL Transform plugin (#4991)

CI

  • [CI] Update action config to support run CI on fork repo (#5065)
  • [E2E] [FtpFile] Add ftp file e2e test case (#4647)

Docs

  • [Connector-V2][Clickhouse] Reconstruct the clickhouse connector doc (#5085)
  • [Connector-V2][Doris] Reconstruct the Doris connector document (#4903)
  • [Connector-V2][HDFS] Refactor connector-v2 docs using unified format HDFS (#4871)
  • [Connector-V2][Http] Reconstruct the Http connector document (#4962)
  • [Connector-V2][Hudi] Reconstruct the Hudi connector document (#4905)
  • [Connector-V2][Jdbc] Change the line boundary store value type to BigDecimal (#4900)
  • [Connector-V2][Jdbc-DB2、Jdbc-Vertica] Improve DB2 Source Vertica Source & DB2 Sink Vertica Sink document (#5102)
  • [Connector-V2][Jdbc-Oracle] Add JDBC Oracle Documentation (#5239)
  • [Connector-V2][Kafka] Reconstruct the kafka connector document (#4778)
  • [Connector-V2][MyHours] Reconstruct the MyHours connector document (#5129)
  • [Connector-V2][PostgreSQL] Refactor connector-v2 docs using unified format PostgreSQL #4590 (#4757)
  • [Connector-V2][S3File] Improve S3File Source & S3File Sink document (#5101)
  • [Connector-V2][SelectDB-Cloud] Reconstruct the SelectDB-Cloud connector document (#5130)
  • [Connector-V2][StarRocks] Reconstruct the StarRocks connector document (#5132)
  • [Connector-V2][StarRocks] StarRocks nodeUrls property name fix (#4951)
  • [README] Remove useless github workflow, and adjust description of 'engineering structure' (#4305)
  • [Zeta] Add savepoint doc (#5081)
  • Update bug-report.yml (#5160)
  • Add Value types in Java to Schema feature (#5087)
  • Documentation and partial word optimization (#4936)
  • Change repository name from incubator-seatunnel to seatunnel (#4868)
  • Update .asf.yaml (#5242)
  • Update the Readme (#4968)
  • Set Up with Kubernetes, dockerfile document error in constructing docker image (#5022)
  • Fix Dockerfile and seatunnel-flink.yaml in Set Up with Kubernetes (#4793)
  • Change the transform website url (#4954)
  • Update quick-start-seatunnel-engine.md (#4943)
  • Remove incubating keyword in document (#5257)
Assets 2