Skip to content

2.3.0-beta-Release
Compare
Choose a tag to compare
@CalvinKirs CalvinKirs released this 09 Nov 03:09
· 1635 commits to dev since this release
2.3.0-beta
f57b897

2.3.0 Beta

[Connector V2]

[New Connector V2 Added]

  • [Source] [Kafka] Add Kafka Source Connector (2953)
  • [Source] [Pulsar] Add Pulsar Source Connector (1980)
  • [Source] [S3File] Add S3 File Source Connector (3119)
  • [Source] [JDBC] [Phoenix] Add Phoenix JDBC Source Connector (2499)
  • [Source] [JDBC] [SQL Server] Add SQL Server JDBC Source Connector (2646)
  • [Source] [JDBC] [Oracle] Add Oracle JDBC Source Connector (2550)
  • [Source] [JDBC] [GBase8a] Add GBase8a JDBC Source Connector (3026)
  • [Source] [JDBC] [StarRocks] Add StarRocks JDBC Source Connector (3060)
  • [Sink] [Kafka] Add Kafka Source Connector (2953)
  • [Sink] [S3File] Add S3 File Sink Connector (3119)

[Improve & Bug Fix]

  • [Source] [Fake]

    • [Improve] Supports direct definition of data values(row) (2839)
    • [Improve] Improve fake source connector: (2944)
      • Support user-defined map size
      • Support user-defined array size
      • Support user-defined string length
      • Support user-defined bytes length
    • [Improve] Support multiple splits for fake source connector (2974)
    • [Improve] Supports setting the number of splits per parallelism and the reading interval between two splits (3098)

  • [Source] [Clickhouse]

    • [Improve] Clickhouse Source random use host when config multi-host (3108)

  • [Source] [FtpFile]

    • [BugFix] Fix the bug of incorrect path in windows environment (2980)
    • [Improve] Support extract partition from SeaTunnelRow fields (3085)
    • [Improve] Support parse field from file path (2985)

  • [Source] [HDFSFile]

    • [BugFix] Fix the bug of incorrect path in windows environment (2980)
    • [Improve] Support extract partition from SeaTunnelRow fields (3085)
    • [Improve] Support parse field from file path (2985)

  • [Source] [LocalFile]

    • [BugFix] Fix the bug of incorrect path in windows environment (2980)
    • [Improve] Support extract partition from SeaTunnelRow fields (3085)
    • [Improve] Support parse field from file path (2985)

  • [Source] [OSSFile]

    • [BugFix] Fix the bug of incorrect path in windows environment (2980)
    • [Improve] Support extract partition from SeaTunnelRow fields (3085)
    • [Improve] Support parse field from file path (2985)

  • [Source] [IoTDB]

    • [Improve] Improve IoTDB Source Connector (2917)
      • Support extract timestamp、device、measurement from SeaTunnelRow
      • Support TINYINT、SMALLINT
      • Support flush cache to database before prepareCommit

  • [Source] [JDBC]

    • [Feature] Support Phoenix JDBC Source (2499)
    • [Feature] Support SQL Server JDBC Source (2646)
    • [Feature] Support Oracle JDBC Source (2550)
    • [Feature] Support StarRocks JDBC Source (3060)
    • [Feature] Support GBase8a JDBC Source (3026)

  • [Sink] [Assert]

    • [Improve] 1.Support check the number of rows (2844) (3031):
      • check rows not empty
      • check minimum number of rows
      • check maximum number of rows
    • [Improve] 2.Support direct define of data values(row) (2844) (3031)
    • [Improve] 3.Support setting parallelism as 1 (2844) (3031)

  • [Sink] [Clickhouse]

    • [Improve] Clickhouse Support Int128,Int256 Type (3067)

  • [Sink] [Console]

    • [Improve] Console sink support print subtask index (3000)

  • [Sink] [Enterprise-WeChat]

    • [BugFix] Fix Enterprise-WeChat Sink data serialization (2856)

  • [Sink] [FtpFile]

    • [BugFix] Fix the bug of incorrect path in windows environment (2980)
    • [BugFix] Fix filesystem get error (3117)
    • [BugFix] Solved the bug of can not parse '\t' as delimiter from config file (3083)

  • [Sink] [HDFSFile]

    • [BugFix] Fix the bug of incorrect path in windows environment (2980)
    • [BugFix] Fix filesystem get error (3117)
    • [BugFix] Solved the bug of can not parse '\t' as delimiter from config file (3083)

  • [Sink] [LocalFile]

    • [BugFix] Fix the bug of incorrect path in windows environment (2980)
    • [BugFix] Fix filesystem get error (3117)
    • [BugFix] Solved the bug of can not parse '\t' as delimiter from config file (3083)

  • [Sink] [OSSFile]

    • [BugFix] Fix the bug of incorrect path in windows environment (2980)
    • [BugFix] Fix filesystem get error (3117)
    • [BugFix] Solved the bug of can not parse '\t' as delimiter from config file (3083)

  • [Sink] [IoTDB]

    • [Improve] Improve IoTDB Sink Connector (2917)
      • Support align by sql syntax
      • Support sql split ignore case
      • Support restore split offset to at-least-once
      • Support read timestamp from RowRecord
    • [BugFix] Fix IoTDB connector sink NPE (3080)

  • [Sink] [JDBC]

    • [BugFix] Fix JDBC split exception (2904)
    • [Feature] Support Phoenix JDBC Source (2499)
    • [Feature] Support SQL Server JDBC Source (2646)
    • [Feature] Support Oracle JDBC Source (2550)
    • [Feature] Support StarRocks JDBC Source (3060)

  • [Sink] [Kudu]

    • [Improve] Kudu Sink Connector Support to upsert row (2881)

  • [Sink] [Hive]

    • [Improve] Hive Sink supports automatic partition repair (3133)

[Connector V1]

[New Connector V1 Added]

[Improve & Bug Fix]

  • [Sink] [Spark-Hbase]
    • [BugFix] Handling null values (3099)

[Starter & Core & API]

[Feature & Improve]

  • [Improve] [Sink] Support define parallelism for sink connector (2941)
  • [Improve] [all] change Log to @slf4j (3001)
  • [Improve] [format] [text] Support read & write SeaTunnelRow type (2969)
  • [Improve] [api] [flink] extraction unified method (2862)
  • [Feature] [deploy] Add Helm charts (2903)
  • [Feature] [seatunnel-text-format] (2884)

[Bug Fix]

  • [BugFix] Fix assert connector name error in config/plugin_config file (3127)
  • [BugFix] [starter] Fix connector-v2 flink & spark dockerfile (3007)
  • [BugFix] [core] Fix spark engine parallelism parameter does not working (2965)
  • [BugFix] [build] Fix the invalidation of the suppression file of checkstyle in the win10 (2986)
  • [BugFix] [format] [json] Fix jackson package conflict with spark (2934)
  • [BugFix] [build] Fix the invalidation of the suppression file of checkstyle in the win10 (2986)
  • [BugFix] [build] Fix the invalidation of the suppression file of checkstyle in the win10 (2986)
  • [BugFix] [seatunnel-translation-base] Fix Source restore state NPE (2878)

[Docs]

  • Add coding guide (2995)

[SeaTunnel Engine]

[Feature & Improve]

[Cluster Manager]

  • Support Run SeaTunnel Engine in stand-alone.
  • Support Run SeaTunnel Engine cluster.
  • Do not rely on third-party services(zookeeper etc) to realize the master-worker architecture.
  • Autonomous cluster (non centralized).
  • Automatic discovery of cluster members.

[Core]

  • Support submit Job to SeaTunnel Engine in local mode.
  • Support submit Job to SeaTunnel Engine in cluster mode.
  • Support Batch Job.
  • Support Stream Job.
  • Supports batch stream integration, and the batch stream integration feature of all SeaTunnel V2 Connectors can be guaranteed in SeaTunnel Engine.
  • Support Distributed Snapshot algorithm Chandy Ramport algorithm and Two-phase Commit. Exactly-Once semantics based on these implementations.
  • Support pipeline granularity job scheduling, Ensure that the job can be started under limited resources.
  • Support pipeline granularity job restore.
  • Sharing threads between tasks to achieve real-time synchronization of a large number of small datasets.

Contributors

slf4j
Assets 2