The open source high performance ELT framework powered by Apache Arrow

This process illustrates how to structure and manipulate relational databases effectively, demonstrating key SQL operations and transformations within an Informatica environment. The provided images and detailed SQL commands serve as a comprehensive guide for implementing and understanding these database management tasks.

A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

Framework to write ETL Pipelines controlled by a central config store.

Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage and metadata. Runs and scales everywhere python does.

Logstash - transport and process your logs, events, or other data

Global Biotic Interactions provides access to existing species interaction datasets

The Frank!Framework is an easy-to-use, stateless integration framework which allows (transactional) messages to be modified and exchanged between different systems.

(Spatial) data harmonisation with hale studio (formerly HUMBOLDT Alignment Editor)

Documentation for the TriplyDB and TriplyETL products

DIRECT, the Data Integration Run-time Execution Control Tool, is a data logistics control framework that can be used to monitor, log, audit and control data integration / ETL processes.

Shift is a high performance better alternative to Airbyte, Singer, Meltano

A tool for building feature stores.

Airflow DAGs for the Stellar ETL project

Stellar ETL will enable real-time analytics on the Stellar network

Flow PHP - data processing framework

This repository contains Data Engineering solution using ETL (Extract, Transform, Load) implementation for the sales data analysis of Apple products. The solution is designed to handle diverse data formats and is implemented on Databricks using PySpark, Python, and Databricks utilities.Factory Method Design Pattern has been implemented for reading.

A lightweight ETL (extract, transform, load) library and data integration toolbox for .NET.

Data transformation framework for ETL processing with SQL-like syntax and GIS extensions, based on Apache Spark