ETL workflow and data analysis. ETL-workflow using prefect and pygrametl (SCD, slow changing dimension). Product classification based on product name.
-
Updated
Nov 15, 2019 - Python
ETL workflow and data analysis. ETL-workflow using prefect and pygrametl (SCD, slow changing dimension). Product classification based on product name.
Introduction to the data pipeline management with Airflow. Airflow schedule and maintain numerous ETL processes running on a large scale Enterprise Data Warehouse
Repository for playing with spark
A simple data processing framework for a quick, no-frills setup of a local data pipeline.
utility to enable flexible ETL scenarios, supports golang plug-in for built-in consumer|transformer|producer options
Data Tweak is a simplified, lightweight ETL framework based on Apache Spark.
Python package that enables customized loading of data from a CSV file into a MySQL database
The repository contains Structured Query Language (SQL) Scripts. The Multiple SQL scripts for various projects which includes data cleaning, data pre-processing, data processing, data transformation and insights gaining through Query Language.
Collection of pkgs to build pipelines in JS/TS
Apache Spark based 'Dist' utility to supplement Data Cooker ETL tool
ETL project for EIA alternative fuels
Effectively demonstrate the ETL process using data on Covid-19, Python, and a non-relational database (MongoDB). Created a web scraper to automate the pulling of data.
A repository containing a Power BI project leveraging the "global_superstore_2016" dataset, showcasing visualizations and insights derived from the data for global sales analysis and forecasting.
Simple and extensible PySpark ETL framework
Project uses Pandas to create multiple DataFrames from CSV files containing social media data on presidential candidates, cleaned those DataFrames, then used SQL to create a relational database to join everything together.
An extension that registers all pharmacies in Argentina.
This project is about doing analysis on the World Economics Data. The factors such as Corruption , Tourism,Unemployment and Cost of Living how it affects Economics and also doing comparision in between countries.
Add a description, image, and links to the etl-framework topic page so that developers can more easily learn about it.
To associate your repository with the etl-framework topic, visit your repo's landing page and select "manage topics."