Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.
-
Updated
Aug 26, 2022 - Python
Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
Classwork projects and home works done through Udacity data engineering nano degree
One framework to develop, deploy and operate data workflows with Python and SQL.
Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow
Project demonstrating how to automate Prefect 2.0 deployments to AWS ECS Fargate
ETL pipeline combined with supervised learning and grid search to classify text messages sent during a disaster event
Code examples showing flow deployment to various types of infrastructure
Solution for the Ultimate Student Hunt Challenge (1st place).
Apache Spark Guide
Learning from multiple companies in Silicon Valley. Netflix, Facebook, Google, Startups
Deploy a Prefect flow to serverless AWS Lambda function
ETL Pipeline / ML Pipeline of Disaster Data provided by figure8
Data Engineering pipeline hosted entirely in the AWS ecosystem utilizing DocumentDB as the database
A end-to-end real-time stock market data pipeline with Python, AWS EC2, Apache Kafka, and Cassandra Data is processed on AWS EC2 with Apache Kafka and stored in a local Cassandra database.
Marshmallow serializer integration with pyspark
Let your pipe lines flow thru the Python code in xonsh.
Spotify API를 이용한 K-POP 인기 탐색 분석 대시보드
A data engineering pipeline for digital marketers.
Add a description, image, and links to the data-engineering-pipeline topic page so that developers can more easily learn about it.
To associate your repository with the data-engineering-pipeline topic, visit your repo's landing page and select "manage topics."