Python SQL Parser and Transpiler

Python interface to Hive and Presto. 🐝

Uses tokenized query returned by python-sqlparse and generates query metadata

Better SQL in Jupyter. 📊

Datafaker is a large-scale test data and flow test data generation tool. Datafaker fakes data and inserts to varied data sources. 测试数据生成工具

基于 scrapy-redis 的通用分布式爬虫框架

🎉🎉🐳 Datawhale大数据处理导论教程 | 大数据技术方向的开篇课程🎉🎉

大数据分析可视化实践

Simple DDL Parser to parse SQL (HQL, TSQL, AWS Redshift, BigQuery, Snowflake and other dialects) ddl files to json/python dict with full information about columns: types, defaults, primary keys, etc. & table properties, types, domains, etc.

An Eyeriss Chip (researched by MIT, a CNN accelerator) simulator and New DNN framework "Hive"

Example code for running Spark and Hive jobs on EMR Serverless.

python ETL framework

This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We will be using pyspark & sparksql for the development. At the end of the course we also cover few case studies.

A luigi powered analytics / warehouse stack

Educational notes,Hands on problems w/ solutions for hadoop ecosystem

a set of scripts to pull meta data and data profiling metrics from relational database systems

使用 python 操作大数据的各种组件

This repository focuses on gathering and making a curated list resources to learn Hadoop for FREE.

Set of iPython and Jupyter extensions to improve user experience

Hive API server (offloads most API calls from hived) implemented using Python+SQL