8000 GitHub - dominicci/gcp-batch-ingestion-bigquery: Uses Cloud Build to deploy a scalable batch ingestion pipeline consisting of GCS, Cloud Functions, Dataflow and BigQuery · GitHub
[go: up one dir, main page]

Skip to content

About

Uses Cloud Build to deploy a scalable batch ingestion pipeline consisting of GCS, Cloud Functions, Dataflow and BigQuery

Resources

Stars

0 stars

Watchers

0 watching

Forks

 
 

Repository files navigation

Uses Cloud Build to deploy a scalable batch ingestion pipeline consisting of GCS, Cloud Functions, Dataflow and BigQuery

See cloudbuild.yaml for an idea of what this CI/CD pipeline does. Essentially it runs a container for each step of the build using Cloud Build and deploys each component of the pipeline to GCP. It also uses Terraform to create the initial buckets for deploying all the binaries to. See the infra.tf file for all that nonsense.

Here's what it looks like:

screen shot 2018-09-04 at 5 33 17 pm

About

Uses Cloud Build to deploy a scalable batch ingestion pipeline consisting of GCS, Cloud Functions, Dataflow and BigQuery

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

0