- Course overview
- Course details
- Prerequisites
Course overview
About this course
In this course, you learn about data engineering on Google Cloud, the roles and responsibilities of data engineers, and how those map to offerings provided by Google Cloud. Participants will also learn about ways to address data engineering challenges.
Audience
-
Data engineers
-
Database administrators
-
System administrators
Course details
Module 1: Data Engineering Tasks and Components
• The role of a data engineer
• Data sources versus data sinks
• Data formats
• Storage solution options on Google Cloud
• Metadata management options on Google Cloud
• Sharing datasets using Analytics Hub
Module 2: Data Replication and Migration
• Replication and migration architecture
• The gcloud command-line tool
• Moving datasets
• Datastream
Module 3: The Extract and Load Data Pipeline Pattern
• Extract and load architecture
• The bq command-line tool
• BigQuery Data Transfer Service
• BigLake
Module 4: The Extract, Load, and Transform Data Pipeline Pattern
• Extract, load, and transform (ELT) architecture
• SQL scripting and scheduling with BigQuery
• Dataform
Module 5: The Extract, Transform, and Load Data Pipeline Pattern
• Extract, transform, and load (ETL) architecture
• Google Cloud GUI tools for ETL data pipelines
• Batch data processing using Dataproc
• Streaming data processing options
• Bigtable and data pipelines
Module 6: Automation Techniques
• Automation patterns and options for pipelines
• Cloud Scheduler and Workflows
• Cloud Composer
• Cloud Run Functions
• Eventarc
Prerequisites
- Prior Google Cloud experience at the fundamental level using Cloud Shell and accessing products from the Google Cloud console.
- Basic proficiency with a common query language such as SQL.
- Experience with data modeling and extract, transform, load (ETL) activities.
- Experience developing applications using a common programming language such as Python.
Enquiry
Course : Data Engineering on Google Cloud
Enquiry
request for : Data Engineering on Google Cloud
