arrow_back
Course Index
GCP Course Index
Study Matrials
GCP Study Material
GCP Data Engineer Videos
Session - 1 What is Data Engineering, Google Cloud, Course Overview
Session - 2 Why we need Cloud, Traditional vs Cloud, GCP Introduction, Why learn GCP, Notes Accessing
Session - 3 How to Interact with GCP Console, Cloud Shell, Cloud SDK, Third Party Tools
Session - 4 Console Walkthrough Console Overview, setup free Tier Account , Project Setup, Enable Billing, Enable APIs
Session - 5 GCP Locations Regions, Zones, Selection Criteria, Importance
Session - 6 GCP IAM & Admin Introduction,Identities, Roles
Session - 7 GCP IAM & Admin Identities, Roles, Roles Types, Policy
Session - 8 GCP IAM & Admin Policy, Resource Hirarchy, Role Creation and Assignement, service account understanding
Session - 9 Linux Basics ls, cp, mkdir, mv, cd, pwd, rm, touch, nano, rmdir, cat, head
Session - 10 Google Cloud Storage Introduction, naming, location, storage class(standard, nearline, coldline, archive), retention policies
Session - 11 Google Cloud Storage Bucket creation(Console, Shell), access control, Interacting Cloud Storage via Cloud Shell, commands
Session - 12 Cloud SQL Introduction, creating demo-instance, Interact with Cloud SQL via Workbench, Cloud Shell
Session - 13 Cloud SQL Interact with Cloud SQL via Workbench, Cloud Shell
Session - 14 Cloud SQL Database Migration - Manual Method, Automated using DMS
Session - 15 BigQuery Introduction, Architecture, working culture, creating sample dataset, table
Session - 16 BigQuery different ways of creating table in BigQuery (Editor,Console, Shell, SDK), basic queries
Session - 17 BigQuery Storage optimization techniques, partitioning (various types), clustering
Session - 18 BigQuery spotify case study and assignements
Session - 19 BigQuery working nested fields and repeated fields for better optimizations
Session - 20 BigQuery views, materialized views and authorized views
Session - 21 DataProc Hadoop and Spark Introduction, What is DataProc, Cluster Types and Cluster Creation
Session - 22 DataProc pyspark, pyspark jobs, extract step - how to read the data from multiple data sources (csv, json, txt, avro, parquet, mysql, bigquery)
Session - 23 DataProc load step - how to load the data to multple data sinks (csv, json, txt, avro, parquet, mysql, bigquery)
Session - 24 DataProc transform step - how to perform various transformations-1
Session - 25 DataProc transform step - how to perform various transformations-2, and assignements
Session - 26A DataProc creating a pyspark job and submit to dataproc cluster
Session - 26B DataProc End to End batch pipeline (dataproc, bigquery, gcs)
Session - 27 Dataflow Introduction, Apache Beam pipeline Introduction, differemce between dataproc and dataflow
Session - 28 Dataflow extract step - how to read the data from multiple data sources (csv, json, txt, avro, parquet, mysql, bigquery)
Session - 29 Dataflow load step - how to load the data to multple data sinks (csv, json, txt, avro, parquet, mysql, bigquery)
Session - 30 Dataflow transform step - how to perform various transformations-map, filter, pardo, groupbykey, combineperkey ..and assignements
Session - 31 Dataflow creating beam pipeline, creating beam pipeline from templates(gcs to bigquery), pubsub
Session - 32 Dataflow End to End Steaming pipeline creation (pubsub, dataflow, gcs, bigquery)
Session - 33 ComposerAirflow introduction, What is DAG, how to create DAG, composer enviromenet creation, cron job format
Session - 34 ComposerAirflow level_1_dag creation, logs checking and error fixing
Session - 35 ComposerAirflow level_2_dag creation, levele_3_dag creation and assignements
Session - 36 Datafusion Introduction and pipeline creation assigment, BigTable Overview for knowledge, types of databases
Session - 37 Cloud Functions Introduction, why we need it, creating cloud function - gcs trigger and pubsub trigger - assignement
Session - 38 Terraform introduction, terraform actions, lifecycle, creating infrastructure using terraform dataproc, bigquery dataset, variables for dynamic code
Session - 39 final session Resume Assistance, Interview Prep and Certication Ready session