1. Introduction to BigData

1. What is Data?
2. What is Database?
3. What is BigData?
4. What are the challenges of BigData?
5. Why Traditional Databases Doesn't handle Bigdata

2. Introduction to Hadoop

1. What is Hadoop?
2. How Hadoop overcome bigdata challenges
3. Hadoop Architecture
4. Hadoop Daemons
5. HDFS
6. YARN
7. MapReduce

3. Introduction to Spark

1. Spark Architecture
2. Spark internals
3. Spark RDD
4. Spark DataFrame
5. Spark Streaming

4. Basics of Cloud Computing

1. What is Cloud?
2. Types of Cloud deployment models

A. Private Cloud
B. Public Cloud
C. Hybrid Cloud

3. Types of Cloud Services

A. IaaS – Infrastructure as a Service
B. PaaS – Platform as a Service
C. SaaS – Software as a Service

5. Cloud computing Platforms / Vendors

1. Azure
2. AWS – Amazon Web Services
3. GCP – Google Cloud Platform etc

6. Introduction to Azure

1. Azure Portal Walkthrough

A. What is Subscription?
B. What is a Resource Group?
C. What is a Resource?

2. Overview of Azure Resources / Services

A. Data Factory
B. Azure Data bricks
C. BLOB Storage, Data Lake Storage Gen1 and Gen2
D. Azure SQL Server, SQL Database
E. Key Vault
F. Function App
G. Logic Apps
H. Log Analytics Workspace

7. Introduction to Azure Databricks

1. Azure Databricks Architecture
2. Azure Databricks Main Concepts

8. Creating an Azure Databricks Service

1. Creating a Databricks worspace in Azure Portal
2. Databricks service using the Azure CLI(command-line interface)
3. Databricks service using Azure Resource Manager(ARM) templates
4. Ading users and groups to the workspace
5. Creating a cluster from the user interface(UI)
6. Getting started with notebooks and jobs Azure Databricks

9. Databricks Cluster Management

1. Creating and configuring clusters
2. Managing Clusters

A. Displaying clusters
B. Starting a cluster
C. Terminating a cluster
D. Delete a cluster
E. Cluster Information
F. Cluster logs
G. Cluster access control

3. Types of Clusters

A. All pupose clusters
B. Job cluster

4. Databricks Pools

A. Databricks without pools
B. Databricks with Pools

5. Clusters Mode

A. Standard
B. High Concurrency
C. Single Node

6. Autoscalling
7. Databricks runtime versions
8. Multiuser Clusters

10. Databricks Utilities

1. credentials utility
2. Data utility
3. FilSystem utility
4. Jobs utitlity
5. Library utility
6. Notebook utility
7. secrets utility
8. widgets utility
9. Limitations of utilities

11. Databricks Integration with

1. Blob strorage storage
2. Azure Datalake storage gen2
3. Azure SQL Database
4. Synapse
5. snowflake

12. Databricks- Handle multiple file formats

1. CSV Data
2. JSON Data
3. parquet files
4. Excel files
5. XML files Data
6. ORC file format
7. AVRO files

13. Databricks - Batch Processing

1. Historical Data load
2. Incremental Data load
3. Date Transformations
4. Aggregations
5. Join Operations
6. window functions
7. union operations

14. Databricks – Streaming API

1. What is streaming?
2. Process streaming using Pyspark API
3. Handling bad records
4. Stream data into Gen2lake
5. Load the data into Tables

15. Databricks - Lakehouse (Delta Lake)

1. Difference between Data lake and Delta Lake
2. Introduction to Deltalake
3. Features of DeltaLake
4. How to create delta table
5. How to DML operations in Delta Table
6. Merge statements
7. Handling SCD Type1 and Type2
8. Handling Data Deduplication in delta tables
9. Handling streaming Data in Delta lake

16. Workflows in Databricks

1. Introduction to workflows
2. Create,run and manage Databricks jobs
3. Schedule Databricks jobs
4. Monitor Databricks Jobs