AWS Services for DE — Live Bootcamp | Grow Data Skills

Light
Dark

AWS Services Mastery For Data Engineering With AI Bootcamp

AWS Services For Data Engineering With Projects

By Shashank In Grow Data Skills

Self Paced
English
Completion Certificate

Demo Class Video

AWS Services For Data Engineering With Projects

1 Year

Validity (From the date of Enrollment)

68 Hours

Duration

16

Sessions

5

Projects

INR 4500

This course includes

Content Duration68 Hours
Total Video Sessions16
Total Industry Projects5
AWS Cloud
Quality Assignments & quizzes after each module
Interview preparation guide
Dedicated placement assistance
Resume & Linkedin profile making
Doubt support on private discord community
Certificate of completion

Show More

Tech stack you'll learn

AWS Services For Data Engineering With Projects

Course Content

AWS Services Covered - AWS IAM, AWS S3, AWS S3 Tables, AWS Lambda, AWS CodeBuild, GitHub, GitHub Actions, AWS SNS

✅ Class - 1

Importance of Cloud
On Premises Vs Cloud
Why Cloud for Data Engineering?
AWS – Regions & Availability Zones
AWS S3 - Simple Storage Service Fundamentals
AWS S3 Storage Classes
Role of S3 in Data Engineering & Pipeline Design
AWS Free Tier Account Setup
Amazon Q AI Assistant Usage
Root Use vs IAM User
Setup for billing alerts
General Purpose Buckets
Directory Buckets
Table Buckets - Iceberg & It's fundamentals
Vector Buckets - Vector Embedding & RAG Fundamentals
Create S3 Bucket from AWS Console
Properties & Configuration of S3 Bucket
Lifecycle Management
Create folder object in S3 bucket from AWS Console
Upload/Download/Delete files from S3 bucket from AWS Console
AWS CLI Setup
Create IAM User, Roles & Policies and Access Credentials
AWS CLI Secrets Configuration
Interact with AWS S3 service from command line - Create, Delete, Copy, Make Bucket

✅ Class - 2

Access AWS S3 from Python Program using boto3 library
AWS Lambda Architecture & Fundamentals
Where Lambda Fits in Data Engineering & Pipeline Architecture
Create first lambda function from AWS console
Lambda configurations and IAM role permissions
Testing of lambda function with Test event
Version & Aliases for AWS Lambda
Weighted Aliases for production deployment
Setup S3 Notifcation Event trigger for Lambda
Process S3 Notifcation Event trigger event in Lambda
Lambda logs in cloudwatch
Packaging and deployment of Lambda function in zip format
Concept of Layers in AWS Lambda
Creation and Deployment of Custom Layer
Lambda function execution with custom layer package

✅ Class - 3

Fundamental of AWS Lambda Durable function
Setup AWS Lambda Durable Function which will trigger another Lambda as worker
Execution of AWS Lambda Durable function
Setup Lambda Function along with external Python dependency
Package Lambda function & external Python dependency for deployment
CICD fundamentals for data engineering
Create GitHub repository for code versioning
Design AWS Lambda function to read csv file from S3 based on S3 create event trigger
Setup project in GitHub repository with branching
Create AWS CodeBuild project for CICD of AWS Lambda function deployment
AWS CodeBuild project environment setup for deployment via buildspec.yml and aws cli under shell script
Testing of CICD job in AWS CodeBuild through feature branch and main branch of GitHub

AWS Services Covered - AWS EC2, AWS SQS, AWS EventBridge, EventBridge Pipes, EventBridge Scheduler, AWS RDS, AWS Aurora, AWS Secrets Manager, AWS Glue

✅ Class - 1

AWS SNS Fundamentals
How SNS Works?
Where SNS Fits in Data Engineering
Real-World Pipeline Example (With SNS) : E-Commerce Data Platform
Key role of SNS in Data Pipeline
Setup AWS SNS from console
Standard SNS vs FIFO
SNS Configuration Properties
Create Topic in SNS
Publish message on SNS topic from console
Subscription protocols in SNS
Email subscriber setup for SNS Topic
Create AWS Lambda to send customized messages on SNS topic using boto3 library
AWS EC2 Fundamentals
Setup EC2 Instance on AWS Console
SSH Access of EC2 Instance
AWS SQS Fundamentals
Standard Queue vs FIFO Queue
Core parameters and configurations of SQS - Visibility Timeout, Message Retention Period, Maximum Message Size, Delivery Delay, Receive Message Wait Time
Dead Letter Queue
Difference between AWS SQS vs Kafka
AWS SQS Setup from console
Publish and Consume messages from SQS via console
Create lambda function for mock data generator
Create Event Bridge scheduler to invoke lambda function every minute
Create consumer lambda function and set SQS as trigger to showcase auto polling/consumption of events
Create consumer lambda function and programmatically poll messages from SQS to showcase long polling
Dead Letter Queue Redrive Task

✅ Class - 2

AWS Event Bridge Pipe Fundamentals
Setup SQS to store data for logistics domain
Create AWS Lambda to generate mock data and publish in SQS
Setup AWS Event Bridge Pipe where SQS is source
Apply Filter and Enrichment
Write Lambda function which will be used by Enrichment step for transformation
Add target lambda in AWS Event Bridge Pipe
AWS Glue Fundamentals
What is Glue Data Catalog?
How Does It Work with Glue?
What is a Glue Crawler?
How Does It Handle Schema Evolution?
Setup and create Glue Crawler to crawl CSV files in S3 bucket
Create database in Glue Data Catalog
Registering Table for Crawled S3 data via Glue Crawler
Create Glue Job from scratch to read and process CSV data read through Glue Catalog Table
Data Quality Checks in Glue Job
Publish Data Quality Results
Working with Glue Dynamic Frames
Glue job configurations & properties
Setup Glue crawler to crawl Json data in S3 Bucket and register table for the same in Glue Data Catalog
Setup Glue crawler to crawl partitioned data in S3 Bucket and register table for the same in Glue Data Catalog
Concept of Job Bookmarking & Advantage of it for incremental data processing
Create Glue job using Visual ETL by enabling Job Bookmarking to read incremental data from S3, process it and write data in partitioned manner in output S3 Bucket

✅ Class - 3

AWS RDS Fundamentals
AWS Aurora Architecture
Setup Aurora MySQL
Understanding of Writer instance and Read replicas
AWS Aurora MySQL setup properties
VPC & Subnet Configuration in AWS Aurora
Setting up Security group for AWS Aurora - Inbound & Outbound rules
Connectivity of Writer Instance & Read Replicas from DBeaver
Create username & password secrets of AWS Aurora in AWS Secret Manager service
Create S3, Glue, STS VPC endpoints to create AWS Aurora connection from AWS Glue
Create JDBC based connection for AWS Aurora in AWS Glue
Publish mock IOT data in AWS Aurora table
Setup Glue crawler to crawl AWS Aurora table and register it into Glue Data Catalog
Create Glue Job with Job Bookmarking enabled to read & process incremental data from AWS Aurora table via Glue Data Catalog
Create medallion architecture based data pipeline with AWS Glue, Triggers & Workflows - Telecom Data Domain
Create mock data generator script to generate mock Telecom Data files
Upload CSV files in S3 bucket
Create Glue crawler to crawl raw telecom data and register them as table in Glue Data Catalog
Setup Glue job to prepare bronze layer by reading raw data from Glue Data Catalog, filter & data quality checks and write into Bronze layer folder in parquet format
Setup Glue job to prepare silver layer by reading bronze layer data from S3 path, transform it and write into Silver layer folder in parquet format
Setup Glue job to prepare gold layer by reading silver layer data from S3 path, transform it and write into Gold layer folder in parquet format
Create 3 conditional Triggers to work based on Success status of Crawler & Glue Job
Create Glue workflow with On Demand trigger, attach conditional triggers & Glue job execution as action

AWS Services Covered - AWS Athena, AWS Redshift, Amazon MWAA (Managed Airflow), AWS EMR, AWS Kinesis, Kinesis Streams, DynamoDB, DynamoDB Streams, Kinesis Firehose

✅ Class - 1

AWS Athena Fundamentals
AWS Athena Architecture Breakdown
Where Does AWS Athena Fit in a Data Pipeline?
AWS Athena Query Editor Walkthrough
AWS Glue Data Catalog + Athena Integration
Configure Athena To Save Query Results in S3
Create External Database in AWS Athena
Create External Tables in AWS Athena to Query S3 Data
Create External Tables in AWS Athena for CSV, Json formats
MSCK Repair Table to get latest partitions
S3 Table Buckets & Apache Iceber
Create Iceberg Tables in Athena
Ingest data in iceberg table
Querying iceberg table in Athena
Event driven data pipeline setup for Shopkartdata
Ingest Data In S3
Invoke lambda based on S3 create object trigger
Lambda will invoke Glue Job
Glue job performs Transformations and Data Quality Check, Writes data to S3 in Parquet format
Athena will query the data from S3 for analytics and reporting
AWS Redshift Fundamentals
AWS Redshift Architecture Breakdown
Massively Parallel Processing (MPP)
Two-tier cluster architecture
Leader node
Compute nodes
Columnar storage
Data distribution styles (KEY, ALL, EVEN)
Sort keys
Compression encodings

✅ Class - 2

Setup provisioned Redshift Cluster
Setup serverless Redshift cluster, Network & Security
Create workgroup in Redshift serverless cluster
Create namespace in Redshift serverless cluster
Redshift serverless cluster query editor walkthrough
Create database
Create schema
Create internal tables, column encoding, data distribution
Copy command to load csv data into Redshift internal tables
Unload query to write data into single csv file
Unload query to write multipart parquet data along with manifest file
Copy parquet data into internal table using manifest file
Create external database
Create external schema
Create external tables and query data (Redshift Spectrum)
Create materialized view
Refresh materialized view
AWS EMR Fundamentals and Architecture
Setup EMR Cluster
SSH on EMR Cluster
SSH Tunneling for EMR to access application endpoints
AWS Managed Airflow Fundamentals and Architecture
Setup Airflow Environment
Airflow UI walkthrough
Create airflow dag & deploy to run bash operators
Insurance Data Pipeline orchestrated via Airflow
Daily insurance data json file will land in S3 bucket
Create spark job to read insurance file passed as parameter from Airflow
Spark job performs transformations and write data into Iceberg tables
Setup EMR cluster and enabled Iceberg
Create parameterized Airflow dag which will access input date as parameter, checks for file availability using S3KeySensor Operator, Submits spark job on EMR using EMRAddStepOperator, waits for spark job to complete by using EMRStepSensor operator

✅ Class - 3

EMR Serverless Fundamentals
EMR Studio
EMR Serverless Runtime Application Setup
Airflow orchestrated Spark batch job to run on EMR Serverless
AWS DynamoDB Fundamentals
Create DynamoDB Table
DynamoDB table operations from console
Data ingestion, updation, deletion in DynamoDB table from Python application
DynamoDB Stream
AWS Kinesis Data Stream Fundamentals
Setup AWS Kinesis Data Stream
AWS Data Firehose Fundamental
Ride hailing CDC pipeline
Create DynamoDB table
Enable DynamoDB Stream for CDC
Enable Kinesis Data Stream in DynamoDB to sync CDC events from DynamoDB table to Kinesis
Create Data Firehose stream to sync CDC events from Kinesis data stream to S3
Create Athena table and run analytical queries

AWS Services Covered - AWS Step Functions

✅ Class - 1

What is AWS Step Functions?
Core Components & State Types in AWS Step Function
Where Does Step Functions Fit in Data Pipelines?
FreshCart Quick Commerce Data Pipeline on AWS orchestrated by AWS Step Function
Mock Data Preparation and S3 Setup
Lambda Function to Validate Manifest Files
Lambda Function for File-Level Profiling
Lambda Function to Summarize Profiling Results
Glue Job to Curate Gold Layer Aggregated Data
Athena Queries for Data Quality and Reconciliation
Lambda Function to Monitor Athena Query Status and Decision Making
Lambda Function to Prepare Success and Failure Messages
SNS Email Subscription for Success and Failure Notifications
Event-Driven AWS Step Functions Workflow for FreshCart Quick Commerce Data Pipeline
Integration of Lambda, Glue, Athena, and SNS using Step Functions (Invoke API, StartJobRun, Map, Parallel, Choice, Retry/Catch)
Manual Trigger of Step Function Using Test Events

AWS Services Covered - AWS DMS, Amazon Bedrock, Amazon Q

✅ Class - 1

What is an LLM?
Foundation Model
How a Foundation Model is Born
Inference
Token
Auto-regressive Generation
Context Window
How Text Becomes Tokens
Context Window Sizes
LLM Workflow
LLM Architecture
Input Prompt
Tokenization
Embeddings + Position Information
Transformer Blocks
Masked Self-Attention
Feed Forward Network
Next-Token Prediction
Prompting Basics — The Interface to Foundation Models
System Prompt
User Prompt
Temperature
Embeddings
Cosine Similarity
Vector Database
Semantic Space — Text Becomes Points
RAG — Retrieval Augmented Generation
Foundation Model Providers Landscape
What is AWS Bedrock?
Base Models & The Converse API
Where Bedrock Fits in the Data Engineering Pipeline
Real-World Use Cases for Data Engineers
Retail Support Intelligence Pipeline Using AWS Bedrock
AWS Step Function Orchrestrated data pipeline for Customer Support Analytics
AWS Glue job to process bronze layer data, standardize it and write it into silver layer
AWS Lambda Function to use Amazon Nova Lite LLM model for each processing each row of customer support ticket. Model classifies the issue, assesses sentiment and urgency, redacts PII, generates a summary, and recommends a next action
AWS Lambda Function to Amazon Titan Embedding LLM Model to generate embeddings for enriched support ticket data
AWS Lambda Function to generate cosine similarity between different pairs of support tickets and keep top-3
AWS Glue job to process LLM enriched, embedded and cosine similarity data along with silver layer data to prepare final fact table
AWS Lambda Function to perform Athena queries and prepare Operational summary for leadership using Amazon Nova Lite LLM model

AWS Services Covered
S3, Lambda, IAM, CLOUDWATCH, EC2, SNS, SQS
Event Bridge Scheduler, Event Bridge Pipe, Kinesis, Kinesis Firehose, DynamoDB, SNS, SQS
Step Function, EMR, GLUE, RDS, ATHENA, REDSHIFT, AWS Code Build For CICD, AWS CloudFormation
AWS S3 Tables, AWS DMS, AWS AppFlow, Amazon Bedrock, Amazon Q
Industry Project - 1 : Industrial Maintenance RAG Assistant
Tech Stack: Python, AWS S3, AWS S3 Vectors, AWS Bedrock, AWS Lambda, AWS IAM, AWS DynamoDB, Static Frontend (HTML/CSS/JS), AWS CLI, Text LLM, Embedding LLM
Industry Project - 2 : Retail Brokerage Data Processing Pipeline
Tech Stack: Python, AWS Glue (PySpark), AWS Step Functions, AWS EventBridge, AWS S3, AWS Redshift Serverless, AWS CloudFormation, AWS CodeBuild, Amazon SNS, PyTest, GitHub
Industry Project - 3 : Retail Banking Analytics Data Platform — Transaction Processing, Risk Scoring & Portfolio Intelligence
Tech Stack: Python, PySpark, EMR Serverless, Apache Airflow (MWAA), AWS S3, Apache Iceberg, AWS Glue Data Catalog, AWS Athena, AWS IAM
Industry Project - 4 : Real-Time Ticket Booking & Payment Analytics Platform
Tech Stack: Python, AWS Kinesis Data Streams, AWS Glue Streaming ETL, Spark Structured Streaming, AWS SQS, AWS Redshift Serverless, AWS S3, AWS CloudFormation, AWS Secrets Manager, Amazon VPC, n8n, OpenRouter (Frontier LLM), JavaScrip
Industry Project - 5 : Ride-Sharing Operations & Driver Performance Analytics Platform
Tech Stack: Python, AWS S3, EMR Serverless (PySpark), AWS Step Functions, AWS EventBridge, AWS Glue Data Catalog, AWS Athena, FastAPI, AWS Lambda Function URL, AWS CloudFormation, AWS CodeBuild, AWS SNS, n8n, Telegram Bot, JavaScript, GitHUb

Attention Seeking Resume Preparation and Interview Strategies
Strategies To Crack Tech Interviews
Linkedin Profile Making
How To Expand Your Professional Network On Linkedin
How To Use Various Job Portals
How To Approach For Referrals

Course Schedule

Mode of the Course:

Self Paced

Course Duration:

68 Hours

Total Sessions:

16

Total Projects:

5

Course Validity:

1 year (Starting From Day Of Enrollment)

Programming Language Used:

Python

Prerequisite:

Basic Python, SQL, Spark OR You Can Check 👉

Data Engineering & Python & SQL

⚠️ Important Notice :

The video may not work on Linux due to DRM restrictions. It is only accessible on Chrome when using Windows or macOS.

Workaround: To access the video on Linux, you can create a Windows virtual machine (VM) and watch the video through the VM. Alternatively, you can use our Android or iOS application to view the video on your mobile device.

Instructor

Data Engineering With AWS

Shashank Mishra

AI Data Engineer Ex-Amazon, Expedia, Paytm, Prophecy & McKinsey & Company

Shashank Mishra is a AI Data Engineer with 10+ years of experience building scalable data platforms, real-time data pipelines, cloud-native architectures, and AI-powered data solutions for global organizations.

An MCA graduate from NIT Allahabad, he specializes in modern multi-cloud Data Engineering and its growing intersection with AI. His areas of focus include RAG pipelines, Agentic AI platforms, AI-powered analytics, intelligent data workflows, and production-grade AI applications.

Beyond industry work, Shashank is the educator behind GrowDataSkills, where he creates practical, project-driven content for Data & AI professionals. His community includes 185K+ YouTube subscribers, 25K+ Instagram followers, and 190K+ LinkedIn professionals.

Having mentored 32,000+ learners, he focuses on helping professionals build industry-ready skills, transition into Data Engineering from diverse backgrounds, and prepare for careers at leading technology companies.

AWS Services Mastery For Data Engineering With AI Bootcamp (Self Paced)

INR 4500

Copyright © 2026 Regex Data Learning Pvt Ltd. All Rights Reserved.