Essential Sql - Azure Data Factory And Data Engineering

What you'll learn
Learn how to construct ELT solutions using Azure Data Factory and SQL Server.
Implement various ELT solutions using Mapping Data Flows, Copy Activities, and Stored Procedures.
Develop the skills to build and troubleshoot Azure Data Factory pipelines with confidence.
Enhance pipeline reliability by mastering the use of parameters to optimize your data workflows.
Learn efficient methods for storing and structuring data using Azure Storage services.
Understand the distinctive roles played by Azure Data Factory, Data Storage, SQL Server, and Data Bricks in the creation of an ELT solution.
Requirements
Familiarity with SQL and basic programming skills can be beneficial.
Proficiency in programming will be valuable when working with scripting and data transformation tasks
Experience in troubleshooting and debugging technical issues will aid in building and troubleshooting Azure Data Factory pipelines.
Description
Use Azure data factory to automate data engineering tasks. This is a great course to introduce yourself to Azure Data Factory's capabilities. We'll look at the data factory from a data engineer perspective.Through examples we'll work side by side to create an Extract Transform and Load process to ingest movie rating data.We are going to explore three different ways to transform your data.The first is by using Azure Data Mapping flows. These are great no-code ways to do ETL.We'll then look at how you can do the same transformations using Python. This way, if you love Python, you know a solid way to use Azure data factory to work with your data.Lastly, we'll look at how you can create pipelines to use your knowledge of SQL and stored procedures.What you'll like about this course is that once you learn one way to transform the data, you can use that knowledge to learn about the other methods. So, if you're a SQL expert but soft on python, you can learn the SQL way first, then go back and try it using Python. In the end you come out learning and appreciating alternative methods to ingesting, transforming, and storing data.
Overview
Section 1: Introduction
Lecture 1 Welcome to the Course
Lecture 2 Who Should Take Azure Data Factory?
Lecture 3 Important! What to Expect from the Course.
Lecture 4 How to Take the Course
Section 2: Set Up Azure
Lecture 5 Setup Azure Account
Lecture 6 Setup Azure Resource Group
Lecture 7 Setup Azure Storage Account
Lecture 8 Create Microsoft Data Factory
Lecture 9 Test Data Factory to Storage Link
Lecture 10 Setup Azure Database
Lecture 11 Set up Azure Data Studio
Lecture 12 Test Azure Database
Lecture 13 Setup Microsoft DevOps
Section 3: Case Study - Using Data Mapping Flows to Refine Data
Lecture 14 Medallion Architecture and Bronze Layer Introduction
Lecture 15 Bronze Layer - Setup Data Lake Folders to support the Medallion Architecture
Lecture 16 Bronze Layer - Prep data using Data Mapping Flows.
Lecture 17 Bronze Layer - Execute Mapping Data Flow using ADF Pipeline.
Lecture 18 Silver Layer - Introduction
Lecture 19 Silver Layer - Building Mapping Flow Part 1 - Data Types
Lecture 20 Silver Layer - Building Mapping Flow Part 2 - Filter and Deduplication
Lecture 21 Silver Layer - Building Mapping Flow Part 3 - Parsing Addresses
Lecture 22 Silver Layer - Building Mapping Flow Part 4 - Touchups and Final Check
Lecture 23 Gold Layer - Introduction
Lecture 24 Gold Layer - Build Dimensions and Fact table using Mapping Flow
Lecture 25 Gold Layer - Automate Dimensions and Fact table Builds using ADF
Lecture 26 Gold Layer - Refactor ADF Pipeline and then See Results in PowerBI
Section 4: Case Study - SQL and the Azure Data Factory Copy Activity to Refine Data
Lecture 27 Medallion Architecture and Bronze Layer Introduction
Lecture 28 Bronze Layer - Using ADF Copy Activity
Lecture 29 Silver Layer - Introduction
Lecture 30 Silver Layer - Land Data into SQL
Lecture 31 Silver Layer - Create Address Split Query
Lecture 32 Silver Layer - Create Stored Procedure Transform Loaded Data
Lecture 33 Silver Layer - Execute Stored Proc from Data Factory Pipeline
Lecture 34 Gold Layer - Introduction
Lecture 35 Gold Layer - Dimension Building
Lecture 36 Gold Layer - Build the Sales Order Fact
Lecture 37 Gold Layer - Move Table to Gold (Production)
Lecture 38 Gold Layer - Review Start Schema in PowerBI
Section 5: ADF Pipeline and Activities Reference
Lecture 39 Azure Data Factory - Introduction
Lecture 40 Azure Data Factory - Variables and Parameters
Lecture 41 Azure Data Factory - Parameters and Return Values
Lecture 42 Azure Data Factory - If Activity
Lecture 43 Azure Data Factory - Switch Activity
Lecture 44 Azure Data Factory - For Each Activity
Lecture 45 Azure Data Factory - Get Meta Data and IF
Lecture 46 Azure Data Factory - Filter Activity
Section 6: Resources
Lecture 47 Product Database
Aspiring Data Engineers: Individuals who are new to data engineering and want to learn how to use Azure Data Factory to build data solutions.,Business Analysts: Analysts who work with data to make business decisions. Understanding data engineering can help them better utilize and secure data for decision-making.,Business Intelligence Developers: BI developers seeking to automate data pipelines for their reporting and analytics needs using Azure Data Factory.,Cloud Architects: Professionals responsible for designing and implementing cloud solutions. They may want to understand how Azure Data Factory fits into their architecture.,Data Analysts: Analysts who want to expand their skills by learning how to move and transform data using Azure Data Factory for better analysis.,Data Architects: Professionals responsible for designing data architectures and systems. They need to ensure that the data solutions they design are secure, scalable, and well-managed.,Data Scientists: Data scientists who want to work with clean and well-organized data, making Azure Data Factory knowledge beneficial for data preparation.,Database Administrators: DBAs looking to integrate Azure Data Factory into their data management processes, especially for data movement and ETL (Extract, Transform, Load) tasks.,Software Developers: Developers who want to gain expertise in building data solutions and working with databases. They might be interested in integrating data management and security practices into their software development workflows.,System Administrators: IT administrators responsible for managing Azure accounts and resources may need to understand Azure Data Factory for resource allocation and monitoring.

Homepage

Код:

https://www.udemy.com/course/essential-sql-azure-data-factory-and-data-engineering/

Buy Premium From My Links To Get Resumable Support,Max Speed & Support Me

No Password - Links are Interchangeable

Essential Sql: Azure Data Factory And Data Engineering	Литература, искусство и культура, книги	2023-11-16
Essential SQL Azure Data Factory and Data Engineering	Литература, искусство и культура, книги	2023-12-26
Azure Data Factory Deployment Essentials - A DevOps Approach	Литература, искусство и культура, книги	2023-12-13
Azure Data Factory By Example Practical Implementation For Data Eng...	Литература, искусство и культура, книги	2024-03-24
Data Engineering on Microsoft Azure: The Definitive Guide	Литература, искусство и культура, книги	2024-04-09

www.prizrak.ws

Меню навигации

Пользовательские ссылки

Информация о пользователе

Essential Sql - Azure Data Factory And Data Engineering

Сообщений 1 страница 1 из 1

Поделиться12023-11-15 22:19:36

Похожие темы