After taking this course, you will be able to describe two different approaches to converting raw data into analytics-ready data. One approach is the Extract, Transform, Load (ETL) process. The other contrasting approach is the Extract, Load, and Transform (ELT) process. ETL processes apply to data warehouses and data marts. ELT processes apply to data lakes, where the data is transformed on demand by the requesting/calling application.
ofrecido por


Acerca de este Curso
Computer and IT literacy.
Qué aprenderás
Describe and contrast Extract, Transform, Load (ETL) processes and Extract, Load, Transform (ELT) processes.
Explain batch vs concurrent modes of execution.
Implement an ETL pipelinethrough shell scripting.
Describe data pipeline components, processes, tools, and technologies.
Habilidades que obtendrás
- Extraction, Transformation And Loading (ETL)
- Apache Kafka
- Apache Airflow
- Data Pipelines
Computer and IT literacy.
ofrecido por

IBM
IBM is the global leader in business transformation through an open hybrid cloud platform and AI, serving clients in more than 170 countries around the world. Today 47 of the Fortune 50 Companies rely on the IBM Cloud to run their business, and IBM Watson enterprise AI is hard at work in more than 30,000 engagements. IBM is also one of the world’s most vital corporate research organizations, with 28 consecutive years of patent leadership. Above all, guided by principles for trust and transparency and support for a more inclusive society, IBM is committed to being a responsible technology innovator and a force for good in the world.
Programa - Qué aprenderás en este curso
Data Processing Techniques
ETL or Extract, Transform, and Load processes are used for cases where flexibility, speed, and scalability of data are important. You will explore some key differences been similar processes, ETL and ELT, which include the place of transformation, flexibility, Big Data support, and time-to-insight.
ETL & Data Pipelines: Tools and Techniques
Extract, transform and load (ETL) pipelines are created with Bash scripts that can be run on a schedule using cron. Data pipelines move data from one place, or form, to another. Data pipeline processes include scheduling or triggering, monitoring, maintenance, and optimization. Furthermore, Batch pipelines extract and operate on batches of data. Whereas streaming data pipelines ingest data packets one-by-one in rapid succession. In this module, you will learn that streaming pipelines apply when the most current data is needed. You will explore that parallelization and I/O buffers help mitigate bottlenecks. You will also learn how to describe data pipeline performance in terms of latency and throughput.
Building Data Pipelines using Airflow
The key advantage of Apache Airflow's approach to representing data pipelines as DAGs is that they are expressed as code, which makes your data pipelines more maintainable, testable, and collaborative. Tasks, the nodes in a DAG, are created by implementing Airflow's built-in operators.
Building Streaming Pipelines using Kafka
Apache Kafka is a very popular open source event streaming pipeline. An event is a type of data that describes the entity’s observable state updates over time. Popular Kafka service providers include Confluent Cloud, IBM Event Stream, and Amazon MSK. Additionally, Kafka Streams API is a client library supporting you with data processing in event streaming pipelines.
Reseñas
- 5 stars67,41 %
- 4 stars23,59 %
- 3 stars4,49 %
- 2 stars3,37 %
- 1 star1,12 %
Principales reseñas sobre ETL AND DATA PIPELINES WITH SHELL, AIRFLOW AND KAFKA
Perfect environment to make experiments! Very easy and powerful in use.
Nice intro to ETL and Data Pipelines. Beginner level easy to follow hands on Airflow and Kafka.
Excellent introduction to this topics. Labs contain all you need to know how to start using this type of technologies. Highly recommended.
It's one of the most challenging courses I've been enrolled!
Preguntas Frecuentes
¿Cuándo podré acceder a las lecciones y tareas?
¿Qué recibiré si me suscribo a este Certificado?
¿Tienes más preguntas? Visita el Centro de Ayuda al Alumno.