In this blog we are going to see about Azure Event Grid in a detail manner along with examples. In today's digital world, the ability to seamlessly integrate and respond to events in real-time is crucial for enterprises. Azure Event Grid, a fully managed eventing service offered by Microsoft Azure, provides a scalable and efficient solution for handling event-driven scenarios. As an Azure data engineer, understanding the capabilities and benefits of Azure Event Grid is essential for leveraging its power to build robust data integration pipelines.
Hope you are excited. For an Azure Data Engineer, this topic is very important. Because most of the time they are going to use Azure Event Grid in their work. It will be very helpful when they are working in event based tasks. And it will be very helpful when they are working in Azure Event hubs. Lets gets started,
In simple words it is called as a Powerful Eventing Service for Cloud-based Data Integration. Azure Event Grid is a cloud-based service that simplifies the development of event-driven architectures by providing a publish-subscribe model for routing events. It acts as a centralized eventing backbone, enabling seamless communication between event publishers and subscribers. The service allows developers to define and publish events from various Azure services or custom applications, and subsequently, these events can be consumed by interested subscribers.
Don't get confused by the above statements, let we make it very simple. " Event grid will trigger something based on the event. The something nothing but the endpoint" , for example it can be Azure Function, webhook etc.
Now look at the below diagram,
From the above diagram , you wont forgot about the process, that is as simple as that event trigger will trigger the endpoint based on the event occurred.
Now let us see this with an example,
Example 1 :
This is a very Important Example so give more attention!!!, so that you will understand better. Assume you are using Azure event grid with captured event enabled. The captured event will create files to ADLS consists of data present in azure event hub based some time and also some size of data that is fixed by the user. The size here mentioned is the file will be created once the file size is met , which is predefined on creation of captured events. In captured events we can also select the format of files that needs to be generated.
So we can create a event gird in Azure event hubs, by going to event option in the azure event hub namespace. So only one event grid can be created for the entire Azure event hub namespace.
After creating that we need to create the Event Subscription. Over there we need to give option as captured file. And select the endpoint. Let us assume here that we are selecting the endpoint as azure functions.
In the endpoint we need to select the required azure function name. And give create.
Thus now whenever a file is captured by the captured event , event grid will trigger the Azure Function with the file details , such as file path, event count, sequence number, event Time etc. So that we can use the Azure function to receive this event and process it and sent it to a table. Such that we can use the file path to process the file further.
Example 2:
Let us assume we need to call the Azure logic apps whenever some event happened. For that we can go to event grid and create a event grid. Now select that and create a event subscription. And give the end point as the Web hook and give the http trigger url of the azure logic app. Assume the logic app is capable of sending the mail to the required mail id's .
Now the important thing here is, Since the endpoint is the web hook, the event grid expects the reply back from the azure event grid. so it will wait for 10 minutes . within that time there should be a reply back from Azure logic apps, then only the deployment will get success. Otherwise it will be failed.
That can be done using Postman within 10 minutes. That is the Event grid will send the "validation key" as well as "reply back url". So we need to use postman to POST the validation key back to the reply back url.
Or else you can add this flow in Azure logic app itself like whenever validation key arrives it should return back the validation key, else the business logic. Since it is a one time process for deployment we can do this in Postman itself.
Some of the other use cases of Azure Event Grid are:
1) Data Integration: Azure Event Grid plays a crucial role in data integration scenarios, allowing real-time updates and seamless communication between different data sources and destinations. For example, it can be used to trigger data pipelines in Azure Data Factory whenever new data arrives in a storage account or notify subscribers when a database record is updated.
2) Serverless Applications: Event Grid enables serverless applications to respond dynamically to events and perform reactive actions. For instance, an Azure Function can be triggered by an event in Event Grid, allowing the application to process the event payload and take appropriate actions in response.
3) IoT Event Processing: With its scalability and efficient event handling capabilities, Azure Event Grid is an excellent choice for IoT event processing. It can receive events from IoT devices, process them in real-time, and trigger downstream actions such as storing data in Azure Cosmos DB or sending alerts to subscribers.
These are only examples, but always remember the two examples that i explained, that will be very useful to understand the concept and also you will come across most of the time.
Thus in this blog we saw about the event grid and also with two important examples. Hope this is useful to you. Keep following this blog to get similar interesting content.
Thank You !!!
0 Comments