Category Archives: Uncategorized
20. Azure Event Hubs and Stream Analytics – Partitions Now in this chapter I want to talk about partitions. Now when it comes to Azure Event Hubs, you can actually have partitions in place and then you can also make use of these partitions when it comes to your Azure Steep Antics job, they’re actually… Read More »
18. Azure Stream Analytics – The importance of time So in the previous chapter, we had looked at the metric of the watermark delay. In this chapter, I just want to stress the fact about how important time is when it comes to Azure Stream antics, because all of the queries that you actually write… Read More »
13. Azure Data Factory – Pipeline Failures Now in this chapter, I just want to give a quick note when it comes to your pipeline failure runs. Now, what I’ve done is I’ve created two pipelines, pipeline one and pipeline two. So if I go on to pipeline one here I just have a simple… Read More »
10. Azure Data Factory – Monitoring – Alerts and Metrics Now, in this chapter, I just want to go through another aspect that is available in the monitoring section of Azure Data Factory, and that’s when it comes on to alerts and metrics. First of all, if you go on to your dashboards here, you… Read More »
7. Azure Synapse – Workload Management Now in this chapter, I just want to go through workload management, which is available as part of your dedicated SQL pool. This is something that we had actually implemented earlier on. So when we were looking at loading data in a SQL pool that time, we had created… Read More »
4. Azure Monitor – Alerts Hi and welcome back. Now in this chapter, I want to go through a special note when it comes to alerts that you can create in Azure Monitor. So if I go on to the monitoring service, so remember, Azure Monitor can be used to gather all of the metrics… Read More »
DP-203 Data Engineering on Microsoft Azure – Monitor and optimize data storage and data processing
1. Best practices for structing files in your data lake Now, in this chapter, I want to give some best practices when it comes to structuring your files. And this is when it comes to building your data lake. So normally when you design a data lake, you might go ahead and create something known… Read More »
DP-203 Data Engineering on Microsoft Azure – Design and Implement Data Security Part 6
17. Lab – Azure Synapse – External Tables Authorization via Azure AD Authentication Now, in this chapter, I want to go through virtual Network service endpoints. Now, what is the purpose of having service endpoints in place? So here in this example, let’s say you have a virtual machine that is part of a virtual… Read More »
DP-203 Data Engineering on Microsoft Azure – Design and Implement Data Security Part 5
14. Lab – Azure Data Lake – Role Based Access Control Now in this chapter I want to explain rolebased access control. Now we know that Azure Active Directory is our identity provider. So here we can create users and the user can actually log into Azure account. Now let’s say that we do define… Read More »
DP-203 Data Engineering on Microsoft Azure – Design and Implement Data Security Part 4
11. Lab – Azure Synapse – Azure AD Authentication – Creating a user Now in the last chapter we had seen how to define a SQL Active Directory admin when it comes to Azure Synapse. Now in this chapter, let’s create a new user in Azure Active Directory and give that user access onto our… Read More »
