The constructs of monitoring keep evolving at a rapid pace, especially in cloud and hybrid architectures. It is important to remain updated about changes on the monitoring front to ensure that you are using the latest and greatest of the tools and services available at your disposal. With DevOps culture being absorbed in all aspects of IT management, monitoring cannot be far behind. This chapter will focus on the aspects on continuous monitoring and upcoming new features and changes in Azure monitoring.

Continuous Monitoring

In an environment where DevOps processes and tools are used for end-to-end deployment of infrastructure and applications, it is important to incorporate monitoring in all phases of the IT life cycle. This is often referred to as Continuous Monitoring aligned with the DevOps terminology of Continuous Integration and Continuous Deployment. The goal is to identify issues early on during the application life cycle and take necessary remedial actions.

Full Stack Visibility

Azure Monitor can be integrated with your application from the initial phase of development through IDEs like Visual Studio and Visual Studio code. When the code is deployed, it can be used with DevOps tools to identify possible issues based on monitoring information. For ongoing maintenance and management after deployment, you could use the out-of-the-box monitoring features available through Azure Monitor or integrate it with your inhouse tools as discussed in Chapters 4 and 5 of this book. Full stack visibility of your business-critical applications can be achieved through components like Azure DevOps, Release pipeline integration with Azure monitoring, Application Insights, and Live Metrics. Figure 6-1 tries to draw the overall picture in this regard.

Figure 6-1
figure 1

Constructs of full scale visibility of continuous monitoring

Full size image

As the diagram depicts, the full stack visibility configuration is linked to the life cycle of the code starting from the repository it resides in that can be linked to Azure DevOps to create a continuous integration and delivery pipeline. Quality gates in the Azure DevOps pipelines enable monitoring of important health and performance metrics as the application is deployed across multiple environments. Additionally, Live Metrics and Application Insights take care of the monitoring once the application is deployed to Azure.

Continuous Deployment and Monitoring

Azure pipelines can be used for continuous deployment of both your code and applications. Monitoring can be integrated to your application release pipelines so that the collected data can be used for decision-making, that is, whether the code should be deployed to a target environment or if a rollback should be triggered.

  • The template “Azure App Service deployment with continuous monitoring” available in Azure pipelines can be used to configure application insights for your App Service during deployment.

  • The template has prebuilt alerts rules for the failed requests, service availability, server response times, and exceptions. These rules can be configured based on your requirements during pipeline setup.

  • Pre-deployment and post-deployment gates can be configured to use “Query Azure Monitor alerts” so that you can access both Azure Monitor as well as Azure Application Insights alerts.

  • Update evaluation option settings based on desired business logic. For example, you can set a timeout for the gate to fail based on the alert configuration.

  • Post-configuration, you can review the release logs to gain insights on the behavior of the gates.

You can also use options like status monitor and live metrics to get real-time information on your application performance. For historical data, it is recommended that you leverage Application Insights. Application Insights can also be used to learn the application behavior over a period and fine-tune it accordingly. The key aspect is to identify the integration point for monitoring from which you get the information relevant to make right decisions.

Relevance of Infrastructure Monitoring

In previous chapters we covered the different ways to configure monitoring for your infrastructure. Here let us summarize what we have learned to reiterate the importance of monitoring infrastructure, which is the lifeblood of your applications.

  • The metrics available by default would be your starting point to collect monitoring data. The relevant metrics should be identified and used to create alerts or pinned to Azure Dashboards for constant monitoring.

  • Azure Monitor for VM or VM insights (Preview) offers additional information about health, performance, and automated dependency mapping. This feature gives a holistic view about your VM as well as its dependencies. Furthermore, the information is streamed to Log analytics and can be used to run custom queries to gainer deeper insights.

  • Monitoring solutions in the Log analytics workspace can be used to gain better visibility into the status of your applications and virtual machines connected to your workspace. Solutions like Antimalware assessment, Azure activity logs, NSG analytics, and AD health check are available out of the box, where information is represented in prebuilt graphs for instant visibility to the service status.

  • Applications using microservices architecture through services like AKS, ACI, etc., can use Azure Monitor for containers as well as container monitoring solutions available in Log analytics to keep an eye on the health of a container-hosting infrastructure.

  • Deployment of Infrastructure using the Infrastructure as Code approach will help you to incorporate monitoring components from day 1 of the deployment. This can be done using popular approaches like ARM template or Terraform via DevOps pipelines.

  • Azure resource group monitor is a service that is in preview that can be used to get a bird’s-eye view of dependent components in a resource group. You can view information on active alerts, health, and performance issues, etc., for all resources in your resource group using this service.

Continuous Changes

Microsoft Azure is an ever-evolving platform with over thousands of new features being released across various services each year. It is expected that some of the services, features, and options may change over period; get better; or even getting deprecated. Hence it is of the utmost importance to always review the Azure products by region (https://azure.microsoft.com/en-us/global-infrastructure/services/) website every time there is a new deployment. To that effect we will explore some of the latest and upcoming features in this chapter. Although these may change, no conversation is complete without looking at “What’s new?” Let us now put this another way.

  • If you do not want to use a “preview” feature for your production workloads and since you are unsure of how long it might take to be globally available (GA) as well, check the Azure products by region site for a region and its services. You may however, at times get a confirmation from the Azure product team if any of the public preview features are available with production support. This may help in considering the new service or feature to be included in the project or solution.

  • Should you wish to keep a track of what is new and evaluate features for your solution and change the way it is currently configured, you may want to bookmark the Azure updates website: https://azure.microsoft.com/en-us/updates/.

Now let’s look at some of the interesting enhancements.

Azure Monitor Status Blog

Monitor Azure Monitor using Azure Monitor status blog (Figure 6-2), if you are experiencing service issues, keeps you updated on the developments. The current URL is: https://techcommunity.microsoft.com/t5/Azure-Monitor-Status/bg-p/AzureMonitorStatusBlog. It not only provides ongoing issues and developments but also tries to suggest any workarounds that can be implemented by customers to provide relief.

Migrate from Classic Alerts Easily

The new Azure Monitor alerts are better with added granularity and multiple filter capability. You can also create a single rule that spans across multiple resources in various resource groups in one Azure region. This will help with the ease of rules management and quick deployment of alerts. All existing classic rules will be migrated by Microsoft starting in July 2019 if it has not already done so, using the tools provided to existing Azure customers.

Azure Monitor Now Supports Containers in China

Microsoft recently rolled out support for Azure Monitor for containers in Azure China. It provides end-to-end Kubernetes monitoring for AKS from infrastructure to workloads.

Identify Open or Bound Ports on Your Virtual Machines

This has always been a point of focus for security and operational teams from the security and troubleshooting aspects. With Azure Monitor you can now easily analyze which ports are open and active. The Microsoft Azure Monitor product team has also included an Azure VM workbook that includes active ports, failed connections, open ports, connections overview, traffic, etc. You can access this workbook once you enable the Insights (preview) monitoring from the Virtual machine settings pane. Once done, go to the Log analytics workspace, select Virtual Machines(preview) ➤ Map ➤ view workbooks. Figure 6-2 depicts the various network-related workbooks from Azure Monitor Virtual Machines Insights (preview).

Figure 6-2
figure 2

Virtual machine workbook in Azure Monitor

Full size image

Alternatively, you can use a Log analytics query and list the open ports. Here is a simple example to list the number of open ports. Figure 6-3 presents a chart of a sample query and the possible outputs.

Figure 6-3
figure 3

Log analytics query and output for Virtual machine open ports

Full size image

Advanced Scoping with Additional Resources

Azure Monitor now includes advanced scoping for additional resources by embedding a “Logs” search from the resource menu. Users will not have to choose any specific Log analytics workspace, as all logs are automatically aggregated from various workspaces that include logs that are associated with the resource selected. Figure 6-4 represents the Log analytics workspace scoped to Virtual network. You can simply navigate to it by selecting the resources (Virtual Network in this case) and then Logs under the Monitoring section.

Figure 6-4
figure 4

Advanced scoping for Virtual Network

Full size image

However, for this to work, Access control mode of the workspace has to be set to “Use resource or workspace permissions.”

Application Change Analysis with Azure Monitor

It is difficult to track and identify which changes cause a failure when multiple teams work together. App services Diagnose and Solve Problems now contains several traces and rules that can help you to identify a failure of the application, including IP configuration rule changes, incorrect connection strings, any binary modifications, or any web configuration file modifications. You can navigate by clicking on “Diagnose and solve problems” of your App Service as shown in Figure 6-5.

Figure 6-5
figure 5

Diagnose and solve problems of your App Service

Full size image

Figure 6-6 is an output of a failure where the client failed to access the website since it was restricted by a Networking rule in the App Service configuration.

Figure 6-6
figure 6

App Service IP restriction in Diagnose and solve problems

Full size image

Similarly, you can identify various failure scenarios, including application changes or various HTTP 4xx errors, as depicted in Figure 6-7.

Figure 6-7
figure 7

HTTP 4xx errors in Diagnostics and solve problems

Full size image

Summary

The items discussed in this chapter are merely a fraction of the vast flow of feature releases. There are continuous improvements in terms of new services supporting new regions, new features of the services being released in additional regions, and more and more capabilities are made globally available every other week. With the pace at which Azure is expanding its services and features, it opens up new scenarios of how a solution is designed, planned, and implemented. The same goes with Azure Monitor. In the coming days, expect to see several changes that make this scalable, multi-cloud, highly available, reliable monitoring platform even more agile and robust.