Observability: End to End Monitoring (Part 1)

Common requested feature from Observblity solution from customers:

• Open Telemetry based

• Verbose logging support

• Retention period customisation

• Cost Efficient

• Alerts/Notification customisation

• Absorb High Volume scale

• Analytics and Holistic Dashboards

First important question is What all components one should typical monitor specifically for Cloud Native Apps to have holistive view of entire estate across multiple moving parts

• Infra/Platform Monitoring - Log Analytics

• App Monitoring - App Insights

• UI/UX - App Insights

• Dependency Monitroing - App Insights with LA

• Database - Az Monitor

• Edge Devices - Az Monitor

• Dashboard - Az Grafana

In This scenario we are using Azure Monitor’s different feature to covers all important components of cloud native systems, Here is High level of Data collection Pillars of Azure Monitor Solution

• Monitor Security posture of all azure subscription in single pane glass via enable via Grafana dashboad

• Create Azure managed Grafana & add azure monitor workspace as Data sources.

• In Dashboards section you will find many builtin dashboard for azure resource including Defender for Cloud, Choose Defender for Cloud to see al alerts and recommendation from MDC.

Infrastructure Monitoring capablities using Azure Monitor

Enable Traffic Analysis:

1. Ensure that you have Network Watcher deployed in the same region and have a Log Analytics workspace set up.

2. Navigate to the Azure Portal and go to Network Watcher. Then, select either NSG Flow Logs or VNet Flow Logs.

3. Choose the Network Security Group (NSG) or Virtual Network you want to monitor.

4. Click Enable for Flow Logs ,select storage account and retention period

5. In same flogs logs plane, toggle Traffic Analytics to On and select Log Analytic workspace

Traffic Anlaysis of your Virtual Network on Azure, including the following information:

• Who is connecting to the network?

• Where are they connecting from?

• Which ports are open to the internet?

• What's the expected network behavior?

• Are there any sudden rises in traffic?

Total volume of traffic in your Network Recorded information about ingress and egress IP

Below screenshot reflects Top VM/server with most egress communication with Internet

Workloads receving Traffic from Internet on different Ports

Traffic consumption of workloads on Azure based on IP

Map view of Malicious Traffic blocked all over world via Intenet

Virtual Machines Performance Metrics

Container-based workloads can be effectively monitored using Azure Monitor's Container Insights. This tool provides a detailed overview of container health, including the status of underlying nodes and performance utilization of both containers and the supporting infrastructure.

Additionally, it offers logging and tracing capabilities for application pods and their peripheral services, enabling you to track and analyze application performance and troubleshoot issues efficiently.

High Level Dashboard view Infra components of Application(via native Az Monitor)

Kubernetes Cluster Overview in Grafana via Prometheus Metrics (via Azure Managed Prometheus)

Web, application, and database logs/traces can be configured as custom logs using Log Analytics in Azure Monitor. In this example, logs are parsed to highlight specific details such as URLs, raw data, and HTTP status codes. This setup can be further customized based on the specific requirements of the application or analytics team, allowing for tailored insights and analysis.

Custom logs in data collection rules via Azure Monitor can be used to capture any time series-based text logs. This feature allows you to define specific log formats and structures, enabling the collection and analysis of time-stamped data that is crucial for monitoring and troubleshooting applications. By configuring custom logs, you can tailor the data collection to meet the specific needs of your organization or application, ensuring that relevant information is captured and available for analysis.

I will delve deeper into each component involved in end-to-end monitoring in the next blog post.