Introducing Auxiliary Logs: A New Addition to Azure Monitor Logs for Cost-Effective Telemetry Data Management

Microsoft announce a significant enhancement to your multi-tier strategy for optimal consumption and cost management: Auxiliary Logs.

What Are Auxiliary Logs?

Auxiliary Logs are designed specifically for verbose logs that don’t require immediate analysis but still need to be retained for future reference. This new plan offers an inexpensive solution for managing and consuming these logs without compromising on capabilities.

Specific Security Log Source Examples for Auxiliary Logs

Auxiliary Logs are ideal for verbose logs or logs that are generated in high volume but are not frequently queried or required for real-time analysis. Here are some specific types of security logs that may benefit from being stored in the Auxiliary Logs tier:

1. Firewall Logs: Detailed logs of all network traffic and security events captured by firewalls. These logs can be voluminous but are essential for historical analysis and troubleshooting.

2. Intrusion Detection System (IDS) Logs: Logs from IDS devices that monitor network or system activities for malicious activities or policy violations. While critical, these logs can generate large volumes of data.

3. VPN Logs: Logs that track VPN connections, including connection times, user activities, and bandwidth usage. Useful for compliance and security audits but typically high-volume and not frequently accessed.

4. Endpoint Detection and Response (EDR) Logs: Logs from EDR solutions that provide detailed information on endpoint activities, including file access, process execution, and user behavior. Important for in-depth investigations but often very detailed. (Yes there are reasons not to ingest “ALL” EDR logs” to the highest tier)

5. Web Server Access Logs: Logs that record all requests made to a web server. These logs can help in analyzing web traffic patterns and detecting anomalies but are generally high-volume.

Always validate the Log tier type against the needs of your analytic rules etc to ensure suitability!

Azure Monitor’s Multi-Tier Strategy

Azure Monitor Logs now supports three distinct plans:

1. Analytics Plan: For advanced data analysis and real-time monitoring.

2. Basic Plan: For essential logging needs at a lower cost.

3. Auxiliary Plan: The new addition for verbose and low-value logs, optimized for cost-efficiency.

This multi-tier strategy allows you to store all your logs in one place, retain different data types as needed, and benefit from a cost-effective pricing model.

Key Features of Auxiliary Logs

1. Cost Optimization

• Ingest low-value or verbose logs into the Auxiliary table to reduce costs.

2. Long-Term Retention

• Retain data for up to 12 years at a low cost, ensuring compliance and historical analysis needs are met.

3. Flexible Data Access

• Access the last 30 days of data with queries or search for older data using search jobs.

4. Advanced Data Aggregation (Preview)

• Use summary rules to aggregate data and ingest results into a table with an Analytics plan, perfect for dashboards, alerts, or complex analysis.

Key Limitations of Auxiliary Logs

A data collection rule that sends data to a table with an Auxiliary plan:
- Can only send data to a single table.
- Can't include a transformation

This means, at least for now, you can't use AMA to ingest eg. Firewall log and forward to a Auxiliary log table. This to be confirmed.

Pricing and Availability

During the initial public preview, billing for Auxiliary Logs (ingestion, long-term retention, query, and search job) is not yet enabled. The start date for billing will be announced on Azure Updates, with advanced notice provided to current users.

Auxiliary Logs are currently in public preview and may have some regional availability limitations, which are detailed in our documentation.

Pricing - Azure Monitor | Microsoft Azure

By integrating Auxiliary Logs into your Azure Monitor strategy, you can achieve unparalleled flexibility and cost efficiency. Dive into the future of telemetry data management with Azure Monitor’s comprehensive multi-tier approach.

Original Announcement

Public Preview: New Azure Monitor Auxiliary Logs Plan (microsoft.com)

If you looking to save money and optimize archive settings on mass, see my post on how to bulk configure your Archive Settings within a LAWS environment, enabling an easy setup for low-cost archive logs. -

#MicrosoftSecurity

#MicrosoftLearn

#CyberSecurity

#MicrosoftSecurityCopilot

#Microsoft

#MSPartnerUK

#msftadvocate