Log Buffer on AMA Server when Sentinel is Down

JC 40 Reputation points

We don’t plan to store any logs on the syslog server as we are planning to set it as bridge for log forwarding.

Noticed that it can only holds 1GB. Is this correct or is this really 10GB as per this thread -

  2. Siva shunmugam Nadessin 10,895 Reputation points • Microsoft External Staff • Moderator

    Hello Janel Cabile,

    Thank you for reaching out to the Microsoft Q&A forum. 

    When investigated it looks like you’re referring to the “offline” buffer that the Azure Monitor Agent (AMA) uses on your Linux syslog-forwarder when Sentinel can’t ingest data. Here’s what we know from the public guidance and community threads:

    1. AMA itself doesn’t expose a knob to directly change the on-disk log queue size.
    2. By default, AMA will spool data to the local VM’s disk (in a folder under /var/lib/azuremonitoragent/ or similar) and this queue is observed by many customers to top out around ~1 GB.
    3. There are some community reports of seeing up to ~10 GB, but those usually end up being the size of the underlying VM OS disk or an artifact of how the syslog engine’s own queue (rsyslog or syslog-ng) is configured, not a built-in “10 GB limit” in AMA.
    4. If you truly need a bigger buffer, your best approach is to increase the VM’s disk size or configure your syslog daemon’s queue parameters (for example, SystemMaxUse/QueueDirectory in rsyslog) so that it buffers more before handing logs off to AMA.

    What you can try next:

    • Check which syslog engine you’re using (rsyslog vs. syslog-ng) and inspect its own queue settings.

    • Verify the actual disk usage under the AMA data folder (e.g. du -sh /var/lib/azuremonitoragent/*).

    • If you bump the VM’s OS disk from, say, 30 GB to 100 GB, you’ll proportionally get more “offline” buffering headroom.

    Follow-up questions to narrow this down:

    1. Which Linux distro & version is your forwarder VM running?
    2. Are you using rsyslog or syslog-ng (or something else) to capture logs?
    3. How did you measure “1 GB” vs. “10 GB”? (e.g. did you inspect a folder size, check a syslog param, watch a metric?)
    4. What AMA agent version are you on?
    5. Can you confirm the path where you’re seeing this buffer filling up?

    Reference docs:

    Hope that helps clarify why you’re seeing ~1 GB by default and points you at the places to tweak if you need more!

  3. JC 40 Reputation points

    Hi,

    See below response:

    1. Which Linux distro & version is your forwarder VM running? RHEL
    2. Are you using rsyslog or syslog-ng (or something else) to capture logs? rsyslog
    3. How did you measure “1 GB” vs. “10 GB”? (e.g. did you inspect a folder size, check a syslog param, watch a metric?)

    we are just referring to the config on this file 10-azuremonitoragent-omfwd.conf

Sign in to comment

Answer accepted by question author

Siva shunmugam Nadessin 10,895 Reputation points • Microsoft External Staff • Moderator

Hello Janel Cabile

The 10‑azuremonitoragent‑omfwd.conf file controls the rsyslog forwarding queue, not the Azure Monitor Agent’s own internal spool. While rsyslog can be configured to buffer more data on disk, AMA itself maintains a separate, internal offline buffer (under /var/lib/azuremonitoragent) whose size is not configurable.

Values such as “10 GB” referenced in rsyslog configs do not represent a supported or guaranteed AMA retention limit.

 References:

  Syslog & CEF ingestion using AMA https://learn.microsoft.com/azure/sentinel/connect-cef-syslog-ama

  AMA Linux troubleshooting (rsyslog focus) Troubleshoot syslog issues with Azure Monitor Agent on Linux - Azure Monitor | Microsoft Learn

  CEF & Syslog AMA troubleshooting Troubleshoot CEF and Syslog via AMA connectors in Microsoft Sentinel | Microsoft Learn

  1. JC 40 Reputation points

    I see, so the only thing that we can check is the config file in rsyslog such as SystemMaxUse/QueueDirectory

  2. JC 40 Reputation points

    If 10‑azuremonitoragent‑omfwd.conf file controls the rsyslog forwarding queue, then will it have the same purpose as SystemMaxUse?

  3. JC 40 Reputation points

    If 10‑azuremonitoragent‑omfwd.conf file controls the rsyslog forwarding queue, then will it have the same purpose as SystemMaxUse?

  4. Siva shunmugam Nadessin 10,895 Reputation points • Microsoft External Staff • Moderator

    Hello Janel Cabile,

    The 10‑azuremonitoragent‑omfwd.conf file and SystemMaxUse serve similar protective purposes but apply to different components. SystemMaxUse limits disk usage for systemd‑journald (log storage), whereas 10‑azuremonitoragent‑omfwd.conf controls rsyslog’s forwarding queue to Azure Monitor Agent. They do not govern the same buffer and changing one does not affect the other.

    So,

    SystemMaxUse controls how long logs are kept; 10‑azuremonitoragent‑omfwd.conf controls how logs are buffered before forwarding.

  5. Siva shunmugam Nadessin 10,895 Reputation points • Microsoft External Staff • Moderator

    Hello Janel Cabile,

    Kindly 👁 User's image
    this can be beneficial to other community members.

Sign in to comment

1 additional answer

  1. AI answer

    The provided information describes how Syslog/CEF collection with the Azure Monitor Agent (AMA) works, but does not specify any numeric buffer size (1 GB, 10 GB, or otherwise) for log storage or in‑memory buffering on the AMA/syslog forwarder when Microsoft Sentinel or the Log Analytics workspace is unavailable. It only states that:

    • Syslog/CEF sources send logs to a Linux log forwarder (rsyslog/syslog‑ng) on port 514 (or another configured port).
    • The syslog daemon forwards logs to AMA (TCP 28330 or Unix domain socket, depending on AMA version).
    • AMA parses and sends logs to the Sentinel/Log Analytics workspace according to Data Collection Rules (DCRs).

    No limits or guarantees about how much data can be buffered or queued locally on the AMA server are documented in the supplied material. Therefore, the specific 1 GB vs. 10 GB value cannot be confirmed or corrected from this context.

    References:

    AI-generated content may be incorrect. Read our transparency notes for more information.
Sign in to answer

Your answer