AWS Storage Blog

Achieving sub-10ms latency and 94% cost savings with Diskless Kafka using AutoMQ and Amazon FSx for NetApp ONTAP

As more organizations adopt Diskless Kafka—a cloud-based messaging queue that builds entirely on object storage like Amazon Simple Storage Service (Amazon S3)—they gain significant cost and operational advantages. But for latency-sensitive workloads, this architecture faces a key question: how to keep millisecond-level write latency while preserving the cost benefits of S3? This is the core […]

Building persistent memory for multi-agent AI systems with Amazon S3 Vectors

The most capable multi-agent AI systems share a common trait: they give agents the right context at the right time. When agents lack access to shared history, including what other agents discovered, what tasks are already complete, and what decisions were made in previous sessions, they might duplicate work, contradict each other, and burn through […]

Amazon S3 audit logging, Part 2: Centralized logging and analysis of S3 data events in AWS CloudTrail for security and compliance

This is Part 2 of our three-part series on Amazon S3 audit logging, focusing on identity-driven security investigations. In Part 1, we covered S3 server access logs for HTTP-level performance analysis and cost attribution. When a security incident occurs—an unauthorized download, a bulk deletion, or suspicious access from an unfamiliar location—the first question is always, […]

Amazon S3 audit logging, Part 1: Analyzing server access logs with Amazon Athena for performance insights

Organizations storing sensitive data must maintain complete visibility into how it’s accessed, by whom, and what changes occur over time. Regulatory frameworks demand detailed audit trails, security teams need rapid answers during investigations, and finance teams require granular cost attribution. Yet as data grows from terabytes to petabytes, the scale that makes centralized storage attractive […]

How to configure user storage quotas for the AWS Transfer Family on Amazon FSx for NetApp ONTAP

Managing storage efficiently in a multi-user environment is a critical yet often overlooked aspect of cloud infrastructure design. When you combine AWS Transfer Family with Amazon FSx for NetApp ONTAP, you unlock a robust, enterprise-grade file transfer solution. However, without effective quota controls, a single user or process can consume disproportionate storage, leading to performance […]

Data discovery: How to find out what’s on your Amazon FSx for NetApp ONTAP volumes

Enterprise storage administrators manage hundreds of terabytes, and sometimes petabytes, of file data spanning business units, applications, and users. As that storage grows, so does the challenge of understanding what is actually stored in it. Administrators are asked to make capacity decisions, identify archive candidates, track storage costs, and support compliance reviews — but with […]