AWS S3 monitoring with Topologix

Data & Storage

Two tiers of S3 telemetry per bucket. The storage tier — size broken down by storage class plus object count — is always on, free, and reported daily by AWS (it can lag 24–48 hours). The request tier — request rate, 4xx/5xx errors and rates, bytes in both directions, first-byte and total latency — appears only on buckets where the paid, opt-in request metrics are enabled.

Requirements

  • An IAM access key with s3:ListAllMyBuckets, s3:GetBucketLocation, cloudwatch:GetMetricData, and cloudwatch:ListMetrics — the managed AmazonS3ReadOnlyAccess + CloudWatchReadOnlyAccess pair covers it.
  • The integration is region-scoped: only buckets in the drawer’s Region are listed. Buckets elsewhere get another integration on the same key.
  • For the request tier: enable Request metrics per bucket (AWS console → bucket → Metrics) — an opt-in, paid CloudWatch feature that creates the EntireBucket filter the fetcher reads.

1 · Set up the integration

  1. Open the Integrations pageClick Integrations in the toolbar and pick AWS S3 from the catalogue.
  2. Create a read-only IAM userAWS Console → IAM → create a user with AmazonS3ReadOnlyAccess + CloudWatchReadOnlyAccess and generate an access key.
  3. Enter the credentialsAccess Key ID, Secret Access Key, and the Region whose buckets this integration should list.
  4. Create, then verifyClick Create, then reopen the integration and click Test — it lists buckets and reports the in-region count against the account total (“Connected — 3 bucket(s) in eu-west-1 (12 total in account)”), capping location lookups at 25 to stay fast.
The integration catalogue — AWS S3 found via search.
The integration catalogue — AWS S3 found via search.
The AWS S3 configuration drawer with example values. Every screen in this guide is the real product.
The AWS S3 configuration drawer with example values. Every screen in this guide is the real product.
Target on a private network? Route it through a proxy agent. Proxy agents run inside your network — behind NAT, on a management VLAN, in an air-gapped segment — and reach targets the Topologix server can't. The agent polls the server every 15 seconds for its assignments, runs the checks locally, and pushes results back every 5 seconds, so no inbound firewall rules are ever needed. Set a default proxy on the integration; every metric source can override it. If a proxy goes offline, its sources are marked stale within ~90 seconds and the last known values stay visible.

2 · Add it to a flow

  1. Select the storage nodeClick the node, open Metrics, then + Add and choose the AWS S3 integration.
  2. Pick the bucketsA multi-select list of the region’s buckets (with creation date) creates one source per ticked bucket; or type a bucket name directly.
  3. SavePick the Request metrics period (1 minute to 24 hours — storage metrics are reported daily by AWS regardless of this setting) and click Add Source.
Live canvas screenshots for this integration require a real AWS S3 environment, so this guide shows the configuration screens only — the flow steps above work exactly like every other integration.

Good to know

  • The fetcher first discovers which storage classes the bucket actually reports via ListMetrics, then reads BucketSizeBytes per class and NumberOfObjects (under AllStorageTypes) with a 48-hour lookback — the card shows the per-class breakdown and when the daily sample was taken.
  • The request tier probes AllRequests under FilterId: EntireBucket: no data means request metrics aren’t enabled, and those fields read as absent — never as a misleading zero. Error rates are computed from 4xx/5xx over AllRequests.

All integrations →

See it on your own infrastructure

Book a demo and we will walk you through Topologix with your systems in mind: on our cloud, or self-hosted in your own network.

Book a demoContact us