OpenTelemetry (OTLP) monitoring with Topologix

Monitoring & Observability

Topologix as the OTLP backend: apps and collectors push OTLP/HTTP telemetry — traces, metrics and logs — to a Topologix proxy on their network, which relays it to the server; no vendor in between. Spans become per-service RED metrics (rate, errors, p95 duration), live dependency edges, and trace→topology auto-discovery (17 services / 22 edges verified against the OpenTelemetry Astronomy Shop demo); pushed metrics chart as their own sources; logs drive per-service alarms with the matching lines one click away. Section 4 below has copy-paste setup for every major technology.

Requirements

• A Topologix proxy the apps can reach — OTLP arrives only through a proxy’s relay: the proxy opens an OTLP/HTTP listener (:4318, the OTLP default port) and forwards to the server. The server itself no longer receives OTLP: its old /api/otel/v1/* and :3001/v1/* paths answer 410 Gone. A self-hosted install can run the proxy on the server host.
• An OTLP/HTTP exporter (protobuf or JSON, gzip accepted) pointed at that proxy — its base endpoint is http://<proxy>:4318 and the standard /v1/traces, /v1/metrics and /v1/logs paths are appended automatically.
• The ingest token, minted server-side when the integration is first saved — exporters authenticate with Authorization: Bearer <token> or the X-Otel-Token header; one token covers all three signals.
• Nothing to poll: this is a push receiver, so no outbound reach into your infrastructure and no credentials to hand over — the proxy only listens for the integration’s token, buffers pushes and relays them.

1 · Set up the integration

1. Open the Integrations pageClick Integrations in the toolbar and pick OpenTelemetry (OTLP) from the catalogue.
2. Save once to mint the ingest tokenThe drawer then shows the token with a Copy button, plus a Regenerate token on save checkbox — regenerating disconnects every currently-pushing collector, so it’s an explicit opt-in.
3. Pick the relay proxyChoose the proxy your apps can reach under Relay proxy — required while the integration is enabled (the save is refused without one). The proxy opens its OTLP listener for this integration’s token within ~15 s of saving. An integration saved before this rule keeps working as stored but its card and Test say “No relay proxy — OTLP only arrives through a proxy; pick one in the integration settings”.
4. Point your exporters at the endpoints panelThe drawer prints the relay proxy’s three push URLs — http://<proxy address>:4318/v1/traces, /v1/metrics, /v1/logs (the address the proxy reported, the port from its settings). Optional: Feed stale after (seconds) — default 90, floor 30 — after which a silent feed reads as stale.
5. TestThe button reports what the receiver has actually seen: “Receiving — last push 4s ago · 17 service(s) · 132840 spans total”, or “Receiver ready — no spans received yet” with exporter pointers.

2 · Add it to a flow

1. Select the nodeClick the node, open Metrics, then + Add and choose the OTLP integration.
2. Pick a modeFour modes across the three signals: Spans — all services (the default — the receiver-wide fleet watch), Spans — per service (one scalar card for one pushing service), Pushed metrics (chart metrics your services push), and Logs — all services (per-service log-match alarms).
3. SaveSet the interval and click Add Source. Breaching services carry exemplar traces you can open in the waterfall viewer directly from the card.

3 · Source modes & topology discovery

All-services alarm

Watches every service pushing to the integration — per-service p95, error rate and request rate computed from the spans themselves, dependency edges, and stale warnings when a service (or the whole feed) goes quiet. A service whose p95 crosses the threshold (default 1000 ms) raises the latency alarm with exemplar traces attached; breach state auto-clears after N healthy checks.

• One all-services source per OTLP integration — its poll drains the receive window, so a second one would see empty windows. Per-service sources are unaffected: they read the accumulated view.
• Absence of data is never green: the fetcher runs even when nothing pushes, so a dead exporter becomes a stale warning — and an integration that has never received a span reads stale immediately.

Per-service metric

Pick a service from the received list and tick any of the 4 metrics — Requests / tick, Error Rate %, Avg Latency (ms), P95 Latency (ms) — for one combined card, with per-metric alarm knobs: P95 alarm (ms), Error alarm (%), and a Volume floor that fires when traffic drops below it.

• Rates are counter deltas per tick from true totals, never from capped samples — a reservoir at its cap can’t flatten the request rate.
• A vanished service is a distinct verdict, not a green 0 — the card says whether the whole feed is silent or just that service stopped pushing (names are case-sensitive).

Pushed metrics

Chart metrics your services push over OTLP — no scrape endpoints, no Prometheus in between. Pick a service, then check any of the metrics the receiver has seen from it: gauges show their latest value, counters their per-tick delta (temporality handled automatically), histograms their per-tick average. Each picked metric can carry its own alarm threshold, and the whole selection renders as one tile-grid card.

• Metric identity is the name plus its datapoint attributes, flattened into keys like http.server.requests{http.route=/checkout} — the same mental model as Prometheus series.
• Cardinality is capped per service with visible drop counters — if an SDK explodes attributes, trim them with collector views rather than raising caps.
• The all-services span card also surfaces each service’s top pushed gauges in a collapsible Pushed metrics section.
• ExponentialHistogram and Summary points are accepted and dropped (counted) — export plain histograms for values you want charted.

Logs — all services

Watches every service pushing OTLP logs — per-service counts of matching log lines per check. The alarm rule is the one the Graylog All Services source uses: Baseline (the default — each service against its usual count for this weekday and hour, provisional for the first weeks) or Count threshold (any service above N matching lines; 0 = any match). Choose the minimum severity that counts (errors only by default). Breaching rows carry a View logs button that opens the actual lines, no external log system involved.

• Log error baselines (server 1.293.1): counts are exact — the receiver counts every pushed record, 0 network calls — a service that pushed nothing in the window reads a measured 0, and the breach, new-type warning, banner, Alerts page and Catalogue Logs section work as described in the Graylog guide.
• Error types come from the record attribute exception.type (read before the 8-attribute cap, so it is never lost); error-level records without it are fingerprinted from the kept lines and marked approx. Up to 50 types per service, the rest (other). With a minimum level of warn or info, the count follows that level but only the error-level records are typed.
• Upgrading: an existing logs source with no match threshold (or 0) switched to Baseline once at boot; a source with an explicit threshold above 0 stays on Count threshold.
• One logs alarm per OTLP integration — its poll drains the log window. It coexists with the span alarm: the two drain independent stores.
• Log lines pushed with trace context also appear in the trace viewer’s Show logs panel, correlated by trace id from the local store — the external Graylog bridge remains as a fallback.
• The retained lines are bounded per service (newest kept); severities map onto the familiar ERR/WARN/INFO coloring.

Trace → topology discovery

“Discover from Traces” builds the full service architecture from the rolling span accumulator (roughly the last hour of pushes) — every pushing service becomes a node, every cross-service parent→child span becomes an edge, and the generated flow arrives named “Architecture from OTLP ingest”, ready for monitoring setup. Tick the Kubernetes clusters and vCenters to place the services on: cluster nodes link to the services whose pods carry the name, VM and ESXi host nodes come from the infrastructure map. OTLP has no historical window and no proxy hop (the store lives on the server), so those controls stay hidden for it.

4 · Instrument your services

Step by step — any technology

OpenTelemetry agents for every language speak the same wire protocol and read the same standard OTEL_* environment variables, so instrumenting a service is never really about the language. It comes down to answering three questions once — where do the traces go, how does the receiver know they’re allowed in, and which signals do we send — and the same answers work for Java, Node.js, Python, or anything else. Only the attach step differs per technology.

1. Get your ingest tokenIn Topologix: Integrations → OpenTelemetry (OTLP) → save once → reopen it. The drawer now shows a token like otel_bf05… with a Copy button — the only credential involved.
2. Pick the endpoint: your relay proxyAlways a Topologix proxy the app can reach: http://<proxy-host>:4318 — the integration’s Endpoints panel prints the exact address. The server does not take OTLP itself (its old /api/otel and :3001/v1 paths answer 410 Gone). OTEL_EXPORTER_OTLP_ENDPOINT is a base URL by spec: the SDK appends /v1/traces itself, so don’t write it.
3. Set the environment block on the serviceHowever you normally set env vars for that service — systemd unit, Dockerfile, compose file, Kubernetes manifest. The block below is the complete set. OTEL_SERVICE_NAME matters more than it looks: it’s how Topologix identifies the service, every process pushing the same name lands on one card, and forgetting it piles everything onto a card called “unknown”.
4. Attach the agent for your technologyThe recipes below — one per language or platform. Steps 1–3 are identical for all of them; only this step changes.
5. Confirm it workedReopen the OTLP integration and click Test — it reports what the receiver actually saw, e.g. “Receiving — last push 4s ago · 17 service(s)”. Your service name then shows up in the All-services alarm card and the Per-service dropdown.

• Why X-Otel-Token? The receiver accepts Authorization: Bearer <token> too, but OTEL_EXPORTER_OTLP_HEADERS is a comma-separated key=value list and Bearer <token> contains a space that several SDKs require URL-encoded (Bearer%20…) — a constant source of silent 401s. X-Otel-Token=… has no space, so it pastes correctly into every SDK. Same security, fewer footguns.
• What lands where? All three signals ride the same endpoint and token. Spans become the RED metrics, dependency edges, and traces; pushed metrics feed the Pushed metrics source mode (and gauge chips on the all-services card); logs feed the Logs — all services alarm and the trace viewer’s correlated-logs panel. Don’t need a signal? Set its exporter to none — optional, not required.
• Why force the protocol? Some SDKs default to OTLP/gRPC, which Topologix doesn’t accept — http/protobuf pins the transport that works.

The canonical environment block — works for every SDK and agent below

OTEL_SERVICE_NAME=checkout-api
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-proxy.example.internal:4318
OTEL_EXPORTER_OTLP_HEADERS=X-Otel-Token=otel_<your-token>
OTEL_TRACES_EXPORTER=otlp
OTEL_METRICS_EXPORTER=otlp
OTEL_LOGS_EXPORTER=otlp

Java

The OpenTelemetry Java agent instruments Spring, JDBC, Kafka, gRPC, and ~100 other libraries with zero code changes — download one jar and attach it. The -javaagent: flag loads it before your application code runs; the agent reads the OTEL_* variables from step 3 and starts pushing spans. No source changes, no rebuild.

• App servers (Tomcat, WildFly…) take the same flag via JAVA_TOOL_OPTIONS=-javaagent:/path/opentelemetry-javaagent.jar.

curl -LO https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar

# attach to any JVM service, env block from above applied
java -javaagent:opentelemetry-javaagent.jar -jar checkout-api.jar

Node.js

The auto-instrumentations meta-package covers http, Express, Fastify, pg, mysql, redis, and more — no code changes, one --require flag.

• Where you can’t edit the start command, set NODE_OPTIONS=--require @opentelemetry/auto-instrumentations-node/register instead.

npm install @opentelemetry/api @opentelemetry/auto-instrumentations-node

# env block from above applied
node --require @opentelemetry/auto-instrumentations-node/register server.js

Python

The distro package plus opentelemetry-bootstrap detects your installed frameworks (Django, Flask, FastAPI, requests, psycopg…) and instruments them at launch.

• The protocol line matters here: the Python distro defaults to OTLP/gRPC, which Topologix does not accept.

pip install opentelemetry-distro opentelemetry-exporter-otlp
opentelemetry-bootstrap -a install

# env block from above applied — keep OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
opentelemetry-instrument python app.py

.NET

OpenTelemetry .NET automatic instrumentation attaches through the CLR profiler — ASP.NET Core, HttpClient, SQL Client, and more, no code changes.

• On Windows, use the OpenTelemetry.DotNet.Auto.psm1 PowerShell module: Install-OpenTelemetryCore then Register-OpenTelemetryForCurrentSession.

# one-time install (Linux/macOS)
curl -sSfL https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet-instrumentation/main/otel-dotnet-auto-install.sh -o otel-dotnet-auto-install.sh
sh otel-dotnet-auto-install.sh
. $HOME/.otel-dotnet-auto/instrument.sh

# env block from above applied
dotnet CheckoutApi.dll

Go

Go has no attach-at-runtime agent — wire the OTLP/HTTP exporter in a few lines at startup. The standard OTEL_EXPORTER_OTLP_* environment variables are honored too, so the env block alone configures a default exporter.

• Set the service name via resource attributes or simply OTEL_SERVICE_NAME; instrument HTTP handlers with otelhttp.NewHandler.

import (
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"
    sdktrace "go.opentelemetry.io/otel/sdk/trace"
)

exp, err := otlptracehttp.New(ctx,
    otlptracehttp.WithEndpointURL("http://otel-proxy.example.internal:4318/v1/traces"),
    otlptracehttp.WithHeaders(map[string]string{"X-Otel-Token": "otel_<your-token>"}),
)
tp := sdktrace.NewTracerProvider(sdktrace.WithBatcher(exp))
otel.SetTracerProvider(tp)

Ruby

Ruby’s OTLP exporter speaks OTLP/HTTP by default — install three gems and enable all instrumentations at boot (Rails: an initializer).

gem install opentelemetry-sdk opentelemetry-exporter-otlp opentelemetry-instrumentation-all

# config/initializers/opentelemetry.rb — env block from above applied
require 'opentelemetry/sdk'
require 'opentelemetry/instrumentation/all'
OpenTelemetry::SDK.configure { |c| c.use_all }

PHP

The opentelemetry PECL extension hooks framework internals; composer packages provide the SDK and per-framework auto-instrumentation (Laravel, Symfony, WordPress…).

• Add the framework package for zero-code spans, e.g. open-telemetry/opentelemetry-auto-laravel.

pecl install opentelemetry            # add extension=opentelemetry to php.ini
composer require open-telemetry/sdk open-telemetry/exporter-otlp

# env block from above applied, plus:
OTEL_PHP_AUTOLOAD_ENABLED=true

OpenTelemetry Collector

For a fleet, point every app at one collector (its OTLP receiver, gRPC or HTTP — apps don’t care) and let the collector push to Topologix. This also converts gRPC-only exporters to the HTTP the receiver accepts.

• Alternatively set the exact URL with traces_endpoint: http://<proxy-host>:4318/v1/traces instead of endpoint.
• Collector config merging replaces arrays wholesale — when adding the exporter to an existing pipeline, restate the full exporters: list, existing entries included.
• The endpoint is always a Topologix relay proxy (http://<proxy-host>:4318) — the integration’s Endpoints panel prints it; the server answers 410 to direct pushes.

collector config — add to your existing pipelines

exporters:
  otlphttp/topologix:
    # base endpoint (the relay proxy) — the exporter appends /v1/traces|metrics|logs itself
    endpoint: http://otel-proxy.example.internal:4318
    compression: gzip
    headers:
      authorization: "Bearer otel_<your-token>"

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlphttp/topologix]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlphttp/topologix]
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlphttp/topologix]

Docker Compose

Containerized services need only the environment block — the same pattern the Astronomy Shop demo uses to push into Topologix.

• The relay proxy on the Docker host itself? OTEL_EXPORTER_OTLP_ENDPOINT: http://host.docker.internal:4318 reaches it from inside containers.

services:
  checkout-api:
    image: yourorg/checkout-api:latest
    environment:
      OTEL_SERVICE_NAME: checkout-api
      OTEL_EXPORTER_OTLP_PROTOCOL: http/protobuf
      OTEL_EXPORTER_OTLP_ENDPOINT: http://otel-proxy.example.internal:4318
      OTEL_EXPORTER_OTLP_HEADERS: X-Otel-Token=otel_<your-token>
      OTEL_TRACES_EXPORTER: otlp
      OTEL_METRICS_EXPORTER: otlp
      OTEL_LOGS_EXPORTER: otlp

Kubernetes

With the OpenTelemetry Operator installed, one Instrumentation resource defines where telemetry goes, and a pod annotation injects the right language agent automatically.

• No Operator? Run a collector as a DaemonSet or sidecar and use the Collector recipe above.

apiVersion: opentelemetry.io/v1beta1
kind: Instrumentation
metadata:
  name: topologix
spec:
  exporter:
    endpoint: http://otel-proxy.example.internal:4318
  env:
    - name: OTEL_EXPORTER_OTLP_PROTOCOL
      value: http/protobuf
    - name: OTEL_EXPORTER_OTLP_HEADERS
      value: X-Otel-Token=otel_<your-token>

then annotate each workload with its language

spec:
  template:
    metadata:
      annotations:
        instrumentation.opentelemetry.io/inject-java: "true"
        # or inject-nodejs / inject-python / inject-dotnet / inject-go

Capture SQL on database spans

Every OpenTelemetry database instrumentation can put the statement it ran onto the span — db.statement in the old semantic conventions, db.query.text in the new ones. Topologix keeps it, but never as the driver sent it: the statement is normalised first — every literal, bind value and IN (…) list collapses to ? — and hashed into a stable fingerprint. That happens on the LAN relay proxy when you push through one, and at ingest otherwise, so raw values never reach the span store, the trace viewer, the AI analysis, or the bot. What you get in return: the SQL printed under each database span in the waterfall, a grouped Statements block per trace (five identical fingerprints in one request is an N+1, visible at a glance), a SQL tab per service ranking its statements by total time, and — when the same database is also a Topologix integration — a one-click lookup of that fingerprint in pg_stat_statements, SQL Server’s Query Store or the HANA plan cache.

• Java — nothing to do. The agent’s JDBC sanitizer (otel.instrumentation.jdbc.statement-sanitizer.enabled, env OTEL_INSTRUMENTATION_JDBC_STATEMENT_SANITIZER_ENABLED) is on by default, so statements already arrive with their literals replaced. Leave it on: Topologix normalises again and the operation is idempotent, so the fingerprint is the same either way.
• Node.js — nothing to do. @opentelemetry/instrumentation-pg, -mysql2 and -tedious (all pulled in by the auto-instrumentations meta-package from the Node.js recipe above) emit db.statement out of the box.
• Python — the psycopg and sqlalchemy instrumentations record the statement; both also accept enable_commenter=True, which appends a sqlcommenter comment to the SQL the database actually executes (see the note below before switching it on).
• .NET — opt-in: the SQL Client instrumentation records command text only when asked, AddSqlClientInstrumentation(o => o.SetDbStatementForText = true). Stored procedures are captured by name regardless of the flag.
• Go — no attach-at-runtime agent, so wrap the driver: otelsql emits the statement on every span, with otelsql.WithSQLCommenter(true) as the optional sqlcommenter.
• What is not kept: db.user and db.query.summary are dropped at the door, and the stored statement is capped at 1024 characters (the fingerprint is computed on the full text first, so truncation never changes it). One honest gap: exception.message can quote the failing statement with its literals — that text is covered by the redaction engine, not by the SQL normaliser.
• sqlcommenter: Topologix parses a traceparent out of the statement comment and strips it before storing, so it never reaches the store and never changes the fingerprint — but it is not how trace↔statement correlation works here. Correlation is by fingerprint, because pg_stat_statements discards comments and keeps only the first text it saw per queryid. On SQL Server and SAP HANA a comment that differs per request is actively harmful: each variant becomes its own plan-cache entry. Prefer static tags there (application, controller, route) or none at all; on HANA pass the trace id in SESSIONVARIABLE:APPLICATIONSOURCE instead of a comment.

Java — already sanitising; this is only the explicit form

OTEL_INSTRUMENTATION_JDBC_STATEMENT_SANITIZER_ENABLED=true

.NET — the command text is opt-in

builder.Services.AddOpenTelemetry().WithTracing(t => t
    .AddSqlClientInstrumentation(o => o.SetDbStatementForText = true));

Python and Go — instrument the driver; the commenter is optional

# Python
from opentelemetry.instrumentation.psycopg import PsycopgInstrumentor
from opentelemetry.instrumentation.sqlalchemy import SQLAlchemyInstrumentor

PsycopgInstrumentor().instrument(enable_commenter=True)
SQLAlchemyInstrumentor().instrument(engine=engine, enable_commenter=True)

// Go — github.com/XSAM/otelsql
db, err := otelsql.Open("pgx", dsn,
    otelsql.WithAttributes(semconv.DBSystemPostgreSQL),
    otelsql.WithSQLCommenter(true))

What the exporter sends, and what Topologix stores

# on the wire
db.system       postgresql
db.name         shop
db.statement    SELECT * FROM orders WHERE customer_id = 4242
                AND email = 'ann@example.com' /*traceparent='00-4bf9…-00f0…-01'*/

# in the span store, the trace viewer, the AI prompt and the bot
db.system       postgresql
db.name         shop
db.statement    SELECT * FROM orders WHERE customer_id = ? AND email = ?
db.fingerprint  <16 hex chars — FNV-1a 64 of the normalised text>
db.operation    SELECT
db.table        orders

Dependencies and blast radius

Every trace backend you connect also feeds one persisted call graph: which service calls which, how often, with how many errors. The Topology tab draws a service's callers and callees at +1 or +2 hops, the Catalogue shows “Calls →” / “Called by ←” on each service, and Blast radius… (on services, externals, VMs, hosts, datastores, devices, databases and business processes) lists everything a change or a maintenance there might touch.

What is collected, and from where

A background dependency sweep reads the service-to-service graph that Discover from Traces already computes, from every enabled Zipkin, OpenTelemetry (OTLP ingest), Dynatrace, Elastic APM and SigNoz integration — through the integration's proxy agent when it has one — and stores one edge per caller → callee pair per owner: calls, errors, a p95 only when a backend measured one, and which integrations reported it. A count nobody measured stays empty, never 0 (Dynatrace's Smartscape edges carry no per-edge counts, so they read “—”). A call to something that is not an instrumented service becomes an external node — an API host, an unmatched database, a queue — unless it matches a monitored database instance, in which case it folds into that instance's node. Externals never enter a runs-on chain. Up to 5,000 edges are kept per owner (the busiest).

Which instance a database call folds into

A database span is matched by its host first (one monitored instance of that engine on the host), then by its db.name — names compare ignoring case and punctuation (crocobetCasinoCommerce, crocobetcasinocommerce and crocobet_casino_commerce are one name; a suffix such as _prod is never stripped). A database held by a primary and its replicas is not ambiguous: the call folds into the primary, both by name and when several instances share the host — a replica is one a primary's replication block names, or a monitored source whose own sample reports it replicates (linked to its primary by that upstream, else by sharing its integration and engine). The external node an earlier sweep drew for that name is then marked stale, its edges with it, as an alias would. Two unrelated instances holding the same name, or a span of another engine, are never guessed: the call stays an external node, and the Catalogue's “database known only by name” gap names the other instance (ambiguous: also on …).

Cadence

Admin → System → Background workers: Dependency sweep interval (default 900 s — 15 min), Dependency lookback (default 3600 s; OTLP ingest ignores it and uses its own in-memory window) and Dependency stale window (default 7 days). The same block shows the sweep's last run — when it started, how long it took, how many integrations it read, each one that failed by name with its error, the edges stored / written / stale — when the next run is due, and a Run now button. The Topology's Calls button starts a sweep too; its tooltip says when the last one ran and how many of your trace sources failed in it. An integration whose sealed credentials cannot be opened is skipped (its stored edges are kept), never read without its secret.

Aliases — when a trace name is really something else

Backends name things their own way: a renamed deployment, an ingress host, a database seen only from its callers. Two ways to say what a name really is — both need edit rights on the integrations' owner (an admin, or the owner); everyone else sees the aliases read-only.

• Catalogue → a service group → Link a trace name…: pick a name from the call graph that is really this service. The group then lists it under Also known as, with × to remove it.
• Topology → an external node → This is actually…: pick the monitored service or database instance (engine and host shown) it really is.
• Aliases are applied first, on every sweep, to every edge — not only new ones. After you save one, a sweep starts; its edges fold into the target and the old edges and the external node are marked stale at once (never deleted). Remove the alias and the next sweep brings the original edges back.

Blast radius — both directions

For a service (or an external or a business process) the blast radius goes both ways across the call graph at 1 or 2 hops: Depends on me (its callers — what breaks if it changes) and I depend on (its callees — what it needs), then down to the pods, K8s nodes, VMs, hosts, datastores and devices they run on, and the business processes any of them belongs to. For a VM, host, datastore, device, database or schema the default is callers only: the services running there and whoever calls them — what a maintenance window touches. Every item carries its path and the reason it is listed, and a confidence: 100 for a call seen by two or more backends, 90 for one, 50 for a stale edge (stale edges are left out unless you tick include stale). Results are capped at 500 services and 500 hardware items, and say so when they are. Export CSV and Open as business process are on the panel.

Stale rules

Nothing here is deleted automatically. A call edge no backend has reported for the stale window is flagged stale — drawn dimmed, lower confidence, kept with its last numbers — and comes back to life the moment a trace shows it again. Removing an integration drops its share of each edge; an edge only it reported stays, as history. An external that no live edge points at any more (an alias folded it into a service, or all its edges aged out) is flagged stale the same way.

Where OTLP- and Dynatrace-only services run

A service seen only through OpenTelemetry or Dynatrace still gets its own node, and its runs-on placement: for OTLP, from the host.ip, host.name, k8s.pod.name and k8s.node.name resource attributes your SDK or collector sends (only those four are kept); for Dynatrace, from the hosts its process groups run on. Each is matched to your pods, K8s nodes and VMs by IP first, then by name, by the same background probe that places Zipkin, Prometheus, Graylog, Elastic APM and SigNoz services (Probe batch size / Probe interval). A placement that has not changed is not rewritten.

The Catalogue — every service in one place

Infrastructure → Catalogue binds everything known about a service together: the integrations that report it (metrics, logs, traces), where it runs, who it calls and who calls it, the databases it uses, its owner and its health. The trace backends are where most services first appear, so this guide carries it; the same page lists every service your other integrations report too.

The list and the service page

The list shows one card per service — names are grouped case-insensitively, so “Checkout” and “checkout” are one card — with its data classes, a health dot, the tier chip and owner, whether it is placed on infrastructure and whether it has live calls. Group by service or by integration; a card loads its detail only when you expand it. The service name (or Open) opens the service page, which has its own link: ?view=catalogue&service=svc:<name> (Copy link in its header). Its sections:

• Identity — every source per integration and data class, with gap chips (“no logs”, “no traces”) and the trace names aliased into it.
• Ownership and Health — below; health with three sparklines (average latency, p95 latency, errors) from the learned baselines, and inbound requests from trace edges (“trace edges, last sweep”). A series nobody measured says “not measured”, never 0.
• Interfaces — the routes it is called on, who calls each one, and what it calls (below).
• Runs on, Calls / Called by, Databases used (the instances it calls, with their catalogued databases), Business processes it belongs to (those you can open), Findings on its VMs, hosts and datastores, Recent vCenter events (last 7 days), Cost share (loaded when opened).
• Change timeline and What changed this week — below.
• The header: favourite star, Blast radius…, Open in Topology, Export ▾. Open in Catalogue reaches the same page from Hypervisors VM rows, the Databases page, the Topology popover, the Business Process editor and blast-radius rows.

Interfaces — what it is called on and what it calls

The service page's Interfaces section is built from traces, not configuration: every inbound route (method + templated path — /wallet/3f2504e0-…/withdraw becomes /wallet/{uuid}/withdraw; also {hex}, {id}, {token}; a path the framework already templated, like /api/{session}/balance, is kept) with its calls, error share, p95, status-code mix and callers (the service whose span called it; a request with no parent span is shown as external ingress), and every outbound peer — another service, a database (its operations keyed on the normalised statement's fingerprint, the same key the database's own statistics use), an HTTP host or a queue — with its top operations. Both instrumentation dialects are read: OpenTelemetry semantic conventions and Spring Micrometer's method / uri / status / outcome / exception tags.

• Sources. OTLP pushes count every span as it arrives. Zipkin is sampled: every 5 minutes the next 25 services (round-robin; Interface inventory: Zipkin services per pass) each get up to 500 traces looking back to their previous visit (5–60 min), so the section says “seen in N traces since …”. A Zipkin reached through a proxy needs proxy ≥ 1.208.0 — with an older one the section says to update it. Dynatrace, Elastic APM and SigNoz do not feed the inventory yet.
• Stored once per flush (Interface inventory: flush interval, default 300 s) in one database transaction; an hourly history is kept 7 days (hourly buckets, on by default). Raw path samples (3 per route) go through your organisation's redaction rules before they are stored; a client's own IP address is never read.
• Health probes — routes matching /health, /healthz, /livez, /readyz, /liveness, /readiness… (also under /actuator/) — are folded into one line and never enter a change contract.
• Not seen recently. A route unseen for 7 days moves to this list (with its last-seen time, calls and callers) and is left out of change contracts; it is deleted 30 days after it was last seen (7 on the built-in SQLite store).
• Bounds. Per service 200 routes and 200 outbound operations (the least recently seen is dropped and counted), 20 callers per route (the rest are counted as “+N”). p95 comes from a sample of 64 durations per route (±2.7 percentage points of rank at 1σ; 32 for outbound, ±3.9) — good for a contract, not an alarm threshold. A value nobody measured shows “—”, never 0.
• The same inventory drives the change page's Contracts card (the routes each caller in a blast radius uses) — see Change management. API: GET /api/catalog/service/{key}/interfaces?probes=1&stale=1, GET /api/catalog/contract?from=svc:a&to=svc:b (topologix/interface-contract/1).

Ownership

Owner, team, tier (critical / important / standard), environment, description, a runbook link (http or https only) and up to 20 tags. Everyone who sees the service reads it; only the integrations' owner or an admin can change it (the same rule as trace-name aliases) — Edit ownership on the card or the page, and Clear ownership in the editor removes the record. Every change is audited (service_meta.set / service_meta.delete) and shows on the service's change timeline. The tier is shown and sorted on — critical services first in a blast radius, “Affects N critical services” on a Hypervisors finding — but it does not change any alert channel.

Health — what the dot means

A background health fold (Admin → System → Background workers → Service health interval, default 300 s) folds what is already measured into one status per service, with up to 8 reasons. The worst input wins; nothing new is polled. Stale placements and stale call edges never change a status, and a snoozed finding is no breach. A status change is recorded once on the timeline, with a 10-minute flap guard.

• Where it shows: the card dot (reasons in its tooltip), the Unhealthy first sort and the health facet, the service page header (“Error · since 15m ago”) with every reason, ⌘K hits, and a thin ring on service and database-instance cards in the Topology (reasons in the card popover; the sidebar has a Health legend).
• Health is display only in this version: no alert channel reads it.

⌘K search, facets and saved views

⌘K (Ctrl+K) anywhere opens the command palette: services, database instances, databases and schemas, VMs, ESXi hosts, IP addresses (a VM's guest IPs, a host's management address, K8s node IPs), externals and integrations — up to 50 hits, exact names first, then prefixes, word starts, substrings and typo-tolerant matches. Enter opens the hit; Tab lists its actions (Open, Blast radius of…, Show in Topology, Copy link, Favourite). With an empty query it shows your favourites, team favourites and recent service pages. The index is built per account on demand and refreshed after each catalogue, infrastructure or dependency sweep (at most 1 min old); you only ever find what your integrations can see.

• Facets over the list: kind (service / instance / has databases / has schemas), engine, data class, environment, tier, health and flags (no placement, no owner, stale, no calls). Values inside a facet are OR-ed, facets are AND-ed, and each count is taken over what the other facets keep. The filter lives in the URL (?f=, &sort=, &group=), so a filtered list can be linked.
• Views ▾ saves the search, facets, grouping and order under a name (up to 50 per user), applies, renames and deletes them. Ticking shared lists a view under Team views for everyone in your organisation — read-only for members, editable by its author or an org admin.

Favourites and team sharing

The star on a card, on the service page or on a ⌘K hit favourites a service, instance, database, schema, VM, host, datastore or external (up to 200). The list opens with a Favourites · Team favourites · Recent strip: a favourite marked Share with team appears under Team favourites for everyone in your organisation (only its author or an org admin changes it), and Recent keeps the last 12 service pages you opened.

Gaps — what the map does not know yet

The Gaps (N) badge in the header (or ?view=catalogue&gaps=1) lists what the Catalogue cannot place or name, one rule per section, each row with the one action that fixes it. Up to 50 rows per rule are shown, then “Show all”.

• Re-probe and the alias fixes need edit rights on the integrations' owner; everyone else sees the candidates read-only. Pin to VM and Add integration open pages that check their own rights.

Change timeline and the weekly diff

The service page's Change timeline merges, newest first, the last 30 days (Load more reads 90; at most 200 entries): ownership edits and clears, health changes with their reasons, aliases set or removed, snoozes on findings of its hardware, vCenter events on its VMs and hosts, calls that started (“Started calling X”) and calls that went stale. Each row links to where it happened.

• What changed this week compares its call edges now with a daily snapshot of the call graph 7 days ago (or the nearest earlier day, which it names): new and gone edges, edges gone stale, error-rate changes (≥ 2× and ≥ 5 points) and call-volume changes (≥ 50 %). The dependency sweep writes the snapshot once a UTC day and keeps 90 days; before the first one it says so (“no snapshot yet — first one on …”) instead of showing zeros.
• The Catalogue header shows “N new edges, M stale this week” across everything you can see.

Exports and the ServiceNow CMDB import

Export ▾ in the header downloads services, instances or databases as CSV, or everything as JSON (topologix/catalogue/1); the service page's menu exports that one service with the instances it calls. A value nobody measured is an empty cell, never 0, and a cell that would start a spreadsheet formula is quoted. Every download is audited (catalog.export), and database names of an integration you cannot see are never in it.

• ServiceNow CMDB… downloads import-ready files for cmdb_ci_service, cmdb_ci_db_instance, cmdb_ci_vmware_instance, cmdb_ci_esx_server, cmdb_rel_ci — one CSV per class, or all of them as one JSON. Every row carries sys_class_name and correlation_id (our key), so an Import Set with a transform map that coalesces on correlation_id updates the same CI on every re-import instead of creating duplicates. Tier maps to business criticality, health to operational status; relationships (Runs on, Virtualized by, Depends on, Uses) are written only between CIs in the same export — load the CI classes first and cmdb_rel_ci last.
• VMs and ESXi hosts ServiceNow Discovery already knows need a second coalesce (VM name, host serial number) or the Identification and Reconciliation Engine. Topologix does not write to the CMDB itself in this version.

Good to know

• Every store (spans, metrics, logs) is bounded at every level and every cap keeps a counter — request/error counts are exact totals, only samples are capped, and the cards render the ingest counters (received, decode errors, drops).
• Service identity comes from the resource attribute service.name — set OTEL_SERVICE_NAME per service, or every app lands on one “unknown” card.
• Signals accepted: traces, metrics, logs over OTLP/HTTP. Still out of scope: OTLP/gRPC, bucket-based histogram percentiles, ExponentialHistogram/Summary (accepted and dropped, with counters).
• OTLP only through a proxy: the server’s former direct paths (/api/otel/v1/traces|metrics|logs and :3001/v1/…) answer 410 Gone with {code: 12, message: “… send to http://<your proxy>:4318/v1/<signal> …”} — a collector stops retrying and its log says where to send instead.
• Relay gotcha: if pushes to the proxy vanish while its log says the relay is listening, check whether another process already binds :4318 — a co-located OpenTelemetry Collector will silently swallow loopback traffic and even answer with valid OTLP responses. The proxy’s settings can move the relay port.