Start with one important request, measure its path from client to response, and test a hypothesis under comparable load. That makes the choice of fix evidence-based.

First define which request is slow.

“The API is slow” is too broad to fix. Identify the endpoint, user workflow, data volume, and when the issue appears: always, under load, or after a change. Check errors and timeouts separately; they may point to a different cause than latency.

  • Record more than the average: include latency distribution such as p50, p95, and p99 for the important endpoint.
  • Compare like with like: request shape, response size, load period, and application version.
  • Set the goal up front: which workflow must improve and what trade-off is acceptable.

First output

One reproducible workflow and a baseline measurement. Without these, the effect of a fix cannot be judged reliably.

Break request time into components.

Use a trace (for example, OpenTelemetry) or another way to follow the request through application code, the database, external services, queues, serialization, and response delivery. The goal is not to collect every graph; it is to locate where time is spent.

01 /

Measure from outside

Capture latency and errors as seen by the client or at service ingress.

02 /

Correlate internally

Link slow requests to traces, logs, and resource metrics from the same period.

03 /

Isolate the segment

Separate SQL, external calls, waiting, CPU, and response construction.

04 /

Recheck under load

Repeat under representative load; a single request may not reveal the problem.

Practical worksheet

A worksheet for one slow request

A template to fill from your own system. Values and conclusions come only after real measurements.

Workflow

Endpoint, user action, response size, and representative load.

Baseline

Response-time distribution, error rate, and when the delay appears.

Time by component

Measure SQL, external calls, waiting, and application processing separately.

Change verification

Repeat under matched load; compare latency, errors, resource use, and correctness.

Use evidence to identify the bottleneck.

The same external latency can have very different causes. Choose a remedy after observing the system, rather than based on a preference for caching, a new framework, or a language.

01 / OPTION

Database

Repeated queries, an expensive plan, or lock waits. Check SQL calls per API request, the slow query log, and the plan via EXPLAIN ANALYZE; in PostgreSQL, pg_stat_statements helps.

02 / OPTION

External service

Time goes to a network call, retries, or partner limits. Check timeouts, concurrency, and whether a synchronous reply is required.

03 / OPTION

App and runtime

CPU, memory, connection pools, or serialization. Profiling and resource metrics help identify the exact segment.

Causes can compound: slow SQL keeps connections busy, which creates queues and timeouts. Fix the primary cause, then measure again.

Causes that are easy to miss

  • PHP-FPM queue. When all workers (pm.max_children) are busy, new requests wait for a free one. Each request looks fast in the logs, yet the client sees latency. Check the pool status page (pm.status_path) and the listen queue counter.
  • Database connection pool. Waiting for a free connection shows up in application metrics as slow SQL, even though the query itself runs fast in the database.
  • Tail latency. A cold cache after a deploy, garbage collector pauses, rare heavy requests, and large uncompressed responses show up in p99 but barely move the average.

Change one hypothesis and verify the effect.

After diagnosis, state a testable change: an index, removal of N+1 queries, fewer external calls, a cache design change, or concurrency tuning. Each change needs an expected effect, correctness checks, and a rollback path.

  • Compare before and after on the same workflows and load.
  • Watch side effects: errors, database load, memory, cost, and data freshness.
  • Improve the critical user path, not just a favorable lab metric.

When does Go help?

If measurements expose a bottleneck in a bounded component, Go may be an implementation option. Compare that path with smaller changes first.

What to prepare for an API review.

One or two critical workflows are enough. Useful inputs include a sanitized request example, expected and observed latency, error graphs, the latest relevant change, and an external dependency diagram. If tracing exists, include several representative slow traces.

  • Which endpoint and user workflow are affected?
  • When did the problem begin, and what changes its frequency?
  • Which metrics exist, and what is still missing for a decision?