Monitoring 📖 12 min read

Grafana Loki Deployment

Grafana Loki stores logs and indexes only a small set of labels, not the text of every line, which makes it much lighter to run than a full-text search stack. This guide runs Loki 3 in Docker Compose with Grafana Alloy collecting host and container logs and Grafana for querying, then covers LogQL, labels, retention, alerting and the errors you're most likely to hit.

What you need: a Linux server with Docker Engine and the Compose plugin (see the Docker Installation Guide), sudo access, and enough disk for the log retention you plan to keep.

Versions: written for grafana/loki:3.7.8, grafana/alloy:v1.20.1 and grafana/grafana:13.2.4 (October 13, 2026, with security fixes), from the Loki, Alloy and Grafana release pages. Newer patch releases may exist; to update, read the release notes, change the tags and run docker compose pull and docker compose up -d. Promtail, the collector older Loki guides use, reached end of life on March 2, 2026, and Grafana's migration guide shows how alloy convert --source-format=promtail turns an existing Promtail config into an Alloy one.

Why Loki Over ELK?

The short version: ELK indexes everything, Loki indexes almost nothing. Loki only indexes labels (like job name, host or container). The log lines themselves are stored compressed in chunks and filtered at query time. Without a large inverted index to keep in memory, Loki needs far less RAM and disk than Elasticsearch for the same logs.

It plugs straight into Grafana, so if you already run Grafana for Prometheus dashboards (see Prometheus Grafana Monitoring), you don't need a separate UI like Kibana. One less thing to update.

The trade-off is real: searches that can't be narrowed down by labels and time range have to scan a lot of data, so Loki is slower than Elasticsearch for broad full-text search. For filtering by service and time range and grepping for error patterns, it works well.

Architecture

  • Loki - stores logs and answers LogQL queries
  • Grafana Alloy - the collector: it reads log files and container logs, adds labels and pushes them to Loki
  • Grafana - where you query and visualise logs

Docker Compose Setup

Create a directory with compose.yaml (below), loki-config.yaml and config.alloy (next sections). In that directory, create a private .env file with install -m 600 /dev/null .env, then edit it with nano .env. Add GRAFANA_ADMIN_PASSWORD=YOUR_LONG_RANDOM_PASSWORD, replacing the placeholder with a unique password from your password manager. Keep .env out of version control.

Editor window showing the configuration from the section Docker Compose Setup
Loki's port is published on 127.0.0.1 only.
services:
  loki:
    image: grafana/loki:3.7.8
    ports:
      - "127.0.0.1:3100:3100"
    volumes:
      - ./loki-config.yaml:/etc/loki/local-config.yaml:ro
      - loki-data:/loki
    command: -config.file=/etc/loki/local-config.yaml
    restart: unless-stopped

  alloy:
    image: grafana/alloy:v1.20.1
    ports:
      - "127.0.0.1:12345:12345"
    volumes:
      - ./config.alloy:/etc/alloy/config.alloy:ro
      - /var/log:/var/log:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - alloy-data:/var/lib/alloy/data
    command: run --server.http.listen-addr=0.0.0.0:12345 --storage.path=/var/lib/alloy/data /etc/alloy/config.alloy
    restart: unless-stopped

  grafana:
    image: grafana/grafana:13.2.4
    ports:
      - "127.0.0.1:3000:3000"
    volumes:
      - grafana-data:/var/lib/grafana
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_ADMIN_PASSWORD:?set GRAFANA_ADMIN_PASSWORD in .env}
    restart: unless-stopped

volumes:
  loki-data:
  alloy-data:
  grafana-data:
  • The Alloy command is the one from Grafana's Alloy Docker docs. --server.http.listen-addr=0.0.0.0:12345 makes Alloy's debugging UI reachable from outside the container, and --storage.path is where Alloy keeps its read positions, so the alloy-data volume stops it re-sending files after a restart.
  • Security: this Loki config has authentication turned off, so anyone who can reach port 3100 can read and write logs. That's why 3100 and Alloy's UI on 12345 are bound to 127.0.0.1; Grafana and Alloy reach Loki over the Compose network. Grafana is also bound to loopback so its login is not sent over plain HTTP to a remote host. Use the SSH tunnel below, or put Grafana behind a reverse proxy with HTTPS before exposing it (the Nginx Web Server Setup shows how).
  • Mounting /var/run/docker.sock lets Alloy read container logs, but it also gives the container full control of the Docker daemon. :ro only stops the socket file being replaced; it doesn't limit the API.
  • If you already run the stack from the Prometheus guide, add the loki and alloy services to that compose file and reuse its Grafana instead of starting a second one on port 3000.

Loki Configuration

Create loki-config.yaml. It's the single-binary, filesystem-backed setup from the configuration that ships in the Loki 3.7 image, plus retention and an Alertmanager address:

Editor window showing the configuration from the section Loki Configuration
With auth_enabled: false anyone who can reach port 3100 can read and write logs, which is why it stays on 127.0.0.1.
auth_enabled: false

server:
  http_listen_port: 3100

common:
  instance_addr: 127.0.0.1
  path_prefix: /loki
  storage:
    filesystem:
      chunks_directory: /loki/chunks
      rules_directory: /loki/rules
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

schema_config:
  configs:
    - from: 2020-10-24
      store: tsdb
      object_store: filesystem
      schema: v13
      index:
        prefix: index_
        period: 24h

compactor:
  working_directory: /loki/compactor
  retention_enabled: true
  retention_delete_delay: 2h
  delete_request_store: filesystem

limits_config:
  retention_period: 744h   # 31 days

ruler:
  alertmanager_url: http://alertmanager:9093

What matters here:

  • store: tsdb with schema: v13 is the index format Loki 3 expects, as in the official configuration examples. Structured metadata, which Loki 3 enables by default (allow_structured_metadata: true), needs this schema.
  • Old Loki 2.x examples use boltdb-shipper, schema v11, shared_store and enforce_metric_name. Loki 3.7 refuses to start with shared_store (field shared_store not found), so don't mix those snippets into this file.
  • The ruler address assumes an Alertmanager container named alertmanager on the same network, as in the Prometheus guide. Change or remove it if you don't have one.

Loki can check a config without starting: run the binary with -config.file=loki-config.yaml -verify-config. Run that check with the pinned version and resolve any reported errors before starting the stack.

Grafana Alloy Configuration

Create config.alloy. It collects two things: *.log files in the host's /var/log, and the logs of every container on the host, read through the Docker API.

// Log files on the host, mounted read-only at /var/log
loki.source.file "varlog" {
  targets = [
    {__path__ = "/var/log/*.log", job = "varlog"},
  ]
  forward_to = [loki.write.local.receiver]

  file_match {
    enabled = true
  }
}

// Container logs, read through the Docker API
discovery.docker "containers" {
  host = "unix:///var/run/docker.sock"
}

discovery.relabel "containers" {
  targets = []

  rule {
    source_labels = ["__meta_docker_container_name"]
    regex         = "/(.*)"
    target_label  = "container"
  }
}

loki.source.docker "containers" {
  host          = "unix:///var/run/docker.sock"
  targets       = discovery.docker.containers.targets
  labels        = {job = "docker"}
  relabel_rules = discovery.relabel.containers.rules
  forward_to    = [loki.write.local.receiver]
}

// Where everything is sent
loki.write "local" {
  endpoint {
    url = "http://loki:3100/loki/api/v1/push"
  }
}
  • loki.source.file tails files. The file_match block, which the docs recommend, lets __path__ be a glob and picks up new files as they appear. Each line gets a filename label with the file's path.
  • discovery.docker finds running containers, discovery.relabel turns Docker's /name into a clean container label, and loki.source.docker reads their logs. This follows Grafana's Docker monitoring example.
  • loki.write pushes to Loki's /loki/api/v1/push endpoint, using the Compose service name loki.
  • Debian 12 stopped installing rsyslog by default (see the Debian 12 release notes), so on Debian-based hosts /var/log may contain few .log files and most system logs live in the journal. Alloy's loki.source.journal component reads the journal instead, with the journal directory mounted into the container.

alloy validate config.alloy checks the file for syntax errors and broken references between components. Run it with the pinned version and resolve any errors before continuing.

Start the Stack

Terminal window with the commands from the section Start the Stack
Validate the Compose file, start the stack, and wait for Loki to answer "ready".
# Checks interpolation and refuses to start without GRAFANA_ADMIN_PASSWORD
docker compose config --quiet
docker compose up -d

# Loki answers "ready" once it has finished starting (this can take about 15 seconds)
curl http://127.0.0.1:3100/ready

On the Docker host, http://127.0.0.1:12345 opens Alloy's UI, which lists each component and whether it's healthy. That's the first place to look if logs don't arrive.

Configure Grafana

  1. From your own computer, run ssh -N -L 3000:127.0.0.1:3000 you@YOUR_SERVER, using your SSH account and server address. Keep that terminal open. Browse to http://127.0.0.1:3000 on your computer and sign in as admin with the password from GF_SECURITY_ADMIN_PASSWORD (without that variable the default is admin/admin, and Grafana asks you to change it).
  2. Go to Connections → Data sources → Add new data source (or search for "Loki" on the Connections page), as described in the Grafana data source docs.
  3. Select Loki.
  4. URL: http://loki:3100
  5. Click Save & test.

Then open Explore, pick the Loki data source and run {job="docker"}.

Querying Logs (LogQL)

Loki uses LogQL, which borrows its selector syntax from PromQL. With the labels from the Alloy config above:

# All lines from the host's log files
{job="varlog"}

# Lines containing "error"
{job="varlog"} |= "error"

# Exclude lines containing "debug"
{job="varlog"} != "debug"

# Regex match
{job="varlog"} |~ "fail(ed|ure)"

# One container's JSON logs, keep only HTTP errors
{container="nginx"} | json | status >= 400

# Error lines per container over 5 minutes
sum by (container) (count_over_time({job="docker"} |= "error" [5m]))

The json example only works if that container writes JSON log lines with a status field; use | logfmt for key=value logs.

Labels Are Key

Loki only indexes labels, so the labels you pick decide how fast queries run and how many streams Loki has to manage:

  • job - what kind of source (here varlog or docker)
  • container or host - where the line came from
  • environment - prod/staging/dev, if you collect from several

Don't use high-cardinality values (user IDs, request IDs, IP addresses) as labels. Each unique combination becomes a separate stream, which bloats the index and slows Loki down. Keep such values in the log line and filter them at query time. Grafana's label best practices go into detail.

Retention

By default Loki keeps logs forever. The compactor and limits_config blocks in the config above turn on deletion after 31 days. From the retention docs:

  • retention_enabled: true is required; without it the compactor only compacts.
  • delete_request_store must be set when retention is enabled. Without it, Loki 3.7 rejects the config with compactor.delete-request-store should be configured when retention is enabled.
  • Chunks are deleted after retention_delete_delay (2 hours here), not immediately, and changing the retention period isn't applied retroactively.

Size the disk for your retention window, and watch it (next sections). If the local disk can't hold the logs you want to keep, or you need more than one Loki instance, move chunks to object storage such as S3 or MinIO as described in the storage docs.

Alerting

Loki's ruler can evaluate LogQL alert rules and send firing alerts to Alertmanager. With auth_enabled: false, Loki runs as the single tenant fake, so rule files go in a fake subdirectory of the rules directory. Save this as loki-rules/rules.yaml:

Editor window showing the configuration from the section Alerting
A Loki ruler alert that fires on a high error rate.
groups:
  - name: error-alerts
    rules:
      - alert: HighErrorRate
        expr: |
          sum(rate({job="docker"} |= "error" [5m])) > 10
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: More than 10 log lines per second contain "error"

Mount it into the Loki container by adding - ./loki-rules:/loki/rules/fake:ro to the loki volumes and restarting. The rule then shows up at http://127.0.0.1:3100/prometheus/api/v1/rules. The threshold is only an example; pick one that matches your normal error rate. The Loki alerting docs cover the ruler in full, and note that Grafana-managed alert rules are the recommended alternative if you'd rather configure alerts in Grafana.

Production Tips

  • Monitor Loki itself. It exposes Prometheus metrics on /metrics, so point the Prometheus server from the Prometheus guide at it and alert on disk usage of the volume that holds /loki.
  • Keep labels few and low-cardinality (see above); it's the most common cause of a slow Loki.
  • Start with this single-binary mode. Loki's scalable and microservice deployment modes exist for much larger volumes and add a lot of moving parts.
  • Back up the loki-data volume only if the logs matter to you; for many home setups, losing old logs is acceptable and the config files are what you need to keep. Stop Loki first (docker compose stop loki) so the copy is consistent.
  • When you remove the stack, docker compose down keeps the volumes. docker compose down -v deletes them, including every stored log line and Alloy's read positions.

Where Loki Falls Short

Full-text search across everything is weaker. If you often search for arbitrary strings without knowing which service produced them, Loki has to scan far more data than an indexed search engine, and such queries can be slow or hit query limits. Loki assumes you know roughly where to look (which job, which time range) before you query.

Log analytics is limited too. LogQL handles rates, counts and grouping by labels or extracted fields well, but complex multi-dimensional analysis of log contents is easier in Elasticsearch and Kibana. If your use case is log analytics rather than log monitoring, ELK may still be the better tool.

Troubleshooting

  • No logs in Grafana. Open Alloy's UI on port 12345 and check that every component is healthy, then run docker compose logs alloy for push errors. Make sure the Explore time range covers the last few minutes, and that your selector matches a label that exists (the label browser in Explore lists them).
  • Loki won't start after copying an older config. Errors like field shared_store not found or messages about boltdb-shipper mean a Loki 2.x snippet; compare with the config above or the Loki upgrade guide.
  • Retention error at startup. compactor.delete-request-store should be configured when retention is enabled means the delete_request_store line is missing.
  • Host log files are missing. Confirm the files match /var/log/*.log (the glob doesn't descend into subdirectories; use /var/log/**/*.log for that), and that the system writes log files at all rather than only to the journal.
  • Container logs are missing. loki.source.docker needs the Docker socket mounted, and it reads logs through the Docker API the same way docker logs does. If docker logs <container> shows nothing, Alloy won't get anything either.
  • "Ingester not ready" right after start. Loki waits a short time after startup before reporting ready; give it a few more seconds.