Custom ClickHouse® Database Monitoring Dashboards

Standard monitoring dashboards are a good start, but every ClickHouse® workload is a little different, and the things that go wrong on your cluster are not always the things a generic dashboard watches. The fix is to build monitoring out of your own SQL. ClickHouse® exposes almost everything about itself in system tables, so anything you can query, you can monitor.

Your database already knows everything

This is the insight that changes how you think about monitoring ClickHouse®. You do not need a separate metrics pipeline to answer most operational questions, because the database already records them. Part counts, merge progress, replication status, query history, memory by query, connection counts, all of it lives in system tables you can query right now.

That means a monitoring dashboard is just a saved query plus a chart. In CHOps monitoring you write the SQL, pick a visualization, and it becomes a panel you can watch. No exporter to configure, no metric names to learn, just the SQL you already know.

Panels I build for almost every cluster

A few custom panels earn their place on nearly every cluster I run.

Parts per table, to catch merges falling behind before they cause errors:

SELECT
    database,
    table,
    count() AS parts
FROM system.parts
WHERE active
GROUP BY database, table
ORDER BY parts DESC
LIMIT 20;

Memory by currently running query, so I can see at a glance if one query is eating the node:

SELECT
    query_id,
    formatReadableSize(memory_usage) AS memory,
    elapsed,
    substring(query, 1, 60) AS query
FROM system.processes
ORDER BY memory_usage DESC;

Query volume and average duration over the last day, bucketed by hour, to see the daily rhythm and spot the hours where things degrade:

SELECT
    toStartOfHour(event_time) AS hour,
    count() AS queries,
    round(avg(query_duration_ms)) AS avg_ms
FROM system.query_log
WHERE type = 'QueryFinish'
  AND event_time > now() - INTERVAL 1 DAY
GROUP BY hour
ORDER BY hour;

Each of these is just a query. Turning it into a watched panel is what makes it monitoring.

Watch what is specific to you

The real power is monitoring the things that are specific to your workload. If you have a table that must stay fresh, write a panel that shows the age of its newest row. If a particular query pattern is your bread and butter, watch its p95 latency. If you depend on a materialized view keeping up, watch its lag. None of these are in a generic dashboard, because they are about your application, not ClickHouse® in general.

This is where SQL-driven dashboards beat fixed ones. You are not limited to the metrics someone else decided were important. You watch what actually matters for the thing you are running.

Pairing custom panels with the basics

I do not throw away the standard health view. I covered the core metrics everyone should watch in my ClickHouse monitoring guide, and those still go on the main screen. The custom panels sit alongside them, covering the workload-specific risks. The combination is what I trust: a standard health view for the universal stuff, and a handful of SQL panels for the things only I know to watch.

And once a panel reliably signals a problem, the natural next step is to turn it into an alert so you do not have to keep looking at it. That is the bridge to email alerts with SQL conditions, where the same query that drives a panel can drive a notification.

If you already know ClickHouse® SQL, you already know how to monitor it. The system tables are right there. The monitoring feature page shows how to turn those queries into panels you can keep an eye on.