Logging Overview

If you need to monitor your cluster, tune performance, or troubleshoot issues, you can check the CockroachDB logs, which include details about notable cluster, node, and range-level events.

Details

When a node starts, shuts down, or processes a cockroach command, it produces a stream of messages about the command's activities. Each message is composed of:

• A payload that contains events either structured in JSON or conveyed in an arbitrary string. For details on structured event types and their fields, see Notable Event Types.
• An envelope that contains event metadata (e.g., severity, date, timestamp, channel). Depending on the log format you specify when configuring logs, the envelope can be formatted either in JSON or as a flat prefix to the message.

Messages are organized into appropriate logging channels and then routed through log sinks. Each sink further processes and filters the messages before emitting them to destinations outside CockroachDB. The mapping of channels to sinks, as well as the processing and filtering done by each sink, is configurable and is intended to support multiple use cases. For more information on configuring sinks and severity levels, see Configure Logs.

Logging channels

Log messages in CockroachDB are directed into logging channels, which can in turn be assigned to output to one or more log sinks.

This allows you to group channels that log related information (e.g., operational, security, or SQL events) into their own sinks. Each sink can output to a predetermined destination where the logs can be collected and parsed. For usage examples, see Logging Use Cases.

Channel	Description
DEV	Uncategorized and debug messages.
OPS	Process starts, stops, shutdowns, and crashes (if they can be logged); changes to cluster topology, such as node additions, removals, and decommissions.
HEALTH	Resource usage; node-node connection events, including connection errors; up- and down-replication and range unavailability.
STORAGE	Low-level storage logs from the Pebble storage engine.
SESSIONS	Client connections and disconnections (when enabled via the server.auth_log.sql_connections.enabled cluster setting); SQL authentication logins/attempts and session/query terminations (when enabled via the server.auth_log.sql_sessions.enabled cluster setting).
SQL_SCHEMA	Database, schema, table, sequence, view, and type creation; changes to table columns and sequence parameters.
USER_ADMIN	Changes to users, roles, and authentication credentials.
PRIVILEGES	Changes to privileges and object ownership.
SENSITIVE_ACCESS	SQL audit events (when enabled via ALTER TABLE ... EXPERIMENTAL_AUDIT).
SQL_EXEC	SQL statement executions (when enabled via the sql.trace.log_statement_execute cluster setting) and uncaught Go panic errors during SQL statement execution. Warning: Logging cluster-wide executions by enabling the sql.trace.log_statement_execute cluster setting will incur considerable overhead and may have a negative performance impact.
SQL_PERF	SQL executions that impact performance, such as slow queries (when enabled via the sql.log.slow_query.latency_threshold and/or sql.log.slow_query.experimental_full_table_scans.enabled cluster settings).
TELEMETRY	Telemetry events for internal usage, except for sampled_query events which may be logged externally to Datadog.
KV_DISTRIBUTION	Data distribution events, such as moving replicas between stores in a cluster, adding replicas to ranges, and removing replicas from ranges.

Logging channels are analogous to logging facilities in Syslog or logging services in Datadog. For more details on the contents of each logging channel, see the Logging reference.

Logging destinations

When using the default logging configuration, the events collected on each logging channel are split into log files as follows:

Filename	Description	Channels
cockroach.log	General CockroachDB log	DEV
cockroach-health.log	Health log	HEALTH
cockroach-kv-distribution.log	Data distribution log	KV_DISTRIBUTION
cockroach-security.log	SQL security log	PRIVILEGES + USER_ADMIN
cockroach-sql-audit.log	SQL access audit log	SENSITIVE_ACCESS
cockroach-sql-auth.log	SQL authentication log	SESSIONS
cockroach-sql-exec.log	SQL execution log	SQL_EXEC
cockroach-sql-slow.log	SQL slow query log	SQL_PERF
cockroach-sql-schema.log	SQL schema change log	SQL_SCHEMA
cockroach-pebble.log	Pebble log	STORAGE
cockroach-telemetry.log	Telemetry log	TELEMETRY

Logging destinations and many other parameters are configurable via YAML. For details, see Configure Logs.

See also

• Configure logs
• Logging Levels and Channels
• Log formats
• Notable Event Types