Publication date: October 25, 2024
Description
CockroachDB diagnostics reporting, if enabled for a cluster in CockroachDB Cloud or in a Self-Hosted deployment, could have been sending the values of the cluster settings specified below through the telemetry payload sent to Cockroach Labs or stored by you locally. This behavior was due to a bug that incorrectly used user-configurable cluster setting redaction code paths when reporting diagnostics. This data is no longer stored by Cockroach Labs and is no longer sent in upcoming patch releases.
The data would be sent only if the cluster setting diagnostics.reporting.enabled was set to true and server.redact_sensitive_settings.enabled was set to false. These are the default values for those cluster settings.
The affected cluster setting in versions 20.2, 21.1, 21.2, 22.1, 22.2, 23.1, and 23.2.0 is:
server.oidc_authentication.client_id
The affected cluster settings in versions 23.2.1+, 24.1, 24.2, and 24.3 are:
server.oidc_authentication.client_idserver.oidc_authentication.client_secret
To prevent the values of these cluster settings from being sent, we recommend either upgrading to specific patch versions as they become available (as detailed in the Statement section) or following the Mitigation steps outlined below. However, even if you do not correct this issue and continue to send this data to Cockroach Labs, it will be automatically discarded by our systems, if received.
If these values were included in diagnostic reporting sent to Cockroach Labs, they were stored in a secure, non-public, restricted Cockroach Labs telemetry database, and have now been deleted. If you need help determining if this information was sent to Cockroach Labs by your CockroachDB clusters, email security@cockroachlabs.com with the subject "TA 133479 - Cluster Review" and a list of the Cluster IDs in question, and we will respond after evaluating what data we may have received.
Statement
The default cluster setting redaction behavior in diagnostics will be resolved in CockroachDB versions 23.1.29, 23.2.14, 24.1.7, 24.2.5, and 24.3.0.
Mitigation
Users of CockroachDB versions 20.2, 21.1, 21.2, 22.1, 22.2, 23.1.0 - 23.1.28, and 23.2.0 are encouraged to:
Set the value of the
diagnostics.reporting.enabledcluster setting tofalseusingSET CLUSTER SETTING:SET CLUSTER SETTING diagnostics.reporting.enabled = false;Rotate any credentials if they have values set for the above cluster settings.
Users of CockroachDB 23.2.1 - 23.2.13, 24.1.0 - 24.1.7, 24.2.0 - 24.2.4, and 24.3-alpha.1+ are encouraged to:
Update either of the following cluster settings using
SET CLUSTER SETTING:- Enable sensitive setting redaction:
SET CLUSTER SETTING server.redact_sensitive_settings.enabled = true;- Disable diagnostic reporting:
SET CLUSTER SETTING diagnostics.reporting.enabled = false;Rotate any credentials if they have values set for the above cluster settings, such as the OIDC client secret.
Even without these mitigations, the information is no longer stored by Cockroach Labs and is automatically discarded by our systems, if received. All existing copies of this data have been deleted. These mitigations are not required if you have already upgraded to v23.1.29, v23.2.14, v24.1.7, v24.2.5, v24.3.0, or later patch versions in their major versions’ series.
Impact
Customer credentials stored in sensitive cluster settings were potentially included in diagnostic payloads periodically sent by cluster nodes to Cockroach Labs for telemetry purposes. Unredacted values of sensitive cluster setting data were stored internally, within a limited access table. The payload information specifies the Cluster ID, but does not include information that directly identifies or maps the Cluster ID to the customer or host cluster.
Was this helpful?
