CockroachDB v19.2 is a required Regular Release.
Refer to Major release types before installing or upgrading for release timing and support details.
On this page, you can read about changes and find downloads for all production and testing releases of CockroachDB v19.2
For key feature enhancements in v19.2 and other upgrade considerations, refer to the notes for v19.2.0.
For details about release types, naming, and licensing, refer to the Releases page.
Be sure to also review the Release Support Policy.
After downloading a supported CockroachDB binary, learn how to install CockroachDB or upgrade your cluster.
Get future release notes emailed to you:
v19.2.12
Release Date: January 19, 2021
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE
, or IMPORT
, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Bug fixes
- Fixed a panic in protobuf decoding. #58876
- Fixed a bug where the
age
function did not normalize the duration for large day orH:M:S
values in the same way PostgreSQL does. #55528
Contributors
This release includes 5 merged PRs by 5 authors. We would like to thank the following contributors from the CockroachDB community:
- kev (first-time contributor)
v19.2.11
Release Date: October 12, 2020
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE
, or IMPORT
, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
General changes
- Links to unimplemented issues that are returned in error messages now use the CockroachLabs redirect/short-link server. #53955
SQL language changes
Bug fixes
- The Cluster Node Map and the Debug Page that lists cluster localities are now available again to non-admin users. This fixes a bug introduced in v19.2.3. #53332
- Fixed a bug that could cause backfills to fail with a vague
TransactionStatus
error as they are completing. #53765 - Fixed a bug that could cause CockroachDB to crash when evaluating queries with window functions with the
GROUPS
mode of framing whenOFFSET FOLLOWING
boundary was used and when the offset was a very large value such that it could result in an integer overflow. #53758 - Fixed a bug that could cause CockroachDB to crash when using a
RANGE
-mode window function with an offset in some cases (e.g,OVER (PARTITION BY b ORDER BY a RANGE 1 PRECEDING
). #54074 - Adding a
NOT NULL
constraint no longer fails with the errorcheck ... does not exist
. #54290 - A
NOT VALID
foreign key constraint no longer fails with the internal errortable descriptor is not valid: duplicate constraint name
. #54290 - Fixed a bug that could cause columns used in an index containing foreign key columns as a prefix to lead to all of the index columns being set to
NULL
or the default value onCASCADE
. #54592 - Fixed a rare bug that could lead to index backfills failing on transaction restarts. #54860
Contributors
This release includes 18 merged PRs by 11 authors.
v19.2.10
Release Date: August 24, 2020
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE
, or IMPORT
, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Bug fixes
- Fixed "column not in input" internal error in some corner cases. #50860
- Previously, CockroachDB could crash when internal memory accounting hit a discrepancy. Now it will report an error instead. #51015
- Fixed incorrect results in some cases involving joins on interleaved tables with limits. #51435
- Previously, CockroachDB could hit an internal error when executing
regexp_replace
builtin. This has been fixed. #51348 - Previously, CockroachDB could hit a "command is too large" error when performing an
UPSERT
operation with many values. Internally, we attempt to perform such operation by splitting it into "batches", but the batching mechanism was broken. This has been fixed. #51612 - CockroachDB previously could crash on some queries with merge joins. This has been fixed. #52074
- Fixed an internal error that could occur when an aggregate function argument contained a correlated subquery with another aggregate function referencing the outer scope. This now returns an appropriate user-friendly error, "aggregate function calls cannot be nested". #52141
- Previously, subtracting months from a
TIMESTAMP
/TIMESTAMPTZ
orDATE
whose date value is greater than 28 could subtract a further year off. This has been fixed. #52155 - Large write requests no longer have a chance of erroneously throwing a "transaction with sequence has a different value" error. #52268
- Previously, CockroachDB could return incorrect results on query that encountered a
ReadWithinUncertaintyInterval
error. This has been fixed. #52252 - Previously, CockroachDB would erroneously restart the execution of empty, unclosed portals after they had been fully exhausted. This has been fixed. #52444
- Previously, executing a large number of statements in a transaction without committing could crash a CockroachDB server. This has been fixed. #52403
Performance improvements
- Queries no longer block during planning if cached table statistics have become stale and the new statistics have not yet been loaded. Instead, the stale statistics are used for planning until the new statistics have been loaded. This improves performance because it prevents latency spikes that may occur if there is a delay in loading the new statistics. #52189
Contributors
This release includes 24 merged PRs by 9 authors.
v19.2.9
Release Date: July 6, 2020
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE
, or IMPORT
, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
General changes
- Schema changes are now logged in greater detail. #50375
Bug fixes
- Fixed a bug that occurred during incremental backups with revision history on a database (or full cluster) when a table in the database being backed up was dropped. When other tables were later created, dropped, or altered, future incremental backups would return an error. #50208
- Fixed a bug in
TRUNCATE
that could leave tables in a state where they could not be renamed. #50714 #50756#50756]
Contributors
This release includes 4 merged PRs by 3 authors.
v19.2.8
Release Date: June 29, 2020
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE
, or IMPORT
, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Security updates
SQL language changes
- The
pg_database
table inpg_catalog
no longer requires privileges on any database in order for the data to be visible. #48766
Bug fixes
- Fixed a bug where
cockroach dump
on a table with collated strings would omit the collation clause for the data insertion statements. #48833 - Manually writing a
NULL
value into thesystem.users
table for the "hashedPassword" column will no longer cause a server crash during user authentication. #48837 - Fixed a memory leak which could affect changefeeds performing scans of large tables. #49162
- Previously, when the value passed to
--drain-wait
was very small, but non-zero,cockroach quit
in certain cases would not proceed to perform a hard shutdown. This has been corrected. This bug was present in v19.1.9, v19.2.7, and v20.1.1. #49364 - Previously, some benign errors were reported as unexpected internal errors by the vectorized execution engine. This is now fixed. #49535
- Fixed a bug causing file descriptors to be leaked during GSS authentication. #49655
- Casting to width-limited strings now works correctly for strings containing Unicode characters. #50160
- Fixed cases in which casting a string to a width-limited string array was not truncating the string. #50169
- Fixed a RocksDB bug that could result in inconsistencies in rare circumstances. #50498
Performance improvements
- Improved the optimizer's estimation of the selectivity of some filters involving a disjunction (
OR
) of predicates over multiple columns. This results in more accurate cardinality estimation and enables the optimizer to choose better query plans in some cases. #50473
Build changes
- Release Docker images are now built on Debian 9.12. #50481
Doc updates
- Updated guidance on node decommissioning. #7304
- Renamed "whitelist/blacklist" terminology to "allowlist/blocklist". #7535
- Updated the Releases navigation in the sidebar to expose the latest Production and Testing releases. #7550
- Fixed scrollbar visibility on Chrome. #7487
Contributors
This release includes 19 merged PRs by 16 authors. We would like to thank the following contributors from the CockroachDB community:
- Drew Kimball (first-time contributor, CockroachDB team member)
- Jackson Owens (first-time contributor, CockroachDB team member)
- James H. Linder (first-time contributor, CockroachDB team member)
v19.2.7
Release Date: May 20, 2020
This page lists additions and changes in v19.2.7 since v19.2.6.
- For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
- To upgrade to v19.2, see Upgrade to CockroachDB v19.2
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE
, or IMPORT
, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Backward-incompatible changes
- Prior to this patch, the phase of server shutdown responsible for range lease transfers to other nodes would give up after 10000 attempts of transferring replica leases away, regardless of the value of
server.shutdown.lease_transfer_wait
. The limit of 10,000 attempts has been removed, so that now only the maximum durationserver.shutdown.lease_transfer_wait
applies. #47691 - The textual error and warning messages displayed by
cockroach quit
under various circumstances have been updated. The messageok
still indicates that the operation has likely succeeded. #47691 cockroach quit
now prints out progress details on its standard error stream, even when--logtostderr
is not specified. Previously, nothing was printed on standard error. Scripts that wish to ignore this output can redirect the standard error stream. #47691- The copy of
system
andcrdb_internal
tables extracted bycockroach debug zip
is now written using the TSV format (inside the ZIP file), instead of an ASCII-art table as previously. #48098
General changes
- Prior to this patch, the phase of server shutdown responsible for range lease transfers to other nodes had a hard timeout of 5 seconds. This timeout is now configurable with the new cluster setting
server.shutdown.lease_transfer_wait
. #47691
SQL language changes
- Columns in the process of being added or removed to a table are now always set to their default or computed value if another transaction concurrently inserts, updates, or upserts a row. This fixes an issue where a column being backfilled would not be properly set by concurrent transactions. #47411
- The
pg_collation
,pg_proc
, andpg_type
tables inpg_catalog
no longer require privileges on any database in order for the data to be visible. #48081 - Histogram collection with
CREATE STATISTICS
is no longer supported on columns with typeARRAY
. Only row count, distinct count, and null count are collected forARRAY
-type columns. #48344 - CockroachDB now correctly reports the type length for the
CHAR
data type. #48643
Command-line changes
- The time that
cockroach quit
waits client-side for the node to drain (i.e., remove existing clients and push range leases away) is now configurable with the flag--drain-wait
. The default value for--drain-wait
is10 minutes
. Note that separate server-side timeouts still apply. Check theserver.shutdown.*
cluster settings for details. #47691 - It is now possible to drain a node without shutting down the process, using
cockroach node drain
. This makes it easier to integrate CockroachDB with service managers and orchestration. It is safe to issuecockroach node drain
and then separately stop the service via a process manager or orchestrator. Without this new command, there is a risk of misconfiguring the service manager to auto-restart the node after it shuts down viacockroach quit
.cockroach node drain
also recognizes the new--drain-wait
flag. #47691 - The commands
cockroach quit
andcockroach node drain
now report a "work remaining" metric on their standard error stream. The value reduces until it reaches 0 to indicate that the graceful shutdown has completed server-side. #47691 - To support clusters that have been configured to use a cluster name, client commands such as
cockroach init
andcockroach quit
now support the--cluster-name
and--disable-cluster-name-verification
flags. Previously it was impossible to run such commands against nodes configured with the--cluster-name
flag. #48017 cockroach debug zip
now properly skips over fully decommissioned nodes. #48095cockroach debug zip
now tries multiple times to retrieve data using SQL if it encounters retry errors. #48098cockroach debug zip
now supports two command-line parameters:--nodes
and--exclude-nodes
. When specified, they control which nodes are inspected when gathering the data. This makes it possible to focus on a group of nodes of interest in a large cluster, or to exclude nodes thatcockroach debug zip
would have trouble reaching otherwise. Both flags accept a list of individual node IDs or ranges of node IDs, e.g.,--nodes=1,10,13-15
. #48098
Bug fixes
- Previously, CockroachDB could incorrectly consider a non-
NULL
value from an interleaved parent table to beNULL
when the interleaved child has aNULL
value in the row with the corresponding index key. This bug has been fixed. #47104 - A data race on AST nodes for
SELECT
statements that include aWINDOW
definition has been fixed. Previously, these queries could have returned incorrect results. #47176 - Previously, a panic could occur when
SHOW RANGES
orSHOW RANGE FOR ROW
was called with a virtual table. This bug has been fixed. #47516 - Fixed incorrect results that could occur when casting negative intervals or timestamps to type decimal. #47521
- Previously, when the query was executed with the row-by-row engine, CockroachDB could incorrectly return no output instead of returning a parsing error in queries with
count(*)
. This bug has been fixed. #47486 - Previously, CockroachDB incorrectly released the memory used in hash aggregation operations, which could lead to node crashes. This bug has been fixed. #47519
- Previously,
PARTITION BY
andORDER BY
columns in window specifications lost qualifications when used inside views. This bug has been fixed. #47716 - Fixed a bug where target column specifications in
IMPORT INTO
were ignored. #47796 - Columns named with reserved keywords and/or other special characters are now correctly handled. #47796
- CockroachDB will now avoid producing a severe "internal error" upon certain privilege check failures with
pg_catalog
built-in functions. #48243 - Fixed a bug causing cascading deletes to fail when deleting thousands of rows across interleaved tables. #48287
- Fixed a rare bug that could lead to data corruption or data loss if a replica was both the source of a snapshot, was being concurrently removed from the range, and if certain specific conditions exist inside RocksDB. #48319
- Fixed a rare bug where stats were not automatically generated for a new table. #48028
- Fixed a rare bug where stats refreshes could fail. CockroachDB now triggers stats refreshing after completing an import job, after the transaction finishes the job commits. #48028
- Fixed an error that occurred when statistics collection was explicitly requested on a column with type
ARRAY
. #48344 - Fixed a rare bug causing a range to deadlock and all the writes to the respective range to timeout. #48304
- Fixed a bug causing HTTP requests to fail with error 503 "
transport: authentication handshake failed: io: read/write on closed pipe
" and continue failing until the node was restarted. #48482 cockroach debug zip
can now successfully avoid out-of-memory errors when extracting very largesystem
orcrdb_internal
tables. #48098cockroach debug zip
will now properly report an error if an error is encountered while writing the end of the output ZIP file. #48098- Fixed a bug that caused vectorized queries on composite data types to occasionally return invalid data. #48464
- Previously,
SHOW STATISTICS USING JSON
output contained incorrect single quotes for strings with spaces inside histograms. This bug has been fixed. #48545 - Fixed a bug preventing timestamps from being closed, which could result in failed follower reads or failure to observe resolved timestamps in changefeeds. #48683
- Fixed a bug that falsely indicated that
kv.closed_timestamp.max_behind_nanos
was almost always growing. #48690 - CockroachDB now prevents dropping databases that contain tables that are currently offline due to
IMPORT
orRESTORE
. Previously, dropping a database in this state could lead to a corrupted schema, which prevented running backups. #48612 - Fixed a bug that caused
cockroach dump
to erroneously include an additionalCREATE UNIQUE INDEX "primary" ... INTERLEAVE IN PARENT
statement in the dump output on tables with interleaved primary keys. This bug made it impossible to re-import dumped data without manual editing. #48777 - Fixed a bug that caused
SHOW CREATE
statements to sometimes show a partitioning step for an index that was previously dropped. #48767 - Fixed a bug that caused changefeeds that were created with a cursor, and that encounter a retryable error before writing their first checkpoint, to perform full-table backfills upon restart. #48752
- Fixed a bug that could cause canceled changefeeds to not stop. #48752
Performance improvements
- Fixed a bug in the histogram filtering logic of the optimizer which was causing inaccurate cardinality estimates for queries with equality predicates on UUIDs and strings, as well as some other types. This bug has existed since histograms were first introduced into the optimizer in the v19.2.0 release. Fixing it improves the optimizer's cardinality estimates and results in better query plans in some cases. #48625
- Histograms used by the optimizer for query planning now have more accurate row counts per histogram bucket, particularly for columns that have many
NULL
values. This results in better plans in some cases. #48645 - Fixed a performance inefficiency in the vectorized execution engine. This fix speeds up all queries that are executed with the vectorized engine, with most noticeable gains on the queries that output many rows. #48733
Contributors
This release includes 47 merged PRs by 21 authors. We would like to thank the following contributors from the CockroachDB community:
- Artem Barger (first-time contributor)
- Jason Brown (first-time contributor)
v19.2.6
Release Date: April 13, 2020
This page lists additions and changes in v19.2.6 since v19.2.5.
- For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
- To upgrade to v19.2, see Upgrade to CockroachDB v19.2
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE
, or IMPORT
, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Backward-incompatible changes
cockroach init
now waits for server readiness and thus no longer fails when a mistaken server address is provided. #46477
Security updates
- The non-authenticated
/health
HTTP endpoint was previously exposing the private IP address of the node, which can be privileged information in some deployments. This has been corrected. Deployments using automation to retrieve a node build details and address details should use/_status/details/local
instead and use a valid admin authentication cookie. #46477 - All users can now view any comments on any object (bypassing other privileges), but modifying comments require write privilege on the target object. Previously, any user could modify any database/table/view/index comment via direct SQL updates to
system.comments
. This was unintended and a form of privilege escalation, and is now prevented. The privileges required for theCOMMENT
statement,pg_description
,col_description()
,obj_description()
, andshobj_description()
are operating as in PostgreSQL and are unaffected by this change. #46833
SQL language changes
- The type checking code now prefers aggregate overloads with string inputs if there are multiple possible candidates due to arguments of unknown type. #46807
- Added an unimplemented error when attempting to
ADD CONSTRAINT
with theEXCLUDE USING
syntax. #46910 - Added support for
CREATE INDEX .... INCLUDE (col1, col2, ...)
, which is an alias that PostgreSQL uses that is analogous to CockroachDB'sSTORING (col1, col2, ...)
syntax. #46910 - Added support for parsing the
REINDEX
syntax, which results in an unimplemented error that explains thatREINDEX
ing is not required in CockroachDB. #46910 CREATE INDEX CONCURRENTLY
andDROP INDEX CONCURRENTLY
are now parsed but return errors. #46805- Improved the error message for unsupported interactions between correlated subqueries and
WITH
clauses. #46971
Command-line changes
- The
cockroach init
command now waits until the node at the provided server address is ready to accept initialization. This also waits for network readiness. This makes it easier to implement initialization scripts by removing the need for a loop. In addition, implementing such a loop is operationally unsafe and is not recommended. #46477 - CockroachDB will now dump the stacks of all goroutines upon receiving
SIGQUIT
prior to terminating. This feature is intended for use while troubleshooting misbehaving nodes. #46478 - The
cockroach start
andcockroach start-single-node
commands no longer initiate a 1-minute hard shutdown countdown after a request to gracefully terminate. This means that graceful shutdowns are now free to take longer than one minute. It also means that deployments where a maximum shutdown time must be enforced must now use a service manager that is suitably configured to do so. #46483 - The
cockroach debug zip
command now creates valid zip files even if some of its requests encounter an error. #46635
Admin UI changes
- Metrics relating to SQL transaction restarts and rollbacks are now properly captured and exported. #46272
- The display options are now saved separately for each authenticated user. Note: When upgrading to a version with this change, all current display customizations for admin users are lost. #46479
Bug fixes
- Fixed a bug where
pg_catalog.pg_indexes
showed the wrong index definition for inverted indexes. #46564 - Fixed a performance bug where
TRUNCATE
would take 2*num columns round trips. This could lead to slow truncate performance in distributed clusters. #46648 - Fixed an internal error that could occur in the optimizer when a
WHERE
filter contained at least one correlated subquery and one non-correlated subquery. #46167 - CockroachDB no longer returns an internal error in some cases when running queries with
CASE
,AND
, orOR
operators via the vectorized execution engine. #46088 - Fixed a bug where various session variables could not be set to the values "on" or "off", only true or false. #46219
- Accesses to
/health
using a non-root authentication token no longer hang when a node is currently under load, or if a system range is unavailable. #46477 - Fixed a race in the vectorized execution engine. #46486
- The
cockroach debug zip
command now properly collects heap profiles. #46482 - The goroutine dump facility now functions properly when logging to files is disabled, e.g., via
--log-dir=
or--logtostderr
. #46482 - Fixed a bug where the vectorized engine could sometimes give an incorrect result when reading from interleaved parents or children. #46495
- Fixed an internal error that could occur when an aggregate inside the right-hand side of a
LATERAL
join was scoped at the level of the left-hand side. #46544 - Fixed an error that incorrectly occurred when an aggregate was used inside the
WHERE
orON
clause of a subquery but was scoped at an outer level of the query. #46544 - Added support for queries with qualified stars that refer to tables in outer scopes. #46538
- Fixed an incorrect "no data source matches prefix" error in some cases involving subqueries that use views. #46537
- Expressions of the form
NOT(a && b)
no longer fail. #46539 - Fixed a bug where operations on an index that contained a collated string in descending order would fail.#46578
- Fixed a bug where the vectorized engine would throw an internal error when executing a query that utilized an inverted index. #46506
- Fixed an internal error that could happen during planning when a column with a histogram was filtered with a predicate of a different data type. #46667
- CockroachDB no longer incorrectly accounts for some RAM usage when computing aggregate functions. #46622
- Fixed an incorrect query result that could occur when a scalar aggregate was called with a null input. #46807
- Benign "outbox next" errors are now only logged when log verbosity is set to 1 or greater. #46837
- Fixed incorrect result with count(*) when grouping on constant columns. #46894
- Customizations of the Admin UI are once again properly saved across sessions. #46479
- Fixed a rare bug causing transactions that have performed schema changes to deadlock after they restart. #46904
Performance improvements
- Less memory is loaded into memory for
crdb_internal.jobs
. #46452 - Virtual tables that access all table descriptors now make fewer round trips. #46950
Contributors
This release includes 42 merged PRs by 16 authors.
v19.2.5
Release Date: March 23, 2020
This page lists additions and changes in v19.2.5 since v19.2.4.
- For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
- To upgrade to v19.2, see Upgrade to CockroachDB v19.2
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE
, or IMPORT
, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Security updates
- Non-licensed users are now able to add more principals to the special superuser role/group
admin
. Creation of additional roles is still a licensed feature. #45395
SQL language changes
- The SQL:2008 syntax
OFFSET <x> ROWS
andFETCH FIRST <y> ROWS ONLY
now accepts parameter values. #45166 - It is now possible to use
GRANT
andREVOKE
to add users to theadmin
role without a valid license. This change aims to enable use of the Admin UI and other privileged features without a license. #45395
Command-line changes
cockroach debug zip
now includes secondary log files (e.g., the RocksDB logs) in the main log directory. Log files in alternate log directories, if different from the main log directory, are not included. #45202cockroach debug zip
now displays its progress differently on the terminal. #45207cockroach debug zip
now properly supports special characters in database and table names. #45207cockroach debug zip
will now apply the--timeout
parameters to the SQL queries it performs. Previously, there was no timeout, which could causecockroach debug zip
to hang. #45207cockroach debug zip
is now able to tolerate more forms of cluster unavailability. In case system ranges are unavailable, we recommend runningcockroach debug zip
on each node's address, to maximize useful data collection. #45207- The
debug.zip
file generated bycockroach debug zip
now contains hex a representation of the marshaled jobs payload, progress, and table descriptors. #46040
Admin UI changes
- The "Log file list" endpoint now includes secondary log files (e.g., the RocksDB logs) in the main log directory. Log files in alternate log directories, if different from the main log directory, are not included. #45202
Bug fixes
- Fixed a "cannot map variable" error that could occur in rare cases involving joins. #44859
- Fixed a bug where
DROP INDEX
jobs waiting for GC TTL might be erroneously deleted early. Usually aDROP INDEX
job would appear as "waiting for GC TTL" to indicate that the data was still on disk. However, this bug might have caused the job to be deleted before the data was actually removed from disk. #44873 - Fixed occasional "concurrent map write" crash. #44874
- Previously, CockroachDB would return an internal error when a
substring
function with non-INT8
start and length arguments was executed with the vectorized execution engine. This is now fixed. #44902 - Fixed incorrect deduplication of impure expressions (e.g.,
gen_random_uuid
) in projections and default values. #44915 - Previously, CockroachDB could return an internal error on the queries that return
INT
columns when the default integer size has been changed. This is now fixed. #44980 - Previously, CockroachDB would return an internal error when a merge join operation was performed with the vectorized execution engine in the case when two sides of the join had comparable but different types in the equality columns (e.g.,
INT2
on the left andINT4
on the right). This is now fixed. #45032 - Previously, CockroachDB could crash when running
EXPLAIN (VEC)
in some edge cases. Now, an internal error is returned instead. #45004 - Fixed an internal error that could occur when
NULLIF
was called with one null argument. #45390 - Some vectorized execution plans that used lookup joins with
DECIMAL
s would previously return incorrect results. This is now fixed. #45548 - AbortSpan records are now cleaned up more aggressively when it is known to be safe. #45553
- It is now possible to create inverted indexes on columns whose names are mixed-case. #45660
- Fixed a bug causing distributed deadlock detection between transactions to stall and fail to resolve a deadlock. #45664
- Fixed a bug where large ranges with slow send rates would hit the timeout in several storage system queues by making the timeout dynamic based on the current rate limit and the size of the data being sent. This affects several storage system queues: the Raft snapshot queue, the replication queue, and the merge queue. #44952
- Previously, drivers that did not truncate trailing zeroes for decimals in the binary format end up having inaccuracies of up to 10^4 during the decode step. Trailing zeroes are now truncated as appropriate. This fixes known incorrect decoding cases with Postgrex in Elixir. #45670
- Range garbage collection will now trigger based on a large abort span, adding defense-in-depth against ranges growing large and eventually unstable. #45744
- Previously, an internal error could occur in CockroachDB when executing queries that contained unordered synchronizers with the vectorized execution engine. This is now fixed. #45739
- Previously, CockroachDB would crash when using a window function with the
RANGE
mode of framing anoffset PRECEDING
oroffset FOLLOWING
boundary where the single column in anORDER BY
clause was also in thePARTITION BY
clause. This is now fixed. #45805 - Previously, CockroachDB could crash when computing window functions with the
RANGE
mode of framing when one of the bounds was eitheroffset PRECEDING
oroffset FOLLOWING
and there wereNULL
values in the single column of anORDER BY
clause. Additionally,RANGE
mode bounds0 PRECEDING
and0 FOLLOWING
could have been handled incorrectly. This is now fixed. #45805 - When considering if a job should be orphaned, CockroachDB used to take the conservative approach when a descriptor ID pointing to non-existent descriptor was found. This caused jobs to hang and be garbage collected. We now disregard these IDs when considering if a job still has work to do. #45895
- The
--set
parameter forcockroach sql
andcockroach demo
is now able to properly override all client-side options, as advertised. #46130 - Expected errors from the vectorized execution engine are no longer mistakenly annotated as unexpected errors. #46092
Performance improvements
- Execution plans are now faster in some cases that involve
IN
/NOT IN
with an empty tuple, or= ANY
with an empty array. #45213 - Improved the selectivity estimation of some predicates containing
OR
, leading to better plan selection by the optimizer, in some cases. #45972 - Improved cardinality estimation in the optimizer for relations with a small number of rows. This may lead to the optimizer choosing a better query plan. #45972
Contributors
This release includes 36 merged PRs by 15 authors.
v19.2.4
Release Date: February 11, 2020
This page lists additions and changes in v19.2.4 since v19.2.3.
- For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
- To upgrade to v19.2, see Upgrade to CockroachDB v19.2
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE
, or IMPORT
, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Security updates
- Previous versions of CockroachDB were incorrectly enabling non-admin SQL users to use the statements details in the Admin UI and the HTTP endpoint
/_status/statements
. This information is sensitive because the endpoint does not hide data that the requester does not have privilege over. This has been corrected by requiring a SQLadmin
user to access the statements details page and the HTTP endpoint. #44354
Bug fixes
- Fixed a bug where CockroachDB would return an internal error when the
substring
function with a negative length was executed via the vectorized engine. CockroachDB now returns a regular query error on executing the function. #44629 - Fixed "no output column equivalent to.." and "column not in input" errors in some cases involving
DISTINCT ON
andORDER BY
. #44598 - Fixed "expected constant FD to be strict" internal error. #44604
- Fixed possibly incorrect query results in various cornercases, especially when
SELECT DISTINCT
is used. #44604 - Fixed a bug where running a query with the
LIKE
operator using the customESCAPE
symbol when the pattern contained Unicode characters could result in an internal error in CockroachDB. #44648 - CockroachDB no longer repeatedly looks for non-existing jobs, which may cause high memory usage, when cleaning up schema changes. #44698
- Fixed "no indexes" internal error in some cases when we
GROUP BY
on a virtual table. #44723 - Fixed invalid query results in some corner cases where part of a
WHERE
clause is incorrectly discarded. #44749 - Fixed a typechecking error where
BETWEEN
would sometimes allow boundary expressions of a different type. #44810 CASE
operators with an unknownWHEN
type no longer return an error. #44818
Contributors
This release includes 15 merged PRs by 10 authors.
v19.2.3
Release Date: February 3, 2020
This page lists additions and changes in v19.2.3 since v19.2.2.
- For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
- To upgrade to v19.2, see Upgrade to CockroachDB v19.2
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE
, or IMPORT
, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Security updates
- SQL admins can now use the new command
cockroach auth-session login
to create authentication tokens with an arbitrary expiration date. Operators should be careful to monitorsystem.web_sessions
and enforce policy-mandated expirations using either SQL queries or the new commandcockroach auth-session logout
. #44110
Enterprise edition changes
- CockroachDB is now more resilient to connection/network errors when reading data from HTTP servers. CockroachDB now attempts to resume the download after such errors. #43687
SQL language changes
extract()
now supports millennium, century, decade, isoyear, isodow, and julian fordate
,timestamp
, andtimestamptz
.DATE_TRUNC
now supports millennium, century, and decade fordate
,timestamp
, andtimestamptz
. #42957- The output of
EXPLAIN
now shows joins where there are no equality columns as "cross" instead of "hash". Cross joins can be very expensive and should be avoided. #43239 - Previously, we error-coded duplicate foreign key constraint names with
42830
or23503
. We changed this to be42710
to be in line with PostgreSQL. #43425 - Vectorized queries that execute only on supported types, even if those types form part of a table with unsupported types are now run through the vectorized engine. This would previously fall back to the row-by-row execution engine. #44014
SHOW RANGES
now shows locality information consistent with the range descriptor when node ID and store ID do not match. #44312- Previously, CockroachDB would return an internal error when using
SET TRACING
with any type other thanSTRING
. Now it will return a regular query error. Additionally, boolean arguments are now supported inSET TRACING
, andtrue
is mapped toon
mode of tracing whereasfalse
is mapped tooff
. #44347
Command-line changes
- Telemetry is now recorded for whenever the command
cockroach demo
is used. #43819 - Three new CLI commands,
cockroach auth-session login
,cockroach auth-session list
, andcockroach auth-session logout
, are now provided to facilitate the management of web sessions. The commandauth-session login
also produces a HTTP cookie which can be used by non-interactive HTTP-based database management tools. It also can generate such a cookie for theroot
user, who would not otherwise be able to do so using a web browser. #44110
Admin UI changes
- We previously introduced a fix on the Admin UI to prevent non-admin users from executing queries. However, this inadvertently caused certain pages requiring table details not to display. This issue has now been resolved. #44191
Bug fixes
- Fixed a bug during planning for some queries that could cause an infinite loop and prevent the query from being cancelled. #43082
- Some incorrect issue links referenced by error hints have been corrected. #43233
EXPLAIN
can now be used with statements that useAS OF SYSTEM TIME
. #43303- This bug may cause jobs to be left indefinitely in
PENDING
state and never run. #43421 - Fixed a bug that prevented error reporting while generating shell autocomplete scripts. #43507
- Fixed a bug that prevented error reporting while reading from
crdb_internal.gossip_nodes
which could lead to lost writes or other undefined behavior. #43502 - Fixed a bug that prevented error reporting while generating plan diagrams rather than silently reporting no result. #43506
- Fixed a bug where an internal error could occur when
CASE
operator operating on distinct although compatible types was executed via the vectorized engine. For example, a query similar toSELECT CASE WHEN false THEN 0:::INT2 ELSE 1:::INT8 END
previously would error out. #43566 - Fixed a bug that prevented error reporting while looking up table descriptors during schema changes and other transactional interactions with schema which could cause lost writes or other undefined behavior. #43503
- Fixed a bug in the parsing logic for
server.host_based_authentication.configuration
, where both single-character strings, and quoted strings containing spaces and separated by commas, were not properly parsed. #43714 - Fixed a bug that prevented error reporting while failing to parse URIs for multi-locality backup. #43501
- It is now possible to perform
ALTER COLUMN SET/DROP NULL
on multiple (different) columns of the same table inside a single transaction. #43645 - CockroachDB now properly rejects
ALTER COLUMN DROP NOT NULL
on a column that is part of the primary key. #43645 - Fixed a bug where errors from JSON containment operations were silently ignored rather than being returned. #43505
- Previously, CockroachDB could incorrectly report that some values were
NULL
when, in fact, they weren't. This could occur only when vectorized execution engine was used withvectorize=experimental_on
and now has been fixed. #43823 - Fixed a bug where CockroachDB returned incorrect results for some aggregate functions when used as window functions with non-default window frame. Note that
MIN
,MAX
,SUM
,AVG
, as well as "pure" window functions (i.e., non-aggregates) were not affected. #43854 - Fixed a bug where CockroachDB could, in some cases, return an internal error when running a query with a
CAST
operation (:::
) if vectorized execution engine is used. #43890 - A SQL row write that is re-issued after already succeeding will no longer throw a duplicate key error when the previous write in its transaction deleted the row. #43937
- On Linux machines, CockroachDB now respects the available memory limit set by the
cgroup
limits which apply to the cockroach process. #43860 - Fixed a bug in which a query shutdown mechanism could fail to fully cleanup the infrastructure when the query was executed via the vectorized engine and the query plan contained wrapped row-by-row processors. #43944
- Fixed a bug introduced in 19.2 that would allow foreign keys to use a unique index on the referenced columns that indexed more columns than were included in the columns used in the foreign key constraint, which allows potentially violating uniqueness in the referenced columns themselves. #43796
- A benign error previously logged at the
ERROR
level is now logged at theINFO
level behind a verbosity(2) flag. This error might have been observed ascontext canceled: readerCtx in Inbox stream handler
. #44037 - Fixed a bug where an internal error could occur when a query with an aggregate function
MIN
orMAX
was executed via the vectorized engine when the input column was eitherINT2
orINT4
type. #44055 cockroach zip
now emits thegoroutine
file in the proper sub-directory when the corresponding call fails with an error. #44065cockroach debug zip
is again able to operate correctly and continue to iterate over all nodes if one of the nodes does not deliver its goroutine dumps. It would previously prematurely and incorrectly stop with an incomplete dump; this was a regression introduced in 19.2. #44065- CDC is no longer susceptible to a bug where a resolved timestamp might be published before all events that precede it have been published in the presence of a range merge. #44077
- Converted a panic in
golang.org/x/text/language/tags.go
when using collated strings to an error. #44117 - CockroachDB now properly supports using
--url
with query options (e.g.,application_name
) but without specifyingsslmode
. The default ofsslmode=disable
is assumed in that case. #44115 - Removed statistics information from backup jobs' payload information to avoid excessive memory utilization when issuing commands such as
SHOW JOBS
. #44190 - Fixed a bug where CockroachDB could crash in special circumstances when vectorized execution engine is used (it was more likely to happen if
vectorize=experimental_on
setting was used). #44196 - Fixed a planning bug related to
FULL
joins between single-row relations. #44228 - Fixed
CopyFrom requires empty destination
internal error. #44245 - Fixed a bug where multiple nodes attempted to populate the results for
CREATE TABLE ... AS
leading to duplicate rows. #44300 - Fixed internal error when mixed types are used with
BETWEEN
. #44319 - Fixed a bug where CockroachDB would return an internal error when a query with
CASE
operator that returns onlyNULL
values was executed via the vectorized engine. #44353 - Fixed an internal error in the planner when table statistics were collected manually using
CREATE STATISTICS
for different columns at different times. #44442 - Fixed a bug that prevented
admin
users from usingBACKUP
,RESTORE
, andIMPORT
. #44456 - Fixed incorrect plans in very rare cases involving filters that aren't constant folded in the optimizer but that can be evaluated statically when running a given query. #44419
Performance improvements
- Adjusted the optimizer's cost of lookup join when the lookup columns aren't a key in the table. This will now cause some queries to switch to using a hash or merge join instead of a lookup join, improving performance in most cases. #43059
- Improved the estimated row count for some lookup joins during planning, leading to better plans in some cases. #43364
- A transaction running into multiple intents from an abandoned conflicting transaction now cleans them up more efficiently. #43588
- Histograms are now collected automatically for all boolean columns, resulting in better query plans in some cases. For tables that aren't being modified frequently, it might be necessary to run
CREATE STATISTICS
manually to see the benefit. #44171
Allocator
- The kv.allocator.range_rebalance_threshold setting, which controls how far away from the mean a store's range count must be before it is considered for rebalance, is now subject to a 2-replica minimum. If, for example, the mean number of replicas per store is 5.6 and the setting is 5%, the store will not be considered for rebalance unless the number of replicas is lesser than 3 or greater than 8. Previously, the bounds would have been 5 and 6. #44359
Doc updates
- Added a Django app development tutorial. #6359 #6365
- Updated the Hibernate app development tutorial with client-side retry logic. #5760
- Documented how to use keyset pagination to iterate through query results. #6114
- Improved the GSSAPI authentication instructions for configuring Active Directory and MIT and for configuring the client. #6115
- Expanded the Kubernetes tutorial to show how to use a custom CA instead of Kubernetes built-in CA when using manual configs. #6232
- Updated the Kubernetes tutorial for compatibility with Helm 3.0. #6121
- Added language-specific connection strings to the instructions on connecting to a CockroachCloud cluster. #6077
- Added Docker as a download option on the full release notes list. #5792
- Updated the
IMPORT
documentation with an example usage ofDELIMITED
with escaping, a note aboutDEFAULT
values, and an explanation of thestrict_quotes
option. #6244 - Added an FAQ on why Cockroach Labs changed the license for CockroachDB. #6154
- Corrected the description of the possible result of clock skew outside the configured clock offset bounds. #6329
- Expanded the data types overview to indicate whether or not a type supports vectorized execution. #6327
Contributors
This release includes 58 merged PRs by 18 authors. We would like to thank the following contributors from the CockroachDB community:
- Roga Pria Sembada (first-time contributor)
v19.2.2
Release Date: December 16, 2019
This page lists additions and changes in v19.2.2 since v19.2.1.
- For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
- To upgrade to v19.2, see Upgrade to CockroachDB v19.2
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE
, or IMPORT
, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Security updates
CockroachDB previously allowed non-authenticated access to privileged HTTP endpoints like
/_admin/v1/events
, which operate usingroot
user permissions and can thus access (and sometimes modify) any and all data in the cluster. This security vulnerability has been patched by disallowing non-authenticated access to these endpoints and restricting access to admin users only.Note:Users who have built monitoring automation using these HTTP endpoints must modify their automation to work using an HTTP session token for an admin user.
Some Admin UI screens (e.g., Jobs) were previously incorrectly displayed using
root
user permissions, regardless of the logged-in user's credentials. This enabled insufficiently privileged users to access privileged information. This security vulnerability has been patched by using the credentials of the logged-in user to display all Admin UI screens.Privileged HTTP endpoints and certain Admin UI screens require an admin user. However,
root
is disallowed from logging in via HTTP and it is not possible to create additional admin accounts without an Enterprise license. This is further discussed here and will be addressed in an upcoming patch revision.Note:Users without an Enterprise license can create an additional admin user using a temporary evaluation license, until an alternative is available. A user created this way will persist beyond the license expiry.
Some Admin UI screens currently display an error or a blank page when viewed by a non-admin user (e.g., Table Details). This is a known limitation mistakenly introduced by the changes described above. This situation is discussed further here and will be addressed in an upcoming patch revision. The list of UI pages affected includes but is not limited to:
- Job details
- Database details
- Table details
- Zone configurations
Note:Users can access these Admin UI screens using an admin user until a fix is available.
The list of HTTP endpoints affected by the first change above includes:
HTTP Endpoint | Description | Sensitive information revealed | Special (see below) |
---|---|---|---|
/_admin/v1/data_distribution |
Database-table-node mapping | Database and table names | |
/_admin/v1/databases/{database}/tables/{table}/stats |
Table stats histograms | Stored table data via PK values | |
/_admin/v1/drain |
API to shut down a node | Can cause DoS on cluster | |
/_admin/v1/enqueue_range |
Force range rebalancing | Can cause DoS on cluster | |
/_admin/v1/events |
Event log | Usernames, stored object names, privilege mappings | |
/_admin/v1/nontablestats |
Non-table statistics | Stored table data via PK values | |
/_admin/v1/rangelog |
Range log | Stored table data via PK values | |
/_admin/v1/settings |
Cluster settings | Organization name | |
/_status/allocator/node/{node_id} |
Rebalance simulator | Can cause DoS on cluster | yes |
/_status/allocator/range/{range_id} |
Rebalance simulatoor | Can cause DoS on cluster | yes |
/_status/certificates/{node_id} |
Node and user certificates | Credentials | |
/_status/details/{node_id} |
Node details | Internal IP addresses | |
/_status/enginestats/{node_id} |
Storage statistics | Operational details | |
/_status/files/{node_id} |
Retrieve heap and goroutine dumps | Operational details | yes |
/_status/gossip/{node_id} |
Gossip details | Internal IP addresses | yes |
/_status/hotranges |
Ranges with active requests | Stored table data via PK values | |
/_status/local_sessions |
SQL sessions | Cleartext SQL queries | yes |
/_status/logfiles/{node_id} |
List of log files | Operational details | yes |
/_status/logfiles/{node_id}/{file} |
Server logs + entries | Many: names, application data, credentials, etc. | yes |
/_status/logs/{node_id} |
Log entries | Many: names, application data, credentials, etc. | yes |
/_status/profile/{node_id} |
Profiling data | Operational details | |
/_status/raft |
Raft details | Stored table data via PK values | |
/_status/range/{range_id} |
Range details | Stored table data via PK values | |
/_status/ranges/{node_id} |
Range details | Stored table data via PK values | |
/_status/sessions |
SQL sessions | Cleartext SQL queries | yes |
/_status/span |
Statistics per key span | Whether certain table rows exist | |
/_status/stacks/{node_id} |
Stack traces | Application data, stored table data | |
/_status/stores/{node_id} |
Store details | Operational details |
"Special" endpoints are subject to the cluster setting server.remote_debugging.mode
. Unless the setting was customized, clients are only able to connect from the same machine as the node.
General changes
- Improved the consistency checker's log output. #42149
- Nodes that have been terminated as the result of a failed consistency check now refuse to restart, making it more likely that the operator notices that there is a persistent issue in a timely manner. #42149
- Client usernames can now be defined to start with a digit; in particular, all-digit usernames are now permitted. #42989
- To curb the spread of corrupted data, outliers are now terminated when the replicas within a range are corrupted. Previously, the leaseholder replica would terminate, regardless of which replicas disagreed with each other. #42149
Enterprise edition changes
SQL language changes
- Added the ability to run
avg()
over intervals. #42679 - Some columns (
typinput
,typoutput
,typreceive
,typsend
,typmodin
,typmodout
,typanalyze
) inpg_type
were incorrectly typed asOID
instead ofREGPROC
. This issue has been resolved. #42784 - It is now valid for
SELECT
andHAVING
to refer to ungrouped columns in the special case when the grouped columns contain the primary key of the table containing the ungrouped column. #42694 - It is now supported to specify selection target aliases as
GROUP BY
columns. Note that theFROM
columns take precedence over the aliases, which are only used if there is no column with that name in the current scope. #42694 - CockroachDB will now report a more accurate error message, hint, and error code if an error is encountered while adding a new column. #42530
Admin UI changes
- Certain web UI pages (like the list of databases or tables) now restrict their content to match the privileges of the logged-in user. #42726
- The event log now presents all cluster settings changes, unredacted, when an admin user uses the page. #42726
- Customization of the UI by users is now only properly saved if the user has write privilege to
system.ui
(i.e., is an admin user). Also, all authenticated users share the same customizations. This is a known limitation and should be lifted in a future version. #42726 - Access to table statistics are temporarily blocked from access by non-admin users until further notice, for security reasons. #42726
- Certain debug pages have been blocked from non-admin users for security reasons. #42726
Bug fixes
- The
TIMESTAMP
data type is now fully supported by the vectorized engine. Previously, an "unsupported type" error was thrown. #42576 - Previously, casting the same type into two or more different precisions/widths from a table in the same
SELECT
query would result in only the first precision specified. For example,SELECT a::decimal(10, 3), a::decimal(10, 1) FROM t
would return both results asa::decimal(10, 3)
. This PR fixes that behavior. #42578 - Fixed a bug that would produce a spurious failure with the error message "incompatible
COALESCE
expressions" when adding or validatingMATCH FULL
foreign key constraints involving composite keys with columns of differing types. #42651 - It is now possible to transfer range leases to lagging replicas. #42761
- Long running transactions which attempt to
TRUNCATE
can now be pushed and will commit in cases where they previously failed or retried forever. #42772 - Fixed a case in which CockroachDB incorrectly determined that a query (or part of a query) that contains an
IS NULL
constraint on a unique index column returns at most one row, possibly ignoring aLIMIT 1
clause. #42791 ALTER INDEX IF EXISTS
no longer fails when using an unqualified index name that does not match any existing index. Now it is a no-op. #42838- Fixed a bug with incorrect handling of Top K sort by the vectorized engine when K is greater than 1024. #42843
- Prevent internal error in some cases when a NULL literal is passed to the
OVERLAPS
operator #42878 - Fixed a bug that caused rare crashes when using built-in functions. #42674
- Other callers to
acquireNodeLease
now do not get erroneously cancelled just because the context of the first caller was cancelled. #42893 - Changefeeds now emit backfill row updates for a dropped column when the table descriptor drops that column. #42669
- Tables that are being restored or imported are no longer considered valid targets in backups and changefeeds. #42735
- Fixed a bug where selecting columns by forcing an
INTERLEAVING
index would return an error instead of returning the correct results. #42901 - CockroachDB now prevents a number of panics from the SQL layer caused by an invalid range split. These would usually manifest with messages containing encoding errors like "found null on not null column". #42859
- Fixed the row count estimate during query planning for some queries with multiple predicates in which the selectivity of one predicate was calculated using a histogram. #42962
- Previously, when executing a built-in function via the vectorized engine that returns an output type unsupported by the vectorized engine, CockroachDB could return an error instead of falling back to the row-by-row execution engine. This is now fixed. #42990
- Empty arrays are now correctly encoded and decoded over the binary protocol. #42974
- Fixed some existing caching issues surrounding role memberships, where users could sometimes see out-of-date role membership information. #43004
- Previously, attempting to parse
0000-01-01 00:00
when involvingtime
did not work aspgdate
does not understand0000
as a year. This is now fixed. #42918 - The allocator now considers stores with very close diversity scores equal (all other things being the same) and doesn't attempt to rebalance. #43077
- The allocator now considers the new store being added when looking for a target in the case of a rebalance. #43077
- CockroachDB did not previously handle date casts from
TIMESTAMP
/TIMESTAMPZ
with time attached for times before the UNIX epoch correctly. For example,'1969-12-30 01:00:00'::timestamp
would round to'1969-12-31'
instead of'1969-12-30'
. This is now fixed. #43022 - CockroachDB is now less likely to hang in an inconvenient or inoperative state if it attempts to access an external HTTP server that is blocked or overloaded. Nodes failing to shut down with
cockroach quit
were a symptom of this bug. #42539
Contributors
This release includes 38 merged PRs by 24 authors. We would like to thank the following contributors from the CockroachDB community:
- Adam Pantel (first-time contributor, CockroachDB team member)
- Oliver Tan (first-time contributor, CockroachDB team member)
- georgebuckerfield (first-time contributor)
v19.2.1
Release Date: November 25, 2019
This page lists additions and changes in v19.2.1 since v19.2.0.
- For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
- To upgrade to v19.2, see Upgrade to CockroachDB v19.2
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE
, or IMPORT
, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
SQL language changes
- Dropping a unique index that was created with
CREATE UNIQUE INDEX
no longer requires theCASCADE
option. #42413 - The vectorized execution engine now supports the
UUID
andTIMESTAMP
data types. #42414 #42416 - The vectorized execution engine now supports
IS NULL
andIS NOT NULL
operators. #42433
Command-line changes
- When restarting a node, CockroachDB now provides a hint to ensure that all
cockroach start
commands include the--join
flag. #42325
Bug fixes
- Reduced the likelihood of out-of-memory errors during histogram collection. #42370
- Fixed an out-of-memory error that could occur when collecting statistics on tables with a string index column. #42380
- Fixed some casts from
OID
toTEXT
. #42257 OUTER HASH
joins are now executed correctly via the vectorized engine whenNULL
values are present. #42438- Fixed a bug in the
crdb_internal.encode_key()
built-in function that could causeSHOW RANGE FOR ROW
statements on tables with dropped indexes to return incorrect results or an error. #42461 - Prevented memory leaks when counting rows during
BACKUP
. #42535
Performance improvements
- The
./cockroach debug check-store
command is now faster. #42150 - Improved statistics estimation during query planning for columns with many
NULL
values. #42436
Contributors
This release includes 12 merged PRs by 8 authors.
v19.2.0
Release Date: November 12, 2019
With the release of CockroachDB v19.2, we’ve made a variety of performance, resiliency, and usability improvements. Check out a comprehensive summary of the most significant user-facing changes and then upgrade to CockroachDB v19.2. You can also read more about these changes in the v19.2 blog post or watch our 19.2 release demo and overview.
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE
, or IMPORT
, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
In addition to v19.2, we're launching Cockroach University, a free, online learning tool for developers and architects who want to gain a fundamental understanding of distributed databases and deep knowledge of CockroachDB’s functionality and architecture. The first course, “Getting Started with CockroachDB,” uses videos, exercises, and quizzes to teach the key characteristics of a distributed SQL database, walking you through how to spin up a CockroachDB instance, run basic queries, and test out CockroachDB’s unique capabilities. For additional information or to register, visit university.cockroachlabs.com.
Summary
This section summarizes the most significant user-facing changes in v19.2.0. For a complete list of features and changes, including bug fixes and performance improvements, see the release notes for previous testing releases.
- CockroachCloud
- Core features
- Enterprise features
- Backward-incompatible changes
- Known limitations
- Documentation
In addition to the features below, SQL support for timestamp objects with a precision value, present in earlier testing releases of v19.2, has been reverted.
CockroachCloud
CockroachCloud will soon offer a completely self-service account creation and management workflow as well as single-region cluster creation in AWS and GCP using a credit card. Sign up for the beta program.
Core features
These features are freely available in the core version and do not require an enterprise license.
The core version of CockroachDB v19.2 uses the Business Source License instead of the Apache 2.0 License. For more information on why we changed our licensing approach and some practical questions and implications, see our blog post and Licensing FAQs. The full details of the license can be found on Github.
Feature | Description |
---|---|
Parallel Commits | CockroachDB's new optimized atomic commit protocol cuts the commit latency of a transaction in half, from two rounds of consensus down to one. Combined with transaction pipelining, parallel commits brings the latency incurred by common OLTP transactions to near the theoretical minimum: the sum of all read latencies plus one round of consensus latency. This especially lowers latency for transactions involving secondary indexes both in multi-region and single-region deployments. |
Vectorized Query Execution | CockroachDB now supports column-oriented ("vectorized") query execution for operations that are guaranteed to execute in memory on tables with supported data types. |
Bulk Import Improvements | CockroachDB now supports efficiently loading large amounts of CSV data into existing tables using the new IMPORT INTO statement.Also, for the previously existing IMPORT statement for bulk import of new tables, using as many files as you have nodes and ordering data in imported files by primary key now increases parallelism and performance. |
Data Replication Reports | Several new and updated system tables can help you query the status of your cluster's data replication, data placement, and replication zone constraint conformance. For example, you can use these reports to see what data is under-replicated or unavailable, to show which of your localities (if any) are vulnerable to data unavailability in more common failure scenarios, and to see if any of your cluster's data placement constraints are being violated. |
Multi-Use CTEs | Common Table Expressions can now be re-used multiple times in the same query via a WITH operator. |
Comprehensive Cost-Based Optimizer | All SQL queries now leverage the cost-based optimizer to choose the lowest cost plans, including DDL statements and window functions that previously leveraged the legacy heuristic planner. |
Ordering Aggregations | Non-commutative aggregate functions are sensitive to the order in which rows are processed. This order can now be controlled with an ORDER BY clause within the function argument list. |
Index Hints for UPDATE & DELETE |
It's now possible to force the use of a specific index for deleting rows and updating rows. |
Streaming with JDBC | CockroachDB now provides limited support for Postgres wire-protocol cursors for implicit and explicit transactions executed to completion. The Java JDBC driver can use this protocol to stream queries with large result sets, providing much faster performance than result paginating with LIMIT ... OFFSET . |
Transaction Latency Graphs | The SQL Dashboard in the Admin UI now provides timeseries graphs of p90 and p99 transaction latencies to complement the per-statement metrics on the Statements page. |
CREATE TABLE AS Improvements |
When using the CREATE TABLE AS statement to create a new table from a selection query, you can now specify the primary key and define column families for the new table. The latter is especially helpful for modifying the primary key of tables for geo-partitioning. Also, CREATE TABLE AS is now usable for larger tables than before. |
Adding NOT NULL Constraint |
It's now possible to add the NOT NULL constraint to an existing column, in addition to the previous ability to set this constraint during table creation. |
Local Testing Improvements | CockroachDB v19.2 includes several usability improvements to running CockroachDB locally for SQL testing and app development. First, the cockroach demo command, which starts a temporary, in-memory CockroachDB cluster with immediate access to an interactive SQL shell, now comes pre-loaded with the movr dataset and a temporary enterprise license that expires after an hour. This means users can try out enterprise features quickly before requesting a license for use in a persistent cluster.Also, the --nodes flag now lets you specify the number of in-memory nodes to create for the demo, with the new --demo-locality flag setting locality information for each node. Alternatively, the --geo-partitioned-replicas flag lets you start a 9-node demo cluster with the Geo-Partitioned Replicas data topology applied automatically to the movr database.Finally, the --with-load command lets you run movr as an active workload against the in-memory cluster. |
Cluster Startup Improvements | There are now distinct methods for starting single-node and multi-node clusters. For multi-node clusters, start each node with the cockroach start command including a --join flag and then run the one-time cockroach init command. Using cockroach start without the --join flag is now deprecated and will be removed in a future release.For single-node clusters, use the new cockroach start-single-node command to start just a single node with replication disabled and without the need for any extra commands. |
Built-in Workload Improvements | Cockroach Labs' fictional vehicle-sharing app, MovR, is now available as a sample workload using the cockroach workload CLI command. Also, the --display-every and --display-format flags now make it possible to control the frequency and format for printing per-operation statistics for several workloads. |
Data Type Improvements | DATE values in CockroachDB are now fully PostgreSQL-compatible, including support for special values (e.g., +/- infinity ). Existing dates outside of the PostgreSQL date range (4714-11-24 BC to 5874897-12-31 ) are converted to +/- infinity dates.Also, you can now use the timezone() and AT TIME ZONE functions to convert a TIMESTAMPTZ into a TIMESTAMP at a specified timezone, or vice versa. |
Interactive SQL Shell Commands | Within the interactive SQL shell, the l , dt , d , and d <table> commands now list all databases in the CockroachDB cluster, show the tables of the current schema in the current database, list the users for all databases, and show details about columns in the specified table, respectively. |
Removing Manual Splits | The new UNSPLIT AT statement removes a split enforcement on a key-value range, at a specified row in a table or index, allowing CockroachDB to merge the range as needed. |
Locking and FOR UPDATE |
Because CockroachDB only supports SERIALIZABLE isolation, locking is not required to order concurrent transactions relative to each other. However, the FOR UPDATE locking clause is now supported in selection queries as a no-op for database migration compatibility only. |
Showing Row Location | The new SHOW RANGE ... FOR ROW statement shows information about a range for a particular row of data. This information is useful for verifying how SQL data maps to underlying ranges, and where the replicas for a range are located. However, note that this is an experimental feature, and the interface and output are subject to change. |
Viewing Node Locality | It's now easy to retrieve the localities of nodes for setting zone configuration constraints via the SHOW LOCALITY statement and the cockroach node status command. |
Viewing Complete Jobs | By default, the SHOW JOBS statement lists all of the types of long-running tasks your cluster has performed in the last 12 hours. You can now use the WHEN COMPLETE option to return only after all specified job ID(s) have completed. |
Viewing Comments for Virtual Tables | Using SHOW TABLES ... WITH COMMENTS on virtual tables in the pg_catalog , information_schema , and crdb_internal schemas now returns useful comments, often with links to further documentation. |
Enterprise features
These features require an enterprise license. Register for a 30-day trial license here, or consider testing enterprise features locally using the cockroach demo
CLI command, which starts an in-memory CockroachDB cluster with a temporary enterprise license pre-loaded.
Feature | Description |
---|---|
Backup & Restore Improvements | CockroachDB now supports locality-aware backup and restore such that each node writes to and restores from files in its locality. This can reduce cloud storage data transfer costs by keeping data within cloud regions and help you comply with data domiciling requirements. Also, using SHOW BACKUP with the new SCHEMAS option adds the number of rows and the schema of the backed up table to the output, and using RESTORE with the new skip_missing_views option restores a table from a view without restoring the view's dependencies. |
Geo-Partitioning Improvements | CockroachDB v19.2 includes several usability improvements to geo-partitioning. First, it's now possible to name partitions identically across indexes of a table (e.g., us-west , us-east ) and use a single command to create zone configurations for all partitions of a given name, saving you multiple steps.Next, it's easy to retrieve the localities of nodes for setting zone configuration constraints via the SHOW LOCALITY statement and the cockroach node status command, both of which do not require an enterprise license.There are now also several ways to view the details of partitions and confirm they are in effect, from the outputs of SHOW CREATE TABLE and the table schemas listed on the Databases page in the Admin UI to the output of SHOW PARTITIONS .Finally, it's now much easier to efficiently query partitioned data; when filtering by the column directly following the partitioned prefix in the primary key, the cost-based optimizer creates a query plan that scans each partition in parallel, rather than performing a costly sequential scan of the entire table. Filtering by the partition value itself can further improve performance by limiting the scan to the specific partition(s) that contain the data that you are querying. |
Backward-incompatible changes
Before upgrading to CockroachDB v19.2.0, be sure to review the following backward-incompatible changes and adjust your application as necessary.
The
cockroach zone
CLI command has been removed. Use the equivalentCONFIGURE ZONE
andSHOW ZONE CONFIGURATIONS
SQL statements for managing replication zones instead.The
IMPORT
SQL statement no longer accepts quotes inside unquoted CSV fields.CONFIGURE ZONE
SQL statements now fail if the user does not have sufficient privileges. If the target is asystem
range, thesystem
database, or a table in thesystem
database, the user must have an admin role. For all other databases and tables, the user must have theCREATE
privilege on the target database or table.This change might be backward-incompatible for users running scripted
CONFIGURE ZONE
statements with restricted permissions. To add the necessary permissions, useGRANT
<privileges> orGRANT
<roles> as a user with an admin role. For example, to grant a user the admin role, runGRANT admin TO <user>
. To grant theCREATE
privilege on a database or table, runGRANT CREATE ON [DATABASE | TABLE] <name> TO <user>
.INT
andFLOAT
columns of less than the max width will now be returned as their own type via the binary protocol. For example, anint4
column will be returned in 32 bits over the pgwire binary protocol instead of 64 bits.The
changefeed.min_high_water
metric has been removed from the Prometheus Endpoint. For detecting high changefeed latency, use thechangefeed.max_behind_nanos
metric instead.
Known limitations
For information about new and unresolved limitations in CockroachDB v19.2, with suggested workarounds where applicable, see Known Limitations.
Documentation
Topic | Description |
---|---|
Performance Benchmarking | Added an overview of CockroachDB's performance profiles (scaling, throughput, latency), based on Cockroach Labs's extensive testing using the TPC-C industry-standard benchmark, as well as detailed instructions for reproducing our TPC-C benchmarking results at different scales. |
Multi-Region Deployment | Updated the tutorial on getting low latency reads and writes in a multi-region cluster to feature two of the most important multi-region data topologies for dramatically reducing the impact of network latency, Geo-Partitioned Replicas and Duplicate Indexes. |
Orchestration with Kubernetes | Expanded the tutorial on Kubernetes single-region deployment to cover running on Amazon's hosted EKS and naming CSR naming requirements for secure deployments. Also updated and expanded the instructions on using Helm. |
Client-Side Transaction Retries | Updated and simplified the client-side transaction logic in the Java, Python, and Go getting started tutorials and code samples. Also added pseudocode to help with the implementation of this logic in other languages as well as instructions for authors of database drivers and ORMs who would like to implement client-side retries in their database driver or ORM for maximum efficiency and ease of use by application developers. |
SQL Tuning with EXPLAIN |
Added a tutorial on how to use EXPLAIN to identify and resolve common SQL performance problems. |
Testing with MovR Dataset | Added an overview of MovR, CockroachDB's fictional vehicle-sharing dataset and application, and updated several SQL pages and examples to use the built-in MovR dataset, for example, Learn CockroachDB SQL. |
Migration from Oracle | Added guidance on migrating from Oracle, including the process of converting schema and exporting data for loading into CockroachDB. |
App Deployment on CockroachCloud | Added a tutorial on running a sample To-Do app in Kubernetes with CockroachCloud as the datastore. The app is written in Python with Flask as the web framework and SQLAlchemy for working with SQL data, and the code is open-source and forkable. |
v19.2.0-rc.4
Release Date: November 7, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
SQL language changes
- CockroachDB now uses the zig-zag join algorithm only in cases where it can benefit performance. #42186
- Added the
sql.defaults.zigzag_join.enabled
cluster setting, which can be used to control the default value of theenable_zigzag_join
session variable and, thus, whether zig-zag joins are allowed. #42182 - The error message generated when a transaction containing DDL is both partially committed and partially rolled back (
XXAOO
) now contains a link to GitHub issue #42061 where this situation is discussed further. #42090
Bug fixes
- CockroachDB now supports GSSAPI authentication from Postgres 11.5 and 12.0 clients. #42189
- Fixed a "cannot map variable to an indexed var" error for certain queries against partitioned tables. #42183
- Fixed "output columnTypes unset after planning" and "inconsistent post-processing" errors for certain queries involving
UNION ALL
. #42184 - Fixed an "invalid indexIdx" error for certain queries involving zig-zag joins. #42182
Contributors
This release includes 6 merged PRs by 3 authors.
v19.2.0-rc.3
Release Date: November 4, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
General changes
- Go 1.12.12 is now required to build CockroachDB from source. #41994
SQL language changes
- Changed the data type of the
system.reports_meta.generated
andsystem.replication_constraint_stats.violation_start
columns fromTIMESTAMP
toTIMESTAMPTZ
. #41915
Bug fixes
- Fixed a rare corruption bug when files are ingested that do not overlap with the in-memory buffer and compaction has fallen behind. #42009
- CockroachDB no longer erroneously transfers leases to replicas that are in the process of being removed, which could lead to ranges being unavailable due to an invalid lease. #42079
Doc updates
- Added a CockroachDB Performance page describing the performance profiles you can expect (scaling, throughput, and latency), based on Cockroach Labs' extensive testing using the industry-standard benchmark TPC-C. #5595
- Documented how to run the built-in
movr
workload. #5713 - Documented how to show comments for virtual tables, including links to further documentation. #5695
- Refreshed the Deploy a Python To-Do App with Flask, Kubernetes, and CockroachCloud. #5694
Contributors
This release includes 4 merged PRs by 3 authors.
v19.2.0-rc.2
Release Date: October 28, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
General changes
- Go 1.12.10 is now required to build CockroachDB from source. #41647
- Named ports in URLs are no longer supported due to a URL parsing security fix in Go 1.12.8. Use port numbers instead. #41647
SQL language changes
- Mutations under
UNION
orUNION ALL
are once again supported. #41500 - Added new
skip
andnullif
options forIMPORT ... DELIMITED DATA
, which ignore the specified number of lines at the beginning of the file and force all empty string fields to be inserted asNULL
in the imported table, respectively. #41656 - It is no longer possible to use
IMPORT INTO
with an interleaved table. This restriction is temporary and will be lifted in a future release. #41651 IMPORT
is no longer allowed during a rolling upgrade. #41733
Bug fixes
- Fixed multiple bugs relating to
COPY FROM PARENT
in replication zones. #41677 - The
system.replication_stats
table no longer incorrectly shows some ranges belonging to table partitions as over-replicated. #41700 - Fixed a bug that resulted in some queries returning an "index join must be against primary index" error. #41723
- Fixed a bug where backfills could lead to certain changefeed row updates never being emitted. #41721
- Fixed a bug that could cause changefeed row updates to be emitted at a timestamp less than or equal to an already forwarded resolved timestamp. #41714
- Fixed a bug in cloud storage sink file naming for changefeeds that could violate ordering in the presence of schema changes. #41693
Performance improvements
- Increased
BACKUP
parallelism in some cases. #41729 - Improved the performance of generating the
system.replication_*
tables for large clusters. #41757 - Improved performance of bulk ingestion. #41768
Doc updates
- Clarified that to upgrade to the next major version upgrade path for rolling upgrades, a cluster must be on a production release of the previous version, as opposed to a testing release (alpha/beta). #5644
- Documented the experimental
SHOW RANGE FOR ROW
statement, which shows information about a range for a particular row of data. This information is useful for verifying how SQL data maps to underlying ranges, and where the replicas for a range are located. #5612 - Improved the usability of the tutorial on deploying a Python To-Do app with Flask, Kubernetes, and CockroachCloud. #5694
- Updated Helm-specific instructions for Kubernetes version 1.16.0 and fixed various errors. #5657, #5659
- Documented the
FOR UPDATE
locking clause, which is supported in selection queries for database migration compatibility. #5658 - Documented using
timezone()
andAT TIME ZONE
functions to convert aTIMESTAMPTZ
into aTIMESTAMP
, or vice versa, in a specified timezone. #5614 - Clarified the required privileges for the
CONFIGURE ZONE
statement. #5609
Contributors
This release includes 32 merged PRs by 19 authors.
v19.2.0-rc.1
Release Date: October 21, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
SQL language changes
INTERVAL(6)
is now supported as an alias forINTERVAL
, for compatibility with third-party tools. #41447- Added the
pg_available_extensions
table, for compatibility with third-party tools. #41447 - The
pg_attrdef()
andpg_get_indexdef()
built-in functions no longer include type annotations on default expressions, andpg_get_indexdef()
now includes the index method in the returnedCREATE INDEX
statement, for compatibility with third-party tools. #41447
Bug fixes
- The
generate_subscripts()
built-in function now behaves properly onint2vector
andoidvector
types. #41447 - Fixed a bug causing the
cluster_logical_timestamp()
built-in function to sometimes return incorrect results. #41439 - Prevented an unlikely race condition where a WAL pending recycling could be deleted, leading to a crash. This bug was present in past 19.2 alphas/betas only, not in any earlier release. #41523
Performance improvements
- Metadata queries that use the
'string'::regclass
form are now much more efficient in the presence of a large number of tables. #41447 - The
pg_get_coldesc()
built-in function now retrieves comments more efficiently. #41447
Doc updates
- Clarified epoch-based vs. expiration-based leases in the Replication Layer architecture documentation. #5578
- Updated the list of Enterprise features. #5599
- Documented the new
--geo-partitioned-replicas
flag of thecockroach demo
command, which starts a 9-node in-memory cluster with the Geo-Partitioned Replicas data topology applied to themovr
database. This lets you quickly simulate one of the most effective data topology patterns for getting low latency in a multi-region cluster. #5587
Contributors
This release includes 12 merged PRs by 10 authors.
v19.2.0-beta.20191014
Release Date: October 14, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
SQL language changes
- The new
SHOW RANGE ... FOR ROW
statement identifies the range containing a specific row of a table or secondary index. This is particularly useful for validating the location of a specific piece of data when using partitions to control data location for improved performance or GDPR compliance. Note that this new statement is experimental and its interface and output are subject to change. #40954 - The
SHOW PARTITIONS
statement now displays inherited as well as explicitly set zone configurations for partitions. Also, it is now possible to retrieve full zone configurations via thefull_config_sql
column in thecrdb_internal.zones
table, and the newzone_id
andsubzone_id
columns in thecrdb_internal.partitions
table point to partitions' corresponding zone configurations incrdb_internal.zones
. #40493
Command-line changes
- When
vmodule
is set toexec_logs=2
to log queries executed by a node, CockroachDB-generated statements are no longer logged alongside client-generated statements. To log CockroachDB-generated statements as well, you must now useexec_logs=3
. #41149
Bug fixes
- Fixed a bug that would cause queries combining SQL mutation statements in sub-queries inside
UNION
to return invalid results or to modify the wrong rows if the transaction is forced to retry. #41307 - Fixed a bug that prevented changes to the
kv.bulk_io_write.concurrent_addsstable_requests
cluster setting from taking effect. #41195 - Fixed a memory leak that affected secondary logging (SQL audit logs, statement execution, and RocksDB logging). #41231
- Fixed a bug where zone configuration changes on tables with existing index zone configurations would not take effect unless the
num_replicas
field was also set. #41261 - Fixed a bug where zone configuration changes on secondary indexes wouldn't perform setting verification, such as constraints with lease preferences or setting only one of
min_range_size
andmax_range size
. #41216 - Fixed a rare data corruption bug in RocksDB caused by newer Linux kernels' handling of
i_generation
on certain filesystems. #41346
Performance improvements
- Stabilized performance of
IMPORT
andRESTORE
by changing the default value of therocksdb.ingest_backpressure.pending_compaction_threshold
cluster setting from 64 GiB to 2 GiB. #41263
Doc updates
- Expanded the architecture documentation to describe the new Parallel Commits protocol, which cuts the commit latency of a transaction in half, from two rounds of consensus down to one. #5394
- The tutorial on getting Low Latency Reads and Writes in a Multi-Region Cluster now features two of the most important multi-region data topologies, Geo-Partitioned Replicas and Duplicate Indexes, to dramatically reduce the impact of network latency. #5558
- Documented the new
cockroach start-single-node
cli command, which starts a single-node cluster with replication disabled for quick SQL testing or app development. #5445 - Documented how to use
SPLIT AT ... WITH EXPIRATION
SQL statement to set an expiration on a range split as well as how to remove a split enforcement viaUNSPLIT AT
. #5360 - Updated hardware recommendations in the Production Checklist. #5472
- Clarified the benefit of merge joins over hash joins. #5464
- Updated the recommendations for improving
IMPORT
performance. #5452 - Documented the
ALTER COLUMN ... SET NOT NULL
SQL statement, which adds a non-null constraint to an existing column. #5456 - Documented how to use the
pg_collation_for()
built-in function, or its alternative syntax formCOLLATION FOR
, to return the locale name of a collated string. #5576 - Documented how to use
SHOW BACKUP SCHEMAS
SQL statement to list the schemas of enterprise backups alongside their contents. #5455 - Updated the SQL statements that are allowed as row sources for table expressions.
- Documented how to use the
SHOW JOBS WHEN COMPLETED
SQL statement to list only just that have completed. #5395 - Documented the
skip_missing_views
option for theRESTORE
SQL statement, which restores a table with a view but does not restore the view's dependencies. - Documented the
--display-every
and--display-format
flags for thecockroach workload
cli command, which control the frequency and format of per-operation statistics. #5424 - Documented the naming requirements for CSR names when orchestrating CockroachDB with Kubernetes. #5422
- Documented the
--watch
flag for thecockroach sql
cli command, which repeats the SQL statements specified with--execute
or-e
until a SQL error occurs or the process is terminated. #5543 - Documented how to use the JDBC driver to retrieve large data sets in chunks using cursors. #5544
- Clarified that
CHANGEFEED
cursors must be in quotes. #5529 - Documented how to show the locality of a node using the
crdb_internal.locality_value()
built-in function. #5540 - Documented how to reference a common table expression (CTE) multiple times in a single query, using a
WITH
operator. #5485
Contributors
This release includes 57 merged PRs by 23 authors.
v19.2.0-beta.20190930
Release Date: September 30, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
In addition to bug fixes and various general, enterprise, SQL, and command-line enhancements, this release includes several major highlights:
- Locality-aware backup and restore: You can now back up or restore a subset of a table's data according to zone configurations (e.g., send your EU backup partition to an EU cloud storage location).
- Import performance improvements: Internal improvements have been made to
IMPORT
to enable more efficient loading of large data sets. - Show partitions using SQL: CockroachDB now supports the
SHOW PARTITIONS
statement, which can be used to identify partitions. It also enhancesSHOW CREATE TABLE
to show constraints as described. - Vectorized Execution: Queries are now automatically faster for users. This work expands upon the SQL vectorized execution prototype, and allows it to be used in production. Note that this does not cover the long-tail of features necessary to use vectorization on all queries. Vectorized execution is turned on by default for a subset of queries and defaults back to DistSQL execution for unsupported queries.
- Improve Streaming Behavior on JDBC Driver: PostgreSQL wire protocol cursors have been implemented. As a result, PGJDBC is able to use streaming behavior to return pagination-related results. Limitations still apply. For more information, see Build a Java App with CockroachDB.
- Improve partitioning user experience: CockroachDB now prevents users from specifying a non-partitioned secondary index on a partitioned table by default to improve performance. Users will have the ability to override this default as it may be necessary in some rare use cases. Additionally, users can now specify the same name for the partition for tables and for all corresponding indexes.
- Improve setting of zone configs via SQL:
ALTER TABLE
now returns an error when users select conflicting or impossible zone constraints due to user error. - Make it easy to focus queries on partitioned data: CockroachDB now automatically limits a query to only the applicable partition without having to specify the particular partition for a geo-distributed table. This prevents the query from conducting a full table scan.
- Better insight into constraint conformance: You can now tell if your cluster is in a healthy state based on the status of conformance to constraints like zone configurations, lease preferences, and replica diversification.
General changes
- Replicas are now added using a Raft learner and go through the normal Raft snapshot process to catch them up, eliminating technical debt. No user-facing changes are expected. #39034
- Atomic replication changes are now enabled by default. #40464
- RocksDB
INFO
logs are now stored with CockroahDB debug logs. #40681
Backward-incompatible changes
- Quotes are no longer accepted inside unquoted CSV fields. #40424
CONFIGURE ZONE
statements now fail if the user does not have sufficient privileges. If the target is asystem
range, thesystem
database, or a table in thesystem
database, the user must have an admin role. For all other databases and tables, the user must have theCREATE
privilege on the target database or table. This change might be backward-incompatible for users running scriptedCONFIGURE ZONE
statements with restricted permissions. To add the necessary permissions, useGRANT
<privileges> orGRANT
<roles> as a user with an admin role. For example, to grant a user the admin role, runGRANT admin TO <user>
. To grant theCREATE
privilege on a database or table, runGRANT CREATE ON [DATABASE | TABLE] <name> TO <user>
. #40879
Enterprise edition changes
- Added support for client certificates for Kafka changefeed authentication. #39832
- Added support for locality-aware backups across multiple backup destinations. #39250
- Added support for displaying creation statements of relations stored in a backups. #39323
- Table statistics for the tables that are being backed up are now also backed up. #39644
- Corrected behavior for unmatched quotes in enclosed fields. #40960
- Added
auth=implicit
setting, which removes access key requirement when listing S3 files for bulk I/O operations. #40836
SQL language changes
- Zone configurations now error out when they conflict with each other. #39081
- Renamed
SHOW EXPERIMENTAL_RANGES
toSHOW RANGES
command, as it is no longer an experimental feature. #39153 - Added the
SHOW RANGES FROM DATABASE
command. #39194 - Added partial support for row limits during portal execution in
pgwire
. #39085 - Added the
SHOW LOCALITY
command. #39058 - Added the
SHOW PARTITIONS
command. #39053 CREATE VIEW
now supports all queries supported by the optimizer, including those containing correlated subqueries. #39169- Mutations are no longer allowed inside views. #39169
- Added the
COLLATION FOR
syntax andpg_collation_for()
built-in function. #39198 - Implemented the
timezone()
built-in andAT TIME ZONE
syntax, which allows converting atimestamptz
into atimezone
at a specified timezone, and converting atimestamp
into atimestamptz
at a specified timezone. #39186 - The new
vectorize_row_count_threshold
cluster setting controls when the vectorized execution engine is used. #39329 - Added a
VEC
option to theEXPLAIN
command, which shows detailed plan information for vectorized plans. #39280 - Partition names can now be reused between different indexes on the same table. #39332
- Columns without an explicit default value now support foreign keys with the
SET DEFAULT
action, in the same way as theSET NULL
andSET DEFAULT NULL
cases. #39136 - Added
SHOW JOB <jobid>
,SHOW JOB <a_expr>
, andSHOW JOBS <selectclause>
. #39324 - Added
SHOW JOB WHEN COMPLETE <a_expr>
andSHOW JOBS WHEN COMPLETE <selectclause>
. #39457 IMPORT INTO
now cleans up any imported rows if it fails. #39459- Updated
SHOW CREATE
to display information about partition and table zone configurations. #39102 - Explicitly closing portals over the wire after partial use is now supported. #39639
- Added progress tracking for new implementation of
IMPORT
. #39646 - CockroachDB now supports an optional frame exclusion clause in the specification of window frames. #28262
- Enabled collection of histograms by default during automatic statistics collection. #39743
- The
force_retry()
function, which previously required admin privileges, is now available to all users to test their transaction retry loops. #39246 - When
DROP CONSTRAINT
is executed in a transaction on a validated constraint, it no longer takes effect immediately, and subsequent writes will now fail if they violate that constraint. This is to enforce consistency when dropping constraints asynchronously in the schema changer. #39648 - The
ALTER PARTITION
statement now supports applying a zone configuration to all the partitions of a table and its indexes that share the same partition name. The syntax for this isALTER PARTITION <partition name> OF INDEX <table name>@*
. #39750 - CockroachDB now supports the
FOR UPDATE
modifier onSELECT
statements for compatibility with third-party tools. It has no effect becauseSERIALIZABLE
transactions do not require explicit write locks to maintain consistency. However, this means thatSELECT FOR UPDATE
cannot be used as an advisory exclusive lock in CockroachDB, as used by some tools.#40206 SHOW PARTITIONS
now shows the full zone configuration statement used to configure the partition. #39814- Added the
pg_my_temp_schema()
built-in for Postgres compatibility. #40194 - Added the overlaps (
&&
), contains (@>
), and contained_by (<@
) operators for arrays. #40194 - Added
pg_get_function_identity_arguments()
built-in. #40194 - Added the
width_bucket()
builtin function. #39263 - Added range size information to
SHOW RANGES
. #40281 - Added replica locality information to
SHOW RANGES
. #39813 - Reordered columns in
SHOW RANGES
output #40501 SHOW CREATE TABLE
now warns users if they have a partitioned table that does not contain zone configurations #40475- Added the
replication_constraint_stats
,replication_stats
, andreplication_critical_localities
system tables. These tables contain information about replication status, constraint conformance, and critical localities. #40625 - Common table expressions that are used a single time no longer create an optimization fence. #40673
EXPLAIN
now has additional output that shows if a query will run with the vectorized execution engine, and if the query will be distributed across nodes. #40652- Column references are no longer allowed in
ROWS
andRANGE
clauses in window functions. #40832 - Added the
pg_function_is_visible
andpg_get_function_result
built-in functions. #40902 - Added parsing for
FOR UPDATE [ OF tables ]
form. #40962 - Mutations under
UNION
orUNION ALL
are now disallowed;WITH
should be used on top of the union operation instead. This restriction is temporary and will be lifted in a future release. #40975 - Added support for the syntax for
oid
wrapper arrays, likeREGCLASS[]
. #40949 EXPLAIN ANALYSE
can now be used as an alias toEXPLAIN ANALYZE
. #41090SHOW PARTITIONS
now displays inherited zone configurations. #40493- Added the
zone_id
andsubzone_id
columns tocrdb_internal.partitions
. These columns form a link to the zone configurations incrdb_internal.zones
that apply to the partitions. #40493 - Renamed the
config_yaml
,config_sql
, andconfig_proto
columns incrdb_internal.zones
toraw_config_yaml
,raw_config_sql
, andraw_config_proto
. #40493 - Added the columns
full_config_sql
andfull_config_yaml
to thecrdb_internal.zones
table. #40493 - Added
FOR ROW
clause toSHOW RANGES
to show range information for a particular row. #40954
Command-line changes
cockroach demo
now accepts a--nodes
flag, which permits customization of the size of the demo cluster. The default remains at1
. #39088cockroach demo
now accepts a--demo-locality
flag, which makes it possible to set node localities. #39454cockroach demo
without an argument is now equivalent tocockroach demo movr
. The previous behavior (no pre-defined dataset) is still available viacockroach demo --empty
. #39298cockroach start
now writes the client URL to the file specified via--listen-url-file
as soon as the server is ready to accept connections. This also happens before the server detaches from the terminal when--background
is specified. #39300- Using
cockroach start
without--join
is now deprecated and this mode of execution will be removed in a later version of CockroachDB. Consider usingcockroach start-single-node
instead or combinecockroach start
withcockroach init
. #28495 - The advisory/informative check performed by
cockroach start
on the validity of addresses contained in the node certificate is now relaxed to focus on the advertised node address, and to tolerate cases when the cert contains an IP address but a hostname is specified as advertised address. #39166 - Added locality information to the output of
cockroach node status
. #39059 - Removed the deprecated
cockroach zone
CLI commands. Please use the equivalentSHOW ZONE CONFIGURATIONS
andCONFIGURE ZONE
commands in a SQL client. #39177 - Added a new
cockroach start-single-node
command to start single-node clusters with replication disabled. #28495 - The
cockroach user
sub-commands are now deprecated. Users and roles can be managed using SQL statements instead. This functionality will be removed in the next release. #39297 cockroach start
now accepts a--cluster-name
flag. If specified, it must match when a node connects to a cluster. This feature is meant for use in combination with the cluster ID, which is still checked in any case, to protect newly created nodes (that do not have a cluster ID yet) from joining the wrong cluster. To introduce a cluster name into an existing cluster without one, the new flag can be temporarily paired with--disable-cluster-name-verification
. #39270cockroach start
now accepts a--sql-addr
flag, which makes it possible to accept connections by clients on a separate TCP address and/or port number from the one used for intra-cluster (node-node) connections. This is aimed to enable firewalling client traffic from server traffic. #39305cockroach workload run
andcockroach workload init
now support loading initial table data usingIMPORT
. #35312cockroach workload
now supports additional command-line parameters to customize the output, to facilitate the integration with third-party testing tools:- For tools that wish to observe the metrics more frequently than every second, a new flag
--display-every
is now supported, which can be used to specify the period between metric reports. This applies to both the JSON and textual output. - For tools that require a different output format than the default, a new
--display-format
argument is supported. For now only the formats "simple" (original output format) and "incremental-json" (RFC3339 timestamps, no summary row) are supported. #37929
- For tools that wish to observe the metrics more frequently than every second, a new flag
- Added
d
,dt
,du
,l
metacommands to the cockroach sql interactive shell. #39141 - Added default cluster locality topologies for multi-node
cockroach demo
clusters. #40221 - Errors in
cockroach demo
are no longer logged to the command-line interface. #40196 - Added
d
client-side command with no arguments as an alias fordt
, for more ease of adoption bypsql
users. #40309 - Added a MovR workload to run with
cockroach demo
. #40271 - Added location information for default localities to populate Admin UI for
cockroach demo
. #40229 cockroach demo
now attempts to contact a license server to obtain a temporary license for demo usage. #40273cockroach demo
now enables telemetry for demo clusters. You can opt out of telemetry by setting theCOCKROACH_SKIP_ENABLING_DIAGNOSTIC_REPORTING
environment variable. For more information, see Diagnostics Reporting. #40273- Deprecated the
--init
flag on theworkload run
command. Useworkload init
instead. #40426 - Added support for a new
DELIMITED
data format to theIMPORT
statement. #40482 - Added an option for
cockroach demo
to automatically apply the geo-partitioned replicas topology to themovr
dataset using the--geo-partitioned-replicas
flag. #40355 cockroach sql
now supports the--watch
flag. When used to specified a duration, the client will repeat the statement(s) specified by-e
until an error occurs. This is intended for simple monitoring scenarios during development and testing. #40594- Added support for redirecting license acquisition to a replacement server (e.g., a suitably configured HTTP proxy), with the environment variable
COCKROACH_DEMO_LICENSE_URL
. This enablescockroach demo
with enterprise features in firewalled network environments. #41029
Admin UI changes
- Metrics about p90 and p99 percentiles of per-application average transaction time are now exposed in the Admin UI. #39429
Bug fixes
- Fixed a potential infinite loop in queries involving reverse scans. #39084
- Upgraded the
grpc
library to fix connection state management bug. #39041 - Properly initialize proposal quota tracking to prevent quota leak which can hang imports or other AddSSTable operations. #39135
crdb_internal.ranges
can now be used inside views. Note that such views can become invalid in future releases ifcrdb_internal.ranges
changes. #39195- Prevented panic due to recording into finished tracing spans caused by acknowledging an earlier failed re-proposal when a later re-proposal exists. #39203
- CockroachDB now properly computes the default port number if an IPv6 address is provided without port number with
--locality-advertise-addr
. #39334 - Unary negatives in constant arithmetic expressions are no longer ignored. #39245
pg_catalog
now shows correct type lengths forINT
andFLOAT
. #39337- Fixed a rare panic (message:
"raft group deleted"
) that could occur during splits. #39571 - Fixed a panic due to incorrect type inference of some
ARRAY(...)
expressions. #39469 - Zone configurations now propagate to non-gossiped system tables. #39638
cockroach dump
now works properly when handlingINTERLEAVED
tables, printing them outside ofCREATE TABLE
statements. #39486- Fixed internal errors generated during execution of some complicated cases of correlated subqueries. #39818
- Using
ORDER BY
inside aggregation functions is now fully supported. #40242 - Lookup joins now spill to disk if the index lookups return more rows than can be stored in memory. #40208
- Fixed an internal planning error that could occur when a
DISTINCT
orGROUP BY
expression was contained in a subquery. #40468 - Fixed an optimizer panic when building array access expressions. #40469
- Fixed faulty error when trying to delete a table and an unrelated index in the same transaction. #40516
ordinal_position
ininformation_schema.columns
now matchespg_attribute.attnum
after a column is dropped. #40485- Fixed fatal error with message
replica descriptor of local store not found
. #39796 - Intents in a read's uncertainty interval are now considered as uncertain as committed values. This removes the potential for stale reads when a causally dependent transaction runs into unresolved intents from a causal ancestor. #40600
- Fixed a cluster migration bug that could occur in clusters running a mixture of v19.1 and v19.2. This bug was signaled by messages of the form:
X at key /Table/54 failed: unexpected value: ...
Affected clusters should be updated to 19.2 or, if 19.1 is preferred, recreated from backup. #40464 - Fixed behavior of
SERIAL
columns on mixed-case tables whenexperimental_serial_normalization=sql_sequence
. #40965 width_bucket
function no longer errors on 0-length arrays #40626- Fixed an error that could occur when the optimizer creates a plan with a lookup semi- or anti-join nested inside another join. #40669
- Fixed a memory corruption error in the optimizer that could cause stack overflow. #40644
- The
cockroach
commands are now able to connect to a server with the environment variableCOCKROACH_URL
. #40824 - Index hints used with
DELETE
andUPDATE
are now taken into consideration. #40715 - Fixed an issue with creating table indexes when the server is running on Windows and the store is on a file system that does not support hard links (e.g., FAT32 or network share) #40875
- Restore correct result generation for queries with index disjunctions on tables with multiple column families. #40898
- Fixed a stack overflow that could occur via use of
WITH
. #40937 - Detailed crash reports now always appear in the main log file. If SQL audit reporting or statement logging was activated, crash reports (i.e., "panic messages") could previously be reported in the wrong file. #40941
- Users must now have admin privileges to cancel, pause, or resume jobs. #40978
- CockroachDB now properly removes excess secondary log files (SQL audit logging, statement execution logging, and RocksDB events). #41020
- CockroachDB now avoids the internal re-use of Replica objects to fix the following crashes:
- #38772 "
found rXXX:{-} [, next=0, gen=0?] in place of the RHS
" - #39796 "
replica descriptor of local store not found in right hand side of split
" - #40470 "
split trigger found right-hand side with tombstone
" - #40257 "
snapshot widens existing replica, but no replica exists for subsumed key
" #40892
- #38772 "
- Fixed panic in
cockroach workload movr run
. #41072 - Resolved an issue with server hanging on
cockroach quit
on Windows. #40994 - The
cockroach zip
,cockroach node
andcockroach user
cockroach
commands now work properly when thedefaultdb
database is manually dropped and the connection URL does not specify a database.cockroach user
is deprecated in 19.2. #41121 cockroach workload init ycsb
now works with--data-loader=import
. #41071- Fixed bug where zone configurations on indexes could leak into configurations on partitions. [#41089][#41089
- Statements inside
[ ... ]
can no longer refer to outer columns or common table expressions. #41158 - The SQL for non-
SELECT
common table expressions now shows up inEXPLAIN
output. #41159 - Fixed bug that prevented changes to
kv.bulk_io_write.concurrent_addsstable_requests
from taking effect. #41195 - Fixed a memory leak that affected secondary logging (SQL audit logs, statement execution, and RocksDB logging). #41231
Performance improvements
- Sped up file-writing during bulk-ingestion. #37528
- Snapshots sent between replicas are now applied more performantly and use less memory. #38932
- Raft entries no longer wait to be applied to the RocksDB storage engine before signaling their success to clients, they now only wait until they are committed in their Raft log. #38954
- Adjusted load-based splitting QPS threshold to avoid over-splitting. #39687
IMPORT
now writes unsorted data directly, reducingIMPORT
's overhead (but potentially increasing RocksDB compaction overhead). #39487- Improved performance for mutation statements with
RETURNING
that are not inside an explicit transaction. #40871
Doc updates
- Added an Oracle migration guide, which guides users through the process of converting their schema and then exporting their data so they can load it into CockroachDB via
IMPORT
. #5902 - Added an overview of MovR, CockroachDB's fictional vehicle-sharing dataset and application, and updated several SQL pages and examples to use the built-in MovR dataset, for example, Learn CockroachDB SQL. #5075, #5216
- Added a tutorial on how to use
EXPLAIN
to identify and resolve common SQL performance problems. #5178
Contributors
This release includes 724 merged PRs by 53 authors. We would like to thank the following contributors from the CockroachDB community:
- Arseni Lapunov
- Elliot Courant
- HyeonGyu Lee (Vazrupe) (first-time contributor)
- Jaewan Park
- Kevin Barbour (first-time contributor)
- Niels Hofmans (first-time contributor)
- Shishir Jaiswal (first-time contributor)
- Taufiq Rahman (first-time contributor)
- Zeming YU (first-time contributor)
- 贾德星 (first-time contributor)
v19.2.0-alpha.20190805
Release Date: August 5, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Enterprise edition changes
- Added a
skip_missing_views
option forRESTORE
to skip restoring views which cannot be restored because their dependencies are not being restored at the same time. #38597
SQL language changes
- Added support for a table reader that performs a loose index scan over the underlying table. The index scan table reader uses information about the index being scanned to skip unnecessary rows while scanning the table, allowing for some optimizations to be used for some types of queries. #38216
- Enabled automatic detection of compression schemes on filenames in
IMPORT
that have parameters after the path. #38521 - Added
check_constraints
table to theinformation_schema
database. #38545 - Added vectorized support for the
IN
selection operator. #38546 - Made some
pgwire
error codes for binary parameter decoding better match Postgres. #38365 - Added support for projection of the
IN
operator. #38613 EXPLAIN
returns more information about lookup joins. #38748- The first statement of a transaction will no longer return a transaction retry error if it is an
UPDATE
orDELETE
(this was already true forINSERT
). #38668 - Non-fatal errors updating jobs while dropping a table are now ignored. #38617
- Added support for
NOT VALID
option for Foreign Key constraints. #38663 - Added cluster setting
sql.stats.histogram_collection.enabled
to enable collection of histograms during statistics collection. #38812 - Common Table Expressions (CTEs) may now be referenced from multiple locations in a query. #38670
CREATE TABLE AS
now allows user-defined primary keys and column families. #38904, #38589EXPLAIN (OPT,ENV)
now returns a URL with the data encoded in the fragment portion. Opening the URL shows a page with the decoded data. Note that the data is processed in the local browser session and is never sent out. #38973
Command-line changes
cockroach dump
will now more clearly refer to issue 37768 when it encounters a table with no visible columns, which (currently) cannot be dumped successfully. #38725- CockroachDB now requires square brackets to specify IPv6 addresses in
--join
,--host
,--listen-addr
and similar flags, for example--listen-addr=[::1]
. The syntax without square brackets had been deprecated since CockroachDB 2.1. #38924
Admin UI changes
- Dashboard graphs now offer a "Last 2 days" option in the time selector. #38852
- Corrected the units displayed for the
MVCC Intent Age
and theMVCC GC Bytes Age
fields in the Range Status page. #38283
Bug fixes
- The
ALTER PARTITION x OF INDEX y CONFIGURE ZONE USING
syntax no longer erroneously modifies the entire index's zone configuration. #38423 - Fixed an issue that prevented restoring some backups if they included tables that were partitioned by columns of a certain types while also interleaved by child tables. #38341
- Restoring multi-line CLI histories in the SQL shell no longer produces terminal anomalies when the cursor is on the last line of the screen. #38505
- Fixed a panic where generators in an aggregate function were not appropriately rejected. #38554
- Handled
database_name
and composite primary key forUNSPLIT ALL
. #38338 - Ranges consisting of only one row (and historical versions of that row) are now correctly up-replicating. #38529
- Nodes that have been down now recover more quickly when they rejoin, assuming they weren't down for much more than the value of the
server.time_until_store_dead
cluster setting (which defaults to 5 minutes). #38484 cockroach sqlfmt
no longer stripsORDER BY
from aggregates. #38623- Corrected issue that caused certain plans that contained both offsets and limits to return more rows than desired. #38667
- Fixed a bug that prevented inverted indexes from being created on JSON columns containing
NULL
values. #38738 - Transaction retries in schema changes are again handled correctly. #38668
- On macOS, the server process can once again start in environments where the
kern.maxfilesperproc
sysctl has been lowered. #38472 - Fixed a bug where dropping an index that a foreign key depends on could cause an inconsistent table state if the schema change were rolled back. #38783
- Fixed an issue where Cost-based optimizer-planned upserts that used lookup join that were run during column mutations on the table being upserted into could cause crashes or other issues. #38637
- Fixed incorrect results, or "unordered span" errors, in some cases involving exclusive inequalities with non-numeric types. #38880
- Prevented OOM conditions during lookup joins between tables with a very large n:1 relationship. #38837
sum_int()
now errors on overflow instead of silently wrapping. #38921- Fixed bug in the optimizer causing a bad index for lookup join in some cases. #39016
- Backfills are no longer performed when a user explicitly sets the default value of a new column to be
NULL
. #38910
Performance improvements
- Introduced new concurrent Raft proposal buffer, which increases the degree of write concurrency supported on a single range. #38343
- Grow stack prior to decoding grpc BatchRequest messages. #38107
- Batched raft entry application and coalesce writes to applied state for the batch. #38568
- Improved performance of the
BACKUP
process. #37887
Build changes
- Building CockroachDB from source now requires node version 10. This was previously implicitly required. Now the requirement is explicit. #38955
- Building CockroachDB from source now requires Go 1.12 or higher. #38379
Doc updates
- Expanded single-region Kubernetes tutorial to cover running on Amazon’s hosted EKS. #5063
- Build an App with CockroachDB examples now include C++ and Clojure application examples for secure clusters. In addition, the client-side transaction logic has been updated in the Java, Python, and Go code examples. #5028, #5084, #5085, #5088
- Updated Common Errors document with specific error strings to aid user searches. #5033
- Documented a known limitation about hanged requests to a restarted node that needs Raft snapshots. #5018
- Updated the
BACKUP
andRESTORE
docs to specify that thesystem.users
table is not included with a backup. To restore the table, you must explicitly include it. #5072
Contributors
This release includes 285 merged PRs by 33 authors. We would like to thank the following contributors from the CockroachDB community:
- Arseni Lapunov (first-time contributor)
- Elliot Courant (first-time contributor)
v19.2.0-alpha.20190701
Release Date: July 1, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
This release contains enhancements and bug fixes to the 19.2.0 series, including:
- Experimental support for using
IMPORT INTO
to ingest CSV data into an existing table. Usage:IMPORT INTO t(a, b) CSV DATA ('gs://acme-co/employees.csv')
. - Adding non-null constraints to existing columns with
ALTER TABLE ... SET NOT NULL
. - Various updates to improve Postgres compatibility.
For more information, see below.
Backward-incompatible changes
- Integer and float columns of less than the max width will now be returned as their own type via the binary protocol. For example, an
int4
column will be returned in 32 bits over the pgwire binary protocol instead of 64 bits. #37295
General changes
- Removed
kv.range_merge.manual_split_ttl
cluster setting. #38004 - Go 1.12.5 is now required to build CockroachDB from source.
SQL language changes
- Added experimental support for using
IMPORT INTO
to ingest CSV data into an existing table. Usage:IMPORT INTO t(a, b) CSV DATA ('gs://acme-co/employees.csv')
. #37451 - Implemented SQL support for timestamp objects to have an optional precision value. #37920
- The logging format for statement execution (as enabled by
sql.trace.log_statement_execute
) and audit logs (as enabled byALTER TABLE .. EXPERIMENTAL AUDIT SET ...
) is extended with one column that reports internal execution retries performed automatically by the server. The valuezero
is logged for the first retry attempt and thus corresponds to statements directly received from a client. #38028 - Added
WITH EXPIRATION
option forSPLIT AT
. #38004 - The
DISTINCT
keyword is now rejected when used in a window function. It was previously accepted, but ignored. #37912 - Added
UNSPLIT ALL
statement. #38023 - The
relam
column of thepg_catalog
tablespg_class
andpg_index
is now filled in correctly. #38164 - CockroachDB now supports the special case
set timezone = 'utc'
as a special alias forset timezone = 'UTC'
. The other time zone names are still case-sensitive as previously, pending resolution of issue #36864. #38195 - Added support for
ALTER TABLE ... ALTER COLUMN ... SET NOT NULL
, which adds a non-null constraint to an existing column. #37554 pg_get_constraintdef
now outputsCHECK
constraints exactly the same way that Postgres does. #38318- The
format_type
Postgres builtin now properly respects its second argument. #38318 SHOW CREATE TABLE
now prints foreign keys more like Postgres does. #38318- The
pg_type_is_visible
builtin from Postgres is now implemented. #38318 - Casting an
int2vector
to a string now produces a Postgres-compatible result. #38318
Command-line changes
- CockroachDB will now print out an error message and an informative hint if the time zone database is unusable. #38197
Admin UI changes
- The snapshots graph on the Replication page in the Web UI now includes learner snapshots. #38147
- Removed version information from login screen. #38140
- Added
Implicit Transaction
flag to Statements and Statement Details page. #36341 - Checking the "skip should queue" checkbox in the Manual Enqueue Range advanced debugging page now works for the GC Queue. #38249
Bug fixes
- Fixed help text that erroneously labeled Encryption at Rest as experimental. #38228
- The
max_retries
column in the statement statistics now properly displays the maximum number of times a statement was retried. #38028 - Previously, due to a bug when estimating result set sizes in the optimizer, queries involving
int
ranges that were very large could result in poor plans being generated. #38036 - Fixed a bug where MVCC value at future timestamp is returned after a transaction restart. #38085
- The YCSB workload will no longer ignore the
--db
option. #38236 - Localities in the
crdb_internal
tables were displayed JSON objects instead of strings. This format stripped them of their order, which is important in localities. This string format now exactly matches what is used in the command line for setting localities. #37944 - Fixed an incorrect type mismatch error when empty array values are used as default values (and potentially in other contexts). #38294
- Fixed a panic that could occur when decoding decimals as query parameters. #38304
- Nulls are now correctly handled by
MIN
,SUM
, andAVG
aggregate functions when used as window functions. #38110 - Prevented a possible panic during
JSONB
binary query parameter parsing. #38346 - Prevented a possible panic during
INET
binary query parameter parsing. #38346 - Prevented a possible panic during
VARBIT
binary query parameter parsing. #38346 - Prevented a possible panic during array binary query parameter parsing. #38346
- Fixed possible deadlock when storage engine write fails. #38170
- Fixed potential reappearance of deleted timeseries data, which could trip consistency checker. #38170
- Removed dependency on
sync_file_range
on Linux platforms on which it returnsENOSYS
, such as WSL (Windows Subsystem for Linux). #38170
Build changes
- A recent
libedit
is now bundled with CockroachDB, which fixes some line editing bugs in the CockroachDB console. On platforms that include libedit as part of the base system, like macOS and FreeBSD, CockroachDB no longer links against the system libedit. #32623
Security
- Only check
CN
on first certificate in file. #38163
Contributors
This release includes 125 merged PRs by 30 authors. We would like to thank the following contributors from the CockroachDB community:
- Utkarsh Gupta (first-time contributor)
v19.2.0-alpha.20190606
Release Date: June 6, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
In addition to various enhancements and bug fixes, with this first alpha release of the v19.2.0 series, we would like to highlight the following changes:
Parallel Commit: The parallel commit feature is a major optimization that halves the latency of distributed transactions, including those caused by secondary indexes. Previously, transactions would involve two synchronous rounds of replication consensus - one for write operations performed in parallel via transaction pipelining and then another for the commit. Now, replication for the commit runs in parallel with other write operations, avoiding an extra synchronous round trip and dramatically increasing overall transaction performance. This is achieved via a new "implicit" commit status that is distributed yet atomic and durable and that resolves to an "explicit" centralized status quickly for the efficiency of conflicting transactions. Detailed documentation is coming soon; in the meantime, see the Parallel Commit RFC.
Business Source License (BSL): This release uses the Business Source License instead of the Apache 2.0 License. For more information on why we changed our licensing approach and some practical questions and implications, see our blog post and Licensing FAQs. The full details of the license can be found on Github.
Backward-incompatible changes
- Removed the deprecated
changefeed.min_high_water
metric; usechangefeed.max_behind_nanos
instead. #36915
General changes
- Added the
kv.range_merge.manual_split_ttl
cluster setting. #37728 - CockroachDB now uses the Business Source License instead of the Apache License. See our blog post for details. #38007
Enterprise edition changes
- You can now alter the zone configuration for a secondary index partition using the syntax
ALTER PARTITION OF INDEX <tablename>@<indexname> CONFIGURE ZONE ...
. #36868
SQL language changes
- Zigzag joins will now be planned by the optimizer in certain cases. They were previously available via the
experimental_enable_zigzag_join
session variable and can still be disabled by setting that value tofalse
. #35911 - The
LATERAL
keyword in aFROM
clause is now supported. #36613 - The
information_schema.columns
table now gives more conformant output forvarbit
,char
,timestamp
, andtime
types. #36598 - Extended the list of advertised OIDs in
pg_type
to include additional array-related types, such as_int2vector
,_regproc
, etc. #36598 - CockroachDB now displays correct array type
oids
foroidvector
andint2vector
types in thepg_type
table, for better PG compatibility. #36598 - CockroachDB now uses the Postgres-compatible name when auto-generating a column alias for a
CAST
expression. #36598 - Locale names in
COLLATE
clauses now have dash characters mapped to underscore when printed as part of SQL syntax. #36598 - Parallel statement execution has been removed because the performance improvement it provided was absorbed by transactional pipelining, which doesn't require client buy in. The
RETURNING NOTHING
syntax will still be accepted but becomes a no-op. #35959 - Only
SELECT
,INSERT
,UPDATE
,UPSERT
,DELETE
,SHOW
,EXPLAIN
are now supported as data sources using theSELECT ... FROM [ ... ]
syntax. #36977 - The
atttypmod
column of thepg_attribute
catalog table is now supported. #37230 - Dates are now fully Postgres-compatible, including support for sentinel values (
+/- infinity
) and the Postgres date range (4714-11-24 BC
to5874897-12-31
). Existing dates outside of this range will be converted to the+/- infinity
dates. #36938 - Added an overload for
now()
that returns a date. #37284 - Added supported for the empty
default_tablespace
session variable for Postgres compatibility. #37333 - Permitted casting empty arrays to any array type. #37356
- Removed the legacy batch-by-batch index backfiller, as well as the
schemachanger.bulk_index_backfill.enabled
cluster setting that could be disabled to activate it. #37213 - Removed the
transform
option fromIMPORT
. #37338 - Added a new
locality_value
built-in function that returns the value of the locality key given as its argument. #37369 - CockroachDB now returns
NaN
instead of an error during invalid decimal operations. #36964 - Foreign keys that are added to an existing table using
ALTER TABLE
will now be validated for existing rows, with improved performance compared to runningADD CONSTRAINT
followed byVALIDATE CONSTRAINT
previously. #37433 SHOW COLUMNS ... WITH COMMENTS
now supports printing out column comments using the optional phraseWITH COMMENT
, e.gSHOW COLUMNS FROM mytable WITH COMMENT
. #37174SHOW DATABASES WITH COMMENTS
now supports printing out database comments using the optional phraseWITH COMMENT
, e.gSHOW DATABASES WITH COMMENT
. #37175CREATE TABLE
statements now always trigger automatic statistics collection on a newly created table. #37611- Added the
split_time
column tocrdb_internal.ranges
andcrdb_internal.ranges_no_leases
. #37610 - Corrected interval math when multiplying or dividing by floats or ints. #37744
- Added the
ALTER TABLE/INDEX ... UNSPLIT AT ...
statement. #37603 - Added support for the time value
24:00
and variants. #37898 - CockroachDB now reports a dedicated SQL error code in the particular situation where a multi-statement transaction is committed but one of the schema change (DDL) operations failed. As in previous versions, CockroachDB leaves the transaction partly committed and partly aborted in that case; the new error code aims to facilitate the recognition of this case in client code. #37791
- Added support for a
SHOW INDEXES FROM DATABASE
command. #37942
Command-line changes
- Included collected goroutine dumps in
cockroach debug zip
. #36813 - Added the
movr
workload to thecockroach workload
command. This workload simulates a fictional ride sharing company and is based on our open-source MovR GitHub repository. #37709
Admin UI changes
- CockroachDB now includes only successfully executed statements in the statement counters. #37264
Bug fixes
- Fixed a roundtrip typecasting issue where an
INET
casted to aSTRING
and back to anINET
would fail. #37938 - Fixed a panic when
json_build_object
is called withBIT
/VARBIT
values. #37057 - Fixed a panic when planning zigzag joins against unique indexes. #36139
RESTORE
now balances the work of ingesting data more evenly between nodes. #36665IMPORT
now balances the work of ingesting data more evenly between nodes. #36665- CockroachDB now computes the result of shifting bit arrays to the right properly and avoids generating invalid bit arrays. #36715
SHOW ZONE CONFIGURATION
no longer emits invalidALTER
syntax in its output when displaying the zone configuration for a table or index partition that is inheriting from the database or the default configuration. #36868SHOW ZONE CONFIGURATION FOR TABLE t PARTITION p
no longer ignores the clausePARTITION p
and now properly displays the zone configuration for that partition instead. #36868- When a node is restarted with another node's address, CockroachDB now performs fewer attempts to communicate with the wrong node. #34197
- It is now possible to write to columns of type
name[]
. #37170 - Permitted planning of window functions within mutation statements (and others that cannot be distributed). #37171
- Avoided a source of internal connectivity problems that would resolve after restarting the affected node. #37204
- Permitted arrays and other sequences to be type-checkable in certain contexts when all of their elements are null. #36673
- CockroachDB now correctly escapes non-finite JSON number values when marshaling to text. #37331
- The
cockroach init
command will now always properly report when a cluster is already initialized, even after the node that it's connecting to is restarted. #37399 - Automated table statistics no longer encounter "batch timestamp must be after replica GC threshold" errors on configurations with low TTL. #36500
- Fixed type inference of columns in subqueries for some expressions of the form
scalar IN (subquery)
. #37578 - Fixed a crash in apply join. #37597
- The
age()
function is now correctly marked as impure, causing it to be unavailable in certain contexts. #37609 - Fixed a panic when constructing the error message for an invalid partitioning. #37689
- Fixed incorrect multiplication of
float
andinterval
types. #37582 - Fixed a case in which
cockroach quit
would return success even though the server process was still running in a severely degraded state. #37668 - Fixed a potential source of (faux) replica inconsistencies that can be reported while running a mixed v19.1 / v2.1 cluster. This error (in that situation only) is benign and can be resolved by upgrading to the latest v19.1 patch release. Every time this error occurs a "checkpoint" is created which will occupy a large amount of disk space and which needs to be removed manually (see
<store directory>/auxiliary/checkpoints
). #37668 - Prevented unlimited memory usage during SQL range deletions #36728
- Dividing floats by zero now returns an error instead of
-Inf
. #37774 - Fixed incorrect results or "incorrectly ordered stream" error in some cases of queries with aggregations. #37713
- A null right operand now causes the sub-operator expression to return null. #37775
- Certain binary encodings of numeric/decimal values no longer result in values that are an order of magnitude off. #35115
- Fixed
GROUP BY
for empty arrays. #37901 - Fixed a race condition that could cause a panic during query planning. #37972
- The
cockroach
command line utilities that internally use a RPC connection (e.g.,cockroach quit
,cockroach init
, etc.) once again properly support passing an IPv6 address literal via the--host
argument. #37977
Performance improvements
- Improved memory efficiency of bulk ingestion and index backfills buffering. #36743
- Fixed a significant performance regression when building CockroachDB with go1.12. #37967
Security improvement
- Stack memory used by CockroachDB is now marked as non-executable, improving security and compatibility with SELinux. #37939
Contributors
This release includes 728 merged PRs by 50 authors. We would like to thank the following contributors from the CockroachDB community:
- George Papadrosou
- Jaewan Park
- Simo Kinnunen (first-time contributor)
- Txiaozhe
- Vijay Karthik
- joowon (first-time contributor)
- lanzao