Core Release Notes

This article describes the release notes for SelectDB Core.

3.0

Note SelectDB Core 3.0 was developed based on Apache Doris 2.0.

November 02, 2023（3.0.3）

New Features

• Supported Doris Clean Label.
• Stream Load cloud API is compatible with Doris community behavior.
• The warehouse had built-in JDBC Driver.

Improvements

• The new optimizer top n supported runtime filter clipping by default.

Bug Fixes

• Fixed the problem of slow query response due to resource leakage in the warehouse.
• Fixed the problem that function statement disabling does not take effect.
• Fixed an issue where schema change would fail when re-arranging.

October 17, 2023（3.0.2）

New Features

• Added JDBC Driver URL whitelist function for S3 protocol import.

Improvements

• Merged all features, improvements and bug fixes from Apache Doris 2.0.2.
• Merged all features, improvements and bug fixes from SelectDB Core 2.3.10 and 2.4.4.
• Reduced the time to obtain metadata during Plan and speeds up Plan generation.
• Added time-consuming statistics for query profile accessing metadata.
• Balanced and optimized resource utilization.

Bug Fixes

• Fixed the issue of read and write errors after setting File Cache TTL.
• Fixed the problem of service unavailability caused by querying the backends table or rowsets table.

September 18, 2023（3.0.1-beta）

Improvements

• Merged all features, improvements and bug fixes from Apache Doris 2.0.1.
• Merged all features, improvements and bug fixes from SelectDB Core 2.4.3.
• Optimized the time consumption of get version.

2.4

Note SelectDB Core 2.4 was developed based on Apache Doris 1.2.

November 14, 2023（2.4.6）

Bug Fixes

• Fixed several compute node memory leaks.

November 06, 2023（2.4.5）

Improvements

• Optimized the balance strategy.

Bug Fixes

• Fixed the query failure problem caused by replace tmp partition.
• Fixed the deadlock problem caused by mixing pthread and bthread.

September 20, 2023（2.4.4）

Improvements

• Merged all features, improvements and bug fixes from SelectDB Core 2.3.9 and 2.3.10.
• Improved the usability of the import function.

Bug Fixes

• Fixed the problem of unresponsive mysql session (requesting truncate table, which contains a large number of partitions).

September 11, 2023（2.4.3）

New Features

• Supported suspension of service due to arrears.

Improvements

• Merged all features, improvements and bug fixes from SelectDB Core 2.3.8.
• Compatible with s3 load endpoint with http string.

Bug Fixes

• Fixed the docker environment time zone issue.

August 28, 2023（2.4.2）

New Features

• Supported the copy into import method to set constant values.

Improvements

• Improved the accuracy of storage information statistics.

Bug Fixes

• Fixed the issue where cloud stream load might fail.

August 3, 2023（2.4.1）

New Features

• Supported on-demand cluster auto start when receiving a request.
• Supported stream load.

Improvements

• Merged all features, improvements and bug fixes from SelectDB Core 2.3.5, 2.3.6 and 2.3.7.

Bug Fixes

• Fixed the issue where inverted index queries could fail.

July 3, 2023（2.4.0）

New Features

• Supported controlling whether SQL uses cache through session variable.
• Supported incremental data synchronization of imported data between clusters.
• Supported cache data preheating function based on table cluster.
• Supported querying cache usage through SQL.
• Supported on-demand cluster auto stop when idle.

Improvements

• Enhanced data security.
• Improved the response speed when the query misses the cache.

Bug Fixes

• Fixed the issue where set default cluster might fail.
• Fixed the issue where the upgrade process could fail.

2.3

Note SelectDB Core 2.3 was developed based on Apache Doris 1.2.

October 16, 2023 (2.3.11)

Improvements

• Optimized the balance strategy.

Bug Fixes

• Fixed the query failure problem caused by replace tmp partition.

September 20, 2023 (2.3.10)

Bug Fixes

• Fixed the problem of time extension in copy into import method.

September 11, 2023 (2.3.9)

Bug Fixes

• Fixed the problem that the in operator for bitmap type columns caused the query to be unavailable.
• Fixed decimal calculation accuracy problem.
• Fixed the bug that when the alter statement changes more columns, the key column changes will also go through lightweight schema change.

August 29, 2023 (2.3.8)

Improvements

• Optimized the speed of importing and uploading small objects.

Bug Fixes

• Fixed the problem of incorrect division accuracy in decimal v3 in some cases (increased 100 times accuracy).

August 2, 2023 (2.3.7)

Bug Fixes

• Fixed the issue where incorrect type inference for decimal v2 calculations resulted in inaccurate precision.
• Fixed the issue where create catalog relied on an external driver, causing the warehouse to fail to start.
• Fixed the issue where indexes will be created for all columns when adding bitmap index (which may cause data import failure).
• Fixed the issue where the default value of the copy into import method does not take effect.

July 25, 2023 (2.3.6)

Improvements

• Improved the usability of the import function.
• The round precision behavior of decimal v2 is supported and is compatible with decimal v3.

Bug Fixes

• Fixed the issue where queries could get stuck.
• Fixed the problem of access failure that may report 403 when importing error_log.

July 19, 2023 (2.3.5)

Improvements

• Reduced the delay when importing small batches of data with partition skew.
• Added the limit configuration of the length of the audit log insert into statement, and the default is 4KB truncation.
• Improved remote IO efficiency and accelerated query speed.

Bug Fixes

• Fixed the issue where an error would be prompted if the column name in the delete statement condition was capitalized.
• Fixed known memory leak issue.
• Fixed the issue where tablet balance could be uneven.
• Fixed the issue where node restart might cause query failure.
• Fixed the problem of OSS S3 load error and added AWS_PROVIDER attribute.

July 7, 2023 (2.3.4)

Bug Fixes

• Fixed the issue where queries might become unavailable.

July 3, 2023 (2.3.3)

Improvements

• Speeded up node restart speed and improved availability.

Bug Fixes

• Fixed several memory leak issues.

June 27, 2023 (2.3.2)

Improvements

• Upgraded the warehouse mariadb client version.
• Reduced the dimension statistics of file cache metrics.
• Added indicators for warehouse Fragment execution.

Bug Fixes

• Fixed the issue where the query is canceled but the hash table is not released during query execution.
• Fixed the bug that inverted index files were not recycled after compaction.
• Added null check to sql parser.
• Fixed the bug that some segments could not be opened after limit optimization.

May 16, 2023 (2.3.0）

New Features

• Supported MySQL TLS connection (closed by default, enabling it requires separate configuration, and requires mysql client 5.0.38 or above).
• Supported Prepare stmt.
• Supported smooth node reduction, reducing query performance jitter during cluster size reduction.
• Supported smooth upgrade and improved the import success rate during version upgrade.

Improvements

• Further improved data security.
• Improved the stability of the copy into import method.
• Provideed better account/data isolation mechanism.

Bug Fixes

• Synchronized several bug fixes of doris-1.2-lts.
• Fixed the problem that hard disk space could not be cleared in time.

2.2

Note SelectDB Core 2.2 was developed based on Apache Doris 1.2.

June 1, 2023（2.2.34）

Bug Fixes

• Fixed the problem of import being stuck due to inconsistent object storage size.

May 15, 2023（2.2.33）

Bug Fixes

• Fixed the issue that caused inaccessibility after JDBC URL or mysql -D was used with cluster.

April 27, 2023（2.2.32）

Improvements

• Added file cache metrics.

Bug Fixes

• Synchronized several bug fixes of doris-1.2-lts.
• Fixed the problem of segment file size being out of order.

April 19, 2023（2.2.31）

New Features

• Supported dryrun testing when configuring external resources (HMS, external stage).
• Supported lightweight schema change to modify the length of Varchar columns.
• Supported the copy into import method of split compressed file.

Improvements

• Disabled routine load.
• Enhanced data security.

Bug Fixes

• Fixed issues related to querying top n.
• Fixed the problem of query failure due to elimination of file cache.
• Fixed the issue where importing a large number of tablets at a time might fail.
• Fixed the issue of inverted index memory instability.
• Fixed the issue where inverted index could cause query failure.
• Fixed the issue where the file was overwritten during copy into, causing the import to get stuck.
• Fixed the issue of "no backend" caused by renaming a cluster with the same name.

March 23, 2023（2.2.21）

Bug Fixes

• Fixed the issue where cumulative compaction may not be triggered.
• Fixed the problem that pure computing plan nodes may cross clusters.
• Fixed name inconsistency after executing rename cluster.
• Fixed the problem that the schema change job cannot fail automatically when executing drop cluster.
• Fixed the issue where base compaction failed due to delete predicate.
• Fixed issue with create table like.

February 28, 2023（2.2.20）

Improvements

• Implemented several cache optimizations.
• Improved copy into import throughput.
• Optimized empty table query performance.

Bug Fixes

• Fixed the bug of incorrect data when executing add inverted index.
• Fixed the issue where parquet import could get stuck.
• Fixed several issues related to data lake analysis in Apache Doris 1.2.4.

February 9, 2023（2.2.1）

New Features

• Supported external stage cross-account authorization.

Improvements

• Optimized copy into upload speed.
• Optimized show roles for cluster and stage permission display.

Bug Fixes

• Fixed the issue where the computing node may get stuck.
• Fixed the problem of resource leakage that may be caused by temp partition.
• Fixed the issue where show cluster cannot be executed except by admin user.

January 17, 2023（2.2.0）

New Features

• Supported inverted index of String type.
• Supported BKD multidimensional numerical index of numeric and date types.
• Supported inverted index and BKD multi-dimensional numerical index of array type.
• Supported dynamic table to dynamically modify schema.

Improvements

• Optimized incremental ADD/DROP INDEX.
• Optimized Index searcher cache.
• Optimized Top n query (ORDER BY c1, c2 LIMIT n) performance.

Bug Fixes

• Fixed the problem of not reading original columns when index filtering.

2.0

Note SelectDB Core 2.0 was developed based on Apache Doris 1.1.

January 5, 2023（2.0.15）

New Features

• Supported rename warehouse.

Improvements

• Optimized copy into delete sign.
• Supported the ability to automatically determine whether the object information is valid when executing copy into to create a stage.
• Optimized partition level tablet balance .
• Added checking of cache configuration parameter types.
• Optimized tablet sink profile.
• Improved the permission system.

Bug Fixes

• Fixed the problem of misjudgment of stale request in sync_tablet_rowsets.
• Fixed load job leak issue.
• Fixed partition level balance bug.

December 14, 2022（2.0.14）

New Features

• Supported the select statement to set cluster through hint.
• Supported retaining several copy job for each table.

Improvements

• Optimized the performance of import and write to s3.
• After the imported Segment index is uploaded to the object storage, it will be entered into the read cache in advance.

Bug Fixes

• Fixed the issue where the data types of planning and execution did not match when importing parquet, causing log import to fail.
• Fixed issue with importing across clusters.

December 2, 2022（2.0.13）

New Features

• Supported the copy into statement additional condition delete on.
• Supported the show data statement to display the data volume of the entire warehouse.

Improvements

• Restored use of show processlist statement.
• Improved the speed of recycling dropped stages.
• Improved the speed of cache data writing to disk.

Bug Fixes

• Fixed the issue of inconsistent warehouse behavior when executing drop cluster.

November 26, 2022（2.0.12）

Improvements

• Supported verifying the validity of object storage information when executing create stage.

Bug Fixes

• Fixed the issue where the write cache may be deleted prematurely.
• Fixed the issue where cache might not be hit.
• Fixed the issue where non-colocate table tablets could be unevenly distributed.