Release 9.0.30.1¶

Disabled audit spool archiving in Audit Server¶

Added support to disable archiving of the audit spool in the audit server. Archiving is now disabled by default to improve performance and reduce unnecessary storage usage. Configuration can be updated to enable it if required.

Portal Role Enhancement: View Operations Configuration¶

Added a check to resource policies to ensure only users with the roles ROLE_SYS_ADMIN, ROLE_ADMIN, and ROLE_READ_ONLY can access Ops connectors.

Ranger Plugin Memory Improvement¶

In-memory object garbage collection has been enhanced. This means less memory is used during rapid policy operations, leading to more efficient and smoother performance.

Fixed ABAC Use Cases for Databricks Runtime 16.4 LTS and Above¶

A fix has been implemented to resolve the failure of ABAC (Attribute-Based Access Control) use cases in Databricks Runtime 16.4 LTS and later versions.

Upgraded Ranger Plugin Version¶

Upgraded the Ranger plugin version in the DataServer to improve performance and reduce memory utilization.

Support for Tracking Table Creation Timestamps to Detect Drop-and-Recreate Events in DBX-SQL¶

• Added support to load table creation timestamps in DBX-SQL, enabling detection of drop-and-recreate table events.
• To enable this feature, add the following property in vars.connector.databricks.sql.analytics.yml to override its default value:
  YAML

  CONNECTOR_DATABRICKS_SQL_ANALYTICS_LOAD_RESOURCES_KEY: "load_multi_thread_pool_like"
  

Improved Error Handling for DESCRIBE Queries on Foreign Catalogs¶

Enhanced error handling in the Databricks Unity Catalog Connector for DESCRIBE queries on foreign catalogs. Clearer error messages are now shown when table metadata cannot be retrieved.

Performance Improvements for Dynamic Ranger Plugin¶

Improved the performance and reduced memory usage of the dynamic Ranger plugin through internal optimizations and cleanup.

Upgraded RocksDB and Improved Shutdown Handling¶

• Upgraded RocksDB from version 6.20.3 to 10.2.1.
• Ps-Core now closes RocksDB gracefully to prevent data corruption during shutdown.

Dremio Base Image Upgraded¶

Updated the Dremio base image to a newer Debian version to address known security vulnerabilities identified in CVE (Common Vulnerabilities and Exposures) report.

Enhanced Solr Integration with Batching and Robust Error Handling¶

Improved Solr integration by adding batching support and enhancing error handling to prevent NullPointerExceptions. Logging has also been optimized for better clarity during Solr operations. These improvements address failures previously observed when adding Lake Formation PolicySync resources to Solr.

Fixed Zero Metrics Display Issue for Databricks Connector in Grafana¶

Resolved an issue where the Resource Loader – Resource Timeline Grafana panel was displaying zero values for metrics related to the Databricks connector. Metrics now reflect accurate usage data.

Added Grafana Alerts for Stuck Service Resource and Permission Loaders¶

Implemented Grafana alerts to detect and notify when the service resource or permission loader becomes stuck in the Databricks connector.

Configurable JDBC SocketTimeout for Databricks SQL Analytics¶

Added support to configure the JDBC SocketTimeout parameter for Databricks SQL Analytics connections. This helps mitigate connection hangs and improves overall network reliability.

Fixed Configuration Sync Issue in BigQuery Connector¶

Resolved an issue where the BigQuery connector failed to resume configuration sync if it was started with empty JSON credentials and the credentials were later re-added via the UI.

Fixed REVOKE and GRANT Failures After Role Property Update in Databricks SQL¶

Resolved improper principal handling that caused REVOKE and GRANT operations to fail. Principals are now consistently stored in the cache, even if unmanaged.

Fixed Syntax Error in Load Dataset Query for BigQuery Connector¶

Corrected a syntax error in the dataset load query, improving resource loading reliability in the BigQuery connector.

Log and Test Data Purging Refinement and Standardisation Enhancement¶

Refined the existing purging mechanism for logs and test data in the Diagnostics Server to further enhance system performance and storage efficiency. Additionally, introduced the privacera_version.txt file in the working directory to align with standard practices and support consistent version tracking.