All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- MeterFilter's applied by the library are no longer explicitly applied and are instead
- Support for spring boot 3.3
- Use static methods to create BeanPostProcessors.
- Ignore explain plan queries, which are generated by tw-reliable-jdbc, as they do not really access the tables and JSQLParser fails to parse them
- Upgrading libraries, mainly the JSqlParser, to support more queries out of the box.
- Added support for multiple threads doing database queries in parallel related to gathering database access statistics.
Previously it wasn't possible as it had the potential to throw a
ConcurrentModificationException
.
- Support for Spring Boot 3.1
- Build against Spring Boot 3.0.6 --> 3.0.7
- Build against Spring Boot 2.7.11 --> 2.7.13
- Build against Spring Boot 2.6.14 --> 2.6.15
- Visibility of
ParsedQuery.SqlOperation
is changed from package protected to public. - Visibility of
ParsedQuery.addOperation
is changed from package protected to public.
- Runtime dependencies are correctly added into POM, for
tw-entrypoints
module.
tw.library.info
metric to provide the version.
- Shadowing and inlining jsqlparser library until spring-data allows to disable
JSqlParserQueryEnhancer
by other means. spring-projects/spring-data-jpa#2989
- Removing quotes around table names provided by JSqlParser. When table name is quoted in sql, it will be returned by JSqlParser also as quoted. But this will create double metrics from sql not having quotes around table names.
- Handling more sql-s the JSqlParser can not parse.
-
A mechanism to disable query parsing through
TwContext
-TasUtils
. -
Flyway integration for TAS. As JSqlParser is not able to parse DDL queries, we will disable the query parsing for Flyway.
-
Implemented a timeout and interruption for TAS SQL parsing. Complex queries in one of our services created long duration heavy CPU burn.
-
Query parsing will use
JSQLParser
complex parsing immediately. Before, our implementation was using simple parsing. AndJSQLParser
implementation tried by defaultsimple
first and thencomplex
, ifsimple
failed. Performance tests showedsimple
parsing for simple queries, is not noticeably faster for simple queries. -
Created a mechanism for a service to provide parsed query information itself and thus skip the query parsing. It can be used for complex queries where parsing is slow or for queries which jsqlparser can not handle. The mechanism can be used via
TableAccessStatisticsParsedQueryRegistry
interface. -
Added more flexibility around query parsing via
TasQueryParsingListener
andTasQueryParsingInterceptor
. -
Supporting parsing queries with
on conflict (...)
clause with multiple parameters. We can remove our own solution, when nextJSQLParser
version would support it.
- Support for Spring Boot 3.0.
- Support for Spring Boot 2.5.
- Counter
EntryPoints_Tas_Parses
, to count per entrypoint, how many parses have been done. It allows to identify, which service components are generating most distinct queries and thus bloating the cache or just burning CPU for parses.
- DAS for database access done outside of entrypoints are now collected in a separate thread. This would allow to collect those metrics even when the service does not have any entrypoints or they are infrequently accessed.
- A bug where bean postprocessor was returning a wrong bean - unwrapped spyql datasource.
-
Counter
EntryPoints_Tas_UncountedQueries
, to count how many queries we were not able to parse table information from. -
Table access statistics can parse tables on queries containing
DATABASE()
function.
- JSqlParser is utilized in a more optimal way, by not executing the parsing in separate threads.
- Hardcoding versions coming from BOMs into POM. That way older dependency resolvers can work without issues as well. Overall this is the most correct way to generate the POM.
-
When SQL parsing fails, we will also add entrypoint dimension into the failure metric, to make it clearer where a problematic SQL may come from.
-
Refactored metrics' constants to a form easily copyable to Grafana.
- Specifying an order for bean post processor, so it will run after Spyql bean post processor.
tw-entrypoints-starter
does not add Spyql data source wrapper anymore. This is now delegated totw-spyql-starter
.
- Spring Boot matrix tests.
- Stop enforcing springboot platform
- Migrating CI to Github Actions.
- Some dependency upgrades.
- Upgraded tw-spyql
- JDK 11+ is required
- Facelift for open-source.
- Small optimizations around tags memory allocations.
- Major optimization around metrics handling, added MeterCache around every metric.
Optimized metrics registration speed.
Added query latency histogram for first table in the query.