Query Groups is a powerful feature which automatically calculates performance statistics on sets of queries. Queries are automatically grouped together using a grouping algorithm which attempts to match queries that performance similar operations. Then, query groups are ranked by volume, execution time, and queue time.
Query Groups can answer questions like:
- my cluster just experienced a sudden increase in latency - which queries are causing this?
- which queries are consuming the most amount of cluster resources?
- which queries are slowly increasing in latency?
How it Works
A "query group" is a collection of queries which have common characteristics. All queries in a query group operate on the same tables. However, queries may have different constants and literals - which are ignored.
Query Grouping Algorithm
The intermix.io platform groups queries by applying the following algorithm:
- comments are ignored
- string constants and literals are ignored
- table names are preserved
- all SQL directives are preserved
The following query:
/* INTERMIX_ID: eyJkYWciOiAi9sb2FkIn0= */ insert into my_schema.my_table (grosysid, groname, user_id, scantime) values (102, 'readers', 105, '2018-08-06 23:20:03');
INSERT INTO INTERMIX_RAW_LOGS MY_TABLE GROSYSID GRONAME USER_ID SCANTIME VALUES
All subsequent queries that match this pattern will be grouped together.
'--' style comments
The algorithm will ignore any text following a '-- comment. Thus, if a query starts with a '--' comment tag, then the entire query is ignored. This result in the query being grouped in the 'Empty Query Group'.
See this article for work-arounds to this issue.
App Tracing Compatibility
App Tracing relies on annotations in the query. Some apps use '--' style annotations (e.g. Looker). These annotations are compatible with Query Grouping since our system is aware of those annotations.
Query Groups Dashboard
Click on the new left-navigation menu item titled "Query Groups" .
The top 50 Query Groups are ranked across three metrics.
|Rank By||How the Rank is Calculated|
|Count||Rank is calculated by measuring total query volume.|
|Execution Time||Rank is calculated by summing the execution time for all queries in the group. This is equivalent to the "average execution time" * "count". Thus, the top ranked group consumed the most amount of query execution time in your cluster.|
|Queue Time||Rank is calculated by summing the queue time for all queries in the group. This is equivalent to the "average queue time" * "count". Thus, the top ranked group consumed the most amount of query queue time in your cluster.|
Description of Metrics
Each query group contains rich information.
Query Groups are sorted by Rank by default. You can also sort by Rank Change and % Change of the metric.
The default time range is 7 days. Choose the pull-down menu to select 24h instead.