Adding the runs_jobs_detail_aggregation endpoint by tmikula-dev · Pull Request #243 · AbsaOSS/EventGate

tmikula-dev · 2026-10-20T13:47:38Z

Overview

This pull request introduces support for window-wide aggregation queries alongside existing paginated named queries, with a focus on the new runs_jobs_detail_aggregation endpoint. It updates the OpenAPI spec, handler validation, query registry, and reader logic to support aggregation queries with a new response shape, and introduces new data models for typed aggregation results.

Release Notes

Added a new runs_jobs_detail_aggregation named-query endpoint providing window-wide summary aggregates plus hourly/daily timeline buckets for runs/jobs data.

Reviewed most of it. Good job, but I noticed several things to improve. Also, when do you anticipate we should remove the stats named queries? They are basically a duplicates of the other ones in named_queries.sql - my recommendation is either now & a ticket (probably preferable to keep PRs small scoped)

lsulak · 2026-10-26T16:39:36Z

+    return _build_json_response(
+        200,
+        {
+            "success": True,


hmm this is a bit weird contract, success is quite visible from the status code, and then also status code is there twice. Is there any situation when it differentiates?

I completely see what you are saying. However this contract was established in the past. And this PR is only a refactor - not creating anything new. So modification in this PR would change the established client contract and that is IMO not a current scope of this task. We can discuss it further and create from this comment a follow-up GH issue. What do you think?

Agree, then maybe create a ticket to revisit this, and do it (if ever) outside of this PR

The issue based on this conversation was created here: #256

lsulak · 2026-10-26T16:48:21Z

+    timeline_daily_sql_key: str
+
+
+SUPPORTED_AGGREGATION_QUERIES: dict[str, AggregationQuery] = {


hmm this really looks comlicated, the whole file. Isn't there already some native aiosql capability to list supported queries? Also, maybe it makes sense to split the existing queries into 2 files, i.e. 2 categories - like job detail type queries and then analytical / aggregation like queries

Update: I could not find it quickly in the aiosql docs so I just opened aiosql code and found this as a public property that you can hopefully use: available_queries against your query, for example when you are doing this: aiosql.from_path(_SQL_DIR, "psycopg2"))

Also the naming is not really explanatory. This is a typical SW development process - we started with named queries, then created data structures and algorithms (loading process, validation etc) around that, and then there was a new type, more analytical queries, and now there are new data structures and algorithms around that - but basically everything is a named query, right? So the names are not descriptive enough. This is technical debt. Not to mention that stats.sql are now duplicated.

I think that this PR is a good time to reduce tech debt, not add much more to it

I tried to make naming conventions better to current query state. See following commit: 4efa108.

I am also implementing suggested approach of aiosql available_queries. Thanks for raising that! But I want to have that in separate PR - I am about to build that on top of this PR. Following PR: #251. The solution is robust and solves several issues that we would face in the future, but for current solution maybe too complex. Share your thoughts please on the added link there.

Ok, thank you, I like it :)

# Conflicts: # Dockerfile

lukasmatta · 2026-11-03T12:29:07Z



+@dataclass(frozen=True)
+class AggregationQueryParams:


At some point, we may want to support filtering by tenant and source (and possibly other parameters as requirements emerge). If it’s a simple change, we can add it within this PR... otherwise, it can done in a separate PR.

Please see my response to similar comment from @lsulak: #243 (comment). The implementation is done in commit: d17382a.

… snapshot

tmikula-dev · 2026-11-03T15:59:56Z

Reviewed most of it. Good job, but I noticed several things to improve. Also, when do you anticipate we should remove the stats named queries? They are basically a duplicates of the other ones in named_queries.sql - my recommendation is either now & a ticket (probably preferable to keep PRs small scoped)

I completely feel, what you are saying or what I feel from the review points. I tried to make the solution more "friendly" by renaming the queries as described in the comment responses. But you are also suggesting some things, that are more on a refactor level or ready to be deeper discussion. If we want to start some better refactored, modifying some contracts etc lets move to follow-up issue and close this one (if possible and good enough) as a v1. Please share your thoughts.

… ones.

feat(aggregation_endpoint): Adding the runs_jobs_detail_aggregation e…

2d58812

…ndpoint

tmikula-dev self-assigned this Oct 20, 2026

tmikula-dev added the enhancement New feature or request label Oct 20, 2026

tmikula-dev requested review from lsulak, oto-macenauer-absa and petr-pokorny-absa as code owners October 20, 2026 13:47

Merge branch 'master' into feature/116-adding-dashboard-aggregated-query

51a2d47

tmikula-dev linked an issue Oct 20, 2026 that may be closed by this pull request

EventStats: compiled queries endpoint for server-side aggregations #116

Closed

tmikula-dev added 3 commits October 20, 2026 17:07

Fixing the Dockerfile librdkafka.

b687ffd

Merge remote-tracking branch 'origin/feature/116-adding-dashboard-agg…

8af1d15

…regated-query' into feature/116-adding-dashboard-aggregated-query

Error message typo.

3c9b3bd

oto-macenauer-absa reviewed Oct 23, 2026

View reviewed changes

Comment thread src/readers/sql/named_queries.sql Resolved

Comment thread src/readers/reader_postgres.py Resolved

Comment thread src/handlers/handler_named_query.py Resolved

Comment thread Dockerfile Resolved

lsulak reviewed Oct 26, 2026

View reviewed changes

Comment thread src/readers/reader_postgres.py Resolved

lsulak reviewed Oct 26, 2026

View reviewed changes

Comment thread src/readers/sql/aggregation_queries.sql Resolved

lsulak reviewed Oct 26, 2026

View reviewed changes

tmikula-dev added 3 commits November 3, 2026 11:15

Merge branch 'master' into feature/116-adding-dashboard-aggregated-query

eed0e59

# Conflicts: # Dockerfile

Implementing review comments: Timestamp validation

513d8a0

Implementing review comments: UTC safety

4b521db

lukasmatta reviewed Nov 3, 2026

View reviewed changes

tmikula-dev added 2 commits November 3, 2026 14:53

Implementing review comments: Summary and timeline queries within one…

1707265

… snapshot

Implementing review comments mentioned by lsulak

d17382a

Implementing review comments: Renaming PaginatedQueries to JobsDetail…

4efa108

… ones.

tmikula-dev added this pull request to stack #252 November 4, 2026 11:06

oto-macenauer-absa approved these changes Nov 10, 2026

View reviewed changes

tmikula-dev mentioned this pull request Nov 10, 2026

Refactoring envelope of successful responses and its client contract #256

Open

Merge branch 'master' into feature/116-adding-dashboard-aggregated-query

7bad42f

tmikula-dev merged commit 641244d into master Nov 11, 2026
10 checks passed

tmikula-dev deleted the feature/116-adding-dashboard-aggregated-query branch November 11, 2026 10:23

		timeline_daily_sql_key: str


		SUPPORTED_AGGREGATION_QUERIES: dict[str, AggregationQuery] = {



		@dataclass(frozen=True)
		class AggregationQueryParams:

Conversation

tmikula-dev commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Release Notes

Related

Uh oh!

coderabbitai Bot commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oto-macenauer-absa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lsulak left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lsulak Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tmikula-dev Nov 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tmikula-dev commented Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

tmikula-dev commented Oct 20, 2026 •

edited

Loading

coderabbitai Bot commented Oct 20, 2026 •

edited

Loading

lsulak Oct 26, 2026 •

edited

Loading

tmikula-dev Nov 4, 2026 •

edited

Loading

tmikula-dev commented Nov 3, 2026 •

edited

Loading