feat: Backends return Backends.QueryError struct for errors by msmithstubbs · Pull Request #3552 · Logflare/logflare

msmithstubbs · 2026-06-02T23:57:07Z

Add a standard %QueryError{} struct to be returned by backend adaptors on an unsuccessful query. This allows flexibility on which error messages are logged and which are show to end users.

Adds %QueryError{} to normalise the shape of errors returned by backend adaptors
Updates BigQuery, Postgres, and Clickhouse adaptors to use QueryError when returning an error
Adaptors set QueryError.kind based on the backend error message:
- A missing field error would be :invalid_query
- Other errors are :backend_query or :connection_error
QueryError.log will log errors, excluding :invalid_query
- User interface layer is not longer responsible for logging backend errors, but may still log user query errors
- If a user has system_monitoring enabled the logged error messages are included
User facing errors only include the backend error message for :invalid_query

EndpointsLive

QueryLive

AlertsLive

Closes O11Y-1819 and O11Y-1970

msmithstubbs · 2026-06-04T06:42:07Z

+      ~s(Field "notthere" does not exist.)
+  """
+  @spec query_error_message(QueryError.t()) :: String.t() | nil
+  def query_error_message(%QueryError{


Error messages could be in the respective backend adaptors but thought it better to keep this in the UI layer.

Ziinc

needs some adjustment for surfacing of errors.
need some distinction between error messages:

end consumer errors - needs to be generic and shouldn't leak information like underlying backend errors
operator user - should surface underlying backend errors, but logged out to Logger and surfaced to user via system.logs with the user_id metadata key set.
admin user - surfaced to admin via Logger, such as connection pool errors, which might not have a user_id associated, and needs the most detail, on par with operator user

Ziinc · 2026-06-04T07:02:16Z

+
+  defp to_query_error(%Ecto.QueryError{message: message} = error) do
+    %QueryError{
+      message: message,


i think we should restrict what is surfaced to users, in the event that sensitive values get surfaced. things like invalid columns etc are columns are fine, probably can match on the message prefix and give a more generic message for the rest.

This applies for all backends, not just for postgres

msmithstubbs · 2026-06-04T07:10:20Z

Make sense.

operator user - should surface underlying backend errors, but logged out to Logger and surfaced to user via system.logs with the user_id metadata key set.

How do we identify an operator user?

msmithstubbs · 2026-06-08T04:57:25Z

Revised approach:

Backend adaptors call Logger for any errors and return QueryError struct
Drop message attr from QueryError
UI layer doesn't log error failures, but handles taking a QueryError struct and determining correct message to show

msmithstubbs · 2026-06-11T01:59:45Z

@Ziinc

end consumer errors - needs to be generic and shouldn't leak information like underlying backend errors

Does this include API consumers? Example:

logflare/test/logflare_web/controllers/endpoints_controller_test.exs

Lines 266 to 267 in b1a389c

    
           assert response.error =~ 
        
                    "Multiple CTEs available (first_cte, second_cte, final_data). You must specify which one to query using `f:name`"

Should this just return a generic error message?

Co-authored-by: depthfirst-app[bot] <184448029+depthfirst-app[bot]@users.noreply.github.com>

Ziinc · 2026-06-22T16:56:39Z

+        {:error, _reason} ->
          conn
-          |> json(%{connected?: false, reason: reason})
+          |> json(%{connected?: false})


@msmithstubbs i think for the test connection pathway, we should still show the reason, otherwise users will not know why their connection is failing when adding a backend

Agreed - figured I'd drop it per the review feedback and pick it up in review.

Ziinc · 2026-06-22T17:02:53Z

+  @spec ch_query_error_kind(term()) :: QueryError.kind()
+  defp ch_query_error_kind(%Ch.Error{code: code}) when code in [47, 62], do: :invalid_query
+
+  defp ch_query_error_kind(%Ch.Error{message: message}) when is_binary(message) do
+    if message =~ "UNKNOWN_IDENTIFIER" or message =~ "SYNTAX_ERROR" do
+      :invalid_query
+    else
+      :backend_error
+    end
+  end


we will need to support more official error codes that make sense so that there is better DX when people are crafting queries

we'll need to narrow them down to error codes relevant to general analytical queries

github-actions Bot assigned msmithstubbs Jun 2, 2026

msmithstubbs changed the title ~~wip: Backends return Backends.QueryError struct for erros~~ wip: Backends return Backends.QueryError struct for errors Jun 2, 2026

msmithstubbs force-pushed the refactor/backend-query-error branch 9 times, most recently from 394c535 to d322ebf Compare June 4, 2026 06:23

msmithstubbs commented Jun 4, 2026

View reviewed changes

msmithstubbs changed the title ~~wip: Backends return Backends.QueryError struct for errors~~ feat: Backends return Backends.QueryError struct for errors Jun 4, 2026

msmithstubbs marked this pull request as ready for review June 4, 2026 06:46

depthfirst-app Bot reviewed Jun 4, 2026

View reviewed changes

Comment thread lib/logflare_web/live/alerts/alerts_live.ex Outdated

depthfirst-app Bot reviewed Jun 4, 2026

View reviewed changes

Comment thread lib/logflare/backends/adaptor/clickhouse_adaptor.ex Outdated

Comment thread lib/logflare/backends/adaptor/postgres_adaptor.ex Outdated

Ziinc requested changes Jun 4, 2026

View reviewed changes

msmithstubbs marked this pull request as draft June 5, 2026 05:44

msmithstubbs force-pushed the refactor/backend-query-error branch from 00e4105 to d4102d9 Compare June 5, 2026 05:59

msmithstubbs force-pushed the refactor/backend-query-error branch 4 times, most recently from f07cfcb to 82ebf3a Compare June 11, 2026 00:57

msmithstubbs force-pushed the refactor/backend-query-error branch 4 times, most recently from 998e94f to 7ba26a4 Compare June 15, 2026 01:37

msmithstubbs force-pushed the refactor/backend-query-error branch 7 times, most recently from 156788b to 01b850d Compare June 17, 2026 04:52

msmithstubbs marked this pull request as ready for review June 17, 2026 05:15

msmithstubbs marked this pull request as draft June 17, 2026 05:16

msmithstubbs commented Jun 17, 2026

View reviewed changes

Comment thread lib/logflare/backends/adaptor/bigquery_adaptor.ex

msmithstubbs commented Jun 17, 2026

View reviewed changes

Comment thread lib/logflare/alerting.ex

msmithstubbs force-pushed the refactor/backend-query-error branch 8 times, most recently from 5c8be24 to 2909c9f Compare June 18, 2026 02:12

msmithstubbs marked this pull request as ready for review June 18, 2026 02:36

msmithstubbs and others added 4 commits June 22, 2026 09:07

Add backend query error normalization

1ad6ec9

Use fallback error message in AlertsLive

4bad81e

Co-authored-by: depthfirst-app[bot] <184448029+depthfirst-app[bot]@users.noreply.github.com>

move query error logging into backend adaptor

d847e2a

bump ci

ef00e2d

msmithstubbs force-pushed the refactor/backend-query-error branch from 2909c9f to ef00e2d Compare June 21, 2026 23:13

Ziinc approved these changes Jun 22, 2026

View reviewed changes

Ziinc merged commit 6ce156c into Logflare:main Jun 22, 2026
13 checks passed

Ziinc mentioned this pull request Jun 22, 2026

fix: add back reason to test connection controller, updated test_connection/1 callback #3624

Merged

This was referenced Jun 22, 2026

refactor: remove duplicate reservation logging #3627

Open

style: add doc to LoggerMetadata #3628

Merged

Uh oh!

Conversation

msmithstubbs commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

EndpointsLive

QueryLive

AlertsLive

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Ziinc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

msmithstubbs commented Jun 4, 2026

Uh oh!

msmithstubbs commented Jun 8, 2026

Uh oh!

msmithstubbs commented Jun 11, 2026

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

msmithstubbs commented Jun 2, 2026 •

edited

Loading