Skip to content

fix: generate header after query also in do_describe_ in DuckDB example #272

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
AlphaJack wants to merge 2 commits into
base: master
Choose a base branch
from
Open

fix: generate header after query also in do_describe_ in DuckDB example #272

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8ae11ac
fix: generate header after query also in do_describe_ in DuckDB example
AlphaJack Jun 16, 2025
0fe686a
Update Cargo.toml
AlphaJack Jun 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ tokio = { version = "1.19", features = ["rt-multi-thread", "net", "macros", "tim
rustls-pki-types = { version = "1.10" }
rusqlite = { version = "0.36.0", features = ["column_decltype"] }
## for duckdb example
duckdb = { version = "1.0.0" }
duckdb = { version = "1", features = ["bundled"] }
Comment thread
AlphaJack marked this conversation as resolved.
Outdated

## for loading custom cert files
rustls-pemfile = "2.0"
Expand Down
48 changes: 43 additions & 5 deletions examples/duckdb.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ impl SimpleQueryHandler for DuckDBBackend {
C: ClientInfo + Unpin + Send + Sync,
{
let conn = self.conn.lock().unwrap();
if query.to_uppercase().starts_with("SELECT") {
if is_result_query(query) {
let mut stmt = conn
.prepare(query)
.map_err(|e| PgWireError::ApiError(Box::new(e)))?;
Expand All @@ -74,6 +74,15 @@ impl SimpleQueryHandler for DuckDBBackend {
}
}

fn is_result_query(query: &str) -> bool {
let query_upper = query.trim().to_uppercase();
query_upper.starts_with("SELECT")
|| query_upper.starts_with("WITH")
|| query_upper.starts_with("EXPLAIN")
|| query_upper.starts_with("DESCRIBE")
|| query_upper.starts_with("FROM")
}

fn into_pg_type(df_type: &DataType) -> PgWireResult<Type> {
Ok(match df_type {
DataType::Null => Type::UNKNOWN,
Expand Down Expand Up @@ -261,7 +270,7 @@ impl ExtendedQueryHandler for DuckDBBackend {
.map(|f| f.as_ref())
.collect::<Vec<&dyn duckdb::ToSql>>();

if query.to_uppercase().starts_with("SELECT") {
if is_result_query(query) {
let rows: Rows<'_> = stmt
.query::<&[&dyn duckdb::ToSql]>(params_ref.as_ref())
.map_err(|e| PgWireError::ApiError(Box::new(e)))?;
Expand All @@ -288,9 +297,21 @@ impl ExtendedQueryHandler for DuckDBBackend {
{
let conn = self.conn.lock().unwrap();
let param_types = stmt.parameter_types.clone();
let stmt = conn
.prepare_cached(&stmt.statement)
let stmt_sql = &stmt.statement;
let mut stmt = conn
.prepare_cached(stmt_sql)
.map_err(|e| PgWireError::ApiError(Box::new(e)))?;

if is_result_query(stmt_sql) {
let _ = stmt
.query([])
.map_err(|e| PgWireError::ApiError(Box::new(e)))?;
} else {
let _ = stmt
.execute([])
.map_err(|e| PgWireError::ApiError(Box::new(e)))?;
}
Copy link
Copy Markdown
Owner

@sunng87 sunng87 Jun 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, we cannot execute it on the describe command, if the query has side-effects, this will break the system.

Copy link
Copy Markdown
Contributor Author

@AlphaJack AlphaJack Jun 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An alternative is to use DuckDB DESCRIBE for queries that return results:

D describe select 1, version(), false::logical;
┌───────────────────────────────────────┬─────────────┬─────────┬─────────┬─────────┬─────────┐
│              column_name              │ column_type │  null   │   key   │ default │  extra  │
│                varchar                │   varchar   │ varchar │ varchar │ varchar │ varchar │
├───────────────────────────────────────┼─────────────┼─────────┼─────────┼─────────┼─────────┤
│ 1                                     │ INTEGER     │ YES     │ NULL    │ NULL    │ NULL    │
│ "version"()                           │ VARCHAR     │ YES     │ NULL    │ NULL    │ NULL    │
│ CAST(CAST('f' AS BOOLEAN) AS BOOLEAN) │ BOOLEAN     │ YES     │ NULL    │ NULL    │ NULL    │
└───────────────────────────────────────┴─────────────┴─────────┴─────────┴─────────┴─────────┘

D select 1, version(), false::logical;
┌───────┬─────────────┬───────────────────────────────────────┐
│   1   │ "version"() │ CAST(CAST('f' AS BOOLEAN) AS BOOLEAN) │
│ int32 │   varchar   │                boolean                │
├───────┼─────────────┼───────────────────────────────────────┤
│   1   │ v1.3.0      │ false                                 │
└───────┴─────────────┴───────────────────────────────────────┘

What are the two do_describe_() expected to return for DDL queries? Because DESCRIBE doesn't support them.

Copy link
Copy Markdown
Owner

@sunng87 sunng87 Jun 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It returns a NoData message in postgres wire protocol.

This approach looks good. Do you know if there is a duckdb API for this? It feels like it should be something like describe on Statement. This is what I expected column_type and column_name should return.

Copy link
Copy Markdown
Owner

@sunng87 sunng87 Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems duckdb does have this capability to get column structure for a statement without running it, but at least duckdb-rs didn't expose this capability as statement API. It would be nice to add it so we can call column_type/column_name either after query or after describe.

Copy link
Copy Markdown
Contributor Author

@AlphaJack AlphaJack Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where could you read that DuckDB can retrieve the column structure for a statement without running it?
In the C API, it seems that duckdb_column_count() and other functions still need a duckdb_result pointer generated by duckdb_query() (doc PDF page 54, doc website).

On the other hand, using DESCRIBE would mean having a new map for type string to FieldInfo datatype

Copy link
Copy Markdown
Owner

@sunng87 sunng87 Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't read from their API. It's just because they are capable with this DESCRIBE statement that returns column structures without executing the statement, which in theory can be implemented as an API.

I agree using DESCRIBE in this scenario can be a little tricky. duckdb-rs has a Type enumeration but we will need to translate it into arrow types then to postgres types.

By the way, I'm going to ask upstream if it's possible to add describe API to statement.

Copy link
Copy Markdown
Owner

@sunng87 sunng87 Jun 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feature request thread in duckdb: duckdb/duckdb#17951


row_desc_from_stmt(&stmt, &Format::UnifiedBinary)
.map(|fields| DescribeStatementResponse::new(param_types, fields))
}
Expand All @@ -304,9 +325,26 @@ impl ExtendedQueryHandler for DuckDBBackend {
C: ClientInfo + Unpin + Send + Sync,
{
let conn = self.conn.lock().unwrap();
let stmt = conn
let mut stmt = conn
.prepare_cached(&portal.statement.statement)
.map_err(|e| PgWireError::ApiError(Box::new(e)))?;

let params = get_params(portal);
let params_ref = params
.iter()
.map(|f| f.as_ref())
.collect::<Vec<&dyn duckdb::ToSql>>();

if is_result_query(&portal.statement.statement) {
let _ = stmt
.query::<&[&dyn duckdb::ToSql]>(params_ref.as_ref())
.map_err(|e| PgWireError::ApiError(Box::new(e)))?;
} else {
let _ = stmt
.execute::<&[&dyn duckdb::ToSql]>(params_ref.as_ref())
.map_err(|e| PgWireError::ApiError(Box::new(e)))?;
}

row_desc_from_stmt(&stmt, &portal.result_column_format)
.map(|fields| DescribePortalResponse::new(fields))
}
Expand Down
Loading