Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -164,5 +164,8 @@ cython_debug/
# Visual Studio Code
.vscode

# Claude AI assistant
.claude/

# ruff stuff
.ruff_cache
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,31 @@
# ChangeLog

## 0.4 (Unreleased)

### 0.4.0

- **MAJOR**: Full PEP 249 (Python Database API v2.0) compliance implementation:
- Add module globals: `apilevel="2.0"`, `threadsafety=1`, `paramstyle="pyformat"`
- Implement module-level `connect()` factory function with full parameter support
- Export all PEP 249 exceptions (`Warning`, `Error`, `DatabaseError`, etc.) at module level
- Create comprehensive type system in `asynch.dbapi_types`:
- Type objects: `STRING`, `NUMBER`, `DATETIME`, `BINARY`, `ROWID`
- Type constructors: `Date`, `Time`, `Timestamp`, `DateFromTicks`, etc.
- ClickHouse to PEP 249 type mapping with `Nullable`/`LowCardinality` support
- Enhance cursor interface:
- Add public `cursor.arraysize` property with getter/setter
- Add `cursor.setoutputsize()` method (renamed from `setoutputsizes`, kept alias)
- Add `cursor.lastrowid` property (returns `None` for ClickHouse)
- Add `cursor.callproc()` method (raises `NotSupportedError` per spec)
- Add `cursor.nextset()` async method (returns `None` for single result sets)
- Fix `cursor.description` to use PEP 249 type objects and return `None` for non-SELECT
- Change `connection.commit()` from raising `NotSupportedError` to no-op behavior
- Fix cursor and connection `__repr__` methods to show clean status strings
- **MAJOR**: SQLAlchemy compatibility support:
- Full DB-API interface compatibility for SQLAlchemy Core and ORM
- Comprehensive test suite covering SQLAlchemy integration patterns
- Support for `clickhouse+asynch://` connection URLs

## 0.3

### 0.3.1
Expand Down
36 changes: 35 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,46 @@ lint:
ruff check --fix $(DIRS)

test:
$(PY_DEBUG_OPTS) pytest
$(PY_DEBUG_OPTS) poetry run pytest

# Run only PEP 249 compliance tests (shows all failures, not just the first)
test-pep249:
$(PY_DEBUG_OPTS) poetry run pytest tests/pep249/ -p no:randomly --no-header --tb=short --override-ini="addopts=-s -vvv"

# Run only SQLAlchemy compatibility tests
test-sqlalchemy:
$(PY_DEBUG_OPTS) poetry run pytest tests/sqlalchemy/ -p no:randomly --no-header --tb=short --override-ini="addopts=-s -vvv"

# Run PEP 249 + SQLAlchemy tests together (TDD workflow — see all failures at once)
test-compat:
$(PY_DEBUG_OPTS) poetry run pytest tests/pep249/ tests/sqlalchemy/ -p no:randomly --no-header --tb=short --override-ini="addopts=-s -vvv"

build: deps clean
poetry build

clean:
rm -rf ./dist

# ClickHouse database management
db-up:
docker compose up -d clickhouse

db-down:
docker compose down

db-logs:
docker compose logs -f clickhouse

db-status:
docker compose ps clickhouse

db-reset:
docker compose down
docker volume rm asynch_clickhouse_data 2>/dev/null || true
docker compose up -d clickhouse

# Connect to ClickHouse CLI (requires database to be running)
db-cli:
docker exec -it asynch_clickhouse clickhouse-client --user default

ci: check test
99 changes: 97 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,15 @@

## Introduction

`asynch` is an asynchronous ClickHouse Python driver with native TCP interface support, which reuses most of [clickhouse-driver](https://github.com/mymarilyn/clickhouse-driver) features and complies with [PEP249](https://www.python.org/dev/peps/pep-0249/).
`asynch` is an asynchronous ClickHouse Python driver with native TCP interface support, which reuses most of [clickhouse-driver](https://github.com/mymarilyn/clickhouse-driver) features and is fully compliant with [PEP249](https://www.python.org/dev/peps/pep-0249/) (Python Database API Specification v2.0).

### ✨ Features

- **Full PEP249 Compliance**: Complete implementation of the Python Database API Specification v2.0
- **SQLAlchemy Compatible**: Works seamlessly with SQLAlchemy Core and ORM
- **Asynchronous**: Built for modern async/await Python applications
- **Type Safety**: Comprehensive type system with proper ClickHouse to Python type mapping
- **Connection Pooling**: Efficient connection management for high-performance applications

## Installation

Expand Down Expand Up @@ -35,7 +43,32 @@ For more details, please refer to the project [CHANGELOG.md](./CHANGELOG.md) fil

## Usage

Basically, a connection to a ClickHouse server can be established in two ways:
### Quick Start with PEP249 Interface

The simplest way to use asynch is through the PEP249-compliant interface:

```python
import asynch

# Create connection using the module-level connect() function
conn = asynch.connect(
host="127.0.0.1",
port=9000,
user="default",
database="default"
)

async def main():
async with conn:
async with conn.cursor() as cursor:
await cursor.execute("SELECT version()")
result = await cursor.fetchone()
print(f"ClickHouse version: {result[0]}")
```

### Advanced Connection Management

For more control, a connection to a ClickHouse server can be established in two ways:

1. with a DSN string, e.g., `clickhouse://[user:password]@host:port/database`;

Expand Down Expand Up @@ -217,6 +250,68 @@ async def use_pool():
await pool.shutdown()
```

### SQLAlchemy Core Integration

`asynch` works well alongside SQLAlchemy Core. Because `asynch` uses `paramstyle = "pyformat"`,
you can compile any SQLAlchemy `ClauseElement` with a ClickHouse dialect and pass the resulting
SQL string and parameter dict directly to an `asynch` cursor.

```python
import sqlalchemy as sa
from sqlalchemy import create_engine, text, select
from asynch import Pool
from asynch.cursors import DictCursor

# Build a throwaway sync engine just to get the ClickHouse dialect object.
# clickhouse-sqlalchemy produces %(name)s pyformat SQL that asynch accepts directly.
_ch_dialect = create_engine("clickhouse+native://user:password@host:9000/database").dialect

async def main():
async with Pool(
minsize=2,
maxsize=10,
host="host",
port=9000,
user="user",
password="password",
database="database",
) as pool:
# --- raw SQL via text() ---
async with pool.connection() as conn:
stmt = text("SELECT :limit rows")
compiled = stmt.compile(dialect=_ch_dialect)
async with conn.cursor(cursor=DictCursor) as cursor:
await cursor.execute(str(compiled), dict(compiled.params))
print(await cursor.fetchall())

# --- SQLAlchemy Core construct ---
metadata = sa.MetaData()
my_table = sa.Table("my_table", metadata, autoload_with=None) # define columns manually
query = select(my_table).where(my_table.c.id > 0).limit(10)

async with pool.connection() as conn:
compiled = query.compile(dialect=_ch_dialect)
async with conn.cursor(cursor=DictCursor) as cursor:
await cursor.execute(str(compiled), dict(compiled.params))
rows = await cursor.fetchall() # list[dict] — column names are keys
print(rows)
```

> **Note on `clickhouse+asynch://`**: A dedicated `create_async_engine("clickhouse+asynch://...")`
> shortcut requires a registered SQLAlchemy async dialect entry point, which is not currently part
> of this package. The pattern above — compile with the sync ClickHouse dialect, execute with an
> asynch cursor — gives you the same result with no extra dependency.

### PEP249 Compliance

`asynch` fully implements the Python Database API Specification v2.0:

- **Module globals**: `apilevel = "2.0"`, `threadsafety = 1`, `paramstyle = "pyformat"`
- **Connection interface**: `connect()`, `commit()`, `rollback()`, `close()`, `cursor()`
- **Cursor interface**: `execute()`, `executemany()`, `fetch*()`, `description`, `rowcount`
- **Type system**: `STRING`, `NUMBER`, `DATETIME`, `BINARY`, `ROWID` type objects
- **Exception hierarchy**: Complete set of standard exceptions (`Error`, `DatabaseError`, etc.)

## ThanksTo

- [clickhouse-driver](https://github.com/mymarilyn/clickhouse-driver), ClickHouse Python Driver with native interface support.
Expand Down
79 changes: 78 additions & 1 deletion asynch/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,81 @@
from asynch.cursors import Cursor, DictCursor
from asynch.pool import Pool

__all__ = ["Connection", "Cursor", "DictCursor", "Pool"]
# Import PEP 249 type objects and constructors
from asynch.dbapi_types import (
STRING, BINARY, NUMBER, DATETIME, ROWID,
Date, Time, Timestamp, DateFromTicks, TimeFromTicks, TimestampFromTicks, Binary
)

# Import PEP 249 exceptions
from asynch.errors import (
Warning,
Error,
InterfaceError,
DatabaseError,
DataError,
OperationalError,
IntegrityError,
InternalError,
ProgrammingError,
NotSupportedError,
)


# PEP 249 module globals
apilevel = "2.0"
threadsafety = 1 # module shareable; connections are per-event-loop
paramstyle = "pyformat" # %(name)s style parameter substitution


def connect(
dsn=None,
user=None,
password=None,
host=None,
port=None,
database=None,
**kwargs,
) -> Connection:
"""Create a new database connection.

This function returns a Connection object. Note that the connection
is not automatically opened - callers must use `await conn.connect()`
or `async with conn` to establish the actual connection.

Args:
dsn: Data Source Name string (takes precedence over individual params)
user: ClickHouse username
password: ClickHouse password
host: ClickHouse server host
port: ClickHouse server port
database: ClickHouse database name
**kwargs: Additional connection parameters

Returns:
Connection: A new Connection object
"""
return Connection(
dsn=dsn,
user=user,
password=password,
host=host,
port=port,
database=database,
**kwargs,
)


__all__ = [
# Core classes
"Connection", "Cursor", "DictCursor", "Pool", "connect",
# Module globals
"apilevel", "threadsafety", "paramstyle",
# Type objects
"STRING", "BINARY", "NUMBER", "DATETIME", "ROWID",
# Type constructors
"Date", "Time", "Timestamp", "DateFromTicks", "TimeFromTicks", "TimestampFromTicks", "Binary",
# Exceptions
"Warning", "Error", "InterfaceError", "DatabaseError", "DataError",
"OperationalError", "IntegrityError", "InternalError", "ProgrammingError", "NotSupportedError",
]
13 changes: 9 additions & 4 deletions asynch/connection.py
Original file line number Diff line number Diff line change
Expand Up @@ -96,11 +96,11 @@ def status(self) -> str:
"""

if not (self._opened or self._closed):
return ConnectionStatus.created
return ConnectionStatus.created.value
if self._opened and not self._closed:
return ConnectionStatus.opened
return ConnectionStatus.opened.value
if self._closed and not self._opened:
return ConnectionStatus.closed
return ConnectionStatus.closed.value
raise ConnectionError(f"{self} is in an unknown state")

@property
Expand Down Expand Up @@ -138,7 +138,12 @@ async def close(self) -> None:
self._closed = True

async def commit(self):
raise NotSupportedError
"""Commit any pending transaction.

ClickHouse does not support transactions, so this is a no-op
that returns None per PEP 249 compliance.
"""
return None

async def connect(self) -> None:
if self._opened:
Expand Down
Loading