pygrpc-native

A gRPC server for Python that does the HTTP/2 and gRPC work in Rust, outside the GIL.
Your handlers stay in Python.

When a Python gRPC server runs short of CPU, the usual answers are more server processes or a rewrite in another language. pygrpc-native moves the gRPC work itself (HTTP/2, flow control, deadlines, cancellation) into Rust, outside the GIL, and leaves your servicers, interceptors and generated code as they are.

• Switching is one line. pygrpc_native.server() and pygrpc_native.aio.server() take the same arguments as grpcio's servers. Clients don't change.
• It behaves like grpcio. Status codes, metadata, deadlines, cancellation, interceptors and ServicerContext work the same way. The few differences are listed below, and anything else is a bug.
• Nothing stops working. Methods it can't serve itself, such as bidirectional streaming or server reflection, are passed to a grpcio server running behind it.
• It's faster under load. With an empty handler it answers about 1.5x as many calls per second as grpcio from one client thread, and about 5x from 32. See Benchmarks.

Quick start

pip install pygrpc-native

Wheels are available for Linux (x86_64, aarch64) and macOS (arm64, x86_64), for CPython 3.10 and later.

Change the line that creates the server:

 from concurrent import futures
-import grpc
+import pygrpc_native

-server = grpc.server(futures.ThreadPoolExecutor(16))
+server = pygrpc_native.server(futures.ThreadPoolExecutor(16))
 greeter_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
 server.add_insecure_port("[::]:50051")
 server.start()
 server.wait_for_termination()

For grpc.aio:

+import pygrpc_native.aio
+
-server = grpc.aio.server()
+server = pygrpc_native.aio.server()
 greeter_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
 server.add_insecure_port("[::]:50051")
 await server.start()
 await server.wait_for_termination()

For TLS, make the credentials with pygrpc_native too, since grpcio's can't be read back:

-creds = grpc.ssl_server_credentials([(key, cert)])
+creds = pygrpc_native.ssl_server_credentials([(key, cert)])
 server.add_secure_port("[::]:50051", creds)

For runnable sync and asyncio servers with a grpcio client, see examples/README.md.

To build from source, you need Rust 1.88 or later and a C compiler:

pip install maturin
maturin develop --release

How it works

Every call reaches Python as an event: an id, the method, the request bytes and who sent it.

grpc client ──HTTP/2──▶ tokio threads (no GIL)
                          │ path → method id
                          ▼
                    event queue ──▶ recv_batch() in Python
                          ◀── respond(call_id, bytes) from any thread

Python picks up calls in batches, so a burst costs one trip across the GIL instead of one per call. server() and aio.server() are built on top of this and run your handlers the way grpcio does.

What runs natively

	Native	grpcio fallback
Unary and server-streaming methods	✓
Client-streaming and bidirectional methods		✓
Code generated by any grpcio-tools, and handlers added with add_generic_rpc_handlers	✓
Server reflection		✓
Interceptors	✓	✓

The fallback is a grpcio server listening on a Unix socket behind this one. Calls that go there work but aren't any faster, and peer() in them returns the socket. The native server listens on a single port.

Server options

	Options
Same as grpcio	grpc.max_receive_message_length, grpc.max_send_message_length, grpc.keepalive_time_ms, grpc.keepalive_timeout_ms, grpc.keepalive_permit_without_calls, grpc.max_concurrent_streams, grpc.max_metadata_size, grpc.absolute_max_metadata_size, grpc.so_reuseport, maximum_concurrent_rpcs
Raise an error	grpc.max_connection_age_ms, grpc.max_connection_age_grace_ms, grpc.max_connection_idle_ms, grpc.max_allowed_incoming_connections, response compression, xds=True
Ignored	Everything else. grpcio also ignores the many options that only clients read. This includes the limits on client pings, which are accepted but not enforced, and grpc.http2.max_frame_size, which grpc-core's flow control overrides anyway.

As in grpcio, a value that isn't an integer is ignored.

Known differences from grpcio

• If a unary handler calls send_initial_metadata() and then fails, the metadata is sent along with the status, so the client sees it as trailing metadata.
• If a client doesn't answer a keepalive ping, the connection is closed after grpc.keepalive_timeout_ms, as gRPC's keepalive docs describe. grpcio waits for grpc.http2.ping_timeout_ms instead, which is a minute by default.
• Response metadata keys starting with grpc- are reserved by gRPC, so a call that sets one ends with INTERNAL. grpcio sends them.
• A response metadata value with non-ASCII text on a key without -bin is sent as is. grpcio ends that call with INTERNAL.

Notes on aio.server()

• Handlers run on the event loop that called start(), including uvloop.
• abort() raises grpc.aio.AbortError. If the client cancels or the deadline passes, the handler gets asyncio.CancelledError, as in grpc.aio.
• Streaming handlers can yield from an async generator or call context.write().
• Handlers that aren't coroutines run in migration_thread_pool with grpcio's sync context, where abort() returns instead of raising.
• The fallback, the options and the differences above are the same as for the sync server.

Notes on TLS

• Credentials come from pygrpc_native.ssl_server_credentials() and hold one key and certificate chain. You can't replace them while the server runs.
• rustls is stricter about certificates than grpcio. For example, it refuses a client certificate marked as a CA.
• auth_context() doesn't have grpcio's ssl_ keys, such as ssl_session_reused, and rare subject attributes can look different in x509_subject.
• TLS ends at the native server. Methods on the fallback see a local insecure connection and no client certificate.

Low-level API

NativeGrpcFront is the layer under server(). You receive each call as an event and answer it by id:

import threading
from pygrpc_native import EVENT_CALL, NativeGrpcFront

front = NativeGrpcFront("0.0.0.0:50051", [("/echo.Echo/Say", False)])
front.start()

def serve():
    while (events := front.recv_batch(64)) is not None:
        for kind, call_id, method_id, payload, client_key in events:
            if kind == EVENT_CALL:
                front.respond(call_id, payload)

threading.Thread(target=serve).start()

Streaming, call details, response metadata, bearer tokens, TLS and metrics are covered in docs/low-level-api.md.

Benchmarks

An empty unary handler on an Apple M4 Pro, called by grpcio clients on the same machine, with pygrpc-native 0.8.0 and grpcio 1.85.1. Calls per second, median of five runs:

Server	1 client thread	32 client threads (8 processes x 4)
grpc.server()	7,449	11,192
grpc.aio.server()	4,766	10,214
pygrpc_native.server()	10,790	56,834
pygrpc_native.aio.server()	9,736	60,869
Low-level API	12,191	62,664

On a 40-vCPU Linux container with pygrpc-native 0.1.0, the low-level API answered about 12x as many calls as grpcio at 32 threads. The benchmark README has all the numbers and explains how they were measured.

Limitations

• NativeGrpcFront only serves unary and server-streaming methods. Other kinds can go through the fallback server, and server() and aio.server() send them there for you.
• SO_REUSEPORT is set only when grpc.so_reuseport is 1. grpcio sets it by default, so binding a port that another server already uses works with grpcio but fails here.
• Each connection allows up to 1024 concurrent streams unless grpc.max_concurrent_streams says otherwise. A connection that hasn't finished the HTTP/2 preface within 120 s is dropped.
• maximum_concurrent_rpcs counts a call to a method on the fallback server only while its handler runs, while grpcio also counts calls waiting for a worker. With server(), such a call over the limit is refused when a worker picks it up rather than as it arrives.
• A method that only generic handlers know is looked up when it is first called, so their service() runs once more for that call. A method none of them knows is looked up on each call before it goes to the fallback server.
• Calls sent back to back on one connection can reach handlers out of order. gRPC doesn't guarantee an order across calls, though grpcio usually keeps it. NativeGrpcFront(ordered=True) keeps it.
• A call whose request metadata is over the limit (16 KiB unless set) is refused. Between grpc.max_metadata_size and that limit, grpcio also refuses some calls at random. This server doesn't.

Compatibility

pygrpc-native follows semantic versioning, and the low-level API also stays compatible within 0.x:

• The 0.1 names, signatures and event tuples of NativeGrpcFront won't change or go away. tests/test_api.py pins them. New parameters are only added after the existing ones, with a default.
• The keys that metrics() returns won't be renamed or removed within 0.x, though new ones may be added.
• server() and aio.server() follow grpcio's API. Any difference from grpcio's behavior is a bug unless this README lists it.

Changes are listed in CHANGELOG.md.

Contributing

See CONTRIBUTING.md. Every commit needs a DCO sign-off (git commit -s).

License

Apache-2.0