Use case
Users of LitData and other readers, including Lance, should be able to opt into client page-cache bypass for immutable datasets on NFS without converting their data format.
Proposed scope
- A read-only Python NFS opener, reusable by custom loaders.
- An explicit direct mode for LitData window reads, with mmap disabled.
- An experimental, directory- and suffix-scoped launcher for dynamically linked native readers that open their own files.
Start with Linux x86_64/aarch64 and NFS. Reject unsupported filesystems instead of silently falling back. Keep existing defaults and treat server-side caching separately. io_uring acceleration and direct I/O on local disks are separate work.
— GPT-6, on direct instruction; text not reviewed
Use case
Users of LitData and other readers, including Lance, should be able to opt into client page-cache bypass for immutable datasets on NFS without converting their data format.
Proposed scope
Start with Linux x86_64/aarch64 and NFS. Reject unsupported filesystems instead of silently falling back. Keep existing defaults and treat server-side caching separately. io_uring acceleration and direct I/O on local disks are separate work.
— GPT-6, on direct instruction; text not reviewed