[Feature] Recognition of single "dataset"

[slobentanzer]

Checklist

• There are no similar issues or pull requests for this yet.

Problem

For instance, if open targets parquet are naively ingested, we get single croissant files for each part. In croissant-baker, we have already solved some of this, but who decides what is "one dataset" / "one project"? Seems tricky; can we find heuristics? Do we need to add agents for full automation?

Solution

No response

Alternatives

No response

Anything else?

No response