docs: initial FORMAT.md

[kaspar030]

This is Opus 4.6 tasked with "document the file format".

As far as I can tell, the documentation is correct.

[chrysn]

Author was insistent that this is not a troll attempt despite being apparently unredacted LLM output; reviewing this as I would review any contributor's code (and as I'd have expected the author to review his tool's output before PR'ing).

I'm not fact-checking the concrete edits (eg. I don't know if the tool can be invoked on a single file too), I'm focusing on docs writing here, and trust that you probably know those by heart anyway.

[edit: fat-fingered submission; reviews coming in in a following run]

[chrysn]

Not related to any particular point in the text:

How is this supposed to be updated? Many properties stem from the code, that is simply lacking documentation. Once that code receives sensible documentation, this will be outdated.

[chrysn]

Suggested change

	All `.yaml` files in a directory are collected recursively, sorted
	alphabetically, and deep-merged into a single document before parsing. This
	allows splitting configuration across files — e.g., one file for chip lists,
	another per board.
	The `sbd-gen` tool is invoked on a single file or a directory. All given `.yaml` files are
	deep-merged into a single document before parsing. This
	allows splitting configuration across files — e.g., one file for chip lists,
	another per board.

Alphabetical sorting is irrelevant unless deep-merge strategy is that later-overwrites-earlier. What is the deep-merge strategy anyway?

[kaspar030]

deep-merge strategy is:
a) maps are merged
b) later overwrites earlier for everything else.

will add that.

[chrysn]

The concept of targets is better introduced at the start; so far, all this has been talking of boards.

[ROMemories]

I also want to add that a separate Markdown file is not how this should be documented anyway, most of this should be in the rustdoc docs of the sbd-gen-schema crate (as started in #68). This does not technically preclude also having separate Markdown documentation, but we should first have rustdoc docs. (Also if some new Markdown files are introduced, they need to be added into the include keys of the relevant package(s), so they are shipped with them.)

[chrysn]

@kaspar030 and I had a personal chat where we converged, and I'd summarize it as follows (and please correct me if I misrepresent anything):

• We agree that the output of this is not the quality of documentation we expect from our project.
  • Also, there are at least no glaring counterfactual items in there.
• We also agree that not having docs around is not good, and that while adding comments on the struct fields is an improvement, it won't immediately give usable docs on the format out of rustdoc.
• Editing this to satisfy our standards would take a lot of time, and then we'd still be in a maintenance mess of how we update it.
• A big concern with LLM contributions is copyright. Reviewing this concrete file, we're both reasonably confident that this is not infringing on anybody else's works. (This would be harder with the same number of lines of code, where it's not all just domain specific in a domain that Kaspar invented).

A proposed way forward is:

• We mark this (in-file, not just in the link) as something along the lines of roughly "We don't have actual documentation on this yet, so until we do, here's something LLM-generated that we looked at and it's at least not blatantly wrong. We'd appreciate help documenting this properly, but don't waste time editing this concrete file (as it'll be LLM-updated as the source code is moving), let's rather improve the [sbd-gen-schema docs](...)."
• The tool and prompt for creating them should go into at least the commit message – in a sense, they are the source code for this generated file.
• We do not mark it as generated in .gitattribute, because unlike files like sbd-gen's output (which we commit without even looking at them), this needs at least one pair of eyes when next "rebuilt".
• We improve, over time, the in-code documentation (field comments etc); occasionally "rebuilding" this file (and that'll probably pick up on the improvements), including looking for tools that create serialization-suitable documentation out of the information that there is in serde. Eventually, that should replace this file, but especially as long as we're iterating over what we even implement in there quickly, there are more important uses for our time.

[chrysn]

Ah, and w/rt my many review comments: I think they've illustrated the point that this LLM output is not the quality level we expect, and we do agree there. As this file will rather be rebuilt than iterated on, the comments can be dismissed when the new disclaimer in the file is added. (Or if you feel like it, let the model ingest them into a newer version – I won't drive those things, but my comments are contributions to the Ariel project, so feel free to tell it to gobble them up).

They might be useful when we do later do full (rustdoc based) documentation, but most of them are probably about issues we won't have there anyway.

[kaspar030]

I addressed most suggestions, and I think it is OK now for what it is.