Add DB test harness

[wagoodman]

This works towards replacing the existing test mocks that sit above the DB layer with a new test harness that supports:

• building a DB for one or more schemas from vunnel test fixture data
• managing DB caches from previous test runs
• be able to update vunnel test fixture material from vunnel caches
• test assertion helpers, pkg builder helpers, and distro constants to keep ver to keep the verbosity of test code to a minimum
• test helpers have completion capabilities on by default (so we check we assert on the entire result) and this does not require the test to be written with the knowledge of the data shapes in the top-level API (which will help with future refactors).
• manage local testdata fixtures or shared fixtures (across packages) to reuse vunnel material
• sub-select vunnel material as input in to the DB build (allowing for fewer duplicate fixtures)

The nice thing about the testdata directories used as input to the builds is that they are simply vunnel workspaces, so should still be 100% interoperable with the vunnel CLI if need be (pending a small vunnel fix inbound for windows compatibility).

This PR starts with replacing only the dpkg matcher tests and mocks with this new approach, however, these refactors will be rolled through the remaining matcher and integration tests.

In terms of writing tests, the helpers keep the verbosity down a lot, while giving us a nice chokepoint for future refactors to take place (instead of full test rewrites). Here's an example of writing a test using the new helpers:

func TestMatcherDpkg_DirectMatch(t *testing.T) {
	dbtest.SharedDBs(t, "all").
		SelectOnly("debian:11/CVE-2024-0727").
		Run(func(t *testing.T, db *dbtest.DB) {
			matcher := Matcher{}

			p := dbtest.NewPackage("openssl", "1.1.1k-1", syftPkg.DebPkg). // vulnerable (< 1.1.1w-0+deb11u2)
											WithDistro(dbtest.Debian11).
											Build()

			db.Match(t, &matcher, p).
				SelectMatch("CVE-2024-0727").
				SelectDetailByType(match.ExactDirectMatch).
				AsDistroSearch()
		})
}

I feel that as we port more matcher tests over we will refine the helper functions considerably.

Note

Reviewers might want to take a look at the dbtest readme that is on the branch. This explains a lot of how this would be used.

TODO

• update CI workflows to upload DB caches

[kzantow]

Is there a benefit of having the Run callback instead of just something like:

db := dbtest.SharedDBs(t, "all").Build()

[willmurphyscode]

I had the same question.

[wagoodman]

There is a Build() method similar to what you have there, only it returns a slice of DBs. One thing baked into this is the ability to build/test with multiple schemas. This allows a single chokepoint where we can control which schemas as used by default for testing (or can be overridden by arguments). Having the Run() method allows us to not really change the test body but we can test multiple subjects -- it's only saving test writers from boilerplate of for each db build call t.Run with the DB schema name and test name....

[kzantow]

This looks like patterns will always query as part of the ID, not the contents? Is this for exploring the results or something else?

[wagoodman]

not against adding that -- lets do that in a follow up PR