Add a generic message reader interface

[meroton-benjamin]

Adds a generic message reader interface for reading proto messages from storage. Many Buildbarn components depend on blobstore.BlobAccess only as a source of proto.Message and this interface allows us to sever that dependency in components which are otherwise not interested in the blobstore.BlobAccess interface.

[EdSchouten]

What's with the msg T argument? Normally you only provide that to ToProto() to indicate the type you're interested in. For MessageReader that should already be implied by the interface itself.

[meroton-benjamin]

msg T argument removed in favor of adding an allocator to the constructor.

[meroton-benjamin]

State of latest version:

• MessageReader moved to pkg/cas
• Non-CAS uses of the MessageReader interface has been removed.
• The storage message has been removed from the function signature and instead the constructor takes an allocator.

Looking at places which reads messages from the CAS in bb-storage, there only seems to be CompletenessCheckingBlobAccess and NestedBlobReplicator. In CompletenessCheckingBlobAccess, it actually only fetches tree objects in a streaming fashion from the CAS while NestedBlobReplicator fetches tree objects in a streaming fashion and Directory/Action objects in a non streaming fashion.

In the last commit here Use message reader in bb-copy a resolution to this has been done by changing the NestedBlobReplicator constructor to:

func NewNestedBlobReplicator(replicator CASReplicator, maximumMessageSizeBytes int, actionReader cas.MessageReader[*remoteexecution.Action], directoryReader cas.MessageReader[*remoteexecution.Directory], treeReader BlobStreamReader, digestKeyFormat digest.KeyFormat)

With:

type CASReplicator interface {
	ReplicateMultiple(ctx context.Context, digests digest.Set) error
}

type BlobStreamReader interface {
	ReadStream(ctx context.Context, d digest.Digest) (io.ReadCloser, error)
}

Which allows me to tie up the loop. Does this look like a good signature and if so I can consume the BlobStreamReader interface from CompletenessCheckingBlobAccess as well.

[EdSchouten]

If you change this line to:

type blobAccessMessageReader[T any, TPtr interface {
        *T
        proto.Message
}] struct {

Then you can use T to refer to a concrete message type, and TPtr whenever you need a pointer to one. That way you can get rid of the allocator function.

[EdSchouten]

If you wanted to, you could change this proto.Message to any. That way you get rid of the dependency on the Protobuf package. Though in this case I'm ambivalent.

[EdSchouten]

I'm just thinking: aren't we going to need an interface like this on the NaiveBuildDirectory side to download files? If so, maybe this should also live in pkg/cas?