Source and computation of w2vbert2_mean_var_stats_emilia.pt

[VartulBahuguna]

I am using DualCodec and noticed that the released checkpoints require the file:

w2vbert2_mean_var_stats_emilia.pt

After inspecting it, it contains:

{
"mean": tensor([1024]),
"var": tensor([1024])
}

I would like to understand exactly how these statistics were computed.

1. Are these the mean and variance of W2V-BERT hidden representations computed on the Emilia dataset?

2. If so, which hidden layer was used (e.g., Layer 16)?

3. Were the statistics computed directly on the hidden states, or on some post-processed representation?

4. If using different training dataset such as LibriSpeech, should separate mean/variance statistics be recomputed?

Thank you.