For example, maybe we figure out 90th percentile of value and clip everything there? Or take a look at distribution of values by channel and consider a maximum cut? I anticipate you'll have to modify the normalization code in the dataloader to calculate some other useful observable. I would test on the small dataset first.
Also may help to review what we did back in 2019 https://www.science.org/doi/10.1126/sciadv.aaw6548
For example, maybe we figure out 90th percentile of value and clip everything there? Or take a look at distribution of values by channel and consider a maximum cut? I anticipate you'll have to modify the normalization code in the dataloader to calculate some other useful observable. I would test on the small dataset first.
Also may help to review what we did back in 2019 https://www.science.org/doi/10.1126/sciadv.aaw6548