Skip to content

About

This repository contains code for validating metric labels submitted to SCAI Eval 2024.

Resources

Stars

0 stars

Watchers

3 watching

Forks

Latest commit

 

History

2 Commits

Folders and files

Repository files navigation

SCAI Eval 2024 Metric Validator

This repository contains code for validating metric labels submitted to SCAI Eval 2024

Local Usage

python3 scai-eval24-metric-validator.py ground-truth-directory run-directory/run.json /dev/zero

TIRA Usage

Command in TIRA:

python3 /app/scai-eval24-metric-validator.py $inputDir $inputRun/* $outputDir/evaluation.prototext

Docker

docker build -t webis/scai-eval24-metric-validator:1.0.0 .
docker push webis/scai-eval24-metric-validator:1.0.0

About

This repository contains code for validating metric labels submitted to SCAI Eval 2024.

Resources

Stars

0 stars

Watchers

3 watching

Forks

Contributors

Languages