[SCHEMA] PET atlas fixes

[bendhouseart]

No description provided.

[bendhouseart]

oh that's failing a lot of the bst tests....

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.07%. Comparing base (308395f) to head (eb1c8c6).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2416   +/-   ##
=======================================
  Coverage   83.07%   83.07%           
=======================================
  Files          22       22           
  Lines        1696     1696           
=======================================
  Hits         1409     1409           
  Misses        287      287           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[bendhouseart]

Removed tpl entity filter for raw rules following Ross' PR to the validator bids-standard/bids-validator#399.

[bendhouseart]

@bendhouseart need to test on Cyrus' atlases examples next.

[effigies]

This is introducing the hemisphere entity into volumetric data. Seems worth a discussion, not a [SCHEMA] PR that describes itself as a fix.

From the data I saw, you have values on a surface, not volumetric data, so the appropriate extension would be .func.gii:

surface_map:
  $ref:
    - meta.templates.deriv.surface
  extensions:
    - .func.gii
    - .json

(I understand that FreeSurfer supports putting surface data in NIfTIs, and NiBabel is capable of reading those files, but it is not a documented NIfTI extension, and I would want to see support from the other major software projects before boosting it in BIDS.)

You would then do:

pet_pet_surface:
  $ref:
    - meta.templates.deriv.surface_map
    - rules.files.raw.pet.pet
  entities:
    $ref:
      - meta.templates.deriv.surface_map.entities
      - rules.files.raw.pet.pet.entities

pet_pet_surface_atlas:
  $ref: rules.files.deriv.imaging.pet_pet_surface
  entities:
    $ref:
      - meta.templates.atlas.base.entities
      - rules.files.deriv.imaging.pet_pet_surface.entities

If it's common to work with CIFTI, you could add .dscalar.nii to the extensions.

[melanieganz]

Dear @bendhouseart ,
I had discussed the above things with @effigies when he was in Cph. I did the changes that Chris requested regarding (hemi and the gifti format instead of nifti). When I test this with the dev validator (https://bids-standard.github.io/bids-validator/dev/) and link it to the ReadTheDocs of this PR (https://bids-specification--2416.org.readthedocs.build/en/2416/schema.json), I still get an error:

Multiple filename rules were found as potential matches. All of them had at least one issue during filename validation.
/tpl-fsaverage/pet/tpl-fsaverage_hemi-L_atlas-NRUSV2A2024_desc-BmaxCOV_pet.func.gii (Rules that matched with issues: rules.files.deriv.imaging.pet_pet_volumetric, rules.files.deriv.imaging.pet_pet_volumetric_atlas, rules.files.raw.pet.pet)

So I am not sure if all issues are resolved? 

[bendhouseart]

Small issues, having removed "anti" selectors following Ross' update with this PR, and yes the dataset I'm validating is a derivative dataset:

{
  "Name": "NRU SV2A atlas converted to BIDS derivatives",
  "BIDSVersion": "1.11.1",
  "DatasetType": "derivative",
  "License": "CC-BY-NC-SA-4.0",
  "GeneratedBy": [
    {
      "Name": "convertNRUatlasses.py",
      "Version": "1.0.0",
      "Description": "Downloads NRU SV2A atlas and renames files to BIDS atlas-derivatives style"
    }
  ]
}

Expand details below to see the error.

Details

anthony@computer:~/Projects/bids-validator$ ./local-run ../bids-examples/atlas-sv2a/ --ignoreNiftiHeaders --ignoreWarnings --schema file:///home/anthony/Projects/bids-validator/atlas_fixe.json --verbose
        [ERROR] MISSING_REQUIRED_ENTITY Missing required entity for files with this suffix.
                /tpl-MNI152/pet/tpl-MNI152_atlas-NRUSV2A2024_desc-BmaxMean_pet.json - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-MNI152/pet/tpl-MNI152_atlas-NRUSV2A2024_desc-BmaxCOV_pet.gii.gz - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-MNI152/pet/tpl-MNI152_atlas-NRUSV2A2024_desc-BmaxSD_pet.json - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-MNI152/pet/tpl-MNI152_atlas-NRUSV2A2024_desc-BmaxSD_pet.gii.gz - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-MNI152/pet/tpl-MNI152_atlas-NRUSV2A2024_desc-BmaxCOV_pet.json - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-MNI152/pet/tpl-MNI152_atlas-NRUSV2A2024_desc-BmaxMean_pet.gii.gz - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-R_atlas-NRUSV2A2024_desc-BmaxCOV_pet.json - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-R_atlas-NRUSV2A2024_desc-BmaxMean_pet.json - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-R_atlas-NRUSV2A2024_desc-BmaxSD_pet.json - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-L_atlas-NRUSV2A2024_desc-BmaxSD_pet.nii.gz - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-L_atlas-NRUSV2A2024_desc-BmaxMean_pet.nii.gz - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-R_atlas-NRUSV2A2024_desc-BmaxMean_pet.nii.gz - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-L_atlas-NRUSV2A2024_desc-BmaxCOV_pet.json - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-R_atlas-NRUSV2A2024_desc-BmaxSD_pet.nii.gz - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-L_atlas-NRUSV2A2024_desc-BmaxCOV_pet.nii.gz - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-R_atlas-NRUSV2A2024_desc-BmaxCOV_pet.nii.gz - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-L_atlas-NRUSV2A2024_desc-BmaxSD_pet.json - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-L_atlas-NRUSV2A2024_desc-BmaxMean_pet.json - sub missing from rule rules.files.raw.pet.pet - rules.files.raw.pet.pet

        Please visit https://neurostars.org/search?q=MISSING_REQUIRED_ENTITY for existing conversations about this issue.

        [ERROR] ENTITY_NOT_IN_RULE Entity not listed as required or optional for files with this suffix
                /tpl-MNI152/pet/tpl-MNI152_atlas-NRUSV2A2024_desc-BmaxMean_pet.json - tpl, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-MNI152/pet/tpl-MNI152_atlas-NRUSV2A2024_desc-BmaxCOV_pet.gii.gz - tpl, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-MNI152/pet/tpl-MNI152_atlas-NRUSV2A2024_desc-BmaxSD_pet.json - tpl, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-MNI152/pet/tpl-MNI152_atlas-NRUSV2A2024_desc-BmaxSD_pet.gii.gz - tpl, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-MNI152/pet/tpl-MNI152_atlas-NRUSV2A2024_desc-BmaxCOV_pet.json - tpl, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-MNI152/pet/tpl-MNI152_atlas-NRUSV2A2024_desc-BmaxMean_pet.gii.gz - tpl, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-R_atlas-NRUSV2A2024_desc-BmaxCOV_pet.json - tpl, hemi, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-R_atlas-NRUSV2A2024_desc-BmaxMean_pet.json - tpl, hemi, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-R_atlas-NRUSV2A2024_desc-BmaxSD_pet.json - tpl, hemi, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-L_atlas-NRUSV2A2024_desc-BmaxSD_pet.nii.gz - tpl, hemi, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-L_atlas-NRUSV2A2024_desc-BmaxMean_pet.nii.gz - tpl, hemi, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-R_atlas-NRUSV2A2024_desc-BmaxMean_pet.nii.gz - tpl, hemi, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-L_atlas-NRUSV2A2024_desc-BmaxCOV_pet.json - tpl, hemi, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-R_atlas-NRUSV2A2024_desc-BmaxSD_pet.nii.gz - tpl, hemi, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-L_atlas-NRUSV2A2024_desc-BmaxCOV_pet.nii.gz - tpl, hemi, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-R_atlas-NRUSV2A2024_desc-BmaxCOV_pet.nii.gz - tpl, hemi, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-L_atlas-NRUSV2A2024_desc-BmaxSD_pet.json - tpl, hemi, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet
                /tpl-fsaverage/pet/tpl-fsaverage_hemi-L_atlas-NRUSV2A2024_desc-BmaxMean_pet.json - tpl, hemi, atlas, desc not in rule rules.files.raw.pet.pet - rules.files.raw.pet.pet

        Please visit https://neurostars.org/search?q=ENTITY_NOT_IN_RULE for existing conversations about this issue.

        [ERROR] EXTENSION_MISMATCH Extension used by file does not match allowed extensions for its suffix
                /tpl-MNI152/pet/tpl-MNI152_atlas-NRUSV2A2024_desc-BmaxCOV_pet.gii.gz - rules.files.raw.pet.pet
                /tpl-MNI152/pet/tpl-MNI152_atlas-NRUSV2A2024_desc-BmaxSD_pet.gii.gz - rules.files.raw.pet.pet
                /tpl-MNI152/pet/tpl-MNI152_atlas-NRUSV2A2024_desc-BmaxMean_pet.gii.gz - rules.files.raw.pet.pet

        Please visit https://neurostars.org/search?q=EXTENSION_MISMATCH for existing conversations about this issue.


          Summary:                         Available Tasks:        Available Modalities:
          20 Files, 5.53 kB                                        PET                  
          0 - Subjects 1 - Sessions                                                     


anthony@computer:~/Projects/bids-validator$ 

Again, not sure what's going on since it lookls like rules.files.raw.pet.pet still is applying to template/derivative folders. Am I misunderstanding what PR 399 to the validator did?

[melanieganz]

I am trying to test the current state with the conversions I made, but I can't get the link to the schema file right when trying to give it as an advanced option to the online validator. How should I link to the schema that you changed above, @bendhouseart? @effigies showed me, but I can't seem to reproduce it.
https://github.com/bids-standard/bids-specification/pull/2416/src/schema.json, but somehow this is not correct.

[bendhouseart]

I am trying to test the current state with the conversions I made, but I can't get the link to the schema file right when trying to give it as an advanced option to the online validator. How should I link to the schema that you changed above, @bendhouseart? @effigies showed me, but I can't seem to reproduce it. https://github.com/bids-standard/bids-specification/pull/2416/src/schema.json, but somehow this is not correct.

This link to the preview should do it -> https://bids-specification--2416.org.readthedocs.build/en/2416/schema.json

I had bundled a schema.json in this PR initially for testing, but it's not necessary given the above.

[melanieganz]

Thanks, I was able to try it now, and I still get an error using this version regarding the filenames:
Multiple filename rules were found as potential matches. All of them had at least one issue during filename validation.

/tpl-fsaverage/pet/tpl-fsaverage_hemi-L_atlas-NRUSV2A2024_desc-BmaxCOV_pet.func.gii (Rules that matched with issues: rules.files.deriv.imaging.pet_pet_volumetric, rules.files.deriv.imaging.pet_pet_volumetric_atlas, rules.files.raw.pet.pet)
 /tpl-fsaverage/pet/tpl-fsaverage_hemi-L_atlas-NRUSV2A2024_desc-BmaxMean_pet.func.gii (Rules that matched with issues: rules.files.deriv.imaging.pet_pet_volumetric, rules.files.deriv.imaging.pet_pet_volumetric_atlas, rules.files.raw.pet.pet)
 /tpl-fsaverage/pet/tpl-fsaverage_hemi-L_atlas-NRUSV2A2024_desc-BmaxSD_pet.func.gii (Rules that matched with issues: rules.files.deriv.imaging.pet_pet_volumetric, rules.files.deriv.imaging.pet_pet_volumetric_atlas, rules.files.raw.pet.pet)
 /tpl-fsaverage/pet/tpl-fsaverage_hemi-R_atlas-NRUSV2A2024_desc-BmaxCOV_pet.func.gii (Rules that matched with issues: rules.files.deriv.imaging.pet_pet_volumetric, rules.files.deriv.imaging.pet_pet_volumetric_atlas, rules.files.raw.pet.pet)
 /tpl-fsaverage/pet/tpl-fsaverage_hemi-R_atlas-NRUSV2A2024_desc-BmaxMean_pet.func.gii (Rules that matched with issues: rules.files.deriv.imaging.pet_pet_volumetric, rules.files.deriv.imaging.pet_pet_volumetric_atlas, rules.files.raw.pet.pet)
 /tpl-fsaverage/pet/tpl-fsaverage_hemi-R_atlas-NRUSV2A2024_desc-BmaxSD_pet.func.gii (Rules that matched with issues: rules.files.deriv.imaging.pet_pet_volumetric, rules.files.deriv.imaging.pet_pet_volumetric_atlas, rules.files.raw.pet.pet)

[bendhouseart]

Oh boo, it's passing validation for the examples here -> bids-standard/bids-examples#552 and bids-standard/bids-examples#557 but my gifti nonsense seems to have broken CI.