Skip to content

Use @ADDMEMORY@ / @ADDTHREADS@ macros appropriately across samtools c… #7902

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wm75 wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Use @ADDMEMORY@ / @ADDTHREADS@ macros appropriately across samtools c… #7902

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3ed329c
Use @ADDMEMORY@ / @ADDTHREADS@ macros appropriately across samtools c…
wm75 Apr 17, 2026
e9242c9
Do necessary wrapper version bumps
wm75 Apr 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions tool_collections/samtools/samtools_ampliconclip/samtools_ampliconclip.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
<expand macro="version_command"/>
<command><![CDATA[
@ADDTHREADS@
@ADDMEMORY@
samtools ampliconclip
$hard_clip_mode
#if $min_length:
Expand All @@ -16,17 +17,17 @@
--tolerance $tolerance
$strand
-b '${input_bed}'
-u
$both_ends
-u
$both_ends
$no_excluded
#if $write_primer_counts:
--primer-counts '${primer_counts}'
#end if
-@ \$addthreads
'${input_bam}'
| samtools collate -@ \$addthreads -O -u -
| samtools fixmate -@ \$addthreads -u - -
| samtools sort -@ \$addthreads -m \${GALAXY_MEMORY_MB:-768}M -T "\${TMPDIR:-.}" -o '${output_bam}'
| samtools fixmate -@ \$addthreads -u - -
| samtools sort -@ \$addthreads -m \$addmemory"M" -T "\${TMPDIR:-.}" -o '${output_bam}'
Copy link
Copy Markdown
Member Author

@wm75 wm75 Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please note that here the change improves things, but the entire pipe using multiple threads at each step seems questionable.
@bernt-matthias maybe you have an idea how to fix this properly?

Copy link
Copy Markdown
Contributor

@bernt-matthias bernt-matthias Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are ampliconclip, collate and fixmate compute intense tasks that need a lot of memory / CPUs?

Maybe just remove the calls of collate, fixmate and sort completelty. It's not documented that these tools run anyway. If we would want to fix all problems that are potentially introduced by ampliconclip the docs suggest that we also should call calmd`... Let the user do this?

Copy link
Copy Markdown
Member Author

@wm75 wm75 Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, I still have never used the tool.
I agree with your points, but then lets do this separately. I will try to understand what's required next week and then open a new PR with those changes.

]]></command>
<inputs>
<param name="input_bed" type="data" format="bed" label="Genetic intervals (in BED format)" />
Expand Down
3 changes: 2 additions & 1 deletion tool_collections/samtools/samtools_fastx/samtools_fastx.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,10 @@
<expand macro="version_command" />
<command><![CDATA[
@ADDTHREADS@
@ADDMEMORY@
## name sort input file if neccessary
#if not $input.is_of_type('qname_sorted.bam', 'qname_input_sorted.bam'):
samtools sort -@ \$addthreads -n '$input' -T "\${TMPDIR:-.}" > input &&
samtools sort -@ \$addthreads -m \$addmemory"M" -n '$input' -T "\${TMPDIR:-.}" > input &&
#else:
ln -s '$input' input &&
#end if
Expand Down
3 changes: 2 additions & 1 deletion tool_collections/samtools/samtools_fixmate/samtools_fixmate.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,11 @@
<expand macro="version_command"/>
<command><![CDATA[
@ADDTHREADS@
@ADDMEMORY@
## name sort input
#if not $bamfile.is_of_type('qname_sorted.bam', 'qnamed_input_sorted.bam'):
samtools sort
-@ \$addthreads -m \${GALAXY_MEMORY_MB:-768}M -T "\${TMPDIR:-.}"
-@ \$addthreads -m \$addmemory"M" -T "\${TMPDIR:-.}"
-n
-O BAM
-o namesorted.bam
Expand Down
3 changes: 2 additions & 1 deletion tool_collections/samtools/samtools_markdup/samtools_markdup.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,11 @@
<expand macro="version_command"/>
<command><![CDATA[
@ADDTHREADS@
@ADDMEMORY@
## coordinate sort input
#if not $bamfile.is_of_type('bam'):
samtools sort
-@ \$addthreads -m \${GALAXY_MEMORY_MB:-768}M -T "\${TMPDIR:-.}"
-@ \$addthreads -m \$addmemory"M" -T "\${TMPDIR:-.}"
-O sam
-o coordsort.sam
'$bamfile' &&
Expand Down
3 changes: 2 additions & 1 deletion tool_collections/samtools/samtools_merge/samtools_merge.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,14 @@
<expand macro="version_command"/>
<command><![CDATA[
@ADDTHREADS@
@ADDMEMORY@
## prepare input (sort sam, link other), determine input ordering
## and check for consistent input ordering
#set sortby=''
#for $i, $bam in enumerate( $bamfiles ):
#if $bam.is_of_type('sam', 'cram', ):
samtools sort
-@ \$addthreads -m \${GALAXY_MEMORY_MB:-768}M -T "\${TMPDIR:-.}"
-@ \$addthreads -m \$addmemory"M" -T "\${TMPDIR:-.}"
-O sam
-o ${i}.sam
'$bam' &&
Expand Down
11 changes: 7 additions & 4 deletions tool_collections/samtools/samtools_slice_bam/samtools_slice_bam.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,18 @@
<expand macro="version_command"/>

<command><![CDATA[
@ADDTHREADS@
@ADDMEMORY@
ln -s '${input_bam}' temp_input.bam &&
ln -s '${input_bam.metadata.bam_index}' temp_input.bam.bai &&

#if str($slice_method.slice_method_selector) == "bed":
samtools view -@ \${GALAXY_SLOTS:-1} -b -L "${input_interval}" -o unsorted_output.bam temp_input.bam &&
samtools view -@ \$addthreads -b -L "${input_interval}" -o unsorted_output.bam temp_input.bam &&
#elif str($slice_method.slice_method_selector) == "chr":
samtools view -@ \${GALAXY_SLOTS:-1} -b -o unsorted_output.bam temp_input.bam
samtools view -@ \$addthreads -b -o unsorted_output.bam temp_input.bam
${ ' '.join( map( lambda x:'"%s"' % ( x ), str( $slice_method.refs ).split(",") ) ) } &&
#elif str($slice_method.slice_method_selector) == "man":
samtools view -@ \${GALAXY_SLOTS:-1} -b -o unsorted_output.bam temp_input.bam
samtools view -@ \$addthreads -b -o unsorted_output.bam temp_input.bam

#for $region in $slice_method.regions:
"${region.chrom}:${region.start}-${region.end}"
Expand All @@ -29,7 +31,8 @@
samtools sort
-O bam
-T "\${TMPDIR:-.}"
-@ \${GALAXY_SLOTS:-1}
-@ \$addthreads
-m \$addmemory"M"
-o '${output_bam}'
unsorted_output.bam
]]></command>
Expand Down
2 changes: 1 addition & 1 deletion tool_collections/samtools/samtools_sort/samtools_sort.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
<expand macro="version_command"/>
<command><![CDATA[
@ADDTHREADS@
@ADDMEMORY@
@ADDMEMORY@
samtools sort
-@ \$addthreads
-m \$addmemory"M"
Expand Down
Loading