Skip to content

Speed optimization validated #34

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mariaelf97 wants to merge 53 commits into
base: main
Choose a base branch
from
Open

Speed optimization validated #34

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
1c4a20f
do not attemp to read the fasta file multiple times
mariaelf97 Jun 12, 2026
d707265
add missing comment
mariaelf97 Jun 12, 2026
7e48113
read reference sequence only once in check-primers function
mariaelf97 Jun 12, 2026
c5000c4
change apply to itertuples for faster operation
mariaelf97 Jun 15, 2026
5267d9e
use pure string reverse compliment instead of Bio Seq reverse complem…
mariaelf97 Jun 15, 2026
55832fe
lint
mariaelf97 Jun 15, 2026
ba2e704
move log message before the primer processing has started
mariaelf97 Jun 15, 2026
d50a638
read count integer coversion not to be repeated in every loop
mariaelf97 Jun 15, 2026
f5b76c3
move ambiguity function outside find_closest_primer_match for more op…
mariaelf97 Jun 15, 2026
d778006
cvhange iterrows with itertuples for faster perfomance
mariaelf97 Jun 15, 2026
64f5d5f
lint
mariaelf97 Jun 15, 2026
ab15c84
parallelize samples for faster performance
mariaelf97 Jun 16, 2026
9a332aa
Efficient merging of fastqfiles
joshuailevy Jun 17, 2026
fbe562b
Efficient merging of fastqfiles-lint
joshuailevy Jun 17, 2026
56d7756
only run reverse search when no forward matches found
mariaelf97 Jun 30, 2026
485d365
lint
mariaelf97 Jun 30, 2026
5c2dd3c
use basic string instead of Bio complimentary reverse
mariaelf97 Jun 30, 2026
9425dfa
have --genomes as an option now that a csv file can be used
mariaelf97 Jul 1, 2026
e43fb8a
read samples and proportions as a cvs file
mariaelf97 Jul 1, 2026
dcecb8a
updating tests
mariaelf97 Jul 1, 2026
a8873ab
Merge branch 'main' into optimization-validated
mariaelf97 Jul 2, 2026
99d4a1a
no need to check if the csv file exists as this results in error when…
mariaelf97 Jul 2, 2026
f8d618c
check the directories after the names are assigned based on whether a…
mariaelf97 Jul 2, 2026
6f8132c
moving directory checking function
mariaelf97 Jul 2, 2026
f550c81
csv file is not true/false type
mariaelf97 Jul 2, 2026
09ff3cd
update tests
mariaelf97 Jul 2, 2026
0509fef
do not store valid combinations to save time
mariaelf97 Jul 2, 2026
16c6e17
lint
mariaelf97 Jul 2, 2026
f445451
assigning proportions based on a multinomial distribution
mariaelf97 Jul 8, 2026
de03609
moving genome quality assessment into the amplicon worker not to read…
mariaelf97 Jul 8, 2026
833eb12
check_primers -do not repeatedly read the fasta file, load sequences …
mariaelf97 Jul 8, 2026
2214c0b
do not need this function imported anymore
mariaelf97 Jul 8, 2026
a0d013a
Adding multifasta option and storing sequences in a dictionary
mariaelf97 Jul 9, 2026
a47f136
update argument validation function
mariaelf97 Jul 9, 2026
2e2f896
updating test files
mariaelf97 Jul 9, 2026
6535f91
update contig names
mariaelf97 Jul 10, 2026
b762713
reference index file
mariaelf97 Jul 10, 2026
2405dcc
lint
mariaelf97 Jul 10, 2026
6d46bcb
add click argument --outdir for check primer function
mariaelf97 Jul 10, 2026
ef59ca7
remove unnecessary imports and only import the worker function in che…
mariaelf97 Jul 10, 2026
bd959c1
adding the multiprocessing function
mariaelf97 Jul 10, 2026
756b7bd
adding worker function for the primer-check parallalization
mariaelf97 Jul 13, 2026
7e460e6
updating sample proportion processing
mariaelf97 Jul 13, 2026
cbcdc2b
update tests
mariaelf97 Jul 13, 2026
df5075b
adding missing flags to the test
mariaelf97 Jul 13, 2026
bfd5e09
remove the extra contig
mariaelf97 Jul 13, 2026
af5ca22
adding the reference flag
mariaelf97 Jul 13, 2026
0d142e9
add the required multifasta file for testing
mariaelf97 Jul 13, 2026
1a677b7
updating test commands
mariaelf97 Jul 13, 2026
59e73b9
reverting the test proportions
mariaelf97 Jul 13, 2026
70146fc
using | pipe sign to distinguish contigs
mariaelf97 Jul 13, 2026
8b50aa8
adjusting gitignore to include vscode generated files
mariaelf97 Jul 14, 2026
6b80479
prevent program exiting if the find valid combinations function failed
mariaelf97 Jul 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ __pycache__/
*.so

# Distribution / packaging
.vscode/
._*
.Python
build/
Expand Down
341 changes: 165 additions & 176 deletions bygul/_cli.py
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions bygul/tests/data/ATM-2FFMD73N3.fasta
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
>hCoV-19/USA/ATM-2FFMD73N3/2021
>ATM-2FFMD73N3
NNNNNNNNNNNNNNNNNNNNNNNNNAACAAACCAACCAACTTTCGATCTCTTGTAGATCT
GTTCTCTAAACGAACTTTAAAATCTGTGTGGCTGTCACTCGGCTGCATGCTTAGTGCACT
CACGCAGTATAATTAATAACTAATTACTGTCGTTGACAGGACACGAGTAACTCGTCTATC
Expand Down Expand Up @@ -497,4 +497,4 @@ GTTAACTTTAATCTCACATAGCAATCTTTAATCAGTGTGTAACATTAGGGAGGACTTGAA
AGAGCCACCACATTTTCACCGAGGCCACTCGGAGTACGATCGAGTGTACAGTGAACAATG
CTAGGGAGAGCTGCCTATATGGAAGAGCCCTAATGTGTAAAATTAATTTTAGTAGTGCTA
TCCCCATGTGATTTTAATAGNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
NNNNNNNNNN
NNNNNNNNNN
2 changes: 1 addition & 1 deletion bygul/tests/data/CA-SEARCH-43254.fasta
View file
Open in desktop

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions bygul/tests/data/reference.fasta.fai
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
NC_045512.2 29903 13 70 71
502 changes: 502 additions & 0 deletions bygul/tests/data/sample_genomes.fasta
View file
Open in desktop

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions bygul/tests/data/sample_proportions.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
sample_name,proportion
ATM-2FFMD73N3,0.54
CA-SEARCH-43254,0.46
34 changes: 25 additions & 9 deletions bygul/tests/data/test_cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ def test_version(self):
def test_simulation(self):
os.system(
"bygul simulate-proportions \
bygul/tests/data/ATM-2FFMD73N3.fasta \
--genomes bygul/tests/data/ATM-2FFMD73N3.fasta \
--primers bygul/tests/data/ARTIC_V4-1.bed \
--reference bygul/tests/data/reference.fasta --redo"
)
Expand All @@ -23,7 +23,7 @@ def test_simulation(self):
def test_simulation_with_simulator(self):
os.system(
"bygul simulate-proportions \
bygul/tests/data/ATM-2FFMD73N3.fasta \
--genomes bygul/tests/data/ATM-2FFMD73N3.fasta \
--primers bygul/tests/data/ARTIC_V4-1.bed \
--reference bygul/tests/data/reference.fasta \
--simulator wgsim --redo"
Expand All @@ -33,7 +33,7 @@ def test_simulation_with_simulator(self):
def test_simulation_with_max_mismatch(self):
os.system(
"bygul simulate-proportions \
bygul/tests/data/ATM-2FFMD73N3.fasta \
--genomes bygul/tests/data/ATM-2FFMD73N3.fasta \
--primers bygul/tests/data/ARTIC_V4-1.bed \
--reference bygul/tests/data/reference.fasta \
--maxmismatch 2 --redo"
Expand All @@ -43,7 +43,7 @@ def test_simulation_with_max_mismatch(self):
def test_simulation_with_readcnt(self):
os.system(
"bygul simulate-proportions \
bygul/tests/data/ATM-2FFMD73N3.fasta \
--genomes bygul/tests/data/ATM-2FFMD73N3.fasta \
--primers bygul/tests/data/ARTIC_V4-1.bed --readcnt 200 \
--reference bygul/tests/data/reference.fasta --redo"
)
Expand All @@ -52,16 +52,17 @@ def test_simulation_with_readcnt(self):
def test_simulation_with_readlength(self):
os.system(
"bygul simulate-proportions \
bygul/tests/data/ATM-2FFMD73N3.fasta \
--genomes bygul/tests/data/ATM-2FFMD73N3.fasta \
--primers bygul/tests/data/ARTIC_V4-1.bed -1 130 \
-2 130 --reference bygul/tests/data/reference.fasta --redo"
"--reference bygul/tests/data/reference.fasta --redo"
)
self.assertTrue(file_exists(".", "results/reads_1.fastq"))

def test_simulation_with_2genomes(self):
os.system(
"bygul simulate-proportions "
"bygul/tests/data/ATM-2FFMD73N3.fasta,"
"--genomes bygul/tests/data/ATM-2FFMD73N3.fasta,"
"bygul/tests/data/KR-SEARCH-120354.fasta "
"--primers bygul/tests/data/ARTIC_V4-1.bed "
"--reference bygul/tests/data/reference.fasta --redo"
Expand All @@ -71,7 +72,7 @@ def test_simulation_with_2genomes(self):
def test_simulation_with_2genomes_proportions(self):
os.system(
"bygul simulate-proportions "
"bygul/tests/data/ATM-2FFMD73N3.fasta,"
"--genomes bygul/tests/data/ATM-2FFMD73N3.fasta,"
"bygul/tests/data/KR-SEARCH-120354.fasta "
"--primers bygul/tests/data/ARTIC_V4-1.bed "
"--reference bygul/tests/data/reference.fasta "
Expand All @@ -82,7 +83,8 @@ def test_simulation_with_2genomes_proportions(self):
def test_simulation_with_metagenomics(self):
os.system(
"bygul simulate-proportions "
"bygul/tests/data/ATM-2FFMD73N3.fasta,"
"--primers bygul/tests/data/ARTIC_V4-1.bed "
"--genomes bygul/tests/data/ATM-2FFMD73N3.fasta,"
"bygul/tests/data/KR-SEARCH-120354.fasta "
"--simulation_mode metagenomics "
"--proportions 0.8,0.2 --redo -1 150 -2 150"
Expand All @@ -92,9 +94,23 @@ def test_simulation_with_metagenomics(self):
def test_simulation_with_mason(self):
os.system(
"bygul simulate-proportions "
"bygul/tests/data/ATM-2FFMD73N3.fasta,"
"--primers bygul/tests/data/ARTIC_V4-1.bed "
"--genomes bygul/tests/data/ATM-2FFMD73N3.fasta,"
"bygul/tests/data/KR-SEARCH-120354.fasta "
"--proportions 0.8,0.2 --redo --simulator mason "
"--reference bygul/tests/data/reference.fasta "
"--illumina-read-length 200"
)
self.assertTrue(file_exists(".", "results/reads_1.fastq"))

def test_simulation_with_csv(self):
os.system(
"bygul simulate-proportions "
"--primers bygul/tests/data/ARTIC_V4-1.bed "
"--csv bygul/tests/data/sample_proportions.csv "
"--multifasta bygul/tests/data/sample_genomes.fasta "
"--redo --simulator mason "
"--reference bygul/tests/data/reference.fasta "
"--illumina-read-length 200"
)
self.assertTrue(file_exists(".", "results/reads_1.fastq"))
Expand Down
2 changes: 1 addition & 1 deletion bygul/tests/data/test_proportions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ def test_prop_with_freyja(self):
# 1. Run the simulation
os.system(
"bygul simulate-proportions "
"bygul/tests/data/BCN-SEARCH-105346.fasta,"
"--genomes bygul/tests/data/BCN-SEARCH-105346.fasta,"
"bygul/tests/data/CA-SEARCH-43254.fasta "
"--primers bygul/tests/data/ARTIC_V4-1.bed "
"--reference bygul/tests/data/reference.fasta "
Expand Down
Loading
Loading