Repository navigation
Error in rownames - AberrantExpression module #606
Description
Hi , I am getting the below error while running aberrant expression module and the command I use to run the module is :
snakemake aberrantExpression --cores 40 -k --rerun-trigger mtime
error message :
[Wed Apr 9 11:32:31 2025]
rule AberrantExpression_pipeline_Counting_filterCounts_R:
input: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline_GRCH37_50million/Output/processed_data/aberrant_expression/v47/outrider/control/total_counts.Rds, /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline_GRCH37_50million/Output/processed_data/preprocess/v47/txdb.db, Scripts/AberrantExpression/pipeline/Counting/filterCounts.R
output: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline_GRCH37_50million/Output/processed_results/aberrant_expression/v47/outrider/control/ods_unfitted.Rds
log: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline_GRCH37_50million/.drop/tmp/AE/v47/control/filter.Rds
jobid: 3
reason: Missing output files: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline_GRCH37_50million/Output/processed_results/aberrant_expression/v47/outrider/control/ods_unfitted.Rds
wildcards: annotation=v47, dataset=control
resources: tmpdir=/tmpError in
rownames<-(*tmp*, value = value[[2]]) :
missing values not allowed in rownames
Calls: OutriderDataSet ... dimnames<- -> dimnames<- -> rownames<- -> rownames<-
Execution halted
[Wed Apr 9 11:32:43 2025]
Error in rule AberrantExpression_pipeline_Counting_filterCounts_R:
jobid: 80
input: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline_GRCH37_50million/Output/processed_data/aberrant_expression/v47/outrider/case/total_counts.Rds, /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline_GRCH37_50million/Output/processed_data/preprocess/v47/txdb.db, Scripts/AberrantExpression/pipeline/Counting/filterCounts.R
output: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline_GRCH37_50million/Output/processed_results/aberrant_expression/v47/outrider/case/ods_unfitted.Rds
log: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline_GRCH37_50million/.drop/tmp/AE/v47/case/filter.Rds (check log file(s) for error details)RuleException:
CalledProcessError in file /tmp/tmpy6272umh, line 48:
Command 'set -euo pipefail; Rscript --vanilla /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline_GRCH37_50million/.snakemake/scripts/tmph3csk9eu.filterCounts.R' returned non-zero exit status 1.
File "/tmp/tmpy6272umh", line 48, in __rule_AberrantExpression_pipeline_Counting_filterCounts_R
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/concurrent/futures/thread.py", line 59, in run
Error inrownames<-(*tmp*, value = value[[2]]) :
missing values not allowed in rownames
Calls: OutriderDataSet ... dimnames<- -> dimnames<- -> rownames<- -> rownames<-
Execution halted
[Wed Apr 9 11:32:43 2025]
Error in rule AberrantExpression_pipeline_Counting_filterCounts_R:
jobid: 3
input: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline_GRCH37_50million/Output/processed_data/aberrant_expression/v47/outrider/control/total_counts.Rds, /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline_GRCH37_50million/Output/processed_data/preprocess/v47/txdb.db, Scripts/AberrantExpression/pipeline/Counting/filterCounts.R
output: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline_GRCH37_50million/Output/processed_results/aberrant_expression/v47/outrider/control/ods_unfitted.Rds
log: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline_GRCH37_50million/.drop/tmp/AE/v47/control/filter.Rds (check log file(s) for error details)RuleException:
CalledProcessError in file /tmp/tmpy6272umh, line 48:
Command 'set -euo pipefail; Rscript --vanilla /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline_GRCH37_50million/.snakemake/scripts/tmp_owmlz5x.filterCounts.R' returned non-zero exit status 1.
File "/tmp/tmpy6272umh", line 48, in __rule_AberrantExpression_pipeline_Counting_filterCounts_R
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/concurrent/futures/thread.py", line 59, in run
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2025-04-09T113228.727115.snakemake.log
Activity
vyepez88 commentedon Apr 10, 2025
Hi, the following error
usually means it ran out of memory.
In addition, check the
fpkmparameter among the aberrantExpression ones. The default and recommended is 1.Finally, check whether your provided gtf file has no empty gene names. You can check the file under
{root}/processed_data/preprocess/{gene_annotation}/gene_name_mapping_{gene_annotation}.tsvfor thisRoshanshafiha commentedon May 13, 2025
Hi @vyepez88 I increased the memory and the error still persist. The other checks are good too but the error message still persist.
whats this line trying to check exactly do you have any idea which part of the script and I could look into it. Thanks
Roshanshafiha commentedon May 15, 2025
Hi @vyepez88 may I know if there is any update on the above issue , thank you.
AtaJadidAhari commentedon May 15, 2025
Hi, can you check to see if you have missing or duplicated sampleIDs?
vyepez88 commentedon May 16, 2025
I still think it's a memory issue. Can you try running 10 samples only?
Roshanshafiha commentedon May 19, 2025
@vyepez88 I have 46 samples , should I remove 36 of them from the annotation file and try running ?
@AtaJadidAhari the sample ID count is the same in the below file and the sample annotation file /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/pipelines/DROP_pipeline_GRCH37_50million/Output/processed_data/aberrant_expression/v47/outrider/control/total_counts.Rds
vyepez88 commentedon May 19, 2025
@Roshanshafiha you can create a new DROP_GROUP with those 10 samples. Otherwise, yes, create a new sample annotation with only 10 samples.
Roshanshafiha commentedon May 19, 2025
@vyepez88 I would also like to add that in the annotation file I have added the INDIVIDUAL_ID column and when I run without this, I don't get this error
vyepez88 commentedon May 19, 2025
that's extremely weird, the
INDIVIDUAL_IDcolumn is never used in DROP. Maybe there's a problem in that column (e.g. a special character or a tab)vyepez88 commentedon Aug 18, 2025
Hi @Roshanshafiha, was this issue sorted out?