GATK APPLYBQSRSPARK¶
ApplyBQSRSpark: Apply base quality score recalibration on Spark; uses output of the BaseRecalibrator tool.
Example¶
This wrapper can be used in the following way:
rule gatk_applybqsr_spark:
input:
bam="mapped/{sample}.bam",
ref="genome.fasta",
dict="genome.dict",
recal_table="recal/{sample}.grp",
output:
bam="recal/{sample}.bam",
log:
"logs/gatk/gatk_applybqsr_spark/{sample}.log",
params:
extra="", # optional
java_opts="", # optional
#spark_runner="", # optional, local by default
#spark_v1.19.1="", # optional
#spark_extra="", # optional
embed_ref=True, # embed reference in cram output
exceed_thread_limit=True, # samtools is also parallized and thread limit is not guaranteed anymore
resources:
mem_mb=1024,
wrapper:
"v1.19.1/bio/gatk/applybqsrspark"
Note that input, output and log file paths can be chosen freely.
When running with
snakemake --use-conda
the software dependencies will be automatically deployed into an isolated environment before execution.
Notes¶
- The java_opts param allows for additional arguments to be passed to the java compiler, e.g. “-XX:ParallelGCThreads=10” (not for -XmX or -Djava.io.tmpdir, since they are handled automatically).
- The extra param allows for additional program arguments for applybqsrspark.
- The spark_runner param = “LOCAL”|”SPARK”|”GCS” allows to set the spark_runner. Set the parameter to “LOCAL” or don’t set it at all to run on local machine.
- The spark_master param allows to set the URL of the Spark Master to submit the job. Set to “local[number_of_cores]” for local execution. Don’t set it at all for local execution with number of cores determined by snakemake.
- The spark_extra param allows for additional spark arguments.
- For more information see, https://gatk.broadinstitute.org/hc/en-us/articles/360057440431-ApplyBQSRSpark-BETA-
Software dependencies¶
gatk4=4.3.0.0
openjdk=11.0.15
snakemake-wrapper-utils=0.5.0
samtools=1.16.1
Input/Output¶
Input:
- bam file
- fasta reference
- recalibration table for the bam
Output:
- recalibrated bam file
Authors¶
- Filipe G. Vieira
Code¶
__author__ = "Filipe G. Vieira, Christopher Schröder"
__copyright__ = "Copyright 2021, Filipe G. Vieira"
__license__ = "MIT"
import tempfile
import random
from pathlib import Path
from snakemake.shell import shell
from snakemake_wrapper_utils.java import get_java_opts
extra = snakemake.params.get("extra", "")
spark_runner = snakemake.params.get("spark_runner", "LOCAL")
spark_master = snakemake.params.get(
"spark_master", "local[{}]".format(snakemake.threads)
)
spark_extra = snakemake.params.get("spark_extra", "")
reference = snakemake.input.get("ref")
embed_ref = snakemake.params.get("embed_ref", False)
exceed_thread_limit = snakemake.params.get("exceed_thread_limit", False)
java_opts = get_java_opts(snakemake)
log = snakemake.log_fmt_shell(stdout=True, stderr=True)
if exceed_thread_limit:
samtools_threads = snakemake.threads
else:
samtools_threads = 1
if snakemake.output.bam.endswith(".cram") and embed_ref:
output = "/dev/stdout --create-output-bam-splitting-index false"
pipe_cmd = " | samtools view -h -O cram,embed_ref -T {reference} -o {snakemake.output.bam} -@ {samtools_threads} -"
else:
output = snakemake.output.bam
pipe_cmd = ""
with tempfile.TemporaryDirectory() as tmpdir:
# This folder must not exist; it is created by GATK
tmpdir_shards = Path(tmpdir) / "shards_{:06d}".format(random.randrange(10**6))
shell(
"(gatk --java-options '{java_opts}' ApplyBQSRSpark"
" --input {snakemake.input.bam}"
" --bqsr-recal-file {snakemake.input.recal_table}"
" --reference {snakemake.input.ref}"
" {extra}"
" --tmp-dir {tmpdir}"
" --output-shard-tmp-dir {tmpdir_shards}"
" --output {output}"
" -- --spark-runner {spark_runner} --spark-master {spark_master} {spark_extra}"
+ pipe_cmd
+ ") {log}"
)