GATK APPLYBQSRSPARK

ApplyBQSRSpark: Apply base quality score recalibration on Spark; uses output of the BaseRecalibrator tool.

URL:

Example

This wrapper can be used in the following way:

rule gatk_applybqsr_spark:
    input:
        bam="mapped/{sample}.bam",
        ref="genome.fasta",
        dict="genome.dict",
        recal_table="recal/{sample}.grp"
    output:
        bam="recal/{sample}.bam"
    log:
        "logs/gatk/gatk_applybqsr_spark/{sample}.log"
    params:
        extra="",  # optional
        java_opts="", # optional
        #spark_runner="",  # optional, local by default
        #spark_v1.1.0="",  # optional
        #spark_extra="", # optional
    resources:
        mem_mb=1024
    wrapper:
        "v1.1.0/bio/gatk/applybqsrspark"

Note that input, output and log file paths can be chosen freely.

When running with

snakemake --use-conda

the software dependencies will be automatically deployed into an isolated environment before execution.

Software dependencies

  • gatk4=4.2
  • openjdk=8
  • snakemake-wrapper-utils=0.1.3

Input/Output

Input:

  • bam file
  • fasta reference
  • recalibration table for the bam

Output:

  • recalibrated bam file

Notes

  • The java_opts param allows for additional arguments to be passed to the java compiler, e.g. “-XX:ParallelGCThreads=10” (not for -XmX or -Djava.io.tmpdir, since they are handled automatically).
  • The extra param allows for additional program arguments for applybqsrspark.
  • The spark_runner param = “LOCAL”|”SPARK”|”GCS” allows to set the spark_runner. Set the parameter to “LOCAL” or don’t set it at all to run on local machine.
  • The spark_master param allows to set the URL of the Spark Master to submit the job. Set to “local[number_of_cores]” for local execution. Don’t set it at all for local execution with number of cores determined by snakemake.
  • The spark_extra param allows for additional spark arguments.
  • For more information see, https://gatk.broadinstitute.org/hc/en-us/articles/360057440431-ApplyBQSRSpark-BETA-

Authors

  • Filipe G. Vieira

Code

__author__ = "Filipe G. Vieira"
__copyright__ = "Copyright 2021, Filipe G. Vieira"
__license__ = "MIT"

import tempfile
import random
from pathlib import Path

from snakemake.shell import shell
from snakemake_wrapper_utils.java import get_java_opts

extra = snakemake.params.get("extra", "")
spark_runner = snakemake.params.get("spark_runner", "LOCAL")
spark_master = snakemake.params.get(
    "spark_master", "local[{}]".format(snakemake.threads)
)
spark_extra = snakemake.params.get("spark_extra", "")
java_opts = get_java_opts(snakemake)

log = snakemake.log_fmt_shell(stdout=True, stderr=True)

with tempfile.TemporaryDirectory() as tmpdir:
    # This folder must not exist; it is created by GATK
    tmpdir_shards = Path(tmpdir) / "shards_{:06d}".format(random.randrange(10 ** 6))

    shell(
        "gatk --java-options '{java_opts}' ApplyBQSRSpark {extra} "
        "--reference {snakemake.input.ref} --input {snakemake.input.bam} "
        "--bqsr-recal-file {snakemake.input.recal_table} "
        "--tmp-dir {tmpdir} --output-shard-tmp-dir {tmpdir_shards} "
        "--output {snakemake.output.bam} "
        "-- --spark-runner {spark_runner} --spark-master {spark_master} {spark_extra} "
        "{log}"
    )