RUNIQ

An efficient way to filter duplicate lines from input. GNU uniq, but faster.

URL: https://github.com/whitfin/runiq

Example

This wrapper can be used in the following way:

rule test_runiq:
    input:
        "unsorted_duplicates.txt",
    output:
        "deduplicated.txt",
    log:
        "test_runiq.log",
    threads: 1
    params:
        extra="",
    wrapper:
        "v9.15.0/utils/runiq"

Note that input, output and log file paths can be chosen freely.

When running with

snakemake --use-conda

the software dependencies will be automatically deployed into an isolated environment before execution.

Notes

This tool handles unsorted files, and finds duplicates faster and with lesser RAM than classic ` sort | uniq `.

Software dependencies

• runiq=2.1.0

Input/Output

Input:

• Path to file with duplicates

Output:

• Path to (de)duplicated data

Params

• extra: Optional parameters

Authors

• Thibault Dayris

Code

# coding: utf-8

"""Snakemake wrapper for runiq"""

__author__ = "Thibault Dayris"
__copyright__ = "Copyright 2026, Thibault Dayris"
__email__ = "thibault.dayris@gustaveroussy.fr"
__license__ = "MIT"

from snakemake.shell import shell

log = snakemake.log_fmt_shell(stdout=False, stderr=True)
extra = snakemake.params.get("extra", "")

shell("runiq {extra} {snakemake.input} > {snakemake.output} {log}")