diff --git a/.travis.yml b/.travis.yml index bc8037e185a6231eed8da45d830148cdce17f087..b77371b03dae59b22b900792fe7f76b2d07faab1 100644 --- a/.travis.yml +++ b/.travis.yml @@ -14,7 +14,7 @@ before_install: - docker pull nfcore/hic:dev # Fake the tag locally so that the pipeline runs properly # Looks weird when this is :dev to :dev, but makes sense when testing code for a release (:dev to :1.0.1) - - docker tag nfcore/hic:dev nfcore/hic:dev + - docker tag nfcore/hic:dev nfcore/hic:1.1.0 install: # Install Nextflow diff --git a/docs/configuration/adding_your_own.md b/docs/configuration/adding_your_own.md index bf7f80811e08de09bfdca2a94ed1187d1906796f..b1703c1e10d064a0d989e2520642d6e77d742e66 100644 --- a/docs/configuration/adding_your_own.md +++ b/docs/configuration/adding_your_own.md @@ -1,25 +1,48 @@ # nf-core/hic: Configuration for other clusters -It is entirely possible to run this pipeline on other clusters, though you will need to set up your own config file so that the pipeline knows how to work with your cluster. - -> If you think that there are other people using the pipeline who would benefit from your configuration (eg. other common cluster setups), please let us know. We can add a new configuration and profile which can used by specifying `-profile <name>` when running the pipeline. The config file will then be hosted at `nf-core/configs` and will be pulled automatically before the pipeline is executed. - -If you are the only person to be running this pipeline, you can create your config file as `~/.nextflow/config` and it will be applied every time you run Nextflow. Alternatively, save the file anywhere and reference it when running the pipeline with `-c path/to/config` (see the [Nextflow documentation](https://www.nextflow.io/docs/latest/config.html) for more). - -A basic configuration comes with the pipeline, which loads the [`conf/base.config`](../../conf/base.config) by default. This means that you only need to configure the specifics for your system and overwrite any defaults that you want to change. +It is entirely possible to run this pipeline on other clusters, though you will +need to set up your own config file so that the pipeline knows how to work with +your cluster. + +> If you think that there are other people using the pipeline who would benefit +from your configuration (eg. other common cluster setups), please let us know. +We can add a new configuration and profile which can used by specifying +`-profile <name>` when running the pipeline. The config file will then be +hosted at `nf-core/configs` and will be pulled automatically before the pipeline +is executed. + +If you are the only person to be running this pipeline, you can create your +config file as `~/.nextflow/config` and it will be applied every time you run +Nextflow. Alternatively, save the file anywhere and reference it when running +the pipeline with `-c path/to/config` (see the +[Nextflow documentation](https://www.nextflow.io/docs/latest/config.html) +for more). + +A basic configuration comes with the pipeline, which loads the +[`conf/base.config`](../../conf/base.config) by default. This means that you +only need to configure the specifics for your system and overwrite any defaults +that you want to change. ## Cluster Environment -By default, pipeline uses the `local` Nextflow executor - in other words, all jobs are run in the login session. If you're using a simple server, this may be fine. If you're using a compute cluster, this is bad as all jobs will run on the head node. -To specify your cluster environment, add the following line to your config file: +By default, pipeline uses the `local` Nextflow executor - in other words, all +jobs are run in the login session. If you're using a simple server, this may be +fine. If you're using a compute cluster, this is bad as all jobs will run on +the head node. + +To specify your cluster environment, add the following line to your config +file: ```nextflow process.executor = 'YOUR_SYSTEM_TYPE' ``` -Many different cluster types are supported by Nextflow. For more information, please see the [Nextflow documentation](https://www.nextflow.io/docs/latest/executor.html). +Many different cluster types are supported by Nextflow. For more information, +please see the +[Nextflow documentation](https://www.nextflow.io/docs/latest/executor.html). -Note that you may need to specify cluster options, such as a project or queue. To do so, use the `clusterOptions` config option: +Note that you may need to specify cluster options, such as a project or queue. +To do so, use the `clusterOptions` config option: ```nextflow process { @@ -28,16 +51,28 @@ process { } ``` - ## Software Requirements -To run the pipeline, several software packages are required. How you satisfy these requirements is essentially up to you and depends on your system. If possible, we _highly_ recommend using either Docker or Singularity. -Please see the [`installation documentation`](../installation.md) for how to run using the below as a one-off. These instructions are about configuring a config file for repeated use. +To run the pipeline, several software packages are required. How you satisfy +these requirements is essentially up to you and depends on your system. +If possible, we _highly_ recommend using either Docker or Singularity. + +Please see the [`installation documentation`](../installation.md) for how to +run using the below as a one-off. These instructions are about configuring a +config file for repeated use. ### Docker -Docker is a great way to run nf-core/hic, as it manages all software installations and allows the pipeline to be run in an identical software environment across a range of systems. -Nextflow has [excellent integration](https://www.nextflow.io/docs/latest/docker.html) with Docker, and beyond installing the two tools, not much else is required - nextflow will automatically fetch the [nfcore/hic](https://hub.docker.com/r/nfcore/hic/) image that we have created and is hosted at dockerhub at run time. +Docker is a great way to run nf-core/hic, as it manages all software +installations and allows the pipeline to be run in an identical software +environment across a range of systems. + +Nextflow has +[excellent integration](https://www.nextflow.io/docs/latest/docker.html) +with Docker, and beyond installing the two tools, not much else is required - +nextflow will automatically fetch the +[nfcore/hic](https://hub.docker.com/r/nfcore/hic/) image that we have created +and is hosted at dockerhub at run time. To add docker support to your own config file, add the following: @@ -46,12 +81,14 @@ docker.enabled = true process.container = "nfcore/hic" ``` -Note that the dockerhub organisation name annoyingly can't have a hyphen, so is `nfcore` and not `nf-core`. - +Note that the dockerhub organisation name annoyingly can't have a hyphen, +so is `nfcore` and not `nf-core`. ### Singularity image + Many HPC environments are not able to run Docker due to security issues. -[Singularity](http://singularity.lbl.gov/) is a tool designed to run on such HPC systems which is very similar to Docker. +[Singularity](http://singularity.lbl.gov/) is a tool designed to run on such +HPC systems which is very similar to Docker. To specify singularity usage in your pipeline config file, add the following: @@ -60,8 +97,10 @@ singularity.enabled = true process.container = "shub://nf-core/hic" ``` -If you intend to run the pipeline offline, nextflow will not be able to automatically download the singularity image for you. -Instead, you'll have to do this yourself manually first, transfer the image file and then point to that. +If you intend to run the pipeline offline, nextflow will not be able to +automatically download the singularity image for you. +Instead, you'll have to do this yourself manually first, transfer the image +file and then point to that. First, pull the image file where you have an internet connection: @@ -76,9 +115,10 @@ singularity.enabled = true process.container = "/path/to/nf-core-hic.simg" ``` - ### Conda -If you're not able to use Docker or Singularity, you can instead use conda to manage the software requirements. + +If you're not able to use Docker or Singularity, you can instead use conda to +manage the software requirements. To use conda in your own config file, add the following: ```nextflow diff --git a/docs/configuration/local.md b/docs/configuration/local.md index d4530fa9007866b32cf2dda77ed780c4fe19f1e8..c3a047fbd856182b6ce823f8f91548b1a2bccc8a 100644 --- a/docs/configuration/local.md +++ b/docs/configuration/local.md @@ -1,13 +1,24 @@ # nf-core/hic: Local Configuration -If running the pipeline in a local environment, we highly recommend using either Docker or Singularity. +If running the pipeline in a local environment, we highly recommend using +either Docker or Singularity. ## Docker -Docker is a great way to run `nf-core/hic`, as it manages all software installations and allows the pipeline to be run in an identical software environment across a range of systems. -Nextflow has [excellent integration](https://www.nextflow.io/docs/latest/docker.html) with Docker, and beyond installing the two tools, not much else is required. The `nf-core/hic` profile comes with a configuration profile for docker, making it very easy to use. This also comes with the required presets to use the AWS iGenomes resource, meaning that if using common reference genomes you just specify the reference ID and it will be automatically downloaded from AWS S3. +Docker is a great way to run `nf-core/hic`, as it manages all software +installations and allows the pipeline to be run in an identical software +environment across a range of systems. -First, install docker on your system: [Docker Installation Instructions](https://docs.docker.com/engine/installation/) +Nextflow has +[excellent integration](https://www.nextflow.io/docs/latest/docker.html) with +Docker, and beyond installing the two tools, not much else is required. +The `nf-core/hic` profile comes with a configuration profile for docker, making +it very easy to use. This also comes with the required presets to use the AWS +iGenomes resource, meaning that if using common reference genomes you just +specify the reference ID and it will be automatically downloaded from AWS S3. + +First, install docker on your system: +[Docker Installation Instructions](https://docs.docker.com/engine/installation/) Then, simply run the analysis pipeline: @@ -15,26 +26,44 @@ Then, simply run the analysis pipeline: nextflow run nf-core/hic -profile docker --genome '<genome ID>' ``` -Nextflow will recognise `nf-core/hic` and download the pipeline from GitHub. The `-profile docker` configuration lists the [nf-core/hic](https://hub.docker.com/r/nfcore/hic/) image that we have created and is hosted at dockerhub, and this is downloaded. +Nextflow will recognise `nf-core/hic` and download the pipeline from GitHub. +The `-profile docker` configuration lists the +[nf-core/hic](https://hub.docker.com/r/nfcore/hic/) image that we have created +and is hosted at dockerhub, and this is downloaded. -For more information about how to work with reference genomes, see [`docs/configuration/reference_genomes.md`](reference_genomes.md). +For more information about how to work with reference genomes, see +[`docs/configuration/reference_genomes.md`](reference_genomes.md). ### Pipeline versions -The public docker images are tagged with the same version numbers as the code, which you can use to ensure reproducibility. When running the pipeline, specify the pipeline version with `-r`, for example `-r 1.0`. This uses pipeline code and docker image from this tagged version. +The public docker images are tagged with the same version numbers as the code, +which you can use to ensure reproducibility. When running the pipeline, +specify the pipeline version with `-r`, for example `-r 1.0`. This uses +pipeline code and docker image from this tagged version. ## Singularity image -Many HPC environments are not able to run Docker due to security issues. [Singularity](http://singularity.lbl.gov/) is a tool designed to run on such HPC systems which is very similar to Docker. Even better, it can use create images directly from dockerhub. -To use the singularity image for a single run, use `-with-singularity`. This will download the docker container from dockerhub and create a singularity image for you dynamically. +Many HPC environments are not able to run Docker due to security issues. +[Singularity](http://singularity.lbl.gov/) is a tool designed to run on such +HPC systems which is very similar to Docker. Even better, it can use create +images directly from dockerhub. + +To use the singularity image for a single run, use `-with-singularity`. +This will download the docker container from dockerhub and create a singularity +image for you dynamically. -If you intend to run the pipeline offline, nextflow will not be able to automatically download the singularity image for you. Instead, you'll have to do this yourself manually first, transfer the image file and then point to that. +If you intend to run the pipeline offline, nextflow will not be able to +automatically download the singularity image for you. Instead, you'll have +to do this yourself manually first, transfer the image file and then point to +that. First, pull the image file where you have an internet connection: > NB: The "tag" at the end of this command corresponds to the pipeline version. -> Here, we're pulling the docker image for version 1.0 of the nf-core/hic pipeline -> Make sure that this tag corresponds to the version of the pipeline that you're using +> Here, we're pulling the docker image for version 1.0 of the nf-core/hic +pipeline +> Make sure that this tag corresponds to the version of the pipeline that +you're using ```bash singularity pull --name nf-core-hic-1.0.img docker://nf-core/hic:1.0 diff --git a/docs/configuration/reference_genomes.md b/docs/configuration/reference_genomes.md index c52faf821ad7e676ed56dc04c92aae165ea573c7..d584c0c8e0c9000b150406665f7d2edc33615edf 100644 --- a/docs/configuration/reference_genomes.md +++ b/docs/configuration/reference_genomes.md @@ -2,16 +2,23 @@ The nf-core/hic pipeline needs a reference genome for alignment and annotation. -These paths can be supplied on the command line at run time (see the [usage docs](../usage.md)), -but for convenience it's often better to save these paths in a nextflow config file. +These paths can be supplied on the command line at run time (see the +[usage docs](../usage.md)), +but for convenience it's often better to save these paths in a nextflow config +file. See below for instructions on how to do this. -Read [Adding your own system](adding_your_own.md) to find out how to set up custom config files. +Read [Adding your own system](adding_your_own.md) to find out how to set up +custom config files. ## Adding paths to a config file + Specifying long paths every time you run the pipeline is a pain. -To make this easier, the pipeline comes configured to understand reference genome keywords which correspond to preconfigured paths, meaning that you can just specify `--genome ID` when running the pipeline. +To make this easier, the pipeline comes configured to understand reference +genome keywords which correspond to preconfigured paths, meaning that you can +just specify `--genome ID` when running the pipeline. -Note that this genome key can also be specified in a config file if you always use the same genome. +Note that this genome key can also be specified in a config file if you always +use the same genome. To use this system, add paths to your config file using the following template: @@ -25,7 +32,8 @@ params { // [..] } } - // Optional - default genome. Ignored if --genome 'OTHER-GENOME' specified on command line + // Optional - default genome. Ignored if --genome 'OTHER-GENOME' specified + // on command line genome = 'YOUR-ID' } ``` @@ -33,16 +41,26 @@ params { You can add as many genomes as you like as long as they have unique IDs. ## illumina iGenomes -To make the use of reference genomes easier, illumina has developed a centralised resource called [iGenomes](https://support.illumina.com/sequencing/sequencing_software/igenome.html). -Multiple reference index types are held together with consistent structure for multiple genomes. -We have put a copy of iGenomes up onto AWS S3 hosting and this pipeline is configured to use this by default. -The hosting fees for AWS iGenomes are currently kindly funded by a grant from Amazon. -The pipeline will automatically download the required reference files when you run the pipeline. -For more information about the AWS iGenomes, see [AWS-iGenomes](https://ewels.github.io/AWS-iGenomes/) +To make the use of reference genomes easier, illumina has developed a +centralised resource called +[iGenomes](https://support.illumina.com/sequencing/sequencing_software/igenome.html). +Multiple reference index types are held together with consistent structure for +multiple genomes. + +We have put a copy of iGenomes up onto AWS S3 hosting and this pipeline is +configured to use this by default. +The hosting fees for AWS iGenomes are currently kindly funded by a grant from +Amazon. +The pipeline will automatically download the required reference files when you +run the pipeline. +For more information about the AWS iGenomes, see +[AWS-iGenomes](https://ewels.github.io/AWS-iGenomes/) -Downloading the files takes time and bandwidth, so we recommend making a local copy of the iGenomes resource. -Once downloaded, you can customise the variable `params.igenomes_base` in your custom configuration file to point to the reference location. +Downloading the files takes time and bandwidth, so we recommend making a local +copy of the iGenomes resource. +Once downloaded, you can customise the variable `params.igenomes_base` in your +custom configuration file to point to the reference location. For example: ```nextflow diff --git a/environment.yml b/environment.yml index 46dae6523bfe0a6eea9e354f978f77f143aa5fad..1a69ba7ac34662b5d0c7f808d50d7dd5b86899c3 100644 --- a/environment.yml +++ b/environment.yml @@ -1,9 +1,10 @@ # You can use this file to create a conda environment for this pipeline: # conda env create -f environment.yml -name: nf-core-hic-1.1.0dev +name: nf-core-hic-1.1.0 channels: - conda-forge - bioconda + - bioconda/label/cf201901 - defaults dependencies: - python=2.7.15 diff --git a/nextflow.config b/nextflow.config index f624219755d1ca2dcb4accbdf8479ac09a4348f9..37a7d3c86ea1e5af718b17bfdd94f6b2bbe3795d 100644 --- a/nextflow.config +++ b/nextflow.config @@ -45,7 +45,7 @@ params { // Container slug. Stable releases should specify release tag! // Developmental code should specify :dev -process.container = 'nfcore/hic:dev' +process.container = 'nfcore/hic:1.1.0' // Load base.config by default for all pipelines includeConfig 'conf/base.config' @@ -102,7 +102,7 @@ manifest { description = 'Analysis of Chromosome Conformation Capture data (Hi-C)' mainScript = 'main.nf' nextflowVersion = '>=0.32.0' - version = '1.1.0dev' + version = '1.1.0' } // Function to ensure that resource requirements don't go beyond