Skip to content
Snippets Groups Projects
Commit 13837a18 authored by nservant's avatar nservant
Browse files

bump v1.1.0

parent 3d578833
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
.travis.yml
+ 1
− 1
View file @ 13837a18
......@@ -14,7 +14,7 @@ before_install:
- docker pull nfcore/hic:dev
# Fake the tag locally so that the pipeline runs properly
# Looks weird when this is :dev to :dev, but makes sense when testing code for a release (:dev to :1.0.1)
- docker tag nfcore/hic:dev nfcore/hic:dev
- docker tag nfcore/hic:dev nfcore/hic:1.1.0
install:
# Install Nextflow
......
docs/configuration/adding_your_own.md
+ 63
− 23
View file @ 13837a18
# nf-core/hic: Configuration for other clusters
It is entirely possible to run this pipeline on other clusters, though you will need to set up your own config file so that the pipeline knows how to work with your cluster.
> If you think that there are other people using the pipeline who would benefit from your configuration (eg. other common cluster setups), please let us know. We can add a new configuration and profile which can used by specifying `-profile <name>` when running the pipeline. The config file will then be hosted at `nf-core/configs` and will be pulled automatically before the pipeline is executed.
If you are the only person to be running this pipeline, you can create your config file as `~/.nextflow/config` and it will be applied every time you run Nextflow. Alternatively, save the file anywhere and reference it when running the pipeline with `-c path/to/config` (see the [Nextflow documentation](https://www.nextflow.io/docs/latest/config.html) for more).
A basic configuration comes with the pipeline, which loads the [`conf/base.config`](../../conf/base.config) by default. This means that you only need to configure the specifics for your system and overwrite any defaults that you want to change.
It is entirely possible to run this pipeline on other clusters, though you will
need to set up your own config file so that the pipeline knows how to work with
your cluster.
> If you think that there are other people using the pipeline who would benefit
from your configuration (eg. other common cluster setups), please let us know.
We can add a new configuration and profile which can used by specifying
`-profile <name>` when running the pipeline. The config file will then be
hosted at `nf-core/configs` and will be pulled automatically before the pipeline
is executed.
If you are the only person to be running this pipeline, you can create your
config file as `~/.nextflow/config` and it will be applied every time you run
Nextflow. Alternatively, save the file anywhere and reference it when running
the pipeline with `-c path/to/config` (see the
[Nextflow documentation](https://www.nextflow.io/docs/latest/config.html)
for more).
A basic configuration comes with the pipeline, which loads the
[`conf/base.config`](../../conf/base.config) by default. This means that you
only need to configure the specifics for your system and overwrite any defaults
that you want to change.
## Cluster Environment
By default, pipeline uses the `local` Nextflow executor - in other words, all jobs are run in the login session. If you're using a simple server, this may be fine. If you're using a compute cluster, this is bad as all jobs will run on the head node.
To specify your cluster environment, add the following line to your config file:
By default, pipeline uses the `local` Nextflow executor - in other words, all
jobs are run in the login session. If you're using a simple server, this may be
fine. If you're using a compute cluster, this is bad as all jobs will run on
the head node.
To specify your cluster environment, add the following line to your config
file:
```nextflow
process.executor = 'YOUR_SYSTEM_TYPE'
```
Many different cluster types are supported by Nextflow. For more information, please see the [Nextflow documentation](https://www.nextflow.io/docs/latest/executor.html).
Many different cluster types are supported by Nextflow. For more information,
please see the
[Nextflow documentation](https://www.nextflow.io/docs/latest/executor.html).
Note that you may need to specify cluster options, such as a project or queue. To do so, use the `clusterOptions` config option:
Note that you may need to specify cluster options, such as a project or queue.
To do so, use the `clusterOptions` config option:
```nextflow
process {
......@@ -28,16 +51,28 @@ process {
}
```
## Software Requirements
To run the pipeline, several software packages are required. How you satisfy these requirements is essentially up to you and depends on your system. If possible, we _highly_ recommend using either Docker or Singularity.
Please see the [`installation documentation`](../installation.md) for how to run using the below as a one-off. These instructions are about configuring a config file for repeated use.
To run the pipeline, several software packages are required. How you satisfy
these requirements is essentially up to you and depends on your system.
If possible, we _highly_ recommend using either Docker or Singularity.
Please see the [`installation documentation`](../installation.md) for how to
run using the below as a one-off. These instructions are about configuring a
config file for repeated use.
### Docker
Docker is a great way to run nf-core/hic, as it manages all software installations and allows the pipeline to be run in an identical software environment across a range of systems.
Nextflow has [excellent integration](https://www.nextflow.io/docs/latest/docker.html) with Docker, and beyond installing the two tools, not much else is required - nextflow will automatically fetch the [nfcore/hic](https://hub.docker.com/r/nfcore/hic/) image that we have created and is hosted at dockerhub at run time.
Docker is a great way to run nf-core/hic, as it manages all software
installations and allows the pipeline to be run in an identical software
environment across a range of systems.
Nextflow has
[excellent integration](https://www.nextflow.io/docs/latest/docker.html)
with Docker, and beyond installing the two tools, not much else is required -
nextflow will automatically fetch the
[nfcore/hic](https://hub.docker.com/r/nfcore/hic/) image that we have created
and is hosted at dockerhub at run time.
To add docker support to your own config file, add the following:
......@@ -46,12 +81,14 @@ docker.enabled = true
process.container = "nfcore/hic"
```
Note that the dockerhub organisation name annoyingly can't have a hyphen, so is `nfcore` and not `nf-core`.
Note that the dockerhub organisation name annoyingly can't have a hyphen,
so is `nfcore` and not `nf-core`.
### Singularity image
Many HPC environments are not able to run Docker due to security issues.
[Singularity](http://singularity.lbl.gov/) is a tool designed to run on such HPC systems which is very similar to Docker.
[Singularity](http://singularity.lbl.gov/) is a tool designed to run on such
HPC systems which is very similar to Docker.
To specify singularity usage in your pipeline config file, add the following:
......@@ -60,8 +97,10 @@ singularity.enabled = true
process.container = "shub://nf-core/hic"
```
If you intend to run the pipeline offline, nextflow will not be able to automatically download the singularity image for you.
Instead, you'll have to do this yourself manually first, transfer the image file and then point to that.
If you intend to run the pipeline offline, nextflow will not be able to
automatically download the singularity image for you.
Instead, you'll have to do this yourself manually first, transfer the image
file and then point to that.
First, pull the image file where you have an internet connection:
......@@ -76,9 +115,10 @@ singularity.enabled = true
process.container = "/path/to/nf-core-hic.simg"
```
### Conda
If you're not able to use Docker or Singularity, you can instead use conda to manage the software requirements.
If you're not able to use Docker or Singularity, you can instead use conda to
manage the software requirements.
To use conda in your own config file, add the following:
```nextflow
......
docs/configuration/local.md
+ 41
− 12
View file @ 13837a18
# nf-core/hic: Local Configuration
If running the pipeline in a local environment, we highly recommend using either Docker or Singularity.
If running the pipeline in a local environment, we highly recommend using
either Docker or Singularity.
## Docker
Docker is a great way to run `nf-core/hic`, as it manages all software installations and allows the pipeline to be run in an identical software environment across a range of systems.
Nextflow has [excellent integration](https://www.nextflow.io/docs/latest/docker.html) with Docker, and beyond installing the two tools, not much else is required. The `nf-core/hic` profile comes with a configuration profile for docker, making it very easy to use. This also comes with the required presets to use the AWS iGenomes resource, meaning that if using common reference genomes you just specify the reference ID and it will be automatically downloaded from AWS S3.
Docker is a great way to run `nf-core/hic`, as it manages all software
installations and allows the pipeline to be run in an identical software
environment across a range of systems.
First, install docker on your system: [Docker Installation Instructions](https://docs.docker.com/engine/installation/)
Nextflow has
[excellent integration](https://www.nextflow.io/docs/latest/docker.html) with
Docker, and beyond installing the two tools, not much else is required.
The `nf-core/hic` profile comes with a configuration profile for docker, making
it very easy to use. This also comes with the required presets to use the AWS
iGenomes resource, meaning that if using common reference genomes you just
specify the reference ID and it will be automatically downloaded from AWS S3.
First, install docker on your system:
[Docker Installation Instructions](https://docs.docker.com/engine/installation/)
Then, simply run the analysis pipeline:
......@@ -15,26 +26,44 @@ Then, simply run the analysis pipeline:
nextflow run nf-core/hic -profile docker --genome '<genome ID>'
```
Nextflow will recognise `nf-core/hic` and download the pipeline from GitHub. The `-profile docker` configuration lists the [nf-core/hic](https://hub.docker.com/r/nfcore/hic/) image that we have created and is hosted at dockerhub, and this is downloaded.
Nextflow will recognise `nf-core/hic` and download the pipeline from GitHub.
The `-profile docker` configuration lists the
[nf-core/hic](https://hub.docker.com/r/nfcore/hic/) image that we have created
and is hosted at dockerhub, and this is downloaded.
For more information about how to work with reference genomes, see [`docs/configuration/reference_genomes.md`](reference_genomes.md).
For more information about how to work with reference genomes, see
[`docs/configuration/reference_genomes.md`](reference_genomes.md).
### Pipeline versions
The public docker images are tagged with the same version numbers as the code, which you can use to ensure reproducibility. When running the pipeline, specify the pipeline version with `-r`, for example `-r 1.0`. This uses pipeline code and docker image from this tagged version.
The public docker images are tagged with the same version numbers as the code,
which you can use to ensure reproducibility. When running the pipeline,
specify the pipeline version with `-r`, for example `-r 1.0`. This uses
pipeline code and docker image from this tagged version.
## Singularity image
Many HPC environments are not able to run Docker due to security issues. [Singularity](http://singularity.lbl.gov/) is a tool designed to run on such HPC systems which is very similar to Docker. Even better, it can use create images directly from dockerhub.
To use the singularity image for a single run, use `-with-singularity`. This will download the docker container from dockerhub and create a singularity image for you dynamically.
Many HPC environments are not able to run Docker due to security issues.
[Singularity](http://singularity.lbl.gov/) is a tool designed to run on such
HPC systems which is very similar to Docker. Even better, it can use create
images directly from dockerhub.
To use the singularity image for a single run, use `-with-singularity`.
This will download the docker container from dockerhub and create a singularity
image for you dynamically.
If you intend to run the pipeline offline, nextflow will not be able to automatically download the singularity image for you. Instead, you'll have to do this yourself manually first, transfer the image file and then point to that.
If you intend to run the pipeline offline, nextflow will not be able to
automatically download the singularity image for you. Instead, you'll have
to do this yourself manually first, transfer the image file and then point to
that.
First, pull the image file where you have an internet connection:
> NB: The "tag" at the end of this command corresponds to the pipeline version.
> Here, we're pulling the docker image for version 1.0 of the nf-core/hic pipeline
> Make sure that this tag corresponds to the version of the pipeline that you're using
> Here, we're pulling the docker image for version 1.0 of the nf-core/hic
pipeline
> Make sure that this tag corresponds to the version of the pipeline that
you're using
```bash
singularity pull --name nf-core-hic-1.0.img docker://nf-core/hic:1.0
......
docs/configuration/reference_genomes.md
+ 32
− 14
View file @ 13837a18
......@@ -2,16 +2,23 @@
The nf-core/hic pipeline needs a reference genome for alignment and annotation.
These paths can be supplied on the command line at run time (see the [usage docs](../usage.md)),
but for convenience it's often better to save these paths in a nextflow config file.
These paths can be supplied on the command line at run time (see the
[usage docs](../usage.md)),
but for convenience it's often better to save these paths in a nextflow config
file.
See below for instructions on how to do this.
Read [Adding your own system](adding_your_own.md) to find out how to set up custom config files.
Read [Adding your own system](adding_your_own.md) to find out how to set up
custom config files.
## Adding paths to a config file
Specifying long paths every time you run the pipeline is a pain.
To make this easier, the pipeline comes configured to understand reference genome keywords which correspond to preconfigured paths, meaning that you can just specify `--genome ID` when running the pipeline.
To make this easier, the pipeline comes configured to understand reference
genome keywords which correspond to preconfigured paths, meaning that you can
just specify `--genome ID` when running the pipeline.
Note that this genome key can also be specified in a config file if you always use the same genome.
Note that this genome key can also be specified in a config file if you always
use the same genome.
To use this system, add paths to your config file using the following template:
......@@ -25,7 +32,8 @@ params {
// [..]
}
}
// Optional - default genome. Ignored if --genome 'OTHER-GENOME' specified on command line
// Optional - default genome. Ignored if --genome 'OTHER-GENOME' specified
// on command line
genome = 'YOUR-ID'
}
```
......@@ -33,16 +41,26 @@ params {
You can add as many genomes as you like as long as they have unique IDs.
## illumina iGenomes
To make the use of reference genomes easier, illumina has developed a centralised resource called [iGenomes](https://support.illumina.com/sequencing/sequencing_software/igenome.html).
Multiple reference index types are held together with consistent structure for multiple genomes.
We have put a copy of iGenomes up onto AWS S3 hosting and this pipeline is configured to use this by default.
The hosting fees for AWS iGenomes are currently kindly funded by a grant from Amazon.
The pipeline will automatically download the required reference files when you run the pipeline.
For more information about the AWS iGenomes, see [AWS-iGenomes](https://ewels.github.io/AWS-iGenomes/)
To make the use of reference genomes easier, illumina has developed a
centralised resource called
[iGenomes](https://support.illumina.com/sequencing/sequencing_software/igenome.html).
Multiple reference index types are held together with consistent structure for
multiple genomes.
We have put a copy of iGenomes up onto AWS S3 hosting and this pipeline is
configured to use this by default.
The hosting fees for AWS iGenomes are currently kindly funded by a grant from
Amazon.
The pipeline will automatically download the required reference files when you
run the pipeline.
For more information about the AWS iGenomes, see
[AWS-iGenomes](https://ewels.github.io/AWS-iGenomes/)
Downloading the files takes time and bandwidth, so we recommend making a local copy of the iGenomes resource.
Once downloaded, you can customise the variable `params.igenomes_base` in your custom configuration file to point to the reference location.
Downloading the files takes time and bandwidth, so we recommend making a local
copy of the iGenomes resource.
Once downloaded, you can customise the variable `params.igenomes_base` in your
custom configuration file to point to the reference location.
For example:
```nextflow
......
environment.yml
+ 1
− 1
View file @ 13837a18
# You can use this file to create a conda environment for this pipeline:
# conda env create -f environment.yml
name: nf-core-hic-1.1.0dev
name: nf-core-hic-1.1.0
channels:
- conda-forge
- bioconda
......
nextflow.config
+ 2
− 2
View file @ 13837a18
......@@ -45,7 +45,7 @@ params {
// Container slug. Stable releases should specify release tag!
// Developmental code should specify :dev
process.container = 'nfcore/hic:dev'
process.container = 'nfcore/hic:1.1.0'
// Load base.config by default for all pipelines
includeConfig 'conf/base.config'
......@@ -102,7 +102,7 @@ manifest {
description = 'Analysis of Chromosome Conformation Capture data (Hi-C)'
mainScript = 'main.nf'
nextflowVersion = '>=0.32.0'
version = '1.1.0dev'
version = '1.1.0'
}
// Function to ensure that resource requirements don't go beyond
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment