diff --git a/.github/.dockstore.yml b/.github/.dockstore.yml
new file mode 100644
index 0000000000000000000000000000000000000000..030138a0ca97a91f378c4cd4d55e79ac4de1dc55
--- /dev/null
+++ b/.github/.dockstore.yml
@@ -0,0 +1,5 @@
+# Dockstore config version, not pipeline version
+version: 1.2
+workflows:
+ - subclass: nfl
+ primaryDescriptorPath: /nextflow.config
diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md
index fda99de7282aba62eefbddccdcfef67fa67bb1a9..25ef2ed3c3c87f3ab115e92de6d830fdb6718b4d 100644
--- a/.github/CONTRIBUTING.md
+++ b/.github/CONTRIBUTING.md
@@ -1,45 +1,128 @@
# nf-core/hic: Contributing Guidelines
-Hi there! Many thanks for taking an interest in improving nf-core/hic.
+Hi there!
+Many thanks for taking an interest in improving nf-core/hic.
-We try to manage the required tasks for nf-core/hic using GitHub issues, you probably came to this page when creating one. Please use the pre-filled template to save time.
+We try to manage the required tasks for nf-core/hic using GitHub issues, you probably came to this page when creating one.
+Please use the pre-filled template to save time.
-However, don't be put off by this template - other more general issues and suggestions are welcome! Contributions to the code are even more welcome ;)
+However, don't be put off by this template - other more general issues and suggestions are welcome!
+Contributions to the code are even more welcome ;)
-> If you need help using or modifying nf-core/hic then the best place to go is the Gitter chatroom where you can ask us questions directly: https://gitter.im/nf-core/Lobby
+> If you need help using or modifying nf-core/hic then the best place to ask is on the nf-core Slack [#hic](https://nfcore.slack.com/channels/hic) channel ([join our Slack here](https://nf-co.re/join/slack)).
## Contribution workflow
-If you'd like to write some code for nf-core/hic, the standard workflow
-is as follows:
-1. Check that there isn't already an issue about your idea in the
- [nf-core/hic issues](https://github.com/nf-core/hic/issues) to avoid
- duplicating work.
- * If there isn't one already, please create one so that others know you're working on this
-2. Fork the [nf-core/hic repository](https://github.com/nf-core/hic) to your GitHub account
-3. Make the necessary changes / additions within your forked repository
-4. Submit a Pull Request against the `dev` branch and wait for the code to be reviewed and merged.
+If you'd like to write some code for nf-core/hic, the standard workflow is as follows:
-If you're not used to this workflow with git, you can start with some [basic docs from GitHub](https://help.github.com/articles/fork-a-repo/) or even their [excellent interactive tutorial](https://try.github.io/).
+1. Check that there isn't already an issue about your idea in the [nf-core/hic issues](https://github.com/nf-core/hic/issues) to avoid duplicating work
+ * If there isn't one already, please create one so that others know you're working on this
+2. [Fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the [nf-core/hic repository](https://github.com/nf-core/hic) to your GitHub account
+3. Make the necessary changes / additions within your forked repository following [Pipeline conventions](#pipeline-contribution-conventions)
+4. Use `nf-core schema build .` and add any new parameters to the pipeline JSON schema (requires [nf-core tools](https://github.com/nf-core/tools) >= 1.10).
+5. Submit a Pull Request against the `dev` branch and wait for the code to be reviewed and merged
+If you're not used to this workflow with git, you can start with some [docs from GitHub](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests) or even their [excellent `git` resources](https://try.github.io/).
## Tests
-When you create a pull request with changes, [Travis CI](https://travis-ci.org/) will run automatic tests.
+
+When you create a pull request with changes, [GitHub Actions](https://github.com/features/actions) will run automatic tests.
Typically, pull-requests are only fully reviewed when these tests are passing, though of course we can help out before then.
There are typically two types of tests that run:
-### Lint Tests
-The nf-core has a [set of guidelines](http://nf-co.re/guidelines) which all pipelines must adhere to.
+### Lint tests
+
+`nf-core` has a [set of guidelines](https://nf-co.re/developers/guidelines) which all pipelines must adhere to.
To enforce these and ensure that all pipelines stay in sync, we have developed a helper tool which runs checks on the pipeline code. This is in the [nf-core/tools repository](https://github.com/nf-core/tools) and once installed can be run locally with the `nf-core lint <pipeline-directory>` command.
If any failures or warnings are encountered, please follow the listed URL for more documentation.
-### Pipeline Tests
-Each nf-core pipeline should be set up with a minimal set of test-data.
-Travis CI then runs the pipeline on this data to ensure that it exists successfully.
+### Pipeline tests
+
+Each `nf-core` pipeline should be set up with a minimal set of test-data.
+`GitHub Actions` then runs the pipeline on this data to ensure that it exits successfully.
If there are any failures then the automated tests fail.
-These tests are run both with the latest available version of Nextflow and also the minimum required version that is stated in the pipeline code.
+These tests are run both with the latest available version of `Nextflow` and also the minimum required version that is stated in the pipeline code.
+
+## Patch
+
+:warning: Only in the unlikely and regretful event of a release happening with a bug.
+
+* On your own fork, make a new branch `patch` based on `upstream/master`.
+* Fix the bug, and bump version (X.Y.Z+1).
+* A PR should be made on `master` from patch to directly this particular bug.
## Getting help
-For further information/help, please consult the [nf-core/hic documentation](https://github.com/nf-core/hic#documentation) and don't hesitate to get in touch on [Gitter](https://gitter.im/nf-core/Lobby)
+
+For further information/help, please consult the [nf-core/hic documentation](https://nf-co.re/hic/usage) and don't hesitate to get in touch on the nf-core Slack [#hic](https://nfcore.slack.com/channels/hic) channel ([join our Slack here](https://nf-co.re/join/slack)).
+
+## Pipeline contribution conventions
+
+To make the nf-core/hic code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.
+
+### Adding a new step
+
+If you wish to contribute a new step, please use the following coding standards:
+
+1. Define the corresponding input channel into your new process from the expected previous process channel
+2. Write the process block (see below).
+3. Define the output channel if needed (see below).
+4. Add any new flags/options to `nextflow.config` with a default (see below).
+5. Add any new flags/options to `nextflow_schema.json` with help text (with `nf-core schema build .`)
+6. Add any new flags/options to the help message (for integer/text parameters, print to help the corresponding `nextflow.config` parameter).
+7. Add sanity checks for all relevant parameters.
+8. Add any new software to the `scrape_software_versions.py` script in `bin/` and the version command to the `scrape_software_versions` process in `main.nf`.
+9. Do local tests that the new code works properly and as expected.
+10. Add a new test command in `.github/workflow/ci.yaml`.
+11. If applicable add a [MultiQC](https://https://multiqc.info/) module.
+12. Update MultiQC config `assets/multiqc_config.yaml` so relevant suffixes, name clean up, General Statistics Table column order, and module figures are in the right order.
+13. Optional: Add any descriptions of MultiQC report sections and output files to `docs/output.md`.
+
+### Default values
+
+Parameters should be initialised / defined with default values in `nextflow.config` under the `params` scope.
+
+Once there, use `nf-core schema build .` to add to `nextflow_schema.json`.
+
+### Default processes resource requirements
+
+Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/%7B%7Bcookiecutter.name_noslash%7D%7D/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.
+
+The process resources can be passed on to the tool dynamically within the process with the `${task.cpu}` and `${task.memory}` variables in the `script:` block.
+
+### Naming schemes
+
+Please use the following naming schemes, to make it easy to understand what is going where.
+
+* initial process channel: `ch_output_from_<process>`
+* intermediate and terminal channels: `ch_<previousprocess>_for_<nextprocess>`
+
+### Nextflow version bumping
+
+If you are using a new feature from core Nextflow, you may bump the minimum required version of nextflow in the pipeline with: `nf-core bump-version --nextflow . [min-nf-version]`
+
+### Software version reporting
+
+If you add a new tool to the pipeline, please ensure you add the information of the tool to the `get_software_version` process.
+
+Add to the script block of the process, something like the following:
+
+```bash
+<YOUR_TOOL> --version &> v_<YOUR_TOOL>.txt 2>&1 || true
+```
+
+or
+
+```bash
+<YOUR_TOOL> --help | head -n 1 &> v_<YOUR_TOOL>.txt 2>&1 || true
+```
+
+You then need to edit the script `bin/scrape_software_versions.py` to:
+
+1. Add a Python regex for your tool's `--version` output (as in stored in the `v_<YOUR_TOOL>.txt` file), to ensure the version is reported as a `v` and the version number e.g. `v2.1.1`
+2. Add a HTML entry to the `OrderedDict` for formatting in MultiQC.
+
+### Images and figures
+
+For overview images and other documents we follow the nf-core [style guidelines and examples](https://nf-co.re/developers/design_guidelines).
diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md
index 8112c95f4b2e8d2c3aeda5b539529608fe9213e8..5ac2f7f52734df2cde27f756dcf7de5e4525463c 100644
--- a/.github/ISSUE_TEMPLATE/bug_report.md
+++ b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -1,31 +1,64 @@
+---
+name: Bug report
+about: Report something that is broken or incorrect
+labels: bug
+---
+
+<!--
+# nf-core/hic bug report
+
Hi there!
-Thanks for telling us about a problem with the pipeline. Please delete this text and anything that's not relevant from the template below:
+Thanks for telling us about a problem with the pipeline.
+Please delete this text and anything that's not relevant from the template below:
+-->
+
+## Check Documentation
+
+I have checked the following places for your error:
-#### Describe the bug
-A clear and concise description of what the bug is.
+- [ ] [nf-core website: troubleshooting](https://nf-co.re/usage/troubleshooting)
+- [ ] [nf-core/hic pipeline documentation](https://nf-co.re/nf-core/hic/usage)
+
+## Description of the bug
+
+<!-- A clear and concise description of what the bug is. -->
+
+## Steps to reproduce
-#### Steps to reproduce
Steps to reproduce the behaviour:
-1. Command line: `nextflow run ...`
-2. See error: _Please provide your error message_
-#### Expected behaviour
-A clear and concise description of what you expected to happen.
+1. Command line: <!-- [e.g. `nextflow run ...`] -->
+2. See error: <!-- [Please provide your error message] -->
+
+## Expected behaviour
+
+<!-- A clear and concise description of what you expected to happen. -->
+
+## Log files
+
+Have you provided the following extra information/files:
+
+- [ ] The command used to run the pipeline
+- [ ] The `.nextflow.log` file <!-- this is a hidden file in the directory where you launched the pipeline -->
+
+## System
+
+- Hardware: <!-- [e.g. HPC, Desktop, Cloud...] -->
+- Executor: <!-- [e.g. slurm, local, awsbatch...] -->
+- OS: <!-- [e.g. CentOS Linux, macOS, Linux Mint...] -->
+- Version <!-- [e.g. 7, 10.13.6, 18.3...] -->
+
+## Nextflow Installation
+
+- Version: <!-- [e.g. 19.10.0] -->
-#### System:
- - Hardware: [e.g. HPC, Desktop, Cloud...]
- - Executor: [e.g. slurm, local, awsbatch...]
- - OS: [e.g. CentOS Linux, macOS, Linux Mint...]
- - Version [e.g. 7, 10.13.6, 18.3...]
+## Container engine
-#### Nextflow Installation:
- - Version: [e.g. 0.31.0]
+- Engine: <!-- [e.g. Conda, Docker, Singularity or Podman] -->
+- version: <!-- [e.g. 1.0.0] -->
+- Image tag: <!-- [e.g. nfcore/hic:1.0.0] -->
-#### Container engine:
- - Engine: [e.g. Conda, Docker or Singularity]
- - version: [e.g. 1.0.0]
- - Image tag: [e.g. nfcore/hic:1.0.0]
+## Additional context
-#### Additional context
-Add any other context about the problem here.
+<!-- Add any other context about the problem here. -->
diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml
new file mode 100644
index 0000000000000000000000000000000000000000..887f04598ba6bb758d35d2ca79012ccc8129ff34
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/config.yml
@@ -0,0 +1,8 @@
+blank_issues_enabled: false
+contact_links:
+ - name: Join nf-core
+ url: https://nf-co.re/join
+ about: Please join the nf-core community here
+ - name: "Slack #hic channel"
+ url: https://nfcore.slack.com/channels/hic
+ about: Discussion about the nf-core/hic pipeline
diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md
index 1f025b779cf127b420c972c1e385e4efcdd56321..2e01a5fe11f6ed4f3e5bfb4bcaff8c8b7bdc56d5 100644
--- a/.github/ISSUE_TEMPLATE/feature_request.md
+++ b/.github/ISSUE_TEMPLATE/feature_request.md
@@ -1,16 +1,32 @@
+---
+name: Feature request
+about: Suggest an idea for the nf-core website
+labels: enhancement
+---
+
+<!--
+# nf-core/hic feature request
+
Hi there!
-Thanks for suggesting a new feature for the pipeline! Please delete this text and anything that's not relevant from the template below:
+Thanks for suggesting a new feature for the pipeline!
+Please delete this text and anything that's not relevant from the template below:
+-->
+
+## Is your feature request related to a problem? Please describe
+
+<!-- A clear and concise description of what the problem is. -->
+
+<!-- e.g. [I'm always frustrated when ...] -->
+
+## Describe the solution you'd like
+
+<!-- A clear and concise description of what you want to happen. -->
-#### Is your feature request related to a problem? Please describe.
-A clear and concise description of what the problem is.
-Ex. I'm always frustrated when [...]
+## Describe alternatives you've considered
-#### Describe the solution you'd like
-A clear and concise description of what you want to happen.
+<!-- A clear and concise description of any alternative solutions or features you've considered. -->
-#### Describe alternatives you've considered
-A clear and concise description of any alternative solutions or features you've considered.
+## Additional context
-#### Additional context
-Add any other context about the feature request here.
+<!-- Add any other context about the feature request here. -->
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
index 473c41dcc78813e0ed7867d92322be210da06b0c..1b480fa1ad0db48ab9e5e605f11ec859944778be 100644
--- a/.github/PULL_REQUEST_TEMPLATE.md
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -1,15 +1,26 @@
-Many thanks to contributing to nf-core/hic!
+<!--
+# nf-core/hic pull request
-Please fill in the appropriate checklist below (delete whatever is not relevant). These are the most common things requested on pull requests (PRs).
+Many thanks for contributing to nf-core/hic!
+
+Please fill in the appropriate checklist below (delete whatever is not relevant).
+These are the most common things requested on pull requests (PRs).
+
+Remember that PRs should be made against the dev branch, unless you're preparing a pipeline release.
+
+Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/hic/tree/master/.github/CONTRIBUTING.md)
+-->
## PR checklist
- - [ ] This comment contains a description of changes (with reason)
- - [ ] If you've fixed a bug or added code that should be tested, add tests!
- - [ ] If necessary, also make a PR on the [nf-core/hic branch on the nf-core/test-datasets repo]( https://github.com/nf-core/test-datasets/pull/new/nf-core/hic)
- - [ ] Ensure the test suite passes (`nextflow run . -profile test,docker`).
- - [ ] Make sure your code lints (`nf-core lint .`).
- - [ ] Documentation in `docs` is updated
- - [ ] `CHANGELOG.md` is updated
- - [ ] `README.md` is updated
-
-**Learn more about contributing:** https://github.com/nf-core/hic/tree/master/.github/CONTRIBUTING.md
+
+- [ ] This comment contains a description of changes (with reason).
+- [ ] If you've fixed a bug or added code that should be tested, add tests!
+ - [ ] If you've added a new tool - add to the software_versions process and a regex to `scrape_software_versions.py`
+ - [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/hic/tree/master/.github/CONTRIBUTING.md)
+ - [ ] If necessary, also make a PR on the nf-core/hic _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
+- [ ] Make sure your code lints (`nf-core lint .`).
+- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker`).
+- [ ] Usage Documentation in `docs/usage.md` is updated.
+- [ ] Output Documentation in `docs/output.md` is updated.
+- [ ] `CHANGELOG.md` is updated.
+- [ ] `README.md` is updated (including new tool citations and authors/contributors).
diff --git a/.github/markdownlint.yml b/.github/markdownlint.yml
new file mode 100644
index 0000000000000000000000000000000000000000..8d7eb53b07463c24bd981a479a7d0591fabf7463
--- /dev/null
+++ b/.github/markdownlint.yml
@@ -0,0 +1,12 @@
+# Markdownlint configuration file
+default: true
+line-length: false
+no-duplicate-header:
+ siblings_only: true
+no-inline-html:
+ allowed_elements:
+ - img
+ - p
+ - kbd
+ - details
+ - summary
diff --git a/.github/workflows/awsfulltest.yml b/.github/workflows/awsfulltest.yml
new file mode 100644
index 0000000000000000000000000000000000000000..e63ea697aa1c0b27438360e3186e6ddaf2b4afee
--- /dev/null
+++ b/.github/workflows/awsfulltest.yml
@@ -0,0 +1,43 @@
+name: nf-core AWS full size tests
+# This workflow is triggered on published releases.
+# It can be additionally triggered manually with GitHub actions workflow dispatch.
+# It runs the -profile 'test_full' on AWS batch
+
+on:
+ workflow_run:
+ workflows: ["nf-core Docker push (release)"]
+ types: [completed]
+ workflow_dispatch:
+
+jobs:
+ run-awstest:
+ name: Run AWS full tests
+ if: github.repository == 'nf-core/hic'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Setup Miniconda
+ uses: conda-incubator/setup-miniconda@v2
+ with:
+ auto-update-conda: true
+ python-version: 3.7
+ - name: Install awscli
+ run: conda install -c conda-forge awscli
+ - name: Start AWS batch job
+ # TODO nf-core: You can customise AWS full pipeline tests as required
+ # Add full size test data (but still relatively small datasets for few samples)
+ # on the `test_full.config` test runs with only one set of parameters
+ # Then specify `-profile test_full` instead of `-profile test` on the AWS batch command
+ env:
+ AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+ AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+ TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
+ AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
+ AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
+ AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
+ run: |
+ aws batch submit-job \
+ --region eu-west-1 \
+ --job-name nf-core-hic \
+ --job-queue $AWS_JOB_QUEUE \
+ --job-definition $AWS_JOB_DEFINITION \
+ --container-overrides '{"command": ["nf-core/hic", "-r '"${GITHUB_SHA}"' -profile test --outdir s3://'"${AWS_S3_BUCKET}"'/hic/results-'"${GITHUB_SHA}"' -w s3://'"${AWS_S3_BUCKET}"'/hic/work-'"${GITHUB_SHA}"' -with-tower"], "environment": [{"name": "TOWER_ACCESS_TOKEN", "value": "'"$TOWER_ACCESS_TOKEN"'"}]}'
diff --git a/.github/workflows/awstest.yml b/.github/workflows/awstest.yml
new file mode 100644
index 0000000000000000000000000000000000000000..92dfa0ddccbebce1ffae4aca4d631f295cb0e601
--- /dev/null
+++ b/.github/workflows/awstest.yml
@@ -0,0 +1,39 @@
+name: nf-core AWS test
+# This workflow is triggered on push to the master branch.
+# It can be additionally triggered manually with GitHub actions workflow dispatch.
+# It runs the -profile 'test' on AWS batch.
+
+on:
+ workflow_dispatch:
+
+jobs:
+ run-awstest:
+ name: Run AWS tests
+ if: github.repository == 'nf-core/hic'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Setup Miniconda
+ uses: conda-incubator/setup-miniconda@v2
+ with:
+ auto-update-conda: true
+ python-version: 3.7
+ - name: Install awscli
+ run: conda install -c conda-forge awscli
+ - name: Start AWS batch job
+ # TODO nf-core: You can customise CI pipeline run tests as required
+ # For example: adding multiple test runs with different parameters
+ # Remember that you can parallelise this by using strategy.matrix
+ env:
+ AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+ AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+ TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
+ AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
+ AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
+ AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
+ run: |
+ aws batch submit-job \
+ --region eu-west-1 \
+ --job-name nf-core-hic \
+ --job-queue $AWS_JOB_QUEUE \
+ --job-definition $AWS_JOB_DEFINITION \
+ --container-overrides '{"command": ["nf-core/hic", "-r '"${GITHUB_SHA}"' -profile test --outdir s3://'"${AWS_S3_BUCKET}"'/hic/results-'"${GITHUB_SHA}"' -w s3://'"${AWS_S3_BUCKET}"'/hic/work-'"${GITHUB_SHA}"' -with-tower"], "environment": [{"name": "TOWER_ACCESS_TOKEN", "value": "'"$TOWER_ACCESS_TOKEN"'"}]}'
diff --git a/.github/workflows/branch.yml b/.github/workflows/branch.yml
new file mode 100644
index 0000000000000000000000000000000000000000..92136a144c3e48bc1a11a7c893f2d69f958dd65d
--- /dev/null
+++ b/.github/workflows/branch.yml
@@ -0,0 +1,37 @@
+name: nf-core branch protection
+# This workflow is triggered on PRs to master branch on the repository
+# It fails when someone tries to make a PR against the nf-core `master` branch instead of `dev`
+on:
+ pull_request_target:
+ branches: [master]
+
+jobs:
+ test:
+ runs-on: ubuntu-latest
+ steps:
+ # PRs to the nf-core repo master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches
+ - name: Check PRs
+ if: github.repository == 'nf-core/hic'
+ run: |
+ { [[ ${{github.event.pull_request.head.repo.full_name}} == nf-core/hic ]] && [[ $GITHUB_HEAD_REF = "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
+
+
+ # If the above check failed, post a comment on the PR explaining the failure
+ # NOTE - this doesn't currently work if the PR is coming from a fork, due to limitations in GitHub actions secrets
+ - name: Post PR comment
+ if: failure()
+ uses: mshick/add-pr-comment@v1
+ with:
+ message: |
+ Hi @${{ github.event.pull_request.user.login }},
+
+ It looks like this pull-request is has been made against the ${{github.event.pull_request.head.repo.full_name}} `master` branch.
+ The `master` branch on nf-core repositories should always contain code from the latest release.
+ Because of this, PRs to `master` are only allowed if they come from the ${{github.event.pull_request.head.repo.full_name}} `dev` branch.
+
+ You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page.
+
+ Thanks again for your contribution!
+ repo-token: ${{ secrets.GITHUB_TOKEN }}
+ allow-repeats: false
+
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
new file mode 100644
index 0000000000000000000000000000000000000000..5553430d3c1f667b2dc0462cf709859df1b1d903
--- /dev/null
+++ b/.github/workflows/ci.yml
@@ -0,0 +1,57 @@
+name: nf-core CI
+# This workflow runs the pipeline with the minimal test dataset to check that it completes without any syntax errors
+on:
+ push:
+ branches:
+ - dev
+ pull_request:
+ release:
+ types: [published]
+
+jobs:
+ test:
+ name: Run workflow tests
+ # Only run on push if this is the nf-core dev branch (merged PRs)
+ if: ${{ github.event_name != 'push' || (github.event_name == 'push' && github.repository == 'nf-core/hic') }}
+ runs-on: ubuntu-latest
+ env:
+ NXF_VER: ${{ matrix.nxf_ver }}
+ NXF_ANSI_LOG: false
+ strategy:
+ matrix:
+ # Nextflow versions: check pipeline minimum and current latest
+ nxf_ver: ['20.04.0', '']
+ steps:
+ - name: Check out pipeline code
+ uses: actions/checkout@v2
+
+ - name: Check if Dockerfile or Conda environment changed
+ uses: technote-space/get-diff-action@v4
+ with:
+ FILES: |
+ Dockerfile
+ environment.yml
+
+ - name: Build new docker image
+ if: env.MATCHED_FILES
+ run: docker build --no-cache . -t nfcore/hic:dev
+
+ - name: Pull docker image
+ if: ${{ !env.MATCHED_FILES }}
+ run: |
+ docker pull nfcore/hic:dev
+ docker tag nfcore/hic:dev nfcore/hic:dev
+
+ - name: Install Nextflow
+ env:
+ CAPSULE_LOG: none
+ run: |
+ wget -qO- get.nextflow.io | bash
+ sudo mv nextflow /usr/local/bin/
+
+ - name: Run pipeline with test data
+ # TODO nf-core: You can customise CI pipeline run tests as required
+ # For example: adding multiple test runs with different parameters
+ # Remember that you can parallelise this by using strategy.matrix
+ run: |
+ nextflow run ${GITHUB_WORKSPACE} -profile test,docker
diff --git a/.github/workflows/linting.yml b/.github/workflows/linting.yml
new file mode 100644
index 0000000000000000000000000000000000000000..bef81e61905d8f265c06b7001c3c17d56d03ba4a
--- /dev/null
+++ b/.github/workflows/linting.yml
@@ -0,0 +1,77 @@
+name: nf-core linting
+# This workflow is triggered on pushes and PRs to the repository.
+# It runs the `nf-core lint` and markdown lint tests to ensure that the code meets the nf-core guidelines
+on:
+ push:
+ pull_request:
+ release:
+ types: [published]
+
+jobs:
+ Markdown:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ - uses: actions/setup-node@v1
+ with:
+ node-version: '10'
+ - name: Install markdownlint
+ run: npm install -g markdownlint-cli
+ - name: Run Markdownlint
+ run: markdownlint ${GITHUB_WORKSPACE} -c ${GITHUB_WORKSPACE}/.github/markdownlint.yml
+ YAML:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v1
+ - uses: actions/setup-node@v1
+ with:
+ node-version: '10'
+ - name: Install yaml-lint
+ run: npm install -g yaml-lint
+ - name: Run yaml-lint
+ run: yamllint $(find ${GITHUB_WORKSPACE} -type f -name "*.yml")
+ nf-core:
+ runs-on: ubuntu-latest
+ steps:
+
+ - name: Check out pipeline code
+ uses: actions/checkout@v2
+
+ - name: Install Nextflow
+ env:
+ CAPSULE_LOG: none
+ run: |
+ wget -qO- get.nextflow.io | bash
+ sudo mv nextflow /usr/local/bin/
+
+ - uses: actions/setup-python@v1
+ with:
+ python-version: '3.6'
+ architecture: 'x64'
+
+ - name: Install dependencies
+ run: |
+ python -m pip install --upgrade pip
+ pip install nf-core
+
+ - name: Run nf-core lint
+ env:
+ GITHUB_COMMENTS_URL: ${{ github.event.pull_request.comments_url }}
+ GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ GITHUB_PR_COMMIT: ${{ github.event.pull_request.head.sha }}
+ run: nf-core -l lint_log.txt lint ${GITHUB_WORKSPACE} --markdown lint_results.md
+
+ - name: Save PR number
+ if: ${{ always() }}
+ run: echo ${{ github.event.pull_request.number }} > PR_number.txt
+
+ - name: Upload linting log file artifact
+ if: ${{ always() }}
+ uses: actions/upload-artifact@v2
+ with:
+ name: linting-log-file
+ path: |
+ lint_log.txt
+ lint_results.md
+ PR_number.txt
+
diff --git a/.github/workflows/linting_comment.yml b/.github/workflows/linting_comment.yml
new file mode 100644
index 0000000000000000000000000000000000000000..90f03c6f91ba0c05d12daff7e15840145a63937f
--- /dev/null
+++ b/.github/workflows/linting_comment.yml
@@ -0,0 +1,29 @@
+
+name: nf-core linting comment
+# This workflow is triggered after the linting action is complete
+# It posts an automated comment to the PR, even if the PR is coming from a fork
+
+on:
+ workflow_run:
+ workflows: ["nf-core linting"]
+
+jobs:
+ test:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Download lint results
+ uses: dawidd6/action-download-artifact@v2
+ with:
+ workflow: linting.yml
+
+ - name: Get PR number
+ id: pr_number
+ run: echo "::set-output name=pr_number::$(cat linting-logs/PR_number.txt)"
+
+ - name: Post PR comment
+ uses: marocchino/sticky-pull-request-comment@v2
+ with:
+ GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ number: ${{ steps.pr_number.outputs.pr_number }}
+ path: linting-logs/lint_results.md
+
diff --git a/.github/workflows/push_dockerhub_dev.yml b/.github/workflows/push_dockerhub_dev.yml
new file mode 100644
index 0000000000000000000000000000000000000000..d6fc716fb947d262e0613ccab07655d387a98d1f
--- /dev/null
+++ b/.github/workflows/push_dockerhub_dev.yml
@@ -0,0 +1,28 @@
+name: nf-core Docker push (dev)
+# This builds the docker image and pushes it to DockerHub
+# Runs on nf-core repo releases and push event to 'dev' branch (PR merges)
+on:
+ push:
+ branches:
+ - dev
+
+jobs:
+ push_dockerhub:
+ name: Push new Docker image to Docker Hub (dev)
+ runs-on: ubuntu-latest
+ # Only run for the nf-core repo, for releases and merged PRs
+ if: ${{ github.repository == 'nf-core/hic' }}
+ env:
+ DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
+ DOCKERHUB_PASS: ${{ secrets.DOCKERHUB_PASS }}
+ steps:
+ - name: Check out pipeline code
+ uses: actions/checkout@v2
+
+ - name: Build new docker image
+ run: docker build --no-cache . -t nfcore/hic:dev
+
+ - name: Push Docker image to DockerHub (dev)
+ run: |
+ echo "$DOCKERHUB_PASS" | docker login -u "$DOCKERHUB_USERNAME" --password-stdin
+ docker push nfcore/hic:dev
diff --git a/.github/workflows/push_dockerhub_release.yml b/.github/workflows/push_dockerhub_release.yml
new file mode 100644
index 0000000000000000000000000000000000000000..eda09ccfb6fcdd5791c56b6875343dc2bbf9c8c0
--- /dev/null
+++ b/.github/workflows/push_dockerhub_release.yml
@@ -0,0 +1,29 @@
+name: nf-core Docker push (release)
+# This builds the docker image and pushes it to DockerHub
+# Runs on nf-core repo releases and push event to 'dev' branch (PR merges)
+on:
+ release:
+ types: [published]
+
+jobs:
+ push_dockerhub:
+ name: Push new Docker image to Docker Hub (release)
+ runs-on: ubuntu-latest
+ # Only run for the nf-core repo, for releases and merged PRs
+ if: ${{ github.repository == 'nf-core/hic' }}
+ env:
+ DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
+ DOCKERHUB_PASS: ${{ secrets.DOCKERHUB_PASS }}
+ steps:
+ - name: Check out pipeline code
+ uses: actions/checkout@v2
+
+ - name: Build new docker image
+ run: docker build --no-cache . -t nfcore/hic:latest
+
+ - name: Push Docker image to DockerHub (release)
+ run: |
+ echo "$DOCKERHUB_PASS" | docker login -u "$DOCKERHUB_USERNAME" --password-stdin
+ docker push nfcore/hic:latest
+ docker tag nfcore/hic:latest nfcore/hic:${{ github.event.release.tag_name }}
+ docker push nfcore/hic:${{ github.event.release.tag_name }}
diff --git a/.gitignore b/.gitignore
index 46f69e414ba5d72f679f5140fee33188d84422f8..aa4bb5b375a9021f754dbd91d2321d16d1c0afc7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -3,4 +3,7 @@ work/
data/
results/
.DS_Store
-tests/test_data
+tests/
+testing/
+testing*
+*.pyc
diff --git a/.travis.yml b/.travis.yml
deleted file mode 100644
index 381c4cf48de25ed8d8011fee2573ec6883fbc402..0000000000000000000000000000000000000000
--- a/.travis.yml
+++ /dev/null
@@ -1,37 +0,0 @@
-sudo: required
-language: python
-jdk: openjdk8
-services: docker
-python: '3.6'
-cache: pip
-matrix:
- fast_finish: true
-
-before_install:
- # PRs to master are only ok if coming from dev branch
- - '[ $TRAVIS_PULL_REQUEST = "false" ] || [ $TRAVIS_BRANCH != "master" ] || ([ $TRAVIS_PULL_REQUEST_SLUG = $TRAVIS_REPO_SLUG ] && [ $TRAVIS_PULL_REQUEST_BRANCH = "dev" ])'
- # Pull the docker image first so the test doesn't wait for this
- - docker pull nfcore/hic:dev
- # Fake the tag locally so that the pipeline runs properly
- - docker tag nfcore/hic:dev nfcore/hic:latest
-
-install:
- # Install Nextflow
- - mkdir /tmp/nextflow && cd /tmp/nextflow
- - wget -qO- get.nextflow.io | bash
- - sudo ln -s /tmp/nextflow/nextflow /usr/local/bin/nextflow
- # Install nf-core/tools
- - pip install --upgrade pip
- - pip install nf-core
- # Reset
- - mkdir ${TRAVIS_BUILD_DIR}/tests && cd ${TRAVIS_BUILD_DIR}/tests
-
-env:
- - NXF_VER='0.32.0' # Specify a minimum NF version that should be tested and work
- - NXF_VER='' # Plus: get the latest NF version and check that it works
-
-script:
- # Lint the pipeline code
- - nf-core lint ${TRAVIS_BUILD_DIR}
- # Run the pipeline with the test profile
- - nextflow run ${TRAVIS_BUILD_DIR} -profile test,docker
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 0b88e32553c46ba971a20c744dde85f0a74339d5..b84d61496f6c71409a4dd0df010a196e400cc338 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,4 +1,16 @@
# nf-core/hic: Changelog
-## v1.0dev - <date>
-Initial release of nf-core/hic, created with the [nf-core](http://nf-co.re/) template.
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## v1.0dev - [date]
+
+Initial release of nf-core/hic, created with the [nf-core](https://nf-co.re/) template.
+
+### `Added`
+
+### `Fixed`
+
+### `Dependencies`
+
+### `Deprecated`
diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md
index 21096193adc83331ff86beea517b1d0e37e35c09..405fb1bfd7f0d022a8cee036dab43e72b8af871d 100644
--- a/CODE_OF_CONDUCT.md
+++ b/CODE_OF_CONDUCT.md
@@ -34,13 +34,13 @@ This Code of Conduct applies both within project spaces and in public spaces whe
## Enforcement
-Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team on the [Gitter channel](https://gitter.im/nf-core/Lobby). The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team on [Slack](https://nf-co.re/join/slack). The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
## Attribution
-This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [https://www.contributor-covenant.org/version/1/4/code-of-conduct/][version]
-[homepage]: http://contributor-covenant.org
-[version]: http://contributor-covenant.org/version/1/4/
+[homepage]: https://contributor-covenant.org
+[version]: https://www.contributor-covenant.org/version/1/4/code-of-conduct/
diff --git a/Dockerfile b/Dockerfile
index 4d28385b74a7fd65cebe0848cd39097a9518864a..1a375e9e6d0474746e7ba47ea2da86430f9db935 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,7 +1,17 @@
-FROM nfcore/base
+FROM nfcore/base:1.12.1
LABEL authors="Nicolas Servant" \
- description="Docker image containing all requirements for nf-core/hic pipeline"
+ description="Docker image containing all software requirements for the nf-core/hic pipeline"
+# Install the conda environment
COPY environment.yml /
-RUN conda env create -f /environment.yml && conda clean -a
+RUN conda env create --quiet -f /environment.yml && conda clean -a
+
+# Add conda installation dir to PATH (instead of doing 'conda activate')
ENV PATH /opt/conda/envs/nf-core-hic-1.0dev/bin:$PATH
+
+# Dump the details of the installed packages to a file for posterity
+RUN conda env export --name nf-core-hic-1.0dev > nf-core-hic-1.0dev.yml
+
+# Instruct R processes to use these empty files instead of clashing with a local version
+RUN touch .Rprofile
+RUN touch .Renviron
diff --git a/README.md b/README.md
index 192fa0de6b9cfd727673b531668abdfdb38c3e2d..30fc4b365283cbe229f28b142c3033f2376c12bb 100644
--- a/README.md
+++ b/README.md
@@ -1,31 +1,90 @@
-# nf-core/hic
-**Analysis of Chromosome Conformation Capture data (Hi-C)**
+# 
-[](https://travis-ci.org/nf-core/hic)
-[](https://www.nextflow.io/)
+**Analysis of Chromosome Conformation Capture data (Hi-C)**.
-[](http://bioconda.github.io/)
+[](https://github.com/nf-core/hic/actions)
+[](https://github.com/nf-core/hic/actions)
+[](https://www.nextflow.io/)
+
+[](https://bioconda.github.io/)
[](https://hub.docker.com/r/nfcore/hic)
-
+[](https://nfcore.slack.com/channels/hic)
+
+## Introduction
+
+<!-- TODO nf-core: Write a 1-2 sentence summary of what data the pipeline is for and what it does -->
+**nf-core/hic** is a bioinformatics best-practise analysis pipeline for
+
+The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It comes with docker containers making installation trivial and results highly reproducible.
+
+## Quick Start
+
+1. Install [`nextflow`](https://nf-co.re/usage/installation)
+
+2. Install any of [`Docker`](https://docs.docker.com/engine/installation/), [`Singularity`](https://www.sylabs.io/guides/3.0/user-guide/) or [`Podman`](https://podman.io/) for full pipeline reproducibility _(please only use [`Conda`](https://conda.io/miniconda.html) as a last resort; see [docs](https://nf-co.re/usage/configuration#basic-configuration-profiles))_
+
+3. Download the pipeline and test it on a minimal dataset with a single command:
+
+ ```bash
+ nextflow run nf-core/hic -profile test,<docker/singularity/podman/conda/institute>
+ ```
+
+ > Please check [nf-core/configs](https://github.com/nf-core/configs#documentation) to see if a custom config file to run nf-core pipelines already exists for your Institute. If so, you can simply use `-profile <institute>` in your command. This will enable either `docker` or `singularity` and set the appropriate execution settings for your local compute environment.
+
+4. Start running your own analysis!
+
+ <!-- TODO nf-core: Update the example "typical command" below used to run the pipeline -->
+
+ ```bash
+ nextflow run nf-core/hic -profile <docker/singularity/podman/conda/institute> --input '*_R{1,2}.fastq.gz' --genome GRCh37
+ ```
+
+See [usage docs](https://nf-co.re/hic/usage) for all of the available options when running the pipeline.
-### Introduction
-The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It comes with docker / singularity containers making installation trivial and results highly reproducible.
+## Pipeline Summary
+By default, the pipeline currently performs the following:
-### Documentation
-The nf-core/hic pipeline comes with documentation about the pipeline, found in the `docs/` directory:
+<!-- TODO nf-core: Fill in short bullet-pointed list of default steps of pipeline -->
-1. [Installation](docs/installation.md)
-2. Pipeline configuration
- * [Local installation](docs/configuration/local.md)
- * [Adding your own system](docs/configuration/adding_your_own.md)
- * [Reference genomes](docs/configuration/reference_genomes.md)
-3. [Running the pipeline](docs/usage.md)
-4. [Output and how to interpret the results](docs/output.md)
-5. [Troubleshooting](docs/troubleshooting.md)
+* Sequencing quality control (`FastQC`)
+* Overall pipeline run summaries (`MultiQC`)
+
+## Documentation
+
+The nf-core/hic pipeline comes with documentation about the pipeline: [usage](https://nf-co.re/hic/usage) and [output](https://nf-co.re/hic/output).
<!-- TODO nf-core: Add a brief overview of what the pipeline does and how it works -->
-### Credits
+## Credits
+
nf-core/hic was originally written by Nicolas Servant.
+
+We thank the following people for their extensive assistance in the development
+of this pipeline:
+
+<!-- TODO nf-core: If applicable, make list of people who have also contributed -->
+
+## Contributions and Support
+
+If you would like to contribute to this pipeline, please see the [contributing guidelines](.github/CONTRIBUTING.md).
+
+For further information or help, don't hesitate to get in touch on the [Slack `#hic` channel](https://nfcore.slack.com/channels/hic) (you can join with [this invite](https://nf-co.re/join/slack)).
+
+## Citations
+
+<!-- TODO nf-core: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi. -->
+<!-- If you use nf-core/hic for your analysis, please cite it using the following doi: [10.5281/zenodo.XXXXXX](https://doi.org/10.5281/zenodo.XXXXXX) -->
+
+You can cite the `nf-core` publication as follows:
+
+> **The nf-core framework for community-curated bioinformatics pipelines.**
+>
+> Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.
+>
+> _Nat Biotechnol._ 2020 Feb 13. doi: [10.1038/s41587-020-0439-x](https://dx.doi.org/10.1038/s41587-020-0439-x).
+> ReadCube: [Full Access Link](https://rdcu.be/b1GjZ)
+
+In addition, references of tools and data used in this pipeline are as follows:
+
+<!-- TODO nf-core: Add bibliography of tools and data used in your pipeline -->
diff --git a/Singularity b/Singularity
deleted file mode 100644
index 927866f0ec891fab132980e50a91d78d7f887eaa..0000000000000000000000000000000000000000
--- a/Singularity
+++ /dev/null
@@ -1,18 +0,0 @@
-From:nfcore/base
-Bootstrap:docker
-
-%labels
- MAINTAINER Nicolas Servant
- DESCRIPTION Singularity image containing all requirements for the nf-core/hic pipeline
- VERSION 1.0dev
-
-%environment
- PATH=/opt/conda/envs/nf-core-hic-1.0dev/bin:$PATH
- export PATH
-
-%files
- environment.yml /
-
-%post
- /opt/conda/bin/conda env create -f /environment.yml
- /opt/conda/bin/conda clean -a
diff --git a/assets/email_template.html b/assets/email_template.html
index bf19807e4a21c86f969a7ed5570b479574d9e4ee..177bccd2d802d3a11e2e6ca606e72ac19968126b 100644
--- a/assets/email_template.html
+++ b/assets/email_template.html
@@ -11,6 +11,8 @@
<body>
<div style="font-family: Helvetica, Arial, sans-serif; padding: 30px; max-width: 800px; margin: 0 auto;">
+<img src="cid:nfcorepipelinelogo">
+
<h1>nf-core/hic v${version}</h1>
<h2>Run Name: $runName</h2>
diff --git a/assets/email_template.txt b/assets/email_template.txt
index 59d7b549d375db84dae99a2e186b2a9df109f2d1..a951c5e7f965fa5829707fc84f4351495995190f 100644
--- a/assets/email_template.txt
+++ b/assets/email_template.txt
@@ -1,6 +1,12 @@
-========================================
- nf-core/hic v${version}
-========================================
+----------------------------------------------------
+ ,--./,-.
+ ___ __ __ __ ___ /,-._.--~\\
+ |\\ | |__ __ / ` / \\ |__) |__ } {
+ | \\| | \\__, \\__/ | \\ |___ \\`-._,-`-,
+ `._,._,'
+ nf-core/hic v${version}
+----------------------------------------------------
+
Run Name: $runName
<% if (success){
@@ -17,23 +23,6 @@ ${errorReport}
} %>
-<% if (!success){
- out << """####################################################
-## nf-core/hic execution completed unsuccessfully! ##
-####################################################
-The exit status of the task that caused the workflow execution to fail was: $exitStatus.
-The full error message was:
-
-${errorReport}
-"""
-} else {
- out << "## nf-core/hic execution completed successfully! ##"
-}
-%>
-
-
-
-
The workflow was completed at $dateComplete (duration: $duration)
The command used to launch the workflow was as follows:
diff --git a/conf/multiqc_config.yaml b/assets/multiqc_config.yaml
similarity index 79%
rename from conf/multiqc_config.yaml
rename to assets/multiqc_config.yaml
index f2a738c43be4dae15db5075017559607c66c0542..41468cab303a5894aa01e0823790b22cb44c95cd 100644
--- a/conf/multiqc_config.yaml
+++ b/assets/multiqc_config.yaml
@@ -3,5 +3,9 @@ report_comment: >
analysis pipeline. For information about how to interpret these results, please see the
<a href="https://github.com/nf-core/hic" target="_blank">documentation</a>.
report_section_order:
- nf-core/hic-software-versions:
+ software_versions:
order: -1000
+ nf-core-hic-summary:
+ order: -1001
+
+export_plots: true
diff --git a/assets/nf-core-hic_logo.png b/assets/nf-core-hic_logo.png
new file mode 100644
index 0000000000000000000000000000000000000000..37461d9a32ae1f73d9090a3a2387cf8997c9a0ed
Binary files /dev/null and b/assets/nf-core-hic_logo.png differ
diff --git a/assets/sendmail_template.txt b/assets/sendmail_template.txt
index fd1cd7396e07c870807d05843d5bd5a74e49c2d8..bdf905878111122e2d6b6983f72e6a04c78e97b5 100644
--- a/assets/sendmail_template.txt
+++ b/assets/sendmail_template.txt
@@ -1,11 +1,53 @@
To: $email
Subject: $subject
Mime-Version: 1.0
-Content-Type: multipart/related;boundary="nfmimeboundary"
+Content-Type: multipart/related;boundary="nfcoremimeboundary"
---nfmimeboundary
+--nfcoremimeboundary
Content-Type: text/html; charset=utf-8
$email_html
---nfmimeboundary--
+--nfcoremimeboundary
+Content-Type: image/png;name="nf-core-hic_logo.png"
+Content-Transfer-Encoding: base64
+Content-ID: <nfcorepipelinelogo>
+Content-Disposition: inline; filename="nf-core-hic_logo.png"
+
+<% out << new File("$projectDir/assets/nf-core-hic_logo.png").
+ bytes.
+ encodeBase64().
+ toString().
+ tokenize( '\n' )*.
+ toList()*.
+ collate( 76 )*.
+ collect { it.join() }.
+ flatten().
+ join( '\n' ) %>
+
+<%
+if (mqcFile){
+def mqcFileObj = new File("$mqcFile")
+if (mqcFileObj.length() < mqcMaxSize){
+out << """
+--nfcoremimeboundary
+Content-Type: text/html; name=\"multiqc_report\"
+Content-Transfer-Encoding: base64
+Content-ID: <mqcreport>
+Content-Disposition: attachment; filename=\"${mqcFileObj.getName()}\"
+
+${mqcFileObj.
+ bytes.
+ encodeBase64().
+ toString().
+ tokenize( '\n' )*.
+ toList()*.
+ collate( 76 )*.
+ collect { it.join() }.
+ flatten().
+ join( '\n' )}
+"""
+}}
+%>
+
+--nfcoremimeboundary--
diff --git a/bin/__pycache__/scrape_software_versions.cpython-36.pyc b/bin/__pycache__/scrape_software_versions.cpython-36.pyc
deleted file mode 100644
index 07062ad2b5afae9e2bc03e055e9776da9b619a1c..0000000000000000000000000000000000000000
Binary files a/bin/__pycache__/scrape_software_versions.cpython-36.pyc and /dev/null differ
diff --git a/bin/markdown_to_html.py b/bin/markdown_to_html.py
new file mode 100755
index 0000000000000000000000000000000000000000..a26d1ff5e6de3c09385760e76cc40f11a512b3a4
--- /dev/null
+++ b/bin/markdown_to_html.py
@@ -0,0 +1,91 @@
+#!/usr/bin/env python
+from __future__ import print_function
+import argparse
+import markdown
+import os
+import sys
+import io
+
+
+def convert_markdown(in_fn):
+ input_md = io.open(in_fn, mode="r", encoding="utf-8").read()
+ html = markdown.markdown(
+ "[TOC]\n" + input_md,
+ extensions=["pymdownx.extra", "pymdownx.b64", "pymdownx.highlight", "pymdownx.emoji", "pymdownx.tilde", "toc"],
+ extension_configs={
+ "pymdownx.b64": {"base_path": os.path.dirname(in_fn)},
+ "pymdownx.highlight": {"noclasses": True},
+ "toc": {"title": "Table of Contents"},
+ },
+ )
+ return html
+
+
+def wrap_html(contents):
+ header = """<!DOCTYPE html><html>
+ <head>
+ <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css" integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous">
+ <style>
+ body {
+ font-family: -apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Helvetica Neue",Arial,"Noto Sans",sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";
+ padding: 3em;
+ margin-right: 350px;
+ max-width: 100%;
+ }
+ .toc {
+ position: fixed;
+ right: 20px;
+ width: 300px;
+ padding-top: 20px;
+ overflow: scroll;
+ height: calc(100% - 3em - 20px);
+ }
+ .toctitle {
+ font-size: 1.8em;
+ font-weight: bold;
+ }
+ .toc > ul {
+ padding: 0;
+ margin: 1rem 0;
+ list-style-type: none;
+ }
+ .toc > ul ul { padding-left: 20px; }
+ .toc > ul > li > a { display: none; }
+ img { max-width: 800px; }
+ pre {
+ padding: 0.6em 1em;
+ }
+ h2 {
+
+ }
+ </style>
+ </head>
+ <body>
+ <div class="container">
+ """
+ footer = """
+ </div>
+ </body>
+ </html>
+ """
+ return header + contents + footer
+
+
+def parse_args(args=None):
+ parser = argparse.ArgumentParser()
+ parser.add_argument("mdfile", type=argparse.FileType("r"), nargs="?", help="File to convert. Defaults to stdin.")
+ parser.add_argument(
+ "-o", "--out", type=argparse.FileType("w"), default=sys.stdout, help="Output file name. Defaults to stdout."
+ )
+ return parser.parse_args(args)
+
+
+def main(args=None):
+ args = parse_args(args)
+ converted_md = convert_markdown(args.mdfile.name)
+ html = wrap_html(converted_md)
+ args.out.write(html)
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/bin/markdown_to_html.r b/bin/markdown_to_html.r
deleted file mode 100755
index abe1335070d84f0d9a17dae7b6d482341f7f59a8..0000000000000000000000000000000000000000
--- a/bin/markdown_to_html.r
+++ /dev/null
@@ -1,51 +0,0 @@
-#!/usr/bin/env Rscript
-
-# Command line argument processing
-args = commandArgs(trailingOnly=TRUE)
-if (length(args) < 2) {
- stop("Usage: markdown_to_html.r <input.md> <output.html>", call.=FALSE)
-}
-markdown_fn <- args[1]
-output_fn <- args[2]
-
-# Load / install packages
-if (!require("markdown")) {
- install.packages("markdown", dependencies=TRUE, repos='http://cloud.r-project.org/')
- library("markdown")
-}
-
-base_css_fn <- getOption("markdown.HTML.stylesheet")
-base_css <- readChar(base_css_fn, file.info(base_css_fn)$size)
-custom_css <- paste(base_css, "
-body {
- padding: 3em;
- margin-right: 350px;
- max-width: 100%;
-}
-#toc {
- position: fixed;
- right: 20px;
- width: 300px;
- padding-top: 20px;
- overflow: scroll;
- height: calc(100% - 3em - 20px);
-}
-#toc_header {
- font-size: 1.8em;
- font-weight: bold;
-}
-#toc > ul {
- padding-left: 0;
- list-style-type: none;
-}
-#toc > ul ul { padding-left: 20px; }
-#toc > ul > li > a { display: none; }
-img { max-width: 800px; }
-")
-
-markdownToHTML(
- file = markdown_fn,
- output = output_fn,
- stylesheet = custom_css,
- options = c('toc', 'base64_images', 'highlight_code')
-)
diff --git a/bin/scrape_software_versions.py b/bin/scrape_software_versions.py
index 0e98e07fea0933a455bd576100e8b026cb684e18..feb158ae01211f5e71e63c8c74a3573882f9a6a4 100755
--- a/bin/scrape_software_versions.py
+++ b/bin/scrape_software_versions.py
@@ -5,35 +5,50 @@ import re
# TODO nf-core: Add additional regexes for new tools in process get_software_versions
regexes = {
- 'nf-core/hic': ['v_pipeline.txt', r"(\S+)"],
- 'Nextflow': ['v_nextflow.txt', r"(\S+)"],
- 'FastQC': ['v_fastqc.txt', r"FastQC v(\S+)"],
- 'MultiQC': ['v_multiqc.txt', r"multiqc, version (\S+)"],
+ "nf-core/hic": ["v_pipeline.txt", r"(\S+)"],
+ "Nextflow": ["v_nextflow.txt", r"(\S+)"],
+ "FastQC": ["v_fastqc.txt", r"FastQC v(\S+)"],
+ "MultiQC": ["v_multiqc.txt", r"multiqc, version (\S+)"],
}
results = OrderedDict()
-results['nf-core/hic'] = '<span style="color:#999999;\">N/A</span>'
-results['Nextflow'] = '<span style="color:#999999;\">N/A</span>'
-results['FastQC'] = '<span style="color:#999999;\">N/A</span>'
-results['MultiQC'] = '<span style="color:#999999;\">N/A</span>'
+results["nf-core/hic"] = '<span style="color:#999999;">N/A</span>'
+results["Nextflow"] = '<span style="color:#999999;">N/A</span>'
+results["FastQC"] = '<span style="color:#999999;">N/A</span>'
+results["MultiQC"] = '<span style="color:#999999;">N/A</span>'
# Search each file using its regex
for k, v in regexes.items():
- with open(v[0]) as x:
- versions = x.read()
- match = re.search(v[1], versions)
- if match:
- results[k] = "v{}".format(match.group(1))
+ try:
+ with open(v[0]) as x:
+ versions = x.read()
+ match = re.search(v[1], versions)
+ if match:
+ results[k] = "v{}".format(match.group(1))
+ except IOError:
+ results[k] = False
+
+# Remove software set to false in results
+for k in list(results):
+ if not results[k]:
+ del results[k]
# Dump to YAML
-print ('''
-id: 'nf-core/hic-software-versions'
+print(
+ """
+id: 'software_versions'
section_name: 'nf-core/hic Software Versions'
section_href: 'https://github.com/nf-core/hic'
plot_type: 'html'
description: 'are collected at run time from the software output.'
data: |
<dl class="dl-horizontal">
-''')
-for k,v in results.items():
- print(" <dt>{}</dt><dd>{}</dd>".format(k,v))
-print (" </dl>")
+"""
+)
+for k, v in results.items():
+ print(" <dt>{}</dt><dd><samp>{}</samp></dd>".format(k, v))
+print(" </dl>")
+
+# Write out regexes as csv file:
+with open("software_versions.csv", "w") as f:
+ for k, v in results.items():
+ f.write("{}\t{}\n".format(k, v))
diff --git a/conf/awsbatch.config b/conf/awsbatch.config
deleted file mode 100644
index 79078c7bd03ef1c4131ebfb5d46bf621150d74c1..0000000000000000000000000000000000000000
--- a/conf/awsbatch.config
+++ /dev/null
@@ -1,13 +0,0 @@
-/*
- * -------------------------------------------------
- * Nextflow config file for AWS Batch
- * -------------------------------------------------
- * Imported under the 'awsbatch' Nextflow profile in nextflow.config
- * Uses docker for software depedencies automagically, so not specified here.
- */
-
-aws.region = params.awsregion
-process.executor = 'awsbatch'
-process.queue = params.awsqueue
-executor.awscli = '/home/ec2-user/miniconda/bin/aws'
-params.tracedir = './'
diff --git a/conf/base.config b/conf/base.config
index 23c9e4a820a7261cdbd2379235a182492bed0499..0b2ea226659efbfec4eb0289d8880a6e9df3c8b6 100644
--- a/conf/base.config
+++ b/conf/base.config
@@ -11,31 +11,41 @@
process {
- container = params.container
-
// TODO nf-core: Check the defaults for all processes
cpus = { check_max( 1 * task.attempt, 'cpus' ) }
- memory = { check_max( 8.GB * task.attempt, 'memory' ) }
- time = { check_max( 2.h * task.attempt, 'time' ) }
+ memory = { check_max( 7.GB * task.attempt, 'memory' ) }
+ time = { check_max( 4.h * task.attempt, 'time' ) }
- errorStrategy = { task.exitStatus in [143,137] ? 'retry' : 'finish' }
+ errorStrategy = { task.exitStatus in [143,137,104,134,139] ? 'retry' : 'finish' }
maxRetries = 1
maxErrors = '-1'
// Process-specific resource requirements
- // TODO nf-core: Customise requirements for specific processes
- withName: fastqc {
- errorStrategy = { task.exitStatus in [143,137] ? 'retry' : 'ignore' }
+ // NOTE - Only one of the labels below are used in the fastqc process in the main script.
+ // If possible, it would be nice to keep the same label naming convention when
+ // adding in your processes.
+ // TODO nf-core: Customise requirements for specific processes.
+ // See https://www.nextflow.io/docs/latest/config.html#config-process-selectors
+ withLabel:process_low {
+ cpus = { check_max( 2 * task.attempt, 'cpus' ) }
+ memory = { check_max( 14.GB * task.attempt, 'memory' ) }
+ time = { check_max( 6.h * task.attempt, 'time' ) }
}
- withName: multiqc {
- errorStrategy = { task.exitStatus in [143,137] ? 'retry' : 'ignore' }
+ withLabel:process_medium {
+ cpus = { check_max( 6 * task.attempt, 'cpus' ) }
+ memory = { check_max( 42.GB * task.attempt, 'memory' ) }
+ time = { check_max( 8.h * task.attempt, 'time' ) }
}
-}
-
-params {
- // Defaults only, expecting to be overwritten
- max_memory = 128.GB
- max_cpus = 16
- max_time = 240.h
- igenomes_base = 's3://ngi-igenomes/igenomes/'
+ withLabel:process_high {
+ cpus = { check_max( 12 * task.attempt, 'cpus' ) }
+ memory = { check_max( 84.GB * task.attempt, 'memory' ) }
+ time = { check_max( 10.h * task.attempt, 'time' ) }
+ }
+ withLabel:process_long {
+ time = { check_max( 20.h * task.attempt, 'time' ) }
+ }
+ withName:get_software_versions {
+ cache = false
+ }
+
}
diff --git a/conf/igenomes.config b/conf/igenomes.config
index d19e61f4b1cf67878013c16065bac62d1254a74d..31b7ee6130c97ff9ea026fa312e8a2354dac565c 100644
--- a/conf/igenomes.config
+++ b/conf/igenomes.config
@@ -9,139 +9,413 @@
params {
// illumina iGenomes reference file paths
- // TODO nf-core: Add new reference types and strip out those that are not needed
genomes {
'GRCh37' {
- bed12 = "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Annotation/README.txt"
+ mito_name = "MT"
+ macs_gsize = "2.7e9"
+ blacklist = "${projectDir}/assets/blacklists/GRCh37-blacklist.bed"
+ }
+ 'GRCh38' {
+ fasta = "${params.igenomes_base}/Homo_sapiens/NCBI/GRCh38/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Homo_sapiens/NCBI/GRCh38/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Homo_sapiens/NCBI/GRCh38/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Homo_sapiens/NCBI/GRCh38/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Homo_sapiens/NCBI/GRCh38/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Homo_sapiens/NCBI/GRCh38/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Homo_sapiens/NCBI/GRCh38/Annotation/Genes/genes.bed"
+ mito_name = "chrM"
+ macs_gsize = "2.7e9"
+ blacklist = "${projectDir}/assets/blacklists/hg38-blacklist.bed"
}
'GRCm38' {
- bed12 = "${params.igenomes_base}/Mus_musculus/Ensembl/GRCm38/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Mus_musculus/Ensembl/GRCm38/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Mus_musculus/Ensembl/GRCm38/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Mus_musculus/Ensembl/GRCm38/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Mus_musculus/Ensembl/GRCm38/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Mus_musculus/Ensembl/GRCm38/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Mus_musculus/Ensembl/GRCm38/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Mus_musculus/Ensembl/GRCm38/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Mus_musculus/Ensembl/GRCm38/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Mus_musculus/Ensembl/GRCm38/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Mus_musculus/Ensembl/GRCm38/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Mus_musculus/Ensembl/GRCm38/Annotation/README.txt"
+ mito_name = "MT"
+ macs_gsize = "1.87e9"
+ blacklist = "${projectDir}/assets/blacklists/GRCm38-blacklist.bed"
}
'TAIR10' {
- bed12 = "${params.igenomes_base}/Arabidopsis_thaliana/Ensembl/TAIR10/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Arabidopsis_thaliana/Ensembl/TAIR10/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Arabidopsis_thaliana/Ensembl/TAIR10/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Arabidopsis_thaliana/Ensembl/TAIR10/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Arabidopsis_thaliana/Ensembl/TAIR10/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Arabidopsis_thaliana/Ensembl/TAIR10/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Arabidopsis_thaliana/Ensembl/TAIR10/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Arabidopsis_thaliana/Ensembl/TAIR10/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Arabidopsis_thaliana/Ensembl/TAIR10/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Arabidopsis_thaliana/Ensembl/TAIR10/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Arabidopsis_thaliana/Ensembl/TAIR10/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Arabidopsis_thaliana/Ensembl/TAIR10/Annotation/README.txt"
+ mito_name = "Mt"
}
'EB2' {
- bed12 = "${params.igenomes_base}/Bacillus_subtilis_168/Ensembl/EB2/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Bacillus_subtilis_168/Ensembl/EB2/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Bacillus_subtilis_168/Ensembl/EB2/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Bacillus_subtilis_168/Ensembl/EB2/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Bacillus_subtilis_168/Ensembl/EB2/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Bacillus_subtilis_168/Ensembl/EB2/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Bacillus_subtilis_168/Ensembl/EB2/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Bacillus_subtilis_168/Ensembl/EB2/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Bacillus_subtilis_168/Ensembl/EB2/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Bacillus_subtilis_168/Ensembl/EB2/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Bacillus_subtilis_168/Ensembl/EB2/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Bacillus_subtilis_168/Ensembl/EB2/Annotation/README.txt"
}
'UMD3.1' {
- bed12 = "${params.igenomes_base}/Bos_taurus/Ensembl/UMD3.1/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Bos_taurus/Ensembl/UMD3.1/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Bos_taurus/Ensembl/UMD3.1/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Bos_taurus/Ensembl/UMD3.1/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Bos_taurus/Ensembl/UMD3.1/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Bos_taurus/Ensembl/UMD3.1/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Bos_taurus/Ensembl/UMD3.1/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Bos_taurus/Ensembl/UMD3.1/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Bos_taurus/Ensembl/UMD3.1/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Bos_taurus/Ensembl/UMD3.1/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Bos_taurus/Ensembl/UMD3.1/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Bos_taurus/Ensembl/UMD3.1/Annotation/README.txt"
+ mito_name = "MT"
}
'WBcel235' {
- bed12 = "${params.igenomes_base}/Caenorhabditis_elegans/Ensembl/WBcel235/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Caenorhabditis_elegans/Ensembl/WBcel235/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Caenorhabditis_elegans/Ensembl/WBcel235/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Caenorhabditis_elegans/Ensembl/WBcel235/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Caenorhabditis_elegans/Ensembl/WBcel235/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Caenorhabditis_elegans/Ensembl/WBcel235/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Caenorhabditis_elegans/Ensembl/WBcel235/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Caenorhabditis_elegans/Ensembl/WBcel235/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Caenorhabditis_elegans/Ensembl/WBcel235/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Caenorhabditis_elegans/Ensembl/WBcel235/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Caenorhabditis_elegans/Ensembl/WBcel235/Annotation/Genes/genes.bed"
+ mito_name = "MtDNA"
+ macs_gsize = "9e7"
}
'CanFam3.1' {
- bed12 = "${params.igenomes_base}/Canis_familiaris/Ensembl/CanFam3.1/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Canis_familiaris/Ensembl/CanFam3.1/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Canis_familiaris/Ensembl/CanFam3.1/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Canis_familiaris/Ensembl/CanFam3.1/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Canis_familiaris/Ensembl/CanFam3.1/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Canis_familiaris/Ensembl/CanFam3.1/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Canis_familiaris/Ensembl/CanFam3.1/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Canis_familiaris/Ensembl/CanFam3.1/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Canis_familiaris/Ensembl/CanFam3.1/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Canis_familiaris/Ensembl/CanFam3.1/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Canis_familiaris/Ensembl/CanFam3.1/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Canis_familiaris/Ensembl/CanFam3.1/Annotation/README.txt"
+ mito_name = "MT"
}
'GRCz10' {
- bed12 = "${params.igenomes_base}/Danio_rerio/Ensembl/GRCz10/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Danio_rerio/Ensembl/GRCz10/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Danio_rerio/Ensembl/GRCz10/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Danio_rerio/Ensembl/GRCz10/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Danio_rerio/Ensembl/GRCz10/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Danio_rerio/Ensembl/GRCz10/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Danio_rerio/Ensembl/GRCz10/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Danio_rerio/Ensembl/GRCz10/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Danio_rerio/Ensembl/GRCz10/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Danio_rerio/Ensembl/GRCz10/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Danio_rerio/Ensembl/GRCz10/Annotation/Genes/genes.bed"
+ mito_name = "MT"
}
'BDGP6' {
- bed12 = "${params.igenomes_base}/Drosophila_melanogaster/Ensembl/BDGP6/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Drosophila_melanogaster/Ensembl/BDGP6/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Drosophila_melanogaster/Ensembl/BDGP6/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Drosophila_melanogaster/Ensembl/BDGP6/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Drosophila_melanogaster/Ensembl/BDGP6/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Drosophila_melanogaster/Ensembl/BDGP6/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Drosophila_melanogaster/Ensembl/BDGP6/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Drosophila_melanogaster/Ensembl/BDGP6/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Drosophila_melanogaster/Ensembl/BDGP6/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Drosophila_melanogaster/Ensembl/BDGP6/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Drosophila_melanogaster/Ensembl/BDGP6/Annotation/Genes/genes.bed"
+ mito_name = "M"
+ macs_gsize = "1.2e8"
}
'EquCab2' {
- bed12 = "${params.igenomes_base}/Equus_caballus/Ensembl/EquCab2/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Equus_caballus/Ensembl/EquCab2/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Equus_caballus/Ensembl/EquCab2/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Equus_caballus/Ensembl/EquCab2/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Equus_caballus/Ensembl/EquCab2/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Equus_caballus/Ensembl/EquCab2/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Equus_caballus/Ensembl/EquCab2/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Equus_caballus/Ensembl/EquCab2/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Equus_caballus/Ensembl/EquCab2/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Equus_caballus/Ensembl/EquCab2/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Equus_caballus/Ensembl/EquCab2/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Equus_caballus/Ensembl/EquCab2/Annotation/README.txt"
+ mito_name = "MT"
}
'EB1' {
- bed12 = "${params.igenomes_base}/Escherichia_coli_K_12_DH10B/Ensembl/EB1/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Escherichia_coli_K_12_DH10B/Ensembl/EB1/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Escherichia_coli_K_12_DH10B/Ensembl/EB1/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Escherichia_coli_K_12_DH10B/Ensembl/EB1/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Escherichia_coli_K_12_DH10B/Ensembl/EB1/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Escherichia_coli_K_12_DH10B/Ensembl/EB1/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Escherichia_coli_K_12_DH10B/Ensembl/EB1/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Escherichia_coli_K_12_DH10B/Ensembl/EB1/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Escherichia_coli_K_12_DH10B/Ensembl/EB1/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Escherichia_coli_K_12_DH10B/Ensembl/EB1/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Escherichia_coli_K_12_DH10B/Ensembl/EB1/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Escherichia_coli_K_12_DH10B/Ensembl/EB1/Annotation/README.txt"
}
'Galgal4' {
- bed12 = "${params.igenomes_base}/Gallus_gallus/Ensembl/Galgal4/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Gallus_gallus/Ensembl/Galgal4/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Gallus_gallus/Ensembl/Galgal4/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Gallus_gallus/Ensembl/Galgal4/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Gallus_gallus/Ensembl/Galgal4/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Gallus_gallus/Ensembl/Galgal4/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Gallus_gallus/Ensembl/Galgal4/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Gallus_gallus/Ensembl/Galgal4/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Gallus_gallus/Ensembl/Galgal4/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Gallus_gallus/Ensembl/Galgal4/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Gallus_gallus/Ensembl/Galgal4/Annotation/Genes/genes.bed"
+ mito_name = "MT"
}
'Gm01' {
- bed12 = "${params.igenomes_base}/Glycine_max/Ensembl/Gm01/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Glycine_max/Ensembl/Gm01/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Glycine_max/Ensembl/Gm01/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Glycine_max/Ensembl/Gm01/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Glycine_max/Ensembl/Gm01/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Glycine_max/Ensembl/Gm01/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Glycine_max/Ensembl/Gm01/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Glycine_max/Ensembl/Gm01/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Glycine_max/Ensembl/Gm01/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Glycine_max/Ensembl/Gm01/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Glycine_max/Ensembl/Gm01/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Glycine_max/Ensembl/Gm01/Annotation/README.txt"
}
'Mmul_1' {
- bed12 = "${params.igenomes_base}/Macaca_mulatta/Ensembl/Mmul_1/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Macaca_mulatta/Ensembl/Mmul_1/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Macaca_mulatta/Ensembl/Mmul_1/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Macaca_mulatta/Ensembl/Mmul_1/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Macaca_mulatta/Ensembl/Mmul_1/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Macaca_mulatta/Ensembl/Mmul_1/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Macaca_mulatta/Ensembl/Mmul_1/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Macaca_mulatta/Ensembl/Mmul_1/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Macaca_mulatta/Ensembl/Mmul_1/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Macaca_mulatta/Ensembl/Mmul_1/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Macaca_mulatta/Ensembl/Mmul_1/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Macaca_mulatta/Ensembl/Mmul_1/Annotation/README.txt"
+ mito_name = "MT"
}
'IRGSP-1.0' {
- bed12 = "${params.igenomes_base}/Oryza_sativa_japonica/Ensembl/IRGSP-1.0/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Oryza_sativa_japonica/Ensembl/IRGSP-1.0/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Oryza_sativa_japonica/Ensembl/IRGSP-1.0/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Oryza_sativa_japonica/Ensembl/IRGSP-1.0/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Oryza_sativa_japonica/Ensembl/IRGSP-1.0/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Oryza_sativa_japonica/Ensembl/IRGSP-1.0/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Oryza_sativa_japonica/Ensembl/IRGSP-1.0/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Oryza_sativa_japonica/Ensembl/IRGSP-1.0/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Oryza_sativa_japonica/Ensembl/IRGSP-1.0/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Oryza_sativa_japonica/Ensembl/IRGSP-1.0/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Oryza_sativa_japonica/Ensembl/IRGSP-1.0/Annotation/Genes/genes.bed"
+ mito_name = "Mt"
}
'CHIMP2.1.4' {
- bed12 = "${params.igenomes_base}/Pan_troglodytes/Ensembl/CHIMP2.1.4/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Pan_troglodytes/Ensembl/CHIMP2.1.4/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Pan_troglodytes/Ensembl/CHIMP2.1.4/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Pan_troglodytes/Ensembl/CHIMP2.1.4/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Pan_troglodytes/Ensembl/CHIMP2.1.4/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Pan_troglodytes/Ensembl/CHIMP2.1.4/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Pan_troglodytes/Ensembl/CHIMP2.1.4/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Pan_troglodytes/Ensembl/CHIMP2.1.4/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Pan_troglodytes/Ensembl/CHIMP2.1.4/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Pan_troglodytes/Ensembl/CHIMP2.1.4/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Pan_troglodytes/Ensembl/CHIMP2.1.4/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Pan_troglodytes/Ensembl/CHIMP2.1.4/Annotation/README.txt"
+ mito_name = "MT"
}
'Rnor_6.0' {
- bed12 = "${params.igenomes_base}/Rattus_norvegicus/Ensembl/Rnor_6.0/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Rattus_norvegicus/Ensembl/Rnor_6.0/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Rattus_norvegicus/Ensembl/Rnor_6.0/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Rattus_norvegicus/Ensembl/Rnor_6.0/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Rattus_norvegicus/Ensembl/Rnor_6.0/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Rattus_norvegicus/Ensembl/Rnor_6.0/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Rattus_norvegicus/Ensembl/Rnor_6.0/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Rattus_norvegicus/Ensembl/Rnor_6.0/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Rattus_norvegicus/Ensembl/Rnor_6.0/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Rattus_norvegicus/Ensembl/Rnor_6.0/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Rattus_norvegicus/Ensembl/Rnor_6.0/Annotation/Genes/genes.bed"
+ mito_name = "MT"
}
'R64-1-1' {
- bed12 = "${params.igenomes_base}/Saccharomyces_cerevisiae/Ensembl/R64-1-1/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Saccharomyces_cerevisiae/Ensembl/R64-1-1/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Saccharomyces_cerevisiae/Ensembl/R64-1-1/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Saccharomyces_cerevisiae/Ensembl/R64-1-1/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Saccharomyces_cerevisiae/Ensembl/R64-1-1/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Saccharomyces_cerevisiae/Ensembl/R64-1-1/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Saccharomyces_cerevisiae/Ensembl/R64-1-1/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Saccharomyces_cerevisiae/Ensembl/R64-1-1/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Saccharomyces_cerevisiae/Ensembl/R64-1-1/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Saccharomyces_cerevisiae/Ensembl/R64-1-1/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Saccharomyces_cerevisiae/Ensembl/R64-1-1/Annotation/Genes/genes.bed"
+ mito_name = "MT"
+ macs_gsize = "1.2e7"
}
'EF2' {
- bed12 = "${params.igenomes_base}/Schizosaccharomyces_pombe/Ensembl/EF2/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Schizosaccharomyces_pombe/Ensembl/EF2/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Schizosaccharomyces_pombe/Ensembl/EF2/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Schizosaccharomyces_pombe/Ensembl/EF2/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Schizosaccharomyces_pombe/Ensembl/EF2/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Schizosaccharomyces_pombe/Ensembl/EF2/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Schizosaccharomyces_pombe/Ensembl/EF2/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Schizosaccharomyces_pombe/Ensembl/EF2/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Schizosaccharomyces_pombe/Ensembl/EF2/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Schizosaccharomyces_pombe/Ensembl/EF2/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Schizosaccharomyces_pombe/Ensembl/EF2/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Schizosaccharomyces_pombe/Ensembl/EF2/Annotation/README.txt"
+ mito_name = "MT"
+ macs_gsize = "1.21e7"
}
'Sbi1' {
- bed12 = "${params.igenomes_base}/Sorghum_bicolor/Ensembl/Sbi1/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Sorghum_bicolor/Ensembl/Sbi1/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Sorghum_bicolor/Ensembl/Sbi1/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Sorghum_bicolor/Ensembl/Sbi1/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Sorghum_bicolor/Ensembl/Sbi1/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Sorghum_bicolor/Ensembl/Sbi1/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Sorghum_bicolor/Ensembl/Sbi1/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Sorghum_bicolor/Ensembl/Sbi1/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Sorghum_bicolor/Ensembl/Sbi1/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Sorghum_bicolor/Ensembl/Sbi1/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Sorghum_bicolor/Ensembl/Sbi1/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Sorghum_bicolor/Ensembl/Sbi1/Annotation/README.txt"
}
'Sscrofa10.2' {
- bed12 = "${params.igenomes_base}/Sus_scrofa/Ensembl/Sscrofa10.2/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Sus_scrofa/Ensembl/Sscrofa10.2/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Sus_scrofa/Ensembl/Sscrofa10.2/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Sus_scrofa/Ensembl/Sscrofa10.2/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Sus_scrofa/Ensembl/Sscrofa10.2/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Sus_scrofa/Ensembl/Sscrofa10.2/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Sus_scrofa/Ensembl/Sscrofa10.2/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Sus_scrofa/Ensembl/Sscrofa10.2/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Sus_scrofa/Ensembl/Sscrofa10.2/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Sus_scrofa/Ensembl/Sscrofa10.2/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Sus_scrofa/Ensembl/Sscrofa10.2/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Sus_scrofa/Ensembl/Sscrofa10.2/Annotation/README.txt"
+ mito_name = "MT"
}
'AGPv3' {
- bed12 = "${params.igenomes_base}/Zea_mays/Ensembl/AGPv3/Annotation/Genes/genes.bed"
- fasta = "${params.igenomes_base}/Zea_mays/Ensembl/AGPv3/Sequence/WholeGenomeFasta/genome.fa"
- gtf = "${params.igenomes_base}/Zea_mays/Ensembl/AGPv3/Annotation/Genes/genes.gtf"
- star = "${params.igenomes_base}/Zea_mays/Ensembl/AGPv3/Sequence/STARIndex/"
+ fasta = "${params.igenomes_base}/Zea_mays/Ensembl/AGPv3/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Zea_mays/Ensembl/AGPv3/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Zea_mays/Ensembl/AGPv3/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Zea_mays/Ensembl/AGPv3/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Zea_mays/Ensembl/AGPv3/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Zea_mays/Ensembl/AGPv3/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Zea_mays/Ensembl/AGPv3/Annotation/Genes/genes.bed"
+ mito_name = "Mt"
+ }
+ 'hg38' {
+ fasta = "${params.igenomes_base}/Homo_sapiens/UCSC/hg38/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Homo_sapiens/UCSC/hg38/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Homo_sapiens/UCSC/hg38/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Homo_sapiens/UCSC/hg38/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Homo_sapiens/UCSC/hg38/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Homo_sapiens/UCSC/hg38/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Homo_sapiens/UCSC/hg38/Annotation/Genes/genes.bed"
+ mito_name = "chrM"
+ macs_gsize = "2.7e9"
+ blacklist = "${projectDir}/assets/blacklists/hg38-blacklist.bed"
+ }
+ 'hg19' {
+ fasta = "${params.igenomes_base}/Homo_sapiens/UCSC/hg19/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Homo_sapiens/UCSC/hg19/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Homo_sapiens/UCSC/hg19/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Homo_sapiens/UCSC/hg19/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Homo_sapiens/UCSC/hg19/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Homo_sapiens/UCSC/hg19/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Homo_sapiens/UCSC/hg19/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Homo_sapiens/UCSC/hg19/Annotation/README.txt"
+ mito_name = "chrM"
+ macs_gsize = "2.7e9"
+ blacklist = "${projectDir}/assets/blacklists/hg19-blacklist.bed"
+ }
+ 'mm10' {
+ fasta = "${params.igenomes_base}/Mus_musculus/UCSC/mm10/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Mus_musculus/UCSC/mm10/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Mus_musculus/UCSC/mm10/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Mus_musculus/UCSC/mm10/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Mus_musculus/UCSC/mm10/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Mus_musculus/UCSC/mm10/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Mus_musculus/UCSC/mm10/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Mus_musculus/UCSC/mm10/Annotation/README.txt"
+ mito_name = "chrM"
+ macs_gsize = "1.87e9"
+ blacklist = "${projectDir}/assets/blacklists/mm10-blacklist.bed"
+ }
+ 'bosTau8' {
+ fasta = "${params.igenomes_base}/Bos_taurus/UCSC/bosTau8/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Bos_taurus/UCSC/bosTau8/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Bos_taurus/UCSC/bosTau8/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Bos_taurus/UCSC/bosTau8/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Bos_taurus/UCSC/bosTau8/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Bos_taurus/UCSC/bosTau8/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Bos_taurus/UCSC/bosTau8/Annotation/Genes/genes.bed"
+ mito_name = "chrM"
+ }
+ 'ce10' {
+ fasta = "${params.igenomes_base}/Caenorhabditis_elegans/UCSC/ce10/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Caenorhabditis_elegans/UCSC/ce10/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Caenorhabditis_elegans/UCSC/ce10/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Caenorhabditis_elegans/UCSC/ce10/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Caenorhabditis_elegans/UCSC/ce10/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Caenorhabditis_elegans/UCSC/ce10/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Caenorhabditis_elegans/UCSC/ce10/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Caenorhabditis_elegans/UCSC/ce10/Annotation/README.txt"
+ mito_name = "chrM"
+ macs_gsize = "9e7"
+ }
+ 'canFam3' {
+ fasta = "${params.igenomes_base}/Canis_familiaris/UCSC/canFam3/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Canis_familiaris/UCSC/canFam3/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Canis_familiaris/UCSC/canFam3/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Canis_familiaris/UCSC/canFam3/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Canis_familiaris/UCSC/canFam3/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Canis_familiaris/UCSC/canFam3/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Canis_familiaris/UCSC/canFam3/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Canis_familiaris/UCSC/canFam3/Annotation/README.txt"
+ mito_name = "chrM"
+ }
+ 'danRer10' {
+ fasta = "${params.igenomes_base}/Danio_rerio/UCSC/danRer10/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Danio_rerio/UCSC/danRer10/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Danio_rerio/UCSC/danRer10/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Danio_rerio/UCSC/danRer10/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Danio_rerio/UCSC/danRer10/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Danio_rerio/UCSC/danRer10/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Danio_rerio/UCSC/danRer10/Annotation/Genes/genes.bed"
+ mito_name = "chrM"
+ macs_gsize = "1.37e9"
+ }
+ 'dm6' {
+ fasta = "${params.igenomes_base}/Drosophila_melanogaster/UCSC/dm6/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Drosophila_melanogaster/UCSC/dm6/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Drosophila_melanogaster/UCSC/dm6/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Drosophila_melanogaster/UCSC/dm6/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Drosophila_melanogaster/UCSC/dm6/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Drosophila_melanogaster/UCSC/dm6/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Drosophila_melanogaster/UCSC/dm6/Annotation/Genes/genes.bed"
+ mito_name = "chrM"
+ macs_gsize = "1.2e8"
+ }
+ 'equCab2' {
+ fasta = "${params.igenomes_base}/Equus_caballus/UCSC/equCab2/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Equus_caballus/UCSC/equCab2/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Equus_caballus/UCSC/equCab2/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Equus_caballus/UCSC/equCab2/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Equus_caballus/UCSC/equCab2/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Equus_caballus/UCSC/equCab2/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Equus_caballus/UCSC/equCab2/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Equus_caballus/UCSC/equCab2/Annotation/README.txt"
+ mito_name = "chrM"
+ }
+ 'galGal4' {
+ fasta = "${params.igenomes_base}/Gallus_gallus/UCSC/galGal4/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Gallus_gallus/UCSC/galGal4/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Gallus_gallus/UCSC/galGal4/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Gallus_gallus/UCSC/galGal4/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Gallus_gallus/UCSC/galGal4/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Gallus_gallus/UCSC/galGal4/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Gallus_gallus/UCSC/galGal4/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Gallus_gallus/UCSC/galGal4/Annotation/README.txt"
+ mito_name = "chrM"
+ }
+ 'panTro4' {
+ fasta = "${params.igenomes_base}/Pan_troglodytes/UCSC/panTro4/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Pan_troglodytes/UCSC/panTro4/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Pan_troglodytes/UCSC/panTro4/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Pan_troglodytes/UCSC/panTro4/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Pan_troglodytes/UCSC/panTro4/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Pan_troglodytes/UCSC/panTro4/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Pan_troglodytes/UCSC/panTro4/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Pan_troglodytes/UCSC/panTro4/Annotation/README.txt"
+ mito_name = "chrM"
+ }
+ 'rn6' {
+ fasta = "${params.igenomes_base}/Rattus_norvegicus/UCSC/rn6/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Rattus_norvegicus/UCSC/rn6/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Rattus_norvegicus/UCSC/rn6/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Rattus_norvegicus/UCSC/rn6/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Rattus_norvegicus/UCSC/rn6/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Rattus_norvegicus/UCSC/rn6/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Rattus_norvegicus/UCSC/rn6/Annotation/Genes/genes.bed"
+ mito_name = "chrM"
+ }
+ 'sacCer3' {
+ fasta = "${params.igenomes_base}/Saccharomyces_cerevisiae/UCSC/sacCer3/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Saccharomyces_cerevisiae/UCSC/sacCer3/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Saccharomyces_cerevisiae/UCSC/sacCer3/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Saccharomyces_cerevisiae/UCSC/sacCer3/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Saccharomyces_cerevisiae/UCSC/sacCer3/Sequence/BismarkIndex/"
+ readme = "${params.igenomes_base}/Saccharomyces_cerevisiae/UCSC/sacCer3/Annotation/README.txt"
+ mito_name = "chrM"
+ macs_gsize = "1.2e7"
+ }
+ 'susScr3' {
+ fasta = "${params.igenomes_base}/Sus_scrofa/UCSC/susScr3/Sequence/WholeGenomeFasta/genome.fa"
+ bwa = "${params.igenomes_base}/Sus_scrofa/UCSC/susScr3/Sequence/BWAIndex/genome.fa"
+ bowtie2 = "${params.igenomes_base}/Sus_scrofa/UCSC/susScr3/Sequence/Bowtie2Index/"
+ star = "${params.igenomes_base}/Sus_scrofa/UCSC/susScr3/Sequence/STARIndex/"
+ bismark = "${params.igenomes_base}/Sus_scrofa/UCSC/susScr3/Sequence/BismarkIndex/"
+ gtf = "${params.igenomes_base}/Sus_scrofa/UCSC/susScr3/Annotation/Genes/genes.gtf"
+ bed12 = "${params.igenomes_base}/Sus_scrofa/UCSC/susScr3/Annotation/Genes/genes.bed"
+ readme = "${params.igenomes_base}/Sus_scrofa/UCSC/susScr3/Annotation/README.txt"
+ mito_name = "chrM"
}
}
}
diff --git a/conf/test.config b/conf/test.config
index a03678b36aa77f315c7e31909247c57242922d61..02f48cb2238a0a51c70442ee2d0571f364d1218c 100644
--- a/conf/test.config
+++ b/conf/test.config
@@ -4,19 +4,22 @@
* -------------------------------------------------
* Defines bundled input files and everything required
* to run a fast and simple test. Use as follows:
- * nextflow run nf-core/hic -profile test
+ * nextflow run nf-core/hic -profile test,<docker/singularity>
*/
params {
- // Limit resources so that this can run on Travis
+ config_profile_name = 'Test profile'
+ config_profile_description = 'Minimal test dataset to check pipeline function'
+ // Limit resources so that this can run on GitHub Actions
max_cpus = 2
max_memory = 6.GB
max_time = 48.h
+
// Input data
// TODO nf-core: Specify the paths to your test data on nf-core/test-datasets
// TODO nf-core: Give any required params for the test so that command line flags are not needed
- singleEnd = false
- readPaths = [
+ single_end = false
+ input_paths = [
['Testdata', ['https://github.com/nf-core/test-datasets/raw/exoseq/testdata/Testdata_R1.tiny.fastq.gz', 'https://github.com/nf-core/test-datasets/raw/exoseq/testdata/Testdata_R2.tiny.fastq.gz']],
['SRR389222', ['https://github.com/nf-core/test-datasets/raw/methylseq/testdata/SRR389222_sub1.fastq.gz', 'https://github.com/nf-core/test-datasets/raw/methylseq/testdata/SRR389222_sub2.fastq.gz']]
]
diff --git a/conf/test_full.config b/conf/test_full.config
new file mode 100644
index 0000000000000000000000000000000000000000..921372eec05fe6147c540a39463c54f0ebf09bce
--- /dev/null
+++ b/conf/test_full.config
@@ -0,0 +1,22 @@
+/*
+ * -------------------------------------------------
+ * Nextflow config file for running full-size tests
+ * -------------------------------------------------
+ * Defines bundled input files and everything required
+ * to run a full size pipeline test. Use as follows:
+ * nextflow run nf-core/hic -profile test_full,<docker/singularity>
+ */
+
+params {
+ config_profile_name = 'Full test profile'
+ config_profile_description = 'Full test dataset to check pipeline function'
+
+ // Input data for full size test
+ // TODO nf-core: Specify the paths to your full test data ( on nf-core/test-datasets or directly in repositories, e.g. SRA)
+ // TODO nf-core: Give any required params for the test so that command line flags are not needed
+ single_end = false
+ input_paths = [
+ ['Testdata', ['https://github.com/nf-core/test-datasets/raw/exoseq/testdata/Testdata_R1.tiny.fastq.gz', 'https://github.com/nf-core/test-datasets/raw/exoseq/testdata/Testdata_R2.tiny.fastq.gz']],
+ ['SRR389222', ['https://github.com/nf-core/test-datasets/raw/methylseq/testdata/SRR389222_sub1.fastq.gz', 'https://github.com/nf-core/test-datasets/raw/methylseq/testdata/SRR389222_sub2.fastq.gz']]
+ ]
+}
diff --git a/docs/README.md b/docs/README.md
index d7dbdac40b9452baa3d7d7747d264077276fb679..a6889549c7f27bda0aed81947685713781fe2d1b 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -1,12 +1,10 @@
# nf-core/hic: Documentation
-The nf-core/hic documentation is split into the following files:
+The nf-core/hic documentation is split into the following pages:
-1. [Installation](installation.md)
-2. Pipeline configuration
- * [Local installation](configuration/local.md)
- * [Adding your own system](configuration/adding_your_own.md)
- * [Reference genomes](configuration/reference_genomes.md)
-3. [Running the pipeline](usage.md)
-4. [Output and how to interpret the results](output.md)
-5. [Troubleshooting](troubleshooting.md)
+* [Usage](usage.md)
+ * An overview of how the pipeline works, how to run it and a description of all of the different command-line flags.
+* [Output](output.md)
+ * An overview of the different results produced by the pipeline and how to interpret them.
+
+You can find a lot more documentation about installing, configuring and running nf-core pipelines on the website: [https://nf-co.re](https://nf-co.re)
diff --git a/docs/configuration/adding_your_own.md b/docs/configuration/adding_your_own.md
deleted file mode 100644
index bf7f80811e08de09bfdca2a94ed1187d1906796f..0000000000000000000000000000000000000000
--- a/docs/configuration/adding_your_own.md
+++ /dev/null
@@ -1,86 +0,0 @@
-# nf-core/hic: Configuration for other clusters
-
-It is entirely possible to run this pipeline on other clusters, though you will need to set up your own config file so that the pipeline knows how to work with your cluster.
-
-> If you think that there are other people using the pipeline who would benefit from your configuration (eg. other common cluster setups), please let us know. We can add a new configuration and profile which can used by specifying `-profile <name>` when running the pipeline. The config file will then be hosted at `nf-core/configs` and will be pulled automatically before the pipeline is executed.
-
-If you are the only person to be running this pipeline, you can create your config file as `~/.nextflow/config` and it will be applied every time you run Nextflow. Alternatively, save the file anywhere and reference it when running the pipeline with `-c path/to/config` (see the [Nextflow documentation](https://www.nextflow.io/docs/latest/config.html) for more).
-
-A basic configuration comes with the pipeline, which loads the [`conf/base.config`](../../conf/base.config) by default. This means that you only need to configure the specifics for your system and overwrite any defaults that you want to change.
-
-## Cluster Environment
-By default, pipeline uses the `local` Nextflow executor - in other words, all jobs are run in the login session. If you're using a simple server, this may be fine. If you're using a compute cluster, this is bad as all jobs will run on the head node.
-
-To specify your cluster environment, add the following line to your config file:
-
-```nextflow
-process.executor = 'YOUR_SYSTEM_TYPE'
-```
-
-Many different cluster types are supported by Nextflow. For more information, please see the [Nextflow documentation](https://www.nextflow.io/docs/latest/executor.html).
-
-Note that you may need to specify cluster options, such as a project or queue. To do so, use the `clusterOptions` config option:
-
-```nextflow
-process {
- executor = 'SLURM'
- clusterOptions = '-A myproject'
-}
-```
-
-
-## Software Requirements
-To run the pipeline, several software packages are required. How you satisfy these requirements is essentially up to you and depends on your system. If possible, we _highly_ recommend using either Docker or Singularity.
-
-Please see the [`installation documentation`](../installation.md) for how to run using the below as a one-off. These instructions are about configuring a config file for repeated use.
-
-### Docker
-Docker is a great way to run nf-core/hic, as it manages all software installations and allows the pipeline to be run in an identical software environment across a range of systems.
-
-Nextflow has [excellent integration](https://www.nextflow.io/docs/latest/docker.html) with Docker, and beyond installing the two tools, not much else is required - nextflow will automatically fetch the [nfcore/hic](https://hub.docker.com/r/nfcore/hic/) image that we have created and is hosted at dockerhub at run time.
-
-To add docker support to your own config file, add the following:
-
-```nextflow
-docker.enabled = true
-process.container = "nfcore/hic"
-```
-
-Note that the dockerhub organisation name annoyingly can't have a hyphen, so is `nfcore` and not `nf-core`.
-
-
-### Singularity image
-Many HPC environments are not able to run Docker due to security issues.
-[Singularity](http://singularity.lbl.gov/) is a tool designed to run on such HPC systems which is very similar to Docker.
-
-To specify singularity usage in your pipeline config file, add the following:
-
-```nextflow
-singularity.enabled = true
-process.container = "shub://nf-core/hic"
-```
-
-If you intend to run the pipeline offline, nextflow will not be able to automatically download the singularity image for you.
-Instead, you'll have to do this yourself manually first, transfer the image file and then point to that.
-
-First, pull the image file where you have an internet connection:
-
-```bash
-singularity pull --name nf-core-hic.simg shub://nf-core/hic
-```
-
-Then transfer this file and point the config file to the image:
-
-```nextflow
-singularity.enabled = true
-process.container = "/path/to/nf-core-hic.simg"
-```
-
-
-### Conda
-If you're not able to use Docker or Singularity, you can instead use conda to manage the software requirements.
-To use conda in your own config file, add the following:
-
-```nextflow
-process.conda = "$baseDir/environment.yml"
-```
diff --git a/docs/configuration/local.md b/docs/configuration/local.md
deleted file mode 100644
index 657422ffdaed246d92580b427e7e56cdabbf5cab..0000000000000000000000000000000000000000
--- a/docs/configuration/local.md
+++ /dev/null
@@ -1,46 +0,0 @@
-# nf-core/hic: Local Configuration
-
-If running the pipeline in a local environment, we highly recommend using either Docker or Singularity.
-
-## Docker
-Docker is a great way to run `nf-core/hic`, as it manages all software installations and allows the pipeline to be run in an identical software environment across a range of systems.
-
-Nextflow has [excellent integration](https://www.nextflow.io/docs/latest/docker.html) with Docker, and beyond installing the two tools, not much else is required. The `nf-core/hic` profile comes with a configuration profile for docker, making it very easy to use. This also comes with the required presets to use the AWS iGenomes resource, meaning that if using common reference genomes you just specify the reference ID and it will be automatically downloaded from AWS S3.
-
-First, install docker on your system: [Docker Installation Instructions](https://docs.docker.com/engine/installation/)
-
-Then, simply run the analysis pipeline:
-```bash
-nextflow run nf-core/hic -profile docker --genome '<genome ID>' --design '<path to your design file>'
-```
-
-Nextflow will recognise `nf-core/hic` and download the pipeline from GitHub. The `-profile docker` configuration lists the [nf-core/hic](https://hub.docker.com/r/nfcore/hic/) image that we have created and is hosted at dockerhub, and this is downloaded.
-
-For more information about how to work with reference genomes, see [`docs/configuration/reference_genomes.md`](reference_genomes.md).
-
-### Pipeline versions
-The public docker images are tagged with the same version numbers as the code, which you can use to ensure reproducibility. When running the pipeline, specify the pipeline version with `-r`, for example `-r 1.0`. This uses pipeline code and docker image from this tagged version.
-
-
-## Singularity image
-Many HPC environments are not able to run Docker due to security issues. [Singularity](http://singularity.lbl.gov/) is a tool designed to run on such HPC systems which is very similar to Docker. Even better, it can use create images directly from dockerhub.
-
-To use the singularity image for a single run, use `-with-singularity`. This will download the docker container from dockerhub and create a singularity image for you dynamically.
-
-If you intend to run the pipeline offline, nextflow will not be able to automatically download the singularity image for you. Instead, you'll have to do this yourself manually first, transfer the image file and then point to that.
-
-First, pull the image file where you have an internet connection:
-
-> NB: The "tag" at the end of this command corresponds to the pipeline version.
-> Here, we're pulling the docker image for version 1.0 of the nf-core/hic pipeline
-> Make sure that this tag corresponds to the version of the pipeline that you're using
-
-```bash
-singularity pull --name nf-core-hic-1.0.img docker://nf-core/hic:1.0
-```
-
-Then transfer this file and run the pipeline with this path:
-
-```bash
-nextflow run /path/to/nf-core-hic -with-singularity /path/to/nf-core-hic-1.0.img
-```
diff --git a/docs/configuration/reference_genomes.md b/docs/configuration/reference_genomes.md
deleted file mode 100644
index 1fafa8feeaaef766fa294b9d1de1bd32ba1f7dc2..0000000000000000000000000000000000000000
--- a/docs/configuration/reference_genomes.md
+++ /dev/null
@@ -1,49 +0,0 @@
-# nf-core/hic: Reference Genomes Configuration
-
-The nf-core/hic pipeline needs a reference genome for alignment and annotation.
-
-These paths can be supplied on the command line at run time (see the [usage docs](../usage.md)),
-but for convenience it's often better to save these paths in a nextflow config file.
-See below for instructions on how to do this.
-Read [Adding your own system](adding_your_own.md) to find out how to set up custom config files.
-
-## Adding paths to a config file
-Specifying long paths every time you run the pipeline is a pain.
-To make this easier, the pipeline comes configured to understand reference genome keywords which correspond to preconfigured paths, meaning that you can just specify `--genome ID` when running the pipeline.
-
-Note that this genome key can also be specified in a config file if you always use the same genome.
-
-To use this system, add paths to your config file using the following template:
-
-```nextflow
-params {
- genomes {
- 'YOUR-ID' {
- fasta = '<PATH TO FASTA FILE>/genome.fa'
- }
- 'OTHER-GENOME' {
- // [..]
- }
- }
- // Optional - default genome. Ignored if --genome 'OTHER-GENOME' specified on command line
- genome = 'YOUR-ID'
-}
-```
-
-You can add as many genomes as you like as long as they have unique IDs.
-
-## illumina iGenomes
-To make the use of reference genomes easier, illumina has developed a centralised resource called [iGenomes](https://support.illumina.com/sequencing/sequencing_software/igenome.html).
-Multiple reference index types are held together with consistent structure for multiple genomes.
-
-We have put a copy of iGenomes up onto AWS S3 hosting and this pipeline is configured to use this by default.
-The hosting fees for AWS iGenomes are currently kindly funded by a grant from Amazon.
-The pipeline will automatically download the required reference files when you run the pipeline.
-For more information about the AWS iGenomes, see https://ewels.github.io/AWS-iGenomes/
-
-Downloading the files takes time and bandwidth, so we recommend making a local copy of the iGenomes resource.
-Once downloaded, you can customise the variable `params.igenomes_base` in your custom configuration file to point to the reference location.
-For example:
-```nextflow
-params.igenomes_base = '/path/to/data/igenomes/'
-```
diff --git a/docs/images/nf-core-hic_logo.png b/docs/images/nf-core-hic_logo.png
new file mode 100644
index 0000000000000000000000000000000000000000..274eb3dc3f3db879c7f3cbc3fd8f49a705a9a3fb
Binary files /dev/null and b/docs/images/nf-core-hic_logo.png differ
diff --git a/docs/installation.md b/docs/installation.md
deleted file mode 100644
index 70c4a6d472f51c28925ab2bedf4f7ec4b468ad7e..0000000000000000000000000000000000000000
--- a/docs/installation.md
+++ /dev/null
@@ -1,110 +0,0 @@
-# nf-core/hic: Installation
-
-To start using the nf-core/hic pipeline, follow the steps below:
-
-1. [Install Nextflow](#1-install-nextflow)
-2. [Install the pipeline](#2-install-the-pipeline)
- * [Automatic](#21-automatic)
- * [Offline](#22-offline)
- * [Development](#23-development)
-3. [Pipeline configuration](#3-pipeline-configuration)
- * [Software deps: Docker and Singularity](#31-software-deps-docker-and-singularity)
- * [Software deps: Bioconda](#32-software-deps-bioconda)
- * [Configuration profiles](#33-configuration-profiles)
-4. [Reference genomes](#4-reference-genomes)
-
-## 1) Install NextFlow
-Nextflow runs on most POSIX systems (Linux, Mac OSX etc). It can be installed by running the following commands:
-
-```bash
-# Make sure that Java v8+ is installed:
-java -version
-
-# Install Nextflow
-curl -fsSL get.nextflow.io | bash
-
-# Add Nextflow binary to your PATH:
-mv nextflow ~/bin/
-# OR system-wide installation:
-# sudo mv nextflow /usr/local/bin
-```
-
-See [nextflow.io](https://www.nextflow.io/) for further instructions on how to install and configure Nextflow.
-
-## 2) Install the pipeline
-
-#### 2.1) Automatic
-This pipeline itself needs no installation - NextFlow will automatically fetch it from GitHub if `nf-core/hic` is specified as the pipeline name.
-
-#### 2.2) Offline
-The above method requires an internet connection so that Nextflow can download the pipeline files. If you're running on a system that has no internet connection, you'll need to download and transfer the pipeline files manually:
-
-```bash
-wget https://github.com/nf-core/hic/archive/master.zip
-mkdir -p ~/my-pipelines/nf-core/
-unzip master.zip -d ~/my-pipelines/nf-core/
-cd ~/my_data/
-nextflow run ~/my-pipelines/nf-core/hic-master
-```
-
-To stop nextflow from looking for updates online, you can tell it to run in offline mode by specifying the following environment variable in your ~/.bashrc file:
-
-```bash
-export NXF_OFFLINE='TRUE'
-```
-
-#### 2.3) Development
-
-If you would like to make changes to the pipeline, it's best to make a fork on GitHub and then clone the files. Once cloned you can run the pipeline directly as above.
-
-
-## 3) Pipeline configuration
-By default, the pipeline loads a basic server configuration [`conf/base.config`](../conf/base.config)
-This uses a number of sensible defaults for process requirements and is suitable for running
-on a simple (if powerful!) local server.
-
-Be warned of two important points about this default configuration:
-
-1. The default profile uses the `local` executor
- * All jobs are run in the login session. If you're using a simple server, this may be fine. If you're using a compute cluster, this is bad as all jobs will run on the head node.
- * See the [nextflow docs](https://www.nextflow.io/docs/latest/executor.html) for information about running with other hardware backends. Most job scheduler systems are natively supported.
-2. Nextflow will expect all software to be installed and available on the `PATH`
- * It's expected to use an additional config profile for docker, singularity or conda support. See below.
-
-#### 3.1) Software deps: Docker
-First, install docker on your system: [Docker Installation Instructions](https://docs.docker.com/engine/installation/)
-
-Then, running the pipeline with the option `-profile docker` tells Nextflow to enable Docker for this run. An image containing all of the software requirements will be automatically fetched and used from dockerhub (https://hub.docker.com/r/nfcore/hic).
-
-#### 3.1) Software deps: Singularity
-If you're not able to use Docker then [Singularity](http://singularity.lbl.gov/) is a great alternative.
-The process is very similar: running the pipeline with the option `-profile singularity` tells Nextflow to enable singularity for this run. An image containing all of the software requirements will be automatically fetched and used from singularity hub.
-
-If running offline with Singularity, you'll need to download and transfer the Singularity image first:
-
-```bash
-singularity pull --name nf-core-hic.simg shub://nf-core/hic
-```
-
-Once transferred, use `-with-singularity` and specify the path to the image file:
-
-```bash
-nextflow run /path/to/nf-core-hic -with-singularity nf-core-hic.simg
-```
-
-Remember to pull updated versions of the singularity image if you update the pipeline.
-
-
-#### 3.2) Software deps: conda
-If you're not able to use Docker _or_ Singularity, you can instead use conda to manage the software requirements.
-This is slower and less reproducible than the above, but is still better than having to install all requirements yourself!
-The pipeline ships with a conda environment file and nextflow has built-in support for this.
-To use it first ensure that you have conda installed (we recommend [miniconda](https://conda.io/miniconda.html)), then follow the same pattern as above and use the flag `-profile conda`
-
-#### 3.3) Configuration profiles
-
-See [`docs/configuration/adding_your_own.md`](configuration/adding_your_own.md)
-
-## 4) Reference genomes
-
-See [`docs/configuration/reference_genomes.md`](configuration/reference_genomes.md)
diff --git a/docs/output.md b/docs/output.md
index 9de7067b987fe10ef7435c135bbd58545f13ecd2..4a7372e92d17cf7897c93fb403d6bd41c2bc2f29 100644
--- a/docs/output.md
+++ b/docs/output.md
@@ -1,41 +1,63 @@
# nf-core/hic: Output
+## :warning: Please read this documentation on the nf-core website: [https://nf-co.re/hic/output](https://nf-co.re/hic/output)
+
+> _Documentation of pipeline parameters is generated automatically from the pipeline schema and can no longer be found in markdown files._
+
+## Introduction
+
This document describes the output produced by the pipeline. Most of the plots are taken from the MultiQC report, which summarises results at the end of the pipeline.
+The directories listed below will be created in the results directory after the pipeline has finished. All paths are relative to the top-level results directory.
+
<!-- TODO nf-core: Write this documentation describing your workflow's output -->
## Pipeline overview
+
The pipeline is built using [Nextflow](https://www.nextflow.io/)
and processes data using the following steps:
-* [FastQC](#fastqc) - read quality control
-* [MultiQC](#multiqc) - aggregate report, describing results of the whole pipeline
+* [FastQC](#fastqc) - Read quality control
+* [MultiQC](#multiqc) - Aggregate report describing results from the whole pipeline
+* [Pipeline information](#pipeline-information) - Report metrics generated during the workflow execution
## FastQC
-[FastQC](http://www.bioinformatics.babraham.ac.uk/projects/fastqc/) gives general quality metrics about your reads. It provides information about the quality score distribution across your reads, the per base sequence content (%T/A/G/C). You get information about adapter contamination and other overrepresented sequences.
-For further reading and documentation see the [FastQC help](http://www.bioinformatics.babraham.ac.uk/projects/fastqc/Help/).
+[FastQC](http://www.bioinformatics.babraham.ac.uk/projects/fastqc/) gives general quality metrics about your sequenced reads. It provides information about the quality score distribution across your reads, per base sequence content (%A/T/G/C), adapter contamination and overrepresented sequences.
-> **NB:** The FastQC plots displayed in the MultiQC report shows _untrimmed_ reads. They may contain adapter sequence and potentially regions with low quality. To see how your reads look after trimming, look at the FastQC reports in the `trim_galore` directory.
+For further reading and documentation see the [FastQC help pages](http://www.bioinformatics.babraham.ac.uk/projects/fastqc/Help/).
-**Output directory: `results/fastqc`**
+**Output files:**
-* `sample_fastqc.html`
- * FastQC report, containing quality metrics for your untrimmed raw fastq files
-* `zips/sample_fastqc.zip`
- * zip file containing the FastQC report, tab-delimited data file and plot images
+* `fastqc/`
+ * `*_fastqc.html`: FastQC report containing quality metrics for your untrimmed raw fastq files.
+* `fastqc/zips/`
+ * `*_fastqc.zip`: Zip archive containing the FastQC report, tab-delimited data file and plot images.
+> **NB:** The FastQC plots displayed in the MultiQC report shows _untrimmed_ reads. They may contain adapter sequence and potentially regions with low quality.
## MultiQC
-[MultiQC](http://multiqc.info) is a visualisation tool that generates a single HTML report summarising all samples in your project. Most of the pipeline QC results are visualised in the report and further statistics are available in within the report data directory.
-The pipeline has special steps which allow the software versions used to be reported in the MultiQC output for future traceability.
+[MultiQC](http://multiqc.info) is a visualization tool that generates a single HTML report summarizing all samples in your project. Most of the pipeline QC results are visualised in the report and further statistics are available in the report data directory.
+
+The pipeline has special steps which also allow the software versions to be reported in the MultiQC output for future traceability.
+
+For more information about how to use MultiQC reports, see [https://multiqc.info](https://multiqc.info).
+
+**Output files:**
+
+* `multiqc/`
+ * `multiqc_report.html`: a standalone HTML file that can be viewed in your web browser.
+ * `multiqc_data/`: directory containing parsed statistics from the different tools used in the pipeline.
+ * `multiqc_plots/`: directory containing static images from the report in various formats.
+
+## Pipeline information
-**Output directory: `results/multiqc`**
+[Nextflow](https://www.nextflow.io/docs/latest/tracing.html) provides excellent functionality for generating various reports relevant to the running and execution of the pipeline. This will allow you to troubleshoot errors with the running of the pipeline, and also provide you with other information such as launch commands, run times and resource usage.
-* `Project_multiqc_report.html`
- * MultiQC report - a standalone HTML file that can be viewed in your web browser
-* `Project_multiqc_data/`
- * Directory containing parsed statistics from the different tools used in the pipeline
+**Output files:**
-For more information about how to use MultiQC reports, see http://multiqc.info
+* `pipeline_info/`
+ * Reports generated by Nextflow: `execution_report.html`, `execution_timeline.html`, `execution_trace.txt` and `pipeline_dag.dot`/`pipeline_dag.svg`.
+ * Reports generated by the pipeline: `pipeline_report.html`, `pipeline_report.txt` and `software_versions.csv`.
+ * Documentation for interpretation of results in HTML format: `results_description.html`.
diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md
deleted file mode 100644
index 927d1b712a15d979f05fc3d74f1c16ddfd9d457c..0000000000000000000000000000000000000000
--- a/docs/troubleshooting.md
+++ /dev/null
@@ -1,30 +0,0 @@
-# nf-core/hic: Troubleshooting
-
-<!-- TODO nf-core: Change this documentation if these parameters/errors are not relevant for your workflow -->
-
-## Input files not found
-
-If only no file, only one input file , or only read one and not read two is picked up then something is wrong with your input file declaration
-
-1. The path must be enclosed in quotes (`'` or `"`)
-2. The path must have at least one `*` wildcard character. This is even if you are only running one paired end sample.
-3. When using the pipeline with paired end data, the path must use `{1,2}` or `{R1,R2}` notation to specify read pairs.
-4. If you are running Single end data make sure to specify `--singleEnd`
-
-If the pipeline can't find your files then you will get the following error
-
-```
-ERROR ~ Cannot find any reads matching: *{1,2}.fastq.gz
-```
-
-Note that if your sample name is "messy" then you have to be very particular with your glob specification. A file name like `L1-1-D-2h_S1_L002_R1_001.fastq.gz` can be difficult enough for a human to read. Specifying `*{1,2}*.gz` wont work give you what you want Whilst `*{R1,R2}*.gz` will.
-
-
-## Data organization
-The pipeline can't take a list of multiple input files - it takes a glob expression. If your input files are scattered in different paths then we recommend that you generate a directory with symlinked files. If running in paired end mode please make sure that your files are sensibly named so that they can be properly paired. See the previous point.
-
-## Extra resources and getting help
-If you still have an issue with running the pipeline then feel free to contact us.
-Have a look at the [pipeline website](https://github.com/nf-core/hic) to find out how.
-
-If you have problems that are related to Nextflow and not our pipeline then check out the [Nextflow gitter channel](https://gitter.im/nextflow-io/nextflow) or the [google group](https://groups.google.com/forum/#!forum/nextflow).
diff --git a/docs/usage.md b/docs/usage.md
index 95d863f688afbf78f738249aade28bd711deb60b..76803ddcd3e60bf2dc060ced21e610abf5cff261 100644
--- a/docs/usage.md
+++ b/docs/usage.md
@@ -1,58 +1,19 @@
# nf-core/hic: Usage
-## Table of contents
-
-* [Introduction](#general-nextflow-info)
-* [Running the pipeline](#running-the-pipeline)
-* [Updating the pipeline](#updating-the-pipeline)
-* [Reproducibility](#reproducibility)
-* [Main arguments](#main-arguments)
- * [`-profile`](#-profile-single-dash)
- * [`awsbatch`](#awsbatch)
- * [`conda`](#conda)
- * [`docker`](#docker)
- * [`singularity`](#singularity)
- * [`test`](#test)
- * [`--reads`](#--reads)
- * [`--singleEnd`](#--singleend)
-* [Reference genomes](#reference-genomes)
- * [`--genome`](#--genome)
- * [`--fasta`](#--fasta)
- * [`--igenomesIgnore`](#--igenomesignore)
-* [Job resources](#job-resources)
-* [Automatic resubmission](#automatic-resubmission)
-* [Custom resource requests](#custom-resource-requests)
-* [AWS batch specific parameters](#aws-batch-specific-parameters)
- * [`-awsbatch`](#-awsbatch)
- * [`--awsqueue`](#--awsqueue)
- * [`--awsregion`](#--awsregion)
-* [Other command line parameters](#other-command-line-parameters)
- * [`--outdir`](#--outdir)
- * [`--email`](#--email)
- * [`-name`](#-name-single-dash)
- * [`-resume`](#-resume-single-dash)
- * [`-c`](#-c-single-dash)
- * [`--custom_config_version`](#--custom_config_version)
- * [`--max_memory`](#--max_memory)
- * [`--max_time`](#--max_time)
- * [`--max_cpus`](#--max_cpus)
- * [`--plaintext_email`](#--plaintext_email)
- * [`--multiqc_config`](#--multiqc_config)
-
-
-## General Nextflow info
-Nextflow handles job submissions on SLURM or other environments, and supervises running the jobs. Thus the Nextflow process must run until the pipeline is finished. We recommend that you put the process running in the background through `screen` / `tmux` or similar tool. Alternatively you can run nextflow within a cluster job submitted your job scheduler.
-
-It is recommended to limit the Nextflow Java virtual machines memory. We recommend adding the following line to your environment (typically in `~/.bashrc` or `~./bash_profile`):
+## :warning: Please read this documentation on the nf-core website: [https://nf-co.re/hic/usage](https://nf-co.re/hic/usage)
+
+> _Documentation of pipeline parameters is generated automatically from the pipeline schema and can no longer be found in markdown files._
+
+## Introduction
+
+<!-- TODO nf-core: Add documentation about anything specific to running your pipeline. For general topics, please point to (and add to) the main nf-core website. -->
-```bash
-NXF_OPTS='-Xms1g -Xmx4g'
-```
-<!-- TODO nf-core: Document required command line parameters to run the pipeline-->
## Running the pipeline
+
The typical command for running the pipeline is as follows:
+
```bash
-nextflow run nf-core/hic --reads '*_R{1,2}.fastq.gz' -profile docker
+nextflow run nf-core/hic --input '*_R{1,2}.fastq.gz' -profile docker
```
This will launch the pipeline with the `docker` configuration profile. See below for more information about profiles.
@@ -67,6 +28,7 @@ results # Finished results (configurable, see below)
```
### Updating the pipeline
+
When you run the above command, Nextflow automatically pulls the pipeline code from GitHub and stores it as a cached version. When running the pipeline after this, it will always use the cached version if available - even if the pipeline has been updated since. To make sure that you're running the latest version of the pipeline, make sure that you regularly update the cached version of the pipeline:
```bash
@@ -74,181 +36,93 @@ nextflow pull nf-core/hic
```
### Reproducibility
+
It's a good idea to specify a pipeline version when running the pipeline on your data. This ensures that a specific version of the pipeline code and software are used when you run your pipeline. If you keep using the same tag, you'll be running the same version of the pipeline, even if there have been changes to the code since.
First, go to the [nf-core/hic releases page](https://github.com/nf-core/hic/releases) and find the latest version number - numeric only (eg. `1.3.1`). Then specify this when running the pipeline with `-r` (one hyphen) - eg. `-r 1.3.1`.
This version number will be logged in reports when you run the pipeline, so that you'll know what you used when you look back in the future.
+## Core Nextflow arguments
-## Main arguments
+> **NB:** These options are part of Nextflow and use a _single_ hyphen (pipeline parameters use a double-hyphen).
### `-profile`
-Use this parameter to choose a configuration profile. Profiles can give configuration presets for different compute environments. Note that multiple profiles can be loaded, for example: `-profile docker` - the order of arguments is important!
-If `-profile` is not specified at all the pipeline will be run locally and expects all software to be installed and available on the `PATH`.
+Use this parameter to choose a configuration profile. Profiles can give configuration presets for different compute environments.
-* `awsbatch`
- * A generic configuration profile to be used with AWS Batch.
-* `conda`
- * A generic configuration profile to be used with [conda](https://conda.io/docs/)
- * Pulls most software from [Bioconda](https://bioconda.github.io/)
-* `docker`
- * A generic configuration profile to be used with [Docker](http://docker.com/)
- * Pulls software from dockerhub: [`nfcore/hic`](http://hub.docker.com/r/nfcore/hic/)
-* `singularity`
- * A generic configuration profile to be used with [Singularity](http://singularity.lbl.gov/)
- * Pulls software from singularity-hub
-* `test`
- * A profile with a complete configuration for automated testing
- * Includes links to test data so needs no other parameters
+Several generic profiles are bundled with the pipeline which instruct the pipeline to use software packaged using different methods (Docker, Singularity, Podman, Conda) - see below.
-<!-- TODO nf-core: Document required command line parameters -->
-### `--reads`
-Use this to specify the location of your input FastQ files. For example:
+> We highly recommend the use of Docker or Singularity containers for full pipeline reproducibility, however when this is not possible, Conda is also supported.
-```bash
---reads 'path/to/data/sample_*_{1,2}.fastq'
-```
-
-Please note the following requirements:
-
-1. The path must be enclosed in quotes
-2. The path must have at least one `*` wildcard character
-3. When using the pipeline with paired end data, the path must use `{1,2}` notation to specify read pairs.
-
-If left unspecified, a default pattern is used: `data/*{1,2}.fastq.gz`
+The pipeline also dynamically loads configurations from [https://github.com/nf-core/configs](https://github.com/nf-core/configs) when it runs, making multiple config profiles for various institutional clusters available at run time. For more information and to see if your system is available in these configs please see the [nf-core/configs documentation](https://github.com/nf-core/configs#documentation).
-### `--singleEnd`
-By default, the pipeline expects paired-end data. If you have single-end data, you need to specify `--singleEnd` on the command line when you launch the pipeline. A normal glob pattern, enclosed in quotation marks, can then be used for `--reads`. For example:
-
-```bash
---singleEnd --reads '*.fastq'
-```
+Note that multiple profiles can be loaded, for example: `-profile test,docker` - the order of arguments is important!
+They are loaded in sequence, so later profiles can overwrite earlier profiles.
-It is not possible to run a mixture of single-end and paired-end files in one run.
+If `-profile` is not specified, the pipeline will run locally and expect all software to be installed and available on the `PATH`. This is _not_ recommended.
+* `docker`
+ * A generic configuration profile to be used with [Docker](https://docker.com/)
+ * Pulls software from Docker Hub: [`nfcore/hic`](https://hub.docker.com/r/nfcore/hic/)
+* `singularity`
+ * A generic configuration profile to be used with [Singularity](https://sylabs.io/docs/)
+ * Pulls software from Docker Hub: [`nfcore/hic`](https://hub.docker.com/r/nfcore/hic/)
+* `podman`
+ * A generic configuration profile to be used with [Podman](https://podman.io/)
+ * Pulls software from Docker Hub: [`nfcore/hic`](https://hub.docker.com/r/nfcore/hic/)
+* `conda`
+ * Please only use Conda as a last resort i.e. when it's not possible to run the pipeline with Docker, Singularity or Podman.
+ * A generic configuration profile to be used with [Conda](https://conda.io/docs/)
+ * Pulls most software from [Bioconda](https://bioconda.github.io/)
+* `test`
+ * A profile with a complete configuration for automated testing
+ * Includes links to test data so needs no other parameters
-## Reference genomes
+### `-resume`
-The pipeline config files come bundled with paths to the illumina iGenomes reference index files. If running with docker or AWS, the configuration is set up to use the [AWS-iGenomes](https://ewels.github.io/AWS-iGenomes/) resource.
+Specify this when restarting a pipeline. Nextflow will used cached results from any pipeline steps where the inputs are the same, continuing from where it got to previously.
-### `--genome` (using iGenomes)
-There are 31 different species supported in the iGenomes references. To run the pipeline, you must specify which to use with the `--genome` flag.
+You can also supply a run name to resume a specific run: `-resume [run-name]`. Use the `nextflow log` command to show previous run names.
-You can find the keys to specify the genomes in the [iGenomes config file](../conf/igenomes.config). Common genomes that are supported are:
+### `-c`
-* Human
- * `--genome GRCh37`
-* Mouse
- * `--genome GRCm38`
-* _Drosophila_
- * `--genome BDGP6`
-* _S. cerevisiae_
- * `--genome 'R64-1-1'`
+Specify the path to a specific config file (this is a core Nextflow command). See the [nf-core website documentation](https://nf-co.re/usage/configuration) for more information.
-> There are numerous others - check the config file for more.
+#### Custom resource requests
-Note that you can use the same configuration setup to save sets of reference files for your own use, even if they are not part of the iGenomes resource. See the [Nextflow documentation](https://www.nextflow.io/docs/latest/config.html) for instructions on where to save such a file.
+Each step in the pipeline has a default set of requirements for number of CPUs, memory and time. For most of the steps in the pipeline, if the job exits with an error code of `143` (exceeded requested resources) it will automatically resubmit with higher requests (2 x original, then 3 x original). If it still fails after three times then the pipeline is stopped.
-The syntax for this reference configuration is as follows:
+Whilst these default requirements will hopefully work for most people with most data, you may find that you want to customise the compute resources that the pipeline requests. You can do this by creating a custom config file. For example, to give the workflow process `star` 32GB of memory, you could use the following config:
-<!-- TODO nf-core: Update reference genome example according to what is needed -->
```nextflow
-params {
- genomes {
- 'GRCh37' {
- fasta = '<path to the genome fasta file>' // Used if no star index given
- }
- // Any number of additional genomes, key is used with --genome
+process {
+ withName: star {
+ memory = 32.GB
}
}
```
-<!-- TODO nf-core: Describe reference path flags -->
-### `--fasta`
-If you prefer, you can specify the full path to your reference genome when you run the pipeline:
-
-```bash
---fasta '[path to Fasta reference]'
-```
-
-### `--igenomesIgnore`
-Do not load `igenomes.config` when running the pipeline. You may choose this option if you observe clashes between custom parameters and those supplied in `igenomes.config`.
-
-## Job resources
-### Automatic resubmission
-Each step in the pipeline has a default set of requirements for number of CPUs, memory and time. For most of the steps in the pipeline, if the job exits with an error code of `143` (exceeded requested resources) it will automatically resubmit with higher requests (2 x original, then 3 x original). If it still fails after three times then the pipeline is stopped.
+See the main [Nextflow documentation](https://www.nextflow.io/docs/latest/config.html) for more information.
-### Custom resource requests
-Wherever process-specific requirements are set in the pipeline, the default value can be changed by creating a custom config file. See the files hosted at [`nf-core/configs`](https://github.com/nf-core/configs/tree/master/conf) for examples.
+If you are likely to be running `nf-core` pipelines regularly it may be a good idea to request that your custom config file is uploaded to the `nf-core/configs` git repository. Before you do this please can you test that the config file works with your pipeline of choice using the `-c` parameter (see definition above). You can then create a pull request to the `nf-core/configs` repository with the addition of your config file, associated documentation file (see examples in [`nf-core/configs/docs`](https://github.com/nf-core/configs/tree/master/docs)), and amending [`nfcore_custom.config`](https://github.com/nf-core/configs/blob/master/nfcore_custom.config) to include your custom profile.
-If you are likely to be running `nf-core` pipelines regularly it may be a good idea to request that your custom config file is uploaded to the `nf-core/configs` git repository. Before you do this please can you test that the config file works with your pipeline of choice using the `-c` parameter (see definition below). You can then create a pull request to the `nf-core/configs` repository with the addition of your config file, associated documentation file (see examples in [`nf-core/configs/docs`](https://github.com/nf-core/configs/tree/master/docs)), and amending [`nfcore_custom.config`](https://github.com/nf-core/configs/blob/master/nfcore_custom.config) to include your custom profile.
+If you have any questions or issues please send us a message on [Slack](https://nf-co.re/join/slack) on the [`#configs` channel](https://nfcore.slack.com/channels/configs).
-If you have any questions or issues please send us a message on [`Slack`](https://nf-core-invite.herokuapp.com/).
+### Running in the background
-## AWS Batch specific parameters
-Running the pipeline on AWS Batch requires a couple of specific parameters to be set according to your AWS Batch configuration. Please use the `-awsbatch` profile and then specify all of the following parameters.
-### `--awsqueue`
-The JobQueue that you intend to use on AWS Batch.
-### `--awsregion`
-The AWS region to run your job in. Default is set to `eu-west-1` but can be adjusted to your needs.
+Nextflow handles job submissions and supervises the running jobs. The Nextflow process must run until the pipeline is finished.
-Please make sure to also set the `-w/--work-dir` and `--outdir` parameters to a S3 storage bucket of your choice - you'll get an error message notifying you if you didn't.
+The Nextflow `-bg` flag launches Nextflow in the background, detached from your terminal so that the workflow does not stop if you log out of your session. The logs are saved to a file.
-## Other command line parameters
+Alternatively, you can use `screen` / `tmux` or similar tool to create a detached session which you can log back into at a later time.
+Some HPC setups also allow you to run nextflow within a cluster job submitted your job scheduler (from where it submits more jobs).
-<!-- TODO nf-core: Describe any other command line flags here -->
+#### Nextflow memory requirements
-### `--outdir`
-The output directory where the results will be saved.
-
-### `--email`
-Set this parameter to your e-mail address to get a summary e-mail with details of the run sent to you when the workflow exits. If set in your user config file (`~/.nextflow/config`) then you don't need to speicfy this on the command line for every run.
-
-### `-name`
-Name for the pipeline run. If not specified, Nextflow will automatically generate a random mnemonic.
-
-This is used in the MultiQC report (if not default) and in the summary HTML / e-mail (always).
-
-**NB:** Single hyphen (core Nextflow option)
-
-### `-resume`
-Specify this when restarting a pipeline. Nextflow will used cached results from any pipeline steps where the inputs are the same, continuing from where it got to previously.
-
-You can also supply a run name to resume a specific run: `-resume [run-name]`. Use the `nextflow log` command to show previous run names.
-
-**NB:** Single hyphen (core Nextflow option)
-
-### `-c`
-Specify the path to a specific config file (this is a core NextFlow command).
-
-**NB:** Single hyphen (core Nextflow option)
-
-Note - you can use this to override pipeline defaults.
-
-### `--custom_config_version`
-Provide git commit id for custom Institutional configs hosted at `nf-core/configs`. This was implemented for reproducibility purposes. Default is set to `master`.
+In some cases, the Nextflow Java virtual machines can start to request a large amount of memory.
+We recommend adding the following line to your environment to limit this (typically in `~/.bashrc` or `~./bash_profile`):
```bash
-## Download and use config file with following git commid id
---custom_config_version d52db660777c4bf36546ddb188ec530c3ada1b96
+NXF_OPTS='-Xms1g -Xmx4g'
```
-
-### `--max_memory`
-Use to set a top-limit for the default memory requirement for each process.
-Should be a string in the format integer-unit. eg. `--max_memory '8.GB'`
-
-### `--max_time`
-Use to set a top-limit for the default time requirement for each process.
-Should be a string in the format integer-unit. eg. `--max_time '2.h'`
-
-### `--max_cpus`
-Use to set a top-limit for the default CPU requirement for each process.
-Should be a string in the format integer-unit. eg. `--max_cpus 1`
-
-### `--plaintext_email`
-Set to receive plain-text e-mails instead of HTML formatted.
-
-### `--multiqc_config`
-Specify a path to a custom MultiQC configuration file.
diff --git a/environment.yml b/environment.yml
index 42c74b4d75896f5a8c24758f2d28e44dbd14b14e..9d3cc672fa52d0b8a8388fbce901768c9e4c1870 100644
--- a/environment.yml
+++ b/environment.yml
@@ -1,9 +1,15 @@
+# You can use this file to create a conda environment for this pipeline:
+# conda env create -f environment.yml
name: nf-core-hic-1.0dev
channels:
- conda-forge
- bioconda
- defaults
dependencies:
+ - conda-forge::python=3.7.3
+ - conda-forge::markdown=3.1.1
+ - conda-forge::pymdown-extensions=6.0
+ - conda-forge::pygments=2.5.2
# TODO nf-core: Add required software dependencies here
- - fastqc=0.11.8
- - multiqc=1.6
+ - bioconda::fastqc=0.11.8
+ - bioconda::multiqc=1.7
diff --git a/main.nf b/main.nf
index 97a07deedf81033e2d56bd4d66631446cecbe1cd..c481dd57affdc947c3024e974caa455cafa54307 100644
--- a/main.nf
+++ b/main.nf
@@ -9,164 +9,159 @@
----------------------------------------------------------------------------------------
*/
-
def helpMessage() {
// TODO nf-core: Add to this help message with new command line parameters
+ log.info nfcoreHeader()
log.info"""
- =======================================================
- ,--./,-.
- ___ __ __ __ ___ /,-._.--~\'
- |\\ | |__ __ / ` / \\ |__) |__ } {
- | \\| | \\__, \\__/ | \\ |___ \\`-._,-`-,
- `._,._,\'
-
- nf-core/hic v${workflow.manifest.version}
- =======================================================
Usage:
The typical command for running the pipeline is as follows:
- nextflow run nf-core/hic --reads '*_R{1,2}.fastq.gz' -profile docker
+ nextflow run nf-core/hic --input '*_R{1,2}.fastq.gz' -profile docker
Mandatory arguments:
- --reads Path to input data (must be surrounded with quotes)
- --genome Name of iGenomes reference
- -profile Configuration profile to use. Can use multiple (comma separated)
- Available: conda, docker, singularity, awsbatch, test and more.
+ --input [file] Path to input data (must be surrounded with quotes)
+ -profile [str] Configuration profile to use. Can use multiple (comma separated)
+ Available: conda, docker, singularity, test, awsbatch, <institute> and more
Options:
- --singleEnd Specifies that the input is single end reads
+ --genome [str] Name of iGenomes reference
+ --single_end [bool] Specifies that the input is single-end reads
- References If not specified in the configuration file or you wish to overwrite any of the references.
- --fasta Path to Fasta reference
+ References If not specified in the configuration file or you wish to overwrite any of the references
+ --fasta [file] Path to fasta reference
Other options:
- --outdir The output directory where the results will be saved
- --email Set this parameter to your e-mail address to get a summary e-mail with details of the run sent to you when the workflow exits
- -name Name for the pipeline run. If not specified, Nextflow will automatically generate a random mnemonic.
+ --outdir [file] The output directory where the results will be saved
+ --publish_dir_mode [str] Mode for publishing results in the output directory. Available: symlink, rellink, link, copy, copyNoFollow, move (Default: copy)
+ --email [email] Set this parameter to your e-mail address to get a summary e-mail with details of the run sent to you when the workflow exits
+ --email_on_fail [email] Same as --email, except only send mail if the workflow is not successful
+ --max_multiqc_email_size [str] Threshold size for MultiQC report to be attached in notification email. If file generated by pipeline exceeds the threshold, it will not be attached (Default: 25MB)
+ -name [str] Name for the pipeline run. If not specified, Nextflow will automatically generate a random mnemonic
AWSBatch options:
- --awsqueue The AWSBatch JobQueue that needs to be set when running on AWSBatch
- --awsregion The AWS Region for your AWS Batch job to run on
+ --awsqueue [str] The AWSBatch JobQueue that needs to be set when running on AWSBatch
+ --awsregion [str] The AWS Region for your AWS Batch job to run on
+ --awscli [str] Path to the AWS CLI tool
""".stripIndent()
}
+// Show help message
+if (params.help) {
+ helpMessage()
+ exit 0
+}
+
/*
* SET UP CONFIGURATION VARIABLES
*/
-// Show help emssage
-if (params.help){
- helpMessage()
- exit 0
+// Check if genome exists in the config file
+if (params.genomes && params.genome && !params.genomes.containsKey(params.genome)) {
+ exit 1, "The provided genome '${params.genome}' is not available in the iGenomes file. Currently the available genomes are ${params.genomes.keySet().join(", ")}"
}
// TODO nf-core: Add any reference files that are needed
// Configurable reference genomes
-fasta = params.genome ? params.genomes[ params.genome ].fasta ?: false : false
-if ( params.fasta ){
- fasta = file(params.fasta)
- if( !fasta.exists() ) exit 1, "Fasta file not found: ${params.fasta}"
-}
//
// NOTE - THIS IS NOT USED IN THIS PIPELINE, EXAMPLE ONLY
-// If you want to use the above in a process, define the following:
+// If you want to use the channel below in a process, define the following:
// input:
-// file fasta from fasta
+// file fasta from ch_fasta
//
-
+params.fasta = params.genome ? params.genomes[ params.genome ].fasta ?: false : false
+if (params.fasta) { ch_fasta = file(params.fasta, checkIfExists: true) }
// Has the run name been specified by the user?
-// this has the bonus effect of catching both -name and --name
+// this has the bonus effect of catching both -name and --name
custom_runName = params.name
-if( !(workflow.runName ==~ /[a-z]+_[a-z]+/) ){
- custom_runName = workflow.runName
+if (!(workflow.runName ==~ /[a-z]+_[a-z]+/)) {
+ custom_runName = workflow.runName
}
-
-if( workflow.profile == 'awsbatch') {
- // AWSBatch sanity checking
- if (!params.awsqueue || !params.awsregion) exit 1, "Specify correct --awsqueue and --awsregion parameters on AWSBatch!"
- if (!workflow.workDir.startsWith('s3') || !params.outdir.startsWith('s3')) exit 1, "Specify S3 URLs for workDir and outdir parameters on AWSBatch!"
- // Check workDir/outdir paths to be S3 buckets if running on AWSBatch
- // related: https://github.com/nextflow-io/nextflow/issues/813
- if (!workflow.workDir.startsWith('s3:') || !params.outdir.startsWith('s3:')) exit 1, "Workdir or Outdir not on S3 - specify S3 Buckets for each to run on AWSBatch!"
+// Check AWS batch settings
+if (workflow.profile.contains('awsbatch')) {
+ // AWSBatch sanity checking
+ if (!params.awsqueue || !params.awsregion) exit 1, "Specify correct --awsqueue and --awsregion parameters on AWSBatch!"
+ // Check outdir paths to be S3 buckets if running on AWSBatch
+ // related: https://github.com/nextflow-io/nextflow/issues/813
+ if (!params.outdir.startsWith('s3:')) exit 1, "Outdir not on S3 - specify S3 Bucket to run on AWSBatch!"
+ // Prevent trace files to be stored on S3 since S3 does not support rolling files.
+ if (params.tracedir.startsWith('s3:')) exit 1, "Specify a local tracedir or run without trace! S3 cannot be used for tracefiles."
}
// Stage config files
-ch_multiqc_config = Channel.fromPath(params.multiqc_config)
-ch_output_docs = Channel.fromPath("$baseDir/docs/output.md")
+ch_multiqc_config = file("$projectDir/assets/multiqc_config.yaml", checkIfExists: true)
+ch_multiqc_custom_config = params.multiqc_config ? Channel.fromPath(params.multiqc_config, checkIfExists: true) : Channel.empty()
+ch_output_docs = file("$projectDir/docs/output.md", checkIfExists: true)
+ch_output_docs_images = file("$projectDir/docs/images/", checkIfExists: true)
/*
* Create a channel for input read files
*/
- if(params.readPaths){
- if(params.singleEnd){
- Channel
- .from(params.readPaths)
- .map { row -> [ row[0], [file(row[1][0])]] }
- .ifEmpty { exit 1, "params.readPaths was empty - no input files supplied" }
- .into { read_files_fastqc; read_files_trimming }
- } else {
- Channel
- .from(params.readPaths)
- .map { row -> [ row[0], [file(row[1][0]), file(row[1][1])]] }
- .ifEmpty { exit 1, "params.readPaths was empty - no input files supplied" }
- .into { read_files_fastqc; read_files_trimming }
- }
- } else {
- Channel
- .fromFilePairs( params.reads, size: params.singleEnd ? 1 : 2 )
- .ifEmpty { exit 1, "Cannot find any reads matching: ${params.reads}\nNB: Path needs to be enclosed in quotes!\nIf this is single-end data, please specify --singleEnd on the command line." }
- .into { read_files_fastqc; read_files_trimming }
- }
-
+if (params.input_paths) {
+ if (params.single_end) {
+ Channel
+ .from(params.input_paths)
+ .map { row -> [ row[0], [ file(row[1][0], checkIfExists: true) ] ] }
+ .ifEmpty { exit 1, "params.input_paths was empty - no input files supplied" }
+ .into { ch_read_files_fastqc; ch_read_files_trimming }
+ } else {
+ Channel
+ .from(params.input_paths)
+ .map { row -> [ row[0], [ file(row[1][0], checkIfExists: true), file(row[1][1], checkIfExists: true) ] ] }
+ .ifEmpty { exit 1, "params.input_paths was empty - no input files supplied" }
+ .into { ch_read_files_fastqc; ch_read_files_trimming }
+ }
+} else {
+ Channel
+ .fromFilePairs(params.input, size: params.single_end ? 1 : 2)
+ .ifEmpty { exit 1, "Cannot find any reads matching: ${params.input}\nNB: Path needs to be enclosed in quotes!\nIf this is single-end data, please specify --single_end on the command line." }
+ .into { ch_read_files_fastqc; ch_read_files_trimming }
+}
// Header log info
-log.info """=======================================================
- ,--./,-.
- ___ __ __ __ ___ /,-._.--~\'
- |\\ | |__ __ / ` / \\ |__) |__ } {
- | \\| | \\__, \\__/ | \\ |___ \\`-._,-`-,
- `._,._,\'
-
-nf-core/hic v${workflow.manifest.version}"
-======================================================="""
+log.info nfcoreHeader()
def summary = [:]
-summary['Pipeline Name'] = 'nf-core/hic'
-summary['Pipeline Version'] = workflow.manifest.version
-summary['Run Name'] = custom_runName ?: workflow.runName
+if (workflow.revision) summary['Pipeline Release'] = workflow.revision
+summary['Run Name'] = custom_runName ?: workflow.runName
// TODO nf-core: Report custom parameters here
-summary['Reads'] = params.reads
-summary['Fasta Ref'] = params.fasta
-summary['Data Type'] = params.singleEnd ? 'Single-End' : 'Paired-End'
-summary['Max Memory'] = params.max_memory
-summary['Max CPUs'] = params.max_cpus
-summary['Max Time'] = params.max_time
-summary['Output dir'] = params.outdir
-summary['Working dir'] = workflow.workDir
-summary['Container Engine'] = workflow.containerEngine
-if(workflow.containerEngine) summary['Container'] = workflow.container
-summary['Current home'] = "$HOME"
-summary['Current user'] = "$USER"
-summary['Current path'] = "$PWD"
-summary['Working dir'] = workflow.workDir
-summary['Output dir'] = params.outdir
-summary['Script dir'] = workflow.projectDir
+summary['Input'] = params.input
+summary['Fasta Ref'] = params.fasta
+summary['Data Type'] = params.single_end ? 'Single-End' : 'Paired-End'
+summary['Max Resources'] = "$params.max_memory memory, $params.max_cpus cpus, $params.max_time time per job"
+if (workflow.containerEngine) summary['Container'] = "$workflow.containerEngine - $workflow.container"
+summary['Output dir'] = params.outdir
+summary['Launch dir'] = workflow.launchDir
+summary['Working dir'] = workflow.workDir
+summary['Script dir'] = workflow.projectDir
+summary['User'] = workflow.userName
+if (workflow.profile.contains('awsbatch')) {
+ summary['AWS Region'] = params.awsregion
+ summary['AWS Queue'] = params.awsqueue
+ summary['AWS CLI'] = params.awscli
+}
summary['Config Profile'] = workflow.profile
-if(workflow.profile == 'awsbatch'){
- summary['AWS Region'] = params.awsregion
- summary['AWS Queue'] = params.awsqueue
+if (params.config_profile_description) summary['Config Profile Description'] = params.config_profile_description
+if (params.config_profile_contact) summary['Config Profile Contact'] = params.config_profile_contact
+if (params.config_profile_url) summary['Config Profile URL'] = params.config_profile_url
+summary['Config Files'] = workflow.configFiles.join(', ')
+if (params.email || params.email_on_fail) {
+ summary['E-mail Address'] = params.email
+ summary['E-mail on failure'] = params.email_on_fail
+ summary['MultiQC maxsize'] = params.max_multiqc_email_size
}
-if(params.email) summary['E-mail Address'] = params.email
-log.info summary.collect { k,v -> "${k.padRight(15)}: $v" }.join("\n")
-log.info "========================================="
+log.info summary.collect { k,v -> "${k.padRight(18)}: $v" }.join("\n")
+log.info "-\033[2m--------------------------------------------------\033[0m-"
+// Check the hostnames against configured profiles
+checkHostname()
-def create_workflow_summary(summary) {
- def yaml_file = workDir.resolve('workflow_summary_mqc.yaml')
- yaml_file.text = """
+Channel.from(summary.collect{ [it.key, it.value] })
+ .map { k,v -> "<dt>$k</dt><dd><samp>${v ?: '<span style=\"color:#999999;\">N/A</a>'}</samp></dd>" }
+ .reduce { a, b -> return [a, b].join("\n ") }
+ .map { x -> """
id: 'nf-core-hic-summary'
description: " - this information is collected when the pipeline is started."
section_name: 'nf-core/hic Workflow Summary'
@@ -174,21 +169,24 @@ def create_workflow_summary(summary) {
plot_type: 'html'
data: |
<dl class=\"dl-horizontal\">
-${summary.collect { k,v -> " <dt>$k</dt><dd><samp>${v ?: '<span style=\"color:#999999;\">N/A</a>'}</samp></dd>" }.join("\n")}
+ $x
</dl>
- """.stripIndent()
-
- return yaml_file
-}
-
+ """.stripIndent() }
+ .set { ch_workflow_summary }
/*
* Parse software version numbers
*/
process get_software_versions {
+ publishDir "${params.outdir}/pipeline_info", mode: params.publish_dir_mode,
+ saveAs: { filename ->
+ if (filename.indexOf(".csv") > 0) filename
+ else null
+ }
output:
- file 'software_versions_mqc.yaml' into software_versions_yaml
+ file 'software_versions_mqc.yaml' into ch_software_versions_yaml
+ file "software_versions.csv"
script:
// TODO nf-core: Get all tools to print their version number here
@@ -197,82 +195,81 @@ process get_software_versions {
echo $workflow.nextflow.version > v_nextflow.txt
fastqc --version > v_fastqc.txt
multiqc --version > v_multiqc.txt
- scrape_software_versions.py > software_versions_mqc.yaml
+ scrape_software_versions.py &> software_versions_mqc.yaml
"""
}
-
-
/*
* STEP 1 - FastQC
*/
process fastqc {
tag "$name"
- publishDir "${params.outdir}/fastqc", mode: 'copy',
- saveAs: {filename -> filename.indexOf(".zip") > 0 ? "zips/$filename" : "$filename"}
+ label 'process_medium'
+ publishDir "${params.outdir}/fastqc", mode: params.publish_dir_mode,
+ saveAs: { filename ->
+ filename.indexOf(".zip") > 0 ? "zips/$filename" : "$filename"
+ }
input:
- set val(name), file(reads) from read_files_fastqc
+ set val(name), file(reads) from ch_read_files_fastqc
output:
- file "*_fastqc.{zip,html}" into fastqc_results
+ file "*_fastqc.{zip,html}" into ch_fastqc_results
script:
"""
- fastqc -q $reads
+ fastqc --quiet --threads $task.cpus $reads
"""
}
-
-
/*
* STEP 2 - MultiQC
*/
process multiqc {
- publishDir "${params.outdir}/MultiQC", mode: 'copy'
+ publishDir "${params.outdir}/MultiQC", mode: params.publish_dir_mode
input:
- file multiqc_config from ch_multiqc_config
+ file (multiqc_config) from ch_multiqc_config
+ file (mqc_custom_config) from ch_multiqc_custom_config.collect().ifEmpty([])
// TODO nf-core: Add in log files from your new processes for MultiQC to find!
- file ('fastqc/*') from fastqc_results.collect().ifEmpty([])
- file ('software_versions/*') from software_versions_yaml
- file workflow_summary from create_workflow_summary(summary)
+ file ('fastqc/*') from ch_fastqc_results.collect().ifEmpty([])
+ file ('software_versions/*') from ch_software_versions_yaml.collect()
+ file workflow_summary from ch_workflow_summary.collectFile(name: "workflow_summary_mqc.yaml")
output:
- file "*multiqc_report.html" into multiqc_report
+ file "*multiqc_report.html" into ch_multiqc_report
file "*_data"
+ file "multiqc_plots"
script:
rtitle = custom_runName ? "--title \"$custom_runName\"" : ''
rfilename = custom_runName ? "--filename " + custom_runName.replaceAll('\\W','_').replaceAll('_+','_') + "_multiqc_report" : ''
+ custom_config_file = params.multiqc_config ? "--config $mqc_custom_config" : ''
// TODO nf-core: Specify which MultiQC modules to use with -m for a faster run time
"""
- multiqc -f $rtitle $rfilename --config $multiqc_config .
+ multiqc -f $rtitle $rfilename $custom_config_file .
"""
}
-
-
/*
* STEP 3 - Output Description HTML
*/
process output_documentation {
- publishDir "${params.outdir}/Documentation", mode: 'copy'
+ publishDir "${params.outdir}/pipeline_info", mode: params.publish_dir_mode
input:
file output_docs from ch_output_docs
+ file images from ch_output_docs_images
output:
file "results_description.html"
script:
"""
- markdown_to_html.r $output_docs results_description.html
+ markdown_to_html.py $output_docs -o results_description.html
"""
}
-
-
/*
* Completion e-mail notification
*/
@@ -280,8 +277,8 @@ workflow.onComplete {
// Set up the e-mail variables
def subject = "[nf-core/hic] Successful: $workflow.runName"
- if(!workflow.success){
- subject = "[nf-core/hic] FAILED: $workflow.runName"
+ if (!workflow.success) {
+ subject = "[nf-core/hic] FAILED: $workflow.runName"
}
def email_fields = [:]
email_fields['version'] = workflow.manifest.version
@@ -299,54 +296,140 @@ workflow.onComplete {
email_fields['summary']['Date Completed'] = workflow.complete
email_fields['summary']['Pipeline script file path'] = workflow.scriptFile
email_fields['summary']['Pipeline script hash ID'] = workflow.scriptId
- if(workflow.repository) email_fields['summary']['Pipeline repository Git URL'] = workflow.repository
- if(workflow.commitId) email_fields['summary']['Pipeline repository Git Commit'] = workflow.commitId
- if(workflow.revision) email_fields['summary']['Pipeline Git branch/tag'] = workflow.revision
+ if (workflow.repository) email_fields['summary']['Pipeline repository Git URL'] = workflow.repository
+ if (workflow.commitId) email_fields['summary']['Pipeline repository Git Commit'] = workflow.commitId
+ if (workflow.revision) email_fields['summary']['Pipeline Git branch/tag'] = workflow.revision
email_fields['summary']['Nextflow Version'] = workflow.nextflow.version
email_fields['summary']['Nextflow Build'] = workflow.nextflow.build
email_fields['summary']['Nextflow Compile Timestamp'] = workflow.nextflow.timestamp
+ // TODO nf-core: If not using MultiQC, strip out this code (including params.max_multiqc_email_size)
+ // On success try attach the multiqc report
+ def mqc_report = null
+ try {
+ if (workflow.success) {
+ mqc_report = ch_multiqc_report.getVal()
+ if (mqc_report.getClass() == ArrayList) {
+ log.warn "[nf-core/hic] Found multiple reports from process 'multiqc', will use only one"
+ mqc_report = mqc_report[0]
+ }
+ }
+ } catch (all) {
+ log.warn "[nf-core/hic] Could not attach MultiQC report to summary email"
+ }
+
+ // Check if we are only sending emails on failure
+ email_address = params.email
+ if (!params.email && params.email_on_fail && !workflow.success) {
+ email_address = params.email_on_fail
+ }
+
// Render the TXT template
def engine = new groovy.text.GStringTemplateEngine()
- def tf = new File("$baseDir/assets/email_template.txt")
+ def tf = new File("$projectDir/assets/email_template.txt")
def txt_template = engine.createTemplate(tf).make(email_fields)
def email_txt = txt_template.toString()
// Render the HTML template
- def hf = new File("$baseDir/assets/email_template.html")
+ def hf = new File("$projectDir/assets/email_template.html")
def html_template = engine.createTemplate(hf).make(email_fields)
def email_html = html_template.toString()
// Render the sendmail template
- def smail_fields = [ email: params.email, subject: subject, email_txt: email_txt, email_html: email_html, baseDir: "$baseDir" ]
- def sf = new File("$baseDir/assets/sendmail_template.txt")
+ def smail_fields = [ email: email_address, subject: subject, email_txt: email_txt, email_html: email_html, projectDir: "$projectDir", mqcFile: mqc_report, mqcMaxSize: params.max_multiqc_email_size.toBytes() ]
+ def sf = new File("$projectDir/assets/sendmail_template.txt")
def sendmail_template = engine.createTemplate(sf).make(smail_fields)
def sendmail_html = sendmail_template.toString()
// Send the HTML e-mail
- if (params.email) {
+ if (email_address) {
try {
- if( params.plaintext_email ){ throw GroovyException('Send plaintext e-mail, not HTML') }
- // Try to send HTML e-mail using sendmail
- [ 'sendmail', '-t' ].execute() << sendmail_html
- log.info "[nf-core/hic] Sent summary e-mail to $params.email (sendmail)"
+ if (params.plaintext_email) { throw GroovyException('Send plaintext e-mail, not HTML') }
+ // Try to send HTML e-mail using sendmail
+ [ 'sendmail', '-t' ].execute() << sendmail_html
+ log.info "[nf-core/hic] Sent summary e-mail to $email_address (sendmail)"
} catch (all) {
- // Catch failures and try with plaintext
- [ 'mail', '-s', subject, params.email ].execute() << email_txt
- log.info "[nf-core/hic] Sent summary e-mail to $params.email (mail)"
+ // Catch failures and try with plaintext
+ def mail_cmd = [ 'mail', '-s', subject, '--content-type=text/html', email_address ]
+ if ( mqc_report.size() <= params.max_multiqc_email_size.toBytes() ) {
+ mail_cmd += [ '-A', mqc_report ]
+ }
+ mail_cmd.execute() << email_html
+ log.info "[nf-core/hic] Sent summary e-mail to $email_address (mail)"
}
}
// Write summary e-mail HTML to a file
- def output_d = new File( "${params.outdir}/Documentation/" )
- if( !output_d.exists() ) {
- output_d.mkdirs()
+ def output_d = new File("${params.outdir}/pipeline_info/")
+ if (!output_d.exists()) {
+ output_d.mkdirs()
}
- def output_hf = new File( output_d, "pipeline_report.html" )
+ def output_hf = new File(output_d, "pipeline_report.html")
output_hf.withWriter { w -> w << email_html }
- def output_tf = new File( output_d, "pipeline_report.txt" )
+ def output_tf = new File(output_d, "pipeline_report.txt")
output_tf.withWriter { w -> w << email_txt }
- log.info "[nf-core/hic] Pipeline Complete"
+ c_green = params.monochrome_logs ? '' : "\033[0;32m";
+ c_purple = params.monochrome_logs ? '' : "\033[0;35m";
+ c_red = params.monochrome_logs ? '' : "\033[0;31m";
+ c_reset = params.monochrome_logs ? '' : "\033[0m";
+
+ if (workflow.stats.ignoredCount > 0 && workflow.success) {
+ log.info "-${c_purple}Warning, pipeline completed, but with errored process(es) ${c_reset}-"
+ log.info "-${c_red}Number of ignored errored process(es) : ${workflow.stats.ignoredCount} ${c_reset}-"
+ log.info "-${c_green}Number of successfully ran process(es) : ${workflow.stats.succeedCount} ${c_reset}-"
+ }
+
+ if (workflow.success) {
+ log.info "-${c_purple}[nf-core/hic]${c_green} Pipeline completed successfully${c_reset}-"
+ } else {
+ checkHostname()
+ log.info "-${c_purple}[nf-core/hic]${c_red} Pipeline completed with errors${c_reset}-"
+ }
+
+}
+
+def nfcoreHeader() {
+ // Log colors ANSI codes
+ c_black = params.monochrome_logs ? '' : "\033[0;30m";
+ c_blue = params.monochrome_logs ? '' : "\033[0;34m";
+ c_cyan = params.monochrome_logs ? '' : "\033[0;36m";
+ c_dim = params.monochrome_logs ? '' : "\033[2m";
+ c_green = params.monochrome_logs ? '' : "\033[0;32m";
+ c_purple = params.monochrome_logs ? '' : "\033[0;35m";
+ c_reset = params.monochrome_logs ? '' : "\033[0m";
+ c_white = params.monochrome_logs ? '' : "\033[0;37m";
+ c_yellow = params.monochrome_logs ? '' : "\033[0;33m";
+
+ return """ -${c_dim}--------------------------------------------------${c_reset}-
+ ${c_green},--.${c_black}/${c_green},-.${c_reset}
+ ${c_blue} ___ __ __ __ ___ ${c_green}/,-._.--~\'${c_reset}
+ ${c_blue} |\\ | |__ __ / ` / \\ |__) |__ ${c_yellow}} {${c_reset}
+ ${c_blue} | \\| | \\__, \\__/ | \\ |___ ${c_green}\\`-._,-`-,${c_reset}
+ ${c_green}`._,._,\'${c_reset}
+ ${c_purple} nf-core/hic v${workflow.manifest.version}${c_reset}
+ -${c_dim}--------------------------------------------------${c_reset}-
+ """.stripIndent()
+}
+
+def checkHostname() {
+ def c_reset = params.monochrome_logs ? '' : "\033[0m"
+ def c_white = params.monochrome_logs ? '' : "\033[0;37m"
+ def c_red = params.monochrome_logs ? '' : "\033[1;91m"
+ def c_yellow_bold = params.monochrome_logs ? '' : "\033[1;93m"
+ if (params.hostnames) {
+ def hostname = "hostname".execute().text.trim()
+ params.hostnames.each { prof, hnames ->
+ hnames.each { hname ->
+ if (hostname.contains(hname) && !workflow.profile.contains(prof)) {
+ log.error "====================================================\n" +
+ " ${c_red}WARNING!${c_reset} You are running with `-profile $workflow.profile`\n" +
+ " but your machine hostname is ${c_white}'$hostname'${c_reset}\n" +
+ " ${c_yellow_bold}It's highly recommended that you use `-profile $prof${c_reset}`\n" +
+ "============================================================"
+ }
+ }
+ }
+ }
}
diff --git a/nextflow.config b/nextflow.config
index 5f363c6caace1d84440605b20a729f09a6dd0d29..44cd92c0ab9b26a7fe1f73ffadee0b170710dc14 100644
--- a/nextflow.config
+++ b/nextflow.config
@@ -3,83 +3,111 @@
* nf-core/hic Nextflow config file
* -------------------------------------------------
* Default config options for all environments.
- * Cluster-specific config options should be saved
- * in the conf folder and imported under a profile
- * name here.
*/
// Global default params, used in configs
params {
- // Container slug. Stable releases should specify release tag!
- // Developmental code should specify :latest
- container = 'nfcore/hic:latest'
-
// Workflow flags
// TODO nf-core: Specify your pipeline's command line flags
- reads = "data/*{1,2}.fastq.gz"
- singleEnd = false
+ genome = false
+ input = "data/*{1,2}.fastq.gz"
+ single_end = false
outdir = './results'
+ publish_dir_mode = 'copy'
// Boilerplate options
name = false
- multiqc_config = "$baseDir/conf/multiqc_config.yaml"
+ multiqc_config = false
email = false
+ email_on_fail = false
+ max_multiqc_email_size = 25.MB
plaintext_email = false
+ monochrome_logs = false
help = false
- igenomes_base = "./iGenomes"
+ igenomes_base = 's3://ngi-igenomes/igenomes/'
tracedir = "${params.outdir}/pipeline_info"
- clusterOptions = false
- awsqueue = false
- awsregion = 'eu-west-1'
- igenomesIgnore = false
+ igenomes_ignore = false
custom_config_version = 'master'
+ custom_config_base = "https://raw.githubusercontent.com/nf-core/configs/${params.custom_config_version}"
+ hostnames = false
+ config_profile_description = false
+ config_profile_contact = false
+ config_profile_url = false
+
+ // Defaults only, expecting to be overwritten
+ max_memory = 128.GB
+ max_cpus = 16
+ max_time = 240.h
+
}
+// Container slug. Stable releases should specify release tag!
+// Developmental code should specify :dev
+process.container = 'nfcore/hic:dev'
+
// Load base.config by default for all pipelines
includeConfig 'conf/base.config'
// Load nf-core custom profiles from different Institutions
-includeConfig "https://raw.githubusercontent.com/nf-core/configs/${params.custom_config_version}/nfcore_custom.config"
+try {
+ includeConfig "${params.custom_config_base}/nfcore_custom.config"
+} catch (Exception e) {
+ System.err.println("WARNING: Could not load nf-core/config profiles: ${params.custom_config_base}/nfcore_custom.config")
+}
profiles {
- awsbatch { includeConfig 'conf/awsbatch.config' }
- conda { process.conda = "$baseDir/environment.yml" }
+ conda { process.conda = "$projectDir/environment.yml" }
debug { process.beforeScript = 'echo $HOSTNAME' }
docker {
docker.enabled = true
- process.container = params.container
+ // Avoid this error:
+ // WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
+ // Testing this in nf-core after discussion here https://github.com/nf-core/tools/pull/351
+ // once this is established and works well, nextflow might implement this behavior as new default.
+ docker.runOptions = '-u \$(id -u):\$(id -g)'
}
singularity {
singularity.enabled = true
- process.container = {"shub://${params.container.replace('nfcore', 'nf-core')}"}
+ singularity.autoMounts = true
+ }
+ podman {
+ podman.enabled = true
}
test { includeConfig 'conf/test.config' }
+ test_full { includeConfig 'conf/test_full.config' }
}
// Load igenomes.config if required
-if(!params.igenomesIgnore){
+if (!params.igenomes_ignore) {
includeConfig 'conf/igenomes.config'
}
+// Export these variables to prevent local Python/R libraries from conflicting with those in the container
+env {
+ PYTHONNOUSERSITE = 1
+ R_PROFILE_USER = "/.Rprofile"
+ R_ENVIRON_USER = "/.Renviron"
+}
+
// Capture exit codes from upstream processes when piping
process.shell = ['/bin/bash', '-euo', 'pipefail']
timeline {
enabled = true
- file = "${params.tracedir}/nf-core/hic_timeline.html"
+ file = "${params.tracedir}/execution_timeline.html"
}
report {
enabled = true
- file = "${params.tracedir}/nf-core/hic_report.html"
+ file = "${params.tracedir}/execution_report.html"
}
trace {
enabled = true
- file = "${params.tracedir}/nf-core/hic_trace.txt"
+ file = "${params.tracedir}/execution_trace.txt"
}
dag {
enabled = true
- file = "${params.tracedir}/nf-core/hic_dag.svg"
+ file = "${params.tracedir}/pipeline_dag.svg"
}
manifest {
@@ -88,16 +116,16 @@ manifest {
homePage = 'https://github.com/nf-core/hic'
description = 'Analysis of Chromosome Conformation Capture data (Hi-C)'
mainScript = 'main.nf'
- nextflowVersion = '>=0.32.0'
+ nextflowVersion = '>=20.04.0'
version = '1.0dev'
}
// Function to ensure that resource requirements don't go beyond
// a maximum limit
def check_max(obj, type) {
- if(type == 'memory'){
+ if (type == 'memory') {
try {
- if(obj.compareTo(params.max_memory as nextflow.util.MemoryUnit) == 1)
+ if (obj.compareTo(params.max_memory as nextflow.util.MemoryUnit) == 1)
return params.max_memory as nextflow.util.MemoryUnit
else
return obj
@@ -105,9 +133,9 @@ def check_max(obj, type) {
println " ### ERROR ### Max memory '${params.max_memory}' is not valid! Using default value: $obj"
return obj
}
- } else if(type == 'time'){
+ } else if (type == 'time') {
try {
- if(obj.compareTo(params.max_time as nextflow.util.Duration) == 1)
+ if (obj.compareTo(params.max_time as nextflow.util.Duration) == 1)
return params.max_time as nextflow.util.Duration
else
return obj
@@ -115,7 +143,7 @@ def check_max(obj, type) {
println " ### ERROR ### Max time '${params.max_time}' is not valid! Using default value: $obj"
return obj
}
- } else if(type == 'cpus'){
+ } else if (type == 'cpus') {
try {
return Math.min( obj, params.max_cpus as int )
} catch (all) {
diff --git a/nextflow_schema.json b/nextflow_schema.json
new file mode 100644
index 0000000000000000000000000000000000000000..21e29d7e5fc226e377bef6fa7e4d9b498d281eed
--- /dev/null
+++ b/nextflow_schema.json
@@ -0,0 +1,259 @@
+{
+ "$schema": "http://json-schema.org/draft-07/schema",
+ "$id": "https://raw.githubusercontent.com/nf-core/hic/master/nextflow_schema.json",
+ "title": "nf-core/hic pipeline parameters",
+ "description": "Analysis of Chromosome Conformation Capture data (Hi-C)",
+ "type": "object",
+ "definitions": {
+ "input_output_options": {
+ "title": "Input/output options",
+ "type": "object",
+ "fa_icon": "fas fa-terminal",
+ "description": "Define where the pipeline should find input data and save output data.",
+ "required": [
+ "input"
+ ],
+ "properties": {
+ "input": {
+ "type": "string",
+ "fa_icon": "fas fa-dna",
+ "description": "Input FastQ files.",
+ "help_text": "Use this to specify the location of your input FastQ files. For example:\n\n```bash\n--input 'path/to/data/sample_*_{1,2}.fastq'\n```\n\nPlease note the following requirements:\n\n1. The path must be enclosed in quotes\n2. The path must have at least one `*` wildcard character\n3. When using the pipeline with paired end data, the path must use `{1,2}` notation to specify read pairs.\n\nIf left unspecified, a default pattern is used: `data/*{1,2}.fastq.gz`"
+ },
+ "single_end": {
+ "type": "boolean",
+ "description": "Specifies that the input is single-end reads.",
+ "fa_icon": "fas fa-align-center",
+ "help_text": "By default, the pipeline expects paired-end data. If you have single-end data, you need to specify `--single_end` on the command line when you launch the pipeline. A normal glob pattern, enclosed in quotation marks, can then be used for `--input`. For example:\n\n```bash\n--single_end --input '*.fastq'\n```\n\nIt is not possible to run a mixture of single-end and paired-end files in one run."
+ },
+ "outdir": {
+ "type": "string",
+ "description": "The output directory where the results will be saved.",
+ "default": "./results",
+ "fa_icon": "fas fa-folder-open"
+ },
+ "email": {
+ "type": "string",
+ "description": "Email address for completion summary.",
+ "fa_icon": "fas fa-envelope",
+ "help_text": "Set this parameter to your e-mail address to get a summary e-mail with details of the run sent to you when the workflow exits. If set in your user config file (`~/.nextflow/config`) then you don't need to specify this on the command line for every run.",
+ "pattern": "^([a-zA-Z0-9_\\-\\.]+)@([a-zA-Z0-9_\\-\\.]+)\\.([a-zA-Z]{2,5})$"
+ }
+ }
+ },
+ "reference_genome_options": {
+ "title": "Reference genome options",
+ "type": "object",
+ "fa_icon": "fas fa-dna",
+ "description": "Options for the reference genome indices used to align reads.",
+ "properties": {
+ "genome": {
+ "type": "string",
+ "description": "Name of iGenomes reference.",
+ "fa_icon": "fas fa-book",
+ "help_text": "If using a reference genome configured in the pipeline using iGenomes, use this parameter to give the ID for the reference. This is then used to build the full paths for all required reference genome files e.g. `--genome GRCh38`.\n\nSee the [nf-core website docs](https://nf-co.re/usage/reference_genomes) for more details."
+ },
+ "fasta": {
+ "type": "string",
+ "fa_icon": "fas fa-font",
+ "description": "Path to FASTA genome file.",
+ "help_text": "If you have no genome reference available, the pipeline can build one using a FASTA file. This requires additional time and resources, so it's better to use a pre-build index if possible."
+ },
+ "igenomes_base": {
+ "type": "string",
+ "description": "Directory / URL base for iGenomes references.",
+ "default": "s3://ngi-igenomes/igenomes/",
+ "fa_icon": "fas fa-cloud-download-alt",
+ "hidden": true
+ },
+ "igenomes_ignore": {
+ "type": "boolean",
+ "description": "Do not load the iGenomes reference config.",
+ "fa_icon": "fas fa-ban",
+ "hidden": true,
+ "help_text": "Do not load `igenomes.config` when running the pipeline. You may choose this option if you observe clashes between custom parameters and those supplied in `igenomes.config`."
+ }
+ }
+ },
+ "generic_options": {
+ "title": "Generic options",
+ "type": "object",
+ "fa_icon": "fas fa-file-import",
+ "description": "Less common options for the pipeline, typically set in a config file.",
+ "help_text": "These options are common to all nf-core pipelines and allow you to customise some of the core preferences for how the pipeline runs.\n\nTypically these options would be set in a Nextflow config file loaded for all pipeline runs, such as `~/.nextflow/config`.",
+ "properties": {
+ "help": {
+ "type": "boolean",
+ "description": "Display help text.",
+ "hidden": true,
+ "fa_icon": "fas fa-question-circle"
+ },
+ "publish_dir_mode": {
+ "type": "string",
+ "default": "copy",
+ "hidden": true,
+ "description": "Method used to save pipeline results to output directory.",
+ "help_text": "The Nextflow `publishDir` option specifies which intermediate files should be saved to the output directory. This option tells the pipeline what method should be used to move these files. See [Nextflow docs](https://www.nextflow.io/docs/latest/process.html#publishdir) for details.",
+ "fa_icon": "fas fa-copy",
+ "enum": [
+ "symlink",
+ "rellink",
+ "link",
+ "copy",
+ "copyNoFollow",
+ "move"
+ ]
+ },
+ "name": {
+ "type": "string",
+ "description": "Workflow name.",
+ "fa_icon": "fas fa-fingerprint",
+ "hidden": true,
+ "help_text": "A custom name for the pipeline run. Unlike the core nextflow `-name` option with one hyphen this parameter can be reused multiple times, for example if using `-resume`. Passed through to steps such as MultiQC and used for things like report filenames and titles."
+ },
+ "email_on_fail": {
+ "type": "string",
+ "description": "Email address for completion summary, only when pipeline fails.",
+ "fa_icon": "fas fa-exclamation-triangle",
+ "pattern": "^([a-zA-Z0-9_\\-\\.]+)@([a-zA-Z0-9_\\-\\.]+)\\.([a-zA-Z]{2,5})$",
+ "hidden": true,
+ "help_text": "This works exactly as with `--email`, except emails are only sent if the workflow is not successful."
+ },
+ "plaintext_email": {
+ "type": "boolean",
+ "description": "Send plain-text email instead of HTML.",
+ "fa_icon": "fas fa-remove-format",
+ "hidden": true,
+ "help_text": "Set to receive plain-text e-mails instead of HTML formatted."
+ },
+ "max_multiqc_email_size": {
+ "type": "string",
+ "description": "File size limit when attaching MultiQC reports to summary emails.",
+ "default": "25.MB",
+ "fa_icon": "fas fa-file-upload",
+ "hidden": true,
+ "help_text": "If file generated by pipeline exceeds the threshold, it will not be attached."
+ },
+ "monochrome_logs": {
+ "type": "boolean",
+ "description": "Do not use coloured log outputs.",
+ "fa_icon": "fas fa-palette",
+ "hidden": true,
+ "help_text": "Set to disable colourful command line output and live life in monochrome."
+ },
+ "multiqc_config": {
+ "type": "string",
+ "description": "Custom config file to supply to MultiQC.",
+ "fa_icon": "fas fa-cog",
+ "hidden": true
+ },
+ "tracedir": {
+ "type": "string",
+ "description": "Directory to keep pipeline Nextflow logs and reports.",
+ "default": "${params.outdir}/pipeline_info",
+ "fa_icon": "fas fa-cogs",
+ "hidden": true
+ }
+ }
+ },
+ "max_job_request_options": {
+ "title": "Max job request options",
+ "type": "object",
+ "fa_icon": "fab fa-acquisitions-incorporated",
+ "description": "Set the top limit for requested resources for any single job.",
+ "help_text": "If you are running on a smaller system, a pipeline step requesting more resources than are available may cause the Nextflow to stop the run with an error. These options allow you to cap the maximum resources requested by any single job so that the pipeline will run on your system.\n\nNote that you can not _increase_ the resources requested by any job using these options. For that you will need your own configuration file. See [the nf-core website](https://nf-co.re/usage/configuration) for details.",
+ "properties": {
+ "max_cpus": {
+ "type": "integer",
+ "description": "Maximum number of CPUs that can be requested for any single job.",
+ "default": 16,
+ "fa_icon": "fas fa-microchip",
+ "hidden": true,
+ "help_text": "Use to set an upper-limit for the CPU requirement for each process. Should be an integer e.g. `--max_cpus 1`"
+ },
+ "max_memory": {
+ "type": "string",
+ "description": "Maximum amount of memory that can be requested for any single job.",
+ "default": "128.GB",
+ "fa_icon": "fas fa-memory",
+ "hidden": true,
+ "help_text": "Use to set an upper-limit for the memory requirement for each process. Should be a string in the format integer-unit e.g. `--max_memory '8.GB'`"
+ },
+ "max_time": {
+ "type": "string",
+ "description": "Maximum amount of time that can be requested for any single job.",
+ "default": "240.h",
+ "fa_icon": "far fa-clock",
+ "hidden": true,
+ "help_text": "Use to set an upper-limit for the time requirement for each process. Should be a string in the format integer-unit e.g. `--max_time '2.h'`"
+ }
+ }
+ },
+ "institutional_config_options": {
+ "title": "Institutional config options",
+ "type": "object",
+ "fa_icon": "fas fa-university",
+ "description": "Parameters used to describe centralised config profiles. These should not be edited.",
+ "help_text": "The centralised nf-core configuration profiles use a handful of pipeline parameters to describe themselves. This information is then printed to the Nextflow log when you run a pipeline. You should not need to change these values when you run a pipeline.",
+ "properties": {
+ "custom_config_version": {
+ "type": "string",
+ "description": "Git commit id for Institutional configs.",
+ "default": "master",
+ "hidden": true,
+ "fa_icon": "fas fa-users-cog",
+ "help_text": "Provide git commit id for custom Institutional configs hosted at `nf-core/configs`. This was implemented for reproducibility purposes. Default: `master`.\n\n```bash\n## Download and use config file with following git commit id\n--custom_config_version d52db660777c4bf36546ddb188ec530c3ada1b96\n```"
+ },
+ "custom_config_base": {
+ "type": "string",
+ "description": "Base directory for Institutional configs.",
+ "default": "https://raw.githubusercontent.com/nf-core/configs/master",
+ "hidden": true,
+ "help_text": "If you're running offline, nextflow will not be able to fetch the institutional config files from the internet. If you don't need them, then this is not a problem. If you do need them, you should download the files from the repo and tell nextflow where to find them with the `custom_config_base` option. For example:\n\n```bash\n## Download and unzip the config files\ncd /path/to/my/configs\nwget https://github.com/nf-core/configs/archive/master.zip\nunzip master.zip\n\n## Run the pipeline\ncd /path/to/my/data\nnextflow run /path/to/pipeline/ --custom_config_base /path/to/my/configs/configs-master/\n```\n\n> Note that the nf-core/tools helper package has a `download` command to download all required pipeline files + singularity containers + institutional configs in one go for you, to make this process easier.",
+ "fa_icon": "fas fa-users-cog"
+ },
+ "hostnames": {
+ "type": "string",
+ "description": "Institutional configs hostname.",
+ "hidden": true,
+ "fa_icon": "fas fa-users-cog"
+ },
+ "config_profile_description": {
+ "type": "string",
+ "description": "Institutional config description.",
+ "hidden": true,
+ "fa_icon": "fas fa-users-cog"
+ },
+ "config_profile_contact": {
+ "type": "string",
+ "description": "Institutional config contact information.",
+ "hidden": true,
+ "fa_icon": "fas fa-users-cog"
+ },
+ "config_profile_url": {
+ "type": "string",
+ "description": "Institutional config URL link.",
+ "hidden": true,
+ "fa_icon": "fas fa-users-cog"
+ }
+ }
+ }
+ },
+ "allOf": [
+ {
+ "$ref": "#/definitions/input_output_options"
+ },
+ {
+ "$ref": "#/definitions/reference_genome_options"
+ },
+ {
+ "$ref": "#/definitions/generic_options"
+ },
+ {
+ "$ref": "#/definitions/max_job_request_options"
+ },
+ {
+ "$ref": "#/definitions/institutional_config_options"
+ }
+ ]
+}