Skip to content
Snippets Groups Projects
Unverified Commit ac74763a authored by Nicolas Servant's avatar Nicolas Servant Committed by GitHub
Browse files

Merge pull request #101 from nf-core/dev

New version 1.3.0
parents 52e5f048 16384675
No related branches found
No related tags found
No related merge requests found
Showing
with 491 additions and 188 deletions
...@@ -3,3 +3,4 @@ version: 1.2 ...@@ -3,3 +3,4 @@ version: 1.2
workflows: workflows:
- subclass: nfl - subclass: nfl
primaryDescriptorPath: /nextflow.config primaryDescriptorPath: /nextflow.config
publish: True
...@@ -9,9 +9,7 @@ Please use the pre-filled template to save time. ...@@ -9,9 +9,7 @@ Please use the pre-filled template to save time.
However, don't be put off by this template - other more general issues and suggestions are welcome! However, don't be put off by this template - other more general issues and suggestions are welcome!
Contributions to the code are even more welcome ;) Contributions to the code are even more welcome ;)
> If you need help using or modifying nf-core/hic then the best place to ask is on the nf-core > If you need help using or modifying nf-core/hic then the best place to ask is on the nf-core Slack [#hic](https://nfcore.slack.com/channels/hic) channel ([join our Slack here](https://nf-co.re/join/slack)).
Slack [#hic](https://nfcore.slack.com/channels/hic) channel ([join our Slack here](https://nf-co.re/join/slack)).
## Contribution workflow ## Contribution workflow
...@@ -20,8 +18,9 @@ If you'd like to write some code for nf-core/hic, the standard workflow is as fo ...@@ -20,8 +18,9 @@ If you'd like to write some code for nf-core/hic, the standard workflow is as fo
1. Check that there isn't already an issue about your idea in the [nf-core/hic issues](https://github.com/nf-core/hic/issues) to avoid duplicating work 1. Check that there isn't already an issue about your idea in the [nf-core/hic issues](https://github.com/nf-core/hic/issues) to avoid duplicating work
* If there isn't one already, please create one so that others know you're working on this * If there isn't one already, please create one so that others know you're working on this
2. [Fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the [nf-core/hic repository](https://github.com/nf-core/hic) to your GitHub account 2. [Fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the [nf-core/hic repository](https://github.com/nf-core/hic) to your GitHub account
3. Make the necessary changes / additions within your forked repository 3. Make the necessary changes / additions within your forked repository following [Pipeline conventions](#pipeline-contribution-conventions)
4. Submit a Pull Request against the `dev` branch and wait for the code to be reviewed and merged 4. Use `nf-core schema build .` and add any new parameters to the pipeline JSON schema (requires [nf-core tools](https://github.com/nf-core/tools) >= 1.10).
5. Submit a Pull Request against the `dev` branch and wait for the code to be reviewed and merged
If you're not used to this workflow with git, you can start with some [docs from GitHub](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests) or even their [excellent `git` resources](https://try.github.io/). If you're not used to this workflow with git, you can start with some [docs from GitHub](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests) or even their [excellent `git` resources](https://try.github.io/).
...@@ -32,14 +31,14 @@ Typically, pull-requests are only fully reviewed when these tests are passing, t ...@@ -32,14 +31,14 @@ Typically, pull-requests are only fully reviewed when these tests are passing, t
There are typically two types of tests that run: There are typically two types of tests that run:
### Lint Tests ### Lint tests
`nf-core` has a [set of guidelines](https://nf-co.re/developers/guidelines) which all pipelines must adhere to. `nf-core` has a [set of guidelines](https://nf-co.re/developers/guidelines) which all pipelines must adhere to.
To enforce these and ensure that all pipelines stay in sync, we have developed a helper tool which runs checks on the pipeline code. This is in the [nf-core/tools repository](https://github.com/nf-core/tools) and once installed can be run locally with the `nf-core lint <pipeline-directory>` command. To enforce these and ensure that all pipelines stay in sync, we have developed a helper tool which runs checks on the pipeline code. This is in the [nf-core/tools repository](https://github.com/nf-core/tools) and once installed can be run locally with the `nf-core lint <pipeline-directory>` command.
If any failures or warnings are encountered, please follow the listed URL for more documentation. If any failures or warnings are encountered, please follow the listed URL for more documentation.
### Pipeline Tests ### Pipeline tests
Each `nf-core` pipeline should be set up with a minimal set of test-data. Each `nf-core` pipeline should be set up with a minimal set of test-data.
`GitHub Actions` then runs the pipeline on this data to ensure that it exits successfully. `GitHub Actions` then runs the pipeline on this data to ensure that it exits successfully.
...@@ -55,8 +54,75 @@ These tests are run both with the latest available version of `Nextflow` and als ...@@ -55,8 +54,75 @@ These tests are run both with the latest available version of `Nextflow` and als
* A PR should be made on `master` from patch to directly this particular bug. * A PR should be made on `master` from patch to directly this particular bug.
## Getting help ## Getting help
For further information/help, please consult the [nf-core/hic documentation](https://nf-co.re/nf-core/hic/docs) and
don't hesitate to get in touch on the nf-core Slack [#hic](https://nfcore.slack.com/channels/hic) channel
([join our Slack here](https://nf-co.re/join/slack)).
For further information/help, please consult the [nf-core/hic documentation](https://nf-co.re/hic/docs) and don't hesitate to get in touch on the nf-core Slack [#hic](https://nfcore.slack.com/channels/hic) channel ([join our Slack here](https://nf-co.re/join/slack)). For further information/help, please consult the [nf-core/hic documentation](https://nf-co.re/hic/usage) and don't hesitate to get in touch on the nf-core Slack [#hic](https://nfcore.slack.com/channels/hic) channel ([join our Slack here](https://nf-co.re/join/slack)).
## Pipeline contribution conventions
To make the nf-core/hic code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.
### Adding a new step
If you wish to contribute a new step, please use the following coding standards:
1. Define the corresponding input channel into your new process from the expected previous process channel
2. Write the process block (see below).
3. Define the output channel if needed (see below).
4. Add any new flags/options to `nextflow.config` with a default (see below).
5. Add any new flags/options to `nextflow_schema.json` with help text (with `nf-core schema build .`).
6. Add any new flags/options to the help message (for integer/text parameters, print to help the corresponding `nextflow.config` parameter).
7. Add sanity checks for all relevant parameters.
8. Add any new software to the `scrape_software_versions.py` script in `bin/` and the version command to the `scrape_software_versions` process in `main.nf`.
9. Do local tests that the new code works properly and as expected.
10. Add a new test command in `.github/workflow/ci.yaml`.
11. If applicable add a [MultiQC](https://https://multiqc.info/) module.
12. Update MultiQC config `assets/multiqc_config.yaml` so relevant suffixes, name clean up, General Statistics Table column order, and module figures are in the right order.
13. Optional: Add any descriptions of MultiQC report sections and output files to `docs/output.md`.
### Default values
Parameters should be initialised / defined with default values in `nextflow.config` under the `params` scope.
Once there, use `nf-core schema build .` to add to `nextflow_schema.json`.
### Default processes resource requirements
Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.
The process resources can be passed on to the tool dynamically within the process with the `${task.cpu}` and `${task.memory}` variables in the `script:` block.
### Naming schemes
Please use the following naming schemes, to make it easy to understand what is going where.
* initial process channel: `ch_output_from_<process>`
* intermediate and terminal channels: `ch_<previousprocess>_for_<nextprocess>`
### Nextflow version bumping
If you are using a new feature from core Nextflow, you may bump the minimum required version of nextflow in the pipeline with: `nf-core bump-version --nextflow . [min-nf-version]`
### Software version reporting
If you add a new tool to the pipeline, please ensure you add the information of the tool to the `get_software_version` process.
Add to the script block of the process, something like the following:
```bash
<YOUR_TOOL> --version &> v_<YOUR_TOOL>.txt 2>&1 || true
```
or
```bash
<YOUR_TOOL> --help | head -n 1 &> v_<YOUR_TOOL>.txt 2>&1 || true
```
You then need to edit the script `bin/scrape_software_versions.py` to:
1. Add a Python regex for your tool's `--version` output (as in stored in the `v_<YOUR_TOOL>.txt` file), to ensure the version is reported as a `v` and the version number e.g. `v2.1.1`
2. Add a HTML entry to the `OrderedDict` for formatting in MultiQC.
### Images and figures
For overview images and other documents we follow the nf-core [style guidelines and examples](https://nf-co.re/developers/design_guidelines).
---
name: Bug report
about: Report something that is broken or incorrect
labels: bug
---
<!-- <!--
# nf-core/hic bug report # nf-core/hic bug report
...@@ -7,6 +13,13 @@ Thanks for telling us about a problem with the pipeline. ...@@ -7,6 +13,13 @@ Thanks for telling us about a problem with the pipeline.
Please delete this text and anything that's not relevant from the template below: Please delete this text and anything that's not relevant from the template below:
--> -->
## Check Documentation
I have checked the following places for your error:
- [ ] [nf-core website: troubleshooting](https://nf-co.re/usage/troubleshooting)
- [ ] [nf-core/hic pipeline documentation](https://nf-co.re/hic/usage)
## Description of the bug ## Description of the bug
<!-- A clear and concise description of what the bug is. --> <!-- A clear and concise description of what the bug is. -->
...@@ -22,6 +35,13 @@ Steps to reproduce the behaviour: ...@@ -22,6 +35,13 @@ Steps to reproduce the behaviour:
<!-- A clear and concise description of what you expected to happen. --> <!-- A clear and concise description of what you expected to happen. -->
## Log files
Have you provided the following extra information/files:
- [ ] The command used to run the pipeline
- [ ] The `.nextflow.log` file <!-- this is a hidden file in the directory where you launched the pipeline -->
## System ## System
- Hardware: <!-- [e.g. HPC, Desktop, Cloud...] --> - Hardware: <!-- [e.g. HPC, Desktop, Cloud...] -->
...@@ -35,7 +55,7 @@ Steps to reproduce the behaviour: ...@@ -35,7 +55,7 @@ Steps to reproduce the behaviour:
## Container engine ## Container engine
- Engine: <!-- [e.g. Conda, Docker or Singularity] --> - Engine: <!-- [e.g. Conda, Docker, Singularity, Podman, Shifter or Charliecloud] -->
- version: <!-- [e.g. 1.0.0] --> - version: <!-- [e.g. 1.0.0] -->
- Image tag: <!-- [e.g. nfcore/hic:1.0.0] --> - Image tag: <!-- [e.g. nfcore/hic:1.0.0] -->
......
blank_issues_enabled: false
contact_links:
- name: Join nf-core
url: https://nf-co.re/join
about: Please join the nf-core community here
- name: "Slack #hic channel"
url: https://nfcore.slack.com/channels/hic
about: Discussion about the nf-core/hic pipeline
---
name: Feature request
about: Suggest an idea for the nf-core/hic pipeline
labels: enhancement
---
<!-- <!--
# nf-core/hic feature request # nf-core/hic feature request
...@@ -5,7 +11,6 @@ Hi there! ...@@ -5,7 +11,6 @@ Hi there!
Thanks for suggesting a new feature for the pipeline! Thanks for suggesting a new feature for the pipeline!
Please delete this text and anything that's not relevant from the template below: Please delete this text and anything that's not relevant from the template below:
--> -->
## Is your feature request related to a problem? Please describe ## Is your feature request related to a problem? Please describe
...@@ -25,4 +30,3 @@ Please delete this text and anything that's not relevant from the template below ...@@ -25,4 +30,3 @@ Please delete this text and anything that's not relevant from the template below
## Additional context ## Additional context
<!-- Add any other context about the feature request here. --> <!-- Add any other context about the feature request here. -->
...@@ -10,12 +10,18 @@ Remember that PRs should be made against the dev branch, unless you're preparing ...@@ -10,12 +10,18 @@ Remember that PRs should be made against the dev branch, unless you're preparing
Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/hic/tree/master/.github/CONTRIBUTING.md) Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/hic/tree/master/.github/CONTRIBUTING.md)
--> -->
<!-- markdownlint-disable ul-indent -->
## PR checklist ## PR checklist
- [ ] This comment contains a description of changes (with reason) - [ ] This comment contains a description of changes (with reason).
- [ ] `CHANGELOG.md` is updated
- [ ] If you've fixed a bug or added code that should be tested, add tests! - [ ] If you've fixed a bug or added code that should be tested, add tests!
- [ ] Documentation in `docs` is updated - [ ] If you've added a new tool - add to the software_versions process and a regex to `scrape_software_versions.py`
- [ ] If necessary, also make a PR on the [nf-core/hic branch on the nf-core/test-datasets repo](https://github.com/nf-core/test-datasets/pull/new/nf-core/hic) - [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/hic/tree/master/.github/CONTRIBUTING.md)
- [ ] If necessary, also make a PR on the nf-core/hic _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
- [ ] Make sure your code lints (`nf-core lint .`).
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker`).
- [ ] Usage Documentation in `docs/usage.md` is updated.
- [ ] Output Documentation in `docs/output.md` is updated.
- [ ] `CHANGELOG.md` is updated.
- [ ] `README.md` is updated (including new tool citations and authors/contributors).
# Markdownlint configuration file # Markdownlint configuration file
default: true, default: true
line-length: false line-length: false
no-duplicate-header: no-duplicate-header:
siblings_only: true siblings_only: true
no-inline-html:
allowed_elements:
- img
- p
- kbd
- details
- summary
name: nf-core AWS full size tests name: nf-core AWS full size tests
# This workflow is triggered on push to the master branch. # This workflow is triggered on published releases.
# It can be additionally triggered manually with GitHub actions workflow dispatch.
# It runs the -profile 'test_full' on AWS batch # It runs the -profile 'test_full' on AWS batch
on: on:
release: workflow_run:
types: [published] workflows: ["nf-core Docker push (release)"]
types: [completed]
workflow_dispatch:
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
jobs: jobs:
run-awstest: run-awstest:
...@@ -13,7 +26,7 @@ jobs: ...@@ -13,7 +26,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Setup Miniconda - name: Setup Miniconda
uses: goanpeca/setup-miniconda@v1.0.2 uses: conda-incubator/setup-miniconda@v2
with: with:
auto-update-conda: true auto-update-conda: true
python-version: 3.7 python-version: 3.7
...@@ -23,13 +36,6 @@ jobs: ...@@ -23,13 +36,6 @@ jobs:
# Add full size test data (but still relatively small datasets for few samples) # Add full size test data (but still relatively small datasets for few samples)
# on the `test_full.config` test runs with only one set of parameters # on the `test_full.config` test runs with only one set of parameters
# Then specify `-profile test_full` instead of `-profile test` on the AWS batch command # Then specify `-profile test_full` instead of `-profile test` on the AWS batch command
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
run: | run: |
aws batch submit-job \ aws batch submit-job \
--region eu-west-1 \ --region eu-west-1 \
......
name: nf-core AWS test name: nf-core AWS test
# This workflow is triggered on push to the master branch. # This workflow is triggered on push to the master branch.
# It runs the -profile 'test' on AWS batch # It can be additionally triggered manually with GitHub actions workflow dispatch.
# It runs the -profile 'test' on AWS batch.
on: on:
push: workflow_dispatch:
branches:
- master
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
jobs: jobs:
run-awstest: run-awstest:
...@@ -14,7 +23,7 @@ jobs: ...@@ -14,7 +23,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Setup Miniconda - name: Setup Miniconda
uses: goanpeca/setup-miniconda@v1.0.2 uses: conda-incubator/setup-miniconda@v2
with: with:
auto-update-conda: true auto-update-conda: true
python-version: 3.7 python-version: 3.7
...@@ -23,17 +32,10 @@ jobs: ...@@ -23,17 +32,10 @@ jobs:
- name: Start AWS batch job - name: Start AWS batch job
# For example: adding multiple test runs with different parameters # For example: adding multiple test runs with different parameters
# Remember that you can parallelise this by using strategy.matrix # Remember that you can parallelise this by using strategy.matrix
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
run: | run: |
aws batch submit-job \ aws batch submit-job \
--region eu-west-1 \ --region eu-west-1 \
--job-name nf-core-hic \ --job-name nf-core-hic \
--job-queue $AWS_JOB_QUEUE \ --job-queue $AWS_JOB_QUEUE \
--job-definition $AWS_JOB_DEFINITION \ --job-definition $AWS_JOB_DEFINITION \
--container-overrides '{"command": ["nf-core/hic", "-r '"${GITHUB_SHA}"' -profile test --outdir s3://'"${AWS_S3_BUCKET}"'/hic/results-'"${GITHUB_SHA}"' -w s3://'"${AWS_S3_BUCKET}"'/hic/work-'"${GITHUB_SHA}"' -with-tower"], "environment": [{"name": "TOWER_ACCESS_TOKEN", "value": "'"$TOWER_ACCESS_TOKEN"'"}]}'hic/work-'"${GITHUB_SHA}"' -with-tower"], "environment": [{"name": "TOWER_ACCESS_TOKEN", "value": "'"$TOWER_ACCESS_TOKEN"'"}]}' --container-overrides '{"command": ["nf-core/hic", "-r '"${GITHUB_SHA}"' -profile test --outdir s3://'"${AWS_S3_BUCKET}"'/hic/results-'"${GITHUB_SHA}"' -w s3://'"${AWS_S3_BUCKET}"'/hic/work-'"${GITHUB_SHA}"' -with-tower"], "environment": [{"name": "TOWER_ACCESS_TOKEN", "value": "'"$TOWER_ACCESS_TOKEN"'"}]}'
...@@ -2,7 +2,7 @@ name: nf-core branch protection ...@@ -2,7 +2,7 @@ name: nf-core branch protection
# This workflow is triggered on PRs to master branch on the repository # This workflow is triggered on PRs to master branch on the repository
# It fails when someone tries to make a PR against the nf-core `master` branch instead of `dev` # It fails when someone tries to make a PR against the nf-core `master` branch instead of `dev`
on: on:
pull_request: pull_request_target:
branches: [master] branches: [master]
jobs: jobs:
...@@ -23,13 +23,22 @@ jobs: ...@@ -23,13 +23,22 @@ jobs:
uses: mshick/add-pr-comment@v1 uses: mshick/add-pr-comment@v1
with: with:
message: | message: |
## This PR is against the `master` branch :x:
* Do not close this PR
* Click _Edit_ and change the `base` to `dev`
* This CI test will remain failed until you push a new commit
---
Hi @${{ github.event.pull_request.user.login }}, Hi @${{ github.event.pull_request.user.login }},
It looks like this pull-request is has been made against the ${{github.event.pull_request.head.repo.full_name}} `master` branch. It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `master` branch.
The `master` branch on nf-core repositories should always contain code from the latest release. The `master` branch on nf-core repositories should always contain code from the latest release.
Because of this, PRs to `master` are only allowed if they come from the ${{github.event.pull_request.head.repo.full_name}} `dev` branch. Because of this, PRs to `master` are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch.
You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page. You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page.
Note that even after this, the test will continue to show as failing until you push a new commit.
Thanks again for your contribution! Thanks again for your contribution!
repo-token: ${{ secrets.GITHUB_TOKEN }} repo-token: ${{ secrets.GITHUB_TOKEN }}
......
...@@ -8,6 +8,9 @@ on: ...@@ -8,6 +8,9 @@ on:
release: release:
types: [published] types: [published]
# Uncomment if we need an edge release of Nextflow again
# env: NXF_EDGE: 1
jobs: jobs:
test: test:
name: Run workflow tests name: Run workflow tests
...@@ -20,29 +23,31 @@ jobs: ...@@ -20,29 +23,31 @@ jobs:
strategy: strategy:
matrix: matrix:
# Nextflow versions: check pipeline minimum and current latest # Nextflow versions: check pipeline minimum and current latest
nxf_ver: ['19.10.0', ''] nxf_ver: ['20.04.0', '']
steps: steps:
- name: Check out pipeline code - name: Check out pipeline code
uses: actions/checkout@v2 uses: actions/checkout@v2
- name: Check if Dockerfile or Conda environment changed - name: Check if Dockerfile or Conda environment changed
uses: technote-space/get-diff-action@v1 uses: technote-space/get-diff-action@v4
with: with:
PREFIX_FILTER: | FILES: |
Dockerfile Dockerfile
environment.yml environment.yml
- name: Build new docker image - name: Build new docker image
if: env.GIT_DIFF if: env.MATCHED_FILES
run: docker build --no-cache . -t nfcore/hic:1.2.2 run: docker build --no-cache . -t nfcore/hic:1.3.0
- name: Pull docker image - name: Pull docker image
if: ${{ !env.GIT_DIFF }} if: ${{ !env.MATCHED_FILES }}
run: | run: |
docker pull nfcore/hic:dev docker pull nfcore/hic:dev
docker tag nfcore/hic:dev nfcore/hic:1.2.2 docker tag nfcore/hic:dev nfcore/hic:1.3.0
- name: Install Nextflow - name: Install Nextflow
env:
CAPSULE_LOG: none
run: | run: |
wget -qO- get.nextflow.io | bash wget -qO- get.nextflow.io | bash
sudo mv nextflow /usr/local/bin/ sudo mv nextflow /usr/local/bin/
......
...@@ -19,6 +19,34 @@ jobs: ...@@ -19,6 +19,34 @@ jobs:
run: npm install -g markdownlint-cli run: npm install -g markdownlint-cli
- name: Run Markdownlint - name: Run Markdownlint
run: markdownlint ${GITHUB_WORKSPACE} -c ${GITHUB_WORKSPACE}/.github/markdownlint.yml run: markdownlint ${GITHUB_WORKSPACE} -c ${GITHUB_WORKSPACE}/.github/markdownlint.yml
# If the above check failed, post a comment on the PR explaining the failure
- name: Post PR comment
if: failure()
uses: mshick/add-pr-comment@v1
with:
message: |
## Markdown linting is failing
To keep the code consistent with lots of contributors, we run automated code consistency checks.
To fix this CI test, please run:
* Install `markdownlint-cli`
* On Mac: `brew install markdownlint-cli`
* Everything else: [Install `npm`](https://www.npmjs.com/get-npm) then [install `markdownlint-cli`](https://www.npmjs.com/package/markdownlint-cli) (`npm install -g markdownlint-cli`)
* Fix the markdown errors
* Automatically: `markdownlint . --config .github/markdownlint.yml --fix`
* Manually resolve anything left from `markdownlint . --config .github/markdownlint.yml`
Once you push these changes the test should pass, and you can hide this comment :+1:
We highly recommend setting up markdownlint in your code editor so that this formatting is done automatically on save. Ask about it on Slack for help!
Thanks again for your contribution!
repo-token: ${{ secrets.GITHUB_TOKEN }}
allow-repeats: false
YAML: YAML:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
...@@ -29,13 +57,44 @@ jobs: ...@@ -29,13 +57,44 @@ jobs:
- name: Install yaml-lint - name: Install yaml-lint
run: npm install -g yaml-lint run: npm install -g yaml-lint
- name: Run yaml-lint - name: Run yaml-lint
run: yamllint $(find ${GITHUB_WORKSPACE} -type f -name "*.yml") run: yamllint $(find ${GITHUB_WORKSPACE} -type f -name "*.yml" -o -name "*.yaml")
# If the above check failed, post a comment on the PR explaining the failure
- name: Post PR comment
if: failure()
uses: mshick/add-pr-comment@v1
with:
message: |
## YAML linting is failing
To keep the code consistent with lots of contributors, we run automated code consistency checks.
To fix this CI test, please run:
* Install `yaml-lint`
* [Install `npm`](https://www.npmjs.com/get-npm) then [install `yaml-lint`](https://www.npmjs.com/package/yaml-lint) (`npm install -g yaml-lint`)
* Fix the markdown errors
* Run the test locally: `yamllint $(find . -type f -name "*.yml" -o -name "*.yaml")`
* Fix any reported errors in your YAML files
Once you push these changes the test should pass, and you can hide this comment :+1:
We highly recommend setting up yaml-lint in your code editor so that this formatting is done automatically on save. Ask about it on Slack for help!
Thanks again for your contribution!
repo-token: ${{ secrets.GITHUB_TOKEN }}
allow-repeats: false
nf-core: nf-core:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Check out pipeline code - name: Check out pipeline code
uses: actions/checkout@v2 uses: actions/checkout@v2
- name: Install Nextflow - name: Install Nextflow
env:
CAPSULE_LOG: none
run: | run: |
wget -qO- get.nextflow.io | bash wget -qO- get.nextflow.io | bash
sudo mv nextflow /usr/local/bin/ sudo mv nextflow /usr/local/bin/
...@@ -55,11 +114,19 @@ jobs: ...@@ -55,11 +114,19 @@ jobs:
GITHUB_COMMENTS_URL: ${{ github.event.pull_request.comments_url }} GITHUB_COMMENTS_URL: ${{ github.event.pull_request.comments_url }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_PR_COMMIT: ${{ github.event.pull_request.head.sha }} GITHUB_PR_COMMIT: ${{ github.event.pull_request.head.sha }}
run: nf-core -l lint_log.txt lint ${GITHUB_WORKSPACE} run: nf-core -l lint_log.txt lint ${GITHUB_WORKSPACE} --markdown lint_results.md
- name: Save PR number
if: ${{ always() }}
run: echo ${{ github.event.pull_request.number }} > PR_number.txt
- name: Upload linting log file artifact - name: Upload linting log file artifact
if: ${{ always() }} if: ${{ always() }}
uses: actions/upload-artifact@v2 uses: actions/upload-artifact@v2
with: with:
name: linting-log-file name: linting-logs
path: lint_log.txt path: |
lint_log.txt
lint_results.md
PR_number.txt
name: nf-core linting comment
# This workflow is triggered after the linting action is complete
# It posts an automated comment to the PR, even if the PR is coming from a fork
on:
workflow_run:
workflows: ["nf-core linting"]
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Download lint results
uses: dawidd6/action-download-artifact@v2
with:
workflow: linting.yml
- name: Get PR number
id: pr_number
run: echo "::set-output name=pr_number::$(cat linting-logs/PR_number.txt)"
- name: Post PR comment
uses: marocchino/sticky-pull-request-comment@v2
with:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
number: ${{ steps.pr_number.outputs.pr_number }}
path: linting-logs/lint_results.md
name: nf-core Docker push (dev)
# This builds the docker image and pushes it to DockerHub
# Runs on nf-core repo releases and push event to 'dev' branch (PR merges)
on:
push:
branches:
- dev
jobs:
push_dockerhub:
name: Push new Docker image to Docker Hub (dev)
runs-on: ubuntu-latest
# Only run for the nf-core repo, for releases and merged PRs
if: ${{ github.repository == 'nf-core/hic' }}
env:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_PASS: ${{ secrets.DOCKERHUB_PASS }}
steps:
- name: Check out pipeline code
uses: actions/checkout@v2
- name: Build new docker image
run: docker build --no-cache . -t nfcore/hic:dev
- name: Push Docker image to DockerHub (dev)
run: |
echo "$DOCKERHUB_PASS" | docker login -u "$DOCKERHUB_USERNAME" --password-stdin
docker push nfcore/hic:dev
name: nf-core Docker push name: nf-core Docker push (release)
# This builds the docker image and pushes it to DockerHub # This builds the docker image and pushes it to DockerHub
# Runs on nf-core repo releases and push event to 'dev' branch (PR merges) # Runs on nf-core repo releases and push event to 'dev' branch (PR merges)
on: on:
push:
branches:
- dev
release: release:
types: [published] types: [published]
jobs: jobs:
push_dockerhub: push_dockerhub:
name: Push new Docker image to Docker Hub name: Push new Docker image to Docker Hub (release)
runs-on: ubuntu-latest runs-on: ubuntu-latest
# Only run for the nf-core repo, for releases and merged PRs # Only run for the nf-core repo, for releases and merged PRs
if: ${{ github.repository == 'nf-core/hic' }} if: ${{ github.repository == 'nf-core/hic' }}
...@@ -24,15 +21,7 @@ jobs: ...@@ -24,15 +21,7 @@ jobs:
- name: Build new docker image - name: Build new docker image
run: docker build --no-cache . -t nfcore/hic:latest run: docker build --no-cache . -t nfcore/hic:latest
- name: Push Docker image to DockerHub (dev)
if: ${{ github.event_name == 'push' }}
run: |
echo "$DOCKERHUB_PASS" | docker login -u "$DOCKERHUB_USERNAME" --password-stdin
docker tag nfcore/hic:latest nfcore/hic:dev
docker push nfcore/hic:dev
- name: Push Docker image to DockerHub (release) - name: Push Docker image to DockerHub (release)
if: ${{ github.event_name == 'release' }}
run: | run: |
echo "$DOCKERHUB_PASS" | docker login -u "$DOCKERHUB_USERNAME" --password-stdin echo "$DOCKERHUB_PASS" | docker login -u "$DOCKERHUB_USERNAME" --password-stdin
docker push nfcore/hic:latest docker push nfcore/hic:latest
......
files_unchanged:
- .github/ISSUE_TEMPLATE/bug_report.md
- .github/PULL_REQUEST_TEMPLATE.md
...@@ -3,6 +3,32 @@ ...@@ -3,6 +3,32 @@
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## v1.3.0 - 2021-22-05
* Change the `/tmp/` folder to `./tmp/` folder so that all tmp files are now in the work directory (#24)
* Add `--hicpro_maps` options to generate the raw and normalized HiC-Pro maps. The default is now to use cooler
* Add chromosome compartments calling with cooltools (#53)
* Add HiCExplorer distance decay quality control (#54)
* Add HiCExplorer TADs calling (#55)
* Add insulation score TADs calling (#55)
* Generate cooler/txt contact maps
* Normalize Hi-C data with cooler instead of iced
* New `--digestion` parameter to automatically set the restriction_site and ligation_site motifs
* New `--keep_multi` and `keep_dup` options. Default: false
* Template update for nf-core/tools
* Minor fix to summary log messages in pipeline header
### `Fixed`
* Fix bug in stats report which were not all correcly exported in the results folder
* Fix recurrent bug in input file extension (#86)
* Fix bug in `--bin_size` parameter (#85)
* `--min_mapq` is ignored if `--keep_multi` is used
### `Deprecated`
* `--rm_dup` and `--rm_multi` are replaced by `--keep_dups` and `--keep_multi`
## v1.2.2 - 2020-09-02 ## v1.2.2 - 2020-09-02
### `Added` ### `Added`
......
# Contributor Covenant Code of Conduct # Code of Conduct at nf-core (v1.0)
## Our Pledge ## Our Pledge
In the interest of fostering an open and welcoming environment, we as In the interest of fostering an open, collaborative, and welcoming environment, we as contributors and maintainers of nf-core, pledge to making participation in our projects and community a harassment-free experience for everyone, regardless of:
contributors and maintainers pledge to making participation in our project
and our community a harassment-free experience for everyone, regardless of
age, body size, disability, ethnicity, gender identity and expression, level
of experience, nationality, personal appearance, race, religion, or sexual
identity and orientation.
## Our Standards - Age
- Body size
- Familial status
- Gender identity and expression
- Geographical location
- Level of experience
- Nationality and national origins
- Native language
- Physical and neurological ability
- Race or ethnicity
- Religion
- Sexual identity and orientation
- Socioeconomic status
Examples of behavior that contributes to creating a positive environment Please note that the list above is alphabetised and is therefore not ranked in any order of preference or importance.
include:
* Using welcoming and inclusive language ## Preamble
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include: > Note: This Code of Conduct (CoC) has been drafted by the nf-core Safety Officer and been edited after input from members of the nf-core team and others. "We", in this document, refers to the Safety Officer and members of the nf-core core team, both of whom are deemed to be members of the nf-core community and are therefore required to abide by this Code of Conduct. This document will amended periodically to keep it up-to-date, and in case of any dispute, the most current version will apply.
* The use of sexualized language or imagery and unwelcome sexual attention An up-to-date list of members of the nf-core core team can be found [here](https://nf-co.re/about). Our current safety officer is Renuka Kudva.
or advances
* Trolling, insulting/derogatory comments, and personal or political attacks nf-core is a young and growing community that welcomes contributions from anyone with a shared vision for [Open Science Policies](https://www.fosteropenscience.eu/taxonomy/term/8). Open science policies encompass inclusive behaviours and we strive to build and maintain a safe and inclusive environment for all individuals.
* Public or private harassment
* Publishing others' private information, such as a physical or electronic We have therefore adopted this code of conduct (CoC), which we require all members of our community and attendees in nf-core events to adhere to in all our workspaces at all times. Workspaces include but are not limited to Slack, meetings on Zoom, Jitsi, YouTube live etc.
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a Our CoC will be strictly enforced and the nf-core team reserve the right to exclude participants who do not comply with our guidelines from our workspaces and future nf-core activities.
professional setting
We ask all members of our community to help maintain a supportive and productive workspace and to avoid behaviours that can make individuals feel unsafe or unwelcome. Please help us maintain and uphold this CoC.
Questions, concerns or ideas on what we can include? Contact safety [at] nf-co [dot] re
## Our Responsibilities ## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable The safety officer is responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behaviour.
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior. The safety officer in consultation with the nf-core core team have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
Members of the core team or the safety officer who violate the CoC will be required to recuse themselves pending investigation. They will not have access to any reports of the violations and be subject to the same actions as others in violation of the CoC.
## When are where does this Code of Conduct apply?
Participation in the nf-core community is contingent on following these guidelines in all our workspaces and events. This includes but is not limited to the following listed alphabetically and therefore in no order of preference:
- Communicating with an official project email address.
- Communicating with community members within the nf-core Slack channel.
- Participating in hackathons organised by nf-core (both online and in-person events).
- Participating in collaborative work on GitHub, Google Suite, community calls, mentorship meetings, email correspondence.
- Participating in workshops, training, and seminar series organised by nf-core (both online and in-person events). This applies to events hosted on web-based platforms such as Zoom, Jitsi, YouTube live etc.
- Representing nf-core on social media. This includes both official and personal accounts.
## nf-core cares 😊
nf-core's CoC and expectations of respectful behaviours for all participants (including organisers and the nf-core team) include but are not limited to the following (listed in alphabetical order):
- Ask for consent before sharing another community member’s personal information (including photographs) on social media.
- Be respectful of differing viewpoints and experiences. We are all here to learn from one another and a difference in opinion can present a good learning opportunity.
- Celebrate your accomplishments at events! (Get creative with your use of emojis 🎉 🥳 💯 🙌 !)
- Demonstrate empathy towards other community members. (We don’t all have the same amount of time to dedicate to nf-core. If tasks are pending, don’t hesitate to gently remind members of your team. If you are leading a task, ask for help if you feel overwhelmed.)
- Engage with and enquire after others. (This is especially important given the geographically remote nature of the nf-core community, so let’s do this the best we can)
- Focus on what is best for the team and the community. (When in doubt, ask)
- Graciously accept constructive criticism, yet be unafraid to question, deliberate, and learn.
- Introduce yourself to members of the community. (We’ve all been outsiders and we know that talking to strangers can be hard for some, but remember we’re interested in getting to know you and your visions for open science!)
- Show appreciation and **provide clear feedback**. (This is especially important because we don’t see each other in person and it can be harder to interpret subtleties. Also remember that not everyone understands a certain language to the same extent as you do, so **be clear in your communications to be kind.**)
- Take breaks when you feel like you need them.
- Using welcoming and inclusive language. (Participants are encouraged to display their chosen pronouns on Zoom or in communication on Slack.)
## nf-core frowns on 😕
The following behaviours from any participants within the nf-core community (including the organisers) will be considered unacceptable under this code of conduct. Engaging or advocating for any of the following could result in expulsion from nf-core workspaces.
- Deliberate intimidation, stalking or following and sustained disruption of communication among participants of the community. This includes hijacking shared screens through actions such as using the annotate tool in conferencing software such as Zoom.
- “Doxing” i.e. posting (or threatening to post) another person’s personal identifying information online.
- Spamming or trolling of individuals on social media.
- Use of sexual or discriminatory imagery, comments, or jokes and unwelcome sexual attention.
- Verbal and text comments that reinforce social structures of domination related to gender, gender identity and expression, sexual orientation, ability, physical appearance, body size, race, age, religion or work experience.
### Online Trolling
The majority of nf-core interactions and events are held online. Unfortunately, holding events online comes with the added issue of online trolling. This is unacceptable, reports of such behaviour will be taken very seriously, and perpetrators will be excluded from activities immediately.
All community members are required to ask members of the group they are working within for explicit consent prior to taking screenshots of individuals during video calls.
## Procedures for Reporting CoC violations
Project maintainers have the right and responsibility to remove, edit, or If someone makes you feel uncomfortable through their behaviours or actions, report it as soon as possible.
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
## Scope You can reach out to members of the [nf-core core team](https://nf-co.re/about) and they will forward your concerns to the safety officer(s).
This Code of Conduct applies both within project spaces and in public spaces Issues directly concerning members of the core team will be dealt with by other members of the core team and the safety manager, and possible conflicts of interest will be taken into account. nf-core is also in discussions about having an ombudsperson, and details will be shared in due course.
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an
appointed representative at an online or offline event. Representation of a
project may be further defined and clarified by project maintainers.
## Enforcement All reports will be handled with utmost discretion and confidentially.
Instances of abusive, harassing, or otherwise unacceptable behavior may be ## Attribution and Acknowledgements
reported by contacting the project team on
[Slack](https://nf-co.re/join/slack). The project team will review
and investigate all complaints, and will respond in a way that it deems
appropriate to the circumstances. The project team is obligated to maintain
confidentiality with regard to the reporter of an incident. Further details
of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good - The [Contributor Covenant, version 1.4](http://contributor-covenant.org/version/1/4)
faith may face temporary or permanent repercussions as determined by other - The [OpenCon 2017 Code of Conduct](http://www.opencon2017.org/code_of_conduct) (CC BY 4.0 OpenCon organisers, SPARC and Right to Research Coalition)
members of the project's leadership. - The [eLife innovation sprint 2020 Code of Conduct](https://sprint.elifesciences.org/code-of-conduct/)
- The [Mozilla Community Participation Guidelines v3.1](https://www.mozilla.org/en-US/about/governance/policies/participation/) (version 3.1, CC BY-SA 3.0 Mozilla)
## Attribution ## Changelog
This Code of Conduct is adapted from the [Contributor Covenant][homepage], ### v1.0 - March 12th, 2021
version 1.4, available at
[https://www.contributor-covenant.org/version/1/4/code-of-conduct/][version]
[homepage]: https://contributor-covenant.org - Complete rewrite from original [Contributor Covenant](http://contributor-covenant.org/) CoC.
[version]: https://www.contributor-covenant.org/version/1/4/code-of-conduct/
FROM nfcore/base:1.10.2 FROM nfcore/base:1.14
LABEL authors="Nicolas Servant" \ LABEL authors="Nicolas Servant" \
description="Docker image containing all software requirements for the nf-core/hic pipeline" description="Docker image containing all software requirements for the nf-core/hic pipeline"
## Install gcc for pip iced install ## Install gcc for pip iced install
RUN apt-get update && apt-get install -y gcc g++ && apt-get clean -y RUN apt-get update && apt-get install -y gcc g++ && apt-get clean -y
# Install the conda environment
COPY environment.yml / COPY environment.yml /
RUN conda env create --quiet -f /environment.yml && conda clean -a RUN conda env create --quiet -f /environment.yml && conda clean -a
# Add conda installation dir to PATH (instead of doing 'conda activate') # Add conda installation dir to PATH (instead of doing 'conda activate')
ENV PATH /opt/conda/envs/nf-core-hic-1.2.2/bin:$PATH ENV PATH /opt/conda/envs/nf-core-hic-1.3.0/bin:$PATH
# Dump the details of the installed packages to a file for posterity # Dump the details of the installed packages to a file for posterity
RUN conda env export --name nf-core-hic-1.2.2 > nf-core-hic-1.2.2.yml RUN conda env export --name nf-core-hic-1.3.0 > nf-core-hic-1.3.0.yml
# Instruct R processes to use these empty files instead of clashing with a local version # Instruct R processes to use these empty files instead of clashing with a local version
RUN touch .Rprofile RUN touch .Rprofile
......
# ![nf-core/hic](docs/images/nfcore-hic_logo.png) # ![nf-core/hic](docs/images/nf-core-hic_logo.png)
**Analysis of Chromosome Conformation Capture data (Hi-C)**. **Analysis of Chromosome Conformation Capture data (Hi-C)**.
[![GitHub Actions CI Status](https://github.com/nf-core/hic/workflows/nf-core%20CI/badge.svg)](https://github.com/nf-core/hic/actions) [![GitHub Actions CI Status](https://github.com/nf-core/hic/workflows/nf-core%20CI/badge.svg)](https://github.com/nf-core/hic/actions)
[![GitHub Actions Linting Status](https://github.com/nf-core/hic/workflows/nf-core%20linting/badge.svg)](https://github.com/nf-core/hic/actions) [![GitHub Actions Linting Status](https://github.com/nf-core/hic/workflows/nf-core%20linting/badge.svg)](https://github.com/nf-core/hic/actions)
[![Nextflow](https://img.shields.io/badge/nextflow-%E2%89%A519.10.0-brightgreen.svg)](https://www.nextflow.io/) [![Nextflow](https://img.shields.io/badge/nextflow-%E2%89%A520.04.0-brightgreen.svg)](https://www.nextflow.io/)
[![install with bioconda](https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg)](https://bioconda.github.io/) [![install with bioconda](https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg)](https://bioconda.github.io/)
[![Docker](https://img.shields.io/docker/automated/nfcore/hic.svg)](https://hub.docker.com/r/nfcore/hic) [![Docker](https://img.shields.io/docker/automated/nfcore/hic.svg)](https://hub.docker.com/r/nfcore/hic)
...@@ -14,7 +14,7 @@ ...@@ -14,7 +14,7 @@
## Introduction ## Introduction
This pipeline is based on the This pipeline was originally set up from the
[HiC-Pro workflow](https://github.com/nservant/HiC-Pro). [HiC-Pro workflow](https://github.com/nservant/HiC-Pro).
It was designed to process Hi-C data from raw FastQ files (paired-end Illumina It was designed to process Hi-C data from raw FastQ files (paired-end Illumina
data) to normalized contact maps. data) to normalized contact maps.
...@@ -24,6 +24,10 @@ In practice, this workflow was successfully applied to many data-sets including ...@@ -24,6 +24,10 @@ In practice, this workflow was successfully applied to many data-sets including
dilution Hi-C, in situ Hi-C, DNase Hi-C, Micro-C, capture-C, capture Hi-C or dilution Hi-C, in situ Hi-C, DNase Hi-C, Micro-C, capture-C, capture Hi-C or
HiChip data. HiChip data.
Contact maps are generated in standard formats including HiC-Pro, and cooler for
downstream analysis and visualization.
Addition analysis steps such as compartments and TADs calling are also available.
The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool
to run tasks across multiple compute infrastructures in a very portable manner. to run tasks across multiple compute infrastructures in a very portable manner.
It comes with docker / singularity containers making installation trivial and It comes with docker / singularity containers making installation trivial and
...@@ -31,28 +35,30 @@ results highly reproducible. ...@@ -31,28 +35,30 @@ results highly reproducible.
## Pipeline summary ## Pipeline summary
1. HiC-Pro data processing ([`HiC-Pro`](https://github.com/nservant/HiC-Pro))
1. Mapping using a two steps strategy to rescue reads spanning the ligation 1. Mapping using a two steps strategy to rescue reads spanning the ligation
sites (bowtie2) sites ([`bowtie2`](http://bowtie-bio.sourceforge.net/bowtie2/index.shtml))
2. Detection of valid interaction products 2. Detection of valid interaction products
3. Duplicates removal 3. Duplicates removal
4. Create genome-wide contact maps at various resolution 4. Generate raw and normalized contact maps ([`iced`](https://github.com/hiclib/iced))
5. Contact maps normalization using the ICE algorithm (iced) 2. Create genome-wide contact maps at various resolutions ([`cooler`](https://github.com/open2c/cooler))
6. Quality controls and report (MultiQC) 3. Contact maps normalization using balancing algorithm ([`cooler`](https://github.com/open2c/cooler))
7. Addition export for visualisation and downstream analysis (cooler) 4. Export to various contact maps formats ([`HiC-Pro`](https://github.com/nservant/HiC-Pro), [`cooler`](https://github.com/open2c/cooler))
5. Quality controls ([`HiC-Pro`](https://github.com/nservant/HiC-Pro), [`HiCExplorer`](https://github.com/deeptools/HiCExplorer))
6. Compartments calling ([`cooltools`](https://cooltools.readthedocs.io/en/latest/))
7. TADs calling ([`HiCExplorer`](https://github.com/deeptools/HiCExplorer), [`cooltools`](https://cooltools.readthedocs.io/en/latest/))
8. Quality control report ([`MultiQC`](https://multiqc.info/))
## Quick Start ## Quick Start
i. Install [`nextflow`](https://nf-co.re/usage/installation) 1. Install [`nextflow`](https://nf-co.re/usage/installation) (`>=20.04.0`)
ii. Install either [`Docker`](https://docs.docker.com/engine/installation/) 2. Install any of [`Docker`](https://docs.docker.com/engine/installation/), [`Singularity`](https://www.sylabs.io/guides/3.0/user-guide/), [`Podman`](https://podman.io/), [`Shifter`](https://nersc.gitlab.io/development/shifter/how-to-use/) or [`Charliecloud`](https://hpc.github.io/charliecloud/) for full pipeline reproducibility _(please only use [`Conda`](https://conda.io/miniconda.html) as a last resort; see [docs](https://nf-co.re/usage/configuration#basic-configuration-profiles))_
or [`Singularity`](https://www.sylabs.io/guides/3.0/user-guide/)
for full pipeline reproducibility (please only use [`Conda`](https://conda.io/miniconda.html)
as a last resort; see [docs](https://nf-co.re/usage/configuration#basic-configuration-profiles))
iii. Download the pipeline and test it on a minimal dataset with a single command 3. Download the pipeline and test it on a minimal dataset with a single command
```bash ```bash
nextflow run nf-core/hic -profile test,<docker/singularity/conda/institute> nextflow run nf-core/hic -profile test,<docker/singularity/podman/shifter/charliecloud/conda/institute>
``` ```
> Please check [nf-core/configs](https://github.com/nf-core/configs#documentation) > Please check [nf-core/configs](https://github.com/nf-core/configs#documentation)
...@@ -61,35 +67,17 @@ If so, you can simply use `-profile <institute>` in your command. ...@@ -61,35 +67,17 @@ If so, you can simply use `-profile <institute>` in your command.
This will enable either `docker` or `singularity` and set the appropriate execution This will enable either `docker` or `singularity` and set the appropriate execution
settings for your local compute environment. settings for your local compute environment.
iv. Start running your own analysis! 4. Start running your own analysis!
```bash ```bash
nextflow run nf-core/hic -profile <docker/singularity/conda/institute> --reads '*_R{1,2}.fastq.gz' --genome GRCh37 nextflow run nf-core/hic -profile <docker/singularity/podman/shifter/charliecloud/conda/institute> --input '*_R{1,2}.fastq.gz' --genome GRCh37
``` ```
See [usage docs](https://nf-co.re/hic/usage) for all of the available options when running
the pipeline.
## Documentation ## Documentation
The nf-core/hic pipeline comes with documentation about the pipeline, The nf-core/hic pipeline comes with documentation about the pipeline: [usage](https://nf-co.re/hic/usage) and [output](https://nf-co.re/hic/output).
found in the `docs/` directory:
1. [Installation](https://nf-co.re/usage/installation)
2. Pipeline configuration
* [Local installation](https://nf-co.re/usage/local_installation)
* [Adding your own system config](https://nf-co.re/usage/adding_own_config)
* [Reference genomes](https://nf-co.re/usage/reference_genomes)
3. [Running the pipeline](docs/usage.md)
4. [Output and how to interpret the results](docs/output.md)
5. [Troubleshooting](https://nf-co.re/usage/troubleshooting)
The nf-core/hic pipeline comes with documentation about the pipeline which
you can read at [https://nf-co.re/hic/usage](https://nf-co.re/hic/usage) or
find in the [`docs/` directory](docs).
For further information or help, don't hesitate to get in touch on For further information or help, don't hesitate to get in touch on [Slack](https://nfcore.slack.com/channels/hic).
[Slack](https://nfcore.slack.com/channels/hic).
You can join with [this invite](https://nf-co.re/join/slack). You can join with [this invite](https://nf-co.re/join/slack).
## Credits ## Credits
...@@ -100,9 +88,7 @@ nf-core/hic was originally written by Nicolas Servant. ...@@ -100,9 +88,7 @@ nf-core/hic was originally written by Nicolas Servant.
If you would like to contribute to this pipeline, please see the [contributing guidelines](.github/CONTRIBUTING.md). If you would like to contribute to this pipeline, please see the [contributing guidelines](.github/CONTRIBUTING.md).
For further information or help, don't hesitate to get in touch on the For further information or help, don't hesitate to get in touch on the [Slack `#hic` channel](https://nfcore.slack.com/channels/hic) (you can join with [this invite](https://nf-co.re/join/slack)).
[Slack `#hic` channel](https://nfcore.slack.com/channels/hic)
(you can join with [this invite](https://nf-co.re/join/slack)).
## Citation ## Citation
...@@ -113,8 +99,14 @@ You can cite the `nf-core` publication as follows: ...@@ -113,8 +99,14 @@ You can cite the `nf-core` publication as follows:
> **The nf-core framework for community-curated bioinformatics pipelines.** > **The nf-core framework for community-curated bioinformatics pipelines.**
> >
> Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, > Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.
Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.
> >
> _Nat Biotechnol._ 2020 Feb 13. doi: [10.1038/s41587-020-0439-x](https://dx.doi.org/10.1038/s41587-020-0439-x). > _Nat Biotechnol._ 2020 Feb 13. doi: [10.1038/s41587-020-0439-x](https://dx.doi.org/10.1038/s41587-020-0439-x).
> ReadCube: [Full Access Link](https://rdcu.be/b1GjZ)
In addition, references of tools and data used in this pipeline are as follows:
> **HiC-Pro: An optimized and flexible pipeline for Hi-C processing.**
>
> Nicolas Servant, Nelle Varoquaux, Bryan R. Lajoie, Eric Viara, Chongjian Chen, Jean-Philippe Vert, Job Dekker, Edith Heard, Emmanuel Barillot.
>
> Genome Biology 2015, 16:259 doi: [10.1186/s13059-015-0831-x](https://dx.doi.org/10.1186/s13059-015-0831-x)
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment