When you produce some results for a particular project, you may want to share them with others who are working with you. With gblk you can share specific archives of your `.borg` repository with others or pull archives created by others.
If many people works with gblk, the push and pull commands may not work properly yet !
Indeed, the remote folder rigths may blocks you if another people has created this folder and you whant to update it.
If you work with other people you might have trouble: if you pull a gblk remote folder update by other, you will lose your unpulled commited results. And if you make a push, you will erase others changes
:::
When you produce some results for a particular project, you may want to share them with others who are working with you. With gblk you can share your `.borg` repository with other or update it with changes done by others.
::: {.callout-important}
When pushing archives to a remote folder, you lose the file deduplication between them. So make sure to push only the archives you want to share with other.
::: {.callout-note}
The choice of sharing the complete archive folder `.borg` was made to always benefits from borg's deduplication
:::
:::
::: {.callout-warning}
::: {.callout-important}
The way of duplicating and storing an archive folder is not the way borg was intended to be used. Indeed the replacment of a borg repository by an older one is considered suspicious and borg blocks the executions of its commands for this archive. See [this page](https://borgbackup.readthedocs.io/en/stable/faq.html?highlight=attack#this-is-either-an-attack-or-unsafe-warning) for details.
When pulling archives from a remote folder, you can pull archives produced by others for an unrelated project or even **tar file** not produced by borg. Make sure to specify the good remote path for your project.
:::
:::
## Remotes
## Remotes
...
@@ -826,7 +819,7 @@ You can see that *globally* defined remote are tagged with `(global)`.
...
@@ -826,7 +819,7 @@ You can see that *globally* defined remote are tagged with `(global)`.
# gblk `clone`, `push` and `pull` commands
# gblk `clone`, `push` and `pull` commands
# Clone command
## Clone command
This command can be used after a `git clone` command. If a gblk archive folder is available for the project, then you can execute the following command:
This command can be used after a `git clone` command. If a gblk archive folder is available for the project, then you can execute the following command:
...
@@ -834,29 +827,31 @@ This command can be used after a `git clone` command. If a gblk archive folder i
...
@@ -834,29 +827,31 @@ This command can be used after a `git clone` command. If a gblk archive folder i
gblk clone [HOSTNAME:]PATH
gblk clone [HOSTNAME:]PATH
```
```
Where `PATH` is the path of a gblk archive folder. The `PATH` must point to a folder with this structure (corresponding to a borg remote archive):
Where `PATH` is the path of a gblk archive folder. The `PATH` must point to a folder containg tar files:
```sh
```sh
PATH
PATH
├── archive_list
├── tar_file1
├── config
├── ...
├── data/
└── tar_filen
└── README
```
```
::: {.callout-note}
::: {.callout-note}
Clone muste be used with a *path* and not a remote name.
Clone must be used with a *path* and not a remote name.
:::
::: {.callout-important}
When cloning archives from a remote folder, you can pull archives produced by others for an unrelated project or even **tar file** not produced by borg. Make sure to specify the good path for your project.
:::
:::
Here are the steps that the clone command execute:
Here are the steps that the clone command execute:
1. Checks if `.borg` folder already exists.
1. Checks if the current folder (or any parent folder) is a git repository
2. Checks if the remote directory already exists and contains `data`, `config` and `archive_file` inside.
2. Checks if the results folder exits
3. Copies with rsync the remote directory into `.borg`
3. Checks if the remote dir exits and contains tar file
4. Creates the local config file `.borg/.gblkconfig` using the name of the archive
4. Initialize a .borg repository (`gblk init`)
5. Add a local remote called `origin` into the local config file
5. Add a local remote called `origin` into the local config file
6. Creates (or append in) a `.gitignore` file containing `.borg`, `.tmp`, `.mount` inside and a `results/.gitignore` containing `*` and `!.gitignore`.
6. Pull every archive from that remote path
7. Create hooks if needed
Here is the help of the clone commands:
Here is the help of the clone commands:
...
@@ -898,27 +893,20 @@ OPTIONS:
...
@@ -898,27 +893,20 @@ OPTIONS:
To copy the content of the `.borg` folder into another location you can use the `push` command like this:
To copy the content of the `.borg` folder into another location you can use the `push` command like this:
```sh
```sh
gblkt push KEY
gblk push KEY ARCHIVE
```
```
Where `KEY` is the name of a *global* or *local* remote. **Path are not supported!**.
Where `KEY` is the name of a *global* or *local* remote. **Path are not supported!**.
And `ARCHIVE` is an archive name defined in the `.borg` repository
Here are the steps that the push command execute:
Here are the steps that the push command execute:
1. Checks if the *remote folder* (defined by the remote) exists
1. Checks if the current directory (or a parent directory) is a git repository, contain a results folder and a `.borg` repository
2. Checks if the *remote archive folder* (folder that will contains the archives) exits. It corresponds to the *path* of the remote and the name of the folder containing the `.borg` folder.
2. Checks if the *remote folder* (defined by the remote) exists
3. If it doesn't exists:
3. Checks if the *remote folder* already contains the archive we want to push. If it exists, stop the process
- go to step 5
4. Creates a tar archives and copy it in the local or a remote filesystem. If the tar archive should be exported to a remote file system, then the tar file is created locally (in .tmp folder at the project root) and then exported remotely using rsync.
4. If it exists:
- Checks if the borg id is the same between the remote and the local borg archive. Stops the command if not.
- gblk uses the `archive_list` file on the *remote archive folder* to compare it with the current archive list saved and show to the users the differences between the remote and the local archive lists.
- Then it asks the user whether or not to continue the push.
5. Copies with rsync the content of the current `.borg` folder into the *remote archive folder*
::: {.callout-warning}
The pull command can delete s old archive inside it !
:::
# Pull command
# Pull command
...
@@ -927,40 +915,50 @@ The pull commands allow to replace the content of your `.borg` archive into a re
...
@@ -927,40 +915,50 @@ The pull commands allow to replace the content of your `.borg` archive into a re
It can be used with the following command:
It can be used with the following command:
```sh
```sh
gblk pull KEY
gblk pull KEY ARCHIVES
```
```
Where `KEY` is the name of a *global* or *local* remote. **Path are not supported!**.
Where `KEY` is the name of a *global* or *local* remote. **Path are not supported!**.
And `ARCHIVE` is an archive name defined in the `.borg` repository
Here are the steps that the pull command execute:
Here are the steps that the pull command execute:
1. Checks if the *remote folder* (defined by the remote) exists
2. Checks if the *remote archive folder* (folder that will contains the archives) exits. If it doesn't, stops the pull.
1. Checks if the current directory (or a parent directory) is a git repository, contain a results folder and a `.borg` repository
3. Checks if the borg id is the same between the remote and the local borg archive folder. Stops the pull if not.
2. Checks if the *remote folder* (defined by the remote) exists and contains the archive to pull
4. gblk uses the `archive_list` file on the *remote archive folder* to compare it with the current archive list saved and show to the users the differences between the remote and the local archive lists.
3. Check if the archive to pull is not already defined in our `.borg` folder. If it is, execute the next step
5. Asks the user whether or not to continue the pull
4. Import the archive in our `.borg` repository
6. Saves and clean `.borg` folder. The content of the old `.borg` folder is saved inside the `.tmp` folder and can be recovered with the `gblk restore` command if something goes wrong afterward.
7. Copies with rsync the content of the *remote archive folder* into the current `.borg` folder.
8. Removes borg manifest timestamp and the cache associated to the current project archive folder.
::: {.callout-note}
::: {.callout-note}
The pull command saves the inital content of the `.borg` folder inside the `.tmp` directory. **Remember to delete it if the pull is sucessful**
You can you the `--all` flag to pull all archives in the remote directory instead of one particular archives.
:::
:::
# Gblk restore command
Here is the help of the pull command:
This command can be used to restore your 'old' `.borg` folder the way it was before the last pull command. It can help to recover from an error that occured during the pull.
```sh
> gblk pull --help
USAGE:
gblk pull [OPTIONS] <KEY> [ARCHIVE]
To restore your old `.borg` folder, enter the command
ARGS:
<KEY>
The name of the remote to use
```sh
<ARCHIVE>
gblk restore
The name of the archive to pull
OPTIONS:
-a, --all
An option indicating to download all archive not already present in .borg folder
-h, --help
Print help information
```
```
# Gblk clean command
# Gblk clean command
After a pull or after using `gblk mount` with the `--diff` flag, temporary files are stored inside the `.tmp` directory. You can remove those with the command:
@@ -1230,14 +1230,14 @@ If you work with other people you might have trouble: if you pull a gblk remote
...
@@ -1230,14 +1230,14 @@ If you work with other people you might have trouble: if you pull a gblk remote
When you produce some results for a particular project, you may want to *share them with others* who are working with you. With gblk you can share your `.borg` repository with others or update it with chnages done by others.
When you produce some results for a particular project, you may want to *share them with others* who are working with you. With gblk you can share your `.borg` repository with others or update it with chnages done by others.
::: {.callout-note}
::: {.callout-important}
The choice of sharing the complete archive folder `.borg` was made to always benefits from borg's deduplication
When pushing archives to a remote folder, you lose the file deduplication between them. So make sure to push only the archives you want to share with other.
:::
:::
. . .
. . .
::: {.callout-warning}
::: {.callout-important}
The way of *duplicating and storing an archive folder* is not the way *borg was intended to be used*. Indeed the replacement of a borg repository by an older one *is considered suspicious* and **borg blocks the executions of its commands for this archive**. See [this page](https://borgbackup.readthedocs.io/en/stable/faq.html?highlight=attack#this-is-either-an-attack-or-unsafe-warning) for details.
When pulling archives from a remote folder, you can pull archives produced by others for an unrelated project or even **tar file** not produced by borg. Make sure to specify the good remote path for your project.
- `[HOSTNAME:]`: Optional, can be defined as `USER@HOSTNAME`. Hosts defined in `ssh config file` can be used
- `[HOSTNAME:]`: Optional, can be defined as `USER@HOSTNAME`. Hosts defined in `ssh config file` can be used
- `PATH`: path of a gblk archive folder. The `PATH` must point to a folder with this structure (corresponding to a borg remote archive):
- `PATH`: the path of a gblk archive folder. The `PATH` must point to a folder containg tar files:
```sh
```sh
PATH
PATH
├── archive_list
├── tar_file1
├── config
├── ...
├── data/
└── tar_filen
└── README
```
```
. . .
. . .
...
@@ -1384,79 +1383,45 @@ clone must be used with a *path* and *not a remote name*.
...
@@ -1384,79 +1383,45 @@ clone must be used with a *path* and *not a remote name*.
Here are the steps that the clone command execute:
Here are the steps that the clone command execute:
:::: {.fragment .fade-in .small-margin}
:::: {.fragment .fade-in .small-margin}
1. Checks if `.borg` folder already exists.
1. Checks if the current folder (or any parent folder) is a git repository
::::
::::
:::: {.fragment .fade-in .small-margin}
:::: {.fragment .fade-in .small-margin}
2. Checks if the remote directory already exists and contains `data`, `config` and `archive_file` inside.
2. Checks if the results folder exits
::::
::::
:::: {.fragment .fade-in .small-margin}
:::: {.fragment .fade-in .small-margin}
3. Copies with rsync the remote directory into `.borg`
3. Checks if the remote dir exits and contains tar file
::::
::::
:::: {.fragment .fade-in .small-margin}
:::: {.fragment .fade-in .small-margin}
4. Creates the local config file `.borg/.gblkconfig` using the name of the archive
4. Initialize a .borg repository (`gblk init`)
::::
::::
:::: {.fragment .fade-in .small-margin}
:::: {.fragment .fade-in .small-margin}
5. Add a local remote called `origin` into the local config file
5. Add a local remote called `origin` into the local config file
::::
::::
:::: {.fragment .fade-in}
:::: {.fragment .fade-in}
6. Creates a `.gitignore` and a `results/.gitignore`
6. Pull every archive from that remote path
:::: {.columns}
::: {.column width="50%"}
```sh
# .gitignore
.borg
.tmp
.mount
```
:::
::: {.column width="50%"}
```sh
# results/.gitignore
*
!.gitignore
```
:::
::::
::::
:::: {.fragment .fade-in}
7. Create hooks if needed
::::
## gblk push
## gblk push
To copy the content of the `.borg` folder into another location you can use the `push` command like this:
To copy the content of the `.borg` folder into another location you can use the `push` command like this:
```sh
```sh
gblkt push KEY
gblk push KEY ARCHIVE
```
```
Where `KEY` is the name of a *global* or *local* remote. **Path are not supported!**.
Where:
- `KEY` is the name of a *global* or *local* remote. **Path are not supported!**
::: {.callout-warning}
- `ARCHIVE` is an archive name defined in the `.borg` repository
The push command might deletes old archives inside the remote archive folder folder !
:::
## gblk push {.unnumbered .unlisted}
::: {.callout layout="minimal"}
::: {.callout layout="minimal"}
Here are the steps that the push command execute:
Here are the steps that the push command execute:
::: {.incremental}
::: {.incremental}
1. Checks if the *remote folder* (defined by the remote) exists
1. Checks if the current directory (or a parent directory) is a git repository, contain a results folder and a `.borg` repository
2. Checks if the *remote archive folder* (folder that will contains the archives) exits. It corresponds to the *path* of the remote and the name of the folder containing the `.borg` folder.
2. Checks if the *remote folder* (defined by the remote) exists
3. If it doesn't exists:
3. Checks if the *remote folder* already contains the archive we want to push. If it exists, stop the process
- go to step 5
4. Creates a tar archives and copy it in the local or a remote filesystem. If the tar archive should be exported to a remote file system, then the tar file is created locally (in .tmp folder at the project root) and then exported remotely using rsync.
4. If it exists:
- Checks if the borg id is the same between the remote and the local borg archive. Stops the command if not.
- gblk uses the `archive_list` file on the *remote archive folder* to compare it with the current archive list saved and show to the users the differences between the remote and the local archive lists.
- Then it asks the user whether or not to continue the push.
5. Copies with rsync the content of the current `.borg` folder into the *remote archive folder*
:::
:::
:::
:::
...
@@ -1467,50 +1432,34 @@ The pull commands allow to replace the content of your `.borg` archive into a re
...
@@ -1467,50 +1432,34 @@ The pull commands allow to replace the content of your `.borg` archive into a re
It can be used with the following command:
It can be used with the following command:
```sh
```sh
gblk pull KEY
gblk pull KEY ARCHIVES
```
```
Where `KEY` is the name of a *global* or *local* remote. **Path are not supported!**.
Where:
- `KEY` is the name of a *global* or *local* remote. **Path are not supported!**.
::: {.callout-note}
- `ARCHIVE` is an archive name defined in the `.borg` repository
The pull commands saves the inital content of the `.borg` folder inside the `.tmp` directory. **Remember to delete it if the pull is sucessful**
:::
## gblk pull {.unnumbered .unlisted}
::: {.callout layout="minimal"}
::: {.callout layout="minimal"}
Here are the steps that the pull command execute:
Here are the steps that the pull command execute:
::: {.incremental}
::: {.incremental}
1. Checks if the *remote folder* (defined by the remote) exists
1. Checks if the current directory (or a parent directory) is a git repository, contain a results folder and a `.borg` repository
2. Checks if the *remote archive folder* (folder that will contains the archives) exits. If it doesn't, stops the pull.
2. Checks if the *remote folder* (defined by the remote) exists and contains the archive to pull
3. Checks if the borg id is the same between the remote and the local borg archive folder. Stops the pull if not.
3. Check if the archive to pull is not already defined in our `.borg` folder. If it is, execute the next step
4. gblk uses the `archive_list` file on the *remote archive folder* to compare it with the current archive list saved and show to the users the differences between the remote and the local archive lists.
4. Import the archive in our `.borg` repository
5. Asks the user whether or not to continue the pull
6. Saves and clean `.borg` folder. The content of the old `.borg` folder is saved inside the `.tmp` folder and can be recovered with the `gblk restore` command if something goes wrong afterward.
7. Copies with rsync the content of the *remote archive folder* into the current `.borg` folder.
8. Removes borg manifest timestamp and the cache associated to the current project archive folder.
:::
:::
:::
:::
## gblk restore
::: {.callout-note}
You can you the `--all` flag to pull all archives in the remote directory instead of one particular archives.
This command can be used to restore your 'old' `.borg` folder the way it was before the last pull command. It can help to recover from an error that occured during the pull.
:::
To restore your old `.borg` folder, enter the command
```sh
gblk restore
```
<p><p>
## gblk clean
## gblk clean
After a `pull` or after using `gblk mount` with the `--diff` flag, temporary files are stored inside the `.tmp` directory. You can remove those with the command:
After a `push` or after using `gblk mount` with the `--diff` flag, temporary files are stored inside the `.tmp` directory. You can remove those with the command: