@@ -25,6 +25,7 @@ the `--diff` option of the `gblk mount` command.
...
@@ -25,6 +25,7 @@ the `--diff` option of the `gblk mount` command.
You can optionally install [ImageMagick](https://imagemagick.org/script/index.php). ImageMagick aims to perform various operation on images. This tool is needed if you plan to create diff of image with gblk with the `--diff` option of the `gblk mount` command.
You can optionally install [ImageMagick](https://imagemagick.org/script/index.php). ImageMagick aims to perform various operation on images. This tool is needed if you plan to create diff of image with gblk with the `--diff` option of the `gblk mount` command.
If you want to share your borg folder whith gblk `clone`, `push` and `pull`, `rsync` must be installed on your computer and on remote machines where you want to store/share you results. `ssh` program must be installed to send the archived results to remote servers.
## Installation
## Installation
...
@@ -646,4 +647,236 @@ Filtering options:
...
@@ -646,4 +647,236 @@ Filtering options:
-s, --stats Print statistics for the deleted archive
-s, --stats Print statistics for the deleted archive
--force Force deletion of corrupted archives, use `--force --force` in case `--force`
--force Force deletion of corrupted archives, use `--force --force` in case `--force`
does not work
does not work
```
```
\ No newline at end of file
## sharing gblk repositories
When you produce some results for a particular project, you may want to share them with others who are working with you. With gblk you can share your `.borg` repository with other or update it with chnages done by others.
> Note: The choice of sharing the complete archive folder `.borg` was made to always benefits from borg's deduplication
> **⚠ Warning**: The way of duplicating and storing an archive folder is not the way borg was intended to be used. Indeed the replacment of a borg repository by an older one is considered suspicious and borg blocks the executions of its commands for this archive. See [this page](https://borgbackup.readthedocs.io/en/stable/faq.html?highlight=attack#this-is-either-an-attack-or-unsafe-warning) for details.
### Remotes
In order to be able to share your archived results in the `.borg` folder, you use remotes like in git. Remotes can be defined *globally* or *locally*:
-*local* remotes are only defined for a given project
-*global* remotes are available for any gblk projects. It can be usefull if you often want to store your gblk results on a same location.
> Note: If a *global* remote have the same name as a *local* remote, only the local remote can be used for this project.
The file that will store the local remotes is `.borg/.gblkconfig` inside the root of your project folder.
The file that will sotre the global remote is `~/.gblkconfig` in your `HOME` folder
A remote is defined by a *name* and a *path*. The *name* is used to identify the remotes and the *path* this the path pointed by the remote. An *path* on a remote server can be defined like this `[USER@]HOSTNAME:PATH`. Note that hostnames defined in `ssh config file` can be used here. The *path* must point to an existing directory.
Below is the help of the remote command:
```sh
gblk remote --help
```
It display the following message.
```
gblk-remote
This command can be used to add a new remote for push and pull commands
USAGE:
gblk remote <SUBCOMMAND>
OPTIONS:
-h, --help Print help information
SUBCOMMANDS:
add Add or update a remote in the configuration file
help Print this message or the help of the given subcommand(s)
rm Remove the configuration of a given key in prune
show Display globally and locally defined remotes
```
### Creation of a new remote
To add a new local remote you can use the following command:
```sh
gblk remote add KEY VALUE
```
Where `KEY` Is the name of the remote and `VALUE` The URL it points to.
To define a globally defined remote you can enter the following commands:
```sh
gblk remote add KEY VALUE --global
```
> Note: This command can also be used to uptade the value of an existing commands
### Removing a remote
To remove a remote named `KEY` you can use this command
```sh
gblk remote rm KEY
```
You can add the flag `--global` at the end of the previous command to remove a globally defined remote.
### Displaying remotes
To display remotes you can use the command:
```sh
gblk remote show
```
It will display *locally* and *globally* defined remotes. Note that only unmasked `globally` remotes are displayed (e.g the global remotes that don't share a name with any local remotes).
Here is an example of what the commands can display:
```
local: /path/to_destination_folder
psmn: psmn:/home/user/test_folder (global)
```
You can see that *globally* defined remote are tagged with `(global)`.
## gblk `clone`, `push` and `pull` commands
## Clone command
This command can be used after a `git clone` command. If a gblk archive folder is available for the project, then you can execute the following command:
```sh
gblk clone [HOSTNAME]:PATH
```
Where `PATH` is the path of a gblk archive folder. The `PATH` must point to a folder with this structure (corresponding to a borg remote archive):
```
PATH
├── archive_list
├── config
├── data/
└── README
```
> Note: clone muste be used with a *path* and not a remote name.
Here are the steps that the clone command execute:
1. Checks if `.borg` folder already exists.
2. Checks if the remote directory already exists and contains `data`, `config` and `archive_file` inside.
3. Copies with rsync the remote directory into `.borg`
4. Creates the local config file `.borg/.gblkconfig` using the name of the archive
5. Add a local remote called `origin` into the local config file
6. Creates (or append in) a `.gitignore` file containing `.borg`, `.tmp`, `.mount` inside and a `results/.gitignore` containing `*` and `!.gitignore`.
7. Create hooks if needed
Here is the help of the clone commands:
```sh
gblk-clone
Clones a repository given a destination
USAGE:
gblk clone [OPTIONS] <PATH>
ARGS:
<PATH>
The path pointing to a borg folder
OPTIONS:
-c, --compression <COMPRESSION>
The compression to use automatically at each commit if hooks are created
[default: lz4]
-h, --help
Print help information
-H, --hooks
If specified, hooks are created inside `.git/hooks repository`
-m, --mode <MODE>
The checkout mode used by gblk automatically after a git checkout: soft or hard. This
option is only used if hooks are created. The hard mode will delete every file in your
results folder and extract those corresponding to the commit targeted by the checkout.
The soft mode will only update files that existed in the targeted checkout
[default: hard]
```
### Push command
To copy the content of the `.borg` folder into another location you can use the `push` command like this:
```sh
gblkt push KEY
```
Where `KEY` is the name of a *global* or *local* remote. **Path are not supported!**.
Here are the steps that the push command execute:
1. Checks if the *remote folder* (defined by the remote) exists
2. Checks if the *remote archive folder* (folder that will contains the archives) exits. It corresponds to the *path* of the remote and the name of the folder containing the `.borg` folder.
3. If it doesn't exists:
- go to step 5
4. If it exists:
- Checks if the borg id is the same between the remote and the local borg archive. Stops the command if not.
- gblk uses the `archive_list` file on the *remote archive folder* to compare it with the current archive list saved and show to the users the differences between the remote and the local archive lists.
- Then it asks the user whether or not to continue the push.
5. Copies with rsync the content of the current `.borg` folder into the *remote archive folder*
> **⚠ Warning**: The pull command can delete s old archive inside it !
## Pull command
The pull commands allow to replace the content of your `.borg` archive into a remote folder.
It can be used with the following command:
```sh
gblk pull KEY
```
Where `KEY` is the name of a *global* or *local* remote. **Path are not supported!**.
Here are the steps that the pull command execute:
1. Checks if the *remote folder* (defined by the remote) exists
2. Checks if the *remote archive folder* (folder that will contains the archives) exits. If it doesn't, stops the pull.
3. Checks if the borg id is the same between the remote and the local borg archive folder. Stops the pull if not.
4. gblk uses the `archive_list` file on the *remote archive folder* to compare it with the current archive list saved and show to the users the differences between the remote and the local archive lists.
5. Asks the user whether or not to continue the pull
6. Saves and clean `.borg` folder. The content of the old `.borg` folder is saved inside the `.tmp` folder and can be recovered with the `gblk restore` command if something goes wrong afterward.
7. Copies with rsync the content of the *remote archive folder* into the current `.borg` folder.
8. Removes borg manifest timestamp and the cache associated to the current project archive folder.
> Note: The pull command saves the inital content of the `.borg` folder inside the `.tmp` directory. **Remember to delte it if the pull is sucessful
## Gblk restore command
This command can be used to restore your 'old' `.borg` folder the way it was before the last pull command. It can help to recover from an error that occured during the pull.
To restore your old `.borg` folder, enter the command
```sh
gblk restore
```
## Gblk clean command
After a pull or after using `gblk mount` with the `--diff` flag, temporary files are stored inside the `.tmp` directory. You can remove those with the command: