Borg User Manual

Table of Contents

Next: , Up: (dir)   [Contents][Index]

Borg User Manual

The Borg assimilate Emacs packages as Git submodules. Borg is a bare-bones package manager for Emacs packages.

This manual is for Borg version 3.2.0 (v3.2.0-7-g541f384+1).

Copyright (C) 2016-2021 Jonas Bernoulli <jonas@bernoul.li>

You can redistribute this document and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.


Next: , Previous: , Up: Top   [Contents][Index]

1 Introduction

The Borg assimilate Emacs packages as Git submodules.

Borg is a bare-bones package manager for Emacs packages. It provides only a few essential features and should be combined with other tools such as Magit, epkg, use-package, and auto-compile.

Borg assimilates packages into the ~/.emacs.d repository as Git submodules. An assimilated package is called a drone and a borg-based ~/.emacs.d repository is called a collective.

It is possible to clone a package repository without assimilating it. A cloned package is called a clone.

To learn more about this project, also read the blog post 1 in which it was announced.


Next: , Previous: , Up: Top   [Contents][Index]

2 Installation

Borg can be used by itself, in which case it has to be bootstrapped. This is how Borg originally was intended to be used exclusively but nowadays you may also choose to use it merely as a secondary package manager, in which case Borg itself is installed using Package.


Next: , Up: Installation   [Contents][Index]

2.1 Use as secondary package manager

To use Borg as a secondary package manager alongside Package, begin by installing the former using the latter. Borg is only available from Melpa, so you have to add that first.

(add-to-list 'package-archives
             (cons "melpa" "https://melpa.org/packages/")
             t)

Then you have to M-x package-refresh-contents RET before you can M-x package-install RET borg RET. Doing that should add a verbose variant of this to your init file:

(custom-set-variables
 '(package-selected-packages '(borg)))

Then you have to make the two package managers aware of one another by replacing the call to package-initialize in your init file with this:

(if (require 'borg-elpa nil t)
    (borg-elpa-initialize)
  (package-initialize))

Just like Package (aka Elpa) defaults to installing packages in elpa/, Borg defaults to installing them in borg/. (When using both package managers, that is. If used by itself, then Borg defaults to using lib/.) If you want to use another directory, then you can do so by setting the Git variable borg.drones-directory.

You should also add a Makefile containing:

BORG_SECONDARY_P = true
include $(shell find -L elpa -maxdepth 1 -regex '.*/borg-[.0-9]*' |\
  sort | tail -n 1)/borg.mk

Despite its title, many things mentioned in the next section are relevant even when Borg was installed the way we just did. Just saying.


Previous: , Up: Installation   [Contents][Index]

2.2 Bootstrapping

(As mentioned in the next section you can use a location other than ~/.emacs.d. Never-the-less most of the subsequent examples just talk about ~/.emacs.d and if you use something else, then obviously have to substitute that. The same applies to the string lib and the variables user-init-file and user-emacs-directory, which also might not be appropriate depending on your choices.)


Next: , Up: Bootstrapping   [Contents][Index]

2.2.1 Bootstrapping using a seed

To get started clone the repository of the emacs.g collective. A "collective" is a starter-kit and/or configuration seed that relies on Borg as the package manager. Most users end up using emacs.g merely as a bootstrapping seed and do not merge upstream changes after that.

This collective already assimilates a few drones in addition to borg itself, namely magit, epkg, use-package, auto-compile, git-modes and diff-hl, as well as their dependencies. These drones are not required by borg but their use is highly recommended.

Clone the emacs.g repository to either ~/.emacs.d, or for testing purposes to any other location. This repository contains a Makefile that imports lib/borg/borg.mk and defines an additional target whose purpose is to make that file and lib/borg/borg.sh available. Run make bootstrap-borg to clone the borg repository. That does not completely setup the borg repository but it makes the latest version of the mentioned files available. Now that these files are available you can run make bootstrap to get and configure all submodules (including the borg submodule) and to build all drones.

git clone git@github.com:emacscollective/emacs.g.git ~/.emacs.d
cd ~/.emacs.d
make bootstrap-borg
make bootstrap

If you cloned to somewhere other than ~/.emacs.d, then you can use that configuration using emacs -Q --load /path/to/emacs.g/init.el.

For drones whose upstream repositories are located on Github or Gitlab the emacs.g collective uses the ssh protocol by default, which is a problem if you don’t have accounts there and have not properly setup your keys. See Using https URLs.

When running the bootstrap target you might get errors about certain commits not being available, but fear not! In Missing commits and repositories we discuss these complications.

During package compilation you may notice the submodules relating to those packages become dirty due to the compilation outputs not being ignored in those submodules. For this reason it is useful to ignore these outputs globally, for example in your ~/.config/git/ignore file:

*.elc
*-autoloads.el
dir

You may discover more things that you’ll want to ignore this way as you use borg.


Next: , Previous: , Up: Bootstrapping   [Contents][Index]

2.2.2 Bootstrapping from scratch

If you don’t want to base your configuration on the emacs.g starter-kit described in the previous section, then you have to do a few things manually.

git init ~/.emacs.d
cd ~/.emacs.d

By default Borg installs packages inside the lib/ subdirectory, but since you are starting from scratch, you may choose something else by setting the Git variable borg.drones-directory locally for this repository.

Then you should add a Makefile containing at least:

DRONES_DIR = $(shell git config "borg.drones-directory" || echo "lib")

-include $(DRONES_DIR)/borg/borg.mk

bootstrap-borg:
        @git submodule--helper clone --name borg --path $(DRONES_DIR)/borg \
        --url git@github.com:emacscollective/borg.git
        @cd $(DRONES_DIR)/borg; git symbolic-ref HEAD refs/heads/master
        @cd $(DRONES_DIR)/borg; git reset --hard HEAD

Now you are probably tempted to run make bootstrap-borg, but that is for bootstrapping from a seed, and what we are doing right now is to bootstrap from scratch. In the process we are creating a seed but we are not there yet. Instead run this:

git submodule add --name borg git@github.com:emacscollective/borg.git lib/borg

Now that borg is available we can build all the assimilated packages (currently just borg itself) using make bootstrap.

Now it is time to tell Emacs to initialize Borg instead of Package by adding a simple init.el file containing at least:

(when (< emacs-major-version 27)
  (setq package-enable-at-startup nil))
(add-to-list 'load-path (expand-file-name "lib/borg" user-emacs-directory))
(require 'borg)
(borg-initialize)

Beginning with Emacs 27.1, package-enable-at-startup has to be disabled earlier, in early-init.el:

(setq package-enable-at-startup nil)

Now you could create the initial commit but you could also delay that.

git commit -m "Assimilate borg"

Now it is time to assimilate some other essential packages. You could do so using M-x borg-assimilate, but you would quickly notice that doing so without the help of the epkg package is quite cumbersome, so lets manually install that and its dependency first:

git submodule add --name closql git@github.com:emacscollective/closql.git lib/closql
git submodule add --name emacsql git@github.com:skeeto/emacsql.git lib/emacsql
git submodule add --name epkg git@github.com:emacscollective/epkg.git lib/epkg
git config -f .gitmodules submodule.emacsql.no-byte-compile emacsql-pg.el
echo /epkgs/ >> .gitignore
git add .gitignore .gitmodules
make all
git commit -m "Assimilate epkg and dependencies"

Once you have done that and restarted Emacs, then you can Magit using Borg as described in Assimilation. As part of that you should tell Magit status buffers to display submodules:

(with-eval-after-load 'magit
  (magit-add-section-hook 'magit-status-sections-hook
                          'magit-insert-modules
                          'magit-insert-stashes
                          'append))

Finally (look, nobody forced you to do this from scratch ;-) I strongly suggest that you make yourself familiar with my auto-compile package.


Next: , Previous: , Up: Bootstrapping   [Contents][Index]

2.2.3 Migrating a legacy configuration

If you are currently using Package and want to gently ease into using Borg alongside that, then you can proceed as described in Use as secondary package manager.

If on the other hand you are already manually using Git modules, then you should proceed as described in Bootstrapping from scratch. Obviously "from scratch" does not apply this time around, so just skip steps like git init.


Next: , Previous: , Up: Bootstrapping   [Contents][Index]

2.2.4 Using your configuration on another machine

Getting started using your existing configuration on another machine works the same way as described in Bootstrapping using a seed. The only difference is that instead of starting by cloning someone else’s repository you start by cloning your own repository.


Next: , Previous: , Up: Bootstrapping   [Contents][Index]

2.2.5 Using https URLs

For drones whose upstream repositories are located on Github or Gitlab the emacs.g collective uses the ssh protocol by default, which is a problem if you don’t have accounts there and have not properly setup your keys.

Luckily this can easily be fixed using the following global rules.

git config --global url.https://github.com/.insteadOf git@github.com:
git config --global url.https://gitlab.com/.insteadOf git@gitlab.com:

If you don’t want to configure this globally, then you can also configure Borg itself to prefer the https URLS.

(setq borg-rewrite-urls-alist
      '(("git@github.com:" . "https://github.com/")
        ("git@gitlab.com:" . "https://gitlab.com/")))

This does not affect packages that have already been assimilated. During bootstrapping you have to change the URLs for packages that are assimilated by default.

cd ~/.emacs.d
sed -i "s|git@github.com:|https://github.com/|g" .gitmodules
sed -i "s|git@gitlab.com:|https://gitlab.com/|g" .gitmodules
git commit -m "Use https URLs for Github and Gitlab"

If you have already run make bootstrap, then you also have to edit .git/config.

cd ~/.emacs.d
sed -i "s|git@github.com:|https://github.com/|g" .git/config
sed -i "s|git@gitlab.com:|https://gitlab.com/|g" .git/config

Previous: , Up: Bootstrapping   [Contents][Index]

2.2.6 Missing commits and repositories

While bootstrapping it can happen that a commit or even a complete repository is missing. That can be scary but understanding what is going on and how to deal with it goes a long way.

So what does it look like when that happens? Here are two typical examples that you might see when running make bootstrap.

Some commit is missing:

--- [hl-todo] ---

Cloning into '/home/you/.emacs.d/lib/hl-todo'...
remote: Enumerating objects: 44, done.
remote: Counting objects: 100% (44/44), done.
remote: Compressing objects: 100% (29/29), done.
remote: Total 367 (delta 23), reused 36 (delta 15), pack-reused 323
Receiving objects: 100% (367/367), 71.98 KiB | 837.00 KiB/s, done.
Resolving deltas: 100% (158/158), done.
fatal: Could not parse object '02bda39bf2d2b77168255d9e49c78b95b0bd314a'.
futile: Checkout of '02bda39bf2d2b77168255d9e49c78b95b0bd314a' into submodule path 'lib/hl-todo' failed
HEAD is now at 3bba459 README.md: Cosmetics

Some repository is missing:

--- [paren-face] ---

Cloning into '/home/you/.emacs.d/lib/paren-face'...
ERROR: Repository not found.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:tarsius/broken-url.git' into submodule path '/home/you/.emacs.d/lib/paren-face' failed
futile: Clone of any remote into submodule path 'lib/paren-face' failed

Both of these packages are not essential and I recommend that you delay figuring out what is going on exactly and that you instead proceed bootstrapping the rest of your configuration. Once you are done with that, then you have a personalized Emacs setup that allows you to efficiently investigate the above issues.

To skip a package you have to "deinit" the submodule and also remove the empty directory that represents the no longer initialized module:

$ git submodule deinit lib/paren-face/
Cleared directory 'lib/paren-face'
error: could not lock config file .git/modules/paren-face/config: No such file or directory
warning: Could not unset core.worktree setting in submodule 'lib/paren-face'
Submodule 'paren-face' (git@github.com:tarsius/broken-url.git) unregistered for path 'lib/paren-face'
$ rm -r lib/paren-face
$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
        deleted:    lib/paren-face

no changes added to commit

Ignore the irrelevant warnings and that the package is now considered deleted.

Now make bootstrap has to resume where it left of, more or less. This make target is equal to these three commands:

$ git submodule init
$ lib/borg/borg.sh
$ make build

The first already succeeded at this point and the second failed somewhere along the way, so lets run that again. It will make some noise about the packages that is has already dealt with, but you can just ignore that. Then it should silently skip over the package that you have just disabled and proceed with the rest, proceeding towards the end of the alphabet.

So now that everything is peachy you can start Emacs and look into what went wrong. Here are some things that could have happened and how to deal with them.

The commit that you have referenced was removed from the upstream repository. Pick another commit instead and resume the bootstrap.

$ git submodule update --init lib/hl-todo
Submodule 'hl-todo' (git@github.com:tarsius/hl-todo.git) registered for path 'lib/hl-todo'
fatal: remote error: upload-pack: not our ref 02bda39bf2d2b77168255d9e49c78b95b0bd314a
Fetched in submodule path 'lib/hl-todo', but it did not contain 02bda39bf2d2b77168255d9e49c78b95b0bd314a. Direct fetching of that commit failed.
$ cd lib/hl-todo
$ git reset --hard
$ cd ../..
$ git add lib/hl-todo
$ git commit -m '"Update" hl-todo'

The repository was moved. Update the url and resume the bootstrap.

$ git config --file .gitmodules submodule.paren-face.url git@github.com:tarsius/paren-face.git
$ git submodule update --init lib/paren-face
Submodule 'paren-face' (git@github.com:tarsius/paren-face.git) registered for path 'lib/paren-face'
Cloning into '/home/you/.emacs.d/lib/paren-face'...
Submodule path 'lib/paren-face': checked out 'eb4a51b8ef455e0914108105e7c0008d675457cc'
$ git add .gitmodules
$ git commit -m "paren-face: Update url"

The repository was removed and no copy remains available anywhere. This is the worst case scenario and unlikely to become a reality. Remove the package for good.

$ git rm lib/goner
$ git commit -m "Remove goner"

Next: , Previous: , Up: Top   [Contents][Index]

3 Startup

The user-init-file, ~/.emacs.d/init.el, has to contain a call to borg-initialize. It should also set package-enable-at-startup to nil unless you really want to use both borg and package at the same time.

Function: borg-initialize

This function initializes assimilated drones using borg-activate.

To skip the activation of the drone named DRONE, temporarily disable it by setting the value of the Git variable submodule.DRONE.disabled to true in ~/.emacs.d/.gitmodules.

Command: borg-activate clone

This function activates the clone named CLONE by adding the appropriate directories to the load-path and to Info-directory-list, and by loading the autoloads file, if it exists.

Unlike borg-initialize, this function ignores the Git variable submodule.DRONE.disabled and can be used to activate clones that have not been assimilated.


Next: , Previous: , Up: Top   [Contents][Index]

4 Assimilation

A third-party package is assimilated by adding it as a submodule and, if necessary, by configuring it in ~/.emacs.d/init.el. Built-in packages are assimilated merely by configuring them.

To begin the assimilation of a third-party package use the command borg-assimilate, which adds the package’s repository as a submodule and attempts to build the drone.

A safer alternative is to first clone the package without assimilating it, using borg-clone. This gives you an opportunity to inspect the cloned package for broken or malicious code before it gets a chance to run arbitrary code. Later you can proceed with the assimilation using borg-assimilate, or remove the clone using borg-remove.

Building the drone can fail, for example due to missing dependencies. Failure to build a drone is not considered as a failure to assimilate. If a build fails, then a buffer containing information about the issue pops up. If the failure is due to unsatisfied dependencies, then assimilate those too, and then build any drone which previously couldn’t be built by using the Emacs command borg-build or make lib/DRONE. Alternatively you can just rebuild everything using make build.

If you wish to avoid such complications, you should use the command epkg-describe-package before assimilating a package. Among other useful information, it also provides a dependency tree.

Once the packages have been added as submodules and the drones have been built, the assimilation is completed by creating an assimilation commit.

If you assimilate a single package, then it is recommended that you use a message similar to this:

Assimilate foo v1.0.0

Or if one or more dependencies had to be assimilated, something like:

Assimilate foo and dependencies

Assimilate foo v1.0.0
Assimilate bar v1.1.0
Assimilate baz v0.1.0

It’s usually a good idea not to assimilate unrelated packages in the same commit, but something like this might make sense:

Assimilate ido and extensions

Assimilate flx               v0.6.1-3-gae0981b
Assimilate ido-at-point      v1.0.0
Assimilate ido-ubiquitious   v3.12-2-g7354d98
Assimilate ido-vertical-mode v0.1.6-33-gb42e422
Assimilate smex               3.0-13-g55aaebe

The command borg-insert-update-message can be used to generate such commit messages.

C-c C-b [in git-commit-mode buffer]     (borg-insert-update-message)

This command insert information about drones that are changed in the index. Formatting is according to the commit message conventions described above.

Command: borg-assimilate package url &optional partially

This command assimilates the package named PACKAGE from URL.

If epkg is available, then only the name of the package is read in the minibuffer and the url stored in the Epkg database is used. If epkg is unavailable, the package is not in the database, or if a prefix argument is used, then the url too is read in the minibuffer.

If a negative prefix argument is used, then the submodule is added but the build and activation steps are skipped. This is useful when assimilating a package that requires special build steps. After configuring the build steps use borg-build to complete the assimilation.

Command: borg-clone package url

This command clones the package named PACKAGE from URL, without assimilating it. This is useful when you want to inspect the package before potentially executing malicious or broken code.

Interactively, when the epkg package is available, then the name is read in the minibuffer and the url stored in the Epkg database is used. If epkg is unavailable, the package is unknown, or when a prefix argument is used, then the url is also read in the minibuffer.

Command: borg-remove clone

This command removes the cloned or assimilated package named CLONE, by removing the working tree from borg-drones-directory, regardless of whether that repository belongs to an assimilated package or a package that has only been cloned for review using borg-clone. The Git directory is not removed.

Command: borg-build clone &optional activate

This command builds the clone named CLONE. Interactively, or when optional ACTIVATE is non-nil, then also activate the drone using borg-activate.

Function: borg-update-autoloads clone &optional path

This function updates the autoload file for the libraries belonging to the clone named CLONE in the directories in PATH. PATH can be omitted or contain file-names that are relative to the top-level of CLONE’s repository.

Function: borg-byte-compile clone &optional path

This function compiles the libraries for the clone named CLONE in the directories in PATH. PATH can be omitted or contain file-names that are relative to the top-level of CLONE’s repository.

Function: borg-makeinfo clone

This function generates the Info manuals and the Info index for the clone named CLONE.

Function: borg-batch-rebuild &optional quick

This function rebuilds all assimilated drones in alphabetic order, except for Org which is rebuilt first. After that it also builds the user init files using borg-batch-rebuild-init.

This function is not intended for interactive use; instead it is used by the Make target build described in the following section.

When optional QUICK is non-nil, then this function does not build drones for which submodule.DRONE.build-step is set, assuming that those are the drones that take longer to be built.

Function: borg-batch-rebuild-init

This function builds the init files specified by the Make variable INIT_FILES, or if that is unspecified init.el and LOGIN.el, where LOGIN is the value of the variable user-real-login-name. If a file does not exist, then it is silently ignored.

This function is not intended for interactive use; instead it is used by the Make targets build-init and (indirectly) build, which are described in Make targets.


Next: , Previous: , Up: Top   [Contents][Index]

5 Updating drones

Borg does not provide an update command. By not doing so, it empowers you to update to exactly the commit you wish to update to, instead of to "the" new version.

To determine the drones which you might want to update, visit the Magit status buffer of the ~/.emacs.d repository and press f m to fetch inside all submodules. After you have done so, and provided there actually are any modules with new upstream commits, a section titled "Modules unpulled from @{upstream}" appears.

Each subsection of that section represents a submodule with new upstream commits. Expanding such a subsection lists the new upstream commits. These commits can be visited by pressing RET, and the status buffer of a submodule can be visited by pressing RET while point is inside the heading of the respective submodule section. To return to the status buffer of ~/.emacs.d press q.

Inside the status buffer of a submodule, you can pull the upstream changes as usual, using F u. If you wish you can inspect the changes before doing so. And you can also choose to check out another commit instead of the upstream HEAD.

Once you have "updated" to a new commit, you should also rebuild the drone using the command borg-build. This may fail, e.g. due to new dependencies.

Once you have resolved all issues you should create an "update commit". You can either create one commit per updated drone or you can create a single commit for all updated drones, which ever you find more appropriate. However it is recommended that you use a message similar to:

Update foo to v1.1.0

Or for multiple packages:

Update 2 drones

Update foo to v1.1.0
Update bar to v1.2.1

The command borg-insert-update-message can be used to generate such commit messages.

To update the Epkg package database use the command epkg-update.


Next: , Previous: , Up: Top   [Contents][Index]

6 Patching drones

By using Borg you can not only make changes to assimilated packages, you can also keep track of those patches and share them with others.

If you created some commits in a drone repository and are the maintainer of the respective package, then you can just push your changes to the "origin" remote. You don’t have to do this every time you created some commits, but at important checkpoints, such as after creating a release, you should record the changes in the ~/.emacs.d repository. To do so proceed as described in Updating drones.

But for most packages you are not the maintainer and if you create commits for such drones, then you have to create a fork and push there instead. You should configure that remote as the push-remote using git config remote.pushDefault FORK, or by pressing b C M-p in Magit. After you have done that you can continue to pull from the upstream using F u in Magit and you can also push to your fork using P p.

Of course you should also occasionally record the changes in the ~/.emacs.d repository. Additionally, and ideally when you first fork a drone, you should also record information about your personal remote in the super-repository by setting submodule.DRONE.remote in ~/.emacs.d/.gitmodules.

Variable: submodule.DRONE.remote "NAME URL"

This variable specifies an additional remote named NAME that is fetched from URL. This variable can be specified multiple times. Note that "NAME URL" is a single value and that the two parts of that value are separated by a single space.

make bootstrap automatically adds all remotes that are specified like this to the DRONE repository by setting remote.NAME.url to URL and using the standard value for remote.NAME.fetch.

Variable: borg.pushDefault FORK

This variable specifies a name used for push-remotes. Because this variable can only have one value it is recommended that you use the same name, FORK, for your personal remote in all drone repositories in which you have created patches that haven’t been merged into the upstream repository (yet). A good value may be your username.

For all DRONES for which one value of submodule.DRONE.remote specifies a remote whose NAME matches FORK, make bootstrap automatically configures FORK to be used as the push-remote by setting remote.pushDefault to FORK.


Next: , Previous: , Up: Top   [Contents][Index]

7 Make targets

The following make targets are available by default. To use them you have to be in ~/.emacs.d in a shell. They are implemented in borg.mk, which is part of the borg package.

Command: help

This target prints information about the following targets.

Command: all
Command: build

This target rebuilds all assimilated drones in alphabetic order, except for Org which is rebuilt first. After that it also builds the user init files, like build-init does.

Command: build-init

This target builds the init files specified by the Make variable INIT_FILES, or if that is unspecified init.el and LOGIN.el, where LOGIN is the value of the variable user-real-login-name. If a file does not exist, then it is silently ignored.

If you publish your ~/.emacs.d repository but would like to keep some settings private, then you can do so by putting them in a file ~/.emacs.d/LOGIN.el. The downside of this approach is that you will have to somehow synchronize that file between your machines without checking it into Git.

Command: quick

This target builds most drones. Excluded are all drones for which the Git variable submodule.DRONE.build-step is set, assuming that those are the drones that take longer to build.

It also builds the init files as described above.

Command: lib/DRONE

This target builds the drone named DRONE.

Command: tangle-init

This target tangles (creates) init.el from init.org. You obviously don’t have to use such a file if you don’t want to.

Command: clean

This target removes all byte-code files inside ~/.emacs.d.

Command: clean-init

This target removes byte-code files for init files.

Command: bootstrap-borg

This target bootstraps borg itself.

Command: bootstrap

This target attempts to bootstrap the drones. To do so it runs git submodule init, borg.sh (which see), and make build.

If an error occurs during the borg.sh phase, then you can just run that command again to process the remaining drones. The drones that have already been bootstrapped or that have previously failed will be skipped. If a drone cannot be cloned from any of the known remotes, then you should temporarily remove it using git submodule deinit lib/DRONE. When done with borg.sh also manually run make build again.


Next: , Previous: , Up: Top   [Contents][Index]

8 Variables

Variable: borg.drones-directory DIRECTORY

This Git variable can be used to override the name of the directory that contains the drone submodules. If specified, the value has to be relative to the top-level directory of the repository.

Note that if you change the value of this variable, then you might have to additionally edit ~/.emacs.d/Makefile.

The values of the following variables are set at startup and should not be changed by the user.

Variable: borg-drones-directory

The value of this constant is the directory beneath which drone submodules are placed. The value is set based on the location of the borg library and the Git variable borg.drones-directory if set, and should not be changed.

Variable: borg-user-emacs-directory

The value of this constant is the directory beneath which additional per-user Emacs-specific files are placed. The value is set based on the location of the borg library and should not be changed. The value is usually the same as that of user-emacs-directory, except when Emacs is started with emacs -q -l /path/to/init.el.

Variable: borg-gitmodules-file

The value of this constant is the .gitmodules file of the super-repository.

The value of this Make variable has to be set in ~/.emacs.d/Makefile.

Variable: INIT_FILES

A list of init files to be build by the Make targets build and build-init. See Make targets.

The values of these borg-specific Git variables have to be set in the file ~/.emacs.d/.gitmodules. The variables borg.pushDefault and submodule.DRONE.remote are described in Patching drones.

Variable: borg.collective REMOTE

This variable specifies the name used for remotes that reference a repository that has been patched by the collective. If a NAME matches REMOTE, then it is configured as the upstream of the current branch of the respective DRONE.

If the file .hive-maint exists, then this variable has the same effect as borg.pushDefault. This special case is only useful for maintainers of the collective (but not for maintainers of individual drones).

Because most repositories used to maintain Emacs packages follow some common-sense conventions, Borg usually does not have to be told how to build a given drone. Building is done using borg-build, which in turn usually does its work using borg-update-autoloads, borg-byte-compile, and borg-makeinfo.

However some packages don’t follow the conventions either because they are too complex to do so, or for the sake of doing it differently. But in either case resistance is futile; by using the following variables you can tell Borg how to build such packages.

Variable: submodule.DRONE.build-step COMMAND

By default drones are built using the lisp functions borg-update-autoloads, borg-byte-compile, and borg-makeinfo, but if this variable has one or more values, then DRONE is built using these COMMANDs instead.

Each COMMAND can be one of the default steps, an S-expression, or a shell command. The COMMANDs are executed in the specified order.

If a COMMAND matches one of default steps, then it is evaluated with the appropriate arguments. Otherwise if the COMMAND begins with a parenthesis, then it is evaluated as a lisp expression. Otherwise it is assumed to be a shell command and executed with shell-command.

[submodule "mu4e"]
        path = lib/mu4e
        url = git@github.com:djcb/mu.git
        build-step = test -e ./configure || autoreconf -i
        build-step = ./configure
        build-step = make -C mu4e > /dev/null
        build-step = borg-update-autoloads
        load-path = mu4e

To skip generating "autoloads" (e.g. using use-package to create "autoloads" on the fly), just provide the required build steps to build the package, omitting borg-update-autoloads. Borg silently ignores a missing "autoloads" file during initialization (borg-initialize).

[submodule "multiple-cursors"]
        path = lib/multiple-cursors
        url = git@github.com:magnars/multiple-cursors.el.git
        build-step = borg-byte-compile

Note that just because a package provides a Makefile, you do not necessarily have to use it.

Even if make generates the Info file, you might still have to add borg-makeinfo as an additional build-step because the former might not generate an Info index file (named dir), which Borg relies on.

Variable: borg-build-shell-command

This variable can be used to change how shell commands specified by submodule.DRONE.build-step are run. The default value is nil, meaning that each build step is run unchanged using shell-command.

If the value is a string, then that is combined with each build step in turn and the results are run using shell-command. This string must contain either %s, which is replaced with the unchanged build step, or %S, which is replaced with the result of quoting the build step using shell-quote-argument.

If the value is a function, then that is called once with the drone as argument and must return either a string or a function. If the returned value is a string, then that is used as described above.

If the value returned by the first function is another function, then this second function is called for each build step with the drone and the build step as arguments. It must return a string or nil. If the returned value is a string, then that is used as described above.

Finally the second function may execute the build step at its own discretion and return nil to indicate that it has done so.

Notice that if the value of this variable is a function, this function must a) be defined in a drone; and b) be registered as an autoload. This is because build happens in a separate Emacs process started with -Q --batch, which only receives the name of the function.

Variable: submodule.DRONE.load-path PATH

This variable instructs borg-activate to add PATH to the load-path instead of the directory it would otherwise have added. Likewise it instructs borg-byte-compile to compile the libraries in that directory. PATH has to be relative to the top-level of the repository of the drone named DRONE. This variable can be specified multiple times.

Normally Borg uses elisp/ as the drone’s load-path, if that exists; otherwise lisp/, if that exists; or else the top-level directory. If this variable is set, then it overrides the default location. Therefore, to add an additional directory, you also have to explicitly specify the default location.

[submodule "org"]
        path = lib/org
        url = git://orgmode.org/org-mode.git
        build-step = make
        load-path = lisp
        load-path = contrib/lisp
        info-path = doc
Variable: submodule.DRONE.no-byte-compile PATH

This variable instructs borg-byte-compile to not compile the library at PATH. PATH has to be relative to the top-level of the repository of the drone named DRONE. This variable can be specified multiple times.

Sometimes a drone comes with an optional library which adds support for some other third-party package, which you don’t want to use. For example emacsql comes with a PostgreSQL back-end, which is implemented in the library emacsql-pg.el, which requires the pg package. The standard Borg collective emacs.g assimilates emacsql, for the sake of the epkg drone, which only requires the SQLite back-end. To avoid an error about pg not being available, emacs.g instructs Borg to not compile emacsql-pg.el. (Of course if you want to use the PostgreSQL back-end and assimilate pg, then you should undo that.)

Variable: submodule.DRONE.recursive-byte-compile BOOLEAN

Setting this variable to true instructs borg-byte-compile to compile DRONE’s directories recursively. This isn’t done by default because there are more repositories in which doing so would cause issues than there are repositories that would benefit from doing so.

Unfortunately many packages put problematic test files or (usually outdated) copies of third-party libraries into subdirectories. The latter is a highly questionable thing to do, but the former would be perfectly fine, if only the non-library lisp files did not provide a feature (which effectively turns them into libraries) and/or if a file named .nosearch existed in the subdirectory. That file tells functions such as normal-top-level-add-subdirs-to-load-path and borg-byte-compile to ignore the containing directory.

Variable: borg-byte-compile-recursive

Setting this variable to a non-nil value instructs borg-byte-compile to compile all drones recursively. Doing so is discouraged.

Variable: submodule.DRONE.info-path PATH

This variable instructs borg-initialize to add PATH to Info-directory-list. PATH has to be relative to the top-level of the repository of the drone named DRONE.

Variable: submodule.DRONE.no-makeinfo PATH

This variable instructs borg-makeinfo to not create an Info file for the Texinfo file at PATH. PATH has to be relative to the top-level of the repository of the drone named DRONE. This variable can be specified multiple times.

Variable: submodule.DRONE.disabled true|false

If the value of this variable is true, then it is skipped by borg-initialize.

Variable: borg-rewrite-urls-alist

An alist that can optionally be used to rewrite certain URLs. Each element has the form (ORIG . BASE). Each URL that starts with ORIG is rewritten to start with BASE instead. See Using https URLs.


Next: , Previous: , Up: Top   [Contents][Index]

9 Low-level functions

You normally should not have to use the following low-level functions directly. That being said, you might want to do so anyway if you build your own tools on top of Borg.

Function: borg-worktree clone

This function returns the top-level of the working tree of the clone named CLONE.

Function: borg-gitdir clone

This function returns the Git directory of the clone named CLONE.

It always returns BORG-USER-EMACS-DIRECTORY/.git/modules/CLONE, even when CLONE’s Git directory is actually located inside the working tree.

Function: borg-get clone variable &optional all

This function returns the value of the Git variable submodule.CLONE.VARIABLE defined in ~/.emacs.d/.gitmodules. If optional ALL is non-nil, then it returns all values as a list.

Function: borg-get-all clone variable

This function returns all values of the Git variable submodule.CLONE.VARIABLE defined in ~/.emacs.d/.gitmodules as a list.

Function: borg-load-path clone

This function returns the load-path for the clone named CLONE.

Function: borg-info-path clone &optional setup

This function returns the Info-directory-list for the clone named CLONE.

If optional SETUP is non-nil, then it returns a list of directories containing texi and/or info files. Otherwise it returns a list of directories containing a file named dir.

Function: borg-drones &optional include-variables

This function returns a list of all assimilated drones.

The returned value is a list of the names of the assimilated drones, unless optional INCLUDE-VARIABLES is non-nil, in which case elements of the returned list have the form (NAME . PLIST).

PLIST is a list of paired elements. Property names are symbols and correspond to a VARIABLE defined in the Borg repository’s .gitmodules file as submodule.NAME.VARIABLE.

Each property value is either a string or a list of strings. If INCLUDE-VARIABLES is raw then all values are lists. Otherwise a property value is only a list if the corresponding property name is a member of borg--multi-value-variables. If a property name isn’t a member of borg--multi-value-variables but it does have multiple values anyway, then it is undefined with value is included in the returned value.

Function: borg-clones

This function returns a list of all cloned packages.

The returned value includes the names of all drones, as well as the names of all other repositories that are located directly inside borg-drones-directory but aren’t tracked as submodules.

Function: borg-read-package prompt &optional edit-url

This function reads a package name and the url of its upstream repository from the user, and returns them as a list.

When the epkg package is available, then the user is only prompted for the name of the package, and the upstream url is retrieved from the Epkg database. If the package isn’t in the database then the url has to be provided by the user. If optional EDIT-URL is non-nil, then the url from the database, if any, is provided as initial input for the user to edit.

PROMPT is used when prompting for the package name.

Function: borg-read-clone prompt

This function reads the name of a cloned package from the user.

There exist a few more functions, but those are considered to be internal and might therefore change in incompatible ways without that being noted in the change log.

Function: borg--maybe-absorb-gitdir pkg
Function: borg--maybe-reuse-gitdir pkg
Function: borg--restore-worktree pkg
Function: borg--call-git pkg &rest args
Function: borg--expand-load-path drone path
Function: borg--sort-submodule-sections

Next: , Previous: , Up: Top   [Contents][Index]

Appendix A Command Index

Jump to:   A   B   C   H   L   Q   T  
Index Entry  Section

A
all: Make targets

B
bootstrap: Make targets
bootstrap-borg: Make targets
borg-activate clone: Startup
borg-assimilate package url &optional partially: Assimilation
borg-build clone &optional activate: Assimilation
borg-clone package url: Assimilation
borg-insert-update-message: Assimilation
borg-remove clone: Assimilation
build: Make targets
build-init: Make targets

C
clean: Make targets
clean-init: Make targets

H
help: Make targets

L
lib/DRONE: Make targets

Q
quick: Make targets

T
tangle-init: Make targets

Jump to:   A   B   C   H   L   Q   T  

Next: , Previous: , Up: Top   [Contents][Index]

Appendix B Function Index

Jump to:   A   B   C   H   L   Q   T  
Index Entry  Section

A
all: Make targets

B
bootstrap: Make targets
bootstrap-borg: Make targets
borg--call-git: Low-level functions
borg--expand-load-path: Low-level functions
borg--maybe-absorb-gitdir: Low-level functions
borg--maybe-reuse-gitdir: Low-level functions
borg--restore-worktree: Low-level functions
borg--sort-submodule-sections: Low-level functions
borg-activate: Startup
borg-assimilate: Assimilation
borg-batch-rebuild: Assimilation
borg-batch-rebuild-init: Assimilation
borg-build: Assimilation
borg-byte-compile: Assimilation
borg-clone: Assimilation
borg-clones: Low-level functions
borg-drones: Low-level functions
borg-get: Low-level functions
borg-get-all: Low-level functions
borg-gitdir: Low-level functions
borg-info-path: Low-level functions
borg-initialize: Startup
borg-load-path: Low-level functions
borg-makeinfo: Assimilation
borg-read-clone: Low-level functions
borg-read-package: Low-level functions
borg-remove: Assimilation
borg-update-autoloads: Assimilation
borg-worktree: Low-level functions
build: Make targets
build-init: Make targets

C
clean: Make targets
clean-init: Make targets

H
help: Make targets

L
lib/DRONE: Make targets

Q
quick: Make targets

T
tangle-init: Make targets

Jump to:   A   B   C   H   L   Q   T  

Previous: , Up: Top   [Contents][Index]

Appendix C Variable Index

Jump to:   B   I   S  
Index Entry  Section

B
borg-build-shell-command: Variables
borg-byte-compile-recursive: Variables
borg-drones-directory: Variables
borg-gitmodules-file: Variables
borg-rewrite-urls-alist: Variables
borg-user-emacs-directory: Variables
borg.collective: Variables
borg.drones-directory: Variables
borg.pushDefault: Patching drones

I
INIT_FILES: Variables

S
submodule.DRONE.build-step: Variables
submodule.DRONE.disabled: Variables
submodule.DRONE.info-path: Variables
submodule.DRONE.load-path: Variables
submodule.DRONE.no-byte-compile: Variables
submodule.DRONE.no-makeinfo: Variables
submodule.DRONE.recursive-byte-compile: Variables
submodule.DRONE.remote: Patching drones

Jump to:   B   I   S  

Footnotes

(1)

https://emacsair.me/2016/05/17/assimilate-emacs-packages-as-git-submodules.