| GITPKG(1) | General Commands Manual | GITPKG(1) |
gitpkg - export a Debian source package from nominated git revisions
gitpkg branch [origbranch]
If gitpkg is run in a git(1) repo with a single 'branch' specified, then it will do a git-archive export of that branch to the DEB_DIR directory. If the package is Debian native it will simply create a source package from it. If the package has a Debian version, then an orig tarball will be expected to already exist for it. If an orig tarball does not already exist then what happens next depends on the value of the gitpkg.create-fake-orig configuration option (described below).
If gitpkg is invoked with two branches specified, then the first branch will be exported as the unpacked complete source, while the second branch will be exported for the orig.tar.gz. This allows all local changes to the source to be recorded in the resulting diff.gz if a pristine upstream branch exists in the repository. If an orig tarball already exists for the version at 'branch' then what happens next depends on the value of the gitpkg.force-overwrite-orig configuration option (described below).
The 'branch' should always have a debian/ dir and may be any tree-ish object that is accepted by git-archive(1). The 'origbranch', if supplied, should usually not have a debian/ dir.
Almost all gitpkg configuration is handled using git-config(1) now. The following configuration options are supported:
If this option is set to 'true', then we will recursively export all submodules in the selected superproject revisions. If it is set to 'false', they will be treated as they were prior to gitpkg 0.31 and silently ignored. If this option is not set and there are submodules present in the revisions being exported, then the user will be prompted for the correct action to take.
At present this is an all or nothing selection, there is no way to selectively exclude only certain submodules from export, but if anyone ever has a real repo where that is the desired action then it should not be a difficult feature to add. As with everything git though, the range of possible use cases makes it folly to prematurely 'support' straw man use cases, and we should wait to see what other problems real use cases actually need solved.
Note that in a repo with submodules gitpkg must be run from a directory that is directly managed by the superproject, not one that belongs to a submodule. Any git commands run in directories that are managed by submodules will use the configuration for, and act on, only that submodule - and it is almost impossible to know or be sure which (of possibly many) containing superproject is the one the user really wanted to export. See the details of the gitpkg.allow-subproject-export option below for more on this.
Available to hook scripts as EXPORT_SUBMODULES.
So if we detect that gitpkg is invoked from a directory which is managed by a submodule then we need the user to tell us what they really wanted to do, and we can't rely on the treeish selected for export to be in any way persistently unique to just a single repo somewhere in the superproject tree.
If this option is not set, the user will be prompted to confirm whether they really intended to export just the submodule, with the option to abort and instead run gitpkg from the desired superproject.
If this option is set to 'true', then we will trust the user understands what they were doing, knows which subproject repo they invoked gitpkg in, and intends to export just that submodule (and possibly any other submodules that it in turn is a superproject for). And if they ever get that wrong, they get to keep all the pieces.
If this option is set to 'false', then we will consider it to always be an error to try and directly export the submodule it is set for and immediately fail out. Note that it must be set in the submodule configuration, as the superproject configuration will not be read.
In theory we could make this option indicate the root of the superproject that should be exported, but that just replaces an 'obvious' failure mode with a somewhat more insidious one that assumes an immutable relationship between them, with a silent awkward failure if that ever changes.
If some subproject really can or should be exported as a package independently of the superproject it is a submodule of, then best practice is probably to be exporting it (and actively working on it) from its own independent working directory, with only selected versions checked out for read-only use as a submodule. Especially in the case of Debian packages, where managing them as a submodule of a superproject would mean that you have debian directories and control files nested in the superproject source.
Available to hook scripts as ALLOW_SUBPROJECT_EXPORT. Though it's probably of little use to them, as it won't be set unless we are running in a submodule, and none of them will be run before the usual decision to abort execution.
gitpkg.orig-gz-opts
gitpkg.orig-xz-opts
gitpkg.orig-bz2-opts
User defined scripts can be invoked from a number of points during the package build process. They are sourced into gitpkg as bash shell snippets, in most cases in a subshell, so they can read state variables and perform external actions, but cannot alter the running configuration once a build is in progress. If a hook returns with a non-zero status, then gitpkg will be terminated. (Hooks that do terminate gitpkg should take some care not to leave too much of a mess, but also should leave enough clues intact for the user to diagnose and fix whatever the problem was. Useful and informative error messages should be barked to stderr before exiting in this way.)
Hook scripts may be installed on the host system outside of the repo tree, or sourced from version controlled files in the repo itself. Both methods have advantages and risks for different use cases. Hook scripts are activated by the local admin, by setting each relevant git-config(1) option with the path to the script to be executed. Paths may be absolute or relative to the directory which that hook is called from. If a hook is set, the script must exist when it is called. Care should be taken to only enable them for use by trusted source trees when hooking into files in the repo itself. Usually you should enable them on a per-repo basis with git-config(1) rather than at a --global or --system level.
You should avoid complicated in-package hook arrangements becoming essential for exporting your package source. If you need them to create a particular package correctly, and need strict version binding with the source being released, and they aren't useful to any other package at all ... then you're quite probably doing something, or several things, quite wrong. Else you're in such deep shit working around some broken build system that you don't need me to tell you about it. Either way, local admin has to enable your hooks before they can run, so if you want to be friendly to others (and yourself), then keep the 'normal' packaging work strictly inside the usual package building tools, and leave the gitpkg hooks free for other local admins to wrap whatever automation it is they need around things. If a particular version of the package source needs some particular actions performed on it prior to the first source package build, then the PREBUILD_TARGET option from above is most probably what you want rather than one of these hooks. Other people can use that again later without needing to have gitpkg around. The aim is for this to Help You. For some values of All Of You. So do be careful to avoid letting it screw other people over if the hook isn't called, and/or let them know what they need to do instead if it isn't. Ok then, there's the barb to watch out for, so back to the point again:
The available hook points are listed below in roughly the order that they would usually be invoked:
This hook is able to modify the gitpkg configuration variables for subsequent operations. It can perform operations on the repo if needed, but since it needs to be committed to the repo before it will ever be called, that may not be so useful here in practice. Basically, it can do anything it pleases, it's just a shell script, nothing else has really begun yet, and it has been sourced into the topmost shell level of gitpkg.
Its operation is different from the admin-config-hook in only one respect, the path to this hook must be relative to the TLD of the repo, and the revision of the file that will be sourced is checked out from the 'branch' tree-ish that gitpkg was requested to export. The file must exist in that version at the path given.
Available to hook scripts as PACKAGE_CONFIG_HOOK.
This can be used by the local admin to override any package specific options, that may have been set by the package-config-hook, with site specific configuration. This is a policy control, not a security one. Security was all over when you let the package-config-hook run, this just lets you override it without having to fake up a new commit changing the package hook.
This is the last hook to run that is able to modify the gitpkg configuration and set environment options that will be visible to later hooks. Available to hook scripts as ADMIN_CONFIG_HOOK. This may be overridden on the command line with the --admin-config-hook=path option.
This can be used to do things like invoke pristine-tar or prefetch an existing orig tarball from some foreign source. It may perform operations on the repo if any such are desired, or any other last minute check that needs to be done before we actually get about the task of exporting the source we want packaged.
Available to hook scripts as PRE_EXPORT_HOOK.
This hook is only invoked if the upstream 'origbranch' actually is exported from the repository. If an existing orig.tar is found or has been created by some earlier hook (and it is not being overwritten, see force-overwrite-orig above), then the operations this hook would perform are presumed to have already happened for this tarball and it is skipped.
It is not safe to assume that this hook will be executed before or after deb-export-hook, and it may in fact be run in parallel with it at some point in the future. They both will be entered after pre-export-hook returns, and exit-hook will not begin until (at least) after both have returned. What else happens in the middle of all that we make no firm promises about at this stage.
Available to hook scripts as ORIG_EXPORT_HOOK.
The following variables are made available for hook scripts, in addition to those already listed as shadowing a git-config option from above. Not all of them are valid/useful at all hook points, see the hook documentation above for the exceptions applying to specific hooks.
These variables have been available to hooks since gitpkg version 0.13
These variables have been available to hooks since gitpkg version 0.24
$ gitpkg --my-option=foo --option2 --opt=oops --opt='bar baz'
Will give:
${GITPKG_AOPTS[my-option]} = 'foo'
${GITPKG_AOPTS[option2]} = ''
${GITPKG_AOPTS[opt]} = 'bar baz'
$ gitpkg --my-option=foo --option2 --opt=oops --opt='bar baz'
Will give:
${GITPKG_IOPTS[0]} = '--my-option=foo'
${GITPKG_IOPTS[1]} = '--option2'
${GITPKG_IOPTS[2]} = '--opt=oops'
${GITPKG_IOPTS[3]} = '--opt=bar baz'
There are convenience functions in repo-config-helper (see below for details) which can be used to inspect this array and obtain all the value(s) for a specific option.
A range of new support functions were added to repo-config-helper in gitpkg version 0.30
- require_bash_version()
- trim_array()
- have_commandline_option()
- have_any_of_these_commandline_options()
- extract_values_for()
- extract_value_for()
- extract_bool_for()
- get_option_values()
- get_option_value()
- get_bool_value()
These are described in more detail in the Hook Library Helpers section below.
There are some canned hook scripts for various tasks available in /usr/share/gitpkg/hooks which currently include:
$ git config gitpkg.exit-hook /usr/share/gitpkg/hooks/cowpoke-exit-hook
Additional git-config(1) configuration options:
Default is to just go ahead and do it if this hook is set.
You can use --cowpoke= (or equivalently, either --cowpoke or --no-cowpoke) to override and clear all configured .options without needing to pass some arbitrary new one. This will not override any other --cowpoke=arg options which are passed on the command line. As a multi-valued option, all empty values which are passed for it will simply be ignored aside from causing the configuration file defaults to be ignored.
If no arch is set or passed on the command line it will default to whatever is configured in cowpoke.conf, which in turn will default to what dpkg-architecture(1) returns for DEB_BUILD_ARCH on the machine that is running gitpkg.
If no dist is set or passed on the command line it will default to whatever is configured in cowpoke.conf, and if not set there it will default to whatever cowbuilder is locally configured for on the remote machine.
The <dist> name used here may also be a locally defined identifier for any specially configured cowpoke chroot that is available, such as those set up for building backports or using extra packages only available from a local repository. See the cowpoke(1) description of its --dist option for details of that.
These options will be appended to any that are specified in DEBBUILDOPTS in the build host's pbuilderrc. To clear any already preset options, first pass it with an empty argument (i.e. --dpkg-bp= --dpkg-bp=option ...).
If set to 'false' or 'no', it is the equivalent of adding the cowpoke(1) option --no-sign-source-changes.
The default if this option is not set at all is defer to SIGN_SOURCE_CHANGES in the cowpoke.conf, or if it is not set to the pbuilder(8) configuration SOURCE_ONLY_CHANGES setting.
All other values passed with this option are an error.
This setting may be overridden on the command line with --sign-source-changes[=arg] where legal values for arg and their actions are the same as described above. Using --no-sign-source-changes has the equivalent effect to using --sign-source-changes=no.
The --no-sign option will cancel the use of any keyid that was set with .sign-key in the git config, or by a previous command-line option, but this just means gitpkg will not pass a keyid to cowpoke, it does not override cowpoke's own configuration which may still specify a default key to use for the given arch/dist. To also override any SIGN_KEYID that is set in cowpoke.conf you can instead use --sign= (with an empty argument), which will be passed through to cowpoke and clear the signing keyid, preventing cowpoke from offering the option of signing the packages after they are built.
The --no-upload option will cancel the use of any upload queue that was set with .upload-to in the git config, or by a previous command-line option, but this just means gitpkg will not pass an upload queue to cowpoke, it does not override cowpoke's own configuration which may still specify a default queue to use for the given arch/dist. To also override any UPLOAD_QUEUE that is set in cowpoke.conf you can instead use --upload= (with an empty argument), which will be passed through to cowpoke and clear the upload queue, preventing cowpoke from offering the option of uploading the packages after they are signed.
The --no-return option will cancel a .return-dir set in the git config, or by a previous command-line option, but this just means gitpkg will not pass a return dir to cowpoke, it does not override cowpoke's own configuration which may still specify a RETURN_DIR. Because cowpoke recognises --return (with no value assignment) as a discrete option, to override and clear RETURN_DIR that is set in cowpoke.conf from here you must pass the empty assignment verbatim in the .options configuration, or on the command line with --cowpoke=--return= which will bypass the normal gitpkg handling of the --return option.
$ git config gitpkg.exit-hook /usr/share/gitpkg/hooks/dpkg-buildpackage-exit-hook
Additional git-config(1) configuration options:
Default is to just do it if this hook is set.
You can use --dpkg-bp= (or equivalently, either --dpkg-bp or --no-dpkg-bp) to override and clear all configured .options without needing to pass some arbitrary new one. This will not override any other --dpkg-bp=arg options which are passed on the command line. As a multi-valued option, all empty values which are passed for it will simply be ignored aside from causing the configuration file defaults to be ignored.
If set to the special value 'signed' then the <package>_source.changes file, and any .dsc or .buildinfo files referenced by it, will be signed regardless of any -uc, -us, -ui options being used to suppress signing of the files created by dpkg-buildpackage. This makes it possible to do a full binary build, creating source and .deb files for inspection and testing, but then sign only the files needed to do a source only upload to another build farm.
If set without a value (or to an empty string), then the behaviour is the same as if it was set to 'signed' and --dpkg-bp=-uc --dpkg-bp=-us were passed. i.e. Only the <package>_source.changes file (and the files referenced by it) will be signed, not the binary files created by the build. This is probably what most people want when exporting a package with the intent of doing a "source only" upload.
The default if unset is the same as setting this option to 'false' or 'no', no <package>_source.changes file will be created by this hook and the behaviour is simply what would be expected from dpkg-buildpackage(1) with any otherwise configured options passed to it.
All other values passed with this option are an error.
This setting may be overridden on the command line with --sign-source-changes[=arg] where legal values for arg and their actions are the same as described above. Using --no-sign-source-changes has the equivalent effect to using --sign-source-changes=no.
Since those two tools don't use the same configuration, that may not be the same key for all users on all machines, but we do respect and use DEB_SIGN_KEYID from the environment (if it is set with no other overriding configuration), when calling debsign, as it would otherwise ignore that configuration option from dpkg-buildpackage. The usual debsign default is instead DEBSIGN_KEYID and set in the devscripts.conf file.
You can override the signing key on the command line with the --sign=keyid option. The --no-sign option will cancel the use of any keyid that was set with .sign-key in the git config, or by a previous command-line option, but this just means gitpkg will not pass a keyid to dpkg-buildpackage(1) and it will revert to its own determination of what key should be used for signing. To actually not sign a package you'll want to use something like --dpkg-bp=-uc --dpkg-bp=-us to explicitly disable signing.
$ git config gitpkg.pre-export-hook /usr/share/gitpkg/hooks/pristine-tar-pre-export-hook
If a pristine-tar branch is not found in the repo, then gitpkg will be terminated.
To enable it:
$ git config gitpkg.deb-export-hook /usr/share/gitpkg/hooks/quilt-patches-deb-export-hook
The contents of debian/source/git-patches may include comments (on any line beginning with a #), empty lines, and expressions of a range of commits. The revision ranges may include $DEB_VERSION, $UPSTREAM_VERSION, $DEB_REF or $UPSTREAM_REF. The first pair will be substituted with the version of the package being exported, the second pair with those version strings after mangling by sanitise_git_ref to remap them to a legal git refname. Using the sanitised versions is to be preferred in most cases. For example:
# Export all commits between these two treeishes, # based on the version of the package being exported. upstream/$UPSTREAM_REF..patches/$DEB_REF
To enable it:
$ git config gitpkg.deb-export-hook /usr/share/gitpkg/hooks/debcherry-deb-export-hook
In order to use this hook, a ${DEB_ORIG}.commit file must be created which contains the treeish of the exported upstream source in the repository. This will be created automatically (if this hook is enabled) when you export an upstream tarball by passing both branch and origbranch to gitpkg, or if you use the pristine-tar-pre-export-hook, which determines an appropriate commit corresponding to the tarball. If your upstream tarball is created using some other mechanism you will need to ensure that file is created yourself.
If using this hook, you may wish to document that in your repository with something similar to the text in /usr/share/doc/gitpkg/examples/README.debcherry-export as a convenience to other users. Your package will still be exportable without this hook enabled, it just won't have the upstream patches individually separated out into a quilt series.
These are additional shell code snippet files which are also found in /usr/share/gitpkg/hooks, for operations which may usefully be shared by several hook scripts. Usually these would be sourced by other scripts rather than being hooked to directly.
It is designed to query multi-valued command line options, filling an array with all of the values passed to gitpkg for some option which were recorded in GITPKG_IOPTS (or any similar array).
See the content of that file itself for more detailed documentation on these functions, their operation, and calling conventions.
If you intend to call gitpkg from your own scripts, then you should note that there are two situations when it may prompt interactively by default. There is no One True Sane Default for these cases, so it's better to just ask the user and continue than to make them start the whole process again in the likely case where they have called gitpkg directly. For details, see the gitpkg.force-overwrite-orig and gitpkg.create-fake-orig config options above. You should set both explicitly to the behaviour that you desire from them if gitpkg should never become interactive.
Though gitpkg explicitly does not try to force any particular workflow procedure upon you in order to make full use of it, it probably is worth making quick mention of at least one simple way to manage Debian packages in git.
One common repo structure is to keep pristine upstream source on one branch, which is updated either directly from an upstream repo or by importing tar archives to it periodically, with the Debian patched source on another branch. In this situation the task of preparing a new upstream release from a tarball might look a bit like this:
Check out the upstream branch
$ cd myrepo
$ git checkout upstream
Remove all old upstream files from the repo
$ rm -rf $(all_files_except .git)
Unpack the new tarball in their place
$ tar zxf $new_upstream.tar.gz
Let git figure out what is renamed/new/gone by itself.
Make sure you don't have things like vim .swp files lurking
in the tree still at this point.
$ git add .
$ git commit -a
$ git tag v$upstream_version
Prepare the Debian branch
$ git checkout debian
$ git merge upstream
$ $(update changelog and other debian patches etc.)
$ git commit -a
$ git tag v${upstream_version}-$debian_version
Make a release
$ gitpkg v${upstream_version}-$debian_version v$upstream_version
$ cd ../deb-packages/mypackage && dpkg-buildpackage ...
git-debimport(1), git-debcherry(1), git(1), git-archive(1), git-config(1), git-format-patch(1), gitattributes(5), dpkg-source(1), cowpoke(1).
gitpkg was written by Ron <ron@debian.org>.
| September 29, 2023 |