This article was originally written for devspotlight
Managing Secrets with git-crypt
Many software projects use secrets – usually keys to external APIs or credentials to access an external resource such as a database. Your application needs these keys at runtime, so you need to be able to provide them when you deploy your application, or as a step in preparing your deployment environment.
In this article, I’m going to show you how to use git-crypt so that you can safely keep your application secrets in your source code repositories, even if they’re public.
The Problem with Application Secrets
Most projects have some sort of secret keys or credentials. For example, if your application is hosted on Heroku, you might provide an API key to your Heroku application using a command like this:
$ heroku config:set API_KEY=my-sooper-sekrit-api-key
By running this command before you (re)deploy your application, you give it an environment variable at runtime called
API_KEY with the value
my-sooper-sekrit-api-key. However, keeping track of these secret values outside of Heroku (or wherever you deploy your application) is still a challenge.
I always try to set up my projects so that I can run a single command to deploy them from scratch without any separate, manual steps. For our example, this means I need to store the value
my-sooper-sekrit-api-key somewhere so that my deployment code can use it (in this case, to run the
heroku config:set... command above).
My project source code is always stored in git, and usually hosted on github.com or bitbucket.com or some other source code hosting service. I could store my API_KEY value in my source code repository, however, there are some downsides to this:
- I can’t share my repository with anyone else unless I’m comfortable with them accessing my secrets. This means all my application repositories with secrets in them need to be private.
- Presumably many staff members at github/bitbucket/wherever would also have access to my secrets, which I might not be okay with (depending on the secret).
- It’s easy to forget about the secrets in a private repository, if I later choose to make it public. So I could accidentally disclose important secrets.
I could store my secrets somewhere separate from my application source code, but this has its own problems:
- I need a way to get my secrets from wherever they’re stored, at or before deployment time, and give my deployment code access to them.
- My secrets may not be stored as robustly as my source code. For example, I could keep secrets in a
.envfile on my laptop, and make sure I never check that in to the git repository. However, if I lose that file (such as if my laptop gets damaged/stolen), then I also lose that secret.
Git-crypt aims to solve this problem by encrypting your secrets whenever you push them to your git repository, and decrypting them whenever you pull them. This happens transparently, from your point of view. So the secrets are in cleartext as far as you and your deployment code are concerned, but nobody else can read them, even if your source code is in a public github repository.
Let’s look at an example.
1. Install git-crypt.
There are instructions for Linux, Mac and Windows on the git-crypt install page
If, like me, you’re using a Mac with Homebrew installed, you can run:
$ brew install git-crypt
2. Create a new git repository.
$ mkdir myproject
$ cd myproject
$ git init
$ echo "This is some text" > file.txt
$ git add file.txt
$ git commit -m "Initial commit"
Now we have a git repository containing a single text file.
3. Set up the repository to use git-crypt.
$ git-crypt init
You should see the output:
Before we do anything else, please run the following command:
$ git-crypt export-key ../git-crypt-key
This command creates a copy of the git-crypt symmetric key that was generated for this repository. We’re putting it in the directory above this repository so that we can re-use the same key across multiple git repositories.
By default, git-crypt stores the generated key in the file
.git/git-crypt/keys/defaultso you can achieve the same result by running
cp .git/git-crypt/keys/default ../git-crypt-key
git-crypt-key file is important. It’s the key which can unlock all the encrypted files in our repository. We’ll see how to use this key later on.
4. Tell git-crypt which files to encrypt.
Imagine our application needs an API key, and we want to store it in a file called
Before we add that file to our repository, we will tell git-crypt that we want the
api.key file to be encrypted whenever we commit it.
We do that using the
.gitattributes file. This is a file we can use to add extra metadata to our git repository. It’s not specific to git-crypt, so you might already have a .gitattributes file in your repository. If so, just add the relevant lines—don’t replace the whole file.
In our case, we don’t have a .gitattributes file, so we need to create one. The .gitattributes file contains lines of the form:
[file pattern] attr1=value1 attr2=value2
For git-crypt, the file pattern needs to match all the files we want git-crypt to encrypt, and the attributes are always the same:
diff, both of which we set to
So, our .gitattributes file should contain this:
api.key filter=git-crypt diff=git-crypt
Create that file, and add and commit it to your git repository:
$ echo "api.key filter=git-crypt diff=git-crypt" > .gitattributes
$ git add .gitattributes
$ git commit -m "Tell git-crypt to encrypt api.key"
I’ve used the literal filename
api.key in my .gitattributes file, but it can be any file pattern that includes the file(s) you want to encrypt, so I could have used
*.key, for instance. Alternatively, you can just add a line for each file you want to encrypt.
It can be easy to make a mistake in your .gitattributes file if you’re trying to protect several files with a single pattern entry. So, I strongly recommend reading this section of the git-crypt README, which highlights some of the common gotchas.
5. Add a secret.
Now that we have told git-crypt we want to encrypt the
api.key file, let’s add that to our repository.
It’s always a good idea to test your setup by adding a dummy value first, and confirming that it’s successfully encrypted, before committing your real secret.
$ echo "dummy value" > api.key
We haven’t added
api.key to git yet, but we can check what git-crypt is going to do by running:
$ git-crypt status
You should see the following output:
not encrypted: .gitattributes
not encrypted: file.txt
So, even though the api.key file has not yet been committed to our git repository, this tells you that git-crypt is going to encrypt it for you.
Let’s add and commit the file:
$ git add api.key
$ git commit -m "Added the API key file"
6. Confirm our secret is encrypted.
We’ve told git-crypt to encrypt
api.key, and we’ve added
api.key to our repository. However, if we look at
api.key, nothing seems different:
$ cat api.key
The reason for this is that git-crypt transparently encrypts and decrypts files as you push and pull them to your repository. So, the
api.key file looks like a normal, cleartext file.
$ file api.key
api.key: ASCII text
One way to confirm that your files really are being encrypted is to push your repository to github. When you view the
api.key file using the github web interface, you’ll see that it’s an encrypted binary file rather than text.
An easier way to see how the repository would look to someone without the decryption key is to run:
$ git-crypt lock
Now if we look at our
api.key file, things are different:
$ file api.key
$ cat api.key
You will see some different garbage output to what I get, but it’s clear the file is encrypted. This is what would be stored on github.
To go back to having a cleartext
api.key file, run:
$ git-crypt unlock ../git-crypt-key
../git-crypt-key file is the one we saved earlier using
Let’s do a quick review of where we are now.
- Initialize git-crypt on a git repository using
- Use file patterns in
.gitattributesto tell git-crypt which files to encrypt
git-crypt lockwill encrypt all the specified files in our repository
git-crypt unlock [path to keyfile]will decrypt the encrypted files
git-crypt-key file is very important. Without it, you won’t be able to decrypt any of the encrypted files in your repository. Anyone who has a copy of that file has access to all of the encrypted secrets in your repository. So you need to keep that file safe and secure.
Re-using Your git-crypt key file
git-crypt init and
git-crypt export-key to create our
git-crypt-key file. But, if we have to have a separate key file for each of our repositories, then we haven’t improved our secret management very much.
Fortunately, it’s very easy to use the same git-crypt key file for multiple git repositories.
To use an existing key file, just use
git-crypt unlock instead of
git-crypt init when you set up your git repository to use git-crypt, like this:
$ mkdir my-other-project # At the same directory level as `myproject`
$ cd my-other-project
$ git init
$ echo "Something" > file.txt
$ git add file.txt
$ git commit -m "initial commit"
$ git-crypt unlock ../git-crypt-key
If you run the
git-crypt unlockcommand before adding any files to your git repository, you will see a message like this:
fatal: You are on a branch yet to be born
Error: 'git checkout' failed
git-crypt has been set up but existing encrypted files have not been decrypted
This still works just fine, but it’s a bit confusing, so I made sure to add and commit at least one file before running
Re-using your git-crypt key file is convenient, but it does mean that if anyone else gets a copy of your key file, all of your encrypted secrets are exposed.
This is the same kind of security trade-off as using a password manager like LastPass or 1password. Rather than managing multiple secrets (passwords), each with its own risk of exposure, you keep them all in a secure store, and use a single master password to unlock that.
The idea here is that it’s easier to manage one important secret than many lesser secrets.
When NOT to Use git-crypt
Git-crypt is a great way to keep the secrets your applications need right in the git repository, alongside the application source code. However, like every other security measure, it’s not always going to be appropriate or advisable.
Here are some things to consider to decide whether it’s the right solution for your particular project:
- git-crypt is designed for situations where the majority of the files in your git repository can remain in cleartext, and you just need to encrypt a few files that contain secrets. If you need to encrypt most or all of the files in your repository, then other solutions may be a better fit.
- There is no easy way to revoke access to the secrets in a repository once someone has the key file, and no easy way to rotate (i.e. replace) a key file (although changing the git-crypt key file doesn’t help much unless you also rotate all of the actual secrets in the repository).
- git-crypt only encrypts the contents of files. Therefore, it’s not suitable if the metadata of your repository is also sensitive (i.e. filenames, modification dates, commit messages, and so on).
- Some GUI git applications may not work reliably with git-crypt. (Although the specific case of Atlassian SourceTree, mentioned in the README, has been fixed.)
There is more information in this section of the git-crypt README.
A Better Way to Use git-crypt
Rather than managing your git-crypt key file directly, there is a better way to manage encrypted repositories by integrating git-crypt with gpg, so that you can use your gpg private key to decrypt the git repository. This also allows you to add multiple collaborators to a git repository without transmitting any secrets between the parties. However, this requires a more complicated setup, so we’ll save that for another article.