Cargo is an incredible package manager for Rust. It only takes a *.toml file, and a few entries, and Cargo handles everything from downloading your packages to compiling your binary. Additionally, unlike other package managers, you don’t need any artifact repository to upload your libraries or artifacts to. It is perfectly fine just to define git dependencies, and Cargo takes care of cloning a specific tag or branch and building your own package based on the source code at hand.

Cargo git dependencies work incredibly well with artifacts hosted on GitHub because the platform is easy to use and open. Other GitHub alternatives also work pretty well as long as the repository is public. Many companies, however, don’t open-source everything they do. They often use self-hosted versions of GitLab or at least their cloud offer for various reasons, like your employer probably does. This is when you quickly begin to ask yourself how to handle private GitLab dependencies in Cargo because they can become an issue.

Things that work locally don’t tend to work as well in your GitLab pipelines. Add Docker to the mix, and you end up with many issues that you first need to solve. Gladly, there is a straightforward solution, which you will learn about in this article.

Handling Private GitLab Dependencies in Cargo

As you probably already know, Cargo allows you to specify dependencies as git dependencies. Instead of using crates.io to pull in the dependency and its metadata, Cargo uses git to clone the repository and check out specific branches or tags. There are no drawbacks to using this method at all because the Rust compiler wants source code, nevertheless. Rust builds are always platform-specific, so the compiler always compiles everything for you to create the best binary for the platform at hand.

The most common git dependency is probably one pointing at a public GitHub repository directly, as you see below:

[package]
authors = ["You and probably you? <[email protected]>"]
edition = "2021"
name = "my-awesome-project"
publish = false
version = "0.1.0"

[dependencies]
my-private-dependency = {git = "https://github.com/project/my-awesome-lib.git", tag = "v1.0.0"}

This dependency is straightforward. It just uses HTTPS to clone the repository, and as long as this repository is public, you have no issues. But what if you use your company’s GitLab and also want to use ssh to clone projects because you cannot constantly juggle around tokens? In this case, Cargo also allows you to use ssh clone URLs.

There are only two catches to it:

1. Cargo does not support git shorthand URLs
2. Cargo’s built-in ssh library does not support using your local ssh configuration (usually in ~/.ssh/config)

The first point can be solved pretty quickly. If your clone URL looks like this: [email protected]:group/othergroup/a-team/my-awesome-lib.git you have to convert it to an ssh URL like this: ssh://[email protected]/group/othergroup/a-team/my-awesome-lib.git. You can take a look below to see what you need to specify inside your Cargo.toml:

[package]
authors = ["You and probably you? <[email protected]>"]
edition = "2021"
name = "my-awesome-project"
publish = false
version = "0.1.0"

[dependencies]
my-private-dependency = {git = "ssh://[email protected]/group/othergroup/a-team/my-awesome-lib.git", tag = "v1.0.0"}

This alone does not make everything work magically, though. You still need to use the ssh key associated with your GitLab account to authenticate yourself. GitLab uses this key to verify whether you have access to a specific repository. As you have learned before, though, Cargo does not use your local ssh config because it simply can’t. But there gladly is a setting to circumvent Cargo’s built-in ssh library.

You can make Cargo use your local installation of git by performing the following three steps:

1. Create a folder named .cargo within your project
2. Create a file called config within this new folder
3. Add the following lines to this file:

[net]
git-fetch-with-cli = true

With this setting in place, Cargo will use your local git CLI and also authenticate itself properly, so you can easily pull in all kinds of private repositories (as long as you have the correct access rights).

Using Private Cargo GitLab Dependencies in Pipelines

Your private repository can now be fetched locally, but sadly, GitLab pipelines will still give you a few headaches. The main issue is that GitLab pipelines, runners, and ssh keys are not the best friends. You either have to manage ssh keys on all runners or work with file-based CI/CD variables that are all not optimal, especially as GitLab already has a solution for accessing other repositories.

Every pipeline job has a token associated with it, stored within the environment variable CI_JOB_TOKEN. This token has advanced access rights to everything within your group. This means that this token can be used to authenticate a git clone request through an HTTPS URL without juggling around tokens, passwords, or else. Additionally, it is a one-time token. It is only valid for as long as your pipeline runs and loses its validity after that. Even if you leak this token accidentally, no one can use it if your pipeline is already finished.

What’s missing now is to find out how to go from ssh to HTTPS. Gladly, git allows you to rewrite specific URLs. This feature is pretty neat, as it precisely satisfies the requirement at hand: Use ssh locally but HTTPS within a pipeline job.

If you put the lines you see below into the before_script section of your job or just use the full block to extend from, all jobs that have this code associated with it can pull dependencies from private GitLab repositories without issues.

.git-access:
  before_script:
    - git config --global credential.helper store
    - echo "https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.mycompany.com" > ~/.git-credentials
    - git config --global url."https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.mycompany.com".insteadOf ssh://[email protected]

How to Handle Private GitLab Dependencies in Cargo with Docker

There is only one major issue left: Docker. It can make a lot of sense to build your binary directly within Docker itself. Rust always compiles your code for the platform it is currently used on by default. As long as you do not need to specify another build target (and that can be difficult sometimes), it is way easier to build your binary on the target platform directly.

If you use Docker to build your project, you will face the same issues as before. Cargo within Docker will have problems authenticating itself because it has no access to your ssh keys by default, and within your pipeline, there are no ssh keys to share. So what now?

Docker has had BuildKit for some time now. It is a newer build engine allowing you to share the ssh-agent with the host system through its Docker secrets. This is exactly what you need to be able to build your container locally. Using it is also pretty straightforward and does not require too many adjustments.

The first thing you need to do to enable Cargo to pull in your private GitLab dependencies within a build container is to add a slight modification to your cargo commands in your Dockerfile. The RUN command allows you to add a reference to a secret of type ssh. This might not sound intuitive, but what it does is connect the ssh-agent within the Docker agent to your local ssh-agent. You can see an example below:

RUN --mount=type=ssh cargo build --release

Only commands that reference the secret get access to the ssh keys. All other commands don’t even know of the presence of the mount, which makes this method very secure. Additionally, no references or keys are cached by Docker, so there is no unintentional leaking of your precious auth information.

To make this secret mount work, you must tell Docker where to look for keys. But first, you need to ensure that BuildKit is enabled by setting an environment variable called DOCKER_BUILDKIT. After that, you can pass a flag called —ssh with a path to your ssh key to Docker, as seen below:

DOCKER_BUILDKIT=1 docker build -t my-awesome-project:latest --ssh default=~/.ssh/id_rsa .

And that’s it. Now your container can also build locally without issues. Only GitLab pipelines are still an issue right now, but that can also be solved.

Remember the trick with rewriting the ssh URL to the HTTPS URL and using the GITLAB_CI_TOKEN as a means of authentication? The same trick can be done within the container.

First, add a build argument to your Dockerfile. You can call it anything you like, but for the sake of this guide, it will simply be called CI_JOB_TOKEN, like the environment variable within your pipeline job. You can define the build argument like this within your Dockerfile:

ARG CI_JOB_TOKEN

Don’t set a default value; this makes it way easier to execute conditional logic based on its presence.

Whenever you build your Docker container now, you add a flag (--build-arg) to the build, like seen below:

DOCKER_BUILDKIT=1 docker build -t my-awesome-project:latest --ssh default=~/.ssh/id_rsa --build-arg=CI_JOB_TOKEN=$CI_JOB_TOKEN .

The only thing missing now is the conditional logic that decides how the dependency shall be pulled: Through ssh when building locally and through HTTPS when the container is built within your pipeline. A simple bash script is enough to make this work, so take a look at the script below:

if [ -z ${CI_JOB_TOKEN} ]; then
  mkdir -p -m 0700 ~/.ssh && ssh-keyscan gitlab.mycompany.com >> ~/.ssh/known_hosts

  cargo build --release;
else
  git config --global credential.helper store;
  echo "https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.mycompany.com" > ~/.git-credentials;
  git config --global url."https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.mycompany.com".insteadOf ssh://[email protected];

  cargo build --release;
fi

The logic is relatively simple. If the environment variable CI_JOB_TOKEN (your build argument) is not set, the dependency is pulled with the ssh URL (making use of the connected ssh-agent). If the environment variable is set, a git URL rewrite is configured, and the CI_JOB_TOKEN is used to authenticate against GitLab’s HTTPS API.

Assuming that the script is called docker-build.sh, the RUN command to build your project within your Dockerfile can simply look like this:

RUN --mount=type=ssh ./docker_build.sh

And that’s it. Congratulations. You can now work with Cargo dependencies pointing to private GitLab repositories both locally and within your pipelines, as well as within Docker, no matter the environment.

Summary

Cargo’s git dependencies are a great way to also include proprietary libraries you build at your company, but especially these private repositories on GitLab can cause issues in your build process. Thankfully, there are ways around these issues that you have learned about here.

You learned how to:

1. Declare git dependencies for repositories through ssh
2. Handle private GitLab repositories in your pipelines
3. Combine both ways to make Docker builds work both locally and in your pipelines

Now, you should know how to handle private GitLab dependencies in Cargo and also have learned a few things more about Docker and its secrets.