You have to execute the command from the root folder of your git repository/project:
$ ls -a
./
../
.git/
.gitignore
.gitlab-ci.yml
Makefile
$ gitlab-runner exec docker <job_name>
You are using docker commands which you shouldn't as gitlab-runner already uses those under the hood.
See $ gitlab-runner exec docker --help for more info on the command.
You have to execute the command from the root folder of your git repository/project:
$ ls -a
./
../
.git/
.gitignore
.gitlab-ci.yml
Makefile
$ gitlab-runner exec docker <job_name>
You are using docker commands which you shouldn't as gitlab-runner already uses those under the hood.
See $ gitlab-runner exec docker --help for more info on the command.
I also came across a similar issue when using VS Code with remote containers and Docker Desktop (WSL2).
I debugged the problem by adding in a --pre-get-sources-script=ls -l "repo_directory", i.e. in the OP's case:
# docker run --name=runner --privileged -t --rm \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp/.gitlab-runner/:/etc/gitlab-runner \
-v
{PWD} \
--workdir $PWD \
--pre-get-sources-script=ls -l /home/fox/Work/docker/core-application/../.git/modules/core-application \
gitlab/gitlab-runner exec docker \
--builds-dir /tmp/builds/ Rspec
In my case, the directory listing (ls -l ...) came up empty! I'm assuming that it mounts the directory as a volume within the container as the same name (/fubar:/fubar), which in my case isn't going to work as it is Docker Desktop with vscode remote containers.
My fix was was to add my named volume as a parameter to a secondary directory, and then setting the --clone-url to point to that directory.
Something like:
# pwd
/workspace/fubar
# gitlab-runner --debug exec docker \
--docker-volumes vscode-workspace:/workspace2 \
--clone-url=file:///workspace2/fubar \
validate;
[output truncated]
...
Starting container 3d49d8e8b977177f34beeae3ae5f09eba288332e109ad79f133e0f160377908b ... job=1 project=0
Fetching changes...
Initialized empty Git repository in /builds/project-0/.git/
Created fresh repository.
Checking out d1a39a4d as detached HEAD (ref is feature/FU-123)...
Skipping Git submodules setup
Executing build stage build_stage=restore_cache job=1 project=0
Skipping stage (nothing to do) build_stage=restore_cache job=1 project=0
Executing build stage build_stage=download_artifacts job=1 project=0
Skipping stage (nothing to do) build_stage=download_artifacts job=1 project=0
Executing build stage build_stage=step_script job=1 project=0
Executing "step_script" stage of the job script
...
I had originally tried changing the --pre-get-sources-script to be cp -R /workspace2/fubar /workspace/ but that complained that the target directory was read-only. Using the --clone-url=file:///workspace2/fubar worked, so I've left it that way.
How to use gitlab-runner exec docker correclty - GitLab CI/CD - GitLab Forum
Use GitLab CI to run tests locally? - Stack Overflow
How to use the docker gitlab-runner to exec stages locally - Stack Overflow
Trouble using gitlab-runner exec docker - How to Use GitLab - GitLab Forum
Since a few months ago this is possible using gitlab-runner:
gitlab-runner exec docker my-job-name
Note that you need both docker and gitlab-runner installed on your computer to get this working.
You also need the image key defined in your .gitlab-ci.yml file. Otherwise won't work.
Here's the line I currently use for testing locally using gitlab-runner:
gitlab-runner exec docker test --docker-volumes "/home/elboletaire/.ssh/id_rsa:/root/.ssh/id_rsa:ro"
Note: You can avoid adding a
--docker-volumeswith your key setting it by default in/etc/gitlab-runner/config.toml. See the official documentation for more details. Also, usegitlab-runner exec docker --helpto see all docker-based runner options (like variables, volumes, networks, etc.).
Due to the confusion in the comments, I paste here the gitlab-runner --help result, so you can see that gitlab-runner can make builds locally:
gitlab-runner --help
NAME:
gitlab-runner - a GitLab Runner
USAGE:
gitlab-runner [global options] command [command options] [arguments...]
VERSION:
1.1.0~beta.135.g24365ee (24365ee)
AUTHOR(S):
Kamil Trzciński <ayufan@ayufan.eu>
COMMANDS:
exec execute a build locally
[...]
GLOBAL OPTIONS:
--debug debug mode [$DEBUG]
[...]
As you can see, the exec command is to execute a build locally.
Even though there was an issue to deprecate the current gitlab-runner exec behavior, it ended up being reconsidered and a new version with greater features will replace the current exec functionality.
Note that this process is to use your own machine to run the tests using docker containers. This is not to define custom runners. To do so, just go to your repo's CI/CD settings and read the documentation there. If you wanna ensure your runner is executed instead of one from gitlab.com, add a custom and unique tag to your runner, ensure it only runs tagged jobs and tag all the jobs you want your runner to be responsible of.
Unofficial alternative gitlab-ci-local
There's an unofficial package which looks promising, with more features than the official gitlab-runner exec: https://github.com/firecow/gitlab-ci-local
It allows you to run gitlab pipelines locally as shell executor or docker executor.
Deprecated since 2024-05 (see edit)
Only for gitlab-runner < 17.0.
- See gitlab issue.
- See gitlab docu
I use this docker-based approach:
Edit: 2024-05
docker run --entrypoint bash --rm -w "PWD:$PWD" -v /var/run/docker.sock:/var/run/docker.sock gitlab/gitlab-runner:v15.11.1 -c 'git config --global --add safe.directory "*";gitlab-runner exec docker test'
Thanks to user Ivo Smits who reported a successful test with gitlab-runner:v15.11.1
Edit: 2022-10
For all git versions > 2.35.2. You must add safe.directory within the container to avoid fatal: detected dubious ownership in repository at.... This also true for patched git versions < 2.35.2. The old command will not work anymore.
Details
0. Create a git repo to test this answer
mkdir my-git-project
cd my-git-project
git init
git commit --allow-empty -m"Initialize repo to showcase gitlab-runner locally."
1. Go to your git directory
cd my-git-project
2. Create a .gitlab-ci.yml
Example .gitlab-ci.yml
image: alpine
test:
script:
- echo "Hello Gitlab-Runner"
3. Create a docker container with your project dir mounted
docker run -d \
--name gitlab-runner \
--restart always \
-v "
PWD" \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:latest
(-d) run container in background and print container ID
(--restart always) or not?
(-v ") Mount current directory into the current directory of the container - Note: On Windows you could bind your dir to a fixed location, e.g. PWD"
-v "${PWD}:/opt/myapp". Also $PWD will only work at powershell not at cmd
(-v /var/run/docker.sock:/var/run/docker.sock) This gives the container access to the docker socket of the host so it can start "sibling containers" (e.g. Alpine).
(gitlab/gitlab-runner:latest) Just the latest available image from dockerhub.
4. Execute with
Avoid fatal: detected dubious ownership in repository at... More info
docker exec -it -w $PWD gitlab-runner git config --global --add safe.directory "*"
Actual execution
docker exec -it -w $PWD gitlab-runner gitlab-runner exec docker test
# ^ ^ ^ ^ ^ ^
# | | | | | |
# (a) (b) (c) (d) (e) (f)
(a) Working dir within the container. Note: On Windows you could use a fixed location, e.g. /opt/myapp.
(b) Name of the docker container
(c) Execute the command "gitlab-runner" within the docker container
(d)(e)(f) run gitlab-runner with "docker executer" and run a job named "test"
5. Prints
...
Executing "step_script" stage of the job script
$ echo "Hello Gitlab-Runner"
Hello Gitlab-Runner
Job succeeded
...
Note: The runner will only work on the commited state of your code base. Uncommited changes will be ignored. Exception: The .gitlab-ci.yml itself does not have be commited to be taken into account.
Note: There are some limitations running locally. Have a look at limitations of gitlab runner locally.
gitlab-runner exec works as its own standalone command and runs the job in the context of where the executable is called -- it will not cause the job to be run on a registered runner. It is solely a local operation, so you must have the binary available.
You can run gitlab-runner exec inside of your runner container or a new container if you want.
For example (assuming Linux platform):
docker run --rm \
-v $(pwd):/workspace \ # mount directory to the container
--workdir /workspace \ # set working directory
-v /var/run/docker.sock:/var/run/docker.sock \ # enable docker
--privileged \
--entrypoint="gitlab-runner" \ # set entrypoint
gitlab/gitlab-runner:latest exec docker MY_JOB
Or use doker exec to run the command on your existing container:
docker exec -it my-running-container-name gitlab-runner exec docker linting
Though, you will want to make sure the .gitlab-ci.yml file and project files are present in the container when you start it...
since gitlab 17, the feature is removed https://docs.gitlab.com/runner/commands/#gitlab-runner-exec-removed