Answered by DazWilkin
I think file systems aren't shared between jobs (create_zip, google_cloud_services). You may want to prove this and consider moving the actions/download-artifact to google_cloud_services.
Working code
name: CI Workflow
on:
push:
branches:
- main
env:
PROJECT_ID: "flights-429420"
BUCKET_NAME: "flights-cicd-bucket"
create_zip:
runs-on: ubuntu-latest
needs: test
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Install zip
run: sudo apt-get install zip
- name: Create jobs.zip
run: |
cd app
zip -r ../jobs.zip jobs
- name: Upload zip file
uses: actions/upload-artifact@v3
with:
name: jobs
path: jobs.zip
google_cloud_services:
runs-on: ubuntu-latest
needs: create_zip
steps:
- uses: actions/checkout@v3
- name: Download zip file artifact
uses: actions/download-artifact@v3
with:
name: jobs
path: ./downloads
- name: Authenticate with GCP using Service Account
uses: 'google-github-actions/auth@v2'
with:
credentials_json: ${{ secrets.GCP_ACCOUNT_KEY }}
- name: Setup GCP CLI
uses: 'google-github-actions/setup-gcloud@v2'
with:
project_id: ${{ env.GCP_PROJECT_ID }}
install_components: 'gsutil'
- name: Create GCS bucket if not exists
run: |
if ! gsutil ls -b gs://${{ env.BUCKET_NAME }}/; then
gsutil mb -l EU gs://${{ env.BUCKET_NAME }}/
fi
- name: Upload files to GCS
run: |
gsutil cp ./downloads/jobs.zip gs://${{ env.BUCKET_NAME }}/jobs.zip
gsutil cp app/main.py gs://${{ env.BUCKET_NAME }}/main.py
gcloud storage objects update gs://${{ env.BUCKET_NAME }}/main.py --content-type=application/octet-stream
Answer from KurczakChrupiacy2 on Stack OverflowGitHub
github.com › google-github-actions › upload-cloud-storage
GitHub - google-github-actions/upload-cloud-storage: A GitHub Action for uploading files to a Google Cloud Storage (GCS) bucket. · GitHub
The upload-cloud-storage GitHub Action uploads files to a Google Cloud Storage (GCS) bucket.
Starred by 264 users
Forked by 56 users
Languages TypeScript 99.0% | JavaScript 1.0%
Videos
GitHub Actions Deployments to Google Cloud Storage | Software ...
11:33
How to use Github Actions with Google's Workload Identity Federation ...
GitHub Actions with Google Cloud
04:10
How to deploy Google Cloud Functions with GitHub Actions - YouTube
11:49
Using GitHub Actions to Deploy My Website To Google Cloud - YouTube
GitHub
github.com › google-github-actions › upload-cloud-storage › blob › main › action.yml
upload-cloud-storage/action.yml at main · google-github-actions/upload-cloud-storage
A GitHub Action for uploading files to a Google Cloud Storage (GCS) bucket. - upload-cloud-storage/action.yml at main · google-github-actions/upload-cloud-storage
Author google-github-actions
GitHub
github.com › google-github-actions › upload-cloud-storage › actions
Workflow runs · google-github-actions/upload-cloud-storage
November 21, 2024 - A GitHub Action for uploading files to a Google Cloud Storage (GCS) bucket. - Workflow runs · google-github-actions/upload-cloud-storage
Author google-github-actions
GitHub
github.com › google-github-actions › upload-cloud-storage › blob › main › CHANGELOG.md
upload-cloud-storage/CHANGELOG.md at main · google-github-actions/upload-cloud-storage
A GitHub Action for uploading files to a Google Cloud Storage (GCS) bucket. - google-github-actions/upload-cloud-storage
Author google-github-actions
GitHub
github.com › google-github-actions › upload-cloud-storage › issues
google-github-actions/upload-cloud-storage
A GitHub Action for uploading files to a Google Cloud Storage (GCS) bucket. - google-github-actions/upload-cloud-storage
Author google-github-actions
GitHub
github.com › google-github-actions › upload-cloud-storage › blob › main › README.md
upload-cloud-storage/README.md at main · google-github-actions/upload-cloud-storage
The upload-cloud-storage GitHub Action uploads files to a Google Cloud Storage (GCS) bucket.
Author google-github-actions
GitHub
github.com › google-github-actions › upload-cloud-storage › issues › 320
Error: google-github-actions/upload-cloud-storage failed with: invalid response body while trying to fetch https://sts.googleapis.com/v1/token: read ECONNRESET · Issue #320 · google-github-actions/upload-cloud-storage
April 1, 2023 - Run google-github-actions/upload-cloud-storage@v1 Warning: The "process_gcloudignore" option is true, but no .gcloudignore file was found. If you do not intend to process a gcloudignore file, set "process_gcloudignore" to false.
Author adaszko
GitHub
github.com › google-github-actions › upload-cloud-storage › releases
Releases · google-github-actions/upload-cloud-storage
November 30, 2022 - A GitHub Action for uploading files to a Google Cloud Storage (GCS) bucket. - google-github-actions/upload-cloud-storage
Author google-github-actions
GitHub
github.com › google-github-actions › upload-cloud-storage › blob › main › package.json
upload-cloud-storage/package.json at main · google-github-actions/upload-cloud-storage
A GitHub Action for uploading files to a Google Cloud Storage (GCS) bucket. - upload-cloud-storage/package.json at main · google-github-actions/upload-cloud-storage
Author google-github-actions
Top answer 1 of 2
4
You could use the following Github Action to have the same files in the destination as in the source:
name: Copy files to GCS
on:
push:
branches: [main]
jobs:
copy-files-gcs:
name: Copy directory to GCS
runs-on: ubuntu-latest
# Add "id-token" with the intended permissions.
permissions:
contents: 'read'
id-token: 'write'
steps:
- name: Checkout
uses: actions/checkout@v3
- id: 'auth'
uses: 'google-github-actions/auth@v0'
with:
credentials_json: '${{ secrets.GCP_CREDENTIALS }}'
- name: 'Set up Cloud SDK'
uses: 'google-github-actions/setup-gcloud@v0'
- name: 'Use gcloud CLI'
run: 'gsutil -m rsync -R -d kavach gs://<your-bucket>'
The key for syncing the files in destination folder as you are looking for is the -d flag for rsync which deletes the files in the destination that are not present in the source.
2 of 2
2
You could try to delete files from bucket using gsutil as follows:
- name: Set up Cloud SDK
uses: google-github-actions/setup-gcloud@master
with:
project_id: ${{ secrets.GCP_PROJECT_ID }}
service_account_key: ${{ secrets.GCP_SA_KEY }}
export_default_credentials: true
- name: Delet files from bucket
run: gsutil rm gs://my-bucket/my-folder/**
GitHub
github.com › google-github-actions › upload-cloud-storage › issues › 10
Error occured when using upload-cloud-storage · Issue #10 · google-github-actions/upload-cloud-storage
October 24, 2020 - name: Upload to Google Cloud Storage uses: GoogleCloudPlatform/github-actions/upload-cloud-storage@master with: credentials: ${{ secrets.GCS_CREDENTIALS }} path: build destination: ${{ env.UPLOAD_P...
Author shwuhk
GitHub
github.com › jakub-bacic › upload-cloud-storage
GitHub - jakub-bacic/upload-cloud-storage: This action uploads files/folders to a Google Cloud Storage (GCS) bucket. · GitHub
This action uploads files/folders to a Google Cloud Storage (GCS) bucket. - jakub-bacic/upload-cloud-storage
Author jakub-bacic
GitHub
github.com › google-github-actions › upload-cloud-storage › issues › 241
Error: failed with error code [object Object] · Issue #241 · google-github-actions/upload-cloud-storage
April 4, 2022 - name: Sync on: schedule: - cron: '17/30 * * * *' workflow_dispatch: jobs: sync: runs-on: ubuntu-latest permissions: contents: 'write' id-token: 'write' env: BUCKET_NAME: linux-mirror-db DB_NAME: mirror.sl3 URL_PREFIX: mirror-chunk- # 10MB = 10 * 1024 * 1024 = 10485760 SERVER_CHUNK_SIZE: 10485760 SUFFIX_LENGTH: 3 steps: - uses: actions/checkout@v2 - name: Download CSVs run: | wget https://${BUCKET_NAME}.storage.googleapis.com/fixes.csv wget https://${BUCKET_NAME}.storage.googleapis.com/reported_by.csv wget https://${BUCKET_NAME}.storage.googleapis.com/tags.csv wget https://${BUCKET_NAME}.storag
Author sirdarckcat
Sha
sha.ws › posts › automatic-upload-to-google-cloud-storage-with-github-actions
Automatic upload to Google Cloud Storage with GitHub Actions · sha.ws
June 24, 2020 - GitHub Actions enable simple workflow automation and CI/CD right from GitHub repos. Uploading your content, whether a static website or any other artifact can be easily managed with an Actions workflow. This article walks through everything required to successfully set up your workflow.
GitHub
docs.github.com › en › enterprise-server@3.7 › admin › github-actions › enabling-github-actions-for-github-enterprise-server › enabling-github-actions-with-google-cloud-storage
Enabling GitHub Actions with Google Cloud Storage - GitHub Enterprise Server 3.7 Docs
Under "GitHub Actions", select Enable GitHub Actions. Under "Artifact & Log Storage", select Google Cloud Storage, and enter your bucket's details: