The answer you referred to will likely work (I've also not fully tested it). The Cloud ML Engine team is working hard to simplify this.
In the meantime, if you prefer working with a VM (just remember to shut it off when not in use), follow instructions on this page. For example,
export IMAGE_FAMILY="pytorch-latest-cu91"
export ZONE="us-west1-b"
export INSTANCE_NAME="my-instance"
gcloud compute instances create $INSTANCE_NAME \
--zone=$ZONE \
--image-family=$IMAGE_FAMILY \
--image-project=deeplearning-platform-release \
--maintenance-policy=TERMINATE \
--accelerator='type=nvidia-tesla-v100,count=1' \
--metadata='install-nvidia-driver=True'
Google Cloud
cloud.google.com › blog › topics › developers-practitioners › pytorch-google-cloud-how-deploy-pytorch-models-vertex-ai
PyTorch on Google Cloud: How to deploy PyTorch models on Vertex AI | Google Cloud Blog
September 16, 2021 - Now let’s walk through the deployment of a Pytorch model using TorchServe as a custom container by deploying the model artifacts to a Vertex Endpoint. You can find the accompanying code for this blog post on the GitHub repository and the Jupyter Notebook. Vertex Prediction service is Google Cloud's managed model serving platform.
Google Cloud
cloud.google.com › vertex ai › pytorch integration
PyTorch integration | Vertex AI | Google Cloud Documentation
How to deploy PyTorch models on Vertex AI: Walk through the deployment of a Pytorch model using TorchServe as a custom container, by deploying the model artifacts to a Vertex AI Inference service.
Can we deploy Pytorch models to Google Cloud ML Engine? If so, how? - Stack Overflow
I searched if anybody has. Somebody has provided a work around but couldn't confirm its working. Here are the links - How to setup pytorch in google-cloud-ml https://discuss.pytorch.org/t/pytorc... More on stackoverflow.com
Deploy ML model on GCP
Vertically scaling pandas on gcp is a slog. The issue being is that you will add loads of cores (that it won't use) to get more ram (which mostly just stops it crashing, it's not doing much to improve performance). Is it possible you could implement modin or polars to try and improve your ability to horizontally scale? If you can, as you suggest, split into multiple processing steps then this could help. If possible though I'd look into dataflow or spark to do this sort of work. Cloud run will be a better fit than cloud functions as it will push you to containers which will be easier to later scale out. More on reddit.com
11
6
March 27, 2023PyTorch on Google Cloud: How to deploy PyTorch models on Vertex AI - AI Discussions - DeepLearning.AI
here is the link - https://cloud.google.com/blog/topics/developers-practitioners/pytorch-google-cloud-how-deploy-pytorch-models-vertex-ai by Google on how to deply pytorch models using vertex ai , but I’m not understanding a lot of things , need a dedicated tutorial on this particularly , ... More on community.deeplearning.ai
0
September 30, 2023[D] Best way to host an image classification model for predictions on Google Cloud
If you are not opposed to using AWS, their sagemaker service makes deploying models pretty straightforward. More on reddit.com
10
3
February 24, 2019Videos
00:48
Deploying PyTorch, deploying Anthos, & more! #Shorts #GoogleCl...
18:51
Operationalize Distributed Training with PyTorch on Google Cloud ...
04:18
PyTorch on Vertex AI - YouTube
20:00
How to easily build, deploy, and manage ML models with Vertex AI ...
01:59:32
How to Deploy a Machine Learning Model to Google Cloud for 20% ...
01:56
Deploying PyTorch Models in Production: PyTorch Playbook Course ...
PyTorch
pytorch.org › get-started › cloud-partners
Start via Cloud Partners
Select preferences and run the command to install PyTorch locally, or get started quickly with one of the supported cloud platforms. ... Cloud platforms provide powerful hardware and infrastructure for training and deploying deep learning models.
Medium
medium.com › @lukas.hauzenberger › training-and-serving-pytorch-models-in-the-google-cloud-with-vertex-ai-pipelines-dedf21dc4e35
Training and serving PyTorch models in the Google Cloud with Vertex AI pipelines | by Lukas Hauzenberger | Medium
April 8, 2023 - Furthermore PyTorch is becoming more and more popular with each iteration and many google services still primarily support Tensorflow or at least make it much easier to implement something when the code is written in Tensorflow. Therefore I feel like a guide like this could help many PyTorch fans. It will give you step-by-step instructions on how to automatically train and deploy a PyTorch model in the GCP with Vertex AI pipelines.
Omi AI
omi.me › blogs › ai-integrations › how-to-integrate-pytorch-with-google-cloud-platform
How to Integrate PyTorch with Google Cloud Platform – Omi AI
February 3, 2025 - gcloud ai-platform versions create v1 --model pytorch_model --framework pytorch --python-version 3.7 --runtime-version 2.2 --origin gs://your-bucket-name/model.pt ... Alternatively, you can use Python with the requests library to send HTTP POST ...
Google Cloud
cloud.google.com › blog › topics › developers-practitioners › pytorch-google-cloud-how-train-pytorch-models-ai-platform
PyTorch on Google Cloud: How To train PyTorch models on AI Platform | Google Cloud Blog
May 6, 2021 - Learn how to build, train and deploy PyTorch models at scale on Google Cloud.
Medium
medium.com › google-cloud › step-by-step-serving-pytorch-models-with-a-custom-handler-on-vertex-ai-5ada1d01c534
Step-by-Step: Serving PyTorch Models with a Custom Handler on Vertex AI | by Remigiusz Samborski | Google Cloud - Community | Medium
June 7, 2025 - Vertex AI needs all your model artifacts — the model weights, configuration, and our new handler.py — to be in one location on Google Cloud Storage. Create a local directory that contains the model files and your handler.
GitHub
gist.github.com › judytraj007 › b0c51a47edd4d690e2ba91b11ffa6063
Deploying pytorch model to Google Cloud Functions · GitHub
Deploying pytorch model to Google Cloud Functions. GitHub Gist: instantly share code, notes, and snippets.
Reddit
reddit.com › r/googlecloud › deploy ml model on gcp
r/googlecloud on Reddit: Deploy ML model on GCP
March 27, 2023 -
Hello experts,
What is the most practical way to serve an ML model on GCP for daily batch predictions. The received batch has to go through multiple preprocessing and feature engineering steps before being fed to the model to produce predictions. The preprocessing is done using pandas (doesn't utilize distributed processing). Therefore, I am assuming a vertically scalable instance has to be triggered at inference time. Based on your experience, what should I use? I am thinking cloud functions that consist of multiple preprocessing steps and then calls the model for predictions.
Top answer 1 of 5
4
Vertically scaling pandas on gcp is a slog. The issue being is that you will add loads of cores (that it won't use) to get more ram (which mostly just stops it crashing, it's not doing much to improve performance). Is it possible you could implement modin or polars to try and improve your ability to horizontally scale? If you can, as you suggest, split into multiple processing steps then this could help. If possible though I'd look into dataflow or spark to do this sort of work. Cloud run will be a better fit than cloud functions as it will push you to containers which will be easier to later scale out.
2 of 5
2
I have deployed an ML Model using Vertex API in Gcp but it was far too costly for us but will scale to very high demand. If its a few predictions daily, cloud run seemed to be our most cost effective option at the time and that's what we settled on.
Google Open Source
opensource.googleblog.com › 2022 › 09 › accelerate-your-models-to-production-with-google-cloud-and-pytorch_0905763892.html
Accelerate your models to production with Google Cloud and PyTorch | Google Open Source Blog
September 12, 2022 - Cloud TPUs are custom accelerators designed by Google, optimized for perf/TCO with large scale ML workload PyTorch/XLA also enables XLA driven optimizations on GPUs. TorchX provides an adapter to run and orchestrate TorchX components as part of Kubeflow Pipelines that you can easily scale on Vertex AI Pipelines. With our OSS contributions to Apache Beam, we have made PyTorch models easy to deploy in batch or stream, data processing pipelines.
Google
docs.cloud.google.com › compute engine › deep learning vm images › create a pytorch deep learning vm instance
Create a PyTorch Deep Learning VM instance | Deep Learning VM Images | Google Cloud Documentation
You can create a PyTorch instance from Cloud Marketplace within the Google Cloud console or using the command line. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in ...
Google Cloud
cloud.google.com › blog › topics › developers-practitioners › pytorch-google-cloud-how-train-and-tune-pytorch-models-vertex-ai
PyTorch on Google Cloud: How To train and tune PyTorch models on Vertex AI | Google Cloud Blog
September 9, 2021 - In the upcoming posts, we will show how you can deploy and serve these PyTorch models on Vertex Prediction service followed by Vertex Pipelines to automate, monitor and govern your ML systems by orchestrating a ML workflow in a serverless manner, and storing workflow's artifacts using Vertex ML Metadata.
Google Cloud
cloud.google.com › blog › products › ai-machine-learning › ai-in-depth-serving-a-pytorch-text-classifier-on-ai-platform-serving-using-custom-online-prediction
AI in Depth: Serving a PyTorch text classifier on AI Platform Serving using custom online prediction | Google Cloud Blog
April 25, 2019 - This class, along with the trained model and its corresponding preprocessing object, will be used to deploy the AI Platform Serving microservices. The following code shows how the Custom Model Prediction class (CustomModelPrediction) for our text classification example is implemented in the model_prediction.py module. ... Your saved (trained) model file: trained_saved_model.pt (see Training and Saving the PyTorch model).
DeepLearning.AI
community.deeplearning.ai › ai discussions
PyTorch on Google Cloud: How to deploy PyTorch models on Vertex AI - AI Discussions - DeepLearning.AI
September 30, 2023 - here is the link - https://cloud.google.com/blog/topics/developers-practitioners/pytorch-google-cloud-how-deploy-pytorch-models-vertex-ai by Google on how to deply pytorch models using vertex ai , but I’m not understanding a lot of things , need a dedicated tutorial on this particularly , ...
GitHub
github.com › OPAYA › PytorchModelServing
GitHub - OPAYA/PytorchModelServing: Deploy pytorch model using Goole Cloud Service · GitHub
$ PROJECT_ID = $(gcloud config list project --format "value(core.projcet)") $ BUCKET="${PROJECT_ID}-ml" #create bucker $ gsutil mb -c regional -l us-central1 gs://${BUCKET} #upload saved model $ gsutil -m cp -R save_model/model_param.pth gs://${BUCKET}
Author OPAYA