You're unlikely to find a consumer gpu option from any of the major cloud hypervisors. More on reddit.com

My system is using the EC2 g4dn.xlarge instance type to run stable diffusion or train lora. Currently I want to find an NVIDIA GPU type version that costs less than g4dn.xlarge. Lower configuration... More on repost.aws

Have you checked what disk type? Using the same disk type and size? Gp3 gives more performance the larger your disk is. Or io3 and provisioned iops could be what you are looking for. It just might be expensive. But yeah I’d look at your disk regarding the issue of being slower on a faster instance type. More on reddit.com

I don't have the exact answer, but I can give you the place to look at if you want to work out the answer on your own. First you should be selecting an opensource LLM. For that I suggest taking a look at: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard For your use case I think it's safe to grab something with 7B parameters or lower. Some suggestions for the model (this is by no mean an exhausting list): - Mistral 7B: https://huggingface.co/mistralai/Mistral-7B-v0.1 - Llama 2 7B: https://huggingface.co/meta-llama/Llama-2-7b-hf - Microsoft Phi-2: https://huggingface.co/microsoft/phi-2 Then to answer your question about hardware take a look here: https://huggingface.co/spaces/Vokturz/can-it-run-llm This should give you a rough idea of the amount of VRAM you need to fit these models on a GPU. For example you are able to run Mistral 7B quantized to 8-bit on a single V100 GPU (p3.2xlarge ec2 instance). More about quantization: https://huggingface.co/docs/accelerate/usage_guides/quantization More on reddit.com