Cloud platform for python
Cloud computing service for Python - Stack Overflow
What is the easiest way to run python scripts in a cloud server? - Stack Overflow
What level of python do cloud engineers require?
Videos
Hello all,
I apologize if this the wrong community to post this in and if it is what is the best community to post this question in?
I am at a new startup and know very little about python. I have been tasked with finding out what is the best cloud platform for running python. Google, Aws, Azure, etc. Functionality, performance, and price are all factors. What are your opinions on the subject?
Since you said that performance is a problem and you are doing web-scraping, first thing to try is a Scrapy framework - it is a very fast and easy to use web-scraping framework. scrapyd tool would allow you to distribute the crawling - you can have multiple scrapyd services running on different servers and split the load between each. See:
- Distributed crawls
- Running Scrapy on Amazon EC2
There is also a Scrapy Cloud service out there:
Scrapy Cloud bridges the highly efficient Scrapy development environment with a robust, fully-featured production environment to deploy and run your crawls. It's like a Heroku for Scrapy, although other technologies will be supported in the near future. It runs on top of the Scrapinghub platform, which means your project can scale on demand, as needed.
As an alternative to the solutions already given, I would suggest Heroku. You can not only deploy easily a website, but also scripts for bots to run.
Basic account is free and is pretty flexible.
This blog entry, this one and this video contain practical examples of how to make it work.
Hello, I'm interested in cloud engineering which usually requires Python that i intend to learn. I read somewhere that you don't need certain aspects e.g. maths/calculation part. So my question is that is there specific areas i should focus on learning for cloud engineering?