Which service fits the most here?
imho you could do a PoC to see which service is more feasible for you. It really depends on how much data you have, what queries and what load you plan to execute.
AWS Redshift is intended for OLAP on top of peta- or exa-bytes scale handling heavy parallel workload. RS can as well aggregate data from other data sources (jdbc, s3,..). However RS is not OLTP, it requires more static server overhead and extra skills for managing the deployment.
So without more numbers and use cases one cannot advice anything. Cloud is great that you can try and see what fits you.
Answer from gusto2 on Stack OverflowHi all,
I have a business request to ETL data from several sources into a data warehouse to run BI tools on (Tableau or PowerBI for example).
The dataset isn't that big, 400GB growing 4GB per week. I can't find any good explanation on if I should use RDS or Redshift to store the data.
Can anyone with experience on this topic guide me please?
Thank you!
Videos
What is the difference between AWS RDS and Redshift?
Redshift is a cloud data warehouse designed for analytical workloads, handling large-scale data queries and complex aggregations.
Is AWS Redshift an RDS?
Should I use RDS or Redshift?
Which service fits the most here?
imho you could do a PoC to see which service is more feasible for you. It really depends on how much data you have, what queries and what load you plan to execute.
AWS Redshift is intended for OLAP on top of peta- or exa-bytes scale handling heavy parallel workload. RS can as well aggregate data from other data sources (jdbc, s3,..). However RS is not OLTP, it requires more static server overhead and extra skills for managing the deployment.
So without more numbers and use cases one cannot advice anything. Cloud is great that you can try and see what fits you.
AWS Redshift is really great when you only want to read the data from the database. Basically, Redshift in the backend is a column-oriented database that is more suitable for analytics. You can transfer all your existing data to redshift using the AWS DMS. AWS DMS is a service that basically needs your bin logs of the existing database and it will automatically transfer your data we don't have to do anything. From my Personal experience Redshift is really great.
RDS is a managed service for Online Transaction Processing databases (OLTP), i.e. a managed service for the usual MySQL, PostgreSQL, Oracle, MariaDB, Microsoft SQL Server or Aurora (Amazon's own relational database)
Redshift is a managed service for data warehousing, i.e. columnar oriented storage, typical for business analytics type of workloads.
It depends on your usecase.
Put it in example:
If you use Select * from <table> more often, go for RDS. (OLTP)
If you use features like SUM(A) more often, go for RedShift. (OLAP)