Hi all,
I have a business request to ETL data from several sources into a data warehouse to run BI tools on (Tableau or PowerBI for example).
The dataset isn't that big, 400GB growing 4GB per week. I can't find any good explanation on if I should use RDS or Redshift to store the data.
Can anyone with experience on this topic guide me please?
Thank you!
amazon web services - AWS Redshift or RDS for a Data warehouse? - Stack Overflow
Deciding between RDS and Redshift for "data warehouse"
DynamoDB or Aurora or RDS?
RDS vs Aurora big price difference
What is the difference between Redshift and RDS?
Performance: Redshift utilizes a columnar storage format and is optimized for complex queries and large datasets, whereas RDS is optimized for high-speed transactions and relational data operations.
Scalability: Redshift can scale to handle petabyte-scale data warehouses, while RDS is more focused on scaling for standard database applications.
Which is better, Aurora or Redshift?
Is Aurora OLAP or OLTP?
Which service fits the most here?
imho you could do a PoC to see which service is more feasible for you. It really depends on how much data you have, what queries and what load you plan to execute.
AWS Redshift is intended for OLAP on top of peta- or exa-bytes scale handling heavy parallel workload. RS can as well aggregate data from other data sources (jdbc, s3,..). However RS is not OLTP, it requires more static server overhead and extra skills for managing the deployment.
So without more numbers and use cases one cannot advice anything. Cloud is great that you can try and see what fits you.
AWS Redshift is really great when you only want to read the data from the database. Basically, Redshift in the backend is a column-oriented database that is more suitable for analytics. You can transfer all your existing data to redshift using the AWS DMS. AWS DMS is a service that basically needs your bin logs of the existing database and it will automatically transfer your data we don't have to do anything. From my Personal experience Redshift is really great.