MongoDB Vs Cassandra Vs PostgreSQL Vs MySQL - Stack Overflow
postgresql - For what type of data it's better to use relational, and for what type of data, non-relational databases? - Database Administrators Stack Exchange
SQL Versus NoSQL Databases: Which to Use, When, and Why
Depends.
More on reddit.comCassandra vs MongoDB vs CouchDB vs Redis vs Riak comparison
I feel like there's as many nosql versions as linux distros... Thanks for the find.
More on reddit.comWhich database performs better - PostgreSQL or Cassandra?
Is Cassandra faster than PostgreSQL?
What is the main difference between PostgreSQL and Cassandra?
Hi everyone,
I’m working on an e-commerce project with a large dataset – 20-30 million products per user, with a few thousand users. Data arrives separately as products, stock, and prices, with updates every 2 hours ranging from 2,000 to 4 million records depending on the supplier.
Requirements:
-
Extensive filtering (e.g., by warehouse,
LIKEqueries, keyword searches). -
High performance for both reads and writes, as users need to quickly search and access the latest data.
I’m deciding between SQL (e.g., PostgreSQL with advanced indexing and partitioning) and NoSQL (e.g., MongoDB or Cassandra) for better scalability and performance with large, frequent updates.
Does anyone have experience with a similar setup? Any advice on structuring data for optimal performance?
Thanks!
may I know how to decide for what services or what type of data/application it's better to use SQL or NoSQL databases?
Use a SQL database if any combination of the following is true:
- You have relational data
- Your schema is under your control
- The schema you're adhering to changes at a rate that's tolerable for you to manage
Use a NoSQL database otherwise. Specifically if your schema is ill-defined and changes more frequently than you're willing to manage.
There's not many differences in use cases otherwise. NoSQL is somewhat a subset of SQL but they compliment each other in different ways too.
What about horizontal scaling? I've read that RDBMS databases have big issues in horizontal scaling for big, rapid growing projects.
Horizontal scaling aka sharding is a little gimmicky, in my honest opinion. There's a lot of room for vertical scaling with servers these days, especially with virtualization and / or cloud services.
MongoDB is one of the most mainstream NoSQL databases that are thought of when the topic of horizontal scaling / sharding comes to mind. Many inexperienced people like to exclaim how sharding makes scaling easier, cheaper, or both. But even the developers of MongoDB claim the contrary, stating sharding is problematic, hard to manage, limiting on what queries can be ran, and they even go as far as claiming vertical scaling is just more practical and cost effective (myth #5):
Sharded clusters also make your data harder to manage, and they add some limitations to the types of queries you can conduct. Sharding is useful if you need it, but it's often cheaper and easier to simply upgrade your hardware!
In fact, that same article discusses how a minimum of 8 servers are needed to properly setup sharding with MongoDB. That's surely going to be more costly with the redundancy in paying for all of the hardware in a full system for each of the 8 servers as opposed to vertically scaling your infrastructure for a specific piece of hardware. Of course there are exception cases, but generally speaking, it's hard for me to see it being cost efficient for the average user.
But even modern SQL databases support various implementations with horizontal scaling, should one feel the need to utilize this methodology. For example, SQL Server has a feature called Availability Groups which automatically synchronizes the data from one server to the other replica servers in the same group. It even offers two modes of synchronization depending on if you prioritize performance over data consistency between servers (similar, though not exactly the same, as the idea of eventual consistency normally seen with NoSQL database synchronization in a sharded topology).
Aside from all of that, when a database is optimally architected, with efficiently designed queries, a SQL database can handle a high level of concurrency for large amounts of data. SQL Server is rated to handle tables with trillions of records. I've personally worked with tables in the 10s of billions on servers that were fairly heavily transactional (1,000s of transactions per minute), with minimal hardware (16 GB of Memory, 8 CPU cores, etc), and most queries ran in under 1 second.
Some key ideas behind this are that data at rest can be any size, and it doesn't really matter from a performance perspective (for most use cases). Proper indexing which uses a B-Tree data structure, allows for optimal searching of any achievable size of data (even by big data standards in 100 years from now). That's because B-Tree's have an O(log(n)) search time complexity. That means if your table has 1 billion rows, in the worst case, it would take log2(1 billion) = 30 to find any subset of the data. If that table grew to 1 trillion rows, log2(1 trillion) = 40 to find any subset. 30 and 40 nodes of a B-Tree takes a few milliseconds for any modern computer, even with the hardware of a cellphone, to search through.
Long story short, there are no differences from a performance perspective in what any modern SQL database is capable of vs a modern NoSQL database. Most data scale problems are usually best solvable on the software side of the equation, not really the hardware side, as most performance problems are caused by poor database design / implementation and / or poorly written code (queries).
Full disclosure: I am an employee of DataStax, and a Cassandra MVP (so I'll do my best to hide any bias).
They've presented on this publicly a few times (see below), but at a high level, Instagram uses Cassandra for two big reasons:
- Scale
- Geographic Awareness
The way Cassandra was designed to work, is that each node in the cluster is a peer. That is, there is not a "primary" or "main" node which handles operations. Every node can handle a read or write operation, with data being replicated or coordinated from additional replicas as required.
This means that scaling horizontally is easy for something like Cassandra. As Instagram serves more than a billion daily users, from all over the world. So using the denormalized, decentralized approach of Cassandra works well in that case. Cassandra distributes data by hashing the partition key(s), and then only sends the data to the instances responsible for those hash ranges. This is why Cassandra doesn't need a sharded approach.
Of course, Instagram has made smart decisions around their data model. This means storing things together which are queried together, and ensuring that a query can be served by a single node (instance).
Instagram also has users around the world. In an effort to cut down on data retrieval times, they utilized data centers in many parts of the world to keep data geographically close to their users. Cassandra makes a good choice for this, due to its ability to separate nodes in a cluster into logical data centers (which are often geographic/cloud regions).
However, they did run into some trouble with quorum operations which started to span multiple regions. To get around that, they opted to only replicate data to certain regions.
That all being said, it's unlikely that you will need to support that type of global usage (at least not right away). So I would say not to concern yourself with Cassandra at this point. If you find yourself needing to go that route at some point, I can say that the migration will be easier if your data model has fewer join operations.
Here's a blog post taken from the DataStax Accelerate conference in 2020: How Instagram uses Cassandra to operate on a Global Scale
Here's a writeup that they did about their database infra from the context of moving from AWS to Facebook's infra: Intstagration Pt. 2: Scaling our Infrastructure to Multiple Data Centers. This article contains some infra-level details about both Cassandra and Postgres (and a few others). It doesn't go into detail on use cases, but does have links to other supporting posts.
More details available on YouTube:
- Cassandra Traffic Management at Instagram
- Cassandra @ Instagram (2014)