Why use MongoDB over Cassandra?
How is Cassandra different from MongoDB?
Are MongoDB and Cassandra the same?
So at work, we no longer can use Mongo because of some licence issues. So we were looking into cassandra.
But more I use it, more it seems like it shouldn't be used as a primary database. Our systems are fairly nascent, so we don't know what all fields we will query with in a table. And given how you can only query with keys in cassandra (or be Okey with secondary indexes), it seems like I will have to keep creating newer tables just to hold mapping between those fields I want to query.
It's just too restrictive for whatever we were doing with mongo.
Are these observations valid? Or can you really use just the cassandra as a primary database?
We are working on a chat application for enterprise (imagine Google Workspace chat or Slack kinda application - for desktop and mobile). Of course we are just getting started, so one might suggest choosing a barebone DB and some basic tools to launch the app, but anticipating traffic, we want to distill the best knowledge available out there and choose the best stack to build our product from the beginning.
For our chat application, where all typical user behaviors are there - messages, spaces, "last seen" or "active" statuses, message notifications, read receipts, etc. we need to choose a database to store all our chats. We also want to enable chat searches, and since search will inevitably lead to random chats, we want that perf to be consistently excellent.
We are planning to use Django (with channels) as our backend. What database is recommended to use with Django to persist the messages? I read that Discord used to use Cassandra, but then it started acting up due to garbage collection, so they switched rto Scylla, and they are very happy with trillions of messages on it. Is ScyllDB a good candidate for our purpose to use with Django? Do these two work together well? Can MongoDB do it (my preferred choice, but I read that it starts acting up with high number of reads or writes at the same time - which would be a basic use case for enterprise chat scenario)?
For a harvesting solution like this, I would recommend a multi-stage approach. Redis is good at real time communication. Redis is designed as an in-memory key/value store and inherits some very nice benefits of being a memory database: O(1) list operations. For as long as there is RAM to use on a server, Redis will not slow down pushing to the end of your lists which is good when you need to insert items at such an extreme rate. Unfortunately, Redis can't operate with data sets larger than the amount of RAM you have (it only writes to disk, reading is for restarting the server or in case of a system crash) and scaling has to be done by you and your application. (A common way is to spread keys across numerous servers, which is implemented by some Redis drivers especially those for Ruby on Rails.) Redis also has support for simple publish/subscribe messenging, which can be useful at times as well.
In this scenario, Redis is "stage one." For each specific type of event you create a list in Redis with a unique name; for example we have "page viewed" and "link clicked." For simplicity we want to make sure the data in each list is the same structure; link clicked may have a user token, link name and URL, while the page viewed may only have the user token and URL. Your first concern is just getting the fact it happened and whatever absolutely neccesary data you need is pushed.
Next we have some simple processing workers that take this frantically inserted information off of Redis' hands, by asking it to take an item off the end of the list and hand it over. The worker can make any adjustments/deduplication/ID lookups needed to properly file the data and hand it off to a more permanent storage site. Fire up as many of these workers as you need to keep Redis' memory load bearable. You could write the workers in anything you wish (Node.js, C#, Java, ...) as long as it has a Redis driver (most web languages do now) and one for your desired storage (SQL, Mongo, etc.)
MongoDB is good at document storage. Unlike Redis it is able to deal with databases larger than RAM and it supports sharding/replication on it's own. An advantage of MongoDB over SQL-based options is that you don't have to have a predetermined schema, you're free to change the way data is stored however you want at any time.
I would, however, suggest Redis or Mongo for the "step one" phase of holding data for processing and use a traditional SQL setup (Postgres or MSSQL, perhaps) to store post-processed data. Tracking client behavior sounds like relational data to me, since you may want to go "Show me everyone who views this page" or "How many pages did this person view on this given day" or "What day had the most viewers in total?". There may be even more complex joins or queries for analytic purposes you come up with, and mature SQL solutions can do a lot of this filtering for you; NoSQL (Mongo or Redis specifically) can't do joins or complex queries across varied sets of data.
I currently work for a very large ad network and we write to flat files :)
I'm personally a Mongo fan, but frankly, Redis and Cassandra are unlikely to perform either better or worse. I mean, all you're doing is throwing stuff into memory and then flushing to disk in the background (both Mongo and Redis do this).
If you're looking for blazing fast speed, the other option is to keep several impressions in local memory and then flush them disk every minute or so. Of course, this is basically what Mongo and Redis do for you. Not a real compelling reason to move.