I am beginner to data mining. This is my understanding, regression is used to predict continuous values. It is a type of classification. Classification is Supervised and Clustering is unsupervised.... More on stats.stackexchange.com

One thing that might help, is thinking of this in terms of decision boundaries in the feature space. Someone else just so happened to post a question with a useful visualization here . Looking at this, we can see that there's basically two overlapping ellipses. Typically in a classification problem, what you're doing is splitting the entire feature space into two regions. The decision boundary is the (possibly non-linear) boundary (a line in this case, since we have a 2d feature space) that splits the universe of possible observations into two categories. Things to this side of the line (men) and things to that side of the line (women). Anomaly detection on the other hand, has different needs. After all, think about it... for the above example, we don't have a third bubble here we're interested in classifying. We're interested in ANY point that seems to be 'unusual' compared to the things we're expecting to see. You might have really weird ratios between weight and height, or maybe you've got some really tall or really short or really heavy people or whatever. Maybe the things you're flagging will end up being data entry errors instead of real human data. Either way, there might be many kinds of 'outliers', there won't be a single bubble you'll expect them to inhabit, so much as you're expecting them to be outside all the known bubbles. You're also likely to have an extremely imbalanced dataset. Maybe you'll have 20,000 records of sounds your turbines make, with 20 samples of 'weird noises' that went before failure. 20 'anomaly' samples is too few to use in a classification approach. So here's generally the approach you'll see instead. From a probabilistic perspective, in the above example we've got two generating distributions, both roughly Gaussian. (This is a very similar picture to the commonly seen 'old faithful' geyser dataset). The idea now, instead of drawing a decision boundary splitting the world of observations into two regions, instead we can train two distributions in the feature space. One for men (centered in the middle of the bubble of men observations, with whatever covariance matrix would best fit the data) and one for women. Here's the cool thing we get from that now: We can use these generating distributions to do classification easily enough. For any point, we can just see if it's a more common thing to see for the 'men' distribution, or the 'women' distribution. We can draw a decision boundary using this, and we've basically just done a maximum likelihood estimation classifier. It'll even be a straight line decision boundary, since you can approximate both covariance matrices as being equal in this case (the ellipses for the men and women observations are very similarly shaped). The math works out where there's a linear MLE decision boundary between two gaussians if the covariance matrices are equal. But, since we learned the full generating distributions instead of just a decision boundary, we get extra power for the extra effort we put in. For any new point, we can ask how likely that point is among the various kinds of things we might expect to see. Is this point a common height/weight observation for men? Hm... no. Is it common for women? Also no. Since we're now talking about a point that's fairly low probability for BOTH of our two classes of things, we call this an anomalous observation. That's what anomaly detection fundamentally is after all. It's not a new category of things. It's observations that fall outside all the known categories of things, if that makes sense. For what it's worth, we can actually draw this decision boundary too, in that above picture of men vs women. Imagine drawing an elipse around the 'men' ellipse. Everything inside is within 2 standard deviations of the mean for that cluster (or whatever you want your 'unlikely' cutoff to be). Now draw another ellipse around the 'women' cluster in the same way. Now take the intersection of the complement of those two sets. You'll get the entire feature space, with two ellipses cut out where our two categories live. This is our anomaly space. The vast wilderness outside what's known. Anything we see in that wilderness is what we want to flag as an anomaly, so you can see why we want slightly different tools than what's normally used in classification. I think you can imagine now too... this is a fairly challenging problem in a high dimensional space, like for generator sounds or whatever, but it's nice having visual examples like this to start with at least. If you'd like to know some of the theoretical background for all of this, the first chapter in Bishop's pattern recognition and machine learning would be a great read. Prereqs are some basic comfort with probability theory, and hopefully at least some comfort with formal proofs. It's not too terrible considering the level of rigor the author uses, check it out if you're wanting to know more background. More on reddit.com

Classification: does the input map to a specific known category? Regression: what's the numerical output given the values for features assuming other output for other data points are known? More on reddit.com

Yes. I had to categorize new products based on a set of features so that we could accurately price it for the market. More on reddit.com