I am trying to impute missing values as the mean of other values in the column; however, my code is having no effect. Does anyone know what I may be doing wrong? Thanks! My code: from sklearn. More on stackoverflow.com

so this code is replacing the value of date with last update if 'yyy' is present in visited column. but the task is if we have date missing for rajesh now we will check does rajesh has any entry in visited column with name yyy if yes then his missing date will get replace with latest update 2022-04-25T15:45:14.47Z+00:00 ... Find the answer to your question by asking. Ask question ... See similar questions with these tags. ... 2 I need to impute ... More on stackoverflow.com

Generally you have 3 options: remove the row entirely, fill the missing value, or some machine learning algorithms can work even when you have missing entries. When you have more than enough data, just remove the rows with empties. Like if I had 2 million rows of data with a random 10% of them empty, I generally wouldn't break a sweat over removing them; it's still a ton of data. 900 is still a pretty good amount of data, so you might consider this. There are a few ways to fill the empties, depending on the data you've got. You could use an average of some kind: mean, median, or mode. The right approach depends on the distribution of the data, so you should plot the data to take a look. For instance, if the attribute has a lot of skewness or otherwise isn't normally distributed, a mean could be the wrong choice. Generally, the idea behind filling the empties with an average is to replace them with benign data close to/at the center of the distribution, so ML algorithms won't be pulled one way or the other. I personally use this approach when I don't have much data, both in terms of total rows as well as columns - too few rows to simply remove them, too few columns to use the next approach, which is imputation. You could impute the missing data, like with k-Nearest Neighbors imputation, through which you give the data to KNN to predict what the missing values ought to be. This approach can provide higher accuracy in certain cases, but I'd only advise it if you have enough columns in your data such that you'd be able to arrive at accurate predictions. For instance, if you have a weather dataset with many columns of pertinent info to predicting temperatures like latitude/longitude, dates, dew point, pressure, etc. etc., and some of your rows have missing data, the imputation would probably work well, because the other attributes in the data are sufficient to predict the empty values. Finally, some algorithms are totally cool with missing values or you can tell them to ignore them. Hope this helps EDIT: I forgot to mention...the cool thing about learning data science is that you get to try all the approaches and compare results. Give them all a shot and see what works best. Record what did and the features of the data. Then when you do another analysis with a different dataset see if a different approach works out better. Compare and contrast depending on the data you have and you'll get the hang of it. More on reddit.com

The way I’ve done it thus far is impute a column based on its distribution (median if the data is skewed) or fill it with the mean. I never drop missing values. This is nowhere near an ideal approach. Depending on the exact nature of your data you'll want to use different techniques to impute missing information, but at the absolute very least you should be using kNN as baseline. Mean and median imputation consistently perform the worst out of common techniques. The only method I'm aware of that has lower performance than mean and median is simply randomly taking another datapoint and filling that in. On the other hand, kNN has been demonstrated in multiple research papers to be a top performer in improving the precision of the final result when used with missing data, and to provide a low degree of distortion. The best technique isn't always kNN, you'll see good performance with predictive mean matching, 1NN, fuzzy mean matching, bayesian PCA, etc depending on the circumstances. However, kNN works as a very decent and more importantly consistently high performing starting point for filling missing values. This hold true when applied across a wide range of datasets both in terms of content and size. One approach worth considering when evaluating which technique to use is to simply drop data on purpose, and then apply a variety of filling techniques as mentioned above. You can then compare the distortion, precision, and errors to the known value and see which method gives the best results on your particular data. This is what I recommend personally. More on reddit.com