Update:
- This Mode function is for dataframes:
my_mode <- function (x, na.rm) {
xtab <- table(x)
xmode <- names(which(xtab == max(xtab)))
if (length(xmode) > 1) xmode <- ">1 mode"
return(xmode)
}
for (var in 1:ncol(cat_df)) {
if (class(cat_df[,var])=="numeric") {
cat_df[is.na(cat_df[,var]),var] <- mean(cat_df[,var], na.rm = TRUE)
} else if (class(cat_df[,var]) %in% c("character", "factor")) {
cat_df[is.na(cat_df[,var]),var] <- my_mode(cat_df[,var], na.rm = TRUE)
}
}
This mode function is for vectors Try this and please let me know.
#define missing values in vector
values <- unique(cat_column)[!is.na(cat_column)]
# mode of cat_column
themode <- values[which.max(tabulate(match(cat_column, values)))]
#assign missing vector
imputevector <- cat_column
imputevector[is.na(imputevector)] <- themode
Answer from TarJae on Stack Overflowr - Mode imputation for categorical variables in a dataframe - Stack Overflow
python - Imputation of missing values for categories in pandas - Stack Overflow
Imputation for categorical values (Specifically via KNN imputation)
Imputation whilst accounting for phylogenetic relatedness
You're going to have to assume a particular model of evolution to do this. There's no universal rule for defining how trait similarity should decay with phylogenetic distance--it depends on the organisms and traits in question.
Brownian Motion is a useful (universally used if not necessarily realistic) starting point. I know Rphylopars is supposed to be able to deal with missing species data (and I think return estimated species means and standard errors). Have you looked into using that to simulate your missing trait values?
More on reddit.comUpdate:
- This Mode function is for dataframes:
my_mode <- function (x, na.rm) {
xtab <- table(x)
xmode <- names(which(xtab == max(xtab)))
if (length(xmode) > 1) xmode <- ">1 mode"
return(xmode)
}
for (var in 1:ncol(cat_df)) {
if (class(cat_df[,var])=="numeric") {
cat_df[is.na(cat_df[,var]),var] <- mean(cat_df[,var], na.rm = TRUE)
} else if (class(cat_df[,var]) %in% c("character", "factor")) {
cat_df[is.na(cat_df[,var]),var] <- my_mode(cat_df[,var], na.rm = TRUE)
}
}
This mode function is for vectors Try this and please let me know.
#define missing values in vector
values <- unique(cat_column)[!is.na(cat_column)]
# mode of cat_column
themode <- values[which.max(tabulate(match(cat_column, values)))]
#assign missing vector
imputevector <- cat_column
imputevector[is.na(imputevector)] <- themode
User Defined Function
Here is the mode function I use with an additional line to choose a single mode in the event there are actually multiple modes:
my_mode <- function(x) {
ux <- unique(x)
tab <- tabulate(match(x, ux))
mode <- ux[tab == max(tab)]
ifelse(length(mode) > 1, sample(mode, 1), mode)
}
# single mode
cat_col_1 <- c(1, 1, 2, NA)
cat_col_1
#> [1] 1 1 2 NA
cat_col_1[is.na(cat_col_1)] <- my_mode(cat_col_1)
cat_col_1
#> [1] 1 1 2 1
# random sample among multimodal
cat_col_2 <- c(1, 1, 2, 2, NA)
cat_col_2
#> [1] 1 1 2 2 NA
cat_col_2[is.na(cat_col_2)] <- my_mode(cat_col_2)
cat_col_2
#> [1] 1 1 2 2 2
DescTools::Mode()
But other folks have written mode functions. One possibility is in the DescTools package and is named Mode().
Because it returns multiple modes in the event there are more than one, you would need to decide what to do in that event.
Here is an example to randomly sample with replacement, the necessary number of modes to replace the missing values.
# single mode
cat_col_3 <- c(1, 1, 2, NA)
cat_col_3
#> [1] 1 1 2 NA
cat_col_3_modes <- DescTools::Mode(cat_col_3, na.rm = TRUE)
cat_col_3_nmiss <- sum(is.na(cat_col_3))
cat_col_3[is.na(cat_col_3)] <- sample(cat_col_3_modes, cat_col_3_nmiss, TRUE)
cat_col_3
#> [1] 1 1 2 1
# random sample among multimodal
cat_col_4 <- c(1, 1, 2, 2, NA, NA)
cat_col_4
#> [1] 1 1 2 2 NA NA
cat_col_4_modes <- DescTools::Mode(cat_col_4, na.rm = TRUE)
cat_col_4_nmiss <- sum(is.na(cat_col_4))
cat_col_4[is.na(cat_col_4)] <- sample(cat_col_4_modes, cat_col_4_nmiss, TRUE)
cat_col_4
#> [1] 1 1 2 2 2 1
Created on 2021-04-16 by the reprex package (v1.0.0)
You can use df = df.fillna(df['Label'].value_counts().index[0]) to fill NaNs with the most frequent value from one column.
If you want to fill every column with its own most frequent value you can use
df = df.apply(lambda x:x.fillna(x.value_counts().index[0]))
UPDATE 2018-25-10 ⬇
Starting from 0.13.1 pandas includes mode method for Series and Dataframes.
You can use it to fill missing values for each column (using its own most frequent value) like this
df = df.fillna(df.mode().iloc[0])
def fillna(col):
col.fillna(col.value_counts().index[0], inplace=True)
return col
df=df.apply(lambda col:fillna(col))