if you need to flatten you can directly use flatten() on numpy.ndarray object

X = X.flatten()

Link to doc: https://numpy.org/doc/stable/reference/generated/numpy.ndarray.flatten.html

It would be nice to see your train_generator code for clarity, but it does not seem to be a torch DataLoader. In this case, you should probably convert your arrays to tensors manually. There are several ways to do so:

• torch.from_numpy(numpy_array) - for numpy arrays;
• torch.as_tensor(list) - for common lists and tuples;
• torch.tensor(array) should also work but the above ways will avoid copying the data when possible.

You are using the right method but in a wrong way :)

nan_to_num is a method of numpy module, not numpy.ndarray. So instead of calling nan_to_num on you data, call it on numpy module giving your data as a paramter:

import numpy as np
data = np.array([1,2,3,np.nan,np.nan,5])
data_without_nan = np.nan_to_num(data)

prints:

array([1., 2., 3., 0., 0., 5.])

In your example:

import numpy as np
val = np.nan_to_num(setTo.ravel())

The problem is that train_test_split(X, y, ...) returns numpy arrays and not pandas dataframes. Numpy arrays have no attribute named columns

If you want to see what features SelectFromModel kept, you need to substitute X_train (which is a numpy.array) with X which is a pandas.DataFrame.

selected_feat= X.columns[(sel.get_support())]

This will return a list of the columns kept by the feature selector.

If you wanted to see how many features were kept you can just run this:

sel.get_support().sum()  # by default this will count 'True' as 1 and 'False' as 0

Using .values on a pandas dataframe gives you a numpy array. This will not contain column names and such. You do this when setting X like this:

X = dataset[['Read?', 'x1', .. ,'x47']].values 

But then you try to get the column names from X (which it does not have) by writing X.columns here:

coeff_df = pd.DataFrame(regressor.coef_, X.columns, columns=['Coefficient'])

So store your column names in a variable or input them again, like this:

coeff_df = pd.DataFrame(regressor.coef_, ['Read?', 'x1', .. ,'x47'], columns=['Coefficient'])