It would be nice to see your train_generator code for clarity, but it does not seem to be a torch DataLoader. In this case, you should probably convert your arrays to tensors manually. There are several ways to do so:

• torch.from_numpy(numpy_array) - for numpy arrays;
• torch.as_tensor(list) - for common lists and tuples;
• torch.tensor(array) should also work but the above ways will avoid copying the data when possible.

It would be helpful if you could post the full stack trace, so that we can see which line your error occurs at. In general, the more information you can provide in a question, the better.

In this case, it looks like your full_model_pipeline may somehow become a numpy array. Since you have a one-element pipeline, you could try changing

full_model_pipeline = Pipeline(steps =[
                        ('full_pipeline',full_pipeline),
                        ('model',LinearRegression())
                    ])

full_model_pipeline.fit(X_train,y_train)

to

model = LinearRegression()
model.fit(X_train, y_train)

There is nothing wrong with your data, your main mistake is that you need to pay attention to the plot function used. This link tells you how to plot your data. https://matplotlib.org/tutorials/introductory/pyplot.html

However I would change the last line to something like:

import matplotlib.pyplot as plt
plt.plot(y_rus)
plt.show()

or something like:

plt.scatter(y_rus)
plt.show()

In general is a bit difficult to understand what exactly you want or the format of the data, so I hope this works. Follow the tutorial it will help you understand what you really need to continue.

You are using the right method but in a wrong way :)

nan_to_num is a method of numpy module, not numpy.ndarray. So instead of calling nan_to_num on you data, call it on numpy module giving your data as a paramter:

import numpy as np
data = np.array([1,2,3,np.nan,np.nan,5])
data_without_nan = np.nan_to_num(data)

prints:

array([1., 2., 3., 0., 0., 5.])

In your example:

import numpy as np
val = np.nan_to_num(setTo.ravel())

The solution:

The given dataset was already an array, so I didn’t need to call .values.

if you need to flatten you can directly use flatten() on numpy.ndarray object

X = X.flatten()

Link to doc: https://numpy.org/doc/stable/reference/generated/numpy.ndarray.flatten.html