Use skiprows=[1] in read_csv to skip the row 1, the conversion to float should be automatic:

df = pd.read_csv('test.csv', sep = ';', decimal = ',', skiprows=[1])

output:

print(df)
     Speed        A
0      700 -72.5560
1      800 -58.9103
2      900 -73.1678
3     1000 -78.2272

print(df.dtypes)
  Speed      int64
A          float64
dtype: object

Why your code did not work

When reading the "[rpm];[N.m]" line, the csv parsers determines that the column(s) are strings, not floats. So the decimal specifier is simply ignored and the -72,556 values remain as string, with the comma.

The following would be useful,

print (heterozygosity_df.columns)

This looks like static typing issue within pandas. What I suspect is heterozygosity_df.['chrI'] is a column in the dataframe. What I think has happened is there's a mix of strings and floats within this column. pandas has set this as a "string" but you are wanting to perform numerical operations. Thus the solution is simply

print(heterozygosity_df.dtypes) # this should state 'chrI' is a "category" or "string"
heterozygosity_df['chrI'] = heterozygosity_df['chrI'].astype(float)
print(heterozygosity_df.dtypes)

If you have multiple changes the syntax is

heterozygosity_df = heterozygosity_df.astype({'chrI':'float', 'egColumn':'category'})

I suspect there will be other errors, e.g. the header is the first row of the data column. This is because again for pandas to automatically assign a column to a "string" means there must be a string value within the column.

From the comments. I see whats happening. The easy solution is ...

replacement = {
    "chrI": 1,
    "chrII": 2,
    "chrIII": 3,
    "chrIV": 4,
    ....
}

heterozygosity_df['chr'] = heterozygosity_df['chr'].str.replace(replacement, regex=True)

From the comments ... good it works ... this is what I would have personally done ..

replacement = {
    "chrI": 1,
    "chrII": 2,
    "chrIII": 3,
    "chrIV": 4, # continue for all chromosomes
}

heterozygosity_df = pd.read_csv("file.tsv", sep="\t", header=None).set_axis(['chr', 'pos', 'het'], axis=1, copy=False)
heterozygosity_df['chr'] = heterozygosity_df['chr'].str.replace(replacement, regex=True).astype('int')

Normally str should be in place because when 'chr' is imported its an object. However, if it works thats the only thing that counts.

I found the mistake. The problem was a thousand separator.

When writing the CSV file, most numbers were below thousand and were correctly written to the CSV file. However, this one value was greater than thousand and it was written as "1,123" which pandas did not recognize as a number but as a string.

The source data file is not clean. You should read in the file first and then parse to float.

import pandas as pd

df = pd.read_csv('kidney_disease.csv')
cols = ['pcv','wc','rc']
df = df[cols]
for col in cols:
    df[col] = pd.to_numeric(df[col],downcast='float',errors='coerce')
print(df.dtypes)

Output

pcv    float32
wc     float32
rc     float32
dtype: object

This will result in nan values where strings could not be converted. You should examine your dataset to see what other cleaning may be required.

These strings have commas as thousands separators so you will have to remove them before the call to float:

df[column] = (df[column].str.split()).apply(lambda x: float(x[0].replace(',', '')))

This can be simplified a bit by moving split inside the lambda:

df[column] = df[column].apply(lambda x: float(x.split()[0].replace(',', '')))

After replacing the ? with np.NaN, you will be able to convert the type of the BareNuc column from object to float (as np.NaN is considered a special floating-point value and cannot be converted to any other type than float). with that, you wouldn’t need to convert to a NumPy array.

df = df.astype({"BareNuc": float})

At the end, all columns will be used as float type, so that wouldn’t affect the resource allocation.