StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POremoving NaN values in python pandas
text
Body
copied!<p>Data is of income of adults from census data, rows look like:</p> <pre><code>31, Private, 84154, Some-college, 10, Married-civ-spouse, Sales, Husband, White, Male, 0, 0, 38, NaN, >50K 48, Self-emp-not-inc, 265477, Assoc-acdm, 12, Married-civ-spouse, Prof-specialty, Husband, White, Male, 0, 0, 40, United-States, <=50K </code></pre> <p>I'm trying to remove all rows with NaNs from a DataFrame loaded from a CSV file in pandas.</p> <pre><code>>>> import pandas as pd >>> income = pd.read_csv('income.data') >>> income['type'].unique() array([ State-gov, Self-emp-not-inc, Private, Federal-gov, Local-gov, NaN, Self-emp-inc, Without-pay, Never-worked], dtype=object) >>> income.dropna(how='any') # should drop all rows with NaNs >>> income['type'].unique() array([ State-gov, Self-emp-not-inc, Private, Federal-gov, Local-gov, NaN, Self-emp-inc, Without-pay, Never-worked], dtype=object) Self-emp-inc, nan], dtype=object) # what?? >>> income = income.dropna(how='any') # ok, maybe reassignment will work? >>> income['type'].unique() array([ State-gov, Self-emp-not-inc, Private, Federal-gov, Local-gov, NaN, Self-emp-inc, Without-pay, Never-worked], dtype=object) # what?? </code></pre> <p>I tried with a smaller <code>example.csv</code>:</p> <pre><code>label,age,sex 1,43,M -1,NaN,F 1,65,NaN </code></pre> <p>And <code>dropna()</code> worked just fine here for both categorical and numerical NaNs. What is going on? I'm new to Pandas, just learning the ropes. </p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload