Databases
StackOverflow2013
Postsdbo
POPrevent pandas from automatically inferring type in read_csv
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POPrevent pandas from automatically inferring type in read_csv
    primarykey
    data
    text
    <p>I have a #-separated file with three columns: the first is integer, the second looks like a float, but isn't, and the third is a string. I attempt to load this directly into python with <code>pandas.read_csv</code></p> <pre><code>In [149]: d = pandas.read_csv('resources/names/fos_names.csv', sep='#', header=None, names=['int_field', 'floatlike_field', 'str_field']) In [150]: d Out[150]: &lt;class 'pandas.core.frame.DataFrame'&gt; Int64Index: 1673 entries, 0 to 1672 Data columns: int_field 1673 non-null values floatlike_field 1673 non-null values str_field 1673 non-null values dtypes: float64(1), int64(1), object(1) </code></pre> <p><code>pandas</code> tries to be smart and automatically convert fields to a useful type. The issue is that I don't actually want it to do so (if I did, I'd used the <code>converters</code> argument). How can I prevent <code>pandas</code> from converting types automatically?</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USCœur
    1. USduckworthd
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POPrevent pandas from automatically inferring type in read_csv
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POPrevent pandas from automatically inferring type in read_csv
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POPrevent pandas from automatically inferring type in read_csv
      1. This table or related slice is empty.
      1. VTUpMod
    1. COIt may not be avoidable. Pandas (and NumPy more generally) doesn't support NaN for integers. Since reading from any CSV leaves the possibility for NaN, it might have been a convenience choice to just have it coerce to float always. Also, your types seem backwards from your printout. It surely *won't* convert floatlike input to an int64, though it presumably *will* convert intlike input to float64.
      singulars
      1. POPrevent pandas from automatically inferring type in read_csv
      1. USely
    2. COI don't believe the order of dtypes listed have anything to do with the order of the columns, but with regard to automatic type coercion, this would be most unfortunate. I was under the impression pandas used NaN anytime information was missing, regardless of type.
      singulars
      1. POPrevent pandas from automatically inferring type in read_csv
      1. USduckworthd
    3. COYes, it does, but then that column will always be of type Object. You can have an Object column that's mostly Ints. However, there are many cases where the NaN coerces it into a float column instead. If you first make an object column and then fill it with possibly-NaN integers, it will stay as object. I think if you just fill it with whatever's there, leaving Pandas to discern the type, it will choose float for any numeric type than has NaNs present. This is not just a limitation of Pandas, but of NumPy and Python entirely. There is no library I'm aware of offering an Int with NaN support.
      singulars
      1. POPrevent pandas from automatically inferring type in read_csv
      1. USely
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload