Databases
StackOverflow2013
Postsdbo
POFilling continuous pandas dataframe from sparse dataframe
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POFilling continuous pandas dataframe from sparse dataframe
    primarykey
    data
    text
    <p>I have a dictionary name date_dict keyed by datetime dates with values corresponding to integer counts of observations. I convert this to a sparse series/dataframe with censored observations that I would like to join or convert to a series/dataframe with continuous dates. The nasty list comprehension is my hack to get around the fact that pandas apparently won't automatically covert datetime date objects to an appropriate DateTime index.</p> <pre><code>df1 = pd.DataFrame(data=date_dict.values(), index=[datetime.datetime.combine(i, datetime.time()) for i in date_dict.keys()], columns=['Name']) df1 = df1.sort(axis=0) </code></pre> <p>This example has 1258 observations and the DateTime index runs from 2003-06-24 to 2012-11-07.</p> <pre><code>df1.head() Name Date 2003-06-24 2 2003-08-13 1 2003-08-19 2 2003-08-22 1 2003-08-24 5 </code></pre> <p>I can create an empty dataframe with a continuous DateTime index, but this introduces an unneeded column and seems clunky. I feel as though I'm missing a more elegant solution involving a join.</p> <pre><code>df2 = pd.DataFrame(data=None,columns=['Empty'], index=pd.DateRange(min(date_dict.keys()), max(date_dict.keys()))) df3 = df1.join(df2,how='right') df3.head() Name Empty 2003-06-24 2 NaN 2003-06-25 NaN NaN 2003-06-26 NaN NaN 2003-06-27 NaN NaN 2003-06-30 NaN NaN </code></pre> <p>Is there a simpler or more elegant way to fill a continuous dataframe from a sparse dataframe so that there is (1) a continuous index, (2) the NaNs are 0s, and (3) there is no left-over empty column in the dataframe?</p> <pre><code> Name 2003-06-24 2 2003-06-25 0 2003-06-26 0 2003-06-27 0 2003-06-30 0 </code></pre>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USChang She
    1. USBrian Keegan
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POFilling continuous pandas dataframe from sparse dataframe
      1. USBrian Keegan
      1. VTFavorite
    2. VO
      singulars
      1. POFilling continuous pandas dataframe from sparse dataframe
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POFilling continuous pandas dataframe from sparse dataframe
      1. USvgoklani
      1. VTFavorite
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload