Databases
StackOverflow2013
Postsdbo
POR: Using the bigmemory library for classification with randomForest
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POR: Using the bigmemory library for classification with randomForest
    primarykey
    data
    text
    <p>has anyone been able to set up a classification (not a regressions) using randomForest AND the bigmemory library. I am aware that the 'formula approach" cannot be used and we have to resort to the "x=predictors, y=response approach". It appears that the big memory library is unable to deal with a response vector that has categorical values (its a matrix, after all). In my case, I have two levels, both represented as characters.</p> <p>According to the bigmemory documentation..."A data frame will have character vectors converted to factors, and then all factors converted to numeric factor levels"</p> <p>Any suggested workarounds to get randomForest classification to work with bigmemory?</p> <pre><code>#EXAMPLE to problem library(randomForest) library(bigmemory) # Removing any extra objects from my workspace (just in case) rm(list=ls()) #first small matrix small.mat &lt;- matrix(sample(0:1,5000,replace = TRUE),1000,5) colnames(small.mat) &lt;- paste("V",1:5,sep = "") small.mat[,5] &lt;- as.factor(small.mat[,5]) small.rf &lt;- randomForest(V5 ~ .,data = small.mat, mtry=2, do.trace=100) print(small.rf) small.result &lt;- matrix(0,1000,1) small.result &lt;- predict(small.rf, data=small.mat[,-5]) #now small dataframe Works! small.mat &lt;- matrix(sample(0:1,5000,replace = TRUE),1000,5) colnames(small.mat) &lt;- paste("V",1:5,sep = "") small.data &lt;- as.data.frame(small.mat) small.data[,5] &lt;- as.factor(small.data[,5]) small.rf &lt;- randomForest(V5 ~ .,data = small.data, mtry=2, do.trace=100) print(small.rf) small.result &lt;- matrix(0,1000,1) small.result &lt;- predict(small.rf, data=small.data[,-5]) #then big matrix Classification Does NOT Work :-( #----------------****************************---- big.mat &lt;- as.big.matrix(small.mat, type = "integer") #Line below throws error, "cannot coerce class 'structure("big.matrix", package = "bigmemory")' into a data.frame" big.rf &lt;- randomForest(V5~.,data = big.mat, do.trace=10) #Runs without error but only regression big.rf &lt;- randomForest(x = big.mat[,-5], y = big.mat[,5], mtry=2, do.trace=100) print(big.rf) big.result &lt;- matrix(0,1000,1) big.result &lt;- predict(big.rf, data=big.mat[,-5]) </code></pre>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USauro
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POR: Using the bigmemory library for classification with randomForest
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POR: Using the bigmemory library for classification with randomForest
      1. This table or related slice is empty.
      1. VTUpMod
    1. COCoerce to factor via `y = as.factor(big.mat[,5])`?
      singulars
      1. POR: Using the bigmemory library for classification with randomForest
      1. USjoran
    2. COI should add that I have no idea if `randomForest` actually supports big.matrix input when the object is truly too large for memory.
      singulars
      1. POR: Using the bigmemory library for classification with randomForest
      1. USjoran
    3. COAs I know, `randomForest` loads all `bigmemory` data in RAM when one calls model building.
      singulars
      1. POR: Using the bigmemory library for classification with randomForest
      1. USDrDom
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload