r - caret with SNOW not using parallel processor on a particular set of data -


i have 2 data frames (here reproducibility) trainfin1 , trainfin2, both sampled same bigger dataset.

i'm trying run cross-validated rpart on them using caret on multiprocessor using dosnow package.

interestingly, trainfin1 trained nicely across 4 processors (finishing in 25 seconds). trainfin2 seems stuck on 1 processor (observed in windows task manager window), , never see finish processing after half hour.

my code below

require(caret) require(rpart) load("trainfin.rdata")  fitcontrol <- traincontrol(method = "repeatedcv", number = 5, repeats = 5)  #setup parallel processing require(dosnow) cl <- makecluster(4, type = "sock") registerdosnow(cl)  #train set.seed(12345) firstset <- train(x = trainfin1[, names(trainfin1) != "happiness"],                   y = trainfin1$happiness,                   method = "rpart2", trcontrol = fitcontrol)  set.seed(12345) secondset <- train(x = trainfin2[, names(trainfin2) != "happiness"],                    y = trainfin2$happiness,                    method = "rpart2", trcontrol = fitcontrol)  stopcluster(cl)

do note avoided use of formula in train , instead feed raw data, avoid caret converting ordinal variables dummy categorical variables (see answer this question). when used formula (i.e. train(happiness ~ ., data = trainfin2, method = "rpart2", trcontrol = fitcontrol)), there seems no issue parallel processing. want avoid using formula per other question.

any suggestions on how can parallel-process data without converting predictors categorical dummies ?


Comments

Post a Comment

Popular posts from this blog

javascript - Google App Script ContentService downloadAsFile not working -

i have web app developed using google app script htmlservice , html form, populating excel sheet in google drive using spreadsheetapp . , 1 section calling contentservice download data excel file. function doget(e) { // read excel sheet //getappfile(); // render application html template return htmlservice.createtemplatefromfile('index').evaluate() .settitle('go smart') .setsandboxmode(htmlservice.sandboxmode.iframe); } function downloaddoublequatecsvfile() { var sheetid = propertiesservice.getscriptproperties().getproperty('sheetid'); var ss = spreadsheetapp.openbyid(sheetid).getactivesheet(); //var ss = spreadsheetapp.getactivespreadsheet().getactivesheet(); var maxcolumn = ss.getlastcolumn(); var maxrow = ss.getlastrow(); var data = ss.getrange(1, 1, maxrow, maxcolumn).getvalues(); if (data.length > 1) { var csv = ""; (var row = 0; row < data.length; row++) { (var c...
Read more

javascript - Function overwritting -

is there way overwrite function or event handling in function? using addeventlistener, can have many event handlers on element want var test=document.getelementbyid("div"); test.addeventlistener("click",function(){alert("im first");},false); test.addeventlistener("click",function(){alert("im second");},false); // alerts "im first" , after "im second" but if want overwrite function/event example based on client width example this function one() { alert("first"); } function two() { alert("second"); } window.onresize = function() { var test = document.getelementbyid("div"); if (window.innerwidth > 500) { test.onclick = 1 } else { test.onclick = two; } } is possible in javascript? in case use effective little known approach. addeventlistener can accept object property handleevent event handler. in case it's easy overwr...
Read more

c# - Exception when attempting to modify Dictionary -

i creating dictionary manage threads , ran exception when attempt clear dead threads after adding threads dictionary. using system; using system.collections.generic; using system.threading; namespace syscat { public class threadmanager { public static readonly threadmanager globalthreadmanager = new threadmanager( ); /// <summary> /// initializes new instance of <see cref="syscat.threadmanager"/> class. /// </summary> public threadmanager( ) { } /// <summary> /// create new threadmanager threads within basetm , deinitializes basetm /// </summary> public threadmanager( threadmanager basetm ) { this.threads = basetm.threads; basetm.threads.clear( ); } private dictionary<guid,thread> threads = new dictionary<guid, thread>( ); /// <summary> /// attempts obtain unused threadid within ...
Read more