I've noticed strange behaviour with launching parallel processes in R that only appears when R is built with icc. The spawned parallel processes are not killed when the main process ends.
Example code is as follows:
library(foreach) library(doMC) registerDoMC(cores=4) d <- rep(1,16) t <- foreach(i=1:4, .combine=c) %dopar% { s <- foreach(1:4, .combine=c) %do% 1*1 } identical(t, d)
Here we see the 4 spawned process are orphaned at the completion of the script.
build$ Rscript HungRProcs.R Loading required package: iterators Loading required package: parallel [1] TRUE build$ ps -elf | grep R F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD 1 S root 1427 2 0 80 0 - 0 worker May15 ? 00:00:00 [SCIF INTR 0] 0 S build 19173 26999 0 80 0 - 35960 poll_s 12:27 pts/1 00:00:00 vim RStats-3.0.3-dw.spec 1 S walling 24425 1 1 80 0 - 51468 hrtime 13:11 pts/5 00:00:00 /home1/00157/walling/software/R-3.1.0/bin/exec/R --slave --no-restore --file=HungRProcs.R --args 1 S walling 24426 1 1 80 0 - 51468 hrtime 13:11 pts/5 00:00:00 /home1/00157/walling/software/R-3.1.0/bin/exec/R --slave --no-restore --file=HungRProcs.R --args 1 S walling 24427 1 1 80 0 - 51468 hrtime 13:11 pts/5 00:00:00 /home1/00157/walling/software/R-3.1.0/bin/exec/R --slave --no-restore --file=HungRProcs.R --args 1 S walling 24428 1 1 80 0 - 51468 hrtime 13:11 pts/5 00:00:00 /home1/00157/walling/software/R-3.1.0/bin/exec/R --slave --no-restore --file=HungRProcs.R --args 0 R walling 24430 21882 0 80 0 - 27561 - 13:11 pts/5 00:00:00 ps -elf 0 S walling 24431 21882 0 80 0 - 25814 pipe_w 13:11 pts/5 00:00:00 grep R
The configure used for the icc build is as follows:
build$ ./configure --prefix=/home1/00157/walling/software/R-3.1.0 CC=icc F77=ifort FC=ifort CXX=icpc
If built with gcc, the spawned processes are terminated when the main process completes. The configure used for the gcc build is as follows:
build$ ./configure --prefix=/home1/00157/walling/software/R-3.1.0 CC=gcc F77=gfortran FC=gfortran CXX=gcc
I have run tests against both R 3.0.3 and 3.1.0, different parallel backends via doMC, doSNOW and straight mclapply. I've also tested with multiple versions of the GNU compiler and Intel compiler and on both Centos 5.10 and 6.5. All tests cases have resulted in the same behaviour.
Any ideas why the compiler would affect proper termination of spawned sub-processes?
Note: This question was asked on Stackoverflow, but has not yet received any replies(http://stackoverflow.com/questions/23919216/hung-parallel-processes-in-r...). I thought I would try here as it seems specific to the intel compiler.