Quantcast
Channel: Intel® C++ Compiler
Viewing all articles
Browse latest Browse all 1616

R Orphaned Processes

$
0
0

I've noticed strange behaviour with launching parallel processes in R that only appears when R is built with icc. The spawned parallel processes are not killed when the main process ends.

Example code is as follows:

library(foreach)
library(doMC)
registerDoMC(cores=4)

d <- rep(1,16)
t <- foreach(i=1:4, .combine=c) %dopar% {
    s <- foreach(1:4, .combine=c) %do% 1*1
}
identical(t, d)

Here we see the 4 spawned process are orphaned at the completion of the script.

build$ Rscript HungRProcs.R
Loading required package: iterators
Loading required package: parallel
[1] TRUE

build$ ps -elf | grep R
F S UID        PID  PPID  C PRI  NI ADDR SZ WCHAN  STIME TTY          TIME CMD
1 S root      1427     2  0  80   0 -     0 worker May15 ?        00:00:00 [SCIF INTR 0]
0 S build    19173 26999  0  80   0 - 35960 poll_s 12:27 pts/1    00:00:00 vim RStats-3.0.3-dw.spec
1 S walling  24425     1  1  80   0 - 51468 hrtime 13:11 pts/5    00:00:00 /home1/00157/walling/software/R-3.1.0/bin/exec/R --slave --no-restore --file=HungRProcs.R --args
1 S walling  24426     1  1  80   0 - 51468 hrtime 13:11 pts/5    00:00:00 /home1/00157/walling/software/R-3.1.0/bin/exec/R --slave --no-restore --file=HungRProcs.R --args
1 S walling  24427     1  1  80   0 - 51468 hrtime 13:11 pts/5    00:00:00 /home1/00157/walling/software/R-3.1.0/bin/exec/R --slave --no-restore --file=HungRProcs.R --args
1 S walling  24428     1  1  80   0 - 51468 hrtime 13:11 pts/5    00:00:00 /home1/00157/walling/software/R-3.1.0/bin/exec/R --slave --no-restore --file=HungRProcs.R --args
0 R walling  24430 21882  0  80   0 - 27561 -      13:11 pts/5    00:00:00 ps -elf
0 S walling  24431 21882  0  80   0 - 25814 pipe_w 13:11 pts/5    00:00:00 grep R

The configure used for the icc build is as follows:

build$ ./configure --prefix=/home1/00157/walling/software/R-3.1.0 CC=icc F77=ifort FC=ifort CXX=icpc

If built with gcc, the spawned processes are terminated when the main process completes. The configure used for the gcc build is as follows:

build$ ./configure --prefix=/home1/00157/walling/software/R-3.1.0 CC=gcc F77=gfortran FC=gfortran CXX=gcc

I have run tests against both R 3.0.3 and 3.1.0, different parallel backends via doMC, doSNOW and straight mclapply. I've also tested with multiple versions of the GNU compiler and Intel compiler and on both Centos 5.10 and 6.5. All tests cases have resulted in the same behaviour.

Any ideas why the compiler would affect proper termination of spawned sub-processes?

Note: This question was asked on Stackoverflow, but has not yet received any replies(http://stackoverflow.com/questions/23919216/hung-parallel-processes-in-r...).  I thought I would try here as it seems specific to the intel compiler.


Viewing all articles
Browse latest Browse all 1616

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>