Fig. 2. Message Passing.
C/C++ files. It runs task in parallel on a “MATLAB Parallel
Machine,” created either locally through fork() or remotely
through rsh, ssh, rexec, or a custom Beowulf Job Manager.
In local mode MATLAB workspace variables are automat-
ically transferred to the slave processes through the fork().
In the remote case, they have to be transferred over sockets.
PLab also supports integration with MOSIX [8] and can mi-
grate slave processes to nodes with smaller load.
PLab supports two modes of parallelization.
emu-
lates a parallel for-loop. For example
is equivalent to the for-loop
;
for
,
;
;
;
end
run in parallel. Two scheduling methods are built into PLab
for
. In the fixed method, the server hands out fixed
number of loop iterations in chucks to each slave. In the dy-
namic method, the number of iterations passed to each slave
at a time varies dynamically trying to make each chunk take
1 second of computing time.
The second mode, peval, is the typical embarrassingly par-
allel eval function.
B. Message Passing
See Fig. 2. Software that makes use of this approach:
MultiMATLAB [35], CMTM [47], DPToolbox [39],
MPITB/PVMTB [22], PMI [34], PTToolbox [26],
MatlabMPI [30], pMatlab [31], and Matlab Paralleliza-
tion Toolkit [25]
This approach provides message passing routines between
MATLAB processes. This enables users to write their own
parallel programs in MATLAB in a fashion similar to writing
parallel programs with a compiled language using MPI. In
fact, for some of the projects [22], [30], the routines pro-
vided are wrappers for MPI routines. This approach has the
advantage of flexibility: in theory, users are able to build any
parallel system in MATLAB that they can build in compiled
languages with MPI. This approach is a superset of the em-
barrassingly parallel approach.
1) MultiMATLAB: MultiMATLAB is perhaps one of the
most well known parallel MATLABs. It was developed by
A. Trefethen, V. Menon, C. Chang, G. Czajkowski, C. Myers,
and L. Trefethen at Cornell University. In MultiMATLAB, a
MATLAB user can start MATLAB processes on other ma-
chines and then pass commands and data between these var-
ious processes. The most important commands are:
• Init—initialize the MultiMATLAB environment;
• Nproc—returns the number of processes in the envi-
ronment; equivalent of MPI Comm size();
• ID—returns the rank of the process; equivalent of MPI
Comm rank();
• Send/Recv—sends and receives data between
processes;
• Eval—evaluates a string in every process in the
environment.
This is essentially the MPI “Basic Six” without MPI Fi-
nalize(), and with Eval added. The Send and Recv commands
in MultiMATLAB are blocking, so one is not able to take ad-
vantage of pipelining.
MultiMATLAB is built upon the P4 device of MPICH
[24], an implementation of MPI developed at Argonne Na-
tional Laboratory and Mississippi State University. Because
of that, it runs over a heterogeneous network of workstations.
C. Back-End Support
Software that makes use of this approach: NetSolve [13],
DLab [38], Matpar [42], PLAPACK [45], PARAMAT [43],
and MATLAB
P [27]
This approach uses MATLAB as a front end for a parallel
computation engine. Computation is done on the engine, usu-
ally making use of high-performance numerical computation
libraries like ScaLAPACK [9]. For some projects e.g. [13],
the data reside in MATLAB and are passed to the engine and
back. And for some projects, e.g., [27], the data reside on the
server and are passed back to MATLAB only upon request.
The latter approach has the advantage that there is less data
traffic. This is important for performance when data sets are
large.
The advantage of this approach is that it only requires one
MATLAB session (therefore, only one license), and it usually
does not require the end user to have knowledge of parallel
programming.
1) NetSolve: NetSolve is actually much more than a
parallel MATLAB. It is a project at University of Tennessee,
Knoxville, “that aims to bring together disparate computa-
tional resources connected by computer networks” (from
their Web page). An interface to the NetSolve system exists
for MATLAB. The user can submit jobs to the system, and
then NetSolve will search for computational resources on
a network and choose the best one for the task. It supports
fault-tolerance and load-balancing. In some sense, it is a
very user-friendly batch system.
Jobs can be submitted as
CHOY AND EDELMAN: PARALLEL MATLAB: DOING IT RIGHT 333