Finally, there is an option to combine iterations through disc. If set to
Yes, at the end of every iteration all MPI slaves will write out a large file with
their accumulated results. The MPI master will read in all these files, combine
them all, and write out a new file with the combined results. All MPI salves
will then read in the combined results. This reduces heavy load on the network,
but increases load on the disc I/O. This will affect the time it takes between
the progress-bar in the expectation step reaching its end (the mouse gets to the
cheese) and the start of the ensuing maximisation step. It will depend on your
system setup which is most efficient. This option was originally implemented to
circumvent bugs on the network cards on our old cluster at LMB. Nowadays,
we prefer not to use this option, as it tends to be very slow when refinements
reached high resolutions.
2.3 On-the-fly processing: running scheduled jobs
The
Schedule button can be used to prepare jobs for future execution. Because
expected output files of scheduled jobs are already available for input into other
jobs from their
Browse buttons, it is possible to build a “pipeline” of multiple,
consecutive jobs in the
Scheduled jobs list. One can execute the scheduled
jobs using the Autorun menu on the top left of the GUI. The program will
ask you how many times you want to run the scheduled jobs. Providing a
value larger than one, will result in an iterative execution of all the scheduled
jobs. If you choose to do this, there will be another question of how many
minutes should at least pass between each repetition of all scheduled jobs. (If
the execution of all scheduled jobs takes less than this time, the program will
wait with executing the next round of scheduled jobs.) The following jobtypes
will only process micrographs that were not finished yet in a previous iteration:
Import , Motion correction , CTF estimation , Auto-picking , Particle extraction
and Movie refinement . All other jobtypes will restart their calculation from
scratch again after each iteration of executing the scheduled jobs.
This setup allows convenient on-the-fly processing of micrographs, as they are
being collected. One can set up a script that copies micrographs, or micrograph
movies from the microscope computer onto a processing computer (preferably
with one or more suitable GPUs for fast processing). (Because copying large
files may take some time, and you do not want to import micrographs/movies
that have not yet finished copying, this script could copy the files over with
a temporary filename, and only once they have arrived entirely could the file
then be renamed to their final filename.) A list of scheduled jobs starting with
for example an
Import of all Micrographs/*.mrcs files, could be followed by
Motion correction , CTF estimation , Auto-picking (provided you already have
suitable templates, otherwise someone would have to manually pick some par-
ticles after a few micrographs have been acquired),
Particle extraction and then
2D classification . This list of scheduled jobs could be repeated many times, per-
haps with at least 30 or 60 minutes between each iteration. The list of imported
10