Run multiple processes simultaneously with MParallel

Multi-core CPUs have been around for a very long time, but unfortunately most software is still single-threaded and has no idea how to utilize all that power.

MParallel is an open-source tool which offers one simple workaround, launching multiple processes in separate threads so that you can run them all simultaneously.

Take the Ping console command as a very basic example. To run this normally you’d specify one IP address or domain, wait for a response, enter the next and repeat. Most of the time you’re just waiting around.

MParallel can launch several Ping commands at once, each in their own threads. They may still take a while to complete, but as you’re running them in parallel, they’ll be finished in a fraction of the usual time.

Here’s an example of how this might work.

MParallel.exe -count=3 ping.exe -n 16 fsf.org : ping.exe -n 16 gnu.org : ping.exe -n 16 w3c.org

The first -count=3 switch is for MParallel, and tells the program to use 3 threads.

After that we’ve specified three ping commands, although you’re able to specify whatever combination of applications or commands you’d like to run. The colon characters separate one command from the next.

While this sounds great, there are some immediate issues, like the way text output from each of your multi-threaded commands blends into a jumbled mess in the main console window.

Still, if you’re thinking of automating a set of commands then you probably don’t need to watch the progress of each one individually. And even if you do, there are workarounds: you can redirect output to files, or maybe have MParallel launch each command in a separate console window.

One concern we have with this kind of tool is the potential for disaster. What if a subprocess fails, or hangs, MParallel somehow doesn’t notice and keeps launching more? It’s easy to image how your PC might virtually lock up, or just crash completely.

Scanning through the documentation gave us some reassurance. Threads aren’t launched entirely separately, but as part of a job object, which means if you close MParallel they instantly die, too. There’s also an option to abort pending tasks (or the entire queue) if a single process fails to execute. Or you’re able to have commands automatically abort after a fixed time, ensuring a locked-up process won’t tie up your resources forever.

If you’re more interested in flexibility, MParallel has many other useful options, including the ability to read its commands from a text file or standard input (to accept the output from some other program).

As you’ve probably already guessed, this isn’t a tool for the novice, but if you’re experienced with Windows scripting then MParallel does give you some useful extra abilities. It’s also portable, ultra-compact (a single 208KB) and from a developer you can trust (LameXP’s Muldersoft), so there’s really no reason why you shouldn’t grab a copy immediately.

MParallel is a free application for Windows 7 and later (and possibly earlier, although we didn’t check. Sorry.)