Unix & Linux
bash parallelism
Updated Thu, 29 Sep 2022 05:55:34 GMT

Is there a way to run process parallelly in the loop of a bash script


Possible Duplicate:
Parallelizing a for loop

The original code might look like this:

 for i in *; do something.py $i; done

I was wondering whether I can run these jobs parallelly in the backgroud, such as:

 for i in *; do something.py $i &; done

I tried and found the & here won't work..

Moreover, a better way might be that bash allow 8 jobs(or any number) run together in the queue in background, but I don't know how to do that...

Does anyone have ideas about this? Thanks!




Solution

Simply remove the ; character, so in final :

 for i in *; do something.py $i & done

And for running N instance of your script at the same time, see man 1 parallel

See http://www.gnu.org/software/parallel/





Comments (4)

  • +0 – Added some hints about parallel — Oct 31, 2012 at 03:12  
  • +2 – It isn't necessary to comment after editing a post: its a wiki, the edit history is recorded... — Oct 31, 2012 at 04:06  
  • +0 – Interesting, you don't need the semicolon before done — Oct 31, 2012 at 08:48  
  • +0& is a command terminator like ; and newline: also && and ||. See SHELL GRAMMAR in the bash manpage, or here — Oct 31, 2012 at 13:40