GNU is a free software operating system that is upward-compatible with Unix. GNU parallel is a shell tool for executing jobs in parallel. It uses the lines of its standard input to modify shell commands, which are then run in parallel. – NASA … to be continued…


目录

  1. Parallelize Your Job
    1. Check remote machine network connection.
    2. Execute commands on remote machines
    3. Transfer files to or retrive files from remote machines

Parallelize Your Job

Check remote machine network connection.

1
2
3
parallel -j 200 -N 4 "ping {} >/dev/null 2>&1 && echo 0 {} is connected || echo 1 {} is disconnected" ::: node{0000..2000} 
echo > node.lst; for i in {0000..20000};do echo node${i} >> node.lst;done
parallel -j 200 -N 4 "ping {} >/dev/null 2>&1 && echo 0 {} is connected || echo 1 {} is disconnected" ::: < node.lst

If the executing order is important, then you may need the -k option as following and parallel will execute ping in order.

1
parallel -j 200 -N 4 -k "ping {} >/dev/null 2>&1 && echo 0 {} is connected || echo 1 {} is disconnected" ::: node{0000..2000}

Execute commands on remote machines

1
2
3
4
# obtain remote machine's hostname
parallel -j 200 'ssh {} "echo \$(hostnamectl --static)"' ::: < node.lst
# obtain remote machine's ip
parallel -j 200 'ssh {} "echo \$(hostname -I)"' ::: < node.lst

Transfer files to or retrive files from remote machines

1
parallel -S servers --transfer