Gentle Shutdown Procedure

From athena

For those days you don't need to pull the big switch:

To "drain" a node so that you're not disturbing folks:

pbsnodes -o name_of_the_node

(marks it "offline") or:

mnodectl compute-5-[1-2].local -m state=Draining
(replies: state on node compute-5-1.local updatedstate on node compute-5-2.local updated)


To shutdown one node gracefully either...

 cluster-fork --nodes=compute-0-1 shutdown -h now,
 or push the button on the node in question.

To shutdown a range of nodes via cluster-fork...

 For a subset of nodes (say 0-7):
 cluster-fork --nodes=compute-0-%d:0-7 shutdown -h now

To shutdown all nodes gracefully...

 cluster-fork shutdown -h now

Shooting a node forces a node to reload its OS. To kill...

 shoot-node compute-0-1

If Polyserve nodes need to be shut down:

  Notify: ljones@astro.washington.edu,becker@astro.washington.edu,ajc@astro.washington.edu re: MOPS and Athena5
  Shutdown Athena0 first, then P5, P4, P3, P2 and finally P1, with a two minute delay between each.