Nodeset: add --index command to cluset/nodeset by thiell · Pull Request #637 · clustershell/clustershell · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion bash_completion.d/cluset
3 changes: 3 additions & 0 deletions doc/man/man1/cluset.1
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,9 @@ list node groups from all group sources (\fB\-LL\fP shows nodes and \fB\-LLL\fP
.B \-r\fP,\fB \-\-regroup
fold nodes using node groups (see \-s \fIGROUPSOURCE\fP)
.TP
.BI \-\-index\fB= NODE
output the index of NODE in the nodeset (reverse of \-I/\-\-slice)
.TP
.B \-\-groupsources
list all active group sources (see \fBgroups.conf\fP(5))
.UNINDENT
Expand Down
3 changes: 3 additions & 0 deletions doc/man/man1/nodeset.1
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,9 @@ list node groups from all group sources (\fB\-LL\fP shows nodes and \fB\-LLL\fP
.B \-r\fP,\fB \-\-regroup
fold nodes using node groups (see \-s \fIGROUPSOURCE\fP)
.TP
.BI \-\-index\fB= NODE
output the index of NODE in the nodeset (reverse of \-I/\-\-slice)
.TP
.B \-\-groupsources
list all active group sources (see \fBgroups.conf\fP(5))
.UNINDENT
Expand Down
66 changes: 63 additions & 3 deletions doc/sphinx/tools/cluset.rst
Original file line number Diff line number Diff line change
Expand Up @@ -601,6 +601,66 @@ the resulting node set (or from the resulting range set with ``-R``)::
node[11,13]


.. _cluset-index:

Finding the index of a node
"""""""""""""""""""""""""""

The ``--index`` command is the reverse of :ref:`slicing <cluset-slice>`:
instead of selecting a node by its position, it outputs the zero-based index
of a node within the resulting (ordered) node set. It is the command-line
equivalent of the :meth:`.NodeSet.index` method::

$ cluset --index node5 node[0-9]
5

If the node is not part of the set, an error is printed and a non-zero exit
status is returned, so ``--index`` can also be used to test set membership::

$ cluset --index node42 node[0-9]
ERROR: 'node42' is not in nodeset

It also works in range set mode with ``-R``::

$ cluset -R --index 18 1,5,18-31
2

The position follows the same ordering as ``-e/--expand``. Two aspects of that
ordering are worth knowing, as both can be surprising. First, when a node set
spans several distinct patterns (different name templates), the patterns are
sorted alphabetically by name, not by the order they were written or by the
numbers they contain::

$ cluset -e node[1-4],bmc[10-20]
bmc10 bmc11 bmc12 bmc13 bmc14 bmc15 bmc16 bmc17 bmc18 bmc19 bmc20 node1 node2 node3 node4
$ cluset --index bmc10 node[1-4],bmc[10-20]
0
$ cluset --index node1 node[1-4],bmc[10-20]
11

Second, for multidimensional node sets, the set is traversed as a cartesian
product in which the last dimension varies fastest::

$ cluset -e rack[1-2]node[1-3]
rack1node1 rack1node2 rack1node3 rack2node1 rack2node2 rack2node3
$ cluset --index rack2node1 rack[1-2]node[1-3]
3

For a one-dimensional node set (for example ``node[01-04]``) the index is just
the node's ascending numeric position. A multidimensional node set has no
single obvious order, so ClusterShell flattens it using its own convention,
which is not guaranteed across versions; do not rely on a given index over
time. Within a version it is always deterministic and depends only on the
set's contents.

.. note::

``--index`` reflects ClusterShell's own ordering, so for a multidimensional
node list it is not guaranteed to match the node rank a resource manager
assigns (such as Slurm's ``$SLURM_NODEID``); use the value the resource
manager provides instead.


.. _cluset-groups:

Node groups
Expand Down Expand Up @@ -1015,7 +1075,7 @@ Arithmetic and special operations
"""""""""""""""""""""""""""""""""

All arithmetic operations, as seen for node sets (cf.
:ref:`cluset-arithmetic`:), are available for range sets, for example::
:ref:`cluset-arithmetic`), are available for range sets, for example::

$ cluset -fR 1-14 -x 10-20
1-9
Expand All @@ -1035,8 +1095,8 @@ sets (cf. :ref:`cluset-extended-patterns`). However, as the union operator


Besides arithmetic operations, special operations may be very convenient for
range sets also (cf. :ref:`cluset-special`:).
Below is an example with ``-I / --slice`` (cf. :ref:`cluset-slice`:)::
range sets also (cf. :ref:`cluset-special`).
Below is an example with ``-I / --slice`` (cf. :ref:`cluset-slice`)::

$ cluset -fR -I 0 100-131
100
Expand Down
64 changes: 63 additions & 1 deletion doc/sphinx/tools/nodeset.rst
Original file line number Diff line number Diff line change
Expand Up @@ -441,6 +441,8 @@ on a predefined node count, splitting non-contiguous subsets, choosing fold
axis (for multidimensional node sets) and picking N nodes randomly. They are
all explained below.

.. _nodeset-slice:

Slicing
"""""""

Expand Down Expand Up @@ -598,6 +600,66 @@ the resulting node set (or from the resulting range set with ``-R``)::
node[11,13]


.. _nodeset-index:

Finding the index of a node
"""""""""""""""""""""""""""

The ``--index`` command is the reverse of :ref:`slicing <nodeset-slice>`:
instead of selecting a node by its position, it outputs the zero-based index
of a node within the resulting (ordered) node set. It is the command-line
equivalent of the :meth:`.NodeSet.index` method::

$ nodeset --index node5 node[0-9]
5

If the node is not part of the set, an error is printed and a non-zero exit
status is returned, so ``--index`` can also be used to test set membership::

$ nodeset --index node42 node[0-9]
ERROR: 'node42' is not in nodeset

It also works in range set mode with ``-R``::

$ nodeset -R --index 18 1,5,18-31
2

The position follows the same ordering as ``-e/--expand``. Two aspects of that
ordering are worth knowing, as both can be surprising. First, when a node set
spans several distinct patterns (different name templates), the patterns are
sorted alphabetically by name, not by the order they were written or by the
numbers they contain::

$ nodeset -e node[1-4],bmc[10-20]
bmc10 bmc11 bmc12 bmc13 bmc14 bmc15 bmc16 bmc17 bmc18 bmc19 bmc20 node1 node2 node3 node4
$ nodeset --index bmc10 node[1-4],bmc[10-20]
0
$ nodeset --index node1 node[1-4],bmc[10-20]
11

Second, for multidimensional node sets, the set is traversed as a cartesian
product in which the last dimension varies fastest::

$ nodeset -e rack[1-2]node[1-3]
rack1node1 rack1node2 rack1node3 rack2node1 rack2node2 rack2node3
$ nodeset --index rack2node1 rack[1-2]node[1-3]
3

For a one-dimensional node set (for example ``node[01-04]``) the index is just
the node's ascending numeric position. A multidimensional node set has no
single obvious order, so ClusterShell flattens it using its own convention,
which is not guaranteed across versions; do not rely on a given index over
time. Within a version it is always deterministic and depends only on the
set's contents.

.. note::

``--index`` reflects ClusterShell's own ordering, so for a multidimensional
node list it is not guaranteed to match the node rank a resource manager
assigns (such as Slurm's ``$SLURM_NODEID``); use the value the resource
manager provides instead.


.. _nodeset-groups:

Node groups
Expand Down Expand Up @@ -1031,7 +1093,7 @@ sets (cf. :ref:`nodeset-extended-patterns`). However, as the union operator

Besides arithmetic operations, special operations may be very convenient for
range sets also. Below is an example with ``-I / --slice`` (cf.
nodeset-slice)::
:ref:`nodeset-slice`)::

$ nodeset -fR -I 0 100-131
100
Expand Down
1 change: 1 addition & 0 deletions doc/txt/cluset.txt
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ OPTIONS
-l, --list list node groups, list node groups and nodes (``-ll``) or list node groups, nodes and node count (``-lll``). When no argument is specified at all, this command will list all node group names found in selected group source (see also -s *GROUPSOURCE*). If any nodesets are specified as argument, this command will find node groups these nodes belongs to (individually). Optionally for each group, the fraction of these nodes being member of the group may be displayed (with ``-ll``), and also member count/total group node count (with ``-lll``). If a single hyphen-minus (-) is given as a nodeset, it will be read from standard input.
-L, --list-all list node groups from all group sources (``-LL`` shows nodes and ``-LLL`` adds node count). Like ``-l``, if any nodesets are specified as argument, this command will find node groups these nodes belongs to (individually).
-r, --regroup fold nodes using node groups (see -s *GROUPSOURCE*)
--index=NODE output the index of NODE in the nodeset (reverse of -I/--slice)
--groupsources list all active group sources (see ``groups.conf``\(5))

Operations:
Expand Down
1 change: 1 addition & 0 deletions doc/txt/nodeset.txt
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ OPTIONS
-l, --list list node groups, list node groups and nodes (``-ll``) or list node groups, nodes and node count (``-lll``). When no argument is specified at all, this command will list all node group names found in selected group source (see also -s *GROUPSOURCE*). If any nodesets are specified as argument, this command will find node groups these nodes belongs to (individually). Optionally for each group, the fraction of these nodes being member of the group may be displayed (with ``-ll``), and also member count/total group node count (with ``-lll``). If a single hyphen-minus (-) is given as a nodeset, it will be read from standard input.
-L, --list-all list node groups from all group sources (``-LL`` shows nodes and ``-LLL`` adds node count). Like ``-l``, if any nodesets are specified as argument, this command will find node groups these nodes belongs to (individually).
-r, --regroup fold nodes using node groups (see -s *GROUPSOURCE*)
--index=NODE output the index of NODE in the nodeset (reverse of -I/--slice)
--groupsources list all active group sources (see ``groups.conf``\(5))

Operations:
Expand Down
11 changes: 10 additions & 1 deletion lib/ClusterShell/CLI/Nodeset.py
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,8 @@ def nodeset():
cmdcount = int(options.count) + int(options.expand) + \
int(options.fold) + int(bool(options.list)) + \
int(bool(options.listall)) + int(options.regroup) + \
int(options.groupsources) + int(options.completion)
int(options.groupsources) + int(options.completion) + \
int(options.index is not None)
if not cmdcount:
parser.error("No command specified.")
elif cmdcount > 1:
Expand All @@ -198,6 +199,9 @@ def nodeset():
if options.axis and (not options.fold or options.rangeset):
parser.error("--axis option is only supported when folding nodeset")

if options.index is not None and options.pick:
parser.error("--index cannot be combined with --pick")

if options.groupsource and not options.quiet and class_set == RangeSet:
print("WARNING: option group source \"%s\" ignored"
% options.groupsource, file=sys.stderr)
Expand Down Expand Up @@ -312,6 +316,11 @@ def nodeset():

fmt = options.output_format # default to '%s'

# --index outputs the position of a node in the set (reverse of -I/--slice)
if options.index is not None:
print(fmt % xset.index(options.index))
return

# Display result according to command choice
if options.expand:
xsubres = lambda x: separator.join((fmt % s for s in x.striter()))
Expand Down
3 changes: 3 additions & 0 deletions lib/ClusterShell/CLI/OptionParser.py
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,9 @@ def install_nodeset_commands(self):
dest="regroup", default=False,
help="fold nodes using node groups (see -s "
"GROUPSOURCE)")
optgrp.add_option("--index", action="store", dest="index",
metavar="NODE", type="string",
help="output the index of NODE in the nodeset")
optgrp.add_option("--list-sources", "--groupsources",
action="store_true", dest="groupsources",
default=False,
Expand Down
100 changes: 100 additions & 0 deletions tests/CLINodesetTest.py
Loading