Node Get and List

The Fuzzball scheduler provides commands to list all compute nodes and retrieve detailed information about specific nodes. These commands are essential for monitoring cluster health, capacity planning, and troubleshooting scheduling issues.

Prerequisites

Compute node information requires that a fuzzball admin context is set up and authorized.

$ fuzzball admin context create <context_name> <api_url>

$ fuzzball admin context login -u <admin_username> -p '<admin_password>'

Get Node Details

The fuzzball admin scheduler node get command retrieves detailed information about a compute node.

Basic Usage

fuzzball admin scheduler node get <node-id>

Example Usage

$ fuzzball admin scheduler node get node-worker-01

Example Output

The command returns JSON output containing detailed information for the compute node:

{
  "node": {
    "id": "node-worker-01",
    "hostname": "worker-node-01.cluster.local",
    "status": 2,
    "start_time": {
      "seconds": 1724025600,
      "nanos": 123456789
    },
    "update_time": {
      "seconds": 1724025600,
      "nanos": 987654321
    }
  },
  "resource": {
    "total": {
      "cpu": {
        "cores": 8,
        "sockets": 1
      },
      "mem": {
        "bytes": 17179869184
      },
      "devices": {
        "cpu/x86_64": 1,
        "io.fuzzball/storage": 1
      }
    },
    "free": {
      "cpu": {
        "cores": 6,
        "sockets": 1
      },
      "mem": {
        "bytes": 12884901888
      },
      "devices": {
        "cpu/x86_64": 1,
        "io.fuzzball/storage": 1
      }
    },
    "node": {
      "cores": [
        {"id": 0},
        {"id": 1},
        {"id": 2},
        {"id": 3},
        {"id": 4},
        {"id": 5},
        {"id": 6},
        {"id": 7}
      ],
      "free_cores": 6,
      "memory_total_bytes": 17179869184,
      "memory_free_bytes": 12884901888,
      "memory_bytes_by_core": 2147483648,
      "numa_nodes": [
        {
          "cores": [0, 1, 2, 3, 4, 5, 6, 7],
          "free_cores": 6,
          "memory_total_bytes": 17179869184,
          "memory_free_bytes": 12884901888,
          "memory_bytes_by_core": 2147483648
        }
      ],
      "sockets": [
        {
          "cores": [0, 1, 2, 3, 4, 5, 6, 7],
          "free_cores": 6,
          "numa_nodes": [0]
        }
      ]
    },
    "annotations": {
      "fuzzball.io/provisioner_backend": "kubernetes",
      "fuzzball.io/provisioner_definition_id": "worker-nodes"
    },
    "plugin_devices": {
      "list": {
        "cpu/x86_64": {
          "devices": [
            {
              "id": "cpu/x86_64",
              "numa_node": 0
            }
          ]
        },
        "io.fuzzball/storage": {
          "devices": [
            {
              "id": "nvme0",
              "numa_node": 0
            }
          ]
        }
      }
    }
  }
}

List All Compute Nodes

The fuzzball admin scheduler node list command retrieves information about all compute nodes in the cluster.

Basic Usage

$ fuzzball admin scheduler node list

Example Output

The command returns JSON output containing detailed information for all compute nodes

{
  "nodes": [
    {
      "node": {
        "id": "node-worker-01",
        "hostname": "worker-node-01.cluster.local",
        "status": 2,
        "start_time": {
          "seconds": 1724025600,
          "nanos": 123456789
        },
        "update_time": {
          "seconds": 1724025600,
          "nanos": 987654321
        }
      },
      "resource": {
        "total": {
          "cpu": {
            "cores": 8,
            "sockets": 1
          },
          "mem": {
            "bytes": 17179869184
          },
          "devices": {
            "cpu/x86_64": 1,
            "io.fuzzball/storage": 1
          }
        },
        "free": {
          "cpu": {
            "cores": 6,
            "sockets": 1
          },
          "mem": {
            "bytes": 12884901888
          },
          "devices": {
            "cpu/x86_64": 1,
            "io.fuzzball/storage": 1
          }
        },
        "node": {
          "cores": [
            {"id": 0},
            {"id": 1},
            {"id": 2},
            {"id": 3},
            {"id": 4},
            {"id": 5},
            {"id": 6},
            {"id": 7}
          ],
          "free_cores": 6,
          "memory_total_bytes": 17179869184,
          "memory_free_bytes": 12884901888,
          "memory_bytes_by_core": 2147483648,
          "numa_nodes": [
            {
              "cores": [0, 1, 2, 3, 4, 5, 6, 7],
              "free_cores": 6,
              "memory_total_bytes": 17179869184,
              "memory_free_bytes": 12884901888,
              "memory_bytes_by_core": 2147483648
            }
          ],
          "sockets": [
            {
              "cores": [0, 1, 2, 3, 4, 5, 6, 7],
              "free_cores": 6,
              "numa_nodes": [0]
            }
          ]
        },
        "annotations": {
          "fuzzball.io/provisioner_backend": "kubernetes",
          "fuzzball.io/provisioner_definition_id": "worker-nodes"
        },
        "plugin_devices": {
          "list": {
            "cpu/x86_64": {
              "devices": [
                {
                  "id": "cpu/x86_64",
                  "numa_node": 0
                }
              ]
            },
            "io.fuzzball/storage": {
              "devices": [
                {
                  "id": "nvme0",
                  "numa_node": 0
                }
              ]
            }
          }
        }
      }
    }
  ]
}

Understanding Node Status

Status Codes

Code	String	Description
1	Not Ready	Node is not ready
2	Ready	Node is active and operational
3	Offline	Node is unreachable
4	Cordoned	Node is unschedulable

Resource Calculation

Calculate utilization by comparing total vs free resources:
CPU utilization: (total.cpu.cores - free.cpu.cores) / total.cpu.cores * 100%
Memory utilization: (total.mem.bytes - free.mem.bytes) / total.mem.bytes * 100%

Filtering and Querying

Use jq to filter node information.

Find active nodes:

$ fuzzball admin scheduler node list | jq '.nodes[] | select(.node.status == 2)'

Get resource summary:

$ fuzzball admin scheduler node list | jq '.nodes[] | {id: .node.id, hostname: .node.hostname, cpu_free: .resource.free.cpu.cores, mem_free_gb: (.resource.free.mem.bytes / 1073741824 | floor)}'

Count nodes by status:

$ fuzzball admin scheduler node list | jq '.nodes | group_by(.node.status) | map({status: .[0].node.status, count: length})'