Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

misc api and commandset #411

Merged
merged 17 commits into from
Sep 22, 2023
Merged

misc api and commandset #411

merged 17 commits into from
Sep 22, 2023

Commits on Sep 22, 2023

  1. Don't list subsets as resources in icfg

    To avoid these outputs:
    
    root@dev2n1:~/dev/om3# ./om3 foo00* resource ls
    OBJECT NODE   RID             TYPE
    foo003 dev2n3 fs#1            fs.flag
    foo003 dev2n3 fs#2            fs.tmpfs
    foo003 dev2n3 subset#task:goo <status is not found>
                                  ^^^^^^^^^^^^^^^^^^^^^
    cvaroqui committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    12fbcb9 View commit details
    Browse the repository at this point in the history
  2. network handlers and command rework

    * Rename GET /networks to GET /networks
    * Add GET /network/ip
    * Convert "network ls" to the tab output
    * Remove "network status", now all info is avail via "network ls"
    * Add "network ip ls"
    
    Example:
    
    $  ./om3 network ls
    NAME    TYPE          NETWORK       FREE USED SIZE PCT
    backend routed_bridge 172.10.8.0/22 0    0    0    0
    default bridge        10.22.0.0/16  0    0    0    0
    lo      lo                          0    0    0    0
    
    $ ./om3 network ip ls
    OBJECT   NODE   RID  IP           NET_NAME NET_TYPE
    nginx002 dev2n1 ip#1 172.10.8.56  backend  routed_bridge
    nginx001 dev2n1 ip#1 172.10.8.57  backend  routed_bridge
    nginx002 dev2n3 ip#1 172.10.10.94 backend  routed_bridge
    nginx002 dev2n2 ip#1 172.10.9.94  backend  routed_bridge
    cvaroqui committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    d7789ff View commit details
    Browse the repository at this point in the history
  3. Fix the network usage metrics

    $ ./om3 net ls
    NAME    TYPE          NETWORK       SIZE  USED FREE  USE_PCT
    local   routed_bridge 172.11.8.0/30 4     0    4     0
    backend routed_bridge 172.10.8.0/22 1024  4    1020  0.390625
    lo      lo                          1     0    1     0
    default bridge        10.22.0.0/16  65536 0    65536 0
    cvaroqui committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    cb7165e View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    f405fce View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    667174a View commit details
    Browse the repository at this point in the history
  6. Support fnmatch in the resource selector

    * Add resourceid.Match(s1, s2 string)
    * Add resourceid.T.Match()
    * Make resource.T.MatchRID() use resourceid.T.Match()
    * Use resourceid.Match() in the GET /resource* api handlers
      instead of the simple rid equality test
    
    Example:
    
    $ ./om3 '*' resource status ls --rid f*0
    OBJECT  NODE   RID  TYPE    STATUS
    drbd001 dev2n1 fs#0 fs.flag down
    drbd001 dev2n2 fs#0 fs.flag down
    drbd001 dev2n3 fs#0 fs.flag down
    cvaroqui committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    8877740 View commit details
    Browse the repository at this point in the history
  7. Replace the yaml implementation for one that uses the json tags

    So we don't have to:
    1/ add yaml tags to all serialized struct
    2/ add yaml tags to api.yaml schemas using a go-specific extension
    cvaroqui committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    3015f9b View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    695e7e4 View commit details
    Browse the repository at this point in the history
  9. om pool command and api tidy up

    * Use the tab renderer for "pool ls" and "pool volume ls"
    * Fix the PoolList not reporting the "shm" pool
    * Use a int64 to report pool usage and vol size in bytes
    * Add Pool and Size to instance.Config so "volume ls" doesn't
    have to go to disk to read this on every request
    * Add a path.Relations type to ease conversion to/from string
    slice
    cvaroqui committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    8f9b251 View commit details
    Browse the repository at this point in the history
  10. Add a go routine to nmon to janitor a pool status cache

    Refresh every minute or when the node or cluster conf changes.
    Make sure only one refresh runs at a time.
    
    GET /pool is now served by this cache, and it now contains the
    volume count.
    cvaroqui committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    e4228da View commit details
    Browse the repository at this point in the history
  11. Remove the api import from nodeselector

    Dedicate the package to the nodesinfo.json management.
    
    The only caller of GET /nodesinfo is nodeselector, so move the
    api func call there.
    cvaroqui committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    beb4b3b View commit details
    Browse the repository at this point in the history
  12. Add the "vol" cobra cmd alias for "volume"

    As in "pool volume ls".
    cvaroqui committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    89bbd7d View commit details
    Browse the repository at this point in the history
  13. Avoid Marshal/Unmarshal data in the core/output renderer

    This Marshal concrete > Unmarshal in a any-type var was done to
    produce a unstructured dataset with keys aligned with the json
    tags.
    
    But this method has poor performance issue and the resulting data
    has unexpected glitches (for example big int64 value end up in
    float-like format, as seen in "pool ls" outputs).
    
    This patch add a unstructed.L type and a Unstructured() map[string]any
    type-func to every type passed to the core/output renderer with
    tab as the default.
    
    The unstructed type will also allow commands to mangle the
    dataset for unit changes and pretty prints, as values can change
    type and even new keys can be added.
    cvaroqui committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    0e63468 View commit details
    Browse the repository at this point in the history
  14. Error on tab render on a type not implementing Unstructured()

    Instead of stacking.
    
    Example:
    
    $ ./om3 foo002 print st -o tab=.
    unstructured interface is not implemented: object.Digest
    cvaroqui committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    35c30f9 View commit details
    Browse the repository at this point in the history
  15. Generalize the Unstructured() use

    * Remove handlers and commands (unnecessarily fined-grained)
    
      instance config
      instance monitor
      instance status
      resource config
      resource monitor
      resource status
    
    * Use plurals for
    
      GET /objects
      GET /instances
      GET /resources
      GET /pools
      GET /pool/volumes
      GET /networks
      GET /network/ips
    
    * Don't return lists. Use a wrapper struct like
      {"kind": "NodeList", "items": []Node{}}
    cvaroqui committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    34b1ab8 View commit details
    Browse the repository at this point in the history
  16. Add a kind property to each api list items

    So it's easy to understand what kind of element users are
    reading data of.
    cvaroqui committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    1370d40 View commit details
    Browse the repository at this point in the history
  17. Report "pool ls" quantities in human readable form by default

    Mangle the items received from the api to add bin_free,
    bin_used and bin_size as 3-digits+unit approximation of
    the original int64 values.
    
    Example:
    
    {
            "bin_free": "2.12g",
            "bin_size": "30.3g",
            "bin_used": "26.6g",
    }
    
    For original values:
    
    {
            "free": 2278817792,
            "size": 32574881792,
            "used": 28615196672,
    }
    cvaroqui committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    2bcdc72 View commit details
    Browse the repository at this point in the history