Skip to content

Latest commit

 

History

History
103 lines (86 loc) · 3.07 KB

README-dev.md

File metadata and controls

103 lines (86 loc) · 3.07 KB
Raw

Experimental wandb client

https://paper.dropbox.com/doc/Cling-CLI-Refactor-lETNuiP0Rax8yjTi03Scp

Play along

pip install --upgrade git+git://github.com/wandb/client-ng.git#egg=wandb-ng

Or from pypi:

Code organization

wandb/sdk                - User accessed functions [wandb.init()] and objects [WandbRun, WandbConfig, WandbSummary, WandbSettings]
wandb/sdk_py27           - Generated files [currently by strip.sh]
wandb/backend            - Support to launch internal process
wandb/interface          - Interface to backend execution 
wandb/proto              - Protocol buffers for inter-process communication and persist file store
wandb/internal           - Backend threads/processes
wandb/apis               - Public api (still has internal api but this should be moved to wandb/internal)
wandb/cli                - Handlers for command line functionality
wandb/stuff              - Stuff copied from wandb-og, needs to be refactored or find a place
wandb/agent              - agent/super agent stuff
wandb/framework/keras    - keras integration
wandb/framework/pytorch  - pytorch integration

Code checks

  • Reformat: tox -e reformat
  • Type check: tox -e mypy
  • Misc: tox

Tasks

  • Improve hybrid (poor connectivity mode) - jhr
  • Add metadata sync - jhr
  • Add system metrics - jhr
  • Add summary metrics mirror - jhr
  • Add file sync
  • Add media logging
  • Add keras framework
  • Fix grouping / job_type / run naming
  • Fix file descriptor close (message + hang on pix2pix)
  • Fix colab console redir error
  • Fix issue in pix2pix with logging
  • Add code saving
  • Add pytorch framework
  • Add other frameworks
  • Basic CLI functionality
  • Support modes
  • Decide on persistant storage
  • Offline sync - jhr

Progress

API:

  • wandb.init() basic
  • wandb.log() basic
  • wandb.join() basic
  • wandb.run basic
  • wandb.config basic
  • wandb.save()
  • wandb.restore()
  • wandb.init() full
  • wandb.log() full
  • wandb.join() full
  • wandb.run full
  • wandb.config full
  • wandb.sweep()
  • wandb.agent()
  • wandb.controller()

CLI:

  • wandb login
  • wandb sync

Functionality:

  • system metrics
  • console log
  • offline
  • Unit tests
  • code coverage

Goals:

  • standardize all CLI->backend updates
  • reorganize code to avoid different contexts (run_manager)
  • jupyter simplification using request queues or RPC
  • utilize more standard methods for background process
  • better isolation for extended features (system monitoring, git logging)
  • offline support improvements: enforce constraints at sync time (code logging, etc)
  • internal api becomes fully internal, only used by "internal" process
  • telemetry of all operations

Bonus:

  • multi-language synchronizer
  • schema'ed binary? cloud? log for offline run data
  • less (no) dependance on local filesytem
  • type annotations
  • cleaned up logger