Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat] refactor data storage to allow saving on a per-window basis #862

Open
wants to merge 17 commits into
base: master
Choose a base branch
from

Conversation

shouldsee
Copy link
Contributor

@shouldsee shouldsee commented Sep 2, 2022

Description

  • changed eager mode logic
  • changed data saving and loading logic
    • old logic in SimpleJsonStorage
    • per-window logic in SimpleWindowJsonStorage
    • using hard-coded MRO to switch between the logic for now.
  • added a utility script for testing
  • added auto directory creation
  • added DEBUG utility printing

Motivation and Context

#861 . changing towards a per-window caching scheme

How Has This Been Tested?

  • rm -rf ./test_data/*
  • start server python server.py -env_path test_data
  • python test_simple.py create plots an save
  • closed server
  • reopen server
  • check env:test is correctly loaded

Screenshots (if appropriate):

tree test_data/
test_data/
├── test
│   ├── jsons
│   │   ├── 0.json
│   │   ├── 1.json
│   │   ├── 2.json
│   │   ├── 3.json
│   │   └── 4.json
│   └── reload.json
└── view

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.

…to these modes

    parser.add_argument('-cache_type', metavar='cache_type', type=str,
                        default=DEFAULT_CACHE_TYPE,
                        help='''specify how the received data should be synced
                        between memory and disk.
                          - JPE/OneJsonPerEnv:             one json per environment
                          - JPW/OneJsonPerWindow:          one json per window
                          - JPWA/OneJsonPerWindowAutoSave: one json per window,
                                                           autosave to disk when plotting''')
shouldsee and others added 9 commits September 3, 2022 16:17
…to these modes

    parser.add_argument('-cache_type', metavar='cache_type', type=str,
                        default=DEFAULT_CACHE_TYPE,
                        help='''specify how the received data should be synced
                        between memory and disk.
                          - JPE/OneJsonPerEnv:             one json per environment
                          - JPW/OneJsonPerWindow:          one json per window
                          - JPWA/OneJsonPerWindowAutoSave: one json per window,
                                                           autosave to disk when plotting''')
Copy link
Contributor

@JackUrb JackUrb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @shouldsee. Thanks for putting up a PR. I'll admit, this one is pretty difficult to review for a number of reasons.

  • Class naming here is somewhat obtuse. B, NIF, etc
  • Without explanations for these classes both inline and in some kind of overall explanation, it's hard to follow your intent and methodology.
  • Overall, these additions should exist outside the server.py file, which is already bloated (and part of why we're trying to refactor).
  • Addition of Makefile is complicating the devX here.
  • Last, there are some comment-ed out changes (like in demo.py), which we really shouldn't be maintaining in the codebase, alongside conditionals such as if 1:

I'm open to the feature change you're proposing here, but I'm definitely not ready to fully review, let alone accept, in the current state. Would love to see it all in a more final state.

@@ -20,7 +20,7 @@ class TextPane extends React.Component {
switch (e.type) {
case 'keydown':
case 'keypress':
e.preventDefault();
// e.preventDefault(); /// this would prevent entering text into input box
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC this was trying to prevent text panes from submitting on enter. Plus, the goal is to send keypress events to the backend, not necessarily to have them render up-front here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see. I was putting forms into visdom window, and .preventDefault would make the form inputting fairly laggy, hence commented out.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps there's value in a new FormPane component, likely a parent class for the PropertiesPane. This would then be able to accomplish what you're attempting in #868, but in a cleaner way

@@ -0,0 +1,21 @@
clean:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm somewhat against adding yet another scripting flow to the visdom codebase. This should either be part of the package.json scripts, or we should be having a discussion about the alternate we should be using instead.

'Visdom failed to handle a handler for {}: {}'
''.format(message, e)
'Visdom failed to handle a handler for {}: {}.'
'\n traceback:{}'.format(message, e, tbmsg)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nice-to-have addition

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants