Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial higher limits and default behavior for private tool resource restrictions #132

Closed
wants to merge 1 commit into from
Closed

Add initial higher limits and default behavior for private tool resource restrictions #132

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 14 additions & 5 deletions src/apps/tools/private.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,21 @@

(defn- restrict-private-tool-container
"Restrict the networking, CPU shares, and memory limits for the tool's container."
[{:keys [pids_limit memory_limit] :or {pids_limit (cfg/private-tool-pids-limit)
memory_limit (cfg/private-tool-memory-limit)}
[{:keys [pids_limit memory_limit max_cpu_cores] :or {pids_limit (cfg/private-tool-pids-limit)
memory_limit (cfg/private-tool-memory-limit)
max_cpu_cores (cfg/private-tool-max-cpu-cores)}
:as container}]
(assoc container :network_mode "none"
:pids_limit (restrict-private-tool-setting pids_limit (cfg/private-tool-pids-limit))
:memory_limit (restrict-private-tool-setting memory_limit (cfg/private-tool-memory-limit))))
:max_cpu_cores (restrict-private-tool-setting max_cpu_cores (cfg/tool-max-cpu-cores))
:pids_limit (restrict-private-tool-setting pids_limit (cfg/private-tool-pids-limit))
:memory_limit (restrict-private-tool-setting memory_limit (cfg/tool-memory-limit))))

(defn- set-private-tool-defaults
"Set the default pid/memory/cpu restrictions for a private tool, if they're unset"
[{:keys [pids_limit memory_limit max_cpu_cores] :as container}]
(assoc container :pids_limit (or pids_limit (cfg/private-tool-pids-limit))
:memory_limit (or memory_limit (cfg/private-tool-memory-limit))
:max_cpu_cores (or max_cpu_cores (cfg/private-tool-max-cpu-cores))))

(defn- restrict-private-tool-time-limit
"Restrict the tool's time limit setting."
Expand Down Expand Up @@ -69,7 +78,7 @@
restrict-private-tool
(assoc :implementation (ensure-default-implementation user implementation))
persistence/add-tool)]
(containers/add-tool-container tool-id (restrict-private-tool-container container))
(containers/add-tool-container tool-id (restrict-private-tool-container (set-private-tool-defaults container)))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The basic gist here is: set-private-tool-defaults should set anything that isn't provided to the (lower) default limits, where restrict-private-tool-container has been repurposed to restrict things down to the higher maximum limits. I've kept it so the latter will still default the values to the default limits, but I wanted to be slightly more explicit, particularly because ultimately we may want to move restrict-private-tool-container to another namespace and apply it to all tools, not just private ones.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original intent was to restrict private tools to a maximum of the apps.tools.private.memory-limit config (16GB), whether the user provided no limit, or if the provided limit was higher than this configured max.

So is it now the intent to allow the user to set a higher limit for private tools, up to the apps.tools.memory-limit max for all tools (32GB)? Or do we still want to restrict private tools to a lower limit than public tools (of course, an admin should still be able to lift the limit to the apps.tools.memory-limit max)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That all tools would be subject to the same effective maximum was my understanding, yeah -- with the prior limits for private tools becoming the defaults instead (and we'll also be hiding this slightly behind something saying it's advanced behaviour). Not much benefit to adding an interface for this unless it can be adjusted both up and down, and I don't think we want to bring the defaults down (though we could -- VICE defaults to 2 cores and I forget how much memory, but I think less than the current private tool defaults). It's also worth noting that the trajectory here is towards this being only the max, adding a field also for the minimum (default whatever value we consider acceptable as the smallest slot), and then allowing users to select a value between them at runtime. Once again there, we'd probably peg the defaults somewhere like these older values, I think

(perms-client/register-private-tool shortUsername tool-id)
(tools/get-tool shortUsername tool-id))))

Expand Down
15 changes: 15 additions & 0 deletions src/apps/util/config.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,21 @@
[props config-valid configs]
"apps.tools.private.memory-limit" (* 16 1024 1024 1024)) ;; 16GB

(cc/defprop-optlong private-tool-max-cpu-cores
"The number of cpu cores to use when adding new private tools"
[props config-valid configs]
"apps.tools.private.max-cpu-cores" 4.0)

(cc/defprop-optint tool-memory-limit
"The maximum memory limit, in bytes, that a (private) tool may be created with"
[props config-valid configs]
"apps.tools.memory-limit" (* 32 1024 1024 1024)) ;; 32GB

(cc/defprop-optlong tool-max-cpu-cores
"The maximum number of max cpu cores that a (private) tool may be created with"
[props config-valid configs]
"apps.tools.max-cpu-cores" 16.0)

(cc/defprop-optstr workspace-root-app-category
"The name of the root app category in a user's workspace."
[props config-valid configs]
Expand Down