DM-46885: Set up a Prompt Processing deployment configured for ComCam. #3793
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
erinleighh
force-pushed
the
tickets/DM-46885
branch
2 times, most recently
from
21f8612 to
9fb674d
Compare
bsmartradio
force-pushed
the
tickets/DM-46885
branch
from
3ae8669 to
6eacdce
Compare
hsinfang
approved these changes
Oct 25, 2024
|
|
||
| s3: | ||
| imageBucket: rubin-summit | ||
| endpointUrl: https://s3dfrgw.slac.stanford.edu |
There was a problem hiding this comment.
https://sdfembs3.sdf.slac.stanford.edu
| imageBucket: rubin-summit | ||
| endpointUrl: https://s3dfrgw.slac.stanford.edu | ||
|
|
||
| raw_microservice: http://172.24.5.144:8080/presence |
|
|
||
| imageNotifications: | ||
| kafkaClusterAddress: prompt-processing-2-kafka-bootstrap.kafka:9092 | ||
| topic: rubin-prompt-processing-prod |
There was a problem hiding this comment.
rubin-summit-notification
| # BLOCK-271 is photon transfer curve calibrations | ||
| # BLOCK-295 is the daily calibration sequence as of May 27, 2024 | ||
| main: >- | ||
| (survey="BLOCK-306")=[${PROMPT_PROCESSING_DIR}/pipelines/LSSTComCam/ApPipe.yaml, |
There was a problem hiding this comment.
These survey names are for LATISS and are unlikely to be the same for LSSTComCam. See the Slack thread . It's okay to put in a simpler placeholder for now.
| @@ -15,11 +15,11 @@ Prompt Proto Service is an event driven service for processing camera images. Th | |||
| | prompt-proto-service.additionalVolumeMounts | list | `[]` | Kubernetes YAML configs for extra container volume(s). Any volumes required by other config options are automatically handled by the Helm chart. | | |||
| | prompt-proto-service.affinity | object | `{}` | Affinity rules for the prompt processing pods | | |||
| | prompt-proto-service.alerts.server | string | `"usdf-alert-stream-dev-broker-0.lsst.cloud:9094"` | Server address for the alert stream | | |||
| | prompt-proto-service.alerts.topic | string | None, must be set | Topic name where alerts will be sent | | |||
| | prompt-proto-service.alerts.topic | string | `""` | Topic name where alerts will be sent | | |||
There was a problem hiding this comment.
This commit of doc update should be squashed with the commit where you update the values
| image: | ||
| pullPolicy: IfNotPresent | ||
| # Overrides the image tag whose default is the chart appVersion. | ||
| tag: 4.5.1 |
| @@ -62,7 +62,7 @@ prompt-proto-service: | |||
| # -- A factor by which to multiply `baseSize` for refcat datasets. | |||
| refcatsPerImage: 4 | |||
| # -- A factor by which to multiply `baseSize` for templates and other patch-based datasets. | |||
| patchesPerImage: 4 | |||
| patchesPerImage: 6 | |||
There was a problem hiding this comment.
I don't know what's the optimal values, but I probably would start with those of LSSTComCamSim. Or where did you get these values that I probably missed?
There was a problem hiding this comment.
These values came from LATISS but I will update them to LSSTComCamSim's.
| # -- The URI to a microservice that maps image metadata to a file location. | ||
| # If empty, Prompt Processing does not use a microservice. | ||
| raw_microservice: "" | ||
|
|
There was a problem hiding this comment.
Any reason to delete this?
There was a problem hiding this comment.
Nope, that was an accident. Thanks for catching that!
| @@ -102,7 +98,6 @@ prompt-proto-service: | |||
| # -- Server address for the alert stream | |||
| server: "usdf-alert-stream-dev-broker-0.lsst.cloud:9094" | |||
| # -- Topic name where alerts will be sent | |||
| # @default -- None, must be set | |||
There was a problem hiding this comment.
The None default seems good to me. Any reason not?
| @@ -143,17 +138,14 @@ prompt-proto-service: | |||
| memoryRequest: "2Gi" | |||
| # -- The maximum memory limit for the full pod (see `containerConcurrency`). | |||
| memoryLimit: "8Gi" | |||
| # -- To acommodate scheduling problems, Knative waits for a request for twice `worker.timeout`. | |||
| # This parameter adds extra time to that minimum (seconds). | |||
| extraTimeout: 10 | |||
erinleighh
force-pushed
the
tickets/DM-46885
branch
from
6eacdce to
9b3f433
Compare
erinleighh
force-pushed
the
tickets/DM-46885
branch
from
ad8ea76 to
28f2c09
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.