Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added description and comment columns in all existing tables and apis #284

Merged
merged 1 commit into from
Jan 3, 2025

Conversation

sauraww
Copy link
Collaborator

@sauraww sauraww commented Nov 14, 2024

Problem

Added description and comment columns in all existing tables and apis

Solution

Provide a brief summary of your solution so that reviewers can understand your code

Environment variable changes

What ENVs need to be added or changed

Pre-deployment activity

Things needed to be done before deploying this change (if any)

Post-deployment activity

Things needed to be done after deploying this change (if any)

API changes

Endpoint Method Request body Response Body
API GET/POST, etc request response

@sauraww sauraww requested a review from a team as a code owner November 14, 2024 11:01
@sauraww sauraww force-pushed the feat/add-description-and-commit branch from e80798d to 8f36918 Compare November 14, 2024 11:02
@@ -0,0 +1,19 @@
-- This file should undo anything in `up.sql`
-- contexts table
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these needed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No , I will remove it

@sauraww sauraww force-pushed the feat/add-description-and-commit branch 11 times, most recently from a37e9b2 to dfc8172 Compare November 14, 2024 15:02
Copy link
Collaborator

@mahatoankitkumar mahatoankitkumar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets make the fields non-nullable, as per the discussion.

Copy link
Collaborator

@Datron Datron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Description should be mandatory. I think comment too but we can discuss this

@@ -378,9 +378,21 @@ fn construct_new_payload(
},
)?;

let description = res
.get("description")
.and_then(|val| val.as_str())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This returns result right? We should handle that

let description = res
.get("description")
.and_then(|val| val.as_str())
.map(|s| s.to_string());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can do this inside and_then

Comment on lines 9 to 10
pub description: Option<String>,
pub comment: Option<String>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This two should be mandatory in this case, the frontend should just send ""

@sauraww sauraww force-pushed the feat/add-description-and-commit branch 14 times, most recently from c3b2999 to 0dd0666 Compare November 21, 2024 12:57
ALTER TABLE public.type_templates ADD COLUMN IF NOT EXISTS description TEXT DEFAULT '' NOT NULL;
ALTER TABLE public.type_templates ADD COLUMN IF NOT EXISTS change_reason TEXT DEFAULT '' NOT NULL;

ALTER TABLE public.functions RENAME COLUMN function_description TO description;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can break function ui for sometime

@@ -207,6 +208,8 @@ fn create_ctx_from_put_req(
tenant_config: &TenantConfig,
) -> superposition::Result<Context> {
let ctx_condition = req.context.to_owned().into_inner();
let description = req.description.to_owned().unwrap();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should not unwrap here,

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, unwrapping here is not safe, it is bound to break here

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed

let mut req_mut = req.into_inner();
if req_mut.description.is_none() {
req_mut.description = ensure_description(
Value::Object(req_mut.context.clone().into_inner().into()),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can pass directly as ContextType

log::info!("context put failed with error: {:?}", err);
err
})?;
let description = req_mut.description.unwrap_or_default();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can first clone only the description and change_reason above and then pass req_mut in put function

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are taking this out for add_config_version , we need to pass description and change_reason there also.

&state,
tags,
transaction_conn,
req_mut.description.unwrap().clone(),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no unwrap

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unwrap is safe as we have already ensured that description is going to be there in the above code.

.first::<Context>(transaction_conn)?;
delete_context_api(ctx_id.clone(), user.clone(), transaction_conn)?;
let description = context.description;
let change_reason = format!("Deleted context by {}", user.username);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change_reason should come from user request , as why he is deleting this context

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But in the delete request we don't have a reqBody .
We can leave it for now .

let version_id = add_config_version(&state, tags, transaction_conn)?;
let context = contexts_table
.filter(context_id.eq(ctx_id.clone()))
.first::<Context>(transaction_conn)?;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we return 404 or 204

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not return 204 if successful, 200 with the object that was deleted

.first()
.map(|r| r.description.clone())
.unwrap_or_else(|| "Default description".to_string());
let change_reason = "Priority recomputation".to_string(); // Use a constant or computed value
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't change reason come from api request

@@ -80,23 +80,51 @@ async fn create(

let result = fetch_default_key(&key, &mut conn);

let (value, schema, function_name, created_at_val, created_by_val) = match result {
let description = match &result {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we should have separated create and update api

@@ -176,27 +220,26 @@ async fn delete_dimension(
) -> superposition::Result<HttpResponse> {
let name: String = path.into_inner().into();
let DbConnection(mut conn) = db_conn;
let dimension_data: Dimension = dimensions::dsl::dimensions
.filter(dimensions::dimension.eq(&name))
let dimension_data = dimension_table
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there should be change reason for deleting dimension

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we do not take reason from the user, we should at least add that this user deleted the dimension

Copy link
Collaborator

@Datron Datron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Can you use map_err and ? instead of match statements to unwrap Result
  • Set a character limit to description and change_reason (not in database but in backend & frontend)
  • Re-consider using textarea for input
  • nitpick and out of scope (need not be done): separate out create and update for APIs so description is mandatory for creates but not updates

@@ -207,6 +208,8 @@ fn create_ctx_from_put_req(
tenant_config: &TenantConfig,
) -> superposition::Result<Context> {
let ctx_condition = req.context.to_owned().into_inner();
let description = req.description.to_owned().unwrap();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed

Comment on lines +383 to +400
let description = match res.get("description") {
Some(Value::String(s)) => Some(s.clone()),
Some(_) => {
log::error!("construct new payload: Description is not a valid string");
return Err(bad_argument!("Description must be a string"));
}
None => None,
};

// Handle change_reason
let change_reason = res
.get("change_reason")
.and_then(|val| val.as_str())
.map(|s| s.to_string())
.ok_or_else(|| {
log::error!("construct new payload: Change reason not present or invalid");
bad_argument!("Change reason is required and must be a string")
})?;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why have different processing flows for the same kind of data?

let version_id = add_config_version(&state, tags, transaction_conn)?;
let context = contexts_table
.filter(context_id.eq(ctx_id.clone()))
.first::<Context>(transaction_conn)?;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not return 204 if successful, 200 with the object that was deleted

@@ -10,6 +10,8 @@ pub struct CreateReq {
pub schema: Option<Map<String, Value>>,
#[serde(default, deserialize_with = "deserialize_option")]
pub function_name: Option<Value>,
pub description: Option<String>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing description is optional because we use this for update as well?

@@ -11,6 +11,8 @@ pub struct CreateReq {
pub schema: Value,
#[serde(default, deserialize_with = "deserialize_option")]
pub function_name: Option<Value>,
pub description: Option<String>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is create and update combined here?

Comment on lines 90 to 94
let existing_template = type_templates::table
.filter(type_templates::type_name.eq(&type_name))
.first::<TypeTemplates>(&mut conn)
.optional()
.map_err(|err| {
log::error!("Failed to fetch existing type template: {}", err);
db_error!(err)
})?;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use map_err and convert the error type into bad_argument and then use '?' to throw the error

Comment on lines +376 to +381
let description = match req.description.clone() {
Some(desc) => desc,
None => experiment.description.clone(),
};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use ? operator

@@ -11,6 +11,7 @@ use superposition_types::{
enum Dimensions {
Os(String),
Client(String),
#[allow(dead_code)]
VariantIds(String),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If not used lets get rid of it

Comment on lines +175 to +196
<textarea
placeholder="Enter description"
class="textarea textarea-bordered w-full max-w-md"
value=description_rs.get_untracked()
on:change=move |ev| {
let value = event_target_value(&ev);
description_ws.set(value);
}
/>
</div>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use the custom Input component?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also limit the size of description, makes our UI more predictable

Comment on lines +192 to +212
<textarea
placeholder="Enter change_reason"
class="textarea textarea-bordered w-full max-w-md"
value=change_reason_rs.get_untracked()
on:change=move |ev| {
let value = event_target_value(&ev);
change_reason_ws.set(value);
}
/>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should change reason be so large? Maybe input is sufficient and encourages brevity

Copy link
Collaborator

@ayushjain17 ayushjain17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partial review

Comment on lines 16 to 20
pub name: String,
pub context: Exp<Condition>,
pub variants: Vec<Variant>,
pub description: String,
pub comment: String,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is also being used for update as well, right ?
description we planned to keep optional for updates

@@ -86,6 +92,8 @@ pub struct ExperimentsResponse {
#[derive(Deserialize, Debug)]
pub struct ConcludeExperimentRequest {
pub chosen_variant: String,
pub description: String,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does concluding a experiment require a description ?

@@ -176,6 +186,8 @@ pub struct ExpListFilters {
#[derive(Deserialize, Debug)]
pub struct RampRequest {
pub traffic_percentage: u64,
pub description: String,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here, ramp should not require a description change

@@ -189,11 +201,15 @@ pub struct VariantUpdateRequest {
#[derive(Deserialize, Debug)]
pub struct OverrideKeysUpdateRequest {
pub variants: Vec<VariantUpdateRequest>,
pub description: String,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is supposed to be optional right ?

Comment on lines +892 to +899
description: description.clone(),
change_reason: change_reason.clone(),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cloning is not needed here

@@ -16,6 +18,8 @@ pub struct TypeTemplateResponse {
pub created_at: String,
pub last_modified: String,
pub created_by: String,
pub description: Option<String>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the response description should be mandatory, as its giving a response from db where it is already mandatory

@@ -11,6 +11,8 @@ pub struct CreateReq {
pub schema: Value,
#[serde(default, deserialize_with = "deserialize_option")]
pub function_name: Option<Value>,
pub description: Option<String>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dimension create request should not have description as optional, it should be mandatory here

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the same one was used for updating

@@ -176,27 +220,26 @@ async fn delete_dimension(
) -> superposition::Result<HttpResponse> {
let name: String = path.into_inner().into();
let DbConnection(mut conn) = db_conn;
let dimension_data: Dimension = dimensions::dsl::dimensions
.filter(dimensions::dimension.eq(&name))
let dimension_data = dimension_table
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we do not take reason from the user, we should at least add that this user deleted the dimension

Comment on lines 81 to 116
let dimension_name = create_req.dimension.to_string();

let existing_dimension = match dimension_table
.filter(dimension.eq(&dimension_name))
.first::<Dimension>(&mut conn)
.optional()
{
Ok(dim) => dim,
Err(e) => {
log::warn!(
"Failed to fetch dimension {}: {:?}. Proceeding as if it does not exist.",
dimension_name,
e
);
None
}
};

let description = match &create_req.description {
Some(desc) if !desc.trim().is_empty() => desc.clone(),
_ => {
if let Some(existing_dim) = &existing_dimension {
existing_dim.description.clone()
} else {
// No description provided and no existing dimension
// Return an error
log::error!(
"No description provided and no existing dimension for {}.",
dimension_name
);
return Err(bad_argument!(
"No description provided for the new dimension {}.",
dimension_name
));
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is should ideally be in a transaction, lets separate out the create and update requests, its difficult to verify these changes
separating these out will make all these logics way simpler

in insert, description is mandatory => create entity with the request as is
in update, description is optional => if description is sent use it, else fetch from db and use the old description in the update query

@sauraww sauraww force-pushed the feat/add-description-and-commit branch 3 times, most recently from aeb97be to a9ecce7 Compare December 18, 2024 13:36
@sauraww sauraww changed the base branch from main to saas December 18, 2024 13:37
let existing_context = contexts_table
.filter(context_id.eq(context_id_value))
.first::<Context>(transaction_conn)
.optional()?; // Query the database for existing context
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why we are making this optional ?
it should return db error

match existing_context {
Some(ctx) => Ok(ctx.description),
None => Err(superposition::AppError::NotFound(format!(
"Description Not Provided in existing context",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

error should be something different because if context is there , description will also be there
so if db gives not found error , then we will return description not provided
if something else error then we can say something went wrong , or could not fetch description

let mut req_mut = req.into_inner();

// Use the helper function to ensure the description
if req_mut.description.is_none() {
Copy link
Collaborator

@pratikmishra356 pratikmishra356 Dec 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not a blocker , also this can be argued
here can we do like this ,
let description = either from request_mut or if it is none then from ensure_description , also assign in req_mut.description
so that here description will be of type and we will not have to use unwrap_or_default
also declare change_reason here only, so that we wont have to cline the whole req_mut

let description = if req.description.is_none() {
ensure_description(ctx_condition_value.clone(), conn)?
} else {
req.description.expect("Description should not be empty")
Copy link
Collaborator

@pratikmishra356 pratikmishra356 Dec 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a suggestion
I know this will work , but if we can use match expression or abstract the some part without expect or unwrap
would be great

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expect will cause a panic, this error message won't be logged properly as well. We should throw an AppError here

log::error!(
"Failed to execute query while recomputing weight, error: {err}"
);
db_error!(err)
})?;
}
let version_id = add_config_version(&state, tags, transaction_conn)?;
let description = contexts_new_weight.iter().map(|(_, _, description, _)| description.clone()).collect::<Vec<String>>().join(",");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of using change_reason and description of all contexts,
lets add only this
description: weight recomputed
change_reason: weight recomputed

@@ -75,12 +81,24 @@ pub struct FunctionsInfo {
pub code: Option<String>,
}

#[derive(Serialize)]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are not using this anymore

@sauraww sauraww force-pushed the feat/add-description-and-commit branch 3 times, most recently from 15db62c to a5b0055 Compare December 18, 2024 19:47
@Datron Datron requested a review from pratikmishra356 January 2, 2025 10:11
@sauraww sauraww force-pushed the feat/add-description-and-commit branch 4 times, most recently from 8a27413 to ab85eb7 Compare January 3, 2025 08:10
Copy link
Collaborator

@Datron Datron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Main concern: the error message for a context description not being found.

Nitpicks: no expect calls though they are safe since it throws off clippy checks

log::error!("construct new payload: Description is not a valid string");
return Err(bad_argument!("Description must be a string"));
}
None => None,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sauraww construct_new_payload function is used by reduce, I don't know if we can just get the description like this. I think we'll need to combine all the descriptions being reduced into one or

insert a new description

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you suggesting that we take it as input in the reduce api itself ?

} else {
req.description
.clone()
.expect("Description should not be empty")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No expects, throw an error here

@@ -198,6 +198,15 @@ fn create_ctx_from_put_req(
tenant_config: &TenantConfig,
) -> superposition::Result<Context> {
let ctx_condition = req.context.to_owned().into_inner();
let description = if req.description.is_none() {
let ctx_condition_value = Value::Object(ctx_condition.clone().into());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: would json!(ctx_condition) work?

Comment on lines 379 to 381
Err(diesel::result::Error::NotFound) => Err(superposition::AppError::NotFound(
"Description not provided in existing context".to_string(),
)),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NotFound error message is confusing. A context was not found, but this message indicates only the description is missing. Probably based on the message the error code should be bad request

let description = if req.description.is_none() {
ensure_description(ctx_condition_value.clone(), conn)?
} else {
req.description.expect("Description should not be empty")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expect will cause a panic, this error message won't be logged properly as well. We should throw an AppError here

} else {
put_req
.description
.expect("Description should not be empty")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No expect

@@ -1,6 +1,6 @@
[print_schema]
file = "schema.rs"
patch_file = "schema.patch"
#patch_file = "schema.patch"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need schema.patch right?

@sauraww sauraww force-pushed the feat/add-description-and-commit branch from ab85eb7 to 8d03179 Compare January 3, 2025 10:49
@sauraww sauraww dismissed stale reviews from ayushjain17 and mahatoankitkumar January 3, 2025 11:14

unavailable

@sauraww sauraww merged commit 8311f36 into saas Jan 3, 2025
4 checks passed
@sauraww sauraww deleted the feat/add-description-and-commit branch January 3, 2025 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants