Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature - auto-commit all outputs to special git branch #22

Open
7flash opened this issue Dec 11, 2024 · 1 comment
Open

Feature - auto-commit all outputs to special git branch #22

7flash opened this issue Dec 11, 2024 · 1 comment
Labels

Comments

@7flash
Copy link

7flash commented Dec 11, 2024

Issue Description

Hi! In this feature request, I would like to propose a feature which is needed to make it possible implementing more advanced workflows with devai.

Currently, devai allows to override existing files and even allows to append output content into existing markdown files. However, more advanced workflows require to compare multiple variations of outputs. For example, invoking the same prompt but with different temperatures, can be very useful for coding tasks, allowing you to compare in between of "conservative" or "creative" changes over the same file using your favorite git interface.

@jeremychone jeremychone changed the title fr: auto-commit all outputs to special git branch FR - auto-commit all outputs to special git branch Dec 11, 2024
@jeremychone
Copy link
Owner

jeremychone commented Dec 11, 2024

This is a great feature request.

So, here is how it is going to work to fit into the existing model.

  • All sections will have access to another global scope, which is going to be config.
  • This is, by default, the representation of
    • the .devai/config.toml
    • overridden by the eventual # Config section.
  • Then, in the # Before All and # Data Lua sections, one can override those with a modified or completely different one.
  • When done at the # Before All, this means that all inputs will receive the new one.
  • When done at the # Data level, it will impact ONLY this input.

And the last point there is where you can do what you were asking.

Because now, the # Before All can reshape (add) the inputs to inject some specific config,
for example, an array [ {input: input_1, config: config_a}, {input: input_1, config: config_b}].

-- # Before All
-- Here we assume we have only one input (one -f file or one -i)
-- Otherwise, just need to build a new list and do 2x or nx for each input

local config_a = { genai = { model = config.genai.model, temperature = 1} }
local config_b =  { genai = { model = config.genai.model, temperature = 0} }

return devail.before_all({
   before_all = "Any data I want all inputs to track to have",
   inputs = {
       {input = inputs[1], config = config_a},
       {input = inputs[1], config = config_b},
   }
})

 -- For the items item above, if an item has .input, the .input becomes .input; otherwise, the item becomes the input.

And then, each data section will have the .input and .config as above

-- # Data

local file = utils.file.load(input.path)

-- By default, the config sent above, so input 1 will be config_a, and input 2 will be config_b.

-- Here, if we wanted, we could even keep overriding the config before the genai exec.
return devail.data({
   data = { file = file },
   config = another_config, -- this will bypass the config of this input track
})

And then, in the output, we have the following in the scope:

-- # Output

-- ai_response - with the config returned by # Data
-- config      - still has access to config, now the one returned by # Data or the default one
-- input       - which will also have the config as {input, config}

Hope that makes sense, but this will allow us to fully utilize the map-reduce for this feature, which has many benefits.

By the way, at some point, I will add ai_response.duration and ai_response.usage (for the tokens).

@jeremychone jeremychone changed the title FR - auto-commit all outputs to special git branch Feature Request - auto-commit all outputs to special git branch Dec 11, 2024
@jeremychone jeremychone changed the title Feature Request - auto-commit all outputs to special git branch Feature - auto-commit all outputs to special git branch Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants