Please add LLaVA

[Xyndra]

Please I want LLaVA.
https://llava-vl.github.io/

[eletroswing]

I loved the idea of ​​having this opensource and completely free

[AltayYuzeir]

Maybe even combination between LLaVA 1.5 and DeepSeek Coder.

[Anon2578]

Yes this is a very cool idea I’ll be here for it once it’s up and ready

[eletroswing]

@AltayYuzeir how llava 1.5 and deepseek coder can be implemented to work together or what it would look like in theory?

[AltayYuzeir]

Hmmm, perhaps I was inaccurate in my statement. I have been thinking about it, and I cannot comprehend direct communication on the neural network level between the two LLMs. I came up with two concepts:

1. If LLaVA is asked to produce a detailed description of the drawing, also including spacing between elements and alignments, the description can be passed on DeepSeek Coder
2. Maybe similar to LLaVA-Med, LLaVA can be fine-tuned on UI layouts, HTML and CSS

[eletroswing]

@AltayYuzeir I'm going to create a dataset of 'images to tailwind' using deepseek coder, and then we can train this custom model, what do you think?

[AltayYuzeir]

I am not the best AI or UI programmer, but I believe such resource will be beneficial in any case for the future. As far as I understand it, Tailwind is a high-level framework on top of CSS, featuring custom classes absent in native CSS. Tailwind is not explicitly supported on DeepSeek Coder, as per info from their GitHub page. I was thinking it might be perhaps more flexible to use DeepSeek to generate more general webpages built on top of raw HTML and CSS. Please, correct me if I am wrong.

[eletroswing]

@AltayYuzeir You are right! Similar to what vercel is doing with v0.dev. Anyway, I'm going to continue creating the dataset using tailwind used in public projects!

[AltayYuzeir]

I guess one could make a dataset for fine tuning based on UIs made with Dash (Python) and Shiny (R)?
I wonder about the capabilities of DeepSeek Coder for Dash and Shiny specifically, since they are libraries for their respective languages, but are also frameworks, which function quite differently than most packages.

[orkutmuratyilmaz]

@eletroswing and @AltayYuzeir, supporting Ollama can solve all these requirements at once. Please check #52

[bocklucas]

#99