Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Visualize SignWriting more pictographically #114

Open
AmitMY opened this issue Nov 3, 2023 · 1 comment
Open

[Feature] Visualize SignWriting more pictographically #114

AmitMY opened this issue Nov 3, 2023 · 1 comment

Comments

@AmitMY
Copy link
Contributor

AmitMY commented Nov 3, 2023

Problem

People without previous SignWriting experience have a hard time understanding SignWriting notation that looks like this:
image

Even when paired with a video producing these signs.

Description

We should support an alternative view to SignWriting, perhaps showing pictographs of the relevant signs.

For example:

SignWriting Pictograph

Probably the easiest way to get a POC done: https://github.com/lllyasviel/ControlNet
(data should be then annotated with a textual prompt "man" or "woman" signing, and can be longer like "man with short hair signing")
image

Alternatives

Once videos are generated directly from SignWriting, we can also show the video for each sign.

Additional context

This can be server side only, doesn't have to be client-side, since it is not necessary for the translation, only assists in visualization. It will also make it faster, less heavy for the client, and cacheable

Data can be extracted from:

This feature is somewhat inspired by https://www.youtube.com/watch?v=Eq3bUFgEcb4 - I don't think that notation must die, but certainly we can give alternative notation to different use cases

@AmitMY
Copy link
Contributor Author

AmitMY commented Nov 9, 2023

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant