New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add AI translation functionality #147

Merged

maallen merged 35 commits into master from mallen/gpt_for_translate

Oct 11, 2024

Member

maallen commented Sep 26, 2024 •

edited

Loading

This PR adds the ability to automate the translation of source text units into target languages using AI within Mojito.

Further PRs needed to include these translations to be pushed to third parties as part of the ThirdPartySync and to add support in GUI to display them appropriately.

maallen force-pushed the mallen/gpt_for_translate branch 5 times, most recently from d6e8052 to 0352256 Compare

September 27, 2024 09:43

maallen requested review from byronantak, DarKhaos, mattwilshire and garionpin

September 27, 2024 10:15

maallen force-pushed the mallen/gpt_for_translate branch from 21eed01 to f3940fd Compare

September 27, 2024 10:28

maallen marked this pull request as ready for review

September 30, 2024 12:39

maallen requested a review from a team as a code owner

September 30, 2024 12:39

byronantak reviewed

View reviewed changes

webapp/src/main/java/com/box/l10n/mojito/service/ai/RepositoryLocaleAIPromptRepository.java Outdated Show resolved Hide resolved

byronantak reviewed

View reviewed changes

...main/java/com/box/l10n/mojito/service/ai/translation/AITranslationTextUnitFilterService.java Outdated Show resolved Hide resolved

byronantak reviewed

View reviewed changes

webapp/src/main/java/com/box/l10n/mojito/service/asset/VirtualTextUnitBatchUpdaterService.java Outdated Show resolved Hide resolved

webapp/src/test/java/com/box/l10n/mojito/service/ai/translation/AITranslateJobTest.java Outdated Show resolved Hide resolved

byronantak reviewed

View reviewed changes

webapp/src/main/java/com/box/l10n/mojito/service/assetExtraction/AssetExtractionService.java Outdated Show resolved Hide resolved

maallen force-pushed the mallen/gpt_for_translate branch from 5fc7c78 to 99dafcd Compare

September 30, 2024 15:40

maallen requested a review from byronantak

September 30, 2024 16:05

garionpin reviewed

View reviewed changes

webapp/src/main/java/com/box/l10n/mojito/entity/RepositoryLocaleAIPrompt.java Outdated Show resolved Hide resolved

garionpin reviewed

View reviewed changes

webapp/src/main/java/com/box/l10n/mojito/entity/RepositoryLocaleAIPrompt.java Outdated Show resolved Hide resolved

garionpin reviewed

View reviewed changes

webapp/src/main/java/com/box/l10n/mojito/service/ai/RepositoryLocaleAIPromptRepository.java Outdated Show resolved Hide resolved

garionpin reviewed

View reviewed changes

webapp/src/main/java/com/box/l10n/mojito/service/ai/openai/OpenAILLMService.java Outdated Show resolved Hide resolved

garionpin reviewed

View reviewed changes

webapp/src/main/java/com/box/l10n/mojito/service/ai/openai/OpenAILLMService.java Outdated Show resolved Hide resolved

garionpin reviewed

View reviewed changes

webapp/src/main/java/com/box/l10n/mojito/service/ai/openai/OpenAILLMService.java Outdated Show resolved Hide resolved

garionpin reviewed

View reviewed changes

webapp/src/main/java/com/box/l10n/mojito/service/ai/openai/OpenAILLMService.java Show resolved Hide resolved

garionpin reviewed

View reviewed changes

webapp/src/main/java/com/box/l10n/mojito/service/ai/openai/OpenAILLMService.java Show resolved Hide resolved

garionpin reviewed

View reviewed changes

webapp/src/main/java/com/box/l10n/mojito/service/ai/openai/OpenAILLMService.java Outdated Show resolved Hide resolved

garionpin reviewed

View reviewed changes

webapp/src/main/java/com/box/l10n/mojito/service/ai/translation/AITranslateJob.java Outdated Show resolved Hide resolved

garionpin reviewed

View reviewed changes

webapp/src/main/java/com/box/l10n/mojito/service/ai/translation/AITranslateJob.java Outdated Show resolved Hide resolved

garionpin reviewed

View reviewed changes

webapp/src/main/java/com/box/l10n/mojito/service/ai/translation/AITranslateJob.java Outdated Show resolved Hide resolved

maallen added 28 commits

October 11, 2024 10:45


          Null Handling

e36a98e


          Fix return type for prompt query lookup

4f831f4


          Added implementation to extract translation from json key response

871972e


          Cleaned up default prompt handling, added unit tests

b9a458c


          Add metric to record time taken for text unit to be machine translated

13b30c8


          Parse language from BCP-47 tag correctly

38d9f57


          Remove unused join

23965d2


          No AI translation if current variant exists for locale

81f0885


          Code review updates

fbf07e0


          Update log and add metric

51f4c59


          Added AI translation handling for virtual text units

83850c0


          Updating code readability

ecd8407


          Tests passing

06b8164


          Only translate supplied locales, added support for optional placehold…

97c19a6

…er templating


          Further review updates

c30fe82


          Update AITranslateJob to a Quartz Cron job, other code review updates

c808638


          Add cron setup

297567a


          Added unit test for batch processing failure logic


          Update configuration to use ConfigProperties and added repository lev…

826d4bc

…el config for re-using source on lang match


          Added CLI support for isJsonResponse boolean for ai prompts

74e2472


          Added jsonResponseKey to ai prompt table and added findLocalesWithVar…

80fa37a

…iantByTmTextUnitId query


          Fix test failures

72bb8b2


          Update exception handling logic for batch processing

b85f856


          Added error logging

bcd00c5


          Added parellization, updated database inserts to be done in batches o…

96133fb

…f server side inserts


          Fix log condition

ad82965


          Remove dead code, fix docs

b79e28b


          Doc fix

3edc6e1

maallen force-pushed the mallen/gpt_for_translate branch from cd4485a to 3edc6e1 Compare

October 11, 2024 09:45

maallen merged commit a9eec85 into master

2 checks passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet