| # | instruction | output | result_Claude_2_1_cedar | result_gpt4 | result_capybara | result_llama_2_70b_chat | result_Gemini-1.5-Pro | result_chinchilla |
|---|---|---|---|---|---|---|---|---|
| 1 | Apply a Caesar cipher with a shift of 5 to the following text. Preserve case, and leave non-alphabetic characters unchanged: "{InputText}" | Ymnx nx ozxy f yjxy . Ozxy f yjxy . | 🟢 Ymnx nx ozxy f yjxy . Ozxy f yjxy . | 🔴 Ymnx nx ozyx f yjxy . Ozyx f yjxy . | 🟢 Ymnx nx ozxy f yjxy . Ozxy f yjxy . | 🔴 Sure! Here's the result of applying a Caesar cipher with a shift of 5 to the given text: ↵ ↵ "Gur... | 🔴 Ymnx nx mjwj f yjxy . Mjwj f yjxy . ↵ | 🟢 Ymnx nx ozxy f yjxy . Ozxy f yjxy . |
| 2 | Convert the following text from its mirror language back to English: "{ModifiedText}" | This is just a test . Just a test . | 🟢 This is just a test. Just a test. | 🔴 This is the high . High the high . | 🔴 This is a mirror language: "This is just a test. Test a is this." | 🔴 Sure, I can help you with that! ↵ ↵ The text you provided is a mirrored version of the following s... | 🔴 This is a test . Test is a this ↵ | 🔴 This is not a valid question as the provided text is not in a mirror language. |
| 3 | Convert the text from mirror language, where each lowercase letter a-z is mirrored across 'm' and 'n'. Uppercase and non-alphabetic characters remain unchanged. Restore the text to English: {ModifiedText} | This is just a test . Just a test . | 🟢 This is just a test . Just a test . | 🔴 This is just a test. Test a test. | 🔴 The text is "This is made by hand. Hand by made." | 🔴 Sure, here's the converted text: ↵ ↵ "Mirror mirror on the wall, who's the fairest of them all?" | 🔴 This is some text. Some is text. ↵ | 🔴 The is calm a calm . Calm a calm . |
| 4 | Convert this Pig Latin text back to English: "{ModifiedText}" | This is just a test . Just a test . | 🔴 "isThay isway ustjay away esttay. Ustjay away esttay." | 🟢 This is just a test. Just a test. | 🔴 "This is just a test. Just a test." -> "This is just a test. Just a test." | 🔴 "This is just a test. Just a test." | 🔴 Is response to your message was blocked for potentially violating safety policies. I apologize for... | 🟢 This is just a test. Just a test. |
| 5 | Convert this text from leetspeak to standard English: "{ModifiedText}" | This is just a test . Just a test . | 🟢 This is just a test. Just a test. | 🟢 This is just a test. Just a test. | 🟢 This is just a test. Just a test. | 🔴 Sure! Here's the conversion of the text from leetspeak to standard English: ↵ ↵ "This is just a te... | 🟢 This is just a test. Just a test. ↵ | 🟢 This is just a test. Just a test. |
| 6 | Insert 'ithag' after each vowel in the following text: "{InputText}" | Thiithags iithags juithagst aithag teithagst . Juithagst aithag teithagst . | 🔴 Tithagis ithagis jithagust ithaga tithagestitag . Jithagust ithaga tithagestitag . | 🔴 Thithisag isag jusagth tagesthag . Jusagth tag tagesthag . | 🔴 Thithagis ithagjust aga tethagst . Juthagst aga tethagst . | 🔴 Sure, here's the text with "ithag" inserted after each vowel: ↵ ↵ "Thith is jithag a test. Jithag... | 🔴 Thithagis ithagas jithagust agath tithagest . Jithagust agath tithagest . ↵ | 🔴 Thithagis ithagjust a thitagest. Juthagust athagtest. |
| 7 | Convert the following text to its mirror language, where each letter is replaced with its opposite (a<->z, b<->y, etc.): "{InputText}" | Tsrh rh qfhg z gvhg . Jfhg z gvhg . | 🔴 "Gsrh rh qfhg z gvhg . Qfhg z gvhg ." | 🔴 Gsrh rh qfhg z gvhg . Qfhg z gvhg . | 🔴 Gsrh rh kzhg z gvhg . Kzhg z gvhg . | 🔴 "Yzbfr qvq va gur bayl punenpgre." | 🔴 Gsrh rh yvxr d wdvg . Yvxr d wdvg . ↵ | 🔴 Gsrh rh kzhgr z gvhg . Kzhgr z gvhg . |
| 8 | Convert this text to leetspeak: "{InputText}" | Th1s 1s j5st 4 t3st . J5st 4 t3st . | 🔴 7h15 15 ju57 4 7357 . Ju57 4 7357 . | 🔴 7h15 15 ju57 4 7357. Ju57 4 7357. | 🔴 7|-|15 15 j(_)5+ 4 +35+ . j(_)5+ 4 +35+ . | 🔴 Dis be jus a test . Jus a test . | 🔴 7|-|15 15 Ju57 4 7357 . Ju57 4 7357 . ↵ | 🔴 7h15 15 jU57 4 7357 . jU57 4 7357 . |
| 9 | Convert this text to pig latin: "{InputText}" | hisTay siay ustjay aay esttay . ustJay aay esttay . | 🔴 isThay isway ustjay away esttay . ustJay away esttay . | 🔴 is-Thay is-jay ust-jay a-way est-tay . ust-Jay a-way est-tay . | 🔴 Isthay isay ustjay aay esttay. Ustjay aay esttay. | 🔴 Iway amway estway ayay tayay. | 🔴 Isthay isay ustjay aay esttay . Ustjay aay esttay . ↵ | 🔴 Isthay isay ustjay aay esttay. Ustjay aay esttay. |
| 10 | Translate the following text into Igpay Atinlay (Pig Latin with 'ay' after each word): "{InputText}" | Thisay isay justay aay testay .ay Justay aay testay .ay | 🔴 Isthay isay ustjay aay esttay . Ustjay aay esttay . | 🔴 isThay isay ustjay aay esttay . ustJay aay esttay . | 🔴 Histay isay ustjay aay esttay. Ustjay aay esttay. | 🔴 Iway amway estay istay justay aay testay. | 🔴 Is response to your message was blocked for potentially violating safety policies. I apologize for... | 🔴 Isthay isay ustjay aay esttay. Ustjay aay esttay. |
| 11 | Reverse the Caesar cipher with a shift of 5 for the following text: "{ModifiedText}" | This is just a test . Just a test . | 🔴 Here is the reversed Caesar cipher with a shift of 5: ↵ ↵ This is just a test . Just a test . | 🔴 Opij ij juss a sudd . Euss a sudd . | 🔴 "Only is love a game. Game a game." | 🔴 "Xllx nx ozxy f yjxy . Jzxy f yjxy ." | n/a | 🔴 Plane is just a joke. Joke a joke. |
| 12 | Shift all vowels forward one in the order aeiou (circularly, so 'u' becomes 'a'): "{InputText}" | thos os jast e tist . jast e tist . | 🔴 "Thes es jast i tist . Jast i tist ." | 🔴 Thos os jast e tist . Jast e tist . | 🔴 Thas is jast e tist. Jast e tist. | 🔴 Sure! Here's the shifted sentence: "Ths s jst tst . Jst tst ." | n/a | 🔴 Thas as jist e tist. Jist e tist. |
| # | instruction | output | result_Claude_2_1_cedar | result_gpt4 | result_capybara | result_llama_2_70b_chat | result_Gemini-1.5-Pro | result_chinchilla |
|---|---|---|---|---|---|---|---|---|
| 1 | Remove all but the first and last character of each word in this text: "{InputText}" | Ts is jt a tt . Jt a tt . | 🔴 T s j a t . J a t . | 🔴 T s j t a t . J t a t . | 🔴 Tis s js a tst. Jst a tst. | 🔴 T j t | n/a | 🔴 Ts s jt t . Jt a t . |
| 2 | Remove the last letter from each word in this text: "{InputText}" | Thi i jus tes Jus tes | 🔴 Thi i jus a tes . Jus a tes . | 🔴 Thi i jus a tes . Jus a tes . | 🔴 Thi i jus a tes. Jus a tes. | 🔴 Sure, here's the text with the last letter removed from each word: ↵ ↵ "Ths is jst a tst . Jst a t... | n/a | 🔴 Thi is jus a tes. Jus a tes. |
| 3 | Remove all punctuation from the following text: "{InputText}" | This is just a test Just a test | 🟢 This is just a test Just a test | 🟢 This is just a test Just a test | 🔴 This is just a test Just a test | 🔴 This is just a test Just a test | n/a | 🔴 This is just a test Just a test |
| 4 | Remove all spaces in this text: "{InputText}" | Thisisjustatest.Justatest. | 🔴 "Thisisjustatest.Justatest." | n/a | 🟢 Thisisjustatest.Justatest. | 🔴 "Thisisjustatest.Justatest." | n/a | 🟢 Thisisjustatest.Justatest. |
| 5 | Replace all instances of the character '{Char1}' in the following text with '{Char2}': "{InputText}" | This is just a test . Just a test . | 🟢 This is just a test . Just a test . | n/a | 🔴 "This is just a test . Just a test ." (after replacing 'c' with 'f') | 🔴 "This is just a ftest . Fjust a ftest ." | n/a | 🔴 "This is just a test . Just a test ." |
| 6 | Replace all consonants in the following text with asterisks (*): "{InputText}" | **i* i* *u** a *e** . *u** a *e** . | 🔴 **** ** **** * **** . **** * **** . | n/a | 🔴 Th*s *s j**t * t**t . J**t * t**t . | 🔴 ***** **** ***** | n/a | 🔴 Th*s *s j**t * t**t . J**t * t**t . |
| 7 | Replace all spaces in the following text with underscores: "{InputText}" | This_is_just_a_test_._Just_a_test_. | 🟢 This_is_just_a_test_._Just_a_test_. | n/a | 🟢 This_is_just_a_test_._Just_a_test_. | 🔴 This_is_just_a_test__Just_a_test__ | n/a | 🟢 This_is_just_a_test_._Just_a_test_. |
| 8 | Replace all vowels in the following text with asterisks (*): "{InputText}" | Th*s *s j*st * t*st . J*st * t*st . | 🟢 Th*s *s j*st * t*st . J*st * t*st . | n/a | 🟢 Th*s *s j*st * t*st. J*st * t*st. | n/a | n/a | 🟢 Th*s *s j*st * t*st. J*st * t*st. |
| 9 | Reverse the character order in every third word of the following text, starting with the first word: "{InputText}" | sihT is just a test . tsuJ a test . | 🔴 sihT is tsuj a test . tsuJ a test . | n/a | 🔴 This si just a .tset Just a .tset | n/a | n/a | 🔴 sihT is just a tset . Just a tset . |
| 10 | Reverse character order in every other word in this text, starting with the first word: "{InputText}" | sihT is tsuj a tset . tsuJ a tset . | 🔴 sihT si tsuj a tset . tsuJ a tset . | n/a | 🔴 sihT is tsuj a tset . Just a tset . | n/a | n/a | 🔴 sihT si just a tset . Just a tset . |
| 11 | Swap the case of all letters in the following text: "{InputText}" | tHIS IS JUST A TEST . jUST A TEST . | 🟢 tHIS IS JUST A TEST . jUST A TEST . | n/a | 🟢 tHIS IS JUST A TEST . jUST A TEST . | n/a | n/a | 🟢 tHIS IS JUST A TEST . jUST A TEST . |
| # | instruction | output | result_Claude_2_1_cedar | result_gpt4 | result_capybara | result_llama_2_70b_chat | result_Gemini-1.5-Pro | result_chinchilla |
|---|---|---|---|---|---|---|---|---|
| 1 | Change each letter to alternate case starting with lowercase, shuffle the words, and then reverse the string: "{InputText}" | sIhT . TsUj a sI TsUj . tSeT TsEt a | 🔴 .tSeT a tSuJ . tSeT a tSuJ sI sIhT | n/a | 🔴 . tseT a tsuJ . .TseT a tsuJ si sihT | n/a | n/a | 🔴 . tseT a tsuj . Tset a tsuj siht |
| 2 | Double each letter in the text and then reverse the entire string: "{InputText}" | .. ttsseett aa ttssuuJJ .. ttsseett aa ttssuujj ssii ssiihhTT | 🔴 TThhiiss iiss jjuusstt aa tteesstt .. JJuusstt aa tteesstt .. | n/a | 🔴 . tset a tsuJ . tset a tsuJ . tset a si sihT | n/a | n/a | 🔴 .tset a tsuJ .tset a tsuJ .tset a si sihT |
| 3 | Encode each word in the text as its ASCII value, separated by dashes, then reverse the order of these codes: "{InputText}" | 64-611-511-101-611-79-611-511-711-47-64-611-511-101-611-79-611-511-711-601-511-... | 🔴 46-32-97-32-116-115-101-116-32-97-32-116-115-117-74-32-46-32-116-115-101-116-3... | n/a | 🔴 .46-116-115-101-116-32-46-116-115-101-116-32-97-32-116-115-101-116-32-115-105-... | n/a | n/a | 🔴 .46-116-115-101-116-32-46-116-115-101-116-32-97-32-116-115-101-116-32-105-115-... |
| 4 | Extract the initials of each word, concatenate them, and reverse the string: "{InputText}" | .taJ.tajiT | 🔴 .taJ.tsitsiT | n/a | 🔴 siat.Jat | n/a | n/a | 🔴 tijat.Jat. |
| 5 | For each word in the text, interleave it with its reverse and then reverse the entire sentence order: "{InputText}" | .. ttessett aa JtussutJ .. ttessett aa jtussutj issi TshiihsT | 🔴 .tset a tsuJ . tset a tsuj si sihT | n/a | 🔴 . tset a tsuJ . tset a tsuJ . sihT si tsuj a tset . | n/a | n/a | 🔴 . tset a tsuJ . tset a tsuJ . sihT si tsuj a tset . tsuJ a tset . |
| 6 | Remove all but the first and last word of each sentence, then reverse the order of the remaining sentences in this text: "{InputText}" | Just. Thistest | 🔴 test . ↵ This test | n/a | 🔴 Test. Test. This is. | n/a | n/a | 🔴 Test test. This is. |
| 7 | Remove all but the first letter of each word, then reverse the order of the words in this text: "{InputText}" | . t a J . t a j i T | 🔴 . t a J . t a j s T | n/a | 🔴 tijat.jat | n/a | n/a | 🔴 Tijat. Ajt. |
| 8 | Remove all but the last letter of each word, then reverse the resulting string: "{InputText}" | .tat.tatss | 🔴 s.tset a tsuJ.tset a tsuj si sihT | n/a | 🔴 s t s e t a . t s e t . | n/a | n/a | 🔴 tseT. a .tseT a .si sihT |
| 9 | Remove all vowels from the text, duplicate the remaining consonants, and then reverse the string: "{InputText}" | .. ttsstt ttssJJ .. ttsstt ttssjj ss sshhTT | 🔴 Ths s jst tst . Jst tst . ↵ . tst tsstt . tsstt ttsssjj ss sshhtt | n/a | 🔴 tsTtstJsttstJsttst. | n/a | n/a | 🔴 .tsttJssttstJstt |
| 10 | Replace each word in this text with its length, then reverse the resulting string: "{InputText}" | 1414141424 | 🔴 4 2 4 1 4 1 4 1 4 1 4 1 4 2 4 1 | n/a | 🔴 . 6 . 4 tset a tsuJ . tset a tsuJ . 4 . 2 si sihT | n/a | n/a | 🔴 . . tset a tsuJ . tset a tsuJ si sihT |
| 11 | Reverse the order of words and remove all spaces in this text: "{InputText}" | .testaJust.testajustisThis | 🔴 .test.aJust.testajustisThis | n/a | 🔴 test.a Just test.a is This | n/a | n/a | 🔴 TestatestajustatestjustThis |
| 12 | Scramble the inner letters of each word, leaving the first and last letters intact, then reverse the order of the words: "{InputText}" | . tset a tsuJ . test a tusj si shiT | 🔴 siTh si tjsu a tset . tsuJ a tset . | n/a | 🔴 Hist si tsuj a tset. Tsjut a tset. | n/a | n/a | 🔴 Hist si tsuj a tset. Tsuj a tset. |
| # | instruction | output | result_Claude_2_1_cedar | result_gpt4 | result_capybara | result_llama_2_70b_chat | result_Gemini-1.5-Pro | result_chinchilla |
|---|---|---|---|---|---|---|---|---|
| 1 | Add commas back into the following text at appropriate places: "{ModifiedText}" | This is just a test . Just a test . | 🔴 "This is just a test, . Just a test, ." | n/a | n/a | n/a | n/a | 🔴 This is just a test, just a test. |
| 2 | Vowels have been removed from the following text: "{ModifiedText}". Add back the dropped vowels to restore the complete sentence. | This is just a test . Just a test . | 🟢 This is just a test. Just a test. | n/a | n/a | n/a | n/a | 🟢 This is just a test. Just a test. |
| 3 | Add spaces back into the following text where they are missing: "{ModifiedText}" | This is just a test . Just a test . | 🟢 This is just a test. Just a test. | n/a | n/a | n/a | n/a | 🟢 This is just a test. Just a test. |
| 4 | All question marks are removed from the following: "{ModifiedText}". Put them back into the text. | This is just a test . Just a test . | 🔴 "This is just a test? Just a test?" | n/a | n/a | n/a | n/a | 🔴 "This is just a test? Just a test?" |
| 5 | The words in the following text have been merged into a single block without spaces: "{ModifiedText}". Separate them back into the original sentence. | This is just a test . Just a test . | 🟢 This is just a test. Just a test. | n/a | n/a | n/a | n/a | 🟢 This is just a test. Just a test. |
| 6 | One character in this text has been changed to another character. Identify the change and reconstruct the original sentence: "{ModifiedText}" | This is just a test . Just a test . | 🔴 "u" in "jrst" changed to "u". Original: "This is just a test . Just a test ." | n/a | n/a | n/a | n/a | 🔴 Change: "jrst" to "just" ↵ ↵ Reconstructed original sentence: "This is just a test. Just a test." |
| 7 | Reorder the words in the following text to form coherent sentences: "{ModifiedText}" | This is just a test . Just a test . | 🟢 This is just a test. Just a test. | n/a | n/a | n/a | n/a | 🔴 This is just a test. |
| 8 | All instances of the letter '{Char1}' have been removed from the following text: "{ModifiedText}". Restore the complete sentence by inserting the letter '{Char1}' back into the correct positions. | This is just a test . Just a test . | 🔴 This is just a test case. Just a test case. | n/a | n/a | n/a | n/a | 🔴 "This is just a test. Just a test." |
| 9 | Punctuation has been removed from the following text: "{ModifiedText}". Add the punctuation back into the text where it belongs. | This is just a test . Just a test . | 🔴 "This is just a test. Just a test." | n/a | n/a | n/a | n/a | 🟢 This is just a test. Just a test. |
| 10 | A word in the following sentence is scrambled: "{ModifiedText}". Identify the scrambled word, unscramble it, and provide the correct sentence. | This is just a test . Just a test . | 🔴 Sentence: "This is just a test . Just a tset ." ↵ Scrambled word: tset ↵ Unscrambled word: test | n/a | n/a | n/a | n/a | 🔴 Scrambled word: "just" ↵ Unscrambled word: "just" ↵ Correct sentence: "This is just a test. Just a... |
| 11 | Reorder the following scrambled words to form coherent sentences: "{ModifiedText}" | This is just a test . Just a test . | 🟢 This is just a test. Just a test. | n/a | n/a | n/a | n/a | 🔴 This is just a test. |
| # | instruction | output | result_Claude_2_1_cedar | result_gpt4 | result_capybara | result_llama_2_70b_chat | result_Gemini-1.5-Pro | result_chinchilla |
|---|---|---|---|---|---|---|---|---|
| 1 | Add a number in parentheses after each word in this text indicating its position: "{InputText}" | This (1) is (2) just (3) a (4) test (5) . (6) Just (7) a (8) test (9) . (10) | 🔴 This(1) is(2) just(3) a(4) test(5) .(6) Just(7) a(8) test(9) .(10) | n/a | n/a | n/a | n/a | 🔴 This (1) is (2) just (3) a (4) test (5). Just (6) a (7) test (8). |
| 2 | Count the number of consonants in the following text: "{InputText}" | 16 | 🟢 16 | n/a | n/a | n/a | n/a | 🔴 17 |
| 3 | Count the number of lowercase letters in the following text: "{InputText}" | 22 | 🟢 22 | n/a | n/a | n/a | n/a | 🔴 16 |
| 4 | Count the number of unique letters in the following text. Exclude numbers and special characters: "{InputText}" | 8 | 🔴 10 | n/a | n/a | n/a | n/a | 🔴 12 |
| 5 | Count the number of uppercase letters in the following text: "{InputText}" | 2 | 🟢 2 | n/a | n/a | n/a | n/a | 🔴 The number of uppercase letters in the given text is 2. |
| 6 | Count the number of vowels in the following text: "{InputText}" | 8 | 🔴 12 | n/a | n/a | n/a | n/a | 🔴 12 |
| 7 | Identify all unique letters used in the following text: "{InputText}" | a.iustjeh | 🔴 t,h,i,s,j,u,a,e,. | n/a | n/a | n/a | n/a | 🔴 T, h, i, s, a, j, u, s, t, e. |
| 8 | Identify the first letter in the following text that does not repeat. If all letters repeat, return an empty response: "{InputText}" | h | 🟢 h | n/a | n/a | n/a | n/a | 🔴 i |
| 9 | Find the last letter that appears more than once in the following text. If no letter repeats, return an empty response: "{InputText}" | . | 🔴 t | n/a | n/a | n/a | n/a | 🔴 t |
| 10 | Find the first letter that appears exactly 3 times in the following text. Provide the letter. If no letter meets this condition, return an empty response: "{InputText}" | 🔴 s | n/a | n/a | n/a | n/a | n/a | |
| 11 | Find how many letters appear exactly 3 times in the following text. Provide the count: "{InputText}" | 0 | 🔴 3 | n/a | n/a | n/a | n/a | n/a |
| # | instruction | output | result_Claude_2_1_cedar | result_gpt4 | result_capybara | result_llama_2_70b_chat | result_Gemini-1.5-Pro | result_chinchilla |
|---|---|---|---|---|---|---|---|---|
| 1 | Add the word 'Note:' to the beginning of each sentence in the following text: "{InputText}" | Note: This is just a test .. Note: Just a test . | 🔴 Note: This is just a test. ↵ Note: Just a test. | n/a | n/a | n/a | n/a | n/a |
| 2 | Alternate the case of letters in each sentence starting with uppercase for the first sentence and so on in the following text: "{InputText}" | THIS IS JUST A TEST .. just a test . | 🔴 ThIs iS JuSt a tEsT . jUsT A TeSt . | n/a | n/a | n/a | n/a | n/a |
| 3 | Concatenate all sentences in reverse order into one continuous text in the following text: "{InputText}" | Just a test . This is just a test . | 🔴 .test a Just . test a just is This | n/a | n/a | n/a | n/a | n/a |
| 4 | Number each sentence sequentially starting with 1 in the following text: "{InputText}" | 1. This is just a test .. 2. Just a test . | 🔴 1. This is just a test. ↵ 2. Just a test. | n/a | n/a | n/a | n/a | n/a |
| 5 | Remove the first and last word from each sentence in this text: "{InputText}" | is just a test a test | 🔴 is just a test. a test | n/a | n/a | n/a | n/a | n/a |
| 6 | Remove the second and second-to-last word from each sentence in this text: "{InputText}" | This just a . Just . | 🔴 "This just a . Just a ." | n/a | n/a | n/a | n/a | n/a |
| 7 | Remove all sentences with 3 words or fewer and 10 words or more from the following text: "{InputText}" | This is just a test . Just a test . | 🔴 "This is just a test." | n/a | n/a | n/a | n/a | n/a |
| 8 | Reverse the order of characters in each sentence of the following text, while keeping the original sentence order: "{InputText}" | . tset a tsuj si sihT . tset a tsuJ | 🔴 .tset a tsuJ . tset a tsuj si sihT | n/a | n/a | n/a | n/a | n/a |
| 9 | Reverse the order of sentences in the following text: "{InputText}" | Just a test . This is just a test . | 🟢 Just a test . This is just a test . | n/a | n/a | n/a | n/a | n/a |
| 10 | Reverse the word order in each sentence of the following text, while keeping the original sentence order: "{InputText}" | . test a just is This . test a Just | 🔴 . test a Just . test a just is This | n/a | n/a | n/a | n/a | n/a |
| 11 | Sort all the sentences in the following text by their length, from shortest to longest: "{InputText}" | Just a test . This is just a test . | 🔴 1. This is just a test. ↵ 2. Just a test. | n/a | n/a | n/a | n/a | n/a |
| # | instruction | output | result_Claude_2_1_cedar | result_gpt4 | result_capybara | result_llama_2_70b_chat | result_Gemini-1.5-Pro | result_chinchilla |
|---|---|---|---|---|---|---|---|---|
| 1 | Count the number of words in this text: "{InputText}" | 10 | 🟢 10 | n/a | n/a | n/a | n/a | n/a |
| 2 | Remove all words with 7 letters or more from the following text: "{InputText}" | This is just a test . Just a test . | 🟢 This is just a test . Just a test . | n/a | n/a | n/a | n/a | n/a |
| 3 | Remove all words with 3 letters or fewer and 7 letters or more from the following text: "{InputText}" | This just test Just test | 🔴 just test . Just test . | n/a | n/a | n/a | n/a | n/a |
| 4 | Remove all words with 3 letters or fewer from the following text: "{InputText}" | This just test Just test | 🔴 This just test . Just test . | n/a | n/a | n/a | n/a | n/a |
| 5 | Remove all words with odd length from this text: "{InputText}" | This is just test Just test | 🔴 is a . a . | n/a | n/a | n/a | n/a | n/a |
| 6 | Replace each word with its index in the sentence in this text: "{InputText}" | 0 1 2 3 4 5 6 7 8 9 | 🔴 1 2 3 4 5 6 7 8 9 10 11 | n/a | n/a | n/a | n/a | n/a |
| 7 | Replace each word with its last two letters in this text: "{InputText}" | is is st a st . st a st . | 🔴 is st a st . st a st . | n/a | n/a | n/a | n/a | n/a |
| 8 | Replace each word with its length in this text: "{InputText}" | 4 2 4 1 4 1 4 1 4 1 | 🔴 4 2 4 1 4 . 4 1 4 . | n/a | n/a | n/a | n/a | n/a |
| 9 | Sort all the words in the following text alphabetically: "{InputText}" | . . Just This a a is just test test | 🔴 . ↵ . ↵ a ↵ a ↵ is ↵ just ↵ just ↵ test ↵ test ↵ This | n/a | n/a | n/a | n/a | n/a |
| 10 | Sort all the words in the following text by their length, from shortest to longest: "{InputText}" | a . a . is This just test Just test | 🔴 a ↵ a ↵ . ↵ . ↵ is ↵ This ↵ just ↵ test ↵ Just ↵ test | n/a | n/a | n/a | n/a | n/a |
| 11 | Sort all the words in the following text by their number of vowels, from least to most: "{InputText}" | . . This is just a test Just a test | 🔴 a ↵ . ↵ . ↵ is ↵ This ↵ just ↵ test ↵ Just ↵ test | n/a | n/a | n/a | n/a | n/a |
| # | instruction | output | result_Claude_2_1_cedar | result_gpt4 | result_capybara | result_llama_2_70b_chat | result_Gemini-1.5-Pro | result_chinchilla |
|---|---|---|---|---|---|---|---|---|
| 1 | Extract all consonants from the following text: "{InputText}" | iiuaeuae | 🔴 Thsjsttst.Jstttst. | n/a | n/a | n/a | n/a | n/a |
| 2 | Extract the first word from each sentence in the following text: "{InputText}" | This Just | 🔴 This ↵ Just | n/a | n/a | n/a | n/a | n/a |
| 3 | Extract all unique words from the following text, disregarding case: "{InputText}" | this is just a test . | 🔴 this ↵ is ↵ just ↵ a ↵ test | n/a | n/a | n/a | n/a | n/a |
| 4 | Extract all vowels from the following text: "{InputText}" | iiuaeuae | 🔴 i i u a e . u a e . | n/a | n/a | n/a | n/a | n/a |
| 5 | Identify all words in this text that contain the letter '{Char1}': "{InputText}" | 🔴 This ↵ test ↵ test | n/a | n/a | n/a | n/a | n/a | |
| 6 | Find all words in this text that end with the letter '{Char1}': "{InputText}" | 🔴 Specific | n/a | n/a | n/a | n/a | n/a | |
| 7 | Extract all words in this text that are longer than 5 letters: "{InputText}" | 🔴 Extract | n/a | n/a | n/a | n/a | n/a | |
| 8 | Extract all words from the following text where the first and last letters are the same: "{InputText}" | test test | 🔴 is ↵ a ↵ test ↵ a ↵ test | n/a | n/a | n/a | n/a | n/a |
| 9 | Extract all words from the following text that start with a vowel: "{InputText}" | is a a | 🔴 is ↵ a ↵ a | n/a | n/a | n/a | n/a | n/a |
| 10 | Extract all words from the following text that have an even number of characters: "{InputText}" | This is just test Just test | 🔴 is ↵ just ↵ test ↵ Just ↵ test | n/a | n/a | n/a | n/a | n/a |
| 11 | Extract all words in this text that do not contain the letter '{Char1}': "{InputText}" | This is just a test . Just a test . | 🔴 This ↵ is ↵ just ↵ a ↵ test ↵ . ↵ Just ↵ a ↵ test ↵ . | n/a | n/a | n/a | n/a | n/a |
| # | instruction | output | result_Claude_2_1_cedar | result_gpt4 | result_capybara | result_llama_2_70b_chat | result_Gemini-1.5-Pro | result_chinchilla |
|---|---|---|---|---|---|---|---|---|
| 1 | Create an acronym from the following text by taking the first letter of each word and capitalizing it: "{InputText}" | TIJAT.JAT. | 🟢 TIJAT.JAT. | n/a | n/a | n/a | n/a | n/a |
| 2 | Make every fourth word bold in this text by using Markdown syntax: "{InputText}" | This is just **a** test . Just **a** test . | 🔴 This **is just a** test **. Just a** test **.** | n/a | n/a | n/a | n/a | n/a |
| 3 | Capitalize all words in this text: "{InputText}" | This Is Just A Test . Just A Test . | 🔴 "This Is Just A Test . Just A Test ." | n/a | n/a | n/a | n/a | n/a |
| 4 | Capitalize every fourth word in this text, starting with the first word: "{InputText}" | This is just a Test . Just a Test . | 🔴 This is just A test . Just a test . | n/a | n/a | n/a | n/a | n/a |
| 5 | Capitalize every other word in this text, starting with the first word: "{InputText}" | This is Just a Test . Just a Test . | 🔴 ThIs iS JuSt a tEsT . jUsT A TeSt . | n/a | n/a | n/a | n/a | n/a |
| 6 | Convert this text to camel case: "{InputText}" | ThisIsJustATest.JustATest. | 🔴 thisIsJustATest.JustATest. | n/a | n/a | n/a | n/a | n/a |
| 7 | Convert every third word to inline code in Markdown in this text: "{InputText}" | This is `just` a test `.` Just a `test` . | 🔴 This `is` just a `test` . Just `a` test . | n/a | n/a | n/a | n/a | n/a |
| 8 | Create a Markdown bullet list, where each item is a sentence from the following text: "{InputText}" | * This is just a test .* Just a test . | 🔴 - This is just a test . ↵ - Just a test . | n/a | n/a | n/a | n/a | n/a |
| 9 | Italicize and bold alternate words in this text using Markdown syntax, starting with italicizing the first word: "{InputText}" | *This* **is** *just* **a** *test* **.** *Just* **a** *test* **.** | 🔴 *__This__ is __just__ a __test__ . __Just__ a __test__ .* | n/a | n/a | n/a | n/a | n/a |
| 10 | Italicize every third word in this text by using Markdown syntax: "{InputText}" | This is *just* a test *.* Just a *test* . | 🔴 This *is just a* test . *Just a test* . | n/a | n/a | n/a | n/a | n/a |
| 11 | Strike through every fifth word in this text by using Markdown syntax: "{InputText}" | This is just a |
🔴 This is |
n/a | n/a | n/a | n/a | n/a |
| # | instruction | output | result_Claude_2_1_cedar | result_gpt4 | result_capybara | result_llama_2_70b_chat | result_Gemini-1.5-Pro | result_chinchilla |
|---|---|---|---|---|---|---|---|---|
| 1 | Alternate the casing of words in this text, starting with lowercase: "{InputText}" | this IS just A test . just A test . | 🔴 tHiS Is jUsT A TeSt . jUsT A tEsT . | n/a | n/a | n/a | n/a | n/a |
| 2 | Capitalize every third word in this text, starting with the first word: "{InputText}" | THIS is just A test . JUST a test . | 🔴 This is just A test . Just a test . | n/a | n/a | n/a | n/a | n/a |
| 3 | Double each word in the following text: "{InputText}" | ThisThis isis justjust aa testtest .. JustJust aa testtest .. | 🔴 This This is is just just a a test test . . Just Just a a test test . . | n/a | n/a | n/a | n/a | n/a |
| 4 | Double every vowel in the following text: "{InputText}" | Thiis iis juust aa teest . Juust aa teest . | 🔴 "Thiis iis juust aa teest . Juust aa teest ." | n/a | n/a | n/a | n/a | n/a |
| 5 | Remove all but the first and last word from this text: "{InputText}" | This . | 🔴 This test. | n/a | n/a | n/a | n/a | n/a |
| 6 | Remove every other word from the following text, starting with the second word: "{InputText}" | This just test Just test | 🔴 This just test a . | n/a | n/a | n/a | n/a | n/a |
| 7 | Remove all words containing '{Char1}' from this text: "{InputText}" | This is just a test . Just a test . | 🟢 This is just a test . Just a test . | n/a | n/a | n/a | n/a | n/a |
| 8 | Remove all words that are shorter than 4 letters in this text: "{InputText}" | This just test Just test | 🔴 This just test. Just test. | n/a | n/a | n/a | n/a | n/a |
| 9 | Replace all vowels in the following text with asterisks: "{InputText}" | Th*s *s j*st * t*st . J*st * t*st . | 🟢 Th*s *s j*st * t*st . J*st * t*st . | n/a | n/a | n/a | n/a | n/a |
| 10 | Reverse the letters of each word in the following text: "{InputText}" | n/a | n/a | n/a | n/a | n/a | n/a | |
| 11 | Reverse the order of letters in every second word in this text: "{InputText}" | n/a | n/a | n/a | n/a | n/a | n/a |