Skip to content

Self-built transformer architecture, mainly from papers I've studied.

Notifications You must be signed in to change notification settings

NicolaiHerforth/nherformer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Nherformer (NH from my name, and it's a transformer, duh)

A transformer model inspired by the GPT approach and the "Attention is all you need" paper on transformers. This model is exclusively trained on Shakespeare's works for research and learning purposes.

Attention Head Implementation

The Attention Head is implemented as a separate class for better code clarity, though this is not typically considered best practice.

Outputs

Very First 100 Token Output

V:

Whowinin ses, o emome, creve buey hapusthinto y,
HitoOUTIUKELoncoudalt bse t! IUKIUSemn,
LEN wis, ig witse'ngr,
OUEdel nthas
Antheilo w quler andwily INEE VI isimay s ar. amarel.
Bejs o wicht ' s siesces ound bar, t,



PFire m g de brngo orkes;
Warstarses t d no'scelasent theeBo, w be WA:
To tloucoproy hindsourimeafothe, ksthin;
ICor psstoubis, k hirdanomathenell ctr
Fr.
The t sth gatowe nd, oulosuce,?
Difewis fo y thamyoowixt LO:
DE:
Angse had ase,
HESS get spr!

Second output at 0.2M parameters

&NUS:
How whe'e, it: in there comper,
Modle' bunded ciusise, thee likes: the entoce but may,
Tervancost of so proth gning,
Nevery man with offer see there desedy
the corforneded pilacaly rerperdnous adain the pone your with berorts.

DUKE XENCUS:
Sing agood puarr'd Orewicel:
In stimesper, which there lady;
And I'll that to coning!
Becounders, for the Ses advies:
Is!
I coudd in chonour is abour.
mostere wear is Godamquering Romounted your getives under sotter?

QUEEN:
UlrisK, raing four fouthbeer

Third Output at 1.6M parameters

NICOLAI:
Confessurn, if but pits young of hencely,
We are the gives love,
In Bulindiers, most singly, sweeth enjoy; and before next is all th.

GREMIO:
My fie! madam!
Are Earl--

Appaid:
And ear tell it: said, come wouin strike
hide beinks Keeps Lords of honourable God rigood, and not unwanted teralives, never dore of note better vow, their necks away of we lord, a moss well gate
To till that,'Lay his ruly,
A hand
Toak that I will know your lifted
To chivo our vices and find
But in my words such not comp

Fourth Output at 14M Parameters

HBushy,
You may with me, that my gage, though he is an
immedent of the patron a barnellor,--by this nine land,--my
think of is talk: there's a
dying may be negling; which done affrights the waters drink.

PORIZEL:
Nod virtue!
Hear me whip your came
In broken out this blame and more wonded
Than shore short was spark'd of yonder yent;
That, man, if he be yester'd and undertake
To your actor; I am in dangerous,
Ere we have shook to hear the watch. Think no more than you was

And lastly fifth output at 52M parameters

NICOLAI:
Know, good mother, child?

CATESBY:
I have, my liege.

KING RICHARD III:
Off with his son George's head!

NORFOLK:
My lord, the enemy is past the marsh
After the battle let George Stanley die.

KING RICHARD III:
A thousand hearts are great within my bosom:
Advance our standards, set upon our foes
Our ancient word of courage, fair Saint George,
Inspire to mock itself a sad's power.

KING RICHARD II:
Where lies was, sir, in the king; and himself,
One whereon thou shalt thereby be accessaryar John,
Where it perceives it, how it went.

QUEEN MARGARET:
I told your majesty as much before:
This proveth Edward's daughter, I would be pleasing,
Doth cloud my joys with danger and my state depose
Did from my hopes at an unstainst request.

QUEEN ELIZABETH:
My Lord of Gloucester's death, my Lord of York
Is yet to become my hate:
Is this the king's Richard's brother's name?

KING RICHARD III:
A king! he's name, my king, and my name
Wherein of England's purse to keep him:
And if he remember to himself a duke
That cries 'Retire to be; and
The one of's throne where, cry 'gainst the other;
Thereof doth a king crowns mother,
So loving-shapen thus, be led since we have ta'en prepare.

Lord Mayor:
Do, good my lord, take shall be your ben venuto.

GLOUCESTER:
Good morrow, brother Richard, will you hence.

DUCHESS OF YORK:
I'll have some yet be patient till then.

QUEEN ELIZABETH:
 reasonsititchers have no more soul shall thee harm.

ARCHBISHOP OF YORK:
Good madam, be not angry with the king.

QUEEN ELIZABETH:
Pitchers have reasons strong my fortune.

DUCHESS OF YORK:
Why should calamity be full

About

Self-built transformer architecture, mainly from papers I've studied.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages