wargreymon@sh.itjust.works to Machine Learning@lemmy.ml · 13 days agoCan gpt generate a gpt model?message-squaremessage-square8fedilinkarrow-up18arrow-down13file-text
arrow-up15arrow-down1message-squareCan gpt generate a gpt model?wargreymon@sh.itjust.works to Machine Learning@lemmy.ml · 13 days agomessage-square8fedilinkfile-text
minus-squarekeepthepacelinkfedilinkarrow-up6·13 days agoNot directly no. It may be able to code one (the code is relatively short and well known) and give training program, and then you would need to spend a few trillion tokens to make it generate data.
minus-squareAwkwardLookMonkeyPuppet@lemmy.worldlinkfedilinkEnglisharrow-up1·13 days agoWhere can we see this well known code? I’d like to see how it works.
minus-squarekeepthepacelinkfedilinkarrow-up1·13 days agoHere is an implementation in pytorch: https://github.com/lyeoni/gpt-pytorch/blob/master/model.py Here is one in pure C that karpathy started: https://github.com/karpathy/llm.c
Not directly no.
It may be able to code one (the code is relatively short and well known) and give training program, and then you would need to spend a few trillion tokens to make it generate data.
Where can we see this well known code? I’d like to see how it works.
Here is an implementation in pytorch:
https://github.com/lyeoni/gpt-pytorch/blob/master/model.py
Here is one in pure C that karpathy started:
https://github.com/karpathy/llm.c
Thanks!