We are thrilled to introduce LLM360, an initiative to open source LLMs that fosters transparency, trust, and collaborative research. When releasing models under LLM360, we strive to make all the details of LLM accessible to everyone.
Nice! It feels like a direct answer to Karpathy comment on Mistral, where he said it is nice to call it “open weight” but not “open source” because we still don’t know the dataset and the training code. LLM360 seem to be fully open source by that definition and releases even the checkpoints!
Performance wise, a bit lagging (under a Llama2 of the same size) but all the tools are there to improve it!
Nice! It feels like a direct answer to Karpathy comment on Mistral, where he said it is nice to call it “open weight” but not “open source” because we still don’t know the dataset and the training code. LLM360 seem to be fully open source by that definition and releases even the checkpoints!
Performance wise, a bit lagging (under a Llama2 of the same size) but all the tools are there to improve it!