
Generative Pretrained Transformers
The project focus on Training a Generative Pretrained Transformers(GPT) aka foundation model from scratch using plain text. The foundation model than can be finetuned on a specific task like text generation, summarization, translation, and more. The project leverages the power of GPT-2 architecture, a state-of-the-art language model, to generate human-like text based on a given prompt.
Disclaimer This project is intended for learning purposes. Training a production-level LLM requires extensive computational power and data. Please conduct thorough research before implementing this in any production environment.
Learn more