Everything I understand about chatgpt
Summary
A synthesized collection of research and notes on ChatGPT's architecture, training data, RLHF process, business context, and potential use cases.
Key quotes
LLMs are generative mathematical models of the statistical distribution of tokens in the vast public corpus of human-generated text
ChatGPT is fine-tuned from a model in the GPT-3.5 series.
This document serves as a research aggregator for understanding the technical and organizational background of OpenAI’s ChatGPT. It includes details on the model’s autoregressive nature and its reliance on Azure infrastructure.