BETA RELEASE

Specifics about ChatGPT's Architecture

Secondary index Data Science Stack Exchange
Published: 2023-02-03 Accessed: 2026-05-02
Original source ↗ Archived copy ↗

Summary

A community discussion regarding the architectural specifications of ChatGPT, linking it to GPT-3.5 and GPT-3 parameters.

Key quotes

ChatGPT is a fine-tuned version of GPT-3.5 (text-davinci-002).
Number of layers: 96 Number of attention heads: 96 Dimensions of its hidden layers: 12288 Sequence length: 2048 Number of parameters: 175B

This Stack Exchange thread discusses the specific model architecture of the public version of ChatGPT. The top answer suggests it shares the architecture of GPT-3/3.5.