A Set on edgi
How ChatGPT Actually Works
ChatGPT predicts the next token. The surprising part is what training and feedback can build on top of that.
-
ChatGPT
ChatGPT is an artificial intelligence chatbot that generates text, speech, and images in response to user prompts.
-
Hallucination (artificial intelligence)
An AI hallucination is a false or misleading statement generated by an artificial intelligence model and presented as true fact.
-
Embedding (machine learning)
An embedding is a list of numbers that represents complex data like words or images so a computer can work with them mathematically.
-
Self-Supervised Learning
Self-supervised learning trains AI models by turning raw data into its own practice tests, bypassing the need for manual human labeling.
-
Attention (machine learning)
Attention is a method in machine learning that calculates how important every component in a sequence is relative to the others.
-
Transformer (machine learning)
Transformers process entire sequences at once using self-attention and positional encodings, replacing sequential neural networks.
-
Generative pre-trained transformer
Generative pre-trained transformers learn broad language patterns on unlabeled data by predicting the next token before adapting to tasks.
-
Neural Scaling Law
Neural scaling laws describe how model error reliably drops as parameters, training datasets, and computing power increase.