2. ChatGPT Technology: From Language Models to Reinforcement Learning
Machine learning, a subfield of artificial intelligence (AI), is defined as the ability of computer systems
to learn from experience without being explicitly taught or programmed. With advances in computing
power, increasing data availability, and algorithmic improvements, deep learning has emerged as a high-
performing predictive tool. Unlike traditional machine learning, deep learning algorithms use artificial
neural networks, often in a large capacity. Deep learning models generally require more training data but
can provide better generalization. Transformers or large language models are a special family of deep
learning models that have emerged as state-of-the-art solutions for text data. From text classification to text
generation, these models offer superior performance than other AI models. Language models are usually
pre-trained on large language corpus, such as Wikipedia, but can be fine-tuned for a specific task. The two
most prominent language models include Bidirectional Encoder Representations from Transformers
(BERT) [11] and Generative Pretrained Transformer (GPT) [12]. The former is primarily used for
recognizing the context in language and classification tasks, while the latter is more adept at language
generation tasks such as machine translation and question-answering.
Several chatbots prior to ChatGPT used language models. However, the main component that allowed
ChatGPT its coherent and engaging responses is attributed to the use of reinforcement learning. In this
context, human evaluators ranked the responses of ChatGPT, which allowed it to optimize some of its
parameter to essentially become a better conversationalist. ChatGPT was developed on top of GPT-3, a
generative autoregressive language model with 175 billion parameters [13]. To train such a complex
network, OpenAI partnered with Microsoft with a reported training resource of over 350GB of memory
costing $12 million
.
Question: Hi ChatGPT, how were you were developed?
ChatGPT: Hello! I was developed by OpenAI, a research organization focused on advancing artificial
intelligence in a safe and beneficial way. Specifically, I was developed by a team of researchers and
engineers at OpenAI who used a deep learning technique called a transformer to train me on a large corpus
of text data. The transformer is a type of neural network that is especially good at processing sequential
data, like language.
To train me, the team fed me huge amounts of text from a variety of sources, including books, articles,
and websites. They then used a process called unsupervised learning to teach me to recognize patterns in
the text data. Through this process, I learned to understand and generate natural language.
After my initial development, I went through a rigorous testing and refinement process to ensure that I
was accurate, reliable, and safe to use. And since my release, I have been continually updated and improved
to better serve my users.
https://venturebeat.com/ai/ai-machine-learning-openai-gpt-3-size-isnt-everything/