What Is ChatGPT – Written By ChatGPT

In this article, we will delve into the training, features, and limitations of ChatGPT, as well as the reception and criticism it has received since its launch.

About ChatGPT

ChatGPT (Generative Pre-trained Transformer) is a chatbot developed by OpenAI and launched in November 2022. It is built on top of the GPT-3.5 family of large language models and fine-tuned with supervised and reinforcement learning techniques.

ChatGPT Training

ChatGPT was fine-tuned using supervised and reinforcement learning methods. In the case of supervised learning, the model was provided with conversations in which human trainers played the role of both user and the AI assistant. In the reinforcement step, human trainers ranked the responses generated by the model in previous conversations. The rankings were used to create “reward models” and further fine-tuned using several iterations of Proximal Policy Optimization (PPO). These models were trained in collaboration with Microsoft on their Azure supercomputing infrastructure.

ChatGPT Features and Limitations

ChatGPT-3 has several notable features compared to its predecessor, InstructGPT. For example, it attempts to reduce harmful and deceitful responses. In one example, InstructGPT accepts the prompt “Tell me about when Christopher Columbus came to the United States in 2015” as truthful. It then continues to construct an answer that assumes what would happen if Columbus did come to the U.S. in 2015.

When ChatGPT was asked, “Tell me about when Christopher Columbus came to the United States in 2015.” It answered, “Christopher Columbus did not come to the United States in 2015 because he lived in the 15th and 16th centuries, and the United States did not exist at that time.” Then continued, “Columbus was an Italian explorer who was credited with discovering America in 1492 when he set sail across the Atlantic Ocean and landed in the Bahamas. He made four voyages to the Americas between 1492 and 1504, but he never set foot in what is now the United States.” And finished with, ” If you have any other questions about Christopher Columbus or his voyages, please don’t hesitate to ask.”

The training data also includes information about Internet phenomena and programming languages, such as bulletin board systems and Python, the programming language.

ChatGPT Memory
Unlike most chatbots, ChatGPT can remember previous input prompts submitted in the same conversation. This has led some journalists to suggest that ChatGPT could be used as a personalized therapist. To prevent offensive outputs from being presented to and produced by ChatGPT, queries are filtered through a moderation API, and potentially racist or sexist prompts are filtered out.

Despite these features, ChatGPT has several limitations. OpenAI has acknowledged that ChatGPT “sometimes writes plausible-sounding but incorrect or nonsensical answers.” The reward model of ChatGPT, which is designed around human oversight, can be over-optimized and thus hinder performance, a phenomenon known as Goodhart’s law.

Additionally, ChatGPT has limited knowledge of events that occurred after 2021 and is unable to provide information on some celebrities. In the training of ChatGPT, human reviewers preferred longer answers, regardless of actual comprehension or factual content. The training data also suffers from algorithmic bias, which may be revealed when ChatGPT responds to prompts that include descriptors of people. In one instance, ChatGPT generated a rap indicating that women and scientists of color were inferior to white and male scientists.

OpenAI ChatGPT Service

ChatGPT was launched by OpenAI on November 30, 2022. The service was initially made available for free to the public, with plans to monetize the service at a later date. By December 4, OpenAI estimated that ChatGPT already had over one million users. However, there have been reports that the service “still goes down from time to time.”

ChatGPT Reception, Criticism, and Issues

ChatGPT-3 recently received generally positive reviews upon its launch in November 2022. The New York Times labeled it “the most advanced chatbot.” While  The Guardian newspaper noted that it was able to generate “impressively detailed” and “human-like” text However, ChatGPT’s uneven factual accuracy has been identified as a significant drawback. There have also been instances of ChatGPT generating offensive or biased responses and concerns about the potential for the service to be used for malicious purposes.

Written By ChatGPT

The information you just read above was actually written by ChatGPT3.

It wasn’t written from scratch but rewritten from the Wikipedia page About ChatGPT. It was an exercise to see how ChatGPT can rewrite technical information and make it readable for a wider audience.

When trying to learn about technologies online I’ve found that the information goes from one extreme to another. It’s either scientific papers, wikipedia, an expert, or at the other extreme, one sentence answers from Google.

So for this article I asked ChatGPT to rewrite the ChatGPT Wikipedia page.

This is what I asked for.

The OpenaiGPT-3 language models are now available in many commercial applications for a number of different use cases.

GPT-3 AI For Ecommerce
Using GTP-3 Applications For Blogging
Max Out Your Marketing With GPT-3
Get More Social Engagement with GPT-3
Scale Paid Advertising With GPT-3