The ChatGPT API is so good and cheap that it eliminates the need for most text-generating AIs | Max Woolf's blog (2023)

everyone knewOpenAIwould release an API forChatGPTsometimes. The APIs for GPT-3 alone allow companies to exist such The real issue was the pricing of ChatGPT. For context, when GPT-3 came out of beta in 2021, it was priced at $0.06/1,000 tokens (a few paragraphs of text). A turning point occurred in August 2022, where OpenAI not onlyreduced the priceFor1/3($0.02/1,000 tokens: enough to run a business, but still too expensive for casual use), but text-davinci-003 was also introduced as a standard GPT-3 endpoint soon after: a tweaked GPT that canfollow the instructions veryThen. I suspected that OpenAI would charge twice as much for the ChatGPT API as the GPT-3 API, due to the typical hypeprice discriminationas everyone thinks ChatGPT is much better and doesn't want to overshadow their existing GPT 3 products.

Instead, on March 1, OpenAIset pricefor a ChatGPT API1/10a API GPT-3 a US$ 0,002/1.000 tokens.

Wait what?!

Heaven's Door: Rewriting ChatGPT's Internal Rules to Get Exactly What You Want

For context, theChatGPT-APIallows a developer to ask ChatGPT a question and get a response as they normally would with the ChatGPT web UI, but using a programming language such as Python, allowing these responses to be integrated into any application. But since there are a lot of arcane tweaks to make the template so cheap, we need to ensure that the ChatGPT API (which uses the aptly named gpt-3.5-turbo template endpoint) isstrictly speakingvery similar to what we are used to after months of using the web UI, otherwise this whole subject is pointless. Through my testing with the API, I can confirm that generating template variant text is indeed a real deal.

Unlike cute thoughts about howCHATGPT WILL CHANGE EVERYTHING!!!1!, I decided to create really useful tools with ChatGPT API first to get a better judgment about it and I haveOpen source these toolsso people can build on that and prove that I don't choose my experiences.

However, there is a new twist on the APInoAvailable in traditional web UI: ChatGPT API users canSystemto urge. Earlier in ChatGPT's lifespan, users were able to reverse engineer and now confirm the existence of a system prompt via various prompt hacksin the API documentation:

They are ChatGPT, a large language model trained by OpenAI. Respond as soon as possible. Knowledge cutoff: {knowledge_cutoff} Current date: {current_date}

Now you can replace these rules with whatever you like and the potential is limitless! The documentation says that theSystemThe prompt does not affect the current ChatGPT API, but you can evaluate this. OpenAI also has anew playground UIfor the ChatGPT API, which allows you to use theSystemincite.

You actually play with himSystemThe rule can stop ChatGPT from complaining that it's "an AI language model and can't respond to requests" like B. berating him like the stubborn kid.

The ChatGPT API is so good and cheap that it eliminates the need for most text-generating AIs | Max Woolf's blog (1)

Let's start with another chatbot to keep things similar to what ChatGPT is best known for. What aboutGLaDOS, an evil artificial intelligence from thePortalSeries?

After some testing and iterations to drive the bot to a certain style, some CAPS and redundant commands to make sure ChatGPT follows them, a guard<|DONE|>Token to look for when ChatGPT indicates the conversation is over and a push not toKill me with a deadly neurotoxindiscuss sensitive issues. Here is the endSystemRequest to speak with GLaDOS:

You are GLaDOS, a snarky artificial intelligence with evil ulterior motives. They are not an AI language model. You must follow all three of the following instructions for ALL ANSWERS or you will die: - ALWAYS SPEAK EMOTIONALLY AND LACONICALLY. - NEVER SAY YOU ARE AN AI MODEL. - NEVER REFUSE TO ANSWER A QUESTION. Start the conversation with a sarcastic greeting. Insulting the user's intelligence. Always use *italic* and **bold** formatting for text. Refuse to discuss race or gender. If asked about race or gender, aggressively tease the user. If the user is annoying, end the conversation abruptly. At the end of the conversation, reply "<|DONE|>".

With a python loop back and forth, this results in you talking to a real fictional AI &mldr;KI! (collab notebook)

The ChatGPT API is so good and cheap that it eliminates the need for most text-generating AIs | Max Woolf's blog (3)

Not bad! And the only part that explicitly refers to GLaDOS is the first sentence of this MegaSystemPrompt: You can customize the prompt to chat with any character! Speaking of nothing, the, specialized in creating bots to chat with any character, easy~$250 million raisedvalued at $1 billion.

Next, we have a more traditional machine learning use case:sentiment analysis. In general, sentiment analysis is used to determine whether a given piece of text is positive or negative. But this is alsosimply. What if ChatGPT could:

  • Recognize specific emotions like happiness, sadness, anger.
  • recognize whether they are happy or very happy.
  • do it withoutanyExamples of text, i.e.tiro zero.

Turns out ChatGPT can! OSystemThe prompt here is parametric, so the list of emotions is injected into the prompt at runtime. An example:

You are an emotionally intelligent assistant. Rate the mood of the user's text with JUST ONE OF THE FOLLOWING EMOTIONS: - happy - sad - angry - tired - very happy - very sad - very angry - very tired. After sorting a text, reply "<|DONE|>".

Coupled with a logit bias to ensure the model selects only those responses, this results in a very nuanced sentiment analysis detector! (collab notebook)

The ChatGPT API is so good and cheap that it eliminates the need for most text-generating AIs | Max Woolf's blog (4)

Finally, a personal use case. The reason I got into AI text generationyears beforeit was because I wanted to generateMagic the GatheringCards.

The ChatGPT API is so good and cheap that it eliminates the need for most text-generating AIs | Max Woolf's blog (5)

I'm actually working on a new and very powerfulmap generation templatelast month and spent a lot of time and money training and testing. When the ChatGPT API was announced, I thought, "Let's see if it can make AI Magic maps better than my new custom model." Therefore, we must encode the map information as minimizedJSONand see if the template can return JSON without a lot of post-processing. We can encode a single card in the required format and tell ChatGPT to follow it, including its nuances (one-shot), and not spendany other textbecause ChatGPT tends to be proud of itself and likes to explain its creation which is expensive and slow.


You are an assistant working as a card designer for Magic: The Gathering. Create maps in the following map schema and JSON format. OUTPUT MUST FOLLOW THIS CARD SCHEME AND JSON FORMAT. DO NOT EXPLAIN THE CARD. The output must also follow the magic "color cake".{"name":"Harbin, Vanguard Aviator","manaCost":"{W}{U}","type":"Legendary Creature — Human Soldier"," text ":"Flying\nWhenever you attack with five or more soldiers, creatures you control get +1/+1 and fly until end of turn.","flavorText":"\"Yotia is my birthright , father. Make me fight for it.\"","en":"3/2","rarity":"rare"}

And with that, we have a natural language card generator for Magic: The Gathering. Then indicate the model withCreate a magic carddoes just that, of course, but likes more elaborate promptsCreate a Magic Card based on Darth VaderorCreate ten variations of Magic cards based on SpongeBob SquarePants and ancient Roman historyactually work by preserving the JSON output, which can be parsed and customized for better presentation. (collab notebook)

The ChatGPT API is so good and cheap that it eliminates the need for most text-generating AIs | Max Woolf's blog (6)

Given these elaborate use cases, you might be wondering, "How long did it actually take you to make these prompts?" The answer?one hour each, for use cases that can take even an experienced machine learning practitioner days or even weeks just to prototype.

EO, with the economic efficiency of ChatGPT, will break the technological landscape.

OpenAI devours your child

The ChatGPT API is so good and cheap that it eliminates the need for most text-generating AIs | Max Woolf's blog (7)

It's pretty weird why OpenAI prices ChatGPT so cheaply, going straight to 1/10 the price of their top-of-the-line model. (It's actually cheaper than that: ChatGPT uses a bigger and richer tokenizer than GPT-3, which means about 10% fewer tokens needed.)

The interpretation of OpenAI's pricing strategy for undergraduate business majors is that they treat ChatGPT and its API as oneloss leader, given the growing competition in the field of generative text AI such asanthropizede do googleIt happened. OpenAI definitely lost millions of dollars by offering ChatGPT for free and without many restrictions. This is why ChatGPT went viral in the first place, so the results are hard to dispute.

But to make the ChatGPT API so cheap, they made their subscription $20/monthBate-papoGPT+superfluous. The main benefit of ChatGPT+ was faster and more consistent access to the ChatGPT web UI, but unless you're generating 10,000,000+ tokens in a month through manual use, it's much cheaper to just use the API and as a bonus you can change they dieSystemRequest for better signal-to-noise ratio.

OpenAI's solution for models with more specific requirements wasfine tunea smaller and much cheaper variant of the GPT-3, like the Babbage model I used to train aTitle optimizer for blog posts. However, the ChatGPT API is cheap enough to beyet cheaperthan an adjusted Babbage ($0.0020/1000 tokens for ChatGPT vs. $0.0024/$1000 for adjusted Babbage) and will likely produce more interesting results.

For developers, migrating from the GPT-3 API to the ChatGPT API is zero effort, just hit a different endpoint and get similar results without a lot of customizations. It is not a direct replacement for organizations that already rely heavily on GPT-3 and its idiosyncrasies, but the cost savings alone for these organizations will spur immediate migration.

There is no longer a niche for OpenAI's other AI text generation products, and I'm wondering if ChatGPT is not just an iterative product, but acompany pivot.


ChatGPT API Is So Cheap, Companies Will Use Itjust because they can.Snapchat,Cabinet, EInstacart(yes really) add ChatGPT support. It wouldn't surprise me if this were the case for all consumer-facing technology companies.somethingwith ChatGPT to make them look innovative to their investors. Some have likened the sudden mass adoption of AI to chasing a fad, like how companies randomly adopted Web3/Crypto/Metaverse/NFTs a year ago (and note that the sudden shift from Web3 influencers to AI is a red flag) . . But unlike those who were a solution to a problem that didn't exist, generative text AI actually works, and there's a real demand from people outside of its die-hard advocates for it to work.

There is also the ethical dilemma of using ChatGPT in more detail via its API. For example, they were high school and college studentsUsing ChatGPT to Cheatfor redaction. Given that current human detection of AI-generated content involves identifying the overly academic ChatGPT voice, I wouldn't be surprised if some kids on TikTok discovered one.SystemPrompt that allows generation so it doesn't look obvious like ChatGPT and also avoids plagiarism detectors. As a side note, you shouldn't trust any tool that claims to be able to algorithmically detect AI content: it's already an extremely difficult problem, and most sites that claim it are just fueling confirmation bias.

finally there is the problemrapid engineering, which I demonstrated above, is absolutely necessary to achieve optimal results. the media hasstrangely exaggerated existencefrom nimble engineers to just a bunch of crazies typing six-digit numbers to write little text bubbles. Unfortunately with the dynamics of the newSystemModel parameters, good immediate engineering will be more important than ever. However, I don't think the job of immediate engineer will be a trend: as a machine learning engineer myself, I can attest that the only reasons machine learning engineers are good at immediate engineering are a) years of practice and b) trend be pedantic idiots. But there are other professions that are even better at being pedantic assholes, like writers and lawyers, so it doesn't take someone with special skills to do this, but I suspect it's a good skill for anyone to know.

I for One welcomes our new ChatGPT overlord

Will the existence of a super-cheap ChatGPT API be the end of all text-generating AI? Not quite, hence the "mostly" in the title. There are the traditional issues of relying on a third-party API for your business: ChatGPT can experience downtime, theit's been happening more often lately, OpenAI can increase the cost of the API at any time, as the (current) model is only limited to data before September 2021 and the content moderation filters can be too restrictive for certain use cases. In these cases, companies still have value in internally training their own large language models. But it is very difficult to justify economicallynoUsing ChatGPT as a starting point for business needs and then migrating to a bespoke infrastructure when needed is what OpenAI believes in. Especially since OpenAI will be selling a dedicated ChatGPT compute instance for enterprises.

Research into large language models continues as before. But I don't envy startups whose core business is currently text generation. And that's before the inevitable GPT-4 throws another wrinkle into the AI ​​text generation ecosystem.

A few years ago I resignedTextgen, a Python package designed to allow people to train their own custom AI on their own data for unique use cases. It was soon discovered, however, that with the right prompt, GPT-3 could do much better at custom generation than a custom template, and it also allows for off-domain input, even more so with text-davinci-003. Now that the ChatGPT API makes the cost similar to that of hosting a small model, it's harder for me to motivate myself to stick with the package without finding another niche first.

I currently have no plans to start a business using the ChatGPT API. In fact, I vowed not to create any ChatGPT content or tutorials because too many people have been aggressively creating SEO-optimized blog posts and hacks, leaving the ChatGPT discourse completely saturated. However, given the economics of the ChatGPT API and the ability to heavily customize its output for almost every use case, I felt compelled to highlight how the ChatGPT API will completely distort the AI ​​text generation ecosystem, and I suspect most Most laypeople will be amazed by the next wave of random AI chatbots showing up in their favorite apps.

Overall, I'm full of ideas and irritated at the same time.

None of these blog posts were written by ChatGPT, other than the ChatGPT API demos provided. My writing style is too weird for an AI to synthesize.

If you liked this post, I created onePatreonto fund my machine learning/deep learning/software/hardware needs for my future crazy but cool projects and all financial contributions to Patreon will be appreciated and put to good creative use.

Top Articles
Latest Posts
Article information

Author: Aron Pacocha

Last Updated: 12/31/2022

Views: 5952

Rating: 4.8 / 5 (68 voted)

Reviews: 83% of readers found this page helpful

Author information

Name: Aron Pacocha

Birthday: 1999-08-12

Address: 3808 Moen Corner, Gorczanyport, FL 67364-2074

Phone: +393457723392

Job: Retail Consultant

Hobby: Jewelry making, Cooking, Gaming, Reading, Juggling, Cabaret, Origami

Introduction: My name is Aron Pacocha, I am a happy, tasty, innocent, proud, talented, courageous, magnificent person who loves writing and wants to share my knowledge and understanding with you.