Aside: How ChatGPT Models Work for Chatbots

[00:00] Okay, let's take an overview of how the GPT-4 model works with chatbots. So we're going to use one function to make requests to the API, and in a conversation, we'll use that function multiple times. But we need to think about what we send in the prompt, because there's actually a big problem for chatbots that we have to overcome.

[00:19] And I want to illustrate that for you right here. So what I've got here is a simple call to the API using the createCompletion endpoint and the TextDaVinci003 model, which is the model we used in the previous project. And I chose it here because it should look pretty familiar by now,

[00:37] and I can use it to illustrate the point that I want to make about chatbots without getting caught up in new syntax we haven't studied yet. Now, I'm going to come in here to this prompt, and I'm going to ask a question. Where were the 2001 Wimbledon Tennis Championships held? OK, let's call that function, and we'll hit Save and open up the console.

[01:01] And we get the answer. The 2001 Wimbledon Tennis Championships were held at the All England Lawn Tennis and Croquet Club in Wimbledon, London, England. OK, so it's a correct answer. Now let's ask it who won that year.

[01:16] And I'll hit Save, and it tells us the Philadelphia 76ers won the 1982 NBA Championship. And this is what's called a hallucination. The AI makes up a linguistically plausible answer when it doesn't know the right answer.

[01:33] And we'll talk more about hallucinations later in this course. Now, it figures that it doesn't know the answer. The question, who won it that year, only makes sense in the context of the previous question, which had the keywords Wimbledon and 2001. And that exposes a big problem for chatbots,

[01:51] which is that models have no memory of past completions. And that means that all relevant information must be sent with each API request. So I need to refactor my second question to include all of the information that the model needs to know, i.e. who won Wimbledon in 2001.

[02:12] Let's run that. And it says Goran Ivanicevic won Wimbledon in 2001. And that is true with the only problem being that it's assumed I was talking about the men's championships. The women's championships were won by Venus Williams that year, and she beat the Belgian Justine Henin in three sets.

[02:31] So there's just a little example of the biases picked up by AI models as they're fed data from the open internet. OK, so when it comes to chatbots, in order for the model to interact properly so the conversation flows and remains logical, the model needs to know the context of the conversation. And we achieve that by sending the conversation

[02:51] as it exists so far with each request. So let's look at a diagram of how the main AI business logic is going to work with this chatbot. So the first thing that we'll need to do is store the conversation in an array. And that is what's represented here by this box. And this does have to be an array.

[03:11] The GPT-4 model and the endpoint we're going to be using have much stricter rules and syntax than the TextDaVinci 003 model and the CreateChatCompletion endpoint. Now, once we've set up this array, we'll use some special OpenAI syntax to instruct the chatbot and tell it how we want it to behave.

[03:28] And then we'll store that instruction in an object at the first index of the array. And I've just put the curly braces either side of instructions just to make it clear that that will be an object and that this is an array of objects. So now we've got the instructions at the first index of the array.

[03:47] We will take some input from the user. And the first thing to do with that input is to render it to the DOM. Then we'll use some OpenAI syntax to format it. And again, we'll save it in an object in conversation array. Once that's done, we're going to send the entire conversation array off to the OpenAI API.

[04:07] And what we'll get back, of course, is the response. And what we'll need to do then is render the completion to the DOM and then place the completion in an object with the correct syntax that we'll be discussing shortly. And then we'll be adding it to conversation array. So right now, conversation array has got three objects.

[04:25] We've got the instructions, the user's input, and the first response from the API. And then as you can imagine, this process continues. We take the user's reply. We render it to the DOM. We place it in an object. We store that object in conversation array. And then we send it off to the API. We get back our response.

[04:43] And we render it to the DOM, place it in an object, and then store the object in conversation array. And that process continues with the user's response. And we just keep repeating as needed. And this can go on and on until you reach the token limit. But the token limit with the GPT-4 model is extremely generous.

[05:03] So it would have to be a very, very long conversation indeed to reach that limit. Okay, that's the overview. Let's get to work.