Do I need a prompt framework?

Do you need a prompt framework like RACE (role, action, context, expectation), RTF (role, task, format), or COSTAR (context, objective, style, tone, audience, and response)?

In short? It depends on what you’re doing with AI.

Most large language models today actually don't need a prompt framework anymore for them to work well. You don't need to specify that you need the AI to act like a professional market researcher with 20+ years of experience in order for them to write you a good draft questionnaire or discussion guide.

Instead, what you should focus on is context and structure.

Context really just means the information that you would give a junior researcher about the task that you need them to perform. Let's take an example. Let's say you need a junior researcher to come up with a good draft questionnaire.

What information would you give them? In this case, you'd probably give them the industry, the customer type, the objective of the questionnaire, what is the business question that the questionnaire is supposed to address, the audience that the questionnaire is supposed to be designed for, whether you want the demographics at the beginning or at the end or split between the two, the kind of screener that you want, and maybe the length of the questionnaire, and whether you want to have some open ended questions included in the questionnaire as well.

That is context. That is the kind of information that you would want to include in a good prompt for your AI in order for it to create a good questionnaire.

Now let's look at structure.

Structure means how you want the AI to output the task.

Let's take a look at the questionnaire again. Do you need the questionnaire to be in Word format or Excel? Do you have examples of previous questionnaires? Examples are useful so that it can tell whether you need to have programming in a certain way, whether you have variable names in parentheses at the beginning of each question or after the question text, or whether you have your scales running positive to negative or in numerical values or in word values.

So, if you consider a prompt in terms of putting together information for a junior staff member in order for them to execute this task well, you are going to be putting together a much better prompt than if you are going to your AI and simply saying, “I need a questionnaire for a concept test for five concepts about dog food.”

If you were to give that to a junior staff member, they wouldn't do a good job giving you a draft questionnaire, either.

Bonus tip. After the AI gives you its output, ask it to give itself a grade. You would be surprised how often the grade is not an A or 100%! Once it gives you a grade, you can do one of two things: you can either tell it to go back and do the task again, but this time improve the grade, or you can ask the AI, "What do you need from me in order for you to improve your grade on this task?"

Then it gives you questions, you answer those questions, and you tell it to go and do the task again, this time improving its grade. Don’t forget to ask for the AI to grade itself again to compare the grades from the first to the second time!

So, when do you need a framework? The role/task/format frameworks are best used for agents. Agents are where you need to explain the role it needs to play and task it needs to do, and the specific format it needs to follow whenever it executes. If you ask AI to help you write a prompt to build an agent, you’ll find it uses the role/task/format framework itself!

Did you find this email helpful? Email it to your friend and tell them to sign up for the weekly Tuesday tips! And if you have a burning question about AI, reply to this email and let me know so I can address it in a future edition.