Think. Try. Transform #7
Prompting - What do (a few) people think?
Prompt engineering:
‘The career of all careers.’
‘800K a year.’
‘You don’t need a computer science degree. If you studied language or humanities, you’re at an advantage.’
‘Everyone using AI needs to learn prompt engineering.’
If you have been following AI developments for the last year or so, you would have undoubtedly heard some of the above hype around prompt engineering.
I’ll be honest - I got swept up in this too. I scoured Indeed and LinkedIn Job for vacancies at OpenAI and Anthropic. No luck, unfortunately! Many of these jobs still seemed to require a Ph.D. in Computer Science.
I enrolled in a Coursera Course: Prompt Engineering for ChatGPT. It was useful, to a degree, but I seemed to have learned more from just playing with AI.
Discussions about the utility of effective prompting have not gone away. But what does effective prompting look like? Does it require an understanding of the technical jargon associated with prompt engineering? Or is it intuitive? Limited only by our creativity, imagination, and specificity.
With more than a million teachers now using Magic School AI, there is certainly an appetite for pre-built prompts that perform specific tasks. And why wouldn’t there be? Time is precious and many teachers will jump at the opportunity to save a few minutes in a day. But does expediting the prompting result in the best outcomes? Or should teachers spend time learning how to prompt by interacting with AI models and seeing what they are truly capable of?
The purpose of this week’s blog is to try and answer those questions. I decided to pose three questions to a number of my connections on LinkedIn. I wanted to get a sense of what educators, ex-educators and a few people who are education-adjacent are doing at this point in time. I specifically reached out to users who have been thinking about and using AI for some time.
I appreciate this approach does not reach all four corners of the education globe, but my hope was that whoever you are and wherever you work, if you are interested in using AI, there will be something for you below.
Think.
Q1: What are your thoughts on prompt engineering? Do you think teachers must have an understanding of some of the techniques?
There seems to be a growing trend towards specific prompt engineering skills not being necessary. The ability of the models, having improved drastically with the likes of GPT-4, has meant that complex prompts do not necessarily yield the best results. Instead, simply interacting with an LLM, aiming towards a desired end, was encouraged.
"As time goes by I think the skill of prompt engineering per se may not be massively important in the educational sector; I do think though it is important for teachers to understand how different approaches can lead to different results."
"LLMs and LMMs are getting better at interpreting and enriching vague and incomplete instructions... They're getting better at understanding the underlying intention of the user."
"Models are improving in how they handle prompt interpretation... my answer is basically 'no'."
"[Prompt engineering is] necessary if they want to reap the benefits. But not too much into the Science that they lose the ease of use... Striking the balance is very important."
"I have mixed feelings about 'prompt engineering.'... Busy teachers should focus on a few key steps: defining a role for the AI, giving clear instructions for their goal, and specifying the desired output... mastering prompting takes time."
"I think prompt engineering is ultimately critical thinking. It’s your ability to articulate an idea, a challenge, directions or anything. People who can articulate with people or machines they can shape the narrative. It’s the difference between consumers and creators. You’ll also save a lot of money when you aren’t dependent on wrapper apps"
A key theme that has emerged is the art of questioning. Fortunately for teachers, this is something we should be good at. We have an innate ability to engage the minds of our students; so why not apply the same thinking and methodology when interacting with a LLM? That way, we can explore complex concepts and encourage the LLM to produce unique outputs. This should encourage teachers to take the leap to use AI models and tools - they already have many of the skills required!
"It's definitely important to understand how to prompt correctly to get the output you want... Teachers are already excellent at asking questions, they just need to use the skills they have."
"Although some experts say that prompt engineering will not be very crucial soon, I think teacher should know about the standards of writing effective prompts."
"I think people should learn some basic prompting skills... Teachers should learn how to prompt both to save some time for themselves, to help their students, and to understand what their students are often doing."
Try.
Q2: Do you have any ‘go-to’ prompts? If so, could you share one?
Several teachers did not have specific prompts they use regularly. Instead, they engage with LLMs conversationally and use whatever prompts will get them closer to their desired outcome. For unique scenarios and interactions, it seems that using unique prompts is the way to go.
"Not especially, in that I would emphasise using AI tools is more about a conversation than a single prompt."
"No 'go-to' prompts have emerged. I use whatever prompts are appropriate for the task."
"All my problems are unique so I create a unique prompt... Go to prompts are good to show the art of what's possible... but better they create and experiment with their own as prompt results seem to vary."
For those who have repetitive tasks, however, you may want to leverage Chat GPT’s facility to create your own GPTs (Plus users only).
"I've created my own GPTs using ChatGPT Plus account which means I always get good and quick results from my prompts."
Others have a few tricks up their sleeves or frameworks they follow. Metacognitive approaches are particularly popular, such as asking the LLM to elicit from the user what information it needs to best support a task. You don’t simply seek an answer or ask for a task to be completed. Instead, the LLM is invited to participate in a reflective process. What’s particularly interesting about this approach is it mirrors methods that may be used in the classroom to guide a student to complete a task.
Other techniques involve giving the LLM a role. This can make the interactions more engaging, but can also result in improved quality of output as the LLM aims to mirror an expert. These types of technique are common in prompt engineering courses and I have had personal experience of some excellent outputs when clearly defining a role in a prompt.
Framework for prompting have also been widely discussed and many people have strong opinions about these. There is no doubt, however, that many teachers like the familiarity of a framework, and so applying either the SPARK or TRAIN model could be very helpful.
"Two good prompts... 'I am doing [add task]. What information do you need from me in order to support me in the best way through this task?' And 'I want you to work in a metacognitive way, foregrounding your thinking throughout'."
"I enjoy customizing my prompt by beginning in a broader narrative and iterating as I progress, depending on the nature of the activity."
"You are an experienced [subject] teacher, write me 10 retrieval practice questions based on [learning objectives] that are aligned to Bloom's taxonomy...' Or: 'You are a master at linking the curriculum to real-world problems. Write a real-world scenario/problem for [learning objectives]..."
"I frequently use the same template: 'you are an expert curriculum designer/ you are a professional career coach, and you will + task/instruction'"
"I always give the [LLM] a persona, point and output. And I always say please and thank you."
"I usually use the SPARK framework (my situation, my problem, my aspiration, how I want the results). I admittedly don't always get to the kismet -- something special added on."
"I have my SPARK method, an empathy-driven approach where you share your situation, problem, aspirations, results and then kismet is your AI tools sharing ideas, solutions, and strategies"
"Use the STAIR model… :)" You can more details in this online book.
Transform.
Q3: What is one tip you have for teachers when prompting LLMs?
The information you feed the LLM is key. Don’t assume any prior knowledge and be precise and clear in your prompting. These were key themes that came out of the responses to this question. Just as we cannot expect our students to draw upon knowledge that they have not been taught, the same is true of LLMs. They are bound by what they know, and although reasoning and the ability to produce novel content is improving, I think assuming that their knowledge is limited, and therefore producing clearer prompts, is essential.
"Contextualize [your] prompts, provide as much information as possible so that it provides the result that is most appropriate."
"Use a devil's advocate type prompt... WHAT you prompt the AI to do is more important than HOW you prompt it."
"Express yourself clearly. Think thoroughly. Do not leave anything for AI to choose."
"Provide as much context as possible to the request - assume the model knows nothing and be as explicit as possible, repeating yourself where necessary."
"Start with a command verb like 'create', 'design', or 'determine' that has intention... anticipate the need for multiple iterations, distilling down the commands like a funnel.
Another key theme was the importance of striking up a conversation with a LLM. Yes, it is important to prompt clearly, but if you do not initially get your desired outcome, don’t assume it’s due to a limitation on the part of the LLM. Continue interacting and engaging, adding context and refining your ideas. This, again, should be natural for teachers as it is our job to effectively communicate with out students. We should do the same with LLMs. This should be another area in which we encourage teachers to explore these tools as they have the required skillset. Communicating with a LLM should not be a barrier to adoption.
"Have conversations with the AI... You should have a conversation with it... you'll walk away with many good ideas, improved productivity, and higher quality work."
"Use natural language, prompt the way you would ask another colleague a question."
"See AI as a partner, not as a servant... prompting it to tell you what it needs from you in order to give you the best support."
And finally, there were some fantastic general tips for prompting LLMs. These really highlighted the dynamic and experimental spirit that is necessary for teachers to harness the best capabilities of AI. Innovation is crucial and it will only come through a process of trial and error. The more you use AI, the more you will find uses for AI.
"Those who are into it will push their own boundaries."
"Don't hesitate to recognise when a 'chat' isn't progressing as intended... The key is to deconstruct our original intent and identify what's being misinterpreted or lost in translation."
"Ask open-ended questions to foster critical thinking and creativity."
"Experimenting is key, always spare some time to try out different prompting to improve your skills."
Overall, it is clear there are various ways teacher can go about prompting a LLM. For me, that is what makes this technology so exciting. Sure, there are several tricks and tips that could be adhered to, but we should not feel limited in any way when interacting with a LLM. The possibilities really are endless, bound only by our creativity and imagination.
It’s appropriate to leave the final word about prompting to Ethan Mollick. He posted this recently on LinkedIn:
‘…If you are not writing prompts for other people to use, you don’t need to do anything other than give the AI a role, some context, and have a conversation. For the vast majority of cases, prompt engineering isn’t worth it and will go away. For almost all interactions I have with AI, I don’t use any prompting tricks… It isn’t worth it. If you use any advanced LLM enough you can eventually understand what it is ‘thinking’ and how to guide it conversationally. People make this too hard.’
Additional Resources
Ethan Mollick’s Blog - One Useful Thing: Working with AI:Two paths to prompting
Open AI’s Prompt Engineering Tips
Mark Anderson - The Little Book Of Generative AI Prompts For Teachers
Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4 (For the TL;DR version, see page 5)