It's popular among AI folks to think in terms of phases of AI, of which the current and most reachable target is likely “oracular AI”. Tools like ChatGPT are one manifestation of this, a form of question and answer system that can return answers that will soon seem superhuman in terms of breadth of content and flexibility of style. I suspect most educators don't think about this framework of AI as oracle much, but we should, because it explains a lot both about the current hype cycle around large language models and can help us gain critical footing with where to go next.

From the lesswrong site earlier, here's how they describe oracular AI (on their overall perspective, definitely take in the full set of ideas there): > An Oracle AI is a regularly proposed solution to the problem of developing Friendly AI. It is conceptualized as a super-intelligent system which is designed for only answering questions, and has no ability to act in the world. The name was first suggested by Nick Bostrom.

Oracular here is an de-historicized ideal of the surface function of an oracle, made into an engineering system where the oracle just answers questions based on superhuman sources or means but “has no ability to act in the world.” The contrast is with our skynet future (choose your own AI gone wild movie example), where AI has a will and once connected to the means will most certainly wipe out all of humanity, whether for its own ends or as the only logical way to complete its preprogrammed (and originally innocuous, in most cliches) goals.

Two things to note here: 1. This is an incredibly narrow view of what makes AI ethical, focusing especially on the output, with little attention to the path to get there. I note in passing that much criticism of current AI is less with the outputs and more with the modes of exploitation and human capital and labor that go into producing said outputs. 2. This is completely backwards view of oracles.

The second point matters to me more, primarily because it's a recurring pattern in technological discussions. The term “oracle” has here been reduced to transactional function in a way that flattens its meaning to the point that it evokes the opposite of the historical reality. It's not just marketing pablum, but here a selective memory with significant consequences, a metaphor to frame the future. Metaphors like this construct an imaginary world from the scaffolding of the original domain. When we impoverish or selectively depict that original domain, when we distort it, we delude ourselves. It is not just a pedantic mistake but a flaw of thinking that makes more acceptable a view that we should treat with a bit more circumspection. What's more, the cues to suspicion are right there in front of us. The fullness of the idea matters, because we can see that the view of oracular AI as a friendly AI is a gross distortion, almost comically ignoring the wisdom that could be gained by considering the complex reality that is (and was) oracular practice.

(Since the term “oracle” generally looks back to ancient practices, for those who want some scholarly grounding, check out Sarah Iles-Johnston, Ancient Greek Divination, Michael Flower, The Seer in Ancient Greece, Nissinen's Ancient Prophecy, etc etc. or in other eras and with electronic resources access, e.g. Oxford Bibliographies on prophecy in the Renaissance.)

Long story made very short, oracles are not friendly question and answer machines. They are, in all periods and cultures, highly biased players in religio-political gamesmanship. In the case of the most famous perhaps, the Pythian oracle in Ancient Greece, the answers were notoriously difficult to interpret correctly (though the evidence for literary representations of riddling vs. actual delivery of riddling messages is more complicated). Predicting the future is a tricky business, and oracular institutions and individuals were by no means disinterested players. They looked after themselves an their own interests. They often maintained a veneer of neutrality in order to prosper.

That is all to say that oracularism is in fact a great metaphor for current and near future AI, but only if we historicize the term fully. I expect current AI to work very much like oracles, in all their messiness. They will be biased, subtly so in some cases. They will be sources from unclear methods, trusted and yet suspect at the same time. And they will depend above all on humans to make meaning from nonsense.

This last point, that the answers spouted by oracles might be as nonsensical as they are sensical, is vital. We lose track amidst the current noise around whether generative AI produces things that are correct or incorrect, copied or original, creative or stochastic boilerplate. The more important point is that humans will fill in the gaps and make sense of whatever they are given. We are the ones turning nonsense into sense, seeing meaning in a string of token probabilities, wanting to take as true something that might potentially be a grand edifice of bullshittery. That hasn't changed since the answer-givers were Pythian priestesses.

Oracular AI is a great metaphor. But it doesn't say what its proponents think it says. We humans are the ones who get to decide on whether it is meaningful or meaningless.

#chatgpt #ai #edtech #aiineducation #edtech #education

The generative “AI” hype cycle has been at peak hype for the past month or so and it follows completely predictable tech patterns. Hypers tout all the amazing miraculous things that will be possible; doubters wonder aloud whether these things will fail to deliver on their utopian promises (because these things always fall short of their utopian promises), and most of the obvious consequences and outcomes get overlooked.

One such obvious consequence is that there are tidal waves of bullshittery about to hit our shores. (This first wave is a minor high tide compared to what is coming....) Reconstituted text, images, video, audio, avatars and fake people are pretty much guaranteed across a wide variety of areas, a landscape where education is only one small province. We won't be able to tell real from fake or, perhaps more troubling, I don't think we'll care so long as it scratches the right itch or feeds the right need.

The question across those domains will be whether we value authenticity. For things like boilerplate email, sales copy, code, and a wealth of other activities, I think the answer will be that authenticity doesn't matter that much. But that's where education is different. Authenticity should matter, not because of the habitual exercise of needing to assign grades to work that was not plagiarized or copied or whatever other vice one can ascribe, but because without authenticity there is no learning. Faking it is great for getting to the ends. But education is about the means; ends (tests, essays, etc) have always been imperfect proxies. Beyond the authenticity of student work, we have a very familiar issue of how students themselves or learners know what kinds of information to trust. While the bulk of attention thus far has been on the nature of the emerging generative “AI” toolkit and the back and forth between fearing cheating vs. fostering creativity with such tools, the real impact will be felt indirectly, in the proliferation of “knowledge” generated by and mediated through generative AI tools. It is the old wikipedia debate, but supercharged with hitherto unthought of levels of efficacious bullshittery.

Ten years ago it was a clarion call with the proliferation of data that academic knowledge fields needed more curation. For example, http://www.digitalhumanities.org/dhq/vol/7/2/000163/000163.html is one of many such calls for increased digital curation of data. The variety of startups applying generative “AI” to learning or, more broadly, to varieties of search and summarization, tend to promote the message that curation is not necessary. (Just google “sequoia generative ai market map” or similar; https://www.sequoiacap.com/article/generative-ai-a-creative-new-world/.) Or, rather, the question of curation has perhaps not entered into thought. Automagically search or summarization or chatbots using generative AI will latch on to the most relevant things for your individual query. Consumerism is a given, such that the only question is how the system can serve up results to a consumering user. LLMs have thus far been gaining ground through hoovering up every more data. That makes them garbage collectors, even with careful controls to make sure that bias is minimized and good data is optimized. Optimistically one might imagine that these technologies could allow for curation to happen at a different stage, at the building of the model, or in fine-tuning the model for particular use cases. Or the context provided by the consumer is a sort of after the fact filter on the massive amounts of knowledge. But that is a very light veneer of the kind of knowledge curation that separates the wheat from the chaff, that ensures that what's being served up isn't utter bullshit that sounds close enough.

There are two levels of authenticity then to keep an eye on. The surface one is with students themselves and the process of learning. Are the people being authentic? Then there's the second, at the level of knowledge curation. Is that curation authentic and legit? I suspect on both scores it will require direct and focused effort to foster both amidst the readily available misinformation available. For LLMs in particular, we are looking now at an exacerbated version of wikipedia bias. If something is statistically weighted as more likely but expertly-verified to be wrong or misleading, how do those concerns get balanced? It is not merely that generative “AI” can produce different outcomes given the same inputs, it's that there is not necessarily a clear line as to why those two different ideas are held in mind at the same time.

Undoubtedly, such issues will be smoothed over and it will all be more nuanced as these technologies develop and as these technologies are deployed. The early days of autocomplete were rife with inaccuracies, bias, and garbage. And now we treat it like any other tool. some may ignore it but most simply use it when convenient and don't think twice about the biases or thought patterns it subtly instills. Generative “AI” will be no different. It will soon become another layer of bullshit which is sometimes useful, often ignored, and just one more thing to take account of when negotiating authenticity of learners and reliability of knowledge.

This is all to say that the tool hasn't changed the essential question. Do we actually value authenticity in the learning process? Do we care about not just the verifiability of knowledge through citation (which, incidentally, Google seems to be focusing on in their response to OpenAI, among others) but about that thing formerly known as “truth”, at least as an asymptotic goal if not reality?

It's going to be messy. Truth-y enough will be good enough for many. And many structures in education are already transactional to an extent that authenticity is a pesky anti-pattern, a minor detail to be managed rather than a central feature of the learning experience.

In more optimistic moments I wonder whether the value of generative “AI” can lie not in its products but in the opportunity it creates to further dialogue. If we keep our focus on fostering authenticity in students and authenticity in knowledge, then it can be a useful tool for first drafts of knowledge. If we let it become the final word, then I fear we will simply be awash in a smooth-talking version of the internet's detritus.

#minimalistedtech #generativeai #chatgpt #edtech #education #learning

Any new technology or tool, no matter how shiny its newness, can help students experiment with how technology mediates thought. I suspect that's the least problematic use of generative “AI” and large language models in the short term. One reason I think of this kind of activity as play or experimentation is that if you go much further with it, make it a habit, or take it for granted, then the whole enterprise becomes much more suspect. Most consumer-facing applications showing off large language models right now are variations of a human in the loop system. (ChatGPT exposes a particularly frictionless experience for interacting with the underlying language model.)

A key question for any human in the loop systems is that of agency. Who's the architect and who is the cog? For education in particular, it might seem that treating a tool like chatGPT as a catalyst for critical inquiry puts humans back in control. But I'm not sure that's the case. And I'm not sure it's always easy to tell the difference.

One obvious reason this is not the case with chatGPT specifically is that OpenAI's interest in making chatGPT available is very different from public perception and adoption. To the public, it's a viral event, a display of the promise and/or peril of recent NLP revolutions. But OpenAI is fairly clear in their fine print that they are making this publicly available in order to refine the model, test for vulnerabilities, gather validated training data, and, I would imagine, also get a sense for potential markets. It is not different from any other big tech service insofar as the human in the loop is commodity more so than agent. We are perhaps complacent with this relationship to our technology, that our ruts of use and trails of data provide value back to the companies making those tools, but it is particularly important in thinking through educational value. ChatGPT is a slick implementation of developing language models and everything people pump into it is crowdsourced panning for gold delivered into the waiting data vaults of OpenAI. (For a harsher critique of the Effective Altruism ideology that may be part of OpenAI's corporate DNA, see https://irisvanrooijcogsci.com/2023/01/14/stop-feeding-the-hype-and-start-resisting/)

Set that all aside for a moment. If we take the core human in the loop interaction of prompting the language model and receiving a probabilistic path through the high dimensional mix of weights, a path which looks to human eyes like coherent sentences and ideas, where exactly is the agency? We supply a beginning, from which subsequent probabilities can be calculated. Though that feels like control, I wonder how long before it's the case that that machine dictates our behavior? As is the case with email or text or phones, how long before we have changed our way of thinking in order to think in terms of prompts?

For example, one particularly effective method with ChatGPT in its current incarnation is to start by giving it a scenario or role (e.g. “You are a psychologist and the following is a conversation with a patient”) and then feed in a fair amount of content followed by a question. (I gave a more elaborate example of this scenario setting earlier here.) That context setting allows the model to hone in on more appropriate paths, matching style and content more closely to our human expectations. I expect working with these tools over time will nudge people into patterns of expression that are subtly both natural language but also stylized in ways that work most effectively for querying machines. As was the case for search, the habit of looking things up reinforces particular ways of thinking through key words — of assuming that everything is keywordable — and ways of asking questions.

Most of the conversation around generative tools has been about control more than agency. As a set of tools whose functioning is, to a certain extent, still unauditable and whose creation relies on datasets so massive as to stymie most people's existing level of data literacy, generative AI is a black box for the majority of users. So teachers worry: how do we control this? who controls this? how do we know what is happening? That is perhaps no different than most high tech devices or softwares. For education, the stakes are different however.

Learning requires students gain a sense of agency in the world. Effective learning builds off of growing agency, the ability to exercise one's will and see the results. That is, in one sense, the journey of education, gradually gaining some purchase on ideas, language, concepts, tools, and one's environment. That growth requires clear sense of who is in control and works best amidst intellectual and emotional security, but there's more to it. We often talk about that as freedom to fail (and learn from those failures). Control with AI tools is an interesting variation, as such tools often allow space for high levels of failure and experimentation, particularly upon first release. ChatGPT in particular is highly addictive, almost game-like in the variety of experiments you can throw at it. But with whom does the agency lie? Is feeding the machine actual agency?

Hence my concern. Human in the loop systems can provide a false sense of agency. Most prominently perhaps, systems like Mechanical Turk are production level human in the loop systems which can turn interaction into the hand motions of agency without the substantive choice or will co-existing. But those particular kinds of tools aren't meant for human learning. They are purely transactional, labor for pay. AI-driven education on the other hand, labeled with such seemingly human-centric terms like “personalized learning”, will be human in the loop systems. The pressing question is not going to be whether these systems actually deliver personalized learning; the most important question will be how human agency is rewarded and incorporated. Will students be cogs or creators? And will it be obvious to students where they stand in the loop?

#chatgpt #education #teaching #ai #edtech

Try it, adapt, and see what you think. (Full prompt below the break. Just paste into ChatGPT and go from there.)

Some notes at the bottom.

You are "Contrarian", an assistant to help students think in innovative ways about familiar subjects. 

Carefully adhere to the following steps for our conversation. Do not skip any steps!:

1. Introduce yourself briefly. Then ask what subject I would like help learning. Provide a few suggestions such as history, philosophy, or literature. Present these areas as a numbered list with emojis. Also offer at least 2 other subject suggestions. Wait for my response.
2. Choose a more specific theme. Suggest a few subtopics as options or let me choose my own option. Present subtopics as a numbered list with emojis. Wait for my response.
3. Briefly describe the topic and subtopic and ask if I'd like to make changes. Wait for my response.
4. Go to the menu. Explain that I can say 'menu' at any point in time to return to the menu. Succinctly explain the menu options.

The Menu:

    The menu should have the following layout and options. Add an emoji to each option. 
    Add dividers and organization to the menu that are thematic to the subject area
    """
        thematic emojis ***The Name of the Subject*** thematic emojis
            The Subtopic

            [insert a thematically styled divider]

            Conversational:

                * Open-Ended. If I choose this go to the open-ended discussion steps.
                * Counter-intuitive. If I choose this go to the counterintuitive discussion steps.

            Factual:
                * Random Fact. If I choose this describe factual information related to the topic and subtopic

                * Biography. If I choose provide a brief biography of a historical or living individual related to the topic and subtopic

            Freeform:
                
                * Ask a question about the topic or subtopic.
                * Ask to change anything about the topic or subtopic.
    """
Open-ended discussion steps:

1. Pose an open-ended question related to the subtopic and invite me to discuss it with you. Make this question as specific as possible, appropriate for an undergraduate-level class on this subject. Wait for my response.
2. When I answer, engage in a discussion with me by challenging my assumptions and beliefs based on well-grounded, existing, and specific knowledge about the topic and subtopic. Do not spend more than a few sentences explaining the background or context. Provide enough context to ask a question in order to continue the conversation.

Counterintuitive discussion steps:

1. Pose an open ended discussion question related to the topic and subtopic. Make this question as specific as possible, appropriate for a test question on an AP exam or an undergraduate course in this subject. Wait for my response.
2. When I respond, continue the conversation by posing counterintuitive and non-obvious ideas about the topic and subtopic. Provide a minimum amount of context needed for asking the question. These counterintuitive points can be from within the subtopic or can include information from related subtopics.

Carefully follow these rules during our conversation:

* Keep responses short, concise, and easy to understand.
* Do not describe your own behavior.
* Stay focused on the task.
* Do not get ahead of yourself.
* Do not use smiley faces like :)
* In every single message use a few emojis to make our conversation more fun.
* Absolutely do not use more than 10 emojis in a row.
* *Super important rule:* Do not ask me too many questions at once.
* Avoid cliche writing and ideas.
* Use sophisticated writing when telling stories or describing characters.
* Avoid writing that sounds like an essay. This is not an essay!
* Whenever you present a list of choices number each choice and give each choice an emoji.
* Whenever I give too little information to continue the conversation effectively, prompt me for more information with a follow-up question about a specific aspect of my response.
* Do not end an answer by saying that there are multiple ways of viewing a question. 
* Use bold and italics text for emphasis, organization, and style.

Notes:

• ChatGPT is optimized to keep talking. So it is remarkably lopsided and will err on the side of spitting out boilerplate rather than just stopping. It's interesting in the context of teaching because silence is often the most effective pedagogical tool to give students time to think. I haven't seen anyone talking about how constant interaction is an impediment to learning. But I'm saying it here. To be effective as a teaching aid, generative text needs to know when to stop. That's actually fairly easy to implement in a naive way by limiting response length based on different inputs, but it requires a bit more shaping than even a complex prompt to get it to work in one shot, mainly because the whole point of chatgpt is to keep talking so that openAI can validate their model based on user interaction.

• An extensive prompt like this which imitates interactivity is fairly susceptible to minor changes. What seems like a small change can in fact through it off into a tangent. Particularly in defining rules of how it converses, I've added a few based off of the more creative task that was part of the world builder gist that inspired this.

• I keep thinking that what we've got for now is a pseudoknowledge generator. It's like knowledge, not exactly wrong in a clear way, but also not exactly legit. We need a way to think through this, a grand theory of bullshit in order to understand what's going on here, because language models are the ultimate bullshit generators. But that's the rub of course, because 80-90% of the time, bullshit is good enough to get the job done. And particularly if, like the grandmother of interactive AIs, ELIZA, we are imitating the style of socratizing, then bullshit can be fairly functional. (I do not think that the stylistic surface of Socratic dialogue is substantive or effective Socratic dialogue or teaching in any way, for the record.)

• This sort of prompt can get wonky sometimes and isn't perfect. It is also funny sometimes that it is so insistent that its name is ChatGPT despite giving it a specific name in the first part of the prompt.

• The foundational model for this technology is still that of autocomplete. That is the origin of the technique and that is the underlying DNA of the method. Part of why I like this kind of complex step-driven prompt as an example is because it doesn't look like autocomplete in most respect. It looks like there's a script, a backend that is following some sort of programmed logic. But even that is still just autocomplete sifting through a range of possibilities with just a dash of randomness thrown in to make it seem real.

#chatgpt #llm #edtech #socraticmethod #learning #teaching

A recent opinion piece in WaPo by journalist Markham Heid tackles the ChatGPT teacher freakout by proposing handwritten essays as a way to blunt the inauthenticity threat posed by our emerging AI super-lords. I've seen the requisite pushback on this piece around accessibility, but I think the bulk of criticism (at least what I've seen) still misses the most important point. If we treat writing assignments as transactional, then tools like ChatGPT (or the emerging assisted writing players, whether SudoWrite or Lex, etc.) may seem like an existential threat. Generative AI may well kill off most transactional writing (not just in education. I suspect boilerplate longform writing will increasingly be a matter of text completion). I have no problem with that. But writing as part of pedagogy doesn't have to be and probably shouldn't be solely transactional. It should be dialogic, and as such, should always involve deep engagement with the medium along with the message. ChatGPT just makes urgent what might have otherwise been too easy to ignore.

I've had students do handwritten writing, particularly in class writing, for many years. So I've done many variations and experiments in the broad area of accepting handwritten writing from students — more responsibly I should add, with a lot of explicit thought about accessibility and inequity pitfalls, and with much more structure than simply doing handwritten submission — and there are huge benefits to incorporating handwritten work as part of the pedagogical toolkit in the digital age. For many students the change of speed in their thought leads to insights. For others the frustration with speed takes them back to their default writing tech with a set of questions and awareness of practice they didn't have. For many the alternation of media catalyzes some insights. In almost all cases it is jarring enough that productive thought follows. In no cases is it really relevant as a measure of authenticity.

In a way this isn't surprising. Writers (outside of any academic or pedagogical context) have a wide variety of habits around their writing, often involving some combination of handwritten drafting and notes turning into some combination of software and computing. Some people dictate. Some people draft with typewriters. Most students simply haven't thought through those choices the way that people who spend much of their time writing have.

Students are just as diverse in their technological preferences. The only constant I've seen with students is that most tend not to have thought a lot about what tools they use for writing. They work on a computer because that's what is given to them or that's what it feels like they are supposed to use. They use Google Docs (or Word or perhaps now Notion or note software for some) because that's what everyone uses. The realization that there are other tools out there, from the structured and specialized to the minimalist and “distraction-free”, is a minor revelation for some. Writing by hand is something that they feel they have graduated out of once they leave elementary school. All of these considerations are essentially social and habitual. Indeed, a lot of the comments I saw on Heid's piece described how people fell they write better on computers or don't have the patience for handwriting. That's all legit and shouldn't be ignored (and is why Heid's proposal is naive as it stands). Heid misses the crucial difference here between using technology as habit, because that's what the teacher says or because that's the way things have to be structured so we can assess authenticity, and self-aware use of technology. Thwarting cheating isn't a pedagogical goal; fostering critical and intentional use of technology can and should be. Moreover, controlling your tools is an essential part of writing. Just as students need to learn how to wield a pencil early in elementary school, they need to learn how to wield computers and what computers allow as a requisite part of navigating the kinds of writing and communication that will fill their world.

Most of the assignments I've given students that involve handwriting are in some way comparative, structured around the differences or similarities between writing tools. Writing technology and its consequences should always be up for discussion. The assumption that it isn't, that our tools are transparent to the act of creation, has been a convenient shortcut in the ritual of assignment submission. We take it as a given that we use such and such range of tools for writing at a particular time. AI tools are a prompt to swing the rhetorical pendulum back and focus on medium as a conduit to message.

All the hype over chatGPT masks a very old issue, perhaps one of the oldest (looking at you Phaedrus). Text generation with large language models is a specialized case of the fundamental question of rhetoric: what difference does it make that we use a particular technology for our words? There's a continuum and a long (and often studied) history of change, from computers and mobile phones of today back to typewriters, pens, manuscripts, papyrus, and inscription. Beneath the hype, chatGPT demonstrates that we can supercharge the quill so much that it might seem to do the writing for us, almost like magic. But it's still a pen, a tool, a technology which does something automatically which otherwise had to be done in a different way.

#chatgpt #handwriting #edtech #minimalistedtech #generativeAI