Generative AI and assessment

The following options could make it more difficult for AI technologies to produce satisfactory results, without requiring significant redesign:

• Ask students to use sources that require an institutional subscription within the assignment (ChatGPT does not currently have access to these).
• Modify questions to be more applied to each student’s context (e.g. “discuss the relevance of this topic to your own, local situation”).
• Require staged submission of an outline, research notes, drafts and final version
• Use more iterative processes of assessment such as student peer review which leads to revisions of the work.

Use of technologies to produce text and other media as part of student submissions (or, indeed, as part of the process of developing such submissions) needs to be thoughtfully supported and/or limited, and clearly and openly acknowledged. Some examples of building AI tools into assessment design are given below:

• Allow students to use AI tools but require that they provide the prompt that they used and show via Track Changes (and comments) which changes they made to improve the output and why.
• Generate AI content and provide it to students for them to evaluate (with predefined criteria) and revise.

Some formats are more difficult for AI tools to generate than others, with text currently the simplest. Similarly, some genres of writing are more difficult for AI technologies. Some ideas for tasks based onalternative formats:

• Ask students to create oral presentations, videos, multimedia resources.
• Ask students to incorporate more personal reflections tied to unit concepts.
• Ask students to take part in live interviews (face to face or online). An interview or ‘viva voce’ can test a student’s understanding, gauging their ability to respond to questions and prompts in conversation. They can also be asked to elaborate or make connections between their responses to different questions. Bear in mind that live conversations are more difficult and anxiety-provoking for some students than others (although this is also true for other formats such as essays and exams).
• 关于assessm的更多信息ent types and formats, seeChoosing assessment tasks.

Rather than only assessing the final output or “product” of students’ work, it is possible to also assess the process through which they produce it. For example:

• In mathematics, students may be required to “show their working”, and the way they solve a problem can be more important than whether or not they arrive at a satisfactory answer.
• In art or architecture, sketches can be compiled into a folio to show stages of the journey towards producing a finished piece. Each item of evidence of the student’s journey can be annotated or can form the basis of reflective writing, generating further insight into how the student was thinking about their work and making connections between ideas.

In this way, assessors can see a continuity of development that is specific to each student and makes sense as a whole. It may be possible to include the use of AI technologies as part of this process (as long as it is appropriately acknowledged and reflected on) but the primary goal is to engage with the way the student thinks about the unit and their work.

Similarly, process-oriented assessment approaches can also include collaboration with other students, as long as the submissions are structured to clearly indicate each individual’s contribution and insights. For more on this, see our Teach HQ resources ongroup and peer assessmentandcollaborative learning和MEA模块Assessing Teamwork.

An additional benefit is that these approaches to assessment can promote a culture of valuing the process of learning above the outcome (e.g. grades). It may create space for students to take risks and learn from their mistakes.

In designing an assessment, choices are made (whether explicitly or not) about which forms of knowledge are valued. These choices are informed not only by the goals of the educator and learning outcomes in a unit, but also by practical considerations such as the size of the cohort, how much time there is or what technologies and systems are available, accreditation requirements, etc. Often, it has been simpler and more efficient to focus on simpler assessment formats such as multiple choice exams and standardised essay questions that produce clear and familiar responses.

However, AI technologies like ChatGPT are reshaping this landscape because of their capacity to generate satisfactory responses to those same kinds of questions. In response, it may now be necessary to target forms of knowledge and expression that are more difficult for generative AI technologies - critical thinking, evaluation or creativity, for example.

The table below shows example assessments that arecurrentlymore or less difficult for ChatGPT (many of these examples are also applicable to other generative AI tools).

Difficulty for AI technologies	Task	Suggestion
	Closed answers (e.g. MCQ or short answer exam questions that ask students to define, list or reproduce)	ChatGPT is good at synthesising information to produce convincing answers to closed or abstract questions. Where feasible, consider alternative formats that align with unit learning outcomes.
	Essays without personalised application	ChatGPT can produce convincing text in many different genres and styles as requested (although it may find accurate referencing difficult). Modify questions to require personalised application. Combine with modifications such as assessing process or personal reflection.
	Essays with personalised application	ChatGPT is poor at some kinds of personalised application or contextualisation of ideas. However, students may also find this type of writing difficult and scaffolding this kind of writing is important. Combine with modifications such as assessing process or personal reflection and scaffold students’ abilities to write in this way.
	Showing individual working process	ChatGPT does not reveal how it produces its outputs, so it cannot produce an account of its own process. Replicating that would be significant work for students. Consider introducing staged submissions where students explicitly respond to feedback on drafts, or where they are asked to submit a portfolio of the work they have done on the way to their final submitted work.
	Showing collaborative working process	Producing a misleading account of how students worked together to produce an output would involve significant work. Consider group tasks (with or without a final, individual submission) where students are asked to reflect on the process of completing the work and what they have learned from it.
	Multimodal artefacts (e.g. a document that contains text + images) as creative response.	ChatGPT currently only produces text. This text could be combined with other AI tools (e.g. DALL-E) but in many cases this requires thoughtful engagement with the material to compose something that is more than the sum of its parts. Consider asking students to produce images (e.g. an infographic) or video recordings as part or all of a response to a question. These should not just be reproducing text in a different format (e.g. a recording of reading text aloud) but should make use of the format as an alternative way of expressing ideas. Note:Multimodal work is often more ambiguous than text and may require more marker training and time. It may also require more technical support.
	Interview / viva voce	目前不可能的repl的人工智能技术icate. Consider asking students to participate in a live (synchronous) conversation (on campus or online) about what they have learned or about their submitted work. Note:This may require scheduling for each student, and may involve a greater time commitment for marking.

While targeting higher order thinking may reduce the risk of misuse of generative AI tools, it is not sufficient to only consider the kind of thinking required to produce a response to an assessment task (e.g. by targeting higher order categories ofBloom's taxonomy). For example, in some circumstances, it is possible for ChatGPT to represent or imitate each of these categories of learning (e.g. it can create new ideas by synthesising unusual combinations of information sources). However, there are other assessment characteristics that can be associated with higher and lower risk of inappropriate use of generative AI. The following diagrams illustrate some of these characteristics in relation to risk.

Click on the plus icons to learn more about the definition of each term.

These diagrams are a starting point for considering modifications to an assessment. They are primarily based on consideration of ChatGPT but many of these ideas are applicable to a broader range of AI technologies. Each aspect requires consideration in relation to the aims and learning outcomes of the unit. See also the Teach HQ page on “Choose the right assessment task”.

In many higher education models, high stakes, heavily weighted summative assessment is the primary source of data about student performance and progress. Where the evidence used as a basis for judging student performance has been partly or wholly generated by technologies, this can lead to misrepresentation of student capability. Adopting a varied approach in designing an assessment regime, where data about student performance is gathered fromformative and diagnostic tasks, can help to create a strong relationship with students, motivate them in their learning and provide information about whether summative assessment performances are consistent with other work.

While generative AI tools pose a significant challenge for some assessment formats when used for high stakes, summativeassessment of learning(e.g. end of unit essays, etc.), they may be less problematic for alternative formats and lower stakes, formativeassessment for learning.

Selecting a range of tasks with varied weighting can allow students to demonstrate learning in a lower-risk environment, giving you the opportunity to gauge your students' understanding while also lowering the incentive for them to rely on tools to generate responses.

Formative assessment that isn’t weighted also allows for opportunities to check understanding and build a picture of student capability. Where formative assessment helps students prepare for summative assessment, it may also increase their confidence to engage authentically in those summative assessment tasks.

Programmatic assessmentdraws upon the notion of assessmentforlearning. Programmatic assessment involves a series of interconnected assessment methods that build upon each other. Rather than focus on assessing individual kinds of knowledge or skills using a single assessment method, it focuses on assessing broader attributes using multiple methods.

Programmatic assessment supports a holistic assessment frame that is most likely to assure assessment tasks so that they are diverse, authentic, provide relevant evidence of student learning and minimise risk of inappropriate use of generative AI technologies.金博宝188欢迎你indicates that programmatic assessment has many benefits for students and educators, including increased confidence in assessment decisions, reduced emotional burden and early identification of at-risk students.

Authentic assessment focuses on assessing student learning against learning outcomes aligned with “real-world” contexts and industry expectations (including, but not limited to. work-integrated learning). Authentic assessment tasks require students to apply knowledge and skills in meaningful ways. Designing authentic assessments involves not just focusing on authentic tasks but also on educator judgement of the relevance of the tasks and how they can contribute to students’ sense of belonging to a profession or society. Examples of authentic assessment can include case studies, exhibitions, reflective portfolios, and problem-based inquiries. As well as creating opportunities for valuable and relevant experiences,金博宝188欢迎你also supports the role of authentic assessment (when done well) in improving engagement and preventing students from engaging in academic misconduct.

Future-focused or “sustainable” assessmentadvocates assessing the knowledge, skills, and attitudes that students would need throughout their lives beyond graduation. This kind of assessment focuses on developing the ways in which students learn to contribute to their capacities as lifelong learners. A key principle of future-focused assessment is to develop students’ capabilities to make evaluative judgments about what constitutes good work. Some examples of future-focused assessment tasks across a range of disciplines can be found onthe Monash University Be Inspired pages.

Each of these assessment elements has the potential to reduce the risk of breaches of academic integrity in relation to generative AI technologies:

• By incorporating multiple assessment formats and targeting a range of different kinds of knowledge, programmatic assessment has the potential to capture learning that cannot be replicated by tools like ChatGPT.
• 学生不太可能参与学术管理信息系统conduct when they find assessment tasks authentic, personally relevant and more readily applicable to real-world scenarios
• Future-focused assessment can help students understand how they might learn to work with new technologies like ChatGPT in ways that contribute to their personal and professional development over time.