1. They hallucinate. They make up sources. They cite work with no relevance to the subject. Example: a paper about colonial religion referenced a book about Spurgeon, a 19th century evangelist.
2. The usually do pretty shallow analysis of texts.
3. Fundamentals of Chicago Manual of Style such as page numbers, headers, double spaced first line indented seems to be beyond many of these programs. In a sense, these minutiae do not teach either research methods or writing. They do teach attention to detail.
One approach that some universities are tsking is scaffolding: breaking the process apart into individual parts.
1. Show me the question you are going to answer..
2. How did you find sources to answer that question. AI may be a perfectly fine method of finding sources. (Are your old enough to remember Reader's Guide to Periodical Literature?)
3. Your sources: explain their relevance to this question; what do they say about the topic?
4. What are the topic sentences for each paragraph? What is the sequence of topics you are going to cover?
4. Show me your bibliography. Get it in Chicago Manual of Style. Again, using AI to do the housekeeping is quite mechanical.
6. Write the paper filling in the paragraphs for each topic sentence.
7. Show that you can do the formatting for Chicago Manual of Style: header, page numbers, paragraph formatting.
8. Submit the paper.
The student must complete each step that can often be done by AI. You can ask the student to explain how he/she/indeterminate made those decisions and provide feedback for how it is supposed to be done. The full laziness of prompting AI and doing a copy pastae into Word is eliminated. Even if every step is AI-assisted, the student learns what each step looks like and can presumably do it in a post-EMP world using dead trees sources.
Thoughts?
