Sometimes a source really is too good to be true!
Depending on the Generative AI, it might be telling the user what it thinks they want to hear. ChatGPT and other generative AIs, as of June 2023, do make up citations that don't exist. They might seem like they could be real (the author might even be a real person who writes about that topic a lot, the publisher might exist, the journal or periodical might focus on publishing articles of that topic) but the overall source/citation is not real. This is because many AIs are not smart enough to present real sources, but rather string together likely word combinations found in patterns of text engineers have fed it with. Some AIs are not connected to the internet, so cannot present you with links to real sources.
Here's an example from Google Bard's Generative AI from 2023. All of these book recommendations (citations) are made up. The first book does exist, but it's written by a different author. A quick Google search for the titles helped us verify what was real:
While a lot of generative AIs are not search engines, Perplexity AI combines a language model with a search engine and provides link to sources where it is generating answers from, making it easier to fact-check.
Just like when using Wikipedia for research, we need to use this AI tools wisely. We recommend the SIFT Method, which involves lateral reading, for evaluating a citations from an AI. If you can't find any mention of the exact source using a search engine, that should be a red flag. Made up sources have real-world implications and consequences if not verified. Take this news story, for example:
We also recommend that faculty take measures to verify student sources. College & Research Libraries recommends requiring students to submit full-text copies of cited sources, for example.
It's better to use AI like ChatGPT for tasks like these:
- Brainstorming for keywords or phrasings to use in your research or writing
- Gleaning ideas about your topic you may not have thought about
- Interpreting or summarizing concepts or other works you might be struggling to understand on your own