Post by Università Bocconi

Have you ever heard someone say they didn’t understand something, so they asked ChatGPT for clarification? Or seen someone rely on AI to make decisions, even for things like a pet’s diet? At what point did AI-generated responses start being treated as a source of truth? And more importantly: are we becoming too comfortable trusting tools that we don’t fully understand?     Research Fellow Tanise Ceron explains why the answer requires caution. #LargeLanguageModels are powerful tools that support writing, summarizing and organizing information, but relying on them as if they were authoritative sources is a risky shortcut. The issue is not simply that these systems make mistakes. It is that their errors are often difficult to detect. Research has shown that #LLMs can hallucinate references, misinterpret sources, fabricate facts and struggle with less frequent knowledge. They can also display compliance, agreeing with users even when the answer is incorrect. These limitations are structural, not occasional.    A key factor lies in the data they are trained on. Most of it comes from the internet, including blogs, forums and other content that has not been verified. Only a small share originates from high-quality sources such as books, peer-reviewed research or curated databases. As a result, models developed by different organizations often produce similar outputs, but also similar biases and inaccuracies. This is why comparing answers across multiple systems does not guarantee reliability. If one model is wrong, others are likely to replicate the same error. The apparent consensus can create a false sense of #accuracy. The distinction between tasks becomes crucial. Low-risk uses, such as paraphrasing or translation, can enhance productivity with limited exposure to misinformation. High-risk uses, such as retrieving facts, explaining concepts, or making decisions, require careful verification against authoritative sources. In these cases, the convenience of #GenerativeAI should not replace critical evaluation.    Another important aspect is how these tools reshape cognitive processes. By providing immediate answers, they reduce the need to search, compare and interpret information. This lowers cognitive effort, but it also increases dependence on outputs that have not undergone editorial or peer review. The point is not to reject these technologies. It is to recognize their limits and use them responsibly. We need a higher level of scrutiny than with traditional media, not a lower one.    Efficiency is valuable. But if it leads to unverified knowledge, the trade-off becomes problematic. The real challenge is not whether we use LLMs, but whether we develop the critical awareness needed to question them.