Know Thyself; Know Thine LLM

• June 22, 2026
• Tadhg Patrick Lawless

The role of human judgement in AI readiness

In casual conversation with a Malta-based tech recruiter earlier this month, I learned that while many local companies sense an urgent need to onboard AI talent and to integrate AI tools into their processes in some way, many of them lack a precise understanding of why they would do so. I came away mulling over the recent experience of many candidates, that hiring managers are being mandated to do what it takes to get their companies AI-enabled as quickly as possible, without those organisations first having established a clear AI strategy or a roadmap towards making meaningful gains from their use of AI.

A satisfyingly congruent finding is illustrated by Cisco in its recent AI Readiness Index report, where researchers found nearly 70% of organisations ranking AI as a top IT budget priority, with just 34% feeling very confident in their ability to effectively monetise their AI use cases.

Further, in a report on AI Adoption by Small and Medium-Sized Enterprises, OECD analysts observed that SMEs consistently cite a lack of skills as a major impediment to AI adoption, with over 50% reporting that their employees lack the necessary aptitudes to use generative AI in a way which would bring material benefits to the organisation.

the old Delphic motto “Know thyself” stands forth, now hand-in-hand with a new variant “Know thine LLM”

The information paints an overall picture in which organisations face a twin-challenge of, on one hand reliably identifying how AI can add value to their business, and on the other ensuring their workers possess the domain-knowledge and skills needed to execute.

For organisations seeking to have their employees make use of generative AI to increase productivity, maximising the ability of those teams to work effectively with AI tools may add far more value than expected, in comparison to the potential gains from choosing the right strategy and toolkit alone.

The famous Dunning-Kruger paper Unskilled and Unaware of It reinforced the ancient understanding that true knowledge is closely associated with clear judgement of one’s own ability to know. Kruger and Dunning were able to demonstrate that those who are unskilled in a particular domain tend to lack the metacognitive ability to realise how unskilled they are. To paraphrase the enduring wisdom of Confucius: to know when you know, and when you do not know – this is true knowledge.

When it comes to using AI well, the same principle holds. In a 2024 research paper, Andrew Caplin and his team demonstrated that the workers who gained the most value from the use of AI tools were those who exhibited the most accurate assessment of their own knowledge and judgement. They showed that workers who lacked confidence in their own judgement were more likely to defer to the judgement of an AI and thereby commit Type I errors (false positives/accepting a false signal) when the AI was wrong about something. Meanwhile, workers who were excessively confident in their own judgement were less likely to defer to an AI, thereby committing Type II errors (false negative/rejecting a true signal) when the AI had judged correctly.

It may be fair to say that this does not only apply to the user’s self-beliefs around the knowledge domain they are querying with the use of AI. Rather, a key consideration in this regard would be the accuracy of their beliefs around what AI itself is, what it is capable of and what it is not capable of. Thus, the user finds cause to apply Confucius’ maxim twice over: first to themselves, then to the AI tool. They must reflect on how likely they are to be correct in their judgement, while also forming a view of how likely the AI is understand the context, take into account influencing factors and to provide reliable output on the topic.

One key to mitigating the risk of these judgement-errors, Caplin and his team proposed, lay in calibration training for employees (training to increase the rate at which their confidence level matches the objective quality of their judgements) – especially where the costs of direct upskilling may be too prohibitive or time-consuming for an organisation to commit to.

In practice, calibration training for specific activities can be designed on a case-by-case or team-by-team basis. However, one thing we can calibrate for at a larger scale is the understanding of AI technology itself. This alone may yield a considerable return on AI (ROAI) by increasing the effectiveness with which the average employee can leverage the use of any generative AI tool.

workers who gained the most value from the use of AI tools were those who exhibited the most accurate assessment of their own knowledge and judgement

A current example of what could be described as a very large-scale calibration drill would be Malta’s recently launched AI for All programme. Designed to impart foundational AI literacy to an entire population, this is an example of a rapid AI literacy-scaling solution implemented with the future in mind. In a matter of weeks after opening, AI for All had attracted tens of thousands of Maltese and Malta-resident learners, providing a boost to national AI literacy which could have structural impacts on Malta’s talent pool and competitiveness well through 2030 and beyond.

Assuming that a sufficient proportion of the population engages with the programme, Malta may see it increase the baseline “AI readiness” of her average SME, purely by sharpening the ability of the average employee to make profitable use of AI technology in general and reducing the likelihood of Type I and II judgement errors concerning outputs.

For the more than half of SMEs who see their employees lacking the expertise to gainfully use AI tools in the workplace, this evolving landscape may come to shape the behaviour of those hiring managers who are seeking to lock-in AI expertise as quickly as possible. In their search, the old Delphic motto “Know thyself” stands forth, now hand-in-hand with a new variant “Know thine LLM”. Candidates who exemplify both of these standards may find themselves ahead of the competition.

References and Sources

Academic Research

Kruger, J., & Dunning, D. (1999). Unskilled and Unaware of It: How Difficulties in Recognising One’s Own Incompetence Lead to Inflated Self-Assessments. Journal of Personality and Social Psychology, 77(6), 1121–1134.

Caplin, A., Deming, D. J., Li, S., Martin, D. J., Marx, P., Weidmann, B., & Ye, K. J. (2024). The ABC’s of Who Benefits from Working with AI: Ability, Beliefs, and Calibration. NBER Working Paper No. 33021. National Bureau of Economic Research.

Shore, A., Tiwari, M., Tandon, P., & Foropon, C. (2024). Building Entrepreneurial Resilience During Crisis Using Generative AI: An Empirical Study on SMEs. Technovation, 135, 103063.

Aristidou, A., & Langer, C. (2025). AI and the Economy: A Review of the Evidence and the SME Research Gap. Stanford Digital Economy Lab.

International Organisations and Industry Research

Organisation for Economic Co-operation and Development (OECD). (2025). AI Adoption by Small and Medium-Sized Enterprises.

Cisco Systems. (2025). Cisco AI Readiness Index 2025: Realizing the Value of AI.

IBM. (2026). The Biggest AI Adoption Challenges for 2026.

The Register. (2025). Cisco: Most Companies Don’t Know What They’re Doing with AI.

National Initiatives and Programme Sources

Government of Malta. AI4ALL Malta.

Other

Confucius, Analects 2:17