On the Frontlines: The Potential, and Perils, of ChatGPT

As it turns out, ChatGPT has great potential to assist internal auditors in most stages of their work — from the risk assessment, through the planning of an engagement, to the report writing. The key limitations and risks in using Chat GPT are the concern over data confidentiality and the risk of incorrect information provided by the bot. With these in mind, let’s explore the possibilities of using ChatGPT in internal auditing and how it can help auditors maximize efficiency in their role.

Periodic Planning and Risk Assessment

During the planning and risk assessment stage, ChatGPT could help auditors with creating or fine-tuning their risk assessment methodology and with ideas on ways to perform the risk assessment. Additionally, it can provide ideas on how to measure or quantify specific types of risks. However, when it comes to the content of the risk assessment and rating, it is essential to note that these tasks are still organization-specific and should be done without relying on ChatGPT, as there is no guarantee that the information provided is correct. If we want (and we definitely do!) to avoid feeding ChatGPT with any confidential information, the use of the bot for assessing the risks of the organization is quite limited.

Planning the Engagement

ChatGPT can be used during the planning of the engagement stage to inquire about possible risks, controls, and what could go wrong in the audited process in general. It also can provide suggestions about possible effective controls and best practices and can assist in drafting an audit program. The first version you get will probably be generic, but an iterative approach with sub-questions could help.

For instance, instead of asking Chat GPT to “draft an audit program for XY process,” we should start with feeding the tool with the context (size and type of the organization, maturity of the process, key elements, etc.). Then we can lead it through the same thinking process we go through when creating the audit program:

• What are the objectives of the XY process?
• What are the sub-processes of the XY process?
• What are the key risks in sub-processes 1, 2, and 3?

The key is to understand that while Chat GPT can draft the audit program, the best quality output can be reached by a piecemeal approach — asking specific questions and challenging the responses.

Fieldwork

During the fieldwork stage, ChatGPT in its current version cannot provide much assistance. For any meaningful output, one would have to feed it with information, which is better to avoid. However, looking at the near future, the integration of AI into data analysis tools (including Excel) could be of great help. Imagine writing into the Excel cell: "Please compare these two (or more) tables and list all exceptions/differences." An AI-powered language model built into the data analytics tools would be a game-changer for users of such tools.

Reporting

ChatGPT has great potential use in writing internal audit reports due to its ability to produce short, concise, and compelling text with clear wording. However, the significant limitation here is that we should refrain from feeding ChatGPT with confidential organizational information. So, in its current version, ChatGPT could not really be used for internal audit report writing, except for polishing and summarizing the general background information (if not confidential) or used on sentence segments when you cannot find the proper word to express your thoughts. For example: “Polish this text to sound persuasive” or “Write this paragraph in one sentence.”

Audit Recommendations Follow-Up

Currently, there is no big showcase of ChatGPT usage for the audit recommendations follow-up, mainly because of data confidentiality, as mentioned above.

Learning the Tool

Even in its current form and limited by data confidentiality risk, ChatGPT can be a valuable tool in the internal audit process. If not already doing so, auditors should practice using the tool to get a flavor for its possibilities and limits, as this technology is here to stay. Use it during engagement planning as a brainstorming buddy, methodology advisor, research tool, or language expert. And for the fieldwork, reporting, and follow-up, just be patient and prepare for the future: If the confidentiality of your data is secured, the use of such an engine will grow exponentially. It will, however, never be limitless. As advanced as it might be, it will never be able to replace the thinking process of an internal auditor, loaded with experience, organizational knowledge, and the ability to form relationships, all of which involve written and unwritten rules. As the saying goes, "We will not be replaced by AI, but by people using AI." Let’s use this powerful tool to enhance our abilities and improve our outcomes.

Any views expressed in this post are personal and do not reflect the position of the UNIDO.