Introduction
In any Quality Management System (QMS), ensuring that the QA Policy document is both compliant and comprehensive becomes one of the most important requirements. This has driven the exploration of innovative technologies, including the use of AI for streamlining the compliance assessment of a QMS. The article outlines the practical implementation of an AI-powered tool designed to analyze QA Policy document(s) against predefined criteria, a concept that was recently tested on the NATO QA Hub. The document analyzer is available only to the users on the QA Hub assigned with ETF profile manager role. For the rest of the user this resource will not be accessible. The process is fairly straightforward: upload a document, let the AI analyze it, and obtain a comprehensive assessment within a minute. Below, we walk you through the process and discuss its potential benefits and limitations.
Step 0: Navigate to the document analyzer
Click the button highlighted in the screenshot below.
Step 1: Uploading the QA Policy Document
Users begin by selecting their QA Policy document in PDF format. The platform currently supports files up to 2MB in size. Once the document is uploaded, it is converted into plain text, making it easier for the AI to parse and analyze it. Please, do not upload policies bigger than 40-50 pages. To be efficient and fit into the limits, just keep the main body and exclude the annexes. If the processes detailed in the annexes are briefly described in the policy main body, the algorithm will determine that the standards are met without further analyzing the annexes. This would be redundant and a waste of resources. Again, the tool is meant to offer an overview and not a deep dive assessment. We are not there, yet.
Step 2: AI-Powered Analysis
Once uploaded, the AI leverages GPT-4 technology to analyze the document against 34 predefined Quality Assurance criteria. You can find the list [here]. This script has been tailored to understand the unique structure of NATO accredited ETF`s QA Policies, quickly identifying areas of compliance and those requiring attention.
Step 3: Results and Insights
In less than a minute, the AI generates a detailed report. This report highlights the areas where the policy meets the required standards and flags sections that may need further refinement. The results provide an initial overview for the QA Managers, allowing them to early identify potential generic non-conformities and focus their efforts further if needed.
Potential Applications
QA Teams of Experts (ToE): This tool can be a significant time-saver by providing an initial assessment of QA policy documents. It allows the team to quickly gauge the document's strengths and weaknesses before conducting a more in-depth review.
QA Managers: For QA Managers, the tool acts as a preliminary check, ensuring that their policies are aligned with NATO's quality standards` requirements before formal submission or review, while also maintaining the health of their respective QMS.
Limitations and Considerations
The AI tool's main limitation is its cost. Analyzing documents, especially larger ones, can become expensive, approximately $2-$3.5 per document, subject to the number of pages. With around 43 QA Managers registered on the platform, even minimal usage could lead to substantial costs. To mitigate this, a single-use policy for registered QA Managers has been implemented. While the tool is not intended for everyday use, it can be useful for quick checks before significant milestones, such as Institutional Accreditation. For the moment the limit will be set to one time usage per year. However, this rule is not set in stone as we try to remain flexible as much as possible. To submit out-of-cycle requests, please email the QA Hub admin with the appropriate justification.
Additionally, the results may not be 100% accurate for every document, as this is tool is still in the development phase (minimum viable product). We are working to optimize it further and your feed-back is more than welcomed in the dedicated section below.
End result generic example
Conclusion
While the AI tool is still in its early stages, it represents a significant step forward in leveraging technology for NATO Quality Assurance processes. By making this tool available on the NATO QA Hub, we aim to provide QA Managers with a useful resource that enhances efficiency and ensures compliance. As the technology evolves and becomes more cost-effective, we anticipate broader adoption and more innovative applications, potentially transforming the QA business.
Bonus:
The source code used for this application is free and it can be modified to suit slightly different ideas related to AI text analysis. If you are tech-savvy enough, please feel free to take a look into the code or download it.
Should you have any questions, please do not hesitate to use the comment form below.