Key performance indicators for deployed AI systems are established to measure their effectiveness.
Tier II
Optional extra depth on top of the baseline. Organizations with more mature AI governance, or more exposure, may choose to adopt these for more thorough control over AI than the baseline alone provides.
Why
AI systems are almost always adopted with the expectation that they will increase productivity, output, and automation of repetitive tasks. However, such benefits are rarely tracked and measured. Without such measurements, the actual benefits of deployed AI systems are hard to gauge against the costs of these systems.
How
The organization should formulate KPIs that compare the AI system against the situation in which it is not used. Such measurements should include the time spent checking and fixing AI output. Time spent setting up and fine-tuning the AI system should also be considered. Measurements should include the total cost of the AI system (or the percentage of purchased usage/tokens).
Sources
- NIST AI RMF MANAGE 1.1
- NIST AI RMF MAP 1.4