By Author Tom Wilson
This is part 2 of a 4-part primer on operational analysis. In part 1 ([WIL14B]), I introduced operational analysis and
focused on applying it to evaluation at the system level. In this part, I will discuss evalation at the resource level. Part 3
will describe using operational analysis for modeling. Part 4 will investigate some other topics that are not found in the
literature.
Figure 1 illustrates how a “system” can be viewed from several levels of detail. At the lowest level, a single resource
can be analyzed without its queue; this is the purple server of resource 2. At the next level, a resource can be analyzed
with its queue; this is the yellow box around resource 2. At another level, the entire computing system can be analyzed;
this is the grey box. At the highest level, the users of the system can be analyzed along with the computing system; this
is the brown box. Other levels of detail are possible, such as collections of resources (e.g., resources 2-4).
Benchmarking AI models from an ethical angle involves ensuring that the evaluation processes promote fairness,...
Find out more