Abstract
In 2017 Ocean Networks Canada (ONC), a research infrastructure operator,
sought to redefine its core reporting metrics. We asked, “which metrics
should we hold as key, essential metrics to drive our organizational
priorities and decision making?” This question helped us define a
collection of eleven sets of yardsticks, some inward-looking, others
squarely focused on societal outcomes. Here, we introduce the individual
metrics adopted, insights they are helping us glean and some of their
inherent challenges. ONC’s core funding agency, the Canada Foundation
for Innovation (CFI), continues to emphasize scientific output as a
primary criterion. We measure this by counting peer-reviewed
presentations and publications resulting from use the facility and ONC’s
data archives. But this seemingly clear-cut metric has been a thorny one
to define, track and grow. Training and support for post-secondary
students is another core reporting metric, however this measurement is
also fraught with ambiguities. Some of the easier metrics to track are
those specifically related to facility operations, such as reliability
and user satisfaction. But we were perplexed by the question of how to
measure “optimal use” of the facility, as mandated by CFI. Optimal use
is hard to define for an underwater infrastructure design like ONC’s,
which can be flexibly extended with no hard limits on hardware capacity,
archive volume or data access. When it comes to societal benefit, our
approach has been twofold. One set of metrics examines technology
transfer, grants and contracts. Another set focuses on our engagements
and active collaborations with governmental, indigenous and
non-governmental organizations. However, some outcomes remain
challenging to measure. While it is straightforward to count up our
external interactions and collaborations, how can we quantify their
current and future societal impact? These and related questions will be
explored.