By Ron Dolin, Research Fellow, Stanford Law School
I was at a gathering a while back that happened to include the General Counsel of a Fortune 100 company. I asked him if he measured ROI on his legal spend. "No,” he said, “I can’t.” Why not? "I can’t measure quality.”
I suspect, though, that he meant “I can’t measure quality yet.” ROI means measuring return, and return requires estimating value. Value is defined as the level of quality in the context of its cost. Therefore, measuring quality is key.
One way for corporate counsel to understand how to do this is to study engineers, who look at quality through standardized metrics and benchmarks: For instance, when it comes to hard drives and memory, the CPU FLoating-point Operations Per Second (FLOPS), or Mean Time To Failure (MTTF), allow for an apples-to-apples comparison of components. The problem is that when different suppliers use different quality metrics, there is simply no straightforward way to compare components. And that’s the dilemma that the legal profession faces.
Standardized quality metrics would enable corporate counsel to create a value curve for legal services, so they are able to judge what constitutes an appropriate level of quality within a specified cost framework. Given the market changes happening in legal services and the shift from a producer-driven market to a consumer-driven market, it’s up to corporate counsel to push the use of standardized quality metrics forward.
Whether we use some form of peer evaluation mechanism with statistical sampling, or an enhanced machine learning framework that correlates aspects of work product with some notion of success, moving the legal system forward requires, in part, the use of measurements.
Many lawyers may worry that as we increase the role of automation in the legal system, the new efficiencies gained will be offset by decreases in quality. In fact, quality metrics and automated evaluation are the best way to guarantee that the high quality of lawyering does not drop off.
This is because in order to truly understand the quality of lawyering, we need to apply quantitative techniques, machine learning, and empirical analysis. Once there are standard quality metrics in law, we can unlock the efficiency gains that technology has provided in other industries.
To read a longer version of this article on Dolin’s blog, click here .