I continue to explore the intersection of Sherlock Holmes, innovation and compliance by starting today with the story The Adventure of Silver Blaze. In this story a valuable race horse is stolen just before the big race in which the owner has literally bet the estate to try and get himself out of debt. If Holmes and Watson cannot find the stolen horse, the owner Colonel Ross, will lose everything he owns and be forced to declare bankruptcy. The story informs today’s post on interpreting data.
Interestingly this matter turns on a clue which was not present or “curious incident of the dog in the night-time”. It was because the barn’s guard dog did not bark, from that Holmes is able to deduce it was the Stable Foreman who stole the horse, hoping to hobble it with a slight hamstringing. Holmes notes that because the dog did not bark, no stranger was present. As Holmes explains: “I had grasped the significance of the silence of the dog, for one true inference invariably suggests others…. Obviously, the midnight visitor was someone whom the dog knew well.”
I thought about the insight of the clue which was not was the clue itself when considering another innovation in compliance, which has largely bedeviled compliance practitioners. Today’s blog post is informed by two articles from the MIT Sloan Management Review, “Is Your Company Ready for HR Analytics” by Bart Baesens, Sophie De Winnie and Luc Sels and “Why Big Data Isn’t Enough” by Sen Chai and Willy Shih (‘Big Data’ article).
Obviously, data analytics can be a valuable tool for the Chief Compliance Officer (CCO) or compliance professional. Yet many wonder not only what the data might mean but what it might not mean. Another issue is how to leverage it for your key compliance customer base: your employees or, as the Department of Justice (DOJ) would say, to operationalize your compliance program. One key insight is that you must match up your data to areas not often considered by the compliance professional, the employee network dynamics. This can be as straight-forward as an international subsidiary’s employees’ loyalty which is to their local organization and not to the US corporation. This can not only help shape behaviors but can place a cohesive band around the compliance insights you might receive and try to implement.
The next area is around the concepts of big data and data analytics. These are not simply panaceas. They are certainly valuable tools, but they do not make the decisions for you. It is not simply the greater amount of the data, the more robust the insights and findings. Compliance professionals need to have some caution about the limitations and proper use of data analytics. Moreover, just as your business and operational conditions evolve and change, you must realize the data you are considering is a static shot-in-time, so the compliance professional must interpret the data based upon such factors as compliance expertise, knowledge of the problem and your organization. This interpretive role is akin to Holmes understanding the lack of an event, a dog barking, as a significant factor. This means you must apply some insight to the lack of hotlines calls beyond simply believing there is nothing untoward out there to report.
Always remember that statistical performance itself is not the goal but the insights you can draw from the data. Interpretability for the compliance professional means that any compliance decision to operationalize based upon data analytics should be properly motivated and can be explained simply to all stakeholders involved; literally from the Board Room, to senior and middle management all the way down to the front-line troops who are fully operationalizing the insight. This move towards simplicity discourages the use of overly complex analytical models that focus more on statistical performance than on proper business insight. For compliance to succeed beyond the simple legal response to laws, such as the Foreign Corrupt Practices Act (FCPA), and move into a true business enabler, this type of simplicity is required.
Finally, you must be aware of internal biases in your data and work towards testing your data insight with those who will implement the solutions. You must be aware of deviations and that they might mean nothing going forward. In the Big Data article, it stated, “A pitfall in studying large datasets with billions of observational data points is that large deviations are often more attributable to the noise than to the signal itself; searches of large datasets inevitably turn up coincidental patterns that have no predictive power.” Ben Locwin refers to this as “white noise”. In his paper entitled “Better risk-based thinking will help produce better risk-based monitoring”, he notes that for the compliance officer the issue is that “you need to know what to fix first; and this usually goes wrong in the form of companies being unable to differentiate the signal from the noise. To not do it properly leads to a lot of organizations that I’ve seen expending a tremendous amount of resource and capital on trying to fix what actually isn’t the problem.”
This leads to the next interpretive problem, that of false correlations. The Big Data article noted, “It’s important to recognize that the number of data points required for statistically significant results needs to increase as the number of variables grows. Otherwise, there will be a greater risk of false correlations.” The authors concluded, “researchers must be mindful of both sample size and sample variation.” I certainly recognize this is not an area many lawyers received training in but it points to the need for every compliance practitioner to work with their internal data resources.
Finally, compliance professionals need to be aware of the systematic biases in simple data collection. This can most easily show up when your data comes from disparate sources, if it has been collected from different technologies, at different times or simply aggregated from multiple sources. You must find a way to standardize this data so you can remove any distortions. Moreover, the average lifespan of your data model may be only two to three years. However, given the impact of compliance decisions on both the company and its employees, you will need to take feedback and loop it into your model going forward. From the theoretical approach, it is important that analytical models are constantly backtested by contrasting the predictions against reality, so that any degradation in performance can be immediately noticed and acted upon. The DOJ would call this feedback, where you take information from your designed program and loop it back into your compliance program on a regular basis.
Whether your interpretive basis is an algorithm or the well-practiced eye of a seasoned compliance professional, the continued mining of large corporate data bases for insights to improve a compliance program will continue. Sometimes the answer will present itself to you but sometimes you will need to ascertain why the dog didn’t bark by delving more deeply to come up with an appropriate solution.
[View source.]
