›Business Intelligence Advisors' software is having success finding red flags in executives' behavior on earnings calls
SATURDAY, MARCH 5, 2011 By VITO J. RACANELLI
Last month's smashing victory by IBM's advanced supercomputer Watson over two human champs on the TV game show Jeopardy! might have some portfolio managers worried about a looming Terminator-like risk to their jobs.
Putting that aside, however, what if a computer attempted to rate the quality and reliability of corporate managers' responses on conference calls? Could a Watson or a Terminatrix T-X help a portfolio manager by combing through thousands of company earnings calls in search of encouraging—and alarming—signs?
Sounds implausible, but something like that has been developed by Business Intelligence Advisors, a Boston-based outfit that draws on staffers with Central Intelligence Agency experience.
Regular readers might remember an article Barron's published five years ago on BIA's training of fund managers to do behavioral analysis of responses from corporate executives: "Is Your CEO Lying?" June 26, 2006[the article is posted in #msg-11961951].
More recently, BIA has been scaling up and mechanizing what essentially had been a bespoke practice, with clients paying for training and analysis of specific conference calls or corporate presentations.
Under that system, says BIA CEO Richard Leggett, its analysts were producing some 250 conference-call reports each quarter. Given the intense labor required and the universe of 5,000 U.S. publicly traded companies, there was no realistic way to increase the company's capabilities without adding some software muscle or recruiting a stock-savvy Watson.
Early in its history—Business Intelligence Advisors was founded in 2001—as their analysts produced research reports, they instituted a system to identify and tag words and sentences (including about 270,000 of the latter) found in conference-call transcripts. From 2002 to 2009, these were put into a computerized database.
Then, Leggett says, the company developed "an algorithm that models the methodology, using linguistic pattern-recognition software." Unlike a word- search list, such programs look for response patterns that, BIA says, are related to certain behavioral indicators. Using methodology developed by the CIA as a noninvasive, nonconfrontational alternative to a polygraph test, the company decided to focus on 32 of them, covering a wide range of verbal and nonverbal behaviors.
Among them: behavior that indicates "a lack of confidence and uncertainty all the way to the other end of the spectrum, [such as] someone's actually trying to avoid disclosing or disclosing something that might not be correct." Despite its origins, Leggett is at pains to note that the program–called the BIA Earnings Call Analytics Platform–isn't equivalent to a lie detector.
Beginning in late 2010, BIA began selling its computer-generated research to analyze conference calls. Typically, these reports are produced for a client within five minutes after a transcript is available. The report initially provides a decile score, ranging from 1 to 10 of the conference call based on the 32 behavioral indicators. It also notes any key topics that raise the most behavioral concerns and provides the scores for the previous four quarterly calls.
In the ranking, each company's call is graded in relation to about 3,000 other calls for that quarter. Companies that score a 1 exhibit the least potentially problematic behavior; those with a 10, the most. The outliers, like 9s and 10s, are further scrutinized by human analysts in an additional report.
Says Leggett: "Management knows more about their business than anybody else. You can glean insights by the way executives talk about the topics of interest to the market." BIA doesn't evaluate the facts on the call, but rather the "quality of the delivery…of the commentary." It's a "filter to know where the information is highly reliable and high-quality or where it is not as reliable and you need to do more work," he observes.
The service isn't cheap. "Our relationships tend to average in the low six-figure range on an annual basis…depending on which services the client wants," Leggett says. "It's consumption-based and can cost considerably more."
Is it worth it? A back test on the bespoke rankings conducted by Harvard Business School Prof. Christopher Malloy, a BIA consultant, suggests that there is a correlation between the rankings and future stock-price performance.
According to Malloy's work, the stocks of companies exhibiting the highest-concern rankings underperformed the market by about 16%, while those of companies that raised the fewest red flags tended to outperform the market by about 15%. Some 60% of the high-risk companies were laggards, and 61% of the low-risk outfits were outperformers.
That study was done on analyst-generated call evaluations in 2003 through 2008. Business Intelligence Advisors has recently begun sending rankings generated by the BIA Earnings Call Analytics Platform to Malloy for another study.
"I'm looking at about 27 months worth of machine-generated data," Malloy tells Barron's. While it's a shorter period, there are actually many more ranked calls—about 3,200 per quarter—in the sample than there were in the previous study. Malloy says that the preliminary results "are consistent with the original study."
Barron's spoke with a few money managers that have kicked the tires or have used BIA's research.
Marshall Kaplan, who runs Fundamental Equity Advisors for Morgan Stanley Smith Barney, has employed BIA in the past and says he plans to be an "a la carte" user of its research. "In the past few years, behavioral finance and management credibility have become more important," he comments. Philip Facchina, a partner at Ramsey Asset Management, which oversees about $500 million in assets, has also used BIA and is considering using the new product. "The BIA process and approach yields insights that others don't have or seek to explore…and which don't seem to correlate with other fundamental factors you may normally monitor," he says.
Yet the head of one very large and successful hedge fund says that it has examined the research and didn't see the value added.
BIA didn't explain all the workings of its proprietary software to Barron's but gave a few examples of what's being measured, that is, some of the building blocks of those 32 behaviors.
"Protest statements" are very significant warnings, Leggett maintains. Here, "protest" doesn't convey the conventional meaning, but instead connotes a statement that doesn't convey information. Rather than directly answer a question, it is designed "to convince you of something."
Another tip-off is "qualified" language, such as "fairly normal" or "pretty typical" revenue growth. These suggest that there are some aspects that aren't so typical or normal. "Investors have to do some work at figuring out what parts aren't so typical." Other worrisome factors include overly courteous behavior, long delays before addressing a question, refusals to answer or the invoking of religion[LOL].
On the July-quarter conference call by Cisco Systems (ticker: CSCO) last year, someone asked whether a decline in the tech company's operating margin was specific to its own business or more related to the economy. In response, CEO John Chambers said: "…in terms of Cisco-specific, we feel really good. We are competing. We are winning.… My confidence has never been higher…. The cautious growth next quarter of 18%-20% is not bad at all. So that would be how I would answer your question."
In BIA's computer-generated report, the Cisco call was graded a 10, the most concerning. The analysis stated: "Vigorous defense of current performance signals considerable stress. Inadvertently reveals weakness in certain areas of the business. Fails to offer insight into how either company-specific or economic factors are impacting their outlook." Cisco's score made it the 29th most-concerning company out of 2,679 analyzed by BIA over 90 days. Since that call on Aug. 11 the company's stock is down 22%.
Leggett says that a big change in a ranking can be worrisome. too. In BIA's report on Best Buy's (BBY) call for its second quarter, ended in August, the ranking rose to a 10 from 6 previously.
During the Sept. 14 call, in response to a question about revenue growth, a Best Buy executive said that the company expected it to pick up as it always does historically in the second half, even against tough comparisons. But later, the same executive notes that his optimism is balanced against a "consumer still under pressure." The obvious question, Leggett notes: "If history is going to repeat itself, then why do you need to balance your optimism?" Since then, Best Buy stock is down 10%.
Lest Barron's be accused of throwing stones while living in a glass house, we'll note that News Corp. (NWS) received a score of 9 at its last conference call on Feb. 3. The stock is up 2% since then and at 52 week highs. News Corp. owns Dow Jones, which publishes Barron's.
Despite his computerized system's apparent success, Leggett cautions that "there is no way that the system is going to ever be as good as humans."
Of course, that's similar to what some folks were saying before IBM's Big Blue beat Gary Kasparov in chess 14 years ago or Watson trounced his rivals last month. And if BIA's Earnings Call Analytics Platform becomes successful, we can already see the perfect TV show for it: To Tell the Truth.‹
News 
Market Data 
Discover 
