The Genius Index: One Scientist's Crusade to Rewrite Reputation Rules By Guy Gugliotta 05.22.09
In the scientific community, journal citations are the coin of the realm. Illustration: Ronald Kurniawan
Jorge Hirsch had been getting screwed. For years. At a scientific conference in 1989, he presented a paper arguing that the generally accepted theory of low-temperature superconductors—the BCS theory—was wrong. Most researchers at the time held that under certain low-temperature conditions, vibrations in a metal's crystal lattice can allow electrons to become attracted to one another, which drops electrical resistance to zero—a superconducting state. Hirsch said this "electron-phonon interaction" in fact had nothing to do with superconductivity. He was a youngish up-and-comer then, but physics rarely forgives apostasy. After his fateful presentation, similar conferences stopped inviting him to speak. Colleagues no longer sought him out for collaboration. Grants dried up. High-visibility journals shunned his papers.
It's not that Hirsch wasn't getting his work published. He was. And other physicists were still citing his research, implying some acceptance of his views. Hirsch just wasn't able to get his papers into the really high-visibility journals—places like Science, Nature, and, for a solid-state physicist, Physical Review Letters. There's a clear pecking order, established and reinforced by several independent rating systems. Chief among them: the Journal Impact Factor.
Hirsch, like his peers, understood that if he wanted to get to the front ranks of his discipline, he had to publish in journals with higher JIFs. But this struck him as unfair. Like most physicists, he's pretty smart, and not shy about it. The system that had shunned him was wrong, Hirsch concluded. It shouldn't be about where he published; it should be about his work. So in 2003, Hirsch decided that he—oh, and science, too!—needed a system that would rate researchers directly.
After two years of number-crunching in his cluttered office at UC San Diego, Hirsch had it—an invention important enough to warrant publication in the (very prestigious) Proceedings of the National Academy of Sciences. In his 2005 article, Hirsch introduced the h-index (named after himself, of course). The key was focusing not on where you published but on how many times other researchers cited your work. In practice, you take all the papers you've published and rank them by how many times each has been cited. Say paper number one has been cited 10,000 times. Paper number two, 8,000 cites. Paper number 32 has 33 citations, but number 33 has received just 28. You've published 32 papers with more than 32 citations—your h-index is 32.
Or to put it more technically, the h-index is the number n of a researcher's papers that have been cited by other papers at least n times. High numbers = important science = important scientist.
In its nearly four years of life, the relatively simple, flexible h-index has become the most talked-about metric in the very hot science of rating scientists and their research, a discipline that has flourished in ways Hirsch had never imagined. The h-index was the biggest splash in a flood of Internet-enabled rating systems—growth and decay chronometrics, semiometric measures, hub/authority metrics. Schools and labs use such ratings to help them make grants, bestow tenure, award bonuses, and hire postdocs. In fact, similar statistical approaches have become standard practice in Internet search algorithms and on social networking sites. These numbers do for scientists what U.S. News & World Report does for colleges and Bill James' Sabermetrics did for baseball: They quantify reputation.
Or at least they try. Either way, the numbers have the stark, uncluttered feel that scientists love. "I'm a physicist," Hirsch says. "Some people put a lot of weight on subjective criteria, and I don't like that at all."
The science citation revolution began more than 50 years ago. Eugene Garfield, then a young librarian pursuing a PhD in structural linguistics, started wondering about that most prosaic of bibliographic tools: the footnote. Most people think of footnotes as reaching backward in time to a document's sources. But Garfield realized that they could reach forward, too—future footnotes would cite the original article. "The citation becomes the subject," says Garfield, now 83 and enjoying his stature as the founding father of modern citation analysis. "It was a radical approach to retrieving information."
Some three decades before the concept of the hyperlink and the World Wide Web crossed anybody's mind, Garfield had figured out how to connect the immense body of scientific knowledge into a network. In the early 1960s he began publishing The Science Citation Index; Garfield sold the first edition, five volumes of arcane hard-copy reference, to academic libraries for $500.
Over the years, as the index expanded, Garfield worked furiously to figure out how to select and compare the journals it included. He found his answer in a simple equation: Divide the number of citations the journal gets over the prior two years by the number of articles it published. He named it the Journal Impact Factor, and in 1975 he decided to reveal how it worked. Only librarians seemed to care. "It was clear that most people didn't have the vaguest idea what I was talking about," Garfield says.
Still, it was a tidy little business, and in 1992 Garfield sold it to what's now called Thomson Reuters. But it didn't get much traction until five years later, when the company dumped the entire database onto the Internet. Today, it's part of the ISI Web of Knowledge, and its arrival online gave scientists easy access to a colleague's oeuvre so they could see whether it was published in good journals. Suddenly, the scientific life had a scoreboard.
The Web of Knowledge now comprises 700 million cited references from 23,000 journals published since 1804. It's used by 20 million researchers in nearly 100 countries. Anyone—scientist, dean, lab director—can sort the entries and tell someone's fortune. Nothing approaches it for breadth and longevity. Though the Journal Impact Factor has competitors, it remains the gold standard. "You may not like the database, but it has not been replaced," Garfield says.
That said, the JIF has problems. Review journals, which round up current research, are generally cited far more often than those that publish new findings. And even though the JIF is meant to rate only journals, most people use it as a proxy for rating individual scientists anyway.
Worse, the system can be gamed. Editors have been known to encourage authors to cite articles from their journal (which has the aroma of extortion—"Help us with our JIF if you want to get published"). Journals that print obituaries, editorials, and letters catch a break, because these articles are included in the JIF numerator but not counted in the denominator. There had to be a better way.
Even change-averse academia admitted that Hirsch had come up with a different mousetrap. "The h-index does seem to be able to identify good scientists, and it is becoming widely used informally," the journal Nature reported in 2007, perhaps somewhat grudgingly. The index makes it harder to cheat. Researchers have to be champion self-citers to move their own numbers, and editors have no reason to influence the system.
On the other hand, Hirsch acknowledges that the h-index has its own intrinsic weaknesses. It's kind to older folks, for example, but not great to younger scientists. If a scientist writes six brilliant papers and dies, his h-index will never be higher than six, even if each paper is cited 10,000 times. And by putting the onus on individuals, it encourages researchers to write about sexy topics and hew close to the conventional wisdom—exactly what Hirsch was trying to avoid. Plus, it has trouble apportioning credit on papers with multiple authors. (Complicated math might sort researchers by their respective contributions ... maybe.)
While the problems of the various citation-based ranking schemes might seem (ahem) academic, their strategies are increasingly the coin of the online realm. Understanding and quantifying reputation is the best approach to navigating the tsunami of information on the Internet. That's why Google founders Larry Page and Sergey Brin cited Eugene Garfield in their academic work on PageRank, the algorithm that powers their company's search engine. "Articles cited by this article" and "articles that cite this article" are really just outbound and inbound links. Today, citation analysis has come full circle. Eigenfactor and SCImago actually use variations of PageRank to evaluate scientific journals. And the introduction of Google Scholar, a search tool designed specifically for academic research, provided a whole new set of citation data; it can help calculate h-index, as well as a newer ranking system called the g-index that gives more weight to articles with higher citation counts.
The depth of data available on the Internet and the increasing importance of online archives and publications like the ArXiv for physics and the Public Library of Science for biomedicine have made possible new metrics: number of downloads, rate of downloads over time (chronometrics), and even levels of funding and numbers of doctoral students working in a lab (nonbibliometric performance indicators). So powerful are these new kinds of measurements that they're finding applications beyond ranking scientists. One recent article suggested that citation analysis could be used to predict the direction of scientific innovation.
Not even Web flavor-of-the-month Twitter is immune. The microblogging site displays the number of users following a given Twitterer, and the number of users that person follows. So you can see where this is going: Various Web sites track Twitter use, factoring in those numbers as well as "re-tweets," the number of times a person's posts get recapitulated, to come up with ranks for users. The more important any given Web application becomes, the more its users want ranks.
"The Impact Factor was never supposed to be used to rank scientists," says Marie McVeigh, senior manager for bibliographic policy at Thomson Reuters. She's right, of course, but it's too late now. At least the h-index seems to give people the right numbers. Edward Witten, genius cosmologist at the Institute for Advanced Study, scores 120, the highest of all physicists. A Brief History of Time theorist Stephen Hawking gets 67. And Hirsch? He rates a 52, which according to his 2005 paper makes him "outstanding" and "likely to be found only at the top universities or major research laboratories." See? The system works.
Guy Gugliotta (guygugliotta@yahoo.com) wrote about mutated microorganisms in issue 15.09.
AWFUL NEWS! .. 6.5 Quake Shakes Buildings on California Coast By THE ASSOCIATED PRESS .. January 10, 2010
FRESNO, Calif (AP) -- A magnitude-6.5 earthquake struck off the coast of Northern California Saturday afternoon, shaking buildings, knocking out power in several coastal communities and leaving a trail of broken windows and dishes south of the Oregon border.
The powerful quake hit at about 4:27 p.m. PST about 22 miles from Ferndale, the U.S. Geological Survey said.
Authorities in the nearby city of Eureka and other area communities said no major injuries have been reported. [GOOD] But several people received minor cuts and scrapes from broken glass at the Bayshore Mall in Eureka, and an elderly person fell and broke a hip, authorities said.
''We're mostly getting reports of bumps, bruises and hits on the head,'' said Laurie Watson Stone, a spokeswoman for St. Joseph Hospital, a 146-bed hospital in Eureka. ''The emergency room is busy, but we haven't heard of any major injuries.''
Amanda Nichols, a dispatcher for Eureka Police Department, said she received a report that an infant was struck in the head with some flying debris at the mall.
Pacific Gas & Electric Co. spokesman J.D. Guidi said power outages were widespread across most of Humboldt County, affecting about 25,000 customers.
Nearly 10,000 remained without power some five hours after the quake, and some could remain without power through Sunday, said PG&E spokeswoman Janna Morris.
No damage was done to the company's former nuclear power plant outside Eureka, Morris said.
Several traffic lights fell and numerous residents reported water, gas and sewer leaks, Humboldt County Office of Emergency Services spokeswoman Jo Wattle said.
''People have chimneys down, [BAD!]and we're hearing about minor property damage and lots of glassware broken,'' Wattle said. ''People are really shaken up. It was shaking pretty good, then it had a big jolt to it at the end.''
According to the USGS, the quake hit at a depth of nearly 10 miles. At least nine aftershocks of magnitude-3.0 or above followed in the hours after the quake, the biggest registering at 6:21 p.m. PST at a magnitude of 4.5.
Police in Ferndale said the earthquake caused stucco to fall off City Hall and broke shop windows, strewing the historic downtown streets with glass shards.
''I thought a tire had blown off my truck because it was so hard to keep control of the vehicle,'' Officer Lindsey Frank said. ''Power lines were swaying, and I could see people in the fields trying to keep their balance.''
Eureka city spokesman Gary Bird said because the earthquake hit shortly before dark, only the city's old town received thorough surveys for damage. Authorities there found fallen bricks and parapets that had fallen off old structures, causing damage to adjacent buildings, he said.
''There are some frayed nerves, but I think we've come through this pretty well for the magnitude of earthquake we've had,'' Bird said.
Televisions tumbled and objects were knocked off walls in Arcata, a small town that's home to Humboldt State University, one resident said.
''The whole town is kind of freaked out right now,'' said Judd Starks, the kitchen manager at a bar and restaurant known as The Alibi. ''All the power is out, people are out walking around.''
The quake was felt as far south as Capitola in central California, and as far north as central Oregon, USGS geophysicist Richard Buckmaster said.
It struck about 270 miles north of San Francisco in an area where earthquakes ranging from 7.0 to 7.3 have struck periodically since the 1800s, causing damage and injuries. In 1964 a tsunami generated by a great 9.2-magnitude earthquake near Anchorage, Alaska, washed away 11 people in Crescent City, 80 miles to the north of Arcata.
There is a small chance -- 5 to 10 percent -- of another magnitude-6.5 temblor or larger hitting the area over the next week, but the odds dramatically decrease as time passes, the USGS said.
There's also a 78 percent chance of a strong and potentially damaging aftershock magnitude-5 or larger over the same period. The earthquake probabilities are based on statistical observations of past earthquakes in California and are not predictions, the USGS said.
California is one of the world's most seismically active regions. More than 300 faults crisscross the state, which sits atop two of Earth's major tectonic plates, the Pacific and North American plates. About 10,000 quakes each year rattle Southern California alone, although most of them are too small to be felt.
The damage created by an earthquake depends greatly on where it hits. A 7.1 quake -- much stronger than Saturday's 6.5 temblor -- hit the Mojave Desert in 1999 but caused only a few injuries and no deaths.
By contrast the 1994 Northridge earthquake under Los Angeles' San Fernando Valley was magnitude 6.7. It killed 72 people, injured more than 9,000 and caused $25 billion in damage in the metropolitan area. And the magnitude-7.1 quake in October 1989 struck just before the third game of the World Series at Candlestick Park. The quake, centered in the Santa Cruz Mountains on the San Andreas fault, caused nearly $3 billion in damage.
Dan Bowermaster of San Francisco was with relatives in Eureka when Saturday's quake hit, moving the refrigerator in his cousins' home about 3 feet. He said he had been in several moderate and large quakes throughout California but had never felt anything as strong as this one.
''It was extremely unsettling, it was shaking in kind of a circular way,'' he said.
Sandra Hall, owner of Antiques and Goodies in Eureka, said furniture fell over, nearly all her lamps broke and the handful of customers in her store got a big scare. She said it was the most dramatic quake in the 30 years the store has been open.
''We'll be having a sale on broken china for those who like to do mosaics,'' she said.
WONDERFUL SENSE OF HUMOR AMID ADVERSITY!
Dalton reported from Los Angeles. Associated Press writers Alicia Chang in Los Angeles and Bob Christie in Phoenix contributed to this report.
Market Data 
Markets 
AI
