AI Is Selecting Reviewers in China
AI Is Selecting Reviewers in China
AI Is Selecting Reviewers in China
SOURCE: NIEHS
HIDDEN LINKS work is also at the heart of an ongoing lawsuit activities.
The 13 facilities supported by the EPA and brought by environmental groups seeking to But Kimberly Gray, who manages the
the NIEHS are scattered in cities across the force the EPA to ban all uses of the pesticide. NIEHS’s contribution to the centres, says that
country and employ hundreds of research- “They’re just jaw-dropping studies,” says her agency cannot afford to support them on
ers in disciplines such as toxicology, genetics Lisa Satterwhite, a molecular geneticist with its own without making significant changes.
and brain development (see ‘Connecting the the children’s centre at Duke University in For now, she says, the NIEHS is trying to
dots’). The centres’ ability to follow people Durham, North Carolina. “We could not have maximize the research that the centres have
from before birth to adulthood has revealed anticipated there would be this built-in natural already completed, by supporting their com-
surprising connections between common experiment.” munity outreach, and looking for ways to
chemicals and health. Each of the facilities also works with local keep their study cohorts going. The centres
Research by the Columbia centre suggests groups to educate communities about the find- are also eligible to compete for NIEHS grants
that the widely used pesticide chlorpyrifos ings of their studies, many of which address against other long-term epidemiological
harms the development of children’s brains. environmental harms that disproportionately studies of all types.
Chlorpyrifos is used to treat a broad array of affect people in low-income neighbourhoods. Linda McCauley, who leads the children’s
food crops, and until 2001, it was legal in the “I cannot think of an equivalent network that centre at Emory University in Atlanta, Georgia,
United States for use indoors against insects could do the same work,” says Aparna Bole, a is spending her remaining money on commu-
such as cockroaches. In 2012, Columbia scien- paediatrician at Rainbow Babies and Children’s nity outreach. Grants from the US National
tists reported that children who were exposed Hospital in Cleveland, Ohio. Institutes of Health — the NIEHS’s parent —
to high levels of the pesticide in the womb had After the children’s centres’ long-term or other funders could help her continue to do
lower IQs and altered brain structure com- grants from the EPA and the NIEHS expire, research, but the outreach programme at her
pared to those with low exposure (V. A. Rauh the facilities will have until July 2020 to spend centre has no other source of financial support.
et al. Proc. Natl. Acad. Sci. USA 109, the remainder of the money. The additional “All these community stakeholders have
7871–7876; 2012). cash that the NIEHS has scraped together will been such critical partners for this work
Last year, Hawaii became the first US state allow some of the centres to perform outreach, nationally and there’s no funding,” she says.
to ban agricultural use of chlorpyrifos — and graduate students to finish dissertations and “They’re the ones being hurt the most.” ■
G RANTS
B Y D AV I D C Y R A N O S K I Natural Science Foundation of China (NSFC) funding agencies, including some in North
is world-leading, but others are sceptical about America and Europe, have trialled simple AI
C
hina’s largest funder of basic science is whether AI can improve the process. systems, some of which match keywords in
piloting an artificial intelligence (AI) Choosing researchers to peer review project grant applications to those in publications of
tool that selects researchers to review proposals or publications is time-consuming other scientists to identify potential reviewers.
grant applications, in an attempt to make the and prone to bias. Several academic publish- The NSFC is building a more sophisticated
process more efficient, faster and fairer. Some ers are experimenting with AI tools to select system that will crawl online scientific-
researchers say the approach by the National reviewers and carry out other tasks. And a few literature databases and scientists’ personal
3 1 6 | NAT U R E | VO L 5 6 9 | 1 6 M AY 2 0 1 9
©
2
0
1
9
S
p
r
i
n
g
e
r
N
a
t
u
r
e
L
i
m
i
t
e
d
.
A
l
l
r
i
g
h
t
s
r
e
s
e
r
v
e
d
.
IN FOCUS NEWS
web pages, using natural-language processing Science Foundation. The NSFC is struggling Thomas Hansteen, an adviser to the council.
to glean detailed information about the publi- to process applications and find appropriate But not everyone is convinced that AI
cations or research projects of potential refer- reviewers, says Li. “The challenge is not hav- should be used in the review process. Susan
ees. The system will use semantic analysis of the ing enough people,” he says. “AI will solve that.” Guthrie, a science-policy specialist at research
text to compare the grant application with this Li also wants the tool to reduce bias in organization RAND Europe in Cambridge,
information and identify the best matches, says reviewer selection. In China, scientists try to UK, notes that the Canadian Institutes of
agency head Li Jinghai, who is based in Beijing. lobby for their projects, he says. “AI can’t be Health Research has run into challenges with
An early version of the tool selected at least corrupted.” an algorithm used for reviewer selection.
one member of each of nearly 44,000 panels Bias can also be an issue in countries where The Canadian agency hired RAND Europe
that approved projects last year, says Yang Wei, applicants are asked to suggest experts who in 2016 to carry out a meta-analysis of stud-
the agency’s former head, who presented data could review their proposals. The Swiss ies on grant peer review. Partly on the basis
on the pilot at a meeting on scholarly commu- National Science Foundation has found that of that report, the agency concluded that the
nication in Hangzhou last month. Panels are reviewers who were algorithm sometimes selected reviewers who
composed of between three and seven people. recommended by the had conflicts of interest or were otherwise
The system is already cutting the time admin- applicants were more
“Because the not qualified to evaluate the proposal. “While
istrative staff have to spend looking for refer- likely to endorse a pressure is so algorithm-based matching sounded attrac-
ees, says Yang. A similar approach will be used project than were high, China had tive, there is a limit at this stage of artificial
this year to select reviewers, he says. referees chosen by no choice but intelligence to what it can possibly achieve,”
The NSFC has become a world leader the foundation. to find the best an independent expert panel concluded.
in reforming grant-review processes, says The NSFC’s pilot way.” Elizabeth Pier, a policy researcher at
Patrick Nédellec, director of the international- AI system works on Education Analytics in Madison, Wisconsin,
cooperation department of the French CNRS, websites written in Chinese characters, but Li thinks AI will not remove selection bias. She
Europe’s largest basic-research agency. The wants it to be able to crawl English-language fears that AI systems end up replicating the
NSFC is being forced to innovate as the num- sites in the future. biases ingrained in human judgements, rather
ber of grant applications keeps growing, says “NSFC’s reform plan is ambitious, forward- than avoiding them.
Nédellec, who attended a meeting last Septem- looking and comprehensive,” says Manfred Li says that the NSFC also plans to introduce
ber at which Li discussed the agency’s reform Horvat, a science-policy adviser at the Vienna a credit system that will reward researchers for
plans. “Because the pressure is so high, China University of Technology, who also heard Li’s good, fair and timely reviews — although he
has no choice but to find the best way,” he says. talk last September. wouldn’t comment on the nature of the rewards.
In the past five years, the number of appli- Other countries are following China’s lead. But statistician John Ioannidis of Stanford
cations the NSFC receives has increased by Last month, the Research Council of Norway University in California says it will be difficult
roughly 10% a year. In 2018, the organization started using natural-language processing to to evaluate whether reviewers have made good
evaluated 225,000 grant applications — almost cluster about 3,000 proposals into groups and decisions because it can take decades for an
6 times the number received by the US National match them to the best reviewer panels, says idea to be considered “great or a waste”. ■
P OLITICS
R
ising intolerance towards intellectuals unusual move for India’s research community, science and a rise in politicians and public
and minority groups in India has which rarely comments on political or social figures making unscientific claims. The BJP
prompted scientists there to speak out issues. It calls on voters to “reject those who includes new technology in its manifesto,
ahead of the country’s mammoth general elec- lynch or assault people, those who discrimi- but some worry that it prioritizes technology
tion. More than 200 scientists have signed an nate against people because of religion, caste, ahead of basic science. The Congress party has
open letter appealing to citizens to reject the gender, language or region”. promised to boost spending on science, but
discrimination and violence being promoted The letter does not mention any political there are doubts over whether it can deliver
by some extremist groups. party. But since the BJP formed a govern- on this.
The election is a contest between the rul- ment in 2014, there has been a rise in attacks The BJP’s election manifesto states that it will
ing Hindu nationalist Bharatiya Janata Party by Hindu right-wing groups In India against launch major programmes in artificial intelli-
(BJP), led by Prime Minister Narendra Modi, Muslims and other minority groups that eat gence, robotics, supercomputers and genom-
and the main opposition, the secular Indian beef — Hindus consider cows sacred. Extreme ics for human health, but the manifesto does
National Congress — the Congress party. right-wing groups were also blamed for the not mention how much it will spend on these
Nearly 900 million people are eligible to vote deaths of three intellectuals, between 2013 and endeavours. Last year, Modi also announced a
in this election, which began on 11 April and is 2015, who campaigned for scientific reasoning. mission to send humans to space by 2022.
1 6 M AY 2 0 1 9 | VO L 5 6 9 | NAT U R E | 3 1 7
©
2
0
1
9
S
p
r
i
n
g
e
r
N
a
t
u
r
e
L
i
m
i
t
e
d
.
A
l
l
r
i
g
h
t
s
r
e
s
e
r
v
e
d
.