Pakistanis angry over detentions in Times Sq. case Monday, May 24, 2010
ISLAMABAD – Relatives of three men detained by Pakistan for alleged links to the suspect in the attempted Times Square bombing say the men are innocent.
They
AFP - Thursday, August 6TAIPEI (AFP) - - Taiwan's Beijing-friendly government on Wednesday denied boycotting an Australian film festival amid a row over the e
BERLIN (Reuters) - Chancellor Angela Merkel suffered a double blow on Thursday as a senior party ally in east German
Minister seeks closure of anti-Berlusconi websites Wednesday, December 16, 2009
ROME (AFP) - – The Italian government moved Tuesday to close down Internet sites encouraging further violence against Prime Minister Silvio Berlusconi, who
By ELAINE KURTENBACH,AP Business Writer AP - Wednesday, March 18SHANGHAI - Asia's stock market rally seemed to be running out of steam Wednesday, despite an
Edition:
U.S.
Africa
Arabic
Argentina
Brazil
Canada
China
France
Germany
India
Italy
Japan
Latin America
Mexico
Russia
Spain
United Kingdom
Home
Business
Business Home
Economy
Technology
Media
Small Business
Legal
Deals
Earnings
Social Pulse
Business Video
The Freeland File
Markets
Markets Home
U.S. Markets
European Markets
Asian Markets
Global Market Data
Indices
M&A
Stocks
Bonds
Currencies
Commodities
Futures
Funds
peHUB
World
World Home
U.S.
Brazil
China
Euro Zone
Japan
Mexico
Russia
India Insight
World Video
Reuters Investigates
Decoder
Politics
Politics Home
Election 2012
Issues 2012
Candidates 2012
Tales from the Trail
Political Punchlines
Supreme Court
Politics Video
Tech
Technology Home
MediaFile
Science
Tech Video
Tech Tonic
Social Pulse
Opinion
Opinion Home
Chrystia Freeland
John Lloyd
Felix Salmon
Jack Shafer
David Rohde
Bernd Debusmann
Nader Mousavizadeh
Lucy P. Marcus
David Cay Johnston
Bethany McLean
Edward Hadas
Hugo Dixon
Ian Bremmer
Lawrence Summers
Susan Glasser
The Great Debate
Steven Brill
Jack & Suzy Welch
Breakingviews
Equities
Credit
Private Equity
M&A
Macro & Markets
Politics
Breakingviews Video
Money
Money Home
Tax Break
Lipper Awards 2012
Global Investing
MuniLand
Unstructured Finance
Linda Stern
Mark Miller
John Wasik
James Saft
Analyst Research
Alerts
Watchlist
Portfolio
Stock Screener
Fund Screener
Personal Finance Video
Money Clip
Investing 201
Life
Health
Sports
Arts
Faithworld
Business Traveler
Entertainment
Oddly Enough
Lifestyle Video
Pictures
Pictures Home
Reuters Photographers
Full Focus
Video
Reuters TV
Reuters News
Article
Comments (1)
Slideshow
Counterparties: Today's Best Links
The gambler who broke Atlantic City
Don Johnson used relatively tiny advantages to win $6 million from an Atlantic City casino in one night. It wasn't the first such haul for Johnson. Here's how he did it. Read more at Counterparties
Dallas Fed: Break up big banks
Private equity's bigges threat is itself
Follow Reuters
Facebook
Twitter
RSS
YouTube
Read
New high in U.S. autism rates inspires renewed debate
1:46pm EDT
Urine-soaked eggs a spring taste treat in China city
|
11:40am EDT
Exclusive: Brazil to rally BRICS against rich countries
28 Mar 2012
Exclusive: U.S. sees lifetime cost of F-35 fighter at $1.45 trillion
1:28am EDT
Pentagon sees mass layoffs if budget cuts prevail
4:08pm EDT
Discussed
248
Poll: Americans angry with Obama over gas prices
224
Cheney recovering after heart transplant: spokeswoman
218
Black friend defends shooter of Florida teen
Watched
Congressman dons a hoodie, gets kicked off House floor
Wed, Mar 28 2012
Horror hits the runway in Japan
Fri, Mar 23 2012
Elephant runs away from circus
12:16am EDT
Robo-readers: the new teachers' helper in the U.S.
Tweet
Share this
Email
Print
Related News
Insight: Public schools sell empty classroom seats abroad
Thu, Mar 8 2012
Suspect told police he killed randomly in Ohio school
Wed, Feb 29 2012
Analysis & Opinion
Ultra-religious schools test Israel’s high-tech future
Jim Yong Kim and Dartmouth’s culture of sexual assault
Related Topics
U.S. »
Tech »
Lifestyle »
1 of 2. An artist's impression shows a fictional robo-teacher. American high school students are terrible writers, and one education reform group thinks it has an answer: robots. Or, more accurately, robo-readers - computers programmed to scan student essays and spit out a grade.
Credit: Reuters/Brice Hall
By Stephanie Simon
Thu Mar 29, 2012 5:21pm EDT
(Reuters) - American high school students are terrible writers, and one education reform group thinks it has an answer: robots.
Or, more accurately, robo-readers - computers programmed to scan student essays and spit out a grade.
The theory is that teachers would assign more writing if they didn't have to read it. And the more writing students do, the better at it they'll become - even if the primary audience for their prose is a string of algorithms.
That sounds logical to Mark Shermis, dean of the College of Education at the University of Akron. He's helping to supervise a contest, set up by the William and Flora Hewlett Foundation, that promises $100,000 in prize money to programmers who write the best automated grading software.
"If you're a high school teacher and you give a writing assignment, you're walking home with 150 essays," Shermis said. "You're going to need some help."
But help from a robo-reader?
"Wow," said Thomas Jehn, director of the Harvard College Writing Program. He paused a moment.
"It's horrifying," he said at last.
Automated essay grading was first proposed in the 1960s, but computers back then were not up to the task. In the late 1990s, as technology improved, several textbook and testing companies jumped into the field.
Today, computers are used to grade essays on South Dakota's student writing assessments and a handful of other high-stakes exams, including the TOEFL test of English fluency, taken by foreign students.
But machines do not grade essays on either the SAT or the ACT, the two primary college entrance exams. And American teachers by and large have been reluctant to turn their students' homework assignments over to robo-graders.
The Hewlett contest aims to change that by demonstrating that computers can grade as perceptively as English teachers - only much more quickly and without all that depressing red ink.
Automated essay scoring is "nonjudgmental," Shermis said. "And it can be done 24/7. If students finish an essay at 10 p.m., they get feedback at 10:01."
Take, for instance, the Intelligent Essay Assessor, a web-based tool marketed by Pearson Education, Inc. Within seconds, it can analyze an essay for spelling, grammar, organization and other traits and prompt students to make revisions. The program scans for key words and analyzes semantic patterns, and Pearson boasts it "can 'understand' the meaning of text much the same as a human reader."
Jehn, the Harvard writing instructor, isn't so sure.
He argues that the best way to teach good writing is to help students wrestle with ideas; misspellings and syntax errors in early drafts should be ignored in favor of talking through the thesis. "Try to find the idea that's percolating," he said. "Then start looking for whether the commas are in the right place." No computer, he said, can do that.
What's more, Jehn said he worries that students will give up striving to craft a beautiful metaphor or insightful analogy if they know their essays will not be read, but scanned for a split second by a computer program.
"I like to know I'm writing for a real flesh-and-blood reader who is excited by the words on the page," Jehn said. "I'm sure children feel the same way."
Even supporters of robo-grading acknowledge its limitations.
A prankster could outwit many scoring programs by jumbling key phrases in a nonsensical order. An essay about Christopher Columbus might ramble on about Queen Isabella sailing with 1492 soldiers to the Island of Ferdinand -- and still be rated as solidly on topic, Shermis said.
Computers also have a hard time dealing with experimental prose. They favor conformity over creativity.
"They hate poetry," said David Williamson, senior research director at the nonprofit Educational Testing Service, which received a patent in late 2010 for an Automatic Essay Scoring System.
But Williamson argues that automated graders aren't meant to identify the next James Joyce. They don't judge artistic merit; they measure how effectively a writer communicates basic ideas. That's a skill many U.S. students lack. Just one in four high-school seniors was rated proficient on the most recent national writing assessment.
The Hewlett Foundation kicked off its robo-grading contest by testing several programs already on the market. Results won't be released for several weeks, but Hewlett officials said they did very well.
Hewlett then challenged amateurs to come up with their own algorithms.
The contest, hosted on the data science website Kaggle.com, has drawn hundreds of competitors from all walks of life. They have until April 30 to write programs that will judge essays studded with awkward phrases such as, "I slouch my bag on to my shoulder" or "When I got my stitches some parts hurted."
The goal is to get the computer to give each essay the same score a human grader would.
Martin O'Leary, a glacier scientist at the University of Michigan, has been working on the contest for weeks.
Poring over thousands of sample essays, he discovered that human graders generally don't give students extra points for using sophisticated vocabulary. So he scrapped plans to have his computer scan the essays for rare words.
Instead, he has his robo-grader count punctuation marks. "The number of commas is a very strong predictor of score," O'Leary said. "It's kind of weird. But the more, the better."
As he digs into the data, O'Leary has run into a dismaying truth: The human graders he's trying to match are inconsistent. They disagree with one another on the merits of a given essay. They award scores that seem random. Indeed, studies have shown that human readers are influenced by factors that should be irrelevant, such as how neatly a student writes.
"The reality is, humans are not very good at doing this," said Steve Graham, a Vanderbilt University professor who has researched essay grading techniques. "It's inevitable," he said, that robo-graders will soon take over.
O'Leary won't mind when that day comes. He tests his program against student prose that has already been graded by a teacher. When the scores diverge, O'Leary reads the essay to find out why.
"More often than not," he said, "I agree with the computer."
(Editing by Jonathan Weber and Philip Barbara)
U.S.
Tech
Lifestyle
Tweet this
Link this
Share this
Digg this
Email
Reprints
We welcome comments that advance the story through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can flag it to our editors by using the report abuse links. Views expressed in the comments do not represent those of Reuters. For more information on our comment policy, see http://blogs.reuters.com/fulldisclosure/2010/09/27/toward-a-more-thoughtful-conversation-on-stories/
Comments (1)
mrtcbear wrote:
Edition:
U.S.
Africa
Arabic
Argentina
Brazil
Canada
China
France
Germany
India
Italy
Japan
Latin America
Mexico
Russia
Spain
United Kingdom
Back to top
Reuters.com
Business
Markets
World
Politics
Technology
Opinion
Money
Pictures
Videos
Site Index
Legal
Bankruptcy Law
California Legal
New York Legal
Securities Law
Support & Contact
Support
Corrections
Connect with Reuters
Twitter
Facebook
LinkedIn
RSS
Podcast
Newsletters
Mobile
About
Privacy Policy
Terms of Use
AdChoices
Copyright
Our Flagship financial information platform incorporating Reuters Insider
An ultra-low latency infrastructure for electronic trading and data distribution
A connected approach to governance, risk and compliance
Our next generation legal research platform
Our global tax workstation
Thomsonreuters.com
About Thomson Reuters
Investor Relations
Careers
Contact Us
Thomson Reuters is the world's largest international multimedia news agency, providing investing news, world news, business news, technology news, headline news, small business news, news alerts, personal finance, stock market, and mutual funds information available on Reuters.com, video, mobile, and interactive television platforms. Thomson Reuters journalists are subject to an Editorial Handbook which requires fair presentation and disclosure of relevant interests.
NYSE and AMEX quotes delayed by at least 20 minutes. Nasdaq delayed by at least 15 minutes. For a complete list of exchanges and delays, please click here.