Saturday, January 19, 2013

A Grammar Checker Fails the Test

The Virtual Writing Tutor Bombs Badly

A fellow ESL teacher recently sent me this message on Facebook about the VirtualWritingTutor, an English grammar checker for ESL learners.

I tried the VirtualWritingTutor with a text I ask my students to edit (English 100). It missed a lot of mistakes - especially with the present continuous verb forms. I think that you'll need to add even more error detection rules! 

Here's the text I tried out:
You have a pet? I do. I have a cats. It have five year old. I buy the animal for my childs. My son and daughter are very happy when they got Tommy. Tommy is the cat’s name. My son love the cat very much. He brings the cat in her bed at night and the cat sleep there. He is feeding the cat every day. Sometimes my daughter feed the cat. Last week, the cat go outside and don’t come back for two day. We not closing the door. Usually, we always close the door. This weekend, my friend is feed the cat because we going on a trip. We go at New York.

The Virtual Writing Tutor--a work in progress

Here was my reply

Thank you very much for this. Obviously, there are many more rules to write. The rules I am writing are based on authentic errors written by my own Francophone 100B and 101A students and based on the two student corpora on Lextutor.com . For some reason, the errors teachers submit to test the system almost always catch it off guard. I am not sure if it is how teachers reformulate errors from memory, the fact that different writing tasks produce a different range of errors, or just the sheer chaos of L2 writing.

In response to the text you posted, I have added a couple dozen rules to the system. Instead of catching 4 errors and throwing up false alarms, the system will now catch 17 errors in your text. Some errors I won’t try to catch simply because they involve understanding sentences that come before and after. This system is set up to look for overt sentence-level errors.

New Error Detection Rules Added

You have a pet?

I added a rule to catch the lack of an auxiliary with “You have a pet?” and a second rule for “You have pets?”

It have five year old.

I had a rule to catch “It have five years old” and many variations but not “it have five year old.” I modified the rule to catch the extra variation.

I buy the animal for my childs.

I created an exception for Childs in the proper name capitalization rule, reasoning that high frequency pluralization errors are more of a problem than low frequency family name capitalization errors. I will have to create a separate rule for the family name Childs. I created a rule to catch “childs” with the exception that it should be ignored if “childs” appears before another noun. In such a case, it will assume that the learner needs a possessive form. I won’t try to catch the “I buy” tense error. it is too broad of an assumption to expect all storytelling to use the past tense when dramatic stories can use the present simple also. More to the point, I have been focusing on sentence-level errors only.

My son and daughter are very happy when they got Tommy.

I created a rule to compare the verb at the beginning of the sentence with the verb at the end of the sentence. I have 36 of these rules so far. I’ll need many more, I figure.

My son love the cat very much.

I created an additional condition for the 3rd-person rule. It is easier to teach than to program because of the danger of false alarms. For one, you have to add an exception for the subjunctive. Also, we need an exception for all past forms, past participles, modals and infinitives that look like present simple forms.

I fixed the adjective order false alarm for “cat very much” by adding an exception that looks for a determiner before “cat”.

He brings the cat in her bed at night and the cat sleep there.

Now the system catches the third person singular error.

He is feeding the cat every day.

The system was able to catch “Every day, he is feeding the cat”. It is easier to program the system to catch aspect errors when the adverb of frequency is at the beginning or in the middle of the sentence. I have nevertheless added the new rule.

Sometimes my daughter feed the cat.

The reason the system didn’t catch “feed” is that it is also the low-frequency but nevertheless past tense of verb “to fee” which is excluded as being potentially correct. I had to create a separate rule for “feed”.

Last week, the cat go outside and don’t come back for two day.

This one was tough because of the two verb errors, but I have cracked it. I’ll need to develop more rules along this line.

This weekend, my friend is feed the cat because we going on a trip.

This was helpful for me. I set up a new group of rules to deal with future plans and arrangements. The second part was simply a case of catching a missing auxiliary after “because,” a case that is very unlikely to be a question with auxiliary-fronting

Stumping the Virtual Writing Tutor

Do you have any error-correction activities or sentences that stump the VirtualWritingTutor? If you do, please share them by adding them in a comment below. I will certainly add any missing rules in short order.

5 comments:

  1. My name is Bezen. Would you mind if you correct me my grammar please?



    Shark

    Shark is an absolute monster ocean animal. It has multiple rows of teeth, and while it lose teeth on a regular basis, new teeth continue to grow in and replace those it lose. It feeds on a variety of marine life, including small sharks and sea turtles. While you are swimming, avoid being in the water during early morning and late afternoon, since sharks actively feed at those times. And also do not swim if you are bleeding, even if the cut or injury is minor. In general, shark is the most dangerous and attacker ocean animal.



    Uses Smart Phone


    Smartphone is the most amzaing techonloy, and a very useful product for communication. This product has a lot of applications and you can send an e-mail or text, search a lot of information very esily and quicly. You can use samartphon as a computer for search some information on the internate, you can do your homework, a record high DVD quality video (with sound), you can take pictures, record sound and or hook up as a web cam, and also when you are traveling you can use the GPS to find the locations easly. In addition, you can use smartphone for other lucxury things like, listenning music, playing games and lock and unlock your home. In general, smartphone is the best product and it can makes your life easier.

    ReplyDelete
  2. This comment has been removed by a blog administrator.

    ReplyDelete
  3. This is great advice. If you're looking for professional editing and proofreading, check out WordsRU.com. They are thorough, quick and are ready to answer any questions you might have even after the edit

    ReplyDelete
  4. This is very helpful advice for a DIY proofreading project. To get your work professionally edited I suggest WordsRU.com. They are quick, friendly and very thorough in their work.

    Academic Editing and Proofreading Services
    Academic Editing Services
    Academic Proofreading Services
    Article Editing Services
    Article Writing Services
    Journal Article Editing Services
    Book Editing Services

    ReplyDelete
  5. this article is good interesting and easily understand to everyone.And every person can known how to Best Spell Checker for Skype to read this article its very help full for everyone. check my site and Make your documents free of errors with Grammer Checker

    ReplyDelete

Introduction to My Blog

In this blog you will learn about me and my adventures in my program.