Voozh

Machine Comprehension Test (MCTest)

A Challenge Dataset for the Machine Comprehension of Text Home | Results | Data

MCTest is a freely available set of 660 stories and associated questions intended for research on the machine comprehension of text. The data was gathered using Mechanical Turk. Details on the data, and the process for gathering and cleaning it, can be found in our paper, published at EMNLP 2013.

Contact

The dataset was generated by Matthew Richardson, Christopher J.C. Burges, and Erin Renshaw. You can find our contact information on our respective home pages.

An Ongoing Resource

This website will maintain the set of published results based on MCTest, including links to the publication for each result. Authors are also encouraged to provide per-question scores for their algorithm, allowing others to more easily build on their work. We will make sure per-question scores available for general download alongside the primary MCTest dataset.

Sample

Here is a sample story and associated questions, taken from the MC500 training set:

James the Turtle was always getting in trouble. Sometimes he'd reach into the freezer and empty out all the food. Other times he'd sled on the deck and get a splinter. His aunt Jane tried as hard as she could to keep him out of trouble, but he was sneaky and got into lots of trouble behind her back.

One day, James thought he would go into town and see what kind of trouble he could get into. He went to the grocery store and pulled all the pudding off the shelves and ate two jars. Then he walked to the fast food restaurant and ordered 15 bags of fries. He didn't pay, and instead headed home.

His aunt was waiting for him in his room. She told James that she loved him, but he would have to start acting like a well-behaved turtle.After about a month, and after getting into lots of trouble, James finally made up his mind to be a better turtle.

1) What is the name of the trouble making turtle?
    A) Fries
    B) Pudding
    C) James
    D) Jane

2) What did James pull off of the shelves in the grocery store?
    A) pudding
    B) fries
    C) food
    D) splinters

3) Where did James go after he went to the grocery store?
    A) his deck
    B) his freezer
    C) a fast food restaurant
    D) his room

4) What did James do after he ordered the fries?
    A) went to the grocery store
    B) went home without paying
    C) ate them
    D) made up his mind to be a better turtle

URL: https://mattr1.github.io/mctest/

⇱ Machine Comprehension Test (MCTest)

Contact

An Ongoing Resource

Sample