November 13, 2017 Science Fiction Makes You Stupid, part 2
Two weeks ago I posted an excerpt of my essay “The Genre Effect: A Science Fiction (vs. Realism) Manipulation Decreases Inference Effort, Reading Comprehension, and Perceptions of Literary Merit,” which I co-wrote with cognitive psychologist Dan Johnson and is newly published by the journal Scientific Study of Literature. I titled the post “Science Fiction Makes You Stupid,” but a more accurate title would have been: “Readers Who Are Stupid Enough to be Biased Against Science Fiction Read Science Fiction Stupidly.”
I wasn’t expecting the post to draw attention beyond my usual readers, but it quickly became the third most viewed in this blog’s six-year history. I responded to a range of excellent comments last week, and, to continue that conversation, I am including below the four texts that Dan and I used in the experiment. Since they aren’t included in the actual journal publication, I can post them here unabridged.
The conclusions that Dan and I draw refer specifically to these texts and so only tentatively to the larger genres of science fiction and narrative realism, which are each vast and diverse. We needed short passages, no more than 1,000 words each, and so the texts are also necessarily flash fiction. We had originally tried to alter actual published stories, but that produced too many variables. At the sentence level, our two texts vary only according to setting-revealing words and phrases, which then produce two drastically different story worlds, one set in a contemporary small town, another in a futuristic space station. That means any generalizations we suggest about the larger genres are limited to setting-based definitions. While narrative realism may include a lot of things, it typically includes a contemporary, seemingly real-world setting. But science fiction is trickier. Only a subset includes a futuristic, other-worldly setting, so our study is limited to that subset.
By some definitions of science fiction, setting isn’t sufficient, since it is only a surface element. I tend to agree, though I still consider space westerns a form of science fiction. But the setting variations in the two experimental texts can produce more than just surface differences. When I tried a mini-version of the experiment unofficially with one of my advanced creative writing classes, one student inferred deeper levels of significance from the setting details, saying: “The main character feels a tension with humanity and artificial life; feels conflicted about the technological changes around him, the role that pain and messiness play in this structured, manipulated world.” No one described the setting of the narrative realism version as having as much significance.
The question of quality might come in too, since setting-defined science fiction, while still a subset of science fiction, might indicate a lower level of merit. Personally, I consider both texts to be at best mediocre. I wrote them solely for the purpose of experimental manipulation. I’ve published about four dozen short stories in literary journals, including one anthologized in Best American Fantasy, but I would never submit these texts anywhere, regardless of genre.
Finally, I’m including below the two additional texts that feature what Dan and I termed “theory-of-mind explanations.” These are topic-sentence-like statements usually at the beginning of paragraphs that declare overtly what characters are thinking and feeling. According to another study’s definition, “literary” fiction promotes theory-of-mind inferences, and so, we hypothesized, these statements should lower that inferencing and so also lower impressions of “literary” merit. That’s not what happened, which we discuss in the study.
For now, here are the four texts:
TEXT 1: “Narrative Realism”
Jim takes a deep breath, bracing himself before pushing open the glass door. Mrs. Moyers glances at him once and then drops her eyes to her menu, which she continues reading with improbable intensity as Jim walks past her booth. Sally—a woman Jim dated back in high school—squeaks the heel of her sneaker as she pivots and vanishes into the shadows of the kitchen. An older waitress Jim doesn’t know by name eventually plods over to his table, slaps a menu on his placemat without a word or glance, and then continues to the next booth where she chats and giggles a full minute before taking orders.
Jim’s letter to the editor appeared that morning. It isn’t a long piece, barely a half column, a fraction of the other Braxton Herald opinion page contributors. He was awake in bed just a few hours ago, staring at the shadows of his ceiling slowly ebbing to pink, when the delivery kid’s bicycle rattled onto the gravel of his driveway. The paper thunked against the screen door and skidded to the porch step where Jim leaned to pluck it up minutes later. There was no going back to sleep.
He reread the letter under the glare of the kitchen light with the coffee machine hissing behind him. The words looked so official, the font so formal, nothing like the sheet of hand-written stationary he’d slapped into an envelope the week before. His hand had cramped from gripping the pen so hard, each letter gouged into the yellow of the paper. He’d ripped up the first two drafts, scattering their fluttery shreds across the kitchen tiles after he missed the wastebasket both times. If only he’d flung the pen in too, kicked the metal can skidding across the carpet.
He didn’t skim to the bottom paragraph before crumpling his Herald into a jagged ball and shoving it into the same can. Now another copy sprawls across the unbussed table beside him, a smear of ketchup bloodying the masthead. That may be another jutting from Mrs. Moyers’ open purse, the pages rerolled into a tight cylinder like a weapon. When the waitress finally returns, she frowns down at him, sharpened pencil tip stabbing tiny wounds into her order pad.
“So,” she says, “find anything you like? Or is our selection too small for you?”
Jim feels his lip quiver as he tries to smile. “It all looks good to me.”His voice is wavering too. “But I think I’ll just have my usual. A cheeseburger.”
She slashes a few marks across the pad in her palm, her eyes still on Jim. “Sure that’s not too”—she pauses before puffing her voice into a posh accent: “Par-o-ch-i-al for you?”
Jim’s face is heating up now. “Well now,” he says, “where’d you go and learn a big word like that?” His laugh is sharp. He leans back, letting his arms settle across the booth back. The metal is cool but comfortable, just the right height. “Did you sound it out all by yourself?”
The waitress stands open-mouthed, eyes round, before retreating, her heels echoing against the tiled floor. Mrs. Moyers is suddenly standing and fishing bills from her purse. Sally—he didn’t notice her slinking back from the kitchen before—is drifting further down the counter, neck rigid as if afraid she might glance over by accident. He’d never said a mean word to her the whole semester they’d dated. But that was quite a few years and Herald editions ago.
Jim glances at the neighboring table again. The puddle of ketchup has expanded into the headline. The paper is ruined. Even if he jumped up and started mopping the pages, he couldn’t repair the damage. He kicks his boots up on the booth seat opposite him instead. His soles might leave a dirt smear, but nobody else will be sitting there anytime soon.
He smooths his hand across the wrinkles of his placemat, remembering how good it felt writing that letter. When Sally broke up with him senior year, he didn’t go and hunt down that runt of a junior she’d fallen for in her stupid Ceramics class. He didn’t jab the kid in the face till his bones of Jim’s fists throbbed and the calloused skin was wet with the other kid’s blood. Just yesterday his jerk of a neighbor let her poodle crap in his lawn again, not ten minutes after Jim had mown it, the air still thick with the scent of cuttings. But Jim didn’t yell at her, didn’t shout in her wrinkled face till his throat was raw.
Jim arranges his fork and knife along the edges of the placemat now, gets everything flush and straight, the fluorescent lights in the polished metal. His meal will be here soon enough.
TEXT 2: “Science Fiction”
Corporal Jones takes a deep breath, bracing himself before stepping through the airlock. Engineer Grady glances at him once and then drops her eyes to her mobile screen, which she continues reading with improbable intensity as Jones walks past her booth. Sally—a four-armed Alpha-Centarian Jones dated back at the Academy—squeaks the heel of her anti-gravity boot as she pivots and vanishes into the shadows of the galley. An ensign on server duty who Jim doesn’t know by name eventually plods over to his table, grudgingly projects a holographic menu over his placemat without a word or glance, and then continues to the next booth where she chats and giggles a full minute before taking orders.
Jones’ message to Command appeared that morning. It isn’t a long piece, barely a full screen, a fraction of the other Colony Morale Survey respondents. He was awake in his bunk just a few hours ago, staring at the gray of his sky-replicating ceiling slowly ebbing to pink, when the satellite dish mounted above his quarters started grinding into position to receive the day’s messages relayed from Earth. The download light on his mobile screen plinked as Jim logged on seconds later. There was no going back to sleep.
He reread it under the glare of his desk light with the air duct hissing behind him. The words looked so official, the font so formal, nothing like the electronic form he’d filled out by hand before slapping the SEND key the week before. His hand had cramped from gripping the stylus so hard, each letter denting the glow of the screen. He’d deleted the first two drafts, dragging their blinking icons to the cartoon wastebasket in the screen corner. If only he could have flung the stylus in too, kicked a real metal can skidding across the floor.
He didn’t skim to the bottom paragraph before stabbing the CLOSE button and dragging the Survey to the same wastebasket icon. Now another Survey copy blinks from the unbussed table beside him, a smear of ketchup bloodying the table’s built-in screen. That may be another Survey glowing from Grady’s screen, the pages as bright as an exploding plasma grenade.
When the ensign finally returns, she frowns down at him, steely stylus tip stabbing tiny wounds into her order screen. “So,” she says, “find anything you like? Or is our selection too small for you?”
Jim feels his lip quiver as he tries to smile. “It all looks good to me.” His voice is wavering too. “But I think I’ll just have my usual. A cheeseburger.”
She slashes a few marks across the screen in her palm, her eyes still on Jim. “Sure that’s not too”—she pauses before puffing her voice into a posh accent: “Par-o-ch-i-al for you?”
Jim’s face is heating up now too. “Well now,” he says, “where’d you go and learn a big word like that?” His laugh is sharp. He leans back, letting his arms settle across the booth back. The metal is cool but comfortable, just the right height. “Did you sound it out all by yourself?”
The ensign stands open-mouthed, eyes round, before retreating, her boots echoing against the metal floor. Grady is suddenly standing and fishing mess hall tokens from her beltpack. Sally—he didn’t notice her slinking back from the kitchen before—is drifting further down the counter, neck rigid as if afraid she might glance over by accident. He’d never said a mean word to her the whole semester they’d dated. But that was quite a few years and Command Surveys ago.
Jim glances at the neighboring table again. The puddle of ketchup has seeped into a crack in the screen. The viewer is ruined. Even if he jumped up and started mopping the table, he couldn’t repair the damage. He kicks his boots up on the booth seat opposite him instead. His soles might leave a smear, but nobody else will be sitting there anytime soon.
He smooths his hand across the computer-generated wrinkles of his holographic placemat, remembering how good it felt writing how he felt for once. When Sally broke up with him, he didn’t go and hunt down that runt of an android she’d fallen for in her stupid Programming class. He didn’t jab the ugly droid in the face till the bones of Jim’s fists throbbed and the calloused skin was wet with synthetic blood. Just yesterday his jerk of a bunkmate let his pet space squid crap on his desk again, not ten minutes after Jim had cleaned it, the air still thick with the scent of ammonia. But Jim didn’t yell at him, didn’t shout in his wrinkled face till his throat was raw.
Jim arranges his fork and knife along the edges of the placemat now, gets everything flush and straight, the fluorescent lights in the polished metal. His meal will be here soon enough.
TEXT 3: “Narrative Realism with Theory-of-Mind Explanations”
Jim knows everyone in the diner will be angry at him. He takes a deep breath, bracing himself before pushing open the glass door. Mrs. Moyers glances at him once and then drops her eyes to her menu, which she continues reading with improbable intensity as Jim walks past her booth. Sally—a woman Jim dated back in high school—squeaks the heel of her sneaker as she pivots and vanishes into the shadows of the kitchen. An older waitress Jim doesn’t know by name eventually plods over to his table, slaps a menu on his placemat without a word or glance, and then continues to the next booth where she chats and giggles a full minute before taking orders.
He knows why they’re mad. They consider him a traitor for insulting the town in their local newspaper. He called them all “parochial” and “small minded.” Jim’s letter to the editor appeared that morning. It isn’t a long piece, barely a half column, a fraction of the other Braxton Herald opinion page contributors. He was awake in bed just a few hours ago, staring at the shadows of his ceiling slowly ebbing to pink, when the delivery kid’s bicycle rattled onto the gravel of his driveway. The paper thunked against the screen door and skidded to the porch step where Jim leaned to pluck it up minutes later. He was too anxious. There was no going back to sleep.
He regretted writing the letter even before he reread it under the glare of the kitchen light with the coffee machine hissing behind him. The words looked so official, the font so formal, nothing like the sheet of hand-written stationary he’d slapped into an envelope the week before.
He’d been angry when he wrote it. His hand had cramped from gripping the pen so hard, each letter gouged into the yellow of the paper. He’d ripped up the first two drafts, scattering their fluttery shreds across the kitchen tiles after he missed the wastebasket both times. If only he’d flung the pen in too, kicked the metal can skidding across the carpet.
He wishes he’d never written the letter, but it’s too late; everyone’s seen it. He didn’t skim to the bottom paragraph before crumpling his Herald into a jagged ball and shoving it into the same can.
Now another copy sprawls across the unbussed table beside him. It’s as if they’re all thinking about killing him, the way a smear of ketchup is bloodying the masthead. That may be another jutting from Mrs. Moyers’ open purse, the pages rerolled into a tight cylinder like a weapon.
When the waitress finally returns, she frowns down at him, sharpened pencil tip stabbing tiny wounds into her order pad. “So,” she says, “find anything you like?” Then she quotes a word from his letter, in case there is any doubt she read it. “Or is our selection too small for you?”
Jim tries to be friendly, but he’s still nervous. His lip quivers as he tries to smile. “It all looks good to me.”His voice is wavering too. “But I think I’ll just have my usual. A cheeseburger.”
She slashes a few marks across the pad in her palm, her eyes still on Jim. “Sure that’s not too”—she pauses before puffing her voice into a posh accent, using another word from his letter: “Par-o-ch-i-al for you?”
Jim’s face is heating up now too, but not with embarrassment. He’s getting angry. “Well now,” he says, “where’d you go and learn a big word like that?”His laugh is sharp. Anger makes him confident. He leans back, letting his arms settle across the booth back. The metal is cool but comfortable, just the right height. “Did you sound it out all by yourself?”
The waitress is shocked. She stands open-mouthed, eyes round, before retreating, her heels echoing against the tiled floor.
Others heard his comment, and that makes them fear what he might say to them if given the chance. Mrs. Moyers is suddenly standing and fishing bills from her purse. Sally—he didn’t notice her slinking back from the kitchen before—is drifting further down the counter, neck rigid as if afraid she might glance over by accident. He’d never said a mean word to her the whole semester they’d dated. But that was quite a few years and Herald editions ago.
His friendships with everyone in Braxton are ruined. He just has to accept that he’s friendless now. Jim glances at the neighboring table again. The puddle of ketchup has expanded into the headline. The paper is ruined too. Even if he jumped up and started mopping the pages, he couldn’t repair the damage. He kicks his boots up on the booth seat opposite him instead. His soles might leave a dirt smear, but nobody else will be sitting there anytime soon.
Maybe writing that letter wasn’t such a bad idea. He’d spent too much of his life not letting himself get angry. He smooths his hand across the wrinkles of his placemat, remembering how good it felt writing how he felt for once. When Sally broke up with him senior year, he didn’t go and hunt down that runt of a junior she’d fallen for in her stupid Ceramics class. He didn’t jab the kid in the face till his bones of Jim’s fists throbbed and the calloused skin was wet with the other kid’s blood. Just yesterday his jerk of a neighbor let her poodle crap in his lawn again, not ten minutes after Jim had mown it, the air still thick with the scent of cuttings. But Jim didn’t yell at her, didn’t shout in her wrinkled face till his throat was raw.
At least now things are the way they should be. Jim arranges his fork and knife along the edges of the placemat now, gets everything flush and straight, the fluorescent lights in the polished metal. His meal will be here soon enough.
TEXT 4: “Science Fiction with Theory-of-Mind Explanations”
Corporal Jones knows everyone in the space station mess hall will be angry at him. He takes a deep breath, bracing himself before stepping through the airlock. Engineer Grady glances at him once and then drops her eyes to her mobile screen, which she continues reading with improbable intensity as Jones walks past her booth. Sally—a four-armed Alpha-Centarian Jones dated back at the Academy—squeaks the heel of her anti-gravity boot as she pivots and vanishes into the shadows of the galley. An ensign on server duty who Jim doesn’t know by name eventually plods over to his table, grudgingly projects a holographic menu over his placemat without a word or glance, and then continues to the next booth where she chats and giggles a full minute before taking orders.
He knows why they’re mad. He called the crew “parochial” and “small minded.” They consider him a traitor for insulting the base in an official report. Jones’ message to Command appeared that morning. It isn’t a long piece, barely a full screen, a fraction of the other Colony Morale Survey respondents. He was awake in his bunk just a few hours ago, staring at the gray of his sky-replicating ceiling slowly ebbing to pink, when the satellite dish mounted above his quarters started grinding into position to receive the day’s messages relayed from Earth. The download light on his mobile screen plinked as Jim logged on seconds later. He was too anxious. There was no going back to sleep.
He regretted writing the report even before he reread it under the glare of his desk light with the air duct hissing behind him. The words looked so official, the font so formal, nothing like the electronic form he’d filled out by hand before slapping the SEND key the week before. He’d been angry when he wrote it. His hand had cramped from gripping the stylus so hard, each letter denting the glow of the screen. He’d deleted the first two drafts, dragging their blinking icons to the cartoon wastebasket in the screen corner. If only he could have flung the stylus in too, kicked a real metal can skidding across the floor.
He wishes he’d never answered the questionnaire, but it’s too late; everyone’s seen it. He didn’t skim to the bottom paragraph before stabbing CLOSE button and dragging the Survey to the same wastebasket icon. Now another Survey copy blinks from the unbussed table beside him. It’s as if that report will get him killed, the way a smear of ketchup is bloodying the table’s built-in screen. That may be another Survey glowing from Grady’s screen, the pages as bright as an exploding plasma grenade.
When the ensign finally returns, she frowns down at him, steely stylus tip stabbing tiny wounds into her order screen. “So,” she says, “find anything you like?” Then she quotes a word from his report, in case there was any doubt she read it. “Or is our selection too small for you?”
He’s trying to be friendly, but he’s still nervous. Jim’s lip quiver as he tries to smile. “It all looks good to me.” His voice is wavering too. “But I think I’ll just have my usual. A cheeseburger.”
She slashes a few marks across the screen in her palm, her eyes still on Jim. “Sure that’s not too”—she pauses before puffing her voice into a posh accent, using another world from his report: “Par-o-ch-i-al for you?”
Jim’s face is heating up now too, but not with embarrassment. He’s getting angry.“Well now,” he says, “where’d you go and learn a big word like that?” Anger makes him confident. His laugh is sharp. He leans back, letting his arms settle across the booth back. The metal is cool but comfortable, just the right height. “Did you sound it out all by yourself?”
The ensign is shocked. She stands open-mouthed, eyes round, before retreating, her boots echoing against the metal floor. Others heard his comment, and that makes them fear what he might say to them if given the chance. Grady is suddenly standing and fishing mess hall tokens from her beltpack. Sally—he didn’t notice her slinking back from the galley before—is drifting further down the counter, neck rigid as if afraid she might glance over by accident. He’d never said a mean word to her the whole semester they’d dated. But that was quite a few years and Command Surveys ago.
His relationships with everyone in the base are ruined. He just has to accept that he’s friendless now. Jim glances at the neighboring table again. The puddle of ketchup has seeped into a crack in the screen. The viewer is ruined too. Even if he jumped up and started mopping the table, he couldn’t repair the damage. He kicks his boots up on the booth seat opposite him instead. His soles might leave a smear, but nobody else will be sitting there anytime soon.
Maybe writing that report wasn’t such a bad idea. He’d spent too much of his life not letting himself get angry. He smooths his hand across the computer-generated wrinkles of his holographic placemat, remembering how good it felt writing how he felt for once. When Sally broke up with him, he didn’t go and hunt down that runt of an android she’d fallen for in her stupid Programming class. He didn’t jab the ugly droid in the face till his bones of Jim’s fists throbbed and the calloused skin was wet with synthetic blood. Just yesterday his jerk of a bunkmate let his pet space squid crap on his desk again, not ten minutes after Jim had cleaned it, the air still thick with the scent of ammonia. But Jim didn’t yell at him, didn’t shout in his wrinkled face till his throat was raw.
At least now things are the way they should be. Jim arranges his fork and knife along the edges of the placemat now, gets everything flush and straight, the fluorescent lights in the polished metal. His meal will be here soon enough.
- 13 comments
- Posted under Uncategorized
Permalink # Anti-SF Bias Leads to Poorer Reading Comprehension, Study Finds | The Digital Reader said
[…] saw the academics work with around 150 participants who were given a text of 1,000 words to read. In each version of the text, a character enters a public eating area and interacts with the people there, after his negative […]
Permalink #
Alicia Nordwell
said
Just using certain terminology does not make a story science fiction. Maybe, if you’d used actual science fiction instead of some poorly-converted piece using what you believe constitutes science fiction terms that included such writing fails as a light making a sound, readers wouldn’t exhibited such poor comprehension and exhibit such a bias against the actual genre.
Permalink #
Chris Gavaler
said
I addressed this in other comments and in another post, but, yes, I agree that setting-defined SF does not define all SF. It does define some though, and so our results apply only to that subcategory. I originally attempted what you suggest, using an “actual science fiction” piece (one by Robert Heinlein), but it was impossible to create a meaningful variant, that is one that differs in only one aspect, the one being tested. That’s the difficulty of designing a scientific study. That said, perhaps you’re right. Perhaps the poor comprehension we documented is only a product of our specific texts. Impossible to know without conducting more studies.
Permalink #
Clinton Keith Borror
said
I suspect that your study, at least as it is now, isn’t going to lead to any insights into whether or not science fiction reduces reading comprehension. I suspect your study, as it is now, will describe whether or not poorly explained scenes, such as writing about a dog wearing a toaster for a hat, will reduce reading comprehension, because the science fiction text is nothing like any science fiction text I have ever seen that was not a rough draft. The reader is fully capable of relating to and fully understanding the “NARRATIVE REALISM” text. I don’t understand some of the parts of the SCIENCE FICTION text. They baffle me. A good example is: Why is someone eating a cheeseburger on a space station that contains a mixture of humans and aliens? Star Trek might explain such an event through machines that can turn energy into any food. Other science fiction stories may include similarly bizarre events as a source of comedy, but your science fiction text contains no comedy or explanation so as a reader, I find myself feeling like I am, rather than reading about actual people and aliens, I am reading about poorly written characters of a rough draft who I don’t care about enough to empathize with. I generally love science fiction though. I empathize with aliens, and man-eating monsters and anything that feels real to me, but I don’t empathize with the characters in the science fiction texts, and it’s not because they’re on a space station containing aliens. It’s because the characters don’t feel real to me. I made another post, but it was more poorly thought out than this one.
Permalink #
Chris Gavaler
said
That’s a fair critique, Clinton. If you found the SF characters less engaging, that would reduce overall effort, etc. I do think though that SF, as part of world-building, often has a range of under-explained elements, things that require greater amount of inferencing from a reader (assuming there’s no infodumping in the story). Your question above is the kind of theory-of-world effort we were trying to trigger: “Why is someone eating a cheeseburger on a space station that contains a mixture of humans and aliens?” And then you created your own inference based on the subgenre the story most resembled: “Star Trek might explain such an event through machines that can turn energy into any food.” That’s what we termed theory-of-world, and we were trying to trigger it to a higher degree than in the NR text, where the world is (mostly) a given and so requires little inferencing. Our hypothesis is that many reader, obviously non-SF readers, are biased against stories that require high theory-of-world inferencing because they don’t associate theory-of-world with “literary.” And in your case, the effort you used on theory-of-world reduced the realism of the characters–even though the characters behaved identically and should otherwise produce the same theory-of-mind inferences about their internal thoughts and feelings, which is typically what creates a realistic character
Permalink #
Michael Mooney
said
I think a big flaw in your study is that the Sci-Fi story is simply worse then the narrative realism story, because you don’t put the effort into detailing the setting. There are lots of unanswered questions that we have to look at ourselves.
Is the 4-armed person an alien, or a mutant? Who is Command? Are they on a Space Station? If they are, then why is there a Colony Morale Survey? Are they Military? Is Jones a veteran? And so on, and so forth.
Not only that, but a lot of stuff doesn’t translate over all that well.
Why is Jones manually writing on a computer? That’s just weird and disjarring. How is the ketchup effecting the holoscreen anyways? I mean, computers are valuable. You think that’d be a priority to clean up. While a newspaper getting dirty, well who cares? Aren’t survey’s supposed to be private? Was it published for some reason? Things don’t make sense in the Sci-Fi version and it really effects how good the story is.
As a result, the Sci-Fi story is of lower quality, and of course people will engage less with a lower quality story. And honestly, some of the ‘translations’ to sci-fi felt really forced. As if you had simply taken the first sci-fi like word that popped in your head and replaced the Narrative Realism word with that.
Permalink #
Chris Gavaler
said
I think the NR text works better than the SF too, but your reasons for prefering it are one of the points of the study: “There are lots of unanswered questions that we have to look at ourselves.” That’s what we mean by theory-of-world, the kinds of big inferences that SF readers often have to make while NR readers don’t because the NR world is simply the world as they know it direclty. So the SF readers had to exert effort in two directions, theory of world and theory of mind, while the NR readers only had to worry about theory of mind. And yet–and this seems key–the SF readers reported exerting less overall effort than the NR readers. The SF readers left the story feeling that it was simpler, when, as you point out very effectively above, it’s actually a lot more complex because of all those unanswered questions. Also, the forced translations you note would be unknown to either sets of readers since each read only one story. But I do agree that some of those translation feel cliche or silly to me too, and that, rather than the higher theory of world demands, could account for the lower impression of literary quality. I’m working on a new pair of texts right now that I hope will correct for these issues when we run a similar experiment next summer.
Permalink #
Michael Mooney
said
I don’t disagree with the idea that the SF readers have to put in more effort, but I think the reason that they are putting in less overall effort is because it’s a worse story.
People react to quality, and when something is low quality, they don’t really try. Because why should you care about the characters, plot, or setting, when the author clearly didn’t care about them? So the reader is more likely to skim, or to simply not care about what they’ve read. It won’t make an impression on them, and they won’t put effort in because they don’t want to.
And I really have to emphasize how bad the sci-fi version was. For example, an Ensign is a decently high rank in the military. A mere corporal talking back to them is nonsensical, as is the ensign being on a serving duty at all.
Basically it’s really obvious that less effort was put into the sci-fi story, and it really shows. Particularly when you read both stories side by side, it looks like you wrote the narrative realism story first, and then merely translated everything with a sci-fi word. Rather then write two stories in two different settings with the same plot. So of course the readers aren’t going to engage as much when their is a vast difference in quality between the two versions.
In summary, the method you used to write your sci-fi version sabotaged your results, because that method will result in a low quality sci-fi story, and people don’t respond well to low quality.
Permalink #
Chris Gavaler
said
You may well be right, Michael. The two texts I’m drafting now will avoid this problem because I’m writing them simultaneously and their word-for-word content will have an even higher overlap. While I also intend them both to have a high level of quality, their level will be the same regardless. If they then produce different results than the first texts, the reason will likely be what you describe. If they produce the same results, then theory-of-world effort and bias against anything with an SF element will look like the determining factors. Until we run the experiment, we can’t know.
Permalink #
Michael Mooney
said
Well I look forward to your future work.
Though, as a suggestion, why don’t you ask your readers to rate the quality of your story as one of your questions? That might give you some idea how much of an impact that a genre shift is having on the quality of the story. Or the popularity of the genre perhaps.
Permalink #
Chris Gavaler
said
We did, and the SF readers significantly rated the SF story lower than the NR readers rated the NR story. But, as you pointed out, there’s no way to determine WHY they ranked it lower–was it theory-of-world or other qualities? Hopefully I’ll be able to find out next time.
Permalink #
Matthew B. Tepper
said
Your “test” reminds me of parallel texts, equally mediocre and derivative, that were cited back in the 1950s to indicate a specifically bad and unimaginative form of science fiction. First, the “regular” example:
“Hoofs [sic] drumming, Bat Durston came galloping down through the narrow pass at Eagle Gulch, a tiny gold colony 400 miles north of Tombstone. He spurred hard for a low overhang of rim-rock … and at that point a tall, lean wrangler stepped out from behind a high boulder, six-shooter in a sun-tanned hand.
“‘Rear back and dismount, Bat Durston,’ the tall stranger lipped thinly. ‘You don’t know it, but this is your last saddle-jaunt through these here parts.'”
And the supposedly science fictional example:
“Jets blasting, Bat Durston came screeching down through the atmosphere of BBllzznaj, a tiny planet seven billion light years from Sol. He cut out his super-hyper-drive for the landing … and at that point, a tall, lean spaceman stepped out of the tail assembly, proton gun-blaster in a space-tanned hand.
“‘Get back from those controls, Bat Durston,’ the tall stranger lipped thinly. ‘You don’t know it, but this is your last space trip.'”
And the tagline to both? “Sound alike? They should — one is merely a western transplanted to some alien and impossible planet. If this is your idea of science fiction, you’re welcome to it! YOU’LL NEVER FIND IT IN GALAXY!” (That was the magazine, founded in 1950, which featured these comparative examples on its back covers for several early issues.)
In other words, I think your examples are jejune and ridiculous, and that by extension your research — and how can I say this tactfully — is not especially rigorous.
Permalink #
Chris Gavaler
said
As I commented above, yes, neither text is good fiction–which poses no problem to our research as long as both are, as you say, equally “jejune and ridiculous.” But the SF version may be objectively worse than the NR, and that would pose a problem. One we’re addressing in our current research where I’ve created one text and alter it by one appearance of only one word in each version: “daughter” in the NR, “robot” in the SF. Otherwise the two texts are word-for-word identical. I don’t see a way to get any more rigorous.