Utah Foreign Language Association · Utah Valley University · February 14, 2020

How to Give Effective Feedback
to Korean Learners

Meaningful, manageable, constant, and timely

Subin OhBrigham Young University
둘제Sp는 인관Sp에P 행복과과 관련된문제다.
Overview

Today

  1. The problemWhy Korean learners struggle most with writing
  2. Dynamic written corrective feedbackWhat it is and what we know from ESL
  3. The study19 intermediate learners, 12 weeks
  4. FindingsAccuracy, fluency, complexity
  5. For your classroomLimitations and next steps
01 · The problem

Learners call writing the hardest skill

Jeon, 2004; Jin, 2009; Yoon, 2013
Listening
Speaking
Reading
Writinghardest!

Learners ask for a better way to improve their writing, yet writing is rarely taught successfully in KFL classrooms.

01 · The problem

Why writing falls behind

1

Motivation

Learners come to Korean through family, friends, and Hallyu. Their reasons rarely involve writing.

Jee, 2015; Wang, 2016
2

Difficulty

Writing demands accuracy as well as fluency and complexity, with a low tolerance for error.

Kang, 2009; Yoon, 2013
3

Classroom time

Courses stress oral communication, so writing gets a small share of class time.

Jin, 2015; Jeong, 2012
01 · The problem

Teachers need a method they can use tomorrow

What teachers face

  • Writing instruction is a small part of teacher training
  • Short workshops are often the only further training
  • Methods tested in research are hard to carry into real classrooms
Jin, 2009; Yoon, 2013

What is needed

A clear, efficient method that any language teacher can start using right away, without long-term training.

01 · The problem

Why written corrective feedback?

It builds accuracy

Since Truscott (1996), most studies find that WCF helps learners write more accurately.

Ashwell, 2000; Ferris, 2004; Van Beuningen et al., 2008

It saves class time

Writing, feedback, and revision happen individually, outside the class period.

It is individual

Each learner gets feedback on their own errors and becomes aware of their own patterns.

02 · Background

Does WCF work? A long debate

1984
Semke

141 learners of German: no method of correction improved accuracy

1996
Truscott

WCF is ineffective and possibly harmful; abandon it

2001 – 2004
Ferris; Chandler

Learners revise more accurately with feedback

2010 →
Evans; Hartshorn

The question shifts from whether to how

02 · Background

Which kind of feedback?

Scope
Focusedone error typevs.Comprehensiveall errors
Findings for focused feedback “may be too focused to be practical.”Evans et al., 2010, p. 386
Form
Directteacher writes the correctionvs.Indirectteacher marks, learner corrects

No consensus in ESL or in KFL; a multi-step feedback-revision cycle works better than one-time feedback (Chandler, 2003).

02 · DWCF

Dynamic written corrective feedback

Comprehensive, indirect, coded feedback on short, frequent writing, repeated until each text is error-free.Evans et al., 2010; Hartshorn et al., 2010 · skill acquisition theory, DeKeyser, 2007

1
Write

A paragraph on a set topic in 10 minutes, in class

→
2
Code

Teacher marks errors with codes; returns it next class

→
3
Revise

Learner corrects the draft and resubmits

↺
4
Repeat

Until the paragraph is error-free

Learners also keep tally sheets of their error types and error rates.

02 · DWCF

Four qualities of good feedback

Meaningful

Holistic feedback; learners note the errors they make

Manageable

One paragraph, written in 10 minutes

Constant

Writing several days every week

Timely

Feedback arrives the next class day

Feedback on one or two long essays a semester cannot do this.

02 · DWCF

Evidence so far, and the gap

Where DWCF has been tested

Evans et al., 2010ESL, intensive program
Hartshorn et al., 2010ESL, intensive program
Evans et al., 2011ESL, university
Hartshorn & Evans, 2012, 2015ESL, intensive program
Kurzer, 2018ESL, three levels
Akiyama & Fleshler, 2013Japanese: grammar items only

The gap

Mostly advanced ESL learners. Never tested with learners of Korean, or at the intermediate level.

03 · The study

Research questions

RQ1

To what extent does DWCF increase linguistic accuracy for intermediate learners of Korean as a foreign language?

RQ2

Will DWCF affect the fluency and complexity of their writing?

03 · The study

Participants

19

intermediate learners in a third-year Korean course at a private university in the western U.S.

  • All native English speakers
  • Most had lived in South Korea for 18–24 months
  • Same teacher Tuesday and Thursday; separate sections Monday, Wednesday, Friday
Treatment · DWCF
n = 9  ·  5 men, 4 women
Control · traditional
n = 10  ·  6 men, 4 women

Intact sections; students were not randomly assigned.

03 · The study

Two ways to write for 12 weeks

Treatment · DWCFControl · traditional
WritingParagraph, in classEssay, take-home
Time per draft10 min37.5 min
Number of writings306
Total writing time300 min225 min
FeedbackIndirect, codedDirect
RevisionUntil error-freeOnce
03 · The study

Error codes built for Korean

Writing style
SpSpelling
hnrHonorific form
WCWord choice
STSentence structure or translated sentence
engWrite this in Korean
?Meaning unclear
Grammar
PParticle
irrIrregular inflection
TTense
C/ACausative vs. active
P/APassive vs. active
ㅇㄱㅈ이/그/저
Grammatical word choice
but-는데, -지만
b/c-아서/어서, -(으)니까, -기 때문에
and-고, -아서/어서, -(으)ㄴ데/는데
NP-는 것, -기

24 codes in all, written for features English codes do not cover.

03 · The study

What a coded sentence looks like

Draft
둘제Sp는 인관Sp에P 행복과 과 관련된 문제다.
Second thing is a problem related to happiness of human.
Revision
둘째는 인간의 행복과 관련된 문제다.

Codes point to the error; the learner finds the correction.

03 · The study

Cycles overlap through the week

Mon
Wed
Fri
Mon
Wed
P1
Write
Feedback
Revise
Feedback
Error-free
P2
Write
Feedback
Revise
Feedback
P3
Write
Feedback
Revise

A new paragraph starts every class day while earlier ones are still being revised.

03 · The study

Pre- and post-tests

Three 10-minute paragraphs each, on paired topics
Pre-testPost-test
내 인생에 영향을 준 인물A person who has had an impact on my life내 인생에서 가장 중요한 사람The most important person in my life
내 인생의 목표The purpose of my life꼭 이루고 싶은 것The thing I really want to achieve
역사적인 사건A historical event이 세상에 일어난 특별한 사건A special event that happened in the world
03 · The study

What we measured

Accuracy

  • Errors per sentence
  • Error-free clauses per clause
  • Error-free syllables per syllable

Fluency

  • Sentences
  • Eojeols
  • Syllables

Complexity

  • Clause-to-word ratio
  • Subordinate clauses per clause
  • Coordinate and subordinate clauses per sentence

Highlighted: measures not used in earlier DWCF research.

03 · The study

Why count syllables in Korean?

강
ㄱ + ㅏ + ㅇ

Letters stack into one syllable block, so syllables are easy to tell apart.

인관에
2 of 3 syllables wrong · the clause counts as wrong

Small errors in spelling and inflection lower the score a little, not a whole clause.

Error-free syllables give learners credit for what they got right.

03 · The study

Analysis

Two raters

Both counted errors, error-free clauses, and error-free syllables in every test paragraph.

.98error-free syllables
.98error-free clauses
.99errors per sentence

Pearson’s r between raters, all p < .001

Test

Mixed-model repeated measures ANOVA: group (treatment, control) × time (pre, post).

04 · Findings · RQ1 accuracy

Fewer errors per sentence

2.002.503.003.504.004.50Pre-testPost-test2.692.984.082.38
Treatment (DWCF)Control
F(1, 17) = 26.28
p < .001
η²p = .61

The treatment group went from 4.08 to 2.38 errors per sentence. The control group did not improve.

04 · Findings · RQ1 accuracy

More error-free clauses

.300.400.500.600Pre-testPost-test.486.460.321.572
Treatment (DWCF)Control
F(1, 17) = 24.42
p < .001
η²p = .59

Error-free clauses rose from .321 to .572 of all clauses with DWCF, and fell slightly in the control group.

04 · Findings · RQ1 accuracy

More error-free syllables

.800.840.880.920Pre-testPost-test.874.859.808.894
Treatment (DWCF)Control
F(1, 17) = 21.51
p < .001
η²p = .56

The treatment group rose from .808 to .894. Control pre-test (.874) is recomputed from the group total; the thesis table prints .087.

04 · Findings · RQ1 accuracy

All three accuracy measures agree

Errors per sentence
η²p = .61
Error-free clauses
η²p = .59
Error-free syllables
η²p = .56

Large effects on every measure: DWCF helped intermediate KFL learners write more accurately.

04 · Findings · RQ2

Fluency and complexity held steady

MeasureGroup × time
FluencySentencesF(1, 17) = 0.03, p = .875
EojeolsF(1, 17) = 0.13, p = .728
SyllablesF(1, 17) = 0.81, p = .381
ComplexityClause-to-word ratioF(1, 17) = 0.05, p = .822
Subordinate clauses per clauseF(1, 17) = 0.97, p = .340
Clauses per sentenceF(1, 17) = 0.74, p = .402

Thirty short, timed paragraphs did not cost learners fluency or complexity.

05 · Discussion

What this adds

A new language

The accuracy gains found in ESL hold for learners of Korean.

Comprehensive works

Systematic feedback on all errors was effective, matching the many errors learners make in real writing.

The syllable as a unit

Error-free syllables offer a finer measure for Korean writing research.

05 · Discussion

Limitations

More time on task

The treatment group wrote and revised for longer, which may explain part of the gain.

Writing
300240*
Revision
662*147*

Minutes over 12 weeks · * estimated by participants

Design

  • Small sample: 9 and 10 learners
  • Intact classes, not random assignment
  • Different section teachers; the treatment teacher was the researcher
05 · Discussion

Next steps

Research

  • Other settings: universities in Korea, King Sejong Institutes
  • Other proficiency levels
  • Longer studies, beyond one semester
  • More domains of accuracy

In your classroom

  • Short paragraphs, often
  • Coded, indirect feedback by the next class
  • Revise until error-free
  • Learners track their own errors

Thank you

Questions and comments are welcome.

Subin Ohosubin@arizona.edu

Oh, S. (2020). The effect of dynamic written corrective feedback for learners of Korean [Master’s thesis, Brigham Young University]. BYU ScholarsArchive.