voice session · ko-KRSLAT Roundtable 2025
Research presentation

ChatGPT as a Conversation Partner

for Learners of Korean
Subin Oh, Sojeong Chun, & Jieun Ryu
University of Arizona
24th SLATSA RoundtableFebruary 7, 2025
Learner
안녕하세요! 저는 한국어를 배우고 있어요.Hi! I'm learning Korean.
ChatGPT · Voice
반가워요! 오늘은 무슨 이야기를 해 볼까요?Nice to meet you! What shall we talk about today?
listening…
introduction
Research purpose

Can a voice chatbot give Korean learners more time to talk?

Goal

Explore the potential of ChatGPT-3.5 Voice for conversation practice among Korean as a Foreign Language (KFL) learners.

Significance

Address gaps in speaking opportunities for learners of Less Commonly Taught Languages (LCTLs).

research questions
Research questions

Two questions

RQ1 · perceptions
How do KFL learners perceive ChatGPT-3.5 Voice as a conversation partner?
RQ2 · challenges
What challenges arise in the conversation between KFL learners and ChatGPT-3.5 Voice?
method / context
Context and participants

A sixth-semester KFL course at a U.S. university

Participants

12 of 14 enrolled students consented to participate.

Native languages

11 English  ·  1 Chinese

Prior experience

Limited prior use of ChatGPT for language learning.

method / procedure
Procedure

Three stages

Stage 1

Familiarization

Initial interactions with ChatGPT and feedback collection.

→
Stage 2

Instruction

AI literacy session; guidelines for effective ChatGPT use.

setting the contextusing the hold buttoncommands to stay in role
→
Stage 3

Implementation

Three 5–10-minute conversations, then a post-survey.

method / data
Data collection and analysis

From questions to data

Research question
Data source
Analysis
Students’ perceptions of ChatGPT-3.5 Voice as a conversation partner
Post-survey
Technology Acceptance Model (TAM)
Qualitative: two researchers’ thematic coding, cross-checking, finding patternsQuantitative: descriptive statistics
Challenges arising in conversations
Auto-generated transcripts
rq1 / framework
Technology Acceptance Model · Davis, 1989

Why TAM?

  • Widely used in technology research; increasingly applied in foreign language education as new tools emerge.
  • Assesses students’ perceptions of new technologies and their willingness to adopt them.
Perceived Usefulness (PU)
Perceived Ease of Use (PEOU)
Behavioral Intention (BI)
Actual System Use
rq1 / framework
Post-survey design

Modified Technology Acceptance Model 3

Academic Relevance
Output Quality
Technology Anxiety
Perceived Enjoyment
Objective Usability
Perceived Usefulness (PU)
Perceived Ease of Use (PEOU)
Behavioral Intention (BI)
Actual System Use
rq1 / perceptions
Perceived UsefulnessPerceived Ease of UseBehavioral Intention

Useful, with a catch

Academic relevance
that’s what happens in real life.
81.81%believe future 300-level students would benefit
  • Fluency & real-world practice: helps improve spontaneous conversation.
  • Cognitive challenge: students had to rely only on linguistic cues.
Output quality
like talking to an acquaintance you don’t know well.
2.82mean, SD = 0.94 · ambivalent authenticity
  • Native-like speed and fluency, but lacks human-like conversation dynamics.
  • Like chatting formally with an employee, an old acquaintance, or even Google.
rq1 / perceptions
Perceived UsefulnessPerceived Ease of UseBehavioral Intention

Easy to reach, hard to steer

Objective usability
just decide to give up.
  • Strengths: ubiquity, accessibility, and controllability.
  • Role-play limitations: forgets prompts, breaks character, and returns to its default role of providing explanations.
  • Not learner-friendly: difficult to sustain conversations once issues arise.
Enjoyment & anxiety
annoyed but satisfied when I complete it.
  • Robot-like interactions: mentally and physically odd.
  • Confidence booster: no judgment, fear of failure, or embarrassment.
  • Encouraging engagement: made students more proactive in conversations.
rq1 / perceptions
Perceived UsefulnessPerceived Ease of UseBehavioral Intention

A partner alongside people, not instead of them

Positive inclination for future use · 5-point Likert scale
3.64mean (SD = 1.43)
1
2
3
4
5
it would go well paired alongside
[the human interaction.]
Supplementary resource

Useful when classes or human partners are unavailable (e.g., summer, after graduation), but not a replacement for human interaction.

method / data
Data collection and analysis

Now, the conversations themselves

Research question
Data source
Analysis
Students’ perceptions of ChatGPT-3.5 Voice as a conversation partner
Post-survey
Technology Acceptance Model (TAM)
Qualitative: two researchers’ separate thematic codingQuantitative: descriptive statistics
Challenges arising in conversations
Auto-generated transcripts
rq2 / challenges
Challenges in the conversations · share of all coded challenges

Most problems were technical

Technical issues · 71.8%
20.3%
7.7%
Technical issues

Speech recognition errors, cut-off utterances, anomalous responses.

Misunderstandings

Semantic and pragmatic errors, lengthy responses.

AI affordance

Learners frustrated, skipping answers, changing topics.

rq2 / technical issuesspeech recognition errors
Technical issues · speech recognition

The transcript heard something else

Transcript errors
128
Heard as a foreign language
20
Korean heard as English
16
Languages the 20 utterances were heard as
Japanese · 7 of 20MandarinSwedishIndonesianMāoriSpanishThaiCantonese…
rq2 / technical issuesheard as a foreign language
Example

Korean in, Japanese out

transcriptあんに、本国のファンにお得になれよ。detected: Japanese
[Anni, hongoku no fan ni otoku ni nare yo.] (*Become beneficial to the fans of your home country.)
What the student most likely said
아니, 한국어로 “funny” 어떻게 말해요?[Ani, hankwukelo “funny” ettehkey malhayyo?] I mean, how can I say “funny” in Korean?
rq2 / technical issuesanomalous responses
Anomalous responses · case 1

Sounds like the end of a YouTube video

ChatGPT
좋아요와 구독, 댓글 부탁드려요! 즐거운 시간 되셨길 바라요!Please like, subscribe, and leave a comment! I hope you had a great time!
Student 1 (transcript)
Thanks for watching.
Student 2 (transcript)
이 영상은 ChatGPT의 유료광고를 포함하고 있습니다.This video contains paid advertisements for ChatGPT.
Student 3 (transcript)
자막은 설정에서 선택하실 수 있습니다.Subtitles can be selected in the settings.
Pattern

Phrases common on YouTube appear where no one said them.

rq2 / technical issuesanomalous responses
Anomalous responses · case 2

A sentence no learner would say

Student 4 (transcript)
UberInfo는 다른 채널의 채널 작성 내부 레벨에 따라 수정된 자료를 추가로 다시 볼링을 해 보세요.UberInfo will further re-bowl the revised material based on the channel authoring internal level of the other channel.

Out of context

Higher than students’ capacity

Meaning unclear, inadequate, or ungrammatical

Sounds like a manual or a template

rq2 / misunderstandingssemantic · pragmatic · lengthy response
Misunderstandings · lengthy response

Too much, too fast

ChatGPT
그렇군요! 해수욕을 즐기는 것이 익숙하시군요. 싱가포르의 해변 경험을 살려서 한국의 해수욕을 더욱 즐겁게 즐기실 수 있을 거예요. 한국의 해수욕장은 다양한 풍경과 활동으로 여행객들을 매료시킬 거예요. 어떤 종류의 해변 활동을 즐기시나요? 일본 같은 수영, 또는 해변을 산책하는 것을 좋아하시나요?
Student
간단한 단어로 말해주세요. 지금 저는 알 수 없어서 문제 너무 어려워요. 이제 간단한 문장을 만들어주세요Please use simpler words. I can’t understand now, this is too hard for me. Please make shorter sentences from now on.
rq2 / ai affordance
AI affordance

Talking to a machine changes the rules

abrupt topic change
… 너는 “Turbulence”를 다른 곡보다 특별한 이유가 있니?Do you have a special reason for liking “Turbulence” more than other songs?
개인적으로 친한 연예인 있어요?Do you have a personally close celebrity?
express frustration
aahshshahahhahshhahahahahhahahahhshshahahshs
choose not to answer
  • Might be considered pragmatically inappropriate in human-to-human interaction.
  • Suggests learners felt a reduced sense of social obligation when conversing with an AI.
discussion
Discussion

ChatGPT, the tenacious communicator

Human interaction
AA speaks
?B doesn’t understand
?B requests clarification
↺A repairs or rephrases
✓Conversation continues smoothly.
ChatGPT interaction
SUser speaks
×Misunderstanding
…Keeps responding anyway
?User doesn’t understand; frustration
⨯Conversation circles around or stops.
discussion
Discussion · example

It answers instead of asking

바꿀까pakkwulkka · “if I should change”
vs.
바쿠가pakhwuka · ?
Student
지금은 언어학 전공해요. 하지만 나중에 *바쿠가 생각하고 있어요.Right now, I’m majoring in linguistics. But later, I’m thinking about Pakhwu. / But later, Pakhwu is thinking.
ChatGPT
… 어떤 분야에서 바쿠를 공부하고 싶으세요?Which field do you want to study pakhwu in?
implications
Implications and suggestions

What this means for teaching

01 · potential

Speaking practice for LCTLs

ChatGPT-Voice can enhance LCTL speaking with proper guidelines: a fluent partner anytime, anywhere, in a safe environment.

02 · pronunciation

Errors as a diagnostic

Frequent recognition errors may point to pronunciation training that has not been emphasized. Misrecognition patterns could reveal key difficulties.

03 · design

Generative AI for learners

Learning chatbots are restrictive; ChatGPT is flexible but not designed for learners. Training on learner-specific data could make it learner-friendly.

session endedSLAT Roundtable 2025
Thank you!

Questions and comments are welcome.

Subin Oh
osubin@arizona.edu
Sojeong Chun
sojeongj@arizona.edu
Jieun Ryu
University of Arizona