Revision
Report for: ms
111-v2_debriefing-as-learning
Publication recommendations at
this stage for v2
Rev A = 2a. Accept, but require
substantial revision in content and/or form (detailed
in my commentary), and send back revised version
to me for re-evaluation.
Rev C = 3a. Accept with minor modifications to
substance and/or form (see my commentary); another review by me not
necessary.
Rev D = 2b. Accept on condition that
relatively moderate changes in content and/or
form (specified in my commentary) are incorporated into a revised version;
and possibly send back to me for a brief review/verification.
Revs: A, C & D
We would like to, again, thank the
reviewers for their thorough reviews and, in particular, for their
constructive critique and suggestions for improvement. We have processed all
comments and worked on improving the paper accordingly.
Besides processing the reviewer
comments, the main changes to the third version are:
-
The title has been
shortened to Introducing the coaching cycle: A coaching by gaming
perspective of serious gaming.
-
More keywords have
been added.
-
Clearer focus
towards a general approach to serious gaming inspired by current training
practices by the military and, thus, softening the claim that this
framework is a novel approach.
Reviewer A
Rev A = 2a. Accept, but require
substantial revision in content and/or form (detailed
in my commentary), and send back revised version to me for re-evaluation.
222__ A. Importance of topic
11__ B. Aims clearly stated, with
logical structure
11__ C: Aims fully achieved
11__ D. Literature review (incl
jrnl S&G)
0__ E. Debriefing discussion*
11__ F. Quality of ideas, logic,
objectivity
222__ G. Quality of data
0__ H. Quality of method
11__ I. Technical aspects, esp
stats
11__ J. Quality of discussion
11__ K. Reliability of results;
validity of conclusions
222__ L. Organization
11__ M. Coherence / balance
11__ N. Clarity, concepts, logic
222__ P.
Quality of writing (Short, simple sentences, paras, topic sentence,
good grammar, etc)
222__ N. Quality & clarity of
visuals
11__ P. Significance of
contribution to profession/field
General
Comment
“I read the
paper when it originally came in and was disappointed with it. I spent an
hour and a half reviewing the paper again last evening. On the whole, I'm
still disappointed that the authors didn't substantively address some of my
comments and comments of Reviewer B.”
|
Reviewer’s comments
|
Authors’
response
|
|
Comment 1
“I have several concerns, but the overarching one is that
what is being proposed is not new. The paper presumes that coaching
while being an active participant in simulation based (or game based)
training is a brand new idea. I have personally witnessed this approach
dozens of times in military training. Just because it might be a less
common practice in civilian settings does not justify the claim of
discovery of a new approach for instructors. Instructors have changed
scenarios and role played in training (as participants), and even sat in
as worker-trainees, for a very long time.“
|
We agree that, within the context of
military training (and, to some extent, live role-playing exercises in
civilian contexts), the idea of an active instructor or coach is common
practice. Our view, however, is that this practice is an implicit
assumption that has not, by and large, been explicitly described and
grounded in current theories related to game-based training and
cognition/learning. In contrast, there are those (e.g. Klabbers) that
assert that the instructor should not disrupt training, but instead take
a step back and only observe how the game evolves.
We have softened our claim that this part of the
framework is new and further clarified the contribution of the coaching
cycle to the general serious game community (further described in our
response to comment 6).
|
Comment 2
“The argument that instructors should play with their
students so that they can relate to the trainee experience is also not
new, and I would argue that every instructor should go through the
training experiences they are going to provide to students prior to
instructing others with it (and I think that this is also common
practice).“
|
Even if this is common practice within the military
domain, it is largely overlooked in other (civilian) digital game-based
training contexts. At least, we have found very few accounts of
instructors as game players in the literature. Thus, we see that the
general GBL community could benefit from a theoretically grounded
description of this practice.
The clarifications made for some other comment also
relate to this comment.
|
Comment 3
“I also do not think that the presumption that playing
with students will necessarily increase deliberate practice. Deliberate
practice is something that students put into the experience. It is not
something that is put into the students by instructional approaches. It
is an age-old idea to employ instructional approaches that provide
information, practice, and motivate students to try hard. Everybody
knows that a knowledgeable, dedicated, exciting instructor can inspire
student motivation. The interest in using games, gaming approaches or
game characteristics is to attempt to crack the motivation nut without
requiring an exceptional instructor to do so.”
|
We do not claim that playing with students directly
increase deliberate practice. What we claim is that:
1. Deliberate
practice is essential for the development of expertise and coaching is
an important part of increasing deliberate practice
2. Games and
simulations are good for deliberate practice because they offer
continuous feedback and opportunities for repetition
3. Only relying
on in-game automatic feedback could, however, lead to a pattern of
incorrect behaviors that are hard to unlearn
4. Instructors
involved in gameplay can give continuous feedback in a flexible and
unobtrusive way (as a complement to automatic/game feedback)
5. Coaching does
not only occur during gameplay, but is also part of the debriefing (as
summative feedback and reflective exercises), so that students know what
to improve during the next gaming session.
We believe that coaching to increase deliberate practice
has more to do with giving feedback and spotting weak spots in trainees’
performance (both on an individual and a group level) than extrinsic
motivation. We don’t think that games can replace the instructors,
neither in terms of feedback or motivation – at least not with current
level of game AI. A game-based training approach still needs a highly
competent instructor to succeed. Otherwise, there is a risk that the
students learn how to win the game instead of picking up the intended
learning goals. As serious games researchers we must take into account
the situational aspects of serious gaming, i.e. studying not only the
game, but also the physical, social, and organizational contexts in
which it is used, including human actors.
We have clarified these points in the section labeled The power of game-based training.
|
Comment 4
“The goal is to inspire student motivation while reducing
the use of costly human instructors, hence ICW, web-based instruction,
and training games. The authors acknowledge that cost limits what is
possible in training situations (only one instructor and no trainer
operator for civilian fire fighting training) but they do not address
the need to improve and/or maintain training effectiveness while reining
in cost.”
|
As pointed out, we acknowledge the problem, but consider
it outside the scope of this paper to fully examine the
cost-effectiveness of the framework. Our focus is training quality.
We have added some text addressing the cost-effectiveness
problem in the discussion.
|
Comment 5
“A paper that recommends leveraging standard military
approaches to civilian training would be a more reality-grounded
approach, but would probably not be worth publishing in Simulation and
Gaming.”
|
We concur that the paper would be greatly improved if it
was pitched more clearly towards a more general game-based training
audience. We have revised the paper accordingly.
It is our hope that the editor of Simulation & Gaming
finds it appropriate for this journal.
|
Comment 6
“It is obvious that the authors have read a great deal of
literature related to the issues they discuss, but I don't feel that
they have clearly articulated what is new and, importantly, what is the
value (monetary and training effectiveness related) of their
recommendations. An article with increased clarity of these two issues
and data to back them up could be worth publishing.”
|
We have tried to improve the paper according to these
issues (see above comments). To sum up, what is “new” about the
framework is:
1. Current
training practices are explicitly articulated
2. Increased
focus on debriefing than current practices
3. Less focus on
initial lecture/briefing than current practices (thus giving more time
for gaming and debriefing)
4. Integrated
training cycle that combines current training practices with theories
from deliberate practice, experiential learning and game-based learning,
i.e. a theoretically grounded framework
The value of this framework is related to the development
of future serious games. Even though there are promising attempts to
create more advanced AI agents in games we are still far from handling
the complexity and flexibility that are prevalent in today’s training –
at least to the point where the AI system can replace the instructors’
role during game-based training. However, studying what the instructors
are actually doing, we can create requirements for such a system. In
fact, this is an important part of our future work.
|
Reviewer
B
Rev C = 3a. Accept with minor modifications to
substance and/or form (see my commentary); another review by me not
necessary.
222__ A. Importance of topic
11__ B. Aims clearly stated, with
logical structure
11__ C: Aims fully achieved
11__ D. Literature review (incl
jrnl S&G)
0__ E. Debriefing discussion*
11__ F. Quality of ideas, logic,
objectivity
222__ G. Quality of data
0__ H. Quality of method
11__ I. Technical aspects, esp
stats
11__ J. Quality of discussion
11__ K. Reliability of results;
validity of conclusions
222__ L. Organization
11__ M. Coherence / balance
11__ N. Clarity, concepts, logic
222__ P.
Quality of writing (Short, simple sentences, paras, topic sentence,
good grammar, etc)
222__ N. Quality & clarity of
visuals
11__ P. Significance of
contribution to profession/field
General
Comment
“I have
completed my review of your second draft and can see that it is much
improved from the first version. I have a few comments for you—I apologize
to not get these to you earlier. I hope these comments are still helpful.
The
manuscript is easy to read and interesting. I really appreciate the addition
of several sections. I believe you have addressed the reviewers comments. In
general, congratulations on improving your paper considerably.”
|
Reviewer’s comments
|
Authors’
response
|
Comment 1
“Reviewer A indicates that many of the recommendations
have been implemented by militaries (but not perhaps public education).
I have to agree with this statement. You paper is interesting, please
don’t get me wrong, but I believe your paper would have more impact if
you acknowledge prior work or even instances in military training in
which instructors are being/have been brought into the gaming
experience. You do this a bit in the section called “the many roles of
the instructor in game-based training” and I believe you could mention
other work here, primarily work I have done in creating in-game roles
for instructors, peers, and training cadre (this is a shameless plug for
my own work which I hoped you would have found in your literature
research so I hope you don’t mind if I have attached it directly for
your perusal). I have other papers on the topic of multiple roles
in-game for instructors and trainees to provide feedback in-game as well
as to an automated AAR of logged game events. This approach has been
instantiated in two U.S. military games. We have put instructors in
these roles and the 2009 paper discusses data collected with trainees in
the roles, not instructors, but you may be interested nonetheless as
this approach seems to codify your coaching cycle notion.”
|
We have changed the paper to be more overtly targeted
towards more general game-based training readers, for whom military
practices are not widely known.
Your papers are good contributions to our argumentation –
thank you!
|
Comment 2
“Given my point above, in the second paragraph, 1st page,
you mention that “they (instructors) are reduced to mere observers, or,
in the case of distance…” I would suggest that you soften this statement
as my own experience with the US military in designing systems as well
as observing training events would suggest that the instructor is always
an active part of the process, in one way or another.”
|
The sentence has been rephrased.
|
Comment 3
“Paragraph 4 of the first section describes coaching and
deliberate practice—when I read this, I asked myself, how is this
scaleable to groups in the methods you describe?”
|
Our work does not get a clear answer to this question,
but we agree that it is important. We do not wish to promote a framework
that requires one-on-one tutoring, but we also acknowledge that coaching
is difficult with too large groups. As we see it, larger groups have to
rely on (1) automatic feedback from the AI system and (2) peer-to-peer
feedback.
We have commented on this in the discussion.
|
Comment 4
“The third paragraph of your research approach suggests
that you have collected and analyzed empirical data to develop your
concepts. You really need to say more about this in detail (what did the
data tell you?) or delete this paragraph from the paper as the reader
expects to see a data analysis section in the paper that is never
presented.”
|
We have added a paragraph on data analysis in the
research approach section.
|
Reviewer
D
General
Comment
“I have
read your revision of the manuscript, Introducing the coaching cycle:
Changing the instructor role by facilitating coaching for deliberate
practice in game-based training systems, and find it to be an excellent
revision job that takes into account my comments and from my perspective the
comments of other reviewers (though they of course will speak for
themselves). The ms is much improved from version 1.”
|
Reviewer’s comments
|
Authors’
response
|
N/A
|
N/A
|
|
|
MsID =
Revs, A, C & E
Responses to reviewers
Reviewer A
|
Section of Paper
|
Comments
|
Response
|
|
Abstract
|
|
|
|
Introduction
|
|
|
|
Review of the literature
|
Although
games and simulations offer some affordances that other mediated
learning do not have, it is easy to get into the trap of viewing
“simulation” or “game” as some kind of intervention, much like
computer-assisted instruction was once conceived, resulting in media
comparison studies of CAI vs. “traditional” instruction. Richard
Clark’s argument is that the media don’t influence learning, rather it
is the effective use of instructional strategies and tactics. For
example, feedback to learners (correction of error) can be provided in a
variety of contexts for learning, including simulations and games, and
feedback is part of effective instruction (e.g., see Merrill’s First
Principles of Instruction—especially the ‘application’ principle). The
good news here is that the present study is not another “media
comparison” study. But the danger is nonetheless there to treat the
idea of simulation or game as if it is a variable itself (or theoretical
construct, which is not well-defined). There can be ineffective
simulations and games for a lot of reasons, because they do not employ
what is known from instructional and learning theories (again, see
Merrill, and also see Clark’s arguments).
|
The authors concur that this study
is not a media comparison study.
|
|
Research Questions
|
|
|
|
Methodology
|
Yes, but why
would they not also apply what they had learned in the course? And in
fact, you later discover that they did. So this basically renders the
first research question meaningless.
-----------------------------------------------------
With unequal
and relatively small n’s, any kind of statistical inference would be
difficult to justify—especially if the within-group variances are not
equal.
While Rogers’ theory was one of the
main ones, it was not the only theory that the original developers
applied in designing the DSG (personal communication from Dr. Molenda).
Note that this was developed to be
used in the context of a course in which change management was one of
the topics. Furthermore, in this course context, debriefing is
included as part of the use of the DSG. There is a debriefing guide
that comes with the licensed version and is recommended. There is no
debriefing guide in the free version.
So why was
the purple debriefing sheet NOT used that comes with the DSG? It has a
list of discussion items that reinforce many of the concepts and
relationships from Rogers’ theory.
|
Research questions were rewritten to
clarify the questions the researchers investigated.
In rewriting the paper, participants
in the study were viewed as one group.
Paper was modified to reflect that
additional change models were used in the development of the Diffusion
Simulation Game.
The authors were not aware of a
debriefing guide for the Diffusion Simulation Game until reviewers
mentioned it. The paper now includes this information.
|
|
Results
|
Means should
always be accompanied by the N and SD.
Ditto. Note that only 6 students
remained in Group A and only 3 in Group B. Missing data could be
biasing the resulting means. So the means are rather meaningless.
Showing the actual scores, as presented in Table 2 is probably the
best thing.
These are all TABLES (not figures,
at least in APA parlance).
More than
interesting, this makes the first research question meaningless. The
groups do not differ on the one dimension that presumably was
manipulated (i.e., whether on not a theory was applied in playing the
DSG). Given this methodological flaw from which no recovery is
possible, you might as well just talk about everyone as one big group.
The A vs. B distinction is irrelevant.
|
N and SD were added to the results
table.
Corrected labeling to Tables (not
figures).
Revisions to the paper treat all
participants as one group.
|
|
Discussion
|
It makes no
sense to talk about A vs. B comparisons, since the groups are more alike
than different. The theory mismatch argument may be compelling
initially, but if students learn from their experience, that should
override whatever theory they might have in mind. What a different
theory could provide is dissonance that might interfere initially with
learning from the experience.
This is by design in the DSG.
Outcomes are stochastic, not deterministic. Sometimes the “correct”
strategy just does not work. According to Molenda, the probabilities
of success under varying conditions were intended to mimic to some
extent what was likely to work in the real world. E.g. if a strategy
under a given condition works 60 percent of the time, this is modeled
in the game by 5 outcomes of a move, 3 of which are successful and 2
of which are not. So if a player does not play the DSG enough times,
she or he won’t understand the stochastic nature (it worked before but
now it does not, and then it works again the next time???). Repeated
play is necessary to get past the problem of jumping to conclusions
too quickly based on just one or two game plays.
Again, I
have to ask, why were the debriefing materials provided with the DSG NOT
utilized. This debriefing was crucial for the board version of the DSG
and is just as important for the online version. Unfortunately, if the
DSG is played outside a course context, there is no obvious way to
control the debriefing. People just play the game and draw whatever
conclusions they may. If they play the DSG enough, they may learn
something that is consistent with theory about diffusion of
innovations.
|
Thank you for the comment about the
nature of the Diffusion Simulation Game being stochastic. The comment
helped the researchers better understand and interpret survey results.
|
|
Limitations of the study
|
|
|
|
Conclusions
|
|
|
Additional comment on the paper from
Reviewer A:
In general this is an interesting
study and potentially worthy of publication in the Journal of Simulation and
Gaming, with revisions indicated and attention to issues raised in below
comments.
Response: The paper has been
extensively rewritten with attention to issues raised by all reviewers.
Reviewer comments helped the authors clarify the research project and better
articulate results and interpretation of findings.
Responses to Reviewer comments and suggestions
Reviewer C
|
Section of Paper
|
Comments
|
Response
|
|
Abstract
|
None
|
|
|
Keywords
|
“I
don’t think all of these words work as efficient keywords. Try narrowing
down.”
|
Narrowed down to effective keywords
|
|
Introduction
|
1. Suggested clarifying what students wrote. …“reflection
what? Paper Journal?”
2. Used a instead of one
…“was there more than one weakness? Maybe change to a”
|
1. Changed to “wrote a refection paper about the experience”
2. Changed to “a weakness of the study was in the area of
debriefing”
|
|
Review of the literature
|
End of 2nd paragraph. …Thus, as players learn
and experience the rules of a game by assuming a specific role, … “this
reads like a run-on sentence. Try breaking up”
|
Broke up to two sentences
|
|
Research Questions
|
None
|
|
|
Methodology
|
1. Tense, 3rd , 4th, and 7th paragraphs. “since this is your study and this is what you did,
shouldn’t this be in past tense?”
2. Name: The survey consisted of sixteen Likert scale.
“technically it should be referred to as a Likert response format”
|
1. Changed to past tense.
2. Changed to “The survey consisted of sixteen questions”
|
|
Results
|
None
|
|
|
Discussion
|
“In this section it is not clear where the answers to the
research questions are. While I think the author has answered the
questions, the research questions should be specifically addressed here
followed by the evidenced based answer (s)”
|
Research question one was addressed in paragraph 1-4 with
discussions on the issue of game logs, the use of moves, and adopters
gained in both groups. Research question two was addressed from
paragraph 5- 10.
|
|
Limitations of the study
|
None
|
|
|
Conclusions
|
Although some negative comments about the game were
indicated … “make this active voice”
|
Changed to “While there were some negative comments about
the game…”
|
|
Appendix I
|
Likert scale
|
Changed to Likert response format
|
|
Figures
|
Figure 2 Survey Results “although the distribution of
scores graphic is a terrific idea, technically for descriptive
statistics the standard deviation should be presented along with the
mean, even with a small N.”
|
Standard deviation was included in the table. (all
“Figures” were changed to be “Tables” per APA guidelines)
|
Responses to Reviewer comments
and suggestions
Reviewer E
|
Section of Paper
|
Comments
|
Response
|
|
Abstract
|
You are talking about a particular
simulation, not any simulation, correct?
You referred
to it previously as a simulation. The reader may not realize that the
“game” you are referring to hear is the same as the “simulation” you
mentioned previously. It would be helpful to the reader to make this
connection explicit before using the terms interchangeably.
|
A statement was added to indicate
the terms “simulation” and “game” are used interchangeably when used in
reference to the Diffusion Simulation Game
|
|
Introduction
|
Is this
speculation? Do you have evidence to cite?
This sentence seems to be
contradicting itself. How could the majority of business faculty not
use simulations if they are used “extensively” in business-related
courses?
Though I realize this is true, if
you do not provide evidence, then readers must assume you are
speculating. Provide evidence for claims.
Again, be
specific. It was not any simulation they viewed positively, it was the
Diffusion Simulation Game.
|
Literature review has been
extensively rewritten to provide citations for statements.
Changes were made throughout the
article to refer specifically to the Diffusion Simulation Game
|
|
Review of the literature
|
This is
strong language for a generalization about all simulation games.
Learning is always probabilistic due to the stochastic nature of
humans. I have never seen any simulation, or any other instructional
method, that is always effective in meeting its learning objectives with
all learners.
“promoted” may be a better choice of
words than “achieved”, or qualifying it with something like “can be”
instead of “are”.
Though I
have not read Peterson (2010) yet, I wonder if this single piece of
“strong evidence” is based on a single game. This may be something you
want to expand on.
Without more
detail, I would be inclined to discredit a statement like this in the
same way that I would discredit a statement that states that “I-Pads are
good for learning” while disregarding the instructional method being
used with the device. The effectiveness, or lack thereof, of games and
simulations result from the effective (or ineffective) design of the
game. For example, I would not expect that a poorly designed game would
be effective for learning just because it is a game.
What criteria are you using for
this? I would delete “convincingly” and leave it to the reader to
make this judgment. The paper should be as unbiased as possible from
your personal opinion/beliefs.
Being unspecific makes it feel as
though you are generalizing to all simulations, or talking about a
different simulation. I would state “DSG” when talking about that
particular simulation.
Your lit
review, though fairly logical and clear, does not lead to a particular
gap in knowledge as far as I can tell. I would like to see the review
concluded with a more clear gap in knowledge. Why is there a need for
this study?
|
Suggestions for word changes were
adopted in the revisions of the literature review.
For example, the word “convincingly”
was deleted.
The gap in the literature was
defined as the relative low use of simulations in higher education as
well as limited information for educators on how to implement a
simulation/game
|
|
Research Questions
|
If this is what you are attempting
to do, I think your methodology is inappropriate. However, if you are
trying to determine if “the DSG” is a valuable addition, then your
methodology makes more sense.
The gap in
literature (not sure what that is), the goal/purpose (does DSG help
students learn content), and the research questions (will students
perform better when using a change model to guide their gameplay than
students who play the game with no guidance) are not aligned.
These are
significant issues that should be addressed.
Also, if your goal is to assess
learning, game performance is not always the best indicator,
particularly if the fidelity of the game rules to the theory is low.
It is entirely possible that the game was poorly designed and
following the change model would result in a poor game performance.
|
Research questions were rewritten to
accurately reflect what the researchers wanted to investigate and learn
about.
Fidelity of the Diffusion Simulation
Game and the real life change process is addressed in the results and
discussion areas of the paper
|
|
Methodology
|
Before or
after participation in the study?
Is this the same change model which
the game was designed to teach?
Though they were just asked to play
the game, did they have knowledge of the change model? Had they
learned the content in class prior to participating in the study?
You mentioned the importance of
debriefing in your literature review, which is not the same as
reflection. If evidence suggests that it is important in learning via
games, why was this not part of the intervention? Is debriefing not
feasible in an online course?
Really? Did players even know the
gamelog was being saved? Why would they want to delete their gamelog?
I wonder if not making an assumption would be better here, and instead
just state that 3 of the gamelogs could not found.
Though this is true, the DSG was not
solely based on Roger’s theory. It also included other ideas that
were known at the time
They are not implementing the
strategy… they are convincing the staff to implement it.
This may have been why 3 of the
participants logs could not be found… if they started a new game but
did not play it.
This answers my previous question.
This
addresses a topic I mentioned earlier as well. However, debriefing is
not the same as reflection. Whereas reflection allows a player to
consider their own gameplay experience, my understanding of debriefing
is that others gameplay experiences are shared as well. In any case, a
brief definition of debriefing from the literature will clear up the
misunderstanding. You may find that you need to state that you used the
reflective journals “in place of” debriefing for whatever reason.
|
The paper describes that
participants played the game after nine change models were studied and
that the Diffusion of Innovation theory was one of the models studied in
the course.
Information and discussion related
to debriefing was increased in this second version of the paper. The
paper now clearly states that the reflective journal was used in place
of a formal debriefing protocol.
The researchers were unaware that a
debriefing guide for the Diffusion Simulation Game was/is available.
|
|
Results
|
The average
you provide for your participants is based on their “last” gameplay
session. I am guessing that the average you are reporting for past DSG
players includes all gameplays. If so, this comparison is not valid.
At the very least, this fact should be made known. If the average
number of adoptees of past DSG games is based on the “last” gameplay,
then state so. If this is so, is the number of gameplays the same? I
would expect the more you play the game, the better you get.
Also, I
wonder if you are able to get another (better) measure of game
performance. Imagine a situation in which one player has 15 people in
the trial phase and only 5 adopters. Another player has 3 people in the
trial phase and has 8 adopters. Based on your indicator, the second
person performed better. However, if you consider “closeness to
adoption”, the former outperformed the latter.
If you have
the data available, I would be interested in knowing the game
performance based on a closeness to adoption in addition to the total
number of adopters.
So they were not asked to use one
particular change theory? This is what was stated earlier.
So the treatment, or at least the
activity, was not necessarily different from those in group A and
those in group B. I wonder if it is important to report that there
were two groups then. It may be more beneficial to compare those who
actually used a change theory with those who did not; instead of
comparing those who were asked to vs those who were not.
This would
have been good to mention earlier so it is made clear to the reader that
the treatment varied between participants and that the game performance
data being analyzed was of their last gameplay, and I assume you have no
way of knowing how many gameplays led up to that. This makes comparing
performance difficult as those who played more games would likely
perform better.
|
The results section was rewritten to
align results with the research questions.
Closeness to adoption was not
investigated in this study. I agree that this is an interesting aspect
to investigate and is a good area for future research.
Reviewers were absolutely correct
that this research project was an exploratory study.
In the revised paper, participants
were treated as one group.
|
|
Discussion
|
. This makes
sense because they did not have to play for the whole 2 year calendar
(because they won). Is there something else that is being implied here?
In these first two paragraphs you
are giving details about what happened during gameplay (findings) but
not really discussing the meaning of these findings, which is what I
would expect in the discussion. Is there any meaning to these
findings?
The small number is a valid reason
to not do a statistical analysis (and so is the fact that the
treatment was the same for many in both groups due to their prior
learning and use of change models). However, groups not having an
equal number is not a great argument for not using a statistical
analysis because there are methods for correcting for this.
Particularly since participants from
both groups did this.
I wonder if you could provide an
example in which following the prescription of one change model does
not align with the game. Do these change models differ that
dramatically? I would expect that they are all intended to work in
the same real-life situations they are designed for.
This is particularly interesting,
given the amount of time it takes to play the game. Do you have any
data on the average amount of time the game took to play?
Substitute?
Again, reflection does not allow
correction of misunderstandings because there is nobody leading the
debriefing and ensuring that misconceptions are addressed.
An online class, correct?
This was an
attempt at debriefing, though I still think an expert would need to be
involved in the discussion to ensure misconceptions are being addressed
(blind not leading the blind).
|
The discussion
section was rewritten to align with the research questions and to better
clarify the meaning of the findings.
No data is
available on the time it took to play the Diffusion Simulation Game.
Weakness in the
area of debriefing is discussed in the Discussion section.
|
|
Limitations of the study
|
As far as I
can tell, no comparisons were made between group A and group B, and
definitely no statistical comparisons. The treatments were not
necessarily different as well. I am not sure why you need to mention
group A and group B at all. It may make more sense to note that X
participants were explicitly asked to use a change model and the others
were not.
|
Limitations section was rewritten.
Study participants comprised one group.
|
|
Conclusions
|
You have no
evidence that your results are generalizable. This statement would be
more appropriate if it were to state “in this particular graduate-level
online course”
Not all students… be careful not to
imply a generalization. “students in this study”
This is a
generalization that cannot be made based on your study. Not only do you
not have sufficient evidence to say this about simulations in general
(as you were only looking at one particular simulation), you do not have
evidence to say this about online courses (as you were looking at only
one online course).
|
Conclusions were rewritten to limit
generalizability.
|
n
|
