Choosing Qualitative Research: A Primer for Technology Education
Researchers
(DR. H. Edy
Riyadi, M.Pd)
A number of writers have commented on the dearth of
substantive research within the field of technology education, and point to the
expansion of its research agenda as a means of strengthening the discipline.
Waetjen, in his call for good research in technology education, states that
"the plea is to use experimental type research as much as possible" (1992, p. 30).
Interestingly, the three areas of research need outlined in his essay would all
lend themselves to alternative methodologies, including qualitative
methodologies.
More recently, others have called for an expansion in
the types of research methods used. Of the 220 reports included in Zuga's
review of technology education-related research (1994), only 16 are identified as having
used qualitative methods, and Zuga notes that many of those studies were
conducted outside the United States. Johnson (1995) suggests that technology educators
"engage in research that probes for deeper understanding rather than
examining surface features." He notes that qualitative methodologies are
powerful tools for enhancing our understanding of teaching and learning, and
that they have "gained increasing acceptance in recent years" (p. 4).
There are compelling reasons for the selection of
qualitative methodologies within the educational research arena, yet many
people remain unfamiliar with these methods. Researchers trained in the use of
quantitative designs face real challenges when called upon to use or teach
qualitative research (Stallings, 1995). There is,
however, a growing body of literature devoted to qualitative research in
education, some of which is synthesized here. The goals of this article are to
elaborate on the reasons for choosing qualitative methodologies, and to provide
a basic introduction to the features of this type of research.
Qualitative Versus Quantitative
Research Paradigms
Researchers have long debated the relative value of
qualitative and quantitative inquiry (Patton, 1990).
Phenomenological inquiry, or qualitative research, uses a naturalistic approach
that seeks to understand phenomena in context-specific settings. Logical
positivism, or quantitative research, uses experimental methods and quantitative
measures to test hypothetical generalizations. Each represents a fundamentally
different inquiry paradigm, and researcher actions are based on the underlying
assumptions of each paradigm.
Qualitative research, broadly defined, means "any
kind of research that produces findings not arrived at by means of statistical
procedures or other means of quantification" (Strauss and Corbin, 1990, p. 17). Where quantitative researchers seek causal determination, prediction,
and generalization of findings, qualitative researchers seek instead
illumination, understanding, and extrapolation to similar situations.
Qualitative analysis results in a different type of knowledge than does
quantitative inquiry.
Eisner points out that all knowledge, including that
gained through quantitative research, is referenced in qualities, and that
there are many ways to represent our understanding of the world:
There is a kind of continuum that moves from the fictional that is
"true"-the novel for example-to th e highly controlled and
quantitatively described scientific experiment. Work at either end of this
continuum has the capacity to inform significantly. Qualitative research and evaluation
are located toward the fictive end of the continuum without being fictional in
the narrow sense of the term Eisner, 1991, pp. 30-31).
This sentiment echoes that of an earlier writer.
Cronbach (1975) states that "the special task
of the social scientist in each generation is to pin down the contemporary
facts. Beyond that, he shares with the humanistic scholar and the artist in the
effort to gain insight into contemporary relationships" (p. 126).
Cronbach claims that statistical research is not able
to take full account of the many interaction effects that take place in social
settings. He gives examples of several empirical "laws" that do not
hold true in actual settings to illustrate this point. Cronbach states that
"the time has come to exorcise the null hypothesis," because it ignores
effects that may be important, but that are not statistically significant
(1975, p. 124). Qualitative inquiry accepts the complex and dynamic quality of
the social world.
However, it is not necessary to pit these two
paradigms against one another in a competing stance. Patton (1990) advocates a "paradigm of
choices" that seeks "methodological appropriateness as
the primary criterion for judging methodological quality." This will allow
for a "situational responsiveness" that strict adherence to one
paradigm or another will not (p. 39). Furthermore, some researchers believe
that qualitative and quantitative research can be effectively combined in the
same research project (Strauss and Corbin, 1990; Patton, 1990). For example,
Russek and Weinberg (1993) claim that by using both
quantitative and qualitative data, their study of technology-based materials
for the elementary classroom gave insights that neither type of analysis could
provide alone.
Basis for the
Use of a Qualitative Methodology
There are several considerations when deciding to
adopt a qualitative research methodology. Strauss and Corbin (1990) claim that qualitative methods can
be used to better understand any phenomenon about which little is yet known.
They can also be used to gain new perspectives on things about which much is
already known, or to gain more in-depth information that may be difficult to
convey quantitatively. Thus, qualitative methods are appropriate in situations
where one needs to first identify the variables that might later be tested
quantitatively, or where the researcher has determined that quantitative
measures cannot adequately describe or interpret a situation. Research problems
tend to be framed as open-ended questions that will support discovery of new
information. Greene's 1994 study of women in the trades, for example, asked
" What personal characteristics do tradeswomen have in common? In what
way, if any, did role models contribute to women's choices to work in the
trades?"
(p. 524a).
The ability of qualitative data to more fully describe
a phenomenon is an important consideration not only from the researcher's
perspective, but from the reader's perspective as well. "If you want
people to understand better than they otherwise might, provide them information
in the form in which they usually experience it" (Lincoln and Guba, 1985, p. 120). Qualitative research reports, typically rich with detail and insights
into participants' experiences of the world, "may be epistemologically in
harmony with the reader's experience" (Stake, 1978, p. 5) and thus more
meaningful.
Features of
Qualitative Research
Several writers have identified what they consider to
be the prominent characteristics of qualitative, or naturalistic, research
(see, for example: Bogdan and Biklen, 1982; Lincoln and Guba, 1985; Patton, 1990; Eisner, 1991). The list
that follows represents a synthesis of these authors' descriptions of
qualitative research:
1. Qualitative
research uses the natural setting as the source of data. The researcher
attempts to observe, describe and interpret settings as they are, maintaining
what Patton calls an "empathic neutrality" (1990, p. 55).
2. The
researcher acts as the "human instrument" of data collection.
3. Qualitative
researchers predominantly use inductive data analysis.
4. Qualitative
research reports are descriptive, incorporating expressive language and the
"presence of voice in the text" (Eisner, 1991, p. 36).
5. Qualitative
research has an interpretive character, aimed at discovering the meaning events
have for the individuals who experience them, and the interpretations of those
meanings by the researcher.
6. Qualitative
researchers pay attention to the idiosyncratic as well as the pervasive,
seeking the uniqueness of each case.
7. Qualitative
research has an emergent (as opposed to predetermined) design, and researchers
focus on this emerging process as well as the outcomes or product of the
research.
8. Qualitative
research is judged using special criteria for trustworthiness (these will be
discussed in some detail in a later section).
Patton (1990) points out that these are not
"absolute characteristics of qualitative inquiry, but rather strategic
ideals that provide a direction and a framework for developing specific designs
and concrete data collection tactics" (p. 59). These characteristics are
considered to be "interconnected" (Patton, 1990, p. 40) and
"mutually reinforcing" (Lincoln and Guba, 1985, p. 39).
It is important to emphasize the emergent nature of qualitative research
design. Because the researcher seeks to observe and interpret meanings in
context, it is neither possible nor appropriate to finalize research strategies
before data collection has begun (Patton, 1990). Qualitative research proposals
should, however, specify primary questions to be explored and plans for data
collection strategies.
The particular design of a qualitative study depends
on the purpose of the inquiry, what information will be most useful, and what
information will have the most credibility. There are no strict criteria for
sample size (Patton, 1990). "Qualitative studies typically employ multiple
forms of evidence....[and] there is no statistical test of significance to
determine if results 'count'" (Eisner, 1991, p. 39). Judgments about
usefulness and credibility are left to the researcher and the reader.
The Role of the Researcher in
Qualitative Inquiry
Before conducting a qualtitative study, a researcher
must do three things. First, (s)he must adopt the stance suggested by the
characteristics of the naturalist paradigm. Second, the researcher must develop
the level of skill appropriate for a human instrument, or the vehicle through
which data will be collected and interpreted. Finally, the researcher must prepare
a research design that utilizes accepted strategies for naturalistic inquiry
(Lincoln and Guba, 1985).
Glaser and Strauss (1967) and Strauss and Corbin (1990) refer to what they call the
"theoretical sensitivity" of the researcher. This is a useful concept
with which to evaluate a researcher's skill and readiness to attempt a
qualitative inquiry.
Theoretical sensitivity refers to a personal quality of the researcher.
It indicates an awareness of the subtleties of meaning of data. ...[It] refers
to the attribute of having insight, the ability to give meaning to data, the
capacity to understand, and capability to separate the pertinent from that
which isn't (Strauss and Corbin, 1990, p. 42).
Strauss and Corbin believe that theoretical
sensitivity comes from a number of sources, including professional literature,
professional experiences, and personal experiences. The credibility of a
qualitative research report relies heavily on the confidence readers have in
the researcher's ability to be sensitive to the data and to make appropriate
decisions in the field (Eisner, 1991; Patton, 1990).
Lincoln and Guba (1985) identify the characteristics that
make humans the "instrument of choice" for naturalistic inquiry.
Humans are responsive to environmental cues, and able to interact with the
situation; they have the ability to collect information at multiple levels
simultaneously; they are able to perceive situations holistically; they are
able to process data as soon as they become available; they can provide
immediate feedback and request verification of data; and they can explore
atypical or unexpected responses.
Research Design and Data Collection
Strategies
Eisner (1991) claims there is a "paucity of
methodological prescriptions" for qualitative research, because such
inquiry places a premium on the strengths of the researcher rather than on
standardization (p. 169). Lincoln and Guba (1985) provide a fairly detailed outline
for the design of naturalistic inquiry, which includes these general steps:
1. Determine a
focus for the inquiry. This should establish a boundary for the study, and
provide inclusion/exclusion criteria for new information. Boundaries, however,
can be altered, and typically are.
2. Determine
the fit of the research paradigm to the research focus. The researcher must
compare the characteristics of the qualitative paradigm with the goals of the
research.
3. Determine
where and from whom data will be collected.
4. Determine
what the successive phases of the inquiry will be. Phase one, for example,
might feature open-ended data collection, while successive phases will be more
focused.
5. Determine what additional instrumentation may be
used, beyond the researcher as the human instrument.
6. Plan data
collection and recording modes. This must include how detailed and specific
research questions will be, and how faithfully data will be reproduced.
7. Plan which
data analysis procedures will be used.
8. Plan the
logistics of data collection, including scheduling and budgeting.
9. Plan the
techniques that will be used to determine trustworthiness.
Steps one and two have been addressed in previous
sections; the remaining steps will be addressed below.
Sampling Strategies for Qualitative
Researchers
In quantitative inquiry, the dominant sampling
strategy is probability sampling, which depends on the selection of a random
and representative sample from the larger population. The purpose of
probability sampling is subsequent generalization of the research findings to
the population. By contrast, purposeful sampling is the
dominant strategy in qualitative research. Purposeful sampling seeks
information-rich cases which can be studied in depth (Patton, 1990).
Patton identifies and describes 16 types of purposeful
sampling. These include: extreme or deviant case sampling; typical case
sampling; maximum variation sampling; snowball or chain sampling; confirming or
disconfirming case sampling; politically important case sampling; convenience
sampling; and others (1990, pp. 169-183). According to
Lincoln and Guba (1985), the most useful strategy for the
naturalistic approach is maximum variation sampling. This strategy aims at
capturing and describing the central themes or principal outcomes that cut
across a great deal of participant or program variation. For small samples a
great deal of heterogeneity can be a problem because individual cases are so
different from each other. The maximum variation sampling strategy turns that
apparent weakness into a strength by applying the following logic: Any common
patterns that emerge from great variation are of particular interest and value
in capturing the core experiences and central, shared aspects or impacts of a
program (Patton, 1990, p. 172).
Maximum variation sampling can yield detailed
descriptions of each case, in addition to identifying shared patterns that cut
across cases. See Hoepfl (1994) for an illustration of this
strategy applied to technology education research. Several examples of studies
employing case sampling can also be found in the technology education
literature (see Brown, 1995; Hansen, 1995; and Lewis, 1995 and 1997) In spite of
the apparent flexibility in purposeful sampling, researchers must be aware of
three types of sampling error that can arise in qualitative research. The first
relates to distortions caused by insufficient breadth in sampling; the second
from distortions introduced by changes over time; and the third from
distortions caused by lack of depth in data collection at each site (Patton, 1990).
Data Collection Techniques
The two prevailing forms of data collection associated
with qualitative inquiry are interviews and observation.
Interviews
Qualitative interviews may be used either as the
primary strategy for data collection, or in conjunction with observation,
document analysis, or other techniques (Bogdan and Biklen, 1982). Qualitative
interviewing utilizes open-ended questions that allow for individual
variations. Patton (1990) writes about three types of
qualitative interviewing: 1) informal, conversational interviews; 2)
semi-structured interviews; and 3) standardized, open-ended interviews.
An interview guide or "schedule" is a list of questions or
general topics that the interviewer wants to explore during each interview.
Although it is prepared to insure that basically the same information is
obtained from each person, there are no predetermined responses, and in
semi-structured interviews the interviewer is free to probe and explore within
these predetermined inquiry areas. Interview guides ensure good use of limited
interview time; they make interviewing multiple subjects more systematic and
comprehensive; and they help to keep interactions focused. In keeping with the
flexible nature of qualitative research designs, interview guides can be
modified over time to focus attention on areas of particular importance, or to
exclude questions the researcher has found to be unproductive for the goals of
the research (Lofland and Lofland, 1984).
Recording Data. A basic decision going into
the interview process is how to record interview data. Whether one relies on
written notes or a tape recorder appears to be largely a matter of personal
preference. For instance, Patton says that a tape recorder is
"indispensable" (1990, p. 348), while Lincoln and Guba "do not
recommend recording except for unusual reasons" (1985, p. 241). Lincoln
and Guba base their recommendation on the intrusiveness of recording devices
and the possibility of technical failure. Recordings have the advantage of
capturing data more faithfully than hurriedly written notes might, and can make
it easier for the researcher to focus on the interview.
Observations
The classic form of data collection in naturalistic or
field research is observation of participants in the context of a natural
scene. Observational data are used for the purpose of description-of settings,
activities, people, and the meanings of what is observed from the perspective
of the participants. Observation can lead to deeper understandings than
interviews alone, because it provides a knowledge of the context in which
events occur, and may enable the researcher to see things that participants
themselves are not aware of, or that they are unwilling to discuss (Patton, 1990). A skilled
observer is one who is trained in the process of monitoring both verbal and
nonverbal cues, and in the use of concrete, unambiguous, descriptive language.
Sours' (1997) study of teaching and learning
styles provides a good example of descriptive language applied to the
technology classroom.
There are several observation strategies available. In
some cases it may be possible and desirable for the researcher to watch from
outside, without being observed. Another option is to maintain a passive
presence, being as unobtrusive as possible and not interacting with
participants. A third strategy is to engage in limited interaction, intervening
only when further clarification of actions is needed. Or the researcher may
exercise more active control over the observation, as in the case of a formal
interview, to elicit specific types of information. Finally, the researcher may
act as a full participant in the situation, with either a hidden or known
identity. Each of these strategies has specific advantages, disadvantages and
concerns which must be carefully examined by the researcher (Schatzman and Strauss, 1973).
The presence of an observer is likely to introduce a
distortion of the natural scene which the researcher must be aware of, and work
to minimize. Critical decisions, including the degree to which researcher
identity and purposes will be revealed to participants, the length of time
spent in the field, and specific observation techniques used, are wholly
dependent on the unique set of questions and resources brought to each study.
In any case, the researcher must consider the legal and ethical
responsibilities associated with naturalistic observation.
Recording Data. Field researchers rely most heavily on the use of field notes, which
are running descriptions of settings, people, activities, and sounds. Field
notes may include drawings or maps. Acknowledging the difficulty of writing
extensive field notes during an observation, Lofland and Lofland (1984) recommend jotting down notes that
will serve as a memory aid when full field notes are constructed. This should
happen as soon after observation as possible, preferably the same day. In
addition to field notes, researchers may use photographs, videotapes, and audio
tapes as means of accurately capturing a setting.
Gaining Access and Researcher
Obligations
Based on their experience with naturalistic research,
Lofland and Lofland (1984) believe that researchers are more
likely to gain successful access to situations if they make use of contacts
that can help remove barriers to entrance; if they avoid wasting respondents'
time by doing advance research for information that is already part of the
public record; and if they treat respondents with courtesy. Because
naturalistic researchers are asking participants to "grant access to their
lives, their minds, [and] their emotions," it is also important to provide
respondents with a straightforward description of the goals of the research (p.
25).
Other Sources of Data
Another source of information that can be invaluable
to qualitative researchers is analysis of documents. Such documents might
include official records, letters, newspaper accounts, diaries, and reports, as
well as the published data used in a review of literature. In his study of
technology teachers in training, Hansen (1995) analyzed journal entries and memos
written by participants, in addition to interviews. Hoepfl (1994), in her study of closure of
technology teacher education programs, used newspaper reports, university
policy documents, and department self-evaluation data, where available, to
supplement data gained through interviews.
There are some specialized forms of qualitative
research which rely solely on analysis of documents. For example, Gagel (1997) used a process known ashermeneutic
inquiry to investigate the literature on both literacy and technology.
Patton (1990) provides a good overview of the
various theoretical orientations that inform the "rich menu of alternative
possibilities within qualitative research" (p. 65).
Deciding When to Stop Sampling
Qualitative researchers have few strict guidelines for
when to stop the data collection process. Criteria include: 1) exhaustion of
resources; 2) emergence of regularities; and 3) overextension, or going too far
beyond the boundaries of the research (Guba, 1978). The decision
to stop sampling must take into account the research goals, the need to achieve
depth through triangulation of data sources, and the possibility of greater
breadth through examination of a variety of sampling sites.
Analysis of Data
Bogdan and Biklen define qualitative data analysis as
"working with data, organizing it, breaking it into manageable units, synthesizing
it, searching for patterns, discovering what is important and what is to be
learned, and deciding what you will tell others" (1982, p. 145). Qualitative
researchers tend to use inductive analysis of data, meaning that the critical
themes emerge out of the data (Patton, 1990). Qualitative
analysis requires some creativity, for the challenge is to place the raw data
into logical, meaningful categories; to examine them in a holistic fashion; and
to find a way to communicate this interpretation to others.
Sitting down to organize a pile of raw data can be a
daunting task. It can involve literally hundreds of pages of interview
transcripts, field notes and documents. The mechanics of handling large
quantities of qualitative data can range from physically sorting and storing
slips of paper to using one of the several computer software programs that have
been designed to aid in this task (see Brown, 1996, for a description of one of these
programs).
Analysis begins with identification of the themes
emerging from the raw data, a process sometimes referred to as "open
coding" (Strauss and Corbin, 1990). During open
coding, the researcher must identify and tentatively name the conceptual
categories into which the phenomena observed will be grouped. The goal is to
create descriptive, multi-dimensional categories which form a preliminary
framework for analysis. Words, phrases or events that appear to be similar can
be grouped into the same category. These categories may be gradually modified
or replaced during the subsequent stages of analysis that follow.
As the raw data are broken down into manageable
chunks, the researcher must also devise an "audit trail"-that is, a
scheme for identifying these data chunks according to their speaker and the
context. The particular identifiers developed may or may not be used in the
research report, but speakers are typically referred to in a manner that
provides a sense of context (see, for example, Brown, 1996; Duffee and Aikenhead, 1992; and Sours, 1997).
Qualititative research reports are characterized by the use of
"voice" in the text; that is, participant quotes that illustrate the
themes being described.
The next stage of analysis involves re-examination of
the categories identified to determine how they are linked, a complex process
sometimes called "axial coding" (Strauss and Corbin, 1990). The discrete
categories identified in open coding are compared and combined in new ways as
the researcher begins to assemble the "big picture." The purpose of
coding is to not only describe but, more importantly, to acquire new
understanding of a phenomenon of interest. Therefore, causal events
contributing to the phenomenon; descriptive details of the phenomenon itself;
and the ramifications of the phenomenon under study must all be identified and
explored. During axial coding the researcher is responsible for building a
conceptual model and for determining whether sufficient data exists to support
that interpretation.
Finally, the researcher must translate the conceptual
model into the story line that will be read by others. Ideally, the research
report will be a rich, tightly woven account that "closely approximates
the reality it represents" (Strauss and Corbin, 1990, p. 57). Many of the concerns surrounding the presentation of qualitative
research reports are discussed in the section "Judging Qualitative
Research" which follows.
Although the stages of analysis are described here in
a linear fashion, in practice they may occur simultaneously and repeatedly.
During axial coding the researcher may determine that the initial categories
identified must be revised, leading to re-examination of the raw data.
Additional data collection may occur at any point if the researcher uncovers
gaps in the data. In fact, informal analysis begins with data collection, and
can and should guide subsequent data collection. For a more detailed yet very
understandable description of the analysis process, see Simpson and Tuson (1995).
The Product of Qualitative Data
Analysis
In their classic text Discovery of Grounded
Theory, Glaser and Strauss (1967,/a>)
describe what they believe to be the primary goal of qualitative research: the
generation of theory, rather than theory testing or mere description. According
to this view, theory is not a "perfected product" but an
"ever-developing entity" or process (p. 32). Glaser and Strauss claim
that one of the requisite properties of grounded theory is that it be
"sufficiently general to be applicable to a multitude of diverse
situations within the substantive area" (p. 237).
The grounded
theory approach described by Glaser and Strauss represents a somewhat extreme
form of naturalistic inquiry. It is not necessary to insist that the product of
qualitative inquiry be a theory that will apply to a "multitude of diverse
situations." Examples of a more flexible approach to qualitative inquiry
can be gained from a number of sources. For example, both Patton (1990) and Guba (1978) state, in the same words, that
"naturalistic inquiry is always a matter of degree" of the extent to
which the researcher influences responses and imposes categories on the data.
The more "pure" the naturalistic inquiry, the less reduction of data
into categories.
Figure 1 illustrates one interpretation of the
relationship between description, verification, and generation of theory-or, in
this case, the development of what Cronbach (1975) calls "working hypotheses,
" which suggests a more tractable form of analysis than the word
"theory." According to this interpretation, a researcher may move
between points on the description/ verification continuum during analysis, but
the final product will fall on one particular point, depending on the degree to
which it is naturalistic.
Figure 1. Description,
verification and generation of working hypotheses in qualitative research.
In keeping with a naturalistic stance, the researcher
might conclude that, to the extent that findings are based on information from
a variety of diverse situations, they may be applicable to a
larger substantive area. However, their applicability to a particular situation
is wholly dependent upon the conditions of the situation and the usefulness of
the research findings to individual readers.
Judging Qualitative Research
The Role of the Reader
Those who are in a position to judge or use the
findings of a qualitative inquiry must play a different type of role than
people who review quantitative research. This is because "there are no
operationally defined truth tests to apply to qualitative research" (Eisner, 1991, p. 53). Instead,
researcher and readers "share a joint responsibility" for
establishing the value of the qualitative research product (Glaser and Strauss, 1967, p. 232). "Pragmatic validation [of qualitative research] means that the
perspective presented is judged by its relevance to and use by those to whom it
is presented : their perspective and actions joined to the
[researcher's] perspective and actions" (Patton, 1990, p. 485).
Eisner (1991) believes that the following three
features of qualitative research should be considered by reviewers:
Coherence: Does the story make sense? How have the conclusions
been supported? To what extent have multiple data sources been used to give
credence to the interpretation that has been made? (p. 53).
Related to coherence is the notion of "structural
corroboration," also known as triangulation (p. 55).
Consensus: The condition in which the readers of a work concur
that the findings and/or interpretations reported by the investigator are
consistent with their own experience or with the evidence presented (p. 56).
Finally, reviewers must assess the report's:
Instrumental Utility: The most important test of any
qualitative study is its usefulness. A good qualitative study can help us
understand a situation that would otherwise be enigmatic or confusing (p. 58).
A good study can help us anticipate the future, not in
the predictive sense of the word, but as a kind of road map or guide.
"Guides call our attention to aspects of the situation or place we might
otherwise miss" (Eisner, 1991, p. 59).
Addressing Trustworthiness in
Qualitative Research
The basic question addressed by the notion of
trustworthiness, according to Lincoln and Guba, is simple: "How can an
inquirer persuade his or her audiences that the research findings of an inquiry
are worth paying attention to?" (1985, p. 290). When judging qualitative
work, Strauss and Corbin (1990) believe that the "usual canons
of 'good science'...require redefinition in order to fit the realities of
qualitative research" (p. 250). Lincoln and Guba (1985, p. 300) have
identified one alternative set of criteria that correspond to those typically
employed to judge quantitative work (see Table 1).
Table 1
Comparison of criteria for judging
the quality of quantitative versus qualitative research
|
Conventional terms
|
Naturalistic terms
|
|
internal validity
|
Credibility
|
|
external validity
|
Transferability
|
|
Reliability
|
Dependability
|
|
Objectivity
|
Confirmability
|
Smith and Heshusius (1986) sharply criticize those writers,
like Lincoln and Guba, who they believe have adopted a stance of
"detente" with rationalists. They are particularly incensed by
Lincoln and Guba's use of "comparable criteria," which to their eyes
look little different than the conventional criteria they supposedly replace.
In either case, there must be a "belief in the assumption that what is
known-be it an existent reality or an interpreted reality-stands independent of
the inquirer and can be described without distortion by the inquirer" (p.
6). Smith and Heshusius claim that naturalistic research can offer only an
"interpretation of the interpretations of others, " and that to
assume an independent reality is "unacceptable" for the qualitative
researcher (p. 9).
Their stance is a strong one, because the only reality
it accepts is a completely mind-dependent one, which will vary from individual
to individual; in other words, for Smith and Heshusius, there is no "out
there" out there. For these researchers, it would not be possible to
choose a best interpretation from among the many available, because no
technique or interpretation can be "epistemologically privileged" (p.
9). To maintain this stance would seem to negate the value of doing research at
all, because it prohibits the possibility of reconciling alternative
interpretations.
Therefore, it is important to determine which criteria
are consistent with the naturalistic paradigm, yet which allow for a
declaration that "good science" has been carried out. In the following
sections, conventional and naturalistic criteria will be discussed, with the
goal of selecting criteria which are appropriate for judging the overall
trustworthiness of a qualitative study.
Internal Validity versus Credibility
In conventional inquiry, internal validity refers to
the extent to which the findings accurately describe reality. Lincoln and Guba
(1985) state that "the determination
of such isomorphism is in principle impossible" (p. 294), because one would have to know
the "precise nature of that reality" and, if one knew this already,
there would be no need to test it (p. 295). The conventional researcher must
postulate relationships and then test them; the postulate cannot be proved, but
only falsified. The naturalistic researcher, on the other hand, assumes the
presence of multiple realities and attempts to represent these multiple
realities adequately. Credibility becomes the test for this.
Credibility depends less on sample size than on the richness
of the information gathered and on the analytical abilities of the researcher (Patton, 1990). It can be
enhanced through triangulation of data. Patton identifies four types of
triangulation: 1) methods triangulation; 2) data triangulation; 3)
triangulation through multiple analysts; and 4) theory triangulation. Other
techniques for addressing credibility include making segments of the raw data
available for others to analyze, and the use of "member checks," in
which respondents are asked to corroborate findings (Lincoln and Guba, 1985, pp. 313-316).
External Validity / Generalizability
versus Transferability
In conventional research, external validity refers to
the ability to generalize findings across different settings. Making
generalizations involves a trade-off between internal and external validity (Lincoln and Guba, 1985). That is, in
order to make generalizable statements that apply to many contexts, one can
include only limited aspects of each local context.
Lincoln and Guba (1985) admit that generalizability is
"an appealing concept," because it allows a semblance of prediction
and control over situations (pp. 110-111). Yet they
suggest that the existence of local conditions "makes it impossible to
generalize" (p. 124). Cronbach (1975) discusses the problem by saying: The
trouble, as I see it, is that we cannot store up generalizations and constructs
for ultimate assembly into a network. It is as if we needed a gross of dry
cells to power an engine and could only make one a month. The energy would leak
out of the first cells before we had half the battery completed (p. 123).
According to Cronbach, "when we give proper weight to local
conditions, any generalization is a working hypothesis, not a conclusion"
(p. 125).
In the naturalistic paradigm, the transferability of a
working hypothesis to other situations depends on the degree of similarity
between the original situation and the situation to which it is transferred.
The researcher cannot specify the transferability of findings; he or she can
only provide sufficient information that can then be used by the reader to
determine whether the findings are applicable to the new situation (Lincoln and Guba, 1985). Other
writers use similar language to describe transferability, if not the word
itself. For example, Stake (1978) refers to what he calls
"naturalistic generalization" (p. 6). Patton suggests that
"extrapolation" is an appropriate term for this process (1990, p.
489). Eisner says it is a form of "retrospective generalization" that
can allow us to understand our past (and future) experiences in a new way
(1991, p. 205).
Reliability versus Dependability
Kirk and Miller (1986) identify three types of
reliability referred to in conventional research, which relate to: 1) the
degree to which a measurement, given repeatedly, remains the same; 2) the
stability of a measurement over time; and 3) the similarity of measurements
within a given time period (pp. 41-42). They note that "issues of
reliability have received little attention" from qualitative researchers,
who have instead focused on achieving greater validity in their work (p. 42).
Although they give several examples of how reliability might be viewed in
qualitative work, the essence of these examples can be summed up in the
following statement by Lincoln and Guba (1985): "Since there can be no
validity without reliability (and thus no credibility without dependability), a
demonstration of the former is sufficient to establish the latter" (p.
316).
Nevertheless, Lincoln and Guba do propose one measure which might
enhance the dependability of qualitative research. That is the use of an
"inquiry audit," in which reviewers examine both the process and the
product of the research for consistency (1985, p. 317).
Objectivity versus Confirmability
Conventional wisdom says that research which relies on
quantitative measures to define a situation is relatively value-free, and
therefore objective. Qualitative research, which relies on interpretations and
is admittedly value-bound, is considered to be subjective. In the world of
conventional research, subjectivity leads to results that are both unreliable
and invalid. There are many researchers, however, who call into question the
true objectivity of statistical measures and, indeed, the possibility of ever
attaining pure objectivity at all (Lincoln and Guba, 1985; Eisner, 1991).
Patton (1990) believes that the terms
objectivity and subjectivity have become "ideological ammunition in the
paradigms debate." He prefers to "avoid using either word and to stay
out of futile debates about subjectivity versus objectivity."
Instead, he strives for "empathic neutrality" (p. 55). While
admitting that these two words appear to be contradictory, Patton points out
that empathy "is a stance toward the people one encounters, while
neutrality is a stance toward the findings" (p. 58). A researcher who is
neutral tries to be non-judgmental, and strives to report what is found in a
balanced way.
Lincoln and Guba (1985) choose to speak of the
"confirmability" of the research. In a sense, they refer to the
degree to which the researcher can demonstrate the neutrality of the research
interpretations, through a "confirmability audit." This means
providing an audit trail consisting of 1) raw data; 2) analysis notes; 3) reconstruction
and synthesis products; 4) process notes; 5) personal notes; and 6) preliminary
developmental information (pp. 320 -321).
With regard to objectivity in qualitative research, it may be useful to
turn to Phillips (1990), who questions whether there is
really much difference between quantitative and qualitative research: Bad work
of either kind is equally to be deplored; and good work of either kind is
still-at best-only tentative. But the good work in both cases will be
objective, in the sense that it has been opened up to criticism, and the
reasons and evidence offered in both cases will have withstood serious
scrutiny. The works will have faced potential refutation, and insofar as they
have survived, they will be regarded as worthy of further investigation (p.
35).
Discussion and Conclusion
The increased interest in qualitative research in
recent years warrants a basic understanding of this paradigm on the part of all
technology education researchers. This overview of qualitative research methods
and issues represents a starting point only for those who are interested in
using and/or reviewing qualitative research. Readers can choose from a growing
body of literature on the topic for further guidance.
The decision to use qualitative methodologies should
be considered carefully; by its very nature, qualitative research can be
emotionally taxing and extraordinarily time consuming. At the same time, it can
yield rich information not obtainable through statistical sampling techniques.
In the past, graduate students contemplating the use of qualitative inquiry
were told that they would have to "sell" the idea to members of their
research committees, who would probably view qualitative research as inferior
to quantitative research. Fortunately, in most universities that belief has
changed, to the point where qualitative research is the paradigm of choice in
some schools. In spite of this growing acceptance, new researchers may still
encounter difficulties in finding faculty advisors who are skilled in this type
of research.
Qualitative researchers have a special responsibility
to their subjects and their readers. Since there are no statistical tests for
significance in qualitative studies, the researcher bears the burden of
discovering and interpreting the importance of what is observed, and of
establishing a plausible connection between what is observed and the
conclusions drawn in the research report. To do all of this skillfully requires
a solid understanding of the research paradigm and, ideally, guided practice in
the use of qualitative observation and analysis techniques.
There are many useful research designs, the selection
of which depends on the research questions being asked. Most importantly,
technology educators must rise to the challenge to find and use rigorous,
appropriate research techniques that address the significant questions facing
the field.
References
Bogdan, R. C., & Biklen, S. K. (1982). Qualitative
research for education: An introduction to theory and methods. Boston:
Allyn and Bacon, Inc.
Brown, D. C. (1996). Why ask why: patterns and themes of causal
attribution in the workplace. Journal of Industrial Teacher Education,33(4),
47-65.
Gagel, C. (1997). Literacy and technology: reflections and insights for
technological literacy. Journal of Industrial Teacher Education,34(3),
6-34.
Hansen, R. E. (1995). Teacher socialization in technological
education. Journal of Technology Education, 6(2),
34-45.
Johnson, S. D. (1995, Spring). Will our research hold up under
scrutiny? Journal of Industrial Teacher Education, 32(3),
3-6.
Lewis, T. (1997). Impact of technology on work and jobs in the printing
industry - implications for vocational curriculum. Journal of Industrial Teacher Education, 34(2),
7-28.
Lewis, T.
(1995). Inside three workplace literacy initiatives: possibilities and limits
of vocational institutions. Journal
of Industrial Teacher Education, 33(1), 60-82.
Tidak ada komentar:
Posting Komentar