\A)**2 is 18.20.
The final calculation gives the value of the "omega squared" term as 0.5617
or 56.17%. Thus, approximately 56% of the total variance is accounted for
by the unfoldment method in terms of the two experimental treatments.
So far, it has only seemed more economical to study Chinese characters in related groups, and also more logical to teach by unfolding the meaningful parts of a character. But more tangible proof is needed. Although we have cited numerous research articles in support of our intuition, some empirical validations are nonetheless necessary. Thus, the purpose of this chapter is to design a psychological experiment to collect pertinent data, and to apply statistical analysis in order to compare the different methods for learning characters.
This is a 3x2 factorial experiment. In this design, the two independent factors are teaching method and organization of character set, with the dependent variable being the number of correctly recalled character. Table 5.1 shows the six different treatment conditions which are defined as follows:
In this condition, the subject is first shown the form of the character in the modern script, is taught its meaning, shown the proper stroke order for writing the character, and finally again given the meaning of the character. A set of 40 characters is used. In this case, the characters are randomized before the teaching session commences. Thus, this treatment condition is referred to as: Stroke-Random (see Fig. 5.1).
This treatment condition is the same as Stroke-Random, except that the same set of characters are now ordered. Specifically, the characters are temporally ordered according to a story as defined in the narrative teaching method of Treatment Condition 3, to be considered next (see Fig. 5.2).
This condition is the same as Stroke-Random, except that a story linking the 40 characters is added (see Appendix 5.1). This story, which acts as an "advanced organizer", is presented first, before the teaching of the characters begins. Since a narrative is employed, it is called Narrative-Room
This is the same as Stroke-Ordered, except that the story is narrated first.
The subject is shown the form of the character both in modern script and in Small Seal. The meaning of the character is then unfolded in terms of the meanings of its component parts (i.e., the formal elements). This method of successive unfoldment is applied to the same set of 40 randomized characters. Thus, the treatment condition is defined as Unfoldment-Random .
This is the same teaching method as used in Unfoldment-Random, except on a set of 40 hierachically arranged characters (see Fig. 5.3).
Sixty-three subjects from Humboldt State University were used for experiment. These are college students ranging from freshmen to graduate students. None of them had any knowledge of the chinese language.
A set of 40 test characters was used throughout the experiment. The size of this set was determined through a pilot study so as not to cause a ceiling effect on recall. From this source of 40 characters, three different test sets were produced, namely:
In addition, a story is provided as the context for the "NARRATIVE" teaching method. This material is entitled, "A Mid-Summer Night's Dream at Tokyo International Airport". It was purposely written as a mystery story so that it would be more interesting than the ordinary stories in a traditional textbook. It was hoped that thereby this teaching method would be given a reasonable chance of success.
A Chinese character possesses three distinctive properties: form, meaning, and sound (Lew, 1923). In order to isolate the effects of these attributes, we have decided to investigate only the effects of the first two of these three properties at the present.
Regarding form, each test character was designed to be 1.5" square, written in black ink on a standard 5"x8" card, from which transparencies were produced for use with an overhead projector. Fig. 5.4 shows a sample card, on which is written a test character in modern script on the left, with its counterpart in Small Seal on the right. Near the bottom of the card is shown the proper stroke order in modern script. On the reverse side of the card is presented the successive unfoldment of the meaning of the character, based on the hierachical analysis of the formal elements involved. After completion of the unfoldment process, the overall meaning is recapitulated to provide an integrated effect.
The Small Seal character were included as an integral part of the teaching method only in treatment conditions Unfoldment-Random and Unfoldment-Ordered, but were not part of the teaching method under the first four treatment conditions, although they were visually present on the cards presented to the subjects. The intent of this was to keep the visual stimuli consistens throughout. We have included the Small Seal characters in the above-mentioned treatment conditions because we recognize that these are powerful symbols which reveal explicitly the formal elements involved and that these are valuable learning aids.
Part of the design considerations were these criteria pertaining to form: diversity in form, a wide range of strokes, and a high frequency of usage. All these criteria were incorporated in our choice of the test set of characters. Specifically, three families of characters were chosen, headed by three significs: One, Sun and Rain.
Regarding meaning, diversity and richness in the semantic aspect of the test characters were taken into account as part of the design considerations.
In addition, the design of the test set had to satisfyrequirements of Chinese orthography. By that we mean that there had to exist in the test set not only a subset of derived significs and a rich subset of related phonetics, but also a large body of related phonograms so that the experiment was related to the theory of Chinese orthography, and the results of the test thus shed some light on the nature of Chinese ortography.
Last, but not least, an important design consideration was the means of delivery during testing, specifically consistency and reliability, since six different treatment conditions were involved, and each treatment condition was to be administered to different groups of test subjects and at different times. In this regard, consistency and reliability were achieved by producing audio tapes in advance. Simultaneously, fairness in the comparison of the different teaching methods was rendered by providing equal time for the teaching of the cortresponding characters in each of the different teaching methods. Again, this was made possible by means of pre-recorded test tapes.
Each treatment condition was composed of two phases: teaching, and testing, for which the procedures were as follows:
1) The teaching tape was firstpresented to the subjects in conjunction with the transparencies of the test cards. One-by-one each character was taught until all 40 characters were finished.
2) The subjects were then asked to count in unison backwards from 33 to zero in intervals of three. This served to prevent the subjects from rehearsing.
3) The "recall" test tape was then played in conjunction with the set of "test character" transparancies. The test characters were presented in the same order as the corresponding teaching tape. Thus, each "test character" for recall was given equal time, about 20 seconds.
The test data of the six different treatment conditions are summarized in Table 5.2. Each subject, randomly assigned to one of the conditions, served in only one of the treatment conditions, and thus contributed to a single observation. Hence, the experiment is a completely randomized design, corresponding to the fixed-effects model (Keppel, 1973).
In Table 5.2 are summarized in Table 5.2. Each subject, randomly assigned to one of the conditions, served in only one of the treatment conditions, and thus contributed to a single observation. Hence, the experiment is a completely randomized design, corresponding to the fixed-effects model (Keppel, 1973).
IN Table 5.2 are summarized the means, standard deviation, and sample sizes for each of the six treatment conditions as per the experimental design of Table 5.1. While condition Stroke-Random gives the lowest mean, condition Unfoldment-Ordered provides the highest mean. Fig. 5.5 is a plot of the actual number of correct recalls versus the treatment conditions. This plot not only shows the dispersion of the individual recall scores, but also indicates the mean and standard deviation for each treatment condition.
Table 5.5 is a summary of the Two-Factor analysis of variance, of which the F ratio is indicated in the last column. It is important to note that, in our case, not only are main effects of "teaching method" and "character organization" present, but there also exists a "method x organization" interaction. The F ratio clearly suggests that "teaching method" exerts the predominant influence.
Table 5.6a is a summary of the analysis of "character organization" at different levels of "teaching method". The F test states that the simple main effects of "character organisation" at levels a1 and a3 are significant, but this is not true at a2. Table 5.6b shows that the simple main effects at b1 and b2 are both significant. Fig. 5.6b in conjunction with the pairwise tests for treatment conditions four and six shows that the main locus of the interaction is the effect of ordering on the unfoldment method.
Since an interaction effect is present, the locus of interaction must be found. Fig. 5.6 is a plot of the "mean" versus "teaching method" and "character organization" is revealed by the appearance of the non-parallel lines in the double-classification plot of the cell "means"
A pairwise comparison was performed on the following:
(1) For the Stroke method: Random vs. Ordered (Temporal).
(2) For the Narrative method: Random vs. Ordered (Temporal).
(3) For the Unfoldment method: Random vs. Ordered (Hierachical).
(4) For Randomized Character Set: Stroke vs. Narrative.
(5) For Randomized Character Set: Stroke vs. Unfoldment.
(6) For Ordered Character Set: Stroke vs. Narrative.
(7) For Ordered Character Set: Narrative vs. Unfoldment.
Table 5.7 is a summary of the analysis, in which the corresponding MS\A, F, and significance level are indicated, respectively. We thus conclude from Table 5.7 that the Unfoldment-Ordered (Hierachical) combination provided the best result. Note that the differences shown are significant at p < 0.01. That is, F(1, 57) = 11.56 at p < 0.01 for case (3), and F(1, 57) = 36.15 at p < 0.01 for case (7).
In considering the magnitude of the treatment effect (for treatment condition
Unfoldment-Ordered in reference to Narrative-Ordered
), the index (omega squared) is used as an estimate, which is
essentially a correlation ratio contrasting the variability due to the
experimental manipulations to the total variability in the experiment. Since
the calculated SS\a = 645 and MS\(S/A) = 269.1, therefore MS\A = 645 and
MS\(S/A) = 14.2, for which s, the harmonic mean, is 10.47 and (IMAGE T)
is 22.03, therefore, (\A)**2 is 18.20.
The final calculation gives the value of the "omega squared" term as 0.5617
or 56.17%. Thus, approximately 56% of the total variance is accounted for
by the unfoldment method in terms of the two experimental treatments.
The reason that the treatment of "Narrative-Ordered (TEMPORAL)" was used as the reference is bacause it is the condition closest to the traditional teaching. Therefore, it is more realistic to compare the new teaching method against that.
Fig. 5.1 shows the serial position of the characters under randomized conditions. Fig. 5.2 shows the serial position arranged in the temporal order of the story narrated (Appendix vi). Fig 5.3 shows the serial position in the hierachical order of the characters (Tseng and Wang, 1983).
Fig. 5.9 is a summary statement for all the randomized cases. Recalls are plotted against the serial positions of Fig. 5.1 for treatment conditions Stroke-Random, Narrative-Random, and Unfoldment-Random. The interpretation of the results of these curves will appear in the Discussion section.
Likewise, Fig. 5.10 contains the two cases for the temporally ordered. While Curve Stroke-Ordered is for recalls of treatment condition Stroke-Ordered, Curve Narrative-Ordered corresponds to treatment condition Narrative-Ordered . In the same way, Fig. 5.11 plots the recall of the hierachically ordered (Fig. 5.3).
Intellectually, one can appreciate tha the "number of recalls" should be inversely proportional to the "number of strokes" that a character has. But, to what extent is this statement true for the different treatment conditions? That should be interesting to know. For that, we have calculated the correlation coefficient pertaining to each of the six conditions, and the results are tabulated in Table 5.9. It was found that the influence of the stroke order on recall is relatively small, no more than 25%. Therefore, the main cause must be somewhere else in order to account for the substantially higher recall score of the new teaching method, the Unfoldment-Hierarchical method.
It is of interest to note (Fig. 5.9) that in case of randomized characters, the curves of all three treatment conditions Stroke-Random, Narrative Random and Unfoldment-Random exhibit the same pattern of a recency effect, but not the primacy effect. Regarding Fig. 5.10, a primary effect was observed for Stroke-Ordered and Narrative-Ordered in the form of a primacy effect, but no recency effect.
It is generally supposed that Narrative is a positive factor for learning. If so, it may be applied to the Unfoldment cases also, as the two processes are not mutually exclusive. The question, then, is: "should there be an increase or a decrease in the recall performance?"
We shall define the following case as Narrative + (Unfoldment-Hierachical) , and examine it in detail. In this case, the story was first narrated to provide the subjects with a general framework before commencing the Unfoldment-Hierarchical teaching. It was hoped that such an additional effort should help, but the result of the experiment was dissappointing, which is clearly reflected by the recall mean of 17.2 (with s.d. = 4.53 and N = 5), which is much less than 27.6. A One-Factor Anova is used to determine its signifiance. The result of that analysis is shown in Table 5.8, which indicates that it is statistically significant at p < 0.01. Comments regarding this negative contribution of the Narrative will be made in the Discussion section following.
The result revealed that while Unfoldment teaching of the Randomized set of characters resulted in higher recall scores, even greater gain is possible by applying Unfoldment to the Ordered (Hierarchical) set of characters. We see that the contribution due to hierarchical ordering of the character set is significant. This component of learning, since it is derived linguistic knowledge about the character set, is gained independently of the effort of the learner. In other words, the learner gets this free when it is embodied in the teaching as an internal "advanced organizer".
Treatment condition Narrative-Ordered may be regarded as the condition closest to the traditional way of teaching. By comparing the new teaching method, Unfoldment-Ordered, with Narrative-Ordered, we can get some idea about how good the new method can be. The comparison reveals that the mean number of characters recalled under treatment condition Unfoldment-Ordered is 176% that of Narrative-Ordered (i.e., 27.6 vs. 16.5), which is a sizable difference, indeed.
5) Fig. 5.11 contains the curve, Unfoldment-Ordered, with recalls plotted against hierarchically ordered characters. The Curve reveals that in general recall decreases as the hierarchical order of a character increases.
6) In reference to Table 5.1, it was surprising at first to note that the recall of Narrative-Ordered was actually worse than that of Narrative-Random, but further consideration suggests that this may be due to over-loading of the mind. As the task of learning 40 characters is already very demanding, any additional learning, such as the learning of an extra, external "organizer", would only further tax the mind. Thus, learning effectiveness and efficiency decrease due to the interference effect. This seems to indicate that the Narrative is a poor "advanced organizer"
This negative effect w4as further substantiated in the case of Narrative + (Unfoldment-Hierarchical), where the result of 17.2 as the recall mean reflects a decrease of 37.7% from that of Unfoldment-Ordered. This experiment is a vivid demonstration that the external "advanced organizer" clashes with the internal one.
We need to examine the relative merits of these two types of advanced organizers. The central question is: Is the investment of acquiring the advanced organizer for one set of characters worthwhile in terms of its transferability to the learning of another set(s)? In this regard, we see that an internal "advanced organizer" has the merit of transferability, while the external ones do not.
To summarize, the results of the 3x2 factorial experiment were most encouraging, since it not only showed that the method of teaching is important, but also the organization (of characters to be learned). In fact, the interaction of these two independent factors caused even greater gains. Specifically, using the traditional Narrative-Ordered (Temporal) case as the reference, the comparison revealed that the new teaching method by Unfoldment on the hierarchically ordered set of characters is 176% that of the referenced. This is a sizeable improvement indeed.