AANNOOVVAA:: FFuullll FFaaccttoorriiaall DDeessiiggnnss IInnttrroodduuccttiioonn ttoo AANNOOVVAA:: FFuullll FFaaccttoorriiaall DDeessiiggnnss •• IInnttrroodduuccttiioonn…………………………………………………………………………………………………… pp.. 22 •• MMaaiinn EEffffeeccttss ……………………………………………………………………..........…………....………… pp.. 99 •• IInntteerraaccttiioonn EEffffeeccttss ……………………………………………………....……....…………………….. pp.. 1122 •• MMaatthheemmaattiiccaall FFoorrmmuullaass aanndd CCaallccuullaattiinngg SSiiggnniiffiiccaannccee …… pp.. 2200 •• RReessttrriiccttiioonnss •• FFiisshheerr AAssssuummppttiioonnss……………………………………………………………………………… pp.. 3355 •• FFiixxeedd,, CCrroosssseedd EEffffeeccttss…………………………………………………………………… pp.. 3366 •• 11 IInnttrroodduuccttiioonn ttoo AANNOOVVAA AAnnaallyyssiiss ooff vvaarriiaannccee ((AANNOOVVAA)) is a statistical technique used to investigate and model the relationship between a response variable and one or more independent variables. Each explanatory variable (ffaaccttoorr) consists of two or more categories (lleevveellss). ANOVA tests the nnuullll hhyyppootthheessiiss that the population means of each level are equal, versus the aalltteerrnnaattiivvee hhyyppootthheessiiss that at least one of the level means are not all equal. EXAMPLE 1: A 2003 study was conducted to test if there was a difference in attitudes towards science between boys and girls. FFaaccttoorr : gender with LLeevveellss : boys and girls UUnniitt (Experimental Unit or Subject): each individual child RReessppoonnssee VVaarriiaabbllee: Each child’s score on an attitude assessment. NNuullll HHyyppootthheessiiss: boys and girls have the same mean score on the assessment. AAlltteerrnnaattiivvee HHyyppootthheessiiss: boys and girls have different mean scores on the assessment. 22 IInnttrroodduuccttiioonn ttoo AANNOOVVAA Example 1 can be analyzed with ANOVA or a two-sample t-test discussed in introductory statistics courses. In both methods the experimenter collects sample data and calculates averages. If the means of the two levels are “significantly” far apart, the experimenter will accept the alternative hypothesis. While their calculations differ, AANNOOVVAA aanndd ttwwoo--ssaammppllee tt--tteessttss aallwwaayyss ggiivvee iiddeennttiiccaall rreessuullttss iinn hhyyppootthheessiiss tteessttss ffoorr mmeeaannss wwiitthh oonnee ffaaccttoorr aanndd ttwwoo lleevveellss.. Unfortunately, modeling real world phenomena often requires more than just one factor. In order to understand the sources of variability in a phenomenon of interest, AANNOOVVAA ccaann ssiimmuullttaanneeoouussllyy tteesstt sseevveerraall ffaaccttoorrss eeaacchh wwiitthh sseevveerraall lleevveellss.. Although there are situations where t-tests should be used to simultaneously test the means of multiple levels, doing so create a multiple comparison problem. Determining when to use ANOVA or t- tests is discussed in all the suggested texts at the end of this tutorial. 33 IInnttrroodduuccttiioonn ttoo AANNOOVVAA KKeeyy sstteeppss iinn ddeessiiggnniinngg aann eexxppeerriimmeenntt iinncclluuddee:: 11)) IIddeennttiiffyy ffaaccttoorrss ooff iinntteerreesstt and a rreessppoonnssee vvaarriiaabbllee.. 22)) DDeetteerrmmiinnee aapppprroopprriiaattee lleevveellss for each explanatory variable. 33)) DDeetteerrmmiinnee aa ddeessiiggnn structure. 44)) RRaannddoommiizzee the order in which each set of conditions is run and collect the data. 55)) OOrrggaanniizzee tthhee rreessuullttss in order to draw appropriate conclusions. This presentation will discuss how to organize and draw conclusions for a specific type of design structure, the ffuullll ffaaccttoorriiaall ddeessiiggnn. This design structure is appropriate for ffiixxeedd eeffffeeccttss and ccrroosssseedd ffaaccttoorrss, which are defined at the end of this tutorial. Other design structures are discussed in the AANNOOVVAA:: AAddvvaanncceedd DDeessiiggnnss tutorial. 44 IInnttrroodduuccttiioonn ttoo MMuullttiivvaarriiaattee AANNOOVVAA EXAMPLE 2: Soft Drink Modeling Problem (Montgomery p. 232): A soft drink bottler is interested in obtaining more uniform fill heights in the bottles produced by his manufacturing process. The filling machine theoretically fills each bottle to the correct target height, but in practice, there is variation around this target, and the bottler would like to understand better the sources of this variability and eventually reduce it. The engineer can control three variables during the filling process (each at two levels): FFaaccttoorr AA: Carbonation with LLeevveellss : 10% and 12% FFaaccttoorr BB: Operating Pressure in the filler with LLeevveellss : 25 and 30 psi FFaaccttoorr CC: Line Speed with LLeevveellss: 200 and 250 bottles produced per minute (bpm) UUnniitt: Each bottle RReessppoonnssee VVaarriiaabbllee: Deviation from the target fill height SSiixx HHyyppootthheesseess wwiillll bbee ssiimmuullttaanneeoouussllyy tteesstteedd The steps to designing this experiment include: 11)) IIddeennttiiffyy ffaaccttoorrss ooff iinntteerreesstt and a rreessppoonnssee vvaarriiaabbllee.. 22)) DDeetteerrmmiinnee aapppprroopprriiaattee lleevveellss for each explanatory variable. 55 IInnttrroodduuccttiioonn ttoo MMuullttiivvaarriiaattee AANNOOVVAA 33)) DDeetteerrmmiinnee aa ddeessiiggnn ssttrruuccttuurree: Design structures can be very complicated. One of the most basic structures is called the ffuullll ffaaccttoorriiaall ddeessiiggnn. This design tests every combination of factor levels an equal amount of times. To list each factor combination exactly once 1st Column--alternate every other (20) row 2nd Column--alternate every 2 (21) rows test A B C 3rd Column--alternate every 4th (=22) row 1 10% 25 200 2 12% 25 200 TThhiiss iiss ccaalllleedd aa 2233 ffuullll ffaaccttoorriiaall ddeessiiggnn ((ii..ee.. 33 3 10% 30 200 ffaaccttoorrss aatt 22 lleevveellss wwiillll nneeeedd 88 rruunnss)).. EEaacchh rrooww iinn 4 12% 30 200 tthhiiss ttaabbllee ggiivveess aa ssppeecciiffiicc ttrreeaattmmeenntt tthhaatt wwiillll bbee rruunn.. FFoorr eexxaammppllee,, tthhee ffiirrsstt rrooww rreepprreesseennttss aa 5 10% 25 250 ssppeecciiffiicc tteesstt iinn wwhhiicchh tthhee mmaannuuffaaccttuurriinngg pprroocceessss 6 12% 25 250 rraann wwiitthh AA sseett aatt 1100%% ccaarrbboonnaattiioonn,, BB sseett aatt 2255 7 10% 30 250 ppssii,, aanndd lliinnee ssppeeeedd,, CC,, iiss sseett aatt 220000 bbmmpp.. 8 12% 30 250 *If there were four factors each at two levels there would be 16 treatments. *If factor C had 3 levels there would be 2*2*3 = 12 treatments. 66 IInnttrroodduuccttiioonn ttoo MMuullttiivvaarriiaattee AANNOOVVAA 44)) RRaannddoommiizzee the order in which each set of test conditions is run and collect the data. In this example the tests will be run in the following order: 7, 4, 1, 6, 8, 2, 3. run A B C If the tests were run in the original order test Results Carb Pressure speed test order, time would be ccoonnffoouunnddeedd 3 1 10% 25 200 -4 (aalliiaasseedd) with factor C. 7 2 12% 25 200 1 Randomization doesn’t guarantee 8 3 10% 30 200 -1 that there will be no confounding 2 4 12% 30 200 5 between time and a factor of interest, 6 5 10% 25 250 -1 however, it is the best practical 4 6 12% 25 250 3 technique available to protect against 1 7 10% 30 250 2 confounding. 5 8 12% 30 250 11 IInn tthhee ffoolllloowwiinngg sslliiddeess AA-- wwiillll rreepprreesseenntt ccaarrbboonnaattiioonn aatt tthhee llooww lleevveell ((1100%% ccaarrbboonnaattiioonn)) aanndd AA++ wwiillll rreepprreesseenntt ccaarrbboonnaattiioonn aatt tthhee hhiigghh lleevveell ((1122%% ccaarrbboonnaattiioonn)).. IInn tthhee ssaammee mmaannnneerr BB++,, BB--,, CC++ aanndd CC-- wwiillll rreepprreesseenntt ffaaccttoorrss BB aanndd CC aatt hhiigghh aanndd llooww lleevveellss.. 77 IInnttrroodduuccttiioonn ttoo MMuullttiivvaarriiaattee AANNOOVVAA 55)) OOrrggaanniizzee tthhee rreessuullttss in order to draw appropriate conclusions. RReessuullttss aarree tthhee ddaattaa ccoolllleecctteedd ffrroomm rruunnnniinngg eeaacchh ooff tthheessee 88 == 2233 ccoonnddiittiioonnss.. FFoorr tthhiiss eexxaammppllee tthhee RReessuullttss ccoolluummnn iiss tthhee oobbsseerrvveedd ddeevviiaattiioonn ffrroomm tthhee ttaarrggeett ffiillll hheeiigghhtt iinn aa pprroodduuccttiioonn rruunn ((aa ttrriiaall)) ooff bboottttlleess aatt eeaacchh sseett ooff tthheessee 88 ccoonnddiittiioonnss.. OOnnccee wwee hhaavvee ccoolllleecctteedd oouurr ssaammpplleess ffrroomm oouurr 88 rruunnss,, wwee ssttaarrtt oorrggaanniizziinngg tthhee rreessuullttss bbyy A B C Results ccoommppuuttiinngg aallll aavveerraaggeess aatt llooww aanndd hhiigghh 10% 25 200 -4 lleevveellss.. 12% 25 200 1 TToo ddeetteerrmmiinnee wwhhaatt eeffffeecctt cchhaannggiinngg tthhee lleevveell 10% 30 200 -1 ooff AA hhaass oonn tthhee rreessuullttss,, ccaallccuullaattee tthhee aavveerraaggee vvaalluuee ooff tthhee tteesstt rreessuullttss ffoorr AA-- aanndd AA++.. 12% 30 200 5 WWhhiillee tthhee oovveerraallll aavveerraaggee ooff tthhee rreessuullttss ((ii..ee.. 10% 25 250 -1 tthhee GGrraanndd MMeeaann)) iiss 22,, tthhee aavveerraaggee ooff tthhee 12% 25 250 3 rreessuullttss ffoorr AA-- ((ffaaccttoorr AA rruunn aatt llooww lleevveell)) iiss 10% 30 250 2 ((--44 ++ --11 ++ --11 ++ 22))//44 == --11 12% 30 250 11 55 iiss tthhee aavveerraaggee vvaalluuee ooff tthhee tteesstt rreessuullttss ffoorr AA++ ((ffaaccttoorr AA rruunn aatt aa hhiigghh lleevveell)) Grand Mean 2 88 MMaaiinn EEffffeeccttss The B and C averages at low and high levels also calculated. A B C Results A Avg. B Avg. C Avg. 10% 25 200 -4 low -1 -.25 .25 12% 25 200 1 high 5 4.25 3.75 10% 30 200 -1 12% 30 200 5 10% 25 250 -1 TThhee mmeeaann ffoorr BB-- iiss ((-- 44 ++ 11 ++ --11 ++ 33))//44 == --..2255 12% 25 250 3 TThhee mmeeaann ffoorr CC-- iiss ((-- 44 ++11++--11++55))//44 == ..2255 10% 30 250 2 12% 30 250 11 Notice that each of these eight trial results are used multiple times to calculate six different averages. This can be effectively done because the full factorial design is bbaallaanncceedd. For example when calculating the mean of C low (CC--), there are 2 A highs (AA++) and 2 A lows (AA--), thus the mean of A is not confounded with the mean of C. This balance is true for all mean calculations. 99 MMaaiinn EEffffeeccttss Often the impact of changing factor levels are described as effect sizes. A MMaaiinn EEffffeeccttss is the difference between the factor average and the grand mean. Subtract the A Avg. B Avg. C Avg. A Effect B Effect C Effect grand low -1 -.25 .25 -3 -2.25 -1.75 mean (2) high 5 4.25 3.75 from 3 2.25 1.75 each cell Effect of AA++ == average of factor A+ minus the grand mean = 5 – 2 = 3 Effect of CC-- == ..25 – 2 = -1.75 Effect sizes determine which factors have the most significant impact on the results. Calculations in ANOVA determine the significance of each factor based on these effect calculations. 1100