public class PostHocGUI
extends javax.swing.JFrame
implements java.awt.event.ActionListener, java.awt.event.FocusListener, java.awt.event.ItemListener
PostHoc - GUI Version |
NOTE: This API assumes the reader has studied the API for AnovaGUI and has a general understanding of the ANOVA test.
An independent variable is manipulated over levels. In HCI, the levels of an independent variable are often called test conditions. In statistics references, they are often called treatments. An independent variable must have at least two levels, as in the mouse vs. touchpad example above. However, an independent variable often has more than two levels. An example might be feedback mode with levels auditory, visual, tactile, none. If the independent variable has more than two levels, a significant F-test only indicates that at least one level is significantly different from one other level. It does not indicate which levels differ significantly from one another. To determine which levels (test conditions) different significantly from one another, post hoc pairwise comparisons are performed.
This utility supports four post hoc comparisons tests:
Launch PostHocGUI from GoStats as follows:
PostHocGUI accommodates data in several formats. If the data are organized in a rectangular table, just (i) select the data file, (ii) choose the desired post hoc test, and (iii) click "Analyse." The assumption in this case is that the data are organized in an n × k table, with n rows and k columns. n is the number of participants, k is the number of test conditions. This is the usual format for data in a within-subjects experiment. With k conditions there are k × (k - 1) / 2 comparisons.
11 11 21 16 18 11 22 15 17 10 18 13 19 15 21 20 13 17 23 10 10 15 15 20 14 14 15 13 13 14 19 18 19 18 16 12 10 17 21 18 10 19 22 13 16 14 18 20 10 20 17 19 10 13 21 18 20 17 14 18 18 17 17 14An AVOVA test (not shown) reveals a significant outcome. However, since there were four test conditions, post hoc comparisons are warranted. The pairwise comparisons are performed by PostHocGUI as follows: (click to enlarge)
Of the six possible comparisons, only two are significant. The data in columns 1 and 3 are significantly different, as are the data in columns 2 and 3. Note: Full details for this example are found in section 6.2.4 Post Hoc Comparisons in MacKenzie's Human-Computer Interaction: An Empirical Research Perspective (2nd edition).
If counterbalancing was used and the data set includes a right-hand column for the group code, select "1" in the combobox for "Ignore 'n' right-hand columns."
For between-subjects experiments, the data are sometimes organized in a vertical column. In this case, select the option "Vertical data in left-hand column." You also need to select the number of right-hand columns to ignore, for example, 1, 2, or 3 for experiments with 1, 2, or 3 between-subjects factors. Note that the post hoc tests only apply to the groups for the first between-subjects factor. The number of groups is automatically determined, according to the codes for the first between-subjects factor (column 2).
As often occurs, the results of different post hoc tests are slightly different.
The primary source for coding these tests was Chapter 21 in D. J. Sheskin's Handbook of Parametric and Nonparametric Statistical Procedures, 5th ed., CRC Press, 2011, pp. 886-999. Sheskin uses the following data set throughout the chapter:
8 7 4 10 8 8 9 5 7 10 8 5 9 5 7There are 3 groups of participants with 5 participants in each group. Each group was tested on a different condition, with the scores for each condition/group appearing in separate columns. There were 15 total participants.
As well as describing the calculations leading to the F-test, Sheskin demonstrates the various post hoc comparisons that often follow a statistically significant ANOVA. Sheskin's calculations are embedded in this utility. His data set is in the file sheskin-ex1.txt, slightly rearranged as per our usual organization for between-subjects experiments:
8 G1 10 G1 9 G1 10 G1 9 G1 7 G2 8 G2 5 G2 8 G2 5 G2 4 G3 8 G3 7 G3 5 G3 7 G3A post hoc analysis is performed on this data set as follows:
The results are consistent between the four post hoc tests. There is a significant difference between groups 1 and 2 and between groups 1 and 3. There is no significant difference between groups 2 and 3.
The most difficult calculation is that for the critical difference. The values above are consistent with those in the Sheskin reference. As further verification, the same test was undertaken with StatView (now JMP). The results follow:
Again, the results are consistent. There are minor deviations between some of the critical
values, however. For example, the Bonferroni-Dunn critical value calculated by GoStats is 2.44.
The critical value in Sheskin's handbook is 2.43 (p. 907) and by StatView 2.423 (see
above). The differences may be due to rounding or to the use of embedded lookup tables in
StatView versus the use of methods of the Statistics
class in GoStats.
Significance is at the α = .05 level. For post hoc comparisons, the alpha level is "family-wise". This implies that the overall alpha for the set of comparisons is α = .05. To maintain the overall alpha, the individual comparisons use a more stringent alpha level. The Bonferroni-Dunn test, for example, uses .05 / n, where n is the number of comparisons. With three comparisons, α = .05 / 3 = .0167. Thus, a comparison is only deemed significant if p < .0167. See above. The other tests have different approaches to maintaining the family-wise alpha level. See Sheskin for complete details.
Note that if the above data are re-organized into the within-subjects format for StatView (compact variables), the result of the pairwise comparisons test is the same. This is not as precise since the residual mean squares are different for within-subjects data than for between-subjects data. However, the differences are minor. Sheskin provides additional details.
CROSSHAIR_CURSOR, DEFAULT_CURSOR, E_RESIZE_CURSOR, HAND_CURSOR, ICONIFIED, MAXIMIZED_BOTH, MAXIMIZED_HORIZ, MAXIMIZED_VERT, MOVE_CURSOR, N_RESIZE_CURSOR, NE_RESIZE_CURSOR, NORMAL, NW_RESIZE_CURSOR, S_RESIZE_CURSOR, SE_RESIZE_CURSOR, SW_RESIZE_CURSOR, TEXT_CURSOR, W_RESIZE_CURSOR, WAIT_CURSOR
BOTTOM_ALIGNMENT, CENTER_ALIGNMENT, LEFT_ALIGNMENT, RIGHT_ALIGNMENT, TOP_ALIGNMENT
Modifier and Type | Method and Description |
---|---|
void |
actionPerformed(java.awt.event.ActionEvent ae) |
void |
focusGained(java.awt.event.FocusEvent fe) |
void |
focusLost(java.awt.event.FocusEvent fe) |
void |
itemStateChanged(java.awt.event.ItemEvent ie) |
getAccessibleContext, getContentPane, getDefaultCloseOperation, getGlassPane, getGraphics, getJMenuBar, getLayeredPane, getRootPane, getTransferHandler, isDefaultLookAndFeelDecorated, remove, repaint, setContentPane, setDefaultCloseOperation, setDefaultLookAndFeelDecorated, setGlassPane, setIconImage, setJMenuBar, setLayeredPane, setLayout, setTransferHandler, update
addNotify, getCursorType, getExtendedState, getFrames, getIconImage, getMaximizedBounds, getMenuBar, getState, getTitle, isResizable, isUndecorated, remove, removeNotify, setBackground, setCursor, setExtendedState, setMaximizedBounds, setMenuBar, setOpacity, setResizable, setShape, setState, setTitle, setUndecorated
addPropertyChangeListener, addPropertyChangeListener, addWindowFocusListener, addWindowListener, addWindowStateListener, applyResourceBundle, applyResourceBundle, createBufferStrategy, createBufferStrategy, dispose, getBackground, getBufferStrategy, getFocusableWindowState, getFocusCycleRootAncestor, getFocusOwner, getFocusTraversalKeys, getIconImages, getInputContext, getListeners, getLocale, getModalExclusionType, getMostRecentFocusOwner, getOpacity, getOwnedWindows, getOwner, getOwnerlessWindows, getShape, getToolkit, getType, getWarningString, getWindowFocusListeners, getWindowListeners, getWindows, getWindowStateListeners, hide, isActive, isAlwaysOnTop, isAlwaysOnTopSupported, isAutoRequestFocus, isFocusableWindow, isFocusCycleRoot, isFocused, isLocationByPlatform, isOpaque, isShowing, isValidateRoot, pack, paint, postEvent, removeWindowFocusListener, removeWindowListener, removeWindowStateListener, reshape, setAlwaysOnTop, setAutoRequestFocus, setBounds, setBounds, setCursor, setFocusableWindowState, setFocusCycleRoot, setIconImages, setLocation, setLocation, setLocationByPlatform, setLocationRelativeTo, setMinimumSize, setModalExclusionType, setSize, setSize, setType, setVisible, show, toBack, toFront
add, add, add, add, add, addContainerListener, applyComponentOrientation, areFocusTraversalKeysSet, countComponents, deliverEvent, doLayout, findComponentAt, findComponentAt, getAlignmentX, getAlignmentY, getComponent, getComponentAt, getComponentAt, getComponentCount, getComponents, getComponentZOrder, getContainerListeners, getFocusTraversalPolicy, getInsets, getLayout, getMaximumSize, getMinimumSize, getMousePosition, getPreferredSize, insets, invalidate, isAncestorOf, isFocusCycleRoot, isFocusTraversalPolicyProvider, isFocusTraversalPolicySet, layout, list, list, locate, minimumSize, paintComponents, preferredSize, print, printComponents, remove, removeAll, removeContainerListener, setComponentZOrder, setFocusTraversalKeys, setFocusTraversalPolicy, setFocusTraversalPolicyProvider, setFont, transferFocusDownCycle, validate
action, add, addComponentListener, addFocusListener, addHierarchyBoundsListener, addHierarchyListener, addInputMethodListener, addKeyListener, addMouseListener, addMouseMotionListener, addMouseWheelListener, bounds, checkImage, checkImage, contains, contains, createImage, createImage, createVolatileImage, createVolatileImage, disable, dispatchEvent, enable, enable, enableInputMethods, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, getBaseline, getBaselineResizeBehavior, getBounds, getBounds, getColorModel, getComponentListeners, getComponentOrientation, getCursor, getDropTarget, getFocusListeners, getFocusTraversalKeysEnabled, getFont, getFontMetrics, getForeground, getGraphicsConfiguration, getHeight, getHierarchyBoundsListeners, getHierarchyListeners, getIgnoreRepaint, getInputMethodListeners, getInputMethodRequests, getKeyListeners, getLocation, getLocation, getLocationOnScreen, getMouseListeners, getMouseMotionListeners, getMousePosition, getMouseWheelListeners, getName, getParent, getPeer, getPropertyChangeListeners, getPropertyChangeListeners, getSize, getSize, getTreeLock, getWidth, getX, getY, gotFocus, handleEvent, hasFocus, imageUpdate, inside, isBackgroundSet, isCursorSet, isDisplayable, isDoubleBuffered, isEnabled, isFocusable, isFocusOwner, isFocusTraversable, isFontSet, isForegroundSet, isLightweight, isMaximumSizeSet, isMinimumSizeSet, isPreferredSizeSet, isValid, isVisible, keyDown, keyUp, list, list, list, location, lostFocus, mouseDown, mouseDrag, mouseEnter, mouseExit, mouseMove, mouseUp, move, nextFocus, paintAll, prepareImage, prepareImage, printAll, removeComponentListener, removeFocusListener, removeHierarchyBoundsListener, removeHierarchyListener, removeInputMethodListener, removeKeyListener, removeMouseListener, removeMouseMotionListener, removeMouseWheelListener, removePropertyChangeListener, removePropertyChangeListener, repaint, repaint, repaint, requestFocus, requestFocusInWindow, resize, resize, revalidate, setComponentOrientation, setDropTarget, setEnabled, setFocusable, setFocusTraversalKeysEnabled, setForeground, setIgnoreRepaint, setLocale, setMaximumSize, setName, setPreferredSize, show, size, toString, transferFocus, transferFocusBackward, transferFocusUpCycle
public void actionPerformed(java.awt.event.ActionEvent ae)
actionPerformed
in interface java.awt.event.ActionListener
public void focusGained(java.awt.event.FocusEvent fe)
focusGained
in interface java.awt.event.FocusListener
public void focusLost(java.awt.event.FocusEvent fe)
focusLost
in interface java.awt.event.FocusListener
public void itemStateChanged(java.awt.event.ItemEvent ie)
itemStateChanged
in interface java.awt.event.ItemListener