Lecture 1a. DESCRIPTIVE AND INFERENTIAL STATISTICS

• Experimentalists use randomized experiments.
• Differentialist uses observational studies and correlations.
• Randomized experiments
  • random and representative sampling
  • independent and dependent variables
  • double-blind experiments
  • confounds
  • causality
• Causality: Randomized controlled experiments allow for strong claims about causality. You can predict, prevent bad, promote good things.
• Strong causal claims require
  • true independent variables
  • random and representative samples
  • no confounds
• Population: the entire collection of cases we want to generalize (e.g. all children in the U.S.).
• Sample: a subset of the population.
• Parameter: Numerical measurement that describes a population characteristic.
• Statistic: numerical measurement that describes a characteristic of a sample.
• Descriptive statistics: procedures to summarize, organize and simplify data.
• Inferential Statistics: technique to generalize about population parameters based on sample statistics.
• Independent variable: variable manipulated by the experimenter.
• Dependent variable: variable affected by the independent variable.

Lecture 1b. OBSERVATIONAL STUDIES.

• Study vs. experiment.
• Patterns of correlations.
• Quasi-independent variables:
• Intelligence testing
  • Intelligence models have been proposed based on correlations across different tests.
  • These are studies as no variable is being manipulated.
  • No arguments can be made about causality.
• Effects of concussion
  • Quasi-independent variables
    • Treatment
      • Suffered a sports-related concussion.
      • Control group.
• Dependent variable
  • Neural measures.
  • Cognitive measures.
• Athletes aren't randomly assigned.
• Confounds
  • Prior concussions.
  • Prior hits to the head.
  • Different personality types (aggressive).
  • Subtle personality types, characteristics (ie.they might be really fast and get into those situations more often.)
  • These could lead to the outcomes seen in the dependent variables.
• These are observational studies, not randomized experiments. Therefore, causality arguments are weak.
• Important concepts
  • Study vs. experiment.
  • Patterns of correlations.
  • Quasi-independent vs independent variables.

Lecture 2a. DESCRIPTIVE STATISTICS

• Histograms
  • Summary statistics.
  • Tools for inferential statistics.
• Histograms concepts.
  • What is distribution.
  • The normal distribution.
  • Many non-normal distributions
    • skew
    • kurtosis
• Rather than average or summarize information, show the entire distribution.
• A bias is a difference in measurements (i.e. thermometer wand measures higher than the glass thermometer).
• Histograms often revel important information.
  • Sex differences in spatial reasoning.
    • In some statistics the mean is higher in men, BUT
    • The variance in each group is far greater than the gap between groups.
    • Observing the histograms, you see the difference was in the peaks. Men had a flatter distribution. More woman scored in the mean than men.
  • The pill: The mean suggested the pill causes slightly higher blood pressure in the mean for women who took the pill.
    • The histogram showed it raised the blood pressure for all women. The whole graph was moved 4 units to the right.
    • Those histograms also showed 2 peaks. One for smokers and one for non-smokers.
• Examples
  • Uniform: flat, not a bell curve. Data was uniformly spaced.
  • Positively skewed: the positive end is flatter, or has skew.
  • Negatively skewed: the negative end if flatter, or has skew.
  • Skew is where there are few in the distribution.
• Summarization of a distribution
  • Central tendency.
  • Variability.
  • Skew.
  • Kurtosis (a peak or flat curve).
  • In mathematics these are known as the four "moments" of the mean.
• Important concepts.
  • What is a distribution.
  • The normal distribution.
  • Many non-normal distributions.
    • Skew
    • Kurtosis

Lecture 2b. SUMMARY STATISTICS

• Summary statistics.
  • Central tendency (mean, median, mode).
  • Variability (standard deviation and variance).
  • Skew.
  • Kurtosis.
• Some people don't want the entire distribution. Marketers want to know what product sells best, the mean. They want central tendency.
• Measures of central tendency.
  • Measurement of the center point of distribution. It must be representative.
  • Mean: average M = ∑(X) / N.
  • Median: middle score.
  • Mode: most common score.
• Mean is used when the distribution is normal.
• Median is preferred when there are extreme scores in the distribution.
• Mode is the score that occurs most often, the peak of a histogram.
• Variability: range and diversity of a distribution - width.
  • Standard deviation: average deviation from the mean.
  • Variance: standard deviation squared.
    • Deviations sum to zero. If you square them the negative sign drops out.
  • M = mean = average = ∑(X) / N.
  • X - M = deviation.
  • ∑(X - M)² = sum of deviations² = sum of squares = SS.
  • ∑(X - M)² / N = variance = mean squared = MS.
  • √(MS) = standard deviation = SD.
• SUMMARY STATISTICS
  • Central tendency (mean, medium, mode).
  • Variability (standard deviation, variance).
  • Skew.
  • Kurtosis.
  • SD² = ∑( X - M )² / N.

Lecture 2c. TOOLS FOR INFERENTIAL STATISTICS

• Tools for inferential statistics:
  • Normal distribution.
  • Z-scores.
  • Percentile rank.
  • Probability.
  • Inferential statistics.
• Normal distribution
  • Bell shaped and symmetrical around the center point.
  • The thermometer wand measured hot. That is a bias.
• Z-scores
  • Z = ( X - M ) / SD.
  • Z of the mean = 0.
  • Z of the standard deviation = 1.
• Percentile rank: percentage of scores that fall at or below a given score.
  • Percentile rank of the mean = 50%
  • i.e. Z of +1, if 34% of the scores are between the mean and a Z-score of + 1, 50% + 34% = 84%.
• You should be able to switch between
  • Raw scored
  • Z-scored
  • Percentile rank
  with relative ease.
• Probability (of an event)
  • P(E) = (# of ways E can be attained) / (# of possible outcomes).
• Probability and normal distribution.
• Inferential statistics
  • Assume a normal distribution.
    • Assume certain values, such as a mean.
    • Conduct an experiment.
    • Do the assumption hold. Either way, an inference can be made.
  • Is is safe to assume a normal distribution.
    • What are you trying to measure.
    • What is the construct.
    • How do you operationalize the construct.
• Tools review
  • Normal distribution.
  • Z-scores.
  • Percentile ranks.
  • Probability.
  • Inferential statistics.

Lecture 3a. INTRODUCTION TO R

• Why R
  • It's free.
  • Works on any platform.
  • Open-source.
  • Flexible.
  • Excellent graphics.
  • Widely used in academia and business.
• To use R in linux (Debian/Ubuntu/Kubuntu)
  • sudo apt-get install r-base r-cran-boot r-cran-class r-cran-cluster r-cran-vgam
  • Create scripts in a text editor, save as .R files.
  • # is a comment.
  • From the command prompt type "R {Enter}".
  • Test: type 5+3 {Enter}
  • The directory you are in is the working directory.
  • The professor wants us to use the levene.test() in install.packages("car"). This is linux. That one only returns the median. The professor asks for the mean. To get the mean, use install.packages("lawstat").
    • Before you can do that you have to "sudo apt-get install r-cran-vgam".
  • install.packages("psych")
  • library(psych)
  • search()
  • list.files()
  • source( "file.name.R", echo=T )
  • help( function.name )
  • quit()
• For Windows: Download and install R
  • http://www.r-project.organize
  • Click on the CRAN link.
  • Choose a country and city.
  • Select your operating system.
• From the R console types
  • >install.packages("psych")
    >library(psych) # load the package
         # denotes a comment.
    >search() # lists loaded packages.
• "File" menu
  • "New Document".
  • R editor appears.
  • Write a line of code.
  • Press return or enter.
  • Result is returned.
• For example, use R as a calculator:
  • >5+6 {Enter}
    >11
• R editor
  • Write several lines of code.
  • Save as a "script".
  • Execute script.
  • Results are returned in the R console.
• Set you preferences
  • Establish a "working directory"
    • Click on R.
    • Click on "Preferences".
    • Click on "Startup".
    • Click on "Change" directory.
    • Choose a "Path" ( i.e ~/Common/Language/R )

Lecture 3b. WRITING A SCRIPT

• Script
  • Several lines of code.
  • Composed in the R editor.
  • Comment and save for later.
• Goals for this script
  • Read data into a data.frame.
  • Explore the contents of the data.frame.
  • Plot histograms.
  • Get descriptive statistics.
• On the course web sight, get and use the text file
  • stats1.ex.01.txt
• First line(s) of code should be comments.
  • # Statistics One, Lecture 3, example script.
  • # Read data, plot histograms, get descriptives.
• Read data into a data.frame.
  • We will name the data.frame "ratings".
    • ratings <- read.table("stats1.ex.01.txt", header = T)
• Explore the contents of the data.frame.
  • class(ratings)
    • R will return "data.frame"
  • name(ratings)
    • R will return "RedTruck" "WoopWoop" "HobNob" "RedTruck"
• Plot histograms
  • hist(RedTruck)
    • R returns a histogram for RedTruck.
• Plot four histograms on one page.
  • layout( matrix( c( 1,2,3,4 ), 2, 2, byrow = TRUE ) )
  • hist( ratings$WoopWoop, xlab = "Rating" )
  • hist( ratings$RedTruck, xlab = "Rating" )
  • hist( ratings$HobNob, xlab = "Rating" )
  • hist( ratings$FourPlay, xlab = "Rating" )
    • R returns 4 histograms on one page.
• Explain code
  • layout( matrix( c
    • c = concatenate.
  • c( 1,2,3,4 ), 2, 2
    • histograms for items 1 - 4, in a 2 x 2 matrix, by row.
  • xlab: x axis label = Rating.
    • help(hist) gives all the options available for hist.
  • $ pick out one variable in ratings data.frame.
• Get descriptive statistics
  • describe( ratings )
• Functions used"
  • read.table
  • class
  • names
  • hist
  • describe
• Learn more about functions
  • In R types
    • >help(hist)
    • >help(read.table)
  • Or google "how to do something in R".
• Final Product
  • Script
  • Histograms
  • Descriptive statistics

COMPLETE SAMPLE SCRIPT

# Statistics One, Lecture 3, example script
# Read data, plot histograms, get descriptives
install.packages("psych")
library( psych )
#Verify
search()

# Read the data into a data.frame called ratings
ratings <- read.table("~/Common/Language/R/stats1.ex.01.txt", header = T)

# What type of object is ratings?
class( ratings )
# List the names of the variables in the data.frame called ratings
names( ratings )

# Print 4 histograms on one page
layout( matrix( c( 1,2,3,4 ), 2, 2, byrow = TRUE ) )

# Plot histograms
hist( ratings$WoopWoop, xlab = "rating" )
hist( ratings$RedTruck, xlab = "rating" )
hist( ratings$HobNob, xlab = "rating" )
hist( ratings$FourPlay, xlab = "rating" )

# Descriptive statistics for the variables in the data.frame called ratings
describe( ratings )

• To run a file ending in .R
  • list.files()
    • "lec3b.example.R"
  • source( "filename.R", echo = T )

ASSIGNMENT ONE CODE

# Statistics One, Week One, Assignment 1
# Designed sports training on working memory study
# Read data, plot histograms, get descriptives

library(psych)

# Read the data into a data.frame called training
training <- read.table("~/Common/Language/R/daa.01.txt", header = T)

# Break training data into two subsets
training_des <- subset( training, subset=( cond=="des" ) )
training_aer <- subset( training, subset=( cond=="aer" ) )

# What type of object is training?
class( training )
# What type of object is training_des, training_aer?
class( training_des )
class( training_aer )
# List the names of the variables in the data.frame called training
names( training )
names( training_des )
names( training_aer )

# Print 8 histograms on one page
layout( matrix( c(1,2,3,4,5,6,7,8), 2, 4, byrow = TRUE ) )

hist( training_des$pre.wm.s )
hist( training_des$post.wm.s )
hist( training_des$pre.wm.v )
hist( training_des$post.wm.v )
hist( training_aer$pre.wm.s )
hist( training_aer$post.wm.s )
hist( training_aer$pre.wm.v )
hist( training_aer$post.wm.v )

# Descriptive statistics for the variables in the data.frame called training
describe( training_des )
describe( training_aer )

ASSIGNMENT ONE TEST

QUIZ ONE

Lecture 4a: CORRELATIONS

• Correlation
  • A statistical procedure used to measure and describe the relationships between two variables.
  • Correlations can range between +1 and -1.
    • +1 is a perfect positive correlation.
    • -1 is a perfect negative correlations.
    • 0 is no relationship between two variables at all.
• Example: working memory capacity (X) is strongly correlated with SAT score (Y). If I know a persons score on a working memory task then I can predict their SAT score.
• R scatterplot: plot( SAT ~ WMC )
  • axis Y ~ X
• CAUTION: Accuracy of predictions will depend upon:
  • The magnitude of the correlation.
  • The magnitude of the correlation depends upon:
    • Reliability of X.
    • Reliability of Y.
    • Sampling (random and representative).
  • The validity of the prediction depends upon:
    • Validity of X.
    • Validity of Y.
    • Several other assumptions (in Segment 3).
  • Most important:
    • The correlation coefficient is a sample statistic. It does not apply to individual cases in the sample.
• A more serious example: Intelligence testing and WWI.
• The advent of WWI caused a unique challenge for the U.S. military.
  • How to quickly recruit and assign to positions a large number of new men.
  • Specifically, which men should be designated as officers and/or assigned to officer training.
• Historical coincidence
  • Intelligence testing was also new. It used a scientific approach to individual differences in intelligence.
• The Army Alpha Battery
  • Prominent Psychology researchers were recruited by the military to develop an aptitude test that could be administered to large groups of men and quickly scored.
  • One of them, Robert Yerkes, argued that a specific test measured "native intellectual ability" and was unaffected by culture.
  • However, later research demonstrated that it was clearly culturally biased.
  • What type of statistical information could be presented to support Yerkes' claim?
  • What type of statistical information could be presented to refute Yerkes' claim?
  • The decisive answer was that the test held for one culture group, but not others.
• Baseball and Sabermetrics
  • Which statistic is a better predictor of a player's contribution to the team's offense, AVG or OBP.
  • In data collected from the New York Yankees:
    • R scatterplot: plot( R ~ AVG )
      • r = +.40
    • R scatterplot: plot( R ~ OBP )
      • r = +.72
• Important Concepts
  • What is a correlation?
  • What are they used for?
  • Scatterplots.

Lecture 4b: CALCULATING CORRELATIONS

• Important topics:
  • Pearson product moment correlation coefficient (r).
  • Covariance
• Correlation coefficient (r)
  • r = the degree to which X and Y vary together, relative to the degree to which X and Y vary independently.

  • r =	covariance of X and Y
    	variance of X and Y

• Formulas for r:
  • Raw score formula.
  • Z-score formula.
• Sum of cross products.
  • Review calculation for SS (sum of squares).
    • SS_x = ∑[ ( X - M_x )² ]
  • To calculate SP (sum of cross products )
    • For each subject in the sample, calculate their deviation scores on both the X and Y.
      • ( X - M_x )
      • ( Y - M_y )
    • For each subject multiply the deviation score on X by the deviation score on Y:
      • ( X - M_x ) ( Y - M_y )
    • Then sum the "cross products"
      • SP = ∑[ ( X - M_x ) ( Y - M_y ) ]
• Formula to calculate r:
  • Raw score formula:

    • r =	SPxy
      	√ (SSx SSy)

    • SS_x = ∑( X - M_x )²
    • SS_y = ∑[ ( Y - M_y ) ( Y - M_y ) ]
    • SP_xy = ∑[ ( X - M_x ) ( Y - M_y ) ]

    • r =	∑[ ( X - Mx ) ( Y - My ) ]
      	[ ∑( X - Mx )2 ∑( Y - My )2 ]½

  • Z-score formula:

    • r =	∑( Zx Zy )
      	N

    • Zx =	( X - Mx )
      	SDx

    • Zy =	( Y - My )
      	SDy

    • SDx =	√∑(X - Mx)²
      	N

    • SDy =	√∑(X - My)²
      	N

• Variance and covariance:
  • Variance = MS (mean²) = SS / N
  • Covariance = COV = SP / N
• Correlation is standardized COV
  • Standardized so the value is in the range -1 to 1.
• Note on the denominators.
  • Correlation for descriptive purposes:
    • Divide by N.
  • Correlation for inferential purposes:
    • Divide by N - 1.

Lecture 4c: INTERPRETATION OF CORRELATIONS

• Important Topics
  • Validity of a correlation-based argument.
  • Reliability of a correlation.
• The validity of any argument made on the basis of a correlation analysis depends on these assumptions:
  • Normal distribution for X and Y.
    • To detect for violations:
      • Plot histograms and run descriptive statistics.
  • Linear relationship between X and Y (not quadratic):
    • To detect violations:
      • Examine scatter plots.
      • Plot a histogram of residuals.
  • Homoskedasticity
    • How to detect violations:
      • Examine scatter plots.
      • Plot a histogram of residuals.
• Homoskedasticity
  • The standard deviation is consistent across the regression line, homogeneity of variance.
  • In a scatter plot the distance between a dot and the regression line reflects the amount of prediction error, or the residual.
    • If it is homoskedastic, those distances are not a function of X, not related to the predictive variable. If they are you might have a confound in you study.
    • In a scatter plot, if the data points are scattered randomly in relation to the regression line it is homoskedastic. If the variance changes with X, they are heteroskedastic.
• Reliability of a correlation:
  • Does the correlation reflect more than just a chance of covariance?
  • If the correlation is iffy, how do you determine across samples if it's valid.
  • One approach to this question is to use NHST.
• If there is not a correlation, what are the odds that you would get these results.
• NHST - Null hypothesis significance testing.
  In R's cor.test(), this is the "p-value" of the "cor"..
  • H₀ = null hypothesis: r = 0.
  • H_A = alternative hypothesis: r > 0.
• Assume H₀ is true, then calculate the probability of observing data with these characteristics, given H₀ is true.
  • Thus, p = P(D|H₀) (| = given)
  • If p < α then reject H₀, else retain H₀

• Truth		Retain H0	Reject H0
  	H0 true	Correct decision	Type I error (false alarm)
  	H0 false	Type II error (miss)	Correct decision

• Truth		Retain H0	Reject H0
  	H0 true	p = ( 1 - α )	p = α
  	H0 false	p = β ( 1 - POWER )	p = ( 1 - β ) POWER

• p = P( D | H₀ )
  • Given that the null hypothesis is true, the probability of true, or more extreme data, is p.
    • NOT: The probability of the null hypothesis.
    • In other words, P( D | H₀ ) <> P( H₀ | D )
• NHST can be applied to:
  • r: Is the correlation significantly different from zero?
  • r1 vs. r2: Is one correlation significantly larger than another?
• There are other correlation coefficients.
  • Point biserial r:
    • When one variable is continuous and the other is dichotomous.
  • Phi coefficient:
    • When both variables are dichotomous.
  • Spearman rank correlation:
    • When both variables are ordinal (ranked data).
• Important topics:
  • Validity of a correlation-based argument.
  • Reliability of a correlation.

Lecture 5a: MEASUREMENT

• Reliability and Validity
  • Classical test theory (true score theory):
    • Raw score (X) are not perfect.
    • They are influenced by bias and chance error.
    • In a perfect world, we would obtain a "true score".
      • X = true score + bias + chance error
    • A measure (X) is considered to be reliable as it approaches the true score.
      • The problem is we don't know the true score.
      • So we estimate reliability.
  • Reliability estimates and methods of testing.
    • Test / re-test
      • Measure everyone twice.
      • The correlation between them is an estimate of reliability.
      • If a bias is uniform, we won't detect it.
    • Parallel tests:
      • In the body temperature example, besides the wand, also use an oral thermometer.
      • The correlation between the two tests is still an estimate of reliability but now the bias will be revealed.
    • Inter-item estimates:
      • The most commonly used method in social sciences because it's cheapest.
      • For example, suppose a 20-item survey is designed to measure extraversion.
        • Randomly select 10 items to get sub-set A (X1).
        • The other 10 items become sub-set B (X2).
        • The correlation between them is an estimate of reliability.
  • Construct validity:
    • What is a construct?
      • An ideal "object" that is not directly observable.
        • As opposed to "real" observable objects.
        • For example, "intelligence" is a construct.
    • How do we operationalize a construct?
      • The process of defining a construct to make it observable and quantifiable (i.e. intelligence tests).
    • Construct validity.
      • Content validity.
      • Convergent validity.
      • Divergent validity.
      • Nomological validity.
    • Example construct: verbal ability in children.
      • Operationalize it with a vocabulary test.
        • Content validity: does the test consist of words that children should know?
        • Convergent validity: Does the test correlate with other, established measures of verbal ability? For example, reading comprehension.
        • Divergent validity: does the test correlate less will with measures designed to test a different type of ability? For example, spatial reasoning.
        • Nomological validity: are scores on the test consistent with more general theories, for example, of child development and neuroscience? For example, children with damage or disease to brain regions associated with language development should score lower on the test.

Lecture 5b: SAMPLING

• Sampling error:
  • From the wine example, suppose there are only 300 certified "wine experts" in the entire world.
    • The population N = 300.
    • Using Red Truck, let's assume a normal distribution in the population with a mean of 5.5 and standard deviation of 2.22. The histograms looks different also.
    • Now take a random sample, N = 30, with a mean of 5.93 and standard deviation of 2.45.
    • This is sampling error.
    • Increase the sample to N = 100. Now we get a mean of 5.47 and a standard deviation of 2.19. The histogram look right. This is much closer to the population.
    • Go the other way, a sample of N = 10. Now we bet a mean of 6.00 and standard deviation of 1.7. The histogram doesn't look at all like the population one.
    • Sampling error is the difference between the population and the sample.
    • Typically, we don't know the population parameter. So how do we estimate sampling error?
    • Clearly, sampling error depends on the size of the sample, relative to the population
    • It also depends on the variance in the population. A large variance will lead to greater sampling error.
• Standard error:
  • The estimate of amount of sampling error:

    • SE =	SD
      	SQRT( N )

    • SE: Standard error.
    • SD: standard deviation of the sample.
    • N: Size of the sample.
• Probability histograms:
  • Standard error is the standard deviation of the probability histogram.
  • If a variable X is perfectly normal, we know a lot about it's distribution. We know what scores fall in what regions.
    • It is symmetrical, so 50% of the distribution falls below the mean, 50% above.
    • 68% of the distribution falls within 1 standard deviation of the mean.
  • Previous histograms where plots of individuals. Now we have to consider a distribution of samples.
  • Probability histogram:
    • A distribution of sample means:
    • Assume we took multiple samples of the same size and them plotted all the sample means.
      • N = 10
      • N = 30
      • N = 100
    • Suppose we sampled N = 10, multiple times.
    • This histogram would have a normal curve but the width would be large. Each of the individual samples would contribute varying standard deviations.
    • Samples of N = 100 would have less fluctuation.
  • Standard Error is the distance of 1 standard deviation (Z of 1), in the distribution of sample means.
• Distribution of sample means.
  • The characteristics of this sample means are hypothetical. We don't know the dimensions of the distribution as we do with individual score. We must estimate.
  • We will assume these characteristics:
    • The mean of the distribution of sample means = mean of the population.
    • The variance of the distribution of sample means is less than the variance in the population of individuals

      • σ2mean =	σ2
        	N

      • the variance in the distribution of the sample means = the variance in the population ÷ by the size of the sample.
    • This formula is familiar. So remember, these all have different meanings.
      • σ² (sigma²): population variance.
      • SD²: variance in the sample.
      • SE (standard error): standard deviation of the distribution of samples.
    • The shape of the distribution of sample means is approximately normal (if the samples are large enough).
    • Distribution of sample means:

      σ2M =	σ2
      	N

      • σ²_M is the variance of the distribution of sample means.
      • σ_M is the standard deviation of the distribution of sample means (standard error)
      • σ² is the variance of the population.
      • σ is the standard deviation of the population.
      • N is the sample size.
• Central limit theorem:
  • The mean of the distribution of sample means is the same as the mean of the population.
  • The standard deviation of the distribution of sample means is the square root of the variance of the distribution of sample means, which is σ²_M = σ² / N.
    • Which means SE = SD / √N.
    • (standard error) = (standard deviation) / (square root of the size of the sample).
  • The shape of the distribution of sample means is approximately normal if either (a) N >= 30 or (b) the shape of the population distribution is normal.

Lecture 6a: CORRELATIONS IN R

• Goal of this script in R
  • Histograms
  • Descriptive statistics
  • Scatterplots
  • Correlations
• Data from ImPACT.com
  • A computerized neuropsychological assessment of memory and attention.
  • Used to assess the cognitive effects of head trauma, for example, sports-related concussion.
  • Impact main measures
    • Verbal memory.
    • Visual memory.
    • Visual motor speed.
    • Reaction time.
    • Impulse control.
  • Data available in "stats1.ex.02.txt"
• Write a script

# Statistics One, Lecture 6a, example script
# Read data, plot histograms, get descriptives,
# examine scatter plots, run correlations.
library(psych)

# Read data into a data.frame called "impact"
impact <- read.table("stats1.ex.02.txt", header=T)

# Explore data.frame
class(impact)
names(impact)

# Change default settings for graphics
par( cex = 2, lwd = 2, col.axis = 200, col.lab = 200, col.main = 200, col.sub = 200, fg = 200 )

# Print 5 histograms on one page
layout(matrix(c(1,2,3,4,5,6),2,3, byrow=TRUE))
hist(impact$memory.visual, xlab="Visual memory", main="Histogram", col="red")
hist(impact$memory.verbal, xlab="Verbal memory", main="Histogram", col="green")
hist(impact$speed.vismotor, xlab="Visual-motor speed", main="Histogram", col="blue")
hist(impact$speed.general, xlab="General speed", main="Histogram", col="black")
hist(impact$impulse.control, xlab="Impulse control", main="Histogram", col="purple")

describe(impact)

# Scatterplots
plot(impact$memory.verbal ~ impact$memory.visual, main="Scatterplot",ylab="Verbal memory", xlab="Visual Memory")
abline(lm(impact$memory.verbal ~ impact$memory.visual), col="blue")

# Correlations (one pair at a times)
cor(impact$memory.verbal, impact$memory.visual)
# Again, only add test for significance
cor.test(impact$memory.verbal, impact$memory.visual)
# Correlations (all in a matrix)
cor(impact)

# Install and load new package
install.packages("gclus")
library(gclus)
# Verify
search()

# Scatterplot matrix
library(gclus)
impact.r = abs(cor(impact))
impact.col = dmat.color(impact.r)
impact.o <- order.single(impact.r)
cpairs(impact, impact.o, panel.colors=impact.col, gap=.5, main="Variables Ordered and Colored by Correlation")

Lecture 6b: TEST / RE-TEST RELIABILITY ANALYSIS IN R.

• Assume 40 athletes took the ImPACT test twice. Tests A and B, taken a couple months apart.
• These are normal athletes, not involved in any head injury. See if the scores between the two tests correlate.
• How to structure repeated measures?
  • Typical to add new columns.
  • In R, sometimes it is better to add new rows.
  We will demonstrate both.

# Statistics One, Lecture 6b
# Test / re-test reliability analysis, column format
library( psych )

# Read data into a data.frame called impact.col
impact.col <- read.table( "stats1.ex.03.col.txt", header = T )

# List name of the variables in the data.frame called impact.col
names( impact.col )
search()
describe( impact.col )

# Correlation ( A & B )
cor( impact.col$memory.verbal.A, impact.col$memory.verbal.B )
cor( impact.col$memory.visual.A, impact.col$memory.visual.B )
cor( impact.col$speed.vismotor.A, impact.col$speed.vismotor.B )
cor( impact.col$speed.general.A, impact.col$speed.general.B )
cor( impact.col$impulse.control.A, impact.col$impulse.control.B )

# Statistics One, Lecture 6b, example script
# Test / re-test reliability analysis, row format
library( psych )

# Read data into a data.frame called impact.row
impact.row <- read.table( "stats1.ex.03.row.txt", header = T )

# List name of the variables in the data.frame called impact.row
names( impact.row )
search()

describeBy( impact.row, impact.row$test )

# Correlation ( A & B )
cor( impact.row$memory.verbal[ impact.row$test=="A" ], impact.row$memory.verbal[ impact.row$test=="B" ] )
cor( impact.row$memory.visual[ impact.row$test=="A" ], impact.row$memory.visual[ impact.row$test=="B" ] )
cor( impact.row$speed.vismotor[ impact.row$test=="A" ], impact.row$speed.vismotor[ impact.row$test=="B" ] )
cor( impact.row$speed.general[ impact.row$test=="A" ], impact.row$speed.general[ impact.row$test=="B" ] )
cor( impact.row$impulse.control[ impact.row$test=="A" ], impact.row$impulse.control[ impact.row$test=="B" ] )

QUIZ TWO

ASSIGNMENT TWO CODE

# Statistics One, Week Two Assignment
# Write R script to:
# Provide descriptive statistics for all 8 measures, for each condition.
# Provide an 8x8 correlation matrix for each condition.
library(psych)

# Read the data into a data.frame called assing02
assing02 <- read.table("~/Common/Language/R/daa.02.txt", header = T, row.names = "pid" )
# row.names = "pid", removed the first column from the data.frame.

# Break assing02 data into two subsets
assing02_des <- subset( assing02, subset=( cond=="des" ) )
assing02_aer <- subset( assing02, subset=( cond=="aer" ) )

# This is not necessary, but now break into 4 subsets
# "subset" works on rows. "select" works on columns.
assing02_des_tst1 <- subset( assing02_des, select = c( pre.wm.s1, post.wm.s1, pre.wm.v1, post.wm.v1 ) )
assing02_des_tst2 <- subset( assing02_des, select = c( pre.wm.s2, post.wm.s2, pre.wm.v2, post.wm.v2 ) )
assing02_aer_tst1 <- subset( assing02_aer, select = c( pre.wm.s1, post.wm.s1, pre.wm.v1, post.wm.v1 ) )
assing02_aer_tst2 <- subset( assing02_aer, select = c( pre.wm.s2, post.wm.s2, pre.wm.v2, post.wm.v2 ) )

# What type of object is assing02?
class( assing02 )
# What type of objects are assing02_des, assing02_aer?
class( assing02_des )
class( assing02_des_tst1 )
# List the names of the variables in the data.frame called assing02
names( assing02 )
names( assing02_des )
names( assing02_des_tst1 )

# Descriptive statistics for the variables in the data.frame called training
describe( assing02_des_tst1 )
describe( assing02_des_tst2 )
describe( assing02_aer_tst1 )
describe( assing02_aer_tst2 )

# This is the same thing as the 4 lines of code directly above.
describe( assing02_des [2:9] )
describe( assing02_aer [2:9] )

cor( assing02_des 2:9 ) cor( assing02_aer 2:9 )

ASSIGNMENT TWO TEST

Lecture 7a: INTRODUCTION TO REGRESSION

• Regression equation and "model".
  • A statistical analysis used to predict scores on an outcome variable, based on scores on one or more predictor variables.
  • For example, we can predict how many runs a baseball will score (Y) if we know the players batting average (X).
  • Y = m + bX + e
    • Y is a linear function of X.
    • m = intercept.
    • b = slope.
    • e = prediction error.
  • Ŷ = B₀ + B₁X₁
    • Ŷ = predicted score.
    • Y - Ŷ = e (prediction error).
  • The regression model is used to "model" or predict future behavior.
    • The "model" is just an equation.
  • The goal is to produce better models so we can generate more accurate predictions.
    • Add more predictor variables.
    • Develop better predictor variables.
      • In the baseball example, OBP predictions were an improvement over AVG.
  • Examine why the results got better.
    • Examine residuals.
    • Plot histogram.
    • Scatterplot residuals.
      • Linear relationship between X and Y.
      • Homoscedasticity.
• Ordinary least squares estimation.
  • The values of the coefficients (B) are estimated such that the model yields optimal predictions.
    • Minimize the residuals!
    • The sum of the squared (SS) residuals is minimized.
    • SS.RESIDUAL = ∑( Ý - Y )².
    • ORDINARY LEAST SQUARES estimation.
  • Sum of Squared deviation scores (SS) in variable Y
    • SS.Y
  • Sum of Cross Products (SP.XY)
    • SS.X
    • SS.Y
    • Also called correlation or covariance.
  • Sum of Squared deviation scores (SS) in variable Y.
    • SS.Y = SS.MODEL + SS.RESIDUAL
• Unstandardized regression coefficients.
  • How to calculate B (unstandardized)
    • B = r ( SD_y / SD_x )
• Standardized regression.
  • Standardized regression coefficient = β + r
  • If X and Y are standardized then: (one predictor)
    • SD_y = SD_x = 1
    • B = r ( SD_y / SD_x )
    • β = r
  • This is only true in simple regression. Multiple regression gets worse.
• Estimation of coefficients:
  • Using the linear model function in R:
    lm( Runs ~ OBP )
  • Ŷ = B₀ + B₁X₁
  • Ŷ = -282 + (1044)X
  • Let X = 0.35
  • Ŷ = 83

Lecture 7b: NHST: A closer look (null hypothesis significance testing)

• Logic of NHST
  • First, this probability is the p-value returned by R.
  • H₀ = null hypothesis: e.g., r = 0
    • H_A = alternative hypothesis: e.g., r != 0
  • From the regression standpoint, the unstandardized regression coefficient, B in the regression equation.
    • The slope relating X to Y is zero.
    • H₀ = null hypothesis: e.g., B = 0.
    • H_A = alternative hypothesis: e.g., B != 0.
  • Assume H₀ is true, then calculate the probability of observing data with these characteristics, given the H₀ is true:
    • Thus, p = P(D|H₀)
      • (p is a conditional probability, the probability of obtaining this outcome (these data) given the assumption that the null hypothesis is true.)
    • If p < α, then Reject H₀, else Retain H₀.

    • Truth		Retain H0	Reject H0
      	H0 true	Correct decision	Type I error (false alarm)
      	H0 false	Type II error (miss)	Correct decision

    • Truth		Retain H0	Reject H0
      	H0 true	p = ( 1 - α )	p = α
      	H0 false	p = β ( 1 - POWER )	p = ( 1 - β ) POWER

  • p = P( D | H₀ )
    • Given that the null hypothesis is true, the probability of true, or more extreme data, is p.
      • NOT: The probability of the null hypothesis.
      • In other words, P( D | H₀ ) <> P( H₀ | D )
  • NHST can be applied to:
    • r -- Is the correlation significantly different from zero?
    • B -- Is the slope of the regression line For X significantly different from zero?
  • NHST for B
    • t = B / SE
      • B is the unstandardized regression coefficient.
      • SE = standard errors
      • SE = √ SS.RESIDUAL / (N - 2)
      • It's good if t is close to zero.
• NHST problems
  • Biased by N:
    • p-value is based on t-value
    • t = B / SE
    • SE = √ SS.RESIDUAL / ( N - 2 )
  • Binary outcome:
    • Technically speaking, one must Reject or Retain the Null Hypothesis.
    • What if p = .06?
  • Null "model" is a weak hypothesis
    • Demonstrating that your model does better than NOTHING is not very impressive.
• NHST alternatives
  • Effect size.
    • Correlation coefficient (r).
    • Standardized regression coefficient (B).
    • Model R².
  • Confidence intervals.
    • Sample statistics are "point estimates" Specific to the sample. They will vary as a function of sampling error.
    • Instead report "interval estimates". The width of interval is a function of standard error.
  • Model comparison.
    • Propose multiple models, Model A, Model B.
    • Compare Model R².

Lecture 8a: MULTIPLE REGRESSION

• Intro to multiple regression
  • Multiple regression equation
    • Ŷ = B₀ + B₁X₁ + B₂X₂ + B₃X₃ ... + B_kX_k
    • Ŷ = ∑(B₀ + B_kX_k)
      • Ŷ = predicted value on the outcome variable Y.
      • B₀ = predicted value on Y when all X = 0.
      • X_k = predictor variables.
      • B_k = unstandardized regression coefficients.
      • Y - Ŷ = residual (prediction error).
      • k = the number of predictor variables.
  • Interpretation of regression coefficients
    • Model R and R²
      • R = multiple correlation coefficient.
        • R = r_ŶY.
        • The correlation between the predicted scores and the observed scores.
      • R²
        • The percentage of variance in Y explained by the model.
  • Multiple regression Example
    • Outcome measure (Y).
      • Faculty salary (Y).
    • Predictors (X1, X2, X3)
      • Time since PhD (X1).
      • # of publications (X2)..
      • Gender (X3).
        • Male = 0.
        • Female = 1.
    • Ŷ = 46, 911 + 1,382(time) + 502(pubs) + -3,484(G)
• Standard vs. sequential regression
  • The difference between these approaches is how they handle the correlations among predictor variables.
    • if X1, X2, and X3 are not correlated then type of regression analysis doesn't matter.
    • If predictors are correlated then different methods will return different results.
  • STANDARD
    • All predictors are entered into the regression equation at the same time.
    • Each predictor is evaluated in terms of what it adds to the prediction of Y that is different from the predictability offered by the others.
    • Overlapping areas are assigned to R² but not to any individual B.
  • SEQUENTIAL (aka hierarchical)
    • Predictors are entered into the regression equation in ordered steps; the order is specified by the researcher.
    • Each predictor is assessed in terms of what it adds to the equation at its point of entry.
    • Often useful to assess the change in R² from one step to another.

Lecture 8b: MATRIX ALGEBRA

• A matrix is a rectangular table of known or unknown numbers, e.g.

  • M =	⌈ 1 2 ⌉ | 5 1 | | 3 4 | ⌊ 4 2 ⌋

  • The size, or order, is given by identifying the number of rows and columns, e.g. the order of matrix M is 4x2
  • The transpose of a matrix is formed by rewriting its rows as columns.

  • MT =	⌈ 1 5 3 4 ⌉ ⌊ 2 1 4 2 ⌋

• Two matrices may be added or subtracted only if they are of the same order.

  • M + N =

    ⌈ 1 2 ⌉ | 5 1 | | 3 4 | ⌊ 4 2 ⌋

    +

    ⌈ 2 3 ⌉ | 4 5 | | 1 2 | ⌊ 3 1 ⌋

    =

    ⌈ 3 5 ⌉ | 9 6 | | 4 6 | ⌊ 7 3 ⌋

• Two matrices may be multiplied when the number of columns in the first matrix is equal to the number of rows in the second matrix. If s, then we say they are conformable for matrix multiplication.
  • R = M^T * N      R_ij = ∑ (M^T_ik * N_kj)

  • R = MT * N =

    ⌈ 1 5 3 4 ⌉ ⌊ 2 1 4 2 ⌋

    *

    ⌈ 2 3 ⌉ | 4 5 | | 1 2 | ⌊ 3 1 ⌋

    =

    ⌈ 37 38 ⌉ ⌊ 18 21 ⌋

  • 1*2 + 5*4 + 3*1 + 4*3 = 37
  • 1*3 + 5*5 + 3*2 + 4*1 = 38
  • ∑( R₂ * C₁ ) = 18
  • ∑( R₂ * C₂ ) = 21
• Special type of matrices
  • A square matrix has the same number of rows as columns.

    • D =	⌈ 17 14 5 ⌉ | 13 25 7 | ⌊ 18 32 9 ⌋

  • A square symmetric matrix is such that D = D^T, the entries above the diagonal are symmetric with the entries below the the diagonal (i.e. correlation matrix):

    • D =	⌈ 17 13 18 ⌉ | 13 25 32 | ⌊ 18 32   9 ⌋

  • Diagonal matrices are square matrices with zeros in all off-diagonal cells:

    • D =	⌈ 17  0  0 ⌉ |  0 25  0 | ⌊  0   0  9 ⌋

  • The inverse of a matrix is similar to the reciprocal of a scalar. e.g., the inverse of 2 is ½ and their product = 1.
    • Inverses only exist for square matrices and not necessarily for all square matrices.
  • The identity matrix is an inverse such that D * D^-1 = I

    • I =	⌈ 1 0 0 ⌉ | 0 1 0 | ⌊ 0 0 1 ⌋

  • The determinant of a matrix is a scalar derived from operations on a square matrix, by taking the product of the diagonals relative to the off-diagonals. For example, for a 2x2 matrix A the determinant is denoted as |A| and is obtained as follows:
    • |A| = a₁₁ * a₂₂ - a₁₂ * a₂₁
  • A vector is a matrix with only one row or one column.
• EXAMPLE: A data matrix to a correlation matrix in 10 steps.
  • Raw data matrix
    Subjects as rows, variables as columns. n = 10 (rows), p = 3 (columns).

    Xnp =

    ⌈ 3 2 3 ⌉ | 3 2 3 | | 2 4 4 | | 4 3 4 | | 4 4 3 | | 5 4 3 | | 2 5 4 | | 3 3 2 | | 5 3 4 | ⌊ 3 5 4 ⌋

  • Row vector of sums (totals)

    T1p =

    11p * Xnp =

    [ 1 1 1 1 1 1 1 1 1 1 ] *

    ⌈ 3 2 3 ⌉ | 3 2 3 | | 2 4 4 | | 4 3 4 | | 4 4 3 | | 5 4 3 | | 2 5 4 | | 3 3 2 | | 5 3 4 | ⌊ 3 5 4 ⌋

    = [ 34 35 34 ]

  • Row vector of means
    M_1p = T_1p * N^-1 = [ 34 35 34 ] * 10^-1 = [ 3.4 3.5 3.4 ]
  • Matrix of means

    Mnp =

    1n1 * M1p =

    ⌈ 1 ⌉ | 1 | | 1 | | 1 | | 1 | | 1 | | 1 | | 1 | | 1 | ⌊ 1 ⌋

    * [ 3.4 3.5 3.4 ] =

    ⌈ 3.4 3.5 3.4 ⌉ | 3.4 3.5 3.4 | | 3.4 3.5 3.4 | | 3.4 3.5 3.4 | | 3.4 3.5 3.4 | | 3.4 3.5 3.4 | | 3.4 3.5 3.4 | | 3.4 3.5 3.4 | | 3.4 3.5 3.4 | ⌊ 3.4 3.5 3.4 ⌋47

  • Matrix of deviation scores

    Dnp =

    Xnp - Mnp =

    ⌈ 3 2 3 ⌉ | 3 2 3 | | 2 4 4 | | 4 3 4 | | 4 4 3 | | 5 4 3 | | 2 5 4 | | 3 3 2 | | 5 3 4 | ⌊ 3 5 4 ⌋

    -

    ⌈ 3.4 3.5 3.4 | 3.4 3.5 3.4 | 3.4 3.5 3.4 | 3.4 3.5 3.4 | 3.4 3.5 3.4 | 3.4 3.5 3.4 | 3.4 3.5 3.4 | 3.4 3.5 3.4 | 3.4 3.5 3.4 ⌊ 3.4 3.5 3.4

    ⌉ | | | | | | | | ⌋

    =

    ⌈  -.4 -1.5   -.4 |  -.4 -1.5   -.4 | -1.4   .5    .6 |   .6  -.5    .6 |   .6    .5   -.4 |  1.6   .5   -.4 | -1.4  1.5    .6 |  -.4  -.5  -1.4 |  1.6  -.5    .6 ⌊  -.4  1.5    .6

    ⌉ | | | | | | | | ⌋

  • Sums of squares and Cross-products matrix

    Sxx = Dpn * Dnp =

    ⌈ | ⌊

    -.4 -1.5 -.4

    -.4 -1.5 -.4

    -1.4 .5 .6

    .6 -.5 .6

    .6 .5 -.4

    1.6 .5 -.4

    -1.4 1.5 .6

    -.4 -.5 -1.4

    1.6 -.5 .6

    -.4 1.5 .6

    ⌉ | ⌋

    *

    ⌈ | | | | | | | | ⌊

    -.4 -1.5   -.4  -.4 -1.5   -.4 -1.4   .5    .6   .6  -.5    .6   .6    .5   -.4  1.6   .5   -.4 -1.4  1.5    .6  -.4  -.5  -1.4  1.6  -.5    .6  -.4  1.5    .6

    ⌉ | | | | | | | | ⌋

    =

    ⌈ | ⌊

    10.4 -2.0 -.6

    -2.0 10.5 3.0

    -.6 3.0 4.4

    ⌉ | ⌋

• Variance/covariance matrix.

  Cxx = Sxx * N-1 =

  ⌈ | ⌊

  10.4 -2.0 -.6

  -2.0 10.5 3.0

  -.6 3.0 4.4

  ⌉ | ⌋

  * 10-1 =

  ⌈ | ⌊

  1.04 -.20 -.06

  -.20 1.05 .30

  -.06 .30 .44

  ⌉ | ⌋

• Diagonal matrix of standard deviations.

  Sxx = (Diag(Cxx))1/2 =

  ⌈ | ⌊

  1.02 0 0

  0 1.02 0

  0 0 .66

  ⌉ | ⌋

• Correlation matrix

  Rxx = Sxx-1 * Cxx * Sxx-1 =

  ⌈ | ⌊

  1.02-1 0 0

  0 1.02-1 0

  0 0 .66-1

  ⌉ | ⌋

  *

  ⌈ | ⌊

  1.04 -.20 -.06

  -.20 1.05 .30

  -.06 .30 .44

  ⌉ | ⌋

  *

  ⌈ | ⌊

  1.02-1 0 0

  0 1.02-1 0

  0 0 .66-1

  ⌉ | ⌋

  =

  ⌈ | ⌊

  1.00 -.19 -.09

  -.19 1.00 .44

  -.09 .44 1.00

  ⌉ | ⌋

Lecture 8c: ESTIMATION OF COEFFICIENTS

• Still ORDINARY LEAST SQUARES estimation.
  • But we will use matrix algebra.
• The values of the coefficients (B) are estimated such that the model yields optimal predictions
  • Minimize the residuals!
  • The sum of the squared (SS) residuals is minimized.
  • SS.RESIDUAL = ∑(Ŷ- Y)²
• Estimation of coefficients.
  • Ŷ = B₀ + B₁X₁
    • Y - Ŷ = e (e is the prediction error, or residual.)
  • Ŷ = BX      # In both matrix form and standardized.
    • Ŷ is a [N x 1] vector
      • N = number of cases.
    • B is a [k x 1] vector
      • k = number of predictors.
    • X is a [N x k] matrix.
  • Ŷ = BX
    • To solve for B
    • Replace Ŷ with Y
    • Conform for matrix multiplication:
      • Y = XB
  • Now let's make X square and symmetric.
    • To do this, pre-multiply both sides of the equation by the transpose of X, X'.
  • Y = XB becomes
    • X'Y = X'XB
    • Now to solve for B, eliminate X'X.
    • Pre-multiply by the inverse, (X'X)^-1
  • X'Y = X'XB becomes
    • (X'X)^-1X'Y = (X'X)^-1X'XB
      • Note that (X'X)^-1X'X = I
        • I = identity matrix. Any matrix times the identity matrix returns that matrix.
      • And IB = B
    • Therefore, (X'X)^-1X'Y = B
  • B = (X'X)^-1X'Y
• Example
  • Take the Sums of squares and Cross-products matrix from the previous example

    • ⌈ | ⌊

      10.4 -2.0 -.6

      -2.0 10.5 3.0

      -.6 3.0 4.4

      ⌉ | ⌋

  • The sums of squares are in the diagonal (10.4, 10.5, 4.4), cross-products in the off diagonal.
  • Now call the columns Y, X₁, and X₂
  • Since we used deviation scores:
    • Substitute S_xx for X'X
    • Substitute S_xy for X'Y
  • Therefore, B = (S_xx)^-1S_xy

    • ⌈ ⌊

      10.5 3.0

      3.0 4.4

      ⌉-1 ⌋

      ⌈ ⌊

      -2.0 -.6

      ⌉ ⌋

      =

      ⌈ ⌊

      -0.19 -0.01

      ⌉ ⌋

  • Those are the regression coefficients for this example.

Lecture 9a: MULTIPLE REGRESSION ANALYSIS IN R

• Simple regression has 1 predictor. Multiple regression has 2 or more predictors.
• In this example, we want to predict physical endurance based on
  • Age.
  • Years engaged in active exercise.

# Statistics One, Lecture 9a, example script
# Multiple regression analysis
# We are assuming normal distribution,
# linear relationships among variable,
# homoscedasticity, reliability and validity.

endur <- read.table( "stats1.ex.04.txt", header = T )

# Scatterplot
plot( endur$endurance ~ endur$age, main = "Scatterplot", ylab = "Endurance", xlab = "Age" )
abline( lm( endur$endurance ~ endur$age ), col="blue" )

plot( endur$endurance ~ endur$activeyears, main = "Scatterplot", ylab = "Endurance", xlab = "Active years" )
abline( lm( endur$endurance ~ endur$activeyears ), col="red" )

# Regression analyses (unstandardized)
model1 = lm( endur$endurance ~ endur$age )
summary( model1 )

model2 = lm( endur$endurance ~ endur$activeyears )
summary( model2 )

model3 = lm( endur$endurance ~ endur$age + endur$activeyears )
summary( model3 )

Lecture 9b: MULTIPLE REGRESSION ANALYSIS, STANDARDIZED

• New script, this time with standardized regression coefficients. Then compare the two models.
• To get standardized regression coefficients, use the "scale" function.
• To compare models, use the "anova" function (analysis of variables).

# Statistics One, Lecture 9b, example script
# Multiple regression analysis "standardized"
# Same data as Lecture 9a.

# Regression analyses (standardized)
model1.z = lm( scale( endur$endurance ) ~ scale( endur$age ) )
summary( model1.z )

model2.z = lm( scale( endur$endurance ) ~ scale( endur$activeyears ) )
summary( model2.z )

model3.z = lm( scale( endur$endurance ) ~ scale( endur$age ) + scale( endur$activeyears ) )
summary( model3.z )

# Model comparisons
comp1 = anova( model1.z, model3.z )
comp1
comp2 = anova( model2.z, model3.z )
comp2

ASSIGNMENT THREE CODE

# Statistics One, Week Three Assignment # Run correlation and multiple regression on "daa.03.txt". Then answer 10 questions. round to 2 significant digits.

assign03 <- read.table( "daa.03.txt", header = T)

# Question 1
# Correlation between age and endurance
cor( assign03$age, assign03$endurance )
# -.13
# Question 2
# Unstandardized regression coefficient for age, predicting endurance
assign03.q2 = lm( assign03$endurance ~ assign03$age )
summary( assign03.q2 )
# -.13
# Question 3
# Standardized regression coefficient for age, predicting endurance
assign03.q3.z = lm( scale( assign03$endurance ) ~ scale( assign03$age ) )
summary( assign03.q3.z )
# -.13
# Question 4
# Unstandardized regression coefficient for age, predicting endurance (while also using activeyears to predict endurance)
assign03.q4 = lm( assign03$endurance ~ assign03$age + assign03$activeyears )
summary( assign03.q4 )
# -.26
# Question 5
# Standardized regression coefficient for age, predicting endurance (while also using activeyears to predict endurance)
assign03.q5.z = lm( scale( assign03$endurance ) ~ scale( assign03$age ) + scale( assign03$activeyears ) )
summary( assign03.q5.z )
# -.24
# Question 6
# Correlation between activeyears and endurance
cor( assign03$activeyears, assign03$endurance )
# .34
# Question 7
# Unstandardized regression coefficient for activeyears, predicting endurance
assign03.q7 = lm( assign03$endurance ~ assign03$activeyears )
summary( assign03.q7 )
# .76
# Question 8
# Standardized regression coefficient for activeyears, predicting endurance
assign03.q8.z = lm( scale( assign03$endurance ) ~ scale( assign03$activeyears ) )
summary( assign03.q8.z )
# .34
# Question 9
# Unstandardized regression coefficient for activeyears, predicting endurance (while also using age to predict endurance)
assign03.q9 = lm( assign03$endurance ~ assign03$activeyears + assign03$age )
summary( assign03.q9 )
# .92
# Question 10
# Standardized regression coefficient for activeyears, predicting endurance (while also using age to predict endurance)
assign03.q10.z = lm( scale( assign03$endurance ) ~ scale( assign03$activeyears ) + scale( assign03$age ) )
summary( assign03.q10.z )
# .40

Lecture 10a: MEDIATION ANALYSIS REGRESSION METHOD

• Suppose we have a correlation between two variable. We are looking for a third variable or mechanism that links them. The intent is to demonstrate causality, when randomized experiments aren't possible.
• Moderation: a moderator is a third variable that has control over a correlation or relationship between two variables.
• An example:
  • X: Psychological trait
    • Extraversion
  • Y: Behavioral outcome
    • Happiness
  • M: Mechanism
    • Diversity of life experience
  • Z: Moderator (ZAP! or ZING!)
    • Socio-Economic-Status (SES)
• We want to know why extraversion is correlated with happiness?
• If X and Y are correlated then we can use regression to predict Y from X
  • Y = B₀ + B₁X + e
• If X and Y are correlated BECAUSE of the mediator M then ( X ⇒ M ⇒ Y ):
  • Y = B₀ + B₁M + e
               &
  • M = B₀ + B₁X + e
               or
  • Y = B₀ + B₁M + B₂X + e
  • What will happen to the predictive value of X
  • In other words, will B₂ be significant? If the mediator is valid, B₂ should be insignificant.
• A mediator variable (M) accounts for some (partial mediation) or all (full mediation) of the relationship between X and Y.
• CAUTION! Correlation does not imply causation! There is a big difference between statistical mediation and true causal mediation.
• Run three regression models
  • lm( Y ~ X )
    • Regression coefficient for X should be significant.
  • lm( M ~ X )
    • Regression coefficient for X should be significant.
  • lm( Y ~ X + M )
    • Regression coefficient for M should be significant.
    • If M is very significant, X would have to be insignificant.
• Back to the example. Assume N = 188.
• Participants surveyed and asked to report:
  • Happiness (happy).
  • Extraversion (extra).
  • Diversity of life experiences (diverse).
    
    
  • Assume all are scored on a scale from 1 - 5.
• Results of the first two models:
  • happy = 2.19 + .28(extra).
    • happy = 2.19 at the intercept, where extraversion is zero. As extraversion increases (per unit), add .28 units of happy.
  • diverse = 1.63 + .28(extra).
  • For both, regression coefficient for X (extra) is statistically significant, p < .05.
  • happy = 1.89 + .22(extra) + .19(diverse).
  • ALL regression coefficients statistically significant.
• This is partial mediation because the direct effect on happy, (extra) only dropped a little after the mediator (diverse) was added into the regression equation. Happy did not drop to zero.

Lecture 10b: MEDIATION ANALYSIS PATH ANALYSIS METHOD

• Standard notation for Path include:
  • Rectangles: Observed (manifested) variables (X, Y, M) (predictor, outcome, mediator).
  • Circles: Unobserved variables (e).
  • Triangles: Constants.
  • Arrows: Associations.
• In the diagrams, variables Y and M are said to be endogenous because the cause of variation are inside the model.
• We are saying nothing about causes for our predictor, X. It is said to be exogenous, outside the model.
• Label the paths have been labeled a, b, c, c'.
• The Sobel test
  • z = (B_a * B_b) / √(B_a² * SE_b²) + (B_b² * SE_a²)
• The null hypothesis
  • The indirect effect is zero.
  • (B_a * B_b) = 0
• Mediation analysis is more powerful with
  • true independent variables
  • the incorporation of time i.e. one thing follows another).
• Otherwise, who is to say that happiness doesn't cause extroversion.

Lecture 11a: MODERATION ANALYSIS

• Moderation: Introducing a new variable that can completely change the picture, or the relationship between two variables.
• It might be the context of X and Y.
• The same example:
  • X: Psychological trait
    • Extraversion
  • Y: Behavioral outcome
    • Happiness
  • M: Mechanism
    • Diversity of life experience
  • Z: Moderator (ZAP! or ZING!)
    • Socio-Economic-Status (SES)
• Perhaps, only rich extravert's are happier.
• A moderator variable (Z) has influence over the relationship between X and Y.
  • Suppose X and Y are positively correlated.
  • The moderator (Z) can change that.
• If X and Y are correlated then we can use regression to predict Y from X
  • Y = B₀ + B₁X + e
    • B₁ is the slope of X to Y.
  • If there is a moderator, Z, then B₁ will depend on Z.
    • The relationship between X and Y changes as a function of Z.
• A quick example:
  • In two decades of working testing, the professor has seen a correlation between SAT's and working memory capacity.
  • Not at Princeton.
  • They all have high SAT's, there is nothing to correlate. There isn't a representative sample.
  • The type of University moderates the relationship between WMC and SAT.
• Y = B₀ + B₁X + B₂Z + B₃(X*Z) + e
  • The B₀ and B₁ are the standard formula. The B₂ and B₃ are the moderator.
• We can test for moderation with one regression model
  • lm( Y ~ X + Z + X*Z )
    • Need to create a new column in our data.frame for (X*Z).
    • Let's call it PRODUCT.
• Back to the extrovert example. To simplify, let's make SES categorical:
  • SES = 1 = HIGH SES
  • SES = 0 = LOW SES
• Results before adding PRODUCT (main or first order affects).
  • Ŷ = B₀(1) + B₁(EXTRA) + B₂(SES)
  • Ŷ = 3.04 + .039(EXTRA) + 0.00(SES)
• Results after adding PRODUCT (main or first order affects).
  • Ŷ = B₀(1) + B₁(EXTRA) + B₂(SES) + B₃(PRODUCT)
  • Ŷ = 3.88 + -.20(EXTRA) + -1.69(SES) + 0.47(PRODUCT)
• In this case, a low SES gives you a negative correlation between extraversion and happiness. High SES displays a positive relationship.
• SES moderates the relationship between extraversion and happiness.

Lecture 11b: CENTERING AND DUMMY CODING

• To center means to put our variable predictors (X) in deviation form.
  • X_c = X - M
  • For every predictor variable, the mean will now be zero.
  • The regression constant (intercept) is the predicted score on Y when all the X's are zero.
  • The regression constant now becomes meaningful and easy to interpret.
• Why center?
  • Conceptual
    • Suppose
      • Y = child's language development.
      • X1 = mother's vocabulary.
      • X2 = child's age.
    • The intercept, B₀, is the predicted score on Y when all X's are zero.
    • If X = zero is meaningless, or impossible, B₀ will be difficult to interpret.
    • If X = zero is the average then B₀ (intercept) is easy to interpret.
      • Now mother's vocabulary and child's age are the average vocabulary and age. The intercept is means based and meaningful.
  • But this is for simple regression only.
  • There is another step for moderation analysis. Remember how to interpret multiple regression coefficients in multiple regression.
    • The regression coefficient B₁ is the slope for X1 assuming an average score on X2.
    • No moderation implies the B₁ is consistent across the entire distribution of X2.
    • However, moderation implies the B₁ is NOT consistent across the entire distribution of X2.
    • Where in the distribution of X2 is B₁ most representative?
      • The best place would be in the middle of the distribution of X2
      • If it's centered, we're already there.
    • In this case, the regression constant and the variable slopes will change, but not the moderator slope.
    • Also, the variable slopes are more meaningful. They are the median slopes, not the end slopes.
  • Statistical
    • The product X1*X2 can become highly correlated with one of the predictors, X1 or X2.
      • It can result in multi-collinearity. That's a bad thing.
• Summary
  • If we center our predictors, we can run a moderation analysis in a more appropriate fashion.
  • Now we can look at the moderation effects by running sequential regression. There are two ways to see if it is significant.
    • Look at the main effects.
    • Look at the moderation effect:
      • Evaluate the regression coefficient (slope) for the product term. Is it significant.
      • ΔR² from Model 1 to Model 2, which means do an F-test on the percentage of variance explained on model one versus model two.
• Dummy Coding: A system to code categorical predictors in a regression analysis.
  • What if SES in the earlier example had more than two values (0 and 1).
  • Example (faculty salary)
    • IV: Area of research
      • Cognitive
      • Social
      • Behavioral neuroscience
      • Cognitive neuroscience
    • DV: # of publications.
    • Each Psychology professor is in one of these four areas of research, and how many publications does each researcher have in each area.
  • Question, do cognitive psychology professors publish more than social psychology professors?
  • Is there a significant difference in number of publications across psychology departments?
    • Add in a dummy variable into the data.frame for each area of research, -1 (4 areas get 3 columns). In this case they are C1, C2, and C3
    • Pick one group as the reference group. We pick cognitive. It has zeros for all the codes.
    • Each of the others gets a one in one of the codes. Each is a different code (variable).
  • The regression model is now:
    • Ŷ = B₀ + B₁(C1) + B₂(C2) + B₃(C3)
  • As there were a different number of professors in each area, the numbers aren't exact. You can improve on this with "unweighted effects" coding or "weighted effects" coding.

Lecture 11c: MODERATION ANALYSIS EXAMPLE 2

• Faculty Salary Example
  • DV = salary
  • IVs
    • # of publications
    • Department
      • Psychology
      • Sociology
      • History
• The question is, does department moderate the relationship between publications and between salary?
  • Center continuous predictor.
    • Center publications.
  • Dummy code categorical predictor.
    • Create dummy codes for department.
      • Regular dummy coding with Psych as the reference group.
  • Create moderation terms.
  • Run sequential regression in 2 steps.
    • Main effects.
    • Moderation effect.
• Regression model: Before moderation.
  • Ŷ = B₀ + B₁(PUBS.C) + B₂(C1) + B₃(C2)
• Regression model: Moderation.
  • Ŷ = B₀ + B₁(PUBS.C) + B₂(C1) + B₃(C2) + B₄(C1*PUBS.C) + B₅(C2*PUBS.C)
• The data did show that the department you are in, moderates between publications and salary.
• To find out if the History or Sociology slopes are significant, of if the difference between them is, re-code to make a different reference group re-run the analysis.

Lecture 12a: MEDIATION ANALYSIS IN R

• Write a script in R to test for mediation.
  • Three regression analyses.
    • Outcome = Predictor.
    • Predictor = Mediator.
    • Outcome = Predictor + Mediator.
  • Three regression analyses.
    • lm( Y ~ X )
    • lm( Y ~ M )
    • lm( Y ~ X + M )
  • Fictional data
    • Outcome (Y).
      • Happiness.
    • Predictors (X, M).
      • Extraversion (X).
      • Diversity of life experience (M).

# Statistics One, Lecture 12a, example script
# Mediation Analysis
# X is extraversion
# Y is happiness
# M is diversity of life experience

# Sobel test requires multilevel package.
install.packages( "multilevel" )
library( psych )
library( multilevel )

med <- read.table( "stats1.ex.05.txt", header = T )

# Test shape of distribution and univariate normal assumptions
describe( med )
layout( matrix( c( 1,2,3 ), 1, 3, byrow = TRUE ) )
hist( med$happy )
hist( med$extra )
hist( med$diverse )

#Print scatter.plots to test linear and homoscedasticity assumptions
layout( matrix( c( 1,2,3 ), 1, 3, byrow = TRUE ) )
plot( med$happy ~ med$extra )
abline( lm( med$happy ~ med$extra ) )
plot( med$diverse ~ med$extra )
abline( lm( med$diverse ~ med$extra ) )
plot( med$happy ~ med$diverse )
abline( lm( med$happy ~ med$diverse ) )

# Conduct three regression analyses
model.12a.1 = lm( med$happy ~ med$extra )
model.12a.2 = lm( med$diverse ~ med$extra )
model.12a.3 = lm( med$happy ~ med$extra + med$diverse )
summary( model.12a.1 )
summary( model.12a.2 )
summary( model.12a.3 )

# Sobel test (is indirect path statistically significant?)
indirect.12a = sobel( med$extra, med$diverse, med$happy )
indirect.12a

Lecture 12b: MODERATION ANALYSIS IN R

• Fictional data
  • Outcome (Y)
    • Happiness
  • Predictors (X, Z)
    • Extraversion (X)
    • Socio-Economic Status (SES) (Z)

# Statistics One, Lecture 12b, example script.
# Moderation analysis.
# X is extraversion.
# Y is happiness.
# Z is SES.

mod.12b <- read.table( "stats1.ex.06.txt", header = T )

# Data for mod.12b$mod variable is in not
# calculated here. It is in the data file.
no.mod.model.12b = lm( mod.12b$happy ~ mod.12b$extra + mod.12b$ses )
mod.model.12b = lm( mod.12b$happy ~ mod.12b$extra + mod.12b$ses + mod.12b$mod )

summary( no.mod.model.12b )
summary( mod.model.12b )

# Compare models.
anova( no.mod.model.12b, mod.model.12b )

Lecture 13a: STUDENT'S t-TEST

• If we have randomized controlled experiments, we don't need the regression models from the last lectures. We can do t-tests.
• Two means can be compared using a t-test.
• In this lecture we cover 4 variations of the t-test.
  • z-test
  • t-test (single sample)
  • t-test (dependent)
  • t-test (independent)
• Overview
  • z = (observed - expected) / SE.
  • t = (observed - expected) / SE.
  • SE: Standard error.
• When to use z and t?
  • z
    • When comparing a sample mean to a population mean and the standard deviation of the population is known.
  • Single sample t
    • When comparing a sample mean to a population mean and the standard deviation of the population is not known.
  • Dependent samples t
    • When evaluating the difference between two related samples.
  • Independent samples t
    • When evaluating the difference between two independent samples.

• 	observed	expected	SE
  z	Sample mean	Population mean	SE for a mean
  t (single sample)	Sample mean	Population mean	SE for a mean
  t (dependent)	Difference	Difference	SE for a difference
  t (independent)	Difference	Difference	SE for a difference

  • σ: population standard deviation.
  • μ: population mean
  • SD: sample standard deviation
  • M: sample mean
  • SE: standard error
  • SE_m: standard error for a mean
  • SE_md: standard error for a difference (dependent)
  • SE_Difference: standard error for a difference (independent)
• Exact p-value depends on:
  • Directional or non-directional test?
  • df, degrees of freedom (different t-distributions for different sample sizes).
• Single sample t: Compare a sample mean to a population mean
  • t = ( M - μ ) / SE_m
  • SE²_m = SD² / N
  • SE_m = SD / SQRT( N )
  • SD² = ∑( X - M )² / ( N - 1 ) = SS / df = MS
• Example:
  • Suppose it takes rats just 2 trials to learn how to navigate a maze to receive a food reward.
  • A researcher surgically lesions part of the brain and then test the rats in the maze. Is the number of trials to learn the maze significantly more than 2?
• Effect size (Cohen's d)
  • d = ( M - μ ) / SD

Lecture 13b: DEPENDENT & INDEPENDENT t-TESTS

• Single sample t compares a sample mean to a population mean. Dependent t is a comparison of the same people in two different conditions. Independent compares two separate groups of people.
• Dependent means t
  • The formulas are actually the same as the single sample t but the raw scores are difference scores, so the mean is the mean of the difference scores and SE_m is based on the standard deviation of the difference scores.
• Independent means t
  • Compares two independent groups
  • For example, males and females, control and experimental, drug group and placebo group, etc.
    • t = (M₁ - M₂) / SE_Difference
    • SE²_Difference = SE²_m1 + SE²_m2
    • SE²_m1 = SD²_Pooled / N1
    • SE²_m2 = SD²_Pooled / N2
    • SD²_Pooled = (df₁ / df_Total) * (SD²₁) + (df₂ / df_Total) * (SD²₂)
    •    Notice that this is just a weighted average of the sample variances.

Lecture 14a: GENERAL LINEAR MODEL

• ANOVA: analysis of variance.
• GLM is the mathematical framework used in many common statistical analyses, including multiple regression and ANOVA.
  • ANOVA is typically presented as distinct from multiple regression but IS a multiple regression.
  • It's a special case where the predictors are orthogonal (not correlated).
    • What they are is conditional or categorical.
• The two main characteristics of GLM are
  • Linear: pairs of variables are assumed to have linear relations.
  • Additive: if one set of variables predict another variable, the effects are thought to be additive.
• This doesn't mean we can't test non-linear or non-additive effects (interaction).
  • GLM can accommodate such tests, for example
    • Transformation of variables
      • Transform so non-linear becomes linear.
    • Moderation analysis
      • Fake the GLM into testing non-additive effects (product scores used in moderation analysis).
• Simple regression
  • Y = B₀ + B₁X₁ + e
    • Y = faculty salary
    • X1 = years since PhD
• Multiple regression
  • Y = B₀ + B₁X₁ + B₂X₂ + B₃X₃ + e
    • Y = faculty salary
    • X1 = years since PhD
    • X2 = # of publications
    • X3 = (years x pubs)
• In the last example, "years since PhD" and "# of publications" are continuous variables. But gender or race are conditional or categorical variables.
• One-way ANOVA
  • Y = B₀ + B₁X₁ + e
    • Y = faculty salary
    • X1 = gender
• Factorial ANOVA
  • Y = B₀ + B₁X₁ + B₂X₂ + B₃X₃ + e
    • Y = faculty salary
    • X1 = gender
    • X2 = race
    • X3 = interaction (gender x race)
• ANOVA: Appropriate when the predictors (independent variable or IVs) are all categorical and the outcome (DV) is continuous.
  • Most common application is to analyze data from randomized controlled experiments.
• More specifically, randomized experiments that generate more than 2 means
  • If only 2 means then use:
    • Independent t-test.
    • Dependent t-test.
  • If more than 2 means then use:
    • If they are independent groups, then between groups ANOVA.
    • If the same subjects measured repeatedly, then repeated measures ANOVA.
    • This is analogous to the independent t-test and the dependent t-test.
• NHST may accompany ANOVA
• The test statistic is the F-test (ratio) rather than a t statistic.
  • F = systematic variance / unsystematic variance
• Like t-test, the F-test has a family of F-distributions
  • The distribution to assume depends on
    • Number of subjects per group.
    • Number of groups.
    • This is a ratio of variances, it can't be less than zero.
    • The expected value under the null hypothesis is one.
      • This is analogous to the Z distribution of the normal curve.
    • If we get a number like 5 we are way out on the tail of the curve, we will reject the null hypothesis and claim a significant effect.

Lecture 14b: ONE-WAY ANOVA

• Example: working memory training, how many sessions they trained, outcome is the gain in their scores.
• F-ratio
  • F = systematic variance / unsystematic variance
  • F = between-groups variance / within-groups variance
  • F = MS_Between / MS_Within
  • F = MS_a / MS_s/a (s/a: subject within group).
    • MS_a = SS_a / df_a
    • MS_s/a = SS_s/a / df_s/a
• SS_a = n ∑(Y_j - Y_T)²
  • Y_j are the treatment means.
  • Y_T is the grand mean.
• SS_s/a = n ∑(Y_ij - Y_j)²
  • Y_ij are individual scores.
  • Y_j are the treatment means.
• Degrees of freedom
  • df_A = a - 1.
  • df_s/a = a(n - 1).
  • df_Total = N - 1.
• Effect size
  • η² (eta-squared) is analogous to R² in multiple regression, percentage of variance explained in the outcome variable.
  • η² = SS_A / SS_Total.
• Assumptions:
  • DV is continuous.
  • DV is normally distributed.
  • Homogeneity of variance.
    • Within-groups variance is equivalent for all groups.
      • levene.test()
• Post-hoc test. If levene.test() is significant then homogeneity of variance assumption has been violated.
  • Conduct comparisons using a restricted error term.

Lecture 14c: FACTORIAL ANOVA

• Two IVs (independent variables) or treatments.
• One continuous DV or dependent variable (response).
  • IV = driving difficulty (easy, difficult).
  • IV = conversation difficulty (none, easy, difficult).
    • Talking on a cell phone.
  • DV = errors made in driving simulator.
• Three hypotheses can be tested in this one experiment:
  • More errors in the difficult simulator?
  • More errors with more difficult conversations?
  • More errors due to the interaction of these factors?
• We will calculate three F-ratios.
  • F_A
  • F_B
  • F_AxB
• Terms
  • Main effect: the effect of one IV averaged across the levels of the other IV.
  • Interaction effect: the effect of one IV depends on the other IV (the simple effects of one IV change across the levels of the other IV).
  • Simple effect: the effect of one IV at a particular level of the other IV, if significant.
• Main effects and interaction effect are independent from one another (orthogonal).
  • Note that this is different from studies that don't employ an experimental design.
    • For example, in MR, when predicting faculty salary, the effects of publications and years since the Ph.D. were correlated..
• Remember GLM (general linear model).
  • Multiple regression looked like
    • Y = B₀ + B₁X₁ + B₂X₂ + B₃X₃ + e
      • Y = faculty salary.
      • X1 = years since Ph.D.
      • X2 = # of publications.
      • X3 = (years x pubs).
  • Factorial ANOVA
    • Y = B₀ + B₁X₁ + B₂X₂ + B₃X₃ + e
      • Y = faculty salary.
      • X1 = gender.
      • X2 = race.
      • X3 = interaction (gender x race).
• We want 3 F-ratios.
  • F_A = MS_A / MS_S/AB
  • F_B = MS_B / MS_S/AB
  • F_AxB = MS_AxB / MS_S/AB
• MS (mean squared)
  • MS_A = SS_A / df_A
  • MS_B = SS_B / df_B
  • MS_AxB = SS_AxB / df_AxB
  • MS_S/AB = SS_S/AB / df_S/AB
• df (degrees of freedom)
  • df_A = a - 1
    • a = levels of driving difficulty (2-1).
  • df_B = b - 1
    • b = levels of conversation difficulty (3-1).
  • df_AxB = (a - 1)(b - 1)
    • 1 x 2
  • df_S/AB = ab(n - 1)
    • number of group x subjects in a group minus one.
  • df_Total = abn - 1 = N - 1
• Follow up tests.
  • Main effects
    • Post-hoc tests
    • Only where we have more than 2 groups.
  • Significant interaction effects
    • It's not enough to show there are significant interaction effects, we have to show where they are coming from.
    • Analysis of simple effects
      • Conduct a series of one-way ANOVAs
        • How, where, when and why are the simple effects different.
        • For example, we could conduct 3 one-way ANOVAs comparing high and low spans at each level of other IV.
• Effect size
  • Complete η²
    • η² = SS_effect / SS_total
  • Partial η²
    • η² = SS_effect / (SS_effect + SS_S/AB)
• The assumptions underlying the factorial ANOVA are the same as for the one-way ANOVA.
  • DV is continuous.
  • DV is normally distributed.
  • Homogeneity of variance.
    • Use Levine's test.
• Back to the driving example.
  • Strayer and Johnson (2001) conducted an experiment to examine the effect of talking on a cell-phone on driving.
  • They tested subjects in a driving simulator.
  • Here's the interesting part...
    • They manipulated the difficulty of the driving,
    • AND the difficulty of the conversation.
• To manipulate driving difficulty, they simply made the driving course in the simulator more or less difficult.
• To manipulate conversation difficulty, they included two "talking" conditions:
  • In one, the subject simply had to repeat what they heard on the other line of the phone.
  • In the other, the subject had to think of and then say a word beginning with the last letter of the last word spoken on the phone.
    • e.g. if you hear "ship" then respond a word that begins with p - "pear".
  • And there was a no-talking condition (control).
• Again, a summary of the design.
  • IV 1 = driving difficulty (easy, difficult).
  • IV 2 = conversation difficulty (none, easy, difficult).
  • DV = errors made in driving simulator.
  • We would refer to this as a 2 x 3 factorial design. We will analysis it with a 2 x 3 between groups ANOVA.
• Follow-up tests
  • Post-hoc tests
    • Need to conduct post-hoc tests on the conversation IV.
    • No need for driving difficulty because there's only 2 levels.
  • Simple effects
    • Simple effect of conversation at each level of driving difficulty.
      OR
    • Simple effect of driving difficulty at each level of conversation.
• Simple effects analysis to explain the interaction variable.
  • Look at the effect of one independent variable at each level of the other. Either effect of driving difficulty for each conversation condition, or vice versus.
    • F_{a at bk} = MS_{a at bk} / MS_S/AB
    • MS_{a at bk} = SS_{a at bk} / df_{a at bk}
    • df_{a at bk} = a - 1
    • SS_{a at bk} = n ∑(Y_jk - Y_Bk)²
• We choose simple effect of driving difficulty at each level of conversation.
  • No conversation.
    • F = MS_{A at b1} / MS_S/AB
    • F = 324.9 / 6.267 = 51.84
  • Easy.
    • F = MS_{A at b2} / MS_S/AB
    • F = 592.9 / 6.267 = 94.61
  • Difficult.
    • F = MS_{A at b3} / MS_S/AB
    • F = 1000 / 6.267 = 159.57
• Effect sizes.
  • No conversation.
    • η² = SS_{A at b1} / (SS_{A at b1} + SS_S/AB)
    • η² = 324.9 / (324.9 + 150.4) = .68
  • Easy.
    • η² = SS_{A at b2} / (SS_{A at b2} + SS_S/AB)
    • η² = 529.9 / (529.9 + 150.4) = .80
  • Difficult.
    • η² = SS_{A at b3} / (SS_{A at b3} + SS_S/AB)
    • η² = 1000 / (1000 + 150.4) = .87

Lecture 15a: STUDENT T-TEST AND ANOVA IN R.

• Is it possible for adults to enhance their intelligence by training their working memory.
• Population
  • Healthy adults
• Sample
  • Random selection from the population.
  • Representative of the population.
• Procedure
  • Treatment group engaged in memory training for a half hour every day for weeks.
  • IQ change
    • All subjects completed a test of intelligence before and after training.
• Results were reported two different ways.
  • First approach
    • IV: WM training (training vs. control)
    • DV: Score on intelligence test (pre and post).
  • Second approach
    • IV: WM training (number of sessions).
    • DV: Gain on intelligence test (post - pre).
• Write a script in R to illustrate dependent t
  • IV: Pre and post
    • dependent t-test, within a group, same subject two different conditions.
  • DV: Score
• Write a script in R to illustrate independent t
  • IV: Training group (control, training)
  • DV: Gain
• Write a script in R to illustrate ANOVA
  • IV: Training group (8, 12, 17, 19)
  • DV: Gain

Lecture 15b: T-TEST IN R.

• Write a script in R
  • dependent t
    • IV: Pre and post.
    • DV: Score.
  • independent t
    • IV: Training group (control, training).
    • DV: Gain.

# Statistics One, Lecture 15b, example script.
# Write t-tests in R using working memory training data
# Dependent t, IV: Pre & post.
#     DV: Score.
# Independent t. IV: Training group (control, training).
#     DV: Gain.

# Read data into R
lec15b = read.table( "stats1.ex.07.txt", header = T )

# Print descriptive statistics for the variables
# in wm by training condition (cond)
describeBy( lec15b, lec15b$cond )

# Create subsets of data for control and training conditions
lec15b.c = subset( lec15b, lec15b$train == "0" )
lec15b.t = subset( lec15b, lec15b$train == "1" )

# We need internal descriptive statistics to calculate effect size.
lec15b.c.out = describe( lec15b.c )
lec15b.t.out = describe( lec15b.t )
lec15b.c.out
lec15b.t.out

# Dependent t-tests
t.test( lec15b.c$pre, lec15b.c$post, paired = T )
t.test( lec15b.t$pre, lec15b.t$post, paired = T )

# Cohen's d for dependent t-test
d.c = (lec15b.c.out[ 4, 3 ]) / (lec15b.c.out[ 4, 4 ])
d.t = (lec15b.t.out[ 4, 3 ]) / (lec15b.t.out[ 4, 4 ])

d.c
d.t

# Independent t-tests
t.test( lec15b$gain ~ lec15b$train, var.equal = T )

# Cohen's d for independent t-tests
lec15b.pooled.sd = (79/118 * lec15b.t.out[ 4, 4 ]) + (39/118 * lec15b.c.out[ 4, 4 ])
d.ct = (lec15b.t.out[ 4, 3 ] - lec15b.c.out[ 4, 3 ]) / lec15b.pooled.sd
d.ct

Lecture 15c: ANOVA IN R.

• This time we will use the working memory training, using "number of sessions" and gain on intelligence tests.
• Because we have 4 groups, we want to use a one-way between groups ANOVA.

# Statistics One, Lecture 15c, example script.
# Write ANOVA in R, using memory training data.
# IV: WM training (number of sessions).
# DV: Gain on intelligence test (post - pre).
# One-way between groups ANOVA.

library( psych )
library( gclus)
library( multilevel )
library( lawstat)

# aov( DV ~ IV )
lec15c.aov.model = aov( lec15b.t$gain ~ lec15b.t$cond )
summary( lec15c.aov.model )
lec15c.aov.table = summary( lec15c.aov.model )

# Effect size for ANOVA
lec15c.ss = lec15c.aov.table[[1]]$"Sum Sq"
lec15c.eta.sq = lec15c.ss[1] / ( lec15c.ss[1] + lec15c.ss[2] )
lec15c.eta.sq

# Post-hoc tests
TukeyHSD( lec15c.aov.model )

library( lawstat )
# levene.test( DV, IV, mean )
levene.test( lec15b.t$gain, lec15b.t$cond, location=c("mean") )

Lecture 16a: BENEFITS OF REPEATED MEASURES (within subjects) ANOVA

• Less cost (fewer subjects required). Every subject is exposed to every condition in your experiment.
• More subjects is more statistical power.
  • Variance across subjects may be systematic.
  • If so, it will not contribute to the error term.
• Error term
  • F_A = MS_A / MS_AxS
• MS and F
  • MS_A = SS_A / df_A
  • MS_AxS = SS_AxS / df_AxS
  • F = MS_A / MS_AxS
• Example
  • A classic memory and learning paradigm is AB/AC paired associate learning.
  • Subjects learn a list of paired associates, A-B.
  • Then they learn another list, A-C.
  • Subsequently, recall is impaired when cued with A and asked to recall B.
  • This is known as retroactive interference.
• df
  • df_A = a - 1 = 3 - 1 = 2
  • df_s = n - 1 = 6 - 1 = 5
  • df_AxS = (a - 1)(n - 1) = 10
  • df_Total = (a)(n) - 1 = 17
• MS and F
  • MS_A = 700 / 2 = 350
  • MS_AxS = 2.67 / 10 = .267
  • F = 1310.86
• Post-hoc tests
  • The error term MS_AxS is NOT appropriate.
    • Need to calculate a new error term based on the conditions that are being compared.
  • F_Ψ = MS_ΨA / MS_ΨAxS
  • MS_ΨA = SS_ΨA / df_ΨA
  • MS_ΨAxS = SS_ΨAxS / df_ΨAxS
  • Where Ψ is a subset of.
• If you do lots of those corrections, one for every pair, you can use the Bonferroni method.
• Sphericity assumption
  • Homogeneity of variance.
  • Homogeneity of correlation.
    • r₁₂ = r₁₃ = r₂₃
  • Homogeneity of correlation can be tested with Mauchly's test.
  • If significant then report the p-value from one of the corrected tests
    • Greenhouse-Geisser
    • Huyn-Feldt

Lecture 16b: RISKS OF REPEATED MEASURES ANOVA

• Risks
  • Order effects.
    • Practice effects.
    • Fatigue effects.
  • Counterbalancing scheme to deal with these effects.
  • Missing data.
• Counterbalancing
  • Consider a simple design with just two condition, A1 and A2.
  • One approach is a Blocked Design
    • Subjects are randomly assigned to one of two "order" conditions.
      • A1, A2.
      • A2, A1.
  • Another approach is a Randomized Design.
    • Conditions are presented randomly in a mixed fashion.
      • A1, A2.
      • A2, A1, A1, A2, A2, A1, A2, ...
  • Now suppose a = 3 and a blocked design.
  • There are 6 possible orders (3!).
    • A1, A2, A3.
    • A1, A3, A2.
    • A2, A1, A3.
    • A2, A3, A1.
    • A3, A1, A2.
    • A3, A2, A1.
  • To completely counterbalance, subjects would be randomly assigned to one of 6 order conditions.
  • The number of conditions needed to completely counterbalance becomes large with more conditions..
    • 4! = 24.
    • 5! = 120.
  • With many levels of the IV a better approach is to use a "Latin Squares" design.
  • Latin Squares designs aren't completely counterbalanced but every condition appears every position at least once.
    • For example, if a = 3, then
      • A1, A2, A3.
      • A2, A3, A1.
      • A3, A1, A2.
    • This isn't perfect but it is very close. The number of orderings you need is the number of levels of independent variables.
• Missing data.
  • Relative amount of missing data.
  • Pattern of missing data.
• How much missing data is a lot?
  • There are no hard and fast rules.
  • A rule of thumb is
    • Less than 10% on any one variable is OK.
    • Greater than 10% is not OK.
• Is the pattern random or lawful?
  • For any variable of interest (X) create a new variable (XM).
    • XM = 0 if X is missing.
    • XM = 1 if X is not missing.
  • Conduct a t-test with XM as the IV and X as the DV.
  • If significant then pattern of missing data may be lawful.
• Missing data ~ Remedies
  • Drop all cases without a perfect profile.
    • Drastic.
    • Use only if you can afford it.
  • Keep all cases and estimate the values of the missing data points.
    • There are several options for how to estimate values.
• Missing data ~ Estimation methods
  • Insert the mean.
    • Conservative.
    • Decreases variance and covariance. That means you don't have much to work with.
  • Regression-based estimation.
    • More precise than using the mean but
    • Confusion often arises over which variables to use as predictors in the regression equation.
    • You must clearly spell out what you did and how you did it. Other wise it smells of manufactured data to fit your goals.

Lecture 17a: MIXED FACTORIAL ANOVA

• Mixed factorial designs.
  • One IV (independent variable) is manipulated between groups.
  • One IV is manipulated within groups.
    • Repeated measures variable.
• What's new?
  • Partitioning SS.
  • Formula for F_A, F_B, F_AxB.
  • Error term for post-hoc tests.
  • Approach to simple effects analyses.
  • Assumptions.
• df
  • df_A = a - 1
  • df_B = b - 1
  • df_AxB = (a - 1)(b - 1)
  • df_S/A = a(n - 1)
  • df_BxS/A = a(b - 1)(n - 1)
  • df_Total = (a)(b)(n) - 1
• MS
  • MS_A = SS_A / df_A
  • MS_B = SS_B / df_B
  • MS_AxB = SS_AxB / df_AxB
  • MS_S/A = SS_S/A / df_S/A
  • MS_BxS/A = SS_BxS/A / df_BxS/A
• F
  • F_A = MS_A / MS_S/A
  • F_B = MS_B / MS_BxS/A
  • F_AxB = MS_AxB / MS_BxS/A
• Post-hoc tests on main effects.
  • Post-hoc tests on the between-groups IV are performed in the same way as with a one-way ANOVA.
    • TukeyHSD
  • Post-hoc tests on repeated measures IV are performed in the same way as with a one-way repeated measures ANOVA.
    • Pairwise comparisons with Bonferroni correction.
• Simple effects analyses.
  • Must choose one approach or the other. To report both is redundant.
    • (a) Simple effects of the between groups IV.
    • (b) Simple effects of the repeated IV.
  • Simple effects of the between groups IV.
    • Simple effect of A at each level of B.
      • F_A.at.b1 = MS_A.at.b1 / MS_S/A.at.b1
    • Simple comparisons use the same error term (assuming homogeneity of variance).
      • MS_S/A.at.b1
  • Simple effect on the repeated measures IV.
    • Simple effect of B at each level of A.
      • F_B.at.a1 = MS_B.at.a1 / MS_BxS/A.at.a1
• Assumptions.
  • Each subject provides b scores.
  • Therefore,there are
    • b variances.
    • ( b * (b + 1) / 2 ) - b    covariances (correlations).
    • e.g., if b = 3 then 3 covariances.
    • e.g., if b = 4 then 6 covariances.
  • Between-groups assumptions.
    • The variances do not depend upon the group (homogeneity of variance).
      • If levene.test() is violated, calculate a new restricted error term.
    • The covariances do not depend upon the group (homogeneity of covariance).
      • Box's test of equality of covariance matrices.
      • If violated then report Greenhouse-Geisser or Huynh-Feldt values. Alternatively, consider a moderation analysis.
  • Within-subjects assumptions.
    • Sphericity: the variances of the different treatment scores (b) are the same and the correlations among pairs of treatment means are the same.
    • If violated, report Greenhouse-Geisser or Huynh-Feldt values.

Lecture 17b: MIXED FACTORIAL ANOVA IN R

• The experimental design of another working memory example.
• Mixed factorial design, 3x2, (Ax(BxS)).
  • Between groups IV (independent variable).
    • Memory task (words, sentences, stories).
  • Within groups IV.
    • Phonological similarity (similar, dissimilar).
  • DV (dependent variable).
    • Percentage of words recalled.
• Hypotheses.
  • Main effect of memory task?
    • Memory task (words, sentences, stories).
  • Main effect of similarity?
    • Phonological similarity (similar, dissimilar).
  • DV (dependent variable).
    • Interaction?
• Three F-ratios
  • F_A
  • F_B
  • F_AxB

code for "eta_squared.R"

eta.2 = function(aov.mdl, ret.labels = FALSE)
{ eta.2vector = c()
  labels = c()
  for ( table in summary(aov.mdl) )
  { # each block of factors
    SS.vector = table[[1]]$"Sum Sq"
      # table is a list with 1 entry, 
      # but you have to use [[1]] anyway
    last = length(SS.vector)
    labels = c(labels, row.names(table[[1]])[-last])
      # all but last (error term)
    for (SS in SS.vector[-last])
    { #all but last entry (error term)
      new.etaval = SS / (SS + SS.vector[last])
      eta.2vector = c(eta.2vector, new.etaval)
    }
  }
  if (ret.labels) return(data.frame(eta.2 = 
	eta.2vector, row.names = labels))
  return(eta.2vector)
}

• Using Linux, I needed to make these corrections to code for stats1.ex.08.R
  • eta_squared.R: on the course web sight, see the post Discussion Forums, Problems running the script for Lect 17b sample Script in R
  • Error: "task not found"
    • He puts the column of the data.frame but he doesn't always list which data.frame.
    • [task=="W"] needs to be [e1sr$task=="W"]
    • lines 22, 23, 28 and 29
      • e1sr, that is not the letter "L", it's the number one.
  • Same data.frame issue with the describeBys
    • recall[task=="W"] should be e1sr$recall[e1sr$task=="W"]
    • stim[task=="W"] should be e1sr$stim[e1sr$task=="W"]
    • lines 39 - 41
  • my version requires describeBy(), not describe.by()
    • lines 39 - 41
  • This is linux. The levene.test() in package(car) only returns the median. To get the mean, use package(lawstat)
    • Lines 5 and 19.
• In the lecture, Professor Conway said they we have to reverse S and D in "levels=c("S","D")", line 12. Otherwise, the graph will be backwards. I did, it didn't change anything. The bars are still backwards from the ones he shows on his slides.

#########
# stats1.ex.08.R, Macnamara, Moore, & Conway (2011), 
# Experiment 1, Serial recall
#########
library(psych)
library(lawstat)
source(file="eta_squared.R")

e1sr <- read.table("stats1.ex.08.txt", header = T)

# Omnibus analysis is a 3x2 mixed factorial with task
# and stimuli as the independent variables and serial
# recall as the dependent variable. The three levels 
# of task are word span, reading span, and story span. 
# The two levels of stimuli are phonologically similar 
# and phonologically dissimilar.

stim = factor(e1sr$stim,levels=c("S","D"))  
	#reverse levels (for graphs like the article)

aov.e1sr = aov(e1sr$recall ~ (e1sr$task*e1sr$stim) + 
	Error(factor(e1sr$subject)/e1sr$stim))

summary(aov.e1sr)

eta.2(aov.e1sr, ret.labels=TRUE)

# levene.test()
levene.test( e1sr$recall, e1sr$task, location=c("mean") )

# Simple effects analysis for simple span (i.e., word span)
aov.e1srw = aov(e1sr$recall[e1sr$task=="W"] ~ 
	e1sr$stim[e1sr$task=="W"] + 
	Error(factor(e1sr$subject[e1sr$task=="W"]) / 
	e1sr$stim[e1sr$task=="W"]))
summary(aov.e1srw) 
eta.2(aov.e1srw, ret.labels=TRUE)

# Simple effects analysis for complex 
# span (this is a 2x2 mixed factorial)
aov.e1srnw = aov(e1sr$recall[e1sr$task!="W"] ~ 
	e1sr$task[e1sr$task!="W"] * 
	e1sr$stim[e1sr$task!="W"] + 
	Error(factor(e1sr$subject[e1sr$task!="W"]) / 
	e1sr$stim[e1sr$task!="W"]))
summary(aov.e1srnw) 
eta.2(aov.e1srnw, ret.labels=TRUE)

#######

#Graph
# Bar plot

wspan = describeBy(e1sr$recall[e1sr$task=="W"], 
	group = e1sr$stim[e1sr$task =="W"], mat = T)
rspan = describeBy(e1sr$recall[e1sr$task=="R"], 
	group = e1sr$stim[e1sr$task =="R"], mat = T)
sspan = describeBy(e1sr$recall[e1sr$task=="S"], 
	group = e1sr$stim[e1sr$task =="S"], mat = T)
graphme = cbind(Words = wspan$mean, Sentences = 
	rspan$mean, Stories = sspan$mean)
rownames(graphme) = c("Phonologically Similar", 
	"Phonologically Dissimilar")
se = cbind(wspan$se, rspan$se, sspan$se)

bp = barplot( graphme, beside = TRUE,
        ylim = c(0,1), space = c(0, .5), legend.text = TRUE, 
        args.legend = c(x = "topright") )
abline(h=0)
for (ii in 1:3) 
{ arrows(bp[1, ii], graphme[1,ii] - se[1, ii],
	y1 = graphme[1,ii] + se[1, ii], angle = 90, code = 3)
  arrows(bp[2, ii], graphme[2,ii] - se[2, ii],
  	y1 = graphme[2,ii] + se[2, ii], angle = 90, code = 3)
}

Lecture 18: SUMMARY

• Experimental design.
  • Randomized controlled experiments.
  • Different types of designs in ANOVA lectures.
• Descriptive statistics.
  • Histograms.
  • Summary statistics.
• Correlation & regression.
  • If two variables are correlated, I can use one to predict the other.
• Measurement.
  • Bias.
  • Reliability.
  • Validity.
• Multiple regression.
  • Matrix algebra.
  • Estimation of coefficients.
  • Mediation.
  • Moderation.
• Central limit theorem.
  • Sampling.
  • Sampling error.
  • Standard error.
• NHST.
  • Flaws.
  • Alternatives:
    • Effect size.
    • Model comparison.
• Tests to compare means.
  • Use t-tests if you only have two means.
  • Use ANOVA if there are more than two.

WEEK 6 ASSIGNMENT

• We now return to cognitive training. Suppose we conducted a training experiment in which subjects were randomly assigned to one of three conditions:
  • Working Memory training (WM)
  • Physical Exercise training (PE)
  • Designed Sports training (DS)
• Further assume that we measured spatial reasoning ability before and after training, using two separate measures:
  • SR1 (pretraining)
  • SR2 (posttraining)
• Fictional data are available in the file: DAA.05.txt or daa.05.txt on this computer.
• Write an R script to answer the following questions (the main analysis should be a 3x2 mixed factorial ANOVA). All values should be given with two decimal places.

1. Our theory predicts a significant interaction between time and condition. What is the F ratio for the interaction?
   • 11.51
2. One assumption of the analysis is homogeneity of variance. What is the F ratio for Levene’s test?
   • .62
3. What is the F ratio for the simple effect of time for the WM group?
   • 19.29
4. What is the F ratio for the simple effect of time for the PE group?
   • .04
5. What is the F ratio for the simple effect of time for the DS group?
   • 32.45

# Statistics One, Assignment 6
# A 3x2 mixed factorial ANOVA
# to analysis 3 working memory conditions: WM, PE, DS,
# in pre (SR1) and post (SR2) training.
library( psych )
library( lawstat)
source( file="eta_squared.R" )

ass6 <- read.table( "daa.05.txt", header = T )

aov.ass6 = aov( ass6$SR ~ ( ass6$condition * ass6$time ) + Error( factor( ass6$subject ) / ass6$time) )

summary( aov.ass6 )

levene.test( ass6$SR, ass6$condition, location=c("mean") )

aov.ass6.WM = aov( ass6$SR[ ass6$condition=="WM" ] ~ ass6$time[ ass6$condition=="WM" ] + Error( factor( ass6$subject[ ass6$condition=="WM" ] ) / ass6$time[ ass6$condition=="WM" ] ) )

aov.ass6.PE = aov( ass6$SR[ ass6$condition=="PE" ] ~ ass6$time[ ass6$condition=="PE" ] + Error( factor( ass6$subject[ ass6$condition=="PE" ] ) / ass6$time[ ass6$condition=="PE" ] ) )

aov.ass6.DS = aov( ass6$SR[ ass6$condition=="DS" ] ~ ass6$time[ ass6$condition=="DS" ] + Error( factor( ass6$subject[ ass6$condition=="DS" ] ) / ass6$time[ ass6$condition=="DS" ] ) )

summary( aov.ass6.WM )

summary( aov.ass6.PE )

summary( aov.ass6.DS )

FINAL EXAM