Experimental Data Analyzer

Statistical notes

These notes summarize the statistical definitions and interpretations used by the calculator.

Introduction

Statistical analysis is useful when experimental observations vary from one measurement to another. An experiment may be regarded as deterministic, when the observable is expected to give the same value under the same conditions, or random / non-deterministic, when repeated experiments may produce different outcomes under the same conditions.

A collection of observations can be regarded as a sample from a population. In practice, experimental limitations usually prevent the entire population from being observed, so a sample is used to obtain estimates such as the mean and variance of the population. When population values have an equal chance of being included, the sample is described as random and the resulting quantities are sample estimates.

Numerical descriptive measures help summarize the relative-frequency distribution of a data set. They include measures of central tendency, which describe the centre; measures of variation, which describe spread; and measures of relative standing, which describe the position of an observation within the data set.

Quantities computed from a sample are called statistics, whereas corresponding quantities describing a population are called parameters. It is also useful to distinguish model-independent procedures, such as descriptive statistics and correlation, from model-dependent procedures, such as least-squares fitting.

Sample mean

The sample (arithmetic) mean estimates the value around which the observations centrally cluster.

• The sample mean estimates the value around which central clustering occurs.
• For distributions with very broad tails, the mean may converge poorly as the number of sampled points increases and can be particularly sensitive to very large or very small observations.

Sample variance and standard deviation

The calculator uses the sample variance and sample standard deviation. For n observations:

The use of n − 1 in the sample variance provides an unbiased estimate of the population variance under the stated sampling convention.

An alternative computational form of the variance can reduce the effect of rounding errors. More generally, if a distribution has no finite second moment, variance and standard deviation are not useful measures of its width and estimates may fail to converge consistently as more observations are collected.

For approximately mound-shaped (for example, Gaussian) distributions, the familiar rules of thumb are about 68% within one standard deviation, 95% within two, and almost all observations within three standard deviations of the mean. These are distribution-dependent rules of thumb, not guarantees for arbitrary data.

Skewness and kurtosis

Skewness and excess kurtosis are defined here from standardized central moments:

• Skewness describes the asymmetry of a distribution around its mean. Positive skewness indicates a tail extending toward larger values; negative skewness indicates a tail extending toward smaller values.
• A finite sample can produce non-zero skewness even when the underlying distribution is symmetric. Interpretation therefore benefits from considering the sample standard deviation and the influence of the distribution's tails.
• The mean is sensitive to extreme observations and tends to shift toward the direction of the skewed tail.
• Kurtosis is dimensionless and describes the relative peakedness or flatness of a distribution. The subtraction of 3 makes the normal distribution have excess kurtosis of zero under this convention.
• Positive excess kurtosis is described as leptokurtic, negative excess kurtosis as platykurtic, and zero excess kurtosis as mesokurtic.
• Higher moments are generally less robust than lower moments and can be strongly affected by extreme observations.

Bias-corrected sample skewness and excess kurtosis are also reported separately and explicitly labelled, so the two conventions are not silently conflated.

Least squares and regression

Regression can be used to estimate the mean value of a dependent response y or predict a future value of y from one or more related independent variables. The principal model here is simple linear regression relating y to a single independent variable x.

The regression model describes the mean value of y for a given x by a straight line of means, with individual observations deviating from that line by random errors. The fitted prediction equation is:

The usual simple linear regression assumptions include zero mean error, constant error variance, normally distributed errors, and independence of errors.

The least-squares method selects the fitted line by minimizing the sum of squared deviations between observed and predicted values. This is a stronger criterion than merely minimizing the signed sum of deviations, because different lines can have errors whose signed sum is zero. A straight line can be fitted to any set of points even when the underlying relationship is not linear.

The fitted coefficients act as estimators: the intercept estimates the point where the fitted line crosses the y-axis, while the slope describes the change in the fitted response associated with a change in x. For simple linear regression, the error-variance estimate is associated with n − 2 degrees of freedom.

The calculator reports the Pearson correlation coefficient, coefficient of determination (R²), residuals, Sum of Squared Errors (SSE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE), and Mean Absolute Error (MAE) to describe the fitted model.

• The Pearson correlation coefficient measures the strength and direction of a linear association and ranges from −1 to +1.
• A high correlation does not by itself establish that changes in x cause changes in y.
• The coefficient of determination R² describes the proportion of variation in the response accounted for by the fitted model under the usual regression interpretation.

Chi-squared statistics

A chi-squared calculation for a fitted line can be expressed using the following non-standard form, with the sum of observed and expected values in the denominator, together with an incomplete-gamma significance calculation.

The significance calculation uses the incomplete gamma function with a degrees-of-freedom parameter described as the number of observations minus the number of constraints. Except for special integer cases, the gamma integral does not have a closed-form expression and its cumulative distribution must therefore be obtained by approximation.

When applied to continuous observations versus regression predictions, this expression is not treated as a statistically valid regression goodness-of-fit test. Regression diagnostics therefore use R², SSE, MSE, RMSE, and MAE. A chi-squared goodness-of-fit feature should use an explicitly specified count/expected-frequency or measurement-error model.

The statistical notes define the conventions used by the calculator and distinguish the non-standard chi-squared expression above from a conventional Pearson goodness-of-fit statistic.