Data Preprocessing: Make Your Data Ready for Statistical Analysis
What is Data Preprocessing?
Data preprocessing turns raw data into a form suitable for statistical analysis: handling missing values (deletion or mean or median imputation), standardizing with Z-scores, transforming skewed data with Box-Cox or Johnson transforms, and flagging outliers. The quality of preprocessing directly determines how trustworthy subsequent analysis conclusions are, so it is an unmissable preparation step. Every transformation step is recorded, keeping the process auditable.
When to Use It
Use data preprocessing before control charts, CpK, normality tests or regression, when merging multiple batches or conditions that need consistent definitions, and when normalizing skewed data before routine statistics. It is also useful for producing a clean, uniform data set that downstream tools can consume directly. Standardize the data format first so multiple batches can be analyzed as one set.
How to Use It (Step by Step)
Paste your raw data and choose the operations: fill or drop missing values, apply Z-score standardization, select a Box-Cox or Johnson transform, and flag outliers. The tool shows before and after comparisons: distribution shape and normality P-value before and after transformation, standardized data centered near zero with unit standard deviation, and a clear log of missing-value handling. An AI interpretation explains the effect of each step.
Key Formulas / Example
Z-score standardization: Z = (x - mu) / sigma, which recenters data to a mean of 0 and standard deviation of 1. The Box-Cox transform is y(lambda) = (x^lambda - 1) / lambda for lambda not equal to 0, and y = ln(x) for lambda = 0; it requires positive data, and the tool adds a constant offset automatically when zeros or negatives are present. Keep specification limits in the original scale if you later compute capability, or transform them identically.