Statistics is the discipline of organizing data obtained through observation and measurement, investigating patterns and relationships within those data, and developing knowledge that can support understanding, judgment, and prediction. Calculating means and standard deviations, performing statistical hypothesis tests, and developing predictive equations through regression analysis are all parts of statistics, but none of these activities is its ultimate purpose.
The real world contains variation and uncertainty. Products manufactured under nominally identical conditions may differ in quality. Consumers may respond differently to the same advertisement. Patients receiving the same treatment may experience different outcomes. Statistics provides methods for examining such phenomena: What patterns exist? Could an observed difference be explained by random variation? Which conditions are associated with particular outcomes?
An important characteristic of statistics is that it attempts to obtain knowledge from limited observations while also considering the uncertainty associated with that knowledge.
Statistics is not simply a technique for producing definitive numerical conclusions. It is also a way of clarifying what the available data can tell us, and what remains unknown.
Karl Pearson’s The Grammar of Science — How Does Science Describe the World?
An interesting starting point for understanding the significance of statistics is The Grammar of Science (1892), written by the British mathematician and statistician Karl Pearson (1857–1936).
This book is not a textbook of statistical techniques. It is a work of philosophy of science that examines the nature of scientific knowledge, the meaning of scientific laws, and what can legitimately be inferred from observed facts.
Pearson argued that scientific laws should not be understood as commands governing nature, but rather as concise descriptions of regularities observed in phenomena. Even if a particular sequence of events has been observed repeatedly, this does not establish with absolute certainty that the same relationship must hold under every future circumstance.
For example, repeated observations may reveal a relationship between temperature and the expansion of a material. Under suitable conditions, this relationship can be expressed mathematically.
However, the equation is a model describing the phenomenon within certain observational ranges and assumptions. It does not necessarily remain valid under radically different conditions.
Pearson’s philosophy was strongly influenced by positivism. He emphasized that science describes how phenomena occur rather than providing ultimate explanations of why they occur.
This philosophical position is not universally accepted in contemporary philosophy of science. The nature of causality, scientific realism, theoretical explanation, and the relationship between models and reality continues to be debated.
Nevertheless, the distinction between observed facts and the concepts or models constructed to describe them remains essential to understanding statistics.
Pearson subsequently made major contributions to the development of modern statistics, particularly in the areas of correlation, regression, and the chi-squared test.
Considering his statistical achievements alongside the philosophical questions raised in The Grammar of Science helps us understand statistics as more than a collection of computational methods.
It can be viewed as a discipline concerned with how we describe the observed world and how far we can reasonably extend our conclusions beyond those observations.
What Is Statistics Used For?
Statistics serves many purposes, but its major applications can be organized according to the questions we want to answer.
| Purpose | Main Question | Representative Methods |
|---|---|---|
| Description | What are the characteristics and distribution of the data? | Mean, standard deviation, histogram |
| Comparison | Are there differences between groups or conditions? | Mean comparisons, ANOVA, statistical tests |
| Discovering Relationships | How are two or more variables related? | Correlation analysis, regression analysis |
| Prediction | Can we predict an unknown outcome using available information? | Regression models, time-series analysis |
| Classification | How can observations be assigned to different groups? | Discriminant analysis, cluster analysis |
| Discovering Structures | What underlying characteristics are present among multiple variables? | Principal component analysis, factor analysis |
| Causal Investigation | What happens to an outcome when conditions are deliberately changed? | Design of Experiments, causal inference |
| Decision-Making | Which choice is preferable under uncertainty? | Statistical decision theory, predictive models |
These categories are intended to help beginners understand the field. They are not independent or mutually exclusive branches of statistics.
For example, regression analysis can be used both to describe relationships and to make predictions. Data obtained from designed experiments may also be analyzed using regression models.
The important first step is not to choose a statistical technique, but to clarify what we want to discover.
Descriptive and Inferential Statistics — Summarizing Data and Reasoning Beyond Observations
In introductory statistics, the field is commonly divided into descriptive statistics and inferential statistics.
Descriptive Statistics deals with methods for summarizing the characteristics of observed data.
Suppose a factory measures the weight of 100 manufactured products.
We can calculate the mean weight, standard deviation, maximum, and minimum values. A histogram can show where the measurements are concentrated, whether the distribution is approximately symmetrical, and whether unusually large or small values are present.
Even if two manufacturing processes have the same average product weight, their implications for quality control may differ substantially if one process produces much greater variation.
Therefore, understanding data requires examining not only their central tendency but also their distribution and variability.
Inferential Statistics concerns methods for drawing conclusions about populations or underlying processes using limited observations.
For example, a manufacturer cannot necessarily subject every product to a destructive strength test.
Instead, a sample of products may be selected and tested to estimate the quality of the broader production process.
The entire set of objects or outcomes of interest is called the population, while the subset actually observed is called the sample.
When drawing conclusions about a population from a sample, the sampling procedure is critically important.
A randomly selected sample and a sample assembled from convenient or easily accessible observations may support very different levels of generalization.
For example, measurements taken exclusively from products manufactured by one particular machine may not adequately represent the quality of an entire factory.
Increasing the sample size does not automatically eliminate bias caused by an inappropriate sampling procedure.
Thus, inferential statistics depends not only on mathematical calculations but also on how the data were obtained.
Statistical Hypothesis Testing and Confidence Intervals — How Much Confidence Can We Place in Observed Results?
Suppose two manufacturing methods, A and B, are compared.
The products made using Method A have an average strength of 50 MPa, while products made using Method B have an average strength of 53 MPa.
At first glance, Method B appears to produce stronger products.
However, individual product strengths vary. If only a few products were tested, the observed difference of 3 MPa might partly reflect random variation in the sampled products.
Statistical methods can help evaluate this uncertainty.
Statistical hypothesis testing begins with a hypothesis, such as the assumption that the population mean strengths associated with the two manufacturing methods are equal.
A statistical test then evaluates how unusual the observed data would be under that hypothesis and the assumptions of the statistical model.
A commonly reported result is the p-value.
However, a p-value does not represent the probability that the hypothesis is true. Nor does a small p-value automatically mean that an observed difference is practically important.
A confidence interval expresses uncertainty about an estimated quantity, such as a population mean or the difference between two population means.
For example, suppose the estimated difference between the methods is 3 MPa, with a 95% confidence interval from 1 to 5 MPa.
This is a hypothetical example.
The interval provides information about the uncertainty associated with the estimated difference. In the conventional frequentist interpretation, the confidence level refers to the long-run coverage of the interval construction procedure, rather than the probability that a fixed population parameter lies in a particular observed interval.
Even if a statistically meaningful difference is identified, practical judgment is still required.
Is an improvement of approximately 3 MPa important for the intended product application? Does it justify additional manufacturing costs?
Statistics provides information relevant to decisions, but it does not automatically determine what should be considered valuable.
From Correlation and Regression to Multivariate Analysis
After understanding descriptive statistics and the basic principles of statistical inference, examining relationships between variables provides a natural introduction to a wider range of statistical methods.
For example, to investigate the relationship between processing temperature and product strength, we can construct a scatterplot with temperature on the horizontal axis and strength on the vertical axis.
If strength tends to increase as temperature increases, the variables may exhibit a positive correlation.
Correlation analysis provides numerical measures of relationships between variables. Regression analysis represents such relationships using mathematical models that can also be used for prediction.
A basic linear regression model has the following form:
Y = a + bX + Error
Here, Y is the outcome variable, X is the explanatory variable, and a and b are coefficients estimated from the data.
However, real-world outcomes rarely depend on just one variable.
Product strength may be related to processing temperature, processing time, material composition, pressure, and other conditions.
Multiple regression can incorporate several explanatory variables:
Y = a + b₁X₁ + b₂X₂ + b₃X₃ + … + Error
Other analytical objectives require different methods.
If the goal is to classify products as acceptable or defective, discriminant analysis or another classification method may be appropriate.
If the goal is to summarize numerous measurements using a few indicators, principal component analysis may be useful.
If the objective is to investigate common underlying factors behind questionnaire responses, factor analysis may be appropriate.
In Japanese statistical practice, methods involving multiple variables are often collectively discussed under the broad heading of multivariate analysis.
However, descriptive statistics, hypothesis testing, regression, and multivariate analysis do not always form a fixed sequence in actual research.
This progression is simply one way of understanding the expanding possibilities of statistical analysis.
What Is a Statistical Model?
A statistical model is a mathematical or probabilistic representation of relationships within observed data or of the processes assumed to generate those data.
For example, a regression model relating temperature to electricity demand may be used to predict demand under specified temperature conditions.
However, a model is not the real world itself.
A model that predicts electricity demand using only temperature may fail to account for factors such as the day of the week, time of day, industrial activity, or electricity prices.
Even if the model fits historical data well, its predictive performance may decline when economic or social circumstances change.
When using statistical models, it is important to understand which variables are included, the range of observations on which the model is based, and the assumptions underlying its construction.
This point is closely related to the nature of scientific laws discussed by Karl Pearson in The Grammar of Science.
Expressing observed regularities mathematically can be extremely valuable.
However, obtaining an equation does not mean that it represents an absolute law valid under every possible circumstance.
Statistical models may also be used to examine causal hypotheses when combined with appropriate research designs.
What can legitimately be concluded from a model depends not only on its mathematical form but also on how the data were collected and how the investigation was conducted.
Discovering a Relationship Is Not the Same as Identifying Its Cause
One of the most important distinctions in statistics is the difference between correlation and causation.
For example, ice cream sales and the number of heat-related illnesses may both increase during hot weather.
However, this relationship does not establish that increased ice cream sales cause heat-related illnesses.
Temperature may influence both variables.
Statistical analysis involving multiple variables can help investigate such situations.
However, simply adding more variables to a model does not automatically establish causality.
If the objective is to investigate causal effects, it may be useful to deliberately manipulate conditions through controlled experiments.
Design of Experiments (DOE) provides methods for systematically varying conditions and examining individual effects and interactions.
When experiments are impractical or impossible, causal inference methods using observational data may provide alternative approaches.
The central point is that identifying a statistical relationship and explaining why that relationship exists are different analytical tasks.
Statistics Begins Before Data Collection
Statistics is often associated with analyzing an existing dataset using software.
However, the reliability of statistical results depends heavily on decisions made before the data are collected.
Consider a customer satisfaction survey.
What exactly does “customer satisfaction” mean?
Does it refer to perceived value for money, product performance, ease of use, reliability, customer support, or some combination of these characteristics?
If these concepts are not adequately defined before the survey is created, sophisticated statistical analysis may produce results whose meaning remains unclear.
The selection of respondents, wording of questions, response formats, measurement accuracy, and other design choices can all influence the resulting data.
Sampling, measurement, experimental design, and data quality are therefore essential parts of statistical methodology.
After data collection, it is also useful to investigate the characteristics of the data before applying complex models.
This process is known as Exploratory Data Analysis (EDA).
EDA uses graphical and numerical techniques to examine distributions, unusual observations, unexpected patterns, and relationships between variables.
These findings can guide the selection of appropriate statistical models and suggest directions for further investigation.
Rather than immediately forcing data into a predetermined model, exploratory analysis emphasizes learning from the observations themselves.
The Relationship Between Qualitative Observation and Statistics
Because statistics frequently involves numerical data, it is sometimes treated as an activity separate from qualitative observation and interpretation.
However, in practical research and problem-solving, qualitative and statistical approaches can form parts of a continuous investigative process.
For example, a company may begin by interviewing customers to understand their dissatisfaction with a product.
The findings can be organized using the KJ Method or another qualitative technique to identify recurring concerns and possible underlying issues.
These findings may then be used to design questionnaire items intended to measure the frequency or importance of particular concerns.
Once survey data have been collected, descriptive statistics can summarize the responses.
Factor analysis or regression analysis may then be used to investigate underlying evaluation dimensions or relationships with purchasing behavior.
Conversely, if statistical analysis reveals an unexpected relationship, researchers may return to interviews or direct observation to explore possible explanations.
In this way, qualitative and quantitative analysis do not need to be treated as opposing approaches.
Statistics becomes meaningful when it is connected to the activities of identifying questions, gathering evidence, and interpreting results.
Statistics in the AI Era
Modern software can perform a wide range of statistical calculations.
With the development of large language models, people without extensive statistical training are increasingly able to describe their analytical objectives in natural language and receive assistance in selecting suitable methods.
For example, a user might ask:
“How can I determine whether two manufacturing methods have different defect rates?”
An AI-assisted system could explain what data are required and which statistical methods might be appropriate.
Another user might ask:
“Which of temperature, processing time, and material composition affects product strength?”
The system could suggest approaches such as multiple regression or Design of Experiments and explain the information needed for each.
An LLM may also help interpret statistical outputs, including coefficients, charts, confidence intervals, and hypothesis test results.
However, AI does not guarantee the correctness of statistical reasoning.
An LLM may generate a convincing explanation for calculations that were never performed or mistakenly interpret correlation as causation.
Therefore, numerical calculations should be carried out using reliable statistical software, while the LLM assists with clarifying analytical objectives, selecting methods, explaining results, and identifying potential interpretation problems.
More importantly, statistical findings must still be examined critically.
Even an appropriate statistical method can produce misleading conclusions when the data are biased.
A mathematically correct calculation may provide little practical value if the underlying analytical question is poorly formulated.
As AI makes statistical computation more accessible, the ability to ask meaningful questions and evaluate the evidence supporting conclusions becomes increasingly important.
Statistics in ThinkNavi
ThinkNavi treats statistics not merely as a specialized mathematical subject but as a collection of methods for supporting problem discovery, hypothesis formation, relationship exploration, hypothesis testing, prediction, and decision-making.
For example, after organizing problems and possible causes using the KJ Method or a Cause-and-Effect Diagram, users can consider appropriate measurements and apply Design of Experiments to test causal hypotheses.
Multivariate analysis can help investigate relationships among multiple variables, potentially leading to new hypotheses and further investigation.
ThinkNavi aims to introduce methods such as descriptive statistics, correlation analysis, regression analysis, experimental design, and multivariate analysis according to the questions users want to answer.
It should be possible to understand the purpose, required data, typical outputs, and limitations of a statistical technique without first mastering every mathematical derivation.
At the same time, statistics cannot be reduced to automated calculations.
The questions raised by Karl Pearson in The Grammar of Science about the relationship between observed facts and scientific knowledge remain relevant today.
The purpose of statistics is not simply to transform data through calculations. It is to discover regularities and relationships in the observed world, understand the conditions under which that knowledge is valid, and clarify how reliably it can support our conclusions.
Related Topics
- Descriptive Statistics
- Statistical Hypothesis Testing
- Correlation Analysis
- Regression Analysis
- Multivariate Analysis — Discovering Relationships and Structures Among Multiple Variables
- Design of Experiments (DOE) — Investigating Multiple Factors and Their Interactions Through Systematic Experimentation
- The KJ Method | Building Meaningful Structures from Fragmented Information
- Cause-and-Effect Diagram (Fishbone Diagram) — Systematically Organizing Factors That May Contribute to a Problem
- Scientific Method | Hypotheses, Tests, and Discovery
- Abduction | Reasoning Toward Explanations
- Critical Thinking | Examining Claims and Evidence
References and Further Reading
- Karl Pearson, The Grammar of Science, Walter Scott, 1892.
- R. A. Fisher, The Design of Experiments, Oliver and Boyd, 1935.
- George E. P. Box, J. Stuart Hunter, and William G. Hunter, Statistics for Experimenters: Design, Innovation, and Discovery, Wiley, 2005.
- David S. Moore, George P. McCabe, and Bruce A. Craig, Introduction to the Practice of Statistics, W. H. Freeman.
- Cambridge University Press — Karl Pearson, The Grammar of Science
- Biodiversity Heritage Library — The Grammar of Science, Second Edition
- Australian Bureau of Statistics — What Statistics Are
- NIST/SEMATECH — e-Handbook of Statistical Methods
- NIST — Exploratory Data Analysis
- American Statistical Association — Statement on Statistical Significance and P-Values