Intelligence Density

For more than a decade, the practice of artificial intelligence research has operated under a premise so pervasive that it is rarely examined: that capability is primarily a function of scale. The relationship between model size, training data volume, and benchmark performance has been studied extensively, and the findings have consistently reinforced the intuition that larger is better. Frontier models are defined by their parameter counts. Research budgets are justified by scaling laws. Competitive advantage is measured in training FLOP.

This premise has been productive. The models it has produced are genuinely impressive. But it has also had a narrowing effect on the questions researchers consider worth asking. When scale is the primary lever, the field becomes a resource allocation problem rather than a scientific one. The interesting questions — about the mechanisms of capability, about what intelligence actually requires, about whether the relationship between compute and capability is fundamental or contingent — go systematically underexplored.

This paper proceeds from a different starting point. Rather than asking how to maximise raw capability at any cost, we ask: how much capability can be concentrated into a given computational budget? This is a question about density. It is also a question that requires a measurement framework that current benchmarking practice does not provide.

We introduce the concept of Intelligence Density as a formal measure of this concentration, propose the Intelligence Density Quotient (IDQ) as a composite metric suitable for rigorous comparative evaluation, and describe the measurement methodology that makes such evaluation tractable. The framework developed here forms the theoretical foundation of the CHISEI research programme.

The scaling hypothesis — that model performance improves predictably with increases in model size, data volume, and compute — has significant empirical support. Work on scaling laws has demonstrated consistent power-law relationships between parameter count and performance across a range of tasks and architectures. This regularity has made scaling a reliable engineering strategy and has guided investment decisions across the industry.

However, the empirical strength of the scaling hypothesis at the frontier conceals important limitations. First, scaling laws describe average behaviour across broad benchmark distributions; they do not describe the distribution of capability within a fixed compute envelope. Second, the benchmarks typically used to evaluate frontier models conflate breadth of coverage with depth of capability, making it difficult to isolate the contribution of any specific architectural or methodological choice. Third, and most importantly, scaling laws are predictive within existing paradigms — they do not constrain what might be achievable through approaches that have not yet been explored at scale.

The assumption that scale is the primary driver of capability has a selection effect on research. Work that requires modest compute is less likely to be funded, less likely to produce results that attract attention, and less likely to influence the direction of the field. This is not a consequence of any deliberate policy choice; it is an emergent property of how results are evaluated and resources are allocated. The consequence is that an entire dimension of the capability landscape — the dimension defined by efficiency rather than scale — is systematically underexplored.

We do not argue that scale is unimportant. We argue that it is not the only variable, and that the field has not developed adequate measurement tools to characterise the capability achievable at constrained scales. Developing those tools is the first task of a rigorous research programme in efficient intelligence.

We define Intelligence Density as the ratio of general-purpose capability to the computational resources required to produce and deploy that capability. More precisely, for a model M trained with compute budget C and evaluated across a task set T, the intelligence density D(M, T) measures the breadth and depth of capability relative to the total cost incurred — encompassing training compute, parameter count, and inference cost per query.

This definition has several important properties. First, it is relative: intelligence density is not a property of a model in isolation but of a model in relation to a compute budget. A model that achieves high absolute performance but requires enormous resources may have lower intelligence density than a model with modest absolute performance achieved within a fraction of that budget. Second, it is compositional: the task set T determines which capabilities are measured, and different task sets will yield different density scores. Third, it is agnostic to architecture: the definition does not privilege any particular approach to achieving capability within a budget.

It is important to distinguish intelligence density from related but distinct concepts. Raw parameter efficiency — performance per parameter — captures one dimension of density but ignores inference cost and training compute. Energy efficiency metrics capture deployment cost but not capability breadth. Benchmark compression ratios capture the relationship between training data volume and performance, but not the full compute picture. Intelligence density is a composite that none of these measures adequately captures in isolation.

The concept has a natural interpretation in terms of the research programme it motivates. If intelligence density is the target variable, then the goal is not to achieve the highest possible score on a given benchmark, but to achieve the highest possible score per unit of computational investment. This shifts the research question from "what is the ceiling of capability?" to "where is capability concentrated, and how do we reach it efficiently?"

The Intelligence Density Quotient (IDQ) is a normalised composite score designed to make intelligence density tractable for comparative evaluation across models with different sizes, architectures, and training regimes.

IDQ is computed across four component dimensions. The first is Task Performance Coverage, a breadth-weighted average of performance across a curated evaluation suite spanning reasoning, language understanding, mathematical problem-solving, and code generation. The second is Parameter Efficiency, the ratio of task performance coverage to the logarithm of parameter count, normalised against a reference model of known size and capability. The third is Training Compute Efficiency, performance coverage relative to the total training FLOP budget expressed in petaFLOP-days. The fourth is Inference Cost Efficiency, task performance coverage per unit of inference compute measured in FLOP per output token, normalised for context length.

The four component scores are combined using a geometric mean rather than an arithmetic mean. This choice is deliberate: it penalises models that excel on one dimension at the expense of others, and rewards models that achieve balanced efficiency across the full compute picture. A model with excellent parameter efficiency but poor inference cost efficiency will score lower on IDQ than a model with moderate efficiency across all four dimensions.

IDQ scores are expressed on a normalised scale anchored to a reference baseline. The baseline is defined as the IDQ of a standard open-weight model trained with conventional methodology at the same parameter scale as the model under evaluation. A score above 1.0 indicates higher intelligence density than the baseline; a score below 1.0 indicates lower. This anchoring makes IDQ interpretable as a relative efficiency measure rather than an abstract composite.

It is important to note what IDQ does not measure. It does not capture alignment, safety, robustness, or task-specific specialisation. It is a measure of general-purpose capability relative to compute, not a comprehensive model quality score. Researchers using IDQ should be explicit about this scope and should not treat a high IDQ as sufficient evidence of model quality for deployment purposes.

Reliable measurement of intelligence density requires careful attention to evaluation design. The primary threats to validity are benchmark contamination, distribution mismatch between training and evaluation data, and confounding from differences in evaluation protocol across models. We describe the methodological choices that address each of these threats.

To address contamination, we construct evaluation tasks from held-out data that cannot plausibly appear in any publicly available training corpus. Tasks are generated programmatically from verified-clean seed data using procedures that are documented and reproducible. A subset of evaluation tasks is refreshed at each evaluation cycle to prevent the gradual accumulation of implicit contamination through iterative training on public benchmarks.

To address distribution mismatch, we stratify the evaluation suite across multiple task domains and difficulty levels, and report disaggregated performance in addition to composite scores. This allows researchers to identify domain-specific capability concentrations that would otherwise be obscured by aggregate metrics. It also allows IDQ scores to be recomputed against task subsets of interest, making the composite more adaptable to specific research questions.

Compute costs are measured using standardised accounting procedures. Training compute is reported in total FLOP as reconstructed from architecture specifications and training configurations, following the conventions established in prior compute accounting literature. Inference compute is measured empirically under standardised hardware conditions, with results reported alongside hardware specifications to allow cross-platform comparison. All measurements are reported with confidence intervals derived from multiple evaluation runs.

To ensure reproducibility, we publish the full evaluation suite, measurement code, and reference baseline model alongside this paper. Independent research groups should be able to reproduce IDQ scores for any model for which training specifications are available, and to generate IDQ scores for new models using the published methodology.

The evaluation suite used for IDQ computation consists of four primary task categories, chosen to provide broad coverage of general-purpose capability while remaining computationally feasible to evaluate across many models.

The Reasoning Cluster comprises tasks requiring multi-step logical inference, causal reasoning, and structured problem decomposition. Tasks are drawn from formal reasoning domains where ground-truth evaluation is unambiguous. Performance is measured as a combination of accuracy and solution path quality, where solution paths are evaluated against a reference rubric rather than a single correct output.

The Language Understanding Cluster covers reading comprehension, semantic similarity, discourse coherence, and lexical inference. Tasks range from single-sentence inference to multi-document reasoning. This cluster captures the degree to which a model has internalised the distributional structure of natural language, independent of world knowledge.

The Quantitative Reasoning Cluster addresses mathematical problem-solving across difficulty levels from arithmetic to undergraduate-level proof verification. Performance is evaluated on both final answers and intermediate steps, with partial credit available for correct reasoning that arrives at an incorrect numerical result due to computational error.

The Code Generation Cluster evaluates the ability to produce correct, efficient, and readable code from natural language specifications across three programming languages. Correctness is evaluated using automated test execution; efficiency is measured relative to a reference solution; readability is evaluated by a static analysis rubric that captures naming conventions, modularity, and documentation quality.

Each cluster contributes equally to Task Performance Coverage. Within each cluster, task difficulty is stratified into three tiers, with higher tiers weighted proportionally in the aggregate score. This weighting ensures that IDQ rewards models that handle difficult tasks rather than those that achieve high scores by concentrating capability on easy problems.

The intelligence density framework has direct implications for how AI research programmes should be structured and evaluated. We identify three areas where the framework changes the research landscape in ways that merit explicit discussion.

First, intelligence density as a target variable makes certain research questions tractable that are currently marginalised. Questions about architectural choices that improve capability without increasing parameter count — knowledge distillation, structured pruning, mixture-of-experts with efficient routing, curriculum learning, and synthetic data generation — all become directly relevant to IDQ optimisation. These are not new research areas, but IDQ provides a unifying metric that allows progress across them to be aggregated and compared.

Second, the framework supports a form of research validation that is currently absent from the field. When the primary metric is raw benchmark performance at frontier scale, there is no principled way to compare a small-scale research contribution with a large-scale engineering effort. IDQ creates a common ground: a research contribution that improves intelligence density by 5% at 1.5B parameters is commensurable with a contribution that improves it by the same amount at 70B parameters, and both can be evaluated against the same baseline. This commensurability is a prerequisite for cumulative scientific progress.

Third, the framework raises important questions about the relationship between intelligence density and practical deployability. Models with high intelligence density are, by definition, capable relative to their compute requirements. This makes them candidates for deployment in resource-constrained environments — edge devices, low-latency applications, and contexts where inference cost is a binding constraint. The research programme motivated by IDQ therefore has a natural connection to the engineering problems of efficient deployment, creating a feedback loop between measurement, optimisation, and application.

This paper has introduced Intelligence Density as a formal framework for measuring the concentration of general-purpose AI capability relative to computational resources. We have proposed the Intelligence Density Quotient as a tractable composite metric, described the evaluation methodology required to measure it reliably, and argued that adopting IDQ as a primary research target opens research directions that the current scale-dominated paradigm systematically forecloses.

Intelligence density is not a replacement for capability as a research objective. Frontier models are important. But they are not the only models, and maximising raw capability at maximum cost is not the only research programme available. We believe that a rigorous science of intelligence density is both achievable and valuable, and that the field will benefit from the development of measurement tools, baselines, and evaluation standards that make comparative progress possible.

The CHISEI programme at SHINGETSU is the initial empirical instantiation of this research direction. CHISEI-1.5B-R0 will apply the IDQ framework to a 1.5-billion-parameter language model, testing whether deliberate architectural and methodological choices can yield intelligence density scores that challenge models an order of magnitude larger. The findings — including negative results — will be published and will form the empirical foundation for subsequent research in the programme.

We invite independent researchers to use the IDQ framework and its associated evaluation suite, to validate or challenge our methodology, and to contribute to the development of measurement standards for efficient intelligence. Progress in this area depends on shared tooling and shared norms. We offer this paper as a contribution to that shared foundation.