> For the complete documentation index, see [llms.txt](https://docs.quantaria.xyz/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.quantaria.xyz/architecture/research-foundations.md).

# Research Foundations

Quantaria combines methods that have each been studied in published work. The figures below are **results from those papers, not Quantaria benchmarks.** They show what each method can do in the setting the authors tested. Quantaria's own results will be published as each model is evaluated.

## Published results

| Result (as reported in the paper)                                                            | Method                                                                                   | Source                                               |
| -------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | ---------------------------------------------------- |
| 200,000x compression of a single dense layer                                                 | Tensor-train factorization                                                               | [arXiv:1509.06569](https://arxiv.org/abs/1509.06569) |
| 14,000x fewer parameters in MERA layers at under 1% accuracy loss, on image classifiers      | MERA layers                                                                              | [arXiv:1711.03357](https://arxiv.org/abs/1711.03357) |
| \~3,000x fewer classical parameters in a hybrid quantum attention layer at +0.26% perplexity | Hybrid quantum attention with disentanglers; part of inference on an IBM Heron processor | [arXiv:2410.17397](https://arxiv.org/abs/2410.17397) |
| 93% memory reduction on LLaMA-7B at a 2-3% accuracy cost                                     | Tensor-network compression                                                               | [arXiv:2401.14109](https://arxiv.org/abs/2401.14109) |
| 27x fewer parameters than Switch Transformer                                                 | MPO mixture-of-experts                                                                   | [arXiv:2203.01104](https://arxiv.org/abs/2203.01104) |
| \~10x smaller KV cache                                                                       | Tensor Product Attention                                                                 | [arXiv:2501.06425](https://arxiv.org/abs/2501.06425) |
| 9x more memory-efficient training than GaLore, with 2-3x faster epochs                       | CoMERA rank-adaptive training                                                            | [arXiv:2405.14377](https://arxiv.org/abs/2405.14377) |
| \~40% fewer parameters at near-baseline perplexity for GPT-2                                 | Training in tensor-train form                                                            | [arXiv:2306.02697](https://arxiv.org/abs/2306.02697) |

## Methods Quantaria builds on

| Quantaria component              | Basis                                                                            |
| -------------------------------- | -------------------------------------------------------------------------------- |
| Unified text and image objective | Transfusion-style training, [arXiv:2408.11039](https://arxiv.org/abs/2408.11039) |
| Tensor Product Attention         | [arXiv:2501.06425](https://arxiv.org/abs/2501.06425)                             |
| MPO mixture-of-experts           | [arXiv:2203.01104](https://arxiv.org/abs/2203.01104)                             |
| Rank-adaptive training           | CoMERA, [arXiv:2405.14377](https://arxiv.org/abs/2405.14377)                     |
| MPS-to-circuit preparation       | Schön et al., [arXiv:quant-ph/0501096](https://arxiv.org/abs/quant-ph/0501096)   |
| Disentangler-based hybrid layers | [arXiv:2410.17397](https://arxiv.org/abs/2410.17397)                             |

## How to read these numbers

* Each result was obtained on the paper's own model, dataset and setup. It does not predict a Quantaria model's performance.
* Results differ in what they measure: parameters, memory, cache size, accuracy or perplexity.
* Quantaria compares each of its models with a dense transformer of the same size on the same data, and publishes the outcome either way.

## Related

* [Tensor-Network Weights](/architecture/tensor-network-weights.md)
* [Quantum Path](/architecture/quantum-path.md)
* [Roadmap](https://github.com/bhavy2908/dataai_docs/tree/main/overview/roadmap.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.quantaria.xyz/architecture/research-foundations.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.