thinletter demo: a query encoder as a 100–350 MiB file in your browser

Mirror of https://thinletter.io/demo. Search 25 656 scientific abstracts (SciDocs, CC BY 4.0) with a compressed query encoder that runs in the browser: the scalar llama.cpp file (wllama, WebAssembly/WebGPU) or the vector-quantised container (our open WebGPU runtime, vqweb/). The corpus, the fp32 index and the model chunks are downloaded once from thinletter.io and cached in the browser; no query leaves your machine. Spaces cannot send the COOP/COEP headers the multi-threaded WebAssembly build needs, so the scalar client is slow here; pick a vq client for the WebGPU path.

Code and recipe: https://github.com/rosecky/embedding-quantization-public (Apache-2.0). Models: https://huggingface.co/thinletter. Contact: info@thinletter.io