Christophe MontagneInterim CTO · Advisory

What Claude, Gemini and ChatGPT actually are/1 min read

08Don't rebuild it: the usual libraries

The open-source stack, from PyTorch to LlamaIndex, and the provider APIs that skip it.

Very few organisations will ever train a transformer from scratch, and none should implement one by hand. A mature ecosystem of open-source libraries covers every layer, and most projects start near the top of it, or skip it entirely by calling a provider's API.

The usual librariesA stack from hardware to applications, with provider APIs as an alternative.Applications and RAGLlamaIndex, LangChainRunning modelsvLLM, llama.cpp, OllamaModels and tokenizersHugging Face Transformers, Tokenizers, Sentence-TransformersFoundationsPyTorch, JAXHardwareGPUs and other specialised chipsProvider APIsor skip the stack:call a hosted modelMost organisations start near the top of the stack, or use an API
  • Foundations. PyTorch[27], and to a lesser extent JAX[28]: the frameworks almost every model is built and trained with.
  • Models and tokenizers. Hugging Face Transformers[29] is the de facto standard for downloading, running and fine-tuning pretrained models in a few lines of code, with its Tokenizers library handling the step described in chapter 05. Sentence-Transformers[30] turns whole sentences into vectors, the basis of most document search and RAG systems.
  • Running models. vLLM[31] serves models efficiently on your own servers; llama.cpp[32], and Ollama[33] built on top of it, run them on ordinary hardware, down to a laptop. These matter when data must stay on your premises.
  • Applications. LlamaIndex[34] and LangChain[35] handle the plumbing of RAG and agent systems: ingesting documents, cutting them into chunks, turning them into vectors, retrieving the relevant ones and passing them to a model.

A naming trap: Llama, LlamaIndex and llama.cpp are three unrelated things. Llama is a family of models from Meta[36], released with open weights: one of several engines you can plug into such a system, alongside Mistral, Qwen or DeepSeek. LlamaIndex is an independent framework that orchestrates RAG. llama.cpp is a tool to run models, Llama or others.

One more caution: "open weights" is not the same as open source. Models such as Llama come with licences and usage conditions that change between versions and deserve a careful read before any client project.