Frequently asked questions

Yes. Try a limited conversation on our home page. For more extensive use, credits can be purchased at https://elastix.ai/dashboard.

Go to elastix.ai/dashboard and sign in to continue with paid usage.

No. Tokens as a Service gives you hosted inference through chat and a familiar API. For on-premise deployments, the PyTorch-compatible flow is designed to replace your existing GPU flow without requiring you to program an FPGA.

The accelerator itself — the compute datapath specialized to one model — not the model's weights or program. Parameterized RTL building blocks are resolved to the target model at compile time and synthesized into a model-specific bitstream. Above the fabric sits a fixed, narrow ISA, so the software stack stays unified across every model.

Both trends push inference toward memory-bound — exactly where a right-sized FPGA datapath beats a compute-heavy GPU. Each card carries ~60 GB HBM; at 4-bit a 1T-parameter model fits across ~10 cards, and ~32 cards interconnect into one fabric for multi-trillion-parameter models. During long-context decode a GPU's compute sits largely idle (dark silicon); ElastixAI provisions bandwidth and capacity to what decode actually demands.

With the current generation of native multi-Tbps links on the FPGAs, cards talk directly over very low-latency point-to-point serial links with no switch. This generation’s upper limit is at least a ~10T model.

Any open- or closed-source model runs well on ElastixAI reconfigurable inference processors. Of course, any closed-source model must be appropriately licensed, but that’s a contractual not a technical issue.

For customers, it takes essentially zero extra effort compared with GPUs. On the ElastixAI side: fine-tunes of supported models require no work; new architectures built from known primitives require none either, since vLLM and PyTorch dispatch to the existing kernel library; genuinely novel primitives take a few days. It is never a blocker — compare that with a multi-year silicon respin for a fixed-silicon vendor.

The customer never touches the FPGA. They keep PyTorch and vLLM; everything FPGA-specific is abstracted below the ElastixAI kernel layer. Model-to-RTL mapping is automated by our compiler.

Modern devices (Altera Agilex, AMD Versal) are no longer pure programmable fabric — they contain hardened, ASIC-class blocks, so the historical FPGA-versus-ASIC overhead is far smaller than decade-old comparisons suggest. The FPGAs ElastixAI targets are roughly an order of magnitude cheaper than a B200 per GB of HBM and per GB/s of bandwidth — the two features that drive inference TCO.

The majority comes from ML-level optimizations, but the three are interdependent — the gains are only realized when they are co-designed. ML optimization sets the ceiling; the runtime, compiler and FPGA architecture determine how much of it is captured.

Three things compound. A bespoke processor per model wastes no silicon or power on generality. Hardware support lets the model use modern ML efficiency techniques, so each token costs fewer operations. And the serving framework is co-tuned to both. Energy is OpEx and a hard physical constraint, which makes this a CFO-level argument as much as an engineering one.

Two inflection points. FPGAs now beat GPUs on cost per GB of memory connected and per GB/s of bandwidth — the features that matter most for inference TCO. And ElastixAI has automated FPGA reconfigurability in a fully PyTorch-compatible flow to make the use of FPGAs instead of GPUs completely transparent. No FPGA knowledge whatsoever is needed to use Elastix reconfigurable processors.

Largely yes, and in some cases we go further. Custom GPU kernel tricks exist because GPUs are fixed hardware; on FPGAs we optimize the hardware logic itself, going beyond what kernel optimization can reach. Speculative decoding, including purpose-trained draft models, is already supported in our stack. Disaggregated serving applies directly, and we particularly shine when paired with GPUs doing prefill.

Two device families, both current-generation and both HBM parts: AMD Xilinx Versal Premium Plus HBM, and Altera Agilex 7M. These are the top of each vendor's high-end line, not legacy UltraScale-class fabric.

No. Everything runs on the FPGA cards today, prefill and decode alike, with no GPU dependency in the production stack. Separately, we have development work in flight on finer-grained disaggregation that would offload selected parts of a workload — not whole prefill or whole decode — to GPUs in the same box. That is not in production yet.

No inference traffic crosses PCIe. PCIe is used only for host-to-card work, such as loading a model onto a box. Cards talk to each other over direct FPGA-to-FPGA links using MCIO over the devices' SerDes, at multiple terabits per second with extremely low latency, using hard IP supplied by the FPGA vendors rather than a TCP/IP stack. It is not limited to one chassis: the fabric spans nodes.

You absolutely can. It is the normal trade-off: raise concurrency and throughput goes up while per-user latency degrades. Because card CAPEX is low and buyers pay a premium for interactivity, running each box at low concurrency is often the better economic point — and with an MoE model, low concurrency activates fewer experts, which behaves like a virtual increase in memory bandwidth. But nothing in the architecture forces low concurrency.

Wear is driven by fabric utilisation and clock frequency, not by how often you reconfigure. Reconfiguration is a short, infrequent operation — a roughly 100 MB bitstream file, a few times a day at most — and does not consume device life. Manufacturers rate these parts for roughly 15–20 years, and we run them at low fabric utilisation.

The ASIC logic held when models were stable. Today the dominant gains come from fast-cadence ML and system innovations — dense to MoE, FlashAttention, FP16 down to ternary, KV-cache compression, sparsity — and each one changes the ideal chip, while silicon design is locked two to three years out. Once memory movement dominates, the ASIC's edge shrinks. The likely end state is hybrid: fixed silicon for invariant primitives, reconfigurable compute for the fast-moving layer. ElastixAI targets that layer.

Our CEO wrote the first paper on practical quantization, Xnor-Net, in 2016; our CTO built custom ML processors at Apple; our Chief Scientist led ML optimization teams at Google, Waymo and Apple. At Xnor.ai before its acquisition by Apple, our CTO and CEO implemented the first efficient, practical DNN to do multi-object recognition on an FPGA in a highly power-constrained environment. Collectively we’ve been thinking about ML and FPGAs for decades.

Kubernetes schedules containers across a fleet, autoscales, does zero-downtime rollouts and self-heals. ElastixAI slots in exactly where GPU serving does: the serving engine is vLLM-compatible and runs in a container, and FPGA cards are exposed to the cluster as a schedulable resource just like GPUs. A customer already on Kubernetes keeps their orchestration layer unchanged and simply schedules pods onto ElastixAI nodes instead of GPU nodes.

Commodity off-the-shelf servers with one to ten PCIe 5 cards per server, air-cooled. The unit of power increment is 0.5–3 kW per box, lower than most GPU servers, so it fits any rack power plan without a redesign. FPGA TDP is under 200 W each, and real-world draw is around 100 W. Elastix is happy to recommend servers, FPGAs, and other components in your inference racks.

Yes, just as GPUs can. An FPGA can load and be running a new model in under a second.

Not per model. For each FPGA a customer owns, they pay a monthly per-card software licence covering the software stack and access to the most up-to-date bitstreams, so every newly supported model is included rather than charged for. That licence fee is built into the TCO comparison against a GPU. A new model release is reconfigured onto existing hardware in hours, and the subscription is what buys that.

GPU inference fleets are typically depreciated over about four years. We expect an FPGA's useful life to be meaningfully longer, because reconfigurability means the card is not obsoleted by a new model architecture the way fixed silicon is — and manufacturers rate these parts at 15–20 years. It is worth modelling, since it changes the shape of the depreciation schedule in a hardware-ownership case.