Optimizing GLM for dual processor Westmere Xeon E5620: 200% gain (3x faster) till now

Running GLM-5.2 on an older dual-socket Xeon E5620 workstation is mainly a performance engineering problem. The model does run, but it does so on a machine with limited memory bandwidth, limited cache capacity, no modern wide vector units and a runtime that has to combine dense transformer work, expert routing and NVMe-backed expert streaming. This means one cannot simply optimize the model in some abstract sense. One has to identify which concrete paths are expensive on the actual machine and then validate every change on the real workload.

This article summarizes the optimization work that has been done on my Colibri fork so far. The focus is on the actual inference path with the model on NVMe storage, measured sequentially, with a fixed prompt and without parallel GLM runs. I am intentionally not using the original repositories tiny test models or synthetic-only benchmarks as the main source of truth, because several apparently reasonable optimizations do not survive contact with the real inference path.

Compared with the older benchmark figures, the current rough result is that the system now reaches about 0.0600 tok/s instead of roughly 0.0200 tok/s in the presently best stable configurations. This corresponds to an overall speedup of about 3 times (approximately 200% gain till now). On modern GPU systems this would still be a slow configuration, but on CPU-only machines it is large enough to make a practical difference.

Not every attempted optimization helped. Some were neutral, some were slower and some overlapped with the still unresolved token-repeat problem that can lead to responses that do not terminate cleanly. I am including those failed attempts as well, because they help to explain which parts of the runtime are actually worth touching and which ideas are probably not promising on this system.

Before discussing the individual changes it makes sense to briefly summarize how the inference flow looks from the perspective of optimization, because names such as qrow_i8, gate/up or attention projection are otherwise not very informative.

The machine that this optimization has been performed on was an dual processor Intel Xeon E5620 (8 physical cores, 16 logical threads) with 20 GByte or DDR3 unbuffered RAM operating on FreeBSD 13.5-RELEASE (p6). The models were stored on an PCIx4 NVMe SSD, the runtime was compiled with clang 19.1.7 using it’s internal -O3 optimizations.

• How The Numbers Were Taken
• What The Runtime Is Actually Doing
• Where We Stand Right Now
• The optimizations
  • FreeBSD RAM Detection And Planning (Success: 0% gain)
  • FreeBSD NUMA Slab Interleave Support (Failed)
  • SSSE3 IDOT Kernels For Non-AVX2 x86 (Success: 26% gain)
  • SSE Activation Quantization For qrow_i8 (Success: 4% gain)
  • RoPE Inverse-Frequency Cache (Success: 7% gain)
  • SSE4.1 RMSNorm Output Pass (Success: 4% gain)
  • SSE4.1 Exact Int4 Attention Projection (Success: 13% gain)
  • Small-n Expert Reuse For Int8 gate/up (Success: 4% gain)
  • Small-n Shared Output Pass For Int8 gate/up (Success: 3% gain)
  • Small-n Direct-Row Gather Removal (Success: 2% gain)
  • SSSE3 4-Way Unroll Experiment (Failed)
  • Forcing 8 OpenMP Threads By Default (Failed)
  • SSSE3 Output-Pair IDOT Driver (Failed)
  • The First Exact Int4 Projection Attempt (Failed)
  • Multi-Row gate/up Fusion (Failed)
  • Thresholded OpenMP For silu(gate) * up (Failed)
  • Broad Int8 gate/up Reuse (Failed)
  • Broad Int8 gate/up Pair Fusion (Failed)
  • Parallel Plain Routing Rows (Failed)
  • sincosf For RoPE (Failed)
  • SSE4.1 qt_addrow On Decode (Failed)
  • SSE4.1 Exact Int8 Attention Projection (Failed)
  • SSE4.1 Expert Accumulation Helpers (Failed)
  • SSSE3 2x2 Int8 IDOT Driver (Failed)
  • Small-n Shared-x Dual-Dot Helper (Failed)
• Conclusion: What Actually Mattered
• References

How The Numbers Were Taken

The important part is not only what was changed, but also how the measurements were taken. Optimization without measurement is worthless.

All runtime measurements were done against the real model on NVMe storage, using the actual coli chat execution path and a fixed prompt. I explicitly avoided parallel GLM runs because the machine simply does not have the headroom for that. Where possible, changes were tested in alternating baseline -> modified -> baseline -> modified sequences so that one-time penalties, cache warmup artifacts, and especially random background noise would not immediately fool the measurement.

For most of the deeper tuning work the prompt was held constant and the benchmark was stopped at a fixed prefill checkpoint (inference was intentionally terminated before full completion to reduce benchmark duration while still gathering meaningful data), which made comparisons slow but trustworthy. Exactness checks were also added for the low-level SIMD paths so that a speedup did not silently change the math.

This distinction turned out to matter quite a lot. Several ideas looked respectable in isolation and still lost once the real model and the real chat path were involved.

What The Runtime Is Actually Doing

At a very high level, each token passes through a sequence of transformer layers. Inside those layers one can roughly separate four kinds of work that matter for this optimization story:

• dense projections for attention and other always-active parts
• positional work such as RoPE
• normalization and elementwise vector operations
• and the Mixture-of-Experts feed-forward path with routing, expert selection and expert matrix multiplications

The last point is the unusual one. GLM-5.2 is not a dense model in which every token activates every feed-forward weight. Instead, a router decides which experts are needed for the current token and only those experts are evaluated. Colibri then tries to keep useful experts in RAM while loading missing ones from NVMe storage when necessary.

This means the forward pass is not just multiplication of a lot of matrices. It is a pipeline that repeatedly does something like the following:

• take the current activations
• normalize and project them for attention
• apply positional rotation
• run routing to decide which experts are needed
• gather or quantize the routed activations
• evaluate the selected expert gate, up and related projections
• combine those results back into the residual stream
• continue to the next layer

Several of the optimized paths sit directly in that sequence. qrow_i8 is part of step 5, where floating-point activations are compressed into int8 rows so that the following integer kernels can work more efficiently. The gate/up paths belong to step 6 inside the selected experts. RoPE belongs to step 3. RMSNorm belongs to step 2 and appears repeatedly across layers. The exact int4 attention projection sits in step 2 as well, in the dense attention-side work that remains active regardless of which experts the router chooses.

There is also an important distinction between prefill and decode. During prefill the model consumes the whole prompt and therefore processes many tokens through many layers at once. During decode it usually extends the answer one token at a time with a warm KV cache. Some optimizations help prefill far more than decode, and some helpers that look interesting on decode barely matter for the larger prompt-ingestion cost. Since the original problem was that the runtime was already very slow while prefilling the first layers, most of the tuning effort naturally focused there.

Once viewed this way, the logic behind the optimization choices becomes less mysterious. One either tries to reduce repeated scalar work in steps that run constantly or one tries to simplify the expert path that dominates large parts of the CPU time on this machine.

Where We Stand Right Now

The latest long running benchmark is the cleanest high-level summary at the moment. The currently stable best runs sit at about 0.0600 tok/s. Note that baseline in the table below is already measured after the optimizations, it is the baseline with respect to the different runtime options that also influence speed massivly.

For the overall historical comparison one instead has to look back to the older benchmark numbers, where all listed configurations were still around 0.0200 tok/s. Relative to that earlier state, the present 0.0600 tok/s result corresponds to roughly a threefold throughput increase.

Configuration	n	tok/s	gen [s]	wall [s]	hit %	RSS [GB]	Speedup
pipe2	16	0.0600	487.0	526.0	16.0	24.6	1.200x
direct_pipe2	14	0.0600	491.0	531.6	15.5	24.6	1.200x
pilot_pipe2	16	0.0600	484.5	523.0	15.5	24.6	1.200x
baseline	15	0.0500	570.0	608.6	15.0	24.7	1.000x
direct	1	0.0500	627.0	669.6	14.0	24.9	1.000x
direct_pilot_pipe2	1	0.0400	515.0	553.7	14.0	24.8	0.800x

The small-n configurations should be treated with care. Most of those runs were terminated because of token repetition and responses that did not stop on their own.

FreeBSD RAM Detection And Planning (Success: 0% gain)

This is not a glamorous optimization, but it had to be fixed.

Before this change, it was necessary to force --ram 19 manually because automatic RAM detection on FreeBSD was not describing the machine correctly. This is not merely inconvenient, it also means the planner may make poor resource decisions and one may end up optimizing around a false view of the available memory. The first step was therefore simply to make the runtime understand the actual memory budget of the operating system it is running on.

There is no direct throughput win to put into a graph here, so I am calling this a 0% speedup. But in practice it was a prerequisite for sane testing, and it removes one of the more annoying pieces of manual babysitting from normal runs.

FreeBSD NUMA Slab Interleave Support (Failed)

This was one of those ideas that made enough architectural sense that it was worth implementing even without a guaranteed speedup.

The machine is NUMA, memory pressure is real and the large slabs are exactly the sort of thing where bad placement can become expensive. So support was added to first-touch large allocations with interleave policy across both NUMA domains. In other words, the runtime can now spread those pages in a controlled way instead of leaving placement entirely to chance.

On the bounded real-model prefill check it did not improve performance. In fact, on that specific run it was slower. So strictly speaking this was a failed speed optimization. Still, I would not call it wasted work. Once the memory configuration changes (much more memory is added) or once larger runs become less memory-starved having the switch already there may become useful.

SSSE3 IDOT Kernels For Non-AVX2 x86 (Success: 26% gain)

This was the first clearly meaningful speedup and one of the easiest to justify.

The local box is old enough that one cannot simply assume newer vector instruction sets, but it still has enough SIMD to do better than the previous scalar-heavy path. So SSSE3 IDOT kernels were added specifically for x86 CPUs that do not have AVX2. The important part was not only to make them fast, but to keep them numerically exact against the existing code.

In practice this path matters because the quantized chat runtime spends a great deal of its time in integer dot products. Once activations have been quantized into int8 rows and the corresponding weights are already stored in low precision, the forward pass repeatedly evaluates a large number of dot products to perform the projection through many quantized weights. If that inner loop is too scalar, the whole model becomes compute-bound very quickly even before storage latency dominates.

SSSE3 is useful here because it gives just enough vector machinery to process several packed small integers at once and to reduce them efficiently without requiring the much newer AVX2 instruction set. On an older Xeon this is the kind of low-level change one would expect to help: it attacks a path that is called constantly and does so using features the CPU already has but the code did not fully exploit before.

That paid off very clearly. On the bounded real-model prefill path this alone improved things by about 1.26x.

SSE Activation Quantization For qrow_i8 (Success: 4% gain)

After the main IDOT work, the next obvious waste was the repeated scalar activation quantization feeding those kernels.

The old path spent too much time turning floating-point activations into the packed int8 rows used later on. That is not flashy work, but when a kernel sits on the critical path often enough, even the boring helper becomes expensive. The change here was therefore simple: use SSE for the max-abs scan and for the packing work instead of leaving the whole thing scalar.

What this means in less implementation-heavy terms is that the runtime first has to compress a normal floating-point activation vector into a quantized form before the cheaper integer arithmetic can begin. If preparing that quantized row is itself slow, one gives back a noticeable fraction of the gain the integer kernels were supposed to provide. In other words, a fast quantized matmul (matrix multiplication) still suffers if its input preparation is lagging behind.

This was therefore a very natural follow-up optimization. The expectation was not that it would revolutionize end-to-end throughput, but rather that it would remove a visible bottleneck sitting directly in front of the newly accelerated IDOT path. That is exactly the kind of surrounding helper one should revisit after improving a core kernel.

This was not a huge change, but it was stable enough to keep. On the real chat path it came out at roughly 1.04x.

RoPE Inverse-Frequency Cache (Success: 7% gain)

This was a nice example of a small structural change beating a more heroic low-level one.

The RoPE code kept recomputing inverse-frequency terms for the same qk_rope/theta combinations over and over again. That is wasteful, especially on a machine where every repeated transcendental operation is more noticeable than it would be on a modern desktop CPU. So rather than trying to be clever inside the trigonometric functions themselves, the better answer was to stop asking for the same values repeatedly.

For readers not living inside transformer kernels every day, this path sits in the positional encoding part of attention. Before the model can compare or mix token states, it rotates parts of the query and key vectors according to their position. Some pieces of that calculation depend only on the model geometry and not on the individual token content. If those pieces are recomputed over and over again, the runtime burns cycles on repeated setup work instead of on the actual per-token math.

This was a good example of why one should first look for repeated work before chasing more exotic tricks. The expectation was straightforward: if a quantity is reused many times and does not change, caching it should cut overhead with relatively little risk. On older CPUs, avoiding unnecessary powf-style work is often a better bargain than trying to force a tiny gain out of already well-tuned arithmetic.

Caching those inverse frequencies per thread turned out to be a clean win. It was not earth-shattering, but a measured 1.07x speedup. And as we will see small gain sums up quickly.

SSE4.1 RMSNorm Output Pass (Success: 4% gain)

The full RMSNorm reduction was not touched here. Only the output pass was.

That matters because it kept the change relatively low-risk. The scalar accumulation stayed as it was, while the final out[i] = x[i] * r * w[i] work was vectorized with SSE4.1. On older x86 hardware that sort of targeted cleanup is often preferable to a grand rewrite because it yields a gain without exploding the complexity or the validation burden.

RMSNorm itself appears all over the model. It is one of those deceptively simple operations that does not look expensive until one sees how often it runs and how wide the vectors are. The reduction part calculates a normalization factor and the output pass then applies that factor together with learned weights across the whole vector. Even if each individual pass is simple the total amount of elementwise work becomes significant across many layers.

The reason this particular optimization was plausible is that the output pass is regular. It is essentially the same formula applied element by element, which maps very naturally to SIMD. Leaving the reduction scalar avoided the more delicate exactness questions while still letting the CPU chew through the wide output loop more efficiently.

The result was modest but real: about 1.04x.

SSE4.1 Exact Int4 Attention Projection (Success: 13% gain)

This one deserves a little more attention because it was not the first attempt at the same general idea.

The attention projection path with exact int4 weights was clearly expensive enough to deserve scrutiny, but the first SIMD version did not help. That could have been the end of the story. Instead, the path was reworked so that the packed int4 decode and accumulation order stayed aligned with the scalar reference while still cutting down the hot loop cost.

This path is easier to understand if one thinks of it as one of the dense projection steps inside attention, except the weights are stored in 4-bit packed form. That compact storage is good for memory traffic but it also means the CPU cannot simply read a convenient array of full integers and multiply away. It first has to unpack nibbles, interpret them correctly, apply scales, and then accumulate results. On an older machine that unpacking cost is very visible.

One would therefore expect a carefully written SIMD path to help, provided it does not subtly change the arithmetic order and thereby the result. That second condition turned out to be the difficult part. The first attempt was exact but not faster. The kept version only became convincing once the packed decode and reduction order were reworked so that the SIMD path matched the scalar behavior closely enough while still reducing the number of wasted operations.

The second try was much more convincing. On the real-model prefill checkpoint it reduced the median from 77 seconds to 68 seconds in the alternating run, which is about 1.13x. In practical terms, this became one of the more useful wins in the session.

Small-n Expert Reuse For Int8 gate/up (Success: 4% gain)

One of the more important observations from profiling was that the routed expert work was not dominated by large row counts. Quite the opposite.

At a deeper prefill checkpoint, roughly 79% of the routed experts were running with nr <= 3. That changed the whole tuning strategy. Instead of optimizing for a broad and elegant general case, it suddenly made much more sense to focus on the tiny-row majority case that the machine actually sees most of the time.

The first kept step based on that was to quantize the activation rows once and reuse them for both gate and up, but only for the small-n case. That avoided some duplicated work without overcomplicating the wider path. The result was about 1.04x.

In terms of model structure, gate and up are two projections used inside the feed-forward expert blocks. If both of them consume the same routed activation rows, quantizing those rows twice is simply duplicated work. The reason the first broad reuse attempt failed, but the small-n one survived, is likely that overhead matters differently depending on how many rows are being handled. For the tiny-row dominant case, the reused preparation work is a larger fraction of the total cost, so saving it is actually noticeable.

Small-n Shared Output Pass For Int8 gate/up (Success: 3% gain)

Once the small-n reuse path was in place, another inefficiency became obvious: even in that case, gate and up still launched separate output passes.

That is the kind of thing one often misses before looking closely, because the code still works and the algorithm is unchanged. But on a machine like this, launching the same style of work twice for a tiny-row dominant workload is the sort of overhead that starts to matter. So the next change merged those passes for the already-specialized small-n path.

The expectation here was fairly direct. Once the quantized activation rows are already shared, the runtime is still doing two very similar sweeps over the outputs, one for gate and one for up. For large batches this extra structure may be hidden beneath the much larger compute load. For very small routed batches, however, those framework costs stand out much more sharply. Merging the output-side handling was therefore an attempt to reduce scheduling and loop overhead in the exact regime this machine sees most often.

Again, this was not dramatic, but it was consistent enough to keep at roughly 1.03x.

Small-n Direct-Row Gather Removal (Success: 2% gain)

After that, there was still one last bit of unnecessary movement left in the same area.

The CPU MoE path copied nr * hidden floats into a temporary buffer and then immediately quantized those copied rows for the int8 gate/up path. Given that the hot case was mostly nr <= 3, it made more sense to stop copying and quantize directly from the original routed rows.

This is one of the less glamorous kinds of optimization. No fancy new instruction set is involved. The code simply stopped moving data into a temporary place when the next step could just as well operate on the original rows. On a wide vector machine that might be a footnote. On an older memory-constrained dual-socket system, unnecessary copies are harder to hide.

One would not expect a huge improvement from this alone, and indeed it is the smallest kept gain in the list. But it fits the general theme very well: once the workload shape is understood, the little pieces of avoidable work around the dominant small-n expert case become worth trimming.

That only saved about 1.02x, so this is the smallest of the kept wins. But as we see small wins sum up surprisingly quick.

SSSE3 4-Way Unroll Experiment (Failed)

This was a rather classical microbenchmark trap.

A wider SSSE3 unroll for dot_i8i8 and dot_i4i8 looked appealing. Unfortunately, once the real-model prefill path was measured, it came out slower than the simpler implementation. That was enough reason to drop it.

Forcing 8 OpenMP Threads By Default (Failed)

This was motivated by the observation that the machine often did not look fully busy at every instant. Since the machine has 8 physical cores but 16 logical threads limiting to the cores may actually reduce resource sharing and thus increase efficiency. On the real chat path the opposite happened: the 16-thread behavior was faster than the patched 8-thread variant. So the idea was discarded.

SSSE3 Output-Pair IDOT Driver (Failed)

The thought here was reasonable enough: if the output side is expensive, maybe pair outputs and share more work there. In practice that version lost on the real chat path.

The First Exact Int4 Projection Attempt (Failed)

The first crack at an SSE4.1 exact int4 attention projection was technically correct and still not worth keeping.

That was mildly frustrating, but also educational. Packed low-bit arithmetic tends to punish premature optimism. The first version preserved exactness and still ended up slightly slower on the real workload. Only the later reworked version actually earned its place.

Multi-Row gate/up Fusion (Failed)

At first glance this looked like the sort of optimization that should obviously help.

Fuse more work, reduce overhead, share intermediate state and surely the machine gets happier. Except here it did not. The real comparison was clearly worse, not marginally noisy-worse but properly worse, so this idea was dropped.

Thresholded OpenMP For silu(gate) * up (Failed)

This was another attempt to improve the glue logic around the expert path rather than the heavy dot products themselves.

A thresholded OpenMP helper for the silu(gate) * up stage sounded plausible, especially given the machine’s intermittent parallelism profile. But once tested on the real model, the median was slightly slower. That is not enough to justify more code and another special case.

Broad Int8 gate/up Reuse (Failed)

The first version of reuse for expert gate/up was too broad.

Conceptually it was fine: the path quantized the same activation rows twice, so why not quantize once and share the result? The answer, at least on this host, was that the general version did not pay for itself. Only after narrowing the idea down to the small-n majority case did it become useful.

The likely explanation is that the general case drags in bookkeeping and structure that are not free. If one tries to optimize every case with the same machinery, the overhead can easily cancel the intended gain. This is precisely why the histogram of routed expert row counts turned out to be so important: it showed that the machine does not need a universally elegant answer here. It needs a very efficient answer for the overwhelmingly common tiny-row case.

This was one of the more useful failures because it directly led to the narrower version that did survive.

Broad Int8 gate/up Pair Fusion (Failed)

The same story repeated for the broader fused gate and up output pass.

As a general optimization it was slightly slower on the deeper prefill checkpoint. Once again the lesson was that the real workload here is lopsided. Broad elegant machinery was not what this machine wanted. Narrow special handling for the common tiny-row case worked better.

Parallel Plain Routing Rows (Failed)

Because thread-state sampling showed periods of low visible parallelism, it was natural to look for serial windows between the heavy kernels.

One such candidate was the plain routing row selection stage in moe(). That work was parallelized experimentally - the idea made a certain amount of sense. Unfortunately the longer alternating run showed the modified path was slower. So even though the motivation was sound, the machine did not reward it.

Not every serial-looking region deserves immediate parallelization. Some phases are simply too small, too synchronization-heavy or too memory-irregular for additional OpenMP structure to help. On a dual-socket system the cost of waking workers, synchronizing them and touching data across domains can easily eat the hoped-for gain.

sincosf For RoPE (Failed)

This was one of the more annoying almost-wins.

After adding the inverse-frequency cache, each new position still paid for one cosf and one sinf per lane. On paper, using sincosf looked attractive and even local microbench behavior was encouraging. On the real benchmark, however, the result was in the noise and leaned slightly slower. That was not enough to keep it.

This also confirmed that the cache mattered much more than the trigonometric call style.

SSE4.1 qt_addrow On Decode (Failed)

This looked good in a model-shaped local microbenchmark and then refused to matter where it counted.

qt_addrow() participates in the exact absorbed-attention helper used on decode-sized batches, so it was not an unreasonable target. But the real-model warm-KV decode test did not improve. If a helper gets faster and the application does not, then the only honest conclusion is that one optimized the wrong thing or optimized too little of it.

Decode-sized work behaves differently from long prefill work. A helper can be visibly faster in isolation and still be too small a fraction of the full decode step to matter. This is why real-model validation remained essential throughout the session.

SSE4.1 Exact Int8 Attention Projection (Failed)

There was a small apparent gain here, but not one I would trust.

The measured improvement vanished into the noise floor. Given the added complexity, that was not good enough. So the exact int8 attention path experiment was discarded.

SSE4.1 Expert Accumulation Helpers (Failed)

The expert path still had large scalar accumulation loops, so accelerating those looked sensible. The pairwise result leaned the wrong way and the apparent raw median win was not stable. Too much uncertainty, too little payoff. Out it went.

SSSE3 2x2 Int8 IDOT Driver (Failed)

This was a more ambitious follow-up to the earlier output-pair idea.

The plan was to share both weight and activation loads across two outputs and two rows, which certainly sounds more substantial than merely pairing outputs. Unfortunately, the measured result was effectively neutral to slightly worse. Neutral code is not free code, so it did not stay.

Small-n Shared-x Dual-Dot Helper (Failed)

After the kept small-n changes there was still the temptation to push the specialization one level further and reuse the loaded activation row inside a dual-dot helper.

That turned out to be one specialization too far. The effect leaned slightly slower and never justified the extra path. So this was the point where it made sense to stop being clever and accept that the simpler small-n optimizations were already the better trade-off.

Conclusion: What Actually Mattered

Looking back over the whole session, the pattern is fairly consistent.

The biggest wins did not come from broad theoretical cleanups. They came from identifying the specific CPU features this old x86 machine still has, enabling them carefully and then focusing on the workload shape the real model actually produces. In practice that meant:

• Using SSSE3 and SSE4.1 where the existing path was still too scalar
• Removing repeated helper work around quantization and RoPE
• Specializing the routed expert path for the tiny nr values that dominate on this host

Just as importantly, several obvious ideas did not help at all. More fusion was not automatically better. More OpenMP was not automatically better. A microbenchmark win was definitely not automatically a real-model win. And some things that are architecturally useful - such as the NUMA support - still do not yet translate into a direct speedup on the current memory setup. Maybe they will when I get a substantial memory upgrade, which in turn will also massively reduce QPI traffic between the NUMA domains.

So there is visible progress: about 200% overall gain till now relative to the earlier benchmark state - roughly a 3x throughput increase. But just as valuable is the narrower search space we are left with. The machine has told us rather clearly which classes of ideas it likes and which ones it does not.

That is the result of an proper optimization session: not only a faster program, but also a much clearer understanding of where not to spend time next.

Of course this is not the end of the story: In future there will be more tries to optimize critical paths and as well as reduce QPI traffic when more RAM is available. Leaving the disk bound operation regime will also show if the discarded optimizations may actually have an effect on a non-disk and memory bound machine.

References

• Fast PCIe x4 NVMe SSD
• Colibri: Running a 744 Billion Parameter LLM with 20 GB of RAM on FreeBSD
• Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin, Attention Is All You Need
• Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, Yunfeng Liu; RoFormer: Enhanced Transformer with Rotary Position Embedding
• Danyang Zhang, Junhao Song, Ziqian Bi, Xinyuan Song, Yingfang Yuan, Tianyang Wang, Joe Yeong, Junfeng Hao, Mixture of Experts in Large Language Models

This article is tagged: Programming, Artificial Intelligence, FreeBSD, Large Language Models, Computational linear algebra, Optimization