8000 Tags · ausimian/emily · GitHub
[go: up one dir, main page]

Skip to content

Tags

Tags

1.4.0

Toggle 1.4.0's commit message
- emily now requires Nx 1.0, and Axon 0.9 / Bumblebee 0.8 for the opt…

…ional

  Axon and Bumblebee integrations. To stay on Nx 0.12, pin
  `{:emily, "~> 1.3.0"}`.
- The native compiler (`native: true`) now compiles `Nx.block`s that
  `Emily.Backend` has no native kernel for, including every Axon 0.9
  activation (GeLU, softmax, ReLU, SiLU and the rest). It lowers the block's
  default implementation, which is exactly what the Evaluator runs, so the
  results are identical. Without this, every Axon or Bumblebee model would
  fall back to the Evaluator under Axon 0.9.

- Under `native: true`, a hook passed in the `:hooks` option (for example
  by `Nx.Serving` when streaming Bumblebee text generation) was silently
  skipped: the defn compiled natively and the hook never ran. A defn with
  a hooked `io_call` now falls back to the Evaluator, which runs the hook.
  Named `io_call`s with no hook registered still compile natively.

1.3.1

Toggle 1.3.1's commit message
- Installing emily from Livebook (`Mix.install`) no longer crashes while

  downloading the precompiled NIF with "Can't set long node name!". The
  download ran on a named peer VM, which fails to boot under a
  longnames-distributed parent when the hostname is not fully qualified.
  The peer now starts without distribution.

1.3.0

Toggle 1.3.0's commit message
- `Emily.Quantization.Transform` now ships with the package. Previously

  it was test-only code. `quantize/3` rewrites every `:dense` layer of an
  Axon or Bumblebee model to run through
  `Emily.Quantization.Layers.quantized_dense/4` and replaces its kernel
  in the `Axon.ModelState` with an `Emily.QuantizedWeight`.
  `quantize_dense_layers/2` and `quantize_model_state/3` do each half
  separately. Options are `:bits`, `:group_size`, `:transpose`, `:mode`
  and `:except`, which keeps matching layers as float `:dense` layers.
  The module is defined only when Axon is available. The quantized
  weights are evaluated before `quantize/3` returns, so the dense
  kernels are freed as soon as the caller drops the original model state
  instead of staying resident until the first forward pass. The
  `qwen3_quantized` livebook now calls `quantize/3` instead of defining
  its own transform.

- Under `native: true`, `Nx.cumulative_sum/2`, `cumulative_product/2`,
  `cumulative_max/2` and `cumulative_min/2` on any axis but the last now
  read their own operand. They previously used the function's first
  input instead, which gave a silently wrong result when the shapes
  matched and an MLX `[slice]` error when they did not.

1.2.0

Toggle 1.2.0's commit message
- `Emily.QuantizedWeight.from_dense/2` accepts `bits: 1` for `"affine"`

  quantization, which MLX 0.32.4 added. Each element dequantizes to its
  group's minimum or maximum, packed 32 to a `u32`. 1-bit weights work
  with `Emily.Quantization.quantized_matmul/2`,
  `quantized_matmul_defn/2` and `dequantize_defn/1`, and
  `Emily.Quantization.defn_supported_bits/0` now includes `1`. On the
  GPU, MLX widens every group's range to include zero, so a group whose
  values are all negative dequantizes to its minimum or `0`.

- Updated the pinned MLX to 0.32.4. Emily picks up its upstream fixes
  without API changes. They cover strided and sliced views, reductions
  over very large arrays, FFTs of size one, empty unsigned sums and
  products on Metal, and a deadlock during eval cleanup. Split-K
  quantized matmul now keeps its partial sums in float32. The release
  also speeds up quantized matvec when the output size isn't a multiple
  of 8, GEMMs with a thin N, and vector SDPA decode, and uses less
  memory for SDPA with head dim 512 on chips before M5.

1.1.1

Toggle 1.1.1's commit message
- Compiling emily as a dependency no longer warns that

  `:public_key.cacerts_get/0` and
  `:public_key.pkix_verify_hostname_match_fun/1` are undefined. The
  warnings came from `mix emily.checksums`, a maintainer-only release
  task that shipped in the hex package. The package now leaves out the
  Mix tasks that only work in a checkout of the repository
  (`emily.checksums`, `bench.native` and `clang.tidy`); `mix emily.doctor`
  is still included.

1.1.0

Toggle 1.1.0's commit message
- `Emily.Fast.scaled_dot_product_attention/4` and

  `scaled_dot_product_attention_with_mask/5` accept `force_fused: true`.
  MLX then always uses one of its fused attention kernels instead of
  choosing per shape, and raises `ArgumentError` if no fused kernel
  supports the shape. This keeps attention memory bounded for
  long-context prefill with head dims 192 and 256 (for example Gemma),
  where MLX otherwise builds the full attention-score matrix, at some
  cost in prefill throughput. Off by default; it works in both eager
  and natively compiled code.
- `Emily.Fast.rope/3` and `rope_with_freqs/4` accept a `{batch}` offset
  as well as a scalar, so each entry along the leading axis starts at its
  own position, for example in batched decode.
- `Emily.Bumblebee.FastKernels.apply/2` accepts `force_fused_sdpa: true`,
  which passes `force_fused: true` to every rewritten attention layer.
- `Emily.Fast.einsum/2` can now be called inside `defn`. Under
  `Emily.Compiler`, evaluated or natively compiled, it still runs MLX's
  `mx::einsum`. Other backends, such as `Nx.BinaryBackend` and EXLA, run a
  composed `Nx.dot/6` fallback that handles diagonals, ellipses,
  broadcasting and implicit output. Equations that `mx::einsum` would
  reject raise `ArgumentError` on every backend, naming the problem.

- `Emily.Fast` kernels can now be called from inside a caller's own
  `defn` or `defnp`, as the module documentation shows. Before, they only
  worked as the top-level function passed to `Nx.Defn.jit`, and raised
  "cannot invoke ... inside defn because it was not defined with defn"
  when nested (#205).
- `Emily.Bumblebee.FastKernels.apply/1` now rewrites the attention and
  RoPE layers of Bumblebee models. Before, it only rewrote RMSNorm and
  LayerNorm: it never recognised Bumblebee's attention and RoPE layers,
  so the fused SDPA and RoPE kernels never ran. Decoder models such as
  Qwen3 now get the fused kernels, whose advantage grows with context
  length. The newly active paths are checked against stock Bumblebee,
  including the `:linear`, `:dynamic`, `:longrope` and `:llama3` RoPE
  scaling strategies, padded and batched position ids, and causal decode
  against a KV cache. Attention is fused even in models whose checkpoint
  sets a nonzero attention dropout rate, such as DistilBERT and other
  BERT-family models, because dropout does nothing at inference (#221).

  **If you train a rewritten model, it no longer applies attention
  dropout.** The fused attention layer has none, whereas the stock
  attention layers that `apply/1` used to leave in place did. Train with
  the stock model, and apply `FastKernels` for inference only.
- `Emily.Fast.rope_with_freqs/4` rotated by the wrong angles on the
  Emily backend: MLX expects periods, not the inverse frequencies the
  function takes. It now matches its documented contract.
- The defn fallback of `Emily.Fast.scaled_dot_product_attention/4` (used
  on non-Emily backends) applied an inverted causal mask, letting each
  query attend to later keys instead of earlier ones.
- The defn fallback of `Emily.Fast.rope/3` and `rope_with_freqs/4` (used
  on non-Emily backends) added the offset after applying `:scale`; it now
  scales the whole position, as MLX does.
- Emily now compiles without warnings on Elixir 1.20. Its type checker
  flagged an unreachable clause in `mix emily.doctor` (#195).

- `Emily.Fast.einsum/2` no longer requires every operand to be on
  `Emily.Backend`. Operands on other backends run the composed
  fallback, or are moved to Emily when mixed with Emily tensors.
- Updated the pinned MLX to 0.32.3. This is a maintenance bump that picks
  up upstream correctness fixes (NaN propagation in arg reductions,
  quantized matmul with a quantized dim that is not a multiple of 32,
  scans and sorts over zero-size axes, quantizing sliced arrays) and
  faster GQA decode attention, all transparently. No API changes.
- Building MLX from source (a git checkout of Emily) now needs Xcode 16
  or later. MLX 0.32.1 started using a Metal standard library
  type that Xcode 15 does not ship. The precompiled NIFs are unaffected
  and still support macOS 14.

1.0.1

Toggle 1.0.1's commit message
- Constrain Nx to the supported 0.12 release line so fresh dependency

  resolution does not select incompatible Nx 0.13 releases.

1.0.0

Toggle 1.0.0's commit message
First stable release. Emily runs `Nx` computations on Apple Silicon

through MLX — set it as the default Nx backend and Bumblebee models run
on the Metal GPU with no further integration. With 1.0 the public API is
stable and follows semantic versioning from here; there are no breaking
changes from 0.7.x, so existing code keeps working. The headline this
cycle is quantized inference — 4-bit layers now stream through the fused
`mx::quantized_matmul` kernel, making native quantized generation ~13×
faster end-to-end and, for the first time, faster than dense (details
below).

- The precompiled NIF now declares an explicit minimum macOS per variant —
  macOS 14 for the AOT build, macOS 26.2 for the JIT build — instead of
  inheriting whateve
8001
r macOS the release runner happened to be on. Published
  artifacts now have a deterministic macOS compatibility floor, and CI
  asserts it on every build.
- Updated the pinned MLX to 0.32.0. This is a maintenance bump that also
  picks up faster small-batch quantized matvec (`qmv_wide`) — accelerating
  the fused quantized path — and broader fused SDPA coverage (asymmetric
  Q/V head dims), both transparently. No API changes.
- Quantized dense layers now use the fused `mx::quantized_matmul` kernel
  instead of dequantizing the full weight to bf16 and running a dense
  matmul. The packed low-bit weights are streamed directly, so a decode
  step no longer re-dequantizes the entire model on every token. On a
  4-bit Qwen3-0.6B this makes native quantized generation roughly 13×
  faster end-to-end — and quantized inference is now *faster* than dense,
  as it should be, rather than slower.

0.7.2

Toggle 0.7.2's commit message
- The README performance section now compares Emily against both

  benchmark baselines — EXLA (host CPU) and EMLX (the older MLX-backed
  Nx backend on the Metal GPU) — instead of EXLA alone, and its
  rule-of-thumb figures (ViT-base, DistilBERT) are reconciled with the
  current benchmark report.
- The benchmark report's environment block now records the Emily
  version the numbers were produced on (0.7.0) and drops a misleading
  run timestamp.
- The `MAINTAINING.md` release runbook is corrected: `mix publisho` is
  no longer described as pushing (it only commits and tags), and the
  obsolete manual draft-promotion step is dropped — `release-nif.yml`
  now publishes the release automatically once the NIFs are built.

0.7.1

Toggle 0.7.1's commit message
- Documentation no longer fails to build over autolink references to the

  hidden `Emily.Native.async_eval/2` and `Emily.Native.fast_rope_int/8`
  NIF stubs in the changelog; both are excluded from ex_doc autolinking.
0