8000 Net.HTTP clients: let callers set timeouts and a response size limit · Issue #484 · kexhq/kex · GitHub
[go: up one dir, main page]

Skip to content

Net.HTTP clients: let callers set timeouts and a response size limit #484

Closed

Description

@akoskovacs

A caller of Net.HTTP cannot say how long a request may take or how big its answer may be. The limits are fixed inside the BEAM runtime, and differ between the two clients:

HTTP.request (kex_intrinsic_nethttp.erl, httpc) Client (kex_intrinsic_nethttpclient.erl)
connect 10 s 10 s
whole request 30 s none
each read — 30 s (?TIMEOUT)
response body unlimited 16 MiB (?BODY_LIMIT)
response headers httpc's 64 KiB (?HEADER_LIMIT)

ClientOptions only has pool, and HTTP.request takes no options.

Why it matters

Fetching from somewhere you don't control, such as link previews from oEmbed providers or webhooks, needs a short deadline and a small cap:

  • No deadline on Client: the 30 s applies to each recv, so a server that sends a byte every 29 s keeps a request, and its process, alive indefinitely.
  • Fixed size limits: a 16 MiB cap is too much for a 2 KB JSON answer. HTTP.request has no cap at all, so the whole body is read into memory.
  • Fixed connect time: 10 s is too long for a request made while a user waits.

Proposal

Request options with today's values as their defaults, so nothing changes for existing callers:

record RequestOptions do
  connectTimeout : Duration = 10.seconds
  # The whole exchange, from connecting to the last byte of the body.
  timeout : Duration = 30.seconds
  maxBodyBytes : Integer = 16777216
  maxHeaderBytes : Integer = 65536
end
  • Client.open(ClientOptions { pool: …, request: RequestOptions { … } }) sets defaults for every request a client makes.
  • client.request(method, url, headers, body, options) and HTTP.request(…, options) override them for one call.
  • Client: enforce timeout as a deadline across all reads, not per recv.
  • HTTP.request: enforce maxBodyBytes too, e.g. {stream, …} or {body_format, binary} checked against Content-Length, aborting once over. Today it has no limit.
  • Errors: exceeding a limit stays Limit, and running out of time stays Timeout. Neither changes kind.

Seen in

Slick: link previews from YouTube, Vimeo, etc. would be fetched in a background process per message. A provider that stalls should give up after a couple of seconds, and an answer larger than a few hundred KB should be refused.

Activity

  1. added this to the 0.4.0 milestone on Oct 14, 2026
  2. akoskovacs commented on Oct 21, 2026

    @akoskovacs
    CollaboratorAuthor

    Solved in #534: RequestOptions (connect timeout, whole-request timeout, response body size, client header size), as client-wide defaults or per-request overrides.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0