Repository navigation
Tags
Tags
fix(client): bound object body reads by stall, not by Timeout (#148) (#… …153) GetObject and GetObjectRange read the response body under the same deadline as the request, so Config.Timeout capped how large an object could be read at a given throughput: a 126 MB object over a 3.8 MB/s link failed at 30s while the store was still streaming. Timeout now covers the request up to the response headers. The body read is bounded by a new Config.ReadIdleTimeout (S3_READ_IDLE_TIMEOUT, default 30s): the read fails only when no data arrives for that long, so a transfer that keeps moving is never cut off. - A stalled body returns the new ErrReadIdleTimeout. - A request timeout still matches errors.Is(err, context.DeadlineExceeded), as it did before. - The caller's own context deadline and cancellation still end a read. - The GetObjectRange path that skips to the offset when a store ignores Range is bounded the same way. Both bounds are timers that cancel one context, not a context deadline: net/http aborts a body read when the request context is canceled, so the context must outlive the headers, and a deadline cannot be lifted. Tests run the real SDK against an HTTP server that trickles or stalls the body. Two deliberate breaks (Timeout still covering the body, the idle timer never restarted) each make the slow-transfer test fail. From review, not changed here: - If the response headers arrive within microseconds of Timeout, the request timer can fire before it is stopped, and that read fails as a request timeout although a response arrived. It still fails as a timeout. - A body read has no total cap: a store sending a byte just inside ReadIdleTimeout keeps the read open until the caller's context ends. That is the intended behavior; a total bound comes from the context. Deferred: PutObject sends its body inside the Timeout deadline and has the same throughput limit on uploads. It needs a different mechanism and is left for a separate change. Closes #148
feat(client): ranged object read reporting the object's total size (#151 ) (#152) Add Client.GetObjectRange(ctx, bucket, key, offset, length), which reads up to length bytes starting at offset. ObjectContent.Size is the whole object's length, taken from Content-Range. - offset < 0, length <= 0, and an end past int64 are refused with ErrInvalidRange before any request. - A store that ignores Range sends the whole object from byte 0. The bytes before offset are read and discarded without being held, and Body is bounded by length. - A Content-Range whose start differs from offset is an error. - An HTTP 416 (offset at or past the end, which includes every range of a zero-byte object) is wrapped as ErrRangeNotSatisfiable 10BC8 , not returned as an empty Body. - Size is -1 when the store reports no total ("bytes 0-99/*"). pkg/client/range_test.go runs the method through the real SDK against an httptest S3 stand-in. Deferred from review: a store that answers 206 without a Content-Range (an RFC 9110 violation) would be taken for one that ignored Range. It could be detected from the raw response status. Also rolls up the open Dependabot PRs: - aws-sdk-go-v2 group (#136) - modelcontextprotocol/go-sdk 1.7.0 -> 1.8.0 (#149) - github/codeql-action init/autobuild/analyze/upload-sarif 4.37.4 -> 4.37.7 (#143, #144, #146, #147), moved together - actions/setup-python 6.3.0 -> 7.0.0 (#145) smithy-go becomes a direct dependency for its ResponseError type. build: make verify's gate sentinel now uses the pre-commit hook's hash (git write-tree over a throwaway index with `git add -A`). The old shasum-of-`git diff` hash never included untracked files, so it could not match the hook for a change that added a file. Closes #151
feat(client): sign presigned URLs against a public endpoint (#109) Presigned URLs were signed against Endpoint, which in clustered deployments is an internal address unreachable from outside the cluster. Add an optional PresignEndpoint (S3_PRESIGN_ENDPOINT) that, when set, builds a dedicated presign client signing against the public-facing host while all data operations keep using Endpoint. Empty PresignEndpoint preserves prior behavior. For path-style addressing the SigV4 query signature does not cover the host, so swapping the base endpoint on the presign client yields a valid signature against the public host. Also pin toolchain go1.26.4 so the build uses the patched standard library, clearing reachable govulncheck findings GO-2026-5039 (net/textproto) and GO-2026-5037 (crypto/x509). TestNew_PresignEndpoint asserts the signed URL host is the public endpoint when set and falls back to Endpoint when empty.
feat(client): add streaming/multipart upload via PutObjectStream (#104) Adds PutObjectStream, which uploads an object from an io.Reader using the AWS SDK transfer manager without buffering the full payload in memory. This unblocks downstream callers that need to stream large or unbounded sources (e.g. query exports) into S3, which the existing []byte-based PutObject cannot do. - ObjectUploader interface (consumer-side, mockable) satisfied by *transfermanager.Client; wired in client.New() - PutObjectStreamInput with optional MaxBytes bound enforced by a counting limitReader that aborts with ErrStreamTooLarge - Library-only capability; no per-operation timeout (caller controls ctx) - Uses feature/s3/transfermanager (feature/s3/manager is deprecated and trips staticcheck SA1019) - Tests: 100% coverage on new code; README documents usage Closes #103
feat: dynamic connection management for runtime add/remove (#71) * feat: add dynamic connection management for runtime add/remove Enable downstream projects (e.g. mcp-data-platform) to dynamically add and remove S3 connections at runtime, matching the pattern established by mcp-trino v1.2.0. Key changes: - Manager.AddConnection: thread-safe with replace semantics, closes stale cached clients, protects default connection - Manager.RemoveConnection: protects default connection, validates existence, returns errors for invalid operations - Thread-safe read methods on Manager (ListConnections, HasConnection, DefaultConnectionName) - Toolkit.RemoveClient: close and evict cached clients - ConnectionManager interface + WithConnectionManager option for Toolkit to report all configured connections (not just cached) * fix: address review findings on dynamic connection management Fixes all 10 issues from critical review: 1. Document lock-during-IO tradeoff on AddConnection with createNow 2. Roll back config on createNow factory failure (new + replace cases) 3. Document RemoveClient scope (locally-added clients only) 4. Eliminate dual-cache: Toolkit no longer caches provider-sourced clients, so Manager.RemoveConnection takes immediate effect 5. Document implicit default connection behavior 6. (resolved by #4) single-cache architecture via provider delegation 7. Remove unused HasConnection from ConnectionManager interface 8. Document intentional close-error discard in Manager; use logger in Toolkit.RemoveClient 9. Strengthen concurrent test: WaitGroup, 3 goroutines, 500 iterations 10. Verify new config took effect in replace test (with config-aware mock)
feat: add Tool.Title, OpenWorldHint=true, and OutputSchema for all to… …ols (#45) Closes #44 ## Changes ### Tool.Title - Add `pkg/tools/titles.go` with default human-readable titles for all 9 tools and a three-level priority chain (default → toolkit → per-registration) - Add `WithTitles(map[ToolName]string)` toolkit-level option - Add `WithTitle(string)` per-registration ToolOption - Wire `Title` field in all 9 tool registration calls ### OpenWorldHint - Change `OpenWorldHint` from `false` to `true` for all 9 tools; S3 tools interact with external object storage and are open-world by definition - Update annotations_test.go to assert `true` ### OutputSchema - Add `pkg/tools/output_schemas.go` with JSON Schema 2020-12 objects for all 9 tools, matching actual response struct fields in types.go - Schemas declare `type` + `properties` only (no `required` / no `additionalProperties: false`) to avoid runtime validation failures on optional or implementation-specific fields - Add `WithOutputSchemas(map[ToolName]any)` toolkit-level option - Add `WithOutputSchema(any)` per-registration ToolOption - Wire `OutputSchema` field in all 9 tool registration calls ### toolConfig / Toolkit structs - Add `title *string` and `outputSchema any` to `toolConfig` - Add `titles map[ToolName]string` and `outputSchemas map[ToolName]any` to `Toolkit`, initialised in `NewToolkit` ### Gosec fixes (pre-existing issues) - `pkg/client/config.go`: add `#nosec G117` to `SessionToken` field - `pkg/multiserver/config.go`: add `#nosec G117` to `SessionToken` field; update `#nosec G304` → `#nosec G304 G703` on both `os.ReadFile` calls - `pkg/client/client.go`: add `#nosec G118` to `context.WithTimeout` return (cancel func is intentionally returned to callers, not a leak) ### Documentation - `docs/reference/tools-api.md`: align all response examples with actual output schemas (fix `body`→`content`, `encoding`→`is_base64`, `key_count`→`count`, `default`→`default_connection`, add missing fields) - `docs/library/quickstart.md`: add "Tool Metadata Customization" section documenting `WithTitles`, `WithTitle`, `WithDescriptions`, `WithOutputSchemas`, and `WithOutputSchema` All checks pass: `make verify` (lint, test, coverage ≥80%, security, deadcode, build-check)
feat: add default icons for all tools (#39) Add Icons field support to tool registrations with a 3-level priority chain: per-registration override, toolkit-level override, and built-in default. All 9 tools now include a default S3 SVG icon in tools/list responses. New WithIcons() toolkit option and WithIcon() tool option allow consumers to override icons at registration time.
Add MCP tool annotations and upgrade CI to sister project standards (#38 ) * Add MCP tool annotations, configurable descriptions, and typed returns - Upgrade go-sdk to v1.3.0 and Go to 1.24; drop legacy mcp-go dependency - Add default tool annotations (ReadOnlyHint, DestructiveHint, IdempotentHint, OpenWorldHint) for all built-in tools per MCP spec - Extract hardcoded descriptions into a centralized defaultDescriptions map - Implement three-level priority chain for both descriptions and annotations: per-registration override > toolkit-level override > default - Add WithDescriptions, WithDescription, WithAnnotations, WithAnnotation options - Return concrete typed pointers from tool handlers instead of any * Upgrade linting, Makefile, and CI to match sister project standards Rewrite .golangci.yml from 5 to 24 linters matching mcp-trino, fix all 263 lint issues across the codebase, upgrade Makefile verify target to run tidy/lint/test/coverage/security/deadcode/build-check, add codecov.yml with 82% project and 80% patch thresholds, and update CLAUDE.md with verification docs and AI-specific rules. * Add tests for tool registration path and new option functions Cover the typed-return wrappers in all 9 register*Tool functions by invoking tools through the MCP server via in-memory transport. Also add tests for WithDescriptions, WithDescription, WithAnnotations, and WithAnnotation option functions. Raises pkg/tools coverage from 77% to 88% and total from 84% to 89%.
Migration to Official MCP SDK (#22) * Update MCP import path to use new `modelcontextprotocol/go-sdk` package * Refactor `toolkit_test.go` and `tools_test.go` to adopt structured input types, update middleware and handler logic, and revise MCP import paths * Refactor `toolkit.go` to add per-tool middleware support, improve tool registration with options, and streamline MCP server integration. * Refactor `sizelimit.go` to update MCP import path, use request pointer, add argument extraction helper, and improve base64 handling logic * Refactor `server.go` to update MCP import path, use new `mcp.Server` implementation, revise toolkit registration logic, and add comprehensive tests for S3 client and configuration handling. * Refactor `readonly.go` to update MCP import path, use request pointer type, and revise interceptor signature * Refactor `toolkit.go` to modularize middleware handling, add helper functions for interceptors, hooks, and transformers, and streamline handler execution logic. * Refactor `put_object.go` to update MCP import path, revise tool registration logic, adopt structured input types, and improve handler design. * Refactor `presign_url.go` to update MCP import path, revise tool registration logic, adopt structured input types, improve handler design, and modularize presigned URL generation logic. * Refactor `prefixacl.go` to update MCP import path, use request pointer type, revise interceptor signature, and add argument extraction helper function. * Refactor `options.go` to use `ToolName` type for tool identifiers, add `WithToolMiddleware` option, and update tests for new middleware and context handling logic. * Refactor `names.go` and `names_test.go` to use `ToolName` type for tool identifiers, update related constants, functions, and tests to improve type safety. * Refactor `middleware.go` and `middleware_test.go` to update MCP import path, redesign middleware interfaces with `Before` and `After` hooks, implement `MiddlewareChain`, and add comprehensive tests for middleware ordering and execution. * Refactor `audit.go` to update MCP import path, redesign `AuditMiddleware` with `Before` and `After` hooks, streamline audit logging logic, and remove argument sanitization function. * Refactor `connections.go` to update MCP import path, revise tool registration logic, adopt structured input types, enhance handler design, and improve error handling. * Refactor `context.go` and `context_test.go` to add `Duration` method, improve type safety with `ToolName` type, and extend test coverage for `GetBool` method. * Refactor `copy_object.go` to update MCP import path, revise tool registration logic, adopt structured input types, improve handler design, and enhance error handling. * Refactor `delete_object.go` to update MCP import path, revise tool registration logic, adopt structured input types, improve handler design, and enhance error handling. * Refactor `errors.go` and `errors_test.go` to update MCP import path, revise result generation logic with pointer-based `TextContent`, remove unused argument extraction functions, and enhance test coverage for result functions. * Refactor `extensions_test.go` and `fuzz_test.go` to update MCP import path, introduce helper functions for request generation, revise middleware tests with `Before` and `After` hooks, improve logging and metrics assertions, and replace legacy fuzz tests with targeted ones for tool and context operations. * Refactor `get_object.go` to update MCP import path, revise tool registration logic, adopt structured input types, improve handler design, streamline input validation, and enhance error handling. * Refactor `get_object_metadata.go` to update MCP import path, revise tool registration logic, adopt structured input types, improve handler design, streamline input validation, and enhance error handling. * Update dependencies: add `modelcontextprotocol/go-sdk v1.2.0`, `google/jsonschema-go v0.3.0`, `golang-jwt/jwt v5.2.2`, `golang.org/x/oauth2 v0.30.0`, and update `google/go-cmp` to `v0.7.0`. * Refactor `interceptor.go` to update MCP import path, switch `CallToolRequest` to pointer-based, and streamline request modification logic. * Refactor `interceptor_test.go` to update MCP import path, switch `CallToolRequest` to pointer-based, introduce `makeRequest` helper, and streamline test logic for interceptors. * Refactor `list_buckets.go` to update MCP import path, revise tool registration logic, adopt structured input types, improve handler design, streamline input validation, and enhance error handling. * Refactor `list_objects.go` to update MCP import path, revise tool registration logic, adopt structured input types, improve handler design, streamline input validation, and enhance error handling. * Refactor `logging.go` to update MCP import path, replace `Wrap` with `Before` and `After` hooks for improved middleware design, revise logging logic, and streamline tool context handling. * Refactor `main.go` to update MCP import path, add context cancellation and signal handling for graceful shutdown, and replace `ServeStdio` with `Run` using `StdioTransport`. * Refactor `metrics.go` to update MCP import path, replace `Wrap` with `Before` and `After` hooks for improved middleware design, revise metrics logic, and streamline tool context handling. * Refactor `put_object.go` to revise tool registration logic, adopt structured input types, improve handler design, modularize input validation and preparation, and enhance error handling. * Refactor `types.go` to define structured input types for S3 tool operations, including `list_buckets`, `list_objects`, `get_object`, `get_object_metadata`, `put_object`, `delete_object`, `copy_object`, `presign_url`, and `list_connections`. * Add `resolver_test.go` to introduce comprehensive unit tests for S3 resolver functionality. * Add `provider_test.go` to introduce comprehensive unit tests for content provider functionality, including caching, content retrieval, streaming, content types, and size handling. * Add `metadata_test.go` to introduce comprehensive unit tests for metadata providers, including caching, existence checks, ETag retrieval, error propagation, and composite provider functionality. * Add `integration_test.sh` to introduce a comprehensive integration test script for MCP-S3 functionality with SeaweedFS, covering key S3 operations and ensuring end-to-end validation. * Update `.gitignore` to exclude `/mcp-s3-test` directory. * Remove `integration_test.sh` script after migration of integration tests to a new framework.
PreviousNext