Repository navigation
Tags
Tags
1.8.1 (Freika#2914) * feat: non-ML stay-point visit detection behind Flipper flag (Freika#2832) * Return 404 instead of 403 for users not in a family The frontend pings family endpoints by default. When the feature is unconfigured, repeated 403s look like brute-force attempts and trip firewall blocks. 404 reflects that the resource simply isn't there for that user. * Remove transitional trial-reminder mailers (Manager owns lifecycle now) Removes the four email types kept temporarily in PR Freika#2508 to let in-flight Sidekiq jobs drain cleanly. Today (2026-05-17) is the earliest safe removal date named in those source comments. Deleted: - MAILER_REGISTRY entries: trial_expires_soon, trial_expired, post_trial_reminder_early, post_trial_reminder_late - Corresponding mailer methods and the should_skip_email? guard - Eight email templates (.html.erb / .text.erb) - All tests and previews specific to these four types - The four types are removed from user_spec "not enqueued" lists too (they're gone; only pure Manager-owned billing types remain) * Harden 1.7.10 release: PII filtering, area validations, posthog guard - Add email, name, first_name, last_name, phone to filter_parameters so request params attached to error-tracking payloads (PostHog, Sentry) and Rails logs are redacted on auth/signup/profile flows. - Skip PostHog initialization when POSTHOG_API_KEY is blank to avoid initializing the client with a nil key. - Validate Area numericality: radius > 0, latitude in [-90, 90], longitude in [-180, 180]. The client-side min on the modal is bypassable via direct PATCH /areas/:id. - Add request specs for PATCH /areas/:id (success, non-positive radius, out-of-range coordinates, cross-user 404). - Document the Cloud-only PostHog telemetry pipeline in CHANGELOG with the field redaction list. * fix: geocode flood (WIP) * fix: timeline calendar points fallback (WIP) * test: cover edge cases for timeline calendar points fallback Adds regression coverage for empty months, points-only bucket clamping, negative UTC offsets, DST spring-forward, and Lite plan out-of-window points. * fix: unblock nightly reverse-geocode rescue and bulk-enqueue dedup NightlyReverseGeocodingJob now passes force: true so points with a lingering dedup key still get re-enqueued. Jobs::Create#bulk_enqueue claims SETNX keys on continue_reverse_geocoding and clears them on start_reverse_geocoding, routing both admin paths through the same dedup. async_reverse_geocode(force:) deletes the key so a forced run doesn't leave a stale claim. * fix: namespace geocode dedup key by class and tighten changelog wording Prefix the Redis dedup key with the class name (geocode:enq:Point:<id>) so a future Place-side SETNX won't silently collide with Point ids. Update the CHANGELOG entry to credit the nightly sweep — not the exhausted job itself — for clearing stale dedup keys. * fix: release dedup key when reverse-geocode enqueue raises If perform_later raises after the SETNX claim (e.g. transient Redis blip between commands), the point was previously locked out for 24 h with no job behind it. Wrap the non-force enqueue path in a rescue that DELs the claimed key(s) and re-raises, so a retry can re-claim immediately instead of waiting for the nightly sweep. * Drop unused points indexes (not_anomaly, track_generation) Frees ~7.4 GB on the points table. - index_points_on_not_anomaly (931 MB): zero lifetime scans; planner always picks (user_id, timestamp) and filters anomaly in heap. - idx_points_track_generation (6.5 GB): used by Track.segment_points_in_sql, but the query runs ~64 times lifetime for 0.7s total. Fallback to index_points_on_user_id_and_timestamp is acceptable. index_points_on_raw_data_archive_id stays — it backs FK enforcement on 44M points referencing 1422 archives, even though idx_scan is currently low. index_points_on_unarchived stays — ARCHIVE_RAW_DATA=true on prod and the monthly archiver query matches it exactly per EXPLAIN. * Bulk-insert track segments (Fixes DAWARICH-P3) Replaces the per-segment .create loop in Tracks::TrackBuilder, TransportationModes::BackfillJob, and Tracks::Reprocessor with a single TrackSegment.insert_all via TrackSegments::BulkInserter. Resolves the N+1 INSERT pattern reported in Sentry DAWARICH-P3 from Tracks::TimeChunkProcessorJob. Each track now issues one batched INSERT instead of N round-trips. - TrackSegments::BulkInserter converts :mode and :confidence symbols to enum integers via .fetch (fail-fast on unknown values). - dominant_mode is computed from segment_data in-memory before insert, removing the need to reload created records. - Returns the input segment_data so callers can inspect what was written. * Drop per-pair ST_Distance round-trip (Fixes DAWARICH-P1) Point#distance_to issues a single-pair ST_Distance SELECT per call. Called inside transportation-mode segmentation loops in TransportationModes::MovementAnalyzer and SourceDataExtractor, this adds ~ms of DB latency per consecutive point pair — flagged in Sentry DAWARICH-P1 as a slow query from Tracks::TimeChunkProcessorJob. Switch both call sites to the existing in-Ruby Haversine path (geocoder_distance / Point#distance_to_geocoder). No DB round-trip, microsecond-scale per pair, same result. * Drop unused :lonlat from AnomalyFilter point fetch (Fixes DAWARICH-NJ) Points::AnomalyFilter#fetch_points_with_context selected id + lonlat + timestamp for every point in a monthly chunk, but Ruby only ever reads point.id afterwards — lonlat is re-fetched inside the calculate_all_speeds CTE by id. For heavy users (>500k points/month) the unused PostGIS geography bytes dominate transfer cost. Dropping :lonlat from the SELECT reduces row payload by ~30+ bytes and addresses the slow SELECT flagged in Sentry DAWARICH-NJ from Points::AnomalyFilterJob. * Fix Map v2 visits ignoring date range when bbox is sent Visits::FindWithinBoundingBox now applies start_at/end_at when both params parse. Since 1.7.10 the Map v2 visits fetch always passes the viewport bbox, which routed every request through this service. The service ignored the date range entirely, so the layer rendered all visits inside the viewport regardless of the selected date filter. Closes Freika#2817 * Align BackfillJob dominant-mode logic with TrackBuilder After rebase TrackBuilder uses Track.pick_dominant_mode (distance-weighted, filters stationary) but BackfillJob still picked the longest-duration segment. Same input would produce different dominant_mode on the two paths. Route BackfillJob through the same in-memory-segment + pick_dominant_mode bridge. Also: spec coverage for TrackBuilder#update_dominant_mode (empty, mixed moving+stationary, stationary fallback, nil distance/duration) and a CHANGELOG entry for the perf work + index drop. * chore: bump gems to close 9 known CVEs bundle-audit reports zero vulnerabilities after these bumps. Direct bumps in Gemfile: - rails ~> 8.0.5 (CVE-2026-33195/33202 ActiveStorage DiskService, CVE-2026-33167/33168/33169/33170/33173/33174/33658) - devise >= 5.0.4 (CVE-2026-40295 open redirect via timeoutable session referrer; CVE-2026-32700) - jwt ~> 2.10.3 (CVE-2026-45363 HMAC bypass) - httparty >= 0.24.0 (CVE-2025-68696 SSRF + API key leakage) - sidekiq-cron >= 2.4.0 (CVE-2025-67202 admin XSS) - aws-sdk-core ~> 3.234 + aws-sdk-s3 ~> 1.208 (CVE-2025-14762; core bumped to satisfy s3 dep graph) - addressable >= 2.9.0 (CVE-2026-35611) - bcrypt >= 3.1.22 (CVE-2026-33306) - faraday >= 2.14.2 (CVE-2026-33637 protocol-relative URI scoping bypass) - json >= 2.19.2 (CVE-2026-33210 format string DoS) - net-imap >= 0.5.14 (CVE-2026-42257/42258 CRLF injection) - zlib >= 3.2.3 (CVE-2026-27820 GzipReader buffer overflow) Transitive bumps pulled in by the above: - aws-partitions, faraday-net_http, fugit, marcel, responders 5791 RSpec examples pass with zero regressions. * fix(spec): scope place factory to test user in visits PUT specs The three "updating name together with place_id" specs created new_place without a user, so the factory assigned a random one. Since f3dde359 added place_id ownership validation, the controller correctly returns 422 for a foreign place_id, the visit name is never updated, and the assertions fail. Pass user: user to the factory so the specs exercise the name-preservation logic they were written for. * feat: non-ML stay-point visit detection behind Flipper flag Add Visits::StayPointDetector, a deterministic single-pass dwell detector, gated per-user by the :stay_point_detection Flipper flag (default OFF). When the flag is off, the existing DBSCAN path is byte-for-byte unchanged (SmartDetect selects the detector, with a rescue->DBSCAN fallback). Also adds: - Visits::ConfidenceScorer: stored 0-100 confidence + breakdown on visits, exposed in Api::VisitSerializer (null when flag off). - PlaceFinder ranking upgrade (exact-name > manual > distance), flag-gated, to stop duplicate "Suggested place" rows. - stay_max_gap_minutes setting; migration adds visits.confidence / confidence_breakdown. No machine learning. Device-evidence (WiFi/charging/velocity) deferred: production fill rates too low (bssid 6.9%, wifi 7%, charging ~5%). * chore: address PR review — clean schema diff, CHANGELOG, upgrade-safe flag registration - Restore db/schema.rb to dev baseline + only the visits.confidence columns; drop the accidental demo-column removal and provider/uid reorder that a stale schema regen had introduced. - Add CHANGELOG entry for the opt-in stay-point detector. - Register :stay_point_detection via a guarded initializer (covers existing installs on upgrade), and remove the db/seeds.rb-only registration. - Add stay_point_detector spec case (m): drift re-anchor across a long gap keeps the original visit start (first vs drift_ref separation). * docs: add CHANGELOG entry; remove duplicate seeds flag registration (initializer covers it) * fix: use newest migration timestamp so schema version stays consistent with dev * fix: families locations history returns 403 (not 404) for non-family users GET /api/v1/families/locations/history now returns 403 Forbidden when the requester is not part of a family, matching the documented swagger contract. The index endpoint keeps its existing 404 (its request + swagger specs assert 404), so the two endpoints get separate before_action guards. * docs: move stay-point detection changelog to 1.8.0 with enable instructions * revert: keep families locations history at 404 for non-family users Restore the controller to a single ensure_user_in_family! guard returning 404 for both index and history; align the swagger spec/yaml to 404 and drop the changelog entry. Reverts the 403 change from 5b11e46. --------- Co-authored-by: Guillermo Guirao Aguilar <ggaguilar@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> * Fix anomaly filter performance issue that caused reverse geocoding to stall * Implement visit place search feature (Freika#2846) * Implement visit place search feature * Update changelog * Bias place search to visit location and cap place name length * Order place search results by distance, nearest first * Add utm parameters to links in monthly digest email * Make native form control icons legible on dark theme (Freika#2857) The dark theme now declares a dark color-scheme so the browser renders calendar/clock indicators light-on-dark instead of a near-invisible dark glyph. Closes Freika#2765 * Size map layout to dynamic viewport height on mobile (Freika#2858) Bottom mobile menu items were hidden behind the browser address bar. The map layout now uses the dynamic viewport height so content stays reachable when mobile browser chrome is visible. Closes Freika#2249 * Remove orphaned tracks when deleting an import (Freika#2856) Deleting an import left tracks with no points behind as empty ghost tracks on the map and timeline. They are now cleaned up, including a migration for existing orphans. Closes Freika#2825 * Support upstream Traccar client nested payload (Freika#2854) The official Traccar client nests coordinates, battery and activity under location.coords / location, one level deeper than Dawarich's own mobile client. Those points were silently dropped. Accept both payload shapes. Closes Freika#2741 * Redraw route after deleting a single point on the map (Freika#2855) Deleting a point via its info card left a stale connecting line through the removed point until reload. The route now redraws immediately. Closes Freika#2844 * fixed default value of RAILS_ENV (Freika#2853) fixed default value of RAILS_ENV to be inline with the change from development to production in 1.3.0 * Add security scanners (Freika#2851) * Add security scanners * Fix CI: bump setup-ruby (knows 3.4.6) and trivy-action - ruby/setup-ruby -> v1.310.0 (older pin lacked Ruby 3.4.6, failing Brakeman and bundler-audit on ubuntu-24.04) - aquasecurity/trivy-action -> v0.36.0, pin trivy binary v0.65.0 (old action's installer aborted before scanning) - authenticate Trivy ghcr DB/policy pulls via GITHUB_TOKEN - make report-only Trivy config job resilient to scanner infra errors * Fix Trivy image build: npm install before docker build assets:precompile runs tailwindcss which requires the daisyui node module; the Dockerfile copies node_modules from the build context, so deps must be installed first (mirrors build_and_push.yml). * Fix Trivy install: authenticate GitHub API via GITHUB_TOKEN The Trivy install script resolves the release tag through the GitHub API, which is rate-limited (60/hr) when unauthenticated and was failing the image job. Pass GITHUB_TOKEN so the lookup is authenticated. * Run Trivy from pinned container; image scan report-only The trivy-action installer (downloads the binary from GitHub releases) flaked deterministically in the image job after the heavy docker build. Switch both Trivy jobs to the pinned ghcr.io/aquasecurity/trivy:0.65.0 container (image scan via docker socket), authenticated to ghcr with GITHUB_TOKEN. Image scan is report-only on the first pass (SARIF still uploads); flip --exit-code to 1 to gate on fixable HIGH/CRITICAL. * feat: Sign in with Apple on the web (cloud only) (Freika#2764) * feat: Sign in with Apple on the web (cloud only) Mirrors the existing iOS Apple Sign-in flow for the web app. Cloud-only; self-hosted instances do not enable the button or routes. Reuses the existing Auth::VerifyAppleToken and Auth::FindOrCreateOauthUser services so the email-collision + verified-email guarantees match the other OAuth providers. Routes & controller - GET /users/auth/apple → Users::AppleOauthController#request_phase - POST /users/auth/apple/callback → Users::AppleOauthController#callback - form_post response_mode (Apple posts the id_token back); CSRF skipped only on callback since Apple is the originator - nonce + state stored in encrypted cookies (httponly, SameSite=None, secure) for the callback round-trip; constant-time state compare - POST callback rate-limited to 20/min per IP (Rack::Attack) Token verification - Auth::VerifyAppleToken now accepts an explicit client_id: kwarg — iOS continues to verify against APPLE_BUNDLE_ID, web verifies against APPLE_WEB_SERVICES_ID User creation - New first_name/last_name columns on users; Apple posts user[name] exactly once on first authorization, so we capture and persist it - Omniauthable#omniauth_name_attrs extracts first/last from any provider's omniauth payload (uses info.name as fallback) so the same pipeline is wired for Google/GitHub when they later provide names - Auth::FindOrCreateOauthUser accepts name_attrs: and writes them on user creation Feature flag & assets - New APPLE_WEB_SIGN_IN_ENABLED constant flips on when all four envs are set (APPLE_WEB_SERVICES_ID, APPLE_WEB_TEAM_ID, APPLE_WEB_KEY_ID, APPLE_WEB_P8_BASE64), and only on cloud - Sign-in / sign-up forms render the Apple button when enabled - Empty apple-developer-domain-association.txt placeholder (real file will be uploaded out-of-band by ops) * Implement web Sign in with Apple flow * Harden Apple web sign-in config and cancellation UX - Require APPLE_WEB_REDIRECT_URI in the enable gate so the feature cannot turn on with an unset redirect URI (the auto-derived fallback can emit http:// behind a proxy, which Apple rejects). - Check the Apple error param before state validation so a cancelled sign-in shows a friendly message even after the state cookie expires. - Add CHANGELOG entry for Sign in with Apple on the web. * Address review: drop dead redirect fallback, remove placeholder, cover rescue paths - Remove the dead apple_oauth_callback_url fallback now that APPLE_WEB_REDIRECT_URI is required by the enable gate. - Remove the committed apple-developer-domain-association.txt placeholder; ops uploads the real file out-of-band before enabling the flag. - Add request specs for the UnverifiedEmail and MissingOauthEmail callback rescue paths. * Make name-columns migration idempotent; drop DEVELOPMENT.md notes - add_column with if_not_exists: true so re-runs are safe. - Revert the Sign in with Apple section added to DEVELOPMENT.md. * feat: per-user changelog widget consent with cloud auto-show (Freika#2849) * feat: per-user changelog widget consent with cloud auto-show Add an opt-in flow for the chibichange "What's New" widget. On self-hosted instances each user is prompted once and their choice is recorded in users.changelog_consent (nullable enum); the widget script is rendered only when granted, so decliners never contact chibichange. On Cloud, signed-in users see the changelog with no prompt. Decline/signed-out keep the existing native GitHub-release badge. * fix: settings-page changelog opt-in, cloud opt-out, 422 on invalid decision Add a Settings -> General control to turn What's New notices on or off, so the navbar prompt's 'change anytime' copy is actually true and Cloud users can opt out. The indicator helper now honours an explicit decline in every mode. The consent endpoint returns 422 on an unknown decision instead of a silent 200. Revert unrelated schema.rb column reorderings, drop the committed plan file, and add a CHANGELOG entry documenting the feature and the CHIBICHANGE_* vars. * fix: make changelog_consent migration idempotent Guard add_column with if_not_exists: true so re-running the migration is safe. * Filter map Places layer to manual, confirmed, and tagged places (Freika#2861) * Filter map Places layer to manual, confirmed, and tagged places Every suggested visit mints a photon Place, so GET /api/v1/places returned a marker for each one and flooded the map's Places layer (both v1 and v2 fetch from this endpoint). Add Place.map_visible(user) — manual OR linked to a confirmed visit OR tagged — and apply it as the default in PlacesController#index. A filter param (all/manual/confirmed/tagged) overrides the default for API consumers that still want the full list. * Document places index filter param in swagger * Update replay panel UI and remove day navigation buttons * Update version * Fix minor stuff * Update changelog consent copy to disclose widget host and data sent, add test for mobile handoff secret fallback * Update chibichange script * fix: harden 1.8.0 orphaned-track cleanup and places search (Freika#2877) * fix: harden 1.8.0 orphaned-track cleanup and places search - Move orphaned-track removal into DataMigrations::DestroyOrphanedTracksJob so the migration enqueues async work instead of holding one long transaction lock over tracks/track_segments at deploy time - Use delete_all for tracks and track_segments in Imports::Destroy (segments first) instead of find_each(&:destroy) - Cap Places::Search query length at 200 chars before forwarding to Photon - Document in the changelog that existing accounts will show far fewer Places-layer markers (suggested-only places hidden, not deleted) * fix: broadcast destroyed track on all delete paths Extract Track.delete_orphaned to bulk-delete segments + tracks and replay the destroyed broadcast (delete_all skips the after_destroy callback), so live maps drop the removed route immediately instead of after a reload. Wire both Imports::Destroy and DestroyOrphanedTracksJob to it. * chore: fix 1.8.0 CI failures (Freika#2878) - Bump puma 7.2.0 -> 8.0.2 (CVE-2026-47736, CVE-2026-47737: PROXY Protocol v1 remote memory exhaustion / header smuggling) - Format public/.well-known/assetlinks.json with Biome * Add a GeoJSON generator for e2e tests, and use it to seed demo data with current timestamps. * test: fix stale export-import spec after user_data files excluded from archive (Freika#2879) The export pipeline skips embedding a user_data export's own file in the archive, but the integration spec still seeded only one non-user_data export with a file while asserting >= 2 restored. Make the second seeded export a points export so two real export files round-trip again. (Re-applies the fix dropped when Freika#2878 squash-merged from a lagging head.) * Enable globe view by default * Update changelog * fix: align activity heatmap weekday labels with their grid rows (Freika#2902) Closes Freika#2896 * fix: keep small monthly distance bars visible and hoverable (Freika#2903) Closes Freika#2864 * fix: refresh anomalies layer after select-area point deletions (Freika#2905) Closes Freika#2790 * fix: ignore countries without qualifying cities in yearly first-time visits (Freika#2906) Closes Freika#2727 * fix: recover imports stuck in the deleting state (Freika#2907) Closes Freika#2835 * chore: point the suggested-visit empty state at place search (Freika#2908) Closes Freika#2852 * fix: strip well-known suffix from OIDC issuer (Freika#2911) Closes Freika#2056 * Update screenshots and timeline view in README * Audit fixes: CI pipeline, Lite plan gating, points-API and stats performance (Freika#2915) * chore: make CI run the RSpec suite * chore: bump oauth2 to 2.0.22 (GHSA-pp92-crg2-gfv9) * fix: correct Lite point-count headers, use spatial index for bbox filter * fix: apply Lite data-window scoping to web /points page * fix: do not mark completed imports failed when post-import processing raises * perf: batch per-visit point counts in timeline day assembly * perf: compute point count and bounds extent in one query * chore: remove debug logs from TimeChunkProcessorJob * perf: single-scan H3 hexagon calculation without AR instantiation * docs: fix DEVELOPMENT.md typo, add native setup section and .env.example * Update changelog * fix: rebuild routes after moving a point instead of patching nearby vertices (Freika#2913) Closes Freika#2150 * Fix Google Timeline import failing on invalid UTF-8 (Freika#2909) * fix: scrub invalid UTF-8 bytes when loading JSON imports Closes Freika#2772 * fix: scrub invalid UTF-8 in load_file_content to cover OwnTracks and TCX * docs: narrow changelog claim to importers actually covered by UTF-8 scrub * Fog of War: hexagon rendering mode (Freika#2899) * feat: add fog_of_war_mode user setting * feat: add Maps::FogHexagons precalculated cell lookup * feat: add fog hexagons API endpoint * feat: plumb fogOfWarMode setting and fog hexagons API client * feat: hexagon rendering mode for fog of war layer * feat: fog of war mode toggle in map settings panel * fix: avoid duplicate fog hexagon fetch on map data reload * refactor: apply fog of war mode immediately from settings dropdown * fix: harden fog hexagon mode after review Clear hexagon fetch state on failure so the layer retries instead of rendering opaque fog, render the saved fog mode in the settings panel, rescue unexpected errors in the fog endpoint, add changelog entry. * fix: require auth on fog endpoint for sharing uuid requests and document it Scope public sharing access to index/bounds so a uuid param on the fog action returns 401 instead of 500, and add the fog endpoint to the Swagger docs. * fix: skip legacy hash-shaped h3_hex_ids in fog hexagons Stats with the legacy { area_too_large: true } shape leaked the hash key as an H3 cell id, which threw client-side and left hexagon fog opaque. Guard the service against non-array shapes, filter invalid cell ids on the client, and cover fog_of_war_mode in the settings round-trip specs. * chore: favicon refresh + e2e onboarding suppression (Freika#2900) * chore: refresh favicons and app icons; skip onboarding modal for e2e users * chore: apply new light/dark favicon sets and rounded navbar logo - Replace all favicons and app icons with the new logo exports, adding dark-mode variants served via prefers-color-scheme media queries in the favicon partial - Fix site.webmanifest to reference non-digested public icon paths (old digested asset URLs were stale) and add maskable icons - Add rounded logo to the navbar brand and bump its sup from α to β * fix: recover from duplicate yearly digests during data recalculation (Freika#2901) Closes Freika#2866 * fix: persist anomalies layer visibility across reloads (Freika#2904) Closes Freika#2791 * Make Map v2 replay playback proportional to real elapsed time (Freika#2910) * fix: drive replay playback proportionally to real elapsed time Closes Freika#2845 * fix: apply replay pacing immediately on seek and speed change Scrubber seeks and mid-segment speed changes kept the previous segment's cached duration until the next point boundary; recompute it on both, preserving in-segment progress so the marker doesn't jump. * Fix marker z-index * fix: disable /admin/flipper rate limiting on self-hosted instances (Freika#2920) The 30 req / 5 min throttle made the feature-flag UI unusable for self-hosters after a few clicks. Keep the throttle for Cloud only. Fixes Freika#2897 * fix: delete location-sharing requests when deleting a family (Freika#2919) Family deletion raised a 500 because Family::LocationRequest rows referencing the family were never cleaned up. Add the association with dependent: :delete_all and a regression spec. Fixes Freika#2916 * 1.8.1 release review follow-ups (Freika#2921) * fix: 1.8.1 release review follow-ups - Bump .app_version to 1.8.1 and finalize the CHANGELOG section - Isolate post-import steps so one failing step no longer skips the rest (each step rescued and reported individually) - Drop a leftover invalid index before the concurrent unique index build in the yearly-digest dedupe migration, so a failed first run can be retried * test: assert post-import failures are reported with the step name * Update changelog * Update env.example * fix: 1.8.1 release review follow-ups, round 2 (Freika#2923) - Read hexagon-stat coordinates in bounded keyset batches so point-heavy months no longer materialize the whole month in worker memory - Notify the user once when a post-import step fails, instead of leaving a "successful" import silently missing stats/tracks/suggestions - Allow the yearly digest calculation up to three attempts when concurrent recalculations collide on the unique index - Build fog-of-war hexagon boundaries lazily per viewport instead of eagerly for every explored cell on each resolution change - Add a 1.8.1 upgrade note about the yearly-digest dedupe migration --------- Co-authored-by: Guillermo Guirao Aguilar <ggaguilar@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: joschreiner <87669137+joschreiner@users.noreply.github.com>
1.8.0 (Freika#2837) * feat: non-ML stay-point visit detection behind Flipper flag (Freika#2832) * Return 404 instead of 403 for users not in a family The frontend pings family endpoints by default. When the feature is unconfigured, repeated 403s look like brute-force attempts and trip firewall blocks. 404 reflects that the resource simply isn't there for that user. * Remove transitional trial-reminder mailers (Manager owns lifecycle now) Removes the four email types kept temporarily in PR Freika#2508 to let in-flight Sidekiq jobs drain cleanly. Today (2026-05-17) is the earliest safe removal date named in those source comments. Deleted: - MAILER_REGISTRY entries: trial_expires_soon, trial_expired, post_trial_reminder_early, post_trial_reminder_late - Corresponding mailer methods and the should_skip_email? guard - Eight email templates (.html.erb / .text.erb) - All tests and previews specific to these four types - The four types are removed from user_spec "not enqueued" lists too (they're gone; only pure Manager-owned billing types remain) * Harden 1.7.10 release: PII filtering, area validations, posthog guard - Add email, name, first_name, last_name, phone to filter_parameters so request params attached to error-tracking payloads (PostHog, Sentry) and Rails logs are redacted on auth/signup/profile flows. - Skip PostHog initialization when POSTHOG_API_KEY is blank to avoid initializing the client with a nil key. - Validate Area numericality: radius > 0, latitude in [-90, 90], longitude in [-180, 180]. The client-side min on the modal is bypassable via direct PATCH /areas/:id. - Add request specs for PATCH /areas/:id (success, non-positive radius, out-of-range coordinates, cross-user 404). - Document the Cloud-only PostHog telemetry pipeline in CHANGELOG with the field redaction list. * fix: geocode flood (WIP) * fix: timeline calendar points fallback (WIP) * test: cover edge cases for timeline calendar points fallback Adds regression coverage for empty months, points-only bucket clamping, negative UTC offsets, DST spring-forward, and Lite plan out-of-window points. * fix: unblock nightly reverse-geocode rescue and bulk-enqueue dedup NightlyReverseGeocodingJob now passes force: true so points with a lingering dedup key still get re-enqueued. Jobs::Create#bulk_enqueue claims SETNX keys on continue_reverse_geocoding and clears them on start_reverse_geocoding, routing both admin paths through the same dedup. async_reverse_geocode(force:) deletes the key so a forced run doesn't leave a stale claim. * fix: namespace geocode dedup key by class and tighten changelog wording Prefix the Redis dedup key with the class name (geocode:enq:Point:<id>) so a future Place-side SETNX won't silently collide with Point ids. Update the CHANGELOG entry to credit the nightly sweep — not the exhausted job itself — for clearing stale dedup keys. * fix: release dedup key when reverse-geocode enqueue raises If perform_later raises after the SETNX claim (e.g. transient Redis blip between commands), the point was previously locked out for 24 h with no job behind it. Wrap the non-force enqueue path in a rescue that DELs the claimed key(s) and re-raises, so a retry can re-claim immediately instead of waiting for the nightly sweep. * Drop unused points indexes (not_anomaly, track_generation) Frees ~7.4 GB on the points table. - index_points_on_not_anomaly (931 MB): zero lifetime scans; planner always picks (user_id, timestamp) and filters anomaly in heap. - idx_points_track_generation (6.5 GB): used by Track.segment_points_in_sql, but the query runs ~64 times lifetime for 0.7s total. Fallback to index_points_on_user_id_and_timestamp is acceptable. index_points_on_raw_data_archive_id stays — it backs FK enforcement on 44M points referencing 1422 archives, even though idx_scan is currently low. index_points_on_unarchived stays — ARCHIVE_RAW_DATA=true on prod and the monthly archiver query matches it exactly per EXPLAIN. * Bulk-insert track segments (Fixes DAWARICH-P3) Replaces the per-segment .create loop in Tracks::TrackBuilder, TransportationModes::BackfillJob, and Tracks::Reprocessor with a single TrackSegment.insert_all via TrackSegments::BulkInserter. Resolves the N+1 INSERT pattern reported in Sentry DAWARICH-P3 from Tracks::TimeChunkProcessorJob. Each track now issues one batched INSERT instead of N round-trips. - TrackSegments::BulkInserter converts :mode and :confidence symbols to enum integers via .fetch (fail-fast on unknown values). - dominant_mode is computed from segment_data in-memory before insert, removing the need to reload created records. - Returns the input segment_data so callers can inspect what was written. * Drop per-pair ST_Distance round-trip (Fixes DAWARICH-P1) Point#distance_to issues a single-pair ST_Distance SELECT per call. Called inside transportation-mode segmentation loops in TransportationModes::MovementAnalyzer and SourceDataExtractor, this adds ~ms of DB latency per consecutive point pair — flagged in Sentry DAWARICH-P1 as a slow query from Tracks::TimeChunkProcessorJob. Switch both call sites to the existing in-Ruby Haversine path (geocoder_distance / Point#distance_to_geocoder). No DB round-trip, microsecond-scale per pair, same result. * Drop unused :lonlat from AnomalyFilter point fetch (Fixes DAWARICH-NJ) Points::AnomalyFilter#fetch_points_with_context selected id + lonlat + timestamp for every point in a monthly chunk, but Ruby only ever reads point.id afterwards — lonlat is re-fetched inside the calculate_all_speeds CTE by id. For heavy users (>500k points/month) the unused PostGIS geography bytes dominate transfer cost. Dropping :lonlat from the SELECT reduces row payload by ~30+ bytes and addresses the slow SELECT flagged in Sentry DAWARICH-NJ from Points::AnomalyFilterJob. * Fix Map v2 visits ignoring date range when bbox is sent Visits::FindWithinBoundingBox now applies start_at/end_at when both params parse. Since 1.7.10 the Map v2 visits fetch always passes the viewport bbox, which routed every request through this service. The service ignored the date range entirely, so the layer rendered all visits inside the viewport regardless of the selected date filter. Closes Freika#2817 * Align BackfillJob dominant-mode logic with TrackBuilder After rebase TrackBuilder uses Track.pick_dominant_mode (distance-weighted, filters stationary) but BackfillJob still picked the longest-duration segment. Same input would produce different dominant_mode on the two paths. Route BackfillJob through the same in-memory-segment + pick_dominant_mode bridge. Also: spec coverage for TrackBuilder#update_dominant_mode (empty, mixed moving+stationary, stationary fallback, nil distance/duration) and a CHANGELOG entry for the perf work + index drop. * chore: bump gems to close 9 known CVEs bundle-audit reports zero vulnerabilities after these bumps. Direct bumps in Gemfile: - rails ~> 8.0.5 (CVE-2026-33195/33202 ActiveStorage DiskService, CVE-2026-33167/33168/33169/33170/33173/33174/33658) - devise >= 5.0.4 (CVE-2026-40295 open redirect via timeoutable session referrer; CVE-2026-32700) - jwt ~> 2.10.3 (CVE-2026-45363 HMAC bypass) - httparty >= 0.24.0 (CVE-2025-68696 SSRF + API key leakage) - sidekiq-cron >= 2.4.0 (CVE-2025-67202 admin XSS) - aws-sdk-core ~> 3.234 + aws-sdk-s3 ~> 1.208 (CVE-2025-14762; core bumped to satisfy s3 dep graph) - addressable >= 2.9.0 (CVE-2026-35611) - bcrypt >= 3.1.22 (CVE-2026-33306) - faraday >= 2.14.2 (CVE-2026-33637 protocol-relative URI scoping bypass) - json >= 2.19.2 (CVE-2026-33210 format string DoS) - net-imap >= 0.5.14 (CVE-2026-42257/42258 CRLF injection) - zlib >= 3.2.3 (CVE-2026-27820 GzipReader buffer overflow) Transitive bumps pulled in by the above: - aws-partitions, faraday-net_http, fugit, marcel, responders 5791 RSpec examples pass with zero regressions. * fix(spec): scope place factory to test user in visits PUT specs The three "updating name together with place_id" specs created new_place without a user, so the factory assigned a random one. Since f3dde359 added place_id ownership validation, the controller correctly returns 422 for a foreign place_id, the visit name is never updated, and the assertions fail. Pass user: user to the factory so the specs exercise the name-preservation logic they were written for. * feat: non-ML stay-point visit detection behind Flipper flag Add Visits::StayPointDetector, a deterministic single-pass dwell detector, gated per-user by the :stay_point_detection Flipper flag (default OFF). When the flag is off, the existing DBSCAN path is byte-for-byte unchanged (SmartDetect selects the detector, with a rescue->DBSCAN fallback). Also adds: - Visits::ConfidenceScorer: stored 0-100 confidence + breakdown on visits, exposed in Api::VisitSerializer (null when flag off). - PlaceFinder ranking upgrade (exact-name > manual > distance), flag-gated, to stop duplicate "Suggested place" rows. - stay_max_gap_minutes setting; migration adds visits.confidence / confidence_breakdown. No machine learning. Device-evidence (WiFi/charging/velocity) deferred: production fill rates too low (bssid 6.9%, wifi 7%, charging ~5%). * chore: address PR review — clean schema diff, CHANGELOG, upgrade-safe flag registration - Restore db/schema.rb to dev baseline + only the visits.confidence columns; drop the accidental demo-column removal and provider/uid reorder that a stale schema regen had introduced. - Add CHANGELOG entry for the opt-in stay-point detector. - Register :stay_point_detection via a guarded initializer (covers existing installs on upgrade), and remove the db/seeds.rb-only registration. - Add stay_point_detector spec case (m): drift re-anchor across a long gap keeps the original visit start (first vs drift_ref separation). * docs: add CHANGELOG entry; remove duplicate seeds flag registration (initializer covers it) * fix: use newest migration timestamp so schema version stays consistent with dev * fix: families locations history returns 403 (not 404) for non-family users GET /api/v1/families/locations/history now returns 403 Forbidden when the requester is not part of a family, matching the documented swagger contract. The index endpoint keeps its existing 404 (its request + swagger specs assert 404), so the two endpoints get separate before_action guards. * docs: move stay-point detection changelog to 1.8.0 with enable instructions * revert: keep families locations history at 404 for non-family users Restore the controller to a single ensure_user_in_family! guard returning 404 for both index and history; align the swagger spec/yaml to 404 and drop the changelog entry. Reverts the 403 change from 5b11e46. --------- Co-authored-by: Guillermo Guirao Aguilar <ggaguilar@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> * Fix anomaly filter performance issue that caused reverse geocoding to stall * Implement visit place search feature (Freika#2846) * Implement visit place search feature * Update changelog * Bias place search to visit location and cap place name length * Order place search results by distance, nearest first * Add utm parameters to links in monthly digest email * Make native form control icons legible on dark theme (Freika#2857) The dark theme now declares a dark color-scheme so the browser renders calendar/clock indicators light-on-dark instead of a near-invisible dark glyph. Closes Freika#2765 * Size map layout to dynamic viewport height on mobile (Freika#2858) Bottom mobile menu items were hidden behind the browser address bar. The map layout now uses the dynamic viewport height so content stays reachable when mobile browser chrome is visible. Closes Freika#2249 * Remove orphaned tracks when deleting an import (Freika#2856) Deleting an import left tracks with no points behind as empty ghost tracks on the map and timeline. They are now cleaned up, including a migration for existing orphans. Closes Freika#2825 * Support upstream Traccar client nested payload (Freika#2854) The official Traccar client nests coordinates, battery and activity under location.coords / location, one level deeper than Dawarich's own mobile client. Those points were silently dropped. Accept both payload shapes. Closes Freika#2741 * Redraw route after deleting a single point on the map (Freika#2855) Deleting a point via its info card left a stale connecting line through the removed point until reload. The route now redraws immediately. Closes Freika#2844 * fixed default value of RAILS_ENV (Freika#2853) fixed default value of RAILS_ENV to be inline with the change from development to production in 1.3.0 * Add security scanners (Freika#2851) * Add security scanners * Fix CI: bump setup-ruby (knows 3.4.6) and trivy-action - ruby/setup-ruby -> v1.310.0 (older pin lacked Ruby 3.4.6, failing Brakeman and bundler-audit on ubuntu-24.04) - aquasecurity/trivy-action -> v0.36.0, pin trivy binary v0.65.0 (old action's installer aborted before scanning) - authenticate Trivy ghcr DB/policy pulls via GITHUB_TOKEN - make report-only Trivy config job resilient to scanner infra errors * Fix Trivy image build: npm install before docker build assets:precompile runs tailwindcss which requires the daisyui node module; the Dockerfile copies node_modules from the build context, so deps must be installed first (mirrors build_and_push.yml). * Fix Trivy install: authenticate GitHub API via GITHUB_TOKEN The Trivy install script resolves the release tag through the GitHub API, which is rate-limited (60/hr) when unauthenticated and was failing the image job. Pass GITHUB_TOKEN so the lookup is authenticated. * Run Trivy from pinned container; image scan report-only The trivy-action installer (downloads the binary from GitHub releases) flaked deterministically in the image job after the heavy docker build. Switch both Trivy jobs to the pinned ghcr.io/aquasecurity/trivy:0.65.0 container (image scan via docker socket), authenticated to ghcr with GITHUB_TOKEN. Image scan is report-only on the first pass (SARIF still uploads); flip --exit-code to 1 to gate on fixable HIGH/CRITICAL. * feat: Sign in with Apple on the web (cloud only) (Freika#2764) * feat: Sign in with Apple on the web (cloud only) Mirrors the existing iOS Apple Sign-in flow for the web app. Cloud-only; self-hosted instances do not enable the button or routes. Reuses the existing Auth::VerifyAppleToken and Auth::FindOrCreateOauthUser services so the email-collision + verified-email guarantees match the other OAuth providers. Routes & controller - GET /users/auth/apple → Users::AppleOauthController#request_phase - POST /users/auth/apple/callback → Users::AppleOauthController#callback - form_post response_mode (Apple posts the id_token back); CSRF skipped only on callback since Apple is the originator - nonce + state stored in encrypted cookies (httponly, SameSite=None, secure) for the callback round-trip; constant-time state compare - POST callback rate-limited to 20/min per IP (Rack::Attack) Token verification - Auth::VerifyAppleToken now accepts an explicit client_id: kwarg — iOS continues to verify against APPLE_BUNDLE_ID, web verifies against APPLE_WEB_SERVICES_ID User creation - New first_name/last_name columns on users; Apple posts user[name] exactly once on first authorization, so we capture and persist it - Omniauthable#omniauth_name_attrs extracts first/last from any provider's omniauth payload (uses info.name as fallback) so the same pipeline is wired for Google/GitHub when they later provide names - Auth::FindOrCreateOauthUser accepts name_attrs: and writes them on user creation Feature flag & assets - New APPLE_WEB_SIGN_IN_ENABLED constant flips on when all four envs are set (APPLE_WEB_SERVICES_ID, APPLE_WEB_TEAM_ID, APPLE_WEB_KEY_ID, APPLE_WEB_P8_BASE64), and only on cloud - Sign-in / sign-up forms render the Apple button when enabled - Empty apple-developer-domain-association.txt placeholder (real file will be uploaded out-of-band by ops) * Implement web Sign in with Apple flow * Harden Apple web sign-in config and cancellation UX - Require APPLE_WEB_REDIRECT_URI in the enable gate so the feature cannot turn on with an unset redirect URI (the auto-derived fallback can emit http:// behind a proxy, which Apple rejects). - Check the Apple error param before state validation so a cancelled sign-in shows a friendly message even after the state cookie expires. - Add CHANGELOG entry for Sign in with Apple on the web. * Address review: drop dead redirect fallback, remove placeholder, cover rescue paths - Remove the dead apple_oauth_callback_url fallback now that APPLE_WEB_REDIRECT_URI is required by the enable gate. - Remove the committed apple-developer-domain-association.txt placeholder; ops uploads the real file out-of-band before enabling the flag. - Add request specs for the UnverifiedEmail and MissingOauthEmail callback rescue paths. * Make name-columns migration idempotent; drop DEVELOPMENT.md notes - add_column with if_not_exists: true so re-runs are safe. - Revert the Sign in with Apple section added to DEVELOPMENT.md. * feat: per-user changelog widget consent with cloud auto-show (Freika#2849) * feat: per-user changelog widget consent with cloud auto-show Add an opt-in flow for the chibichange "What's New" widget. On self-hosted instances each user is prompted once and their choice is recorded in users.changelog_consent (nullable enum); the widget script is rendered only when granted, so decliners never contact chibichange. On Cloud, signed-in users see the changelog with no prompt. Decline/signed-out keep the existing native GitHub-release badge. * fix: settings-page changelog opt-in, cloud opt-out, 422 on invalid decision Add a Settings -> General control to turn What's New notices on or off, so the navbar prompt's 'change anytime' copy is actually true and Cloud users can opt out. The indicator helper now honours an explicit decline in every mode. The consent endpoint returns 422 on an unknown decision instead of a silent 200. Revert unrelated schema.rb column reorderings, drop the committed plan file, and add a CHANGELOG entry documenting the feature and the CHIBICHANGE_* vars. * fix: make changelog_consent migration idempotent Guard add_column with if_not_exists: true so re-running the migration is safe. * Filter map Places layer to manual, confirmed, and tagged places (Freika#2861) * Filter map Places layer to manual, confirmed, and tagged places Every suggested visit mints a photon Place, so GET /api/v1/places returned a marker for each one and flooded the map's Places layer (both v1 and v2 fetch from this endpoint). Add Place.map_visible(user) — manual OR linked to a confirmed visit OR tagged — and apply it as the default in PlacesController#index. A filter param (all/manual/confirmed/tagged) overrides the default for API consumers that still want the full list. * Document places index filter param in swagger * Update replay panel UI and remove day navigation buttons * Update version * Fix minor stuff * Update changelog consent copy to disclose widget host and data sent, add test for mobile handoff secret fallback * Update chibichange script * fix: harden 1.8.0 orphaned-track cleanup and places search (Freika#2877) * fix: harden 1.8.0 orphaned-track cleanup and places search - Move orphaned-track removal into DataMigrations::DestroyOrphanedTracksJob so the migration enqueues async work instead of holding one long transaction lock over tracks/track_segments at deploy time - Use delete_all for tracks and track_segments in Imports::Destroy (segments first) instead of find_each(&:destroy) - Cap Places::Search query length at 200 chars before forwarding to Photon - Document in the changelog that existing accounts will show far fewer Places-layer markers (suggested-only places hidden, not deleted) * fix: broadcast destroyed track on all delete paths Extract Track.delete_orphaned to bulk-delete segments + tracks and replay the destroyed broadcast (delete_all skips the after_destroy callback), so live maps drop the removed route immediately instead of after a reload. Wire both Imports::Destroy and DestroyOrphanedTracksJob to it. * chore: fix 1.8.0 CI failures (Freika#2878) - Bump puma 7.2.0 -> 8.0.2 (CVE-2026-47736, CVE-2026-47737: PROXY Protocol v1 remote memory exhaustion / header smuggling) - Format public/.well-known/assetlinks.json with Biome * Add a GeoJSON generator for e2e tests, and use it to seed demo data with current timestamps. * test: fix stale export-import spec after user_data files excluded from archive (Freika#2879) The export pipeline skips embedding a user_data export's own file in the archive, but the integration spec still seeded only one non-user_data export with a file while asserting >= 2 restored. Make the second seeded export a points export so two real export files round-trip again. (Re-applies the fix dropped when Freika#2878 squash-merged from a lagging head.) * Enable globe view by default * Update changelog --------- Co-authored-by: Guillermo Guirao Aguilar <ggaguilar@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: joschreiner <87669137+joschreiner@users.noreply.github.com>
PreviousNext