8000 GitHub - ange-daumal/my_spotiPy · GitHub
[go: up one dir, main page]

Skip to content

Navigation Menu

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Repository files navigation

my_spotiPy

The existing notebooks stay in notebooks/. The CLI adds an interactive workflow: log in to Spotify → select a playlist → explore and try sorting methods → preview → optionally apply.

Getting started

From the project directory:

poetry install
poetry run python -m my_spotipy demo

The demo runs offline with sample tracks. You can open IPython and return with exit. Spotify actions are disabled in demo mode.

For the existing notebooks, use poetry run jupyter notebook to run Jupyter in the Poetry environment.

The CLI uses ReccoBeats for audio data by default. Spotify supplies playlists and receives changes only after confirmation. No ReccoBeats account is needed.

Credentials in 1Password

An op://vault/item/field reference tells the CLI where to read a value in 1Password; it does not contain the secret. Sections may add a path segment, and copied references may use IDs. Paste the exact reference provided by the app.

  1. On macOS, install the CLI with brew install 1password-cli.
  2. In 1Password, open Settings → Developer → Integrate with 1Password CLI. Touch ID can authorize access.
  3. Create a Spotify item (for example, API Credential) in your chosen vault. Add client_id and client_secret fields containing your Spotify app credentials; use a password field for the secret. Save the item.
  4. Open each field's dropdown menu and choose Copy Secret Reference.
  5. Copy spotipy.config.example.json to spotipy.config.json and replace the references with your own.
{
  "client_id": "op://Private/Spotify/client_id",
  "client_secret": "op://Private/Spotify/client_secret",
  "redirect_uri": "http://127.0.0.1:8765/callback"
}

The CLI reads this file from the current directory and resolves references using op read, without printing secrets or writing their values into the configuration. 1Password may request authorization. Secret rotation takes effect on the next connected launch if the reference stays the same.

Alternatively, export SPOTIPY_CLIENT_ID and SPOTIPY_CLIENT_SECRET, containing either values or references. The legacy c_id and c_secret variables also work. Precedence: SPOTIPY_*, then legacy variables, then JSON configuration. The CLI does not load .env automatically; use op run --env-file .env -- poetry run python -m my_spotipy explore if that file contains 1Password references.

References: 1Password CLI setup, secret references, copying a reference in the UI.

Spotify login

In your app settings in the Spotify dashboard, add this exact Redirect URI:

http://127.0.0.1:8765/callback

Then run:

poetry run python -m my_spotipy login
poetry run python -m my_spotipy explore

The first login opens a browser for Spotify authorization. The callback is captured automatically on port 8765. This differs from Jupyter's usual port 8888 so both can run together. If the port is occupied, configure another port in both Spotify and your JSON configuration. For an older configuration using 8888, update the redirect URI in both places.

OAuth tokens are cached in .spotipy-cli/ with private permissions and refreshed by Spotipy. The client secret stays in 1Password; the Spotify token is a separate local cache.

For a separate browser, run poetry run python -m my_spotipy --no-browser login and paste the full callback URL when prompted. Global options --config and --no-browser go before the command.

Guided menu and IPython

# List playlists, or select one interactively
poetry run python -m my_spotipy playlists
poetry run python -m my_spotipy explore

# Playlist IDs, spotify:playlist:… URIs and URLs are accepted
poetry run python -m my_spotipy explore PLAYLIST_ID

# Open IPython first; use exit to return to the menu
poetry run python -m my_spotipy explore PLAYLIST_ID --console

# Use Spotify audio data if your app still has access
poetry run python -m my_spotipy explore PLAYLIST_ID --audio-source spotify

# Explore your liked tracks
poetry run python -m my_spotipy explore --saved

# Keep the last two tracks in place
poetry run python -m my_spotipy explore PLAYLIST_ID --ignore-last 2

# Preview/export without an interactive session or Spotify writes
poetry run python -m my_spotipy preview PLAYLIST_ID --sort-by name --ascending --no-audio --csv order.csv

Playlist selection shows 25 results at a time in Spotify order. Enter a number to select, n for the next page, or q to go back. Numbering continues across pages. The playlists command lists all playlists.

Use p — Select another playlist from the connected menu to switch without restarting or logging in again. The new playlist gets a fresh working DataFrame, baseline, preview and trial history, using your launch options for sorting and audio enrichment. Switching does not apply the previous preview. Cancelling selection or a loading error keeps your current session. This option also works from liked tracks; it is unavailable in the offline demo.

Fetched data is cached locally in .spotipy-cli/data-cache/, across playlist switches and CLI launches, with these default lifetimes:

Data Cache lifetime
Audio features (ReccoBeats or Spotify) Up to 90 days
Last.fm track/artist tags Up to 30 days
Contributor names Up to 30 days
Popularity Up to 7 days
Successful empty lookups Up to 24 hours
Complete playlist order While Spotify's current snapshot matches
Partial playlist checkpoints Up to 24 hours, with a matching snapshot

HTTP cache directives can shorten these lifetimes. Cache-Control: max-age, Age, Date and Expires determine freshness; no-store and no-cache prevent reuse and remove any older cached value for that lookup. Expired responses are fetched again. This follows the cache guidance in Spotify's documentation and Last.fm's API terms. Old cache records without HTTP metadata retain their original 24-hour lifetime; new fetches use the policy above, without clearing the existing cache.

Audio features and popularity metadata are shared by track ID; contributor names are shared by user ID. Playlist contents are reused only after a live Spotify check confirms the snapshot is unchanged. The earliest HTTP expiry across its pages also limits reuse. Popularity has a separate cache: an unchanged playlist does not keep an old popularity value fresh. Expired Spotify popularity becomes missing and can be filled from ReccoBeats when that enrichment is enabled. Each received playlist page is checkpointed: if a load is interrupted, reopening the same playlist resumes from the next page when the snapshot still matches. A changed snapshot triggers a fresh track listing. Playlist selection and liked-track listings remain live, and applying always checks Spotify's current snapshot.

The CLI prints cache-hit/fetch counts for audio and metadata. Successful empty lookup results are cached too; network failures, rate limits and malformed responses are not. The cache contains fetched source data, not p50 estimates or local edits, and is separate from the OAuth token cache. Unreadable/expired entries are fetched again; a cache write failure does not prevent exploration.

To bypass the data cache and replace entries with fresh results:

poetry run python -m my_spotipy explore --refresh

--refresh bypasses every data-cache lifetime and also applies when switching playlists during that run. --no-audio skips audio/popularity enrichment; --no-genres separately skips Last.fm. The normal r action reloads the playlist with the current snapshot check, retains the session's audio values and looks up genres using their cache; use a fresh launch with --refresh to refetch all enrichment.

The first table immediately shows the first 20 tracks in the proposed final order, with Δ rank: +5 means five places up, -2 means two places down, and 0 means unchanged. Each successful trial compares with the preceding preview; the first preview compares with the loaded Spotify order. The reference is named above the table. Reopening c does not change that reference. Counts cover the whole playlist, and c shows the full result. The added by column identifies contributors when available.

Choose an algorithm directly from the main menu; each choice immediately shows the final order and rank changes:

  • 1 or Enter: descending popularity with alternating energy, high then low (the default, based on the notebook).
  • 2: descending tempo.
  • 3: ascending tempo.
  • 4: alternating energy, starting low.
  • 5: descending tempo with alternating energy, starting low.
  • 6: distribute contributions across the current sorting recipe (optional).
  • 7: custom settings, including starting energy and trailing tracks to keep in place.

Sorting presets start from the working data without reloading Spotify and keep the current trailing-track setting. Contribution distribution runs after the current recipe; choosing another preset disables it again. Use h to restore an earlier trial or o to restore the loaded order. Both immediately display the result. i opens IPython. History stays in memory until applying, reloading or switching playlists establishes a new baseline. 0 or q quits.

Use s for Statistics, e for Export, g for Histogram, and p to select another playlist. a — Apply to Spotify compares the preview against the current Spotify baseline (the last loaded or successfully applied order), then asks [Y/n]: Enter or y applies; n cancels. Invalid answers prompt again. After a successful apply, v shows its result and rank changes; both comparison references reset to the newly applied order. Failed trials and cancelled applies do not advance the references. r reloads Spotify and prepares the launch-time sorting recipe again. Quitting never applies changes. l optionally likes the selected tracks, with a separate explicit yes confirmation.

Statistics, exports and histograms use the preview. A .txt export contains “Title - Artist” lines. Existing export files are not overwritten.

With Last.fm configured, interactive explore loads genres in the background. The menu shows progress each time it is displayed; completed batches are collected when displaying, sorting, viewing statistics or exporting. Genre updates preserve the current order, rank differences and trial history. t starts/resumes genre loading using the cache. Switching playlists or quitting cancels the old background job; completed lookups remain cached. An in-flight request may finish before cancellation takes effect.

Use --wait-genres to wait for genre loading before opening the menu, or --no-genres to disable it. preview, --no-interactive and a launch-time --csv wait for genre loading automatically. Exporting from the menu while the worker is active uses the tags available so far and displays a message. In IPython, call s.collect_genres() to update the shared DataFrames with the latest background results; the worker never edits those DataFrames directly.

In IPython:

df.head()
df[['name', 'artist', 'tempo', 'energy']]

# Prepare an order and share it with the menu
result = s.preview(sort_by='tempo', ascending=True, alternate='energy', ignore_last=2)
s.show_comparison()  # final order and signed rank changes
s.compare()          # before/after positions relative to the preceding preview
s.compare(reference='spotify')  # compare against the last loaded/applied Spotify order
s.stats()

# Optional: spread contributors across the current sorting recipe
result = s.preview(distribute_contributions=True)
s.show_comparison()

# Explicitly disable settings
result = s.preview(sort_by='name', ascending=True, alternate=None,
                   ignore_last=0, distribute_contributions=False)

# Use pandas freely; s.result is the preview shared with the menu
s.result = df.sort_values(['artist', 'name'])
s.export('order.csv')
s.histogram('tempo')

# Explore a selection and optionally like those tracks
selection = df.query('130 <= tempo <= 160 and energy >= 0.6')
s.export('selection.csv', selection)
s.like(selection)  # preview and confirmation: type yes

# Apply the current preview, or pass an explicit DataFrame to s.apply(result)
s.apply()         # preview, then [Y/n]: Enter/y applies, n cancels
df = s.df         # refresh positions after applying/reloading
s.last_applied    # last successfully applied comparison

exit              # return to the menu

s is the shared session. df, result and settings are aliases created when entering the console. If you reassign them, explicitly update the session (s.df = df, s.result = result, s.settings = settings). s.preview() reads the working data/settings and updates s.result. After applying or reloading, refresh df = s.df and result = s.result. Comparison DataFrames use the English column names before and after.

Sorting behavior

Without explicit sorting options, the CLI proposes descending popularity with alternating high/low energy. Explicit --sort-by, --alternate or --ascending options replace that default.

When sorting uses popularity or energy, missing values in the preview are filled with that column's p50 (median), calculated from available values among the tracks being sorted, excluding trailing tracks kept in place. Trials preserve the working data and loaded baseline. The CLI reports counts per column and a combined count without double-counting tracks missing both. The result and CSV export mark estimated values with popularity_imputed and energy_imputed. For popularity, 0 is also treated as unknown for sorting: its p50 uses only strictly positive, observed values. Filled zeros are included in the imputation counts. Loaded values remain unchanged, and an energy of zero remains valid. A column with no usable observations (including popularity containing only zeros/nulls) has no calculable p50 and blocks the sort with an explanation. Other columns, including tempo, are not filled automatically.

A failed trial remains visible in the menu, comparison and apply step, with missing columns, counts and example tracks. The previous preview remains available but does not represent the failed trial. “No moves to apply” describes the current preview; it does not mean the requested sort succeeded.

Tables mark each numeric value filled with the median with * (for example, 36* or 0.174*), including zero popularity treated as missing. An English legend distinguishes this from the artist-level genre fallback. DataFrame and CSV values remain numeric; the *_imputed columns retain the flags.

sort_by uses stable sorting. alternate='energy' alternates groups at/above and below the median; start_with='low' or 'high' chooses the first group. High is the Python API and startup default. The menu suggests low for other alternation methods and high for the notebook method. alternate='tempo' alternates BPM groups. Popularity decreases within each energy group, not necessarily between consecutive tracks; the same principle applies to tempo sorting combined with energy alternation. Every track is retained, even with unequal group sizes. ignore_last preserves the last N tracks in their original order.

position identifies each occurrence, so duplicate tracks remain separate rows. Unavailable tracks, local tracks and episodes remain present. Applying an order requires all rows and unchanged position/uri columns. A filtered DataFrame can be explored, exported or liked, but cannot replace the playlist.

Distribute contributions

This optional method uses Spotify's added_by metadata, which identifies the person who added each track, not its performing artist. added_by stores their Spotify ID, and contributor stores their display name when available (otherwise their ID). The CLI looks up names when loading a playlist; a failed name lookup does not block sorting.

Each contributor's tracks are spread evenly across the playlist, in proportion to how many tracks they contributed. Their relative order from the current sorting recipe is preserved. For example, a contributor with 47 of 112 tracks gets about 8 of the first 20 positions instead of having most of their tracks clustered at the end. This does not give everyone equal counts or add/remove tracks. Global energy alternation and popularity descent may change because contribution distribution takes priority.

Unknown contributors stay in their current slots; trailing tracks excluded by ignore_last also stay in place. If all contributor data is missing, the method explains that it cannot run. One known contributor produces no redistribution. Older backups do not contain this metadata; reload from Spotify before using this method. Missing and zero popularity values are filled with the p50 of strictly positive values in the preview. This is a sorting preference, not a claim that every zero returned by Spotify is missing data.

Applying uses Spotify reorder operations, checks that the loaded version is still current, and backs up the original order under .spotipy-cli/backups/. A failure during requests can leave a partially applied order; the CLI reports this and requires a reload. Avoid modifying the same playlist in another client while applying.

Data availability

Genres from Last.fm (optional)

Set lastfm_api_key in spotipy.config.json to your API key or its op://vault/item/section/field reference. Alternatively set LASTFM_API_KEY (takes priority, also accepts a reference). Genre lookups need only the API key, not Last.fm user authorization or the shared secret. An optional lastfm_secret configuration entry is unused: it is never resolved or sent. The key is resolved lazily from 1Password only when a lookup is missing from the cache, and is never written to the data cache.

When configured, explore and preview add these fields (progressively in interactive explore):

  • genre: a broad category such as Pop, Latin, Jazz, Rock, Metal, Blues, Classical, Piano, Instrumental or Unknown.
  • genre_source: lastfm_track, lastfm_artist (fallback) or Unknown.
  • genre_tags: the source tags used for classification, separated by semicolons.

The CLI first requests track tags using the exact title and primary Spotify performer, with autocorrection disabled. If no recognized genre is found, it tries artist tags. The strongest recognized tag determines the broad category. This is a community-tag approximation, not audio analysis: covers use their actual performer, and the original song's genre is not looked up or inferred. Artist tags may describe an artist's overall work poorly for an individual recording. Local tracks, episodes and unclassified tracks remain Unknown; genre is never filled using p50.

Final-order tables show a genre column. A * marks an artist-level fallback, with an explanation above the table. The DataFrame and CSV retain all three fields so you can inspect the tags and source:

df[['name', 'artist', 'genre', 'genre_source', 'genre_tags']]
df[df.genre.isin(['Piano', 'Classical', 'Instrumental'])]

Only titles, performer names and the Last.fm API key are sent to Last.fm, through a separate HTTPS session. Lookups are paced, with progress and coverage counts. Track and artist tags are cached for up to 30 days, with successful empty results kept for up to 24 hours; Last.fm's HTTP directives can shorten or disable reuse, including within a single load. API/network errors are not cached; temporary failures are retried within the limits below, then uncached lookups stop while cached results remain usable. Sorting and exploration remain available. --no-genres skips these lookups entirely, and --refresh bypasses their cache on the initial load. t resumes using completed cached lookups. Genre enrichment does not change the default sorting method.

Request pacing and recovery

Reads are sequential within each service, with a minimum interval of 0.5 seconds for Spotify and ReccoBeats, and 0.25 seconds for Last.fm, shared across endpoints and background jobs in one CLI process. These are client pacing choices, not guaranteed provider quotas. Last.fm can run in the background while you use the menu or load Spotify/ReccoBeats data for another playlist. Opening a playlist still loads its complete Spotify contents and requested audio/popularity data before offering whole-playlist sorting.

HTTP 429, 500, 502, 503 and 504, connection failures and timeouts receive at most two retries (three attempts total). Last.fm's temporary API errors 11/16 and rate-limit error 29 are handled even with HTTP 200. Retry-After accepts seconds or an HTTP date; otherwise retry delays increase from 2 to 4 seconds. Waits are interruptible, with a 60-second cumulative waiting budget per read. A longer requested delay defers the read without contacting the service early. Exhausted attempts pause that service for at least 30 seconds, or longer when requested, across its endpoints; ReccoBeats popularity cannot immediately bypass an audio cooldown.

Successful pages/batches/tags are cached as they arrive. Reopen the playlist to resume after the cooldown, or use t for genres. Permanent errors such as invalid credentials are not retried. Spotify writes, including reorder operations, are never automatically replayed after an ambiguous failure. Pacing/cooldowns are shared within the running process; cached data and playlist checkpoints also survive a restart.

Audio features and popularity

By default, explore and preview (including --saved) enrich tracks using ReccoBeats. Only track IDs are sent, through an HTTP session separate from Spotify. Spotify credentials and tokens are never sent to ReccoBeats. --audio-source spotify selects Spotify instead; --no-audio disables enrichment, including ReccoBeats popularity retrieval.

The CLI fills tempo, energy, danceability, valence, speechiness, mode, key, acousticness, instrumentalness, liveness and loudness when returned. audio_source records provenance. BPM/energy coverage is shown at loading. Results are matched by Spotify links, not response position or ReccoBeats UUID; duplicates are preserved.

Missing popularity is fetched from ReccoBeats track metadata, preserving values already supplied by Spotify. popularity_source records provenance. Missing tracks, fields not returned (including time_signature) and invalid values remain empty in the loaded data. Providers' values are not assumed identical or equally recent. Network failures and rate limits preserve results already received and allow exploration to continue. Sorting can fill missing popularity/energy using the p50 rules above.

Spotify's Development Mode changes affect playlist routes and remove some fields, including popularity. The CLI requires Spotipy 2.26+ and accepts both item and track responses. Accessible playlists may be restricted to those you own or collaborate on; Premium and authorized users may also be required. See the Spotify migration guide.

Audio Features access is restricted for some apps (Spotify announcement). HTTP 403/404 produces a short explanation and leaves missing audio data empty. A 403 is consistent with access restrictions but does not establish the exact cause. Only handled errors suppress Spotipy's long logged URL; other errors remain visible. The CLI cannot restore access withdrawn by Spotify. Entirely missing audio columns cannot be filled with p50; choose other sorting criteria or supply your own data.

Local verification

poetry run python -m unittest discover -s tests -v
poetry run python -m my_spotipy demo --no-interactive

Tests use simulated APIs and do not modify a Spotify account.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages

0