AI DJ Pro — Complete Technical Guide

AI DJ is a local-first DJ application: your music stays on your machine, playback and mixing run in the desktop app, and a Python DJ brain on your PC chooses tracks, drives auto-mix timing, and generates host (MC) announcements. Optional cloud APIs (OpenAI, ElevenLabs) power AI search, set planning, and voice — but core playback does not require them.

Division of labour

The client and server talk over ws://127.0.0.1:8765. The UI must stay connected for crowd requests and server-driven auto mix to work.

Key source files

Production vs development

• Installed app: Electron spawns bundled aidj-server.exe, waits for http://127.0.0.1:8766/health, then opens the UI.
• Development: Run python aidj_server.py and npm start in electron_mock/ separately.

Default music folder on first install: Documents\AI DJ\Music (Windows) or ~/Documents/AI DJ/Music (macOS).

Setup wizard

On first successful library load, a 4-step setup wizard may appear (unless you already completed it):

1. Music folder — point at your MP3/M4A/FLAC library
2. API keys — optional OpenAI / ElevenLabs for AI and MC voice
3. Crowd requests — LAN URL and QR code for guests
4. Analyze a track — BPM/key for beat grid and auto cues

Completion is stored in browser localStorage as AIDJ_SETUP_COMPLETE.

Where data lives (Windows example)

%APPDATA%\AI_DJ_Demo_Pro\

The client also mirrors the library in IndexedDB (AIDJLibrary) so reopening the library panel can show tracks immediately while refreshing from the server in the background.

Scanning local files

On startup (and when you change the music folder), the server scans AIDJ_MUSIC_DIR for: .mp3, .m4a, .flac, .wav, .ogg, .aac

For each file it builds a library item with title, artist, duration, BPM and musical key (Camelot), genre, album art, beat grid origin, and mix cue.

Library panel (client)

• Search, filter by genre / BPM / Camelot / date added
• Sort and drag-to-reorder (playlist order)
• Double-click — set as starting track or play (mode-dependent)
• Right-click — queue next (auto mode)
• Genre editing, AI genre detect, dedications, remove from playlist
• In manual mode: Load Deck A / B buttons per row
• In auto mode (session active): live/next deck indicators on rows

Request full library sync: client sends get_library → server responds with { type: "library", tracks: [...] }.

Saved playlists & analysis tools

Save the current library view to IndexedDB (named playlists) and reload later. Wire-format playlists from YouTube preserve order; sequential queue mode plays them in list order instead of smart reordering.

Single track import

Paste a URL → Load YouTube (Add) or Replace. Server uses yt-dlp + FFmpeg to download (or stream), analyzes BPM/key, adds to state.lib, and pushes library_update to the client.

Playlist import (stream mode)

1. Resolve — yt_playlist_resolve returns track list without downloading everything
2. Load — yt_playlist_load adds entries as stream URLs: http://127.0.0.1:8766/stream/youtube?id=VIDEO_ID
3. Background preload — parallel BPM/key/genre for tracks that need metadata

Replace during a live session: the library swaps, but the currently playing track keeps going until it ends or you skip — the new list does not hijack the mix mid-song.

Playback URLs

• Local files → file:// or http://127.0.0.1:8766/... (served from music dir only — path traversal blocked)
• YouTube → stream proxy on 8766

First play of a YouTube stream may take 15–30 seconds while the pipeline starts.

Start

1. Load a library (local folder and/or YouTube playlist).
2. Optionally double-click a starting track in the library.
3. Press Start → client sends { type: "start" }.
4. Server sets started = true and the player loop picks the first track (crowd queue, DJ-selected starting track, or first in queue).
5. Server sends incoming → client loads that track on deck A or B and begins playback (auto mix).

During the session

• Pause — freezes client audio and server track clock.
• Next Track (auto only) — skip to next pick with a crossfade.
• Energy slider — crowd energy 0–100; biases BPM targets for smart selection.
• End — goodbye MC (optional) → music fade out → session reset.

Stop

1. Client: { type: "stop" }
2. Optional goodbye MC → client plays TTS wav
3. Server: fade_out (default ~8s) → client fades gains → fade_complete
4. Server: stop_ack — library queue reset, stats finalized

Decks A and B

Two logical decks alternate in auto mode. Server toggles state.deck between A and B each transition. Live deck = on air; other deck = preloaded with the next track before the crossfade.

Toggle via mix control mode (stored in localStorage, synced to server as mix_control_mode).

Use auto for hands-off hosting; manual when you want full DJ control with the crossfader.

Manual crossfader curve

• Center — both gainA and gainB at full level (~80% peak headroom).
• Left of center — Deck A stays full; Deck B fades out toward the left extreme.
• Right of center — Deck B stays full; Deck A fades out toward the right extreme.

This matches hardware DJ mixers with a wide overlap at center. Saved position is in client memory; switching back to auto resets the fader to the live deck.

Auto mix is a cooperation between server timing and client audio engineering.

Server side

After sending incoming, the server waits roughly track duration minus overlap. While paused, that clock does not advance. On timeout or skip, it calls pick_next_track(), flips deck, sends the next incoming.

Client side (playOnDeck)

1. Resolve crossfade length — manual Xfade fader seconds, or auto from outgoing waveform analysis.
2. Resolve mix-in point (Set Cue mode: manual seconds, or auto/drop/chorus from waveform).
3. Snap cue to Phrase or Bar grid (see Session alignment controls, section 11).
4. Beat-align incoming to outgoing (tempo + phase) at mix start.
5. Schedule crossfade; run phase lock during overlap.
6. Update UI: waveforms, deck titles, library deck indicators.

Shock mix (hard transitions)

When Shock mix is enabled (checkbox under the waveforms, next to Align on mix), the client may replace a normal crossfade with a shock transition:

1. Outgoing deck spinback (playback rate ramps down + spinback SFX).
2. Outgoing gain drops quickly; incoming drops in on the beat.

Runs on natural track end and when you press Next Track (not blocked by quick-fade). Requires genre metadata for genre-based detection. Client: evaluateShockTransition() / crossfadeToShockTransition() in renderer.js. Persisted: localStorage keys aidj_auto_shock_transition, aidj_shock_mix_trigger. Shock mix is auto mode only — disabled in manual mix.

Crossfade audio graph

audioA ──► gainA ──┐
                   ├──► gainMusicDuck ──► gainMaster ──► speakers
audioB ──► gainB ──┘
audioMC ──► gainMC ──► (parallel, for host voice)

MC ducking lowers gainMusicDuck during announcements so the crossfader gains on A/B are untouched.

When AIDJ_SMART_SELECTION=1 (default), the server does not pick tracks randomly. find_best_next_track() scores candidates.

Pick priority

1. Manually queued track (queue_next / right-click in library)
2. Crowd queue (FIFO, approved requests)
3. Sequential playlist — if enabled, next in wire order
4. Smart selection — scored best match
5. Fallback — first available track ≠ current

Set arc (energy curve)

Set arc dropdown → server energy_arc_mode. See Session alignment controls (section 11) for how Set arc differs from Phrase, Bar, and Phase lock.

Crowd energy slider (0–100) further nudges target BPM within each phase. Harmonic filter in the library (Δ column vs live deck key) is a UI filter for browsing; smart selection uses its own key logic server-side.

Server analyzes audio (librosa + FFmpeg) to produce peak waveform segments (multi-band colours on deck display), beat grid (BPM + beat_grid_origin), and mix cue (structural mix-in point). Results cached under waveform_cache/ by file hash.

Set Cue modes (client)

Full detail on the faders and mode dropdowns: see section 18 — Xfade, Next, and Set Cue faders.

Xfade modes

Session alignment toolbar

Phrase, Bar, Phase lock, and Set arc — documented in Session alignment controls (section 11). The beat bar dots (1–2–3–4) show live beat position on the playing deck.

These settings sit in the top control bar (next to Start, Pause, and Next Track). They apply in auto mix mode only — not when you are blending decks manually with the crossfader.

Phrase

Options: 16, 24, 32, 48, 64 beats (default: 32). Align mixes to longer musical phrases on the outgoing track's analyzed beat grid.

Phrase controls when the next crossfade is allowed to start (the app waits for the next phrase boundary when possible) and where the incoming track's mix cue snaps (aligned to the same phrase length on the incoming grid). At 128 BPM, 32 beats is roughly 15 seconds (about 8 bars in 4/4) — a common DJ phrase length.

If waiting for a full phrase would run past the waveform fade window, the app falls back to Bar, then to a single beat, so the mix still happens in time. Saved in browser localStorage (persists across sessions on this machine).

Bar

Options: 4 or 8 beats (default: 4). Finer beat-grid alignment when a full phrase is not used or as a fallback.

• 4 = one bar (four beats in 4/4)
• 8 = two bars

Bar affects the same two things as Phrase — crossfade timing and cue snap — but on a shorter grid. Use 8 for slightly looser bar-line mixes; 4 for tighter alignment to every bar. Saved in browser localStorage.

Phase lock

Options: Off, Low, Medium, High (default: Medium in code; UI may show High if you changed it). Keep beats locked during the crossfade overlap.

While both decks are audible, the app briefly nudges the incoming deck's tempo so its beat phase matches the outgoing deck. When the fade finishes, tempo returns to normal.

Phase lock does not change which song plays next or when the fade is scheduled — only how tightly the beats align while the two tracks overlap. Saved in browser localStorage.

Set arc

Options: Auto, Warm up, Peak, Cool down (default: Auto). Tell the server which energy curve to use when choosing the next track in smart / auto playback.

This does not control crossfade length, cue points, or beat alignment. It influences BPM and energy of upcoming picks (and related smart-selection logic). Saved on the server for the active session (sent over WebSocket when you change the dropdown).

How they work together

Outgoing track playing
        │
        ▼
Phrase / Bar ──► wait for grid boundary (when possible)
        │
        ▼
Incoming track cue snapped to same grid idea
        │
        ▼
Crossfade starts (timing also follows waveform / Xfade settings)
        │
        ▼
Phase lock ──► micro-tempo on incoming deck during overlap
        │
        ▼
Set arc ──► (parallel) server picks next track for following transition

Related settings (not in this toolbar)

• Xfade → Manual / Auto Xfade — whether fade length is fixed (fader) or driven by waveform analysis. See section 18.
• Next — fade length when pressing Next Track early. See section 18.
• Set Cue — how mix-in points on tracks are detected (works with Phrase/Bar for snap length). See section 18.
• Shock mix — spinback + hard drop on big BPM/genre jumps (under waveforms, auto mode only). See section 8 and /guide/user#shock-mix.
• Deck effects / hot cues — per-deck knobs and C1–C8 cues. See section 20.
• Manual mix mode — Phrase, Bar, and Phase lock are for auto mix; in manual mode you load decks and blend yourself.

Tips

• Start with Phrase 32, Bar 4, Phase lock Medium or High, Set arc Auto for typical club/party sets.
• Raise Phrase if mixes feel too busy; lower it if you want quicker turnover.
• Use Set arc → Warm up at the start of an event or Cool down for last hour — without touching mix timing controls.

The MC is an optional AI/host voice that speaks between or over tracks.

Generation pipeline

1. Prompt — event context, venue, track title, energy, dedication, Mem0 memories (optional).
2. Line source — OpenAI (AIDJ_LIVE_AI_MC) and/or script templates (default, hype, smooth, wedding, etc.).
3. TTS — ElevenLabs if API key configured; else local pyttsx3.
4. Delivery — server sends { type: "mc", text, wav_path, duck_enabled, duck_level } → client plays audioMC.

Ducking & settings

While MC speaks, music volume ramps down via gainMusicDuck (default ~30%). On end, music restores and client sends mc_done so the server can continue timing.

• Event prompt, venue name, DJ name
• Enable/disable MC, interval minutes, premix/postmix delay seconds
• Template set vs live AI lines; language, tone, voice
• Dedicate — one-shot shout-out tied to a specific library track

Rapid Next Track presses temporarily disable skip to prevent chaos. MC announcements can delay skip processing so crossfade timing stays musical.

Guests on the same Wi‑Fi open the crowd page (default http://<your-LAN-IP>:8767/).

Guest flow

1. Enter song search (artist + title) and optional dedication/name.
2. Optional PIN (AIDJ_CROWD_REQUEST_PIN).
3. Content filters block explicit, commercial, and movie-trailer style queries.

Server processing

1. Fuzzy match local library (AIDJ_CROWD_MIN_MATCH_SCORE, default 0.52).
2. Optional AI verification for borderline queries.
3. If no match → YouTube search + download/stream for crowd.
4. Queue in crowd_play_queue or pending approval if AIDJ_CROWD_REQUIRE_APPROVAL=1.

DJ tools: Crowd link and QR in the top bar, approve/reject pending requests, blocklist for repeat bad queries. Crowd tracks show REQUEST badge in library; play with priority after manual queue.

All AI features require OPENAI_API_KEY unless noted. Keys are stored via server config (not logged).

Library AI Search

Natural-language search: "1980s wedding opener", lyric queries, theme/era/mood. Returns matches from your library plus YouTube suggestions. WebSocket: library_ai_search → library_ai_search_result.

AI Set Builder

Plans a full 6–40 track set from a prompt with energy flow (warmup → build → peak → cooldown), BPM/key smoothing, and marks library vs YouTube import needs. WebSocket: ai_set_builder → ai_set_builder_result.

Genre detection & MC

• Genre: ID3 tags → heuristics → AI detect (OpenAI) → manual override
• MC lines: OpenAI + optional Mem0 for venue-specific memories
• ElevenLabs v3 emotional tags optional for richer TTS

URL: ws://127.0.0.1:8765 (configurable via AIDJ_WS_PORT). On connect: license status, API config, energy arc, mix control mode, MC context. Reconnect with exponential backoff if server restarts.

Important client → server messages

Important server → client messages

Many operations are license-gated (start, library load, YouTube, AI search, set builder, etc.). Set LICENSE_DISABLE=1 in dev config to bypass. Max WebSocket frame size: 32 MiB (large libraries with embedded art).

Config merges: environment variables → user aidj_config.json → bundled aidj_config.json.

Core

Selection / energy

YouTube / preload

Crowd

AI / voice

Client-only (localStorage)

License

Online validation (Gumroad / Paddle / FastSpring / generic REST) + offline crypto + trusted keys. UI modals for activation; periodic re-check (~12h). Dev bypass: LICENSE_DISABLE=1 in config.

Common issues

Logs

• Packaged: %APPDATA%\AI_DJ_Demo_Pro\aidj-server.log
• Dev: terminal running python aidj_server.py
• Client: in-app log panel + browser DevTools console

Three vertical faders sit in the session control bar (beside Master volume), each with an optional mode dropdown. They control when the outgoing track fades, how long the blend lasts, and where the incoming track starts — all on the client in auto mix mode.

In manual mix mode, these faders are disabled (dimmed). You load decks yourself and blend with the horizontal crossfader instead.

Where they live

Session bar (auto mix)
├── Master volume
├── Xfade fader + Manual / Auto Xfade dropdown
├── Next fader                    (no dropdown)
└── Set Cue fader + Manual / Auto Cue / Drop / Chorus dropdown

Each fader has an LED strip that lights segments proportionally to the current value (2–30s range for Xfade and Next; 1–30s for Set Cue in manual mode).

Overview — what each control does

Xfade fader

UI: vertical slider 2–30 seconds (default 8s) + dropdown Manual / Auto Xfade.

• Manual (default): fader sets fixed crossfade length. Fade starts at effective track end − fade length (trailing silence ignored). Orange Fade marker on live deck waveform.
• Auto Xfade: fader disabled. Analyzer finds where energy drops in the tail. Label shows fade start time and duration, e.g. 3:42 · 12s. Deck time counts down to fade instead of to mix.
• Applies to automatic transitions — not directly when you press Next Track (that uses the Next fader).
• Saved in localStorage (xfadeMode for dropdown). Fader position is session UI state (default 8s on load).

Next fader

UI: vertical slider 2–30 seconds (default 8s). Label reads Next 8s or Next Auto. Controls fade timing when you press Next Track before the song would naturally mix out.

Tip: Use a shorter Next (e.g. 4–6s) for quick DJ-style skips; use a longer Xfade (e.g. 12–16s) for smooth end-of-track blends.

Set Cue fader

• Manual: incoming tracks start at that offset (e.g. 1s skips silence; 32s jumps to a breakdown).
• Auto/Drop/Chorus: mix-in from server waveform analysis, aligned to outgoing fade using current Xfade length, snapped to Phrase or Bar grid.
• First track of a session always plays from 0:00 (no cue skip on the opener).
• Live deck: orange Fade line. Incoming deck: green Mix line.
• Auto modes need waveform data — run Analyze missing BPM if label stays …
• Saved in localStorage (cueMode for dropdown).

How the three work together

1. Outgoing fade window — Xfade (Manual or Auto) decides when and how long the live track blends out.
2. Incoming entry point — Set Cue decides where the next track starts.
3. Alignment — In auto cue modes, Set Cue is shifted so the incoming mix point matches the outgoing fade window.
4. Grid snap — Phrase and Bar (section 11) refine when the crossfade starts and where the cue lands.
5. Phase lock — keeps beats aligned during the overlap (separate control).

End-to-end: natural mix vs Next Track

Natural mix:
Live deck → fade trigger (Xfade) → client skip → server incoming
→ load other deck at Set Cue → beat-aligned crossfade → new live deck

Next Track (early skip):
Next Track pressed → server incoming immediately → quickFade=true
→ Next fader length (Manual) or Auto Xfade → crossfade starts now (no phrase wait)

The server's playback timer (AIDJ_OVERLAP_SEC, default 8s) is a rough overlap hint for when to send the next incoming after a transition; the client faders govern the actual audible blend.

Quick reference

AI DJ can split local tracks into four stems using Demucs htdemucs (vocals, drums, bass, other).

Architecture

• aidj-stems (Windows: aidj-stems.exe; macOS: aidj-stems binary) — bundled worker (PyTorch + Demucs + NumPy). Runs as a subprocess only; the main server never imports torch.
• Cache — per-user folder with four WAV files + meta.json per track hash: %AppData%\AI_DJ_Demo_Pro\stems\<hash>\ (Windows) or ~/Library/Application Support/AI_DJ_Demo_Pro/stems/<hash>/ (macOS).
• HTTP — stems stream at http://127.0.0.1:8766/stems/<cache_key>/<role>.wav.
• Queue — one separation job at a time (CPU/GPU heavy).

Both Windows and macOS installers bundle the stem worker when the PyInstaller build step succeeds (build-installer.bat / build-installer.sh step [2b/6]).

Using stems on decks

Each deck has V / D / B / O pads, a Full mix / Stems toggle, and a status chip.

1. Load a local file (not a YouTube stream).
2. Click Separate stems in the library row, or wait for background prepare on load.
3. When status shows Stems ready, click Full mix to switch to Stems.
4. Click pads to mute; Alt+click to solo a stem.

Waveform and BPM analysis still use the full mix file.

MC ducking with stems

When Stems mode is active on a playing deck, MC volume ducking lowers vocals only on that deck instead of ducking the entire mix.

First run & installer size

• First separation downloads htdemucs weights (~80 MB) into the torch hub cache. CPU works; CUDA or Apple Silicon GPU may be used when available.
• The stem worker adds roughly 1.5–2 GB to the installer (PyTorch + Demucs). Build the worker on the same OS and CPU architecture you ship.

Development (from source)

pip install -r requirements-stems.txt
python aidj_server.py

In dev, the server runs stem_worker.py via your Python interpreter instead of the bundled aidj-stems binary.

Stem troubleshooting

Per-deck performance tools live in the Electron client (renderer.js, EchoEffect.js). They work in auto and manual mix.

Effects signal path

eqMix (post-EQ) ──┬──► dry bus ──► effectsOutput ──► analysers ──► gainMusicDuck
                  │
                  └──► parallel wet effects ──► wet mix ──┘

Each deck has 12 effect sends in two UI rows plus a separate Filter panel (low/high/band-pass when enabled).

Echo (electron_mock/EchoEffect.js): dual delay lines with cross-feedback, high-pass on input, low-pass on feedback and wet path. Delay time maps from dotted-eighth to quarter note using deckABPM / deckBBPM when known.

FX bypass toggle

Round FX buttons (deckFxToggleA / deckFxToggleB) appear under the channel fader when the Effects panel is expanded. They toggle effectsAEnabled / effectsBEnabled — wet gains go to zero while knob values are preserved.

Hot cues

• 8 slots per deck (C1–C8); was 4 in earlier builds.
• Shift+click or Shift+1…8 stores the current playhead.
• Cues persist per track URL in localStorage.
• HOT_CUE_COUNT = 8 in renderer.js.

MIDI

Effect and filter knobs are exposed to the MIDI learn system (same pattern as EQ). Hot cue MIDI actions use deck-specific note/CC mappings.

Party (auto, hands-off)

1. Load YouTube playlist or local folder.
2. Run Analyze missing BPM if needed.
3. Set Set arc → Auto, MC on with event prompt.
4. Enable crowd QR.
5. Start — let smart selection and crossfades run.

Club segment (manual)

1. Switch to manual mix mode.
2. Load tracks on Deck A / B from library.
3. Blend with crossfader (center = both decks full); use hot cues and deck effects.
4. MC intro fires when you play a deck.

Planned wedding set

1. AI Set Builder — "wedding cocktail 2h, 90s–today, no explicit".
2. Review results, import missing YouTube tracks.
3. Load as queue with sequential mode.
4. Warm up set arc at start; switch to Peak for dancing.