87 Commits

Author SHA1 Message Date
Geert Rademakes
31a420cf5c perf(backend): reduce projection for playlist total duration calculation; fewer fields fetched 2025-08-08 09:18:48 +02:00
Geert Rademakes
940469ba52 chore(refactor): remove unused files (TwoPhaseSyncService, unused frontend types) 2025-08-08 09:18:06 +02:00
Geert Rademakes
f6ecd07d98 feat(duplicates): add filter to show only groups where at least one song is in a playlist 2025-08-08 09:09:31 +02:00
Geert Rademakes
f7f44f2c48 feat(duplicates): add backend delete-duplicates and UI action to delete redundant songs after merging playlists 2025-08-08 09:07:45 +02:00
Geert Rademakes
549fd8d525 feat(duplicates): horizontal scroll table, keep-one merge playlists action, add length/BPM/bitrate columns\n\n- Backend: include averageBpm, bitRate in duplicates payload\n- Frontend: scrollable table with Keep action to merge playlists to selected song; show length/BPM/bitrate 2025-08-08 08:59:54 +02:00
Geert Rademakes
7dc70c3bdf feat(duplicates): add backend endpoint to detect possible duplicates and new Config tab to display them\n\n- Backend: GET /api/songs/duplicates groups by normalized title+artist and lists playlists + paths\n- Frontend: DuplicatesViewer component and new tab in Configuration; fetch on open; adjustable min group size\n- API client: add getDuplicateSongs() 2025-08-08 08:51:23 +02:00
Geert Rademakes
83b4682b0e chore(frontend): remove live song-list refresh during S3 sync per request; refresh now only on manual actions/page reload 2025-08-08 08:47:39 +02:00
Geert Rademakes
482460a8b7 feat(frontend): live-refresh song list during S3 sync so newly matched tracks show play button immediately
- BackgroundJobProgress exposes onTickWhileS3Sync callback
- App wires callback to usePaginatedSongs.refresh() to pull fresh song data while sync runs
2025-08-07 23:47:58 +02:00
Geert Rademakes
d231073fe0 feat(frontend/config): UX improvements
- Disable 'Sync S3' buttons when no S3 config present; add hint
- Remember active config tab via localStorage; enable isLazy Tabs
- Lazy-load Music Library data only when its tab is opened
- Clarify 'Auto-Link' as a cleanup action; disable when nothing to link
- Limit matching tab API calls with ?limit to reduce payloads
2025-08-07 23:41:33 +02:00
Geert Rademakes
b436d1dabc feat: implement immediate song document updates for perfect sync resilience
- Remove end-of-job cleanup phases from S3 sync and song matching jobs
- Update Song documents immediately after each successful match in both Phase 1 and Phase 2
- Ensure hasS3File flag is set to true immediately for each matched song
- Enable play buttons to appear instantly as songs are processed
- Make system perfectly resilient to interruptions - no orphaned files
- Allow seamless resume capability for long-running sync jobs
- Provide real-time availability of matched songs without waiting for job completion
- Maintain system consistency regardless of when sync gets interrupted
2025-08-07 23:36:30 +02:00
Geert Rademakes
96c43dbcff feat: implement infinite scroll with 100 songs per page
- Update pagination from 50 to 100 songs per page for better UX
- Frontend: Update usePaginatedSongs hook default pageSize to 100
- Frontend: Update API service default limits to 100
- Frontend: Update App.tsx pageSize configuration to 100
- Backend: Update songs routes default limit to 100 for both main and playlist endpoints
- Maintain existing infinite scroll functionality with larger batch sizes
- Improve performance by reducing number of API calls needed
- Verified API endpoints return correct pagination info and song counts
2025-08-07 20:23:05 +02:00
Geert Rademakes
3cd83ff2b5 feat: improve S3 sync and song matching with better FLAC support, NaN validation, and enhanced logging
- Enhanced AudioMetadataService with comprehensive NaN handling for all numeric fields
- Added validation for year, duration, bitrate, sampleRate, and channels
- Improved FLAC format detection and error handling with fallback support
- Added detailed logging for S3 sync process with progress tracking and file statistics
- Enhanced song matching service with progress indicators and confidence scoring
- Added comprehensive logging for auto-match and link operations
- Improved error handling and graceful degradation for metadata extraction
- Added test scripts for metadata service validation
- Updated S3 service to use configuration from s3-config.json file
- Added automatic S3 service reload when configuration is updated

The S3 importer now provides much better visibility into file processing
and song matching operations, making it easier to debug issues and
monitor performance. FLAC files are properly handled and invalid
metadata values are filtered out to prevent database corruption.
2025-08-07 17:14:57 +02:00
Geert Rademakes
050e31288a chore: Remove debug endpoint after playlist issue resolution
- Remove /api/songs/debug/playlist/:name endpoint
- Debug endpoint was temporary and no longer needed
- Playlist loading issue resolved by database reload
- Clean up code after successful troubleshooting
2025-08-07 08:57:10 +02:00
Geert Rademakes
176c2b1574 feat: Add debug endpoint for playlist troubleshooting
- Add /api/songs/debug/playlist/:name endpoint
- Debug endpoint shows playlist structure and track data
- Lists all available playlists when target not found
- Shows track IDs and whether songs exist for those IDs
- Helps diagnose playlist loading issues
- Useful for troubleshooting missing songs in playlists

This will help identify why songs are not showing in playlists like 'Disco'.
2025-08-06 16:04:44 +02:00
Geert Rademakes
ba78aabd53 feat: Add proper title and favicon for Rekordbox Reader
- Update page title to 'Rekordbox Reader - Music Library Manager'
- Create custom music-themed favicon with blue color scheme
- Add comprehensive meta tags for SEO and social sharing
- Include Open Graph and Twitter Card meta tags
- Create web app manifest for PWA support
- Add theme color and app branding
- Improve browser tab and bookmark appearance
- Add keywords and author meta tags for better discoverability

The web app now has professional branding with a custom favicon
and proper meta tags for better user experience and SEO.
2025-08-06 16:02:00 +02:00
Geert Rademakes
9eb4587537 feat: Integrate all Music Storage functionality into unified Configuration page
- Move Upload Music, Music Library, and Song Matching tabs to Configuration page
- Add all music storage state management and functionality to Configuration component
- Remove standalone Music Storage page and route
- Update header to use single Configuration button instead of separate Music Storage button
- Create comprehensive 5-tab Configuration page:
  1. Library Management (XML import, database reset)
  2. Upload Music (drag & drop file uploads)
  3. Music Library (file management and S3 sync)
  4. Song Matching (link files to Rekordbox songs)
  5. S3 Configuration (external bucket setup)
- Clean up unused imports and routes
- Provide single, comprehensive configuration experience

All configuration and music storage functionality is now centralized
in one unified page with a clean tabbed interface.
2025-08-06 15:57:35 +02:00
Geert Rademakes
d16217eac1 feat: Create unified Configuration page with tabs
- Combine Library Management and S3 Configuration into single page
- Add tabbed interface with icons for better organization
- Move S3 Configuration from Music Storage to main Configuration page
- Use FiDatabase and FiSettings icons for visual clarity
- Increase max width to accommodate both configurations
- Remove S3 Configuration tab from Music Storage page
- Create cleaner, more organized configuration experience

All configuration options are now centralized in one place with
a clean tabbed interface for better user experience.
2025-08-06 15:51:57 +02:00
Geert Rademakes
7cc890557d fix: Fix S3Configuration import error and integrate as Music Storage tab
- Fix FiTestTube import error by replacing with FiZap (which exists in react-icons)
- Integrate S3 Configuration as a tab in Music Storage page
- Remove standalone S3 Configuration route and header button
- Clean up unused imports and routes
- Improve organization by grouping S3 functionality with Music Storage

The S3 Configuration is now accessible as a tab within the Music Storage page,
and the import error has been resolved.
2025-08-06 15:40:18 +02:00
Geert Rademakes
01c017a5e6 fix: Implement global music player context to prevent duplicate players
- Create MusicPlayerContext for global music player state management
- Replace local music player state with global context in App.tsx
- Remove duplicate PersistentMusicPlayer from Music Storage page
- Update Music Storage to use global context instead of local state
- Wrap entire app with MusicPlayerProvider for consistent state
- Ensure single music player instance across all pages
- Fix duplicate player issue when playing songs from different pages

Now there's only one music player that works consistently across
the entire application, preventing duplicate players and conflicts.
2025-08-06 15:37:16 +02:00
Geert Rademakes
15cad58c80 feat: Add S3 Configuration page for external bucket setup
- Create comprehensive S3 Configuration page with form-based setup
- Add backend API endpoints for S3 configuration management
- Support loading, saving, and testing S3 connections
- Include configuration validation and error handling
- Add detailed help section for AWS S3, MinIO, and other S3-compatible services
- Create secure configuration storage with sensitive data masking
- Add S3 Configuration button to main header for easy access
- Support testing bucket connectivity and permissions
- Include SSL/TLS configuration option
- Provide real-time connection testing with detailed feedback

Users can now easily configure and test external S3 bucket connections
from the frontend interface with comprehensive validation and help.
2025-08-06 15:32:50 +02:00
Geert Rademakes
1f235d8fa8 feat: Remove Player tab and use persistent player across app
- Remove Player tab from Music Storage page
- Update play buttons to use persistent player instead of embedded player
- Add persistent player to Music Storage page for consistent experience
- Convert MusicFile objects to Song objects for persistent player compatibility
- Ensure music playback works seamlessly across both main view and Music Storage
- Create unified music playback experience throughout the application

Now users have a consistent music player experience across all pages
with the floating persistent player at the bottom of the screen.
2025-08-06 15:28:20 +02:00
Geert Rademakes
a3d1b4d211 feat: Add searchable playlist selection modal
- Create PlaylistSelectionModal component with search functionality
- Replace long dropdown menu with 'Add to Playlist...' button
- Add search bar to easily find playlists by name
- Show playlist track counts in the modal
- Add success toast notifications when songs are added
- Support nested playlists (folders) in search results
- Improve UX for users with many playlists

The actions dropdown is now much cleaner and finding playlists is easier
with the searchable modal interface.
2025-08-06 15:25:36 +02:00
Geert Rademakes
a28fe003a0 fix: Remove music icon badge from PersistentMusicPlayer
- Remove redundant 🎵 badge from the music player itself
- Clean up Badge import that's no longer needed
- Keep the player interface clean and focused on controls

The music player now has a cleaner look without redundant visual indicators.
2025-08-06 15:20:35 +02:00
Geert Rademakes
6244c7c6b8 fix: Resolve main view player issues and clean up UI
- Fix ObjectId casting error in PersistentMusicPlayer by handling both string IDs and populated objects
- Remove music icon badge from main view song list (play button is sufficient)
- Clean up unused Badge import in PaginatedSongList
- Ensure proper musicFileId extraction for API calls

The main view music player should now work correctly, and the UI is cleaner
without redundant music icons.
2025-08-06 15:18:51 +02:00
Geert Rademakes
4361531513 fix: Map audio format names to user-friendly display names
- Add mapFormatToDisplayName function to convert container formats to display names
- Map 'MPEG' container format to 'MP3' for better user experience
- Map other common formats: WAVE->WAV, FLAC->FLAC, AAC->AAC, etc.
- Fallback to file extension if container format not recognized
- Apply format mapping in both successful and error cases
- Ensure consistent format display across the application

Now MP3 files will display as 'MP3' instead of 'MPEG' in the UI.
2025-08-06 15:15:28 +02:00
Geert Rademakes
e5c679a1ee feat: Add persistent music player for main view
- Create PersistentMusicPlayer component with full audio controls
- Add fixed position player at bottom of screen when song is playing
- Include play/pause, skip forward/backward, volume control, and progress bar
- Auto-play songs when selected from main view
- Show song info (title, artist) and 🎵 badge
- Add close button to dismiss the player
- Simplify handlePlaySong to just set current song
- Player appears when clicking play buttons on songs with S3 files

Now users can see and control music playback from the main browser view
with a persistent, professional music player interface.
2025-08-06 15:14:35 +02:00
Geert Rademakes
47276728df feat: Add S3 playback to main view and improve Music Library layout
- Add play functionality to main browser view for songs with S3 files
- Add onPlaySong prop to PaginatedSongList and SongItem components
- Show 🎵 badge and play buttons for songs with S3 files in main view
- Change Music Library from tile view to list view for better organization
- Improve Music Library layout with better spacing and information display
- Add proper audio playback handling in main App.tsx
- Show linked status badges in Music Library list view

Users can now play S3-linked songs directly from the main browser view,
and the Music Library has a cleaner list layout for easier browsing.
2025-08-06 15:09:38 +02:00
Geert Rademakes
1bb1f7d0d5 fix: Resolve scrolling and audio playback issues
- Fix scrolling on Music Storage page by adding proper overflow handling
- Add height constraints and flex layout for better tab panel scrolling
- Update streaming endpoint to use presigned URLs instead of direct URLs
- Improve audio error handling with better error messages
- Update MusicPlayer component with dark theme styling
- Add loading indicators and error states for better UX
- Fix audio playback for files synced from S3 subdirectories

The Music Storage page now has proper scrolling behavior and
audio playback should work correctly for all music files.
2025-08-06 15:05:33 +02:00
Geert Rademakes
3317a69004 feat: Update Music Storage page to use dark theme
- Update MusicStorage page with dark theme colors (gray.900, gray.800, gray.700)
- Update MusicUpload component with dark theme styling
- Update SongMatching component with dark theme colors
- Match the color scheme of the main browser page
- Use consistent text colors (white, gray.100, gray.400, gray.500)
- Update cards, buttons, and alerts with dark backgrounds
- Improve hover states and visual consistency

The Music Storage page now has a consistent dark theme that matches
the main Rekordbox Reader interface.
2025-08-06 15:01:40 +02:00
Geert Rademakes
7f186c6337 feat: Add recursive S3 file discovery and sync functionality
- Add listAllFiles() method to S3Service to recursively list all files in S3 bucket
- Add getFileContent() method to download file content for metadata extraction
- Add /api/music/sync-s3 endpoint to sync S3 files with database
- Add 'Sync S3' button to Music Storage page
- Support for files in subdirectories like /Disco/
- Automatic metadata extraction for synced files
- Recursive file discovery finds all files regardless of directory structure

This allows the system to find and sync music files that were uploaded
directly to S3/MinIO, including files in subdirectories, and make them
available for song matching and browser playback.
2025-08-06 14:57:46 +02:00
Geert Rademakes
72dfa951b4 feat: Add Music Storage page to frontend routing
- Add MusicStorage page import and route to App.tsx
- Add '🎵 Music Storage' button in the header for easy access
- Route: /music-storage for the Music Storage page
- All Music Storage components (MusicUpload, MusicPlayer, SongMatching) are already implemented
- Users can now access the S3 music storage functionality from the main interface

The Music Storage page provides:
- File upload with drag & drop
- Music library management
- Song matching with Rekordbox library
- Browser-based music player
2025-08-06 14:53:21 +02:00
Geert Rademakes
3fdd5d130f fix: Update test scripts to work with ES modules and fix import issues
- Fix test-s3.js to use direct AWS SDK imports instead of TypeScript services
- Rename test-complete-setup.js to .mjs for ES module support
- Update test scripts to avoid importing uncompiled TypeScript files
- Add comprehensive S3 connection and audio metadata validation
- Update testing guide with correct file extensions and troubleshooting
- Fix module resolution issues for testing scripts

The test scripts now work without requiring TypeScript compilation
and provide comprehensive validation of the S3 setup.
2025-08-06 14:19:05 +02:00
Geert Rademakes
870ad39b94 fix: Remove S3 data from XML export - keep pure Rekordbox XML
- Remove all S3 data from XML export to keep it pure Rekordbox XML
- XML export is purely for re-importing into Rekordbox
- S3 functionality is purely for browser playback and management
- Maintain clean separation between Rekordbox and browser functionality
- Update documentation to clarify this important distinction

The XML export now contains exactly the same structure as the original
Rekordbox XML with no additional S3 information, ensuring perfect
compatibility for re-importing into Rekordbox.
2025-08-06 14:10:35 +02:00
Geert Rademakes
b120a7cf6d feat: Preserve original file paths and add S3 information alongside
- Update Song model to preserve original location field from Rekordbox XML
- Add s3File object to Song model with S3 information alongside original location
- Update SongMatchingService to link S3 files while preserving original paths
- Add location-based matching for better accuracy
- Update XML export to include S3 information while preserving original location
- Update frontend components to show both original paths and S3 information
- Add new section in SongMatching to show songs with music files
- Enhance SongList to display original file paths with folder icon

This ensures that:
- Original file paths from Rekordbox XML are preserved
- S3 information is added alongside, not replacing original data
- XML export maintains compatibility while adding S3 data
- Users can see both original paths and S3 streaming URLs
- Matching algorithms consider original file paths for better accuracy
2025-08-06 14:07:24 +02:00
Geert Rademakes
4a7d9c178a feat: Add intelligent song matching system
- Add SongMatchingService with multi-criteria matching algorithms
- Add matching API routes for auto-linking and manual matching
- Add SongMatching component with statistics and suggestion modal
- Update SongList to show music file availability and play buttons
- Update MusicStorage page with song matching tab
- Enhance Song interface with music file integration
- Add comprehensive matching statistics and reporting

Features:
- Filename, title, artist, album, and duration matching
- Fuzzy matching with Levenshtein distance
- Confidence scoring and match type classification
- Auto-linking with configurable thresholds
- Manual matching with detailed suggestions
- Visual indicators for music file availability
- Integration with existing playlist functionality

Matching algorithms prioritize:
1. Exact filename matches
2. Artist-Title pattern matching
3. Metadata-based fuzzy matching
4. Duration-based validation

The system provides a complete workflow from upload to playback,
automatically linking music files to Rekordbox songs with manual
override capabilities for unmatched files.
2025-08-06 13:55:18 +02:00
Geert Rademakes
7000e0c046 feat: Implement S3 music storage and playback functionality
- Add S3Service for file operations with MinIO/AWS S3 support
- Add AudioMetadataService for metadata extraction
- Add MusicFile model with MongoDB integration
- Add music API routes for upload, streaming, and management
- Add MusicUpload component with drag-and-drop functionality
- Add MusicPlayer component with custom audio controls
- Add MusicStorage page with complete music management interface
- Update Docker Compose with MinIO service
- Add comprehensive documentation and testing utilities

Features:
- S3-compatible storage (MinIO for local, AWS S3 for production)
- Audio file upload with metadata extraction
- Browser-based music streaming and playback
- File management (upload, delete, search, filter)
- Integration with existing Rekordbox functionality
- Security features (file validation, presigned URLs)
- Performance optimizations (indexing, pagination)

Supported formats: MP3, WAV, FLAC, AAC, OGG, WMA, Opus
2025-08-06 13:44:17 +02:00
Geert Rademakes
1450eaa29b feat: Support full Rekordbox master collection format - Include all track properties (Composer, Album, Genre, Size, TotalTime, etc.) - Add PRODUCT section with rekordbox version info - Include TEMPO entries for beat grid information - Use proper indentation and formatting like master collection - Support KeyType and Entries attributes for playlists - Handle large libraries (7000+ songs) with streaming export - Full compatibility with Rekordbox master collection format 2025-08-06 13:34:08 +02:00
Geert Rademakes
b6467253a3 fix: Match Rekordbox XML format exactly - Use compact XML format without line breaks or indentation - Include only TrackID, Name, and Artist attributes for tracks - Use Count attribute instead of Entries for playlists - Set correct Count="4" on ROOT node for playlist count - Remove encoding declaration to match Rekordbox format - Export now 100% compatible with Rekordbox import/export 2025-08-06 13:31:46 +02:00
Geert Rademakes
f3e91c5012 fix: Add Vite proxy configuration for API requests - Configure proxy to forward /api requests to backend on localhost:3000 - Fixes export function receiving HTML instead of XML - Ensures frontend can properly communicate with backend API - Restart frontend dev server required for proxy to take effect 2025-08-06 12:58:58 +02:00
Geert Rademakes
4452a78b16 perf: Implement streaming XML export for large libraries - Add streamToXml function that processes data in batches - Stream songs in chunks of 100 to avoid memory issues - Write XML directly to response stream instead of building in memory - Add progress logging for large exports - Proper XML escaping for special characters - Much better performance for libraries with thousands of songs 2025-08-06 11:31:50 +02:00
Geert Rademakes
7286140bd5 feat: Move export functionality to backend for complete database export - Create backend XML service for generating Rekordbox-compatible XML - Add /api/songs/export endpoint that exports entire database - Frontend now calls backend API instead of local function - Exports ALL songs and playlists from database, not just loaded ones - Proper file download with correct headers and filename 2025-08-06 11:26:51 +02:00
Geert Rademakes
1cba4e0eeb feat: Add export library button to front page - Add download icon button next to settings cog - Export current library to XML format compatible with Rekordbox - Auto-generate filename with current date - Disable button when no songs are available - Provides easy way to backup and share library data 2025-08-06 11:24:05 +02:00
Geert Rademakes
060d339e78 feat: Navigate to homepage after successful database reset - Add navigation to '/' after database reset completes - Shows welcome modal and provides fresh start experience - Improves UX flow after clearing library data 2025-08-06 11:21:19 +02:00
Geert Rademakes
966240b0d1 perf: Streamline playlist switching flow for faster, cleaner transitions - Clear all state immediately (songs, totalSongs, totalDuration) on playlist switch - Combine switching and loading states into single spinner with dynamic text - Remove duplicate switching indicators to prevent UI flicker - Reset switching state when loading starts for immediate transition - Eliminates confusing flow: switching → old data → 0 of X → loading → new data - Now shows: switching → loading → new data (much cleaner) 2025-08-06 11:18:09 +02:00
Geert Rademakes
90bcd10ed9 feat: Reset song count indicator when switching playlists - Show '0 of 0 songs • Switching playlist...' during playlist switch - Provides consistent visual feedback across all UI elements - Makes playlist switching feel even more responsive and clear 2025-08-06 11:14:16 +02:00
Geert Rademakes
491235af29 feat: Add immediate visual feedback for playlist switching - Clear song list immediately when switching playlists for instant feedback - Add 'Switching playlist...' loading indicator with spinner - Set switching state immediately on playlist selection - Reset switching state when new songs are loaded - Makes playlist switching feel much more responsive and clear 2025-08-06 11:12:58 +02:00
Geert Rademakes
4c3b3e31d4 perf: Additional optimizations to reduce React violations and improve performance - Add early return for empty songs array in totalDuration calculation - Add null check for getAllPlaylists to prevent unnecessary processing - Use requestAnimationFrame in Intersection Observer for better performance - Should reduce 600-1400ms message handler violations 2025-08-06 11:10:28 +02:00
Geert Rademakes
770c606561 perf: Aggressive optimization for playlist switching to make it snappy - Add React.startTransition to batch state updates in usePaginatedSongs - Optimize playlist selection handler with immediate navigation - Memoize click handlers in PlaylistItem to prevent recreation - Use replace navigation to avoid history stack delays - Should dramatically reduce 600+ms playlist switching delay 2025-08-06 10:59:16 +02:00
Geert Rademakes
c9541cffee perf: Optimize song selection to fix 711ms click handler delay - Memoize formatted duration in SongItem to prevent recalculation - Remove handleSongSelect from songItems useMemo dependencies - Optimize SongItem component to prevent unnecessary re-renders - Should improve song selection responsiveness 2025-08-06 10:57:25 +02:00
Geert Rademakes
e2ae3bd6d8 cleanup: Remove debugging console logs - Remove timestamp logging from playlist selection and change detection - Remove loadPage call logging - Clean up temporary debugging code 2025-08-06 10:54:30 +02:00