63 Commits

Author SHA1 Message Date
Geert Rademakes
e58d42bea2 Fix WebDAV music seeking functionality
- Add proxy endpoint for WebDAV streaming with authentication
- Implement range request support for audio seeking
- Update streaming endpoint to use proxy for WebDAV, presigned URLs for S3
- Add proper HTTP headers for audio streaming (Accept-Ranges, Content-Range)
- Fix music player seeking issue where clicking progress bar would reset playback

The WebDAV integration now supports proper audio seeking by using a backend
proxy that handles WebDAV authentication and range requests, allowing users
to click anywhere on the progress bar to seek to that position in the track.
2025-09-18 08:58:49 +02:00
Geert Rademakes
9de7564c18 Fix WebDAV file listing issue and add AIFF support
- Fix WebDAV service to find all 4,101 MP3 files instead of 1,023
- Add support for AIFF files (.aif, .aiff) in audio detection
- Update audioMetadataService to recognize AIFF formats
- Simplify BackgroundJobProgress component polling logic
- Add maxDepth parameter to WebDAV directory listing
- Add comprehensive test scripts for debugging WebDAV integration

The WebDAV integration now correctly processes all 4,257 audio files
from the music collection, including 4,101 MP3 files and 156 other
audio formats (FLAC, WAV, AIFF, M4A, OGG).
2025-09-17 22:52:15 +02:00
Geert Rademakes
d747830384 Working webdav backend! 2025-09-17 13:22:19 +02:00
Geert Rademakes
7065247277 More webdav compatiblity 2025-09-17 11:30:03 +02:00
Geert Rademakes
218046ec4f webdav setup 2025-09-17 10:55:35 +02:00
Geert Rademakes
96fdf64060 fix(upload): add missing 'id' field to 'To Be Scanned' playlist creation for proper frontend rendering 2025-08-14 14:12:32 +02:00
Geert Rademakes
675c1f8d8f fix(upload): resolve folder selection issue and improve 'To Be Scanned' playlist creation - add debug logging, visual selection indicator, simplify playlist creation logic, support batch uploads with markForScan 2025-08-14 11:55:54 +02:00
Geert Rademakes
7557eddeb4 feat(upload): improve folder browser UX and performance - add loading state, fix chevron visibility, implement backend caching with 5min TTL, add cache invalidation on file operations 2025-08-14 09:17:54 +02:00
Geert Rademakes
a49e628d93 fix: resolve TypeScript syntax error in music routes - use spread operator instead of invalid 'as string[]' syntax 2025-08-14 09:08:50 +02:00
Geert Rademakes
e4ee7230d1 feat(upload): include root folder option in S3 folder dropdown and support empty targetFolder (root) on backend 2025-08-13 17:04:20 +02:00
Geert Rademakes
762ae0730a feat(upload): add S3 folder dropdown (backend /api/music/folders) and use selected folder for uploads; keep text input as fallback 2025-08-13 17:02:58 +02:00
Geert Rademakes
aae57ec55f feat(upload): allow selecting target S3 folder and auto-add uploaded files to 'To Be Scanned' playlist via stub songs for XML export 2025-08-13 17:00:59 +02:00
Geert Rademakes
d87d83eaf6 fix(config): improve auto-link UX; fix song update in background job; upsert MusicFile on complex matching 2025-08-13 16:54:38 +02:00
Geert Rademakes
fe3a7abf32 fix(backend): avoid duplicate key errors during complex matching by upserting MusicFile by s3Key 2025-08-13 16:48:34 +02:00
Geert Rademakes
58eaa50bd2 feat(config): refactor Sync & Matching panel; remove heavy Music Library tab; add Sync, Force Sync, and Clear Links + Force Sync controls; backend supports clearLinks/force 2025-08-13 16:41:59 +02:00
Geert Rademakes
73b4bc6eb1 fix(songs): repair generic song list route; make XML import upsert + delete stale while preserving s3File; unlink musicFiles for removed songs; auto-trigger matching 2025-08-08 16:24:25 +02:00
Geert Rademakes
5f17380816 chore: remove verbose reorder/debug logs and tidy XML export logs; streamline FE reorder console noise 2025-08-08 15:05:10 +02:00
Geert Rademakes
5659dde540 feat(export): honor custom playlist order overlay when exporting Rekordbox XML (Entries + TRACK sequence) 2025-08-08 14:56:28 +02:00
Geert Rademakes
9249a5a4a7 feat(reorder): support dragging multiple selected songs to a specific position and to end (block move) 2025-08-08 14:55:06 +02:00
Geert Rademakes
e2d6d55433 chore(reorder-debug): add server/client logs for move and playlist order slice to diagnose ordering issue 2025-08-08 14:37:45 +02:00
Geert Rademakes
484d191201 fix(playlist-order): honor custom order in playlist songs endpoint by slicing trackIds and mapping results 2025-08-08 14:33:42 +02:00
Geert Rademakes
8136bbb959 fix(reorder): backend move endpoint and frontend use precise move to persist order reliably; refresh after move 2025-08-08 14:30:58 +02:00
Geert Rademakes
61d4ca16de feat(playlists): track custom order in separate 'order' array; reading API honors 'order' while preserving 'tracks' 2025-08-08 14:25:50 +02:00
Geert Rademakes
5a396d774e fix(playlists): markModified on children/tracks to persist order; API response confirms update 2025-08-08 14:20:48 +02:00
Geert Rademakes
5a21243f5b chore(reorder): add client debug logs; refresh playlist after reorder; backend keeps missing ids at end when reordering 2025-08-08 14:15:55 +02:00
Geert Rademakes
32d545959d feat(playlists): support custom per-playlist track order; backend /playlists/reorder; playlist songs endpoint honors order; frontend intra-playlist row DnD with reorder persist 2025-08-08 14:03:03 +02:00
Geert Rademakes
dc11487a9f fix(dnd): bubble drop handler through nested folders; add debug log on handler invoke; harden /playlists/batch insertMany 2025-08-08 13:39:39 +02:00
Geert Rademakes
3bd110884c feat(admin): add /api/music/fix-content-types to correct MIME types; ensure sync sets proper contentType for FLAC and others 2025-08-08 12:04:05 +02:00
Geert Rademakes
5144df2e93 fix(flac): set correct contentType for FLAC and other types in S3 sync; include contentType hint in stream endpoint response 2025-08-08 12:02:03 +02:00
Geert Rademakes
dbf9dbcb8c fix(sync): reference correct processed count in job result; fix(search): keep focus in search while playing by blurring audio and focusing search; cleanup lints 2025-08-08 11:30:59 +02:00
Geert Rademakes
2e21c3b5f5 fix(matching): strip diacritics in matching and quick match so accented letters (e.g., é) match plain equivalents 2025-08-08 11:06:48 +02:00
Geert Rademakes
07044c7594 fix(matching): URL-decode filenames and Rekordbox locations during quick match and location matching (%20, %27 etc.) 2025-08-08 11:00:08 +02:00
Geert Rademakes
31a420cf5c perf(backend): reduce projection for playlist total duration calculation; fewer fields fetched 2025-08-08 09:18:48 +02:00
Geert Rademakes
940469ba52 chore(refactor): remove unused files (TwoPhaseSyncService, unused frontend types) 2025-08-08 09:18:06 +02:00
Geert Rademakes
f7f44f2c48 feat(duplicates): add backend delete-duplicates and UI action to delete redundant songs after merging playlists 2025-08-08 09:07:45 +02:00
Geert Rademakes
549fd8d525 feat(duplicates): horizontal scroll table, keep-one merge playlists action, add length/BPM/bitrate columns\n\n- Backend: include averageBpm, bitRate in duplicates payload\n- Frontend: scrollable table with Keep action to merge playlists to selected song; show length/BPM/bitrate 2025-08-08 08:59:54 +02:00
Geert Rademakes
7dc70c3bdf feat(duplicates): add backend endpoint to detect possible duplicates and new Config tab to display them\n\n- Backend: GET /api/songs/duplicates groups by normalized title+artist and lists playlists + paths\n- Frontend: DuplicatesViewer component and new tab in Configuration; fetch on open; adjustable min group size\n- API client: add getDuplicateSongs() 2025-08-08 08:51:23 +02:00
Geert Rademakes
b436d1dabc feat: implement immediate song document updates for perfect sync resilience
- Remove end-of-job cleanup phases from S3 sync and song matching jobs
- Update Song documents immediately after each successful match in both Phase 1 and Phase 2
- Ensure hasS3File flag is set to true immediately for each matched song
- Enable play buttons to appear instantly as songs are processed
- Make system perfectly resilient to interruptions - no orphaned files
- Allow seamless resume capability for long-running sync jobs
- Provide real-time availability of matched songs without waiting for job completion
- Maintain system consistency regardless of when sync gets interrupted
2025-08-07 23:36:30 +02:00
Geert Rademakes
96c43dbcff feat: implement infinite scroll with 100 songs per page
- Update pagination from 50 to 100 songs per page for better UX
- Frontend: Update usePaginatedSongs hook default pageSize to 100
- Frontend: Update API service default limits to 100
- Frontend: Update App.tsx pageSize configuration to 100
- Backend: Update songs routes default limit to 100 for both main and playlist endpoints
- Maintain existing infinite scroll functionality with larger batch sizes
- Improve performance by reducing number of API calls needed
- Verified API endpoints return correct pagination info and song counts
2025-08-07 20:23:05 +02:00
Geert Rademakes
3cd83ff2b5 feat: improve S3 sync and song matching with better FLAC support, NaN validation, and enhanced logging
- Enhanced AudioMetadataService with comprehensive NaN handling for all numeric fields
- Added validation for year, duration, bitrate, sampleRate, and channels
- Improved FLAC format detection and error handling with fallback support
- Added detailed logging for S3 sync process with progress tracking and file statistics
- Enhanced song matching service with progress indicators and confidence scoring
- Added comprehensive logging for auto-match and link operations
- Improved error handling and graceful degradation for metadata extraction
- Added test scripts for metadata service validation
- Updated S3 service to use configuration from s3-config.json file
- Added automatic S3 service reload when configuration is updated

The S3 importer now provides much better visibility into file processing
and song matching operations, making it easier to debug issues and
monitor performance. FLAC files are properly handled and invalid
metadata values are filtered out to prevent database corruption.
2025-08-07 17:14:57 +02:00
Geert Rademakes
050e31288a chore: Remove debug endpoint after playlist issue resolution
- Remove /api/songs/debug/playlist/:name endpoint
- Debug endpoint was temporary and no longer needed
- Playlist loading issue resolved by database reload
- Clean up code after successful troubleshooting
2025-08-07 08:57:10 +02:00
Geert Rademakes
176c2b1574 feat: Add debug endpoint for playlist troubleshooting
- Add /api/songs/debug/playlist/:name endpoint
- Debug endpoint shows playlist structure and track data
- Lists all available playlists when target not found
- Shows track IDs and whether songs exist for those IDs
- Helps diagnose playlist loading issues
- Useful for troubleshooting missing songs in playlists

This will help identify why songs are not showing in playlists like 'Disco'.
2025-08-06 16:04:44 +02:00
Geert Rademakes
9eb4587537 feat: Integrate all Music Storage functionality into unified Configuration page
- Move Upload Music, Music Library, and Song Matching tabs to Configuration page
- Add all music storage state management and functionality to Configuration component
- Remove standalone Music Storage page and route
- Update header to use single Configuration button instead of separate Music Storage button
- Create comprehensive 5-tab Configuration page:
  1. Library Management (XML import, database reset)
  2. Upload Music (drag & drop file uploads)
  3. Music Library (file management and S3 sync)
  4. Song Matching (link files to Rekordbox songs)
  5. S3 Configuration (external bucket setup)
- Clean up unused imports and routes
- Provide single, comprehensive configuration experience

All configuration and music storage functionality is now centralized
in one unified page with a clean tabbed interface.
2025-08-06 15:57:35 +02:00
Geert Rademakes
15cad58c80 feat: Add S3 Configuration page for external bucket setup
- Create comprehensive S3 Configuration page with form-based setup
- Add backend API endpoints for S3 configuration management
- Support loading, saving, and testing S3 connections
- Include configuration validation and error handling
- Add detailed help section for AWS S3, MinIO, and other S3-compatible services
- Create secure configuration storage with sensitive data masking
- Add S3 Configuration button to main header for easy access
- Support testing bucket connectivity and permissions
- Include SSL/TLS configuration option
- Provide real-time connection testing with detailed feedback

Users can now easily configure and test external S3 bucket connections
from the frontend interface with comprehensive validation and help.
2025-08-06 15:32:50 +02:00
Geert Rademakes
4361531513 fix: Map audio format names to user-friendly display names
- Add mapFormatToDisplayName function to convert container formats to display names
- Map 'MPEG' container format to 'MP3' for better user experience
- Map other common formats: WAVE->WAV, FLAC->FLAC, AAC->AAC, etc.
- Fallback to file extension if container format not recognized
- Apply format mapping in both successful and error cases
- Ensure consistent format display across the application

Now MP3 files will display as 'MP3' instead of 'MPEG' in the UI.
2025-08-06 15:15:28 +02:00
Geert Rademakes
1bb1f7d0d5 fix: Resolve scrolling and audio playback issues
- Fix scrolling on Music Storage page by adding proper overflow handling
- Add height constraints and flex layout for better tab panel scrolling
- Update streaming endpoint to use presigned URLs instead of direct URLs
- Improve audio error handling with better error messages
- Update MusicPlayer component with dark theme styling
- Add loading indicators and error states for better UX
- Fix audio playback for files synced from S3 subdirectories

The Music Storage page now has proper scrolling behavior and
audio playback should work correctly for all music files.
2025-08-06 15:05:33 +02:00
Geert Rademakes
7f186c6337 feat: Add recursive S3 file discovery and sync functionality
- Add listAllFiles() method to S3Service to recursively list all files in S3 bucket
- Add getFileContent() method to download file content for metadata extraction
- Add /api/music/sync-s3 endpoint to sync S3 files with database
- Add 'Sync S3' button to Music Storage page
- Support for files in subdirectories like /Disco/
- Automatic metadata extraction for synced files
- Recursive file discovery finds all files regardless of directory structure

This allows the system to find and sync music files that were uploaded
directly to S3/MinIO, including files in subdirectories, and make them
available for song matching and browser playback.
2025-08-06 14:57:46 +02:00
Geert Rademakes
3fdd5d130f fix: Update test scripts to work with ES modules and fix import issues
- Fix test-s3.js to use direct AWS SDK imports instead of TypeScript services
- Rename test-complete-setup.js to .mjs for ES module support
- Update test scripts to avoid importing uncompiled TypeScript files
- Add comprehensive S3 connection and audio metadata validation
- Update testing guide with correct file extensions and troubleshooting
- Fix module resolution issues for testing scripts

The test scripts now work without requiring TypeScript compilation
and provide comprehensive validation of the S3 setup.
2025-08-06 14:19:05 +02:00
Geert Rademakes
870ad39b94 fix: Remove S3 data from XML export - keep pure Rekordbox XML
- Remove all S3 data from XML export to keep it pure Rekordbox XML
- XML export is purely for re-importing into Rekordbox
- S3 functionality is purely for browser playback and management
- Maintain clean separation between Rekordbox and browser functionality
- Update documentation to clarify this important distinction

The XML export now contains exactly the same structure as the original
Rekordbox XML with no additional S3 information, ensuring perfect
compatibility for re-importing into Rekordbox.
2025-08-06 14:10:35 +02:00
Geert Rademakes
b120a7cf6d feat: Preserve original file paths and add S3 information alongside
- Update Song model to preserve original location field from Rekordbox XML
- Add s3File object to Song model with S3 information alongside original location
- Update SongMatchingService to link S3 files while preserving original paths
- Add location-based matching for better accuracy
- Update XML export to include S3 information while preserving original location
- Update frontend components to show both original paths and S3 information
- Add new section in SongMatching to show songs with music files
- Enhance SongList to display original file paths with folder icon

This ensures that:
- Original file paths from Rekordbox XML are preserved
- S3 information is added alongside, not replacing original data
- XML export maintains compatibility while adding S3 data
- Users can see both original paths and S3 streaming URLs
- Matching algorithms consider original file paths for better accuracy
2025-08-06 14:07:24 +02:00