rekordbox-viewer/S3_STORAGE_README.md

# S3 Music Storage & Playback Feature

This document describes the implementation of S3-compatible storage for music files with browser playback capabilities in the Rekordbox Reader application.

## 🎵 Features

- **S3-Compatible Storage**: Store music files in MinIO (local) or any S3-compatible service
- **Audio Metadata Extraction**: Automatically extract artist, album, title, duration, etc.
- **Browser Playback**: Stream music files directly in the browser
- **File Management**: Upload, delete, and organize music files
- **Rekordbox Integration**: Link uploaded files to existing Rekordbox songs
- **Search & Filter**: Find music by artist, album, genre, or text search

## 🏗️ Architecture

### Backend Components

1. **S3Service** (`src/services/s3Service.ts`)
   - Handles file upload/download/delete operations
   - Generates presigned URLs for secure access
   - Manages S3 bucket operations

2. **AudioMetadataService** (`src/services/audioMetadataService.ts`)
   - Extracts metadata from audio files
   - Validates audio file formats
   - Provides utility functions for formatting

3. **MusicFile Model** (`src/models/MusicFile.ts`)
   - MongoDB schema for music file metadata
   - Links to existing Song model
   - Includes search indexes

4. **Music Routes** (`src/routes/music.ts`)
   - REST API endpoints for music operations
   - File upload handling with multer
   - Streaming and metadata endpoints

### Frontend Components (To be implemented)

1. **Music Upload Component**
   - Drag & drop file upload
   - Progress indicators
   - Batch upload support

2. **Music Player Component**
   - HTML5 audio player
   - Custom controls
   - Playlist integration

3. **Music Library Component**
   - Grid/list view of music files
   - Search and filter
   - Metadata display

## 🚀 Setup Instructions

### 1. Local Development Setup

#### Start MinIO (S3-compatible storage)
```bash
# Using Docker Compose
docker-compose -f docker-compose.dev.yml up -d

# Or manually
docker run -d \
  --name minio \
  -p 9000:9000 \
  -p 9001:9001 \
  -e MINIO_ROOT_USER=minioadmin \
  -e MINIO_ROOT_PASSWORD=minioadmin \
  -v minio_data:/data \
  minio/minio server /data --console-address ":9001"
```

#### Access MinIO Console
- URL: http://localhost:9001
- Username: `minioadmin`
- Password: `minioadmin`

#### Create Music Bucket
```bash
# Using MinIO client
mc alias set myminio http://localhost:9000 minioadmin minioadmin
mc mb myminio/music-files
mc policy set public myminio/music-files
```

### 2. Environment Variables

Create `.env` file in the backend directory:

```env
# MongoDB
MONGODB_URI=mongodb://localhost:27017/rekordbox

# S3/MinIO Configuration
S3_ENDPOINT=http://localhost:9000
S3_ACCESS_KEY_ID=minioadmin
S3_SECRET_ACCESS_KEY=minioadmin
S3_BUCKET_NAME=music-files
S3_REGION=us-east-1

# Server
PORT=3000
NODE_ENV=development
```

### 3. Install Dependencies

```bash
# Install backend dependencies
cd packages/backend
npm install

# Install frontend dependencies (when implemented)
cd ../frontend
npm install
```

## 📡 API Endpoints

### Music File Management

#### Upload Single File
```http
POST /api/music/upload
Content-Type: multipart/form-data

file: [audio file]
```

#### Upload Multiple Files
```http
POST /api/music/batch-upload
Content-Type: multipart/form-data

files: [audio files]
```

#### List Music Files
```http
GET /api/music?page=1&limit=20&search=artist&genre=electronic
```

#### Get Streaming URL
```http
GET /api/music/:id/stream
```

#### Get Presigned URL
```http
GET /api/music/:id/presigned?expiresIn=3600
```

#### Get File Metadata
```http
GET /api/music/:id/metadata
```

#### Delete File
```http
DELETE /api/music/:id
```

#### Link to Song
```http
POST /api/music/:id/link-song/:songId
```

### Response Examples

#### Upload Response
```json
{
  "message": "File uploaded successfully",
  "musicFile": {
    "_id": "64f8a1b2c3d4e5f6a7b8c9d0",
    "originalName": "track.mp3",
    "s3Key": "music/uuid.mp3",
    "s3Url": "music-files/music/uuid.mp3",
    "contentType": "audio/mpeg",
    "size": 5242880,
    "title": "Track Title",
    "artist": "Artist Name",
    "album": "Album Name",
    "duration": 180.5,
    "format": "mp3",
    "uploadedAt": "2024-01-15T10:30:00.000Z"
  }
}
```

#### Streaming Response
```json
{
  "streamingUrl": "http://localhost:9000/music-files/music/uuid.mp3",
  "musicFile": {
    // ... music file metadata
  }
}
```

## 🎵 Supported Audio Formats

- MP3 (.mp3)
- WAV (.wav)
- FLAC (.flac)
- AAC (.aac, .m4a)
- OGG (.ogg)
- WMA (.wma)
- Opus (.opus)

## 🔧 Configuration Options

### S3 Service Configuration

```typescript
const s3Config = {
  endpoint: 'http://localhost:9000',        // MinIO endpoint
  accessKeyId: 'minioadmin',                // Access key
  secretAccessKey: 'minioadmin',            // Secret key
  bucketName: 'music-files',                // Bucket name
  region: 'us-east-1',                      // Region
};
```

### File Upload Limits

- **Single file**: 100MB
- **Batch upload**: 10 files per request
- **Supported formats**: Audio files only

### Streaming Configuration

- **Direct URL**: For public buckets (MinIO)
- **Presigned URL**: For private buckets (AWS S3)
- **Expiration**: Configurable (default: 1 hour)

## 🔒 Security Considerations

### File Access
- Use presigned URLs for secure access
- Set appropriate expiration times
- Implement user authentication (future)

### File Validation
- Validate file types on upload
- Check file size limits
- Sanitize file names

### Storage Security
- Use environment variables for credentials
- Implement bucket policies
- Regular backup procedures

## 🚀 Production Deployment

### AWS S3 Setup
1. Create S3 bucket
2. Configure CORS policy
3. Set up IAM user with appropriate permissions
4. Update environment variables

### MinIO Production Setup
1. Deploy MinIO cluster
2. Configure SSL/TLS
3. Set up monitoring
4. Implement backup strategy

### CDN Integration
- Use CloudFront with S3
- Configure caching policies
- Optimize for audio streaming

## 🧪 Testing

### Manual Testing
1. Start MinIO and MongoDB
2. Upload test audio files
3. Verify metadata extraction
4. Test streaming functionality
5. Check file deletion

### API Testing
```bash
# Test health endpoint
curl http://localhost:3000/api/health

# Test file upload
curl -X POST -F "file=@test.mp3" http://localhost:3000/api/music/upload

# Test streaming
curl http://localhost:3000/api/music/:id/stream
```

## 🔄 Integration with Existing Features

### Rekordbox XML Import
- Match uploaded files with XML entries
- Link files to existing songs
- Maintain playlist relationships

### Playlist Management
- Include music files in playlists
- Stream playlist sequences
- Export playlists with file references

## 📈 Performance Optimization

### Streaming Optimization
- Implement range requests
- Use appropriate cache headers
- Optimize for mobile playback

### Database Optimization
- Index frequently queried fields
- Implement pagination
- Use text search indexes

### Storage Optimization
- Implement file compression
- Use appropriate storage classes
- Monitor storage costs

## 🐛 Troubleshooting

### Common Issues

1. **MinIO Connection Error**
   - Check MinIO is running: `docker ps`
   - Verify endpoint URL
   - Check credentials

2. **File Upload Fails**
   - Check file size limits
   - Verify file format
   - Check bucket permissions

3. **Streaming Issues**
   - Verify bucket is public (MinIO)
   - Check CORS configuration
   - Test with different browsers

4. **Metadata Extraction Fails**
   - Check file format support
   - Verify file integrity
   - Check music-metadata library

### Debug Commands

```bash
# Check MinIO status
docker logs minio

# Test S3 connection
mc ls myminio/music-files

# Check MongoDB connection
mongosh rekordbox --eval "db.musicfiles.find().limit(1)"
```

## 🔮 Future Enhancements

### Planned Features
- [ ] Audio visualization (waveform)
- [ ] Playlist streaming
- [ ] Audio format conversion
- [ ] User authentication
- [ ] File sharing
- [ ] Mobile app support

### Technical Improvements
- [ ] WebSocket streaming
- [ ] Progressive download
- [ ] Audio caching
- [ ] CDN integration
- [ ] Analytics tracking

## 📚 Resources

- [MinIO Documentation](https://docs.min.io/)
- [AWS S3 Documentation](https://docs.aws.amazon.com/s3/)
- [music-metadata Library](https://github.com/Borewit/music-metadata)
- [HTML5 Audio API](https://developer.mozilla.org/en-US/docs/Web/API/HTMLAudioElement)