rekordbox-viewer/packages/desktop-sync/README.md

Rekordbox Sync - Desktop Companion

A desktop application for bidirectional synchronization between a Garage-hosted S3 instance and your local computer, specifically designed for Rekordbox music libraries.

🚀 Features

• Bidirectional S3 Sync: Seamlessly sync files between your local machine and S3 storage
• Incremental Sync: Only sync files that have changed since the last sync
• Automatic Cleanup: Removes temporary files before syncing
• Real-time Monitoring: Continuous sync with configurable intervals
• Error Handling: Robust error handling with automatic retries
• Progress Tracking: Real-time progress updates and file counting
• Cross-platform: Built with Electron for macOS, Windows, and Linux

🔧 Prerequisites

AWS CLI v2

This tool requires AWS CLI v2 to be installed on your system. The AWS CLI provides the aws s3 sync command which offers superior performance and reliability compared to other S3 sync tools.

Installation Options:

Option 1: Automatic Installation (macOS)

npm run install-aws-cli

Option 2: Manual Installation

• Download from: https://awscli.amazonaws.com/
• Follow the installation guide: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

Option 3: Homebrew (macOS)

brew install awscli

Verify Installation

aws --version

📦 Installation

1. Clone the repository

   git clone <repository-url>
   cd rekordbox-reader/packages/desktop-sync
   

2. Install dependencies

   npm install
   

3. Configure environment

   cp .env.example .env
   # Edit .env with your S3 configuration
   

4. Build the application

   npm run build
   

5. Start the application

   npm run dev
   

⚙️ Configuration

Create a .env file in the project root with the following variables:

# S3 Configuration (Garage)
S3_ENDPOINT=http://your-garage-instance:3900
S3_REGION=garage
S3_ACCESS_KEY_ID=your-access-key
S3_SECRET_ACCESS_KEY=your-secret-key
S3_BUCKET_NAME=your-bucket-name
S3_USE_SSL=false

# Sync Configuration
SYNC_LOCAL_PATH=/path/to/your/local/music/folder
SYNC_INTERVAL=30000
SYNC_AUTO_START=false
SYNC_CONFLICT_RESOLUTION=newer-wins

# UI Configuration
UI_THEME=dark
UI_LANGUAGE=en

Garage S3 Configuration

For Garage S3 compatibility, ensure your configuration includes:

• Endpoint: Your Garage instance URL (e.g., http://localhost:3900)
• Region: Usually garage or us-east-1
• SSL: Set to false for local Garage instances

🎯 Usage

Starting Sync

1. Launch the application
2. Click "Start Sync" to begin bidirectional synchronization
3. The app will:
   • Download all files from S3 to local (first time)
   • Upload new/changed local files to S3
   • Start continuous bidirectional sync

Sync Modes

Initial Sync

• Downloads all files from S3 to local
• Ensures local folder matches S3 bucket contents
• Excludes temporary files (.tmp, .temp, .part, .DS_Store)

Continuous Sync

• Monitors both local and S3 for changes
• Automatically syncs new, modified, or deleted files
• Runs every 30 seconds by default
• Maintains bidirectional consistency

Force Full Sync

• Completely resynchronizes all files
• Useful for resolving sync conflicts
• Deletes and re-downloads all files

File Handling

• Temporary Files: Automatically excluded and cleaned up
• Conflict Resolution: Newer timestamp wins by default
• Delete Propagation: Files deleted locally are removed from S3 and vice versa
• Incremental Updates: Only changed files are transferred

🔍 Monitoring

Real-time Status

• Current sync phase (downloading, uploading, watching)
• Progress percentage and file counts
• Transfer speed and ETA
• Error messages and retry attempts

Activity Log

• Detailed AWS CLI output
• File operations and sync events
• Error tracking and resolution

File Counts

• Accurate local file counting
• S3 bucket file statistics
• Sync progress tracking

🛠️ Development

Project Structure

src/
├── main.ts                 # Main Electron process
├── preload.ts             # Preload script for IPC
├── services/
│   ├── awsS3Service.ts    # AWS S3 sync service
│   ├── configManager.ts   # Configuration management
│   ├── fileWatcher.ts     # Local file system monitoring
│   └── syncManager.ts     # Sync orchestration
└── renderer/              # UI components
    ├── index.html
    ├── renderer.js
    └── styles.css

Available Scripts

• npm run dev - Development mode with hot reload
• npm run build - Build TypeScript to JavaScript
• npm run start - Start the built application
• npm run package - Package for distribution
• npm run install-aws-cli - Install AWS CLI (macOS)

Building

npm run build
npm start

🚨 Troubleshooting

Common Issues

AWS CLI Not Found

# Check if AWS CLI is installed
aws --version

# Install if missing
npm run install-aws-cli

Sync Fails to Start

• Verify S3 credentials in .env
• Check network connectivity to Garage instance
• Ensure local sync path exists and is writable

Files Not Syncing

• Check file permissions
• Verify S3 bucket access
• Review activity log for error messages

Performance Issues

• AWS CLI v2 provides optimal performance
• Consider adjusting sync interval
• Monitor network bandwidth usage

Debug Mode

Enable detailed logging by setting environment variables:

DEBUG=* npm run dev

📊 Performance

• AWS CLI v2: Optimized for S3 operations
• Incremental Sync: Only transfers changed files
• Parallel Operations: Efficient file transfer
• Memory Management: Minimal memory footprint
• Network Optimization: Intelligent retry and backoff

🔒 Security

• Credential Management: Secure storage of S3 credentials
• Local Storage: Credentials stored locally, never transmitted
• SSL Support: Configurable SSL/TLS for S3 endpoints
• Access Control: Follows S3 bucket policies and IAM permissions

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

📄 License

MIT License - see LICENSE file for details

🙏 Acknowledgments

• AWS CLI: Powerful S3 sync capabilities
• Electron: Cross-platform desktop framework
• Garage: Self-hosted S3-compatible storage
• Rekordbox: Professional DJ software

---

Note: This tool is designed for personal and professional use with Garage S3 storage. Ensure compliance with your organization's data policies and S3 usage guidelines.