A desktop application for bidirectional synchronization between a Garage-hosted S3 instance and your local computer, specifically designed for Rekordbox music libraries.

🚀 Features

Bidirectional S3 Sync: Seamlessly sync files between your local machine and S3 storage
Incremental Sync: Only sync files that have changed since the last sync
Automatic Cleanup: Removes temporary files before syncing
Real-time Monitoring: Continuous sync with configurable intervals
Error Handling: Robust error handling with automatic retries
Progress Tracking: Real-time progress updates and file counting
Cross-platform: Built with Electron for macOS, Windows, and Linux

🔧 Prerequisites

AWS CLI v2

This tool requires AWS CLI v2 to be installed on your system. The AWS CLI provides the aws s3 sync command which offers superior performance and reliability compared to other S3 sync tools.

Installation Options:

Option 1: Automatic Installation (macOS)

npm run install-aws-cli

Option 2: Manual Installation

Download from: https://awscli.amazonaws.com/
Follow the installation guide: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

Option 3: Homebrew (macOS)

brew install awscli

Verify Installation

aws --version

📦 Installation

Clone the repository

git clone <repository-url>
cd rekordbox-reader/packages/desktop-sync

Install dependencies
```
npm install
```

Configure environment

cp .env.example .env
# Edit .env with your S3 configuration

Build the application
```
npm run build
```
Start the application
```
npm run dev
```

⚙️ Configuration

Create a .env file in the project root with the following variables:

# S3 Configuration (Garage)
S3_ENDPOINT=http://your-garage-instance:3900
S3_REGION=garage
S3_ACCESS_KEY_ID=your-access-key
S3_SECRET_ACCESS_KEY=your-secret-key
S3_BUCKET_NAME=your-bucket-name
S3_USE_SSL=false

# Sync Configuration
SYNC_LOCAL_PATH=/path/to/your/local/music/folder
SYNC_INTERVAL=30000
SYNC_AUTO_START=false
SYNC_CONFLICT_RESOLUTION=newer-wins

# UI Configuration
UI_THEME=dark
UI_LANGUAGE=en

Garage S3 Configuration

For Garage S3 compatibility, ensure your configuration includes:

Endpoint: Your Garage instance URL (e.g., http://localhost:3900)
Region: Usually garage or us-east-1
SSL: Set to false for local Garage instances

🎯 Usage

Starting Sync

Launch the application
Click "Start Sync" to begin bidirectional synchronization
The app will:
- Download all files from S3 to local (first time)
- Upload new/changed local files to S3
- Start continuous bidirectional sync

Sync Modes

Initial Sync

Downloads all files from S3 to local
Ensures local folder matches S3 bucket contents
Excludes temporary files (.tmp, .temp, .part, .DS_Store)

Continuous Sync

Monitors both local and S3 for changes
Automatically syncs new, modified, or deleted files
Runs every 30 seconds by default
Maintains bidirectional consistency

Force Full Sync

Completely resynchronizes all files
Useful for resolving sync conflicts
Deletes and re-downloads all files

File Handling

Temporary Files: Automatically excluded and cleaned up
Conflict Resolution: Newer timestamp wins by default
Delete Propagation: Files deleted locally are removed from S3 and vice versa
Incremental Updates: Only changed files are transferred

🔍 Monitoring

Real-time Status

Current sync phase (downloading, uploading, watching)
Progress percentage and file counts
Transfer speed and ETA
Error messages and retry attempts

Activity Log

Detailed AWS CLI output
File operations and sync events
Error tracking and resolution

File Counts

Accurate local file counting
S3 bucket file statistics
Sync progress tracking

🛠️ Development

Project Structure

src/
├── main.ts                 # Main Electron process
├── preload.ts             # Preload script for IPC
├── services/
│   ├── awsS3Service.ts    # AWS S3 sync service
│   ├── configManager.ts   # Configuration management
│   ├── fileWatcher.ts     # Local file system monitoring
│   └── syncManager.ts     # Sync orchestration
└── renderer/              # UI components
    ├── index.html
    ├── renderer.js
    └── styles.css

Available Scripts

npm run dev - Development mode with hot reload
npm run build - Build TypeScript to JavaScript
npm run start - Start the built application
npm run package - Package for distribution
npm run install-aws-cli - Install AWS CLI (macOS)

Building

npm run build
npm start

🚨 Troubleshooting

Common Issues

AWS CLI Not Found

# Check if AWS CLI is installed
aws --version

# Install if missing
npm run install-aws-cli

Sync Fails to Start

Verify S3 credentials in .env
Check network connectivity to Garage instance
Ensure local sync path exists and is writable

Files Not Syncing

Check file permissions
Verify S3 bucket access
Review activity log for error messages

Performance Issues

AWS CLI v2 provides optimal performance
Consider adjusting sync interval
Monitor network bandwidth usage

Debug Mode

Enable detailed logging by setting environment variables:

DEBUG=* npm run dev

📊 Performance

AWS CLI v2: Optimized for S3 operations
Incremental Sync: Only transfers changed files
Parallel Operations: Efficient file transfer
Memory Management: Minimal memory footprint
Network Optimization: Intelligent retry and backoff

🔒 Security

Credential Management: Secure storage of S3 credentials
Local Storage: Credentials stored locally, never transmitted
SSL Support: Configurable SSL/TLS for S3 endpoints
Access Control: Follows S3 bucket policies and IAM permissions

🤝 Contributing

Fork the repository
Create a feature branch
Make your changes
Add tests if applicable
Submit a pull request

📄 License

MIT License - see LICENSE file for details

🙏 Acknowledgments

AWS CLI: Powerful S3 sync capabilities
Electron: Cross-platform desktop framework
Garage: Self-hosted S3-compatible storage
Rekordbox: Professional DJ software

Note: This tool is designed for personal and professional use with Garage S3 storage. Ensure compliance with your organization's data policies and S3 usage guidelines.