256 lines
6.7 KiB
Markdown
256 lines
6.7 KiB
Markdown
# Rekordbox Sync - Desktop Companion
|
|
|
|
A desktop application for bidirectional synchronization between a Garage-hosted S3 instance and your local computer, specifically designed for Rekordbox music libraries.
|
|
|
|
## 🚀 Features
|
|
|
|
- **Bidirectional S3 Sync**: Seamlessly sync files between your local machine and S3 storage
|
|
- **Incremental Sync**: Only sync files that have changed since the last sync
|
|
- **Automatic Cleanup**: Removes temporary files before syncing
|
|
- **Real-time Monitoring**: Continuous sync with configurable intervals
|
|
- **Error Handling**: Robust error handling with automatic retries
|
|
- **Progress Tracking**: Real-time progress updates and file counting
|
|
- **Cross-platform**: Built with Electron for macOS, Windows, and Linux
|
|
|
|
## 🔧 Prerequisites
|
|
|
|
### AWS CLI v2
|
|
This tool requires AWS CLI v2 to be installed on your system. The AWS CLI provides the `aws s3 sync` command which offers superior performance and reliability compared to other S3 sync tools.
|
|
|
|
#### Installation Options:
|
|
|
|
**Option 1: Automatic Installation (macOS)**
|
|
```bash
|
|
npm run install-aws-cli
|
|
```
|
|
|
|
**Option 2: Manual Installation**
|
|
- Download from: https://awscli.amazonaws.com/
|
|
- Follow the installation guide: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
|
|
|
|
**Option 3: Homebrew (macOS)**
|
|
```bash
|
|
brew install awscli
|
|
```
|
|
|
|
#### Verify Installation
|
|
```bash
|
|
aws --version
|
|
```
|
|
|
|
## 📦 Installation
|
|
|
|
1. **Clone the repository**
|
|
```bash
|
|
git clone <repository-url>
|
|
cd rekordbox-reader/packages/desktop-sync
|
|
```
|
|
|
|
2. **Install dependencies**
|
|
```bash
|
|
npm install
|
|
```
|
|
|
|
3. **Configure environment**
|
|
```bash
|
|
cp .env.example .env
|
|
# Edit .env with your S3 configuration
|
|
```
|
|
|
|
4. **Build the application**
|
|
```bash
|
|
npm run build
|
|
```
|
|
|
|
5. **Start the application**
|
|
```bash
|
|
npm run dev
|
|
```
|
|
|
|
## ⚙️ Configuration
|
|
|
|
Create a `.env` file in the project root with the following variables:
|
|
|
|
```env
|
|
# S3 Configuration (Garage)
|
|
S3_ENDPOINT=http://your-garage-instance:3900
|
|
S3_REGION=garage
|
|
S3_ACCESS_KEY_ID=your-access-key
|
|
S3_SECRET_ACCESS_KEY=your-secret-key
|
|
S3_BUCKET_NAME=your-bucket-name
|
|
S3_USE_SSL=false
|
|
|
|
# Sync Configuration
|
|
SYNC_LOCAL_PATH=/path/to/your/local/music/folder
|
|
SYNC_INTERVAL=30000
|
|
SYNC_AUTO_START=false
|
|
SYNC_CONFLICT_RESOLUTION=newer-wins
|
|
|
|
# UI Configuration
|
|
UI_THEME=dark
|
|
UI_LANGUAGE=en
|
|
```
|
|
|
|
### Garage S3 Configuration
|
|
|
|
For Garage S3 compatibility, ensure your configuration includes:
|
|
- **Endpoint**: Your Garage instance URL (e.g., `http://localhost:3900`)
|
|
- **Region**: Usually `garage` or `us-east-1`
|
|
- **SSL**: Set to `false` for local Garage instances
|
|
|
|
## 🎯 Usage
|
|
|
|
### Starting Sync
|
|
1. Launch the application
|
|
2. Click "Start Sync" to begin bidirectional synchronization
|
|
3. The app will:
|
|
- Download all files from S3 to local (first time)
|
|
- Upload new/changed local files to S3
|
|
- Start continuous bidirectional sync
|
|
|
|
### Sync Modes
|
|
|
|
#### **Initial Sync**
|
|
- Downloads all files from S3 to local
|
|
- Ensures local folder matches S3 bucket contents
|
|
- Excludes temporary files (`.tmp`, `.temp`, `.part`, `.DS_Store`)
|
|
|
|
#### **Continuous Sync**
|
|
- Monitors both local and S3 for changes
|
|
- Automatically syncs new, modified, or deleted files
|
|
- Runs every 30 seconds by default
|
|
- Maintains bidirectional consistency
|
|
|
|
#### **Force Full Sync**
|
|
- Completely resynchronizes all files
|
|
- Useful for resolving sync conflicts
|
|
- Deletes and re-downloads all files
|
|
|
|
### File Handling
|
|
|
|
- **Temporary Files**: Automatically excluded and cleaned up
|
|
- **Conflict Resolution**: Newer timestamp wins by default
|
|
- **Delete Propagation**: Files deleted locally are removed from S3 and vice versa
|
|
- **Incremental Updates**: Only changed files are transferred
|
|
|
|
## 🔍 Monitoring
|
|
|
|
### Real-time Status
|
|
- Current sync phase (downloading, uploading, watching)
|
|
- Progress percentage and file counts
|
|
- Transfer speed and ETA
|
|
- Error messages and retry attempts
|
|
|
|
### Activity Log
|
|
- Detailed AWS CLI output
|
|
- File operations and sync events
|
|
- Error tracking and resolution
|
|
|
|
### File Counts
|
|
- Accurate local file counting
|
|
- S3 bucket file statistics
|
|
- Sync progress tracking
|
|
|
|
## 🛠️ Development
|
|
|
|
### Project Structure
|
|
```
|
|
src/
|
|
├── main.ts # Main Electron process
|
|
├── preload.ts # Preload script for IPC
|
|
├── services/
|
|
│ ├── awsS3Service.ts # AWS S3 sync service
|
|
│ ├── configManager.ts # Configuration management
|
|
│ ├── fileWatcher.ts # Local file system monitoring
|
|
│ └── syncManager.ts # Sync orchestration
|
|
└── renderer/ # UI components
|
|
├── index.html
|
|
├── renderer.js
|
|
└── styles.css
|
|
```
|
|
|
|
### Available Scripts
|
|
- `npm run dev` - Development mode with hot reload
|
|
- `npm run build` - Build TypeScript to JavaScript
|
|
- `npm run start` - Start the built application
|
|
- `npm run package` - Package for distribution
|
|
- `npm run install-aws-cli` - Install AWS CLI (macOS)
|
|
|
|
### Building
|
|
```bash
|
|
npm run build
|
|
npm start
|
|
```
|
|
|
|
## 🚨 Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
**AWS CLI Not Found**
|
|
```bash
|
|
# Check if AWS CLI is installed
|
|
aws --version
|
|
|
|
# Install if missing
|
|
npm run install-aws-cli
|
|
```
|
|
|
|
**Sync Fails to Start**
|
|
- Verify S3 credentials in `.env`
|
|
- Check network connectivity to Garage instance
|
|
- Ensure local sync path exists and is writable
|
|
|
|
**Files Not Syncing**
|
|
- Check file permissions
|
|
- Verify S3 bucket access
|
|
- Review activity log for error messages
|
|
|
|
**Performance Issues**
|
|
- AWS CLI v2 provides optimal performance
|
|
- Consider adjusting sync interval
|
|
- Monitor network bandwidth usage
|
|
|
|
### Debug Mode
|
|
Enable detailed logging by setting environment variables:
|
|
```bash
|
|
DEBUG=* npm run dev
|
|
```
|
|
|
|
## 📊 Performance
|
|
|
|
- **AWS CLI v2**: Optimized for S3 operations
|
|
- **Incremental Sync**: Only transfers changed files
|
|
- **Parallel Operations**: Efficient file transfer
|
|
- **Memory Management**: Minimal memory footprint
|
|
- **Network Optimization**: Intelligent retry and backoff
|
|
|
|
## 🔒 Security
|
|
|
|
- **Credential Management**: Secure storage of S3 credentials
|
|
- **Local Storage**: Credentials stored locally, never transmitted
|
|
- **SSL Support**: Configurable SSL/TLS for S3 endpoints
|
|
- **Access Control**: Follows S3 bucket policies and IAM permissions
|
|
|
|
## 🤝 Contributing
|
|
|
|
1. Fork the repository
|
|
2. Create a feature branch
|
|
3. Make your changes
|
|
4. Add tests if applicable
|
|
5. Submit a pull request
|
|
|
|
## 📄 License
|
|
|
|
MIT License - see LICENSE file for details
|
|
|
|
## 🙏 Acknowledgments
|
|
|
|
- **AWS CLI**: Powerful S3 sync capabilities
|
|
- **Electron**: Cross-platform desktop framework
|
|
- **Garage**: Self-hosted S3-compatible storage
|
|
- **Rekordbox**: Professional DJ software
|
|
|
|
---
|
|
|
|
**Note**: This tool is designed for personal and professional use with Garage S3 storage. Ensure compliance with your organization's data policies and S3 usage guidelines.
|