diff --git a/README.md b/README.md index f576ddd..a61b49d 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,8 @@ A web application for reading, managing, and exporting Rekordbox XML files. This - **Playlist Management**: Create, edit, and organize playlists and folders - **Song Details**: View detailed information about tracks including BPM, key, rating, etc. - **Export Functionality**: Export modified libraries back to XML format +- **Music File Storage**: Upload and stream music files with multiple storage providers +- **Storage Providers**: Support for S3-compatible storage (AWS S3, MinIO) and WebDAV (Nextcloud, ownCloud) - **Responsive Design**: Works on desktop and mobile devices - **Database Storage**: Persistent storage using MongoDB @@ -130,6 +132,34 @@ The frontend is configured to connect to the backend API. The API URL can be con - Development: `packages/frontend/src/services/api.ts` - Production: Environment variable `VITE_API_URL` in Docker +### Storage Configuration + +The application supports multiple storage providers for music files: + +#### S3-Compatible Storage (AWS S3, MinIO) +```env +STORAGE_PROVIDER=s3 +S3_ENDPOINT=http://localhost:9000 +S3_ACCESS_KEY_ID=minioadmin +S3_SECRET_ACCESS_KEY=minioadmin +S3_BUCKET_NAME=music-files +S3_REGION=us-east-1 +S3_USE_SSL=false +``` + +#### WebDAV (Nextcloud, ownCloud) +```env +STORAGE_PROVIDER=webdav +WEBDAV_URL=https://your-nextcloud.com/remote.php/dav/files/username/ +WEBDAV_USERNAME=your-username +WEBDAV_PASSWORD=your-password-or-app-password +WEBDAV_BASE_PATH=/music-files +``` + +You can also configure storage through the web interface at **Configuration โ†’ Storage Configuration**. + +For detailed setup instructions, see [WEBDAV_INTEGRATION.md](./WEBDAV_INTEGRATION.md). + ## ๐Ÿ“Š API Endpoints - `GET /api/health` - Health check diff --git a/WEBDAV_INTEGRATION.md b/WEBDAV_INTEGRATION.md new file mode 100644 index 0000000..6d59c92 --- /dev/null +++ b/WEBDAV_INTEGRATION.md @@ -0,0 +1,258 @@ +# WebDAV Integration for Rekordbox Reader + +This document describes the WebDAV integration that allows Rekordbox Reader to work with self-hosted storage solutions like Nextcloud and ownCloud. + +## ๐ŸŽฏ Overview + +The application now supports two storage providers: +- **S3-Compatible Storage** (AWS S3, MinIO, etc.) +- **WebDAV** (Nextcloud, ownCloud, etc.) + +Users can choose between these providers in the Storage Configuration page. + +## ๐Ÿ—๏ธ Architecture + +### Storage Provider Abstraction + +The system uses a provider abstraction pattern: + +``` +StorageProvider Interface +โ”œโ”€โ”€ S3Service (implements StorageProvider) +โ””โ”€โ”€ WebDAVService (implements StorageProvider) +``` + +### Key Components + +1. **StorageProvider Interface** (`src/services/storageProvider.ts`) + - Defines common operations for all storage providers + - Factory pattern for creating providers + - Configuration loading and validation + +2. **WebDAVService** (`src/services/webdavService.ts`) + - Implements WebDAV operations using the `webdav` npm package + - Supports Nextcloud, ownCloud, and other WebDAV-compatible servers + - Handles file upload, download, listing, and deletion + +3. **Updated Configuration System** + - New `/api/config/storage` endpoints + - Support for both S3 and WebDAV configuration + - Backward compatibility with existing S3 configuration + +4. **Frontend Storage Configuration** + - Provider selection (S3 vs WebDAV) + - Dynamic form fields based on selected provider + - Connection testing for both providers + +## ๐Ÿš€ Setup Instructions + +### 1. Backend Configuration + +The backend automatically detects the storage provider from the configuration file: + +**File: `storage-config.json`** +```json +{ + "provider": "webdav", + "url": "https://your-nextcloud.com/remote.php/dav/files/username/", + "username": "your-username", + "password": "your-password-or-app-password", + "basePath": "/music-files" +} +``` + +### 2. Environment Variables + +You can also configure WebDAV using environment variables: + +```bash +STORAGE_PROVIDER=webdav +WEBDAV_URL=https://your-nextcloud.com/remote.php/dav/files/username/ +WEBDAV_USERNAME=your-username +WEBDAV_PASSWORD=your-password-or-app-password +WEBDAV_BASE_PATH=/music-files +``` + +### 3. Frontend Configuration + +1. Navigate to **Configuration โ†’ Storage Configuration** +2. Select **WebDAV** as the storage provider +3. Fill in your WebDAV server details: + - **URL**: Your Nextcloud/ownCloud WebDAV URL + - **Username**: Your account username + - **Password**: Your password or app password + - **Base Path**: Optional subfolder for music files +4. Click **Test Connection** to verify +5. Click **Save Configuration** to apply + +## ๐Ÿ”ง Nextcloud Setup + +### 1. Enable WebDAV + +WebDAV is enabled by default in Nextcloud. You can verify this in: +- **Settings โ†’ Administration โ†’ Basic settings** + +### 2. Create App Password (Recommended) + +For better security, create an app password: + +1. Go to **Settings โ†’ Personal โ†’ Security** +2. Scroll down to **App passwords** +3. Create a new app password for "Rekordbox Reader" +4. Use this password instead of your main password + +### 3. Get WebDAV URL + +Your WebDAV URL follows this pattern: +``` +https://your-nextcloud.com/remote.php/dav/files/username/ +``` + +Replace: +- `your-nextcloud.com` with your Nextcloud domain +- `username` with your Nextcloud username + +### 4. Create Music Folder (Optional) + +You can create a dedicated folder for music files: +1. In Nextcloud, create a folder called `music-files` +2. Set this as the `basePath` in the configuration + +## ๐Ÿ”ง ownCloud Setup + +The setup is similar to Nextcloud: + +1. Enable WebDAV in ownCloud settings +2. Create an app password for security +3. Use the WebDAV URL: `https://your-owncloud.com/remote.php/dav/files/username/` +4. Configure the same way as Nextcloud + +## ๐Ÿงช Testing + +### Backend Test + +Run the WebDAV test script: + +```bash +cd packages/backend +node test-webdav.js +``` + +Make sure to update the configuration in the test file first. + +### Frontend Test + +1. Go to **Configuration โ†’ Storage Configuration** +2. Select **WebDAV** provider +3. Enter your WebDAV details +4. Click **Test Connection** +5. Verify the connection is successful + +## ๐Ÿ“ File Operations + +The WebDAV service supports all standard file operations: + +- **Upload**: Upload music files to WebDAV storage +- **Download**: Download files for playback +- **List**: List all files and folders +- **Delete**: Remove files from storage +- **Metadata**: Get file information +- **Streaming**: Generate streaming URLs + +## ๐Ÿ”’ Security Considerations + +1. **Use App Passwords**: Don't use your main Nextcloud/ownCloud password +2. **HTTPS Only**: Always use HTTPS URLs for WebDAV connections +3. **Base Path**: Use a dedicated folder for music files +4. **Permissions**: Ensure the WebDAV user has appropriate permissions + +## ๐Ÿ› Troubleshooting + +### Common Issues + +1. **Connection Failed** + - Verify the WebDAV URL is correct + - Check username and password + - Ensure WebDAV is enabled on your server + +2. **Permission Denied** + - Check if the user has write permissions + - Verify the base path exists and is accessible + +3. **SSL/TLS Errors** + - Ensure you're using HTTPS + - Check if the SSL certificate is valid + +4. **File Upload Fails** + - Check available storage space + - Verify file permissions + - Ensure the file format is supported + +### Debug Mode + +Enable debug logging by setting: +```bash +DEBUG=webdav:* +``` + +## ๐Ÿ”„ Migration from S3 + +If you're migrating from S3 to WebDAV: + +1. Export your current configuration +2. Set up WebDAV storage +3. Update the configuration to use WebDAV +4. The application will automatically use the new storage provider + +## ๐Ÿ“Š Performance + +WebDAV performance depends on: +- Network latency to your server +- Server performance and storage type +- File sizes and concurrent operations + +For best performance: +- Use a local or fast Nextcloud/ownCloud instance +- Consider using SSD storage +- Optimize your network connection + +## ๐ŸŽต Supported File Formats + +The WebDAV integration supports all audio formats supported by the application: +- MP3 +- WAV +- FLAC +- M4A +- AAC +- OGG +- OPUS +- WMA + +## ๐Ÿ“ API Endpoints + +### Storage Configuration + +- `GET /api/config/storage` - Get current storage configuration +- `POST /api/config/storage` - Save storage configuration +- `POST /api/config/storage/test` - Test storage connection + +### Legacy S3 Endpoints + +The following endpoints are still available for backward compatibility: +- `GET /api/config/s3` - Get S3 configuration +- `POST /api/config/s3` - Save S3 configuration +- `POST /api/config/s3/test` - Test S3 connection + +## ๐Ÿค Contributing + +When adding new storage providers: + +1. Implement the `StorageProvider` interface +2. Add the provider to `StorageProviderFactory` +3. Update the frontend configuration UI +4. Add appropriate tests +5. Update this documentation + +## ๐Ÿ“„ License + +This WebDAV integration follows the same license as the main Rekordbox Reader project. diff --git a/package-lock.json b/package-lock.json index 6365a91..1dad9b6 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1180,6 +1180,14 @@ "node": ">=6.9.0" } }, + "node_modules/@buttercup/fetch": { + "version": "0.2.1", + "resolved": "https://registry.npmjs.org/@buttercup/fetch/-/fetch-0.2.1.tgz", + "integrity": "sha512-sCgECOx8wiqY8NN1xN22BqqKzXYIG2AicNLlakOAI4f0WgyLVUbAigMf8CZhBtJxdudTcB1gD5lciqi44jwJvg==", + "optionalDependencies": { + "node-fetch": "^3.3.0" + } + }, "node_modules/@chakra-ui/accordion": { "version": "2.3.1", "resolved": "https://registry.npmjs.org/@chakra-ui/accordion/-/accordion-2.3.1.tgz", @@ -4261,9 +4269,13 @@ "version": "1.0.2", "resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz", "integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==", - "dev": true, "license": "MIT" }, + "node_modules/base-64": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/base-64/-/base-64-1.0.0.tgz", + "integrity": "sha512-kwDPIFCGx0NZHog36dj+tHiwP4QMzsZ3AgMViUBKI0+V5n4U0ufTCUMhnQ04diaRI8EX/QcPfql7zlhZ7j4zgg==" + }, "node_modules/base64-js": { "version": "1.5.1", "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz", @@ -4428,6 +4440,11 @@ "node": ">=10.16.0" } }, + "node_modules/byte-length": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/byte-length/-/byte-length-1.0.2.tgz", + "integrity": "sha512-ovBpjmsgd/teRmgcPh23d4gJvxDoXtAzEL9xTfMU8Yc2kqCDb7L9jAG0XHl1nzuGl+h3ebCIF1i62UFyA9V/2Q==" + }, "node_modules/bytes": { "version": "3.1.2", "resolved": "https://registry.npmjs.org/bytes/-/bytes-3.1.2.tgz", @@ -4510,6 +4527,14 @@ "url": "https://github.com/chalk/chalk?sponsor=1" } }, + "node_modules/charenc": { + "version": "0.0.2", + "resolved": "https://registry.npmjs.org/charenc/-/charenc-0.0.2.tgz", + "integrity": "sha512-yrLQ/yVUFXkzg7EDQsPieE/53+0RlaWTs+wBrvW36cyilJ2SaDWfl4Yj7MtLTXleV9uEKefbAGUPv2/iWSooRA==", + "engines": { + "node": "*" + } + }, "node_modules/cliui": { "version": "8.0.1", "resolved": "https://registry.npmjs.org/cliui/-/cliui-8.0.1.tgz", @@ -4716,6 +4741,14 @@ "node": ">= 8" } }, + "node_modules/crypt": { + "version": "0.0.2", + "resolved": "https://registry.npmjs.org/crypt/-/crypt-0.0.2.tgz", + "integrity": "sha512-mCxBlsHFYh9C+HVpiEacem8FEBnMXgU9gy4zmNC+SXAZNB/1idgp/aulFJ4FgCi7GPEVbfyng092GqL2k2rmow==", + "engines": { + "node": "*" + } + }, "node_modules/css-box-model": { "version": "1.2.1", "resolved": "https://registry.npmjs.org/css-box-model/-/css-box-model-1.2.1.tgz", @@ -4730,6 +4763,14 @@ "integrity": "sha512-M1uQkMl8rQK/szD0LNhtqxIPLpimGm8sOBwU7lLnCpSbTyY3yeU1Vc7l4KT5zT4s/yOxHH5O7tIuuLOCnLADRw==", "license": "MIT" }, + "node_modules/data-uri-to-buffer": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-4.0.1.tgz", + "integrity": "sha512-0R9ikRb668HB7QDxT1vkpuUBtqc53YyAwMwGeUFKRojY/NWKvdZ+9UYtRfGmhqNbRkTSVpMbmyhXipFFv2cb/A==", + "engines": { + "node": ">= 12" + } + }, "node_modules/date-fns": { "version": "2.30.0", "resolved": "https://registry.npmjs.org/date-fns/-/date-fns-2.30.0.tgz", @@ -4842,6 +4883,17 @@ "node": ">= 0.8" } }, + "node_modules/entities": { + "version": "6.0.1", + "resolved": "https://registry.npmjs.org/entities/-/entities-6.0.1.tgz", + "integrity": "sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g==", + "engines": { + "node": ">=0.12" + }, + "funding": { + "url": "https://github.com/fb55/entities?sponsor=1" + } + }, "node_modules/error-ex": { "version": "1.3.2", "resolved": "https://registry.npmjs.org/error-ex/-/error-ex-1.3.2.tgz", @@ -5284,6 +5336,28 @@ "reusify": "^1.0.4" } }, + "node_modules/fetch-blob": { + "version": "3.2.0", + "resolved": "https://registry.npmjs.org/fetch-blob/-/fetch-blob-3.2.0.tgz", + "integrity": "sha512-7yAQpD2UMJzLi1Dqv7qFYnPbaPx7ZfFK6PiIxQ4PfkGPyNyl2Ugx+a/umUonmKqjhM4DnfbMvdX6otXq83soQQ==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/jimmywarting" + }, + { + "type": "paypal", + "url": "https://paypal.me/jimmywarting" + } + ], + "dependencies": { + "node-domexception": "^1.0.0", + "web-streams-polyfill": "^3.0.3" + }, + "engines": { + "node": "^12.20 || >= 14.13" + } + }, "node_modules/file-entry-cache": { "version": "8.0.0", "resolved": "https://registry.npmjs.org/file-entry-cache/-/file-entry-cache-8.0.0.tgz", @@ -5422,6 +5496,17 @@ "node": ">=10" } }, + "node_modules/formdata-polyfill": { + "version": "4.0.10", + "resolved": "https://registry.npmjs.org/formdata-polyfill/-/formdata-polyfill-4.0.10.tgz", + "integrity": "sha512-buewHzMvYL29jdeQTVILecSaZKnt/RJWjoZCF5OW60Z67/GmSLBkOFM7qh1PI3zFNtJbaZL5eQu1vLfazOwj4g==", + "dependencies": { + "fetch-blob": "^3.1.2" + }, + "engines": { + "node": ">=12.20.0" + } + }, "node_modules/forwarded": { "version": "0.2.0", "resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.2.0.tgz", @@ -5675,6 +5760,11 @@ "react-is": "^16.7.0" } }, + "node_modules/hot-patcher": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/hot-patcher/-/hot-patcher-2.0.1.tgz", + "integrity": "sha512-ECg1JFG0YzehicQaogenlcs2qg6WsXQsxtnbr1i696u5tLUjtJdQAh0u2g0Q5YV45f263Ta1GnUJsc8WIfJf4Q==" + }, "node_modules/http-errors": { "version": "2.0.0", "resolved": "https://registry.npmjs.org/http-errors/-/http-errors-2.0.0.tgz", @@ -5775,6 +5865,11 @@ "integrity": "sha512-zz06S8t0ozoDXMG+ube26zeCTNXcKIPJZJi8hBrF4idCLms4CG9QtK7qBl1boi5ODzFpjswb5JPmHCbMpjaYzg==", "license": "MIT" }, + "node_modules/is-buffer": { + "version": "1.1.6", + "resolved": "https://registry.npmjs.org/is-buffer/-/is-buffer-1.1.6.tgz", + "integrity": "sha512-NcdALwpXkTm5Zvvbk7owOUSvVvBKDgKP5/ewfXEznmQFfs4ZRmanOeKBTjRVjka3QFoN6XJ+9F3USqfHqTaU5w==" + }, "node_modules/is-core-module": { "version": "2.16.1", "resolved": "https://registry.npmjs.org/is-core-module/-/is-core-module-2.16.1.tgz", @@ -5928,6 +6023,11 @@ "json-buffer": "3.0.1" } }, + "node_modules/layerr": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/layerr/-/layerr-3.0.0.tgz", + "integrity": "sha512-tv754Ki2dXpPVApOrjTyRo4/QegVb9eVFq4mjqp4+NM5NaX7syQvN5BBNfV/ZpAHCEHV24XdUVrBAoka4jt3pA==" + }, "node_modules/levn": { "version": "0.4.1", "resolved": "https://registry.npmjs.org/levn/-/levn-0.4.1.tgz", @@ -6011,6 +6111,16 @@ "node": ">= 0.4" } }, + "node_modules/md5": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/md5/-/md5-2.3.0.tgz", + "integrity": "sha512-T1GITYmFaKuO91vxyoQMFETst+O71VUPEU3ze5GNzDm0OWdP8v1ziTaAEPUr/3kLsY3Sftgz242A1SetQiDL7g==", + "dependencies": { + "charenc": "0.0.2", + "crypt": "0.0.2", + "is-buffer": "~1.1.6" + } + }, "node_modules/media-typer": { "version": "0.3.0", "resolved": "https://registry.npmjs.org/media-typer/-/media-typer-0.3.0.tgz", @@ -6319,6 +6429,47 @@ "node": ">= 0.6" } }, + "node_modules/nested-property": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/nested-property/-/nested-property-4.0.0.tgz", + "integrity": "sha512-yFehXNWRs4cM0+dz7QxCd06hTbWbSkV0ISsqBfkntU6TOY4Qm3Q88fRRLOddkGh2Qq6dZvnKVAahfhjcUvLnyA==" + }, + "node_modules/node-domexception": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/node-domexception/-/node-domexception-1.0.0.tgz", + "integrity": "sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==", + "deprecated": "Use your platform's native DOMException instead", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/jimmywarting" + }, + { + "type": "github", + "url": "https://paypal.me/jimmywarting" + } + ], + "engines": { + "node": ">=10.5.0" + } + }, + "node_modules/node-fetch": { + "version": "3.3.2", + "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-3.3.2.tgz", + "integrity": "sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA==", + "dependencies": { + "data-uri-to-buffer": "^4.0.0", + "fetch-blob": "^3.1.4", + "formdata-polyfill": "^4.0.10" + }, + "engines": { + "node": "^12.20.0 || ^14.13.1 || >=16.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/node-fetch" + } + }, "node_modules/node-releases": { "version": "2.0.19", "resolved": "https://registry.npmjs.org/node-releases/-/node-releases-2.0.19.tgz", @@ -6480,6 +6631,11 @@ "integrity": "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw==", "license": "MIT" }, + "node_modules/path-posix": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/path-posix/-/path-posix-1.0.0.tgz", + "integrity": "sha512-1gJ0WpNIiYcQydgg3Ed8KzvIqTsDpNwq+cjBCssvBtuTWjEqY1AW+i+OepiEMqDCzyro9B2sLAe4RBPajMYFiA==" + }, "node_modules/path-to-regexp": { "version": "0.1.12", "resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.12.tgz", @@ -6624,6 +6780,11 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/querystringify": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/querystringify/-/querystringify-2.2.0.tgz", + "integrity": "sha512-FIqgj2EUvTa7R50u0rGsyTftzjYmv/a3hO345bZNrqabNqjtgiDMgmo4mkUjd+nzU5oF3dClKqFIPUKybUyqoQ==" + }, "node_modules/queue-microtask": { "version": "1.2.3", "resolved": "https://registry.npmjs.org/queue-microtask/-/queue-microtask-1.2.3.tgz", @@ -6939,6 +7100,11 @@ "node": ">=0.10.0" } }, + "node_modules/requires-port": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/requires-port/-/requires-port-1.0.0.tgz", + "integrity": "sha512-KigOCHcocU3XODJxsu8i/j8T9tzT4adHiecwORRQ0ZZFcp7ahwXuRU1m+yuO90C5ZUyGeGfocHDI14M3L3yDAQ==" + }, "node_modules/resolve": { "version": "1.22.10", "resolved": "https://registry.npmjs.org/resolve/-/resolve-1.22.10.tgz", @@ -7675,6 +7841,23 @@ "punycode": "^2.1.0" } }, + "node_modules/url-join": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/url-join/-/url-join-5.0.0.tgz", + "integrity": "sha512-n2huDr9h9yzd6exQVnH/jU5mr+Pfx08LRXXZhkLLetAMESRj+anQsTAh940iMrIetKAmry9coFuZQ2jY8/p3WA==", + "engines": { + "node": "^12.20.0 || ^14.13.1 || >=16.0.0" + } + }, + "node_modules/url-parse": { + "version": "1.5.10", + "resolved": "https://registry.npmjs.org/url-parse/-/url-parse-1.5.10.tgz", + "integrity": "sha512-WypcfiRhfeUP9vvF0j6rw0J3hrWrw6iZv3+22h6iRMJ/8z1Tj6XfLP4DsUix5MhMPnXpiHDoKyoZ/bdCkwBCiQ==", + "dependencies": { + "querystringify": "^2.1.1", + "requires-port": "^1.0.0" + } + }, "node_modules/use-callback-ref": { "version": "1.3.3", "resolved": "https://registry.npmjs.org/use-callback-ref/-/use-callback-ref-1.3.3.tgz", @@ -7821,6 +8004,88 @@ } } }, + "node_modules/web-streams-polyfill": { + "version": "3.3.3", + "resolved": "https://registry.npmjs.org/web-streams-polyfill/-/web-streams-polyfill-3.3.3.tgz", + "integrity": "sha512-d2JWLCivmZYTSIoge9MsgFCZrt571BikcWGYkjC1khllbTeDlGqZ2D8vD8E/lJa8WGWbb7Plm8/XJYV7IJHZZw==", + "engines": { + "node": ">= 8" + } + }, + "node_modules/webdav": { + "version": "5.8.0", + "resolved": "https://registry.npmjs.org/webdav/-/webdav-5.8.0.tgz", + "integrity": "sha512-iuFG7NamJ41Oshg4930iQgfIpRrUiatPWIekeznYgEf2EOraTRcDPTjy7gIOMtkdpKTaqPk1E68NO5PAGtJahA==", + "dependencies": { + "@buttercup/fetch": "^0.2.1", + "base-64": "^1.0.0", + "byte-length": "^1.0.2", + "entities": "^6.0.0", + "fast-xml-parser": "^4.5.1", + "hot-patcher": "^2.0.1", + "layerr": "^3.0.0", + "md5": "^2.3.0", + "minimatch": "^9.0.5", + "nested-property": "^4.0.0", + "node-fetch": "^3.3.2", + "path-posix": "^1.0.0", + "url-join": "^5.0.0", + "url-parse": "^1.5.10" + }, + "engines": { + "node": ">=14" + } + }, + "node_modules/webdav/node_modules/brace-expansion": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.2.tgz", + "integrity": "sha512-Jt0vHyM+jmUBqojB7E1NIYadt0vI0Qxjxd2TErW94wDz+E2LAm5vKMXXwg6ZZBTHPuUlDgQHKXvjGBdfcF1ZDQ==", + "dependencies": { + "balanced-match": "^1.0.0" + } + }, + "node_modules/webdav/node_modules/fast-xml-parser": { + "version": "4.5.3", + "resolved": "https://registry.npmjs.org/fast-xml-parser/-/fast-xml-parser-4.5.3.tgz", + "integrity": "sha512-RKihhV+SHsIUGXObeVy9AXiBbFwkVk7Syp8XgwN5U3JV416+Gwp/GO9i0JYKmikykgz/UHRrrV4ROuZEo/T0ig==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/NaturalIntelligence" + } + ], + "dependencies": { + "strnum": "^1.1.1" + }, + "bin": { + "fxparser": "src/cli/cli.js" + } + }, + "node_modules/webdav/node_modules/minimatch": { + "version": "9.0.5", + "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-9.0.5.tgz", + "integrity": "sha512-G6T0ZX48xgozx7587koeX9Ys2NYy6Gmv//P89sEte9V9whIapMNF4idKxnW2QtCcLiTWlb/wfCabAtAFWhhBow==", + "dependencies": { + "brace-expansion": "^2.0.1" + }, + "engines": { + "node": ">=16 || 14 >=14.17" + }, + "funding": { + "url": "https://github.com/sponsors/isaacs" + } + }, + "node_modules/webdav/node_modules/strnum": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/strnum/-/strnum-1.1.2.tgz", + "integrity": "sha512-vrN+B7DBIoTTZjnPNewwhx6cBA/H+IS7rfW68n7XxC1y7uoiGQBxaKzqucGUgavX15dJgiGztLJ8vxuEzwqBdA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/NaturalIntelligence" + } + ] + }, "node_modules/webidl-conversions": { "version": "7.0.0", "resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-7.0.0.tgz", @@ -8003,7 +8268,8 @@ "mongoose": "^8.2.1", "multer": "^2.0.0-rc.3", "music-metadata": "^8.1.0", - "uuid": "^11.1.0" + "uuid": "^11.1.0", + "webdav": "^5.3.0" }, "devDependencies": { "@types/cors": "^2.8.17", diff --git a/packages/backend/package.json b/packages/backend/package.json index a29e793..2d817cf 100644 --- a/packages/backend/package.json +++ b/packages/backend/package.json @@ -17,7 +17,8 @@ "mongoose": "^8.2.1", "multer": "^2.0.0-rc.3", "music-metadata": "^8.1.0", - "uuid": "^11.1.0" + "uuid": "^11.1.0", + "webdav": "^5.3.0" }, "devDependencies": { "@types/cors": "^2.8.17", diff --git a/packages/backend/src/routes/config.ts b/packages/backend/src/routes/config.ts index 072da59..a08a199 100644 --- a/packages/backend/src/routes/config.ts +++ b/packages/backend/src/routes/config.ts @@ -2,14 +2,16 @@ import express from 'express'; import { S3Client, ListBucketsCommand, HeadBucketCommand } from '@aws-sdk/client-s3'; import fs from 'fs/promises'; import path from 'path'; -import { reloadS3Service } from './music.js'; +import { reloadStorageService } from './music.js'; +import { StorageProviderFactory, StorageConfig } from '../services/storageProvider.js'; const router = express.Router(); -// Path to the S3 configuration file -const CONFIG_FILE_PATH = path.join(process.cwd(), 's3-config.json'); +// Path to the storage configuration file +const CONFIG_FILE_PATH = path.join(process.cwd(), 'storage-config.json'); -interface S3Config { +interface S3Config extends StorageConfig { + provider: 's3'; endpoint: string; region: string; accessKeyId: string; @@ -18,8 +20,56 @@ interface S3Config { useSSL: boolean; } +interface WebDAVConfig extends StorageConfig { + provider: 'webdav'; + url: string; + username: string; + password: string; + basePath?: string; +} + /** - * Get current S3 configuration + * Get current storage configuration + */ +router.get('/storage', async (req, res) => { + try { + // Check if config file exists + try { + const configData = await fs.readFile(CONFIG_FILE_PATH, 'utf-8'); + const config = JSON.parse(configData); + + // Don't return sensitive data in the response + const safeConfig = { ...config }; + if (config.provider === 's3') { + safeConfig.accessKeyId = config.accessKeyId ? '***' : ''; + safeConfig.secretAccessKey = config.secretAccessKey ? '***' : ''; + } else if (config.provider === 'webdav') { + safeConfig.password = config.password ? '***' : ''; + } + + res.json({ config: safeConfig }); + } catch (error) { + // Config file doesn't exist, return default config + res.json({ + config: { + provider: 's3', + endpoint: process.env.S3_ENDPOINT || '', + region: process.env.S3_REGION || 'us-east-1', + accessKeyId: process.env.S3_ACCESS_KEY_ID ? '***' : '', + secretAccessKey: process.env.S3_SECRET_ACCESS_KEY ? '***' : '', + bucketName: process.env.S3_BUCKET_NAME || '', + useSSL: process.env.S3_USE_SSL !== 'false', + } + }); + } + } catch (error) { + console.error('Error loading storage config:', error); + res.status(500).json({ error: 'Failed to load storage configuration' }); + } +}); + +/** + * Get current S3 configuration (legacy endpoint) */ router.get('/s3', async (req, res) => { try { @@ -56,7 +106,88 @@ router.get('/s3', async (req, res) => { }); /** - * Save S3 configuration + * Save storage configuration + */ +router.post('/storage', async (req, res) => { + try { + const newConfig: StorageConfig = req.body; + + // Load existing configuration to merge with + let existingConfig: StorageConfig | null = null; + try { + const existingData = await fs.readFile(CONFIG_FILE_PATH, 'utf-8'); + existingConfig = JSON.parse(existingData); + } catch (error) { + // No existing config file, that's okay + } + + // Merge with existing configuration to preserve passwords/credentials that weren't provided + const mergedConfig = { ...existingConfig }; + + // Only override fields that are actually provided (not empty strings) + Object.keys(newConfig).forEach(key => { + const value = newConfig[key]; + if (value !== '' && value !== null && value !== undefined) { + mergedConfig[key] = value; + } + }); + + // Validate required fields based on provider + if (mergedConfig.provider === 's3') { + const s3Config = mergedConfig as S3Config; + if (!s3Config.endpoint || !s3Config.accessKeyId || !s3Config.secretAccessKey || !s3Config.bucketName) { + return res.status(400).json({ + error: 'Missing required S3 fields: endpoint, accessKeyId, secretAccessKey, bucketName' + }); + } + } else if (mergedConfig.provider === 'webdav') { + const webdavConfig = mergedConfig as WebDAVConfig; + if (!webdavConfig.url || !webdavConfig.username || !webdavConfig.password) { + return res.status(400).json({ + error: 'Missing required WebDAV fields: url, username, password' + }); + } + } else { + return res.status(400).json({ + error: 'Invalid provider. Must be "s3" or "webdav"' + }); + } + + // Save configuration to file + await fs.writeFile(CONFIG_FILE_PATH, JSON.stringify(mergedConfig, null, 2)); + + // Update environment variables for current session + if (mergedConfig.provider === 's3') { + const s3Config = mergedConfig as S3Config; + process.env.S3_ENDPOINT = s3Config.endpoint; + process.env.S3_REGION = s3Config.region; + process.env.S3_ACCESS_KEY_ID = s3Config.accessKeyId; + process.env.S3_SECRET_ACCESS_KEY = s3Config.secretAccessKey; + process.env.S3_BUCKET_NAME = s3Config.bucketName; + process.env.S3_USE_SSL = s3Config.useSSL.toString(); + } else if (mergedConfig.provider === 'webdav') { + const webdavConfig = mergedConfig as WebDAVConfig; + process.env.WEBDAV_URL = webdavConfig.url; + process.env.WEBDAV_USERNAME = webdavConfig.username; + process.env.WEBDAV_PASSWORD = webdavConfig.password; + process.env.WEBDAV_BASE_PATH = webdavConfig.basePath || '/music-files'; + } + + // Reload storage service with new configuration + const reloadSuccess = await reloadStorageService(); + + res.json({ + message: `${mergedConfig.provider.toUpperCase()} configuration saved successfully`, + storageServiceReloaded: reloadSuccess + }); + } catch (error) { + console.error('Error saving storage config:', error); + res.status(500).json({ error: 'Failed to save storage configuration' }); + } +}); + +/** + * Save S3 configuration (legacy endpoint) */ router.post('/s3', async (req, res) => { try { @@ -80,8 +211,8 @@ router.post('/s3', async (req, res) => { process.env.S3_BUCKET_NAME = config.bucketName; process.env.S3_USE_SSL = config.useSSL.toString(); - // Reload S3 service with new configuration - const reloadSuccess = await reloadS3Service(); + // Reload storage service with new configuration + const reloadSuccess = await reloadStorageService(); res.json({ message: 'S3 configuration saved successfully', @@ -94,7 +225,83 @@ router.post('/s3', async (req, res) => { }); /** - * Test S3 connection + * Test storage connection + */ +router.post('/storage/test', async (req, res) => { + try { + const newConfig: StorageConfig = req.body; + + // Load existing configuration to merge with + let existingConfig: StorageConfig | null = null; + try { + const existingData = await fs.readFile(CONFIG_FILE_PATH, 'utf-8'); + existingConfig = JSON.parse(existingData); + } catch (error) { + // No existing config file, that's okay + } + + // Merge with existing configuration to preserve passwords/credentials that weren't provided + const config = { ...existingConfig }; + + // Only override fields that are actually provided (not empty strings) + Object.keys(newConfig).forEach(key => { + const value = newConfig[key]; + if (value !== '' && value !== null && value !== undefined) { + config[key] = value; + } + }); + + + // Validate required fields based on provider + if (config.provider === 's3') { + const s3Config = config as S3Config; + if (!s3Config.endpoint || !s3Config.accessKeyId || !s3Config.secretAccessKey || !s3Config.bucketName) { + return res.status(400).json({ + error: 'Missing required S3 fields: endpoint, accessKeyId, secretAccessKey, bucketName' + }); + } + } else if (config.provider === 'webdav') { + const webdavConfig = config as WebDAVConfig; + if (!webdavConfig.url || !webdavConfig.username || !webdavConfig.password) { + return res.status(400).json({ + error: 'Missing required WebDAV fields: url, username, password' + }); + } + } else { + return res.status(400).json({ + error: 'Invalid provider. Must be "s3" or "webdav"' + }); + } + + // Create storage provider and test connection + const provider = await StorageProviderFactory.createProvider(config); + const connectionTest = await provider.testConnection(); + + if (connectionTest) { + res.json({ + success: true, + message: `${config.provider.toUpperCase()} connection test successful`, + provider: config.provider + }); + } else { + res.status(400).json({ + success: false, + error: `${config.provider.toUpperCase()} connection test failed`, + provider: config.provider + }); + } + + } catch (error) { + console.error('Error testing storage connection:', error); + res.status(500).json({ + error: 'Failed to test storage connection', + details: error instanceof Error ? error.message : 'Unknown error' + }); + } +}); + +/** + * Test S3 connection (legacy endpoint) */ router.post('/s3/test', async (req, res) => { try { @@ -170,4 +377,6 @@ router.post('/s3/test', async (req, res) => { } }); + + export { router as configRouter }; \ No newline at end of file diff --git a/packages/backend/src/routes/music.ts b/packages/backend/src/routes/music.ts index 33a1117..ee8e940 100644 --- a/packages/backend/src/routes/music.ts +++ b/packages/backend/src/routes/music.ts @@ -1,6 +1,6 @@ import express from 'express'; import multer from 'multer'; -import { S3Service } from '../services/s3Service.js'; +import { StorageProviderFactory, StorageProvider } from '../services/storageProvider.js'; import { AudioMetadataService } from '../services/audioMetadataService.js'; import { MusicFile } from '../models/MusicFile.js'; import { Song } from '../models/Song.js'; @@ -24,40 +24,43 @@ const upload = multer({ }); // Initialize services -let s3Service: S3Service; +let storageService: StorageProvider; -// Initialize S3 service with configuration from file -async function initializeS3Service() { +// Initialize storage service with configuration from file +async function initializeStorageService() { try { - s3Service = await S3Service.createFromConfig(); - console.log('โœ… S3 service initialized with configuration from s3-config.json'); + const config = await StorageProviderFactory.loadConfig(); + storageService = await StorageProviderFactory.createProvider(config); + console.log(`โœ… Storage service initialized (${config.provider}) with configuration from storage-config.json`); } catch (error) { - console.error('โŒ Failed to initialize S3 service:', error); - // Fallback to environment variables - s3Service = new S3Service({ + console.error('โŒ Failed to initialize storage service:', error); + // Fallback to S3 with environment variables + storageService = await StorageProviderFactory.createProvider({ + provider: 's3', endpoint: process.env.S3_ENDPOINT || 'http://localhost:9000', accessKeyId: process.env.S3_ACCESS_KEY_ID || 'minioadmin', secretAccessKey: process.env.S3_SECRET_ACCESS_KEY || 'minioadmin', bucketName: process.env.S3_BUCKET_NAME || 'music-files', region: process.env.S3_REGION || 'us-east-1', }); - console.log('โš ๏ธ S3 service initialized with environment variables as fallback'); + console.log('โš ๏ธ Storage service initialized with S3 environment variables as fallback'); } } -// Initialize S3 service on startup -initializeS3Service(); +// Initialize storage service on startup +initializeStorageService(); /** - * Reload S3 service with updated configuration + * Reload storage service with updated configuration */ -export async function reloadS3Service() { +export async function reloadStorageService() { try { - s3Service = await S3Service.createFromConfig(); - console.log('โœ… S3 service reloaded with updated configuration'); + const config = await StorageProviderFactory.loadConfig(); + storageService = await StorageProviderFactory.createProvider(config); + console.log(`โœ… Storage service reloaded (${config.provider}) with updated configuration`); return true; } catch (error) { - console.error('โŒ Failed to reload S3 service:', error); + console.error('โŒ Failed to reload storage service:', error); return false; } } @@ -77,8 +80,8 @@ router.post('/upload', upload.single('file'), async (req, res) => { const targetFolder = typeof req.body?.targetFolder === 'string' ? req.body.targetFolder : undefined; const markForScan = String(req.body?.markForScan || '').toLowerCase() === 'true'; - // Upload to S3 - const uploadResult = await s3Service.uploadFile(buffer, originalname, mimetype, targetFolder); + // Upload to storage + const uploadResult = await storageService.uploadFile(buffer, originalname, mimetype, targetFolder); // Extract audio metadata const metadata = await audioMetadataService.extractMetadata(buffer, originalname); @@ -177,8 +180,8 @@ router.post('/batch-upload', upload.array('files', 10), async (req, res) => { try { const { buffer, originalname, mimetype } = file; - // Upload to S3 - const uploadResult = await s3Service.uploadFile(buffer, originalname, mimetype); + // Upload to storage + const uploadResult = await storageService.uploadFile(buffer, originalname, mimetype); // Extract audio metadata const metadata = await audioMetadataService.extractMetadata(buffer, originalname); @@ -298,7 +301,7 @@ router.get('/folders', async (req, res) => { return res.json({ folders: folderCache.folders }); } - const folders = await s3Service.listAllFolders(''); + const folders = await storageService.listAllFolders(''); const result = ['', ...folders]; // Cache the result @@ -369,7 +372,7 @@ router.get('/:id/stream', async (req, res) => { } // Use presigned URL for secure access instead of direct URL - const presignedUrl = await s3Service.getPresignedUrl(musicFile.s3Key, 3600); // 1 hour expiry + const presignedUrl = await storageService.getPresignedUrl(musicFile.s3Key, 3600); // 1 hour expiry res.json({ streamingUrl: presignedUrl, @@ -393,7 +396,7 @@ router.get('/:id/presigned', async (req, res) => { } const expiresIn = parseInt(req.query.expiresIn as string) || 3600; - const presignedUrl = await s3Service.getPresignedUrl(musicFile.s3Key, expiresIn); + const presignedUrl = await storageService.getPresignedUrl(musicFile.s3Key, expiresIn); res.json({ presignedUrl, @@ -486,8 +489,8 @@ router.delete('/:id', async (req, res) => { return res.status(404).json({ error: 'Music file not found' }); } - // Delete from S3 - await s3Service.deleteFile(musicFile.s3Key); + // Delete from storage + await storageService.deleteFile(musicFile.s3Key); // Delete from database await MusicFile.findByIdAndDelete(req.params.id); diff --git a/packages/backend/src/services/s3Service.ts b/packages/backend/src/services/s3Service.ts index 7ed7da8..da9375f 100644 --- a/packages/backend/src/services/s3Service.ts +++ b/packages/backend/src/services/s3Service.ts @@ -3,8 +3,10 @@ import { getSignedUrl } from '@aws-sdk/s3-request-presigner'; import { v4 as uuidv4 } from 'uuid'; import fs from 'fs/promises'; import path from 'path'; +import { StorageProvider, StorageConfig, UploadResult, FileInfo } from './storageProvider.js'; -export interface S3Config { +export interface S3Config extends StorageConfig { + provider: 's3'; endpoint: string; accessKeyId: string; secretAccessKey: string; @@ -13,21 +15,7 @@ export interface S3Config { useSSL?: boolean; } -export interface UploadResult { - key: string; - url: string; - size: number; - contentType: string; -} - -export interface S3FileInfo { - key: string; - size: number; - lastModified: Date; - contentType?: string; -} - -export class S3Service { +export class S3Service implements StorageProvider { private client: S3Client; private bucketName: string; @@ -113,8 +101,8 @@ export class S3Service { /** * Recursively list all files in the S3 bucket */ - async listAllFiles(prefix: string = ''): Promise { - const files: S3FileInfo[] = []; + async listAllFiles(prefix: string = ''): Promise { + const files: FileInfo[] = []; let continuationToken: string | undefined; do { @@ -263,4 +251,22 @@ export class S3Service { async getStreamingUrl(key: string): Promise { return `${process.env.S3_ENDPOINT}/${this.bucketName}/${key}`; } + + /** + * Test the connection to S3 + */ + async testConnection(): Promise { + try { + // Try to list objects with a limit of 1 to test the connection + const command = new ListObjectsV2Command({ + Bucket: this.bucketName, + MaxKeys: 1, + }); + await this.client.send(command); + return true; + } catch (error) { + console.error('S3 connection test failed:', error); + return false; + } + } } \ No newline at end of file diff --git a/packages/backend/src/services/storageProvider.ts b/packages/backend/src/services/storageProvider.ts new file mode 100644 index 0000000..a166497 --- /dev/null +++ b/packages/backend/src/services/storageProvider.ts @@ -0,0 +1,143 @@ +/** + * Storage Provider Interface + * + * This interface defines the contract that all storage providers (S3, WebDAV, etc.) + * must implement to ensure consistent behavior across different storage backends. + */ + +export interface StorageConfig { + provider: 's3' | 'webdav'; + [key: string]: any; // Allow additional provider-specific config +} + +export interface UploadResult { + key: string; + url: string; + size: number; + contentType: string; +} + +export interface FileInfo { + key: string; + size: number; + lastModified: Date; + contentType?: string; +} + +export interface StorageProvider { + /** + * Upload a file to storage + */ + uploadFile( + file: Buffer, + originalName: string, + contentType: string, + targetFolder?: string + ): Promise; + + /** + * List all files in storage + */ + listAllFiles(prefix?: string): Promise; + + /** + * List all folders in storage + */ + listAllFolders(prefix?: string): Promise; + + /** + * Generate a presigned/secure URL for file access + */ + getPresignedUrl(key: string, expiresIn?: number): Promise; + + /** + * Delete a file from storage + */ + deleteFile(key: string): Promise; + + /** + * Check if a file exists + */ + fileExists(key: string): Promise; + + /** + * Get file metadata + */ + getFileMetadata(key: string): Promise; + + /** + * Get file content as buffer + */ + getFileContent(key: string): Promise; + + /** + * Get streaming URL for a file + */ + getStreamingUrl(key: string): Promise; + + /** + * Test the connection to the storage provider + */ + testConnection(): Promise; +} + +/** + * Storage Provider Factory + * Creates the appropriate storage provider based on configuration + */ +export class StorageProviderFactory { + static async createProvider(config: StorageConfig): Promise { + switch (config.provider) { + case 's3': + const { S3Service } = await import('./s3Service.js'); + return new S3Service(config as any); + + case 'webdav': + const { WebDAVService } = await import('./webdavService.js'); + return new WebDAVService(config as any); + + default: + throw new Error(`Unsupported storage provider: ${config.provider}`); + } + } + + /** + * Load storage configuration from file or environment + */ + static async loadConfig(): Promise { + try { + const configPath = path.join(process.cwd(), 'storage-config.json'); + const configData = await fs.readFile(configPath, 'utf-8'); + return JSON.parse(configData); + } catch (error) { + console.warn('Failed to load storage-config.json, using environment variables as fallback'); + + // Determine provider from environment + const provider = process.env.STORAGE_PROVIDER || 's3'; + + if (provider === 'webdav') { + return { + provider: 'webdav', + url: process.env.WEBDAV_URL || '', + username: process.env.WEBDAV_USERNAME || '', + password: process.env.WEBDAV_PASSWORD || '', + basePath: process.env.WEBDAV_BASE_PATH || '/music-files', + }; + } else { + return { + provider: 's3', + endpoint: process.env.S3_ENDPOINT || 'http://localhost:9000', + accessKeyId: process.env.S3_ACCESS_KEY_ID || 'minioadmin', + secretAccessKey: process.env.S3_SECRET_ACCESS_KEY || 'minioadmin', + bucketName: process.env.S3_BUCKET_NAME || 'music-files', + region: process.env.S3_REGION || 'us-east-1', + useSSL: process.env.S3_USE_SSL !== 'false', + }; + } + } + } +} + +// Import required modules +import fs from 'fs/promises'; +import path from 'path'; diff --git a/packages/backend/src/services/webdavService.ts b/packages/backend/src/services/webdavService.ts new file mode 100644 index 0000000..67a688a --- /dev/null +++ b/packages/backend/src/services/webdavService.ts @@ -0,0 +1,275 @@ +import { createClient, WebDAVClient, FileStat } from 'webdav'; +import { v4 as uuidv4 } from 'uuid'; +import { StorageProvider, StorageConfig, UploadResult, FileInfo } from './storageProvider.js'; + +export interface WebDAVConfig extends StorageConfig { + provider: 'webdav'; + url: string; + username: string; + password: string; + basePath?: string; +} + +export class WebDAVService implements StorageProvider { + private client: WebDAVClient; + private basePath: string; + + constructor(config: WebDAVConfig) { + this.client = createClient(config.url, { + username: config.username, + password: config.password, + }); + this.basePath = config.basePath || '/music-files'; + } + + /** + * Load WebDAV configuration from file or environment + */ + static async loadConfig(): Promise { + try { + const configPath = path.join(process.cwd(), 'storage-config.json'); + const configData = await fs.readFile(configPath, 'utf-8'); + const config = JSON.parse(configData); + + if (config.provider !== 'webdav') { + throw new Error('Configuration is not for WebDAV provider'); + } + + return config; + } catch (error) { + console.warn('Failed to load storage-config.json, using environment variables as fallback'); + return { + provider: 'webdav', + url: process.env.WEBDAV_URL || '', + username: process.env.WEBDAV_USERNAME || '', + password: process.env.WEBDAV_PASSWORD || '', + basePath: process.env.WEBDAV_BASE_PATH || '/music-files', + }; + } + } + + /** + * Create WebDAVService instance with configuration from file + */ + static async createFromConfig(): Promise { + const config = await this.loadConfig(); + return new WebDAVService(config); + } + + /** + * Upload a file to WebDAV + */ + async uploadFile( + file: Buffer, + originalName: string, + contentType: string, + targetFolder?: string + ): Promise { + const fileExtension = originalName.split('.').pop(); + const cleaned = typeof targetFolder === 'string' ? targetFolder.replace(/^\/+|\/+$/g, '') : ''; + const safeFolder = cleaned; + const key = safeFolder + ? `${safeFolder}/${uuidv4()}.${fileExtension}` + : `${uuidv4()}.${fileExtension}`; + + const remotePath = `${this.basePath}/${key}`; + + // Ensure the directory exists + const dirPath = remotePath.substring(0, remotePath.lastIndexOf('/')); + await this.ensureDirectoryExists(dirPath); + + // Upload the file + await this.client.putFileContents(remotePath, file, { + overwrite: true, + headers: { + 'Content-Type': contentType, + }, + }); + + return { + key, + url: remotePath, + size: file.length, + contentType, + }; + } + + /** + * Recursively list all files in the WebDAV directory + */ + async listAllFiles(prefix: string = ''): Promise { + const files: FileInfo[] = []; + const searchPath = `${this.basePath}/${prefix}`.replace(/\/+/g, '/'); + + try { + const response = await this.client.getDirectoryContents(searchPath, { + deep: true, + }); + + // Handle both single item and array responses + const items = Array.isArray(response) ? response : [response]; + + for (const item of items) { + if (item && typeof item === 'object' && 'type' in item && item.type === 'file') { + const fileItem = item as any; + const relativePath = fileItem.filename.replace(this.basePath + '/', ''); + files.push({ + key: relativePath, + size: fileItem.size || 0, + lastModified: new Date(fileItem.lastmod || Date.now()), + contentType: this.getContentTypeFromFilename(fileItem.basename), + }); + } + } + } catch (error) { + console.error('Error listing WebDAV files:', error); + } + + return files; + } + + /** + * List all folders in the WebDAV directory + */ + async listAllFolders(prefix: string = ''): Promise { + const folders = new Set(); + const searchPath = `${this.basePath}/${prefix}`.replace(/\/+/g, '/'); + + try { + const response = await this.client.getDirectoryContents(searchPath, { + deep: true, + }); + + // Handle both single item and array responses + const items = Array.isArray(response) ? response : [response]; + + for (const item of items) { + if (item && typeof item === 'object' && 'type' in item && item.type === 'directory') { + const dirItem = item as any; + const relativePath = dirItem.filename.replace(this.basePath + '/', ''); + if (relativePath) { + folders.add(relativePath); + } + } + } + } catch (error) { + console.error('Error listing WebDAV folders:', error); + } + + return Array.from(folders).sort(); + } + + /** + * Generate a direct URL for file access (WebDAV doesn't support presigned URLs) + */ + async getPresignedUrl(key: string, expiresIn: number = 3600): Promise { + // WebDAV doesn't support presigned URLs, so we return a direct URL + // In a real implementation, you might want to implement token-based access + const baseUrl = (this.client as any).getURL?.() || (this.client as any).toString() || this.config.url; + return `${baseUrl}${this.basePath}/${key}`; + } + + /** + * Delete a file from WebDAV + */ + async deleteFile(key: string): Promise { + const remotePath = `${this.basePath}/${key}`; + await this.client.deleteFile(remotePath); + } + + /** + * Check if a file exists + */ + async fileExists(key: string): Promise { + try { + const remotePath = `${this.basePath}/${key}`; + const response = await this.client.stat(remotePath); + const stat = Array.isArray(response) ? response[0] : response; + return stat && typeof stat === 'object' && 'type' in stat && stat.type === 'file'; + } catch (error) { + return false; + } + } + + /** + * Get file metadata + */ + async getFileMetadata(key: string): Promise { + const remotePath = `${this.basePath}/${key}`; + const response = await this.client.stat(remotePath); + return Array.isArray(response) ? response[0] : response; + } + + /** + * Get file content as buffer + */ + async getFileContent(key: string): Promise { + const remotePath = `${this.basePath}/${key}`; + const arrayBuffer = await this.client.getFileContents(remotePath, { + format: 'binary', + }); + return Buffer.from(arrayBuffer as ArrayBuffer); + } + + /** + * Get streaming URL for a file + */ + async getStreamingUrl(key: string): Promise { + const baseUrl = (this.client as any).getURL?.() || (this.client as any).toString() || this.config.url; + return `${baseUrl}${this.basePath}/${key}`; + } + + /** + * Test the connection to WebDAV + */ + async testConnection(): Promise { + try { + // Try to list the base directory to test the connection + await this.client.getDirectoryContents(this.basePath); + return true; + } catch (error) { + console.error('WebDAV connection test failed:', error); + return false; + } + } + + /** + * Ensure a directory exists, creating it if necessary + */ + private async ensureDirectoryExists(dirPath: string): Promise { + try { + // Check if directory exists + await this.client.stat(dirPath); + } catch (error) { + // Directory doesn't exist, create it + try { + await this.client.createDirectory(dirPath, { recursive: true }); + } catch (createError) { + console.error('Failed to create directory:', dirPath, createError); + throw createError; + } + } + } + + /** + * Get content type from filename + */ + private getContentTypeFromFilename(filename: string): string { + const ext = filename.split('.').pop()?.toLowerCase(); + switch (ext) { + case 'mp3': return 'audio/mpeg'; + case 'wav': return 'audio/wav'; + case 'flac': return 'audio/flac'; + case 'm4a': return 'audio/mp4'; + case 'aac': return 'audio/aac'; + case 'ogg': return 'audio/ogg'; + case 'opus': return 'audio/opus'; + case 'wma': return 'audio/x-ms-wma'; + default: return 'application/octet-stream'; + } + } +} + +// Import required modules +import fs from 'fs/promises'; +import path from 'path'; diff --git a/packages/backend/storage-config.json b/packages/backend/storage-config.json new file mode 100644 index 0000000..ad7cc2c --- /dev/null +++ b/packages/backend/storage-config.json @@ -0,0 +1,7 @@ +{ + "provider": "webdav", + "url": "https://cloud.geertrademakers.nl/remote.php/dav/files/admin", + "username": "admin", + "password": "XPZK2-MGQ5W-7Yetf-nr8gf-s5g5Z", + "basePath": "/Test" +} \ No newline at end of file diff --git a/packages/backend/test-webdav.js b/packages/backend/test-webdav.js new file mode 100644 index 0000000..9067e68 --- /dev/null +++ b/packages/backend/test-webdav.js @@ -0,0 +1,61 @@ +#!/usr/bin/env node + +/** + * WebDAV Integration Test + * + * This script tests the WebDAV service integration with a sample Nextcloud instance. + * Update the configuration below to match your Nextcloud setup. + */ + +import { WebDAVService } from './dist/services/webdavService.js'; + +// Test configuration - update these values for your Nextcloud instance +const testConfig = { + provider: 'webdav', + url: 'https://your-nextcloud.com/remote.php/dav/files/username/', + username: 'your-username', + password: 'your-password-or-app-password', + basePath: '/music-files' +}; + +async function testWebDAVConnection() { + console.log('๐Ÿงช Testing WebDAV connection...'); + console.log('๐Ÿ“‹ Configuration:', { + ...testConfig, + password: '***' // Hide password in logs + }); + + try { + // Create WebDAV service + const webdavService = new WebDAVService(testConfig); + + // Test connection + console.log('๐Ÿ”— Testing connection...'); + const isConnected = await webdavService.testConnection(); + + if (isConnected) { + console.log('โœ… WebDAV connection successful!'); + + // Test listing files + console.log('๐Ÿ“ Testing file listing...'); + const files = await webdavService.listAllFiles(); + console.log(`๐Ÿ“Š Found ${files.length} files`); + + // Test listing folders + console.log('๐Ÿ“‚ Testing folder listing...'); + const folders = await webdavService.listAllFolders(); + console.log(`๐Ÿ“Š Found ${folders.length} folders:`, folders); + + console.log('๐ŸŽ‰ All WebDAV tests passed!'); + } else { + console.log('โŒ WebDAV connection failed'); + } + + } catch (error) { + console.error('โŒ WebDAV test failed:', error.message); + console.error('๐Ÿ’ก Make sure to update the configuration in this file with your Nextcloud details'); + } +} + +// Run the test +testWebDAVConnection().catch(console.error); diff --git a/packages/frontend/src/pages/Configuration.tsx b/packages/frontend/src/pages/Configuration.tsx index c878e6b..86c118a 100644 --- a/packages/frontend/src/pages/Configuration.tsx +++ b/packages/frontend/src/pages/Configuration.tsx @@ -29,7 +29,7 @@ import { FiDatabase, FiSettings, FiUpload, FiLink, FiRefreshCw, FiTrash2 } from import { useNavigate } from "react-router-dom"; import { useXmlParser } from "../hooks/useXmlParser"; import { StyledFileInput } from "../components/StyledFileInput"; -import { S3Configuration } from "./S3Configuration"; +import { StorageConfiguration } from "./StorageConfiguration"; import { MusicUpload } from "../components/MusicUpload"; import { SongMatching } from "../components/SongMatching"; import { api } from "../services/api"; @@ -179,7 +179,7 @@ export function Configuration() { - S3 Configuration + Storage Configuration @@ -236,8 +236,8 @@ export function Configuration() { Upload Music Files - Drag and drop your music files here or click to select. Files will be uploaded to S3 storage - and metadata will be automatically extracted. + Drag and drop your music files here or click to select. Files will be uploaded to your configured storage + (S3 or WebDAV) and metadata will be automatically extracted. @@ -285,10 +285,10 @@ export function Configuration() { - {/* S3 Configuration Tab */} + {/* Storage Configuration Tab */} - + diff --git a/packages/frontend/src/pages/StorageConfiguration.tsx b/packages/frontend/src/pages/StorageConfiguration.tsx new file mode 100644 index 0000000..a1c2e72 --- /dev/null +++ b/packages/frontend/src/pages/StorageConfiguration.tsx @@ -0,0 +1,702 @@ +import React, { useState, useEffect } from 'react'; +import { + Box, + VStack, + HStack, + Text, + Heading, + FormControl, + FormLabel, + Input, + Button, + useToast, + Alert, + AlertIcon, + AlertTitle, + AlertDescription, + Card, + CardBody, + CardHeader, + Spinner, + Badge, + Icon, + Switch, + FormHelperText, + Select, + Divider, +} from '@chakra-ui/react'; +import { FiSettings, FiZap, FiSave, FiCloud, FiServer } from 'react-icons/fi'; + +interface S3Config { + provider: 's3'; + endpoint: string; + region: string; + accessKeyId: string; + secretAccessKey: string; + bucketName: string; + useSSL: boolean; +} + +interface WebDAVConfig { + provider: 'webdav'; + url: string; + username: string; + password: string; + basePath?: string; +} + +type StorageConfig = S3Config | WebDAVConfig; + +interface TestResult { + success: boolean; + message: string; + details?: any; + provider?: string; +} + +export const StorageConfiguration: React.FC = () => { + const [config, setConfig] = useState({ + provider: 's3', + endpoint: '', + region: 'us-east-1', + accessKeyId: '', + secretAccessKey: '', + bucketName: '', + useSSL: true, + }); + + const [isLoading, setIsLoading] = useState(false); + const [isTesting, setIsTesting] = useState(false); + const [isSaving, setIsSaving] = useState(false); + const [testResult, setTestResult] = useState(null); + const [currentConfig, setCurrentConfig] = useState(null); + const toast = useToast(); + + // Load current configuration on component mount + useEffect(() => { + loadCurrentConfig(); + }, []); + + const loadCurrentConfig = async () => { + setIsLoading(true); + try { + const response = await fetch('/api/config/storage'); + if (response.ok) { + const data = await response.json(); + setCurrentConfig(data.config); + + // Handle masked passwords - don't set masked values as initial state + const configWithEmptyPasswords = { ...data.config }; + if (configWithEmptyPasswords.provider === 'webdav' && configWithEmptyPasswords.password === '***') { + configWithEmptyPasswords.password = ''; + } + if (configWithEmptyPasswords.provider === 's3') { + if (configWithEmptyPasswords.accessKeyId === '***') { + configWithEmptyPasswords.accessKeyId = ''; + } + if (configWithEmptyPasswords.secretAccessKey === '***') { + configWithEmptyPasswords.secretAccessKey = ''; + } + } + + setConfig(configWithEmptyPasswords); + } + } catch (error) { + console.error('Error loading storage config:', error); + } finally { + setIsLoading(false); + } + }; + + const handleProviderChange = (provider: 's3' | 'webdav') => { + if (provider === 's3') { + setConfig({ + provider: 's3', + endpoint: '', + region: 'us-east-1', + accessKeyId: '', + secretAccessKey: '', + bucketName: '', + useSSL: true, + }); + } else { + setConfig({ + provider: 'webdav', + url: '', + username: '', + password: '', + basePath: '/music-files', + }); + } + }; + + const handleInputChange = (field: string, value: string | boolean) => { + setConfig(prev => ({ + ...prev, + [field]: value, + })); + }; + + const testConnection = async () => { + setIsTesting(true); + setTestResult(null); + + try { + // For testing, merge current form state with existing config to preserve passwords + const testConfig = { ...currentConfig, ...config }; + + const response = await fetch('/api/config/storage/test', { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + }, + body: JSON.stringify(testConfig), + }); + + const result = await response.json(); + + if (response.ok) { + setTestResult({ + success: true, + message: 'Connection successful!', + details: result, + provider: testConfig.provider, + }); + toast({ + title: 'Connection Test Successful', + description: `${testConfig.provider.toUpperCase()} connection is working properly`, + status: 'success', + duration: 5000, + isClosable: true, + }); + } else { + setTestResult({ + success: false, + message: result.error || 'Connection failed', + details: result, + provider: testConfig.provider, + }); + toast({ + title: 'Connection Test Failed', + description: result.error || `Failed to connect to ${testConfig.provider.toUpperCase()}`, + status: 'error', + duration: 5000, + isClosable: true, + }); + } + } catch (error) { + console.error('Error testing storage connection:', error); + setTestResult({ + success: false, + message: 'Network error or server unavailable', + provider: testConfig.provider, + }); + toast({ + title: 'Connection Test Failed', + description: 'Network error or server unavailable', + status: 'error', + duration: 5000, + isClosable: true, + }); + } finally { + setIsTesting(false); + } + }; + + const saveConfiguration = async () => { + setIsSaving(true); + + try { + // Always send the complete configuration + const configToSave = { ...config }; + + const response = await fetch('/api/config/storage', { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + }, + body: JSON.stringify(configToSave), + }); + + if (response.ok) { + setCurrentConfig(config); + toast({ + title: 'Configuration Saved', + description: `${config.provider.toUpperCase()} configuration has been saved successfully`, + status: 'success', + duration: 3000, + isClosable: true, + }); + } else { + const error = await response.json(); + throw new Error(error.error || 'Failed to save configuration'); + } + } catch (error) { + console.error('Error saving storage config:', error); + toast({ + title: 'Save Failed', + description: error instanceof Error ? error.message : 'Failed to save configuration', + status: 'error', + duration: 5000, + isClosable: true, + }); + } finally { + setIsSaving(false); + } + }; + + const hasChanges = () => { + if (!currentConfig) return true; + + // Create a copy of currentConfig with masked values replaced with empty strings for comparison + const normalizedCurrentConfig = { ...currentConfig }; + if (normalizedCurrentConfig.provider === 'webdav' && normalizedCurrentConfig.password === '***') { + normalizedCurrentConfig.password = ''; + } + if (normalizedCurrentConfig.provider === 's3') { + if (normalizedCurrentConfig.accessKeyId === '***') { + normalizedCurrentConfig.accessKeyId = ''; + } + if (normalizedCurrentConfig.secretAccessKey === '***') { + normalizedCurrentConfig.secretAccessKey = ''; + } + } + + return JSON.stringify(config) !== JSON.stringify(normalizedCurrentConfig); + }; + + const renderS3Config = () => ( + + {/* Endpoint */} + + S3 Endpoint + handleInputChange('endpoint', e.target.value)} + placeholder="https://s3.amazonaws.com or http://localhost:9000 for MinIO" + bg="gray.700" + borderColor="gray.600" + color="white" + _placeholder={{ color: 'gray.400' }} + _focus={{ borderColor: 'blue.400', boxShadow: 'none' }} + /> + + For AWS S3, use: https://s3.amazonaws.com. For MinIO: http://localhost:9000 + + + + {/* Region */} + + Region + handleInputChange('region', e.target.value)} + placeholder="us-east-1" + bg="gray.700" + borderColor="gray.600" + color="white" + _placeholder={{ color: 'gray.400' }} + _focus={{ borderColor: 'blue.400', boxShadow: 'none' }} + /> + + AWS region (e.g., us-east-1, eu-west-1) or 'us-east-1' for MinIO + + + + {/* Access Key */} + + Access Key ID + handleInputChange('accessKeyId', e.target.value)} + placeholder="Your S3 access key" + bg="gray.700" + borderColor="gray.600" + color="white" + _placeholder={{ color: 'gray.400' }} + _focus={{ borderColor: 'blue.400', boxShadow: 'none' }} + /> + + + {/* Secret Key */} + + Secret Access Key + handleInputChange('secretAccessKey', e.target.value)} + placeholder="Your S3 secret key" + bg="gray.700" + borderColor="gray.600" + color="white" + _placeholder={{ color: 'gray.400' }} + _focus={{ borderColor: 'blue.400', boxShadow: 'none' }} + /> + + + {/* Bucket Name */} + + Bucket Name + handleInputChange('bucketName', e.target.value)} + placeholder="music-files" + bg="gray.700" + borderColor="gray.600" + color="white" + _placeholder={{ color: 'gray.400' }} + _focus={{ borderColor: 'blue.400', boxShadow: 'none' }} + /> + + The S3 bucket where music files will be stored + + + + {/* Use SSL */} + + + Use SSL/TLS + + handleInputChange('useSSL', e.target.checked)} + colorScheme="blue" + /> + + Enable for HTTPS connections (recommended for production) + + + + ); + + const renderWebDAVConfig = () => ( + + {/* URL */} + + WebDAV URL + handleInputChange('url', e.target.value)} + placeholder="https://your-nextcloud.com/remote.php/dav/files/username/" + bg="gray.700" + borderColor="gray.600" + color="white" + _placeholder={{ color: 'gray.400' }} + _focus={{ borderColor: 'blue.400', boxShadow: 'none' }} + /> + + Your Nextcloud WebDAV URL (usually ends with /remote.php/dav/files/username/) + + + + {/* Username */} + + Username + handleInputChange('username', e.target.value)} + placeholder="Your Nextcloud username" + bg="gray.700" + borderColor="gray.600" + color="white" + _placeholder={{ color: 'gray.400' }} + _focus={{ borderColor: 'blue.400', boxShadow: 'none' }} + /> + + + {/* Password */} + + Password + handleInputChange('password', e.target.value)} + placeholder="Your Nextcloud password or app password" + bg="gray.700" + borderColor="gray.600" + color="white" + _placeholder={{ color: 'gray.400' }} + _focus={{ borderColor: 'blue.400', boxShadow: 'none' }} + /> + + Use your Nextcloud password or create an app password for better security + + + + {/* Base Path */} + + Base Path (Optional) + handleInputChange('basePath', e.target.value)} + placeholder="/music-files" + bg="gray.700" + borderColor="gray.600" + color="white" + _placeholder={{ color: 'gray.400' }} + _focus={{ borderColor: 'blue.400', boxShadow: 'none' }} + /> + + Subfolder within your WebDAV storage where music files will be stored + + + + ); + + if (isLoading) { + return ( + + + + Loading storage configuration... + + + ); + } + + return ( + + + {/* Header */} + + + + Storage Configuration + + + Configure your storage provider for music file storage and playback. Choose between S3-compatible storage or WebDAV (Nextcloud). + + + + {/* Provider Selection */} + + + Storage Provider + + + + + Select Storage Provider + + + + + + + {config.provider === 's3' + ? 'S3-compatible storage for scalable cloud storage' + : 'WebDAV for self-hosted solutions like Nextcloud' + } + + + + + + + {/* Configuration Form */} + + + + {config.provider === 's3' ? 'S3 Configuration' : 'WebDAV Configuration'} + + + + {config.provider === 's3' ? renderS3Config() : renderWebDAVConfig()} + + + + {/* Test Connection */} + + + + + Test Connection + + + + + + Test your {config.provider.toUpperCase()} configuration to ensure it's working properly before saving. + + + + + {testResult && ( + + + + {testResult.success ? 'Connection Successful' : 'Connection Failed'} + + + {testResult.message} + + {testResult.details && ( + + Details: + + {JSON.stringify(testResult.details, null, 2)} + + + )} + + )} + + + + + {/* Save Configuration */} + + + + + Save Configuration + + + + + + Save your {config.provider.toUpperCase()} configuration to use it for music file storage and playback. + + + + + + {currentConfig && ( + + Configuration Loaded + + )} + + + {!hasChanges() && currentConfig && ( + + + No changes to save + + )} + + + + + {/* Help Section */} + + + Configuration Help + + + + {/* S3 Help */} + + + + S3-Compatible Storage + + + + For AWS S3: + + + โ€ข Endpoint: https://s3.amazonaws.com
+ โ€ข Region: Your AWS region (e.g., us-east-1)
+ โ€ข Access Key: Your AWS access key
+ โ€ข Secret Key: Your AWS secret key
+ โ€ข Bucket: Your S3 bucket name +
+ + + For MinIO (Local Development): + + + โ€ข Endpoint: http://localhost:9000
+ โ€ข Region: us-east-1
+ โ€ข Access Key: minioadmin
+ โ€ข Secret Key: minioadmin
+ โ€ข Bucket: Create a bucket named 'music-files' +
+
+
+ + + + {/* WebDAV Help */} + + + + WebDAV (Nextcloud/ownCloud) + + + + For Nextcloud: + + + โ€ข URL: https://your-nextcloud.com/remote.php/dav/files/username/
+ โ€ข Username: Your Nextcloud username
+ โ€ข Password: Your Nextcloud password or app password
+ โ€ข Base Path: /music-files (optional subfolder) +
+ + + For ownCloud: + + + โ€ข URL: https://your-owncloud.com/remote.php/dav/files/username/
+ โ€ข Username: Your ownCloud username
+ โ€ข Password: Your ownCloud password or app password
+ โ€ข Base Path: /music-files (optional subfolder) +
+ + + Note: For better security, create an app password in your Nextcloud/ownCloud settings instead of using your main password. + +
+
+
+
+
+
+
+ ); +};