From 791c2235c5dcb9c6310461479d8286b7cd8bfe08 Mon Sep 17 00:00:00 2001 From: tcomlab Date: Wed, 15 Apr 2026 14:40:24 +0300 Subject: [PATCH] Expand deployment and sync documentation --- .env.example | 4 +- README.md | 391 ++++++++++++++++++++++++++++++++++++++------------- 2 files changed, 299 insertions(+), 96 deletions(-) diff --git a/.env.example b/.env.example index c2bd038..d39c2a7 100644 --- a/.env.example +++ b/.env.example @@ -29,9 +29,9 @@ OVERWRITE_ARCHIVE_EXTERNAL_LINK=false DEFAULT_STOCK_QUANTITY=1 INVENTREE_STOCK_STATUS=10 PART_IPN_PREFIX=BMB -PART_KEY_FIELDS=filename,name +PART_KEY_FIELDS=content_hash,filename,print_name,name BACKFILL_PAGE_SIZE=50 -POLL_INTERVAL_SECONDS=0 +POLL_INTERVAL_SECONDS=300 SYNC_ON_STARTUP=false HTTP_TIMEOUT_SECONDS=30 diff --git a/README.md b/README.md index 1136b1a..12169b1 100644 --- a/README.md +++ b/README.md @@ -1,147 +1,350 @@ # Bambuddy InvenTree Sync -Small sidecar service for syncing Bambuddy `Archives` into InvenTree. +Sidecar service for syncing successful Bambuddy print archives into InvenTree stock. -It currently: +The service runs as a separate Docker container. It does not modify Bambuddy or InvenTree source code. Bambuddy is used as the print-history source, while InvenTree remains the stock-control system of record. -- creates an InvenTree `Part` automatically when a printed model is first seen; -- creates one InvenTree `StockItem` for each synced Bambuddy archive; -- stores local sync state in SQLite to avoid duplicate stock items; -- accepts Bambuddy webhooks and can also backfill existing archives. +## What It Does -The first version intentionally treats one Bambuddy archive as one printed stock item. Later we can add plate parsing, multi-object quantity detection, filament costing, thumbnails, 3MF attachments, and mapping rules. +For each successful Bambuddy archive, the service: -## Setup +- finds or creates an InvenTree `Part`; +- creates one InvenTree `StockItem` for the Bambuddy archive; +- writes `Weight` and `PrintTime` part parameters when those templates exist in InvenTree; +- uploads the Bambuddy archive thumbnail to the InvenTree part image; +- writes an InvenTree part URL back into Bambuddy archive `external_url`; +- stores sync state in SQLite to prevent duplicate stock items. -1. Copy `.env.example` to `.env`. -2. Fill in: - - `BAMBUDDY_BASE_URL` - - `BAMBUDDY_API_KEY` - - `INVENTREE_BASE_URL` - - `INVENTREE_TOKEN` - - `INVENTREE_PART_CATEGORY_ID` - - `INVENTREE_STOCK_LOCATION_ID` -3. Start the service: +Failed, stopped, running, or still-printing archives are skipped when `SYNC_SUCCESS_ONLY=true`. -```powershell -docker compose up -d --build -``` - -The service listens on `http://localhost:8088`. - -## InvenTree IDs - -For the first version, use numeric IDs for the target InvenTree part category and stock location. Open the desired category/location in InvenTree and copy the ID from the URL or API response. - -Example: - -```env -INVENTREE_PART_CATEGORY_ID=12 -INVENTREE_STOCK_LOCATION_ID=7 -``` - -## Validate Connectivity - -If `SERVICE_API_TOKEN` is set in `.env`, pass it as `X-Service-Token`: - -```powershell -curl.exe -H "X-Service-Token: change-me" http://localhost:8088/validate -``` - -## Backfill Existing Archives - -Run a test with one archive first: - -```powershell -curl.exe -X POST -H "X-Service-Token: change-me" "http://localhost:8088/sync/backfill?max_archives=1" -``` - -When `SYNC_SUCCESS_ONLY=true`, `max_archives` counts import attempts. Archives that are still `printing` or already failed are skipped and do not consume the limit. - -Run full backfill for successful Bambuddy archives: - -```powershell -curl.exe -X POST -H "X-Service-Token: change-me" http://localhost:8088/sync/backfill -``` - -The default behavior is `SYNC_SUCCESS_ONLY=true`, so failed or stopped prints are not imported. - -## Bambuddy Webhook - -In Bambuddy, configure a webhook to: +## Data Flow ```text -http://WINDOWS-SERVER-IP:8088/webhooks/bambuddy +Bambuddy Archives + | + | webhook or polling + v +bambuddy-inventree-sync + | + | InvenTree API + v +Part + StockItem + Parameters + Image + | + | Bambuddy API + v +Archive external_url -> InvenTree Part page ``` -If `WEBHOOK_SHARED_SECRET` is configured, Bambuddy must send this header: +## InvenTree Mapping -```text -X-Sync-Secret: your-secret -``` +| Bambuddy data | InvenTree target | +| --- | --- | +| archive/model identity | `Part` | +| successful print archive | `StockItem` | +| target category | `Part.category` | +| target storage | `StockItem.location` | +| archive id | `StockItem.batch = bambuddy-` | +| filament weight | `Part` parameter `Weight` | +| print time | `Part` parameter `PrintTime` | +| archive thumbnail | `Part.image` | +| InvenTree part page | Bambuddy `external_url` | -The service expects Bambuddy payloads with `event=print_complete` and `data.archive_id`. +## Duplicate Protection -## Part Matching +The service is idempotent: -The service builds a stable key from: +- each Bambuddy archive ID is stored in `data/sync.sqlite3`; +- each `StockItem` gets batch `bambuddy-`; +- rerunning backfill does not create duplicate stock items; +- repeat prints of the same file/model reuse the same `Part` and create new `StockItem` rows. + +The `Part` identity key is controlled by: ```env -PART_KEY_FIELDS=filename,name +PART_KEY_FIELDS=content_hash,filename,print_name,name ``` -That key becomes an InvenTree IPN: +The generated key becomes an InvenTree IPN: ```text BMB- ``` -This means repeat prints of the same file/name reuse the same `Part` and create new `StockItem` rows. To change matching behavior later, edit `PART_KEY_FIELDS`. +## Delete Policy -## InvenTree Part Parameters +If an archive is deleted in Bambuddy after it was synced, the corresponding InvenTree data is not deleted. -For each synced `Part`, the service updates these InvenTree parameters when matching parameter templates exist: +This is intentional. InvenTree is the inventory record, and deleting print history in Bambuddy should not silently remove stock records. The synced `Part`, `StockItem`, parameters, and image remain in InvenTree. -- `Weight`: filament weight per printed item in grams. If a Bambuddy archive has `quantity=2`, the total filament weight is divided by 2. -- `PrintTime`: print duration from Bambuddy. The visible value is formatted as `1h 6m 1s`; `data_numeric` stores the duration in seconds. +## Requirements -## InvenTree Part Images +- Docker with Linux containers enabled. +- Network access from the sync container to Bambuddy API and InvenTree API. +- Bambuddy API key. +- InvenTree API token. +- Existing InvenTree part category ID. +- Existing InvenTree stock location ID. +- Optional InvenTree parameter templates: `Weight` and `PrintTime`. -When `SYNC_PART_IMAGES=true`, the service downloads the Bambuddy archive thumbnail and uploads it to the InvenTree `Part.image` field. +On Windows Server 2022, verify Docker with: -By default, existing InvenTree part images are preserved: - -```env -OVERWRITE_PART_IMAGES=false +```powershell +docker run --rm hello-world +docker run --rm python:3.12-slim python --version ``` -Set `OVERWRITE_PART_IMAGES=true` if Bambuddy thumbnails should replace existing part images. +## Docker Deployment -## Bambuddy External Links +Clone the repository: -When `SYNC_ARCHIVE_EXTERNAL_LINK=true`, the service writes the InvenTree part page URL into Bambuddy archive `external_url`. +```powershell +git clone https://git.tcom.space/tcom/Lab8DATAPROCESSOR.git +cd Lab8DATAPROCESSOR +``` -The default link format is: +Create the runtime config: + +```powershell +Copy-Item .env.example .env +notepad .env +``` + +Start the service: + +```powershell +docker compose up -d --build +``` + +View logs: + +```powershell +docker compose logs --tail=100 +``` + +Stop the service: + +```powershell +docker compose down +``` + +Update to the latest version: + +```powershell +git pull +docker compose up -d --build +``` + +## Configuration + +Example `.env`: + +```env +BAMBUDDY_BASE_URL=http://192.168.1.5:8000/api/v1 +BAMBUDDY_API_KEY=replace-with-bambuddy-api-key + +INVENTREE_BASE_URL=http://192.168.0.3:1337 +INVENTREE_WEB_URL= +INVENTREE_TOKEN=replace-with-inventree-token +INVENTREE_PART_CATEGORY_ID=26 +INVENTREE_STOCK_LOCATION_ID=98 + +SERVICE_API_TOKEN=change-me +WEBHOOK_SHARED_SECRET= + +SYNC_SUCCESS_ONLY=true +SYNC_PART_IMAGES=true +OVERWRITE_PART_IMAGES=false +SYNC_ARCHIVE_EXTERNAL_LINK=true +OVERWRITE_ARCHIVE_EXTERNAL_LINK=false + +DEFAULT_STOCK_QUANTITY=1 +INVENTREE_STOCK_STATUS=10 +PART_IPN_PREFIX=BMB +PART_KEY_FIELDS=content_hash,filename,print_name,name + +BACKFILL_PAGE_SIZE=50 +POLL_INTERVAL_SECONDS=300 +SYNC_ON_STARTUP=false +HTTP_TIMEOUT_SECONDS=30 +DATA_DIR=/data +``` + +Do not commit `.env`. It contains API tokens and is ignored by git. + +## Important Settings + +`BAMBUDDY_BASE_URL` +: Bambuddy API base URL. Use `/api/v1`. + +`INVENTREE_BASE_URL` +: InvenTree root URL or `/api` URL. The service normalizes it internally. + +`INVENTREE_WEB_URL` +: Browser-facing InvenTree URL for Bambuddy external links. If empty, `INVENTREE_BASE_URL` is used. + +`INVENTREE_PART_CATEGORY_ID` +: Existing InvenTree category where auto-created printed parts are placed. + +`INVENTREE_STOCK_LOCATION_ID` +: Existing InvenTree stock location where printed stock items are placed. + +`SYNC_SUCCESS_ONLY` +: When `true`, only successful/completed prints are imported. + +`POLL_INTERVAL_SECONDS` +: Enables automatic periodic backfill. `300` means every 5 minutes. `0` disables polling. + +`SYNC_PART_IMAGES` +: Downloads Bambuddy thumbnail and uploads it to InvenTree `Part.image`. + +`OVERWRITE_PART_IMAGES` +: When `false`, manually set InvenTree part images are preserved. + +`SYNC_ARCHIVE_EXTERNAL_LINK` +: Writes the InvenTree part page URL into Bambuddy `external_url`. + +`OVERWRITE_ARCHIVE_EXTERNAL_LINK` +: When `false`, existing non-InvenTree external links in Bambuddy are preserved. + +## InvenTree IDs + +Use numeric IDs for target category and stock location. You can find them from the InvenTree UI URL or API. + +Example: + +```powershell +curl.exe -H "Authorization: Token YOUR_TOKEN" http://192.168.0.3:1337/api/part/category/26/ +curl.exe -H "Authorization: Token YOUR_TOKEN" http://192.168.0.3:1337/api/stock/location/98/ +``` + +Current known example: + +```env +INVENTREE_PART_CATEGORY_ID=26 +INVENTREE_STOCK_LOCATION_ID=98 +``` + +## Validate Connectivity + +After the container starts: + +```powershell +curl.exe http://localhost:8088/health +curl.exe -H "X-Service-Token: change-me" http://localhost:8088/validate +``` + +Expected result: + +- `/health` returns `status: ok`; +- `/validate` confirms Bambuddy, InvenTree category, and InvenTree location are reachable. + +## Initial Backfill + +Test one successful archive first: + +```powershell +curl.exe -X POST -H "X-Service-Token: change-me" "http://localhost:8088/sync/backfill?max_archives=1" +``` + +Run full backfill: + +```powershell +curl.exe -X POST -H "X-Service-Token: change-me" http://localhost:8088/sync/backfill +``` + +Check status: + +```powershell +curl.exe -H "X-Service-Token: change-me" http://localhost:8088/sync/status +``` + +When `SYNC_SUCCESS_ONLY=true`, still-printing or failed archives are skipped and do not consume `max_archives`. + +## Automatic Operation + +Recommended production mode: + +```env +SYNC_SUCCESS_ONLY=true +POLL_INTERVAL_SECONDS=300 +SYNC_ON_STARTUP=false +``` + +This makes the service check Bambuddy every 5 minutes and import all newly completed prints. + +For near-real-time syncing, also configure a Bambuddy webhook: ```text -/web/part// +http://WINDOWS-SERVER-IP:8088/webhooks/bambuddy ``` -Existing non-InvenTree external links are preserved unless: +Use the `print_complete` event. If `WEBHOOK_SHARED_SECRET` is set, Bambuddy must send: -```env -OVERWRITE_ARCHIVE_EXTERNAL_LINK=true +```text +X-Sync-Secret: your-secret ``` -## Useful Endpoints +Running both webhook and polling is recommended. Webhook gives fast sync, polling catches missed events after restarts or temporary failures. + +## Manual Sync + +Sync one archive: + +```powershell +curl.exe -X POST -H "X-Service-Token: change-me" http://localhost:8088/sync/archive/11 +``` + +Force re-sync one archive: + +```powershell +curl.exe -X POST -H "X-Service-Token: change-me" "http://localhost:8088/sync/archive/11?force=true" +``` + +Force sync is useful after changing image, parameter, or external link behavior. + +## API Endpoints ```text GET /health -GET /sync/status GET /validate +GET /sync/status POST /sync/archive/{archive_id} POST /sync/backfill POST /webhooks/bambuddy ``` Manual sync endpoints require `X-Service-Token` when `SERVICE_API_TOKEN` is set. + +## Troubleshooting + +View recent logs: + +```powershell +docker compose logs --tail=100 +``` + +Rebuild after pulling code changes: + +```powershell +git pull +docker compose up -d --build +``` + +Common issues: + +- `401` from service endpoints: missing or wrong `X-Service-Token`. +- `401` from Bambuddy: wrong `BAMBUDDY_API_KEY`. +- `401` from InvenTree: wrong `INVENTREE_TOKEN`. +- InvenTree `description` length errors: update to the latest service version; descriptions are capped at 250 characters. +- No items imported: check if Bambuddy archives are still `printing` or `failed`. +- Duplicate protection prevents repeated imports: use `force=true` only when you want to refresh parameters/images/links for an existing archive. + +## Backups + +Persistent sync state is stored in: + +```text +./data/sync.sqlite3 +``` + +Back up the `data` directory together with `.env` if you move the service to another server.