# NoteSight - Full LLM & AI Agent Technical Reference > Complete machine-readable technical guide, OpenAPI schemas, integration flows, and code examples for the NoteSight Optical Music Recognition (OMR) system. ## 1. Overview & System Design NoteSight is a distributed, production-grade Optical Music Recognition and interactive sheet music platform. ### Core Processing Flow 1. **Input Ingestion**: Client submits a multi-page PDF or image file (PNG/JPG/TIFF). 2. **Page Extraction & Preprocessing**: `PyMuPDF` converts PDF pages to 300 DPI standardized RGB raster buffers, deskews rotation, and normalizes dimensions without external C system dependencies. 3. **Neural Transcription**: The `homr` neural network (accelerated via ONNX Runtime) processes individual staves, outputting semantic notation tokens (clefs, keys, time signatures, notes, rests, beams, barlines). 4. **Multi-Page Continuous Stitching**: The engine merges discrete measures sequentially across pages, normalizes divisions, renumbers measures, and formats standard MusicXML 3.1 Partwise XML. 5. **Interactive Client**: OpenSheetMusicDisplay (OSMD) renders responsive SVG engraving; Web Audio API executes polyphonic synthesis with synchronized visual playheads. 6. **Vector Rendering**: Verovio compiles the MusicXML into multi-page print-ready vector PDF files on demand. --- ## 2. Authentication & Authorization NoteSight supports both session-based authentication and token-based API access: - **Type**: Bearer JWT (JSON Web Token) - **Header**: `Authorization: Bearer ` - **Token Endpoint**: `POST /api/auth/login` (`username` and `password` as application/x-www-form-urlencoded) - **Registration**: `POST /api/auth/register` (`email`, `password`, `full_name`) - **Public Endpoints (No Auth Required)**: - `GET /api/scores/share/{token}`: Retrieve publicly shared score - `GET /api/health/live`: Public liveness probe - `GET /api/openapi.json`: OpenAPI 3.0 specification - `GET /api/docs`: Swagger documentation UI - `GET /llms.txt`: Standard LLM discovery document --- ## 3. API Reference & Code Examples ### 3.1 Upload Score Document Uploads a PDF or image file to initialize an asynchronous transcription job. - **Method**: `POST` - **Path**: `/api/jobs/upload` - **Headers**: - `Authorization: Bearer ` (optional if demo/guest upload is permitted) - `Content-Type: multipart/form-data` - **Form Parameters**: - `file`: Binary file content (PDF, PNG, JPG, TIFF) - **Response**: `200 OK` ```json { "id": "a1b2c3d4-e5f6-7a8b-9c0d-1e2f3a4b5c6d", "status": "PENDING", "original_filename": "beethoven_sonata.pdf", "page_count": null, "created_at": "2026-10-17T18:00:00Z" } ``` ### 3.2 Poll Job Status Monitors progress until transcription completes. - **Method**: `GET` - **Path**: `/api/jobs/{job_id}` - **Response**: `200 OK` ```json { "id": "a1b2c3d4-e5f6-7a8b-9c0d-1e2f3a4b5c6d", "status": "COMPLETED", "original_filename": "beethoven_sonata.pdf", "page_count": 3, "error_message": null, "created_at": "2026-10-17T18:00:00Z", "completed_at": "2026-10-17T18:00:15Z" } ``` - **Job Status Values**: - `PENDING`: Uploaded, queued in Celery/Redis. - `PROCESSING`: Preprocessing or neural OMR actively running. - `COMPLETED`: MusicXML transcription successfully generated. - `FAILED`: Error occurred (details in `error_message`). ### 3.3 Export MusicXML Retrieves the parsed notation as standard MusicXML. - **Method**: `GET` - **Path**: `/api/scores/export/{job_id}/musicxml` - **Response**: `200 OK` - **Content-Type**: `application/vnd.recordare.musicxml+xml` (or `application/xml`) - **Body**: UTF-8 XML document. ### 3.4 Export Standard MIDI Synthesizes the score into a playable Standard MIDI file. - **Method**: `GET` - **Path**: `/api/scores/export/{job_id}/midi` - **Response**: `200 OK` - **Content-Type**: `audio/midi` ### 3.5 Export Vector PDF (Verovio) Compiles the notation into a high-resolution vector PDF. - **Method**: `GET` - **Path**: `/api/scores/export/{job_id}/pdf` - **Response**: `200 OK` - **Content-Type**: `application/pdf` ### 3.6 Public Shared Score Retrieval Allows public access to scores that have been shared via a token. - **Method**: `GET` - **Path**: `/api/scores/share/{share_token}` - **Authentication**: None required - **Response**: `200 OK` ```json { "share_token": "tok_xyz123abc", "title": "Sonata No. 14 in C-sharp Minor", "author_name": "Ludwig van Beethoven", "musicxml": "...", "allow_playback": true, "allow_fork": true, "view_count": 42, "created_at": "2026-10-17T12:00:00Z" } ``` --- ## 4. Integration Code Snippets ### Python (`httpx` asynchronous client) ```python import asyncio import httpx BASE_URL = "https://notesight.app/api" async def digitize_sheet_music(file_path: str, token: str) -> str: headers = {"Authorization": f"Bearer {token}"} async with httpx.AsyncClient(base_url=BASE_URL, timeout=120.0) as client: # 1. Upload score with open(file_path, "rb") as f: upload_res = await client.post( "/jobs/upload", headers=headers, files={"file": (file_path, f, "application/pdf")}, ) upload_res.raise_for_status() job = upload_res.json() job_id = job["id"] # 2. Poll until completed while True: status_res = await client.get(f"/jobs/{job_id}", headers=headers) status_res.raise_for_status() data = status_res.json() status = data["status"] if status == "COMPLETED": break elif status == "FAILED": raise RuntimeError(f"OMR failed: {data.get('error_message')}") await asyncio.sleep(2.0) # 3. Download MusicXML xml_res = await client.get(f"/scores/export/{job_id}/musicxml", headers=headers) xml_res.raise_for_status() return xml_res.text ``` ### cURL Command Line ```bash # 1. Upload PDF JOB_ID=$(curl -s -X POST "https://notesight.app/api/jobs/upload" \ -H "Authorization: Bearer $TOKEN" \ -F "file=@score.pdf" | jq -r '.id') # 2. Check status curl -s -X GET "https://notesight.app/api/jobs/$JOB_ID" \ -H "Authorization: Bearer $TOKEN" | jq . # 3. Fetch MusicXML when COMPLETED curl -s -X GET "https://notesight.app/api/scores/export/$JOB_ID/musicxml" \ -H "Authorization: Bearer $TOKEN" -o transcription.musicxml ``` --- ## 5. Error Codes & Troubleshooting | HTTP Status | Meaning | Typical Cause & Resolution | |-------------|---------|----------------------------| | `400 Bad Request` | Invalid payload or unsupported format | Ensure file is PDF, PNG, JPG, or TIFF. Verify file is not empty or corrupted. | | `401 Unauthorized` | Missing or invalid Bearer token | Ensure `Authorization: Bearer ` is present for authenticated endpoints. | | `403 Forbidden` | Access forbidden | User quota reached or unverified email. | | `404 Not Found` | Resource not found | Invalid `job_id` or nonexistent shared score token. | | `410 Gone` | Share link expired | The public share token expiration date has passed. | | `413 Payload Too Large` | Upload exceeded maximum limit | Files must be under 50 MB and under 200 pages. | | `429 Too Many Requests` | Rate limit triggered | Exponentially back off polling frequency (e.g. 2–3s intervals). | | `500 Server Error` | OMR processing fault | Unparseable musical layout or corrupted image raster. | --- ## 6. Schema & Specification Links - **OpenAPI 3.0 Manifest**: `https://notesight.app/api/openapi.json` - **Interactive Documentation**: `https://notesight.app/api/docs` - **AI Agent Discovery**: `https://notesight.app/llms.txt` - **Sitemap**: `https://notesight.app/sitemap.xml`