# NoteSight

> Sheet Music Digitization, Optical Music Recognition (OMR), and Interactive Score Rendering Platform.

NoteSight is a modern web-based sheet music sight-reading and digitization platform. It converts sheet music documents (multi-page PDFs or single images in PNG, JPEG, TIFF) into standardized MusicXML, synthesized audio playback (Web Audio & MIDI), and high-resolution vector PDFs via an asynchronous deep learning OMR pipeline.

## Capabilities & Architecture

- **OMR Neural Engine**: Employs deep-learning neural transcription (`homr` with ONNX runtime) to parse staves, clefs, key signatures, time signatures, notes, accidentals, rests, and musical dynamics.
- **Continuous Multi-Page Stitching**: Extracts vector PDF pages at 300 DPI (via PyMuPDF), transcribes pages concurrently or sequentially, renumbers musical measures monotonically, and stitches them into a single consolidated MusicXML document.
- **Interactive Browser Notation**: Renders responsive SVG sheet music in the client using OpenSheetMusicDisplay (OSMD).
- **Audio Synthesis**: Synthesizes polyphonic Web Audio playback with real-time cursor tracking, metronome accompaniment, and variable tempo controls.
- **Verovio Vector PDF & MIDI**: Exports scores back into standard print-ready PDF and Standard MIDI (`.mid`) format.
- **Collaboration & Sharing**: Secure share links, iframe embedding, and WebSocket studio collaboration.

## Quickstart for AI Agents

To digitize sheet music programmatically via the NoteSight API:

1. **Upload Score Document**:
   Send a `POST` request to `/api/jobs/upload` with `multipart/form-data` containing the `file` (PDF or image).
   The response returns a `job_id` and initial status `PENDING`.

2. **Poll Status**:
   Send periodic `GET` requests to `/api/jobs/{job_id}`.
   When the status transitions from `PROCESSING` to `COMPLETED`, the job is ready.

3. **Retrieve MusicXML**:
   Fetch the transcribed notation via `GET /api/scores/export/{job_id}/musicxml`.

4. **Export Audio or Print Format**:
   - Standard MIDI: `GET /api/scores/export/{job_id}/midi`
   - Vector PDF: `GET /api/scores/export/{job_id}/pdf`

## Key API Endpoints

- `POST /api/jobs/upload`: Upload PDF/PNG/JPEG for OMR transcription (returns `job_id`).
- `GET /api/jobs/{job_id}`: Check processing status, page count, and metadata.
- `GET /api/scores/export/{job_id}/musicxml`: Export transcription as MusicXML 3.1.
- `GET /api/scores/export/{job_id}/midi`: Export synthesized MIDI track.
- `GET /api/scores/export/{job_id}/pdf`: Render and download vector PDF via Verovio.
- `GET /api/scores/share/{share_token}`: Public shared score metadata and MusicXML payload.
- `GET /api/health/live`: Public liveness health check probe.
- `GET /api/openapi.json`: Complete OpenAPI 3.0 specification.
- `GET /api/docs`: Interactive Swagger UI documentation.

## Input Constraints & Supported Formats

- **File Formats**: PDF (`.pdf`), PNG (`.png`), JPEG (`.jpg`, `.jpeg`), TIFF (`.tiff`).
- **Maximum File Size**: 50 MB (`MAX_UPLOAD_SIZE_BYTES=52428800`).
- **Resolution**: 300 DPI recommended for optimal optical character recognition accuracy.
- **Maximum Pages**: 200 pages per PDF document.

## Extended Documentation

For full API schemas, request/response samples, code snippets in Python and TypeScript, see:
- [Full LLM Specification](/llms-full.txt): Detailed developer documentation for autonomous agents.
- [OpenAPI Specification](/api/openapi.json): Standard OpenAPI JSON manifest.