Skip to content

perf(local-model): adaptively accelerate slow model downloads - #387

Merged
Angnuo (angnuoli) merged 1 commit into
microsofthackathons:mainfrom
angnuoli:codex/adaptive-model-downloads
Nov 16, 2026
Merged

Angnuo (angnuoli) merged 1 commit into
microsofthackathons:mainfrom
angnuoli:codex/adaptive-model-downloads

Conversation

@angnuoli

Copy link
Copy Markdown
Collaborator

Description

Large pinned ModelScope downloads currently use a single response stream. This change starts with that stream, samples its throughput and attempts bounded range prefetch only when the source is slow. Fast sources retain their original connection.

Changes

  • Sample the initial stream for 1.5 seconds or 32 MiB. Below 16 MiB/s, probe range support and fetch the remaining bytes with up to four requests, 8 MiB parts and a bounded eight-part prefetch window.
  • Preserve the already-read prefix. Unsupported or rate-limited probes keep the existing response without restarting the download or issuing more parallel requests.
  • Retain the native hostname policy, cancellation, full-file size/SHA-256 verification, atomic publication and partial-file cleanup. GitHub runtime downloads and the legacy provider path remain unchanged.
  • Report transfer progress from received bytes with bounded update frequency, rather than from bursts of buffered writes.
  • Add coverage for adaptive selection, request bounds, fallback, malformed ranges, cancellation and native verification/publication behavior.

Type of Change

  • Performance improvement
  • Tests

Testing

The requester will validate the final adaptive revision and explicitly asked to stop further local tests and live benchmarks. The final revision has not been executed locally.

Earlier experiments confirmed range support and showed variable parallel throughput; a complete 328 MB artifact was faster on its original single stream. That evidence motivated keeping fast sources on their existing connection. No fixed speedup is claimed for this final revision.

The compatibility preload is loaded when the Gateway starts. This change does not restart an active download.

@angnuoli
Angnuo (angnuoli) merged commit e2529c3 into microsofthackathons:main Nov 16, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant