A Chunk Map You Can Resume From, and a Final File You Verify Before Believing

A resumable upload fails in two quiet ways. It resumes from a wrong belief about what the server holds, or it publishes an assembled file nobody checked. This article proposes a chunk map and a final acceptance plan. Everything here is a conditional design to adapt, and no part is claimed as implemented.

The artifact: chunk map

For each upload session, store one row per chunk index with these fields:

index, byte offset, expected length
state: pending, sent_unconfirmed, confirmed, or rejected
chunk digest, recorded when the server confirms

The session also records the declared total size, the declared whole-file digest, the chunk count, and a local fingerprint of the selected file (name, size, last modified time, and ideally a digest of a few sampled regions).

A chunk becomes confirmed only when the server's response states the index, length and digest it stored. A sent request alone moves it only to sent_unconfirmed.

Resume branches

On resume, choose exactly one:

Same file. The fingerprint matches. Continue with the map.
Changed file. Size or fingerprint differs. Do not reuse the map. Offer to start a new session and leave the old one for cleanup.
Uncertain chunk. Some chunk is sent_unconfirmed. Ask the server which chunks it holds for this session. Resend only the chunks the server reports absent. If the status query itself fails, stop and show a retry choice to the user. Do not resend automatically, because the earlier write may have succeeded.
Server disagrees. The server reports a chunk the map calls pending, or reports a different digest from the map. Treat the session as suspect and stop.

Final acceptance plan

Assembly is a staging step, never publication. After assembling into a staging location, require all of these:

The chunk count equals the map.
The assembled byte length equals the declared size.
The whole-file digest computed server side equals the declared digest.
Only then move the file to its final record. A single guarded state change marks it available.

A local claim that all chunks were sent does not count as proof of delivery. Only the server-computed digest counts.

Worked hypothetical example

A 50 MiB file with 8 MiB chunks gives 7 chunks: six full chunks (48 MiB) and a last chunk of 2 MiB. The connection drops. The map shows chunks 0 to 3 confirmed, chunk 4 sent_unconfirmed, and chunks 5 and 6 pending.

On resume the fingerprint matches, so the same-file branch applies. The server status says chunks 0 to 4 are present and chunk 4's digest equals the map's expected digest. Chunk 4 becomes confirmed with no resend. Chunks 5 and 6 are sent and confirmed. Assembly then yields 7 chunks and 52428800 bytes, which equals the declared size. The server computes the whole-file digest and it matches the declared one, so the file becomes available.

Variant: if the server had reported chunk 4 absent, only chunk 4 would be resent, one time, then re-checked. If the final digest had mismatched, the staged file stays unpublished, the user sees a failure, and nothing is retried silently.

Stop conditions

Fingerprint changed mid-session.
Server status unavailable.
Final digest, size or count mismatch.
A chunk is rejected twice for the same reason.

Proposed acceptance tests (to be run, not yet performed)

Interrupt after chunk 3 and expect resume to send only chunks that are not confirmed.
Drop the response for chunk 4 and expect a status query before any resend.
Edit the local file between sessions and expect the changed-file branch.
Corrupt one stored chunk in a test environment and expect a digest mismatch with the file unpublished.
Make the status endpoint fail and expect a stop with a manual choice, not a loop.
Complete a normal upload and expect available state only after all three assembly checks.

Defects will remain after these tests. Treat them as a starting set and extend them from real failures you observe.