Give an Import Estimate as a Range You Measured, Not a Finish Time You Promised

A progress bar that says "done at 14:05" creates a promise the import cannot keep. Rows differ in size, remote services slow down, and retries take time. This article proposes a worksheet that produces a range from your own measurements and states what the range leaves out. Every feature below is a design recommendation to adapt, not a description of an existing product.

The worksheet

Record these fields for each import run.

1. Total rows, and whether that total is known or still being counted.
2. Window size: a fixed number of rows per measurement window.
3. For each completed window: rows processed and elapsed seconds, which give rows per second.
4. Rows remaining.
5. Slowest and fastest window rates seen so far.
6. Excluded time: pauses, operator holds and time spent waiting on retries.

The estimate is remaining rows divided by the fastest rate, and remaining rows divided by the slowest rate. Show both ends as the range. Label the range "based on the last N windows" and do not display a single clock time.

Branches

Choose exactly one branch for each run.

Total unknown: show only rows processed and the current rate. Give no completion range. Switch to the range branch once the total is counted.

Fewer than three completed windows: show "measuring" and no range. Two samples are too few to trust.

Total known and at least three windows: use the worksheet range.

The range keeps growing: if the slowest rate drops below half of the first window's rate, mark the estimate as unreliable and ask a person to look at the cause. Do not keep recalculating quietly.

Worked hypothetical example

An import has 12,000 rows and a window size of 600 rows.

Window 1: 120 seconds, 5.00 rows per second. Window 2: 150 seconds, 4.00 rows per second. Window 3: 96 seconds, 6.25 rows per second.

Processed rows: 3 x 600 = 1,800. Remaining: 12,000 - 1,800 = 10,200.

Optimistic end: 10,200 / 6.25 = 1,632 seconds, about 27 minutes. Pessimistic end: 10,200 / 4.00 = 2,550 seconds, about 42.5 minutes.

The display reads: "Roughly 27 to 43 minutes remaining, based on 3 windows. Pauses and retries are not included." Seven windows later, if one window runs at 1.5 rows per second, that is below half of the first window's 5.00 rate. The status changes to "estimate unreliable, review needed" and the range is hidden until a person acknowledges it.

Stop conditions

Stop showing a range when any of these holds:

• The import is paused or held. Show "paused", not a countdown.
• The total row count changes after the range was shown. Restart measuring.
• A window records zero rows processed. Treat it as stalled, not as a rate of zero to divide by.
• An external write has an uncertain outcome. Do not rerun that write automatically. Mark it for review, and keep it out of the rate calculation until its outcome is known.

Proposed acceptance tests (not yet performed)

• Feed the three windows above and check that the display shows 27 and 43 minutes after rounding, with no single clock time.
• Feed two windows and check that the status is "measuring".
• Feed an unknown total and check that no range appears.
• Feed a window of zero rows and check that the status is stalled and no division error occurs.
• Pause the run and check that the countdown disappears and that paused time is excluded from rates.
• Change the total mid-run and check that measuring restarts.

Limits

The range describes the past windows, not the future. A slow remote service, a larger batch of rows or a queue shared with other work can push the real finish outside it. Say that in the interface once, plainly, and let the numbers speak.

Sources