A surprising number of companies have it, the Monday morning rite: export from one system as CSV, open the file, check the columns, fix the date format, repair the three rows with broken umlauts by hand, import into the other system, click away the error messages. It was introduced years ago as a stopgap, “until we have a proper solution”. The stopgap stayed, as stopgaps do. The way out rarely leads through one big rebuild, but through three stages, each of which is worth something on its own.

Why manual imports get more expensive over time

The hour per week is the smallest item. Three side effects cost more. First, the delay: between imports, the target system works with stale data, and everyone who uses it knows that and calls to double-check, just in case. Second, the fragility: every manual step in the routine, every hand-repaired row, is an opportunity for silent errors that surface weeks later. Third, the person dependency: the import belongs to one person, and it stands and falls with them.

The treacherous part is habituation. Because the import somehow works every week, it does not feel like a problem. Yet the ritual is no outlier: in Smartsheet’s “Automation in the Workplace” survey (2017), 42 percent of the roughly 1,000 office workers polled said they spend more than ten hours a week on repetitive manual work such as data entry and data transfer. Only when the responsible person is out for three weeks, or an error produces a wrong statement, does it become visible how much hangs on this ritual.

Stage 1: run the same import automatically

The first stage does not change the import; it merely takes it off the person’s hands. A small program picks up the export, applies the same corrections that used to happen manually, and loads the file, every night instead of every Monday. The format stays, the systems stay, the logic stays.

The charm of this stage is the effort: it is often done in one to two weeks, frequently for under €5,000, because nothing gets reinvented. The prerequisite is the same as with any automation: the person who does the import today has to list their steps completely, once, including the ones that are “only sometimes” necessary. Those sometimes-cases are the real specification.

Stage 2: validate before the import instead of repairing after

The automatic import from stage 1 has inherited a weakness: it is as trusting as the CSV file it receives. Stage 2 puts a check in front of the import. Every row is tested against rules before it reaches the target system: is the article number known? Is the price within a plausible range? Does the customer exist, and only once?

Rows that fail do not block the whole run; they land on a review list that an employee goes through in the morning. This inverts the working principle: instead of checking a hundred rows to find three errors, she gets the three errors served up. In our experience, this is the stage with the biggest tangible effect in daily work, because it does more than save time: it restores trust in the data. Gartner (2020) put a number on what is at stake: poor data quality costs a company an average of $12.9 million per year. That was measured at large corporations, but the mechanics are the same at small scale – the errors that surface late are the expensive ones.

Stage 3: the import disappears

The final stage replaces the file detour with a direct connection: the source system hands changes to the target system continuously, through an interface, event-driven or at short intervals. “Data as of last night” becomes “current data”, and the CSV format with its date and umlaut traps is history.

This stage is the most involved, from a few thousand to over ten thousand euros depending on the systems, and it does not always pay off. If the overnight freshness from stage 2 is enough for daily work, and it is more often than you would think, there is no reason for the rebuild. The appeal of “real time” is rarely worth it when nobody in the company makes a decision that hinges on minutes.

What happens to the special cases

The most common objection to abolishing the manual import goes: “But Ms. Steiner knows what to do with the odd cases.” The objection deserves to be taken seriously, because it is true. Almost every grown import contains five percent of cases handled by judgment: the one supplier with their own numbering format, the bundled line item that has to be split.

The solution is not to automate these cases away but to make them explicit. Whatever can be captured as a rule becomes a rule. Whatever remains genuine judgment lands on the review list, with exactly the person who can judge it. Ms. Steiner does not get replaced; she gets relieved of the assembly-line work and keeps the cases she is genuinely needed for.

When a manual import deserves to stay

Finally, the opposite direction, for completeness: an import that runs once a quarter, covers twenty rows, and is done in ten minutes needs no automation. The development would not pay for itself in years, and every additional piece of software wants maintenance. The same applies when one of the systems involved is due for replacement anyway.

The honest question is not whether an import can be automated; almost any can. The question is whether it runs often enough, moves enough rows, and can do enough damage to justify the three stages. For the Monday ritual with a thousand rows, the answer is almost always yes. For the quarterly twenty-liner, almost always no.