The hot database shrank 45-fold, and payments never stopped · Codelabs
RU|EN|KK
← All cases
CASE 02Moving history into a columnar store

The hot database shrank 45-fold, and payments never stopped

The money journal had grown to hundreds of millions of rows, and everything touching it was slowing down. We left one day in the main database instead of forty-five and sent the whole history into a separate store, streaming, with a lag of seconds. Every report was switched to the new read one at a time, in small releases, and at each step the old answer was reconciled against the new one: the difference had to be zero. It was.

DataMariaDBClickHouse

How it works

The switch was not one release but a series of small ones: each reader of the money journal moved to the new read separately, behind a runtime toggle, with the old path one switch away.

Acceptance at every step meant comparing the old path against the new one on production data with a requirement of zero difference. Along the way three non-obvious problems had to be solved: classifying tombstone rows, filtering legacy roll-ups, and agreeing a single time-zone canon between MySQL and ClickHouse.

Data streams into the columnar store from the replication log with a lag measured in seconds. The hot database now holds one day instead of forty-five, the full history stays available, and payments were never stopped.

In numbers

  • ·hot database: one day instead of 45
  • ·the full history is kept
  • ·reconciliation at every step: difference 0.00
  • ·payments were never stopped

Describe your task — we will tell you what part of it we have already built