loading page

Bit-Grooming: Shave Your Bits with Razor-sharp Precision
  • Charles Zender,
  • Jeremy Silver
Charles Zender

Corresponding Author:[email protected]

Author Profile
Jeremy Silver
University of Melbourne
Author Profile

Abstract

Lossless compression can reduce climate data storage by 30-40%. In general, further reductions require lossy compression that also reduces precision. Fortunately, geoscientific models and measurements generate false precision (scientifically meaningless data bits) that can be eliminated without sacrificing scientifically meaningful data. We introduce Bit Grooming, a lossy compression algorithm that removes the bloat due to false- precision, those bits and bytes beyond the meaningful precision of the data. We evaluated Bit Grooming against competitors Linear Packing, Layer Packing, and GRIB2/JPEG2000.