Low-power non-volatile memories operating down to deep cryogenic temperatures are important for a large spectrum of applications from high-performance computing and electronics interfacing quantum computing hardware to space-based electronics. Despite the potential of Hf0.5Zr0.5O2 (HZO), thanks to its compatibility with complementary metal-oxide-semiconductor (CMOS) back-end-of line processing, only a few studies of HZO based memory devices down to cryogenic operation temperatures exist. Here, we report on analog ferroelectric memory stack fabrication with 10 nm HZO and its detailed characterization under a wide range of pulse amplitudes and frequencies from 300 K down to 4 K. When operated at temperatures below 100 K, HZO devices can support high amplitude voltage pulses, yielding record high remnant polarization Pr of up to 75µC cm-2 at ±7 Vp (14 Vpp) pulse amplitudes accompanied with frequency-dependent memory window between 6 and 8 V. Devices show excellent endurance exceeding 109 cycles of ±5 Vp (10 Vpp) and Pr of 30 µC cm-2 without significant degradation of coercive voltages or loss of polarization at cryogenic temperatures. At least 20 reproducible analog states for temperatures below 100 K with almost ideal linearity of intermediate polarization states in both pulse directions are observed, demonstrating the high potential of analog cryogenic ferroelectric memories, essential for online training in In-Memory-Computing architecture.