We present a multi-way parallel corpus of Math Word Problems (MWPs) in nine languages, including six low-resource languages. To date, this is the largest multilingual MWP dataset available. We utilize this dataset and show the viability of using large language models (LLMs) for autoregressive MWP generation in both monolingual and multilingual setups, particularly for low-resource languages. We also integrate a math constraint satisfaction module with autoregressive text generation. Our extensive evaluations identify several factors that affect autoregressive text generation on LLMs. These include language representation in the LLM, model size, existence of similar languages in the model, and language script. Overall, our results reveal that autoregressive MWP generation on top of LLMs is very promising, even for low-resource languages.