Anomaly detection is an important application in factory environments. The sounds emitted by a manufacturing machine during runtime can be indicative of either normal performance or of mechanical failure. Traditionally, cosine losses are frequently utilized in anomalous sound detection (ASD) algorithms. In this work, we evaluate cosine losses within the context and scope of Deep Metric Learning (DML), and we investigate various ways to modify and potentially boost their performance. The impact of each modification on the performance of ASD systems is studied under extensive experimental settings and ablations utilizing both the DCASE 2022 and DCASE 2023 machine ASD datasets. Additionally, under the framework of DML, we develop a key new insight into the inner workings of cosine losses, and we verify it with an ablation.