Large Language Models (LLMs) excel in various tasks [2], but depending on use case, they rely on carefully crafted prompts that often demand substantial human effort to perfect or find optimal prompting given the task. Building on suggestions from EvoPrompt that seek to explore the search space of LLM prompting using Genetic Algorithms [9], we introduce AgentOptimiser, which leverages a Reinforcement Learning (RL) framework. We empower the agents to optimise prompting by passing a Q-table-like structure that manages state, enabling the LLM to track accuracy measures and corresponding historical prompting. We then seek to enable the model to explore the prompt and temperature space. We find that the AgentOptimiser-optimised GPT-4 variant outperforms other fine-tuned transformer models and vanilla few-shot GPT-4 performance on our commodity news-specific test set. We outline additional opportunities to continue and improve this work. The inclusion of the temperature parameterisation increases the optimisation search space; by introducing a generational fitness function, we increase our model's performance. We outline areas of future research to explore additional metaheuristic approaches that could be explored to better manage the exploration-exploitation trade-off to guide LLM agent task-specific optimisation in financial engineering.

Mechanistic interpretability[9] is the study of taking a trained neural network, and analysing the weights to reverse engineer the algorithms learned by the mode. We propose multiple forms of mechanistic interpretability MI [9] to support financial engineering utilising large language models. As these models scale, their open-endedness and high capacity creates an increasing scope for unexpected and sometimes harmful behaviors. [9] [1] Antrhopic discuss even years after a large model is trained, both creators and users routinely discover model capabilities including problematic behaviors they were previously unaware of. For our usecase it is imperative traders, or decision makers have an insight into the model, that might produce interesting signals to support decision making. We seek to provide mechanistic interpretability exploring BERTology [13] for open models and proposing alternative methods for closed source generative models. We find we can identify hallucinations from generated sentiment justifications from a finetuned GPT3.5 model, when we use an adjusted BLEU scores. When considering the use of these applications in production systems, human review remains critical, but creating a suite of hallucination prevention metrics should be considered.