In the realm of batch reinforcement learning, where agents are confronted with the challenges of partial observability, the phenomena of overfitting and asymptotic bias can greatly affect the learning process. This paper explores effective strategies for addressing these issues and enhancing the performance of agents operating under partial observability. We delve into novel techniques that provide insights into mitigating overfitting while countering the inherent asymptotic bias, thus paving the way for more robust and reliable batch reinforcement learning algorithms.