A Clean Slate for Offline Reinforcement Learning