Offline reinforcement learning datasets collected on real TriFinger robots and with a simulated version of the robot. Trajectories for two dexterous manipulation tasks were collected: Pushing a cube to a goal position on the ground and Lifting it to a goal position and orientation in the air. Several dataset types collected with policies of varying proficiency are available. Pose estimates for the cube are part of the observations. Versions of the datasets including images from three cameras are available as well. The datasets can be loaded with the trifinger_rl_datasets Python package available at https://github.com/rr-learning/trifinger_rl_datasets