During the Second World War, the Nazis converted the castle into a high security prisoner-of-war camp for Allied officers who had become escape risks or who were regarded as particularly dangerous. However, although the castle was heavily guarded and is situated on a rocky outcropping above a river, it boasted one of the highest records of successful escape attempts and became a popular subject in film and literature. Those escape tunnels now form part of a local museum too.

But should you stay at Colditz Schloss (Castle) YHA in Germany, you won’t want to escape – for the past three years the northeastern part of the castle has undergone modernization to be transformed into one of the most up to date youth hostels in Germany, opening in June 2007 with over 160 beds in a variety of rooms, all of which are ensuite.
The historical character of the building has been retained and has a large terrace and an extensive park. And very nice it looks too.
So instead of visiting history, why not just sleep in it instead? Contact and bookings - you don't need to be a YHA member and the upper age limit no longer applies.
Wikipedia: Colditz Castle