The County Gaol was built in 1813 in the style of a medieval castle, and because of this, the name of Salt Lane was changed to Castle Street. Mr. Sandy's was the architect, following the principals of John Howard. It contained 90 cells, and was enlarged in 1839 to give 80 extra cells. This was the time of great political agitation, when during the summer, the gaol was excessively crowded with Chartists from Dudley, with cells intended for one containing three.

Over the centuries, the City had many prisons. There was the gaol at the east end of St Nicholas Street, a Bridewell at the bottom of Cucken Street (Copenhagan Street) and below the gatehouse of the Foregate were cells which, for a long period, were used as a prison for strangers