When in 1272 the garrison passed definitely under Lucca the fortress already existed.
In 1316 it was, together with Coreglia, a stronghold of Guelph resistance against Castruccio Castracani. When he conquered the fortress, the castle was enlarged and became important for the control of the Serchio valley.
The tower had considerable dimensions and could house a large number of soldiers and arms; moreover, the well defended position of the fortress, together with its healthy climate, made it an ideal residence for important Lucchese personalities such as Paolo Guinigi who took refuge here with all his family and entourage during the 1418 pest in Lucca. In 1408 heavy rains caused the collapse of a 38 m long portion of wall, whereas in 1410 the 'Vicario' (Lieutenant), Landuccio Menabuoi, informed Paolo Guinigi that the restoration of castle, rocca and sleeping quarters within the tower had been completed.
In 1430, when the Seigniory of the Guinigi over Lucca ended, the Vicariate of Coreglia fell into the hands of Francesco Sforza and for 4 years Ghivizzano was the centre of a small state; it then became a stable possession of the Lucchese Republic, until the latter became Principality under Napoleon.