It is thought that the men garrisoning the milecastles and turrets came from the neighbouring forts. We know the Roman army frequently practised outposting throughout the empire, which meant that any fort or fortress might have detachments of varying sizes sent off on various tasks. This is confirmed by the Vindolanda writing tablets – a strength report of the last decade of the 1st century AD (so about 30 years before the Wall) reveals that cohors I Tungrorum had 60% of its men outposted, rather than at Vindolanda itself. So, although direct evidence is lacking, it is likely that the forts provided the men for manning the Wall.
Further reading: Breeze and Dobson 2000