I don't know of a way to restrict how far they travel within the existing waypoint system. However, you can try using a script to keep their movement in check. The following example code assumes that the Guard waypoint is the first (and only) waypoint.
_pos = getWPPos [group1,1]
#Loop
~2
; Exit the script if all units in group1 are dead.
? {alive _x} count (units group1) < 1 : exit
; If the leader is farther than 1000 meters from the waypoint, order the group to move to the waypoint.
? (leader group1) distance _pos > 1000 : (units group1) doMove _pos
goto "Loop"
Of course, you can change the 1000 meters to whatever distance suits your needs. I chose to get the distance of the leader from the waypoint's position since I assume that the rest of the squad won't be too far away, since it's an AI squad. I also have it dynamically get the leader of group1 each time just in case the leader dies; that way, the new leader's position will be used instead of the previous dead one's.