At the moment the clamp gizmo works in the way that an unclamped object that is clamped again to the gizmo jumps to the same relative position in relation to the clamp gizmo as the item had before it was unclamped.
A more flexible clamp gizmo could have a different behaviour:
As long as an item is clamped to the gizmo it should for sure stay in the same relative position in relation to the gizmo. If it is unclamped, clamp gizmo and item could be moved independent. If the item is clamped again, it is clamped to the gizmo in exactly the same relative position that the item has at the moment it was when it was clamped again. So the item does not move due to the clamping.