HasBorder

The HasBorder determines whether a border will be drawn around a control's contents.

Control(ControlID).HasBorder = True

You can change the border color using the BorderColor property.