You can change a supervisor's status to Canceled, just like an apprentice, to remove them from actively participating in the program on WorkHands. Find them in your list from your Supervisors page.
Next, visit their page and click the Pencil icon below their profile picture.
A pop-up menu will appear. Select the Apprenticeship category. Find Status in the form, change it to Canceled, and then Save.
Note: If this Supervisor has active apprentices assigned to them as a primary supervisor, you'll want to change each of their apprentices' primary reviewers, too.