Departments - filter out current users from search

When in Settings - Departments, If I choose a Department and Add Members, I can search based on user email which is very helpful, but it would really beneficial if the search results only included results who are not already in the department. This way going through and manually adding a bunch of users one by one based on an email address would be possible.