High: ipcs: Prevent ipc server use after free.

The ipc server registers the bind socket to
the poll loop in order to be alerted to new
connection requests. Upon shutdown, the ipc server
does not remove this poll entry. This patch fixes
this use after free.