libadmin: Don't try to release garbage connection