No you do not have to have the push notification icon on your screen to send push notifications. However, having it there will allow users to see past notifications so it’s a good idea to keep it there.
Your other option for sending push notification is to use the GLOBAL PUSH NOTIFICATIONS feature. With this feature you can send a push notification to one or multiple apps that you have published.
Rocket App Builder Support