Yes, you need to have the web dashboard open (and set to Available!) or be logged in on the mobile app. If no one is logged in, your chat box will be shown as an email form.