Alright, let's kick things off by getting the Firebase CLI (Command-Line Interface) tool installed. This is our gateway to Firebase Hosting. Open up your terminal and type:

npm install -g firebase-tools

Next up, we need to set up our Firebase project in the local directory. Fire up your terminal again and log in to Firebase using your Google account by typing:

firebase login

Once you're logged in, initialize your Firebase project with:

firebase init

During this setup, choose Hosting. You'll need to configure your public directory (where your website’s files live) and decide if you want to use single-page app routing.

Now, let's create a custom error page. Save this file in your public directory, the same place as your website's index.html. You can name it something like 404.html.

In the firebase.json file at the root of your project, we need to tell Firebase Hosting to redirect 404 errors to our custom error page. Add this code:

{
  "hosting": {
    "public": "public",
    "ignore": [
      "firebase.json",
      "**/.*",
      "**/node_modules/**"
    ],
    "rewrites": [
      {
        "source": "**",
        "destination": "/404.html"
      }
    ]
  }
}

This setup tells Firebase Hosting to send all requests for non-existent static files to the /404.html page.

Your website is now ready to roll with Firebase, complete with its custom error page. Deploy your site by running:

firebase deploy

After deploying, Firebase will give you a unique URL. Test it out by adding a path that you know doesn't exist, and make sure Firebase shows your custom 404 error page.