The Gamepad API & Physical Computing
HTML’s Gamepad API has been created specifically with online games in mind. However, this does not mean it cannot be used in other contexts. For instance, gamepads have been hacked to assist people with physical disabilities, repurposed for artistic goals or altered for the sheer fun of it. Whatever your physical computing project, this article will show you how you can use the Gamepad API within TangibleJS’ preferred deployment platform: NW.js.
Background
The Gamepad API’s first draft was published in 2012. That’s pretty old in web years. However, as with any experimental technology, it took a while for browsers to catch on. Today, browser support is pretty good with Edge, Firefox, Chrome and Opera being on board.
Not that we really care though… If, like us, you are using NW.js or Electron as your primary physical computing platform, all you need to know is that Chromium supports the Gamepad API… which it does, of course!
Let’s get started by looking at the kind of gaming hardware we can use.
Hardware
The Gamepad API should theoretically work with any OS-supported game controllers. On Windows, this means all devices compatible with XInput or DirectInput. On Mac, support is a bit more sketchy but still pretty good.
I haven’t found a definitive list of gamepads that are confirmed to work in the Gamepad API but you can easily check for yourself by going to the HTML5 Gamepad Tester.
An interesting alternative to the classic gamepad, especially for the tinkerers that we are, is the game controller emulator. This kind of emulator behaves as a regular gamepad to the host computer but let’s us hook up custom switches and/or analog inputs. One such emulator is called the Multi-Console Cthulhu (link down):
It is very similar in design to the keyboard emulators we looked at in an earlier article. Basically, the Cthulhu is a PC-compatible USB gamepad with no buttons. Instead, the board lets you screw in your own switches to its terminals. When the buttons (or switches) are pressed, the Cthulhu sends matching gamepad events to the host computer. This makes it super easy to build your own game controller (almost) from scratch. You can buy the Cthulhu from Godlike Controls.
While it is probably the best known, the Cthulhu is not the only gamepad emulator. Other examples are the one sold by SmallCab and the Dual Strike from Strike Devices.
Obviously, you can also extract boards from old game controllers and repurpose them. This can be a very cheap option if you are willing to get your hands dirty. The even nerdier option would be to build your own from scratch.
I have been asked about the Wiimote (Wii Remote). Unfortunately, there are no official drivers for the Wiimote on either Mac or Windows. However, there are third party drivers that may work for you. I was able to successfully connect a Wiimote to my Mac (OS X Yosemite) and use it in a browser with an application called WJoy (no longer actively supported). On Windows, HID Wiimote is supposed to do the same. However, on my old Windows 7 box I was not able to make it work. I did read reports of people who were successful, though.
For this demo, I will stay on the safe side and use a good old wired Xbox 360 controller. This controller is officially supported by Microsoft with version 7, 8 and 10 of Windows.
You can get detailed configuration instructions for Windows 7, 8 and 10 on Microsoft’s site. You can also get similar instructions for the Xbox One controller.
By the way, you cannot use a wireless Xbox 360 controller unless you buy the Xbox 360 Wireless Gaming Receiver for Windows. If you have a wireless controller with a nice USB cable (a.k.a. Play & Charge Kit), you may think that you can use that to connect to your PC. You would be wrong. I know, I wasted countless hours trying to make this work…
If your are on Mac, the Xbox 360 wired controller will work just fine if you install the third party 360Controller driver. The driver also supports wireless Xbox 360 controllers as long as you have a Wireless Gaming Receiver.
Be sure to download the latest release from the GitHub repo and NOT the one from the TattieBogle site which is older and wasn’t working for me on OS X Yosemite.
Obviously, if you have another brand/model this tutorial still applies to you. Just make sure that its recognized at the system level. To do that on Windows, open a command prompt and type joy.cpl. This will open a window with a list of all properly configured controllers.
On Mac, it depends on the device’s maker but usually you will see a new icon in the system preferences.
Once you know your device is properly recognized at the system level, you can then check if it works all the way up at the browser level. To do that, launch Chrome and go to the HTML5 Gamepad API Tester. If you can see your controller, you are all set.
Software
As we stated earlier, our goal is to use the Gamepad API in NW.js. So, the first thing to do is setup an NW.js project. If you are not familiar with NW.js or can’t recall how a project is created, please first read Creating a Native App with HTML, CSS and JavaScript.
Retrieving information about controllers
Once ready, the first thing to do would be to retrieve a list of connected gamepads. This is done by listening to the gamedpadconnected event. The gamedpadconnected event is triggered each time a gamepad is connected. However, there is a catch here. Chromium (NW.js’ engine) insists that the user first interacts with the controller before triggering the initial gamedpadconnected event. This is to prevent user fingerprinting. While this makes a lot of sense on the web, it is far from ideal in the context of NW.js. Unfortunately, as of yet, there is no way to bypass this restriction. This means that you will have to request from your users that they press one of the front-facing buttons so the event is triggered. On the Xbox 360 controller, this is one of A, B, X or Y. The other buttons (Start, Select, etc.) cannot be used for this.
An issue has been filed with the Chromium team to request a flag enabling to bypass this security restriction. If you would like to see this feature implemented, please vote for it. The more votes it gets, the more likely it is to ever be implemented.
So, if you add the following code, as soon as the user presses A, B, X or Y, the gamedpadconnected event will be fired:
|
1 2 3 |
window.addEventListener("gamepadconnected", function(e) { console.log(e.gamepad); }); |
The code above allows us to retrieve information about the connected game controller. This is what you should see in the console’s output:
As you can see, the controller features 4 axes: X/Y for the left stick and X/Y for the right stick. It also features 17 buttons: A, B, X, Y, Select, Home, Back, Left Stick Push, Right Stick Push, LB, LT, RB, RT and the 4 buttons of the directional pad. Obviously, this may be different for other makes and models of controllers.
The mapping property indicates whether the controller’s buttons have been remapped to a known layout. Currently, there is only one layout called “standard”. Check out the specification for more details about this layout.
The timestamp property is a DOMHighResTimeStamp representing the last time the data for this gamepad was updated. It is relative to the PerformanceTiming.navigationStart property. The DOMHighResTimeStamp is precise to the millisecond.
It should also be noted that the index property is unique for each device. It also matches the index of that same controller in the array returned by the navigator.getGamepads() function. This function returns a 4-position GamepadList object :
|
1 2 |
var gamepads = navigator.getGamepads(); console.log(gamepads); |
The GamepadList object is basically an array of all the available spots for devices to be connected at. As you can see, some of these spots might be undefined. Currently, Chromium seems to limit the number of concurrent controllers to 4:
When a controller is disconnected, a gamepaddisconnected event is dispatched to the focused window. Beware that this event is only triggered by gamepads that previously sent the gamepadconnected event.
Reacting on button presses and stick moves
For various reasons, even the latest release of the Gamepad API specification does not include events for button presses and stick moves. It seems more discussion is needed. Instead, what has been favoured so far is a polling model. This means that, in lieu of listening to events, we need to be constantly checking the state of the device for changes. The most efficient way to do that, is to use a requestAnimationFrame() based loop.
In the example below we will use this polling technique to move an element anywhere in the page using one of the directional stick. First, let’s write some HTML and CSS:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Gamepad API + NW.js Demo</title> <script src="script.js"></script> <style> #ball { position: absolute; left: calc(50vw - 25px); top: calc(50vh - 25px); background-color: red; width: 50px; height: 50px; border-radius: 25px; } </style> </head> <body> <p>Press a button to activate your game controller.</p> <div id="ball"></div> </body> </html> |
As you can see in the body tag, there is a “ball” div element which is the one we will move around. It has been shaped into a ball using a bit of CSS. To make it move, we will use the following JavaScript (in the script.js file):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
var ball; window.addEventListener("gamepadconnected", function(e) { ball = document.getElementById("ball"); ball.style.backgroundColor = "green"; document.getElementsByTagName("p")[0].innerHTML = e.gamepad.id; updateLoop(); }); function updateLoop() { var gp = navigator.getGamepads()[0]; var left = (gp.axes[0] + 1) / 2 * (window.innerWidth - ball.offsetWidth); var right = (gp.axes[1] + 1) / 2 * (window.innerHeight - ball.offsetHeight); ball.style.left = left + "px"; ball.style.top = right + "px"; requestAnimationFrame(updateLoop); } |
There’s nothing too fancy here. When the gamepadconnected event fires, we change the ball’s color to green and write out the game controller’s name. Then, we start our update loop.
In the updateLoop function, we first grab the instance of the gamepad we want to work with. You might be wondering why we are doing that each frame. The problem with the Gamepad object is that it is only a static snapshot of the gamepad’s current state. In order to get the latest info, we must fetch it each time. Weird.
Anyway, once we have our gamepad object, we can simply retrieve the information from the axes array and update the ball’s position.
It should be noted that while the axes property of the Gamepad object contains an array of numbers, the buttons property contains an array of GamepadButton objects.
This means that if we wanted to change the background color of the body when the “A” button of the Xbox controller is pressed, we could use the pressed property of the desired button. Let’s modify the updateLoop() function to do just that:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
function updateLoop() { var gp = navigator.getGamepads()[0]; var left = (gp.axes[0] + 1) / 2 * (window.innerWidth - ball.offsetWidth); var right = (gp.axes[1] + 1) / 2 * (window.innerHeight - ball.offsetHeight); ball.style.left = left + "px"; ball.style.top = right + "px"; if (gp.buttons[0].pressed) { document.body.style.backgroundColor = "red"; } else { document.body.style.backgroundColor = "white"; } requestAnimationFrame(updateLoop); } |
While the polling approach might not be the most intuitive to some of us, I’m sure you will agree that using the Gamepad API is still pretty easy. Even if some features are still missing (rumble, accelerometer, etc.), it definitely can be used today!
If you need, you can download the code of this simple example. Note: the package.json file is only needed if you want to run the example in NW.js.
I hope you enjoyed this article. As always, I invite you to engage in the conversation by leaving your comments below.
Cheers!
The header photo has been taken by Iwan Gabovitch and is used in accordance with the Creative Commons Attribution 2.0 Generic license.