Implementing a simple websocket server in Python

23 Apr 2022 - tsp
Last update 23 Apr 2022
Reading time 7 mins

Introduction
The example application
- Client side (Browser)
- Server side (Python)
What is possible using WebSockets?

Introduction

This blog article will be just a quick note on how one can quickly hack a WebSocket server using Python with the websockets package and a coroutine per client. So what are WebSockets? Since the early days the web was - and still is - based on a pretty simple protocol - the hypertext transfer protocol (HTTP). When doing an HTTP request a client opens a connection, requests a resource and gets an response immediately. Then the connection is closed again, no state is kept on the server side by HTTP specification between different requests. This is sufficient for most web tasks - but then JavaScript applications (of which I’m not a huge fan of obviously, but there are niches especially in Intranet applications where JavaScript can add more than just convenience value in a reasonable way) emerged which are able to run inside the browser. To allow one to exchange data with the server-side back-end without reloading the page as for traditional HTML forms then the browsers have been extended with XMLHttpRequest which is commonly known under the term AJAX - asynchronous JavaScript and XML. Basically this interface allows scripts to initiate arbitrary HTTP requests as long as they adhere to the security policies which are out of scope for this blog post. This interface is not limited to XML but one can transfer arbitrary data. That way one can implement features like infinite scrolling, load additional information, trigger actions on the server side, execute actions on the backend, etc. directly out of JavaScript by sending an asynchronous request and having a callback being invoked with the request returned a result. This is nice and the way to go for most slow actions triggered by the user inside the browser - but still there is no way for the server side to initiate a data transfer to the browser. For example when one wanted to implement notifications one had to wait for the client to either perform periodic AJAX requests all over again or perform dirty hacks like long polling - which basically is also an AJAX request but where one keeps the TCP connection open by simply not delivering the answer from the server side till either and event occurs or the connection times out.

WebSockets close this gap. They allow bidirectional realtime data exchange between a server and a client (in contrast to WebRTC data channels which are another API that allows one to build peer to peer communication channels between different browser instances). They’re built upon HTTP by utilizing the Upgrade functionality by which one can switch protocols after initial connection establishment. There are a few ways websocket servers can be implemented:

Running them on a separate port (which might be a bad idea due to firewalling issues)
Running them embedded in the webserver which is often not possible due to the internal structure of either the client or the websocket server
Using a proxy module such as Apache’s proxy module and passing it to a server running on a separate port

In the following quick summary I’m following the first approach though this can simply be modified to be hidden behind a proxy module anyways.

The example application

Client side (Browser)

The example application will be pretty straight forward: A webpage that allows a user to submit a string (pure JavaScript, no fallback for clients without JavaScript so basically bad design but it’s only a demo anyways) to the websocket backend which then returns the case-folded version of the string or nothing in case non ASCII characters have been submitted.

On the web side the application will consist of a simple webpage that I’m going to call wstest.html:

<!DOCTYPE html>
<html>
        <head>
                <title> Websocket example </title>
				<script type="text/javascript" src="./wstest.js"></script>
        </head>
        <body>
				<p> Input text to casefold: <input type="text" id="testdata"> <button id="sendtestdata">Send</button> </p>

				<h2> Responses </h2>
				<ul>
				</ul>
        </body>
</html>

The scripts will be contained in a simple JavaScript file wstest.js:

window.addEventListener('load', () => {
	let ws = new WebSocket("wss://www.example.com:1234");

	ws.onmessage = function(event) {
		let messages = document.getElementsByTagName('ul')[0];
		let newMessage = document.createElement('li');
		newMessage.appendChild(document.createTextNode(event.data));
		messages.appendChild(newMessage);
	};

	document.getElementById('sendtestdata').addEventListener('click', () => {
		ws.send(document.getElementById('testdata').value);
	});
});

Server side (Python)

On the server side the application will utilize the asyncio package as well as the simple to use websockets package. This can easily be installed via pip using

pip install websockets

WebSockets provides a simple coroutine based implementation of the WebSockets protocol as well as the most basic implementation of HTTP to only allow the Upgrade request to succeed. The idea behind the server implementation is pretty simple:

Create an SSL context for our SSL certificates
Create the server instance
A handler coroutine will be called (or one can imagine started) for every incoming request. This coroutine can be imagined like a single thread that’s started for every incoming request - the routine will be alive till the connection terminates. This makes implementing most communication stuff pretty simple but one has to take care of synchronization between different coroutines. Of course one looses the ability to do thread pooling or similar stuff that one would have with a full blown threaded implementation - there are libraries for that of course.
Inside the handler one can receive and transmit data as one likes till the connection gets closed or one drops the connection.

First one requires some imports:

#!/usr/bin/env python

import asyncio
import websockets
import ssl

Then one has to initialize the SSL context to handle the SSL certificate and private key:

ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
ssl_context.load_cert_chain("ssl.cer", "ssl.key")

The last step outside the connection handler is to initialize the server and enter the asyncio event loop:

server = websockets.serve(wsHandler, host = '127.0.0.1', port = 1234, ssl = ssl_context)

asyncio.get_event_loop().run_until_complete(server)
asyncio.get_event_loop().run_forever()

Now one can define the handler coroutine itself:

async def wsHandler(socket, path):
        print("[ Connection established: {} ]".format(path))
        while True:
                try:
                        dta = await socket.recv()
                        print("< {}".format(dta))
                        response = dta.casefold()
                        await socket.send(response)
                        print("> {}".format(response))
                except websockets.ConnectionClosed:
                        print("[ Connection closed ]")
                        break
                except Exception as e:
                        pass

As one can see this is really simple. It just informs us on the console that the connection has been established and then - in an infinite loop - awaits new data from the client and pushes the response out via the websocket. Of course a real application that uses websockets would have a little bit different structure since one also wants to asynchronously transmit information back to the client and perform proper authentication.

The full application thus is just:

#!/usr/bin/env python

import asyncio
import websockets
import ssl

async def wsHandler(socket, path):
        print("[ Connection established ]")
        print("[ Path: {} ]".format(path))
        while True:
                try:
                        dta = await socket.recv()
                        print("< {}".format(dta))
                        response = dta.casefold()
                        await socket.send(response)
                        print("> {}".format(response))
                except websockets.ConnectionClosed:
                        print("[ Connection closed ]")
                        break
                except Exception as e:
                        print(e)

ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
ssl_context.load_cert_chain("ssl.cer", "ssl.key")

server = websockets.serve(wsHandler, host = '127.0.0.1', port = 1234, ssl = ssl_context)

asyncio.get_event_loop().run_until_complete(server)
asyncio.get_event_loop().run_forever()

Usually one wants to wrap the whole application in some kind of demonizing framework, add proper logging and error handling and of course interaction with different other components.

What is possible using WebSockets?

Building notification frameworks
Interfacing message queuing systems like MQTT or AMQP brokers
Delivering realtime data from network devices
Building multiplayer games
Delivering data while it gets generated (for example realtime results from data analysis applications)
Building collaborative applications
Providing permanent synchronization of work inside the browser
Returning auto completion results from things like Elasticsearch backends
many, many more