Raspberry Pi’s single-board machines have set the mark for affordable, real-world computing. With its mannequin four, the Raspberry Pi can host net purposes with a production-grade net server, a transactional database system, and dynamic content material by means of scripting. This article explains the set up and configuration particulars with a full code instance. Welcome to net purposes hosted on a really light-weight pc.
The snowfall utility
Imagine a downhill ski space giant sufficient to have microclimates, which might imply dramatically totally different snowfalls throughout the world. The space is split into areas, every of which has units that report snowfall in centimeters; the recorded data then guides selections on snowmaking, grooming, and different upkeep operations. The units talk, say, each 20 minutes with a server that updates a database that helps reviews. Nowadays, the server-side software program for such an utility may be free and production-grade.
This snowfall utility makes use of the next applied sciences:
- A Raspberry Pi 4 operating Debian
- Nginx net server: The free model hosts over 400 million web sites. This net server is straightforward to put in, configure, and use.
- SQLite relational database system, which is file-based: A database, which might maintain many tables, is a file on the native system. SQLite is light-weight but additionally ACID-compliant; it’s suited to low to average quantity. SQLite is probably going probably the most extensively used database system on the earth, and the supply code for SQLite is within the public area. The present model is three. A extra highly effective (however nonetheless free) possibility is PostgreSQL.
- Python: The Python programming language can work together with databases resembling SQLite and net servers resembling Nginx. Python (model three) comes with Linux and macOS methods.
Python features a software program driver for speaking with SQLite. There are choices for connecting Python scripts with Nginx and different net servers. One possibility is uWSGI (Web Server Gateway Interface), which updates the traditional CGI (Common Gateway Interface) from the 1990s.
Several components converse for uWSGI:
- uWSGI is versatile. It can be utilized as both a light-weight concurrent net server or the backend utility server related to an internet server resembling Nginx.
- Its setup is minimal.
- The snowfall utility entails a low to average quantity of hits on the internet server and database system. In common, CGI applied sciences will not be quick by trendy requirements, however CGI performs properly sufficient for department-level net purposes resembling this one.
Various acronyms describe the uWSGI possibility. Here’s a sketch of the three principal ones:
- WSGI is a Python specification for an interface between an internet server on one aspect, and an utility or an utility framework (e.g., Django) on the opposite aspect. This specification defines an API whose implementation is left open.
- uWSGI implements the WSGI interface by offering an utility server, which connects purposes to an internet server. A uWSGI utility server’s most important job is to translate HTTP requests right into a format that an online utility can devour and, afterward, to format the applying’s response into an HTTP message.
- uwsgi is a binary protocol carried out by a uWSGI utility server to speak with a full-featured net server resembling Nginx; it additionally contains utilities resembling a light-weight net server. The Nginx net server “speaks” uwsgi out of the field.
For comfort, I’ll use “uwsgi” as shorthand for the binary protocol, the applying server, and the very light-weight net server.
Setting up the database
On a Debian-based system, you possibly can set up SQLite the standard approach (with % representing the command-line immediate):
% sudo apt-get set up sqlite3This database system is a group of C libraries and utilities, all of which come to about 500KB in dimension. There is not any database server to start out, cease, or in any other case preserve.
Once SQLite is put in, create a database on the command-line immediate:
% sqlite3 snowfall.dbIf this succeeds, the command creates the file snowfall.db within the present working listing. The database title is unfair (e.g., no extension is required), and the command opens the SQLite shopper utility with >sqlite because the immediate:
Enter ".help" for utilization hints.
sqlite>
Create the snowfall desk within the snowfall database with the next command. The desk title, just like the database title, is unfair:
sqlite> CREATE TABLE snowfall (id INTEGER PRIMARY KEY AUTOINCREMENT,
area TEXT NOT NULL,
system TEXT NOT NULL,
quantity DECIMAL NOT NULL,
tstamp DECIMAL NOT NULL);
SQLite instructions are case-insensitive, however it’s conventional to make use of uppercase for SQL phrases and lowercase for person phrases. Check that the desk was created:
sqlite> .schemaThe command echoes the CREATE TABLE assertion.
The database is now prepared for enterprise, though the single-table snowfall is empty. You can add rows interactively to the desk, however an empty desk is ok for now.
A primary have a look at the general structure
Recall that uwsgi can be utilized in two methods: both as a light-weight net server or as an utility server related to a production-grade net server resembling Nginx. The second use is the purpose, however the first is suited to growing and testing the programmer’s request-handling code. Here’s the structure with Nginx in play as the online server:
HTTP uwsgi
shopper<---->Nginx<----->appServer<--->request-handling code<--->SQLite
The shopper could possibly be a browser, a utility resembling curl, or a handmade program fluent in HTTP. Communications between the shopper and Nginx happen by means of HTTP, however then uwsgi takes over as a binary-transport protocol between Nginx and the applying server, which interacts with request-handling code resembling requestHandler.py (described beneath). This structure delivers a clear division of labor. Nginx alone manages the shopper, and solely the request-handling code interacts with the database. In flip, the applying server separates the online server from the programmer-written code, which has a high-level API to learn and write HTTP messages delivered over uwsgi.
I will look at these architectural items and canopy the steps for putting in, configuring, and utilizing uwsgi and Nginx within the subsequent sections.
The snowfall utility code
Below is the supply code file requestHandler.py for the snowfall utility. (It’s additionally out there on my website.) Different capabilities inside this code assist make clear the software program structure that connects SQLite, Nginx, and uwsgi.
The request-handling program
import sqlite3
import cgiPATH_2_DB = '/house/marty/wsgi/snowfall.db'
## Dispatches HTTP requests to the suitable handler.
def utility(env, start_line):
if env['REQUEST_METHOD'] == 'POST': ## add new DB report
return handle_post(env, start_line)
elif env['REQUEST_METHOD'] == 'GET': ## create HTML-fragment report
return handle_get(start_line)
else: ## no different possibility for now
start_line('405 METHOD NOT ALLOWED', [('Content-Type', 'textual content/plain')])
response_body = 'Only POST and GET verbs supported.'
return [response_body.encode()]def handle_post(env, start_line):
kind = get_field_storage(env) ## physique of an HTTP POST request
## Extract fields from POST kind.
area = kind.getvalue('area')
system = kind.getvalue('system')
quantity = kind.getvalue('quantity')
tstamp = kind.getvalue('tstamp')## Missing information?
if (area is not None and
system is not None and
quantity is not None and
tstamp is not None):
add_record(area, system, quantity, tstamp)
response_body = "POST request dealt with.n"
start_line('201 OK', [('Content-Type', 'textual content/plain')])
else:
response_body = "Missing information in POST request.n"
start_line('400 Bad Request', [('Content-Type', 'textual content/plain')])
return [response_body.encode()]def handle_get(start_line):
conn = sqlite3.join(PATH_2_DB) ## connect with DB
cursor = conn.cursor() ## get a cursor
cursor.execute("select * from snowfall")response_body = "<h3>Snowfall report</h3><ul>"
rows = cursor.fetchall()
for row in rows:
response_body += "<li>" + str(row[zero]) + '|' ## major key
response_body += row[1] + '|' ## area
response_body += row[2] + '|' ## system
response_body += str(row[three]) + '|' ## quantity
response_body += str(row[four]) + "</li>" ## timestamp
response_body += "</ul>"conn.commit() ## commit
conn.shut() ## cleanup
start_line('200 OK', [('Content-Type', 'textual content/html')])
return [response_body.encode()]## Add a report from a tool to the DB.
def add_record(reg, dev, amt, tstamp):
conn = sqlite3.join(PATH_2_DB) ## connect with DB
cursor = conn.cursor() ## get a cursorsql = "INSERT INTO snowfall(region,device,amount,tstamp) values (?,?,?,?)"
cursor.execute(sql, (reg, dev, amt, tstamp)) ## execute INSERTconn.commit() ## commit
conn.shut() ## cleanupdef get_field_storage(env):
enter = env['wsgi.enter']
kind = env.get('wsgi.post_form')
if (kind is not None and kind[zero] is enter):
return kind[2]fs = cgi.FieldStorage(fp = enter,
environ = env,
keep_blank_values = 1)
return fs
A relentless at first of the supply file defines the trail to the database file:
PATH_2_DB = '/house/marty/wsgi/snowfall.db'Make positive to replace the trail in your Raspberry Pi.
As famous earlier, uwsgi features a light-weight net server that may host this request-handling utility. To start, set up uwsgi with these two instructions (## introduces my feedback):
% sudo apt-get set up build-essential python-dev ## C header recordsdata, and so on.
% pip set up uwsgi ## pip = Python package deal supervisor
Next, launch a bare-bones snowfall utility utilizing uwsgi as the online server:
% uwsgi --http 127.zero.zero.1:9999 --wsgi-file requestHandler.py The flag --http runs uwsgi in web-server mode, with 9999 as the online server’s listening port on localhost (127.zero.zero.1). By default, uwsgi dispatches HTTP requests to a programmer-defined perform named utility. For overview, this is the total perform from the highest of the requestHandler.py code:
def utility(env, start_line):
if env['REQUEST_METHOD'] == 'POST': ## add new DB report
return handle_post(env, start_line)
elif env['REQUEST_METHOD'] == 'GET': ## create HTML-fragment report
return handle_get(start_line)
else: ## no different possibility for now
start_line('405 METHOD NOT ALLOWED', [('Content-Type', 'textual content/plain')])
response_body = 'Only POST and GET verbs supported.'
return [response_body.encode()]
The snowfall utility accepts solely two request varieties:
- A POST request, if as much as snuff, creates a brand new entry within the snowfall desk. The request ought to embody the ski space area, the system within the area, the snowfall quantity in centimeters, and a Unix-style timestamp. A POST request is dispatched to the
handle_postperform (which I will make clear shortly). - A GET request returns an HTML fragment (an unordered listing) with the information presently within the snowfall desk.
Requests with an HTTP verb apart from POST and GET will generate an error message.
You can use a utility resembling curl to generate HTTP requests for testing. Here are three pattern POST requests to start out populating the database:
% curl -X POST -d "region=R1&device=D9&amount=1.42&tstamp=1604722088.0158753" localhost:9999/
% curl -X POST -d "region=R7&device=D4&amount=2.11&tstamp=1604722296.8862638" localhost:9999/
% curl -X POST -d "region=R5&device=D1&amount=1.12&tstamp=1604942236.1013834" localhost:9999/
These instructions add three information to the snowfall desk. A subsequent GET request from curl or a browser shows an HTML fragment that lists the rows within the snowfall desk. Here’s the equal as non-HTML textual content:
Snowfall report1|R1|D9|1.42|1604722088.0158753
2|R7|D4|2.11|1604722296.8862638
three|R5|D1|1.12|1604942236.1013834
Knowledgeable report would convert the numeric timestamps into human-readable ones. But the emphasis, for now, is on the architectural parts within the snowfall utility, not on the person interface.
The uwsgi utility accepts numerous flags, which may be given both by means of a configuration file or within the launch command. For instance, this is a richer launch of uwsgi as an internet server:
% uwsgi --master --processes 2 --http 127.zero.zero.1:9999 --wsgi-file requestHandler.pyThis model creates a grasp (supervisory) course of and two employee processes, which might deal with the HTTP requests concurrently.
In the snowfall utility, the capabilities handle_post and handle_get course of POST and GET requests, respectively. Here’s the handle_post perform in full:
def handle_post(env, start_line):
kind = get_field_storage(env) ## physique of an HTTP POST request
## Extract fields from POST kind.
area = kind.getvalue('area')
system = kind.getvalue('system')
quantity = kind.getvalue('quantity')
tstamp = kind.getvalue('tstamp')## Missing information?
if (area is not None and
system is not None and
quantity is not None and
tstamp is not None):
add_record(area, system, quantity, tstamp)
response_body = "POST request dealt with.n"
start_line('201 OK', [('Content-Type', 'textual content/plain')])
else:
response_body = "Missing information in POST request.n"
start_line('400 Bad Request', [('Content-Type', 'textual content/plain')])
return [response_body.encode()]
The two arguments to the handle_post perform (env and start_line) characterize the system setting and a communications channel, respectively. The start_line channel sends the HTTP begin line (on this case, both 400 Bad Request or 201 OK) and any HTTP headers (on this case, simply Content-Type: textual content/plain) of an HTTP response.
The handle_post perform tries to extract the related information from the HTTP POST request and, if it is profitable, calls the perform add_record so as to add one other row to the snowfall desk:
def add_record(reg, dev, amt, tstamp):
conn = sqlite3.join(PATH_2_DB) ## connect with DB
cursor = conn.cursor() ## get a cursorsql = "INSERT INTO snowfall(region,device,amount,tstamp) VALUES (?,?,?,?)"
cursor.execute(sql, (reg, dev, amt, tstamp)) ## execute INSERTconn.commit() ## commit
conn.shut() ## cleanup
SQLite routinely wraps single SQL statements (resembling INSERT above) in a transaction, which accounts for the decision to conn.commit() within the code. SQLite additionally helps multi-statement transactions. After calling add_record, the handle_post perform winds up its work by sending an HTTP response affirmation message to the requester.
The handle_get perform additionally touches the database, however solely to learn the information within the snowfall desk:
def handle_get(start_line):
conn = sqlite3.join(PATH_2_DB) ## connect with DB
cursor = conn.cursor() ## get a cursor
cursor.execute("SELECT * FROM snowfall")response_body = "<h3>Snowfall report</h3><ul>"
rows = cursor.fetchall()
for row in rows:
response_body += "<li>" + str(row[zero]) + '|' ## major key
response_body += row[1] + '|' ## area
response_body += row[2] + '|' ## system
response_body += str(row[three]) + '|' ## quantity
response_body += str(row[four]) + "</li>" ## timestamp
response_body += "</ul>"conn.commit() ## commit
conn.shut() ## cleanup
start_line('200 OK', [('Content-Type', 'textual content/html')])
return [response_body.encode()]
A user-friendly model of the snowfall utility would assist further (and fancier) reviews, however even this model of handle_get underscores the clear interface between Python and SQLite. By the way in which, uwsgi expects a response physique to be a listing of bytes. In the return assertion, the decision to response_body.encode() contained in the sq. brackets generates the byte listing from the response_body string.
Moving as much as Nginx
The Nginx net server may be put in on a Debian-based system with one command:
% sudo apt-get set up nginxAs an internet server, Nginx offers the anticipated providers, resembling wire-level safety, HTTPS, person authentication, load balancing, media streaming, response compression, file importing, and so on. The Nginx engine is high-performance and steady, and this server can assist dynamic content material by means of a wide range of programming languages. Using uwsgi as a really light-weight net server is a pretty possibility however switching to Nginx is a transfer as much as industrial-strength hosting with high-volume functionality. Nginx and uwsgi are each carried out in C.
With Nginx in play, uwsgi takes on a communication protocol’s restricted roles and an utility server; it now not acts as an HTTP net server. Here’s the revised structure:
HTTP uwsgi
requester<---->Nginx<----->app server<--->requestHandler.py
As famous earlier, Nginx contains uwsgi assist and now acts as a reverse-proxy server that forwards designated HTTP requests to the uwsgi utility server, which in flip interacts with the Python script requestHandler.py. Responses from the Python script transfer within the reverse route in order that Nginx sends the HTTP response again to the requesting shopper.
Two adjustments deliver this new structure to life. The first launches uwsgi as an utility server:
% uwsgi --socket 127.zero.zero.1:8001 --wsgi-file requestHandler.pySocket 8001 is the Nginx default for uwsgi communications. For robustness, you would use the total path to the Python script in order that the command above doesn’t must be executed within the listing that homes the Python script. In a manufacturing setting, uwsgi would begin and cease routinely; for now, nonetheless, the emphasis stays on how the architectural items match collectively.
The second change entails Nginx configuration, which may be tough on Debian-based methods. The most important configuration file for Nginx is /and so on/nginx/nginx.conf, however this file could have embody directives for different recordsdata, specifically, recordsdata in one in all three /and so on/nginx subdirectories: nginx.d, sites-available, and sites-enabled. The embody directives may be eradicated to simplify issues; on this case, the configuration happens solely in nginx.conf. I like to recommend the straightforward strategy.
However the configuration is distributed, the important thing part for having Nginx speak to the uwsgi utility server begins with http and has a number of server subsections, which in flip have location subsections. Here’s an instance from the Nginx documentation:
...
http
The location subsections are those of curiosity. For the snowfall utility, this is the added location entry with its two configuration strains:
...
server
...
To preserve issues easy for now, make /snowfall the one location within the configuration. With this configuration in place, Nginx listens on port 80 and dispatches HTTP requests ending with the /snowfall path to the uwsgi utility server:
% curl -X POST -d "..." localhost/snowfall ## new POST
% curl -X GET localhost/snowfall ## new GET
The port quantity 80 may be dropped from the request as a result of 80 is the default server port for HTTP requests.
If the configured location had been merely / as a substitute of /snowfall, then any HTTP request with / at first of the trail can be dispatched to the uwsgi utility server. Accordingly, the /snowfall path leaves room for different areas and, due to this fact, for additional actions in response to HTTP requests.
Once you have modified the Nginx configuration with the added location subsection, you can begin the online server:
% sudo systemctl begin nginxThere are different instructions just like cease and restart Nginx. In a manufacturing setting, you would automate these actions in order that Nginx begins on a system boot and stops on a system shutdown.
With uwsgi and Nginx each operating, you need to use a browser to check whether or not the architectural parts cooperate as anticipated. For instance, when you enter the URL localhost/ within the browser’s enter window, then the Nginx welcome web page ought to seem with (HTML) content material just like this:
Welcome to nginx!
...
Thank you for utilizing nginx.
By distinction, the URL localhost/snowfall ought to show the rows presently within the snowfall desk:
Snowfall report1|R1|D9|1.42|1604722088.0158753
2|R7|D4|2.11|1604722296.8862638
three|R5|D1|1.12|1604942236.1013834
Wrapping up
The snowfall utility reveals how free software program parts—a high-powered net server, an ACID-compliant database system, and scripting for dynamic content material—can assist a practical net utility on a Raspberry Pi four platform. This light-weight machine lifts above its weight class, and Debian eases the lifting.
The software program parts within the net utility work properly collectively and require little or no configuration. For larger quantity hits towards a relational database, recall free and feature-rich different to SQLite is PostgreSQL. If you are wanting to play on the Raspberry Pi four—specifically, to discover server-side net programming on this platform—then Nginx, SQLite or PostgreSQL, uwsgi, and Python are value contemplating.
