Problem solving
I enjoy the challenge of finding solutions to problems, and these are five that came up on platforms I have built and maintained, with what was going wrong in each and what I did about it.
Giving every device its own sign-in
I inherited a project where the API had one shared key written into the API itself, which every copy of the app sent with its requests and the server checked against its own. I identified this as a security issue as much as a practical one, because anyone who sniffed the requests from one copy of the app would have the key and could call the API themselves, there was nothing to tell one device from another and the only way to change the key was to change it for every device at once.
I replaced it with a proper sign-in for devices, where an app registers through a handshake endpoint and gets back a signed token that expires, and then sends that token with every request it makes. Devices have their own table and their own authentication guard, separate from the accounts people use in the admin system, and each device record has an access flag so that a single device can be shut out without affecting the rest.
The same code had been used for multiple projects, so the shared key was an issue in several other APIs as well, and I systematically reviewed all of them and updated each one to sign in per device.
Keeping the rest of a batch when one entry fails
The app event log is sent to the API in batches, the original endpoint would work through the batch in order and stop with a 404 at the first entry it could not match, so everything before that entry had been saved, everything after it was never processed and the app had no way of knowing which entries had made it.
I built a replacement so that one bad entry no longer loses the rest of the batch. It validates each
entry on its own and stores the valid ones, and it answers with a 207 Multi-Status listing
what was created and what failed, with every failed entry returned exactly as it was sent along with the
reasons it was rejected. It also recognises an entry it has already stored, so an app that is unsure
what got through can simply send the whole batch again.
Recovering tests that only partly uploaded
A headset uploads a test in two calls, the first to create the test and the second to send its inputs, which can run to several hundred, and that second call is all or nothing. A headset could therefore be holding a test the server did not have, or one it had without all of its inputs, and there was no way of comparing the two.
To sort this out I built an import for the headset's own export file. Before it imports anything it compares every test in the file against the database and puts each one into one of seven states, which include missing on the server, partly uploaded, up to date and in conflict, and then it imports only what is missing.
The import runs through the same code as the API, so a record arrives exactly as it would have if the headset had uploaded it, and keeps the ID the headset gave it. Running the same file a second time changes nothing, and where the server holds more inputs than the file does, the import flags a conflict and leaves that test as it is.
Uploading videos bigger than 100 MB
Video uploads in the admin system used to go through the web server, and at first they failed at 12 MB, which is the default limit of the upload component and was reached before my own validation had even run. I raised the limit to 99 MB, and that was as far as it could go because the site sits behind Cloudflare, which refuses any single request over 100 MB, and nothing in the application can change that.
Where a file only needs storing, the browser now asks the application first, and the application chooses the storage key itself and hands back a presigned URL that allows that one upload for 15 minutes. The browser sends the file straight to object storage, showing its progress as it goes, and the application confirms the file is there under the key it issued before it saves anything.
A video the server has to convert still has to reach the server, so a presigned URL does not help there. For those the browser cuts the file into 20 MB pieces and posts them one at a time, and the server puts them back together and checks what the file really is before converting it, which takes files up to 500 MB.
A file that goes straight to storage is one the application has never read, so there could be anything behind a 3D model's file name, and for that reason stored models are checked for the format's signature and structure before they are ever served.
Copying a course from one tenant to another
On a learning platform I built in a previous role, each client organisation is a tenant with its own subdomain and its own database. Middleware reads the subdomain before the route runs and switches the database connection, cache, file storage and queue over to that tenant's, so a query cannot return another tenant's rows because the request is not connected to their database in the first place.
That separation made one request particularly difficult, which was to copy a course from one tenant to another, because no query can see both databases, every ID is different on the other side and the media sits in separate storage.
I wrote the copy as a background job that switches between the two tenants as it goes, reading from one and writing to the other, and keeps a map of old IDs to new so that a quiz in the copy points at the copied questions. The media is copied on a queue of its own with every file tracked, so anything that fails to copy is flagged. The tenant switching itself is handled by the open-source package stancl/tenancy.