Software Possession for Personal Use

2024-08-19 #software #web

There’s a lot to be frustrated about contemporary software and the modern web. (If you are a software user and you are not frustrated, you can safely skip this article).

There’s a lot to be frustrated about software, the cloud, and Big Tech, especially if you’ve been using computers long enough to remember what the experience used to be like. It was worse, of course, in some ways, more limited and primitive, but much better in others. And that’s the thing: we probably didn’t notice that we were making this trade-off—not before we were all in.

We gave up privacy and control in exchange for convenience and information sharing, but we also gave up performance and offline access, and data durability and ownership. There are things we can do now—things we need to do now, like video conferencing and real-time collaboration—that couldn’t be done with traditional desktop software. But for the things we could do back then, the overall experience has likely gotten worse, and it seems to be getting worse all the time.

There’s a lot to be frustrated about, many problems with contemporary software, and many attempts to work around them:

• Building local-first software.
• Replacing platforms with open protocols.
• Self-hosting free and open-source software.
• Using P2P networks, like BitTorrent and the InterPlanetary File System.
• Using Federated networks, like those powered by ActivityPub.
• Moving from walled gardens to the IndieWeb.
• Home-cooking software as bare-foot developers.

(If you aren’t already familiar with these ideas, reading about them is probably a better use of your time than reading the rest of this article).

The local-first software paper describes the transition from desktop to the cloud, what we lost along the way, and how we could get it back. It emphasizes real-time collaboration as a distinct feature of modern software. The biggest challenge, then, lies in re-imagining applications like Google Docs in a context where there isn’t a server acting as a centralized authority, mediating client interactions. Hence the need to experiment with decentralized data replication and conflict resolution, and the prominent place that CDRTs occupy in this space.

Useful and ground-breaking as real-time collaboration is, looking back at my user journey, it wasn’t that that drew me to replace desktop applications with cloud platforms. What forced me to “surrender” was that I went from using a single computer at home to using another one at the office and carrying a third one in my pocket at all times. I needed my programs to run on all these devices and my data available and synchronized everywhere. Even today, when I look at the home screen on my phone, half of what I see are specialized data synchronization apps: Google Drive, Google Keep, Trello, LastPass, Goodreads, Spotify. Even the ones, like Google Docs and Trello, that are designed for teamwork, I only have them there so I can access the data while I’m away from my laptop. (I realize I’m not the average user, and that’s part of the problem). For me, real-time collaboration typically happens on the desktop, in short-lived sessions where I can tolerate more invasive applications.

I got into platforms because of the device syncing, and that’s something platforms do well. Why do I want out? For all the reasons listed in the local-first paper, of course, but, most of all, because platforms dumb down my computing experience. I can live with the loss of control and privacy during collaboration sessions. I can live with unnecessary roundtrip latency and mandatory internet connection. But I don’t want to live in a world where every 6 months I need to add an extra hop to find an album on Spotify, where Google abruptly retires applications I came to rely on, where Microsoft places ads in my taskbar, where Apple keeps me from running the software I want and Amazon acquires companies to let their products, which I use, rot to death.

As a user, I can’t become proficient with platforms like I did with traditional desktop applications—and continue to do with local-only open-source software. Not only can’t they be adjusted to my needs, but I can’t even adjust myself to their capabilities without the risk of them changing unexpectedly, becoming unusable or altogether unavailable after some time. Platforms get in the way of realizing the Creative Computing and Augmented Human Intellect ideals that made me want to become a programmer. We replaced the bicycle of the mind with a set of brain crutches.

In a recent conference talk by Martin Kleppmann, I perceived a slight shift in the local-first narrative, focusing less on real-time collaboration and more on interoperable data synchronization. This is best illustrated by a diagram in the “endgame” slide:

In this scenario, client applications work independently, without server coordination; the cloud components are only needed to relay data to other devices and users, to the extent allowed by the owner. Users still pay for cloud storage, but sync services and local-first applications communicate through interoperable protocols, so users are free to switch providers, e.g. migrating from AWS to Azure, opting for a P2P network, a self-hosted server, or switching to local-only mode.

Similar to the vision of the Protocols, not Platforms essay, in this setup cloud providers still have a business, only one not built around locking users in. The fact that servers are interoperable and application-agnostic means that multiple implementations of the same app could flourish, catering to different user preferences, thus enabling free and open-source distributed applications without the server administration overhead.

This is a future I look forward to; the talk made me even more enthusiastic about the local-first ideas. But there’s still a lot of way to go before the community agrees on protocols, vendors start providing sync services, and open-source libraries become available to assist application developers in realizing this architecture. I can think of a couple of short-term experiments in this direction, like building serverless client apps that synchronize their data with cr-sqlite or writing open-source adapter libraries to turn cloud services into dumb encrypted data banks. These sound like interesting research projects but less appealing as means to build applications today¹.

One way to “resist” platforms today is to run open-source software on commodity servers —even on home servers. As Kleppmann points out in his talk, this isn’t for everyone and it’s not a general solution: most people don’t have the skills to do system administration or, if they do, it’s not something they necessarily want to spend their time on. Self-hosting is not a general solution, but it’s a solution, and I’m glad to see people out there doing impressive things with it. They’ll be laughing at us from their rocking chairs when cloudpocalypse comes.

I can do some basic sysadmin but it’s not something I particularly enjoy. Since, above all, I like to build software, my take on escaping the cloud is to build my own tools. This is even less practical than self-hosting —I still need to host the system myself, on top of building it. And no one can possibly build all the software they need. And for anything I can put together in my free time, there are probably many better open-source alternatives. But I still think it’s a worthwhile exercise. You may say that reinterpreting the wheel is a hobby of mine.

Self-building is especially tempting for software that falls into what I previously called specialized data synchronization apps: clients that provide a convenient interface to access data stored in a server, with no hard real-time collaboration or multi-user requirements. Note-taking, to-do/to-read/to-watch lists, personal project management, and feed readers are good examples.

I recently published a couple of such tools; to describe them, I wanted to convey that, while they weren’t strictly toy projects, neither were they intended as general-purpose tools, to support the needs of a wider public, or to scale beyond a few users. I settled on the word ‘personal’ to communicate this idea: feedi is a personal feed reader; jorge is a personal (small + opinionated) site generator². So I retrospectively started calling these apps personal software.

If I had to define it, I’d say personal software is software developed by someone for their own use (or, perhaps, for the use of a handful of people), fitting their particular needs and preferences, running and providing data access across devices. This is just a fancy way of saying “dogfooded web apps” but I find that the longer formulation is useful during development, as shown in the appendix below.

Personal software won’t change the world, it’s just a little rest area by the road to a better web. If local-fist software attempts to tame the cloud and self-hosting to resist it, personal software merely tries to escape it. But it’s good to know it’s there, that it’s an option for developers. It may become an option for amateur tinkerers, too³. It’s good to know it’s there, it’s good to tackle a project without pretension—not to start a business, not to acquire some skill or grow a portfolio, but to recover that builder joy, and the user power, of creative computing.

Appendix: my personal software stack

The software development process is an endless stream of decisions. Each decision involves trade-offs which (ideally) should be “calibrated” to the context of the project: its goals, the available resources, and the desired capabilities of the system being produced⁴. Given the definition above, this is how I calibrate personal software projects:

• The project should be narrow enough to be executed by a single person. More importantly, the cognitive load should be small enough to be carried by one person—the system should fit in one head.
• The system should be operable in local-only or self-hosted setups with minimal effort.
• The application should be accessible from multiple devices.
• Some interface simplicity can be sacrificed in favor of implementation simplicity⁵.
• The user experience should satisfy the needs and preferences of its designer, as opposed to those of some generic user (which removes the problem of modeling that user).
• The software doesn’t need to solve the problem of its economic sustainability.

Given my current skill set and preferences, to satisfy those requirements I “instantiate” my projects with these defaults:

• The system is structured as a web application because the browser is a universal front end, allowing the same application to run on desktop and mobile devices with minimal implementation overhead.

  • The program can also provide a command-line interface for administrative tasks or for streamlining some of the functionality when working at the terminal.

• The system is implemented with the Go language because it works well for servers and CLI programs, and compiles to easy-to-distribute binaries. Much can be done with the standard library alone, without external dependencies, and the concurrency model makes it easy to implement features like background tasks, which would require additional components in other ecosystems (e.g. cron, celery, sidekiq).

  • In scenarios where it makes sense to sacrifice operational simplicity in favor of implementation simplicity, I would use Python instead.
• SQLite as the database, because it’s featureful and easy to operate, and personal software shouldn’t need to scale beyond a single server.
• htmx for the front end, because it enables rich interfaces without turning the front end into its own separate application, and with almost no JavaScript.
• Linux on a VPS to deploy the system, because it’s cheap and has low operational overhead (as opposed to using containers or AWS infrastructure)⁶.

Notes