Tiny Buckets

My pain

1. Starting things is hard. I can't believe how hard it is. All this experience I have jumping in to 10-year-old code bases, building a mental model, working within and enhancing the architecture—super valuable and marketable, barely any help with what I'm trying to do now. (Ok, not true. See 3.2. All the experience still warns me off mistakes.)
2. I have found no plausible replacement for Rails on the server, but I want to use Svelte on the client. And Svelte is my canvas: that's where I want to experiment and design, not in Rails. 
3. I've been working (off and on) for several months on a scratch-my-own-itch app ("hovercar") that I can't share with anyone (and that has some major inconveniences for using myself) because there's no data storage. I actually input data by manually editing JSON files.
   1. Shouldn't take so long to whip up a simple CRUD API, right? What's the hold up?
   2. See first point, but also it has taken a few months of developing/using this thing to realize that the correct canonical format for the main data in this thing is probably...drumroll...one text blob or file per user per day. (I.e., the ostensibly-interim defacto format I was actually using.) Every richer model I've considered along the way hasn't felt quite right, and my this isn't quite right feeling is very wise.

What I want

1. A simple, flexible data storage API I can use for rich-client apps. Basically a document database.
2. BUT, I'm not trying to build anything mega-scale, and I don't want to be locked into the pricing or limitations of anyone's cloud. There are a ton of things (sending mail, charging credit cards) that can be handled much more conveniently with a traditional Rails app than some bundle of trendy JAMstack microservices.
3. Anything I want to build should be able to run entirely (or sharded by account, perhaps) on a single server with a SQLite database. SQLite's crazy-fast and easy to deploy, and an in-process database cuts so much overhead vs a client-server one that, e.g., many small queries (n+1 and worse!) are actually just fine.
   1. For resiliency, the idea is nightly backups (or streaming backups when warranted) and a trivial, automated server spin-up process.
4. Some nice way to handle authorization without having to decide on the full user model up front. S3/B2-style keys-scoped-to-bucket-and-prefix have a strong appeal because I can manually assign keys, buckets, and prefixes at first and later decide how people will normally get the keys and what entities might own or manage them and how.
5. Version/conflict management along the lines of HTTP conditional GET and PUT requests. We can leave it up to the client to manage the conflict, but it should be easy for the client to find out that it's trying to save over a version we haven't seen yet.
   1. I did too much research on concurrent editing. It can't be done without substantial loss of generality, and I don't really need concurrent editing or even realtime reading of updates right now.
   2. But I'd like to be able to build realtime reading/notifying of updates in the future.

How I'm solving it

When I started writing this, I was 95% sure I would be building some kind of micro document db or micro S3 (hence the "tiny buckets" title) as a Rails Engine—had most of the schema worked out. Then I could embed and extend it in any Rails app I should happen to build, including starting with an empty app that had nothing but the engine for initial work. (That's all I need to take hovercar to the next stage if I manually create keys.) This SQLite as a document database article points to a couple helpful specific tactics for evolving such a document store.

But from hacker news comments on that article I ran across LiteStore (first released all the way back in 2015!) that may cover everything I need to get started.

Sidebar rant about Rails and Hotwire

Rails is my most productive server-side framework, possibly the most productive current framework in some absolute sense. But Hotwire, Rails' new UI hotness, is so very lame: an ugly, bolted-on DSL to do exactly that set of tedious imperative UI manipulations that the basecamp/hey folks happened to want. It's the opposite of composable and functional, and I don't want it. I don't want to imperatively manage the UI, even with less code. I want to manage the state and have the UI handle itself.