2024 State of the Tools
It’s that time of year again! Since I did a relatively full reset last year, for this year’s State of the Tools I’ll focus on changes and a few highlights. My Zoom studio post discusses my hardware and software for video recording in more detail.
Hardware
My core hardware is the same as year; there are a few tweaks or additions:
Bought a modern gaming rig at work for ML development and testing to increase my lab’s compute capacity and allow me to work on recommender software development while the datacenter machine is busy with bigger jobs and student work. It’s an HP Omen 45L with an i9 14900K and GeForce RTX 4090. After the first of the year, I will also be adding a Geekworm PiKVM for remote management.
The RockPro64 board in my home NAS died, so I replaced it with a cheap NAS/HTPC machine built on the Intel N100 chipset (link). It’s been working great. I also added a Sabrent 4-bay USB 3.2 drive enclosure with a couple of 16TB drives to expand my data storage; now that it’s Intel-based, I switched the main file system from XFS back to (mirrored) ZFS for increased flexibility with external storage expansion.
Bought a Steam Deck for gaming (and software testing).
Bought a 144TB 8-bay Thunderbolt RAID for university data archival and slow storage.
I’ve also been working with Drexel CCI IT to set up the ability to monitor power consumption for my work compute resources; I’ll write more about the details of that another time.
Productivity and Work Management
After experimenting with Logseq for a few months, I have switched to Obsidian for project and task tracking. Logseq’s data model might be a better fit for what I need, but the lack of good documentation for its query language was increasingly frustrating. Obsidian lets me write my custom query and reporting logic in plain JavaScript.
Operating Systems
Nothing new here; still macOS on endpoints, Debian on personal servers, and Ubuntu (grudgingly) on work servers. I’ve got a Windows 11 ARM install in Parallels on my Drexel MacBook, and we still have Windows on a desktop at home. That machine will get a refresh with either Debian or Fedora in 2025, since it is too old to run Windows 11.
I’ve started using Lima for managing Linux VMs on Macs (it provides a WSL2-like experience), and Colima for hosting Docker containers. I’m very happy with both; in particular, Colima’s seamless support for x86_64 Docker containers via Rosetta 2 is very nice.
Data and Code Management
I’ve been moving away from git-annex for data management; it’s nice, but it’s also more manual than I would like for a lot of cases, and while it works ok on macOS, it’s very Linux-oriented.
For synchronizing large data archives (e.g., my academic portfolio or teaching archive), I’ve gone back to Syncthing; once set up, it (mostly) just works, and doesn’t require any manual steps for keeping data in sync.
For assets in my website, I’ve switched back to git-lfs. I set up Charm’s soft-serve to store git + git-lfs data without GitHub’s storage and bandwidth quotas, and after a bit of juggling to figure out how to build it into a Podman container with Tailscale support, it’s been excellent.
I’m mostly using WebDAV for other kinds of data servers these days, especially for DVC remotes and my Kopia backups. Caddy has been great for a lightweight, efficient server (as well as a local development server for web work), and I’m using Apache for more traditional setups (my web site is published with Apache, as is my research group’s data repository). They’re both mostly fine, each has capabilities the other lacks. I did have problems when trying to use Apache mod_dav
as a Kopia remote, though — Kopia makes so many small transfers Apache was spending all its time authenticating them, whereas Caddy handles the load without problems.
I’ve switched from my own scripts on top of a Git repo to chezmoi for managing home directory config files.
Development and Analytics
My development stack is mostly stable these days; mostly Python + Rust + TypeScript (usually in Deno). I’m not using Tcl as much as I was (and no longer use it in my website). Interesting additions include:
- just for task running in various projects.
- Pixi for installing Conda-based software environments; it has a better project and lockfile UX than Conda and
conda-lock
. - Using PyTorch much more, including as the core of most LensKit models.
- DuckDB for large single-node data processing; so far, it’s working better than other things I’ve tried, including Polars and DataFusion, and has a much more stable developer experience.
- uv for non-Conda Python project management.
Tools and Utilities
I’ve picked up or changed a few of the smaller bits in my toolset as well:
- Switched from Barkeeper to Ice after Bartender was sold without transparency.
- Maccy for clipboard history; works as well as or better than Paste for my use cases and is cheap.
- Adopted Parallels Toolbox for a number of tools, including on systems without Parallels. Particular tools I use from it include the system monitor, the disk cleaner, and the uninstaller. I don’t like the Parallels menu bar hider or clipboard history, they are significantly buggier than Ice or Maccy.
- Replaced iStatMenus with the Parallels system monitor, for now; I was having Bluetooth monitoring performance problems with iStatMenus. May revisit this.
- Atuin for command-line history in
zsh
. - Spaceship for my ZSH prompt.
Future
I’m sure my stack will evolve in the next year, but I don’t have any specific changes I’m looking to make.