This post serves as a personal bookmark and a mirror of this fantastic guide by Ok-Pace-1900 on /r/linux_gaming to ensure that this information does not get lost.

I learned the hard way that the GPU I have in a DIY Steam Machine PC, the AMD RX 480, is strictly unsupported by Forza Horizon 6.

Forza Horizon 6 will not work for AMD users with GPUs based on the Polaris or Vega architectures and older (for example Radeon 400 and 500 series players). These architectures are below our minimum supported specification.

I knew that asking for a refund on Steam would be the easy way out. Deciding against it, I did a quick search for the FH201 error code and stumbled on the Reddit post mentioned above. My CPU is good enough for Forza Horizon 6 (Intel i5-10500), so the additional launch options command that worked for me is the following:

VKD3D_FEATURE_LEVEL=12_1 VKD3D_CONFIG=descriptor_heap,no_upload_h_vram RADV_EXPERIMENTAL=heap,sync2 radv_wait_for_vm_map_updates=true %command% 

Simple fix, but the context around this is actually kind of funny. The way a lot of Windows-only games work on SteamOS is via a translation layer referred to as Proton. With this trick, you can pretend that your GPU has some DirectX features that it actually does not have, but it doesn’t matter since it can be successfully emulated via translation to Vulkan, which the GPU supports well!

As a result, I can play Forza Horizon 6 on a hacky SteamOS build, with 1080p low or medium settings. Low settings is a 60 FPS experience, with medium settings some areas like Tokyo can struggle a bit and drop below it to ~40 FPS.

Now all I need to do is to get rid of the urge to splurge on a great GPU, which would also require a case and PSU upgrade…

Slightly off-topic, but can you monitor your gaming PC via Prometheus Node Exporter and visualize it in Grafana? :)

Networking has long been my Achilles heel. I know the very basics, but the more complex areas of networking have been a bit puzzling to me. By the time I figured out how IPv4 works, I found IPv6 and that my ISP supports it.

Back to square one.

That didn’t stop me from learning some bits, and after 8+ years of self-hosting as a hobby, I’ve settled on a setup that works for me and overcomes common residential internet connection nuances, such as dynamic IPv4 addresses and changing IPv6 prefixes. I’m sharing these tips and tricks with the goal of helping out other hobbyists out there that happen to share a similar stack.

Background

My ISP is polite enough to provide a public IPv4 address, and allowing incoming traffic is a toggle in their online self-service. Not perfect, but at least you can do it. However, they charge about 6 EUR a month for the static IP address service, which I am not willing to pay out of principle.

They also support IPv6, which is great, and they provide you a whole /56 slice of it to play with using IPv6 prefix delegation. Unfortunately they have configured the lease time for the prefix to be incredibly short: 26 minutes!

A router reboot or short power outage usually results in the IPv4 address and IPv6 prefix changing, which is really annoying as my services become unavailable for a short time.

Dynamic DNS

A common way to overcome the dynamic IP address limitation is to sign up with a provider to set up a DNS record that changes whenever your home IP address changes. My domain registrar does not have this as a feature, and I’m not interested in using a different provider, so I went in a different direction and built a home-grown script that does the same thing.

Initially, this script relied on a public service that tells you what your IP address is, and based on that I could check if things have changed and I need to update my DNS record. This approach has one glaring catastrophic failure mode though: that provider could lie to you one day and now you’ve pointed your DNS records at the attackers’ servers. :)

I ignored that failure mode for a while, but once I learned about the effectiveness of LLM-based tooling, I decided to give it a go and to build a better solution that takes into account my setup and requirements, while at the same time saving me from the frustration of troubleshooting and debugging this in a late evening. I’m still very limited on available free time, so optimizing for that is a priority for me.

My main networking gear runs OpenWRT, and it supports running shell scripts periodically in a crontab. The router has two WAN interfaces, one for IPv4 and one for IPv6. It already knows what IP address and prefix have been assigned to it, so I don’t have to rely on an external service provider for finding this out.

Handling IPv4 addresses is simple: check the IPv4 address of the WAN interface. Query your existing DNS records, diff it, and if it has changed, push an update in a separate API call. Super simple!

With IPv6, the approach is slightly different. Instead of the WAN interface, I have to get the IPv6 address of the target machine, and make sure that it’s routable over the public internet. When you’ve checked your network settings in an IPv6 network, you may have noticed a lot of different IP addresses there, with lots of letters thrown into the mix.

Here’s an example from the machine that is serving you this blog (likely out of date though!):

    inet6 fdb3:6dad:6dce::f41/128 scope global dynamic noprefixroute 
    inet6 fdb3:6dad:6dce:0:2e0:4cff:fe0c:9ddb/64 scope global noprefixroute 
    inet6 2001:7d0:856c:4000::f41/128 scope global dynamic noprefixroute 
    inet6 2001:7d0:856c:4000:2e0:4cff:fe0c:9ddb/64 scope global dynamic noprefixroute 
    inet6 fe80::2e0:4cff:fe0c:9ddb/64 scope link noprefixroute 

The two relevant ones are the ones that start with 2001:, others are link-local or accessible over the local network only. The shorter one consists of the IPv6 prefix part, and then the unique bit at the end is a predictable suffix that the host gets. The other one also works, but is as far as I understand randomly generated and more difficult to predict when we get around to next sections.

I know that there is probably a better way to do this, but I wanted to keep things simple enough so that I can troubleshoot them if needed. It may be possible to trigger this updater script on events that WAN and WAN6 interfaces send, but I have not validated this theory.

There are many different ways to find the IPv6 address of a particular host, so the script I have just tries multiple approaches to find the one that we’re looking for.

Here’s the script in case you’re interested in setting up something similar. It reads credentials from an .env file and is built around the Zone API. On OpenWRT, the only dependency that you need to install is curl, which to my surprise was not part of the default packages list, probably to save on space.

One lesson I learned from a previous iteration of the script: if you trigger DNS record updates every minute, then Zone will actually reach out to you via e-mail telling you to cut that shit out, politely. It was just one missing if statement, and yet it caused some frustration to engineers far away. Sorry!

Predictable IP addresses

It’s a good idea to set up static IP addresses for your hosts, both for IPv4 addresses and IPv6 prefix delegation via DUID-s.

The OpenWRT GUI LuCI makes it quite simple, just set the addresses as static on the landing page for the hosts that you are interested in forwarding traffic to, and you’re done!

My recommendation here is to also set a predictable IPv6 suffix, otherwise all your IPv6 traffic rules may break once again due to this nuance.

I like to make that host number the same for both IPv4 and IPv6, quick example:

• 192.168.1.2
• 2001:7d0:854f:8e00::2

Here’s a configuration snippet example from /etc/config/dhcp, look for the hostid option:

config host
        option name 'mycoolserver'
        option ip '192.168.1.69'
        list mac '12:34:56:78:90:AB'
        option duid 'yourduidgoeshere'
        option hostid '69'

Apply with service dnsmasq restart.

In LuCI, as of OpenWRT 25.12, look for “IPv6 token”.

Note that due to a bug, it doesn’t seem to be possible to set a numeric IPv6 token via GUI, which is why you will need to add it manually in CLI using the above approach.

Port forwards, traffic rules, potato, potahtoh

Whenever you want to make a local machine accessible on the internet for IPv4, the solution is simple: set up a port forward to that particular machine, and you’re done! It’s a common enough flow for people who’ve set up game servers and the like, and well understood by more novice users.

With IPv6, port forwards don’t help. You’ll have to check one tab over at “Traffic rules” in OpenWRT GUI.

It’s a common misconception that by using IPv6 you are exposing everything to the world as each device gets its own IPv6 address, but turns out that this is not the case in most common setups. By default, OpenWRT forwards only a few types of traffic to IPv6 hosts, such as ICMP packets that make ping work between devices over IPv6 across the public internet. If you are interested in allowing IPv6 clients to access services on your local server that has an IPv6 address, you will have to explicitly allow it by adding a new traffic rule.

There’s one issue with this approach that a lot of users seem to run into: if the IPv6 prefix changes, then all my traffic rules that are pointing to a particular host are automatically broken!

Luckily there is a clever workaround implemented on OpenWRT that bypasses this issue. Assuming that you followed the previous step and set yourself up with a predictable IPv6 suffix, when setting up a traffic rule, set the target device up as ::69/-64, just replace 69 with your actual suffix. The IPv6 prefix can now change, but the ports that you’ve made accessible on this specific host will remain working.

At this point, you should be all set with a reasonably well working setup where you’ve handled the issues with dynamic IPv4 and IPv6 prefix, and you can access your services over the public internet even when things happen.

Limitations

One issue that this setup has is the fact that DNS change propagation takes time. Usually clients will pick up the new records within 5 minutes, but in my professional career I’ve seen some clients take up to 24 hours or longer to finally start sending traffic to the new DNS record. Whenever your IP address changes, there will be a mini-outage. Not catastrophic if you’re just hosting hobby projects and personal services at home, but I wouldn’t host anything mission-critical in such a setup.

When your OpenWRT device is as underpowered as mine, then you may notice that the TLS encryption overhead when curl -ing around can be significant. I have set my dynamic DNS script to run every 5 minutes, and it shows up on the CPU usage graphs on my router.

Wireguard all the things!

I know that Tailscale is a popular method of connecting up your devices and making your personal services privately accessible over that, which significantly reduces your attack surface. Being behind a few updates or not being vigilant enough is less of an issue compared to exposing your services over the public internet.

You don’t necessarily need Tailscale for that though! If you just need a way to access your services over a private and secure network, then setting up a dedicated mini PC or single-board computer is a very good starting point. Let it be the server, allow traffic to move between the clients over the Wireguard interface, and you’re all set!

Alternatively, if you have an OpenWRT router, then you can do it right there, but I found the GUI management setup to be a bit clunky compared to deploying the plain configuration files to clients. When I did do that test, I discovered quickly that my router and its single ARM CPU core with no cryptography extensions is too slow for managing my Wireguard network, with speeds topping out at 20 Mbit/s. 20. The LattePanda IOTA can easily saturate its gigabit link, as does the ThinkPad T430, and even devices like the Orange Pi Zero can handle a theoretical maximum of about 240 Mbit/s over Wireguard measured using wg-bench.

My current Wireguard host is the LattePanda V1, the most unstable computer in my fleet. With a USB adapter, it can push almost half a gigabit of traffic over Wireguard.

If you’re like me, and you like hosting your services over Docker or Podman, then instead of listening on ports for all interfaces on your containers (default behaviour when setting up port forwards), I recommend listening only on the Wireguard interface. This makes the service only accessible over Wireguard, meaning that you only need to set up one port forward and traffic rule to connect to the Wireguard network, and then you have access to all of your services. The attack surface is significantly reduced, the whole Wireguard solution is stable and very small, and unless you leak your private key, you are reasonably secure!

Here’s a snippet from a compose file showcasing how to set this up for IPv4 and IPv6:

ports:
  - 10.69.69.12:2283:2283
  - "[fded:abba:acca::12]:2283:2283"

Want to make the service available over Wireguard and over the local network directly? Just add those to the list! Note that if your local address changes and you don’t update it in the compose file, your container will refuse to start up as it cannot listen to the interface any longer, but you can mitigate that with the static IP addresses step.

When you are going with this route, it is unlikely but still possible that by the time the container starts up, the Wireguard interface is not yet up. To resolve this, you can use systemd to set Wireguard up as a dependency that you will have to wait for before the container starts up.

I manage my Wireguard connection with wg-quick@interfacename service. You can set up a systemd override for Docker, or if you manage your Docker/Podman services via systemd, then you can set it up per-service using this pattern:

# /etc/systemd/system/myimmichserver.service.d/override.conf
[Unit]
Requires=wg-quick@interfacename.service
After=wg-quick@interfacename.service

By the way, systemd overrides are also really useful for ensuring that your storage that your containers rely on is properly mounted. If my service requires the path /immich to be available and mounted, add something like this:

BindsTo=immich.mount
After=immich.mount

If you unmount the mount point, it will also properly bring down the service. The service won’t start if the mount point is missing. I’ve had the issue with containers seeing blank mount points more times than I’d like to admit, and this has eliminated this issue for me.

systemd has received a lot of hate online, and I don’t think it’s fair. The ease with which you can set up dependencies on your system, set up resource limits, make services more restricted to improve the security posture is great and allows me to and avoid all sorts of failure modes. Production services that I’m responsible for make use of these systemd features, with great results.

For services that need to be public, such as Nextcloud and its public shareable links, this approach won’t work, obviously, but for things that only you and your family members use, this is a viable approach.

Conclusion

This setup works well enough for me to confidently host my blog and self-hosted services off of it. I’ve hit a lot of paper cuts and frustrations along the way, but after following this guide, you don’t have to do the same.

Yes, VLAN-s are intentionally missing from this guide. I’ll get to them eventually, maybe by Q4 2037 given my lack of free time.

And no, IPv6 isn’t complicated, it’s just different from what everyone is used to. If we started out with IPv6 right from the get-go, we wouldn’t be having dumb arguments online.

Do you have a need for speed really fast disk performance?

Are you unwilling or unable to buy/rent a machine with fast, NVMe-based storage?

Are you OK with potential data corruption?

Then cache=unsafe is the solution to all your problems!

We had an interesting conundrum at work recently. Our platform does not use a lot of resources, but there are bursts of activity that require a lot of CPU and performant disk IO from our infrastructure. This was previously handled by manually starting some expensive AWS EC2 instances to cope with the load, but this manual process was error-prone due to the human factor (which did end up causing an actual production outage once), and AWS is stupidly expensive for what you get in return.

Around this time I also learned about a Proxmox server that we were underutilizing. My goal was to investigate the resources that we had available and to ensure that we didn’t have to think about taking any manual actions while at the same time not relying on AWS and its expensive resources.

I set up a few VM-s on the Proxmox machine, and did some testing. CPU, memory, that was all fine, but the IO-bound workloads that we had to run during those bursty periods would still be relatively slow. Not much slower than the main infrastructure provider that we were using, but slow enough for a beefy machine to not be able to handle more than a few parallel IO-heavy workloads running at the same time.

We exhausted a few other wild-ass ideas during the investigation:

• Docker on a RAM-backed storage drive
  • online resources did not inspire confidence in this working well, so we didn’t try this
• optimizing the workload to not be IO-heavy
  • unsuccessful after spending a few hours on it, the high IO was a consequence of making an intentional trade-off to reduce CPU load, and the IO requirement was much more manageable
• putting certain folders in the container itself on RAM-backed storage
  • highly container specific, and did not yield the desired results

Then one day I was browsing around Proxmox and noticed an interesting option on the virtual storage drives: setting the cache mode to unsafe. With this one trick, your VM will see really fast disk speeds up to a certain point, and it’s invisible from the perspective of your workloads, no customization needed. In a way, this is like one of the RAM-backed storage options, but for the whole VM.

The major trade-off is that an unexpected shutdown of the VM or the VM host will likely result in data corruption. This is because you’re writing everything to memory first, and eventually the writes will end up on persistent storage, whenever the disks catch up with you. Something happens while changes are in memory, and they are lost.

In our case, the data corruption risk is completely OK, as the workloads are ephemeral, the results of the work are sent to another machine immediately after completion, and the configuration of the machine is largely automated with Ansible.

One instance of our workload would usually result in writing 50 MB to disk, and we observed about 300-500 IOPS of performance from HDD-backed storage. The disks were not able to handle more than one at a time if we cared about execution time.

With the unsafe trick, and on some relatively old hardware (assume DDR3 memory), we saw numbers as high as 15K IOPS and disk throughput of 500+ MB/s. This was more than enough to handle peak loads, and the resources were always on and available on a rented server with a stable price that compared extremely well to AWS.

Cloud service providers have their benefits, sure, but when all you need is raw speed and configurability to make it happen, then owning a physical Linux server (or a few of them for redundancy) is a no-brainer, slam-dunk decision, as long as you have someone in your team that knows how to manage one. Since you’re working with Linux VM-s already in the cloud, then you already have that person in your team, don’t you? :)

I got the opportunity to try out a MacBook Pro with the M3 Pro with 18GB RAM (not Pro).

I’ve been rocking a ThinkPad P14s gen 4 and am reasonably happy with it, but after realizing that I am the only person in the whole company not on a MacBook, and one was suddenly available for use, I set one up for work duties to see if I could ever like using one.

It’s nice.

I’ve used various flavours of Linux on the desktop since 2014, starting with Linux Mint. 2015 was the year I deleted the Windows dual boot partition. Over those years, the experience on Linux and especially Fedora Linux has improved a lot, and for some reason it’s controversial to say that I love GNOME and its opinionated approach to building a cohesive and yet functional desktop environment.

When transitioning over to macOS, I went in with an open mind. I won’t heavily customise it, won’t install Asahi Linux on it, or make it do things it wasn’t meant to do. This is an appliance, I will use it to get work done and that’s it.

With this introduction out of the way, here are some observations I’ve made about this experience so far.

Ergonomics

The first stumbling block was an expected one: all the shortcuts are wrong, and the Ctrl-Super-Alt friendship has been replaced with these new weird ones. With a lot of trial and error, it is not that difficult to pick it up, but I still stumble around with copy-paste, moving windows around, or operating my cursor effectively. It certainly doesn’t help that in terminal windows, Ctrl is still king, while elsewhere it’s Cmd.

Mouse gestures are nice, and not that different from the GNOME experience.

macOS has window snapping by default, but only using the mouse. I had to install a specific program to enable window moving and snapping with keyboard shortcuts (Rectangle), which is something I use heavily in GNOME. Odd omission by Apple.

For my Logitech keyboard and mouse to do the right thing, I did have to install the Logitech Logi+ app, which is not ideal, but is needed to have an acceptable experience using my MX series peripherals, especially the keyboard where it needs to remap some keys for them to properly work in macOS.

I still haven’t quite figured out why Page up/down and Home/End keys are not working as they should be. Also, give my Delete key back!

Opening the laptop with Touch ID is a nice bonus, especially on public transport where I don’t really want my neighbour to see me typing in my password.

The macOS concept of showing open applications that don’t have windows on them as open in the dock is a strange choice, that has caused me to look for those phantom windows and is generally misleading.

Not being able to switch between open windows instead of applications echoes the same design choice that GNOME made, and I’m not a big fan of it here as well. But at least in GNOME you can remap the Alt+Tab shortcut to fix it.

Installing stuff

The default macOS application installation process of downloading a .dmg file, then opening it, then dragging an icon in a window to the Applications folder feels super odd.

Luckily I was aware of the tool brew and have been using that heavily to get everything that I need installed, in a Linux-y way.

Permissions

I appreciate the concern that macOS has about actions that I take on my laptop, but my god, the permission popups get silly sometimes. When a CLI app is doing things and accessing data on my drive, I can randomly be presented with a permissions pop-up, stealing my focus from writing a Slack message.

Day to day work

Video calls work really well, I can do my full stack engineer things, and overall things work, even if it is sometimes slightly different.

The default Terminal app is not good, I’m still not quite sure why it does not close the window when I exit it, that “Process exited” message is not helpful.

Hardware

No contest, the hardware on a MacBook Pro feels nice and premium compared to the ThinkPad P14s gen 4. The latter now feels like a flexible plastic piece of crap.

The screen is beautiful and super smooth due to the higher refresh rate.

The MacBook does not flex when I hold it.

Battery life is phenomenal, the need to have a charger is legitimately not a concern in 90% of the situations I use a MacBook in.

Keyboard is alright, good to type on, but layout is not my preference.

M3 Pro chip is fast as heck. 18 GB of memory is a solid downgrade from 32 GB, but so far it has not prevented me from doing my work.

I have never heard the fan kick on, even when testing a lot of Go code in dozens of containers, pegging the CPU at 100%, using a lot of memory, and causing a lot of disk writes. I thought that I once heard it, but no, that fan noise was coming from a nearby ThinkPad.

The aluminium case does have one downside: the MacBook Pro is incredibly slippery. I once put it in my backpack and it made a loud thunk as it hit the table that the backpack was on. Whoops.

External displays

macOS does not provide scaling options on my 3440x1440p ultra-wide monitor. Even GNOME has that, with fractional scaling!

The two alternatives are to use a lower resolution (disgusting), or increase the text size across the OS so that I don’t suffer with my poor eyesight.

Apple ID and App Store

Never needed those. I like that. Having used an iPhone for a while, I sort of expected this to be a requirement, but no, you can completely ignore those aspects of macOS and work with a local account. Even Windows 11 doesn’t want to allow that!

Language switching

Switching the keyboard language using the keyboard shortcut is broken about 50% of the time, which feels odd given that it’s something that just works on GNOME. This is quite critical for me since I shift between the Estonian and US keyboard a lot when working, as the US layout has the brackets and all the other important characters in the right places for programming and writing, while Estonian keyboard has all the Õ Ä Ö Ü-s that I need.

The day it fell apart

I upgraded to macOS 26.3 Tahoe on 23rd of February. SSH worked in the morning. Upgrade during lunch, come back, bam, broken.

The SSH logins would halt at the part where public key authentication was taking place, the process just hung. I confirmed that by adding -vvv into the SSH command.

With some vibe-debugging with Claude Code, I found that something with the SSH agent service had broken after the upgrade. One reasonably simple fix was to put this in your .zshrc:

export SSH_AUTH_SOCK=$(mktemp -d)/agent.sock
ssh-agent -a "$SSH_AUTH_SOCK" > /dev/null 2>&1
ssh-add --apple-use-keychain ~/.ssh/id_ed25519 2>/dev/null

Then it works in the shell, but all other git integrations, such as all the repos I have cloned and am using via IntelliJ IDEA, were still broken.

Claude suggested that I build my own SSH agent, and install that until this issue is fixed. That’s when I decided to stop.

macOS was supposed to just work, and not get into my way when doing work. This level of workaround is something I expect from working with Linux, and even there it usually doesn’t get that odd, I can roll back a version of a package easily, or fix it by pulling in the latest development release of that particular package.

Conclusion

I went into this experiment with an open mind, no expectations, and I have to admit that a MacBook Pro with M3 Pro chip is not bad at all, as long as it works.

Unfortunately it doesn’t work for me right now. I might have gotten very unlucky with this issue and the timing, but first impressions matter a lot. The hardware can be nice and feel nice, but if the software lets me down and stops me from doing what’s more important, then it makes the hardware useless.

It turns out that I like Linux and GNOME a lot. Things are simple, improvements are constant and iterative in nature, so you don’t usually notice it (with Wayland and Pipewire being rare exceptions), and you have more control when you need to fix something. Making those one-off solutions like a DIY coding agent sandbox, or a backup script, or setting up snapshots on my workstation are also super easy.

If Asahi Linux had 100% compatibility on all modern M-series MacBooks, then that would be a killer combination.¹

Until then, back to the ol’ reliable ThinkPad P14s gen 4 I go. I can live with fan noise, Bluetooth oddities and Wi-Fi roaming issues, but not with something as basic as SSH not working one day.²

In April 2024, I wrote on the Lenovo ThinkPad P14s gen 4 and how it does not suck under Linux.

That is still true. It’s been fantastic, and a very reliable laptop during all that time.

The P14s gen 4 comes with a CPU that is still solid today, the AMD Ryzen 7 PRO 7840U, and that comes with impressive integrated graphics in the form of an AMD Radeon 780M.

I’ve had a Steam Deck.

I’ve also accidentally built a Steam Machine.

I had to put SteamOS on this laptop to see how well it does. I did a quick Bazzite test the last time around, but after being impressed with how well the stock SteamOS image runs on a random machine with an AMD GPU, I had to test that, too.

Installing SteamOS on USB storage

The normal way to install SteamOS on a machine is to take the Steam Deck recovery image and to install it on your own machine that has one NVMe SSD.

I didn’t want to do exactly that, I wanted to run it off of an USB SATA SSD, which the recovery image does not support, as it hard-codes the target SSD for the SteamOS installation to /dev/nvme0n1. There’s a handy project out there that customizes the recovery script to allow you to install SteamOS to any target device, but I learned about that after the fact.

I went a slightly different route: I imaged the SteamOS installation from my DIY Steam Machine build, wrote it to the 4TB USB SSD that I had available for testing, and after that I resized the /home partition to take up the full disk. Bam, clean SteamOS on a USB SSD!

Oh, and before I did that, I did the same process but to a 128 GB Samsung FIT USB 3.0 thumb drive.

It worked.

The game library images did load a bit slowly, but it was a great demonstration of how low you can go with the hardware requirements. I wouldn’t recommend actually installing games on such a setup as that would likely kill the USB thumb drive very quickly.

Performance

I ran the SteamOS setup on this laptop over a USB-C dock that only supports running at up to 4K at 30Hz, so I did testing at 1080p 60Hz setup. You’re unlikely to want to run this setup at 4K anyway, unless you’re a fan of light, easy to run games like Katamari or Donut County.

In most games, the experience was enjoyable. 1080p resolution, maybe change the settings to medium or low in some cases, and you’ll likely have a solid gaming experience.

Forza Horizon 4? No problem, 1080p high settings and a solid, consistent experience.

Need for Speed Hot Pursuit Remastered was an equally enjoyable experience, and I did not have to turn the settings down from high/ultra.

God of War Ragnarök was pushing the setup to the limits. With 1080p, low/medium settings you can expect 30+ FPS. If you include AMD FSR settings in the mix and also enable FSR frame generation, you can have a perfectly enjoyable 50-60 FPS experience. Some UI hints were a bit “laggy” with frame generation, but I’m genuinely surprised how well that rendering trick worked. I’ll admit it, my eyesight is not the best, but given the choice of a crisp but laggy picture, and a slightly blurrier but smoother experience, I’d pick the latter. After a pint of Winter Stout, you won’t even notice the difference.¹

Wreckfest was also heaps fun. It did push the limits of the GPU at times, but running it at 1080p and medium/high settings is perfectly enjoyable.

The observed power usage throughout the heaviest games measured via SteamOS performance metrics (mangohud) were around 30-40 W, with the GPU using up the most of that budget. In most games, the CPU was less heavily loaded, and in the games that required good single thread performance, it could provide it.

The SteamOS revolution

I like SteamOS. It’s intentionally locked down in some aspects (but you can unlock it with one command), and the Flatpak-only approach to software installation will make some people mad, but I like this balance. It almost feels like a proper console-type experience, almost.

Valve does not officially support running SteamOS on random devices, but they haven’t explicitly prevented it either. I love that.

Take any computer from AMD that has been manufactured from the last 5 years, slap SteamOS on it, and there is a very high chance that you’ll have a lovely gaming experience, with the level of detail and resolution varying depending on what hardware you pick.

A top of the line APU from AMD seems to do the job well enough for most casual gamers like myself, and if the AMD Strix Halo based systems were more affordable, I would definitely recommend getting one if you want a small but efficient SteamOS machine.

Last year, we saw the proliferation of gaming-oriented Linux distros.

The Steam Machine is shipping this year.

DankPods is covering gaming on Linux.

2026 has to be the year of the Linux (gaming) desktop.

This is the Garbage PC.

Not too long ago, I received a half-broken Dell Inspiron N5110, sporting 6 GB of RAM, a dual core Intel i3-2110M, and an unsupported NVIDIA GPU of some sort.

One of the hinges was loose from the case because it was screwed into plastic (common issue for these types of laptops), the touchpad did not work, and to insert a drive into it, you had to disassemble the whole machine.

I’m quite confident that I’ve worked on this laptop model in a very distant past, around 2012-2013, and I remember it well because during the disassembly plastic bits were falling off everywhere and the right hinge was broken in exactly the same way.

I’ve always wanted to take a half-broken laptop and to mount it on some acrylic panels (plexiglass) using brass standoffs. I love how bare PCB-s look. They’re just so damn cool, and I can’t be the only one who thinks this way, right?

I also get a good feeling out of taking trash/obsolete parts and making them useful once again. This laptop was destined for the e-waste pile, which meant that it was a fantastic candidate to try this idea on.

The main constraint in this project was time. I’m a parent, I have a job, and sometimes between all that I like to rest, so the amount of available time for this project was about 8 hours spread across a month. I also lack proper tooling to do a good job, so this was achieved using whatever I had available, mainly a cordless jigsaw and a hand drill. On the upside, this means that if I can do it, then you can likely do it as well!

Disassembly and testing

This laptop turned out to be a total pain to work on.

During initial testing, it was very clear that the laptop needed a good thermal paste and pads replacement, as it tried to overheat playing casual videos off of YouTube.

After I disassembled the laptop to bare essentials and put it together as a test run for the “mount it on acrylic panel” idea, the extension board containing two USB ports and the Ethernet port just stopped working. I probably broke something, but annoying nevertheless. On the positive side, the overall size of the build was smaller as a result of this happy little accident.

Using this laptop via the HDMI output only also turned out to be an unnecessary headache. LibreELEC did not play well with it, often resulting in a blank screen, and on Fedora Workstation 43, there was a “ghost” display somewhere that always showed up on the display settings view. So did the internal display, even when it was disconnected.

This caused an issue when trying to get Fedora installed on this machine, as the installation UI would be placed on a screen that was not the HDMI output one that I was actually using. This issue can be mitigated similar to my LattePanda V1 adventure by disabling video outputs completely. For this laptop, I modified kernel parameters via /etc/default/grub and added the following kernel parameters:

video=VGA-1:d video=LVDS-1:d video=VGA-2:d

Yes, it’s possible to modify the display setup on your desktop environment of choice to disable certain outputs that way, but using kernel parameters ensures that if you change monitors, you won’t have to do that all over again.

To make this whole build even more garbage-tier, I used an 256GB SATA SSD with 5 known bad blocks.

The build

I sourced a large 4mm thick plexiglass panel from a hardware store, as that seemed to be the most accessible place where I can get one. In Estonia, these types of panels are often sold in the gardening sections of general hardware stores.

For attaching the motherboard to the board, I sourced an assortment of M2.5 brass standoffs and screws, and multiple sets in case I need more of a specific height (turned out to be a good call on my part). I chose M2.5 because the laptop used screws of that size, and this size is common in the world of Raspberry Pi and other SBC-s, which can be handy for any future dumb ideas experiments.

To mark the positions on the plexiglass, I put the motherboard assembly on it, marked some good spots with an awl ¹, and then drilled holes using a hand drill and 2.5mm drill bit.

I also sourced heat inserts so that I can melt them into the plexiglass assembly, but those didn’t work out very well. I used my Pinecil soldering iron to push these in to the 2mm pre-drilled holes that I set up for these, but I had alignment issues and the threads ended up getting gunked up by the melted remains of the plexiglass, so I could not screw any brass standoffs in there.

I tried to be very careful with getting the drill holes to line up, and it went mostly alright. My recommendation here is to be precise, and don’t screw everything tight before you’ve got screws and standoffs lined up for all planned holes, otherwise you lose the option of wiggling things a bit to get them to line up.

The standoffs and screws were screwed on tight enough to keep things in place, but not too tight to avoid cracking.

For the other panel, I cut out a similarily sized plexiglass panel, marked the holes again, and repeated the process. Since I was using a cordless jigsaw, I positioned the new piece so that the flat side of the plexiglass panel that I bought lined up with the other straight end on the existing assembly, because I will never get a good straight cut with a freehanded jigsaw. That worked out well enough.

For the power button, I reused the small PCB that contains the power button and power LED-s from the original case. To house that, I drilled a small hole with 1cm diameter to slip the ribbon cable in, and I used small pieces of 3M VHB double-sided tape² to secure it to the panel.

And there you have it, the garbage PC. The shine of the plexiglass does a fantastic job of bringing out the beauty of the motherboard and all its components. Standoffs leave plenty of room for the machine to breathe.

Since the extension board is missing, this build relies heavily on one USB port and one eSATA port that also supports USB connectivity. WiFi, Bluetooth, keyboard/mouse, it’s all over USB 2.0 ports.

One thing that I have yet to do is to add a base to the build so that it does not tip over that easily. Double-sided tape plus a wooden trim piece might do the trick.

The experience

This build is using parts that are about 15 years old. For context, that was when dubstep was popular, it was cool to hate on Justin Bieber, rage comics and bad memes were a thing, and the news in Europe were worried about Greece going bankrupt or something.

That does mean that the performance on this machine is not great.

The machine still runs warm, but not nearly as hot as before. In its stock form and before any thermal paste replacements, it ran about 85+°C, but now it doesn’t seem to ever hit 70°C.

As a basic desktop PC, assuming that you’re not trying to run a 1440p or 4K display, the experience feels completely usable!

If you’re thinking about setting this up as a media player PC, then you’re limited to H.264 playback. H.265 was just too much for this machine. If you use Kodi with Jellyfin, then it is luckily possible to enforce transcoding content to H.264, ensuring a smooth experience on the client side.

Other notes

I wish that this laptop supported a “power on with AC attach” type feature that turns the laptop on once the power adapter is connected, that would’ve made it more useful as a crappy little home server. If you don’t mind extended downtime during a power outage, then it can still do that job well enough, but it’s just something I was slightly annoyed with.

The board has a small SATA port that can be converted to a normal SATA port, plus an eSATA port, making it perfectly plausible to add two drives to this and to totally turn it into a home server.

I tried running Windows 11 on it once, but I tried to do that with the official installer and didn’t get past the “lol your hardware is too old” view. I know that you can remove that limitation, but given the 6GB of RAM and Windows 11 being awful with using resources, it was probably for the best to give up here.

The fan is audible when doing things. I did in fact take it apart and added some silicon oil inside the center to give it a fighting chance and it did improve the acoustics, but it’s something to keep in mind if you’re doing similar projects with these old laptops. There does seem to be a way to control the fan by writing values to dell_smm kernel module controls in /sys/class/hwmon/, and if you overwrite the pwm value often enough, like in a 0.1 second loop, then you can definitely overpower the BIOS fan control on this board. The control does not seem to be fine-tuned, it’s either off, on, or full speed, but at least you can get some control over the fan speed if you really need to. Alternatively, you can slap a huge heat sink on the CPU and GPU if you want to, and you should be getting away with it.

The CPU throttles heavily once you hit 80°C while playing back video, so that seems to be the soft temperature ceiling for this laptop.

Conclusion

Overall, I’m happy I did this project. There were way more obstacles and challenges associated with this project that I expected, but the end result looks cool, so that makes it worth it in my view. It was also a good trial run to work with plexiglass and brass standoffs, and I will very likely do something cooler in the future based on this experience.

I hope that this inspires more people to reuse older hardware instead of just throwing it into the e-waste pile, especially with new computer parts sometimes experiencing price spikes due to the economy doing weird things.

If you’ve built something similar, then do share a link to it (ideally in blog post format) and I will happily link to it here!

2026-02-22 update

There was something missing from the garbage PC.

A garbage OS.

So I went ahead, connected the internal display temporarily, and installed Windows 10 on it. The included activation key did not work on it.

The GPU is confirmed to be a NVIDIA GT 525M. Latest supported drivers are version 391.xx, and had to be downloaded manually.

Unless you use higher resolutions like 3440x1440p, the experience is not bad at all, actually, and Kodi does a superb job playing back videos, even in H.265 format.

I hate to say it, but installing a legacy OS over Linux seems to be the best way forward for this particular combination of Dell and NVIDIA horribleness. Just don’t open any shady links on the internet with it, or connect it to the internet at all.

Microsoft seems to allow you to get security updates on it for a year for free, but only if you sign up with a Microsoft account, otherwise it’s paid. Sneaky. Still not falling for it.

2026 June update

The lack of a connected internal display was a real limiting factor in making this setup useful, so I attached it.

This display has some of the most horrific viewing angles of any display that I’ve used in the past decade, so I’ll probably have to add some adjustable hinges to it to make using it bearable. Alternatively, I’m considering getting the cheapest possible monitor arm that’s freely adjustable and attaching it there to turn this into the crappiest iMac-like experience that you can get.

Some of my dumb experiments result in interesting findings and unexpected successes.

Some end up with very predictable failures.

What happens when you have two crappy USB hard drives running btrfs¹ in raid0 mode? Nothing, until something goes wrong on one of the drives.

Here’s what it looks like:

[188574.681476] BTRFS error (device dm-1): bdev /dev/dm-0 errs: wr 0, rd 0, flush 0, corrupt 5114512, gen 0
[188574.681479] BTRFS warning (device dm-1): csum failed root 719 ino 1133 off 175546368 csum 0x899d8def expected csum 0x6bd987b6 mirror 1
[188574.681481] BTRFS error (device dm-1): bdev /dev/dm-0 errs: wr 0, rd 0, flush 0, corrupt 5114513, gen 0
[188574.681484] BTRFS warning (device dm-1): csum failed root 719 ino 1133 off 175550464 csum 0xab3e9209 expected csum 0xb9ea310f mirror 1
[188574.681486] BTRFS error (device dm-1): bdev /dev/dm-0 errs: wr 0, rd 0, flush 0, corrupt 5114514, gen 0
[188574.681489] BTRFS warning (device dm-1): csum failed root 719 ino 1133 off 175554560 csum 0xc2f14e0c expected csum 0x503c7709 mirror 1
[188574.681491] BTRFS error (device dm-1): bdev /dev/dm-0 errs: wr 0, rd 0, flush 0, corrupt 5114515, gen 0
[188574.681494] BTRFS warning (device dm-1): csum failed root 719 ino 1133 off 175558656 csum 0x6ae8a7c9 expected csum 0xb4c26691 mirror 1
[188574.681496] BTRFS error (device dm-1): bdev /dev/dm-0 errs: wr 0, rd 0, flush 0, corrupt 5114516, gen 0
[188574.681498] BTRFS warning (device dm-1): csum failed root 719 ino 1133 off 175562752 csum 0x97db9766 expected csum 0xbe2e8040 mirror 1
[188574.681500] BTRFS error (device dm-1): bdev /dev/dm-0 errs: wr 0, rd 0, flush 0, corrupt 5114517, gen 0
[188574.681503] BTRFS warning (device dm-1): csum failed root 719 ino 1133 off 175566848 csum 0x86568469 expected csum 0x863fbb4a mirror 1
[188574.681505] BTRFS error (device dm-1): bdev /dev/dm-0 errs: wr 0, rd 0, flush 0, corrupt 5114518, gen 0
[189615.784048] BTRFS warning (device dm-1): csum failed root 719 ino 1181 off 23457792 csum 0xb19058fa expected csum 0x5d11a400 mirror 1
[189615.784086] BTRFS error (device dm-1): bdev /dev/dm-0 errs: wr 0, rd 0, flush 0, corrupt 5114519, gen 0
[189615.784119] BTRFS warning (device dm-1): csum failed root 719 ino 1181 off 23461888 csum 0x256ea900 expected csum 0xabd1eafb mirror 1
[189615.784133] BTRFS error (device dm-1): bdev /dev/dm-0 errs: wr 0, rd 0, flush 0, corrupt 5114520, gen 0
[189615.784153] BTRFS warning (device dm-1): csum failed root 719 ino 1181 off 23465984 csum 0x7b10000b expected csum 0x647fd7bf mirror 1
[189615.784165] BTRFS error (device dm-1): bdev /dev/dm-0 errs: wr 0, rd 0, flush 0, corrupt 5114521, gen 0
[189615.784178] BTRFS warning (device dm-1): csum failed root 719 ino 1181 off 23470080 csum 0xef791959 expected csum 0x175f7adc mirror 1

But in a way, this setup worked exactly as expected.

If you want to have a lot of storage on the cheap, or simply care about performance, or both, then running disks in RAID0 mode is a very sensible thing to do. I used it mainly for having a place where I can store a bunch of data temporarily, such a full disk images or data that I can easily replace.

Now I can test that theory out!

I like the Steam Deck. It’s what convinced me that gaming on Linux is actually viable now.

But after playing through games like God of War Ragnarök¹, I felt like I needed an upgrade. I love playing with the Steam Deck, but what I love more is playing without having to worry about playing around with graphics settings a lot. Great story and gameplay can only hide the fact that you’re running at 720p 30Hz on a big screen for a little bit.

I also get to play relatively rarely, so I might as well make it a better, more enjoyable experience. Quality vs quantity.

The specs

I went on a look-out for a used PC with roughly these requirements:

• any modern 6-core CPU or better
  • includes both Intel and AMD as the CPU does not matter much here²
• an AMD GPU that can do 1080p/4K gaming, depending on the game
  • NVIDIA was out of the question due to lack of support on SteamOS
  • Intel GPUs are a risk that I was not willing to take right now
• 16+GB RAM
• has to support an NVMe drive
  • using the SteamOS recovery image method is dependent on this
• acceptable case, PSU and cooling setup
  • if it does not burn the house down and makes the machine cool and quiet, then I’m fine with anything

The AMD GPU being a hard requirement turned out to be an interesting challenge. I wasn’t looking into putting together a custom build, but was rather going for a setup that works and that I can customize according to my specific needs. Turns out that most of the PC-s out there on the market are all based around NVIDIA GPU-s, and AMD builds of this range are relatively rare, with a guesstimate of the ratio being roughly 10 NVIDIA-based machines to 1 AMD-based machine. The good side of this is that the selection process was made way simpler as I got to choose between 3-4 options in the end.

During my search I also saw some machines that I would call absolute overkill, and I almost got one in a bidding war, but eventually I found a more sensible option. It also included a monitor, keyboard, mouse and three SSD-s that I didn’t really need, but the PC itself was decent.

Here’s what I landed on:

• Intel i5-10500
  • 6 cores 12 threads at a reasonable speed (4.2 GHz in real-life use)
• adequate Cooler Master CPU cooler that does a lot of RGB if needed
• 16 GB DDR4 RAM @ 2666 MT/s
  • I soon upgraded this to 32 GB because my brother had some leftover modules from his own memory upgrade
  • I forced the modules to run at 3200 MT/s. It’s memtest-stable so good enough for me.
• AMD RX 6600XT with 8GB VRAM
  • some might scoff at the VRAM amount, but coming from a Steam Deck where 16GB was shared between CPU and GPU, this is plenty!
• 512GB NVMe SSD
• three 256GB SATA SSD-s
  • previous owner put them in as RAID0, which is clever and works well as a game library
• some Gigabyte motherboard that works
  • it really doesn’t matter here
• some Fractal Design case, possibly a Define-series one
  • all I know is that it’s huuuuuuuuuge

All-in-all, it cost me 365 EUR in Estonia in October 2025, and so far I’ve made about 25 EUR back from the SSD sales alone, with some items still up for sale. It’s not as portable as a Steam Deck, but it’s cheaper even if we account for the cost of the game controller and cables/accessories/adapters that you usually need.

The setup

Regarding the operating system choice, I tried both SteamOS from the Steam Deck recovery image, and Bazzite. Both work fine and in the default couch gaming mode you won’t notice a difference, but I ended up defaulting to SteamOS because I had my setup and configuration changes tuned around that. The SteamOS recovery image approach does assume that you have an NVMe drive available, so if you lack one, you’re better off trying Bazzite as that can be installed on any drive.

I replaced the NVMe SSD with a cheap 128GB one and utilized the bigger drive in the LattePanda IOTA setup that now serves as my home server.

As a game library drive, I took a 1TB Samsung SSD that I had around, which roughly matches the storage that I had available on my Steam Deck that I ended up modding with a 1TB M.2 2230 SSD. With games like God of War Ragnarök taking up around 176GB, it’s not going to be the most luxurious arrangement, but for now it’s okay.

The Fractal case that it came up with was one that is fully metal, with sound dampening material present on the side panels. It’s a bit banged up, but still a pretty nice experience if you have the room for storing one in your setup. The case had one flaw that I stumbled upon: the power button on the Fractal case liked to get stuck, which seems to be a common issue with that model. I fixed that with a random power button that I sourced from a local electronic parts supplier for a few euros and that works really well now, with the additional bonus of it being slightly more cat-proof.

The default fan curves on the motherboard were a bit too aggressive, so I had to slightly tune them down, and now the machine is quiet while doing a great job with keeping the internals cool. You can hear a subtle whirring when you’re in the same room with it, but during gaming it stays at reasonable volumes and is not noticeable. Certainly quieter than a Steam Deck would be.

The gaming experience

The AMD GPU is a low/midrange model, but it gets the job done in 1080p gaming, and with a lot of titles it can do 4K with ease. In God of War Ragnarök I stuck with 1080p and cranked the settings, but with games like Need for Speed Hot Pursuit Remastered, I pushed the resolution to 4K with high/ultra settings, and it runs smoothly at 60Hz.

This setup also taught me that Linux supports HDR now, which was news to me! My tech setup usually lags behind the state of the art, mostly because I don’t really see a need to upgrade to the latest and greatest thing out there if the current one works well enough, but this was a really nice surprise. My TV has a crappy HDR implementation, so I don’t get the full HDR experience, but it’s nice to see the TV show that HDR logo when I start up the machine.

Regarding the gaming experience, I’ve only noticed a few sore spots.

For whatever reason, the Need for Speed (2015) just does not start up on anything but an actual Steam Deck. It just doesn’t work here. I can’t be arsed to investigate this yet, the wonky physics in this game are perhaps not worth that effort.

It’s also clear that the choice of an Intel CPU is generally fine, but in God of War Ragnarök it was running too well, so the CPU kept dropping down to lower clock speeds, which then made the game performance inconsistent. Finding that this was the issue was actually quite straightforward: when I first loaded the game, the shader compilation was taking place in the background and even though the CPU was at a constant 100% usage, the game ran quite smoothly. It only started stuttering after that was done, and the integrated mangohud setup helped confirm the issue as its most detailed preset shows the frame time and CPU clock speed graphs really well.

Since this is just a Linux box, then you can of course run a few commands to fix it.³

Here’s how I fixed it.

Create a desktop entry at /home/deck/.local/bin/its-gaming-time.desktop with the contents:

[Desktop Entry]
Name=Gaming Time!
Exec=/home/deck/.local/bin/its-gaming-time
Comment=
Icon=
Type=Application
Terminal=true

Create a file /home/deck/.local/bin/its-gaming-time with the contents:

#!/bin/bash
# Set performance mode
echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

# Set min_perf_pct to 100 (forces min freq = max freq)
echo 100 | sudo tee /sys/devices/system/cpu/intel_pstate/min_perf_pct

Don’t forget to mark the script as executable with chmod +x /home/deck/.local/bin/its-gaming-time.

Note that the script above does require that you have set up passwordless sudo on the SteamOS installation. This can be configured in /etc/sudoers.d/wheel, just make sure that the line starting with %wheel looks like this:

%wheel ALL=(ALL) NOPASSWD: ALL

With all that set up, in desktop mode, right-click on the desktop shortcut, “Add to Steam”, and now you can run this script any time in Steam gaming mode, even while a game is running!

All-in-all, I’m very satisfied with the experience that a cheap gaming PC box provides with SteamOS. The installation is painless, my wireless controllers just work, and aside from a few rare exceptions, my games run really well.

It’s also way easier on my eyes and with the 4K resolution I can actually see oncoming cars better in games like Need for Speed Hot Pursuit Remastered.⁴

The GabeCube

Less than three weeks after buying that gaming PC, the Steam Machine was officially announced.

The rumored specs suggesting a 6 core/12 thread CPU, 16GB DDR5 RAM, and a custom 8GB VRAM AMD GPU that seems to be roughly comparable to an AMD RX 6600XT-ish level of performance.

It seems that I have accidentally built a Steam Machine. Oops.

Of course, the specs and final performance are not public at the time of writing, and the Steam Machine has many benefits (better SteamOS compatibility, good WiFi, smaller size, likely more efficient and quiet), but it’s still interesting how close I got with my setup and selection criteria.

I was slightly disappointed that I got this machine right before that announcement, but then I reminded myself of the fact that I can enjoy games on the big screen right now, and the Steam Machine is scheduled for a release in Q1 2026, which can be as late as 31st of March 2026.

And hey, when the Steam Machine does come out and I decide to get one, the current gaming desktop will make for a very good home server candidate with all the room that it has available, and all the six SATA ports on the motherboard sure look tempting. I’m pretty sure that the Fractal case also allows something crazy like 17+ hard drives installed in it.

This approach of building my own Steam Machine of sorts did lead to me selling my Steam Deck. Better to have someone else enjoy it than having it sit in a box until its battery dies. That also serves as a major sign of confidence for this big box that makes my sparse downtime sessions more fun.

If you have a machine with a modern AMD GPU, then give SteamOS a try, you might be surprised at how well it works. Even a laptop with an AMD APU can do it, as long as you temper your expectations regarding the image quality.

Start of experiment: September 2025

Post last updated: October 2025

Check the live status of my Orange Pi Zero here!

I love the Orange Pi Zero. It’s tiny, uses very little power and it’s just neat! It’s also the subject of the very first post on my blog, which makes it a bit special.

Unfortunately I haven’t really found a good use case for it, given that its performance is quite limited and the CPU is a 32-bit ARM CPU with 4 relatively weak cores, which rules out using it as a Docker container host due to the architectural limitations.

The board is currently not doing much, but I’ll figure out a use case for it.

The board also has a few quirks at the moment that I’ve worked around. For example, rebooting seems to be broken, and it’s unlikely to get fixed any time soon. I resolved it by simply not rebooting it¹, at least not on a regular schedule, anyway. I’m hoping that any short-term power outages at home will take care of the need to reboot it.

It also runs quite hot in its stock Armbian configuration, which I worked around by running powertop --auto-tune on startup as that forces the CPU to always run at its slowest clock speed (480 MHz). Without it, I found that this board can run its CPU at 105°C, and it does have a thermal shutdown feature.

This board has been featured in a few previous posts as well, and even then it was quite underpowered:

• the little Wi-Fi AP that could
• seedbox on a wall
• database optimization adventures on low-end hardware

Now that I have it set up, will it be able to survive to year 2038 and beyond? Only time will tell.

Which issue will we run into first? Plausible options based on my previous experience:

• the cheap 8GB SD card craps out
• component on the board dies from heat-related issues
• the USB power supply dies
  • it’s currently running off of an IKEA extension cord that has two USB-A ports, so hopefully not
• board loses Armbian support completely (currently under community maintenance status)

The board is currently running the latest version of Armbian, and I might occasionally refresh its version from time to time.

If you like Armbian, then please support them! They’re doing great work with keeping all sorts of SBC-s up and running with usable versions of Debian and Ubuntu Linux.

2025-10-20 update

What are the chances that the day I make this project public, AWS suffers a massive outage? Poetic in a way. And yes, my website and the Orange Pi Zero were fully operational.

I liked Ubuntu. For a very long time, it was the sensible default option. Around 2016, I used the Ubuntu GNOME flavor, and after they ditched the Unity desktop environment, GNOME became the default option.

I was really happy with it, both for work and personal computing needs.

Estonian ID card software was also officially supported on Ubuntu, which made Ubuntu a good choice for family members.

But then something changed.

Upgrades suck

Like many Ubuntu users, I stuck to the long-term support releases and upgraded every two years to the next major version. There was just one tiny little issue: every upgrade broke something.

Usually it was a relatively minor issue, with some icons, fonts or themes being a bit funny. Sometimes things went completely wrong.

The worst upgrade was the one I did on my mothers’ laptop. During the upgrade process from Ubuntu 20.04 to 22.04, everything blew up spectacularly. The UI froze, the machine was completely unresponsive. After a 30-minute wait and a forced restart later, the installation was absolutely fucked. In frustration, I ended up installing Windows so that I don’t have to support Ubuntu.

Another family member, another upgrade. This is one that they did themselves on Lubuntu 18.04, and they upgraded to the latest version. The result: Firefox shortcuts stopped working, the status bar contained duplicate icons, and random errors popped up after logging in. After making sure that ID card software works on Fedora 40, I installed that instead. All they need is a working browser, and that’s too difficult for Ubuntu to handle.

Snaps ruined Ubuntu

Snaps. I hate them.

They sound great in theory, but the poor implementation and heavy-handed push by Canonical has been a mess.

Snaps auto-update by default. Great for security¹, but horrible for users who want to control what their personal computer is doing.

Snaps get forced upon users as more and more system components are forcibly switched from Debian-based packages to Snaps, which breaks compatibility, functionality and introduces a lot of new issues. You can upgrade your Ubuntu installation and then discover that your browser is now contained within a Snap, the desktop shortcut for it doesn’t work and your government ID card does not work for logging in to your bank any longer.

Snaps also destroy productivity. A colleague was struggling to get any work done because the desktop environment on their Ubuntu installation was flashing certain UI elements, being unresponsive and blocking them from doing any work. Apparently the whole GNOME desktop environment is a Snap now, and that lead to issues.

The fix was super easy, barely an inconvenience:

• roll back to the previous version of the GNOME snap
• restart
  • still broken
• update to the latest version again
• restart
  • still broken
• restart again
  • it is fixed now

What was the issue?

Absolutely no clue, but a days’ worth of developers’ productivity was completely wasted.

Some of these issues have probably been fixed by now, but if I executed migration projects at my day job with a similar track record, I would be fired.²

Snaps done right: Flatpak

Snaps can be implemented in a way that doesn’t suck for end users. It’s called a Flatpak.

They work reasonably well, you can update them whenever you want and they are optional. Your Firefox installation won’t suddenly turn into a Flatpak overnight.

On the Steam Deck, Flatpaks are the main distribution method for user-installed apps and I don’t mind it at all. The only issue is the software selection, not every app is available as a Flatpak just yet.

Consider Fedora

Fedora works fine.

It’s not perfect, but I like it.

At this point I’ve used it for longer than Ubuntu and unless IBM ruins it for all of us, I think it will be a perfectly cromulent distro go get work done on.

Hopefully it’s not too late for Canonical to reconsider their approach to building a Linux distro.

In November 2024, my blog was down for over 24 hours.

Here’s what I learned from this absolute clusterfuck of an incident.

Lead-up to the incident

I was browsing through photos on my Nextcloud instance. Everything was fine, until Nextcloud started generating preview images for older photos. This process is quite resource intensive, but generally manageable. However, this time the images were high quality photos in the 10-20 MB size range.

Nextcloud crunched through those, but ended up spawning so many processes that it ended up using all the available memory on my home server.

And thus, the server was down.

This could have been solved by a forced reboot. Things were complicated by the simple fact that I was 120 kilometers away from my server, and I had no IPMI-like device set up.

So I waited.

50 minutes later, I successfully logged in to my server over SSH again! The load averages were in the three-digit realm, but the system was mostly operational.

I thought that it would be a good idea to restart the server, since who knows what might’ve gone wrong while the server was handling the out-of-memory situation.

I reboot.

The server doesn’t seem to come back up. Fuck.

The downtime

The worst part of the downtime was that I was simply unable to immediately fix it due to being 120 kilometers away from the server.

My VPN connection back home was also hosted right there on the server, using this Docker image.

I eventually got around to fixing this issue the next day when I could finally get hands-on with the server, my trusty ThinkPad T430. I open the lid and am greeted with the console login screen. This means that the machine did boot.

I log in to the server over SSH and quickly open htop. My htop configuration shows metrics like systemd state, and it was showing 20+ failed services. This is very unusual.

lsblk and mount show that the storage is there. What was the issue?

Well, apparently the Docker daemon was not starting. I was searching for the error messages and ended up on this GitHub issue. I tried the fix, which involved deleting the Docker folder with all the containers and configuration, and restarted the daemon and containers. Everything is operational once again.

I then rebooted the server.

Everything is down again, with the same issue.

And thus began a 8+ hours long troubleshooting session that ran late into the night. 04:00-ish late, on a Monday.

I tried everything that I could come up with:

• used the btrfs Docker storage driver instead of the default overlay one
  • Docker is still broken after a reboot
• replaced everything with podman
  • I could not get podman to play well with my containers and IPv6 networking
• considered switching careers
  • tractors are surprisingly expensive!

I’m unable to put into words how frustrating this troubleshooting session was. The sleep deprivation, the lack of helpful information, the failed attempts at finding solutions. I’m usually quite calm and very rarely feel anger, but during these hours I felt enraged.

The root cause

The root cause will make more sense after you understand the storage setup I had at the time.

The storage on my server consisted of four 4 TB SSD-s, two were mounted inside the laptop, and the remaining two were connected via USB-SATA adapters. The filesystem in use was btrfs, both on the OS drive and the 4x 4TB storage pool. To avoid hitting the OS boot drive with unnecessary writes, I moved the Docker data root to a separate btrfs subvolume on the main storage pool.

What was the issue?

Apparently the Docker daemon on Fedora Server is able to start up before every filesystem was mounted. In this case, Docker daemon started up before the subvolume containing all the Docker images, containers and networks was mounted.

I tested out this theory by moving the Docker storage back to /var/lib/docker, which lives on the root filesystem, and after a reboot everything remained functional.

In the past, I ran a similar setup, but with the Docker storage on the SATA SSD-s that are mounted inside the laptop over a native SATA connection. With the addition of two USB-connected SSD-s, the mounting process took longer for the whole pool, which resulted in a race condition between the Docker daemon startup and the storage being mounted.

Fixing the root cause

The fix for Docker starting up before all of your storage is mounted is actually quite elegant.

The Docker service definition is contained in /etc/systemd/system/docker.service. You can override this configuration by creating a new directory at /etc/systemd/system/docker.service.d and dropping a file with the name override.conf in there with the following contents:

[Unit]
RequiresMountsFor=/containerstorage

The rest of the service definition remains the same and your customized configuration won’t be overwritten with a Docker version update. The RequiresMountsFor setting prevents the Docker service from starting up before that particular mount exists.

You can specify multiple mount points on the same line, separated by spaces.

[Unit]
RequiresMountsFor=/containerstorage /otherstorage /some/other/mountpoint

You can also specify the mount points over multiple lines if you prefer.

[Unit]
RequiresMountsFor=/containerstorage 
RequiresMountsFor=/otherstorage 
RequiresMountsFor=/some/other/mountpoint

If you’re using systemd unit files for controlling containers, then you can use the same systemd setting to prevent your containers from starting up before the storage that the container depends on is mounted.

Avoiding the out of memory incident

Nextcloud taking down my home server for 50 minutes was not the root cause, it only highlighted an issue that had been there for days at that point. That doesn’t mean that this area can’t be improved.

After this incident, every Docker Compose file that I use includes resource limits on all containers.

When defining the limits, I started with very conservative limits based on the average resource usage as observed from docker stats output.

Over the past few months I’ve had to continuously tweak the limits, especially the memory ones, due to the containers themselves running out of memory when the limits were set too low. Apparently software is getting increasingly more resource hungry.

An example Docker Compose file with resource limits looks like this:

name: nextcloud
services:
  nextcloud:
    container_name: nextcloud
    volumes:
      - /path/to/nextcloud/stuff:/data
    deploy:
      resources:
        limits:
          cpus: "4"
          memory: 2gb
    image: docker.io/nextcloud:latest
    restart: always
  nextcloud-db:
    container_name: nextcloud-db
    volumes:
      - /path/to/database:/var/lib/postgresql/data
    deploy:
      resources:
        limits:
          cpus: "4"
          memory: 2gb
    image: docker.io/postgres:16
    restart: always

In this example, each container is able to use up to 4 CPU cores and a maximum of 2 GB of memory. And just like that, Nextcloud is unable to take down my server by eating up all the available memory.

Yes, I’m aware of the Preview Generator Nextcloud app. I have it, but over multiple years of running Nextcloud, I have not found it to be very effective against the resource-hungry preview image generation happening during user interactions.

Decoupling my VPN solution from Docker

With this incident, it was also clear that running your gateway to your home network inside a container was a really stupid idea.

I’ve mitigated this issue by taking the WireGuard configuration generated by the container and moving it to the host. I also used this as an opportunity to get a to-do list item done and used this guide to add IPv6 support inside the virtual WireGuard network. I can now access IPv6 networks everywhere I go!

I briefly considered setting WireGuard up on my openWRT-powered router, but I decided against it as I’d like to own one computer that I don’t screw up with my configuration changes.

Closing thoughts

I have not yet faced an incident this severe, even at work. The impact wasn’t that big, I guess a hundred people were not able to read my blog, but the stress levels were off the charts for me during the troubleshooting process.

I’ve long advocated for self-hosting and running basic and boring solutions, with the main benefits being ease of maintenance, troubleshooting and low cost. This incident is a good reminder that even the most basic setups can have complicated issues associated with them.

At least I got it fixed and learned about a new systemd unit setting, which is nice.

Still better than handling Kubernetes issues.

I was in a pinch.

I needed to make a full disk backup of a PC, but I had no external storage device with me to store it on. The local Wi-Fi network was also way too slow to transfer the disk over it.

All I had was my laptop with an Ethernet port, a Fedora Linux USB stick, and a short Ethernet cable.

I took the following steps:

• boot the target machine up with the Fedora Linux installer in a live environment
• modify the SSH configuration on the target machine to allow root user login with a password
  • it’s OK to do this on a temporary setup like this one, but don’t do it on an actual Linux server
• set a password for the root user on the target machine
  • only required because a live environment usually does not set one for root user
• connect both laptops with the Ethernet cable
• set static IPv4 addresses on both machines using network settings ¹
  • edit the “Wired” connection and open the IPv4 tab
  • example IP address on target: 192.168.100.5
  • example IP address on my laptop: 192.168.100.1
  • make sure to set the netmask to 255.255.255.0 on both!
• verify that the SSH connection works to the target machine
• back up the disk to your local machine using ssh and dd
  • example: ssh root@192.168.100.5 "dd if=/dev/sda" | dd of=disk-image-backup.iso status=progress
  • replace /dev/sda with the correct drive name!

And just like that, I backed up the 120 GB SSD at gigabit speeds to my laptop.

I’ve used a similar approach in the past when switching between laptops by running a live environment on both machines and copying the disk over with dd bit by bit.

You’ll also save time on not having to copy the data over twice, first to an external storage device, and then to the target device.

I’ve recently made an effort to reduce the amount of tech stuff that I own. Lack of space, lack of time and the guilt of storing perfectly functional hardware unused in a box were the main motivators.

This has resulted in experiments with my assortment of USB flash storage that I’ve acquired over the years. They’re too old and cheap to be worth selling, so why not run them to the ground?

I’ve always liked the idea of USB sticks. They’re small and the good ones are actually surprisingly fast, especially compared to hard drives. This is why I started installing operating systems on them. It’s really handy to have a USB stick around with a full operating system on it for testing purposes, and I’m too cheap to buy an IODD device.¹

What I’ve learned is that Linux installed on a USB stick is a much nicer experience compared to a Windows To Go installation, even on the cheaper and crappier ones.

The Windows experience

I’ve used Rufus for creating portable Windows installations in the past, mostly on portable SSD-s. These have come in handy when I’ve had to update the firmware on certain PC-s or associated hardware, like USB-C docks.²

Recently, I took a Samsung FIT 128GB USB 3.0 drive to see how well Windows can operate off of it.

It’s horrible.

Windows will run, but the first time you boot it, you will have to wait for all sorts of driver and update installations, which puts a huge strain on the storage, making the whole experience borderline unusable. Once that process is complete (multiple hours later), it’s actually not that bad, until you receive yet another Windows update.

For a USB stick that you use only occasionally to troubleshoot hardware, Windows is a really poor choice. Even on a proper portable SSD, it will still be relatively slow due to all that driver and update installation taking place in the background.

Eventually, that installation will also probably brick itself in one way or another, and not even due to any storage failures. Simply using the same disk on multiple machines will accrue configuration and driver cruft that likely results in Windows giving up. It’s just a guesstimate on my side, but I would not be surprised if it was true.

After a Windows update run completely fried one USB stick³ that I used for testing, I gave up on it.

The Linux experience

Most Linux distributions have supported a LiveCD/LiveUSB solution for at least 15+ years at this point. It’s a neat idea: your installation media also doubles as an emergency portable installation of that Linux distro, which you can use to quickly test the functionality of a PC, repair an existing installation or install it on a new machine.

The downside of that solution is that the software on it cannot be changed. You can temporarily install new software or update existing packages during that session, or have a separate writeable partition set aside on the USB stick for storing any permanent changes, but it’s a bit of a hassle to think about that nuance every time.

This is why I went ahead and installed Fedora Linux 40 on a Sandisk Ultra 16GB USB 3.0 flash drive (the one that is actually 16GB). The storage speed is still a bottleneck, but it’s much less noticeable during normal use.

After the installation is done, you’re simply using the system as-is. Opening programs will have a slightly longer delay, but it’s still a perfectly cromulent Linux experience.

A fresh installation of Fedora Workstation 40 uses only 3.6GB of storage! On a 16GB USB flash drive, this leaves plenty of room for performing system updates and installing a small number of applications. If you choose the btrfs filesystem during the installation (the default option), then the files will also be compressed to save even more space and increase the effective storage bandwidth.⁴

You’ll only notice that you’re running off of a USB flash drive when installing new packages, performing system updates or downloading large files. At least with Linux you have control over when you want to install the updates.

Using btrfs also allows you to easily check the flash drive for any integrity issues. Simply run sudo btrfs scrub start -B / and you’ll know soon enough if your flash drive is messing with your data.

The backerupper USB stick

One actual use case that I have for a full Linux installation on a USB stick is the backerupper solution: a fully featured Fedora Linux installation, stored on a small USB stick, and with limited access to networked storage.

Most people working in IT know the joys of working with hardware that family members and relatives own. Before doing any destructive changes, it’s always a good idea to make a full disk copy of the storage on the machine you’re fixing. If you screw up, you’ll have a way of setting everything back just as it was before you did any work on it. A full disk image will also mean that nothing will be left behind before you do a fresh installation.

I already have a home server with some spare storage, so using it as a target for any backups makes perfect sense to me.

This solution has already come in handy once when I had to deal with a 10-year-old laptop with a dying hard drive.⁵

Crappy storage is still crappy

Don’t expect any kind of longevity with USB sticks, the wear leveling and endurance is not on the level of proper SSD-s. Linux is nicer to the USB stick regarding the wear and tear, but given enough time, the USB flash drive will still fail.

I will still keep using USB flash sticks for this purpose. I have backups of the USB stick installations on my server, so restoring it will simply mean getting another USB stick and writing the full disk image back to it.

USB sticks are also great for specialized solutions that don’t do a lot of IO operations. LibreELEC, a media-center oriented Linux distro, can be run off of a USB flash drive without any major issues, since its storage demands are very small (unless you install a ton of plugins).

Conclusion

Linux and USB flash storage is a great option for making backups or having a working operating system available for disaster recovery scenarios. Just mind the endurance of the flash storage, and don’t use Windows.

Imagine that you have an unencrypted drive containing your private data and one day it starts throwing a bunch of errors. You have backups of the data so you’ve got that part covered, but would you feel comfortable sending the drive in to be warrantied? You have no control over who has access to that drive, and due to the drive failing you can’t format it as well.

Do you take the financial hit and buy a new drive, or send it in regardless and risk someone looking through your files?

I’ve bought and sold a bunch of hard drives and SSD-s over the years¹, and warrantied a few of them. I encrypt the disk on my personal and work machines because of the obvious security benefits, but for a long time I avoided doing the same for other storage devices.

Then I realized that I can just encrypt them and use almost anything as the key. All you need is a reasonably sized file to pass to cryptsetup as a key-file, refer to that key-file in /etc/crypttab, and you’re good to go!

Be creative! The key file can be anything:

• a dank meme
• a very short low resolution cat video
• an adorable photo of your dog
• a precious family photo
• Twilight fan fiction that you wrote back in 2011
• your letter of resignation at your last job

Just mind the size, cryptsetup wasn’t very happy with a 17MB full resolution image. A JPEG that’s less than a megabyte in size worked well enough for me.

On Linux, you can encrypt a partition with a cat picture using a command like this:

cryptsetup luksFormat /dev/sdx1 --key-file /path/to/cat.jpg

You can make sure that the drive gets automatically decrypted at boot by defining it in /etc/crypttab. Find the UUID of the encrypted partition by running ls -lah /dev/disk/by-uuid and see which one matches with /dev/sdx1. If the UUID is ef277bc2-d953-44c4-88af-8320aca76969, then a line in /etc/crypttab would look like this:

encryptedcatdrive UUID=ef277bc2-d953-44c4-88af-8320aca76969 /path/to/cat.jpg 

Once unlocked, your decrypted partition will be available as /dev/mapper/encryptedcatdrive.

Just make sure that the key file isn’t placed on the encrypted drive itself, otherwise you’ll lock yourself out for good.

For more information on LUKS disk encryption and its capabilities, see this handy Arch Wiki page.

This approach comes with some great benefits and a few downsides.

On the bright side, you don’t have to worry about a stranger getting their hands on your data when selling a storage device or sending it in to be warrantied, assuming that you haven’t posted the key file on social media or anywhere else.² Just do a quick format of the drive, or format a very specific part to clear the LUKS headers, and the contents of the drive are gone!

If you can’t format the drive due to hardware issues, then you should still be safe, the attacker would have to first fix the drive (which isn’t a guaranteed success) and then figure out the key, which will require a lot of effort and time.

The one obvious downside is that if you lose the key file, you lose the data. You can mitigate this by either adding multiple key files as a backup option, or setting a strong password that you can use in case you lose your original key file.

If the drive contains something of value to your family members, then it may be a good idea to specify those details in a will, or by taping the backup password with instructions on the drive itself. If you end up selling the drive before your personal expiration date arrives, then you can simply remove the instructions. Just make sure that your family members or relatives know to plug in the drive to a Linux machine.³

I also like the thrill of hiding a key in plain sight. Did I use the cover photo of this post to encrypt my drives? You’ll never know, and that’s what makes it fun!

If you’re paranoid or targeted by state actors, then you probably shouldn’t follow this advice, except the part where I encourage encrypting stuff.

I like Fedora Linux.

It’s the Linux distro that stopped my habit of distro-hopping. Big deal? Ooh, big deal!

It’s stable in day-to-day use, ships with up-to-date software, and the software selection is adequate out of the box.

It also ships with a fresh Linux kernel version¹, resulting in a good experience even on modern hardware.

I’ve been happily using it for about 4 years at this point, both on personal and work computers.

However, every time I recommend it to my friends, family and colleagues, I have to mention a few things that a new Fedora Linux user might want to do first so that the experience is as smooth as possible.

If it weren’t for those aspects, Fedora would be pretty much perfect.

Set up RPM Fusion

There are some packages that are not available on Fedora by default, likely as a result of copyright laws, lawyers and Red Hat (IBM) not having the appetite to fight it out in the courts.

This is why RPM Fusion exists:

RPM Fusion provides software that the Fedora Project or Red Hat doesn’t want to ship.

The installation instructions are here, and I personally prefer this one-liner setup:

sudo dnf install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm

This step is a prerequisite to the next one.

Enable hardware accelerated video playback

This is the part where I blame lawyers.

To have a good chance at smoothly playing back videos in a browser or using VLC, you have to do a few extra steps, because someone fucked up.

Check this guide for up-to-date information.

Switch to the full version of ffmpeg:

• sudo dnf swap ffmpeg-free ffmpeg --allowerasing

AMD GPU users:

• sudo dnf swap mesa-va-drivers mesa-va-drivers-freeworld
• sudo dnf swap mesa-vdpau-drivers mesa-vdpau-drivers-freeworld

Intel GPU users (Broadwell and newer, 2014+):

• sudo dnf install intel-media-driver

Old Intel GPU users (Ivy Bridge and older, <=2013):

• sudo dnf install libva-intel-driver

NVIDIA GPU users: good luck.²

You can verify the enabled profiles using vainfo.

Here’s the output on my machine before completing the steps (AMD GPU).

Note the lack of H.264 and HEVC profiles.

me@mymachine:~$ vainfo
Trying display: wayland
libva info: VA-API version 1.21.0
libva info: Trying to open /usr/lib64/dri-nonfree/radeonsi_drv_video.so
libva info: Trying to open /usr/lib64/dri-freeworld/radeonsi_drv_video.so
libva info: Trying to open /usr/lib64/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_21
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.21 (libva 2.21.0)
vainfo: Driver version: Mesa Gallium driver 24.1.7 for AMD Radeon 780M (radeonsi, gfx1103_r1, LLVM 18.1.6, DRM 3.57, 6.10.8-200.fc40.x86_64)
vainfo: Supported profile and entrypoints
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileVP9Profile0            :	VAEntrypointVLD
      VAProfileVP9Profile2            :	VAEntrypointVLD
      VAProfileAV1Profile0            :	VAEntrypointVLD
      VAProfileAV1Profile0            :	VAEntrypointEncSlice
      VAProfileNone                   :	VAEntrypointVideoProc

Here’s what it looks like after the tweaks are applied.

me@mymachine:~$ vainfo
Trying display: wayland
libva info: VA-API version 1.21.0
libva info: Trying to open /usr/lib64/dri-nonfree/radeonsi_drv_video.so
libva info: Trying to open /usr/lib64/dri-freeworld/radeonsi_drv_video.so
libva info: Trying to open /usr/lib64/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_21
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.21 (libva 2.21.0)
vainfo: Driver version: Mesa Gallium driver 24.1.7 for AMD Radeon 780M (radeonsi, gfx1103_r1, LLVM 18.1.6, DRM 3.57, 6.10.8-200.fc40.x86_64)
vainfo: Supported profile and entrypoints
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileHEVCMain10             :	VAEntrypointEncSlice
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileVP9Profile0            :	VAEntrypointVLD
      VAProfileVP9Profile2            :	VAEntrypointVLD
      VAProfileAV1Profile0            :	VAEntrypointVLD
      VAProfileAV1Profile0            :	VAEntrypointEncSlice
      VAProfileNone                   :	VAEntrypointVideoProc

Enable Flathub

This step is only necessary if you did not enable third-party repositories during the inital setup of your Fedora Linux installation.

Setting up Flathub is just one command:

flatpak remote-add --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo

Check the Flathub setup instructions for Fedora for up-to-date info.

🇪🇪 Specific to Estonia: ID card support

Estonia has an ID Card system (Eesti ID kaart), and the desktop application is also officially supported on Linux. Ubuntu Linux.

To use the ID card in Fedora, you’ll need to install the necessary packages:

sudo dnf install -y open-eid

Firefox users will need to make sure that the Web eID extension is installed. If it wasn’t installed with the above command, you can also get it here.

The desktop application used for opening and digitally signing documents, DigiDoc4, has a bad habit of breaking sometimes due to changes. Luckily there exists an unofficial version of DigiDoc4 as a Flatpak and it’s available on Flathub.

It’s mentioned in the official open-eid/linux-installer repo as well, which lends some credibility (plus I know the person that built it).

As of 2024-10-14, ID card works on Firefox and the DigiDoc4 client from the Fedora official repository does not crash and burn.

GNOME: fix the “Alt+Tab” behaviour

GNOME is a great desktop environment, but it, too, has its quirks.³

If you’re used to “Alt+Tab” keyboard shortcut simply switching windows, then you’ll have to tweak it in settings.

• Open “Settings”
• Navigate to “Keyboard”
• Click on “View and Customize shortcuts”
• Use the search bar to:
  • disable “Switch applications” shortcut
  • set the “Switch windows” shortcut to “Alt+Tab”

Here’s what it looks like on GNOME 46.

GNOME: minimize, maximize buttons on windows

By default, you’ll only see the “Close” button on windows, but it is possible to also show minimize and maximize buttons.

Install GNOME tweaks with sudo dnf install -y gnome-tweaks.

Open it, navigate to Windows, and toggle Maximize and Minimize settings.

GNOME: app status indicators

Some apps, such as Solaar, like to run in the background and show you a status indicator on your taskbar. This functionality is not present on GNOME by default, but you can easily fix it with the AppIndicator and KStatusNotifierItem Support extension.

GNOME: corner tiling

As of GNOME 46, you cannot do corner tiling out of the box. The best you can do is split windows horizontally, which is fine for single ultra-wide monitor setups. There are situations where corner tiling comes in really handy.

The best solution I have found so far seems to be the Tiling Assistant extension. It’s configurable and will probably suit your workflow well.

At the time of writing, it does have a few small but annoying enough bugs that are a show-stopper for me, but compared to other solutions this one at least works.

Optional: disable SELinux

There are two types of people: those that swear by SELinux, and those that swear due to SELinux.

If you’re running into weird issues and you don’t know why, then see if SELinux is to blame.

I run into issues with SELinux every time I try to move files around with rsync, resulting in a simple rsync -aAXv source/ destination/ command becoming rsync -aAXv --filter='-x security.selinux' source/ destination/. It’s frustrating.

To fix this, I simply disable SELinux. Open up /etc/selinux/config, find the line SELINUX=enforcing and change it to SELINUX=disabled.

You’ll need to restart your machine for it to take effect.

Don’t use ZFS

ZFS is a great filesystem, but not on distros that ship new kernel versions.

This is due to OpenZFS needing to be built against the Linux kernel, and new kernel releases often break compatibility. Those breakages may lead to your ZFS pools not being imported at boot as a result.

You can use ZFS if you use LTS kernels built by someone else, or if you simply don’t update your system, but I don’t recommend either approach.

Use a different distribution, such as Debian, if you really need to use ZFS daily.

If you don’t care for ZFS but like data integrity checks, snapshots and don’t use a lot of disks in an array, then btrfs is totally fine.

Don’t use NVIDIA GPU-s

I’ve never had a good experience with the open source driver nouveau, and the proprietary drivers are a headache and break things.

This might be about to change, but I’ll believe it when I see it.

Conclusion

I hope that this has made your Fedora Linux experience just a little bit better.

Here’s to hoping that IBM does not fuck up Fedora Linux. My expectations are low.

If you have any other tips to share then please contact me! You can find the contact details below.

I helped a family member upgrade to a newer iPhone and make some room so that the internal storage does not run out.

They had Nextcloud installed on the current phone, but due to limitations of the Nextcloud iOS app, the backups only take place if the app itself is open, meaning that we had hundreds of photos and videos that were not yet backed up.¹

The network was slow and I was in a time crunch, so I opted to copy the media off of the phone by connecting the iPhone to my laptop over a Lightning cable. After entering the PIN on the phone again and approving the connection, I could see the iPhone in my file manager, but opening it would show nothing at all. This was very unusual as other iPhones I used in the past just worked on my laptop.

Rebooting the phone and connecting it again did nothing. Different known good Lightning cables did not improve things, either. Even a separate Windows 10 machine could not see any images, so it wasn’t a Linux thing.

Various Apple-related posts online were devoid of useful advice.²

I was not ready to give up just yet and I finally stumbled upon this Ask Ubuntu answer. This had potential, so I gave it a go, and I’m happy to report that it worked on my Fedora Linux machine!

These are the steps that I took. The commands need to be run in a terminal window.

Tested with Fedora Linux 40.

• install the necessary packages
  • on Fedora, run sudo dnf install -y ifuse libimobiledevice libimobiledevice-utils
• connect your iPhone to your PC
  • if your phone is not yet paired, then run idevicepair pair
  • if it’s already paired, verify the connection by running idevicepair validate
• mount your phone to an empty folder of your choice
  • if an empty folder does not exist, run mkdir ~/my-iphone (use any folder name that you want)
  • to mount the phone, run ifuse ~/my-iphone
• you can now browse your media via your file manager or terminal by navigating to ~/my-iphone
  • photos and videos taken by the camera will be in the DCIM folder
  • other folders, such as Photos and Downloads may also interest you
  • to make a full copy of your phone to a folder named iphone-backup, run cp -r ~/my-iphone ~/iphone-backup
  • alternative way to create a full backup: rsync -aAXv ~/my-iphone ~/iphone-backup
• once you’re done, unmount the iPhone by running umount ~/my-iphone
• it is now safe to disconnect your iPhone from the PC

If you have copied everything over and want to make more space on your phone using this method, then you need to delete files under the DCIM folder and the cached thumbnails under the PhotoData folder. iOS seems to generate thumbnails for the photos you take and even if you delete all the actual photos off of the device, then the photo thumbnails will still show up in the Photos app and elsewhere. This confused the hell out of me after I deleted the backed up photos off the phone the first time.

Alternatively, you can just delete files from the Photos app on the phone itself to avoid this nuance.

Linux has its flaws, but at least it provides you with the power and tools needed to get you out of tricky situations.

I probably shouldn’t have written down my notes on the eGPU setup I had years ago.

I’d be lying if I wasn’t considering remaking this setup with everything I’ve learned 6 years later.

Oops.

I got access to an allegedly-faulty AMD Radeon RX 480 and an NVIDIA GTX 1650 for free thanks to my friend, so I ordered the EXP GDC Beast v8.5c, two sorts of 6/8pin power cables, and a 12V 12A DC power supply.

The idea was to take a ThinkPad T430, bolt a GPU to it, and use it as a stationary workstation until the laptop dies or gets too slow for modern tasks. The resale value of the T430 is too low for me to justify selling it, and if there’s anyone out there who can run a T430 into the ground 5+ years from now, then it’s probably going to be me.

Quick specs:

• CPU: Intel i7-3820QM
• RAM: 16GB DDR3
• Storage: 4TB Samsung 870 QVO

Side-note: I also went ahead and replaced the thermal paste with a bootleg Honeywell PTM7950 phase-changing thermal pad. It works just as well as fresh thermal paste, but hopefully it won’t drip out if the laptop is running up-right, in a vertical laptop stand.

Testing

My testing with the GTX 1650 was brief, as the open source nouveau driver crashed on the Wayland desktop¹, the proprietary drivers under Fedora did not seem to work, and under Windows the NVIDIA driver intentionally triggers the error code 43 issue. A tweak exists for the Windows issue, but that didn’t help much either.

The AMD Radeon RX 480 seems to work well out of the box on Windows 11, plug-and-play. However, it wasn’t that stable, but that could be related to the possibly faulty GPU itself or the power supply that can barely drive it under load. Or the fact that Windows itself was running off of a 128GB Samsung USB 3.0 flash drive. Probably the latter, given that the USB stick died as a result of this experiment.

On Fedora Linux 40, I discovered that eGPU-s aren’t really plug and play. Sure, the image is there, but on my ultrawide monitor I saw a whopping 11 frames per second! Turns out that by default GNOME on Wayland doesn’t like to use the eGPU as the main rendering device, even on more legitimate Thunderbolt-based eGPU setups. It’s a common enough problem that there exist multiple solutions to this issue.

I didn’t like any of them², so I disabled the i915 kernel driver responsible for driving the iGPU, which did the trick. The 3440x1440p ultrawide monitor was being rendered by the eGPU and for the most part the experience was very smooth.

The iGPU in the T430 can also run the ultrawide monitor at 60 Hz via the DisplayPort port on the dock, but it can’t run things at 60 FPS at all times because it’s really, really weak.

The eGPU ran things much smoother, which made the whole setup feel more responsive and great to use. It’s not the fastest setup in the world, but the GUI running smoothly certainly makes it feel like one.

I didn’t have a ton of games installed for testing, but I did give Minecraft a try, and it ran just fine. The CPU was struggling when building new parts of the map, but the visuals were doing fine and I could see myself playing on this setup for hours.

There were some things that weren’t running smoothly even with the eGPU. For example, 1080p H.265 video playback was awfully choppy. Using radeontop revealed around 80% GPU usage, so either the decoding engine is crap on this GPU, or the very limited PCIe bandwidth and CPU-GPU data transferring is the culprit.

I also tested LibreELEC to see how it handles the eGPU and I’m happy to report that it renders Kodi on it by default, with the UI running very smoothly.

On Linux I also saw some stability issues and crashes, but those could also be down to the faulty GPU or the PSU being underpowered. I can probably rule out the latter by limiting the GPU clock speeds, and the Arch wiki has some great instructions for that. Those toggles do work, but I noticed that GNOME can be a bit choppier when the GPU switches between lower power states to higher ones.

The physical stability of this setup is questionable, especially if you use a bigger GPU. Most prefabricated cases for this eGPU adapter assume that you’re running it with an ATX/SFX PSU, so those are out of the question for me. There exist some 3D printable designs out there, but they can sometimes be very specific to a particular setup or GPU, so I’d have to design my own.

The power consumption of this setup isn’t great, unfortunately. Using a smart plug I measured the idle power of the eGPU component to be about 20W. Typical desktop usage and video playback results in around 50W and peaks around 120-130W, which is right on the limit of the shoddy PSU I’m using.

If I had the choice to pick any GPU for this setup, then I’d likely opt for a modern AMD GPU that didn’t require additional power via the 6/8 pin power connector as that option would likely be a lot more efficient while yielding similar performance.

I ended up disassembling the setup in the end, partly because I misdiagnosed a Mesa bug to be a GPU issue, and because the RX 480 really was acting weird from time to time, even with LibreELEC.

Not all experiments end up being successful.

Conclusion

I think that these types of eGPU adapters are great even in 2024, but only for a small number of very specific use cases, and with GPU-s that actually work.

If you already have an older laptop around, and a compatible spare GPU collecting dust, then this setup will make sense for desktop or media playback machines, and perhaps gaming if your demands are not very high.

I don’t suggest building this setup from scratch, it’s probably not worth the money and hassle. Used gaming PC-s that have similar specifications to this eGPU build, but with no funny PCIe bandwidth limitations, go for less than 200 EUR at this point.

A family member has a Canon PIXMA MP250 printer, originally released in 2009. It has been a very reliable piece of hardware, especially for a printer.

Then came Windows 10. The printer would not work out of the box with it and the official drivers got stuck during installation. Fiddling with the printer in device manager, trying to install drivers via Windows Update and stars aligning got the printer to work again.

Then came Windows 11. Nothing I did could get it working now, and the printer is not even officially supported by Canon on this version of Windows.

This printer works out of the box on any mainstream Linux distribution, and the family member had been using the printer with Ubuntu for years, but a certain piece of important software in Estonia was not working properly a lot of the time, which triggered a switch to Windows.¹

Quick searches online suggested that I could turn this printer into a network printer, as long as I had some patience and a spare computer.

The idea is simple: take a spare computer, hook it up to the printer, install CUPS on it, configure it and you’re good to go!

I initially tested this setup with a Zimaboard running Fedora Server, but the final solution used an old Raspberry Pi B+ running the latest version of Raspberry Pi OS.

How to set it up

The steps will be similar with other pieces of hardware and Linux distributions, you may just need to adjust a few commands and package names.

The steps are roughly based on these two guides:

• Setting up Network Printer with Raspberry Pi and CUPS
• Raspberry Pi Print Server: Setup a Network Printer

If you’re following along with a Raspberry Pi, set up your OS first. A minimal installation with no graphical interface will be just fine.

Next, install CUPS and optionally the right printer drivers for your printer. The Canon PIXMA MP250 requires the package printer-driver-gutenprint.

sudo apt install -y cups printer-driver-gutenprint

Make sure that CUPS is running.

sudo systemctl enable --now cups.service

Next, allow your user to manage printing. In this example, the username is pi.

sudo usermod -a -G lpadmin pi

Next, make sure that CUPS allows connections from other machines in your network, and then restart it.

sudo cupsctl --remote-any
sudo systemctl restart cups

Find out the IP address of the Pi and navigate to it in your browser. It will look something like https://192.168.0.49:631. The browser will likely warn you about a self-signed certificate. It’s safe to proceed.

If you haven’t already connected your printer to the Raspberry Pi via USB, then do so now.

Use the graphical interface to add a new printer under the Administration section. You’ll need to enter the username and password for your user here. It’s the same user we used earlier, so in this example it’s pi.

If you can’t find your printer model listed in the Add printer view, then you are probably missing the proper driver package, or your printer simply isn’t supported.

After adding your printer, you can use the CUPS web interface to print a test page.

If that worked, then you should now be able to use any other device on the network to print!

On Linux and Windows, open the settings section that handles printing, and add a new printer. This networked printer should appear automagically, no additional configuration required.

I also did testing using a Fairphone 5 running Android 13, and an iPhone X running iOS 16. In both cases the printing just worked.

I’ve used the scanning functionality of this particular printer on Linux and it has worked, however I forgot to test this out using this setup. I’ll try to test that out once I get the chance to try the printer out again.

Hardware tech tips

I first ran this setup with a Zimaboard to test the idea out. It worked very well, but as a simple print server it is a bit overkill.

I then replaced it with a Raspberry Pi B+, and while the printing works, I discovered that a single ARM CPU core is apparently a limiting factor during printing. A simple test page print pegged the CPU at 100%, and that’s with the 700->1000 MHz overclock applied.

If you don’t mind a slower printing speed, then any Raspberry Pi will be absolutely OK. More performant versions of Raspberry Pi will likely fare even better.

If you already have a small home server running, then you may want to hook the printer up to that and set up the printer there. It’s already running 24/7, so it would be a perfect match.

One useful aspect of this setup is that you can queue up print jobs even when the printer itself is powered off. Once you turn it on, the print jobs will start, assuming that you have enough paper and ink.

Free open source software prolongs the useful life of hardware once again. See y’all in October 2025!

2024-06-15 update

This post got featured on:

• XDA Developers
• hackster.io
• Hackaday
• Hacker News

And even the leader of OpenPrinting, Till Kamppeter, chimed in! They also held a talk about OpenPrinting at FOSDEM 2024.

Regarding scanning, seems like there’s a separate project for that called SANE - Scanner Access Now Easy.

Find out more about OpenPrinting here.

2024-11-14 update

A reader shared with me the p910nd project that can be used for a similar purpose. I have not yet tried this myself.

There’s even an OpenWRT page that describes its use on a router!

I got a Steam Deck.

Only took me a year or so of contemplating getting one, and trying out HoloISO, the unofficial SteamOS installer finally convinced me to get one.¹

It took another year to actually get down to writing down my thoughts on it.

This post is written from the perspective of a software developer who used to play video games a lot as a teenager, less so as an adult, and as someone who dabbles with Linux as a hobby.

Overview

The Steam Deck OLED exists now, but back when I got my Steam Deck, it came in three flavours, and in the interests of my budget I opted for the cheapest option that came with only 64 GB of storage (eMMC based).

I also got a 512 GB microSD card to hold all the games. All in all, this setup cost about 470 EUR. Certainly cheaper than the 512 GB NVMe SSD model that sold for 679 EUR at the time of writing. The shipping estimate to Estonia was about 1-2 weeks, and mine arrived in almost exactly 1 week, which was nice. And in one piece as well!

I don’t have the official Steam Deck Dock (try saying that 10 times in a row), but what I do have is a vertical laptop stand and a Dell WD19TB dock. As long as you leave a gap for airflow, then this combination works quite well if you want to use the Steam Deck in a docked mode. The combination of this dock and the Steam Deck also works nicely, the 3440x1440p ultrawide monitor gets picked up fine and even the gigabit Ethernet port works at the rated speed.

However, it seems like the Steam Deck USB-C compatibility can be dependent on your monitor and dock choice. While the Dell WD19TB dock worked great with an ultrawide monitor, a Lenovo Thinkpad USB-C dock would fail to output anything to the same monitor quite often. Turning on the setting to enable safe resolutions for the external display and setting the resolution manually usually fixes it, but it’s still quite annoying.

It’s booting Linux!

Not long after I set the Steam Deck up, I switched to desktop mode and opened a terminal window to confirm that yes, this thing is running Linux. Arch Linux, in fact, or at least a modified version of it.

But in all other aspects, it’s just like a normal computer, with an actual UEFI setup that you can browse. With the Steam Deck, you get the polished experience of a gaming console and the freedom to do whatever you want to with the software and hardware.

You can also boot any x86-based Linux distro, like Fedora Linux.

Or if SteamOS 3 is too limiting for you, then check out projects like Bazzite. The experience will be very similar, but it’s easier to tweak the setup to your liking. Bazzite also allows you to format storage as btrfs with compression enabled, meaning that you can fit more games on your device with a very slight increase in CPU usage. I’m sticking with the official Steam Deck experience for now, but I’m tempted to try Bazzite for the btrfs aspect alone.

The desktop experience on the Steam Deck is quite nice. You can easily boot into it from the graphical interface, and it runs on KDE Plasma. The design language of KDE is an eyesore, but it makes up for it with the responsive UI.

The touchpads are not that big, but they’re plenty good for clicking around. Manually popping up the on-screen keyboard by pressing STEAM + X is a bit annoying at first, but it gets the job done.

Installing software can be done in the graphical environment via the “Discover” app that ships with the desktop environment. One thing you’ll probably notice is that the selection of software is a bit limited, and that’s mainly down to how the Steam Deck is set up by default. All the apps you install are installed via Flatpak and while the selection of apps that are available as a Flatpak grows with time, not everything is yet present there.

I was surprised to see that the Steam Deck Desktop FAQ actually has useful information for an enthusiast like myself, even mentioning what a flatpak is.

Peeking under the hood

During the first couple of months I went the tinkerers’ path, enabled the SSH server, unlocked the root partition for modifications and automated the setup of my Steam Deck. While this thing is running a flavor of Arch Linux, it’s not really designed for these types of manual modifications. I had to mess around to get the Arch default package manager pacman to work, but at one point messing with pacman and running commands that I didn’t know much about, I blew up the OS by force-installing some packages and throwing out a crucial dependency. Whoops.

Well, at least the Steam Deck OS recovery setup is quite straightforward. Download the OS recovery image, put it on an USB stick, boot it on the Steam Deck and re-image the OS. There are also options to attempt repairs or open a terminal window in case you know how to fix the mess you’ve created.

One downside of the SteamOS recovery setup became apparent when Valve released a buggy software update around summer/fall of 2023. The update was buggy for many users, including myself, with games crashing or running poorly. On my Deck the SD card was also automatically reformatted for some reason so I had to redownload all the games I had on that. When I went to download the recovery image, I discovered that you could only go back to the current, buggy version of Steam Deck OS.

By fiddling with the download link I could navigate to a web directory where older recovery images were present. Those weren’t much help though. Although you could boot into an older version during the initial Steam Deck setup, it would auto-update to the newest version automatically, with no obvious way to bypass it.

The real workaround would have been to manually boot into the older version of the OS that sits on the other partition and not update for a while. I unfortunately discovered that after I had re-imaged the Steam Deck, which formats the whole drive.

I’ve now stopped with the tinkering. I still have an SSH server running on it, but it’s there so that I can install my favourite applications as flatpaks and deploy the backup script, via Ansible. That’s it.

It’s booting… Windows?

Don’t worry, Valve doesn’t even officially support running Windows on this thing, and while you can do it, I don’t really recommend it. You’ll be giving up all the nice things about SteamOS while also taking on the hassle of running Windows, including unexpected Windows updates.

Games are too damn big

512 GB of storage on something as small as a fingernail was something I would have never expected to see 10 years ago, but here we are. There are even bigger microSD cards out there, but those also cost a lot more.

And even with this marvel of technology I ran into issues when installing games. Install GTA V, Forza Horizon 4, and God of War, and just like that about half of all that storage is gone.

The 64 GB internal drive was also quite limiting, especially due to shader cache and compatibility layer (Proton) related files taking up a majority of it. The situation has improved now, but I still went ahead and replaced it with a 1 TB M.2 2230 NVMe SSD that I ordered online for 74 EUR in May 2023, which still makes it a cheaper option than the 512 GB model sold at the time.

The new SSD works well, it’s fast and doesn’t seem to affect battery life negatively. Changing it was quite simple, all you need is an appropriate screwdriver, good fingernails or a plastic spudger to get the case off, and some delicate care when working inside the Steam Deck.

Before I replaced my SSD, I tried all sorts of wild-ass ideas to fit more games on my Steam Deck.

First, I tried Nextcloud. The idea is simple: I’m not going to be playing all my games all the time, so it makes sense to offload some of those to my home server and retrieve them over the fast local network whenever I want to play them. This idea makes even more sense if your internet speeds are not that great.

The results were mixed. The syncing part was working surprisingly well with the transfer speeds capping out at 1 Gbit/s over Ethernet and actually being faster than the microSD card itself at times. However, the usability of this setup is not that great. You can select which folders to sync in the Nextcloud Client application, but Steam will still believe that a game is installed even if the game files are missing as long as the app manifest file is present. This can lead to annoying situations where you try to play a game and discover that the games files are not present on your Steam Deck.

The second solution I tried out was iSCSI. I followed this guide on Reddit. Getting iSCSI to work was tricky even for someone like myself who should be able to do this technical stuff properly, but I ended up getting it working. I formatted the iSCSI device as ext4, mounted it and made it a Steam library folder in Steam desktop mode.

The benefit of this solution is that as long as you have access to the server, the iSCSI storage device will show up on your Steam Deck as any other drive and games will just work. If your network is good enough, you can play games that are stored on your server, which might make a lot of sense for games that you generally play in docked mode. Transferring games to your internal storage is also simple, performant and can be done entirely within the Steam UI. The iSCSI setup is not that great mainly due to the technical complexity of the setup, making it a no-go for the more casual audience. The performance over Wi-Fi was also lousy, and it’s also difficult to justify having a good chunk of your home server storage being permanently held up by Steam games.

After all that hassle, I went with the solution that most people would have gone with already: store games on an external SSD. Duh. It has the same benefits as the iSCSI setup, but it’s faster and you can bypass a lot of the technical complexity. The external SSD is also automatically mounted when you connect it to the Steam Deck if it’s formatted as ext4, making it a very convenient option for docked play.

If you happen to have a fast desktop PC with plenty of storage, or a home server, then consider using the Steam local network game transfers. You won’t fit more games onto the Steam Deck, but redownloading them will be much faster.

We really need to start decoupling high resolution textures and other high quality assets from the base game at some point because the current approach is just silly. For a device like Steam Deck you’re rarely going to run those games with the highest quality settings, meaning that those assets will just sit there on the disk taking up space.

A free optional DLC could work well in theory. In practice I’m sure that game publishers would quickly turn a high quality texture pack into a paid DLC.

The part where I play games

Enough about the part where I tinker with the system. Let’s talk about the games.

I had some idea about the game compatibility when I tried out HoloISO. I’m happy to report that the game compatibility is even better on the Steam Deck. This fact alone is a minor miracle. Some years ago I was convinced that the only way to properly play games on Linux was to set up a VM with Windows installed in it.

With the Steam Deck, you download the game and run it. Simple as.²

The Steam Deck is not the most powerful machine in the world, but it has a decent CPU in it. The GPU is more akin to integrated GPU-s that ship in modern AMD Ryzen based laptops, it’s still plenty of power to run games at the native 1280x800 resolution.

God of War (2018) did stutter a bit, but in most situations you can expect around 40-60 FPS.

GTA V was running fine as well, although the game is CPU limited at times and can drop to around 40 FPS in certain sections of the map, especially in GTA Online.

Teardown runs alright after you lower the settings, and it’s best played in docked mode due to some fun mods being designed around keyboard controls. It’s really fun, but it will bring the Steam Deck to its knees when you do some heavy destruction. Totally playable though, and I’ve had a lot of fun with this game.

Wreckfest runs well with medium or high settings. I feel the framerate dropping a bit when heading into the first corner and being part of a 24-car pile-up, but in all other situations it’s smooth enough.

Need for Speed Hot Pursuit (2010) runs very well and is an absolute blast on the Steam Deck. Crank the settings to maximum and enjoy the thrilling police chases.

Need for Speed Most Wanted (2012) is also great on the Steam Deck.

Need for Speed Rivals (2013) runs well after you apply the tweak that forces the game to run at 60 FPS. The game is a bit buggy even on Windows, so expect some bugs and crashes.

More modern Need for Speed titles, such as Need for Speed (2015), Need for Speed Payback (2017) and Need for Speed Heat (2019) are playable, but will typically run anywhere between 40-60 FPS. The handy framerate capping feature of the Steam Deck will help a lot in those cases since a stable 40-45 FPS is a much better experience than a wildly fluctuating and unstable framerate.

More lightweight games run like a charm while using very little power. This includes titles like Vampire Survivors³ and Donut County.

What I really like about the Steam Deck is that performance tuning is very accessible. You can toggle mangohud with various levels of detail to narrow down any performance troubles, and you have great control over the CPU and GPU speeds. All of this can be done while the game is running.

The APU in the Steam Deck tries its best to share the limited power that the system can pull between the CPU and GPU, and in most cases it works well, but there are games where the CPU or the GPU demands can fluctuate a lot. In those scenarios it helps if you lock the GPU clock speed to an amount that keeps the GPU usage to around 80% or less. Being able to play with settings like this is one of the reasons why I got Forza Horizon 4 to be playable within minutes of tinkering.

There are a few games that I like to play but aren’t working well on the Steam Deck, such as GTA III or Vice City, but that is pretty much it. Over the one year that I’ve had the Steam Deck, a lot of games have gained support, either by Valve improving the Proton compatibility layer, or game publishers releasing updates that make the game work on the Steam Deck.

One such example is FlatOut Ultimate Carnage, which seems to have been bought by a new publisher that removed the “Games for Windows Live” mess and got the game “Steam Deck Verified” in the process.

The performance of the Steam Deck was actually quite surprising in a lot of games. For example, I’ve never seen GTA IV run so well on any machine that I’ve owned before.

If you want to see if your favourite games are compatible with the Steam Deck I recommend you check out ProtonDB for a detailed overview. YouTube is your friend if you want to see some specific games being tested on the Steam Deck.

When it comes to emulating PS1, PS2 and PSP games, the Steam Deck is fantastic. There’s plenty of performance to emulate those consoles well, and for whatever reason playing the childhood classics on the Steam Deck feels so right. With PSP the comparison is even more apt as it’s basically a bigger PSP but with a much better battery life.

More intensive games can discharge the battery in about 2-3 hours, but when playing emulated games for older consoles you can expect up to 7-8 hours of gameplay.

Non-Steam PC gaming

It’s clear from the name that this console is set up around Steam. As long as all your games are purchased on Steam, you’ll have a great time.

I do have some games from other sources, such as GOG.

There exist community-supported launchers, such as HeroicGamesLauncher. They work, but might require a bit of setup and fiddling to get things like cloud synced game saves to work properly. In the case of art of rally on GOG, you’ll also lose out on multiplayer daily/weekly challenges.

If you have some older games on CD-s, or legally acquired archived copies of PC games, then the situation gets tricky. There exist solutions like Lutris or Bottles, however I never quite had a good experience with those, especially once you deviate from the happy path.

In my experience, if you want to have the best chance at playing your old PC games, then you’ll need to obtain a version of the game that has a no-CD patch applied to it already, add it to Steam and set a version of the Proton compatibility layer. Some legally dubious sources for these games label these packages as “direct play” as well.

There can also be a difference when running games via gaming or desktop mode. Need for Speed Most Wanted (2005) did not work when started via gaming mode, but it started up just fine in desktop mode.

Proton, the compatibility layer, is great for what it allows us to do, but I feel like the concept and technical implementation is something that novice users will never quite understand. This also becomes apparent when you look up discussions online where novice users try to apply mods to their favourite games. It works out of the box on Windows, but you need to grok the way Proton handles things for it to work on the Steam Deck.

I consider myself competent with computers and stuff, but even I need to sit down and think about what’s going on when working in this area.

Backups

I rely on Ludusavi to collect all my game saves, and I use rsync to back them up to my home server.

I made a desktop entry for the backup script. By right-clicking it in desktop mode and selecting Add to Steam, it will appear in your Steam library like any other game or application. This means that I don’t have to quit gaming mode to back up the Steam Deck.

The backup script also updates all the applications that I’ve installed as a flatpak, and shuts the device down after it’s done.

It’s big.

The Steam Deck is big.

This does mean that Valve can pack a lot of compute power and cooling into it, but it comes with the trade-off of it not fitting that well into a backpack. The hardcase that it ships with is great for preventing damage to the Steam Deck, but it takes up a lot of space.

I don’t find the size uncomfortable, quite the opposite actually, but for others it might be a problem.

I’ve had the opportunity to briefly play on a Nintendo Switch Lite. There’s something that feels special about that form factor. The performance is not great compared to the Steam Deck, but I feel like the Nintendo Switch is in its own class of portable gaming consoles.

I wonder how a Nintendo Switch sized Steam Deck would perform.

The controls

The controls on the Steam Deck are good. It takes the best aspects of the Nintendo Wii U and Xbox 360 controllers and smashes them into one polished experience.

The Steam Deck has some extra mappable buttons on the back. They sound great in theory, but I rarely use them. When I do, I find that the L4 and R4 buttons are more easily reachable compared to L5 and R5 (the lower ones).

When I do use those extra buttons, they are invaluable. In Need for Speed Rivals (2013), I mapped L4 and R4 to the two power-up slots, which left my thumbs free to focus on driving and looking around.

Using those extra buttons does require some readjustment. It was difficult to reprogram my muscle memory and remember that I have those extra buttons there.

Steam also provides lots of controller templates, both Valve-approved and community ones. If the game you want to play doesn’t have official controller support, then it’s highly likely that someone else has created a controller configuration that emulates keypresses and allows you to play your favourite game on the Steam Deck.

Sometimes that doesn’t quite work, in which case you can create your own layout, or improve an existing one.

There’s also the gyro functionality, but I’ve only used it in Aperture Desk Job, which itself is like a fancy demo of the controller capabilities of the Steam Deck. Never quite found a suitable use case for this input method in other games.

The fun factor

Based on what you read, you might think that I got the Steam Deck for the tinkering aspect. While that’s certainly part of the rationale behind getting one, it wasn’t the main one.

I got one as an excuse to finally spend more time resting. I have had a bad habit of trying to be productive at all times, which can lead to exhaustion and eventually burning out.

With the Steam Deck I’ve been able to actually spend some time playing games and turning my brain off.

I love it.

And it gives my cats a great opportunity to snuggle up while I do some illegal street racing. Win-win.

There’s something different about playing games on the Steam Deck. Perhaps it’s related to the memories I had as a teenager playing on my PlayStation Portable a lot, or knowing that I can replay all my favourite console games without having to set up the physical consoles themselves.

The sad reality of adulthood

I love using the Steam Deck. However, I still can’t seem to make much time for it when life gets busy.

That’s my only big gripe with it.

Conclusion

If you’ve been on the edge about getting a Steam Deck, then I hope that this post has helped you make a decision. It has some rough edges, but when it works, it feels magical.

If you already have a gaming PC with a modern AMD GPU in it, then consider trying out projects like Bazzite, HoloISO or ChimeraOS to get a taste of the Steam Deck experience.

Even if I can’t put in as many hours as I’d like to, I still love my Steam Deck and don’t regret getting it at all.

I recently busted out my old ThinkPad T40, the last of the OG IBM ThinkPads.

I picked it up some time around my university days because I liked collecting ThinkPads at the time, and it was a nice complement to my existing ThinkPad T60 and T430.

The battery is dead, but everything else still works. Checking a few online listings, I’m surprised that I can still find batteries sold for this model. Probably old stock that’s been sitting around in a warehouse slowly discharging to death, but hey, you might get lucky!

I also learned that I had replaced the internal IDE hard drive with a 16 GB mSATA SSD via an adapter at one point. I don’t think they even sell SSD-s that small nowadays, but the size feels age-appropriate.

I had already installed a flavor of Linux on this, but had completely forgotten the password.¹ This was a good excuse to wipe it and see if I can install a modern mainstream Linux distro on it.

I went with Debian 12 (Bookworm). It’s the newest Debian release, and unlike many other distros, they still offer 32-bit installers.

It had been a while since I had last installed Debian via the terminal user interface. The experience felt surprisingly snappy.

I went with the good old XFCE desktop environment. It strikes a good balance of being lightweight, simple and usable.

Things were great until I started up Firefox. It works, but man, is it slow. All the years of browser complexity have caught up with this laptop and made the web barely usable.

It does pass the techtipsy test.

This will likely be the last time this laptop runs a modern OS. 32-bit CPU-s are losing support both in x86 and ARM ecosystems, and the year 2038 is not that far away any more.

So long, and thanks for all the fish!

I’ve had the opportunity to try out another new laptop at work. I’ve used a brand new laptop recently, and it was horrible.

But this time I’m pleasantly surprised.

The Lenovo ThinkPad P14s gen 4 has great specs:

• CPU: AMD Ryzen 7 PRO 7840U (8 cores, 16 threads, up to 5.1 GHz)
• GPU: AMD Radeon 780M (integrated)
• RAM: 32GB DDR5, soldered
• SSD: 1 TB NVMe
• Display: 1920x1200 resolution
• Two USB-C ports
• Two USB-A ports
• HDMI out
• A full-sized Ethernet port!

I have used it for about a month at work, and so far I have not noticed anything outright broken about it. The Fedora Linux 39 installation that I copied over from the previous laptop (Dell Latitude 5411) simply works, no modifications required. The CPU is blazing fast. The GPU driver does not crash. No issues with Wi-Fi or Bluetooth so far. USB-C docks and monitors that I have just work. The webcam is also functional and has an acceptable image quality for meetings. CPU and GPU temperature data is present with lm-sensors.

The build quality is what you’d expect from a modern ThinkPad: better than consumer-grade crap, but not as solid as a ThinkPad T430. The case has a soft-feeling light gray finish to it, which will likely look horrible after a few years of use.

The battery life is not great: don’t expect to last a full 8-hour work day on battery with this thing. It’s been years since Apple has released Apple Silicon laptops with great battery life and somehow the rest of the industry still hasn’t caught up.

The fan has a really annoying high-pitched whine to it. You only hear it when pushing the CPU hard, like you’d do when compiling a legacy Java monolith, but in a quiet room it’s really annoying. I’m not even sure that the fan has to be run that hard, the AMD Ryzen CPU inside is very efficient and can regulate its power consumption based on the current temperature. You can tweak the fan curve on this laptop with a basic script.

I haven’t tried functionality that I don’t use myself, such as the fingerprint reader, so I cannot comment on that.

This laptop can also play games. I took an external SSD, installed Bazzite on it (GNOME + gamemode version), set everything up, and installed Forza Horizon 4 on it. It runs, and for an integrated GPU surprisingly well. I could push the game to run with medium-high settings on the internal 1920x1200 display at a solid 60 FPS, with very rare dips below it. That’s even better than what I got with the Steam Deck, and the CPU running near the 5 GHz mark certainly contributed to it. This makes the laptop a great candidate for a LAN party after work, unless you favor more demanding games, in which case you might want to temper your expectations a little bit.

It’s the first time that I’ve had a brand new laptop that just works on Linux. Hats off to the FOSS community for making this happen, AMD for getting their act together, and Lenovo for not completely fucking it up this time.

Oh boy, here I go testing a new laptop again!

Well, it was new back in 2020.

You might remember my article on why I went back to a ThinkPad T430 in 2022. Or that other time when I got a new HP laptop for testing and got so frustrated that I wrote about it.

Well, today I’m writing about the Dell Latitude 5411.

It’s not the newest laptop in the world, but I decided to give it a go because of a few reasons:

• nobody wanted one at work due to the reputation of this laptop (sounds like a jet engine)
• it’s almost 3 years old at this point, which is the Linux compatibility sweet spot (in theory)
• there were some changes coming to the office, which meant more flexible seating arrangements and everything being designed around USB-C, and I wanted to continue my employment at the same company which might have been tricky with a ThinkPad T430

And that’s how I ended up with one.

The Linux compatibility sweet spot

If you have spent any time using new hardware on Linux, then you probably know how painful it can be. You might get lucky and receive hardware that has support right from the get-go, but in most cases there are issues around that may or may not get fixed with time.

If you take the same device a couple of years later, it’s highly likely that the Linux support on it will be much better. The price of the device will also be much lower than before, especially if you get it on the second hand market.

Any machine that falls into this time window is what I like to call “the Linux compatibility sweet spot”. This Dell happens to be one such machine.

Here’s a quick rundown of some specifications:

• CPU: Intel Core i7-10850H 6C/12T, 2.7 GHz/ 5.1 GHz
• RAM: 32GB DDR4
• SSD: 512 GB NVMe
• OS: Fedora Linux 38
• Thunderbolt support
• Wi-Fi, Ethernet based on Intel chips

The Dell Latitude 5411 is powerful, and the components in it seem to be working just fine under Linux. The USB-C port has not had major issues in a work environment, and it also works tolerably with a Dell WD19TB Thunderbolt 3 dock. My ultrawide monitor works, the Ethernet port on the dock can push gigabit speeds, and USB 3 ports work as expected.

Honestly, that’s pretty much it regarding compatibility. It just works most of the time, and that’s why I like this machine.

Oh god, the noise

Not everything about this machine is ideal, though.

One of the first things I noticed with this machine is some kind of electrical noise. Some call it “coil whine”, but all I know is that it’s this high-pitched squealing that’s coming from the laptop. Based on some searching and personal testing I can confirm that the noise is related to the CPU switching between idle and really high turbo boost speeds very quickly. If you remember the spec sheet, you’ll know that this CPU can reach 5.1 GHz. The noise is apparent whenever there’s a load on the CPU, and it’s probably the sound of the power delivery components screaming in pain to deliver all this power that this furnace of a CPU requires.

There is a workaround to the electrical noise: try running the CPU at a lower speed. You can do that by disabling Intel Turbo Boost, for example. There’s an UEFI setting for that, but on Fedora Linux 38 there’s an even easier option in the form of the “Power Profiles” setting. When running the CPU slower, the noise is much less severe.

This machine got the reputation of being a jet engine at work, and I agree with those people as this thing can be damn loud even when it’s seemingly not doing much. This is a result of the CPU demanding a lot of power and the cooling solution not being able to catch up. I feel that the stock fan curve is quite aggressive to prevent any reliability issues after long-term use.

If you have the same laptop and are looking for a fix to that issue, then you’re in luck! There are plenty of solutions out there and I am personally a fan of this Arch Wiki section. I use dell-bios-fan-control to be able to control the fan speed myself, and a simple script to fine-tune the behaviour of the system. The script looks something like this:

#!/bin/bash

set -e

echo "Finding device path."
device_path=$(echo /sys/devices/platform/dell_smm_hwmon/hwmon/hwmon*)
echo "Device path: $device_path"

echo "Testing existence of paths."
cat "$device_path"/temp1_input
cat "$device_path"/pwm1
echo "Testing existence of paths done."

echo "Disabling Dell stock BIOS fan control."
/root/.local/bin/dell-bios-fan-control 0
echo "Done!"

while true; do

  temp=$(cat "$device_path"/temp1_input)

  # Only two fan speed levels seem to exist, anything above 200 is basically
  # equivalent to full speed.
  if ((temp > 50000)); then
    echo 64 >"$device_path"/pwm1
  else
    echo 0 >"$device_path"/pwm1
  fi

  sleep 15
done

The CPU is already temperature limited even in the stock configuration and will throttle under load, which should mean that running the machine at a lower fan speed is just fine. Let’s hope that I’m right.

The damn USB-C port

While the USB-C port generally works well, I do still get annoyed with it. When connecting it to my Dell WD19TB Thunderbolt 3 dock, there’s a good chance that it will do nothing but charge. Restarting the machine with the dock connected will usually work, but it does take away from the portability of the setup.

I’m convinced at this point that USB-C is a cursed standard and will never “just work”. If you disagree, then please look up any article that tries to explain how USB-C works and all the things you have to keep in mind if you want to buy a damn cable for your device that does what you want. And then add Thunderbolt to the mix.

If this aspect starts becoming very annoying, then it might become a dealbreaker for me, but right now it happens infrequently enough that I can ignore it.

Oh, wait, it happened again while writing this article. Damn it all.

Other notes

Here’s a rundown of some observations that I’ve made while using this machine:

• you can actually boot from the microSD card, which is nice
• if you dual boot Fedora (two completely separate installations), then you’ll get two Fedora entries in the boot options list, and although the names are identical, you can set a preferred boot order just fine
• keyboard and touchpad are not the best things out there, but they are at the very least tolerable
• cannot say the same about the clit mouse, it’s not that good
• battery life is what I expected from a laptop that’s not an Apple M1/M2 based laptop: 2-4 hours of use when doing any real work, and up to 8 hours if your work involves staring at the screen with the brightness set to low and Wi-Fi/Bluetooth turned off
• lowest power draw I observed with upower was around 6 watts
• this machine has enough power to be a solid desktop replacement for software development
• the above is true even when you switch turbo boost off, you still have 6 cores running at 2.7 GHz!
• never managed to overheat the machine, even if I accidentally forgot to turn the laptop fan back on while running heavy loads (CPU throttled to around 2.2-2.4 GHz, which is impressive!), or when I accidentally put it in my backpack while it was still running
• “power on with USB-C connected” or any other UEFI setting with a similar purpose should be avoided, you will end up putting it into your backpack while it’s running
• this laptop has a full sized Ethernet port, which is incredibly handy for many IT professionals
• it even has the old Dell barrel plug charging port, which is great for situations where you’ve damaged the only USB-C port on the laptop
• I don’t have too much faith in the construction of the machine, some parts feel very plasticky and might not hold up well 5-10 years from now
• I was the one who actually made the recommendation to purchase this particular laptop model at my current job back in 2020 due to the fleet of ThinkPads being compatibility nightmares with the monitors and docks available, and in that regard it was an improvement. Shame about the fan and electrical noise.

Conclusion

Will I end up switching to a modern laptop now? Maybe. I’ve done so for my day job already, but the jury is still out on whether I might end up with this particular model for my personal use (especially because of the damn Thunderbolt dock issue). There are alternative laptops out there with similar pricing that are also in this Linux compatibility sweet spot.

Should you get one? If the price is right and you don’t mind applying the workarounds for the issues I mentioned, then go for it!

I have restored some faith in modern laptops now. They all suck in different ways, but this one is not the worst one, and that’s good enough for me. Probably. I’ll keep my ThinkPad T430 in an easy to reach spot, just in case.

If you have spent any time in gaming hardware circles, then you’ve probably heard about the Steam Deck, the Linux-based handheld gaming PC built by Valve. Yes, you heard that right: not Windows, but Linux.

I’ve had my fair share of attempts at gaming on Linux. When it works, it’s amazing. When it doesn’t, it’s incredibly frustrating. The last thing I want to do when playing a game is to become a developer and start troubleshooting issues with all the layers between the hardware and the game.

What makes the Steam Deck different is that the compatibility layer (Proton) and all the bits and pieces associated with it are integrated in a way that should make it unnoticeable to the user. Just start up your Steam Deck, pick your game, play, and you’re good to go.

Although I’ve thought about buying a Steam Deck myself for tinkering and testing purposes, I could not justify that purchase. But thanks to videos from the ETAPRIME YouTube channel I learned about the existence of HoloISO, an unofficial SteamOS 3 installer.

HoloISO

HoloISO aims to be as close to the native SteamOS 3 experience as possible. The only reason it exists right now is because Valve does not yet officially support SteamOS 3 as a distro that you can go and install. That might change in the future, but for those that are impatient like myself, HoloISO is a nice option.

To test out HoloISO and give it a good chance at succeeding, I chose my ASRock DeskMini X300. It sports an AMD Ryzen 7 5700G, which is a high-end APU and should result in performance that’s roughly comparable to the Steam Deck in most scenarios. The GPU cores are not on the newest GPU architecture released by AMD, but the higher power budget should make up for any architectural deficiencies.

The installation of HoloISO is very easy, just download the installer and write it to an USB stick. Once you’re booted up, follow the installation instructions and reboot. You should now be at the SteamOS 3 setup screen.

First impressions

I’ll be the first one to admit that my Steam library is a bit out of date, which is why my experience is limited to older titles. I can confirm that BeamNG drive and Dirt Rally 2 work fantastically though.

The only way to play GTA V on Linux without issues used to involve creating a Windows VM and passing a GPU to it. This is why I was very impressed to see that installing and starting up GTA V worked out of the box. I could even go to GTA Online and play around without being kicked out or banned!

To make choosing and playing games a better experience for casual gamers, Steam has also come up with the Deck verified program. You shouldn’t be discouraged if your favourite game is shown as unsupported or not tested, though. Try to start it up and see if it works. Burnout Paradise and Absolute Drift were not officially supported, but worked like a charm.

The unfortunate reality of SteamOS is that not every game will work out of the box. Some older games, such as GTA III and GTA Vice City, would not work at all. You might be able to check out places like ProtonDB and see if a game has a tweak that you can apply. Tweaks can involve passing additional launch options to a game or using another version of the compatibility layer (Proton).

What turned me off of this platform was Forza Horizon 4. On this setup, it did not launch properly, the sound was there but the screen was black. Based on results in YouTube, it seems like Forza Horizon 4 and Forza Horizon 5 do work on Steam Deck, so this issue might be down to a compatibility issue on my hardware or HoloISO distribution of SteamOS 3. I wanted to play that game though, so I had to eventually end the experiment and install Windows 10 again.

As someone who wants to get a good overview of resources usage during gameplay to detect and resolve any performance issues, I tend to use tools like MSI Afterburner. With SteamOS 3, you can easily achieve similar results by enabling the performance overlay. I was very happy to see that addition because that means installing and configuring one less tool on my gaming box.

Quality of life issues

Even when things generally work, I did notice some annoyances when playing games. Sometimes I ran into issues with controls. Steam allows you to pick gamepad templates and layouts, which can involve community-provided setups. Sometimes it would default to a layout that simply did not work. Turning off Steam input altogether would sometimes improve the situation. If that did not work, then picking an alternative template would do the trick most of the time.

One major issue that I ran into was related to my XBOX controllers. Because the XBOX wireless adapter does not work out of the box on Linux, I connected them via Bluetooth. However, the latency was horrible and borderline unplayable. There do exist drivers that you can install, mainly xone and xpadneo, and those do improve the situation.

Desktop mode

One aspect of SteamOS 3 that made me actually consider buying a Steam Deck is the desktop mode. Not only can you play games, but you can also boot into a normal KDE desktop environment. Install a web browser, emulators, tweak the system or do actual productive work on it, it just works!

Hell, you can even install an SSH server and manage your gaming box with Ansible!

Whenever I’ve had a gaming PC, I’ve usually hooked it up to a big screen, installed Windows and Steam on it, and called it a day. If you want to watch some media, open up the browser and you’re good to go. I’m happy to report that something similar can be achieved on SteamOS as well.

At the time of testing, there were some issues that I didn’t expect to have with desktop mode. Firefox is included as a Flatpak and it did not support any form of hardware acceleration for video playback, which made it a no-go for a home theater setup.

After browsing /r/SteamDeck for a while, you’ll notice that a lot of people are into modifying their consoles, and all of that is possible because Valve did not lock anything down. Pop the hood and work with the Linux internals as much as you want to! With other gaming consoles you’re going to have to wait until someone discovers an exploit that allows you to have proper control over hardware that you physically own.

Gaming outside of Steam

With the presence of desktop mode, you’re not only limited to games purchased from Steam.

To see how well I could emulate games that I have purchased on older consoles, such as PS1, PS2 and PSP, I tried setting up some emulators. I’m happy to say that setting them up is just as fiddly as it is on Windows, but they do seem to work quite well, at least for these consoles.

I gave Minecraft a go as well by installing it as a Flatpak from KDE Discovery software manager, but for some reason it didn’t run and crashed on startup. I didn’t look into it further, but it seemed like something that an update can fix.

GOG is a platform similar to Steam, with one notable exception: their games are DRM-free. I don’t use that often, but with a game like art of rally I had to get it from there just for that reason alone (and the developers publishing native Linux ports). Although there isn’t an official client for GOG on Linux, there do exist open source implementations. I gave HeroicGamesLauncher a go and although it is a bit rough around the edges, it got the job done and I could play art of rally just fine.

Caveats

The HoloISO experience tries to be as close to the one provided by SteamOS 3, but it is not 100% there yet. It seems that there are differences present under the hood, such as Steam Deck shipping with the root filesystem being read-only by default. I might not be aware of other differences between the HoloISO and official Steam Deck installation, so keep this in mind when making any decisions based on info from this post.

The choice of hardware will also affect your experience. The requirements regarding GPU-s is relatively strict and you’ll likely have the best experience with a modern AMD GPU since that’s what SteamOS 3 is built around. Intel and NVIDIA GPU-s may work, but are not guaranteed to.

Conclusion

SteamOS 3, even in its HoloISO implementation, is very impressive. So impressive, in fact, that I almost bought a Steam Deck. The uncertainty about support for my favourite games and less than ideal performance on the big screen were what held me back. For now.

In its current form, SteamOS 3 (and by extension the Steam Deck itself) are a tinkerers’ dream. There are some rough corners and caveats that you should be aware of as well. Not everyone is into tinkering and experimentation, and that’s OK.

I sincerely hope that Valve can use the momentum that Steam Deck has achieved to push gaming on Linux even further. After having to reinstall Windows more than I’d like to admit, having my gaming PC run plain Linux and be fully manageable with Ansible just feels so right.

I’ve decided to give a new laptop a test drive, mainly because I had the opportunity at work, and also out of morbid curiosity.

It’s the HP Elitebook 845 G9, and it has pretty good specifications:

• CPU: AMD Ryzen 7 6800U
• RAM: 2x16GB DDR5
• Storage: 512GB KIOXIA NVMe SSD (KBG50ZNV512G)
• OS: Fedora 37, kernel version 6.0.8

This laptop was also shown in a recent LinusTechTips video and praised for its ease of repair and maintenance.

However, I can’t seem to enjoy nice things. This attempt at using a newer laptop has had some frustrating challenges. Perhaps the common denominator is Linux. Or me. Can’t rule anything out.

This is simply my experience with this laptop, including my attempts to overcome certain issues with workarounds. If you find something that’s factually wrong about any part of this post, then please reach out!

Wi-Fi

It all began with the Wi-Fi card not working. The card can be replaced, but the results can be mixed.

An Intel Wi-Fi card that I had lying around would not be recognized by the system at all. The situation was fixed by the reseller who replaced the Realtek RTL8852BE chip with another Realtek one, this time with the Realtek RTL8822CE.

Yes, there exist drivers that you can install, but it’s far from an ideal solution. You will have to reinstall the Wi-Fi driver after every kernel update, and the kernel updates frequently on Fedora Linux.

The HP website allows you to pick between a Realtek and Qualcomm Wi-Fi chip. My recommendation is to go with the Qualcomm option because the Realtek RTL8852BE does not work out of the box.

Airplane mode

This is an issue I didn’t think I would ever have to worry about.

The laptop would switch itself to airplane mode at random on boot. Not on every boot, but about 50% of the time. When you try to disable Airplane mode from GNOME settings, it would flip back on again, no matter how often you tried.

To turn Airplane mode off properly, you have to use the Airplane mode keyboard shortcut, and even then it might take an attempt or two.

Some HP users suspect that the issue might be related to a botched firmware update. There’s a kernel patch in the pipeline, and some users say that blacklisting the hp_wmi kernel module could also fix the issue.

Others say that this behaviour might be related to certain events being triggered on the laptop, such as lid closing or the screen rotating. The proposed fix did not work to resolve this issue on my laptop.

Blacklisting the hp_wmi kernel module was what improved the situation for me.

USB4 and Thunderbolt

This laptop is noteworthy due to its support for USB4, which brings with it support for Thunderbolt, making it one of the first AMD-based laptops to properly support that feature.

At my current employer the general trend has been towards standardization: every meeting room and desk has a single USB-C port that you plug into, and you should be good to go.

I can confirm that USB-C docks seem to work quite well under Linux, especially the one integrated into my Dell monitor at work.

I can also confirm that the Dell WD19TB Thunderbolt 3 dock does work.

On Windows 10.

Not on Linux though.

At some point the errors I saw in kernel logs got quite ridiculous.

When reconnecting the port, the results were slightly different almost every time.

The UEFI settings allow some control over the behaviour of the USB-C ports. You can even disable Thunderbolt support and use them as plain USB-C ports with DisplayPort support.

The behaviour of the dock would also be different depending on how you connect it. If the dock is connected during boot, your USB devices and Ethernet port will all be functional, but there’s no display output. If you boot while the dock is disconnected and connect it once you have reached the GUI, your display will work, but you have no USB or Ethernet support.

Even if one of the two critical functions of the dock work, reconnecting the cable will eventually result in nothing working. You also have a 50% chance of the machine locking up completely.

The reasons for these issues can be any one of these:

• the proper kernel support is not there yet
• HP messed something up with the latest UEFI update for Linux users
• I was using it wrong

Dell WD19TB Thunderbolt dock workaround

After multiple sessions of troubleshooting, I seem to have come up with a solution that seems to work for my use case, which involves connecting a couple of USB devices, an Ethernet cable, and one 1440p monitor. You’re giving up Thunderbolt capabilities and forcing the dock to fall back to USB+DP altmode setting, but in return you’ll get a setup that actually works.

To get the Dell WD19TB Thunderbolt dock working, do the following:

• go to UEFI settings and turn off Thunderbolt mode
• save settings and reboot
• connect your monitor over HDMI

Yup, that was it. I could not get DisplayPort working. Only HDMI.

The UEFI settings do contain a setting that allows you to sacrifice USB and Ethernet bandwidth to allow for higher resolution DP signals to be passed through, but that’s simply not acceptable to me. Based on the description, I think that the setting forces all USB devices to run at USB 2.0 speeds. This setting may fix the issue if you want to use DisplayPort, but I haven’t tried it yet.

Suspend

Doesn’t work. The screen will go black and the power button indicator will keep blinking. Only way to break that cycle is to force the laptop to reset by holding the power button down for an extended amount of time.

On Lenovo laptops I’ve seen an option that allows you to change the S3 state behaviour for Linux compatibility. I haven’t found a way to do something similar on the HP yet.

Stability

This is the straw that broke the camel’s back. When playing back video, I noticed issues with glitching and stuttering. The video would play fine, until a frame froze for a few seconds, and then everything continued as normal.

What ruined everything for me were the random crashes that only occurred when playing back video.

When I checked system logs from a previous boot (journalctl -b -1), I found that something was up with the GPU driver.

This was not a one-off event and once it had happened for the fourth time, I gave up on this machine and switched back to my trusty ThinkPad T430.

Conclusion

Given enough time, I’m sure that at least some of the issues will get fixed, either with a kernel update or HP rolling out a UEFI update that fixes things for Linux users.

It’s also possible that a specific set of UEFI settings will contribute to fixing any of the issues listed here. The number of combinations you can have is too high for me to bother testing all of those, so I won’t be doing that.

In its current state, I cannot recommend this laptop to Linux users. You can get most things working, but you will still be incredibly frustrated with the issues that remain.

Something has to change, and it’s going to be how I choose hardware in the future. Linux will stay, which means that the newest and “greatest” hardware is off the table for me. Honestly, lagging behind the newest hardware 3-5 years (or 10 in my case) doesn’t sound too bad, you’ll still get the performance improvements at a fraction of the original price.

If that’s what’s necessary for a good Linux experience, then so be it.

I’ve always been a fan of tinkering with cooling setups on my computers. I’ve even went as far as writing crappy solutions to make up for deficiencies on the hardware level. After years of dumb experiments I’ve seen how little you can get away with in cooling and how to run your machines as quietly as possible without giving up too much performance.

I also appreciate simplicity, which is why I’m currently running the simplest damn solution you can imagine to control how my ThinkPad T430 and ASRock DeskMini X300 run: a shell script.

It also comes with some added benefits:

• self-documenting
• highly customizable
• low resource usage

These scripts are not “battle-hardened” and using them incorrectly may or may not result in hardware failure. Use at your own risk. The only guarantee I can give is that it works on my machine™.

If you’re someone who doesn’t like tinkering with their computers and wants their machines to “just work”, then this article likely isn’t for you.

ThinkPad T430

With the ThinkPad T430, the script does two things:

• control the fan speed
• preemptively throttle the CPU and GPU

This type of behaviour is optimized for mostly quiet operation. The fan is running at its lowest speed when under a small load, and the CPU/GPU throttling has an added benefit of reducing the overall power usage, too. The difference between non-turbo and turbo speeds on the Intel i7-3820QM can be as much as 13-15 watts.

To enable fan control on ThinkPads, you need to have a file at /usr/lib/modprobe.d/thinkpad_acpi.conf with the following contents:

options thinkpad_acpi fan_control=1

Reboot, and you can now have full control over the fan speed in your ThinkPad!

The script itself looks like this:

#!/bin/bash

set -e

while true; do

  temp=$(cat /sys/class/thermal/thermal_zone1/temp)

  if ((temp > 90000)); then
    echo level 7 >/proc/acpi/ibm/fan
  elif ((temp > 75000)); then
    echo level 5 >/proc/acpi/ibm/fan
  elif ((temp > 60000)); then
    echo level 3 >/proc/acpi/ibm/fan
  elif ((temp > 30000)); then
    echo level 1 >/proc/acpi/ibm/fan
  else
    echo level 0 >/proc/acpi/ibm/fan
  fi

  if ((temp > 97000)); then
    echo 350 >/sys/class/drm/card*/gt_max_freq_mhz
    echo 350 >/sys/class/drm/card*/gt_boost_freq_mhz
  elif ((temp > 90000)); then
    echo 650 >/sys/class/drm/card*/gt_max_freq_mhz
    echo 650 >/sys/class/drm/card*/gt_boost_freq_mhz
  else
    echo 1250 >/sys/class/drm/card*/gt_max_freq_mhz
    echo 1250 >/sys/class/drm/card*/gt_boost_freq_mhz
  fi

  if ((temp > 70000)); then
    echo 1 >/sys/devices/system/cpu/intel_pstate/no_turbo
  else
    echo 0 >/sys/devices/system/cpu/intel_pstate/no_turbo
  fi

  sleep 0.5
done

What we’re doing is reading the temperature of the CPU package and then setting the fan speed and throttling based on that. Note that the temperatures are represented without any decimal places. A reading of 78000 means that the chip is running at 78 °C.

On my ThinkPad T430, the fan speed levels can be roughly described as such:

• level 1: lowest speed, barely audible
• level 3: audible, but tolerable
• level 5 and above: it’s probably loud enough to bother you

The second section controls the integrated GPU. What I’ve found in my testing is that a weak integrated GPU can still chug up to 20 watts, which is a considerable amount, especially on a laptop. The card* wildcard is there because the integrated GPU may change between card0 and card1, and I’m too lazy to make the script any smarter.

The values written there correspond to the maximum GPU clock speed. 1250 happens to be the max clock speed (in MHz) and at that speed the GPU is using a lot of power. 650 is the speed at which the integrated GPU is running most efficiently. 350 is the lowest speed, and you will notice it due to the performance being extra crappy.

With the CPU, I chose to just turn the turbo boost on or off. With turbo boost off, the CPU will top out at 2.7 GHz and around 22 watts of power usage. With turbo boost, the CPU can run anywhere between 3.4-3.7 GHz, depending on the number of cores under load. You’ll likely notice much higher CPU temperatures and power usage climbing to around 35 watts. Fast at short and bursty workloads, and yet efficient and relatively cool at sustained loads.

And at the very end, you have a simple sleep statement. I’ve set it to half a second, but feel free to set it as you see fit. Higher intervals might not be that good of an idea because the script may not be able to respond quickly enough to changing thermals, just keep that in mind.

ASRock DeskMini X300

Although I’ve opted for a more experimental setup with my home server and the ASRock DeskMini X300 is no longer running as one, I think it’s still worthwhile to share how I configured the fan control logic on this machine.

With the ASRock DeskMini X300, my goal was simple: let it run as quietly as reasonably possible without losing too much performance.

This machine was running 24/7 as a server and has a powerful 8-core AMD Ryzen 7 5700G CPU in it. One thing this script is heavily relying on is the fact that Ryzen CPU-s have well-engineered boosting logic in them, meaning that they’re already designed to run at the highest speed possible and only limit their speed if they hit power or thermal limits.

One thing you may need to do first is installing lm-sensors and running sudo sensors-detect. By default the CPU fan speed controls were not properly exposed, but after running sensors-detect and saying yes at every prompt, they were picked up.

The script looks like this:

#!/bin/bash

set -e

# /sys/devices/platform/nct6775.656/hwmon/hwmon3

echo 1 >/sys/devices/platform/nct6775.656/hwmon/hwmon3/pwm2_enable

while true; do

  temp=$(cat /sys/class/hwmon/hwmon4/temp1_input)

  if ((temp > 94000)); then
    echo 170 >/sys/devices/platform/nct6775.656/hwmon/hwmon3/pwm2
  elif ((temp > 90000)); then
    echo 140 >/sys/devices/platform/nct6775.656/hwmon/hwmon3/pwm2
  elif ((temp > 70000)); then
    echo 120 >/sys/devices/platform/nct6775.656/hwmon/hwmon3/pwm2
  elif ((temp > 40000)); then
    echo 30 >/sys/devices/platform/nct6775.656/hwmon/hwmon3/pwm2
  else
    echo 0 >/sys/devices/platform/nct6775.656/hwmon/hwmon3/pwm2
  fi

  sleep 5
done

The hwmon values and which fan you’re controlling is likely different on your own PC. Take a look at the paths found in the script and poke around while also physically looking at your PC. Try enabling PWM control by writing to pwm*_enable handles and then setting the fan speed. As a result of your poking, you should see the CPU fan change its speed. If it doesn’t, try another value.

The values you can write are in the range 0-255, where the higher value corresponds to a higher speed.

I’ve limited the TDP of my CPU to 35 watts using a setting found in UEFI settings. This results in the CPU running cool and quiet even under a full load.

systemd service

To run your fan control script as a systemd service, drop the script to your preferred location (I placed it in /root/.local/bin/fancontrol), then create a file /etc/systemd/system/fancontrol.service with the following contents:

[Unit]
Description=Quick fan control software

[Service]
ExecStart=/root/.local/bin/fancontrol

[Install]
WantedBy=multi-user.target

And once that is done, enable and start your fan control service with sudo systemctl enable --now fancontrol.service.

If you encounter issues or something isn’t working right, check the status of the service with systemctl status fancontrol.service or full logs with journalctl -u fancontrol.service.

Conclusion

If you want something that works out of the box and does the smart stuff for you, then give this Arch Linux wiki page a look. If you’re not into that and want to easily tune the behaviour of your machine, then feel free to use these scripts as a baseline for your own experimentation.

After playing around with fan curves on my ASRock DeskMini X300 based home server, I noticed one interesting thing. At least it was interesting to me.

Background

I had one goal: to try to see if I could turn it into a semi-passive PC by only turning the fan on when the PC was under load while keeping it off for as long as possible. Running the fan only when necessary means less noise and less dust accumulating on the heat sink. No dust, no maintenance required.

The cooling solution used is the Noctua NH-L9a, a low-profile CPU cooler. The two main selling points for me were that it was small enough to fit into the DeskMini, and it is much quieter than the stock AMD Ryzen Stealth cooler.

I started with something reasonable: only kick the fan on when the CPU is over 55°C. This did end up with the fan occasionally turning off, but after around 10-15 seconds it would kick on again. With time, I bumped this limit up by 5-10 degrees each time and ended up with the current configuration. The fan now turns on after 80°C and starts spinning at a modest level at over 90°C.

By turning this limit up, I noticed that the PC could be run in a fanless mode for much longer. This meant that the PC was also much hotter itself and was warm to touch, but nothing extreme or concerning.

Results

I added a couple of lines to my fan control script to track the CPU temperatures and the fan speeds, here are the results:

Based on the data I gathered, the CPU fan was off for 90.7% of the time during this test period. The temperature in the apartment was around 26-30°C, which is pretty much the worst case scenario in my region.

My running theory is that a hotter heat sink is able to dissipate more heat passively. The motherboard and the metal case may also be small contributors to cooling the PC this way. The PC is also acting as a 24/7 home server, which means a lot of idle time that’s interrupted by occasional short-term load. Heat from those bursty loads can be stored in the heat sink relatively easily without it heating up enough to trigger the fan.

This approach will likely not work that well if your workloads are more demanding and result in a higher continuous loads on the system.

Yes, I’m aware that running a PC like this may have negative consequences. I ran the server like this for about a week and while I liked it being a semi-passively cooled server, but decided to go back to the old behaviour.

While there are plenty of fanless designs out there that run hot by design, I decided that I don’t want to take this risk with the DeskMini. It would be a cool experiment, but my main server is probably not the best candidate for it. The CPU could handle it without breaking a sweat, but I’m more concerned about the rest of the components, especially the motherboard. The stickers on the DRAM modules were already warped due to the heat, I prefer not to damage anything else.

For those who want to try something similar (and at your own risk, of course), here’s my fan control script. Just make sure to install lm-sensors first and run sudo sensors-detect so that the controls are exposed by your system. The exact paths and values may be something you need to change to match your system.

#!/bin/bash

set -e

# /sys/devices/platform/nct6775.656/hwmon/hwmon3

echo 1 >/sys/devices/platform/nct6775.656/hwmon/hwmon3/pwm2_enable

while true; do

  temp=$(cat /sys/class/hwmon/hwmon4/temp1_input)

  if ((temp > 94000)); then
    echo 120 >/sys/devices/platform/nct6775.656/hwmon/hwmon3/pwm2
  elif ((temp > 80000)); then
    echo 70 >/sys/devices/platform/nct6775.656/hwmon/hwmon3/pwm2
  else
    echo 0 >/sys/devices/platform/nct6775.656/hwmon/hwmon3/pwm2
  fi

  sleep 5
done

Honorable mentions

I’m not the only one who likes to experiment with the ASRock DeskMini X300. This all started with an idea of fitting a passive cooler into the DeskMini and me discovering that it was already done.

More fun ideas:

• the hotrod of PC cooling
• a reader of FanlessTech actually slapped the passive cooler onto the DeskMini

As some of you might own, I’m still rocking a ThinkPad T430, a laptop model originally released in 2012. It’s not the fastest laptop out there, but it is plenty fast for a number of tasks, even most software development work.

I also try to keep my machines up to date and properly backed up. On my laptop, this means having around a couple of scheduled tasks, such as restic backups. They don’t really take up a lot of time, but if an intensive scheduled task starts while you’re having a video call over Google Meet while also sharing the screen, things can get a bit slow, ruining the experience.

After a couple of these instances, I started thinking about finding a solution.

Scheduled tasks

Here’s a quick rundown of regular tasks that I like to run on my laptop.

Backups using restic: hourly backups to my home server. The file scanning process of the backup is quite intense, eating up a lot of my CPU time.

System updates using dnf and flatpak: usually run daily, because I’m too lazy to install the updates using a GUI, and at the same time I cannot afford to run out of date software.

Backups of critical files from my server to my laptop: there are some files I just cannot afford to lose. For this reason, I make a daily backup of the files to my laptop.

Filesystem maintenance: I don’t trust my storage devices (and neither should you), which is why I run a weekly scrub on my SSD that’s running the btrfs filesystem. If there are errors detected, I will know as soon as possible.

These tasks can be quite intensive, especially on hardware that’s a bit older.

With some tasks, such as system updates, you may also run into random issues. Updating Mozilla Firefox while it’s actively running results in the browser requiring a restart immediately after opening a new tab. This is very annoying and inconvenient when you’re in the middle of your work.

The “time-to-sleep” script

To avoid intensive scheduled tasks running while I’m doing actual work, I tried rescheduling these tasks to a more quiet time. All I needed to do was to edit the systemd timers so that the jobs start after midnight and leave the laptop running.

This solution has quite an obvious downside: my laptop will be powered on pretty much 24/7, and since electricity is super expensive and laptop maintenance a bit of a pain, I’d be wasting resources, time and eventually money.

I could make the last scheduled task make the machine shut down, but what if I have a long-running file copy operation running? It would be interrupted, unless I specifically disabled the scheduled task.

Then it clicked for me.

I know when I’m planning on going to sleep. Sleeping also has a regular cycle as it happens daily. Why not just put all the scheduled daily tasks into one script that I can run whenever I intend to shut down my laptop?

And thus, the time-to-sleep script was born. It’s a simple Bash script that lives in /root/.local/bin/time-to-sleep. To make sure I can also reasonably observe the behaviour of the script, I’ve implemented it as a simple systemd service that lives in /etc/systemd/system/time-to-sleep.service. It’s disabled by default, but can be started manually.

[Unit]
Description=Run maintenance tasks

[Service]
ExecStart=/root/.local/bin/time-to-sleep

I have an alias in ~/.bashrc that I use to start the service.

alias time-to-sleep="sudo systemctl start time-to-sleep.service"

Whenever the time comes to go to sleep, or when I’m just going to do something else and can leave the laptop running at my desk, I just open a terminal window, type time-to-sleep and I’m good to go. If I’m somewhere with a very poor internet connection or need to conserve battery power, I can opt to not run that script.

Going with this approach also saves me a lot of trouble on the technical side as well. I don’t need to think of a magical way to automatically detect if I’m working at the moment, or if I’m running on battery, or if I’m behind a limited internet connection. Sometimes your job becomes so much easier if you make some assumptions about when and where the script runs. You’re in control of choosing when to run it, after all.

In case something goes wrong, I can rely on the set -e line in my script to fail hard. When I come back to my machine, it will be quite noticeable that something has gone wrong, as the laptop will still be running. Who needs a fancy monitoring setup when you have visual evidence of failure?

As for the hourly backups, I solved that problem by only making backups of more important files. It takes much fewer resources compared to the full /home folder backup that runs as part of the time-to-sleep script.

Conclusion

After all this work, I don’t have to worry about any scheduled tasks hogging up all the resources while working. At the same time, I still get all the benefits that the scheduled tasks bring.

Example

In case you need it: here’s an example of the script that I’m running, with all the personal details scrubbed out. Feel free to use it as inspiration for your own time-to-sleep script!

#!/bin/bash

set -e

echo "Restic full home folder backup."
export RESTIC_REPOSITORY='sftp:backupuser@backupserver.lan:/path/to/repo/'
export RESTIC_PASSWORD='someverysecurepasswordgoeshere'

restic unlock

restic backup --verbose --exclude-caches --cleanup-cache \
  --iexclude=/home/*/downloads \
  --exclude=/home/*/.cache \
  --exclude=/home/*/.gradle \
  --exclude=/home/*/.local/share/Trash \
  /home

restic forget --prune \
  --keep-hourly 0 \
  --keep-daily 7 \
  --keep-monthly 0

echo "Restic full home folder backup done."

echo "Backing up important data from the server."

rsync -az --delete-before \
  user@myserver.lan:/path/to/nextcloud/ \
  /storage/backups/nextcloud/

chown -R localuser:localuser /storage/backups/nextcloud/

echo "Backing up important data from the server done."

echo "Updating system."

dnf update -y

flatpak update -y && flatpak uninstall --unused -y

echo "Updating system done."

echo "Scrubbing disk."
btrfs scrub start -B /
echo "Scrubbing disk done."

echo "All done! Going to sleep."

shutdown now

I like having a safety net whenever I’m doing something potentially destructive, which is why I use the btrfs file system for my operating system and my data. Snapshots are one half of my “whoops, there goes all my work” strategy (backups are the other half).

I’ve written about how I use snapshots on btrfs using snapper, but lately I’ve become annoyed with it.

Shortcomings of snapper

snapper is great while you’re on the happy path, but when you wander off of it, it gets a bit frustrating. This is 100% my own personal experience and I cannot rule out any PEBCAK scenarios, but it’s how I felt using the tool.

The snapshots are on the same subvolume, such as /home/.snapshots, which means that I have to specifically exclude the .snapshots folder in every tool and script that I use for making backups. Without that change, the tools ended up scanning the folder and eating up a lot of resources, mainly the CPU and storage.

Backing up snapshots to something like an external backup SSD wasn’t something that snapper supported out of the box as well, meaning that for offline backups I still had to resort to a good old rsync copy. It’s inconvenient and I won’t have a backup that also contained recent snapshots, but it was manageable.

I also reinstalled Fedora Linux on my ThinkPad recently to see how well my Ansible playbooks hold up (quite decently!) and found that setting up snapper is a bit of a pain. I tried to set up snapper again, but it wasn’t as straightforward as I was hoping for, with it not properly picking up the configuration file changes. At the end, it just ran into weird errors when I tried it to prune some snapshots according to the policy I set up. Since I was running out of disk space, I just gave up, uninstalled snapper and destroyed the snapshots manually.

After all that (and a recommendation by a friend) I decided to bite the bullet and try out btrbk.

btrbk

Setting up btrbk on Fedora Linux 36 was a breeze. Install it via dnf install btrbk, copy your configuration to /etc/btrbk/, make sure that the directory used for snapshots exists with mkdir /btrbk_snapshots and you’re good to go! btrbk is just a huge Perl script, so no fancy dependencies are needed.

My btrfs filesystem has three subvolumes: /, /home and /storage. Data stored on these subvolumes has different levels of importance, which is also reflected in the snapshot retention policies.

The configuration on my setup is the following:

timestamp_format        long

volume /
  snapshot_dir /btrbk_snapshots
  subvolume /
    snapshot_preserve_min   12h
    snapshot_preserve       24h
  subvolume /home
    snapshot_preserve_min   48h
    snapshot_preserve       7d
  subvolume /storage
    snapshot_preserve_min   6h
    snapshot_preserve       7d

To take the snapshots, I’ve set up a systemd timer that runs every 5 minutes and runs the following command:

btrbk -c /etc/btrbk/btrbk.conf run --progress

A successful run will have output like this:

--------------------------------------------------------------------------------
Backup Summary (btrbk command line client, version 0.32.1)
    Date:   Sat Jul  9 14:00:00 2022
    Config: /etc/btrbk/btrbk.conf
Legend:
    ===  up-to-date subvolume (source snapshot)
    +++  created subvolume (source snapshot)
    ---  deleted subvolume
    ***  received subvolume (non-incremental)
    >>>  received subvolume (incremental)
--------------------------------------------------------------------------------
/
+++ /btrbk_snapshots/ROOT.20220709T1400
--- /btrbk_snapshots/ROOT.20220708T1300
/home
+++ /btrbk_snapshots/home.20220709T1400
/storage
+++ /btrbk_snapshots/storage.20220709T1400
--- /btrbk_snapshots/storage.20220709T0745
--- /btrbk_snapshots/storage.20220709T0750
--- /btrbk_snapshots/storage.20220709T0755

I love how the output is very concise about what it just did.

To make a backup of my drive to an external SSD that’s also a btrfs filesystem, I created a separate config at /etc/btrbk/backup-to-ext-drive.conf:

timestamp_format        long

target_preserve_min    no
target_preserve        7d

volume /
  snapshot_dir /btrbk_snapshots
  target /mnt/backup/T430
  subvolume /
  subvolume /home
  subvolume /storage

In this example, /mnt/backup/T430 is a btrfs subvolume itself.

Instead of running rsync, my backup script contains the line:

btrbk -c /etc/btrbk/backup-to-ext-drive.conf run --progress

I love how the default behaviour in btrbk is to keep snapshots separate from the subvolumes themselves. Making a full copy of /home or any other filesystem doesn’t require me to always exclude the .snapshots directory, which is nice.

The snapshot names themselves are also human readable. If I accidentally messed up and my changes from an hour ago were lost, I’d just have to go to /btrbk_snapshots, look up a snapshot from an hour ago, and copy whatever I need from the snapshot. This has already saved my butt at least once.

Conclusion

It’s not often that a piece of software gets me that excited, but btrbk has somehow managed to do it. It’s a single script that has all the example use cases documented in its README and the functionality does exactly what I want.

That’s all I really need from software.

Huge thanks to Digital Integrity GmbH for building, maintaining and sharing this tool with the world!

You’ve disabled the boneheaded Lenovo WiFi adapter whitelist using a tool like 1vyrain, installed a better adapter like the Intel Dual Band Wireless-AC 7260, and found that Bluetooth is not working as intended on your ThinkPad T430 running Linux?

Here’s a guide on what you can try to resolve some issues you may encounter.

The integrated Bluetooth adapter

The ThinkPad T430 comes with Bluetooth out of the box. Mine is this one:

Bus 001 Device 004: ID 0a5c:21e6 Broadcom Corp. BCM20702 Bluetooth 4.0 [ThinkPad]

I run Fedora Linux and frankly the experience with the stock adapter is horrible. Getting stuck while pairing a device or the GNOME Bluetooth settings GUI hanging isn’t a rare sight. Sometimes I’ve had to run a script to reset all USB devices to get things sorted out, or restart the machine altogether. This is not fun when a work call started 3 minutes ago and your Bluetooth headset is just not working.

After installing the new Intel WiFi adapter, I thought that I’d just disable the integrated Bluetooth controller via the UEFI settings. That should do the trick, right?

Nope. Apparently this disabled ALL bluetooth devices, including the one provided by the Intel adapter.

Well, no problem, I’ll just physically remove the Broadcom device and that should force the ThinkPad to use the Intel adapter, right?

Nope. Now the machine just shows no Bluetooth devices.

udev to the rescue

If you install the Intel adapter and don’t touch any of the settings, you’ll have two Bluetooth adapters. On my laptop, the OS used the Broadcom adapter by default, which was evident by all GUI interactions interfacing with the Broadcom one and throwing out errors in dmesg -w as a result.

Since disabling the Broadcom bluetooth adapter via other means did not work, I had to look for some solutions. One that I stumbled upon relied on udev rules.

Essentially, what you want to do is drop a file in /etc/udev/rules.d/, name it something like 81-bluetooth-hci.rules and make the contents something like this:

SUBSYSTEM=="usb", ATTRS{idVendor}=="0a5c", ATTRS{idProduct}=="21e6", ATTR{authorized}="0"

Make sure to replace the idVendor and idProduct with the correct values for your Bluetooth adapter. Output of lsusb will likely contain those values.

In my case, the output of lsusb is

Bus 001 Device 004: ID 0a5c:21e6 Broadcom Corp. BCM20702 Bluetooth 4.0 [ThinkPad]

The idVendor and idProduct info is in this string: ID 0a5c:21e6.

After making this change and restarting, you should see thatbluetoothctl list outputs only one Bluetooth device.

[host@T430 ~]$ bluetoothctl list
Controller AC:FD:CE:30:EB:BA T430 [default]

Because Bluetooth is still a hot mess in 2022, I still have to occasionally beat it with a hammer. However, the experience with the Intel adapter is much better and a simple “turn the adapter off and on again in the GUI” solved 99% of the issues I’ve encountered since then.

“Have you tried turning it off and on again?”

If for whatever reason this solution does not apply to you, and you feel like you want to hit your machine with a big hammer without having to restart it, then here’s a script that resets all the USB devices on your machine. I’ve observed that most Bluetooth adapters run over USB, which is why this script works with those as well.

It has helped me countless times when desperate for a quick Bluetooth fix on various desktops and laptops that I’ve used.

Run it at your own risk.

#!/bin/bash

for i in /sys/bus/pci/drivers/[uoex]hci_hcd/*:*; do
  [ -e "$i" ] || continue
  echo "${i##*/}" > "${i%/*}/unbind"
  echo "${i##*/}" > "${i%/*}/bind"
done

Update: I’ve switched to btrbk for managing btrfs snapshots due to certain deficiencies that I had with snapper. Those may have been fixed since then, but btrbk has served me very well, and I’m sticking with it in the foreseeable future.

Friday afternoon. You’re trying out a script that you wrote to mass-rename and move some files around. You finish the script and test it out.

Oops.

All the files now have all the wrong names, and some have been randomly moved 10 folders deep. It’s a mess.

And you didn’t make a backup of your files right before this step because you thought that you wouldn’t need it. It’s just a simple script, after all…

If you’ve found yourself in a similar situation and want to know how to avoid problems in the future, then this article is for you

btrfs: overview

btrfs (butter-fs, bee-tee-arr-eff-ess) is a modern filesystem present in the Linux kernel that has a lot of great features:

• different RAID levels: RAID0, RAID1, RAID10…
• data integrity guarantees
• snapshots
• and many more!

btrfs does have some downsides and has had its fair share of issues and controversy in the past, but at least during the last couple of years it’s been fine in my simple RAID1 + snapshots use cases.

In this post, we’ll focus on the snapshots feature that btrfs offers.

If you’re not sure what snapshots are, then you can think of them as a still image of the state of your filesystem at a certain point in time. You can create a snapshot of the filesystem, delete all your files, and still be able to get your files back just like you left them.

If you’re a developer who has used git and doesn’t mind oversimplified (and probably incorrect) explanations: you can think of snapshots like git commits. You can take snapshots (commit), roll back to a previous snapshot (reset/revert) or view the contents of a previous snapshot (checkout commit).

One thing to keep in mind with snapshots is that they will take up space, especially if you make a lot of them and move files around a lot.

There are many tools available to manage btrfs snapshots. The one I’ve opted for is snapper.

Snapper quick-start

Here’s a quick guide on how to get started with snapshotting your btrfs filesystem. The example is based on a system with a btrfs filesystem mounted on /home

• Install snapper
  • Ubuntu/Debian: sudo apt install snapper
  • Fedora: sudo dnf install snapper
• Setup the configuration: sudo snapper -c home create-config /home
• /etc/snapper/configs now has a text file named home that you can configure to your liking. My personal recommendation is to use the TIMELINE snapshots configuration to configure the maximum number of hourly and daily snapshots.
• snapper ships with two systemd timers that automatically take new snapshots and remove old ones. Make sure that they are enabled:
  • sudo systemctl enable --now snapper-timeline.timer snapper-cleanup.timer
• After a while, you should be able to see snapshots appear in the output of sudo snapper -c home list.
• If you want to browse the files for the old snapshots, go to /home/.snapshots.

Example output of sudo snapper -c home list on my machine:

> sudo snapper -c home list

    # | Type   | Pre # | Date                     | User | Cleanup  | Description | Userdata
------+--------+-------+--------------------------+------+----------+-------------+---------
   0  | single |       |                          | root |          | current     |         
   1  | single |       | K 16 märts 2022 08:12:01 | root | timeline |             |         
 242  | single |       | K 16 märts 2022 13:00:16 | root | timeline |             |         
 302  | single |       | K 16 märts 2022 14:00:05 | root | timeline |             |         
 362  | single |       | K 16 märts 2022 15:00:10 | root | timeline |             |         
 422  | single |       | K 16 märts 2022 16:00:19 | root | timeline |             |         
 461  | single |       | K 16 märts 2022 17:20:29 | root | timeline |             |         
 501  | single |       | K 16 märts 2022 18:00:02 | root | timeline |             |
...         

That’s it!

Snapper tech tips

If you’re new to snapshots, then I recommend keeping an eye on your disk usage and tuning the snapper timeline related configurations to your liking. For my /home folder, my configuration looks like this:

# create hourly snapshots
TIMELINE_CREATE="yes"

# cleanup hourly snapshots after some time
TIMELINE_CLEANUP="yes"

# limits for timeline cleanup
TIMELINE_MIN_AGE="300"
TIMELINE_LIMIT_HOURLY="48"
TIMELINE_LIMIT_DAILY="7"
TIMELINE_LIMIT_WEEKLY="0"
TIMELINE_LIMIT_MONTHLY="0"
TIMELINE_LIMIT_YEARLY="0"

With this configuration, I can undo my screw-ups for up to 7 days, with more granular control for changes done within the last 48 hours.

If you have data that changes more often than other data on the same filesystem, then I recommend utilizing btrfs subvolumes and creating separate snapper configurations for each.

For example: a folder containing gigabytes of cat pictures is unlikely to change all that often, so a longer snapshot retention policy of 24 hourly/30 daily snapshots is a sensible one.

However, the contents of my downloads folder changes more frequently and may contain bigger files that I need temporarily, such as the ISO file for the latest Fedora/Debian release. In a situation like that, having a 6 hourly/7 daily snapshots policy will make more sense, especially if I don’t want to run out of space due to excessive number of snapshots taken for that filesystem.

Further reading: ZFS snapshots

ZFS is a fantastic filesystem that offers many of the features boasted by btrfs, and some that btrfs doesn’t have. ZFS also supports snapshots. To set up automatic snapshot creation and pruning on ZFS, I can recommend sanoid.

Introduction

Before we jump into all the nitty-gritty details, I’d like to go over what we are dealing with here since these topics may be unfamiliar to you. VFIO is quite a niche topic and not everyone knows about it.

GPU passthrough: the process of allowing a VM (virtual machine) to use a dedicated GPU. This allows you to run GPU-heavy workloads inside the VM, such as gaming or anything that benefits from GPU compute power.

VFIO: the framework that allows us to perform this operation.

IOMMU: hardware feature that supports all of this.

virt-manager: GUI application that I use to manage VM-s.

Reasons why you may want to go through all the hassle and use this approach:

• you’re on Linux but want to play games occasionally without rebooting into Windows
• you want to do all your work, gaming and server workloads on one machine
• you simply don’t want to build a dedicated PC for playing games and would like to utilize the existing resources on your main machine
• you want more control over the Windows installation because you simply don’t trust Microsoft
• you want to have more control over the Windows installation and the ability to revert the installation to an earlier point in time using a filesystem that supports snapshots (BTRFS, ZFS)
• you have multiple separate VM-s for different purposes, but you would like to use one single GPU for all of them

One thing to keep in mind is that the hardware you use matters a lot here as that provides the foundation for getting this solution working. Here are some of the requirements you should know about:

• The GPU you want to pass through has to support UEFI. GPU-s from the past 5 years should be good on that, but if you have a really old GPU that you want to use for testing, then that might not work out that well.
• You need to have a good-enough CPU and plenty of RAM. After all, you’re essentially running a full PC within another PC.
• Your motherboard has to support IOMMU and have IOMMU groups that allow you to isolate the GPU and only pass the GPU to the VM. Level1Techs does quite a few motherboard reviews and Wendell goes over the IOMMU groups and suitability for VFIO in them. It also helps if you look around the internet for enthusiasts that have already bought the same motherboard, they may have posted information about the groups. Here’s a handy script that allows you to check your IOMMU groups.
• Some games may ban you if they detect that you are running the game inside a VM. It’s great that they’re trying to do something against cheaters, but unfortunately VFIO users get unfairly treated here. Please do some research beforehand if you play competitive multiplayer games a lot!

I hope that this introduction helped you understand the basics. Now, let’s just jump right into it.

Setup

We’re going to test this setup on hardware that I have covered previously. What makes this setup a bit special is that we’re using an AMD Ryzen 7 5700G APU on an mITX motherboard and a single dedicated GPU, allowing us to do big things in a small package. Yes, there are setups where you can do VFIO with a single GPU and pass it between the host OS and the VM, but that setup might be a bit tricky to use.

The OS is Fedora 34. To get this working, I’ve used various resources:

• beginners guide on Level1Techs forums
• Fedora-specific guide by Wendell from Level1Techs
• this brilliant Arch Wiki guide that covers most of what you need to know about setting this up. It’s not only useful for Arch, a lot of what is written here applies for any distro.

Testing

I decided to do the initial testing with the Nvidia GT 710. It’s slow, buggy on Linux on Nouveau open source drivers, and I had it available. I also recently heard that Nvidia stopped being hostile towards their customers in one aspect by allowing GeForce GPU-s to be used in a Windows 10 VM without having to resort to workarounds.

The testing itself was relatively straightforward. The only issues I had were PEBCAK issues, possibly related to me performing this testing after work. The main issues I ran were either small typos in dracut configuration or the wrong device ID-s being added to kernel boot parameters. Once I discovered and fixed those, it was all smooth sailing.

I’m not going to rewrite a full guide here, please refer to the previously linked resources if you need more details than that. Those resources are also much more likely to receive updates about new finds and features.

The steps I took were roughly these:

• Make sure that IOMMU was enabled in UEFI settings. It’s probably set to auto by default, make sure to set it to enabled.
• Install the GPU that I want to pass through and connect it to a monitor.
• Install virtualization packages on Fedora: sudo dnf install @virtualization
• Enable IOMMU and preload the VFIO kernel module by adding amd_iommu=on rd.driver.pre=vfio-pci to Linux kernel parameters.
  • I use GRUB2, these parameters live in /etc/sysconfig/grub.
  • To apply these changes, you have to regenerate the GRUB configuration. Under Fedora, this is done using grub2-mkconfig -o /etc/grub2-efi.cfg
• Get the device ID-s for the GPU and the related audio device that you’re planning on passing through
  • To see the devices and ID-s, run lspci -nnk. The ID-s look something like 1002:aaf0.
• Bind the GPU to vfio-pci to avoid the GPU driver from taking control of the GPU, otherwise you cannot pass it to the VM.
  • I opted to go for the simple approach and added the device ID-s to kernel parameters: vfio-pci.ids=1002:67df,1002:aaf0
  • Since I went with this approach, then I have to regenerate GRUB configuration again.
• Make sure that the initramfs loads the necessary vfio drivers early in the boot.
  • With Fedora 34, this means creating a file /etc/dracut.conf.d/10-vfio.conf with contents add_drivers+=" vfio_pci vfio vfio_iommu_type1 vfio_virqfd ".
  • Make sure that you don’t have any typos in that.
  • Regenerate the initramfs: dracut -f
• Reboot!
• Using virt-manager, select the VM you want to pass the GPU through and add two PCIe devices: the GPU and its associated audio device.
  • If you don’t have the VM set up yet, then proceed with the normal installation without the GPU passed through yet. Make sure to create an UEFI VM, otherwise you might run into issues. This can be configured in virt-manager _ Overview_ section by selecting the Q35 chipset and setting the firmware to OVMF_CODE.df.
  • If you have passed through the GPU, then make sure to remove the Display Spice device from the VM.
  • If you want to also control the VM, you need to also pass through your USB devices, such as a Logitech wireless receiver.
• Start the VM and cheer once you notice TianoCore appearing on the monitor that’s connected to the passed through GPU!

Results: Nvidia GT 710

Initial testing with Nvidia GT 710 was successful. By successful, I mean that the GPU displayed an image and did not install GPU drivers automatically.

To overcome that last issue, I downloaded the latest official Nvidia GPU drivers and was good to go. The news were true, error code 43 was no longer an issue!

Things got a bit less exciting when I got reminded that this GPU is weak. Very weak. Regardless, I decided to demonstrate its computing prowess by downloading the art of rally demo. Side note: it’s a great game, go try it out!

The demo ran! Not well, but it still ran!

Satisfied with the results, I decided to go for gold and replace this GPU with the AMD RX 570 that I “borrowed” from this PC I built recently.

Results: AMD RX 570

After switching the GPU, I replaced the device ID-s with the new values and continued with the VFIO adventure. This time, the GPU drivers were automatically installed by Windows and everything just worked. That’s not supposed to happen, at least in my experience something always goes wrong. Always.

I installed the latest drivers from AMD-s website and continued with testing.

Furmark? Runs as expected.

GTA IV? A stuttery mess, so yeah, runs as expected.

At this point the VM was assigned 4 CPU cores and I had performed no CPU pinning or optimizations of any kind, so the results were pretty good.

Storage

Because I built this setup on my current workstation/server machine, the storage situation is a bit tricky. The other VM that runs all the services has full access to the two 12TB hard drives and I wasn’t interested in setting up a networked storage setup. The only free spots I had were as follows:

• 120GB partition from the NVMe SSD. Good enough for storing the Windows 10 system files.
• 2x 250GB partitions on the Samsung SATA 1TB SSD-s. Just enough to hold my recently played games, but not much more. Running in striped configuration under Windows.

I later opted to expand the game storage partitions to 375GB, which meant that I had to get rid of the extra overprovisioning space that I had left aside. This setup is fine, but I’m losing out on some of the benefits that come with a virtualized Windows setup.

With regards to passing this storage to the VM, I had two options:

• keep using the SATA virtual disk: works out of the box, but might not perform as well.
• use virtio: you need to manually load and install the drivers for Windows to recognize these disks, but allegedly this has better performance than the SATA implementation.

I started off with SATA and did a comparison against virtio using CrystalDiskMark.

virtio did come ahead in these comparisons, but I would probably have been fine with the SATA performance as well.

Conclusion

One of the main reasons I do these kinds of setups is the technical challenge. It sounds weird, but sometimes I am happier after having completed something technical and novel compared to actually using the setup.

Previously I’ve had mixed results with GPU passthrough, with ThinkPad T430 and eGPU somehow managing to run GTA V in a VM , while in another testing session I ran into issues which turned out to be related to the CPU being faulty (lots of PCIe errors).

I do have more practical future plans with this. The plan is to use this VM as a gaming VM that I can stream games from using Parsec to any device that I wish. One idea is to use the Nvidia Shield TV as a low-power box that is capable of performing streaming. Alternatively, I could also get a super tiny form factor Dell/Lenovo/HP PC that has plenty of power to drive a 4K display and uses a reasonable amount of power. More on that in a future post.

Regarding storage I foresee an upgrade coming soon, either to the drives themselves or the whole setup since ATX motherboards support more SATA ports and expansion cards. If you also pair that up with a case like Masterbox Q500L with a good power supply placement, then you’d still have a relatively small setup.

Introduction

I’ve covered my self-hosting setup in a previous post. It has been a couple of months and the setup has mostly been fine. Sure, the power usage was pretty big under load, and yes, my internet did cut out every time I turned the VM on with all the services starting, but other than that, it has been rock-solid.

However, I soon realized that in most situations, the 12 CPU cores is a bit overkill. Performance was fantastic, but due to the case and the cooling solution that I’m using, the fans did have to ramp up to cool the CPU down. Due to the ongoing GPU drought driven by yet another cryptocurrency bubble, it also felt a bit wasteful to use an AMD RX 560 just to drive two 1080p monitors.

It just so happened that around this time, AMD released the Ryzen 5000 G-series APU-s to retail. Yes, you could buy the 4000 series “OEM-only” APU-s from certain online retailers as well, but with this release, it’s 100% official. The release date rolled around, and the reviews were good. A day later, the CPU was also listed at a local reseller and was actually in stock. After checking the street price for a used Ryzen 9 3900X and the Radeon RX 560, I decided to go for it, as it would be very close to a free “upgrade”.

Prep work

Before performing the purchase, I confirmed that the motherboard that I use (Gigabyte B450 I AORUS PRO WIFI) supports this CPU after a BIOS upgrade. B450 chipset lives in a bit of a gray area where the CPU support is determined by the motherboard manufacturer. No updates? No CPU support for you.

From my previous BIOS updating experiences, I remember that this particular motherboard had some buggy BIOS implementations. I ran into two issues:

• PC would get stuck on POST if you reboot. It’s annoying when making changes to UEFI settings, but with my use case this is not that big of a deal.
• If you remove a boot option, then it is impossible to get it back. Seriously, even a CMOS reset did not help with this one.

This time around, I decided to update the versions one by one to follow the happy path and not run into other types of issues. The bug did resurface right after the first version that I tried, so that was a bit disappointing.

Once I was all up-to-date, I carefully removed the old CPU and plugged in the new one.

Hold on, why would you intentionally downgrade your CPU?

It all came down to “use the right tool for the job”. The Ryzen 9 3900X is a good CPU, no doubts about that, but during my work, I kept an eye on my CPU usage quite often and found that even in the worst-case scenarios, such as IntelliJ IDEA performing indexing, I wasn’t using the CPU to the full extent. It’s great for those scenarios, but I wasn’t hitting them as often as I would have thought.

It also did not help that the CPU did not have an integrated GPU. When all you have to run is a basic desktop setup with 60Hz 1080p monitors, having a dedicated GPU to run them sounds so wasteful. This increased the power use of the whole setup and the GPU could be utilized better elsewhere.

The Ryzen 7 5700G is a downgrade in multi-core performance, mainly due to 4 fewer cores and only 25% of the L3 cache, but according to various reviews and benchmarks, you could actually expect a small single-core performance uplift. Nothing groundbreaking, but good enough for my use case.

Power usage

Because I don’t change my CPU-s too often, I thought I’d take the time to run some benchmarks and measure the power usage of the whole setup. This includes everything: the PC (with two spinning hard drives), standing desk, two 1080p monitors and peripherals.

Please note that these benchmarks are not very scientific and are simply my observations of the change you can expect when performing a switch like this on your own setup. I did my best to make sure that the running conditions are as close as possible by stopping the service VM, waiting until the background CPU usage was normal, running the memory at the same spec etc.

AMD Ryzen 9 3900X + Radeon RX 560

• Idle, sitting on desktop: ~139W
• Full synthetic CPU load using stress: ~253W

AMD Ryzen 7 5700G + integrated Vega 8 GPU

• Idle, sitting on desktop: ~99W (generally between 85-100W)
• Full synthetic CPU load using stress: ~166W

As expected, the change of the CPU and removal of the dedicated GPU results in a big drop when it comes to the power usage of the whole workstation setup. To go even lower, I’d have to switch out the core of the PC to something like the ASRock Deskmini X300 and replace the hard drives with two big SSD-s. I’d go for that as soon as bulk SSD storage became a viable option. Unfortunately, this is not the case, yet.

Performance

Reputable review sites, such as AnandTech, have already done a lot of benchmarks with better testing methodologies, so I instead went ahead and tested out something that I do quite often at work: running tests in poorly written Java projects!

For this, I have two Java projects that have their own test suites. Project A has 900 or so tests, which are mainly parallel in nature. Project B has 2000+ tests, where a good number of them are dependent on a single thread to do all the work.

To run the tests, I just call the “Run all tests” function in IntelliJ and let it do the work of running the tests and measuring the elapsed time. And as mentioned before: this is not scientific, there are probably some errors in here, but the actions performed here are in my opinion a pretty good reflection of day-to-day work, and that’s what matters the most to me. Tests were run twice with a short break between the test runs to let the CPU cool down a bit.

Project A: AMD Ryzen 9 3900X

Test run 1: 10.396s.

Test run 2: 10.494s.

Project A: AMD Ryzen 7 5700G

Test run 1: 15.561s.

Test run 2: 15.690s.

Project B: AMD Ryzen 9 3900X

Test run 1: 364s.

Test run 2: 365s.

Project B: AMD Ryzen 7 5700G

Test run 1: 471s.

Test run 2: 480s.

The performance hit was somewhat expected for project A due to the parallel nature of the tests. However, for project B I expected better results than that due to the alleged improvements in single-thread CPU performance.

The Linux experience

I expected the APU to work as expected, with perhaps some small issues popping up here and there. I’m currently running Fedora 34 with all the latest updates installed and the kernel version at 5.13.6, which should be the best option when trying to get new hardware working on Linux.

The CPU portion of the APU is working relatively well for a CPU that was released just a couple of days ago. I have noticed that for some reason this CPU is constantly clocked at around 4.2GHz on all cores at all times, even if there isn’t too much going on in the system. sensors, the program I use to monitor system temperatures, is also having a small hiccup, as I cannot see the CPU temperatures at all, only the integrated GPU temperatures show up even after running sensors-detect.

nvme-pci-0900
Adapter: PCI adapter
Composite:    +47.9°C  (low  = -40.1°C, high = +89.8°C)
                       (crit = +93.8°C)
Sensor 2:     +47.9°C  (low  = -40.1°C, high = +89.8°C)

iwlwifi_1-virtual-0
Adapter: Virtual device
temp1:        +48.0°C  

amdgpu-pci-0a00
Adapter: PCI adapter
vddgfx:        1.36 V  
vddnb:       999.00 mV 
edge:         +51.0°C  
power1:        6.00 mW 

The GPU seems to be a bit rough around the corners. I’m using the GNOME desktop environment, which means that I’ve learned to endure laggy user interfaces. I default to using Wayland, but I sometimes also use Xorg for those situations where the application I’m trying to run does not work properly under Wayland (looking at you, Slack and your screen sharing functionality…).

The integrated Vega 8 graphics are almost as fast as the dedicated RX 560 GPU under normal light desktop workloads. Once you run something very heavy, like Furmark, the GPU performance tanks and the desktop becomes almost unusable. This is not the case when running some WebGL workloads though, so it could be an isolated case.

The most serious issue I’ve managed to find is that Firefox has some issues with this GPU once you put enough load on the GPU. You may start to notice flickering and UI elements blinking when moving around. A good test case for this is to run Netdata in live mode and have a 1080p Youtube video in another window. I haven’t seen anything like that since running the Nvidia GT 710 with Nouveau drivers under Fedora 34. At least it does not crash the browser or the Wayland session, so it’s not a complete disaster. Still, pretty annoying, so I hope this gets resolved eventually. If I had to guess: the issue is probably related to the Webrender implementation that makes use of the GPU to speed up the browsing experience.

Closing thoughts

Overall, I’m happy that the APU actually works. I’ve had a couple of bad experiences in the past where the Ryzen 3 2200G had stability issues and had to be replaced, so this experience is already a step up from that. The PC runs cooler and quieter, the desktop experience is good enough for me and the CPU performance is acceptable.

I’m sure that the issues that I’ve had with this APU under Linux will be fixed eventually. Each kernel update usually brings lots of improvements to new hardware and I’m sure that this APU is no exception.

As for the BIOS issues, I’m happy that AMD and Gigabyte opted to support the B450 chipset well into 2021, but I’m sure that this choice has come at the cost of QA. If I had to choose between buggy support and having to buy a new motherboard for my new, shiny CPU, then I’d still pick the first one.

Update (2021-08-18)

I’ve had this CPU for a while now, and there have been a couple of kernel updates in the meantime. Here are my notes.

The idle power usage has recently been in between 65-70W for the whole setup. I’ve measured the desktop itself to take up 38W of that.

Although my motherboard does not seem to expose the cTDP option or have ECO mode available, I still managed to limit my TDP to 45W using this trick that /u/The-Stilt mentioned over at Reddit:

The potential lack of the ECO-mode does not prevent you from using it, 
since it is nothing but a preset of PPT/TDC/EDC settings.

Eco-Mode (95W): PPT = 128W, TDC = 80A, EDC = 125A

Eco-Mode (65W): PPT = 88W, TDC = 60A, EDC = 90A

Eco-Mode (45W): PPT = 61W, TDC = 45A, EDC = 65A

This had a great effect on the full load of the system: 3.8GHz all-core while the whole setup consumed 113W.

Update (2021-08-20)

Quick update on the GPU situation: Firefox seems to be fine now, I could not get the flickering to occur again even under worst case scenarios.

Notes and stuff

Here are the outputs for a couple of commands that I could think of.

IOMMU groups

Script taken from Arch wiki.

shopt -s nullglob
for g in `find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V`; do
    echo "IOMMU Group ${g##*/}:"
    for d in $g/devices/*; do
        echo -e "\t$(lspci -nns ${d##*/})"
    done;
done;

IOMMU Group 0:
        00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU Group 1:
        00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU Group 2:
        00:02.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge [1022:1634]
IOMMU Group 3:
        00:02.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge [1022:1634]
IOMMU Group 4:
        00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
        00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus [1022:1635]
        00:08.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus [1022:1635]
        0a:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Cezanne [1002:1638] (rev c8)
        0a:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Renoir Radeon High Definition Audio Controller [1002:1637]
        0a:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor [1022:15df]
        0a:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1 [1022:1639]
        0a:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1 [1022:1639]
        0a:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) HD Audio Controller [1022:15e3]
        0b:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 81)
        0b:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 81)
IOMMU Group 5:
        00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 51)
        00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU Group 6:
        00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:166a]
        00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:166b]
        00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:166c]
        00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:166d]
        00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:166e]
        00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:166f]
        00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1670]
        00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1671]
IOMMU Group 7:
        01:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset USB 3.1 XHCI Controller [1022:43d5] (rev 01)
        01:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset SATA Controller [1022:43c8] (rev 01)
        01:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Bridge [1022:43c6] (rev 01)
        02:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
        02:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
        02:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
        02:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
        02:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
        02:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
        06:00.0 Network controller [0280]: Intel Corporation Wireless-AC 9260 [8086:2526] (rev 29)
        08:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)
IOMMU Group 8:
        09:00.0 Non-Volatile memory controller [0108]: Kingston Technology Company, Inc. U-SNS8154P3 NVMe SSD [2646:5008] (rev 01)

# lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   48 bits physical, 48 bits virtual
CPU(s):                          16
On-line CPU(s) list:             0-15
Thread(s) per core:              2
Core(s) per socket:              8
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       AuthenticAMD
CPU family:                      25
Model:                           80
Model name:                      AMD Ryzen 7 5700G with Radeon Graphics
Stepping:                        0
Frequency boost:                 enabled
CPU MHz:                         3800.000
CPU max MHz:                     4672,0698
CPU min MHz:                     1400,0000
BogoMIPS:                        7585.94
Virtualization:                  AMD-V
L1d cache:                       256 KiB
L1i cache:                       256 KiB
L2 cache:                        4 MiB
L3 cache:                        16 MiB
NUMA node0 CPU(s):               0-15
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full AMD retpoline, IBPB conditional, IBRS_FW, STIBP always-on, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid ap
                                 erfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topo
                                 ext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni 
                                 xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists paus
                                 efilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor smca fsrm

# lspci -nnk
00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne Root Complex [1022:1630]
	Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne Root Complex [1022:1630]
00:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne IOMMU [1022:1631]
	Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne IOMMU [1022:1631]
00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
00:02.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge [1022:1634]
	Kernel driver in use: pcieport
00:02.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge [1022:1634]
	Kernel driver in use: pcieport
00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus [1022:1635]
	Kernel driver in use: pcieport
00:08.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus [1022:1635]
	Kernel driver in use: pcieport
00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 51)
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:5001]
	Kernel modules: i2c_piix4, sp5100_tco
00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:5001]
00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:166a]
00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:166b]
00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:166c]
00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:166d]
00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:166e]
00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:166f]
00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1670]
00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1671]
01:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset USB 3.1 XHCI Controller [1022:43d5] (rev 01)
	Subsystem: ASMedia Technology Inc. Device [1b21:1142]
	Kernel driver in use: xhci_hcd
01:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset SATA Controller [1022:43c8] (rev 01)
	Subsystem: ASMedia Technology Inc. Device [1b21:1062]
	Kernel driver in use: ahci
01:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Bridge [1022:43c6] (rev 01)
	Kernel driver in use: pcieport
02:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
	Kernel driver in use: pcieport
02:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
	Kernel driver in use: pcieport
02:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
	Kernel driver in use: pcieport
02:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
	Kernel driver in use: pcieport
02:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
	Kernel driver in use: pcieport
02:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
	Kernel driver in use: pcieport
06:00.0 Network controller [0280]: Intel Corporation Wireless-AC 9260 [8086:2526] (rev 29)
	Subsystem: Intel Corporation Device [8086:0014]
	Kernel driver in use: iwlwifi
	Kernel modules: iwlwifi
08:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:e000]
	Kernel driver in use: igb
	Kernel modules: igb
09:00.0 Non-Volatile memory controller [0108]: Kingston Technology Company, Inc. U-SNS8154P3 NVMe SSD [2646:5008] (rev 01)
	Subsystem: Kingston Technology Company, Inc. U-SNS8154P3 NVMe SSD [2646:5008]
	Kernel driver in use: nvme
	Kernel modules: nvme
0a:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Cezanne [1002:1638] (rev c8)
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:d000]
	Kernel driver in use: amdgpu
	Kernel modules: amdgpu
0a:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Renoir Radeon High Definition Audio Controller [1002:1637]
	Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Renoir Radeon High Definition Audio Controller [1002:1637]
	Kernel driver in use: snd_hda_intel
	Kernel modules: snd_hda_intel
0a:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor [1022:15df]
	Subsystem: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor [1022:15df]
	Kernel driver in use: ccp
	Kernel modules: ccp
0a:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1 [1022:1639]
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:5007]
	Kernel driver in use: xhci_hcd
0a:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1 [1022:1639]
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:5007]
	Kernel driver in use: xhci_hcd
0a:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) HD Audio Controller [1022:15e3]
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:a0c3]
	Kernel driver in use: snd_hda_intel
	Kernel modules: snd_hda_intel
0b:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 81)
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:b002]
	Kernel driver in use: ahci
0b:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 81)
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:b002]
	Kernel driver in use: ahci

Issue description

You are running a Linux-based machine with an install of ZFS on Linux. Everything seems to work correctly, but after restarting your machine, the ZFS pool is not visible. You can still import your pool manually using zpool import poolname or zpool import -a.

In my situation, this issue does not occur with Debian 10 on an x86-based machine, but it does occur on a Raspberry Pi 4 running Ubuntu Server 20.04.1 LTS.

Setup:

• Raspberry Pi 4 (4GB)
• Ubuntu Server 20.04.1
• 32GB microSD card for root filesystem
• 2x 8TB WD Elements external USB 3.0 drives (ZFS mirror)

Potential causes and solutions

When looking this issue up, there were many potential causes and solutions provided:

• ensure that all ZFS related services are enabled
• add a startup delay in ZFS related configuration files
• just import the pools (duh)
• recreate the ZFS cache file

In my specific scenario, none of these seemed to do the trick.

Investigation

The ZFS pools should be automatically imported on boot by a service named zfs-import-cache.service. The cache file should contain information about the imported pools.

Once you have started your system and don’t see any ZFS pools available, try running systemctl status zfs-import-cache.service or journalctl -u zfs-import-cache.service. This should show you if the automatic import succeeded or not.

The pool tanker is a ZFS mirror that is supported by two 8TB WD Elements external hard drives. They are big, loud and require some time to spin up the platters before you can actually read data off of them. At this point in the investigation, a good friend recommended that I check what disks were recognized by the system at the time when this import service was starting up.

For that, you need to modify the systemd service by running systemctl edit zfs-import-cache.service and put these contents in there:

[Service]
ExecStartPre=/usr/bin/lsblk

This addition will run the lsblk command before the command specified in the systemd service is actually executed. The output of this command will be visible in the service logs. After making this addition, reboot your system and run journalctl -u zfs-import-cache.service again. You should be able to see all the connected drives on the system.

In my case, the system recognized the microSD card and an SSD that was connected over USB, but not the hard drives. This was a very clear indication that the system was not waiting until the drives had spun up before attempting to import the ZFS pool.

Hacky solution

To resolve this issue quickly and move on with more interesting things, you can use the same trick that we used for troubleshooting this issue, but instead of listing the connected hard drives, we can make the systemd service delay its execution by putting a sleep statement in there.

Edit the systemd service again with systemctl edit zfs-import-cache.service and replace the contents with this:

[Service]
ExecStartPre=/usr/bin/sleep 15

This will make the service sleep for 15 seconds before importing the ZFS pool(s). It could be 10, or 30, or whatever value works for you, 15 seconds was chosen here because it is more than enough time for the disks to spin up.

Proper solution (probably)

There are probably better solutions, such as messing around with udev rules or enforcing the delay somewhere else in the ZFS pool import chain of operations, but I have not investigated them at this point. Feel free to contact me with a better solution, and I will update the post accordingly.

I happen to run on an ASRock Fatal1ty B450 Gaming-ITX/ac motherboard that used to run fine, but ever since I installed one UEFI update, it has had this one annoying issue: whenever you reboot, it gets stuck at POST. Cold boots work fine and the shutdown-and-push-the-power-button cycle also does the trick, but it’s not something I want to do all the time, especially when the machine is out of reach.

Before you say it: no, warranty service was not an option here, especially since this motherboard has suffered a fall once and it has this slight curvature in it. No, this separate incident was after this freezing issue started occurring.

Initial troubleshooting

I went through most of the standard hardware troubleshooting procedures:

• clear CMOS
• re-seat RAM
• try a different CPU
• disable features in UEFI, reduce PCIe speeds
• disconnect storage devices

No luck, issue was still there. The POST code was stuck at A0. When taking a look at the ASRock Dr. Debug POST codes list, I saw that A0 falls under

Problem related to IDE or SATA devices. Please re-install IDE and SATA devices. If the problem still exists, please clear CMOS and try removing all SATA devices.

Since I had just done that procedure, and the issue was still there, I was stumped.

Wake-on-LAN

I had tried wake-on-LAN many years ago and was successful with it, so I thought that perhaps this could be an option. After all, I really did not want to get a new motherboard if I could just work around this issue.

I checked the ASRock UEFI for various wake-on-LAN related settings and found a couple of them. Oddly enough, the Wake on LAN setting itself only became available after I enabled booting from LAN in a completely unrelated UEFI section. In addition to the Wake on LAN setting, I also made sure to enable waking up from PS2 and PCIe devices, just in case.

After a quick Google search I found the Arch Linux Wiki page on this topic and saw that I needed to make sure that my network adapter was configured properly for wake-on-LAN to work.

[root@my-machine]# ethtool enp9s0 | grep Wake-on
	Supports Wake-on: pumbg
	Wake-on: g

Super, everything should work then!

I set up an always-on machine to send the wake-on-lan magic packets using wol every minute so that the machine eventually wakes up again. It’s configured, I shut down the problematic machine and wait.

Nothing happens.

Huh, perhaps I missed something.

No, everything was configured properly and the magic packet setting g was on by default.

I checked the wiki page again and two things caught my eye:

Note: Setting one of u, m or b along with g might also be necessary to enable the feature.

It is known that some motherboards are affected by a bug that can cause immediate or random wake-up after a shutdown whenever the BIOS WoL feature is enabled

Well, that’s interesting.

I added * * * * * ethtool -s enp9s0 wol pumbg to my crontab and shut my machine down.

Power supply does a click. Silence. Another click, ethernet port LED-s light up and my machine is powered on again. Success!

As it turns out, I did not even need to enable sending wake-on-LAN magic packets from another machine, the two motherboard issues managed to cancel each other out.

I have a bad habit of testing things whenever a “good” idea pops into my head. This is a short overview of one of them.

The Orange Pi Zero is a SBC (single board computer) that has a slow 32-bit ARM 4 core CPU, 512MB of RAM and no display output. It’s actually quite OK for many tasks, such as reverse proxy (assuming 100Mbit/s is enough for you), low-performance NAS (assuming you are fine with 10-12MB/s file transfer speeds) or a Syncthing relay. Oh, and you can also turn it into a Wi-Fi access point.

Why?

Because Armbian made it really easy to test out and I was interested in seeing what kind of performance an old USB Wi-Fi dongle could offer. This cheap AP could also come into handy in situations where my main access point dies for some reason.

How?

Go to the Armbian device page for your SBC and download the latest image, then write it to your microSD card using the tool of your choice, start your SBC and finish the initial setup.

After all that is done, start armbian-config as root. This tool allows you to do many things over a terminal UI, including a no-hassle method of setting up an Wi-Fi AP.

The option you are looking for is under Network -> Hotspot.

Follow the on-screen instructions to set everything up. After the initial setup is done, make sure to navigate to the same menu again to change the SSID and the Wi-Fi password because the defaults are horribly insecure.

After all this you should be good to go!

Issues and troubleshooting tips

It would not be a proper project if something didn’t go wrong in the process.

Before starting this configuration, run ip a to see if your Wi-Fi device is present. In my case the Orange Pi Zero also has onboard Wi-Fi, but that particular one has poor support, so I use an USB Wi-Fi dongle instead.

If your Wi-Fi device is not showing up, make sure that the correct firmware is present. In my case I had to install firmware-atheros package so that my ath9k_htc based Wi-Fi adapter would start working. dmesg and lsusb -v can help determine what chip your Wi-Fi adapter is using and what driver it needs.

The Orange Pi Zero that I have could set everything up, but it would crash hard with the logs not really showing much. After testing the USB Wi-Fi adapter in all the different USB ports with no success, I opted to use armbian-config menu System -> CPU to limit the CPU clock speed to 480MHz and set the CPU governor to performance to rule out power delivery issues that might be caused by variations in CPU power usage. That seems to have done the trick, because after that change the Wi-Fi AP has been rock solid.

Testing and conclusions

After setting it up and tweaking it a bit I started testing this thing out. The AP had no major issues throughout the day and did not crash.

The performance, however, isn’t anything to write home about. Speed maxes out at 30Mbit/s and is generally lower than that, especially when more than one client is connected. You can do things like Steam Remote Play (stream your game over your home network), but the quality will suffer. It was still quite fun to mess around in GTA V regardless of the image quality and during ~1h of play time there were only 4-5 stutters that were caused by the network.

Would I use this as my main Wi-Fi access point?

Definitely not. Any half-decent router will out-perform this setup.

Can it still be useful?

Definitely. A crappy Wi-Fi AP is better than no AP.

Parts list

• Orange Pi Zero SBC (512MB RAM)
• Orange Pi Zero expansion board (adds two USB 2.0 ports)
• Cheap case for the Orange Pi Zero
• 32GB Sandisk microSD card
• TP-Link WN-722N USB Wi-Fi adapter
• Official Raspberry Pi microUSB power adapter

Update (2020-08-22)

It’s still working and is happily serving as a temporary Wi-Fi access point.

During additional testing I could also hit 40 Mbps speeds when testing in an environment with far fewer competing Wi-Fi access points nearby.

Update (2020-08-30)

It’s still running well and it seems that it won’t be a simple temporary solution after all.

Update (2020-11-30, approximately)

This setup has now been retired. It had 0 issues throughout its lifespan as a Wi-Fi access point, but I replaced it with a more standard Wi-Fi access point so that this board can be used somewhere else.

The setup did surprisingly well and even survived nightly automated updates and restarts, which can be attributed to the good work that the Armbian project has done.