Merge pull request #624 from genebean/kiosk

Setup remote builds and set Pi mostly readonly

README.md

@@ -5,14 +5,18 @@ This repo is a Nix flake that manages most of my setup on macOS and fully manage
 - [Flake structure](#flake-structure)
 - [Formatting and CI](#formatting-and-ci)
 - [Historical bits](#historical-bits)
-- [Adding a new macOS host](#adding-a-new-macos-host)
-  - [Extras steps not done by Nix and/or Homebrew and/or mas](#extras-steps-not-done-by-nix-andor-homebrew-andor-mas)
-    - [Firefox profile switcher](#firefox-profile-switcher)
-    - [Setup sudo via  Touch ID](#setup-sudo-via--touch-id)
-    - [Atuin](#atuin)
-    - [Mouse support](#mouse-support)
-- [Adding a NixOS host](#adding-a-nixos-host)
-  - [Post-install](#post-install)
+- [Host Bootstrapping](#host-bootstrapping)
+  - [Replacements](#replacements)
+    - [Image-based Systems](#image-based-systems)
+    - [Other Systems](#other-systems)
+  - [Net-new Hosts](#net-new-hosts)
+    - [Adding a new macOS host](#adding-a-new-macos-host)
+      - [Extras steps not done by Nix and/or Homebrew and/or mas](#extras-steps-not-done-by-nix-andor-homebrew-andor-mas)
+        - [Setup sudo via Touch ID](#setup-sudo-via-touch-id)
+        - [Atuin](#atuin)
+        - [Mouse support](#mouse-support)
+    - [Adding a NixOS host](#adding-a-nixos-host)
+      - [Post-install](#post-install)
 
 ## Flake structure
 
@@ -48,7 +52,44 @@ CI validation is defined in `.github/workflows/validate.yml` and mirrors what is
 
 This repo historically contained my dot files. Historically symlinked files on Windows are still in `windows/`. Everything else is just in git history now.
 
-## Adding a new macOS host
+## Host Bootstrapping
+
+### Replacements
+
+Sometimes hosts, or their storage, need replacing... sepcially ones that run on SD cards like `kiosk-gene-desk`. When that time comes, here is how to get it back up and running.
+
+#### Image-based Systems
+
+1. install image
+2. boot with wired connection
+3. ssh in but don’t use known hosts file
+4. restore user and host ssh keys
+5. run `mkdir -p ~/.config/sops/age && ssh-to-age -private-key -i ~/.ssh/id_ed25519 > ~/.config/sops/age/keys.txt && ssh-to-age -i ~/.ssh/id_ed25519.pub  > ~/.config/sops/age/pub-keys.txt`
+6. reboot
+7. ssh in as normal
+8. run these commands:
+
+   ```bash
+   mkdir repos
+   cd repos
+   git clone git@github.com:genebean/dots
+   cd dots
+   nix-auth login
+   nix flake update private-flake # needed so private bits are cached properly
+   nixup
+   ```
+
+9. reboot
+
+#### Other Systems
+
+Yeah.... this is not something I have properly documented. Best guess: install like a net-new host but then restore keys and such like on an image based system. Supplement that with restores from restic backups.
+
+### Net-new Hosts
+
+The directions below are all a bit dated and likely incomplete 😔 They will be updated as time make practical.
+
+#### Adding a new macOS host
 
 1. run `xcode-select --install` to install the command-line developer tools (this includes the Apple's stock version of Git).
 2. create ed25519 ssh key via `ssh-keygen -t ed25519`
@@ -108,19 +149,15 @@ This repo historically contained my dot files. Historically symlinked files on W
 26. After the nix command finally works, open a new iTerm window and it should have all the nixified settings in it.
 27. Go into iTerm2's preferences and use the Hack Nerd Mono font so that the prompt and other things look right. You will likely also want to adjust the size of the font.
 
-### Extras steps not done by Nix and/or Homebrew and/or mas
+##### Extras steps not done by Nix and/or Homebrew and/or mas
 
-#### Firefox profile switcher
-
-You will need to link `firefox-profile-switcher-connector` for it to work. The easiest way to do this is to run `brew reinstall firefox-profile-switcher-connector` and follow the directions printed in the terminal.
-
-#### Setup sudo via  Touch ID
+###### Setup sudo via Touch ID
 
 1. run `sudo cp /etc/pam.d/sudo_local{.template,}` - this will generate a popup asking permission
 2. run `sudo nvim /etc/pam.d/sudo_local` and uncomment line as directed by top comments
 3. save via `!w` which will generate a popup asking permission
 
-#### Atuin
+###### Atuin
 
 Nix installs and configures Atuin, but you still need to log into the server:
 
@@ -129,13 +166,13 @@ Nix installs and configures Atuin, but you still need to log into the server:
 3. run `read -s apass` and enter the user password
 4. run `atuin login --key=$akey --password=$apass --username=gene`
 
-#### Mouse support
+###### Mouse support
 
 - [Logitech M720 Triathlon mouse](https://support.logi.com/hc/en-us/articles/360024698414--Downloads-M720-Triathlon-Multi-Device-Mouse)
 
-## Adding a NixOS host
+#### Adding a NixOS host
 
-### Post-install
+##### Post-install
 
 1. clone this repo
 2. create keys for [SOPS](https://georgheiler.com/post/sops/) via `mkdir -p ~/.config/sops/age && nix run nixpkgs#ssh-to-age -- -private-key -i ~/.ssh/id_ed25519 > ~/.config/sops/age/keys.txt && nix run nixpkgs#ssh-to-age --  -i ~/.ssh/id_ed25519.pub  > ~/.config/sops/age/pub-keys.txt`

flake.lock

@@ -41,11 +41,11 @@
         "onchg": "onchg"
       },
       "locked": {
-        "lastModified": 1771280126,
-        "narHash": "sha256-pegK7+4aWBgc7tKK9Us5LYK6lmvEXgrylFva7f4FbUs=",
+        "lastModified": 1774888967,
+        "narHash": "sha256-bbB44tkmxKehBvXYeBruuPp/VnJMwMwyc98tyJ6rWO4=",
         "owner": "aksiksi",
         "repo": "compose2nix",
-        "rev": "58d4b4685a8fe152a46386b63edb7a055f0de8a1",
+        "rev": "901ac12a99c14c6f526487d58588f5c01109f5bb",
         "type": "github"
       },
       "original": {
@@ -351,11 +351,11 @@
         ]
       },
       "locked": {
-        "lastModified": 1774274588,
-        "narHash": "sha256-dnHvv5EMUgTzGZmA+3diYjQU2O6BEpGLEOgJ1Qe9LaY=",
+        "lastModified": 1774875830,
+        "narHash": "sha256-WPYlTmZvVa9dWlAziFkVjBdv1Z6giNIq40O1DxsBmiI=",
         "owner": "nix-community",
         "repo": "home-manager",
-        "rev": "cf9686ba26f5ef788226843bc31fda4cf72e373b",
+        "rev": "7afd8cebb99e25a64a745765920e663478eb8830",
         "type": "github"
       },
       "original": {
@@ -507,11 +507,11 @@
     },
     "nixos-hardware": {
       "locked": {
-        "lastModified": 1774465523,
-        "narHash": "sha256-4v7HPm63Q90nNn4fgkgKsjW1AH2Klw7XzPtHJr562nM=",
+        "lastModified": 1774933469,
+        "narHash": "sha256-OrnCQeUO2bqaWUl0lkDWyGWjKsOhtCyd7JSfTedQNUE=",
         "owner": "NixOS",
         "repo": "nixos-hardware",
-        "rev": "de895be946ad1d8aafa0bb6dfc7e7e0e9e466a29",
+        "rev": "f4c4c2c0c923d7811ac2a63ccc154767e4195337",
         "type": "github"
       },
       "original": {
@@ -554,11 +554,11 @@
     },
     "nixpkgs-unstable": {
       "locked": {
-        "lastModified": 1774273680,
-        "narHash": "sha256-a++tZ1RQsDb1I0NHrFwdGuRlR5TORvCEUksM459wKUA=",
+        "lastModified": 1775034801,
+        "narHash": "sha256-tsecHNsWwr4wSaM2oW9GwafMwE24J+xD8bKDoto3exY=",
         "owner": "nixos",
         "repo": "nixpkgs",
-        "rev": "fdc7b8f7b30fdbedec91b71ed82f36e1637483ed",
+        "rev": "8d029aa64915e54b7846873d9583af4c9fd21ea4",
         "type": "github"
       },
       "original": {
@@ -586,11 +586,11 @@
     },
     "nixpkgs_3": {
       "locked": {
-        "lastModified": 1774388614,
-        "narHash": "sha256-tFwzTI0DdDzovdE9+Ras6CUss0yn8P9XV4Ja6RjA+nU=",
+        "lastModified": 1775002709,
+        "narHash": "sha256-d3Yx83vSrN+2z/loBh4mJpyRqr9aAJqlke4TkpFmRJA=",
         "owner": "nixos",
         "repo": "nixpkgs",
-        "rev": "1073dad219cb244572b74da2b20c7fe39cb3fa9e",
+        "rev": "bcd464ccd2a1a7cd09aa2f8d4ffba83b761b1d0e",
         "type": "github"
       },
       "original": {
@@ -658,11 +658,11 @@
         ]
       },
       "locked": {
-        "lastModified": 1774233120,
-        "narHash": "sha256-txGwTNKNYQT1rFPkxd6imEvQ03SmIyKAXNBaYtB3Jes=",
+        "lastModified": 1775077724,
+        "narHash": "sha256-LGifKfUhZr99hX+vRZZhDDT6+6AyjTbqomq2SgL/Pv8=",
         "owner": "genebean",
         "repo": "private-flake",
-        "rev": "45fca86f711966ee29add03027ee3ffc48992110",
+        "rev": "aef30e8ac1ae465de8d5747931bea4402d042113",
         "type": "github"
       },
       "original": {
@@ -779,11 +779,11 @@
         ]
       },
       "locked": {
-        "lastModified": 1774303811,
-        "narHash": "sha256-fhG4JAcLgjKwt+XHbjs8brpWnyKUfU4LikLm3s0Q/ic=",
+        "lastModified": 1774910634,
+        "narHash": "sha256-B+rZDPyktGEjOMt8PcHKYmgmKoF+GaNAFJhguktXAo0=",
         "owner": "mic92",
         "repo": "sops-nix",
-        "rev": "614e256310e0a4f8a9ccae3fa80c11844fba7042",
+        "rev": "19bf3d8678fbbfbc173beaa0b5b37d37938db301",
         "type": "github"
       },
       "original": {

modules/hosts/nixos/kiosk-gene-desk/default.nix

@@ -10,6 +10,7 @@
   imports = [
     # SD card image
     "${inputs.nixpkgs}/nixos/modules/installer/sd-card/sd-image-aarch64.nix"
+    ./read-only-root.nix
   ];
 
   system.stateVersion = "24.11";
@@ -49,6 +50,29 @@
     };
   };
 
+  nix = {
+    distributedBuilds = true;
+    buildMachines = [
+      {
+        hostName = "hetznix02.technicalissues.us";
+        system = "aarch64-linux";
+        protocol = "ssh-ng";
+        maxJobs = 4;
+        speedFactor = 2;
+        supportedFeatures = [
+          "nixos-test"
+          "benchmark"
+          "big-parallel"
+        ];
+        sshUser = "gene";
+        sshKey = "/root/.ssh/id_ed25519";
+      }
+    ];
+    extraOptions = ''
+      builders-use-substitutes = true
+    '';
+  };
+
   nixpkgs.overlays = [
     (_final: super: {
       makeModulesClosure = x: super.makeModulesClosure (x // { allowMissing = true; });

modules/hosts/nixos/kiosk-gene-desk/home-gene.nix

@@ -17,6 +17,7 @@
         "--hide-crash-restore-bubble"
       ];
     };
+    zsh.history.path = "/tmp/zsh_history_gene"; # needed becaues of read only fs
   };
 
 }

modules/hosts/nixos/kiosk-gene-desk/read-only-root.nix

@@ -0,0 +1,126 @@
+{
+  lib,
+  pkgs,
+  username,
+  ...
+}:
+{
+  # ------------------------------------------------------------------ #
+  # Read-only SD card mounts and tmpfs for writable paths
+  # ------------------------------------------------------------------ #
+  fileSystems = {
+    "/" = lib.mkForce {
+      device = "/dev/disk/by-label/NIXOS_SD";
+      fsType = "ext4";
+      options = [
+        "ro"
+        "noatime"
+        "nodiratime"
+      ];
+    };
+
+    "/boot/firmware" = lib.mkForce {
+      device = "/dev/disk/by-label/FIRMWARE";
+      fsType = "vfat";
+      options = [
+        "ro"
+        "noatime"
+        "nofail"
+        "noauto"
+      ];
+    };
+
+    "/var/log" = {
+      device = "tmpfs";
+      fsType = "tmpfs";
+      options = [
+        "size=64m"
+        "mode=0755"
+        "nosuid"
+        "nodev"
+      ];
+      neededForBoot = true;
+    };
+
+    "/var/lib" = {
+      device = "tmpfs";
+      fsType = "tmpfs";
+      options = [
+        "size=256m"
+        "mode=0755"
+        "nosuid"
+        "nodev"
+      ];
+      neededForBoot = true;
+    };
+
+    "/home/${username}/.cache" = {
+      device = "tmpfs";
+      fsType = "tmpfs";
+      options = [
+        "size=256m"
+        "mode=0700"
+        "uid=1000"
+        "nosuid"
+        "nodev"
+      ];
+    };
+
+    "/home/${username}/.local" = {
+      device = "tmpfs";
+      fsType = "tmpfs";
+      options = [
+        "size=256m"
+        "mode=0700"
+        "uid=1000"
+        "nosuid"
+        "nodev"
+      ];
+    };
+
+    "/home/${username}/.config/chromium" = {
+      device = "tmpfs";
+      fsType = "tmpfs";
+      options = [
+        "size=128m"
+        "mode=0700"
+        "uid=1000"
+        "nosuid"
+        "nodev"
+      ];
+    };
+  };
+
+  # ------------------------------------------------------------------ #
+  # tmpfs for paths that need to be writable at runtime
+  # ------------------------------------------------------------------ #
+
+  # /tmp - NixOS built-in option, cleaner than a manual fileSystems entry
+  boot.tmp.useTmpfs = true;
+  boot.tmp.tmpfsSize = "20%";
+
+  # ------------------------------------------------------------------ #
+  # systemd-journal needs its directory to exist after /var/log tmpfs
+  # is mounted
+  # ------------------------------------------------------------------ #
+  systemd.tmpfiles.rules = [
+    "d /var/log/journal 0755 root systemd-journal -"
+    # create a writable zsh history file in /tmp for gene
+    "f /tmp/zsh_history_gene 0600 ${username} users -"
+  ];
+
+  # ------------------------------------------------------------------ #
+  # Helper scripts for doing a nixos-rebuild
+  # ------------------------------------------------------------------ #
+  environment.systemPackages = [
+    (pkgs.writeShellScriptBin "remount-rw" ''
+      echo "Remounting / read-write..."
+      sudo mount -o remount,rw /
+
+      echo "Starting nix-daemon..."
+      systemctl start nix-daemon.socket nix-daemon.service
+
+      echo "Done. Run 'reboot' when finished."
+    '')
+  ];
+}