🧪

Testing

Static checks, unit and smoke tests, and real boots and installs in QEMU.

• Static: syntax and shellcheck, desktop-file-validate, strict schema compilation, XML and YAML parsing, manifest consistency.
• Unit (pytest): the logic that could silently break: calculator safety, search, markup sanitizing, file operations, labwc config editing, GTK CSS management, autostart rules, the greetd protocol, tray icons, sunrise and sunset, unit and currency conversion, clipboard history (including skipping passwords), OCR language choice, weather parsing, the Spotlight providers, the Night Light schedule.
• Smoke: a real labwc session on wlroots' headless backend in a container. Every app and Settings page opens without exceptions, the tray works end to end, Spotlight answers conversions, emoji, clipboard and project searches, Quick Look previews code, a PDF and a picture, the overview opens with windows in it, and screenshots are saved.
• Image: checks the finished root filesystem and ISO (programs, services, themes, installer, cleanliness, BIOS and UEFI boot records, offline pool, every manifest app).
• Boot: QEMU boots the real ISO through GRUB, with BIOS and with UEFI (OVMF). The QEMU guest agent inside the image lets the test run checks in the live system and launch every default app with the real session environment.
• Install: the real installer, driven by key presses sent through QEMU's monitor (no test hooks inside the installer), installs onto an empty virtual disk. The installed system then boots on its own and is checked: user, login, btrfs layout, snapshots and the snapshot boot menu.
• What the tests have already caught:
  • a heap-corrupting GRUB font on BIOS;
  • a missing Docker CLI;
  • Plymouth hiding the desktop;
  • games unreachable from the session PATH;
  • two garbage-collection bugs, a Files crash and a Settings crash;
  • a 25-second black desktop: the shell asked D-Bus to start BlueZ on machines with no Bluetooth adapter;
  • a theme built for a newer GTK filling logs with CSS errors;
  • the shell passing its gtk4-layer-shell preload on to every app it started, which crashed GTK 3 apps (Firefox, LibreOffice, Geary) launched from the dock.
  • the installer's "strong password" check failing every password, because the cracklib dictionary wasn't installed;
  • installing was impossible: unsquashfs (squashfs-tools) was missing from the image, so Calamares couldn't copy the system to the disk.