Forge

nix flake lock rm -f /var/lib/sysadmin/confirmed # This is a safety mechanism in case ssh dies, will reset after 10 mins if this file is present nixos-rebuild switch --flake .#forgejo
--build-host root@188.245.235.1
--target-host root@188.245.235.1

LitheumCore testnet (from this repo)

Run on the forge box (forge-singapore: ssh root@5.223.68.120) as root. LitheumCore changes must be merged on git.litheum.com first — the box pulls main from both repos.

git -C /root/sysadmin pull git -C /root/LitheumCore pull cd /root/sysadmin ./scripts/deploy-litheumcore --code-only # ships code to all nodes; only changed services restart ./scripts/check-sync # "OK" = all nodes same chain + same binary ./scripts/find-divergence # binary-search the fork point between two nodes

• --code-only skips staking/spam/monitoring. The full ./scripts/deploy-litheumcore is idempotent (already-staked runners are skipped) and safe to run anytime.
• Expect a brief per-node TPS dip in Grafana during restarts — the chain keeps producing; nodes are restarted sequentially with health checks between.
• check-sync compares the block hash 10 blocks behind the slowest tip and the running binary's nix store hash across every node in nodes.nix.
• find-divergence takes two nodes (nodes.nix names or raw IPs) and reports the greatest common block, divergence depth per node, and — on binaries that support lcli block-header — the competing blocks at fork-point+1. The fork-point search itself only needs latest-block-id/block-hash and works on currently deployed nodes.
• Other deploy.sh flags: --genesis-only --runners-only --stake-only --spam-only --monitoring-only; LITHEUMCORE=/path/to/LitheumCore to relocate.
• Quick checks: ssh root@ lcli health · Grafana http://168.144.209.197:3000

Media funding (dapp media expiry)

Dapp media (JS libs, dapp HTML) is bought in bytehours and expires; when it expires, every dapp referencing it breaks. Two scripts watch and refill it (run from the Mac, with the LitheumCore + deployments checkouts beside sysadmin). Since the testnet reset, bytehours are real byte-hours (24 hours per byte per day), the price curve is P(x) = x/40000 ions per bytehour over the OUTSTANDING bytehour stock (it decays as media expires), and media payments ride inside the media message (--max-payment, U256).

export LITHEUM_NETWORK=testnet
./scripts/check-media                # reconcile deployments/*-hash vs list-media
./scripts/refund-media web3 15       # extend one media by 15 days (label = file minus -hash, or 64-hex hash)
./scripts/refund-media --all 15      # extend everything flagged LOW/EXPIRED

• check-media statuses: OK · LOW (under MEDIA_WARN_HOURS, default 48h) · EXPIRED (bh=0 but still in the DB — extendable) · GONE (pruned past the reorg window — only a republish via the Dapps deploy scripts recovers it). Exits 0 all ok / 1 expired or gone / 2 low → cron-friendly.
• refund-media sends a MEDIAEXTEND tx (--extend-media-hash) with the payment in --max-payment, priced like the Dapps deploys: live per-bytehour price from the Runner (× MEDIA_PRICE_MULTIPLIER headroom). It pre-flights the wallet balance (an unaffordable tx is rejected everywhere and then sits in the outbox retrying, blocking later nonces until the daemon restarts), verifies the receipt strictly, and only reports success when list-media shows the expiry actually advanced.
• Runner-only reads (list-media, media-price) go over ssh to $LITHEUM_NODE (default root@runner02.litheum.com). With LITHEUM_NETWORK=local the scripts instead query the local litheumcore daemon directly (its lcli slot is auto-probed via lcli <slot> id). The signing slot follows the same rule: the local core daemon's slot on local chains, the admin TUI's slot 1 elsewhere — export LCLI_SEND_SLOT= to pin it either way.
• refund-media needs a LitheumCore build with --extend-media-hash and --max-payment in send-transaction — rebuild + restart the slot's daemon after pulling one.

DeMSG services (dequic + declaw) on the forge box

One-time setup, in this order (systemd units live in hosts/forge-singapore):

git -C /root/sysadmin pull
nixos-rebuild switch --flake /root/sysadmin#forge-singapore   # units + ports + Rust toolchain
./scripts/setup-demsg 0x<servicemarketplace-address>          # clone DeMSG + build + write configs
# passwords: /root/.litheum/demsg-passwords/{dequic,declaw}.env with PASSWORD=… (chmod 600)
systemctl start dequic declaw

• Ports: TCP 8080/8090/8091 (gateways + declaw /v1 proxy), UDP 4443/4444 (dequic QUIC/WebTransport).
• The services self-register in the on-chain marketplace on start (--register-service), advertising advertised_ip = the box's public IP from the gateway config.
• Verify: journalctl -u dequic -f · curl http://5.223.68.120:8080/pubkey
• Configs: /root/.litheum/{dequic,declaw}{,-gateway}/config.toml (written by setup-demsg).
• Mac-side operator TUIs: testnet-tui-configs/ (admin = lcli slot 1, all Dapps/DeMSG scripts use lcli 1).

Forgejo (git.litheum.com → 5.223.68.120)

on box: nixos-rebuild boot --flake /root/sysadmin#forge-singapore ; reboot

---

Nix Syntax

Comments

line

/* block */

Literals 1 1.5 true false null "hello" ''multi line'' ./foo.nix # path /abs/path

String interpolation "hi ${name}" ''hi ${name}''

Lists — space-separated, no commas [ 1 2 "a" ./file ]

Sets (attrsets) { a = 1; b = "x"; } { a.b = 1; # nested: { a = { b = 1; }; } } rec { a = 1; b = a; } # fields see each other

Lookup set.a set.a.b set.${name} set.a or 0 # default if missing set ? a # true if field exists

Merge { a = 1; } // { b = 2; } # right wins

Functions x: x + 1 { a, b }: a + b { a ? 0, b }: a + b # default { a, ... }: a # ignore extra fields { a, ... }@args: args.a # whole set as args

Call — space, not () f 1 f { a = 1; b = 2; } g h 1 # (g h) 1 left-associative let let x = 1; f = y: y + x; in f 2

inherit — copy a name into a set { inherit x; } # { x = x; } { inherit (set) a b; } # { a = set.a; b = set.b; } with — fields in scope (avoid when learning) with pkgs; [ hello git ] # same as [ pkgs.hello pkgs.git ]

import — evaluate a file, return its expression import ./foo.nix import ./foo.nix { inherit pkgs; }

if / assert if cond then a else b assert cond; expr # abort if cond is false

Operators

• • • / ++ # ++ is list concat == != < > <= >= && || !

Nix has no global namespace. A name exists only if something in the current expression bound it. Ways a name appears

Binding Example

• function argument x: x + 1 → x
• set-pattern argument { pkgs, ... }: pkgs.hello → pkgs
• let let f = …; in f 1 → f
• rec set rec { a = 1; b = a; } → a inside the set
• inherit copies an existing name into a set
• with set; dumps set’s fields into scope (avoid)

In a flake, self is this flake — the same repo, already evaluated.

outputs = { self, nixpkgs, ... }:

Nix passes self in as an input. It is the flake’s own outputs (plus outPath, sourceInfo, …). self.packages.x86_64-linux.litheumcore # a package this flake defined self.nixosModules.litheum # a module this flake defined self.outPath # path to this repo in the store

#NixOS

NixOS is not “a Linux distro with a config file.” It is a function from a Nix expression + a pinned nixpkgs to one store path that is the entire OS.

Nix builds derivations: pure functions inputs → /nix/store/-name. The hash covers the recipe and every input. Change a byte, get a new path. Old paths stay until GC.

A NixOS system is one derivation whose contents are:

• kernel, systemd, packages
• /etc files (including /etc/litheum/config.toml)
• unit files (litheumcore.service)
• a script that activates that tree (switch generation, restart changed units)

configuration.nix is not that system. It is a NixOS module: a function that sets options (services.openssh.enable = true). The module system merges many modules, then nixpkgs turns the result into the derivation above.

derivation - a Nix build recipe: inputs + build command → a store path. a .drv file. Building it produces /nix/store/{hash}-name. package - a derivation that builds software — a recipe whose output is something installable expression - any Nix value: a number, string, path, list, set, function, or a combination of those. A .nix file is one expression. nixpkgs — The package collection (and NixOS modules) everything is built from. input — A flake dependency (usually a git repo), e.g. nixpkgs. Locked by flake.lock. output — A named product of a flake: a package, a NixOS system, a deploy target. contract — Flakes agree on those names (packages, nixosConfigurations, …) so nix build / deploy-rs know what to grab. module — A Nix function that sets NixOS options (services.openssh.enable = true). Many modules merge into one config. recipe / derivation — Same idea: “how to build X.” The .drv is the recipe on disk. Building it produces a store path. store path — /nix/store/{hash}-name. The built result. Nix files are expressions. Almost everything is a function or a set ({ ... }). Modules are just functions that return a set of options.

You write

• functions (args: body)
• sets / attrsets ({ foo = 1; })
• lists, strings, paths
• modules (a function { pkgs, ... }: { services.openssh.enable = true; })

Importing import ./foo.nix # any expression import ./foo.nix { inherit pkgs; } # if foo.nix is a function NixOS also has imports = [ ./bar.nix ]; inside a module — that pulls in more modules to merge, not a normal import.

The flake is the entry point. The CLI looks up outputs by name.

  description = "...";
  inputs = { ... };
  outputs = { self, nixpkgs }: { ... };
}```

outputs are required, others optional

Command                                     -> What it uses
nix run .#hello                             -> packages.<system>.hello
nix build .#hello                           -> same, leaves ./result
nix flake lock                              -> writes flake.lock from inputs
nixos-rebuild build-vm --flake .#hellohost  -> nixosConfigurations.hellohost

On nix flake lock (or the first nix build / nix run / nixos-rebuild if there is no lock yet), nix reads inputs.*.url, clones the repos, hashes them, and writes flake.lock.