# Proxmox CT + Pi-hole over Tailscale — Completed Session

## Context
- Host: `imac-proxmox` (Tailscale IP 100.67.1.32)
- Goal: Create CT `pihole` (CTID 106) running Debian 12 + Tailscale + Pi-hole, then use the Tailscale IP as global DNS for the tailnet.
- Outcome: **Completed successfully**. Pi-hole now blocks ads for all tailnet devices via `100.86.194.20`.

## Pre-flight Discovery

```bash
# Next available CT ID
ssh -o IdentitiesOnly=yes -i ~/.ssh/id_ed25519_imac_proxmox root@imac-proxmox "pvesh get /cluster/nextid"
# -> 106

# Storage accepting templates (jq not installed on host, parse JSON manually)
pvesh get /nodes/localhost/storage --output-format json
# -> hdd-data (content includes "vztmpl")

# Available OS templates
ls -la /var/lib/vz/template/cache/
# -> Only ubuntu-26.04 present. Downloaded Debian 12:
pveam download local debian-12-standard_12.12-1_amd64.tar.zst

# Network bridge
pvesh get /nodes/localhost/network --output-format json
# -> vmbr0 (static 192.168.0.50/24)
```

## SSH Access Hygiene

- Proxmox nodes typically only allow `root` login via key auth. Use `root@<host>` explicitly.
- Specify the Hermes session key: `ssh -o IdentitiesOnly=yes -i ~/.ssh/id_ed25519_imac_proxmox root@imac-proxmox`
- If auth fails after rapid retries, back off briefly before retrying (possible rate limit/lockout).
- If the key is rejected, the user may need to add the public key to `/root/.ssh/authorized_keys`.

## CT Creation

```bash
pct create 106 local:vztmpl/debian-12-standard_12.12-1_amd64.tar.zst \
  --hostname pihole \
  --storage hdd-data --rootfs hdd-data:8 \
  --memory 512 --cores 1 \
  --net0 name=eth0,bridge=vmbr0,ip=dhcp \
  --unprivileged 1 --features nesting=1 \
  --password <TEMP_PASSWORD>

pct start 106
```

## Tailscale in an Unprivileged LXC — Critical Step

By default, unprivileged LXC containers do **not** have access to `/dev/net/tun`, which Tailscale needs to create the `tailscale0` interface.

**Without this fix, `tailscaled` will crash loop with:**
```
getLocalBackend error: createEngine: tstun.New("tailscale0"): CreateTUN("tailscale0") failed; /dev/net/tun does not exist
```

**Fix:** Add these two lines to `/etc/pve/lxc/<VMID>.conf` on the **host**, then restart the CT:

```bash
echo 'lxc.cgroup2.devices.allow: c 10:200 rwm' >> /etc/pve/lxc/106.conf
echo 'lxc.mount.entry: /dev/net/tun dev/net/tun none bind,create=file 0 0' >> /etc/pve/lxc/106.conf
pct reboot 106
```

Then inside the CT:
```bash
curl -fsSL https://tailscale.com/install.sh | sh
systemctl start tailscaled
tailscale up
# User must open the printed auth URL to approve the node
tailscale status
# -> pihole  100.86.194.20
```

## Pi-hole v6 Installation

Pi-hole v6 was released between this skill's creation and this session. Key differences from v5:
- Legacy `/etc/pihole/setupVars.conf` is **auto-migrated** to `/etc/pihole/pihole.toml` during install.
- There is **no `pihole setdns` CLI command** in v6. Edit `pihole.toml` directly.
- The web server and API are now built into `pihole-FTL` (ports 80 and 443).

### Unattended install

```bash
mkdir -p /etc/pihole
cat > /etc/pihole/setupVars.conf << 'EOF'
PIHOLE_INTERFACE=eth0
IPV4_ADDRESS=192.168.0.215
PIHOLE_DNS_1=1.1.1.1
PIHOLE_DNS_2=8.8.8.8
QUERY_LOGGING=true
INSTALL_WEB_SERVER=true
INSTALL_WEB_INTERFACE=true
LIGHTTPD_ENABLED=true
CACHE_SIZE=10000
DNS_FQDN_REQUIRED=true
DNS_BOGUS_PRIV=true
DNSMASQ_LISTENING=all
WEBPASSWORD=<YOUR_PASSWORD>
EOF

curl -sSL https://install.pi-hole.net | bash -s -- --unattended
```

### Configure Pi-hole to answer Tailscale queries

Pi-hole must listen on all interfaces, not just the LAN interface, to respond to queries arriving over the `tailscale0` tunnel:

```bash
# Inside the CT
sed -i 's/listeningMode = "LOCAL"/listeningMode = "ALL"/' /etc/pihole/pihole.toml
systemctl restart pihole-FTL
```

Verify DNS is working over the Tailscale IP:
```bash
dig @100.86.194.20 google.com +short      # Should resolve
dig @100.86.194.20 doubleclick.net +short  # Should return 0.0.0.0 (blocked)
```

### Changing upstream DNS servers in Pi-hole v6

```bash
# Replace Google with Quad9 (or any preferred upstream)
sed -i 's/"8.8.8.8"/"9.9.9.9"/' /etc/pihole/pihole.toml
systemctl restart pihole-FTL
```

## Tailscale DNS Configuration

In the Tailscale admin console (https://login.tailscale.com/admin/dns):

1. Under **Global nameservers**, add the CT's Tailscale IP: `100.86.194.20`
2. Enable **"Override local DNS"**
3. Remove any commercial DNS (e.g., Google `8.8.8.8`) you don't want logging queries.
4. Optionally add Quad9 (`9.9.9.9`) as a fallback upstream.

This forces every device on the tailnet to route DNS through Pi-hole, giving network-wide ad blocking.

## Verification Checklist

| Check | Command | Expected Result |
|-------|---------|-----------------|
| CT running | `pct status 106` | `status: running` |
| Tailscale active | `tailscale status` | `100.86.194.20 pihole` |
| Pi-hole listening | `ss -tlnp \| grep 53` | Port 53 on `0.0.0.0` and `[::]` |
| Web admin up | `curl -k https://100.86.194.20/admin` | HTTP 200, Pi-hole HTML |
| DNS resolves | `dig @100.86.194.20 google.com` | Real IPs |
| Ads blocked | `dig @100.86.194.20 doubleclick.net` | `0.0.0.0` |

## Common Pitfalls

| Pitfall | Cause | Fix |
|---------|-------|-----|
| `tailscaled` crashes on start | Missing `/dev/net/tun` in unprivileged CT | Add TUN device to CT config (see above) |
| Pi-hole doesn't answer Tailscale queries | `listeningMode` set to `LOCAL` | Change to `ALL` in `pihole.toml` |
| `pihole setdns` command not found | Pi-hole v6 removed this CLI | Edit `/etc/pihole/pihole.toml` directly |
| SSH auth fails | Wrong user or missing key | Use `root@host` and the explicit Hermes key |
| `pct restart` command not found | Proxmox has no `restart` subcommand | Use `pct reboot` or `pct stop && pct start` |